[Paper Review] Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers

Chenglei Si, Diyi Yang+1 more|arXiv (Cornell University)|Sep 6, 2024

Wikis in Education and Collaboration41 citations

TL;DR

The authors conduct a large-scale study comparing LLM-generated research ideas with human expert ideas; AI ideas are rated as more novel while feasibility is similar, based on blind reviews from 100+ NLP researchers.

ABSTRACT

Recent advancements in large language models (LLMs) have sparked optimism about their potential to accelerate scientific discovery, with a growing number of works proposing research agents that autonomously generate and validate new ideas. Despite this, no evaluations have shown that LLM systems can take the very first step of producing novel, expert-level ideas, let alone perform the entire research process. We address this by establishing an experimental design that evaluates research idea generation while controlling for confounders and performs the first head-to-head comparison between expert NLP researchers and an LLM ideation agent. By recruiting over 100 NLP researchers to write novel ideas and blind reviews of both LLM and human ideas, we obtain the first statistically significant conclusion on current LLM capabilities for research ideation: we find LLM-generated ideas are judged as more novel (p < 0.05) than human expert ideas while being judged slightly weaker on feasibility. Studying our agent baselines closely, we identify open problems in building and evaluating research agents, including failures of LLM self-evaluation and their lack of diversity in generation. Finally, we acknowledge that human judgements of novelty can be difficult, even by experts, and propose an end-to-end study design which recruits researchers to execute these ideas into full projects, enabling us to study whether these novelty and feasibility judgements result in meaningful differences in research outcome.

Motivation & Objective

• Assess whether state-of-the-art LLMs can generate novel, expert-level research ideas at scale.
• Control for confounders in ideation, writeup format, and review process to enable fair comparison with human experts.
• Provide a standardized evaluation protocol and benchmark data for future ideation agent research.
• Compare three conditions: human-written ideas, AI-generated ideas, and AI-generated ideas reranked by a human expert.
• Investigate limitations of LLM ideation agents and propose end-to-end follow-up studies to study real-world outcomes.

Proposed method

• Use retrieval-augmented generation (RAG) to ground AI ideas with up to 120 retrieved papers per topic and score papers for relevance, empirical content, and potential to inspire new work.
• Generate 4000 seed ideas per topic to maximize candidate quality, followed by deduplication and a top-N selection.
• Rank ideas using a Swiss-system pairwise comparison-based LLM ranker trained on public conference review data, with a human-backed rerank option (AI Ideas + Human Rerank).
• Standardize idea writeups with a fixed template and a style normalization module to remove signal cues from writing style.
• Recruit over 100 NLP researchers as experts for idea writing and blind reviewing, ensuring cross-condition comparability and institutional separation between reviewers and authors.
• Evaluate ideas via a blinded review form with four metrics (novelty, excitement, feasibility, expected effectiveness) plus overall score, using 1–10 scales with rationales.

Figure 1: Overview of our study: we recruit 79 expert researchers to perform blind review of 49 ideas from each of the three conditions: expert-written ideas, AI-generated ideas, and AI-generated ideas reranked by a human expert. We standardize the format and style of ideas from all conditions befor

Experimental results

Research questions

• RQ1 Can current LLMs generate novel research ideas at expert level comparable to or surpassing human experts in novelty?
• RQ2 Do AI-generated ideas differ in perceived novelty, excitement, feasibility, and expected impact compared with human-written ideas?
• RQ3 How reliable is LLM-based ranking of idea quality, and what are the limitations of self-evaluation and diversity in generation?
• RQ4 What is the effect of human reranking on the quality and novelty of AI-generated ideas?
• RQ5 What are the implications of human judgments of novelty for end-to-end execution studies of ideation agents?

Key findings

• AI-generated ideas are rated as more novel than human expert ideas (p<0.05) across multiple tests.
• AI-generated ideas are rated with slightly lower feasibility compared with human ideas, though overall scores are not consistently worse.
• AI ideas plus human rerank show even higher novelty and overall scores than human ideas, indicating value in human-in-the-loop ranking.
• Reviewer analyses reveal that novelty and excitement drive overall scores more than feasibility, highlighting subjectivity in novelty judgments.
• LMM-based ranking of ideas via the agent shows limitations, including lack of diversity at scale and imperfect self-evaluation as evaluators.
• The study provides a standardized protocol and release of agent implementation and reviews to support future end-to-end ideation research.

Figure 2: Comparison of the three experiment conditions across all review metrics. Red asterisks indicate that the condition is statistically better than the Human baseline with two-tailed Welch’s t-tests and Bonferroni correction. All scores are on a 1 to 10 scale. More detailed results are in Sect