Research Brief

LLM-generated research ideas consistently demonstrate a narrower and systematically shifted distribution compared to human ideas, primarily focusing on synthesis and 'bridge-like' opportunities.

Large Language Models (LLMs) are increasingly employed for brainstorming research ideas, yet current evaluations often focus on individual idea attributes like novelty or feasibility. This paper introduces a novel approach to assess the fundamental gap between human and LLM creative ideation. The researchers built a large-scale evaluation framework by taking high-quality human research papers, identifying their likely inspiring prior works, and then prompting LLMs to generate new ideas from these same sets of prior works. By profiling each idea using a new two-axis taxonomy of 'research taste' (opportunity pattern and research paradigm), they quantified the divergence. The key finding is a consistent distributional gap: LLMs disproportionately generate ideas that bridge existing concepts or synthesize information, while human researchers demonstrate a much broader range in how they frame problems and construct contributions. This suggests that while LLMs produce reasonable ideas, their creative spectrum remains narrower and systematically different from human research preferences.

• Developing AI tools that specifically target and expand LLM capabilities in areas where human creativity excels, such as novel problem framing or identifying entirely new research paradigms.
• Designing advanced benchmarking systems for generative AI in scientific discovery, allowing for a more nuanced comparison against human 'research taste' beyond simple metrics.
• Guiding human researchers towards 'white spaces' or less explored avenues in research where current LLMs are least effective, thus optimizing human-AI collaboration.
• Informing the training objectives for future LLMs aimed at scientific ideation, to better cultivate a broader and more diverse 'research taste' beyond mere synthesis.