[orcid=0000-0002-5725-2677]

[orcid=0009-0006-8476-4826]

[orcid=0009-0008-3346-692X]

[orcid=0009-0000-7448-8043]

[]

[orcid=0009-0003-6555-9091]

Idea Generation ,Large Language Models ,AI for Science ,Genetic Algorithm

1 Introduction↩︎

Generating novel and meaningful research ideas is a central cognitive activity in scientific research [1]. Researchers typically synthesize large volumes of previous work, understand existing research directions, identify unresolved problems, and envision new methodological or conceptual advances [2], [3]. This process is inherently time-consuming and intellectually demanding, and has become increasingly challenging with the rapid growth and fragmentation of scientific literature [4]. Recent advances in Large Language Models (LLMs) have demonstrated strong capabilities for knowledge synthesis by integrating effective reasoning and information retrieval techniques, suggesting promising new opportunities to leverage LLMs to support scientific idea generation in literature-intensive research settings [5]–[8].

LLM-assisted scientific idea generation frameworks are built upon the Retrieval-Augmented Generation (RAG) paradigm [9], [10], utilizing external knowledge to contextualize generated concepts. The initial approaches mainly focused on retrieval optimization, employing structured or iterative semantic search to improve the relevance and novelty of the idea [1], [11]. Recently, the focus has shifted to complex generative strategies, ranging from planned retrieval for expanded topical coverage [12], to structured schemes such as chain-based trend modeling [13], prompting-based reasoning [14], [15], and multi-agent collaboration [16], [17].

Despite these advances, most existing methods follow a linear and isolated idea generation paradigm, as illustrated in Figure 1 (a). Each idea is generated and refined independently based on its own retrieval context, with limited exploration among concurrently generated ideas. Meanwhile, retrieval strategies in this paradigm remain predominantly relevance-driven, relying on signals such as entity co-occurrence or citation chains [1], [13]. However, while planned retrieval strategies aim to diversify query formulation [12], [17], they are typically deployed in a static and uniform manner. This rigidity constrains idea evolution to conservative trajectories, limiting the search space. Consequently, the generation process tends to converge prematurely within existing semantic neighborhoods, weakening the diversity of research topics and the systematic exploration of cross-disciplinary research.

To address these limitations, we introduce EvoGens, a population-based heuristic search framework for LLM-assisted scientific idea generation. EvoGens initializes a diverse population of candidate research ideas and iteratively refines them via rank-based mutation and semantic-aware crossover. This parallelized evolutionary process supports continuous recombination and diversification of concepts, enabling more systematic exploration of literature-grounded research directions (see Figure  1 (b)). Taken together, EvoGens is intended to support early-stage conceptual exploration rather than autonomous scientific discovery. Our contributions are summarized as follows:

• We introduce EvoGens, a population-based framework for AI-assisted research ideation that conceptualizes early-stage idea exploration as a population-level evolutionary process. Using selection, mutation, and crossover, it transcends the limitations of linear generation, enabling systematic exploration of synergistic research directions.

• We introduce the Rank-Based Mutation (RBM) mechanism, implemented through differentiated retrieval-plan generation, together with the Semantic-Aware Crossover (SAC) operator to reorganize core conceptual components.

• We conduct extensive experiments on a scientific idea generation benchmark. Results show that EvoGens consistently improves novelty and diversity under the current automatic evaluation protocol, while maintaining comparable quality.

2 Related Work↩︎

2.1 LLM-Driven Idea Generation↩︎

Recent advances in LLMs have sparked growing interest in AI-assisted scientific idea generation within the broader AI for Science paradigm [18], [19]. Most existing work adopts retrieval-augmented generation as the core approach, using external academic knowledge to ground the generation of new research ideas. Representative methods construct structured retrieval mechanisms, such as entity co-occurrence graphs or citation-based associations, to identify relevant concepts for idea synthesis [1], [11]. Building upon this paradigm, subsequent studies introduce more structured retrieval and generation strategies to enhance novelty and diversity, including planning-based retrieval to guide query formulation and expand topical coverage [12], [17], as well as structured or iterative generation schemes such as chain-based modeling of research evolution [13], prompting-based reasoning frameworks [14], [15], and multi-agent discussion settings that simulate collaborative scientific processes [16], [20].

Although these approaches have demonstrated that LLMs can generate scientifically meaningful and sometimes highly novel ideas [21], [22], they also highlight ongoing challenges in reducing redundancy and maintaining sufficient diversity between generated ideas [11]. Existing iterative or retrieval-based methods typically optimize ideas along largely independent generation trajectories, where each idea is refined in terms of its own retrieval context. As a consequence, exploration of isolated ideas is often implicit or indirect [16], and systematic coordination of exploration and refinement across multiple candidates remains underexplored.

These observations highlight the potential of population-based optimization for LLM-based idea generation. Evolutionary algorithms[23], as a prominent example, provide a principled framework for balancing exploration and exploitation through mechanisms such as selection, crossover, and mutation—offering a systematic paradigm that transcends isolated single-trajectory iterations.

2.2 LLM-Based Genetic Algorithms↩︎

Genetic Algorithms (GAs) are a class of evolutionary computation methods inspired by natural selection, extensively used in optimization, machine learning, and conceptual design [7], [24], [25]. By simulating evolutionary operators such as selection, crossover, and mutation, GAs iteratively update a population of candidate solutions to explore the search space and progressively approach promising regions. Compared with single-point search methods, their population-based and parallel exploration mechanisms enable stronger exploration in complex and large-scale search spaces, helping avoid local optima while preserving diversity.

Driven by recent advances in LLMs, researchers are integrating evolutionary principles into LLM-based tasks. Evolutionary algorithms are considered as frameworks for structured generation and selection, rather than strict numerical optimization. Previous studies demonstrate that LLMs can effectively simulate evolutionary operators such as mutation [26] and crossover  [27] in natural language spaces, providing a practical foundation for applying evolutionary search to structured generation tasks. For example, EvoAgent [28] and EvoPrompt [29] leverage evolutionary algorithms to automatically generate task-specific agents or prompts, replacing manual design with population-based search and iterative refinement. In particular, Evolution of Heuristics (EoH) [30] evolves natural-language heuristic “thoughts” together with executable code for automatic heuristic design, while LLaMEA [31] iteratively generates and refines metaheuristic algorithms based on performance feedback. Recent studies have further explored the close integration between LLMs and evolutionary processes, such as Guided Evolution (GE) [32], which uses LLM reasoning to guide evolutionary operations, and LAOS [33], which employs LLMs to adaptively control mutation and crossover based on online optimization feedback.

Evolutionary algorithms have also shown potential in scientific discovery tasks. FunSearch [7] demonstrates that combining LLMs with evolutionary search can effectively explore discrete program spaces for combinatorial optimization. LLM-SR [34] and SGA [6] further extend this paradigm to symbolic regression and physics-driven formula discovery, where evolutionary search in discrete hypothesis spaces is combined with LLM prior knowledge and, in some cases, continuous optimization via differentiable simulators. In general, these studies indicate that the integration of evolutionary principles into LLM-based generation can improve both the efficiency of the exploration and the quality of the solution.

Despite the success of existing LLM-based genetic algorithms, most approaches focus on well-structured generation targets such as formulas, code, or prompts, or executable heuristics, where candidate solutions lie in a relatively constrained search space, fitness functions are directly quantifiable, and optimization objectives are explicit. This is also the case for representative LLM+EA systems such as EoH and LLaMEA, which target automatic heuristic or algorithm design under explicit performance feedback.

In contrast, when generating open-ended scientific ideas in natural language, candidate solutions have diffuse semantic boundaries, evaluation is inherently weaker, and naive crossover or mutation can introduce hallucinations, semantic redundancy [11], [12], or premature convergence. These differences motivate evolutionary strategies tailored to literature-grounded scientific idea generation, including diversity-aware crossover for recombining semantically distinct ideas and retrieval-guided mutation for grounding modifications in relevant knowledge.

2.3 Reasoning and Planning↩︎

The planning and reasoning capabilities of LLMs have become an active research focus for complex generation tasks. Classical approaches such as Chain-of-Thought (CoT) prompting [35] explicitly model intermediate reasoning steps, while Tree-of-Thought (ToT) [36] extends this paradigm by enabling multi-branch exploration and self-evaluation. For knowledge intensive settings, retrieval has been shown to further enhance reasoning effectiveness. Representative frameworks such as Retrieval-Augmented Reasoning  [37] and ReAct [38] integrate reasoning, planning, and external tool use, allowing models to access and incorporate external knowledge during generation. Together, these studies demonstrate that structured reasoning and retrieval-aware planning can substantially improve generation quality in complex tasks.

These paradigms have also been explored in idea generation and hypothesis discovery tasks [12], [17], where retrieval and planning are typically used to improve individual generations under a shared strategy. However, such uniform strategies fail to account for the heterogeneous states and varying maturity levels of candidate ideas within a population, where different ideas may require different degrees of knowledge grounding and semantic expansion. This limitation suggests the need for more adaptive retrieval and planning mechanisms in open-ended idea generation.

Taken together, prior work suggests that LLMs can support idea generation through structured generation, retrieval, reasoning, and planning, while evolutionary methods provide a natural mechanism for exploration and recombination. However, existing approaches still lack a unified framework that combines retrieval-aware planning with population-level evolutionary search for literature-grounded research ideas. Most current methods either follow linear or isolated generation pipelines, or apply optimization strategies primarily designed for more structured search targets with clearer evaluation signals. In contrast, research ideas are open-ended natural language objects with diffuse semantic boundaries and weakly defined fitness, requiring both sustained diversification and adaptive control. This gap motivates EvoGens, which integrates retrieval-aware planning with population-based evolutionary exploration through rank-based mutation and semantic-aware crossover, enabling continuous diversification and structured reorganization of candidate ideas.

3 METHODS↩︎

3.1 EvoGens Framework↩︎

This section introduces the EvoGens framework, as illustrated in Figure 2. The framework consists of three evolutionary modules: (1) Seed Population Initialization, which constructs an initial population of candidate ideas under the research scope defined by a target paper and its reference context; (2) Evolutionary Idea Exploration, which updates the idea population through the Score Module, Rank-Based Mutation (RBM), and Semantic-Aware Crossover (SAC); and (3) Population Update and Abstract Synthesis, which summarizes the evolved population and generates a structured research abstract for the selected ideas.

EvoGens formulates scientific idea generation as an evolutionary search process in natural language space. Candidate ideas are iteratively evolved through mutation, crossover, and selection. Within this process, LLMs serve as the generative engine for seed initialization, mutation, crossover, and abstract synthesis, while a lightweight scoring model provides heuristic control for ranking, filtering, and population update. In addition, a text embedding model is used to instantiate the shared semantic space for similarity computation, clustering, and crossover pairing. The framework is designed to support comparative search and population control within an open-ended ideation process, rather than to provide a definitive assessment of scientific validity or real-world usefulness. Since the scorer is LLM-based, the resulting scores should be interpreted as heuristic comparative signals within a fixed evaluation setting, rather than as stable or definitive measurements of scientific merit.

3.2 Seed Population Initialization↩︎

The Seed Population Initialization module constructs an initial population of candidate ideas grounded in a specific research context, providing the starting point for the subsequent evolutionary process. Specifically, a target paper is used as the primary input, defining the research topic, direction, and domain, and thereby establishing the semantic scope for subsequent evolution.

To provide supporting context, the framework also incorporates related papers drawn from the reference context of the target paper. These papers provide complementary background knowledge and research directions that help initialize the search space. Based on the target paper and its related papers, the system automatically summarizes and refines relevant contextual information from the abstracts of the related papers, and uses it to guide initial idea generation.

Following Nova [12], we adopt a scientific discovery-oriented prompting strategy to encourage the generation of candidate ideas grounded in prior literature. The corresponding seed-generation prompt is provided in Tables 12 and 13 of Appendix 9.

For the initial population, the module combines the provided contextual information with the model’s internal knowledge and incorporates Chain-of-Thought reasoning to improve coherence and interpretability. Meanwhile, self-correction and reflection mechanisms [39] are applied to examine the internal consistency of the generated outputs, thereby reducing hallucination. The resulting seed ideas constitute the initial population. They are subsequently evaluated by the unified Score Module, which is detailed in Section 3.3. This module provides a heuristic signal for relative comparison and population control, rather than an absolute measure of scientific merit. Based on these scores, the initial population is filtered and organized to form the first current population that enters the evolutionary loop.

3.3 Score Module↩︎

Before introducing the scoring mechanism, we first define the population and structured idea representation on which the subsequent evaluation and evolutionary operators operate.

At each iteration \(t\), the system maintains a population \(P_t = \{ i_1^t, i_2^t, \dots, i_N^t \}\) of \(N\) candidate ideas.

Each candidate idea is represented as a structured semantic unit so that it can be evaluated and transformed at the component level during evolution. \(i = \langle T, M_{ot}, D, M_{et}, R \rangle\), corresponding to its Title, Motivation, Description, Method, and Rationale. Here, Method refers to the proposed technical mechanism or intervention, while Rationale captures the reasoning that explains why the proposed method may be useful under the target problem setting. This structured representation enables fine-grained evaluation and controlled transformation during the evolutionary process.

A unified scoring signal is needed to coordinate mutation, crossover, and selection within the evolutionary process. Rather than operating in a purely stochastic manner, these operators are coordinated through explicit scoring signals, allowing the system to promote exploration and diversity while maintaining basic idea quality.

At each iteration, all ideas in the population are evaluated by a unified Score Module, which assigns a scalar score to each idea based on its content. The resulting score serves as a heuristic fitness proxy for within-population control, rather than a validated estimate of scientific merit. It provides a unified reference for relative comparison and decision-making within the population, including ranking, filtering, and the control of generation strategies in mutation and crossover. Importantly, the Score Module itself is not an evolutionary operator. Specifically, we employ an LLM-based scorer to assess each idea along six complementary dimensions: Innovativeness, Feasibility, Significance, Clarity, Effectiveness, and Generalizability [1], [13], [16], [40], [41]. The corresponding scoring prompt is provided in Table 14 of Appendix 9. These dimensions reflect the originality of the idea, the plausibility of its implementation, its potential impact, the clarity of its presentation, its expected ability to address the target problem, and its applicability beyond a narrow setting, respectively. Together, they provide a coarse-grained yet informative signal for evaluating early-stage scientific ideas, where precise quantitative metrics are often unavailable. Each dimension is scored on an integer scale from 1 to 10, and the overall idea score is computed as a weighted linear combination (Eq. 1 ), where \(s_k(i)\) denotes the score of idea \(i\) on the \(k\)-th dimension, and \(w_k\) is the corresponding predefined weight.

\[\text{Score}(i) = \sum_{k=1}^{6} w_k \cdot s_k(i), \label{equa1}\tag{1}\]

6pt

As summarized in Table 1, the weighting scheme reflects relative priorities among different characteristics of scientific ideas, rather than defining a universal or optimal evaluation function. Following prior work on multi-dimensional assessment of research ideas [1], [42], higher weights are assigned to Innovativeness and Significance, as EvoGens primarily targets the exploration of novel and potentially impactful research directions. The remaining dimensions serve as supporting criteria that encourage basic soundness, interpretability, and extensibility, thereby preventing incoherent or degenerate ideas from dominating the population.

We do not claim that this weighting scheme is uniquely optimal; rather, it provides a practical heuristic configuration for within-population control in early-stage idea exploration. The specific weight values are used as practical coefficients for introducing sufficient score differentiation among candidate ideas, which supports effective selection and controlled generation within the evolutionary loop [43]. We do not claim that this weighting scheme is uniquely optimal; rather, it provides a practical heuristic configuration for within-population control in early-stage idea exploration.

While the resulting scores are analyzed as continuous variables, they are discretized into coarse quality tiers during specific evolutionary stages. This discretization supports subsequent rank-based and crossover-related control mechanisms, as detailed in the following sections.

3.4 Rank-Based Mutation (RBM)↩︎

Rank-Based Mutation (RBM) assigns different mutation modes to ideas according to their relative ranking in the current population. Building on the unified scoring signal introduced above, we first rank ideas within the current population and then partition the ranked list into three coarse tiers, which are mapped to different mutation modes. We refer to this mechanism as Rank-Based Mutation (RBM): the score determines relative ordering among ideas, while the actual mutation behavior is assigned according to rank intervals rather than absolute score thresholds [44].

Operationally, RBM transforms an input idea into a literature-grounded variant through retrieval-plan generation, literature retrieval, and conditioned rewriting. From an operator perspective, RBM is implemented through retrieval-plan generation, external knowledge acquisition, and conditioned rewriting, transforming an existing idea into a literature-grounded variant.

Formally, the mutation process first generates a retrieval plan, then retrieves external literature, and finally synthesizes a mutated idea conditioned on the retrieved knowledge. Operationally, RBM takes an input idea, applies rank-conditioned retrieval planning and diversity control, and outputs a literature-grounded mutated idea.

Given a current idea \(i\), the system first generates a mutation-specific retrieval plan under a planning regime \(\rho\) and diversity control \(\tau\): \[p \sim \mathcal{P}(i; \rho, \tau), \label{equa2}\tag{2}\] where \(\mathcal{P}(\cdot)\) (Eq. 2 ) denotes the LLM-based retrieval-plan generation process, and the resulting plan \(p\) specifies the search fields and keywords used for external knowledge acquisition. The system then executes this plan through an academic search engine¹ to obtain retrieved literature: \[k \sim \mathcal{R}(p), \label{equa3}\tag{3}\] where \(\mathcal{R}(\cdot)\) (Eq. 3 ) denotes literature retrieval conditioned on the generated plan. Finally, the retrieved knowledge \(k\) is used to generate a mutated idea via (Eq. 4 ) \[\mathcal{M}_{ut}(i, k) = \mathrm{LLM}(i, k). \label{equa4}\tag{4}\]

Unlike classical genetic algorithms, where mutation is often implemented as a low-probability random perturbation, EvoGens treats mutation as a primary knowledge-acquisition operator for exploring an open-ended idea space. Each mutation is grounded in newly retrieved academic literature, allowing the system to continuously introduce external knowledge and reduce semantic stagnation under limited initial context. Accordingly, mutation is applied to all individuals in the current population before crossover, so that recombination operates on ideas that have already been enriched through knowledge-driven expansion. Therefore, the mutation stage should be understood as retrieval-grounded, selection-guided variation rather than unconstrained random perturbation: diversity is introduced through differentiated planning and external literature acquisition, while remaining bounded by the current idea and retrieved evidence.

In implementation, the ranked population is partitioned into three coarse tiers using a 2:2:1 ratio. The top-ranked ideas are assigned to Conservative mutation, the middle-ranked ideas to Explorative mutation, and the bottom-ranked ideas to Radical mutation. This allocation is used as a practical heuristic to balance stability and diversification, rather than a theoretically optimal ratio. The three mutation modes correspond to progressively stronger retrieval planning regimes and diversity settings. As summarized in Table 2, Conservative mutation emphasizes local refinement using closely related literature; Explorative mutation expands toward adjacent or less-explored topics; and Radical mutation encourages broader semantic deviation by probing semantically distant or unconventional research directions.

Notably, prompt variation and temperature control are applied primarily at the stage of retrieval-plan generation rather than unrestricted final idea synthesis. This design localizes the risk of higher-diversity generation: even when a more exploratory plan produces noisy or weak leads, the resulting effect is typically limited to failed or less relevant retrieval, rather than directly injecting hallucinated content into the final mutated idea. The final mutation step remains conditioned on retrieved academic literature, preserving evidence grounding and traceability.

3.5pt

At the implementation level, the three mutation modes in Table 2 are instantiated with distinct retrieval-plan prompts and corresponding temperature settings, producing low-, medium-, and high-diversity retrieval plans, respectively. The corresponding retrieval-plan prompts for Conservative, Explorative, and Radical mutation are provided in Tables 7, 8, and 9 of Appendix 9. In our implementation, these three modes use temperatures of 0.6, 0.8, and 1.0.

For each idea, the LLM first formulates a mutation-specific retrieval plan specifying search fields and keywords. This plan is then executed through an external academic search engine (in our implementation, Semantic Scholar) to obtain idea-specific new knowledge from the literature. Conditioned on the retrieved content, the LLM generates a refined or transformed idea using a shared mutation-generation prompt, ensuring that mutations remain evidence-based and traceable. The corresponding mutation-generation prompt is provided in Table 10 of Appendix 9.

3.5 Semantic-Aware Crossover (SAC)↩︎

In this stage, EvoGens recombines ideas across semantically distinct regions of the population, where semantic distinctions are determined in the embedding space described below. This design promotes knowledge integration while reducing redundant recombination between highly similar ideas. The crossover process operates on a unified candidate pool that combines the mutated results with the current population, allowing newly generated ideas to interact with existing ones.

To construct diverse and high-quality crossover parents, SAC first embeds the candidate pool into a shared semantic space and performs clustering-based parent selection. Let \(\mathcal{C}_t\) denote the raw candidate set at iteration \(t\), which comprises the current population and their mutated variants. All ideas in \(\mathcal{C}_t\) are embedded into a shared semantic space. Here, the shared semantic space is instantiated as the sentence-embedding space produced by all-MiniLM-L6-v2, and semantic distance is measured by cosine distance between idea embeddings. This embedding space is used only for clustering and parent pairing, rather than as a direct substitute for the idea representation itself. To encourage semantic diversity while retaining strong candidates in parent selection, we apply \(k\)-means clustering to partition the embeddings into \(k\) clusters. From each non-empty cluster, the idea with the highest score (as evaluated by the Score Module) is selected as a representative. This clustering-based selection yields the parent set \(\mathcal{P}_t = \{ i_1, i_2, \dots, i_m \}\), where \(m \leq k\), ensuring that the selected parents span diverse semantic regions while prioritizing high-quality candidates within each region.

After representative parents are selected, SAC forms crossover pairs by greedily matching semantically distant ideas. Based on the selected parent set \(\mathcal{P}_t\), multiple parent pairs are constructed for crossover in a greedy and distance-aware manner. For each idea \(i \in \mathcal{P}_t\) that remains unpaired, we compute its semantic distance to all other unpaired ideas and select the one with the largest distance as its crossover partner. Once a pair is formed, both ideas are removed from the candidate list so that each idea participates in at most one crossover. This procedure yields a set of parent pairs:

\[\mathcal{M}_t^{\text{match}} = \{(i_a^{(1)}, i_b^{(1)}), (i_a^{(2)}, i_b^{(2)}), \dots, (i_a^{(r)}, i_b^{(r)})\},\] which collectively support the generation of multiple offspring within a single evolutionary iteration.

Operationally, SAC takes a pair of semantically distant parent ideas, applies a structured crossover prompt over their complementary components, and outputs a synthesized offspring idea. For each parent pair \((i_a, i_b) \in \mathcal{M}_t^{\text{match}}\), the crossover operator integrates their complementary conceptual components through an LLM-based generation process: \[\mathcal{C}_{\text{cross}}(i_a, i_b) = \text{LLM}(i_a, i_b). \label{equa5}\tag{5}\] For each parent pair \((i_a, i_b)\) (Eq. 5 ), the LLM is prompted with the structured components of both ideas (e.g., motivation, description, method, and rationale) and instructed to synthesize an offspring that recombines complementary elements into a coherent new idea. The corresponding crossover-generation prompt is provided in Table 11 of Appendix 9.

The generated crossover offspring are merged with the mutated ideas to form the candidate pool for subsequent population update. Combined with clustering-based parent selection, this crossover design is intended to reduce redundant recombination while encouraging knowledge integration.

3.6 Population Update and Abstract Synthesis↩︎

This module governs population-level selection and generational transition, consolidating evolved ideas across iterations and producing the final output.

After mutation and crossover, the newly generated ideas are merged into a candidate pool from which the next population is selected. Only ideas generated in the current iteration are eligible for selection into the next population, while ideas from the previous population are not directly carried over. The population update step formalizes how the next generation is selected from the mutated and crossover-produced candidates at iteration \(t\).

\[P_{t+1} = \mathcal{S}_{\text{pop}}\bigl(P_t^{\text{mut}} \cup P_t^{\text{cross}}\bigr) \label{equa6}\tag{6}\]

Here, \(\mathcal{S}_{\text{pop}}(\cdot)\) (Eq. 6 ) denotes the population selection operator that determines the composition of the next generation.

This generational replacement strategy avoids repeatedly selecting unchanged ideas and promotes sustained semantic exploration across iterations. To implement \(\mathcal{S}_{\text{pop}}(\cdot)\) while preserving diversity, the candidate pool is partitioned into \(n\) clusters in the semantic embedding space, and the highest-scoring idea from each cluster is selected to form the next population. If the maximum number of evolutionary iterations has not been reached, \(P_{t+1}\) serves as the new current population for the next iteration; otherwise, it is treated as the final evolved population.

After the final iteration, the final population of candidate research ideas is converted into a structured abstract representation. Following VIRSCI [16], an LLM maps each idea into a standardized template consisting of title, research motivation, method, and idea description, ensuring a consistent semantic structure for downstream analysis.

4 EXPERIMENTS↩︎

4.1 Data↩︎

We evaluate our method using a dataset derived from MAGenIdeas [17], comprising 144 long papers published in ACL 2024. Each instance in MAGenIdeas is centered on a target paper, which contains its title, abstract, and a set of referenced papers that provide an essential background context. This structure mimics authentic scientific discovery, where researchers formulate new ideas by synthesizing existing work with existing literature.

To ensure a fair comparison and accommodate LLMs context window constraints, we standardize the input across all methods. Specifically, each system is provided with the target paper and its top-10 semantically relevant references. This constraint normalizes the “information budget” available to each model, ensuring that performance differences reflect generative capability rather than data access.

4.2 Experimental Configuration↩︎

Within EvoGens, different models are assigned to generation, scoring, and semantic representation roles. Specifically, we use GPT-4o-mini [45] as the generation model for seed idea initialization, rank-based mutation, semantic-aware crossover, and abstract synthesis. GPT-5-mini [46] is adopted as the scoring model in the unified Score Module, where it provides relative score signals for ranking, filtering, and mutation control during the evolutionary process. Consistent with the method design, this scoring model instantiates the unified scorer used during evolution.

In addition, we use all-MiniLM-L6-v2 [47] as the text embedding backbone. Its embeddings instantiate the shared semantic space used for semantic similarity computation, clustering, and parent pairing in the crossover stage.

4.3 Baselines↩︎

To evaluate effectiveness, we compare EvoGens with four representative baseline approaches: AI-Researcher [11], AI-Scientist [20], Future-Idea-Generation [22], and MAGenIdeas [17].

For the first three baselines, AI-Researcher, AI-Scientist, and Future-Idea-Generation, we directly use their official open-source implementations without modifying the core generation pipelines. AI-Researcher is evaluated using its end-to-end idea generation framework, while for AI-Scientist, we extract and evaluate only its research idea generation module, excluding other downstream components such as experiment design and paper writing.

For the MAGenIdeas, we adopt its default experimental settings, using three iterative discussion rounds with four agents. To ensure a comparable number of generated ideas across methods, EvoGens is configured to run for three evolutionary iterations.

4.4 Metrics↩︎

Following the evaluation protocols of AI-Researcher and Nova [11], [12], we examine generated research ideas from three perspectives: Novelty, Diversity, and Quality. Among them, novelty and diversity are used as exploration-oriented automatic proxies that characterize semantic expansion and population spread, rather than fully establishing the scientific usefulness of the generated ideas.

4.4.0.1 Novelty.

  We adopt a semantic similarity–based approach to measure the novelty of generated ideas. Specifically, each generated idea and its corresponding reference papers are mapped to vector representations using the same text embedding model, and the cosine similarity between them is computed.

An idea is considered novel if its maximum similarity to all reference papers is below a predefined threshold \(\theta_n\). Following previous studies [12], [22], we set \(\theta_n = 0.4\). The novelty score is defined as \[\mathrm{Novelty} = \frac{1}{n} \sum_{i=1}^{n} \mathbb{I}\!\left(\max_j \mathrm{sim}(a_i, r_{ij}) < \theta_n \right), \label{n95novelty}\tag{7}\] where \(n\) denotes (Eq. 7 ) the number of generated ideas, \(a_i\) denotes the \(i\)-th generated idea, \(r_{ij}\) denotes its related reference papers, \(\mathrm{sim}(\cdot,\cdot)\) denotes cosine similarity, and \(\mathbb{I}(\cdot)\) denotes an indicator.

4.4.0.2 Diversity.

  Consistent with previous works [11], [12], diversity is measured by the proportion of semantically unique ideas among the generated results. Using the same embedding model and cosine similarity computation as in the novelty metric, we compare each generated idea against the remaining generated ideas. An idea is counted as diverse if its maximum similarity to all other generated ideas is below a duplication threshold \(\theta_d\) (Eq. 8 ). Following prior work, we set \(\theta_d = 0.8\). The diversity score is defined as \[\mathrm{Diversity} = \frac{1}{n} \sum_{i=1}^{n} \mathbb{I}\!\left(\max_{j \neq i} \mathrm{sim}(a_i, a_j) < \theta_d \right). \label{m95d}\tag{8}\] This metric reflects population-level semantic spread rather than direct scientific quality.

4.4.0.3 Relative Quality.

  The quality of generated research ideas is evaluated using a Swiss-system tournament combined with a zero-shot LLM-based ranker, following prior work [11], [12].² Under this protocol, ideas are compared pairwise, and the ranker determines which idea is superior in each comparison. Each idea participates in five rounds of pairwise comparison and receives one point for each win. Let \(s_i\) denote the total tournament score of the \(i\)-th idea. Following prior settings, an idea is regarded as high-quality if \(s_i \geq 5\). The reported quality score in Table 3 is the proportion of high-quality ideas: \[\mathrm{Relative\;Quality} = \frac{1}{n} \sum_{i=1}^{n} \mathbb{I}(s_i \geq 5), \label{r95qua}\tag{9}\] where \(n\) (Eq. 9 ) denotes the total number of generated ideas. This metric is a comparative automatic proxy derived from pairwise ranking, rather than an absolute measure of scientific merit. Accordingly, the quality results reported in this work should be interpreted as relative evidence under a shared automatic ranking setup, not as definitive judgments of scientific validity, feasibility, or long-term impact.

4.5 Implementation↩︎

Considering the cost and evaluation criteria, all methods generate 40 research ideas for comparison in this study. EvoGens is executed for 3 evolutionary iterations with a maximum population size of 40. During parent selection, 30 parent ideas are selected by clustering k-means (k=30) to encourage diversity, retaining the highest-scoring idea from each group. The same clustering-based mechanism is applied during next-generation selection. When the number of candidates is smaller than the cluster count, all candidates are preserved. For each evolutionary operation, both crossover and mutation generate two new offspring ideas. Overall, at most 110 new ideas are generated per iteration.

4.6 Experimental Results↩︎

Table 3 shows that the evolution-inspired generation framework substantially improves the novelty and diversity of the generated research ideas under the current automatic evaluation protocol. In particular, EvoGens exhibits a clear advantage in novelty, which is an important criterion for assessing the potential originality of research ideas.

Compared with the baselines, EvoGens achieves the strongest performance on both Novelty and Diversity, while its quality remains competitive with strong baselines, though not superior to the best-performing quality baseline. This result suggests a trade-off consistent with the exploration-oriented design of EvoGens: the framework primarily promotes conceptual exploration and diversification while maintaining a competitive level of automatic comparative quality.

A turn-level analysis is reported in Table 4, showing the evolution of novelty and diversity across generations for the full EvoGens model. Diversity increases consistently with each turn, with the largest improvement observed from Turn 1 to Turn 2 (0.35 to 0.53), while the gain from Turn 2 to Turn 3 becomes smaller (0.53 to 0.55), indicating diminishing returns in later iterations under the current budget setting. In contrast, novelty exhibits a fluctuating trend and shows a slight decrease in later generations, suggesting that iterative evolution gradually shifts from introducing new variations to reorganizing existing concepts through recombination. Similar behaviors have been reported in prior idea generation frameworks such as NOVA and MAGenIdeas [12], [17]. Given that EvoGens primarily targets diversity and conceptual exploration, we adopt three evolutionary turns as a practical trade-off between exploration gain and computational cost. At this point, diversity has improved substantially, while novelty remains at a competitive level, providing a practical operating point for the generation of early-stage research ideas. The weaker gains after Turn 3 do not necessarily indicate that the search space is exhausted; instead, they more likely reflect diminishing marginal returns under the current operator design and computational budget.

To qualitatively examine how ideas evolve in semantic space, we visualize two representative idea evolution chains using t-SNE, as shown in Figure 3. The plot is not intended to interpret the t-SNE axes themselves, but to illustrate relative semantic displacement during mutation and crossover.

The red chain shows a mutation-only case. It begins with an explorative mutation, which induces a relatively larger semantic shift, and is followed by two conservative mutations that produce smaller local refinements. This pattern is consistent with the intended behavior of rank-based mutation: broader exploration first, then local refinement.

The blue chain shows a mutation-and-crossover case. A radical mutation first produces a larger semantic displacement, after which explorative mutation further reshapes the idea. The final crossover child remains semantically related to its parent ideas while occupying a distinct position, suggesting that crossover in EvoGens performs structured conceptual recombination rather than simple text fusion.

Overall, the two chains show that different mutation modes induce different degrees of semantic change, while crossover yields offspring that are related to, but still distinct from, their parent ideas.

5 ANALYSIS AND DISCUSSION↩︎

5.1 Ablation Study↩︎

To analyze the contribution of each component in EvoGens, we conduct a series of ablation studies by selectively removing the Rank-Based Mutation (RBM) mechanism and/or the Semantic-Aware Crossover (SAC) mechanism. All ablation variants keep the same mutation retrieval budget (\(n=6\) papers retrieved per mutation step) to ensure a fair comparison. Specifically, EvoGens w/o RBM retains the retrieval-plan-based mutation process but removes rank-based differentiation, using a single shared retrieval-plan prompt and a unified temperature setting for all ideas;

EvoGens w/o Crossover removes inter-idea recombination entirely; and EvoGens w/o SAC retains crossover but replaces semantic-aware pairing with random pairing. The results are summarized in Table 5, which reports novelty, diversity, and automatic comparative quality.

5.1.0.1 EvoGens w/o RBM.

  This variant retains the retrieval-plan-based mutation mechanism but removes the rank-based mutation control strategy. All ideas share the same retrieval-plan prompt (Table 15 in Appendix 9) and a unified temperature setting (\(T=0.7\)). Novelty drops dramatically (from 0.40 to 0.06), accompanied by a slight decrease in diversity (from 0.55 to 0.51), while relative quality decreases only slightly (from 0.21 to 0.20). This indicates that the main contribution of RBM does not come from any single radical mutation mode in isolation, but from differentiated mutation control across the population. By assigning conservative, explorative, and radical mutation modes to different rank tiers, RBM provides complementary semantic perspectives that jointly expand the search space. Without such rank-based differentiation, mutation tends to remain more homogeneous, and crossover mainly reinforces convergence toward retrieved references rather than promoting broader exploration.

5.1.0.2 EvoGens w/o RBM & Crossover.

  This variant removes both rank-based mutation control and crossover while retaining a uniform retrieval-plan-based mutation process. All ideas share the same retrieval-plan prompt and temperature setting, with no inter-idea recombination. Both novelty and diversity decline substantially (from 0.40 to 0.05 and from 0.55 to 0.49, respectively), while relative quality also decreases from 0.21 to 0.19. Compared with EvoGens w/o RBM, removing crossover does not reduce novelty further, but it does reduce diversity and relative quality. This suggests that without RBM, crossover contributes little to exploration, yet still helps preserve the overall coherence and competitiveness of generated ideas under the current automatic evaluation protocol.

5.1.0.3 EvoGens w/o Crossover.

  This variant completely removes the crossover operator while keeping the RBM process intact. All ideas evolve solely through rank-based mutation without inter-idea recombination. As a result, both novelty and diversity decrease (from 0.40 to 0.25 and from 0.55 to 0.51, respectively), and relative quality also drops from 0.21 to 0.16. This indicates that differentiated retrieval planning alone is sufficient to introduce substantial semantic divergence even without explicit idea recombination. At the same time, the decline in relative quality suggests that crossover helps preserve coherence and integration across generated ideas, rather than simply adding novelty.

5.1.0.4 EvoGens w/o SAC.

  This variant retains the crossover operator but removes the semantic-aware pairing strategy. Specifically, parent ideas are selected via tournament selection, while crossover pairing is performed randomly, and the full RBM mechanism is preserved. Compared to the full model, diversity decreases moderately (from 0.55 to 0.52), while novelty remains unchanged (0.40), and relative quality also decreases from 0.21 to 0.17. This suggests that RBM plays a dominant role in promoting novelty, while SAC primarily contributes to controlling recombination quality and maintaining balanced semantic coverage during crossover. The drop in relative quality further indicates that the current SAC design is not merely diversity-oriented, but also supports the generation of more competitive offspring under the automatic comparison protocol. At the same time, the remaining gap suggests that semantic-aware pairing is still heuristic, and that more adaptive or quality-aware parent matching strategies remain an important direction for future work.

These ablation results help clarify the functional roles of RBM and SAC within the current automatic evaluation setup, but they should not be interpreted as definitive validation of real-world scientific usefulness.

5.2 Case Study↩︎

Figure 4 presents a representative evolutionary case in EvoGens, showing two seed ideas, the retrieval-grounded mutation process, and the final crossover offspring. Unlike random perturbation, mutation is explicitly mediated by a generated search plan and the retrieved external knowledge.

In the illustrated mutation branch, the system first assigns a mutation strategy and generates a search plan specifying retrieval fields and keywords. This plan is executed through Semantic Scholar, and the retrieved literature is summarized as compressed knowledge cues beside the mutated idea. Guided by this process, the mutated idea “Chaos-Enhanced Evolutionary Idea Mutation” preserves the conceptual evolution framework while replacing linear anomaly modeling with chaos-driven mutation, enabling more radical yet still controlled exploration.

The final offspring, “TRIZ-Evolutionary Cross-Disciplinary Idea Generation”, is produced by recombining complementary components from two parent ideas. It preserves the evolutionary generation framework from one branch while incorporating TRIZ-guided cross-disciplinary retrieval from the other, making cross-disciplinary exploration more structured and controllable. This case shows that mutation in EvoGens is retrieval-grounded instead of random, and that crossover performs mechanism-level conceptual recombination rather than surface-level text fusion.

6 LIMITATIONS↩︎

Despite its effectiveness, EvoGens has several limitations that warrant further investigation. (1) Heuristic Scoring Design.  Although multi-dimensional criteria and aggregation strategies are employed to improve stability, such scores serve only as heuristic fitness proxies for within-population control rather than as optimal or theoretically grounded fitness functions from an evolutionary computation perspective. Designing more robust, interpretable, and scientifically aligned evaluation functions remains an important challenge. We also note that this version does not include a dedicated analysis of scorer variance across repeated LLM evaluations or multi-run search stability. Since both generation and evaluation rely on LLM-based components, assessing such variability remains an important direction for future work. (2) Evaluation Scope.  Current evaluation of EvoGens is limited to scientific idea generation tasks in the natural language processing (NLP) domain. Although this setting is sufficient to show the effectiveness of the proposed framework, further validation in additional research domains remains an important direction for future work. More broadly, all automatic evaluations used in this work provide proxy evidence rather than definitive validation of scientific usefulness. (3) Lack of Human Expert Evaluation.  Due to practical constraints, we did not involve human evaluation of generated ideas. As in existing work, the current assessment mainly relies on automated quality metrics and comparative ranking signals, which may not fully capture the true scientific value, feasibility, or long-term usefulness of research ideas. Incorporating expert judgment remains necessary to better validate real-world scientific utility.

In addition, we do not provide a detailed analysis of computational cost or initialization sensitivity in this version. Although the current settings are sufficient for controlled comparison, understanding these trade-offs more systematically would further strengthen the framework.

7 ETHICS STATEMENT↩︎

This work proposes EvoGens, an LLM-based evolutionary framework for scientific idea generation. The study does not involve human participants, personal data, or sensitive information, and therefore does not raise direct concerns regarding privacy or data protection. A potential ethical consideration concerns the misuse of automatically generated research ideas. Generated ideas may be incomplete, incorrect, or infeasible, and should not be regarded as validated scientific contributions without expert review. Accordingly, EvoGens is designed as an assistant tool to support human researchers rather than an autonomous decision-making system. The framework grounds idea generation in retrieved academic literature and employs multi-dimensional quality assessment to reduce unverifiable or hallucinated content. Overall, EvoGens aims to augment, rather than replace, human scientific creativity. When used responsibly, it can support early-stage scientific exploration without introducing additional ethical risks.

8 Additional Analysis of Evolution Dynamics↩︎

Figure 5 presents a bar-chart comparison between EvoGens and baseline methods in terms of novelty, diversity, and quality. Overall, EvoGens demonstrates clear advantages in both novelty and diversity, indicating a stronger capability to explore and maintain a diverse idea space. Although its quality score is not the highest among all methods, it remains within a competitive and acceptable range, suggesting that the gains in exploration do not come at the cost of severely degraded idea quality.

In addition, we analyze how novelty and diversity evolve across generations for EvoGens (see Figure 6). The results show that novelty stabilizes after the first evolutionary turn, while diversity continues to increase across subsequent turns, with diminishing gains in later iterations. This trend suggests that early evolution primarily contributes to expanding the idea space,whereas later turns help consolidate and redistribute ideas to achieve a more diverse yet stable population.

8.1 Turn-level Evolution Analysis↩︎

We analyze how novelty and diversity evolve across generations for EvoGens and its ablated variants. Table 4 reports the novelty and diversity after each evolutionary turn.

8.2 Effect of Parent Pool Size↩︎

We investigate the effect of the parent pool size used during crossover. As shown in Table 6, using an excessively large parent pool leads to a noticeable degradation in novelty, while smaller to moderate pool sizes maintain stable novelty and comparable diversity. This suggests that overly broad parent selection may introduce redundant or weakly related recombinations, reducing the effectiveness of crossover in generating novel ideas. Based on this observation, we adopt a moderate parent pool size in all experiments to ensure stable evolutionary behavior.

9 Prompts↩︎

This section mainly introduces the prompts used by EvoGens in the generation and evolution of ideas. Specifically, Tables 7, 8, and 9 present the prompts used for the three mutation plans, corresponding to conservative, exploratory, and radical evolution. Table 14 details the prompt for idea evaluation, while Table 10 summarizes the prompt used to guide mutation generation. Finally, Table 11 provides the prompt employed in the semantic-aware crossover stage.

6pt

6pt

6pt

4pt

6pt

6pt

6pt

6pt

6pt

References↩︎

---

1. In this work, literature retrieval is implemented using the Semantic Scholar API.↩︎

2. In our implementation, the zero-shot ranker is instantiated with DeepSeek-V3.↩︎