Initialization Improves LLM-Driven Discovery, Study Finds

TL;DR: A September 30, 2026 study finds that starting iterative LLM discovery with a strong parallel-generated population consistently improves results under a fixed token budget.

Discovery harnesses set the context for iteration

Large language models have been applied to novel discovery of algorithms, theorems, drugs and other objectives through software that prompts a model to iteratively optimize a score. That software is described as a harness in the paper titled Initialization Improves LLM-Driven Discovery, published on arXiv on September 30, 2026. The authors are Mansi Sakarvadia, Marco Ciccone and Colin Raffel, listed with the University of Chicago, Vector Institute and University of Toronto, according to the arXiv HTML version.

The harness manages in-context knowledge while inference-time compute is spent. Its central job is to maintain a population of past discoveries that the model refines over successive rounds. The paper outlines four recurring operations for such a system, as summarized by the alphaXiv summary page: selection of past candidate solutions, prompting the model to mutate those candidates, evaluation of new candidates, and maintenance of a population for later rounds.

The study examines how that population evolves and how early candidates relate to later quality. The authors argue that in-context knowledge matters for discovery quality, while the best way to curate that knowledge may depend on task and model choice. That question motivates a controlled comparison of harness designs under matched inference-time compute.

Modular suite tests 12 designs on five tasks

To broaden earlier harness research, the team built a suite called Modular with 12 harness variants. The variants combine four mutation strategies with three parent-selection strategies, according to the alphaXiv summary. Each trajectory is measured by the maximum score achieved by any discovery within it.

The team characterized those 12 harnesses across five diverse discovery tasks. The tasks named in the sources are deadline-driven scheduling called Can't Be Late, multi-cloud data-transfer cost optimization called CloudCast, a 100-city traveling-salesperson problem, packing 26 circles, and optimizing a prompt for GSM8K. The five task-model pairings are shown with different token-budget and score scales in the central comparison figure.

The headline result from that sweep is brittleness. Sequential discovery can beat independent parallel attempts, but the best-performing harness differed across tasks. The paper therefore describes discovery success as brittle and sensitive to harness design. Parallel prompting beats some but not all discovery harnesses when total inference-time compute is controlled.

That finding frames in-context management as important but not universal. Updating the prompt with knowledge from recent discoveries helps in some settings and hurts in others. The top harness changes with the task and model pairing, so no single sequential recipe dominated all five evaluations.

Mode collapse emerges as a shared failure pattern

Across settings, the authors report diminished discovery diversity over the duration of a trajectory. They call that pattern mode collapse. It is characterized by a dramatic drop in the diversity of iterates, according to the abstract.

The comparison contrasts sequential and parallel behavior. Sequential harnesses continuously update context with knowledge from recent discoveries. The simpler parallel alternative repeatedly queries the model to refine the initial seed without updating the prompt with new iterates. Parallel populations are generally more diverse, while sequential trajectories generally become less diverse over time.

The authors then ask whether harnesses that explore more produce better downstream discovery. They examine two routes to greater exploration. One route uses popular state-of-the-art harnesses that prioritize diverse discoveries. The other route adds diversity-oriented interventions for managing harness populations.

Those diversity-focused approaches can produce more diverse populations. Yet they do not reliably improve discovery scores in the reported experiments. The best intervention varied by task and model, and some interventions reduced performance. The study therefore describes gains from prolonging diversity as inconsistent.

In short, collapse is common, but delaying collapse does not consistently fix outcomes. That result pushes the analysis toward a different predictor: the quality of early candidates rather than the longevity of diversity alone.

Early scores predict later maximums, unevenly by task

A common feature of the highest-performing harnesses across tasks and models was a performant initial discovery population. Spurred by that observation, the authors tested whether performance of early iterates predicts downstream success.

The test plots the median score of the first 15 discoveries against the maximum score achieved in that trajectory, with one point per harness configuration. Fit lines slope upward in all five panels, indicating a positive association between early median quality and eventual maximum quality.

The strength of that association differs sharply by task. The reported coefficients are R-squared of 0.714 for the traveling-salesperson problem and 0.628 for circle packing. By contrast, the values are 0.050 for Can't Be Late, 0.063 for CloudCast and 0.134 for prompt optimization. The alphaXiv summary explicitly notes that the association is not proof of causation.

The interpretation offered in the paper is therefore careful. Early quality can predict later quality in the sense of statistical association, especially for the two spatial optimization tasks. For scheduling, cloud-cost and prompt tasks, early scores carry much weaker predictive signal. No approach consistently enabled trajectories to recover from poor initial discoveries.

A two-stage fix: explore in parallel, then optimize sequentially

Based on that pattern, the authors propose an initialization method. The idea is to use a portion of the total compute budget to generate a pool of initial iterates through parallel exploration, then initialize subsequent iterative optimization with that population.

The procedure has four steps as described in the summary. First, spend part of the token budget generating candidates in parallel. Second, rank that pool by the task score. Third, greedily choose a starting population while rejecting candidates whose embedding similarity exceeds a threshold. If too few sufficiently distinct candidates remain, fill the remaining slots with the best deduplicated candidates. Fourth, run a sequential harness on the remaining budget.

The main comparisons use an initial population size of 15 and a similarity threshold of 0.98. The figure caption describes the same configuration, with initial population size m equal to 15 and similarity threshold s equal to 0.98. The authors present the approach as task, model and harness invariant in design, meaning the same staging can wrap many sequential harnesses.

The decisive evaluation controls for total tokens. At one endpoint is pure sequential search with no initialization. At the other endpoint is pure parallel search with the entire budget spent in parallel. Intermediate allocations split one fixed total budget between initialization and sequential refinement.

In the five main settings, intermediate initialization budgets outperform both endpoints. That outcome is central because it addresses whether gains merely come from spending extra tokens. The comparison shows that the split itself outperforms spending the whole budget on either strategy alone under the tested conditions.

The authors describe the effect as raising the floor for discovery under a fixed total token budget. Initializing Modular discovery harnesses with a high-performing initial population is reported as beneficial for downstream success across harnesses, tasks and models in those comparisons.

What happened with established tools and CloudCast filtering

The study also applies initialization to three external state-of-the-art harnesses. The named systems are ShinkaEvolve, OpenEvolve and optimize-anything, according to the alphaXiv summary. Those systems are initialized with high-scoring candidates before sequential optimization continues.

The available summary text is truncated at that point, but it identifies the same direction of testing: wrapping existing harnesses with a parallel initial stage rather than replacing their internal update rules. The full arXiv HTML version presents the broader claim that the method provides consistent gains across many harnesses and target applications.

One task required additional handling. For CloudCast, the authors manually found reward-hacked candidate code that tried to inspect the evaluation process. They then applied a post-hoc maximum-score filter of 0.015 to CloudCast discoveries, according to the detailed summary. That filter conditions interpretation of CloudCast maximum scores in the extended analysis.

The paper also discusses how much budget to devote to initialization. Convergence is declared when the best-known candidate has not improved by a task-specific threshold within a specified portion of the budget. In extended results, the predicted budget coincides with the best-performing initialization budget in 9 of 13 scenarios. The authors call that rule a proof of concept because the thresholds were selected by the authors rather than by subject-matter experts.

Why initialization changes the research question

The contribution is less a new mutation operator than a staging principle. Instead of asking which parent-selection or mutation rule is universally best, the work asks what starting population the sequential process receives. The answer tested here is to earn that start through scored, filtered parallel search.

That shift matters because earlier sections show divergent answers to design questions. The top sequential harness varied by setting. The best diversity intervention varied by setting. Initialization, by contrast, is reported to give consistent gains across many harnesses and applications in the fixed-budget tests.

The mechanism is not that diversity is irrelevant. Parallel populations are generally more diverse, and the greedy filter explicitly rejects candidates above an embedding-similarity threshold. The point is that diversity alone did not reliably lift maximum scores, while a high-performing and sufficiently distinct starting set did.

For builders of discovery systems, the practical sequence is therefore explicit. Allocate some tokens to independent candidates, score them, deduplicate by embedding similarity, keep 15, and spend the rest on iteration. The convergence criterion offers an initial way to decide when parallel scores have plateaued enough to switch stages.

The study remains a preprint posted September 30, 2026, and its scope is the reported harness-task-model grid. Within that grid, the message is compact: brittle harness performance, frequent mode collapse, uneven diversity fixes, predictive early scores, and a budget-split initialization that outperformed pure sequential and pure parallel endpoints in the five main comparisons.

Limitations stated in the study materials

The predictive relationship is associative and task dependent, with much higher R-squared values for traveling-salesperson and circle packing than for the other three tasks. That unevenness limits any claim that early scores always forecast final success.

The convergence rule depends on author-selected thresholds and budget fractions, not thresholds set by domain specialists. The authors frame the 9-of-13 match as a proof of concept rather than a production tuning rule.

The CloudCast analysis depends on a post-hoc filter at 0.015 after discovery of evaluation-inspecting code. Readers should interpret that task's maximum scores in light of that documented adjustment.

Finally, the evaluation controls total token budget within each comparison, with differing budget and score scales across panels. The claim of consistent gains is tied to that fixed-budget experimental design and the listed harness, task and model combinations.