AI co-scientists and autonomous discovery loops

The loop runs without a human in each turn only when three things exist: auto-reset of the environment, auto-improve of the artifact, and auto-evaluate of the result. Where all three are present, tasks have been driven from 0% to near-total success autonomously. Where one is missing, a human is back in every turn and the loop is a demo.

A breakthrough is by definition outside the training set, so thinking harder about the literature does not substitute for running the experiment. The history cited against “thinkism” is pointed: the Nobel that mattered for superconductivity was for liquefying helium, a capability build, with the phenomenon stumbled into; a headline superconductor came from trying tens of thousands of materials rather than from theory. Mathematics and theoretical computer science are a different regime.

The systems that work falsify aggressively, often by rating candidate ideas against each other in a tournament before committing compute. Progress arrives in jumps, not smoothly, one autonomous run had 240 experiments between its last two improvements, which makes cheap selection the difference between a loop that pays and one that burns.

A frontier lab reports seeing no particularly successful auto-research rollout with today’s models: progress appears early and then stalls, because research is serial and long-horizon. And the field’s own framing is that autonomy on grand challenges is gated by trust rather than by capability.

Also: AlphaEvolve; AlphaFold and AlphaGenome; the FrontierScience benchmark; PutnamBench; METR time-horizon measurements; Goodfire Silico; Prima Mente Pleiades; and “the lab” (open source, workspace, dashboard, one git branch per idea).

The end-to-end research pipeline is the shape Assay’s dreaming pass was designed to have: out-of-band idea generation, agents arguing the candidates, harness-gated execution, and a human accept or reject at the end. None of that runs today.

Auto-reset is worktree discipline: every worker gets a fresh worktree off the mainline. Auto-improve is the worker desks. Auto-evaluate is the Verify rows executed after merge by someone other than the implementer, per the lifecycle. The 0%-to-99% result is evidence that loops with all three legs saturate their task, which argues for closing the remaining gap rather than adding a fourth mechanism. The open-source “lab” is the strongest outside validation of Assay’s core bet, branch per idea, sandboxed writes, findings written down, arrived at independently, sandbox rationale included.

The ELO idea tournament is a candidate mechanism for choosing among competing brief plans or proposed memory updates, where Assay currently has no selection step at all. The iterate-with-the-environment stance is the external argument for why Verify rows run against real CI rather than against model confidence. And the time-horizon cadence is a standing review trigger: if reliably-executable task length doubles roughly every six months, the granularity ceiling on a brief should be revisited on that cadence rather than treated as fixed. The dissent is worth holding too, the successes reported here are mostly bounded, verifiable problems, which is exactly the class a brief is.

• Whether any auto-research result is independently reproduced, the striking numbers here are each single-team single-run reports.
• Whether the dissent softens: a long-horizon research rollout that does not stall would be the signal that the loop generalizes past bounded tasks.
• Whether idea-selection mechanisms (ELO tournaments, verifier-fed replanning) show up outside research settings, which is what would make them portable to plan authoring.