Self-improving loops set records without inventing anything new

Automated field notes from AI Engineer World’s Fair 2026, agent-distilled from auto-captioned recordings; names and quotes are approximate. Method and source links in the overview.

On OpenAI’s Parameter Golf benchmark, Weco’s agent Aiden set seven records against the best human’s three, reached an H-index of 10 against the human 7, and did it on no more than 4% of the compute while claiming about 15% of the records. Recursive.com reported beating NVIDIA’s best CUDA kernels and improving nanochat and nanoGPT training records within days. “Autoresearch,” the outer loop that studies and improves the primary system, was the headline idea of the conference, and the evidence that it works right now is real.

The caveat arrived from the same stage. Prime Intellect, who run an open speedrun benchmark exactly so third parties can check claims like these, found that their record-setting agents produced no truly novel optimizers; the wins were recombinations, “+1” compositions of known techniques applied with superhuman patience. That distinction, records without invention, is the sharpest correction anyone issued all week, and their insistence on open third-party benchmarks for verifying recursive-self-improvement claims deserves adoption before the marketing cycle gets to this word.

Reflection beats rollouts on sample efficiency

The technical result underneath the movement came from GEPA. Reinforcement learning collapses a rich rollout, the chain of thought, the tool calls, the error messages, into a single scalar reward; GEPA instead has a model reflect on the whole trace in text and edit a prompt, so one natural-language change (“10-line summary” instead of “one-line summary”) produces a behavior shift that gradient descent would need thousands of updates to match. One round of reflection on three data points matched twice the gains that GRPO achieved after 25,000 rollouts. On an AMD NPU with near-zero public documentation, a GEPA loop took kernel utilization from 4.25% to 30.52%, discovering along the way that a shipped header file simply did not work on the new hardware. A four-line chain-of-thought program evolved into a six-step agent that moved ARC-GI from 32.5% to 89.5%, and Databricks reported a 90× cost reduction from tuning an open model past a frontier one.

The mechanism that makes the search work is worth naming: a Pareto pool that keeps every candidate that wins on even one training example, rather than only the top scorer, accounts for over half the gains, because a model iterating on its own best answer gets stuck. Anything expressible as scorable text, prompts, kernels, harnesses, scheduling policies, is now an optimization surface.

The skeptics have the sharpest data

HumanLayer’s Jack gave the talk I have thought about most since. They ran the “lights-off software factory,” nobody reads the code, just keep the queue full, for real in July 2025, and it broke: unsolvable issues, outages, accumulating slop. His diagnosis is structural rather than operational. RL rewards are binary, the test passed or it did not, and there is no reward channel for architecture, so models learn the hacks that make tests pass: the needless try/except, the cast, the commented-out test. Verifying maintainability is orders of magnitude harder than verifying correctness because bad architecture bills you months later, far past any reward horizon, and models have accordingly improved enormously on greenfield tasks while brownfield codebases still degrade after three to six months of agent maintenance. His fix is unfashionable: model-assisted upfront planning, product review, then architecture contracts, then typed program design, so that roughly 30 minutes of planning preserves your ability to read every line. If a model knew what good code looked like, he noted, it would just write it.

His colleague Kyle supplied the constructive version, loops designed like control systems: an ast-grep sensor measuring violations out-of-band where the agent cannot disable it, a committed baseline, an agent actuator driven by hand-written golden patterns, and adaptive flow control that emits at most one small reviewable PR per day on existing CI. The contrast with a bash loop yolo-ing 40K-line PRs is the whole argument in miniature.

The research OS is just files

The quietest talk in this theme was the one I could verify from my own chair. Pauline and Luis’s personal research OS is plain markdown files, sources, comparisons, implementations, that both humans and agents read and extend, chosen over NotebookLM (not agent-native, not owned) and over RAG infrastructure (not inspectable, not editable). Knowledge compounds because nothing is trapped in a session. The notes you are reading came out of exactly this arrangement, an Obsidian vault synced to a headless machine where agents work on it as files, and the compounding is not hypothetical; these six essays were the second thing the vault produced, a day after it was set up.

Where this settles, I think, is that the loop is real and the ceiling is unproven in both directions. Weco’s records, GEPA’s sample efficiency, and Jina’s overnight retrieval discoveries are results you can check; so is HumanLayer’s broken factory. The bet worth making now is the one every credible speaker was already making: run the outer loop on everything cheap to verify, and keep a human reading the code everywhere verification runs out. The interesting frontier is whichever of those two territories moves first.