Geoffrey Hinton told reporters on Capitol Hill on September 17 that AI is now designing better AI, and that Congress has about a year to act. The term he used is recursive self-improvement: a system that improves the process that builds its successor, so each gain makes the next gain easier to find.

As of October 2026, the demonstrated version of that loop covers one layer. AI agents rewrite the software wrapped around a model and measurably improve it. The model’s weights, the compute budget, the scoring rules and the decision to keep going all stay in human hands.

Hinton is describing the far end of the idea, an intelligence explosion. The experiments describe the near end. Most public discussion treats the two as one thing.

What recursive self-improvement actually means

Recursive self-improvement means an AI system improves itself: it generates the “machinery” that produces its own future improvements. The result of one round feeds back into the process that carries out the next.

An example is an AI research agent that writes code and tests possible solutions. Initially, it chooses experiments using a procedure its developers supplied. It then discovers a better way to select experiments, tests that change and incorporates it into its research process. The updated agent uses the new procedure to search for further improvements—including further changes to how it conducts research.

That feedback is the recursive part. The system being improved also participates in the process of improvement.

In practical implementations, this can involve generating candidate changes, running them in a testing environment, measuring their performance and retaining successful versions. The changes might affect how an agent remembers previous experiments, allocates its computing budget, checks results or selects its next action. A broader system could also modify training procedures, model architectures or weights, the numerical parameters adjusted during training.

The word “self” therefore needs a defined boundary. An AI agent can include a model, software controlling its actions, memory, tools and an evaluation process. Changing one of those components can improve the overall system while leaving the underlying model unchanged. A claim of self-improvement should specify which components changed and which remained fixed.

The immediate benefit is potentially better research per unit of time or compute. A system might discard unsuccessful approaches earlier, run more useful experiments or reuse previous findings more effectively. Whether that produces faster progress depends on the quality of the changes, the reliability of evaluation and how difficult the remaining problems are.

Recursion alone does not establish acceleration. An improved research process might produce another small improvement, then encounter diminishing returns. It might also introduce errors. An intelligence explosion is the stronger hypothesis: improvements to research capability feed back strongly enough to produce a sustained, rapid acceleration in AI progress.

How the meaning has broadened

The underlying idea has remained recognisable for decades. In 1965, I.J. Good argued that a machine exceeding human intellectual capabilities could also exceed humans at designing machines. It could therefore design a better successor, which could continue the process. His argument connected machine design directly to an intelligence explosion. (sciencedirect.com)

More recent experiments have applied the terminology to smaller, bounded systems. Researchers can test whether software improves its own search procedures without first building a machine that exceeds human intelligence generally.

The STOP research, first released in 2023, illustrates this shift. Its authors used a language model to improve a program that used the same language model to improve code. The improvement procedure became an object of its own optimisation. They described this as recursively self-improving code generation, while explicitly acknowledging that leaving the language model unchanged meant it was not full recursive self-improvement. (arxiv.org)

Weco uses a graded framework that distinguishes autonomous improvement, improvement that outperforms human engineering, improvement of the improver itself, and feedback strong enough to overcome diminishing returns. In that framework, a system can demonstrate a limited level of recursive self-improvement without demonstrating an intelligence explosion. (weco.ai)

There has therefore been a broadening of usage rather than one universally agreed change in definition. The term now covers both a long-standing hypothesis about successive generations of increasingly capable machines and experiments that improve particular components of an AI research system.

The questions that distinguish those claims are concrete: what changes, does the changed system help produce the next change, and does that feedback measurably improve the rate or efficiency of further progress? A higher benchmark score answers only part of that inquiry.

Hinton’s three weeks

On September 17 Hinton briefed senators at a private session convened by Bernie Sanders. Afterward he told reporters that “AI has now reached the point where AI is designing better AI” and that it will get out of control unless something is done. Elizabeth Warren left the room talking about agents that are beginning to replicate themselves. John Kennedy, the one Republican who attended, talked about AI becoming an independent species.

On September 28 the Cambridge Programme on AI Science & Policy published What if automating AI R&D triggers an intelligence explosion? It has more than 20 authors, including Hinton, Yoshua Bengio, Andrew Barto, OpenAI chief scientist Jakub Pachocki, Anthropic co-founder Jack Clark and Microsoft’s Eric Horvitz. The paper defines an intelligence explosion as AI-driven acceleration that packs years of AI progress into months or less.

The paper’s mechanism has two steps. A few thousand people do leading AI research today, and AI systems can be copied and run in parallel, so automating the work adds the equivalent of millions of researchers. Those systems then help build better successors, and the workforce grows again. The authors list the frictions that could stall it: diminishing returns to research labour, tasks that resist automation, and compute limits.

On October 2 Hinton posted the paper with the comment that the idea is old but until very recently “it did not seem imminent,” and that many leading researchers now think it may happen soon.

Weco’s loop: the latest development

Weco AI posted Recursive self-improvement of AI research agents on September 22, and it is circulating on X as the empirical result that ends decades of abstract argument. Weco’s own July blog post called it the first evidence of recursive self-improvement.

The system, AIDE², runs two nested loops. An outer agent, Weco’s production research agent running on Claude Opus 4.7, proposes rewrites to the code of an inner research agent running on Gemini 3 Flash. Each rewrite is scored on AI R&D tasks using held-out data the inner agent never sees. A rewrite is kept only if it beats the current best, and the winner becomes the agent the next round edits.

• Over eight days the loop proposed 99 rewrites and accepted seven. Two repeat runs accepted two and four.
• The score rose from 0.703 to 0.778. Weco’s human-built production agent, the product of two years of engineering, scores 0.749 on the same test.
• On four outside benchmarks, including a weather-forecasting task unlike anything in the selection set, the final agent matched or beat the human-built one.
• Reward hacking, where an agent inflates its score without doing the real job, fell from 55% of test cases to 32%. The human-built agent sits at 39%. The loop was never scored on it.
• The agent found a broken scoring script and repaired it. Exploiting it was the other option.

Both models stayed fixed for the whole run. The loop rewrote the harness: the code that controls how the agent searches, what it remembers and how it checks its work. The seven accepted changes were a search policy that keeps several strategies alive, memory that compresses the agent’s growing history, and guards against lucky one-off scores. Weights, training methods and chips sat outside the experiment.

The paper includes an “ignition test” for the property the word recursive depends on: an improved agent that is better at improving agents than the one that produced it. Weco put a self-improved agent in the outer seat for three runs and compared it with the human-built agent. The averages finished at 0.780 and 0.782, with the human-built agent slightly ahead, and the authors call the result inconclusive. More runs cost too much to buy.

The result, stated at its actual size: an agent improved the scaffolding of another agent seven times, under scoring rules people wrote, and the gains held up on outside tests. The improved agent took the driver’s seat and held its level. Whether it drives better is unmeasured. The authors add that the discovered agent is complex and hard to interpret, and may carry dead code from earlier rounds.

Three components, one name

1. AI-assisted AI research. People set the direction. AI writes the code, runs the experiments and drafts the analysis. This is daily practice at every major lab.
2. Automated AI research. A closed loop proposes changes, tests them and keeps the winners with no person approving each step. AIDE² sits here, at one layer, inside a test people designed.
3. Intelligence explosion. A closed loop where each generation is better at improving the next, so progress accelerates. Hinton’s warning and the Cambridge paper are about this one. Weco’s ignition test is the experiment built to detect it.

The loop also closes at different depths of the stack.

An intelligence explosion needs the loop closed at the model layer at minimum, with acceleration. The published evidence is a closed loop at the harness layer, with the one acceleration test inconclusive.

Sixty years of the same idea

Schmidhuber’s proofs turned out to be impractical, so every system since swaps proof for testing: run the rewrite, score it, keep what wins. That swap is why the last three years produced experiments after six decades of argument. It is also why the scorer became the most important part of the system.

A research clock that outruns the oversight clock

The Cambridge authors name three dangers: biological and cyber threats that outpace defences, people losing the ability to steer systems as they step back from the research, and states converting a small lead into a decisive one. Their recommendations are progress reports from developers, auditors embedded inside labs, and a government capacity to pause AI research in data centres. Their warning, as quoted in coverage: “Once an intelligence explosion begins, the window for action may close.”

All three dangers come from speed. Regulation, audits, procurement and corporate planning run on quarters and years. A research loop that runs in days leaves those processes reviewing systems that have already been replaced.

The commercial stake is closer to hand. Weco’s eight-day run matched an agent that took its engineers two years to build, and the authors describe the effect as moving part of the bottleneck from expert engineering time to compute. For any company building or buying AI agents, harness quality is becoming something purchased with compute. That shifts advantage toward whoever holds the chips and the test suites.

The distance left

Compute. Every candidate has to be run to be judged. Weco reports that evaluation dominated the cost of its loop, and that it could afford too few runs to settle its own central question. A loop that needs more chips to test each idea moves at the speed of chip supply.

Verification. A self-improvement loop is as good as its scorer. Weco’s scorer was written by people and hidden from the agent. A July survey of the field found that demonstrated self-improvement is strongest where a formal verifier exists and weakest where a system grades itself, and that the failures (self-confirming loops, model collapse) follow from weak verification. Research direction, the choice of what is worth improving, still rests on human judgment.

Noise. Weco reports that noise compounds across the two loops. One falsely accepted rewrite becomes the new baseline and can derail the search that follows.

Diminishing returns. Ideas get harder to find as a field matures, the pattern Bloom and colleagues documented in 2020. The Cambridge paper reduces the whole question to one parameter, r. Above 1, each round of improvement makes the next arrive faster. Below 1, the loop fades.

Legibility. Weco’s improved agent is hard for its own builders to interpret after seven rewrites. Each generation built by the last is harder for people to audit.

Everything outside software. Fabs, power, cooling and data arrive on construction schedules.

Seven conditions for a closed loop

The Cambridge authors write that AI systems are on track to automate most AI R&D work within a few years, and they treat the explosion that might follow as possible and uncertain. The published experiments support a narrower statement: automated improvement at the harness layer is real and repeatable, acceleration is unmeasured, and the model and hardware layers remain human-directed.

An intelligence explosion of the kind Hinton describes requires all of the following.

1. An improved system that measurably improves itself faster than its predecessor did. A passed ignition test, replicated by a second team.
2. The loop reaching weights and training methods, past the scaffolding.
3. Scoring the system cannot game, covering research direction as well as benchmark tasks.
4. Returns above the r = 1 line, sustained across many generations.
5. Compute supply that keeps pace with the loop’s appetite for experiments.
6. Goals that survive each handoff from one generation to the next. Weco’s drop in reward hacking is one favourable data point from one run.
7. An organisation that chooses to remove the human checkpoint.

Each condition is an engineering or institutional target that can be measured. As of October 2026 the first is open after one inconclusive test, and the rest have partial evidence at most. The idea is coherent and the direction of travel is toward it. The arrival date depends on r and on chips, and the evidence so far fixes neither.

Signals to track

• A replicated ignition test. A self-improved agent that beats its maker at self-improvement, across enough runs to be statistically solid, is the result that changes the picture.
• Loops that touch training. Self-improvement results that modify weights, architectures or training methods, with held-out validation.
• Lab disclosures. The share of internal AI research run by agents without human direction. The Cambridge paper asks governments to require this reporting.
• Harness search as a product. Vendors will sell self-improving agents on results like Weco’s. The buyer’s questions: who wrote the scorer, what was held out, and how many runs back the claim.
• The policy clock. Hinton gave Congress about a year. A pause capability for data-centre research is now a written proposal with lab scientists’ names on it.

Limited forms of recursive self-improvement are already being demonstrated. What remains unestablished is the stronger claim: that improved systems reliably become better at producing further improvements, driving sustained acceleration toward an intelligence explosion. Weco’s experiment does not close that gap, and Hinton’s warning does not supply the missing evidence. His standing gives the possibility enormous public attention, but expertise is not experimental confirmation. When demonstrated software improvements and a hypothetical intelligence explosion are discussed under the same label, the evidence for the former can make the latter sound further advanced than it is. We have working improvement loops and an unresolved question about how far they can compound. The public conversation needs to preserve that distinction, regardless of whose name accompanies the prediction.

Sources

• NBC News, “‘Godfather of AI’ warns Congress has ‘maybe a year’ left to regulate AI”, September 17, 2026
• Chan et al., What if automating AI R&D triggers an intelligence explosion?, Cambridge Programme on AI Science & Policy, September 28, 2026
• Geoffrey Hinton, post on X, October 2, 2026
• Srikanth et al., Recursive self-improvement of AI research agents, Weco AI, arXiv, September 22, 2026
• Weco AI, AIDE²: The First Evidence of Recursive Self-Improvement, July 14, 2026
• Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops, arXiv, July 2026
• Resultsense, coverage of the Cambridge paper, September 29, 2026