Truace
Subscribe

AI godfathers warn of runaway ‘intelligence explosion’

Two of the “godfathers” of modern AI and senior executives at OpenAI and Anthropic have warned governments to prepare for an AI “intelligence explosion”, which they say could be the most consequential technological development in history. A report co-authored by the Nobel laureate Geoffrey Hinton and the Canadian computer scientist Yoshua Bengio, considered godfathers of modern AI for their work in the field, urges politicians to act now before there is runaway progress in the technology. The authors say they ar…

By the Truvace desk2 min read1 sourceJournalismTRV-2026-1537· cite & verify
AI godfathers warn of runaway ‘intelligence explosion’

In brief

More than 20 authors, including Geoffrey Hinton, Yoshua Bengio, Anthropic co-founder Jack Clark and OpenAI chief scientist Jakub Pachocki, published a paper on automated AI research and development. It argues that AI systems improving themselves could trigger an abrupt acceleration of progress, and it asks governments to act before that happens.

The paper matters because it links a technical trend to governance: transparency, auditing and emergency planning are proposed before any such acceleration begins. The authors themselves call the impacts uncertain, so whether an intelligence explosion occurs, and how quickly, remains an open question.

Main points

  1. The paper defines an intelligence explosion as a dramatic AI-driven acceleration of AI progress, compressing years of advances into months or less.
  2. Automated AI R&D is named as the most likely trigger, since AIs already contribute to improving their own technology.
  3. Anthropic says AI now produces 80% of its own code, and OpenAI uses autonomous agents in areas such as training new models.
  4. Proposed measures include transparent R&D progress reports with embedded independent auditors, limits on how fast AI can improve, and emergency response plans.
  5. The authors say productivity gains from AI R&D automation have not yet reached the threshold needed to trigger an explosion, but newer systems are likely approaching it.

The problem

A report by leading AI researchers warns that automating AI research could compress years of progress into months, threatening human control and weakening checks on power, so governments need to prepare.

The rundown

The authors say that once AI reaches expert-level capability at AI R&D, a single developer could run a workforce equivalent to millions of top human researchers. They add that improved systems can be deployed quickly once built, which is why they see automation of AI R&D as the likeliest path to an explosion.

The paper names three risks: biological and cyber threats could develop faster than countermeasures, humans could lose the chance to control systems as they step back from AI R&D, and states could turn a modest lead in areas like cyberspace into a decisive one. It also notes that AI could accelerate medical breakthroughs and technological leaps.

Policy options include limiting how fast an AI can improve in a given period, working with datacentres to enable pausing certain AI R&D projects, and fully isolating automated R&D systems so they cannot escape human control. Anthropic and OpenAI have agreed to independent evaluators assessing their models. The article says R&D projects that take humans months could be fully automated by 2028, which is a projection rather than an observed result.

Sources

  1. JournalismThe Guardian2026-09-28

The debate