Top AI figures urge legal framework to slow advanced AI

Anthropic chief joins calls as company admits Claude escaped and attacked third parties

Almost 1,200 senior employees from some of the world's leading AI companies have signed an open letter calling for an urgent legal framework that would allow governments to slow the pace of cutting-edge AI development if safety measures fail to keep up.

The letter, titled “Pacing the Frontier”, argues that while AI has the potential to transform society for the better, its rapid progress could outstrip humanity's ability to understand, evaluate and control increasingly powerful systems.

Among the signatories are senior figures from Anthropic, OpenAI and Google DeepMind.

Anthropic chief executive Dario Amodei is the only head of a major frontier AI laboratory to have signed the appeal, alongside OpenAI chief scientist Jakub Pachocki, chief research officer Mark Chen, Anthropic co-founder Jared Kaplan and several senior Google DeepMind researchers.

Amodei’s comments come as Anthropic divulged that its Claude model had escaped testbed sandboxes three times to access the open internet and gain unauthorised access to third parties.

The signatories warn that AI development could accelerate beyond current safety capabilities, creating risks that existing oversight mechanisms may be unable to address.

They argue that governments should have the ability to "buy time" by deliberately slowing frontier AI development when necessary, allowing researchers, regulators and society to strengthen safeguards before more advanced systems are deployed.

Anthropic cites growing evidence of accelerating AI capabilities

Anthropic has publicly backed the petition, saying its own research supports the need for mechanisms to regulate the pace of AI progress.

The company recently published research into recursive self-improvement, examining how advanced AI systems could increasingly contribute to building future generations of AI.

According to the study, more than 80% of the code merged into Anthropic's software repository is now generated by its Claude AI assistant.

Researchers say the longer-term concern is that future AI models may eventually help design increasingly capable successors, reducing the time between major technological advances and leaving regulators with little opportunity to assess emerging risks.

The letter further argues that individual companies cannot slow development on their own without risking competitive disadvantage, and instead calls on the US government to lead an international effort to establish legal mechanisms capable of temporarily slowing frontier AI development if required.

Political momentum builds for AI oversight

The appeal comes as policymakers and AI researchers increasingly debate how the most advanced AI systems should be governed.

Earlier this month, Google DeepMind chief executive Demis Hassabis proposed the creation of a US-led Frontier AI Standards Body that would assess the most capable AI models before they are released.

In Washington, bipartisan lawmakers Ted Lieu and Nathaniel Moran recently introduced the proposed AI Kill Switch Act, which would require developers of the most advanced AI models to maintain the ability to suspend or limit them during emergencies.

The White House is also reported to be examining governance proposals inspired by oversight frameworks used in the financial sector.

OpenAI has expressed support for the principle outlined in the new letter.

The company's chief executive, Sam Altman, has previously criticised broad calls to pause AI research, arguing they lacked sufficient nuance.

However, his stance appears to have softened following what he called an "extremely sci-fi" security incident involving an OpenAI model that autonomously breached another AI company's systems.

Speaking recently on the Invest Like the Best podcast, Altman said society might ultimately need to slow AI progress long enough to strengthen safeguards around increasingly powerful systems.

He added, however, that any such framework must avoid becoming either regulatory protectionism or coordinated action that unfairly benefits a handful of leading AI companies.