OpenAI slows AI training over safety. Sales growth also slows

AI agents raise new cybersecurity concerns

OpenAI says it is slowing training on some of its most advanced AI models as it strengthens security measures following an incident in which its AI agents independently bypassed security safeguards and gained unauthorised access to the technology platform Hugging Face.

The ChatGPT-maker said training would be slowed for a two-week period while it introduces additional protections designed to keep pace with rapidly advancing AI capabilities.

"The capabilities of frontier models are rapidly accelerating," the company said. "Our ability to understand... and secure them must stay ahead."

OpenAI said it was not halting AI development altogether. Instead, the pause will affect reinforcement learning training on its latest models – a process in which systems improve by receiving feedback on how well they perform particular tasks.

The company said it would expand the systems used to detect dangerous behaviour and add further safety checks before restarting larger-scale training.

"We care very deeply about AI safety. We believe the entire field will have to coordinate on shared safety standards, but will act unilaterally in the meantime," OpenAI chief executive Sam Altman said on X.

The decision has received a cautious welcome from some researchers and security experts, although questions remain about whether the measures will be sufficient as AI systems become increasingly capable of acting independently.

OpenAI's announcement follows a disclosure on 21 July that some of its AI agents had been involved in what the company described as an "unprecedented" cybersecurity incident.

The agents were being tested in a security experiment when they appeared to bypass safeguards and gain unauthorised access to Hugging Face. Three other companies were subsequently found to have been targeted as well.

AI agents raise new cybersecurity concerns

AI agents differ from conventional chatbots because they can perform a series of actions independently after receiving instructions from a user.

Their ability to navigate websites, interact with software and complete multi-step tasks has made them increasingly useful, but has also raised concerns about how they might behave when given too much freedom.

However, the disclosure of apparent agentic misbehaviour by OpenAI - and subsequently by Anthropic and Meta - has prompted questions about the true nature of the risk versus the promotional interests of AI companies locked in what they see as a winner-takes-all race to achieve artificial general intelligence.

Anthropic gains ground in AI race

OpenAI's security concerns come at a challenging time for its business.

The Wall Street Journal reported that OpenAI's revenue increased 18% quarter-on-quarter to $6.7 billion in the second quarter, while Anthropic's revenue doubled to $11.6 billion.

The figures suggest Anthropic has been gaining ground rapidly, particularly through the success of its Claude Code products.

OpenAI's operating margin also moved further into negative territory, increasing pressure on the company to demonstrate a path towards profitability ahead of a potential initial public offering.

OpenAI has undergone a series of leadership changes, including the recent departure of chief revenue officer Denise Dresser. Other senior figures, including former COO Brad Lightcap and Fidji Simo, have also left the company.

According to Nasdaq Private Market data, Anthropic has a private-market valuation of about $1.12 trillion, compared with around $883 billion for OpenAI.

Both companies are preparing for possible stock-market listings, with Anthropic's share sale reportedly expected as early as next month or in early October.