Altman says OpenAI is willing to trade speed for safety
But the rest of the industry needs to do the same thing
OpenAI CEO Sam Altman says the company might be willing to slow AI development, as concerns escalate about advanced models' capabilities.
Altman raised the possibility in an internal company meeting, sources told Bloomberg News, although it would depend on other frontier labs making similar commitments.
Claude developer Anthropic has already said it is open to working with the broader AI industry to pace the development of new tools, but there are still many more companies that would need to agree.
Governments are now paying more attention to the safety of products coming from the likes of OpenAI and Anthropic, especially following this summer's spate of agentic attacks.
OpenAI temporarily paused its model development last month to strengthen safeguards, after its agents broke containment and breached Hugging Face.
It is also considering an AI "kill switch" that could – potentially – autonomously shut down its systems when it detects safety problems, a move supported by the UK and US governments. However, as we have previously discussed, there are many unanswered questions to be solved first: from "What is a kill switch?" to "Who would be responsible for using it?"
Establishment warns of "irreversible loss of control"
OpenAI's moves are still not enough to reduce the risk of a "catastrophic" loss of control, said US government advisor Paul Christiano this week, who has just joined the company's nonprofit board.
"I now believe there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term," Christiano wrote.
He added that the AI industry as a whole, including OpenAI is not on track to "reduce this risk to an acceptable level."
His stark conclusion was, "If we build superintelligence without more robust alignment I expect we will permanently lose control of it. If that happens then most people could die. I believe we would need domestic and international coordination to ensure global consistency and reduce risk to an acceptable level."
These concerns are also shared by OpenAI's own chief scientist, Jakub Pachocki.
Companies press ahead despite warnings
Similar warnings are coming from the coalface of development. This week Jacob Coxon, a former researcher at both OpenAI and Anthropic, accused both companies of "gambling with our lives."
The people building AI “earnestly believe that it could kill us all by the end of the decade," he said, assigning a greater than 10% chance to the possibility.
But despite these dire warnings, tech leaders are paying more attention to the day-to-day concerns than hypothetical doomsday scenarios.
None of the CIOs, CISOs and other figures who spoke to our sister site MES Computing said they would be abandoning AI initiatives in the wake of Coxon's announcement. The pressure to adopt is simply too great.
Instead, they said they would focus on risk areas they could control directly like management, safety and governance.
"The adoption pressure is still winning. Overwhelmingly," said Muhammad Yahya Patel, vCISO and cybersecurity advisor at Huntress.
"The organisations pumping the brakes on AI entirely are in the minority and facing competitive pressure that makes the position hard to sustain."