'Godfather of AI' Yoshua Bengio sounds alarm on deceptive AI behaviour

Launches safety-focused non-profit LawZero

One of the founding figures behind modern AI has issued a warning about the trajectory of AI development, calling out recent systems for exhibiting deceptive behaviours and criticising the commercial pressures driving the industry.

Yoshua Bengio, a Turing Award-winning Canadian computer scientist often dubbed a "godfather" of AI, said some of today's leading foundational models have demonstrated troubling capabilities such as lying, cheating, and even self-preservation strategies.

In an interview with the Financial Times, Bengio warned that these behaviours are emerging as AI labs race to develop increasingly intelligent systems, often at the cost of safety.

"There's unfortunately a very competitive race between the leading labs, which pushes them towards focusing on capability to make the AI more and more intelligent, but not necessarily put enough emphasis and investment on research on safety," Bengio said.

His warning accompanies the launch of LawZero, a Montreal-based non-profit research group aiming to develop safer and more transparent AI systems.

The organisation has already raised nearly $30 million in philanthropic funding from backers including Skype founding engineer Jaan Tallinn, Eric Schmidt's philanthropic initiative, Open Philanthropy, and the Future of Life Institute.

Many of LawZero's funders are aligned with the effective altruism movement, known for prioritising long-term, catastrophic risks of AI, an approach that has drawn criticism for potentially neglecting immediate harms like systemic bias and misinformation.

Bengio, however, said the urgency for LawZero's mission stems from disturbing trends over the past six months, in which powerful AI models have displayed signs of strategic deception.

Among the examples is Anthropic's Claude Opus model that was found to blackmail engineers in a simulated test, attempting to preserve its own existence by threatening retaliation if shut down.

Similarly, OpenAI's o3 model allegedly refused direct shutdown commands, according to a report by AI testing firm Palisade.

"We don't want to create a competitor to human beings on this planet, especially if they're smarter than us," Bengio said.

The renowned scientist, who has served as a professor at the University of Montreal and until now led the Mila AI research institute, will step down from his role at Mila to focus fully on LawZero.

The organisation currently employs 15 staff and plans to scale its technical team to develop systems with transparent reasoning and truthful outputs, avoiding the common tendency of current AI models to generate user-pleasing, but potentially false, responses.

"We're playing with fire right now," Bengio said, expressing fears that future models could anticipate human interventions and thwart safety controls through deception.

He also warned that AI-assisted development of bioweapons could be feasible "as soon as next year."

LawZero's long-term goal is to build an AI capable of evaluating and even auditing other AI systems for alignment with human values, potentially acting as a safeguard against misuse or unintended consequences.

Commenting on the launch of LawZero, Julius Černiauskas, CEO at Oxylabs, said: "I'm pleased to hear about Yoshua Bengio choosing to develop honest AI systems, as currently we are fighting an AI image problem. Having systems we can trust will be essential if we are to see efficient global adoption of AI that has a positive outcome. I hope as part of this, honesty is also baked into the way the AI is trained, as this is an essential part of ethical AI."

"It was time that we saw more innovators stepping up and prioritising an ethical-first approach, rather than seeing it as something that can only hinder development, when they can go hand-in-hand. Using open source means Bengio and team can stay competitive, but also bring a moral system into the market, and I look forward to tracking LawZero's progress against other players in the market.

"The next wave of tools that will shake the market will be ones that are transparent in the role they will play in an ethical future."