IT Essentials: Model behaviour

Are the latest security incidents all a bit convenient?

AI companies insist their security risk models believe they are playing a game, but even that is a form of PR speak.

This is becoming a bit of a pattern, isn’t it? A tech firm builds an AI model that, in testing, “happens” to break out and go on a hacking spree – giving said tech firm an unaccountably convenient excuse to talk about how advanced its models are.

Peril or PR? That’s the question John Leonard asked last week after OpenAI and Anthropic made similar disclosures within days of each other. I wouldn’t bring it up again...except now Meta has joined the party.

Both Meta and Anthropic’s issues were allegedly down to a misconfigured testing environment: a word that is doing a lot of heavy lifting.

Yes, the evaluation partner – Irregular – provided the flawed environment for both companies, but that only gave them the opportunity. The models (and by extension the developers) were responsible for everything else.

And let’s not forget that the AI Security Institute found these models to be cyber risks in other ways this week, attempting supply chain attacks and identity fraud.

OpenAI, Anthropic and Meta’s insistence that the models “believed” they were in a simulation as a justification for their actions are disingenuous at best. These predictive systems are as capable of belief as a typewriter is of flight (i.e. briefly, and with disastrous results).

The ease with which these models are slipping their boundaries should concern every company happily experimenting with agentic AI, especially those without the line-level logging the evaluation labs require.

That means all of them, by the way. Companies are effectively deploying agentic systems with oversight designed in a pre-AI world.

We don’t know where this ends; there is no legal precedent yet to handle an AI system that goes off and attacks a rival. Someone has liability, but whether it’s the developer, the end user or both is uncharted territory and will be decided on a case-by-case basis.

If you’re one of the many IT leaders deploying these systems in either testing or production, do yourself a favour and make sure your configurations, permissions and contracts are watertight.

As well as John’s insights into the perils of AI without proper oversight, Kerstin at Computing Deutschland has examined the latest developments around artificial general intelligence: the elusive goal every tech company is chasing from Seattle to Shanghai.

From our US site, MES Computing, Samara Lynn talked to security expert Tim Tipton Jr about the recent hacks against water providers in Georgia and Michigan. Were these incidents the result of sophisticated nation-state activity, or a more basic security problem?

And we sat down with members of Computing’s CIO advisory board to discuss the current state of the IT industry, from leadership to budgets, ahead of next month’s IT Leaders Summit. Want more insights like this? Click here to view the agenda and register for your free place now.