Rising AI costs leading to model swapping, Capgemini finds

Three in five organisations plan to expand their use of open-weight models

Infidelity is all the rage in the world of AI, and the wandering eye is not restricted to chief executives of big tech firms.

Research among senior executives at large organisations by consultancy giant Capgemini found 59% saying that AI expenditure had been significantly above expectations or budget, with the overrun amounting to 24% on average.

This unexpected expense is pushing companies towards cheaper models, particularly open-weight LLMs which can be as much as 15 times cheaper than proprietary models from frontier labs such as OpenAI and Anthropic.

The Capgemini Research Institute found that nearly three in five organisations plan to expand their use of open-weight models. The average price differential was less pronounced than the extreme figure cited above, but tokens for open-weight models were still 87% cheaper than for their closed models.
This finding is supported by figures LLM platform OpenRouter, whose usage data indicates that open-weight models enjoyed 72% of traffic but accounted for just 27% of spending in September.

The propensity to swap models prompts a question about the breadth and depth of the “moat” surrounding the economic model of frontier providers such as Anthropic, which is planning a $1.3 trillion IPO in the near future. The advantage enjoyed by the leading models over their open-weight competitors is measured in weeks or months, and the number of use cases that benefit from using cutting-edge capabilities is small. Moreover, models are largely interchangeable.

Capgemini also found that most organisations struggle to understand the financial impact of their AI deployments. Only around a third of the executives polled were able to estimate indirect costs such as for oversight and governance, while half admitted that AI spending had been approved without adequate visibility of full costs or proper consideration of anticipated business outcomes.

The findings align closely with Computing’s own research, which found AI costs rising to the top of IT leaders’ agenda as cheap and freemium licences are ditched by providers seeking to recoup their massive expenditure on infrastructure.

The industry is reacting to the growing trend for model polyamoury by providing platforms that can switch automatically between LLMs according to cost and capability metrics.

“A lot of customers now realising that actually, even from the frontier model companies, a model that's six months old actually performs as well as the frontier model,” said Rich Radley, VP of field engineering at Databricks, speaking about the company’s Unity Gateway platform.

Databricks' own research has found that organisations in the retail sector the most likely to use multiple LLMs, with 83% using two or more model families.

Nagy Nagy, head of data at Compare the Market told Computing that cost, matching capability to use case and avoiding vendor lock in are all part of his AI calculations: “Today it might be model that's provided by OpenAI. The day after it might be Anthropic, and maybe Kimi might be a good option for us too.”