Ctg sit23 hub banner.jpg

"Performant but brittle": Beware when handing responsibility to AI

AI can dramatically improve operational efficiencies, but don't forget that it has costs as well - even if you can't see them immediately

Image:

AI can dramatically improve operational efficiencies, but don't forget that it has costs as well - even if you can't see them immediately

Artificial intelligence can do amazing things, but it isn’t a silver bullet for every application, explains Harvey Lewis of EY

The drive to replace old tech with shiny new stuff is innate among IT pros - it might even be described as a defining characteristic. But when it comes to artificial intelligence, simpler is often better.

"A lot of the focus [of AI] in recent years has been on topics like deep learning, the application of very deep neural networks to machine learning problems. Those techniques tend to be very, very performant but quite narrow in their application, and very brittle," says Harvey Lewis, an associate partner at professional services giant EY and chief data scientist for tax and law. "Change the context of the situation just a little bit and they actually perform really badly in a lot of cases.

"This pursuit of bigger and better algorithms, for me, was a bit of a fallacy because they became ever more brittle."

Lewis believes it is better to focus on using simpler tools before complex AI systems, even if they're a little less performant.

"Your deep learning algorithms are like Formula One race cars: they are tuned for driving as quickly as possible and set up for particular circuits. As soon as you take them off those circuits and try to drive them on ordinary roads, you're going to have an accident...whereas you can buy a much more conventional motor car and pretty much drive it wherever you want, including around a Formula One racetrack. It is clearly not going to be as fast, but it will get the job done."

Using traditional software over AI may cost you some time, but if you track success using alternative metrics it can often come out ahead. Carbon emissions, for example, are on average much lower.

Large, complex AI models require much more energy to train and run than legacy tools, and that's easy to forget; after all, those emissions are invisible.

But emissions we can't see are just as harmful as those that come from driving a car or taking a flight - activities we've been told to minimise for decades. Lewis explains:

"Ones and noughts don't seem to have any real substance or weight. It becomes a lot easier to put that into the back of your mind and just concentrate on the performance metrics that you're trying to optimise for."

Put it another way: we all understand that limiting our travel is good for the environment, but datacentres and AI were already thought to produce equivalent emissions to the airline industry way back in 2018: in fact, training a single AI model generates about the same amount of carbon as a flight across the USA. With both cloud and AI use rising fast, it's time to acknowledge that tech has a real impact on the environment - even if we can't see it.

Putting it into practice

It's all well and good to talk about this theoretically, but you would be forgiven for asking, ‘How can I stay competitive while limiting AI use?'

EY's approach in its tax practice, where the company considers measures of success aside from raw performance, is a good example .

"Rather than using a deep artificial neural network straight out of the box, maybe we think about decision trees. Maybe we think about optimising across not just performant dimensions, but other dimensions like efficiency, or explainability, or speed - so that when you consider those other dimensions, actually some of those simpler models and methods perform pretty well.

"It's one of the reasons why, for example...we use simpler models; because yes, performance is important to us, but it's not the only consideration. We like to think that when a tax application spits out an answer, we can get an explanation for how it arrived at that. Or that it doesn't take us five weeks to train the model, we can do it in five minutes. Those are the kinds of considerations that are really important."

While some of this sounds technophobic, Lewis is the first to acknowledge that AI has a place, especially in the sustainability sphere; nothing else is quite on the same level when it comes to predicting extreme weather events, for example. But it isn't the panacea some have hoped for when it comes to climate change.

"A lot of people do think that AI can be the silver bullet; but actually, what becomes more important is getting the ideas and the people and the other technologies that we need for combating climate change...and then using AI to help improve those processes.

"The answer to the question [can AI address climate change] isn't, ‘AI is the end, and we should just throw everything we've got at AI'... Actually, we've got to be innovative and agile in all of the other fields, and AI can help us with that."