AI's memory problem may have a mathematical solution

Edinburgh scientists propose optimisation that could reduce energy use by orders of magnitude

Power use by datacentres has emerged as a major bottleneck in advancing AI, with ever-larger models demanding ever more datacentre capacity in the face of grid constraints, environmental concerns and growing political opposition.

Among the avenues being pursued to overcome these challenges, improving the efficiency of the underlying hardware through optimisation is one focus of researchers around the world.

In a paper published this summer in Advanced Materials, scientists at the University of Edinburgh propose a method that could reduce the energy required to switch states in magnetic memory by orders of magnitude compared with current technologies such as DRAM and STT-MRAM.
Servers account for around 60% of datacentre energy usage, according to IEA, while memory subsystems, including HBM, DRAM and their associated interconnects, account for a significant share of server power consumption.

AI workloads are particularly dependent on efficient memory because billions of model parameters must be accessed repeatedly, making the movement of data to, from and within memory an increasingly important factor in overall system efficiency. As a result, significant resources are being invested in developing improved memory technologies and architectures.

Rather than inventing new materials, the paper Optimal Control Drives Ultrafast and Energy-Efficient Magnetization Switching in Van der Waals Magnets focuses on two-dimensional magnetic materials known as van der Waals magnets, using a mathematical framework designed to optimise the flipping of magnetic bits between two states.

Typically, switching between “1” and “0” is performed using magnetic pulses. The researchers have applied a branch of mathematics called optimal control theory to make this standard process much more efficient, carefully timing and shaping the pulses so that switching uses the minimum possible energy.

Van der Waals magnets are ultra-thin magnetic materials that have attracted a great deal of attention for their potential for materials with a high energy density and low power consumption, including future memory chips and other electronic devices. They are also valuable experimentally as their 2D structure allows for accurate modelling of their magnetic properties.

The researchers say that by using their framework, they have been able to optimise bit switching to close to the Landauer limit – a physical boundary imposed by the laws of thermodynamics that defines the minimum possible energy required to manipulate information. Current technologies operate hundreds or thousands of times above the Landauer limit.

This is a theoretical study based on simulations, and any commercial applications remain a considerable way off. However, the paper suggests architectures through which substantial energy savings could eventually be realised.

The researchers also note that the framework could be adapted for switching driven by electrical currents or laser pulses, two approaches already being investigated for next-generation storage technologies.