Azure's Pelican cloud storage system casts shadow over tape

Microsoft is working to add a new tier of disk-based storage that is faster and easier to integrate than tape

Microsoft is working to develop a new class of disk-based storage that is low-cost but faster than tape for its Azure cloud service, according to The Register.

Speaking at an event in Sydney, Mark Russinovich, the CTO of Azure, said that Microsoft is the world's biggest user of tape, which it uses for its Azure Archive software-defined storage service.

The company is "not satisfied" with tape's performance, and wants a faster alternative that will encourage customers to store more data for longer. "We think there is another point in the [storage cost vs. access latency] spectrum that is slightly higher cost, but much more efficient than hard disks," Russinovich told attendees.

Cost is also a factor, with tape enclosures - which need special infrastructure in the data centre - working out to be fairly expensive.

Microsoft's approach is called ‘Project Pelican', and it has been talking about it for years; but the system might actually be put into production in 2018.

The company is aiming to design a rack of over 1,150 3.5" disks, each with 10TB of storage, for a total capacity of 11.5 PB (petabytes). The design also features servers to control the drives.

"What makes this unique is that the disks cannot all spin up at the same time," Russinovich said. "When data is being accessed the server spins up the drives. This allows you to store massive amounts of data with very little power."

There was no mention of when Pelican might be available to customers - but there were some subtle hints that let us peek into the future.

Russinovich called Pelican "the heir" to current cloud storage hardware, and said that it is especially suited to storing massive amounts of ‘cold' (infrequently accessed) data without the speed drawbacks of tape. That could make it useful for, for instance, information that could inform future AI developments. Russinovich said that this data is sometimes discarded today.

Another minor clue was the admission that some Azure data centres aren't set up to house tape libraries, but are ready to take on the new disk-based system.

That's not to say that Pelican will turn tape into a dodo at Azure; but Microsoft is clearly aware of the medium's drawbacks and is working to counter them.