Operating more efficiently – an NHS trust’s hybrid cloud storage strategy

Imperial College Healthcare NHS Trust technical architect on how a mandatory backup project has grown into a collaborative data platform

How do you decide between on-premises or cloud for data storage?

The rule of thumb is to choose cloud storage for infrequently accessed or archived data, in-house for sensitive information or data that requires low latency access. But of course, its much more complex than that, particularly if you're a large, complex organisation in a regulated sector, such as a NHS trust.

Over the past four years Imperial College Healthcare NHS Trust has moved from traditional on-premises storage infrastructure for digital pathology to a hybrid cloud approach that has fundamentally changed how it manages its rapidly growing volumes of data. What began as a mandated backup project has evolved into a comprehensive storage platform handling everything from currently accessed clinical image to departmental archives and CCTV footage.

Technical architect Yusuf Mangera told Computing how his team has achieved a successful balance between cloud and on-premises storage in the face of ever-changing requirements, and how this underpins the Trust's strategy of greater operational flexibility, collaboration with other Trusts and cost savings.

Choosing between cloud and on-prem for long term storage

The original decision to move long-term storage of high-res clinical images to the cloud was driven by both cost and operational considerations, Mangera explained. For high-volume, infrequently accessed data the ongoing costs of on-site NAS, appliances and servers were increasingly unjustifiable. "We didn't want to go around every five years, constantly having to refresh on-premise infrastructure, which involves a year or so of migration activity. Before you actually even start using the storage, you're already about a year into your subscription or your contract."

Mangera calculated that a combination of long-term cloud storage for clinical digital images, CCTV footage, logs and departmental archives, plus hot storage for frequently accessed data could eliminate the need for constant infrastructure refreshes and ease pressure on datacentre space. The Trust chose Wasabi, mostly for reasons of low cost.

For frequently accessed data the trust uses high-performance in-house storage to ensure low latency for active clinical work. But this storage is now part of a hybrid scheme, created with Wasabi alongside consultancy InfraTech Systems, with intelligent tiering of data based on usage patterns. For example, with maternity ultrasound images, patient data remains on-premises for the first 30 days while the patient is being treated in the hospital. After 30 days, data is automatically moved to hot cloud storage where it remains easily accessible but at lower cost than on-premises. Later it is archived to cold cloud storage.

Clinicians can access data in hot storage seamlessly - when they click to open an image, if it's not stored locally it’s automatically downloaded from the cloud within 60 seconds. They rarely notice any difference in responsiveness between cloud and in-house.

Importantly, this approach did not require any complex redevelopment work. "The application requires no re-architecture, no redesign. It's purely middleware," Mangera explained. "It's allowed us to take a lot of pressure off our on-premise infrastructure."

To ensure low latency from cloud-based storage, the Trust invested in a 10-gigabit connection directly to Wasabi's London region, enabling performance comparable to the in-house infrastructure. This investment was crucial to maintaining clinical workflow efficiency while still achieving cost savings.

A platform to build on

The return on investment has been significant, Mangera said, with the cloud storage solution coming in well below what the Trust would have spent on the next five-year cycle of physical infrastructure refreshes. In addition, time and effort has been saved on migrations, physical datacentre space and energy to run the equipment. It has also eliminated a previous scenario whereby departments needed additional storage capacity late in a refresh cycle, forcing expensive short-term purchases.

Wasabi was chosen over alternatives like AWS and Azure because at the time it was approximately 70% cheaper. The Trust also committed to a five-year contract with reserved capacity pricing, ensuring predictable costs as its data volumes grew. Even as cloud prices have changed (mostly upwards), Wasabi remains "one of the cheapest providers on the market today," according to Mangera.

In particular, the provider’s zero egress fee model was a clincher, because of the unpredictable nature of health data.

"It's uncertain when a patient is going to be back in hospital again - five years later, two years later - and how much data you're going to generate," Mangera explained. Without a mechanism stream to protect against spikes, egress fees represented an unacceptable financial risk.

The cloud platform, which is integrated with a Snowflake data warehouse, provides a basis for AI experiments, and also for cooperation with other facilities in the North West London ICB (integrated care board), where four hospitals have come together to collaborate on the rollout of electronic patient records (EPR) and other systems.

Collaboration allows the hospitals to reduce duplication and costs. They are introducing vendor-hosted SaaS services that reduce on-premises infrastructure requirements, and single sign-on across facilities making it easier for staff to move between sites. It’s about simplification and scale, Mangera remarked.

"Where it's hosted in the cloud it's easier for all four hospitals to access. It also pushes some of the maintenance and management to the vendors themselves.”