Morrisons taps SAN for application performance

Retailer monitors traffic at a port level to identify storage bandwidth hogs and prevent arguments between application administrators

Since its acquisition of Safeway in 2005, the Morrisons supermarket chain has expanded to 455 stores across the UK and is gearing up for an anticipated Christmas rush with a range of offers and incentives, both in-store and online, to get shoppers spending money with it rather than rivals Tesco, Asda and Sainsburys.

Supporting those deals is a complex series of networks, storage and applications which can cause major headaches and lost revenue if performance comes up short.

This is one reason why the company is currently deploying technology from Virtual Instruments (VI) to monitor potential problems in its storage area network (SAN).

"We have deployed some of VI's solutions into live environments, but the project is in the first third of the deployment stage and it is early days yet," Morrisons' head of storage Simon Close told Computing.

"We are determining the profiles of the applications in use before we can start to look at them in detail. [VI] allows you to dig in and understand why applications are behaving as they are," he added.

Morrisons has installed VI's SANInsight Fibre TAP Patch Panel System, which attaches to the ports on a Fibre Channel (FC) SAN switch to monitor the data traffic passing through it to provide low-level diagnostic information around application response times and FC protocol errors.

If nothing else, the TAPs give Close sufficient visibility into application traffic to verify problems with individual applications, meaning the IT department can no longer pass the buck.

"The TAPs let us see the demands that some apps are putting on the storage infrastructure, meaning we are not as reliant on the Oracle database administrator or other application teams giving us this information. We have equivalent metrics, so we can say ‘yes, we agree, or ‘we don't agree'. For me, it is all about not being reactive, but being proactive to alleviate bottlenecks before we actually see them, then working out how best to satisfy demand," said Close.

One problem for many companies is monitoring of the performance of applications running in virtual workloads, something Morrisons is doing using VMWare rather than VI tools. The retailer does not run really business-critical apps on virtual servers, though it does have Hypervisors on the Unix side.

"The common problem is that in a shared SAN environment you have one application that runs away with all the resources and leaves nothing for anybody else. So when a huge workload is submitted to the system the system will absorb it, but at the expense of every other application. There is no real way around that; you just have to make sure everything is as isolated as it can be," said Close.

Close first used Virtual Instruments' technology in a previous job at the Lloyds Banking Group, where he worked for eight years prior to joining the retailer.

"We have taken a different approach at Morrisons than Lloyds because it is a smaller environment. We are not tapping every SAN port on the core of the SAN, where we could potentially see every disk drive, because the size of the deployment made it cost prohibitive," said Close. "What we can do is look at timeslide or roving mechanisms able to move through different environments and check for five minutes here or there rather than use hundreds of hardware probes."

Morrisons runs mainly HP SAN (Brocade) equipment, and also uses the native management tools that come with the storage hardware and Unix systems for administration purposes. The VI deployment was just one element in a larger technology refresh that saw a whole storage estate of legacy disk drives and back-up solutions upgraded, with new fibre optic cabling systems and a move from a mixture of 2Gbit/s and 4Gbit/s to 8Gbit/s FC ports as standard, with multiple 8Gbit/s circuits connecting the company's two data centres in West Yorkshire.

Close says the traditional approach to improving application performance - throwing more bandwidth at the problem - is becoming less of an option for cash-strapped IT departments and doesn't always solve the issue anyway.

"That is always a knee-jerk reaction, but the way finances are with a lot of organisations now means it is not as easy to do that as it was five years ago," he said. "Throwing hardware at stuff just moves the bottleneck somewhere else, from the storage to the Unix system or whatever, and it does not really fix the underlying issues. It is also about juggling things, seeing exactly what is going on and deciding whether to make a strategic investment in hardware or negate the need to purchase new hardware at that point in time."