Spinnaker seeks to contain data explosion

Any IT department with mushrooming quantities of network attached storage (Nas) devices from Network Appliance (NetApp) knows the headaches involved in managing 'filer farms'.

If you generate petabytes of data daily in seismic research, you may be considering a grid for your computing platform to replace high-end Unix - and asking what kind of storage infrastructure you can deploy to complement it.

Even if you're not in high-performance computing but are tired of managing distributed home directories, you may dream of provisioning storage capacity on the fly as applications' requirements change.

For all these scenarios, NetApp is developing something called 'storage grids'. This is an idea championed by Spinnaker Networks, the storage vendor NetApp acquired last November.

Spinnaker sold Nas filers that, as well as all the functions of a NetApp product, had a virtualisation layer to create a logical map of all the disks connected to a server and present them as a single storage name space.

Nas virtualisation, particularly across multiple filers, interested NetApp so much it paid $300m (£164m) for the 80-person company, which still has just 10 customers.

Now it has embarked on an integration of the two firms' technologies, says Ron Bianchini, Spinnaker's former chief executive and now the NetApp vice president handling the Spinnaker business.

At the heart of the Spinnaker offering, like all Nas products, is a proprietary file system (SpinFS).

Many Nas devices, particularly low-end ones, run on Windows or Linux, but Spinnaker, like NetApp, has its own proprietary operating system (OS), purpose-built for the box's functions, called SpinOS.

At the moment, both ship on proprietary Spinnaker hardware, but over the next two years they'll be integrated into NetApp technology: SpinOS into Data ONTAP, SpinFS into the WAFL file system.

Not surprisingly, the plan is to merge the Spinnaker team with NetApp's gFiler group developing Nas gateways, which is where its technology should sit to operate across multiple devices.

Bianchini acknowledged that Spinnaker's technology will initially be NetApp-specific, although integration with gFilers will, by default, extend it to supporting the same Hitachi and IBM hardware with which NetApp's gateways currently work.

That merger will also take Spinnaker into protocols beyond the standard network file system (NFS) and common internet file system (CIFS), all open systems that Nas devices must support. Fibre Channel and iSCSI are also on the roadmap, allowing Spinnaker to virtualise blocks as well as files.

But Hamish Macarthur, a director of storage analyst Macarthur Stroud, has identified a deeper level of proprietary tie-in, and one with the potential for conflicts elsewhere in the data centre.

First, Spinnaker will be a complex product to implement, particularly if virtualising across many devices.

He says that, once installed, "Spinnaker will handle all the storage services, like snapshot and migration. Even for simple backups you'll initially mirror with Spinnaker then use something like Veritas to back up offline."

That in itself is not a problem. A customer would have to take the strategic decision to use the Spinnaker platform to manage their entire storage infrastructure, ultimately even any block-level storage area networks they may have.

The problem will come, Macarthur argues, if they adopt a utility computing (UC) model for their data centre.

UC proposes the same kind of virtualised, provisioned-on-the-fly environment as Spinnaker, potentially for both processing and storage.

Why should a customer want a UC offering from, say, IBM or HP managing its servers while a Spinnaker/NetApp gateway does the same for its storage?

And even if they do, will the two systems be able to coexist and interoperate comfortably?

How Spinnaker's proposal will work
Users want to provision storage out of a 'cloud' containing both Nas and San devices without worrying about the specifics of what hardware is being allocated to which applications.

Spinnaker can cluster up to 512 of its boxes and present them as a single, 12PB name space with a single mount point, yet partitionable for security purposes.

It requires no client software to be loaded on hosts, nor is it an in-band virtualisation engine representing a potential bottleneck or single point of failure in the data path; all the virtualisation takes place within the Spinnaker stack.

It currently operates on Spinnaker boxes, will shortly be extended to support NetApp hardware and, through the latter's gateway operations, will support Hitachi and IBM arrays.