Momentous change is coming to IT, says Mike Lynch, chief executive of enterprise search vendor Autonomy, and many of today’s chief information officers (CIOs) are completely unprepared for it and will have to adapt.
To date, corporate IT has been built on the basis of organising data within the rigid structures of a relational database. And while this model has proved enormously powerful, it requires a logical approach to ordering information; whereas human beings are more comfortable with the unstructured ordering of information, says Lynch. The rise of unstructured data will reorder the world of IT, he predicts.
“We’re seeing an explosion of working with unstructured data and human beings are much happier with this,” says Lynch. “The computer is now fitting us and it’s a big change that’s coming.
“We’re talking about a new age of technology where we’re no longer constrained to cramming in rows of information. The next 40 years will be built on human-friendly information.”
Lynch founded Autonomy in 1996, inspired by research he carried out while studying at Cambridge University. Since then he has led the firm through an ambitious programme of global expansion that has taken its market capitalisation to about £2bn. Autonomy specialises in what Lynch calls meaning-based computing, enabling its customers to draw meaning from all kinds of information, structured or otherwise. Autonomy aims to change the interaction between humans and computers, ensuring that machines accommodate us, not the other way around, says Lynch.
Research estimates that unstructured data now makes up between 80 per cent and 90 per cent of all organisational data. Analyst Gartner suggests the worldwide volume of unstructured data is doubling every month.
And businesses are gradually waking up to the possibilities of tapping into this unstructured data. Sean O’Dowd, senior analyst at Financial Insights, says finance firms have huge opportunities to improve their predictive modelling and bring out market-moving analysis ahead of their competitors through mining their unstructured data.
Patricia McGinnis, research manager at Financial Insights, adds that the creation of applications designed to “mine and leverage” unstructured data will prompt ever greater levels of collaboration between strategic business managers and technology professionals.
But, while unstructured data may allow more human-friendly approaches to the way information is dealt with, those increasing pools of unstructured data have some decidedly unfriendly consequences, particularly given the complex regulatory environment in which companies operate today. With vital pieces of information potentially residing in any number of places throughout the organisation, IT leaders need to be confident they have a good handle on that data if they are to meet compliance requirements.
Furthermore, the regulations governing the use and storage of data are becoming increasingly numerous and complex. Businesses leaders are having to learn about new requirements fast to keep up with the evolving landscape and remain compliant. For example, some regulations govern the actual storage of data, including locations where it can be stored. Switzerland, for example, is now demanding that certain types of data on Swiss citizens must not be held outside the country.
The penalties and damage to the brand that result from poor data management have ensured that compliance remains a priority within IT, even as budgets tighten elsewhere, says Lynch. “You don’t have a choice but to invest in technology,” he says. “The areas where the banks haven’t shut off spending are around regulations.”
He predicts that the onslaught of new regulations governing management of data will continue apace. Furthermore, law courts are taking an increasingly hard-line approach. “You now have to turn up in court with everything, includi ng records of phone calls and emails. You have 99 days to organise everything you need or it could be a criminal offence,” he says.
“When you start dealing with unstructured data you’re in a whole new world, and knowing what is going on and being able to stop people doing really silly things is a key priority. We’ve always been concerned with the threat of people breaking into systems, now the big problem is your system walking out the door.”
Lynch expects the current surge in the use of unstructured data will transform the role of the CIO. The CIO is the natural choice among board-level executives to take responsibility for data, he suggests, and that has far-reaching consequences. “The CIO’s job is becoming ever more sophisticated and legal requirements are becoming central to a CIO’s responsibilities,” he says.