Panning for data gold - a guide to information management

Unstructured data can contain valuable nuggets of information

Information is the lifeblood of every business, second only to staff in importance. However, for years organisations have built Byzantine information management systems that pay little heed to how enterprises need to consume that information. As the volume of data being generated has escalated, the problems inherent in this systems-dominated approach to information have become pressing. Executives often spend much more time compiling the information they think they need to support a decision than they do on actually weighing up their options and their possible ramifications.

Today, some organisations are starting to appreciate the benefits of moving the responsibility of information management away from IT and into the business, where its ownership lies, and where the real value of the derived information and knowledge can be fully exploited.

In this model of information management, IT replaces its data stewardship role with that of facilitator, providing the tools to channel the flow of information throughout the enterprise.

At global recruitment consultancy Hudson the ability to easily locate the ideal job candidate provides competitive advantage. On any given day it has three million candidate records in Europe that it may need to scour to unearth the right person for their clients. Hudson had historically relied on a database of candidates that consultants could query, says chief operating officer Laurent Chen. But with the volume of searches approaching 12,000 a day, “using the master database actually slowed down our consultants’ access to key information and hindered productivity,” he says.

By introducing a search appliance from Google, Hudson’s consultants can now search for candidates with particular skills, or similar positions that a candidate might fit, in a flash, says Chen.

Historically, business leaders may have assumed that databases, business intelligence systems and perhaps a few data quality tools would provide suffici ent analytical powers to understand what was going on in the business. Today, however, the burgeoning importance of unstructured data is redefining the toolset required to make sense of enterprise information. Recent research by Coleman Parkes for HP suggested that UK business leaders believe that only 25 per cent of their data is currently unstructured. However, that is at odds with most industry predictions, highlighting a dangerous blindspot in corporate information management plans.

Unstructured data includes emails, documents, and files from third parties that lie outside of internal systems. Ignoring, or underestimating, the amount of unstructured data in a company means that organisations are failing to fully understand their business information and its importance.

“All of our data is unstructured,” says Hudson’s Chen. “However, we have created specific mechanisms to automatically put some structure around the information so that every piece of data we produce will be in the right place and can be found.”

Such approaches are consistent with the information management frameworks constructed by analysts. Gartner defines enterprise information management as an integrated discipline for structuring, describing and governing information assets, regardless of organisational and technological boundaries, in order to improve operational efficiency, promote transparency and enable business insight.

Elsewhere, Ovum also has a framework for unified information management (UIM), but as practice leader Ian Charlesworth explains, the benefits of these frameworks are not necessarily achieved by rigid adherence to the vision, but from the aspiration to impose some order on information management practices.

“The real problem facing organisations is how to join the dots between the vision, the vendor offerings and the user’s demands,” says Charlesworth. “UIM offers the disciplines, organisation and structure that are needed to tie this moving feast together so that when companies are analysing the benefits of a smaller piece of information management technology they can assess it against the concept of a larger framework, to see whether it really fits.”

For many, those frameworks will be founded on tried and tested technologies, such as business intelligence, content management and enterprise search. But the gloomy economy is adding impetus to the need for effective information management, as business leaders strive to make better use of the data they collect, and to reduce the costs associated with that.

“Many organisations will adopt a back-to-basics approach that will resonate strongly throughout 2009 as companies look to cut costs and streamline operations and enter an era of risk assessment and management,” says Charlesworth. “Business intelligence and data quality are going to be in the forefront.”

Countless organisations still have huge data quality issues and many have multiple copies of data assets draining the coffers unnecessarily. A one-off expense of a major data migration and consolidation exercise could bring significant benefits in reducing the overall cost of maintaining disparate data sets.

The Liverpool Women’s Hospital has migrated and consolidated its data using two EMC storage area network devices, and Zafar Chaudry, director of information management, says the whole hospital is now realising the benefits. “For the first time we know where our data is,” he says. “Clinical staff can actually find the information they need because it is kept within a proper structure. In addition, we can protect the data and impose professional security procedures around it.”

According to Charlesworth, organisations have already taken some big steps down the road to UIM, helped by the evolution of some of the core information management technologies. “In the past 18 months we have seen the business intelligence (BI) marketplace cease to exist as a separate entity. BI has gone through maturity to the point where its functionality is delivered through other means – either as part of a business process or embedded in enterprise management functions.”

The notion that BI existed as a specific category of technology was indicative of the problem faced by businesses, he adds; the analytical capabilities that should have been embedded into the applications were either not sufficiently powerful, or only accessible to data associated with that application. Today, BI goes across not only applications, but end-to-end business processes, even extending outside the organisation into the supply chain.

Nevertheless, for BI – or any other information management technology – to be truly effective, users have to be certain they can trust the veracity and accuracy of the data. Over the past two years, master data management (MDM) has been seen in some quarters as the best method for developing a “single version of the truth” across all data types held within the enterprise.

Within many firms the most highly developed data sets exist around customer information, with product data following a close second. Therefore customer and product information management is an important starting point for any organisation embarking on an MDM strategy.

Companies are starting to build a single, intelligent platform to protect, secure and manage information. Applications are no longer siloed – they are connected and melded to support more advanced capabilities. Data is shared across applications and is managed as a corporate asset.

Poupart, one of the UK’s largest fruit suppliers to supermarkets, has brought its disparate data systems together, improving decision making while increasing efficiency across its supply chain. IT manager Matthew Butlin says that access to real time, accurate data is allowing the company to manage deliveries more efficiently and helping to reduce wastage. “We started out with a clear idea of what we wanted to do and then we built the infrastructure to support our vision of centralised, integrated data,” he says. “Conversations are now based on real facts, not fiction.”

Ultimately, Butlin’s plans go to the heart of most information management initiatives – to build a mechanism for collecting corporate data that the users have absolute faith in.

Five information management technologies to look out for

Master data management
Master data management (MDM) consists of the strategies, processes, technologies and people needed to create and maintain a single, inclusive view of all the organisational data that should make up a customer record. MDM removes some of the difficulties associated with data quality, consistent classification and identification of data as well as data-reconciliation issues by providing complete, real-time views of data assets gathered from multiple applications, systems and databases.

Text mining
Text mining is the process of deriving high-quality information from text sources, drawing on information retrieval, data mining, machine learning, statistics and computational linguistics. An automated text mining system can analyse large collections of documents, email and other text sources to discover previously unknown information, relationships or patterns. With more than 80 per cent of information currently stored as text, text mining could have a high commercial value especially with its ability to mine information across different languages.

Data visualisation dashboards
An enterprise dashboard offering an at-a-glance visual display of selected corporate data gathered from across the organisation’s multiple data sources. With new tools to discover, interrogate and analyse business information, an easy-to-use display tool will be needed to allow the data to be viewed, shared and published. Data visualisation dashboard tools will be appearing as part of information management suites and as standalone applications.

New search technologies
Search engines will be entering a new era as vendors look for search algorithms that will uncover and present relationships in the disparate types of structured and unstructured data that exists across an enterprise. These new technologies will move search techniques forward from keyword recognition into fuzzy techniques that imitate the pattern-searching features of the human brain, allowing for inexact definition searches and incomplete search criteria.

Enterprise mashups
This is the trendy terminology for combining information from a variety of sources into a single view for the user. One of the most important aspects of a mashup is that the pieces of information are not displayed in isolation beside each other, but are overlaid and combined into contextual links to help the user make more informed decisions. For example, sales information could be overlaid on a map, and drilling down into the information could show customer records, sales forecasts or stock availability together with free-text customer information all at the click of a button.

Five firms looking to change the way you use information

Visokio
Omniscope from Visokio is a general-purpose data visualisation, reporting and publishing tool that allows users to bring together a variety of data from disparate sources. It lets users query, analyse and explore the data, and can cope with large table of data imported from spreadsheets or databases, together with related images and geographical information.
www.visokio.com

Endeca
Endeca provides users with tools to answer open-ended questions that technologies such as business intelligence, relational databases and search engines fail to answer. Powered by a new class of access-optimised database – the MDEX Engine – Endeca claims to help non-technical people find, analyse and understand complex information in ways that were never before possible.
www. endeca.co m

InforSense
InforSense produces a next-generation business intelligence platform, which it claims delivers mashup technology that can be used to access most of the data that exists inside or outside an organisation. The platform uses predictive and visual analytics to present data in a variety of ways, such as reports or embedded into existing applications and delivered over the web.
www.inforsense.com

Initiate Systems
Initiate produces master data management software that provides a complete, accurate and real-time view of data spread across multiple sources. It claims to improve data integrity, costs, and audit and security controls by accurately identifying duplicate and fragmented records. Having a single view of a customer can also improve customer service, reduce fraud and improve customer loyalty while cutting marketing costs.
www.initiatesystems.com

Clarabridge
Clarabridge’s text analytics software gathers all textual information from a company’s internal and external sources and restructures this data to provide additional customer intelligence. The tools are used to analyse customer feedback, aiming to improve overall operational performance and identify ways to improve customer loyalty.
www.clarabridge.com

In the second part of our guide to information management next week, we talk to IT leaders putting best practice into action.