AI will solve the challenge of unstructured data, says IBM

Text, images and voice are difficult for computers to process - but artificial intelligence is making contextual understanding a possibility

The vast majority of data - as much as 80 per cent - is not set up for machine processing, making it a key problem for businesses; and AI is the key to making this unstructured data useful, says IBM's CMO for North America, Rashmy Chatterjee.

Unstructured data is a catch-all term used to refer to information that does not have a pre-defined data model, or is not organised in a pre-defined manner, which makes it very difficult for computers to understand. Text, images and voice recordings are classic examples. AI systems can assist with this, as they can 'understand' information in text files, interpret the content of images and transcribe conversations. Chatterjee was speaking at VentureBeat's MB 2017 conference.

Understanding of context, like emotions, is not possible with today's computers, but machine learning systems like IBM's Watson APIs can be used to solve that.

Adam Beales of IBM, who works with Watson, was speaking at an AI discussion we attended in London this week. He said that cognitive computing (the simulation of human thought processes) enables Watson to understand large quantities of unstructured data. Beales described the APIs that Chatterjee mentioned as the 'Lego blocks' of AI, which can be used by other systems to enable, for example, natural language processing and visual recognition.

One example of Watson's use with unstructured data sets is with Woodside Energy, an Australian oil and gas company. IBM has used the system to create a virtual advisor that can examine the existing body of data (more than 20,000 documents) created by the original engineers, many of whom have now retired, and present it to new employees.

Other firms are working on the challenge of unstructured data, including Google, Microsoft and Amazon. The work is seen as key to advance the development of artificial intelligence for business.