Beware of big data visualisation tools that claim to help you derive hidden meaning - Pentaho

'We're so good at detecting patterns that we detect patterns that aren't there,' says Wael Elrifai

CIOs shouldn't be fooled by visualisation tools that claim that they can enable enterprises to explore datasets, according to Wael Elrifai, director for enterprise solutions EMEA at Pentaho.

Speaking at Computing's IT Leaders Summit 2015 yesterday at Home House in central London, Elrifai warned CIOs to be wary of the promises of visualisation tools - which have grown in popularity over the past decade - as deriving valuable insight from them is difficult.

"Don't be fooled by visualisations - they're great, but there are [vendors] out there that talk about exploratory capability of visualisation and being able to visually explore data sets, but the reality of the matter is that it's not that easy," he said.

Elrifai said the number of correlations that visualisaton tools, which generally flatten information into two or sometimes three dimensions, enable the user to derive is limited.

"We can draw correlations between two [pieces of data], maybe three. But four? Can we even display five on there? No way," he exclaimed.

"Do real systems only have one real cause? If they did then you'd be able to [understand that] without looking at anything. If you're looking at real systems, these correlations can't be inferred using visualisations; it's just not that simple, you need to approach these by analysing the data directly," said Elrifai.

Elrifai added that another problem with visualisations is that those analysing them may find patterns that they're subconsciously trying to find.

"As human beings we're really good at detecting patterns. We're so good at it that we detect patterns that aren't there," he said.

"When you go into something expecting a solution, you'll find it," he stated.