Hadoop Summit 2014: $50bn of big data market in 2020 will be driven by Hadoop - IDC

Forrester Research finds that 81 per cent of large organisations are "interested in Hadoop"

The big data market is predicted to be valued at $100bn (£60bn) by 2020, and half of this will be driven by Hadoop, according to estimates from analyst firm IDC.

The figures were shared by Rob Bearden, the CEO of Apache Hadoop developer and distributor HortonWorks at a keynote session at Hadoop Summit 2014 in Amsterdam.

IDC predicts that the big data market will be worth $27bn (£16bn) by 2015, of which $6bn (£4bn) will be driven by Hadoop.

And the analyst firm predicts that Hadoop will drive $50bn (£30bn) of the market by 2020, of which $18bn will be on services, $17bn on hardware and $15bn on software.

At the summit, Forrester Research analyst Mike Gualtieri said that 81 per cent of large organisations were interested in Hadoop.

HortonWorks' Bearden explained that there were three main drivers for enterprises opting to implement a Hadoop solution - business, technical and financial drivers.

The business drivers he said are for many companies to switch from "reactive analytics to proactive customer interaction".

"They can get access to mobile, social and clickstream data, and if they do it right they can get information about customers as well as their own supply chain. So they will have better data sets that will ultimately enable them to provide more value to their customers," he said.

Bearden said many enterprises' existing systems are overwhelmed with the amount of data they need to process, while the cost of data systems as a percentage of IT spend continues to rise.

He added that companies thus far have generally been reacting "post-transaction" - analysing data between two days and two weeks after a transaction. He believes that now, organisations are shifting to "pre-transaction insight", and that Hadoop had played a big part in this move.