Yahoo! spins off big data analytics engineers

Yahoo! has spun off its Apache Hadoop engineering team to create a new company, known as Hortonworks, which will focus on helping firms deal with brain-busting volumes of data.

Internet firm Yahoo! played a critical role in the development of Hadoop, which is now widely used in big data projects – a fast-growing trend for firms looking to glean business insight from vast, real-time sources of data.

Hortonworks aims to develop technology and services that will make it easier for companies to store, manage, process and analyse enterprise-relevant data, and will focus on making Hadoop easier to deploy, more robust and easier to integrate and extend.

“We anticipate that within five years, more than half the world’s data will be stored in Apache Hadoop,” said Eric Baldeschwieler, chief executive of Hortonworks.

Hadoop is becoming an increasingly popular tool among data analytic vendors and is being incorporated into products from vendors such as IBM and EMC.

But some market watchers believe Hadoop will remain too complex to be widely used within the enterprise – at least for the next few years.

Hortonworks aims to decrease that complexity and accelerate adoption and is already partnering with storage firm NetApp to develop business-ready service.

“As businesses generate, organise and process more data than ever before, the importance of Apache Hadoop in the enterprise has never been more apparent,” said Val Bercovici, a cloud computing professional at NetApp.

Silicon Valley venture capital firm Benchmark Capital has invested an undisclosed amount in the establishment of Hortonworks.