Hortonworks announces HDP 3.0 and increased integration with Google, Microsoft and IBM cloud services

Hadoop vendor adds containers and deep learning support to its data platform

Hortonworks today announced a new major version of its Hadoop-based Hortonworks Data Platform (HDP). Version 3.0 includes containerisation capabilities, GPU support for deep learning and enhancements to the Hive data warehouse for improved real-time database queries. It will be generally available in the autumn.

The new version of HDP is based on Apache Hadoop 3.1. It is designed for managing data at rest in support of big-data-type applications, both batch and real-time.

The company says the platform is "maturing to meet changing enterprise requirements", which include agile development using containers, support for deep learning and machine learning applications that are often heavily GPU-intensive and require pooling of GPU resources, and better support for real-time querying on Hive. There are also enhancements to security and governance features allowing the lineage of data to be tracked from its place of origin to its storage in the data lake, and improving visibility for auditors.

Hortonworks also announced increased integration with two major public cloud platforms.

HDP is now integrated with Google Cloud Storage, making it easier to deploy HDP on the Google Cloud Platform, according to Hortonworks CTO Scott Gnau. The Hortonworks DataFlow (HDF) streaming analytics platform and DataPlane Service (DPS) metadata management platform are also available on the Google cloud.

On Microsoft Azure, customers can now deploy HDP, HDF and DPS natively. They can choose Microsoft Azure HDInsight, a fully managed service providing Hadoop and Spark. The Hortonworks suite has been available on Azure for a few years but native deployment on Microsoft's IaaS is new, explained Gnau.

"You could deploy on Azure prior to this but now it becomes a better experience for customers. So if a customer wants a prescriptive and managed experience they can go with HDInsight (which is a white label of HDP), or if they want full flexibility and ability to customise they can choose Azure IaaS."

The Hortonworks suite is also supported by AWS and IBM clouds, Gnau added.

"We have a multi-cloud partner ecosystem so customers get maximum choice. Whatever cloud or clouds a customer wants, we can get them there."

The company is also furthering its analytics partnership with IBM via the newly announced IBM Hosted Analytics with Hortonworks (IHAH) data platform, which is an integrated analytics solution available as a service on the IBM Cloud.

"With IHAH, users are given a fully-provisioned data management and analytic environment via the cloud, for very quick setup, provisioning, security and deployment. Specifically, the solution comprises HDP, IBM Db2 Big SQL and the IBM Data Science Experience," said Rob Thomas, general manager at IBM Analytics.