Yahoo takes on Splunk's Hunk to analyse data from Hadoop environment

Yahoo has been a Splunk customer for several years, but now it wants to explore and analyse its Hadoop environment which contains more than 600 petabytes of data

Yahoo has selected Splunk's Hunk, the analytics tool for Hadoop and NoSQL data stores, in order to explore, analyse and visualise data from its Hadoop environment which contains more than 600 petabytes of data.

Yahoo has been a customer of Splunk's for several years and some of its designated teams have been analysing more than 150 terabytes of machine data per day in Splunk Enterprise for use cases including IT operations, applications delivery, security and business analytics.

The internet and technology giant has since wanted to delve deeper into its Hadoop data store and has selected Splunk's big data analytics platform Hunk in order to be able to do this.

Ian Flint, monitoring architect at Yahoo said that Splunk Enterprise and Hunk helped the company to "gain insights into all of our data, whether it is streaming in real-time or historical data at rest".

Yahoo is currently using Hunk in its grid operations group as well as within several other product teams with a view to extending it across the entire enterprise.

"Hunk gives Yahoo deep visibility into our massive Hadoop data stores to help us continuously optimise operational performance," Flint explained.

"Insight we gain from Hunk helps us save millions of dollars per year in hardware provisioning," he added.

Yahoo is also using Splunk Enterprise to unearth more about its product lines including customer preferences, user experience, click rates, performance and IT workflow issues, advertising and marketing campaign popularity and more.

"Splunk Enterprise helps us to maximise revenue by giving product and business teams better insight into our customers, the user experience and any looming issues," said Flint.

Splunk's influence in the enterprise has been growing steadily. In June last year, Computing's own research found that the top ‘specialist' big data vendor that respondents would consider was Splunk, followed by MongoDB, QlikTech and Cloudera.

Fourteen per cent of the sample had gone as far as setting up trial or production implementations with a specialist vendor, among which Splunk, QlikTech, Hortonworks and MongoDB were the front-runners.

However, the number of respondents choosing a specialist vendor for a hypothetical big data project was much smaller than the number of respondents who had chosen a traditional vendor such as Microsoft or IBM.

Computing's Research suggested that volatility in the market meant that there was bound to be a certain amount of waiting and seeing going on as potential customers seek assurances that their choice will still be around two or three years down the line.

But this seemingly hasn't deterred all end users; Computing has spoken to several well-known end users about their use of Splunk in the last year including BSkyB, Credit Suisse and BNP Paribas.

Other high-profile customers include Coca-Cola, NASDAQ, Valve and Shazam.

Readers interested in developments in the big data space should check out Computing's summit on 26 March 2015 on big data and analytics, details can be found here. Qualified end users can attend for free.