Big Data Summit 2014: 'Data scientists are like yetis - there aren't many of them' - HP

The size issue of big data is no longer an problem, but lack of data scientists is, says HP's big data solution marketing director

A shortage in big data skills, particularly for the data scientist role, has led to HP's big data solution marketing director, Dan Wood, likening them to yetis.

Wood, who was speaking at Computing's annual Big Data Summit, told delegates that analyst and media reports suggest that there is a lack of data scientists around, and that these reports must have some element of truth, since he hasn't come across many himself.

"I haven't met that many [data scientists]; maybe they're like yetis - there are not that many data scientists around," he said.

Despite their rarity, Wood explained that data scientists are necessary for businesses to get the most out of their data.

In his presentation, Wood explained that before businesses pick a big data analytics platform, they should think about the use case.

"It's not about how big the data is, how much data there is, it is how you analyse the data and the speed with which data can be moved. I think the industry solved the size issue years ago," he said.

He suggested many different reasons for opting for a big data solution such as complementing CRM intelligence with social sentiment, and emphasised that both structured and unstructured data should be analysed.

When asked by Computing whether HP itself uses its HAVEn platform, Wood said that it did in "several areas of the business". He gave the example of the digital printing division using analytics "to check for failure". He said that HP monitors health information it receives on a regular basis, which it uses to tell the customer that their machine needs servicing. Meanwhile HP.com uses Vertica for clickstream analytics, and Autonomy is used to decode HP's annual employee morale survey, he said.

Wood also revealed that Facebook is using HP's Vertica solution on top of Hadoop to enable the social media network to find trending subjects.