Big data skills gap 'not huge' and even 'exaggerated' say Rackspace and EMC

'Skills are growing with demand for skills' argue big data vendors

The supposed IT skills gap that requires "10,000 data scientists" to fill is "exaggerated", claims storage vendor EMC, while Rackspace describes the figure as "aspirational".

These statements came from a big-data roundtable hosted by Rackspace, at which EMC solutions principal Bernd Kaponig said: "I think from my perspective it's not a huge skills gap, it's exaggerated."

He continued: "I think the skills are growing with the demand for the skills, and tools are also getting easier to use and appeal to a broader audience, including management.

"They also get insight into what's happening; it's not just a black box they don't understand. So I think as an audience we're growing in skills. I'm not too concerned about that."

Rackspace's head of technical product strategy, Toby Owen, was inclined to agree. Owen said that while he "kind-of thinks there is a skills gap", he would not "call it huge".

"The demand for 10,000 data scientists may be a little aspirational for some companies who have goals to do this great, big big-data strategy. But I'd agree that the basic skillsets are there, the tooling is getting better to use, and the toolsets being made are clearer, such as BI tools."

Owen believes that big data platforms are becoming easier for end users to operate without the need for new skillsets. He cited the example of MongoDB, which, though it only started in 2007, is now "one of the top-10 databses in the world".

"But how many people have become experts in it in that time?" he asked.

"It's competing very well with established relational databases that are 40 years old. I think that makes a point for getting help [now] - vendors provide help, and service providers have a place for concentrating knowledge. Not every company is going to be able to afford to hire data scientists or a MongoDB DA [data analyst]."

Chris Harris, technical director at Hadoop developer Hortonworks, said: "The way in which we confront this is not about completely ripping out everything you have. What we've done as an organisation is to partner with many organisations out there. So you could take Microsoft Excel and stick it on top of Hadoop. The majority of people out there know how to use Excel, right?"

DataStax EMEA general manager John Glendenning, said: "The data stacks we have are working perfectly well, and there's a huge investment going into companies [like Rackspace, MongoDB, HortonWorks and EMC], as well as the partner network. We're all doing partner networks; we're all working globally."

Harris said that he's not even sure that the so-called "data scientist" even exists in the hyped-up form promoted by many in the industry.

"The mistake many companies make is that they look for this data scientist; this one guy who is going to change your entire organisation.

"Take a step back, and realise that this one scientist is actually a team of people. Look at the data scientist skillset, look at your team, and look at what's required in terms of being able to move it forward, rather than stall yourself on not being able to find this mythical data scientist person."

Is the big data "skills gap" affecting your organisation, and are these vendors just pulling rank to big up their own products, or is this a fair indication of wider industry attitudes towards the supposed need for 10,000 data scientists? Let us know in the comments below.