Big Data Summit 2014: Shortage of talent endangering big data projects

By Graeme Burton
28 Mar 2014 View Comments

Organisations' big data initiatives could be derailed by a lack of talent to implement them, according to master data management firm Talend's Mark Carter.

In the US alone, there is an estimated shortage of 200,000 big data professionals who, as a result, can attract bumper salaries - typically between $150,000 and $200,000. "These people are as rare as hens' teeth," said Carter, speaking at Computing's Big Data Summit 2014.

Further reading

However, four-fifths of their time is typically spent on administration and office politics because organisations simply aren't set up to do big data. "Organisations have lots of different data silos and he or she has to get all that data together first," said Carter.

Instead, he said, organisations need to leverage the talent that they have in-house and to make the process easier with automated tools, cutting the in-house knowledge of complex technologies like Hadoop and MapReduce in the process.

"Why go outside your organisation when you can use the resources you have got inside?" asked Carter.

Trying to do big data on traditional databases, though, is also a mistake, he warned.

But most organisations are perhaps not as behind in big data as they might think: the vast majority worldwide still have not advanced to the stage where they are able to do real-time analytics.

"Some people seem to think that big data is like the Holy Grail, something to be attained or reached. It's not, it's more like a journey," said Carter.

Reader comments
blog comments powered by Disqus
Windows 10 - will you upgrade?

Microsoft has made an early version of Windows 10 - its next operating system - available for download. The OS promises better integration and harmonisation across platforms, including mobile and desktop. Will your business be upgrading?

39 %
26 %
14 %
21 %