Attracting and retaining data scientists: not easy unless you're in the financial or technology sectors

'Unless you're an Amazon, where you're perceived as a technology company, it will always be hard to attract and retain these types of talents,' says Jan Karstens, CTO at predictive analytics firm Blue Yonder

As an organisation that was founded by a former particle physicist from CERN (the European Organisation for Nuclear Research), a background shared by many of its employees, predictive analytics firm Blue Yonder has little problem hiring and retaining data scientists. For other companies, though, such as the company's customers in the retail and manufacturing sectors, things are not so easy.

"Our customers are asking us all the time: how can we build up this talent?" said Jan Karstens, CTO at Blue Yonder.

With data science jobs at a premium, he told Computing, companies face a series of challenges, first in attracting and second in finding a productive and rewarding place for such candidates. Some sectors start with an inbuilt disadvantage: they are simply not the types of environment that are typically attractive to data scientists. People with strong analytical and statistical skills are often employed in the financial sector as "quants" (short for "quantitative analysts"), where they are rewarded handsomely for their talents. But for retail?

"Unless you're an Amazon, where you're perceived as a technology company, it will always be hard to attract and retain these types of talents," Karstens said.

Then there is the question of size. Organisations that are most likely to employ data scientists are large, and this means that it can be hard to create a productive niche for the new role.

"Even if you are perceived as a technology company, the question is where does this person end up? Let's say you are hiring one or two or three data scientists - they are hardly recognisable in a company of 10,000, 50,000 or 100,000 employees. Which task do I give them and how do I make them relevant to my company?"

Among Blue Yonder's large customers in the fiercely competitive retail sector, Karstens said there is a recogniton of the need for data scientists to give them a competitive edge, but that they struggle to recruit them in sufficient numbers or to train them fast enough. A collaborative approach has proved helpful, he said, with Blue Yonder's data scientists helping to get their in-house counterparts up to speed as they work on predictive analytics issues.

"We sell them our predictive analytics platform and our data scientists work alongside the one, two or three people they've managed to hire so we can give them a head-start in how to approach these sorts of problems," Karstens said.

This assistance is important, Karstens explained, because businesses not only need data scientists who can work with predictive modelling, they must also quickly acquire a good understanding of their organisation's business model to make sure they are asking the right questions.

While data science will eventually become a mainstream role as more people pursue it as a career, for now he feels a mixture of internal hiring and external consultancy is the best option for most.

Another phenomenon that's fuelling the demand for data scientists - and thus contributing to their rarity - is companies becoming excited by the talk of big data, provisioning a technological solution and trying to make it work for them.

"They say ‘if I get Hadoop cluster then I'm doing big data'. But if you start asking them questions, like ‘what data do you have in there', and ‘what are you going to do with that data' there's no response at all, because they've bought into the technology without thinking of the use case," Karstens said.

The shortage of data scientists is unlikely to be solved soon. All of this, and a general lack of understanding about where big data technologies can really make a difference makes it likely that the adoption of analytics-as-a-service for testing analytics, creating pilot studies or scaling up existing programmes will increase.