'Companies will stop hiring data scientists when they realise that the majority bring no value' says data scientist
Data scientists should focus on agile methodology and DevOps if they want to keep their jobs
Data scientists need to stop playing around and start delivering a measurable return for their organisation, or their jobs will be on the line.
That's the view of Gianmario Spacagna, a data scientist for customer and retail banking at Barclays in London, who believes that many of his counterparts spend too much time "playing" and experimenting with data and analytics tools and not enough time focusing on real business needs.
"If you just do analytics, if you just do scripting, if you just make a model and this model does not go into production then at the end of the day you just did research, but your company is not going to profit," he told Computing during the recent Spark Summit Europe.
"It's not a playground. It is not academic. The company wants to make money and you have to solve a problem," he said, adding that many data scientists find it hard to turn their initial explorations into relevant projects and applications.
"This is where data scientists struggle a lot. Companies will stop hiring data scientists, I promise you, when they realise that the majority of them do not bring value," he said.
Spacagna believes that a focus on agile methodology, one of the core principles of which is to "satisfy the customer through early and continuous delivery of valuable software", is essential if data scientists are to remain relevant and not dismissed as the product of industry hype.
"It's how do we apply the scientific methodology of agile to make sure we always deliver something better for the business. This is where data scientists will make the difference," he added. "How do you bring value? You have to know exactly what your business wants."
The data scientist role should be considered as part of a multifunctional DevOps team, he insists.
"In the team you want to cover the whole spectrum of skills. Having DevOps in your team is absolutely fundamental," he said.
"If you're a start-up, the smartest smartest person you want to hire is your DevOps guy, not a data scientist. And you need engineers, machine learning specialists, mathematicians, statisticians, agile experts. You you need to cover everything otherwise you have a very hard time to actually create proper applications that bring value."
Asked what value his team has been able to deliver to Barclays, Spagacna mentioned two projects. First there is a BI analytics "insight engine" that is offered as a free service to all of the bank's business customers. Then there is a project to clean up and "backfill" the retail bank's transactional database, adding in inferred information about the 70 per cent of businesses that are not Barclays' customers.
"Let's say you've got a Barclays debit card and you're spending in a shop and that shop is not our customer. We don't know anything about that shop, so we try to infer from your spending patterns what their business is," he explained.
"Ultimately that will enable further data science applications to be built on top of [the transactional database] because now we have a 10-times higher quality dataset."