The data science skills gap is real, and industry needs both academic and technological input to crunch big data, says dunnhumby CIO

Algorithms without fast and effective application 'defeat the purpose' of big data, says Cosset

The big data skills gap is a real IT industry affliction, and a side of big data information processing which dunnhumby has two factions of scientist to combat, the CIO of dunnhumby, Yael Cosset, has revealed.

Speaking to Computing at Oracle Open World in San Francisco today, Cosset championed the company's Exadata product, which is now central to dunnhumby's ability to meet increasing industry demand for customer data processing, before sharing his views on the skills gap.

"I think as a customer science company, science is the core of our business.

"We probably are very innovative, and many would consider leaders in the industry, around how to absorb and interpret data," said Cosset, "But it was more out of necessity because ,if you don't have good data, no matter how smart your science is, the insight you're going to get is not going to be the best insight you should be able to deliver to you partners."

Citing "science" as the very core of business at dunnhumby, Cosset explained how this goes even deeper, into two main disciplines.

"I'd say there are two key areas our science teams focus on. One is around the pure, academic - which I think is perhaps the wrong term - but how to structure models and algorithms," said Cosset.

"The scientists looking at leveraging new science techniques to find answers we haven't answered in the past."

But on the flipside, Cosset explained how "once you create the best algorithm", there needs to be a place to "execute and apply that science".

He said how - particularly when entering a world which is sure to introduce a land grab for medical data processing from a growing wearables market - speed is of the essence, applying models and leveraging data is the tough part.

"While finding the best way to answer to the question is hard - I'm not saying one side is harder than the other, as my [academic] science team would be very upset with that statement - you can't have one without the other," stated Cosset, diplomatically.

"If you have the most fantastic algorithm, but from production to execution it takes a month to apply that model to your data to come up with a recommendation for five million consumers in wellness, it defeats the purpose.

"So you need pure science innovation, and to be looking for ways to apply these models and leverage this technology to process at scale."