Firms unable to use real-time data due to lack of infrastructure

Jason Nathan, group MD for Data at Dunnhumby said that supermarkets still batch process sales data as their infrastructure can't yet support real-time data flow

Many firms, including supermarkets, are struggling to use real-time information as their infrastructure can't manage the data flow.

That revelation was made by Jason Nathan, group MD for data at data analysis firm Dunnhumby, which was originally set up by Tesco to analyse and extract value from its information.

Speaking at a recent Computing IT Leaders Forum event, Nathan said that many firms are now building new networks to cope with the demands of real-time data.

"The vast majority of retailers stil cannot see sales from their tills at a single moment," said Nathan. "It's batched by hour or even day. So if you've bought something in shop, the majority of that sales data is batch processed later.

"We encourage them to build up that infrastructure, they can then improve their services by not trying to sell customers something they just bought for instance, but that infrastructure isn't quite there yet," he added.

Nathan explained that other companies are attempting to move into the space in an effort to own those data assets on behalf of the retailers.

"And there are loads of players who are trying to find out if you can work out from people's phones which shops they go into, without needing to buy that data."

He argued that this means that the value of that type of data is hugely dimished, if it becomes so easily available. And the increasing numbers of firms who help retailers get access to, analyse and process data, further dimishes the money to be made from it.

"Everybody has to get paid, everybody who contributed data or infrastructure for you to see an ad or other form of content has to get paid. Everyone who contributes to you receiving offers, they all need to get paid. So as there are more people either the value of what they get paid goes down, or they each get a smaller slice of the pie."

And with so many firms involved in the generation of things like retail offers, the return that retailers can expect can also be threatened. For Nathan, this means firms need to have a considered data strategy in place, so they know what they're aiming for, what it's likely cose will be, and its associated return well in advance.

"So you need a data strategy to make it work, otherwise it all costs too much," he concluded.

Nathan was speaking as part of a panel session, which earlier had argued that the democratisation of data can lead to organisational problems.