Analytics needs to 'grow up' and GDPR is just the push it needs, say tech firms

Shawn Rogers of Tibco and Ali Ghodsi of Databricks share their thoughts on the lessons of the Cambridge Analytica scandal

Attitudes to personal data are are changing. The Cambridge Analytica scandal was a wake-up call for many users of Facebook and other social media platforms as they suddenly realised that highly personal information about them was being used in ways that they could never have imagined - and had never given permission for.

The power of massive datasets combined with data mining and machine learning algorithms was never in doubt, but this was the big data on the dark side.

The authorities have been fighting something of a rearguard action against the unrestricted use of personal data, one result of which is legislation like the GDPR, but what of the big data companies themselves? What do they make of the situation? Computing spoke seperately last week to two senior representatives of such organisations. Both agreed that while there may be some short-term costs in compliance, particularly for large organisations who must now prove they can discover and delete personal data across legacy software solos on demand, change is essential if the algorithm economy is to be a positive for society.

But not all tech firms think this way about GDPR. Earlier this month IBM sent senior representitives to the US Congress to lobby representatives against adopting similar privacy laws, urging lawmakers instead to adopt "third-way" regulation "tailored to America's needs" (shades of China's "capitalism with Chinese characteristics", perhaps).

While they would not comment directly on IBM's actions, neither of those we spoke to thought this was a positive step, and both said they saw GDPR as a plus.

Shawn Rogers is senior director of analytic strategy at Tibco. He also heads up a thought leadership group within that company, a position that allows him to explore the shape of things to come without necessarily speaking for his company. He made the point that analytics has grown from a cottage industry within large organisation to something akin to an operational production line, and that the whole sector must now 'grow up' as a result. Asked whether the 'personal information economy' an idea long espoused by Doc Searls and others - might finally become a reality, he was positive.

"I think the personal information economy will find legs. I do a lot of talking about the algorithm economy, and that's why I'm a fan of GDPR and better data management. I think that pushes the algorithm economy forward which will then bring new business models such as the personal information economy," Rogers said.

As consumers grow more savvy about their personal data things will change, he went on.

"I think the word 'economy' is important. We will get to the point where if I give you my information I'll feel differently vested. Not to pick on Mark Zuckerberg, but a lot of users were OK with the trade but then they found the trade was not quite on the level as they'd expected and that really made them angry."

Companies like Facebook, whose current strategy seems to be GDPR avoidance rather than compliance, will need to find ways to better manage their data he believes.

"They're going to have to figure out how to apply laws by region. Tracking people and identifying religions and social classes - that's a scary thing."

Ali Ghodsi, CEO of data and analytics firm Databricks was of similar mind. Mentioning that his company is fully GDPR compliant globally (something insisted on by Microsoft Azure which hosts the Databricks platform as a first-party service) he said big data needs to build trust.

"I think GDPR is a great thing. It's what we need to make sure the technology is being used in the right way. The whole big data technology revolution is going to have massive impact on society over the next 10 or 20 years," Ghodsi said.

"Whatever happens it's going to be used by all big corporations because that's how they will become more efficient. If the public does not trust it and if the data people are sharing is being used in a weird way they'll think it's a Big Brother society.

"It's a no-brainer. Our customers want to be able to do this, they want to find out what information we have and then to be able to delete it. I don't know why that's such a big deal. It's just common sense."

It's up to companies to sell the benefits of data sharing rather than just scooping it up, Ghodsi said.

"You can say 'fine, we'll delete your data if that's what you want, but then I won't be able to serve you quite so well'."