Stop moving data without reason: Unify the analytics experience instead

If organisations focus on unifying the analytics, they'll spend a lot less time moving, replicating and transforming data

Data storage and mobility has been front and centre for a long time. It underpins every form of the digital leap, and was often sold as building data warehouses and data lakes or pooling and aggregating data. This is now re-packaged as cloud migration, which requires moving all that data yet again. Supposedly, moving from your legacy infrastructure to yet another new system is how you layer on intelligent capabilities like machine learning and analytics. To take advantage of new forms of analytics, you need uniform and central storage. Again.

While moving data to the cloud has been transformative in many ways, in a lot of cases it involves mounting and unpredictable opex costs, inevitable technical debt, and interoperability challenges that the industry is yet to iron out.

It's important to take a step back and ask why you are making these changes in the name of digital transformation

It's important to take a step back and ask why you are making these changes in the name of digital transformation. Insight from advanced analytics is the real goal. You want to improve business services, make your company more agile, cut costs and drive economies of scale, while setting yourself up for future innovation.

It is not to have everything in one place just because that's in style.

In reality, modern organisations have so much data in so many different formats and locations that no organisation can realistically be expected to move and store all their data in one place and in one format. Data can and should remain wherever it makes sense to be stored, for optimal data and cost management.

Unified analytics platforms

We've moved on from basic analytics using one structured data source to predictive analytics using real-time data and historical data in a wide variety of formats to model, anticipate and provide proactive action - which has shifted the goalposts for what digital transformation can achieve. Be it for financial, operations and IT management, or even on a consumer level of hyper-personalised touch points, the data generated at these nodes needs to be accessible and usable, not central.

What is needed, then, is a unified analytics platform. In other words, unified analytics that can span the businesses.

Letting the data remain where it makes the most sense.

The best way to describe a unified analytics platform is to take shopping on Amazon as an example. I use my phone or my laptop when I choose. I use the interface that is comfortable and convenient for me, and I go to the Amazon website. I don't mind where the product that I want to buy currently resides, I just care that it's delivered in a timely way.

For Amazon, where that product is stored does matter, just like data management optimisation matters to the IT department. But for the business analyst using their preferred SQL queries or the data scientist coding in Python on a notebook or using TensorFlow, the data location shouldn't matter. What matters is the right answer in time to take proactive action. That's what a unified analytics platform must deliver for companies to achieve success in our digitally transformed world.

Building a data-driven business

Providing a unified analytics experience and, in turn, creating a data-driven business has three key elements. These include:

  1. Storing and keeping all the data where it makes the most sense, spanning HDFS and S3 style Object Storage formats including Parquet, ORC, JSON, Avro, among many others. Analysing complex data types like maps, structs and arrays in their original formats maintains analytical efficiency.
  2. Ensuring that the data science community and the business analyst community can access the data for analysis, no matter where it is stored, using the language and tools that they know and trust, whether that's Python code in a Jupyter notebook or a Tableau visualisation with a SQL back end.
  3. Delivering unified analytical insights that make the most of machine learning at the scale required for accurate predictions and the performance required for proactive actions.

If organisations focus on unifying the analytics, they'll spend a lot less time moving, replicating and transforming data. Instead, they'll be able to focus on using analytics to unlock meaningful trends, patterns and relationships in data. Through delivering accurate insights at speed, organisations can positively influence business outcomes and find data-driven success.

Joy King is VP Vertica product and GTM strategy at Micro Focus