In the beginning there was BI
A big beast in the IT jungle, business intelligence (BI) evolved to fill a niche: meeting the decision-making needs of large and complex organisations, bringing together disparate silos of data, processing them and pushing relevant information out to where it was needed.
For many years BI specialists had no major competitors in a corporate environment that valued their expertise. Commandeering as they did a fearsome array of tools by IBM, Microsoft and Oracle among others, they enjoyed a symbiotic relationship with the board, feeding strategists the information they needed for planning and being granted a favoured status in return.
Then things started to change. Data got big, new technologies arrived and the board’s demands for timely analytics grew. Organisations of all types began to realise that extracting value from the torrents of data streaming in from their websites and other sources every second of every day could give them a real competitive advantage.
BI now found its position challenged. It was forced to adapt to the changed conditions, in particular the need to make sense of very large and rapidly changing sets of data not easily marshalled into the familiar tabular formats that BI systems require. As a result of this adaption a new species emerged: the data scientist.
Hey, can I get your number?
Suddenly, data scientists are hot property. This rapid increase in sexiness might come as a surprise to even the data scientists themselves, although as analytical types with a broad overview, a talent for spotting trends and a firm grasp of numbers, there is no logical reason that it should.
In short, data scientists are in demand because in organisations of every size and sector data is now a fundamental resource. At the same time, data volumes are increasing exponentially, and with organisations increasingly fighting over the same patch of ground, strategists need someone who can read the digital runes and provide them with evidence-based direction to help them get ahead.