How to become a data-driven 'algorithmic enterprise'

Mike Mason, technology activist and adviser to the CTO at ThoughtWorks, explains how to instil a DevOps culture, and the value of open source to the enterprise

The next big thing for IT leaders today is to become a data-driven 'algorithmic enterprise'.

That's the opinion of Mike Mason, technology activist and adviser to the CTO at ThoughtWorks. Speaking to Computing recently, Mason explained that an effective data strategy is needed to solve traditional problems around legacy applications and multiple, conflicting data sets.

"Against a backdrop of increasing speed of technological advancement - which is itself a challenge to which IT leaders must respond - the next big shift is to become a data-driven, algorithmic enterprise," said Mason (pictured).

"Harnessing data is absolutely crucial for incorporating the latest generation of machine intelligence, but most organisations suffer from having data locked up in legacy systems or dispersed across a variety of databases, often with poor data quality and conflicting sources of truth.

"IT leaders must implement an effective data strategy to unlock, preserve and grow their data assets. Data strategy often relies on an underlying platform strategy - such as public or private PaaS - in order to manage and process data, as well as rapidly deliver applications that unlock business value. In our experience, organisations have difficulty gaining full value from cloud, often building superficial private clouds which we caution against.

"With regards to underlying technology strategy, the questions every leader is faced with include: where do I invest, when do I experiment and what can I afford to ignore? Planning across the entire technology ecosystem is no easy task and with the pace of change accelerating, planning cycles are compressed.

"In order to thrive, leaders need to understand the implication of the changing technology landscape. We encourage our clients to assess the efficacy of their technology choices and collaboratively build a roadmap."

Mason explained how to encourage an agile culture, something that he believes can benefit even a more traditional enterprise.

"Agile culture usually begins with grass-roots enthusiasm from developers, and from top-down executive sponsorship by leaders who understand the need for greater agility. Unfortunately this often leaves a 'frozen middle' of managers who don't understand, are defensive about their existing practices, and need significant focus in order to change their ways of working. It's best to start small, with an individual department or team. Teams must be empowered to be creative, make decisions and work together, all while being flexible. As benefits of agile are realised, sharing beneficial results across the entire organisation will generate excitement and help lower the resistance to change."

Giving his top tips for introducing agile into an organisation, Mason said to begin with feedback cycles.

"The first tip for creating an agile culture is to focus on improving feedback cycles. Seek high quality feedback in every part of the process, but most importantly from the end-user or customer of your software. Ensure feedback cycles are short; feedback needs to be given often and early. If project decisions are driven by something other than concrete feedback and evidence, you're on the wrong track. Only by responding to exactly what your customer wants can you set your organisation up for success.

"The second tip for creating an agile culture is to build responsive teams that are empowered to act. If possible, consider small 'two pizza teams' that can own a particular product, or even a key business metric, and who are equipped to move the needle on that metric. This will likely mean breaking down siloes, for example allowing the team to deploy and operate their software using a self-service cloud, rather than throwing the software over the wall to operations. At the end of the day, software is built by people and you need the right talent with the right support. An environment that rewards team collaboration over the heroic acts of individuals is a good place to start."

His third tip was to continually monitor and guide the process, and not think of it as a one-off burst of training.

"The third and final tip for companies seeking to benefit from an agile culture is to continuously provide guidance. Taking risks, learning and sharing information on failures is simply part of the the agile mindset. With each feedback cycle there may be new prioritisation. Listening to what the customer actually wants and understanding what is valued by the customer is key. Remember that a "customer" may be an internal stakeholder group, not necessarily an external party. With focused, ongoing guidance teams are positioned to understand changing needs and how they can adjust to deliver exactly what the customer wants."

[Turn to page two]

How to become a data-driven 'algorithmic enterprise'

Mike Mason, technology activist and adviser to the CTO at ThoughtWorks, explains how to instil a DevOps culture, and the value of open source to the enterprise

Many organisations have been using agile methodologies in software development especially, for some time. But fewer have yet adopted a DevOps culture. Mason advised that this adoption begins by aligning the goals of the two teams.

"DevOps culture is exactly that - a way of thinking about how to bring development and operations closer together to achieve better outcomes. DevOps isn't a team of people or a product that you can install, although good DevOps practices can be significantly enhanced through good tooling.

"Start by aligning both development and operations teams to the same goals - everyone should strive to deliver feature-rich, high-performance production systems that delight customers and that are easy to maintain. Building teams that include an operations person can go a long way to creating understanding and empathy between these two traditionally siloed functions.

"According to the 2016 State of DevOps report, organisations with a healthy DevOps culture:

"The report, in its fifth year, clearly shows that a high-performing IT organisation delivers significant business benefits such as more rapid releases, flexibility to changing business conditions, and the ability to run experiments and gain data to guide business decisions. But DevOps isn't just a thing you do and there is no one-size-fits-all approach. It requires changes across the organisation that impact people, process and culture. It's a journey worth exploring and ThoughtWorks continues to recommend adopting a DevOps culture because it will have a positive benefit for many aspects of an organization."

Open source is also a topic close to Mason's heart, and he believes that it's one of the best ways to encourage innovation. He explained how an internal open source, or inner source culture can benefit organisations.

"Today's quickly changing environment requires rapid innovation and a steady flow of continuous improvements," said Mason. "The collaborative, merit-based nature of open source ensures that the best ideas quickly flourish and can be reinforced by others also contributing their efforts. Some organisations have adopted an 'internal open source' model where teams use a private GitHub-like system to share code across the firm, reaping many of the benefits of using regular open source.

"As with all third party software, however, consumers of open source must ensure the tools are fit for purpose, performant, and free of bugs or security glitches. With security an ever more important topic, it's vital to keep up to date with fixes and enhancements that are made to an open source library, and this will require some amount of ongoing effort.

"A decade ago most enterprise software was closed-source, proprietary and expensive. Fast forward to today and almost every piece of software included in ThoughtWorks Technology Radar is open source. That's not just because we're fans of open source; the world's innovations in software are driven primarily by product companies such as Google, Netflix and Amazon, and they release their software for free.

"Many enterprise organisations open source their important software assets to attract new recruits and validate themselves. This creates a virtuous feedback loop: innovative open source attracts good developers who are in turn more likely to innovate. As a side effect, these companies' frameworks and libraries are some of the most influential in the industry. This represents a big shift in the software development ecosystem and is further proof of the efficacy of open source software," he concluded.