Stop wasting money on digital transformation, start building the team to make it happen

If you want to digitally transform your organisation, build a team that can do it, says Dr Alberto Arribas, Science Fellow at the Met Office's Informatics Lab

Having been around for over 150 years, the Met Office is well known as a weather forecasting organisation but fewer people are aware of the magnitude of the technology operation taking place behind our walls. Not only has the Met Office recently acquired one of the most powerful supercomputers in the world (top ten) but the rest of our IT estate is equally impressive, with as many servers as Ebay and data volumes comparable to Facebook.

As most companies around the world, the Met Office has been facing profound challenges in the last few years: big data from simulations and sensors, social media in mobile devices, machine learning for data analytics. The list is long and not a single area of our operations has been left untouched.

However, the problem is not the change, even if it is happening in a multitude of fronts at the same time, the problem is the rate of change. The speed is such that current processes and approaches cannot adapt and evolve fast enough. So, if you need to digitally transform your business, you are not going to find a solution by trying to do exactly the same things you were doing before but using a different technology. You will need to do things differently. In other words, business as usual is not an option, requiring a fundamental shift at all levels of the organisation.

Because of the size of our technology operations, the Met Office had to face these dilemmas earlier than most. Our answer to the challenges was to create the Informatics Lab (which started operating in full in early 2015 with two aims: making environmental science and data useful; and transforming Met Office's culture and ways of working).

The Informatics Lab has been a success story - picking some awards and developing successful collaborations with the likes of NASA and Amazon in the process. So, two years down the line, what have we learnt that could be useful to others?

To address multi-faceted problems you need a multi-disciplinary team

At the Informatics Lab we created a team with scientists, technologists and designers so we had all the skills we needed within the team.

People do not join the Lab to be the 'scientist' or the 'designer' of the Lab, people join the Lab to solve hard problems together. That means that everybody has to learn enough science, coding, and design. Enough not only to understand whatever the challenge at hand is but, even more importantly, to successfully understand the other members of team and their perspectives too. Therefore, a non-negotiable requirement in the Lab is to be ready to learn and adapt continuously.

The combination of multi-disciplinary skills and the ability to learn quickly is extremely important because, by reducing external dependencies, you reduce downtime and increase productivity, solving problems much faster.

From an organisation's point of view, you almost certainly have most of the people you need in a Lab like ours in your organisation already. At the start of the Lab we recruited from inside the organisation only. We did this because you need people who know what the problems in your organisation are. Leaving me aside (an old hand having been in the organisation 15 years, a bit more on that later on), the sweet spot for us proved to be bright staff that had been in the organisation around three years, long enough so they knew the organisation but not so long that they had been brainwashed into believing that whatever their department was doing was the optimal way of doing it.

We have also kept the team small, always under 10 people so we can minimise management and comms overheads, again in order to boost productivity. You can see how making things happen is the driving principle and we do things in many fronts, well beyond technology and including areas such as: changes to physical space, different ways of organising, recruiting and manage staff and teams. All of these are critical elements to drive successful digital transformation.

[Turn to page 2]

Stop wasting money on digital transformation, start building the team to make it happen

If you want to digitally transform your organisation, build a team that can do it, says Dr Alberto Arribas, Science Fellow at the Met Office's Informatics Lab

Executive support is critical to protect the Lab and ensure gearing with rest of the organisation

If you need to transform an organisation you will need to do things differently and to do things differently you will need to take risks. Of course, you do not want to bring the organisation down while taking risks so you need to create a safe space.

That is what the Lab is, a safe space for staff to take risks and attack problems without fear of missing targets or failure. Actually, taking risks in a controlled way, drastically reduces the overall risk for the organisation as a whole.

To achieve that, the Lab reports directly to the Exec and has independence (budget, staff, IT) from other parts of the business. Crucially, the Lab does not have a target but a mission. This does not mean the Lab is not scrutinised - every quarter its past performance and future projects are critically reviewed at Exec level - but it emphasises that the role of the Lab is to solve strategic problems to help transform the organisation.

Ensuring the correct gearing between the Lab and the rest of the organisation, the flow of questions and know-how from both sides, the linking of teams and projects, is the other side of the Exec's support. This will always be a balancing act and it is not easy but there are some basic principles guiding it: different teams have different roles (operational delivery vs research and prototyping is an obvious one) and the key is to integrate these changing roles within the bigger changes faced by your company. The implication is that you really need to understand your own processes today and where you want to be tomorrow in order to effectively leverage the Lab.

Openness (and developing shiny examples) pays off

Internally and externally we emphasise openness. Our codes are published in GitHub, we regularly give talks, we have open demos sessions inside and outside the organisation, we blog, we tweet. You get the idea.

This achieves two aims: constant exposure helps us find problems quickly but, even more importantly, it helps us to understand what we do. Ask any teacher, you do not really understand anything until you have to explain it to somebody else.

Openness also helps staff development. Members of the Lab regularly present, demo and discuss with technical peers, executive members, marketing managers, scientists, general public. This brings ownership and commitment, creating a positive feedback loop that further strengthens the team and facilitates achieving change.

Finally, if you expose what you do - and particularly if you put some effort into making demos and example entertaining, usable and enjoyable - you collect a lot of positive feedback at all levels, from general public to Executive members … and, when you are trying to change an organisation, nothing helps like getting positive feedback and encouraging from the outside.

Be patient

Even if the Lab is able to create prototypes that provide solutions to problems, it still takes year to bring transformation across the business. Some examples: the use of containers in production took 18 months from the original Lab project, similar timescales were necessary for the use of natural language interfaces in production, changes in the organisation of teams and physical space took over a year, and we expect the new data analytics platforms we have just developed to become part of the next HPC ecosystem in 2020.

So, yes, it takes time and it is not easy but each one of these changes has been positive and the process itself has contributed enormously to create a different atmosphere across the organisation that is helping us to accelerate and achieve the transformation we want and need.

Dr. Alberto Arribas is Science Fellow at Informatics Lab

Computing's DevOps Summit will be held on Wednesday 22nd March 2017 in Central London. Attendance is free for qualifying end users. It will be followed by the DevOps Excellence Awards.