Three key DevOps trends: DataOps, GitOps and hybrid teams
As DevOps matures, new models emerge
As DevOps matures, so it inevitably splinters into sub-genres designed to serve particular niches. Indeed, sometimes seems like barely a month goes by without a new Ops portmanteau popping up. Here are three that have been garnering techies' attention recently: DataOps, GitOps and hybrid DevOps teams.
DataOps
DevOps, as we all know, brought together the fractious Developers and Operations tribes to overcome blockages to code release caused by poor communication, misunderstandings and a reliance on non-standard infrastructure which defied automation.
Many working at the cutting edge of cloud native deployments and machine learning believe a similar revolution is now required with data.
According to this theory, bottlenecks in the pipelines that deliver data stand in the way of progress with analytics in the same way snowflake servers did with code delivery in the pre-DevOpsian era.
DataOps is about bridging the data scientist - DevOps divide in a direct analogy to how the old Dev vs Ops schism was overcome.
Currently data scientists use their own tools and work in their own environments, and when they think they've discovered something with their models they haul it to the wall and chuck it over to the DevOps team for them to turn it into a product, then sometimes get annoyed when this doesn't happen as expected.
DataOps is about getting data scientists, DevOps engineers, data analysts and business analysts all around the same table to focus on data pipelines so that no-one has to chuck anything over any wall. By putting their heads together, they aim to improve the quality of analytics and reduce the cycle time required to produce meaningful insights.
If you like, DataOps is where DevOps meets big data. What that means in practice is lots of automation: if it can be automated it should be automated.
For this, the data stack must be standardised as far as possible, and analysts and DevOps teams must learn to speak the same language, and sit down to agree in advance what constitutes the relevant data and metadata.
Where DevOps has Infrastructure as Code, a foundational concept of DataOps, according to the DataOps Manifesto is "Analytics is Code" - a sort of recursive view of analytics that says the process of analysing data generates its own data and configurations which then need to be recycled into the pipeline. Which means that all data operations that feed business analytics process such as BI should ultimately be modular, automated and easily and instantly repeatable.
Businesses implementing DataOps include Uber, GSK and Shell and their goals include increased efficiency and reduced drudgery by automation, better quality data and faster access to business intelligence and actionable insights.
There has been a steady rise in interest in DataOps over the last couple of years, as this Google Trends report shows.
GitOps
GitOps has been increasing in popularity with the rise of of containers and Kubernetes. As the name suggests, it is based on the versioning and source control system devised by software genius and self-described git, Linus Torvalds.
It can be seen as the logical destination of the idea of Infrastructure as Code.
GitLab defines it thus:
GitOps == IaC + MRs + CI/CD
Or Infrastructure as code plus merge requests, plus continuous integration/continuous delivery.
More prosaically, it is a method for enabling continuous deployment for cloud native applications. Continuous deployment means that when developers make a change to an application it goes live immediately and automatically, using merge requests as the change mechanism for all infrastructure updates which then take place via the CI/CD pipeline.
The central idea is that there's a Git repository that acts as a single source of truth. In this repository are stored all the instructions and configurations and artefacts needed to build the desired infrastructure. Connected to this single source of the truth is an automated CI/CD pipeline to make these instructions a reality, by building the infrastructure in the cloud.
Then if the developer team needs to update an application they just do so in the Git repository and processes take care of everything else.
What is so helpful about GitOps is that developers don't have to switch tools to deploy the application. Everything happens in the version control system they probably already use for coding.
While it was developed with containers and Kubernetes in mind, GitOps can be used with any Infrastructure as Code setup.
The GitOps concept is being deployed by Fidelity Investments, Deutsche Telekom, and the National Bank of Australia, among others, with the aim of speeding up development with their teams able to make changes and updates to complex applications in the cloud in a safe and secure manner.
Hybrid teams
For our third topic, just to make a change, I decided not to go for another -Ops mash-up, although I could have easily picked AIOps, DevSecOps or even NoOps as rising stars in the DevOps firmament.
There has been a fair bit of chat over the last few months about hybrid DevOps teams. Hybrid teams become relevant once the first stages of DevOps have got off the ground
Once again, it's about breaking down barriers. Whereas DevOps brings the Devs and Ops tribes together, and DataOps seeks to open up the club to analysts and data scientists, so the hybrid DevOps team seeks to welcome in more business-oriented roles (BusOps?).
The thinking goes that DevOps is not just a technical undertaking, it's actually a business proposition that calls for skills and experience outside the usual remit of developers, operations specialists and engineers to allow software development and deployment to be an integral part of the overall business strategy.
A survey last year by the DevOps Institute found that process skills and knowledge are now the top skill sought for DevOps initiatives, surpassing even automation skills.
After DevOps has bedded in, the survey's authors point out. "The next logical stage is to establish a process and rules through which the technology will be used."
The ideal candidates for a DevOps role at this stage are people with a holistic view of the business, an understanding of issues face by developers, ops teams, and the possibilities of automation. That's because "human, process, knowledge and automation skills are equally important".
Hybrid teams could be seen as seeking to 'normalise' software production, a step on the journey to making it a key department like finance, HR or marketing.
Author spotlight
John Leonard
More from John Leonard
Ofcom fines TikTok £1.9m for failure to provide child safety information