DevOps: the need for speed and the tools of choice

A summary of Computing's research into DevOps priorities and tools

Engineering, including software engineering, is all about tradeoffs. For example there is often tension between speed, quality and security. One of the questions explored in this year's Computing research was how DevOps, with its emphasis on quick releases and rapid iterations, can achieve speed without adversely affecting quality and security.

There is a marked contrast between agile approaches to software development like DevOps, with their focus on speed and responding to feedback, and the Waterfall approach, which emphasises the quality of finished product. Waterfall practitioners spend a long time nailing down specifications, and an even longer time coding and fine-tuning their applications. When they finally emerge, the applications should be pretty much fully formed; however, there is a risk that the users' requirements will have changed in the time taken to develop them. History is littered with such examples, such as the failed £12bn NHS Patient Management System.

Designed to navigate shifting requirements and take in abrupt changes of course, DevOps is predicated on constant change: create an approximate version of what the customer needs, road test it and tweak it repeatedly in response to feedback - or if necessary chuck it out and start again before too much effort and money has been wasted.

Perfect is the enemy of good

Perfect is the enemy of good in DevOps world where the cardinal sins are inflexibility and delay. But where does that leave quality and security? Moving faster could easily mean sloppiness, resulting in and buggy software and vulnerabilities that can be exploited before they are patched. Too much emphasis on the delivery date could result in a minimum viable product (MVP) which is in fact unviable - at least in the eyes of users.

According to DevOps practitioners who completed a quantitative survey as part of Computing's research programme, it's quality that is the main area of concern - although encouragingly the biggest proportion said that none of these lose out with the adoption of DevOps.

Oddly, 16 per cent thought that speed suffered under DevOps. We can only advise these people to rebuild the wall at once because DevOps obviously isn't working for you.

The importance of initial quality depends very much on the application. If your software is central to aircraft flight controls, or if it is used to power a heart monitor, obviously quality matters hugely more than if it is a video game where however annoying a glitch might be, it won't result in injury or death.

"That's why medical software and aerospace software is so expensive," commented a focus group participant.

For less critical software, metrics should be devised to balance the level of quality with the cost and time taken to make improvements. Calls to a help desk are a simple measure.

"You can sell a product that's going to give you one customer help call a day. It's going to cost you another 90 per cent to develop it so it's one call a month…" a product director in a technology firm explained.

Asked to choose between speed, security and quality in terms of priorities over the next 12 months, improving quality was the main focus 43 per cent of our respondents, with 32 per cent prioritising speed and 22 per cent security. The low priority given to security is interesting. One interviewee felt that this might be down to the fact that the security of software produced under DevOps is seen as being no worse than that using other methodologies, and therefore effort is being concentrated on the more noticeable areas.

Download the Computing DevOps Review 2017

DevOps: the need for speed and the tools of choice

A summary of Computing's research into DevOps priorities and tools

The cloud-DevOps synergy

As a more general priority, security also lagged behind tools, automation and investigating platforms such as cloud, which was top of this list. The synergies between DevOps and cloud are obvious. Cloud, particularly Infrastructure and Platform as a Service (IaaS and PaaS), makes it possible to scale the virtual infrastructure up and down as required by the application and to integrate it with supporting services.

Many security and compliance requirements are taken care of by the cloud provider, too, reducing the need for this part of the testing process, perhaps another reason why security features lower on the list of concerns than might be expected.

Cloud also gives developers access to many native and third-party tools and services. Such is the pace of change in cloud platforms that some people we spoke to believed that AWS has become a specialism in itself.

Cloud providers have been quick to add DevOps-friendly features to their platforms, including so called 'serverless' platforms offering function as a service. This sits well with the microservices architectures which are popular with agile developers. Also available are cloud-based continuous delivery and continuous integration tools, automated testing and deployment, and the list of cloud based DevOps tools grows longer by the day.

Our interviewees saw serverless in particular as a game-changer, at least for simple applications, as it could allow them to leapfrog the need to deploy containers such as Docker.

"Serverless is an evolution of containers. As long as you're operating your cloud service efficiently, there's an argument that you don't need containers," said the head of platform delivery in the travel sector.

"I don't think containers are the utopia people think they are, I think they are going to be outstripped by serverless," that person added.

Last year, everyone was talking about containers, but this year Docker may well be looking over its shoulder. Asked whether they would go for serverless or containers, the answers were pretty evenly split.

Containers are very useful for making applications portable, but were also thought by our interviewees to increase complexity. Serverless cloud was a way of simplifying operations again. Why would you use containers when you've moved to the public cloud?

The attractiveness of serverless is connected to the advance of microservices, which is a way of breaking down applications into component functions. Forty-two per cent of our DevOps adopters were moving in this direction. Why? Well the main reason was to break up complexity as well as enabling faster development and changes.

When it comes to serverless cloud, there is no doubt as to who is the biggest beast in the jungle, AWS Lambda. In fact none of our interviewees mentioned the alternatives from Microsoft, IBM, Oracle and Google, who all have some serious catching up to do.

"I hate to say it Amazon is utterly unassailable. We did a sort of test to see if we could become cloud provider agnostic and effectively if we could run on anything we could, but they're the ones with the API," said a CIO in insurance.

One issue that organisations will inevitably face is deciding on tools. With every developer having his or her favourite some degree of standardisation will be necessary.

Often the choice will come down to best of breed or integrated stack, but there are other considerations too, such as whether the tools are open source.

DevOps tools: going up, down and treading water

Tools ebb and flow in terms of their popularity and it's important to select one that will last. There's a simple rule of thumb one of our interviewees said: "If it's agile and open source it's going up, and if it's an old enterprise brand it's going down".

High among the risers were Docker and AWS Lambda, which we have just discussed. Right at the top though was the online code repository and versioning system GitHub. While not open source itself, GitHub has become central to many open source development tasks.

Atlassian, owner of Jira and Bitbucket, which is rapidly scooping up other DevOps tools like Trello was also one to watch, our respondents said.

"Atlassian is taking over the world," said one.

The old enterprise brand Microsoft is an interesting one. While plenty thought it was going down more said it was on the way up on account of its development tools, IDEs and languages, the new support for open source and the new Dev-friendly features in Azure.

"Microsoft, Amazon and Google. Just tons and tons of new offerings coming out," said one respondent.

The going down list is a veritable who's who of old enterprise brands, headed by Oracle and SAP and also including Apple. Oracle has some catching up to do in the cloud stakes if it wants to attract DevOps. The issue seemed largely around pricing, with one respondent accusing Oracle of "taking the piss".

Apple was seen to have missed the opportunity to appeal to developers as some sort of cool cloud provider. "They'll never come back now," was one comment.

A number of tools and vendors were thought to be treading water. These include configuration stalwarts Puppet and Chef, whose original model has been encroached on by Docker and serverless. It also includes log analytics tool Splunk, which was thought to be a good solution, but pricy.

"The purchasing model for Splunk just doesn't fit with DevOps because they charge for the number of instances which we can scale up," remarked an interviewee.

Download the Computing DevOps Review 2017