Lloyds is streamlining with automation and DevOps

Lean methodologies have more than halved the time taken for testing

Lloyds Banking Group (LBG) is a widespread and complex institution, full of different teams using many different software bundles and apps. That has made change a drawn-out process in the past, but the Group began adopting DevOps in 2015 in an effort to change.

"[It's] a very complex network of people and applications that you have to deal with," said chief engineer Terry Marsh at Computing's DevOps Summit this week. "We want to make sure that we can deliver changes that deliver the best customer experience, but we've got to do that at pace...otherwise we'll be left behind."

Marsh's team used a lean approach to conduct a value stream analysis, working out the best place to focus on to make the improvements that they wanted. As an example, he discussed a set of applications (more than 30) that use a single platform:

"[We] have to make sure that when they go into a production environment, they're not going to impact existing applications. We have a package regression cycle, which means that once a project team have got their code and are ready to be deployed into production, we bundle up the different projects into a package and we put them all into this package regression test environment - which is kept in close synchronisation with production functionality - and we run some functional tests to make sure that all [the] functionality that's in production isn't impacted by these changes."

Running the cycle is time-consuming, but important to avoid failure - and Marsh's team has dramatically lowered the time requirement over the past three years.

"We did a value stream analysis and worked out that the best part to focus on was automating the testing skills…

"We also looked at the environments. We used to have our systems integrated through the other test systems...and what we were finding was that the cycles were being interrupted because the other test systems' weren't always available. So we looked at virtual services and stubs… By building virtual services we were able to run the functionality through on the package regression testing time while improving the environments' stability.

The team also automated the deployment process, and used lean methodologies to look at the overall process and handoffs between teams and testers. The final step was to look at orchestration, which is when the team started to look at DevOps tools, and string together the release and testing automation to push reports out more quickly.

"We managed to lower the duration of our package regression test cycle from nine days to 3.5 days… We've got a 260 per cent overall faster time to market in that test cycle," said Marsh. These were figures he could use to prove that the process works and secure further sponsorship.

"[Using the new system,] our area's been 18 per cent more efficient than it was the previous year," he said. "We've delivered 18 per cent more work, but we've done it for the same amount of money.

"We've had pockets where we've demonstrated that our release automation is five times faster than it used to be, manually; and...our testing automation can be up to 20 times faster."

Marsh and his team has so far rolled out the new regression testing environment for more than 30 applications, but there's still work to do: the team still looks after another about 60 more.