Making movies: How comparethemarket.com got Meerkat Movies up and running in months

Comparethemarket.com CIO James Lomas talks about Meerkat Movies and the company's cautious shift to the cloud

When comparethemarket.com CIO James Lomas was suddenly given the task of getting Meerkat Movies up and running in a matter of months, late in 2014, it seemed the ideal opportunity to test the company's nascent shift towards cloud computing.

Already that autumn – as Lomas had exclusively revealed to Computing – the company had made its first tentative steps towards cloud computing with the shift of its travel insurance comparison tools onto Amazon Web Service (AWS), as a means of testing cloud and what it might demand of comparethemarket.com's IT department.

"Our approach to cloud was somewhat cautious in that we have always had the mindset that the 'promised land' of cloud computing is a commoditised service," says Lomas. In porting the travel insurance comparison service to AWS, comparethemarket.com sought to build all the supporting infrastructure around it at the same time, which could quickly and easily be reused if and when the company chose to shift other services into the cloud.

That meant automating as much as possible, as well as eschewing proprietary tools that might make it more difficult to port to an alternative cloud provider if the need arose. "The initial principle was to automate everything. So we are using Docker, Chef and other tools. But automating everything can take quite a long time," says Lomas.

"We want to be able to, or at least have the possibility of, taking the service from Amazon and maybe putting it on to Google or Microsoft Azure instead, if necessary. So we have tried not to embed ourselves too deeply into some of the software stack that Amazon can give us," adds Lomas.

The open-source tool Docker, in particular, has proven to be popular among comparethemarket.com's developers. "It's a very lightweight way of building an image of an application, sitting within the operating system. It's incredibly simple and quick to build Docker images," says senior software engineer Stuart Greenhall.

By using Docker containers, continues Matthew Collinge, associate director of technology and architecture at comparethemarket.com, "we were basically building once, creating a gold binary, and that's the same thing that runs in both test and production. So you gain confidence throughout the process [and] it was a first step towards running our own platform-as-a-service."

Before the travel insurance comparison service went live on AWS, Lomas's team conducted extensive testing, including how they would respond to an instance simply disappearing, and also testing scalability – almost anything that could possibly happen. "What we took away from that was that automation is key. If you don't automate things, trying to repeat them is very hard, and automation is not necessarily as easy as it looks... you have to coordinate lots of disparate systems," warns Collinge.

But to be able to automate technical processes, he adds, it's important to know how to do them manually. "That means, for example, working out things like what sort of configuration you want to apply to the various services that you are operating, what size instances you want to run, and what size discs you need to configure – the topology of the discs within the instance," says Collinge.

The cloud shift also coincided with a move from Microsoft SQL Server, not to Amazon's own DynamoDB, but to MongoDB. That decision was taken, says Collinge, because "there is currently no good story for SQL Server operating on AWS in a redundant, resilient way. So we took the decision to move to a MongoDB-based storage engine." DynamoDB was eschewed to help avoid AWS vendor lock-in.

[Please turn to page two]

Making movies: How comparethemarket.com got Meerkat Movies up and running in months

Comparethemarket.com CIO James Lomas talks about Meerkat Movies and the company's cautious shift to the cloud

Collinge continues: "MongoDB gives you the ability to replicate data between multiple nodes so that in the event of a loss of a master node, another node can step up and become the master. There's a slight interruption while that happens, but it's automated and takes between 500 milliseconds and a few seconds. With SQL Server there would be a manual intervention, which would involve having to restore the last known good backup, which would mean you've lost 15 to 30 minutes of data.

"When you move to the cloud, you have to appreciate that you could lose an instance at any time and that you need to design that in [to take account of the risk] as opposed to hoping for the best."

What that means, adds Lomas, is building the cloud service so it is resilient to failure – making sure, for example, that if an instance goes down, there is enough capacity within the remaining instances to soak up the load.

Going to the movies

As a result of all this work on the travel insurance comparison service, Lomas's IT team had already built much of the tools and infrastructure they would need to use to build Meerkat Movies – and which may also be used to transfer other comparethemarket.com services to the cloud.

Meerkat Movies is comparethemarket.com's follow-up to the long-running "Orange Wednesday" deal, enabling customers to get two-for-one cinema tickets on Tuesdays and Wednesdays. Customers would need to sign up online, a mobile app would need to be built, and the infrastructure would need to be put in place to ensure that voucher codes could be redeemed by cinemas.

"Having a fixed date meant that we had to talk about the scope, the resource, the number of people we had working on it. In a way, it simplified the whole discussion for us," says Lomas. The team therefore set a target of February to get a rudimentary application up-and-running, giving it just over two months to iron it all out in time for a Good Friday launch – during the ad break in the middle of Coronation Street.

"We went through an inception process to get all the key stakeholders into a room and to agree a common understanding of what's going to be delivered. And also, from that, to work out what might be realistic to deliver in the time frame, in terms of features. So we determined a minimum viable product (MVP) and then it was really a case of cracking on and getting that MVP built," says Greenhall.

The MVP consisted of the features required to get the application running, but is by no means the finished product. Daily discussions and decisions might mean changes to the MVP, depending on the available IT resources, and how quickly work is progressing. "That's quite a healthy conversation to have," says Lomas. "How fast we are going and how much we can deliver? By how much can we further 'flex the scope'?"

Meerkat Movies was also an opportunity to build an application, from scratch, using Node.js. Indeed, Comparethemarket.com has been undertaking the same kind of shift from Microsoft to open source in software tools that it has already performed in databases. "We have historically been a Microsoft .Net shop. We had been through a trialling process and decided that Node.js is a viable alternative, and used it for a few, tentative services. Node.js now sits alongside .Net as a production platform option," says Collinge.

With Node.js, the engineers at comparethemarket.com found they could get up and running more quickly, swapping the Microsoft .Net IDE for a text editor and Node.js instead. "It's a little bit different, but our engineers are quite 'polyglot'. We do a lot of front-end Javascript, so it wasn't too much of a jump for them, provided that they had written in Javascript on the front end before," says Collinge.

[Please turn to page three]

Making movies: How comparethemarket.com got Meerkat Movies up and running in months

Comparethemarket.com CIO James Lomas talks about Meerkat Movies and the company's cautious shift to the cloud

"I guess that as an engineer, if you had experienced only .Net and had never touched any Javascript, there's probably something of a learning curve to overcome in terms of some of Javascript's idioms, but not too much of a jump," he added.

Where experienced Microsoft .Net engineers might struggle, he continued, was in terms of the asynchronous programming within Node.js, but these concepts had also been emphasised in Microsoft .Net with the release of version 4.5 and is something they need to get used to in order to write highly performing applications.

"For me, I think the biggest risk of going down the Node.js route was that it's dynamically typed instead of statically typed, so the importance of testing is that much higher. Fortunately, our principles of pair programming and of writing everything test-first has protected us from some of the pitfalls we could have fallen into," says Collinge.

Greenhall did admit to some nostalgia for the integrated Microsoft .Net development experience. "We have missed IDEs a little bit. Using Visual Studio with C#, if you are very good with the keyboard shortcuts you can build lots of code very quickly and get into a nice flow.

"Now, we basically all use a text editor. The engineers really like it, but sometimes you'll want to do a re-factor and it would be really nice if you could do control-Shift-R and move some stuff around with the wizard... It's just a different way of developing software," says Greenhall.

"Comparethemarket.com's IT department is arranged on the principle of autonomous "two pizza teams" who decide what they are going to deliver and when, in consultation with Lomas, with multiple regular releases and updates following multi-variant tests (MVT) on the different options.

"We try to release 'small and often': we will always try to break down a big thing into much smaller chunks, potentially using feature switching, going live in as short a time as we possibly can," says Lomas. "We will typically 'MVT' official changes, UI changes, and promote the winning variant that does the job best for customers."

Hence, the February deadline on Meerkat Movies. "We wanted to have a production-ready MVP to test on a small sample of people before we got to the television launch in April."

Back to front

Building applications and services in the cloud, too, also changed the way comparethemarket.com approached application development. "Things we used to do last, we now do first," says Lomas.

One of the first applications the IT team built, adds Greenhall, was a monitoring app that would simply check whether services were available. "That grew from a dashboard into something that could monitor key trends. It now monitors factors such as how many members have been created over the day. We would plot them on histograms alongside things like how many codes were claimed, as well as monitoring the availability of our systems," says Greenhall.

"The monitoring we put in place, along with the automation, gives us the confidence to be able to release changes more frequently, and at any time of the day, without affecting customers." says Collinge.

Alongside that, comparethemarket.com uses both New Relic and Splunk: "New Relic monitors how our actual 'tin' is doing and how our services are behaving in terms of throughputs, and how it looks from a customer's point of view. Then we use Splunk, as well, for log aggregation," says Greenhall.

For Lomas, it all forms part of a cautious, step-by-step cloud strategy that enables the organisation and its IT department to focus squarely on applications that make a difference, rather than keeping hardware running. In true build, measure and learn style, the learnings from Meerkat Movies will be applied to help comparethemarket.com's next step into the cloud..

Further info: Computing is holding a DevOps summit on 8 July. Click here for details.