Photobox Group CTO reveals how the company moved nine petabytes to the cloud in six months

Going too fast would mean damaging data centre infrastructure - but too slow and there would be bottlenecks at peak times

Richard Orme, CTO of Photobox, said, "We were adding about a petabyte a year: anything up to about 5-6 million photos a day"

Tom Allen
13 August 2018

Tweet
Facebook
LinkedIn
Send to

Time is money

Efficiency doesn't necessarily equal speed; the planning stage predicted that the data transfer would take as much as 12 months. However, when the work began that started to look optimistic.

"Quite quickly, we found that there were some things that we hadn't picked up in the planning that were causing us a challenge. The first one was: 5.7 billion really small files. Images are quite big, but not in comparison to videos or large data files...

"We experienced some challenges in getting stuff into the Snowballs. They'd fill up pretty quickly and then they'd start to slow down a little bit. That was because the index on the database that runs the file system on the Snowball was just filling up; it never anticipated that the Snowball would take that many files."

AWS, luckily, was monitoring the move closely - the Group was, after all, moving Europe's largest consumer photo database to the cloud. Orme says, "For a period of a week and a half it felt like we were running the Snowball Edge roadmap; they just stopped everything to get us over the line."

That monitoring, with feedback from the Photobox team, saw AWS pushing patches in near-real time. One of these - bringing forward a compression feature that had been due to be rolled out months later - solved the small file challenge.

"Within two or three weeks we got to a point where we were suddenly pulling data out at a much faster rate than we'd believed possible, while still preserving the sanctity of the underlying discs. [It was] incredible stuff, right at the edge of what we could do, but incredibly motivating for the team, because they were just smashing through speed records every day."

We're making savings, our customer serving of photos is now radically faster than it was and our ability to ingest new photos is relatively limitless

These optimisations sped up the entire project, with the end result that 12 months turned into just six and a half. The data transfer that began in January ended in mid-July, and Photobox is celebrating.

"You don't often think of these things as being an opportunity to learn and improve, write new software and push the limits of what can be done - but in the end that's what it turned out to be...

"The net result that we get from having completed this project five months early is just incredible. We're making savings, our customer serving of photos is now radically faster than it was and our ability to ingest new photos is relatively limitless, compared to what it used to be. We don't have any worries going into peak this year."