• Home
  • News
  • Big Data & Analytics
  • DevOps
  • Security
  • GDPR
  • AI & ML
  • Women in Tech
  • Cloud & Infrastructure
  • CIO
  • Deskflix
  • Events
  • Whitepapers
  • Spotlights
  • IT Leaders 250
  • Research
  • Delta
  • Tech Marketing Hub
  • About Us
  • Newsletters
  • Sign in
  •  
      • Newsletters
      • Account details
      • Contact support
      • Sign out
     
     
    • You are currently accessing Computing via your Enterprise account.

      If you already have an account please use the link below to sign in.

      If you have any problems with your access or would like to request an individual access account please contact our customer service team.

      Phone: +44 (0) 1858 438800

      Email: customerservices@incisivemedia.com

      • Sign in
     
  • Follow us
    • Twitter
    • LinkedIn
    • Newsletters
    • Facebook
    • YouTube
  • Register
  • Events
    • Upcoming events
      event logo
      How to overcome your Active Directory consolidation challenges

      In this web seminar we explore how best to go about assessing and modernising your AD as we reveal our research findings into AD health and readiness, consolidation challenges, and strategies for success.

      • Date: 20 Apr 2021
      event logo
      Mitigating Ransomware Attacks

      In this webinar, Javvad Malik, Security Awareness Advocate at KnowBe4, will share practical advice and real-world examples of how you can defend your organisation from the ongoing ransomware threat.

      • Date: 27 Apr 2021
      event logo
      Desklix: Digital Workplace

      The Coronavirus pandemic has had a huge impact on our lives with most organisations making a sudden switch to mass remote working. As restrictions are progressively eased, the impact continues, with organisations having to decide when to allow staff back into offices, what proportion of remote working should be expected, and how all of this should be supported.Take part in this virtual event to put your questions to the experts, and see what your peers have learnt from the pandemic, and how they plan to apply this understanding to 2021 and beyond.

      • Date: 12 May 2021
      event logo
      How to improve security whilst reducing workload and stress

      In this webinar we discuss the existing situation and its solutions, with automation one of the most promising.

      • Date: 13 May 2021
      • Computing UK, London
      View all events
  • Whitepapers
    • LATEST WHITEPAPERS
      Darktrace 120x194
      Cyber AI Response: Threat Report 2019

      This white paper details 7 case studies of attacks that were intercepted and neutralised by Darktrace cyber defense AI, including a zero-day trojan in a manufacturing company's network. Learn how Darktrace Antigena AI Response modules fight back autonomously, no matter where a threat may emerge, extending to the Cloud, Email and SaaS.

      Download
      Darktrace 120x194
      Cyber AI & Darktrace Cloud

      This white paper explores how cloud is a security blind spot for many organisations who struggle with the limited visibility and control in this new environment, where their existing security tools are often not applicable.

      Download
      Find whitepapers
      Search by title or subject area
      View all whitepapers
  • Spotlights
    • Spotlights

      Welcome to Computing's Spotlight section, where we focus in on particularly important themes and topics of enterprise IT.

      Intel logo

       

      Endpoint Management and Security Hub

  • IT Leaders 250
  • Research
  • Delta
  • Tech Marketing Hub
  • About Us
Computing
Computing
  • Home
  • News
  • Big Data & Analytics
  • DevOps
  • Security
  • GDPR
  • AI & ML
  • Women in Tech
  • Cloud & Infrastructure
  • CIO
  • Deskflix
 
    • Newsletters
    • Account details
    • Contact support
    • Sign out
 
 
  • You are currently accessing Computing via your Enterprise account.

    If you already have an account please use the link below to sign in.

    If you have any problems with your access or would like to request an individual access account please contact our customer service team.

    Phone: +44 (0) 1858 438800

    Email: customerservices@incisivemedia.com

    • Sign in
 
  • Cloud and Infrastructure

Photobox Group CTO reveals how the company moved nine petabytes to the cloud in six months

Going too fast would mean damaging data centre infrastructure - but too slow and there would be bottlenecks at peak times

Richard Orme, CTO of Photobox, said, "We were adding about a petabyte a year: anything up to about 5-6 million photos a day"
Richard Orme, CTO of Photobox, said, "We were adding about a petabyte a year: anything up to about 5-6 million photos a day"
  • Tom Allen
  • Tom Allen
  • 13 August 2018
  • Tweet  
  • Facebook  
  • LinkedIn  
  • Send to  
0 Comments

 

Time is money

Efficiency doesn't necessarily equal speed; the planning stage predicted that the data transfer would take as much as 12 months. However, when the work began that started to look optimistic.

"Quite quickly, we found that there were some things that we hadn't picked up in the planning that were causing us a challenge. The first one was: 5.7 billion really small files. Images are quite big, but not in comparison to videos or large data files...

"We experienced some challenges in getting stuff into the Snowballs. They'd fill up pretty quickly and then they'd start to slow down a little bit. That was because the index on the database that runs the file system on the Snowball was just filling up; it never anticipated that the Snowball would take that many files."

AWS, luckily, was monitoring the move closely - the Group was, after all, moving Europe's largest consumer photo database to the cloud. Orme says, "For a period of a week and a half it felt like we were running the Snowball Edge roadmap; they just stopped everything to get us over the line."

That monitoring, with feedback from the Photobox team, saw AWS pushing patches in near-real time. One of these - bringing forward a compression feature that had been due to be rolled out months later - solved the small file challenge.

"Within two or three weeks we got to a point where we were suddenly pulling data out at a much faster rate than we'd believed possible, while still preserving the sanctity of the underlying discs. [It was] incredible stuff, right at the edge of what we could do, but incredibly motivating for the team, because they were just smashing through speed records every day."

We're making savings, our customer serving of photos is now radically faster than it was and our ability to ingest new photos is relatively limitless

These optimisations sped up the entire project, with the end result that 12 months turned into just six and a half. The data transfer that began in January ended in mid-July, and Photobox is celebrating.

"You don't often think of these things as being an opportunity to learn and improve, write new software and push the limits of what can be done - but in the end that's what it turned out to be...

"The net result that we get from having completed this project five months early is just incredible. We're making savings, our customer serving of photos is now radically faster than it was and our ability to ingest new photos is relatively limitless, compared to what it used to be. We don't have any worries going into peak this year."

12

Further reading

How Monzo leveraged AWS and Kubernetes to build an agile retail bank
  • Financial Solutions
  • 09 July 2018
NHS Business Services Authority considering a 24-hour virtual contact centre after trial with AWS
  • Cloud and Infrastructure
  • 11 May 2018
AWS announces support for serverless containers on Kubernetes
  • DevOps
  • 30 April 2018
Facebook tempts US banks with financial data sharing deal
  • Security
  • 07 August 2018
  • Tweet  
  • Facebook  
  • LinkedIn  
  • Send to  
  • Topics
  • Cloud and Infrastructure
  • Photobox Group
  • Cloud and Infrastructure
  • AWS
  • Snowball
  • Datacentre
  • datastrategy

More on Cloud and Infrastructure

Does the choice of enterprise Linux really matter anymore?

We speak to Canonical, Red Hat and SUSE about the place of Linux in a cloud-based future - and what the CentOS EOL foretells

  • Open Source
  • 15 April 2021
Cloud services are a store of valuable IP and sensitive data
Securing your investment in Microsoft 365

There’s a lot of sensitive data contained in Office documents - so it makes sense to take care of it

  • Cloud and Infrastructure
  • 07 April 2021
Covid: The great reset means IT leaders must now think ahead, not just cope with an inconvenient present

  • Cloud and Infrastructure
  • 29 March 2021
Escaping technical debt: moving to the cloud at the Royal Society of Chemistry
Escaping technical debt: moving to the cloud at the Royal Society of Chemistry

'We needed a much more flexible and cost-efficient operation to keep up with the big players in the field', says head of DevOps Chris Callaghan

  • Cloud and Infrastructure
  • 29 March 2021
Industry Voice: In a software-centric world, don't lose sight of the need for secure, manageable hardware

  • Cloud and Infrastructure
  • 26 March 2021
blog comments powered by Disqus
Back to Top

Most read

Covid results emails may breach GDPR
Covid results emails may breach GDPR
Apple and Google block NHS Covid-19 app update over privacy issues
Apple and Google block NHS Covid-19 app update over privacy issues
Microsoft patches five zero-day bugs in April 2021 Patch Tuesday update
Microsoft patches five zero-day bugs in April 2021 Patch Tuesday update
University of Hertfordshire cancels live online teaching following cyber attack
University of Hertfordshire cancels live online teaching following cyber attack
Darktrace reveals plan to float in London at the end of the month
Darktrace reveals plan to float in London at the end of the month
  • Contact
  • Delta
  • Marketing solutions
  • Enterprise IT Events
  • Incisive Media
  • Terms & conditions
  • Policies
  • Careers
  • Privacy Settings
  • Twitter
  • LinkedIn
  • Newsletters
  • Facebook
  • YouTube

im_logo

© Incisive Business Media (IP) Limited, Published by Incisive Business Media Limited, New London House, 172 Drury Lane, London WC2B 5QR, registered in England and Wales with company registration numbers 09177174 & 09178013

Digital publisher of the year
Digital publisher of the year 2010, 2013, 2016 & 2017
Loading