When Facebook opened its data centre in Luleå in northern Sweden, the first it had built outside of the US, it raised more than a few eyebrows – and questions. Why Sweden? Why so far away from any major population centre? And isn’t it something of a risk to fill it with bespoke equipment based on the Open Compute Project, rather than conventional, off-the-shelf servers from HP or Dell?
Luleå was put on the map when Facebook arrived, but there were a number of sound reasons why Facebook alighted there, Niall McEntegart (pictured), head of data centre operations Europe for Facebook, told the Computing Data Centre Summit this week.
First, he said, it offers 100 per cent clean, renewable energy from the area’s many nearby hydro-power plants. “It means that we aren’t exposed to fluctuating power costs,” said McEntegart.
Second, the area is well connected to the kind of fibre-optic broadband backbone that a company like Facebook needs – generating some 10 petabytes of information every day.
Third, the climate in the area provides between eight and 10 months of free-air cooling per year, radically cutting refrigeration costs for such a vast data centre complex.
Finally, the local university – Luleå University of Technology – should be able to provide a conveyor belt of suitably qualified staff to work in the data centre. The area even has connections to two independent power grids, improving redundancy in the power supply and reducing the need for back-up generator equipment.
Open Compute Project
The campus itself will contain three data centre buildings when it is fully complete, each about 30,000 square metres – the equivalent of four-and-a-half Wembley pitches.
The facility is the first in the company to be based entirely on the principles of the Facebook-led Open Compute Project. This encompasses not just the design of the hardware, but the design of power systems and cooling systems, too.
Using free-air cooling does not just cut costs in terms of power consumption, but also saves space. The Open Compute design in Luleå features vertical drainable louvres that take outside air into air intake corridors. This is filtered and humidified, as necessary, before it is piped to the cool-air aisles in the data centre floor itself.
The Open Compute design is also very energy efficient, according to McEntegart. A typical data centre power system loses between 21 per cent and 27 per cent of inputed power at various stages as AC is converted to DC (and back again), but with Open Compute, these losses can be cut to just 7.5 per cent, he says.
The Open Compute hardware, including the racks, has also been simplified in the interests of power efficiency. “It’s literally a piece of sheet metal with a motherboard sitting on top of it. There’s nothing else there. It doesn’t restrict your airflow, which is another source of efficiency. And when it comes to end-of-life, there’s much less there to recycle, so it’s much more sustainable,” says McEntegart.
It is also much cheaper: “Typically, Open Compute hardware is about 38 per cent more efficient and 24 per cent cheaper than more traditional designs; and we use about five or six standard designs for everything. We don’t like to use variation as that adds cost.”
On top of all this, Facebook regularly reviews workflows to identify inefficiencies and to ensure that processes are consistent. Management tools have been built from scratch and as many as half of faults that arise can be fixed with automated tools without ever requiring human intervention, claims McEntegart.
Even the servers are fully front-of-rack accessible, enabling quick and speedy repair.
“Everything happens in the cold aisles; there’s no need to go into the hot aisles at all – there’s no cabling in there,” he says. It is also relatively quiet as the servers do not require internal fans for cooling.
As a result, Facebook has nailed the power usage effectiveness (PUE) figures right down. “We expected, when we built Luleå, to run at 1.06-1.08 PUE. It’s looking like we are going to hit 1.04 or 1.05,” says McEntegart.
And anyone can adopt Open Compute. “It is all open source. Anyone can use the designs that we have built and added to Open Compute – the data centre design, the power design and server designs. We have about 60 OCP partners at the moment and its gaining real momentum. The beauty of it is that no one owns it. It’s there for anyone to use,” says McEntegart.