How to cut waste for a leaner and greener cloud
Cloud wastage is needlessly increasing both carbon footprints and infrastructure costs. Enterprises can boost their ESG profiles and their profits by curbing cloud waste.
Cloud vendors have long positioned their services as a more sustainable alternative to enterprises running their own datacentres and the green cloud message is indeed crafted around a kernel of truth. Many enterprises using a single hyperscale datacentre are going to generate less carbon per compute or storage instance than all of those enterprises running their own smaller, less efficient datacentres.
However, the efficiency of cloud datacentres has to be balanced against the fact that the huge growth in demand for cloud services, particularly since early 2020, has pushed up the carbon emissions generated by the electricity needed to run and cool datacentres. That's before we even get to water consumption needed for cooling or the impact of all the minerals and materials needed to build them in the first place. The fact that the cloud now generates more carbon than aviation should be a wake up call for anyone who thinks that cloud equals clean.
Why is it so hard to get a grip on cloud waste?
Part of the challenge facing both cloud vendors and their customers, is that cloud has been sold since its inception as a more elastic alternative to customer owned infrastructure. The dream was infinite elasticity - to be able to scale up or down as demand requires. Certainly, most enterprises have found is that the scaling up is pretty straightforward. But dialling down? Well, that involves navigating plenty of contractual, financial, psychological and organisational barriers.
Most readers will be familiar with the commercial barriers to the visibility - and potential reduction of cloud infrastructure. The complex and continually changing pricing models, bills consisting of thousands of line items and swingeing data egress charges act almost seem to be designed as a deliberate disincentive for businesses to get a grip on sprawling infrastructure costs and reduce their carbon footprints.
It's easy to separate out the commercial barriers to reining in cloud wastage. It's harder to pin down the more human related ones. Assim Razzaq, founder and CEO of Yotascale attributes these barriers to a clash of priorities and a lack of information available to the developers, platform engineers and anyone else who is creating cloud infrastructure. Razzaq makes the point that it's very difficult to understand decisions about cloud infrastructure out of the enterprise context in which those decisions are made.
"Certain people own applications and products, services, and they understand the characteristics back to performance, cost, security, reliability etc. An engineer doesn't have the time to figure out what all of these line items they can choose from are going to add up to and will make decisions based on what they know when building an application. But that application might behave very differently out in production. You just just don't know how it will behave and operate as it starts serving different workloads."
Razzaq also points out how cloud provisioning decisions are also often made without knowledge of infrastructure decisions being made in parallel.
"There's also Shadow IT. The platform engineering team doesn't even know people are provisioning other infrastructure. Organisations have the agility but the people provisioning infrastructure don't necessarily have the information or education they need to make the most sustainable or cost effective decisions."
Cutting the waste for a more sustainable cloud infrastructure
What can enterprises do to get to grips with cloud waste and make more efficient and sustainable decisions?
It sounds obvious but it worth restating that awareness of the problem and commitment to resolving it has to come from the top. The employees who inadvertently create cloud wastage may not be motivated to resolve the issue if it clashes with other objectives - objectives set by those in leadership positions.
This commitment should extend to educating employees to be aware that, as Razzaq puts it, cloud efficiency is not just about economics, it's also about sustainability. The two are different sides of the same coin. The more efficient infrastructure is the less environmental impact it will have. This matters because an enterprise might know it's wasting thousands of dollars a month but if the relevant departments know it's going to cost them many hours identifying underutilised infrastructure, the problem drags on.
Razzaq also questions the assumption that public cloud is sustainable by definition.
"Cloud is a modern-day smokestack," he says. "The ratio of employee to server used to be one to one. Now it can be one to 10, 200 or 1000."
Whilst the big cloud beasts are certainly all undertaking to make their datacentres more sustainable, regular readers of our Eco warriors series of articles comparing cloud vendors, know that there are considerable differences between vendors in terms of the sustainability of their clouds, and also between the images and narrative that ESG reports typically present, and the hard data usually contained with the appendices.
Cloud customers should also use the sustainability tools that they are increasingly being provided as part of the services they subscribe to. Microsoft offer a Sustainability Calculator, AWS a Carbon Footprint Tool and Google have recently announced a series of new sustainability measures which allow customers to measure their carbon footprint across not just GCP but also Google Cloud Workspace. Customers can also restrict cloud resources to the lowest carbon Google locations.
None of these tools are likely to give a completely accurate measure of cloud carbon footprint, particularly of the indirect emissions associated with cloud, but they are a start - an environmental baseline. They provide the opportunity for cloud customers to reduce the macro wastage of cloud resource. The next step is using specific, AI driven products to identify micro wastage in shared infrastructure which so affects both carbon footprint and also bottom-line costs. This enables those building, deploying and managing cloud applications to make more sustainable decisions, and to shrink their carbon footprint.