Debugging the planet: cloud automation tracks global insect decline

How Kent Wildlife Trust built a scalable, cloud native data pipeline for Bugs Matter app

Kent Wildlife Trust has engineered a cloud-hosted pipeline to measure insect decline, using automation and citizen science to generate actionable ecological intelligence for governments, partners and conservationists.

For Kent Wildlife Trust, gathering ecological evidence is the backbone of the organisation’s mission to reverse wildlife decline. At the centre of this work are Dr Lawrence Ball, Monitoring and Evidence Manager, and Euan McKenzie, Digital Development Officer. Together they are using cloud technology, automated data pipelines and citizen science to uncover a clearer picture of the health of global insect populations

“I oversee the work that ensures we’re evidence-led and evidence-generating as an organisation,” Ball begins. “Using evidence is vital if we’re going to meet our mission increase wildlife abundance and climate resilience across 30% of Kent’s land and sea.”

Ball explains how the mission he describes above rests on three key pillars. The first of these is to implement nature-based land management practices such as grazing bison and working on campaigns to rally public support for those practices.

“Goal two is to inspire and collaborate,” Ball continues. “We know we can't deliver our mission alone, so to inspire others, we increase our engagement with communities, businesses and stakeholders, to reach a large, diverse audience.”

The third pillar is to strengthen and grow, adopting national and international best practice to become more resilient, sustainable and demonstrably impactful.

Kent Wildlife Trust employs around 100–120 staff, supported by thousands of volunteers. Together they deploy an array of monitoring tools including drones, passive acoustic sensors, camera traps, and large-scale citizen science projects. One such project has captured the public imagination.

Bugs Matter

“Bugs Matter was originally a 2004 RSPB idea which involved counting insect splats on number plates to estimate insect abundance,” Ball explains. “They ran it for one year. In 2019 we discussed reviving it, secured lottery funding, rebuilt the app, and launched nationwide in 2021 in partnership with the charity Buglife. The dataset has grown every year since.”

Counting splats on a number plate sounds basic, but it helps to address a major challenge in ecology: measuring all flying insects, not just a single species group. “Most existing methods focus on butterflies, moths or pollinators,” Ball says. “But those groups can show different trends. Bugs Matter monitors thousands of species collectively which gives us a better picture of biomass change.”

He continues: “We acknowledge that there are other factors that could affect how many bugs are being splatted, but we're constantly validating our data. This involves looking at how consistent journeys are over the years and whether we're sampling a representative proportion of the different habitats that we have in the UK. We need representative samples but we're increasingly becoming more confident in these trend statistics that are coming out of the data.”

Ball’s team was successful in a bid for AWS funding to scale the project overseas, and the first international rollout is underway in now underway in the Republic of Ireland, with more countries to follow.

International expansion creates new needs such as multilingual app support and country-specific settings. The team are working on this now and it has proven to be an ideal opportunity to rethink how results are shared.

“In the UK, we used to run all the statistical models ourselves and publish a PDF each year,” Ball recalls. “If this became global, I’d do nothing but produce reports forever.”

Citizen science meets automated data pipeline

Enter Euan McKenzie, Digital Development Officer: To prepare Bugs Matter for a global expansion McKenzie built a fully cloud-hosted, automated data pipeline on an Amazon EC2 instance.

“The pipeline is modularized to make it easier to develop and manage, but all those parts of the workflow share the same environment. The tool that we could use was an R package called targets, which helps with data science pipelines, and it remembers the state of the pipeline, and each time you run it, so the next time you run it, it can just run steps that need to be updated, and skip over steps that don't change.”

The pipeline queries the mobile app API daily, processes submissions drivers make via the mobile app, cleans errors such as GPS dropouts or unclosed journeys and enriches each with environmental variables like temperature, altitude, habitat types, road types and vehicle characteristics.

A complex zero-inflated negative binomial model analyses trends, accounting for the (worryingly) large number of zero-splat journeys. Results are stored in a Postgres database and visualised in a custom R Shiny dashboard which McKenzie and Ball developed together.

The new dashboard is already proving to be transformative. Rather than waiting for annual reports, anyone from citizen scientists to government decision-makers will be able to explore trends by region, country or habitat type.

“If we imagine this being done globally,” Ball says, “governments could see which areas are experiencing steep declines, ask why, and act. That level of insight could be crucial.

Kent Wildlfe Trust is also working on partner-specific logins for organisations like Openreach, one of the UK’s largest van fleets. “They’re encouraging their drivers to take part,” Lawrence says. “A partner login would let them explore their own data.

Datacentre dilemma

User numbers of the Bugs Matter app are still climbing, with more than 10,000 registered participants and over a million miles of journeys logged. It’s a huge win for the concept of citizen science and the contribution to the knowledge of insect decline is clear.

However biodiverse environments are lost in the construction of many datacentres, and the impact on insect populations of the energy and water required to sustain datacentres are becoming more widely understood. Does Kent Wildlife Trust see a contradiction in the concept of cloud powering conservation efforts?

Lawrence Ball doesn’t shy away from the dilemma. “We’re far past the point of avoiding the climate and biodiversity crisis,” he says. “We’re now planning how to adapt to impacts that will happen. Technology and innovation may be critical to survival. If using data centres helps us understand and mitigate ecological collapse, then that trade-off becomes part of a necessary path forward.”