BBC: "Incident response isn't a paper you put on the shelf and just grab when someone starts screaming"

David Jones, head of information security at the BBC, explains how incident response works at the broadcaster, and stresses that organisations must move away from a blame culture when it comes to security breaches

The BBC, an organisation employing roughly 35,000 people worldwide, which sends and receives over 100 million emails monthly and has more than 70TB of email data stored, is under constant cyber attack. Speaking at InfoSec 2014 today in London, David Jones, head of information security at the BBC, described the volume and types of attack he faces.

"We see regular attacks, with very high levels of phishing and malware. We also have reportable incidents, which at the BBC means something which has a measurable financial, reputational, legal or mortal consequence."

He said that the aims of incident response is to improve service resilience, reduce service downtime, identify and remediate vulnerabilities and weaknesses, and educate stakeholders, staff and partners.

In order to achieve these aims, the strategy is divided into four principles: communicating, preparing, understanding and acting, and learning.

Expanding on the third aim - understanding and acting - Jones said that to understand the environment in which the business operates is essential.

"When we delve into the way the business functions and the technology operates, we see areas which over time become greyer. Those are the areas which cause the most grief when an incident occurs."

Jones gave the example of an FTP service, a common technology used by many organisations.

"Ask 20 people who use the service how they use it, and you'll get at least 10 different answers," said Jones. "This sort of technology does the job quietly. People might create an account only intending to use it once, but over time it grows, they share their account with colleagues and it becomes relied on.

"The people administrating it will change over time, so they don't know its history, and no one has documented every new account, so each account may be 10 generations removed from the person who created it. So a vital service which has grown over time is not understood."

Jones said that this makes the job of identifying who needs to be contacted should an incident occur and the service be taken offline extremely difficult. To avoid these issues, Jones and his team devote a lot of energy into understanding the business, its systems and people.

"We spend a lot of time working with technology colleagues to understand the environment and our dependencies. It's essential to perform a very thorough service mapping. You need to understand what happens when you take a server down, this is both a technical and a human issue. Knowing technically what something does isn't the same as knowing what it's used for, and how critical it is. And it might only be critical to some people at very specific times. You need to understand that in order to know how to respond when the incident happens," said Jones.

Allied to the "understanding" part of Jones' strategy is "acting". And a big part of this is having the authority to act.

"Communication with stakeholders is critical," said Jones. "This needs to run before incidents occur, and well after they complete. Often the major problem is finding someone in authority to help you, someone who can take a decision. It's really important to get stakeholders in line, understanding what you do and how you do it, and working with you."

He added that organisations need to agree who can make decisions, and when.

"If the attack is slow with a low impact, we can look into it at great depth before we make a decision. If it's fast and high we can't. So we need to know who has the authority to act in an emergency. And it follows that you need to be ready to act and avoid a blamestorm culture - people need to know that if they're acting on their best information in the best interests of the organisation, they won't be blamed later. This hamstrings so many organisations, because people are terrified that they'll be blamed by their superiors for something later."

Planning is another key area for Jones. He explained that the way an organisation plans for incidents must continually evolve and improve because the attacks themselves also change over time. He gave the example of the recent attacks on the BBC by hacktivists the Syrian Electronic Army.

"The Syrian Electronic Army started out using basic phishing attacks on western media outlets, including the BBC," said Jones. "It started simply, with the intent to get access to Twitter accounts so they could post embarrassing messages. Then they discovered they could get access to webmail, so they put out more attacks. As we responded to that, they moved on to the supply chain with the same attacks, then onto the DNS [Domain Name Servers] providers to have websites reassigned - which they did effectively in an attack against the New York Times.

"Then they started stealing information and publishing it, like invoices from Microsoft to the FBI for searches done on customer data. So those attacks started very simply, and evolved over a year to become increasingly sophisticated," Jones said.

He concluded that organisations should look for responses they can automate - tasks that are quick and simple. For phishing attacks, the BBC now has an automatic strategy in place.

"When we see a phishing attack now, we block the domain. So we stop it coming in, we kill it at the exchange level. Then we perform a search and destroy to remove it from everyone's mailbox."

He admitted that this was not enough though, given that today's users have mobile devices too.

"But users now have iPhones and iPads. So we have to reach them to tell them they're potentially going to be phished in a way we can't block. So the response evolves, and we can initiate that in minutes," Jones added.

He wrapped up by stating that an incident response strategy needs to be running and improving constantly.

"Incident reponse isn't a paper you put on the shelf and just grab when someone starts screaming. You need to keep it running all of the time," he concluded.