Why government IT projects go wrong

The auditor's report is blistering. A child welfare information system, designed to replace six legacy databases and manual records, has been plagued by political meddling, poor project management and changes to function and systems architecture.

Although under development since 1994, it will not come into service until June 2005 at five times the original estimated cost. Another UK government computer scandal hits the headlines? Not quite.

The project in question is run by the State of Florida's Children and Family Services Department. The state auditor's report was published last month.

Government IT project failures are a global phenomenon. This article could have opened with recent examples from India, Thailand or South Africa, not just Europe or the US.

The tendency for public sector IT projects to run late, go over budget or simply fail to deliver, threatens the whole international e-government project, according to the Organisation for Economic Co-operation and Development.

"Unless governments learn to manage the risks connected with large public IT projects, these e-dreams will turn into global nightmares," it warned last year.

Civil servants and contractors working at the two latest UK government IT projects - the Criminal Records Bureau and the Libra case management system for magistrates courts - will gain little comfort from the fact that their problems are not unique to the UK.

If anything, the UK is developing an international reputation for expertise in identifying why public sector IT projects fail, and how to fix them. The Treasury's Office of Government Commerce has received enquiries from half a dozen countries about its key weapon against project disaster: the Gateway reviews.

Step one is to admit the problem. Here, the past couple of years have brought about a quiet recantation. Politicians, civil servants and IT contractors used to fall back on the excuse that government agencies are no more prone to disaster than private companies; just more likely to come out and talk about them.

There is an element of truth in this analysis. According to researcher Standish Group's latest annual Chaos survey (relating to 2000), only 28 per cent of all IT projects in the US, in government and industry alike, hit their targets for budget, functionality and timeliness.

Another 23 per cent were cancelled outright and the rest failed on at least one of the counts. These findings suggest a great deal of failure within the private sector, which is often able to conceal its mistakes.

However, while direct comparisons are difficult, evidence is mounting that the problems are not just due to publicity.

The UK government's definitive report, Successful IT: Modernising Government in Action, is known as the McCartney report after its ministerial sponsor said in 2000: "Government IT projects have too often missed delivery dates, run over budget or failed to fulfil requirements."

This is a tacit admission that the government's record is worse than that of industry benchmarks.

Since the report, a consensus has emerged that government IT projects are trickier than those in the public sector.

Mike Davis, senior research analyst at the Butler Group, and previously a public sector IT manager, identifies the following as the most important reasons.

Multiple stakeholders
A commercial company has a single clear objective: to make profits. A government organisation has to deliver services to the satisfaction of taxpayers, its political masters and staff, who may be organised in a powerful professional group or trade union. This leads to a lack of clarity over goals.

Stricter measures of success
Commercial managers can judge a project's success by the seat of their pants. If business is going well, IT will claim a share of the credit although, as Thomas Landauer of MIT points out in his book The Trouble with Computers, the link is hard to demonstrate.

In the public sector, the only measure of success is in the project process itself, whether it was completed on time and within budget.

Lack of clarity applies particularly when the project is 'business-led': politicians don't like closing hospitals or making people redundant. Talking openly about cost savings can be taboo. Earlier this year, e-envoy Andrew Pinder had to back-pedal over a reported claim that e-government could cut the civil service by 20 per cent.

Policy uncertainty
Government departments have difficulty with long-term planning and are not usually allowed to carry money from one year into another.

Changes in legislation can render a system obsolete overnight; NIRS2 had to be changed to take pensions law into account. A change of government can render a whole class of systems obsolete, such as the programs created to support fund-holding GPs.

Unlike, say, the Crest project failure at the London Stock Exchange, taxpayers see failure in the public sector as a waste of 'their' money.

Government auditors and parliamentary investigators, especially the Commons Public Accounts Committee (PAC), have collected a formidable list of reasons why specific projects went awry. Several repeat themselves over and over again.

Poor project management
In the 1990s, the National Probation Service Information Systems Strategy had seven project managers in seven years. Not surprisingly, it ran over budget, was late going live and short on functions.

This is an extreme example, but Peter Gershon, chief executive of the Office of Government Commerce, cites lack of project management skills as a common thread in disasters. Traditionally, the public sector has not nurtured such management expertise.

Cultural misunderstandings
PAC chairman Edward Leigh explained that the Civil Service habit of promoting classically educated generalists has a lot to answer for. But complex cultural issues run throughout government.

"Most public sector organisations are hierarchies," said Davis. "Projects tend to work across departments. This raises the question of who's going to lead and who's going to have power over another department's staff. These things can't be belittled."

Lack of communication between departments
One classic case was the Post Office Benefits Payment Card project, justified on the basis of a shared business case to which no single stakeholder subscribed.

The current fiasco at the Criminal Records Bureau also demonstrates what happens when different departments have simultaneous hands on the tiller: the priority at the Department for Education and Skills was to get schools opened on time, while the Home Office was concerned with identifying people with criminal records.

Procurement bureaucracy
The process of purchasing a large system can become an end to itself. In the NHS, thanks to safeguards against scandal that were introduced in the early 1990s, a routine hospital information system can take two years or more to procure. The result has been hardware specified at the start of a procurement that is obsolete by the time the contract is signed.

Value for money versus highest bid
The public sector is always under pressure to take the lowest offer. This has led to suppliers winning contracts without the resources to deliver them.

Cultural gaps between IT and business management
This has been identified as a factor in the private sector, but occurs at least as often in government.

Both the Passport Agency and the Probation Service's Crams project went ahead with little consideration for how the new systems would change people's working lives.

Temptation to throw good money after bad
The public sector is notoriously reluctant to cancel projects.

THE MCCARTNEY REPORT

Obviously not all shortcomings occur in all projects. However, the McCartney review, and a follow-up study by Intellect ( formerly the Computing Services and Software Association) came up with a remarkably consistent set of recommendations for avoiding disaster.

Choose your contractor by value for money rather than the lowest bid
The Office of Government Commerce's Peter Gershon explained that value for money is about much more than getting a good purchase price.

"We define procurement as the whole lifecycle process of the acquisition of goods, services and works from third parties. This starts when a requirement is a gleam in the eye and ends with the conclusion of a service contract or the ultimate disposal of an asset," he said.

Grown-up negotiations
Peter Holmes, managing director for government services at Accenture, suggested that in the past there has been a lack of communication between project team and supplier.

This will have to change, he pointed out. On the contractor's side, this means not being ashamed of wanting to make a profit.

"Fear of the private sector's profit motive has too often led to contracts which inhibit the ability to respond to changing circumstances and actively discourage joint problem solving," he said.

Ironically, the McCartney report identified one government agency which went overboard to ensure that all three shortlisted bidders really understood what was involved.

It even paid them £100,000 each from the project budget to compile reports that would allow the agency to assess the viability of the proposal and would form part of the eventual contract. Unfortunately, the contract in question was for the Criminal Records Bureau.

Better project management, especially across agencies

Put someone in charge
Give every project a 'senior responsible owner' answerable for its success, and with the clout to achieve it.

Holmes described the appointment of a senior responsible owner as a "big step forward".

"A few years ago, there were very substantial procurements where decisions were taken at a very low level by the procurement officials, and senior civil servants didn't want to get involved, perhaps for reasons of propriety," he said.

Use industry-standard, commercial, off-the-shelf technology

Partnership
Create an understanding between customer and contractor. "It's very easy to blame suppliers," explained Nick Kalisperas, e-government programme manager at Intellect. "But the fact is that the majority of time they're responding to tenders. It's up to those issuing tenders to make sure that there's flexibility to deal with every eventuality."

Run projects in manageable chunks
Nearly everyone agrees that 'big bang' is a recipe for disaster. The Organisation for Economic Co-operation and Development insisted that "large IT projects should be avoided wherever possible", and urged governments to create "dolphins, not whales".

The OGC agreed: its Gateway reviews have often required IT contracts to be broken up into smaller components as a way of reducing risk.

Set realistic timetables
Managers need to see both ends of the tunnel at once. "Given the choice between delivering 80 per cent functionality in a year or 100 per cent in three years, it's always best to go for the quicker option," concluded Davis.