Overoptimism and a lack of parliamentary scrutiny blight big government computer projects. These systemic, cultural weaknesses have afflicted countless IT projects, from the failed unification of welfare computer systems in the 1980s to the £12.7 billion programme for IT in the NHS.

As long ago as 1984, MPs on the Public Accounts Committee warned senior civil servants against being too optimistic in setting budgets and time frames on large computer projects.

The response of governments to IT failures has been to set up internal committees and programmes to identify problems early in a project’s life. These initiatives are always well intentioned — and there are many of them — but they have had little effect on the biggest and riskiest schemes.

Almost every time there is a failure of a large project to deliver what was promised within the original budget or time frame, the heads of departments and agencies say that the lessons have been learnt. But they have not, or have been pushed to one side because each project is regarded as unique. So ministers, encouraged by potential suppliers and other enthusiasts, approve incrementally larger and riskier schemes, while potential suppliers reach for their calculators and ministers look forward to a series of press releases on the (potential) benefits of the new technology.

None of the problems is the fault of thousands of civil and public servants and contractors who work in information technology for government and keep its uniquely complex machinery running smoothly. One cause of the almost predictable failure of big, risky projects is the way they are approved in the first place — without anyone seriously or effectively challenging whether a scheme is too ambitious.

Overoptimism is Whitehall’s euphemism for offering ministers a project with short timescales, capped costs and in-built simplicity to get the scheme approved. The project’s risks are not considered seriously because those pushing the original concept want the money to be approved — and for all the right reasons. Nobody can argue with the need for an electronic health record to replace paper records that are often lost or unavailable when needed. But the potential for a project to be simple often loses out to complexity.

There needs to be a different way of doing things — the introduction of parliamentary scrutiny before the main contracts are awarded. MPs should scrutinise IT projects and programmes before they are approved formally, instead of when constituents complain.

MPs are not computer experts. But they don’t need to be. They could challenge assumptions about costs and time frames, and test whether officials understand fully the commitments they are about to make. They could soon discover whether the complexity has been underestimated.

The preference of government, however, is for continued secrecy over its biggest schemes — what it calls “mission-critical” projects. The Government is even willing to go to the High Court to stop the publication of early “gateway” reviews — which assess risky projects — on the ID cards scheme. This culture of secrecy and lack of parliamentary scrutiny before a project starts to implode can only contribute to the cycle of failure.

Tony Collins is executive editor of Computer Weekly

Times Recommends

• Lies and cover-up at MG Rover exposed by report
• High oil prices push manufacturing costs up
• Primark founder steps down as chief executive