At a glittering awards dinner at the Hotel Intercontinental on Park Lane, the great and the good of Britain’s biotechnology industry were out in force. In ill-fitting dinner jackets, ponytailed professors rubbed shoulders with their university spin-out peers or sipped champagne with more conventionally dressed City bankers and lawyers. Amid the hubbub and the hors d’oeuvres, it would have been all to easy to believe that this was the annual highlight for a prosperous industry in rude health.

It wasn’t. The backslapping and bonhomie at the Mediscience dinner in June belied a deeper malaise in Britain’s biotech sector. Despite being home to hundreds of biotech start-ups, two of the world’s biggest pharmaceutical companies – GlaxoSmithKline and AstraZeneca – and an almost unmatched pool of talented scientists, Britain’s biotech industry is failing to fulfil its potential.

“We have the second-best science base in the world,” Simon Best, chairman of Britain’s BioIndustry Association, said, “and in the City we have one of the greatest concentrations of financial expertise. But somewhere along the line, something is going wrong.”

The industry, he argues, is failing to deliver growth, failing to attract investment and failing to produce the kind of world-leading, independent companies that have emerged elsewhere.

In the United States, where the biotech revolution began in the 1970s, the market valuations of some biotechnology companies have soared. The San Francisco-based Genentech, for example, is worth $81 billion (£40 billion) and employs nearly 11,000 people.

In the UK, despite a stunning roster of home-grown innovations from cloning to cracking the human genome, the largest biotech player is Vectura, a Wiltshire-based specialist in inhaled medicines that is worth a mere £227 million.

A study by Ernst & Young shows that the total market value of all UK-listed biotech companies is about €14 billion (£9 billion). In the US, the figure is $392 billion. In Switzerland the value of life sciences companies listed on the SWX index stands at just under €15 billion.

Yet the report also illustrates the strength of British science. In terms of the number of scientific paper citations by country, America still leads the field easily, with 37,822 in 2006. Britain comes a respectable second, with 7,565. Switzerland boasts just 2,168, in tenth place.

So why the mismatch of scientific excellence and commercial mediocrity? Nowell Stebbing, a British co-founder of both Amgen and Genentech, cites a number of reasons. For investors, biotechnology is a notoriously volatile industry. The risk of developing a new drug is incredibly high, the timeframe involved is painfully long and the fortunes of small companies often ride on just a handful of experimental compounds.

“In the US there have tended to be larger slugs of money going in at an earlier stage, so that companies are not stillborn,” Dr Stebbing said. “That approach is just not done well here.”

All too frequently, he added, promising British companies have been drip-fed, undermining morale and forcing management to spend valuable time and energy on fundraising rather than on building products and revenues. Dr Stebbing argues that this has created a culture in which investors feel justified to meddle in the day-to-day management of companies, disempowering their executives and creating friction.

This conservatism and propensity to meddle is understandable, given the history of the British biotech sector, which experienced a dizzy heyday in the 1990s when dozens of companies floated and valuations soared, only to be followed by a bust that left many investors with burnt fingers. Their willingness to return was not helped by a scandal at the unfortunately named British Biotech, whose value exceeded £1 billion in 1996 before collapsing amid claims that executives had overegged the prospects of Marimastat, its cancer treatment.

Since then, many creditable UK groups have seen their valuations slide, prompting several of the most promising to be snapped up by foreign predators. Novartis, the Swiss drugs giant, paid £305 million for NeuTec, the Manchester University spin-out that is a specialist in MSRA, the hospital super-bug. Celltech, another British champion, was bought by UCB, of Belgium. Last November 3M, an American company, bought Biotrace International.

Meanwhile, despite its success in other spheres, London’s Alternative Investment Market has failed to achieve critical mass in biotechnology and remains host to a slew of small-cap companies with low valuations and illiquid shares.

None of this is helped by what Dr Stebbing describes as a fundamental cultural gulf that exists between British universities and the corporate world. It is a gulf that the US has managed to bridge.

“We seem to be in a negative spiral,” Dr Stebbing said. “Had there been earlier UK successes, it would have been easier to have more. At the moment, it’s hard to see a way out.”

William Powlett Smith, a partner at Ernst & Young, does not share this gloomy portrayal. He believes that, despite the challenges, there are many UK companies that have good prospects, are well managed and are steadily building credibility. What is needed now, he says, is greater consolidation within the sector. He points to companies such as Renovo, which recently signed a deal with Shire, worth up to $825 million, to co-develop Juvista, a scar treatment drug.

It is an interpretation that the Government hopes is accurate. With the decline of manufacturing, Gordon Brown has placed a high priority on life sciences as a potential mainstay of a future “knowledge-based economy”.

Ultimately, though, it is likely that it will be market forces that help the industry to gain a firmer footing in the UK. “If valuations remain depressed, then we are likely to see more US and overseas investors commit venture capital to these assets,” one senior healthcare banker said. “There is a lot of good science, and therefore value, to be had in UK biotech.”

Another stimulus could come from AstraZeneca, Britain’s second-largest drugs company, whose chief executive David Brennan wants to transform it into a biotechnology specialist. AstraZeneca has made a series of biotech acquisitions recently, including the £702 million purchase last year of Cambridge Antibody Technology. If Mr Brennan’s strategy succeeds in generating new blockbuster medicines, the company’s commitment to biotech could act as a powerful catalyst for the UK industry as a whole.

Times Recommends

• Lies and cover-up at MG Rover exposed by report
• High oil prices push manufacturing costs up
• Primark founder steps down as chief executive