He liked the Ivy so much that he ate it, and then swallowed Le Caprice and their sister restaurants for dessert. Now the rag trade tycoon Richard Caring has devoured London’s most famous nightclub, Annabel’s, and observers wonder what other blue-chip playgrounds for the rich he will gobble up next.

Caring’s bulging portfolio includes the exclusive Wentworth golf club in Surrey and a large part of Camden market. Earlier this month he snapped up the former US Navy headquarters next to the American embassy in London’s Grosvenor Square, promptly flying in Thierry Despont, one of America’s most famous interior designers, to give the building a sumptuous makeover.

Annabel’s seems an odd choice for a 59-year-old recluse who seldom gives interviews and was virtually unknown five years ago. It was at this elite society venue for aristocrats and celebs that Frank Sinatra drank, Jackie Onassis danced and Hugh Grant and Jemima Khan fell in love.

Hardly the place, you might think, for a man who rarely breaks cover from his own palatial home near Kenwood House in north London, dubbed the Versailles of Hampstead. The mansion has a master bedroom with the same floor space as the average four-bedroom detached house. There is a 55ft ballroom, a cinema, a 30-seat dining room, a vast wine cellar and a lake in the two-acre garden.

He lives there with his wife, the former model Jacqueline Stead, the Aldershot-born daughter of a retired major, whom he married at Marylebone register office in 1971. They have two grown-up sons, Jamie, a vice-president of MTV Networks Europe, and Ben.

Caring has been more reticent than ever since being identified in the cash-for-peerages controversy as one of the 12 millionaire businessmen who made secret loans to help the Labour party pay its election bills. The party has to return his £2m loan by February. Last year he denied he had refused to be interviewed by police, his solicitor insisting that Caring, who has not been honoured, had offered full cooperation.

With a perma-tan and slicked-back hair that have prompted comparisons with a younger version of the actor George Hamilton, Caring is said to be suave and personable. “Yes, Richard is charming, but he can be ruthless,” said one associate. “You don’t get to his position without some of that running through you.”

Although a deeply private man often described as secretive, an unexpectedly flamboyant side emerges at his extravagant charity bashes. Two years ago he splashed out a reported £8m in flying 450 guests, including Sir Elton John, Bill Clinton, Tina Turner and Liz Hurley, to his Napoleonic ball in St Petersburg. Less publicised was the rationale behind the event – it raised £13m for the victims of paedophiles. “One paedophile can ruin the lives of of as many as 450 children in as little as three years,” he said. “You can’t read statistics like this and be unmoved.”

With a fortune estimated at £450m by the Sunday Times Rich List, Caring can well afford personal crusades. It enabled him to pay well over the odds for Annabel’s and four associated clubs (£90m), Wentworth (£130m) and the US Navy building (£250m). Last month he sold the Strada restaurant chain for £140m.

His predilection for the grand gesture is something he shares with the retail mogul Sir Philip Green, whose close friendship and long collaboration have been key to their mutual prosperity. (A recognition of this was the red Ferrari Spider sports car he presented to Green on his 50th birthday). Caring was one of the first businessmen in the UK to spot the opportunity of manufacturing cheap clothes in the Far East and supplied the major retailers, including Green.

The two men have been characterised as “brothers” for more than a quarter of a century. It was said Green would not make a move in the retail section without Caring’s approval: “Everything [Green] does is discussed with him. Green’s success is his ability to cut costs and in that, Caring is crucial,” said a colleague. When Green bought the Bhs clothing chain in 2000 he ordered his buyers to go through Caring, who was given a permanent office in the store’s headquarters. After turning round Bhs, the pair pulled off the same trick at Arcadia, the group behind Topshop and other stores.

Caring’s acquisitive eye alighted on Annabel’s while the two friends were celebrating at the nightclub, and he became familiar with the owner’s plight. Mark Birley founded the club in 1963, naming it after his wife, who famously ran off with Sir James Goldsmith, whom she eventually married. Birley, now wheelchair-bound and 77, wanted to hand Annabel’s on to his son and daughter, Robin and India Jane, but a family feud drove a wedge between the children and led Birley to sack Robin from the business.

Last week’s deal was clinched by Caring’s promise not to commercialise the business, which flies in the face of his avowed instincts. Last year Caring scooped up the Ivy and its sibling eateries, Le Caprice, Daphne’s, J Sheekey, Bam-Bou, Rivington and Urban Caprice – and then nabbed another, Scott’s of Mayfair. In a rare interview he declared: “I’ve bought restaurant brands and the world today is brands. There will be a second Le Caprice in New York and we are also looking at Moscow.”

A new upstairs section at the Ivy has unsettled regular clients such as Michael Winner, although the restaurant critic recently praised the unchanged decor downstairs and pronounced the food “pleasing in a totally unostentatious way”.

Fears that Caring will update Annabel’s may deter older members. “The Old Etonian crowd might think, ‘This might be the time to chuck in my card’,” said a patron.

They may well hope Caring will be too preoccupied with the four other posh establishments that came in the job lot. These are Harry’s Bar, the agreeably louche restaurant and bar on South Audley Street; the nearby George, a private dining club popular with hedge fund managers; Mark’s, the most prestigious of the lot, off Berkeley Square; and the Bath and Racquets club, with London’s most elegant gym.

It’s a heck of a lot of fancy joints for the son of a US GI. Caring was born in 1948 to Lou Caringi, an American-Italian soldier forced to convalesce in London during the second world war, and Silvia Parnes, the nurse who cared for him. After the war Lou stayed on, anglicising his name by dropping the final i and establishing a clothing industry that sourced Marks & Spencer and other stores. The family, living in Finchley, north London, had a showroom in the capital’s fashion district off Great Portland Street.

Caring grew up surrounded by the rag trade, learning from his father and eventually joining the business. “His father’s work ethic has always been with him and his children know the value of money, too,” said a former colleague.

The realisation that he could undercut British garment makers led him to set up a base in Hong Kong, where he monopolised the burgeoning cut-price fashion industry. Not content to trade in clothes, he cashed in on the booming Asian property market. But the assertiveness of emerging countries in recent years saw Caring increasingly eclipsed as a middle man. Taking stock, he scaled down his involvement in fashion and looked for a new direction. His new outlook was partly shaped by a narrow shave during the tsunami on Boxing Day 2004.

While holidaying with his family in the Maldives, he was scuba-diving with his sons when the wall of water passed over their heads, wreaking devastation when it hit the shore. That morning his divemaster said the family could either sail to the other side of an atoll or enjoy a more relaxed day’s diving nearby. They decided to take the easy option.

“I don’t know why, but it saved our lives,” he recalled. “There wasn’t anything clever about it, we were just lucky. We were about 100ft down, with the speedboat waiting on the surface. We dived for about 45 minutes and felt nothing. Later we began to get hysterical phone calls from around the world asking, ‘Are you all right?’ My first reaction was, ‘Why wouldn’t we be?’ ” One of the callers was Green, who sent a plane to pick up the family. Caring later gave £1m to the Tsunami Appeal. “It does change the way you think, the way you look at the world,” he reflected. “Experiencing something like that puts things into focus.”

On arriving home, Caring began to spend his money on the things he loved. He loved playing golf at Wentworth so much he bought the club. Now he is said to have his eye on Carluccio’s, the Italian cafe chain. One by one, the gold dominoes are falling into his hands.