Some of America’s wealthiest socialites were facing ruin last night after the arrest of a Wall Street big hitter accused of the largest investor swindle perpetrated by one man.

Shock and panic spread through the country clubs of Palm Beach and Long Island after Bernard Madoff, a trading powerbroker for more than four decades, allegedly confessed to a fraud that will cost his wealthy investors at least $50 billion – perhaps the largest swindle in Wall Street history.

Mr Madoff, 70, a former Nasdaq stock chairman, was apparently turned in by his two sons and arrested on Thursday morning at his Manhattan apartment by the FBI. Andrew Calamari, a senior enforcement official at the US Securities and Exchange Commission, described the scheme as “a stunning fraud that appears to be of epic proportions”.

The FBI’s criminal complaint states that when two federal agents arrived at Mr Madoff’s apartment, he told them: “There is no innocent explanation.” The agents say that he told them “he paid investors with money that wasn’t there”, that he was “broke” and that he expected to go to jail.

Many of his investors came from the enormously wealthy enclaves of Palm Beach, Florida and Long Island, New York, where people had invested billions in Mr Madoff’s firm for decades. He was a fixture on the Palm Beach social scene, and was a member of some of its most exclusive clubs, including the Palm Beach Country Club and Boca Rio Golf Club, where he drummed up much of his business.

The FBI claims that three senior employees of Mr Madoff’s investment firm turned up at his apartment on Wednesday to ask questions about the company’s solvency. Two of them are believed to be his sons, Andrew and Mark, who have worked for their father for two decades.

Mr Madoff told them that he was “finished”, that he had “absolutely nothing”, and that “it’s all just one big lie”. He said the investment arm of his firm was “basically a giant Ponzi scheme”, and that it had been insolvent for years.

A Ponzi scheme, named after the swindler Charles Ponzi, is a fraudulent investment operation that pays abnormally high returns to investors out of money put into the scheme by subsequent investors, rather than from real profits generated by share trading.

The FBI complaint states that Mr Madoff told his sons that he believed the losses from his scheme could exceed $50 billion. If that is the case, his fraud would be far greater than past Ponzi schemes and easily the greatest swindle blamed on a single individual.

There has been scepticism for years on Wall Street over how Mr Madoff managed to pay such consistently high returns. Ponzi schemes inevitably collapse, and Mr Madoff found himself to be no exception. This month, clients asked for $7 billion to be returned, the FBI says.

Mr Madoff ran the scheme separately from his main business and his sons had no involvement in it.

Mr Madoff has been charged with a single count of securities fraud. He declined to enter a plea in Manhattan’s US District Court and was released on $10 million bail. He faces up to 20 years in jail and a $5 million fine if convicted. His lawyer, Dan Horwitz, said that his client was “a person of integrity. He intends to fight to get through this unfortunate event.”

One investor told The Wall Street Journal: “This is going to kill so many people. It’s absolutely awful.” Ira Roth, from New Jersey, said that his family had $1 million invested, and that he was in a state of panic.

Money Central: The 10 most audacious swindles ever

Times Recommends

• ‘Lame duck’ fear for FSA as chief quits
• Storm over bailout of EU's most ailing economy
• Northern Rock to drop 100% savings guarantee