THE Royal Bank of Scotland could be forced to pay fines of up to £5 billion over its part in the mis-selling of toxic financial products in the US.

The bank, which is partly nationalised, is expecting to face a penalty over claims arising from the selling of worthless mortgage-backed securities, and has reportedly set aside funds of £1.9 billion pounds to deal with them.

But it has now emerged that the fines could be much more than RBS anticipated, according to market experts who have examined fines paid by other banks.

The fines relate to $32 billion of mortgage-backed debt bound up in complex financial products sold by RBS in the United States, a practice that became widespread within certain circles and has largely been blamed for instigating the financial and housing crashes of 2008.

The bank already agreed to pay a $99.5 million fine in June to settle claims it misrepresented more than $2 billion of mortgage-backed bonds during the housing bubble between 2005 and 2007.

A federal court in Connecticut is handling the new case which addresses a much larger portion of mortgage-backed securities.

RBS Chief Executive Ross McEwan said in October that the bank would not pay a dividend until it had strengthened its capital position and had more clarity over future misconduct charges.

Expert RBC Capital Markets banks analyst Leigh Goodwin said: "We believe that a $5 billion settlement would be towards the top-end of market expectations for this particular issue, and slightly above the levels where other banks have settled in relation to the assets involved, but it's not inconceivable.

"The FHFA settlement is likely to be the largest of the outstanding litigation and conduct hits that RBS still faces, but there are many others, and they are likely to drag on for many years.

"What is clear is that investors need to factor these costs into their earnings and dividend expectations."

In June the FHFA had totalled about $16.1 billion of settlements, stemming from lawsuits it filed in 2011 to recoup losses on roughly $200 billion of mortgage-backed securities purchased by Freddie Mac and Fannie Mae.

Analysts at Credit Suisse last year estimated European banks would take an $11 billion hit from a raft of mortgage-related litigation costs in the United States.

Shares in RBS, which is 80 percent owned by the British government, fell 1.2 per cent to after news of the possible fines surfaced yesterday. yesterday.

An RBS spokeswoman declined to comment on the claims.

The fines would come on top of penalties paid by RBS in recent years for separate issues.

Last year RBS was fined £56m by regulators over a 2012 software issue left millions of customers unable to access accounts.

RBS, NatWest, and Ulster Bank customers were affected in June 2012 after problems with a software upgrade.

The Financial Conduct Authority fined RBS £42m, and the Prudential Regulation Authority fined the bank £14m.

And in 2013 the bank was handed a £390m fine for "widespread misconduct" in rigging the lending rate known as Libor.

A group of bankers at RBS had been found to be manipulating the rate as recently as November 2010, two years after it was bailed out by the taxpayer and even after regulators had begun to investigate the key benchmark rate.

Regulators found that corrupt payments of more than £100,000 were made to those involved and that the bailed-out bank had "abetted" Swiss bank UBS - fined £940m - in manipulating the rate used to set prices on £300tn of financial contracts around the world, from ordinary household mortgages to business loans.