RBS was the biggest faller in the FTSE-100, closing down 22p at 345.8p, as the 82% state-owned bank also became embroiled in fresh controversy over bonuses for its investment bankers.
Heavyweight investment bank Goldman Sachs highlighted Edinburgh-based RBS's vulnerability to changing banking regulation, and analysts at Espirito Santo said a further equity raising by RBS, which they think is the weakest capitalised bank in the UK, "cannot be ruled out".
The fall in the share price left the taxpayer's stake further adrift of its 500p break-even price, making re-privatisation a more remote prospect in the short term.
But banking shares have had a strong run since last summer and, despite the fall, RBS's shares are still up 75% on six months ago.
Goldman Sachs analyst Frederik Thomasen downgraded his recommendation on RBS from "neutral" to sell".
"Of the UK banks in our coverage, we view Royal Bank of Scotland as most exposed to near-term regulatory risk," he wrote in a note for clients.
Goldman Sachs thinks an ongoing drive by the Financial Services Authority, on the back of the latest Bank of England financial stability report, could cut as much as 25% from RBS earnings.
Analysts at Germany's Berenberg Bank, meanwhile, warned RBS is unlikely to pay a dividend before 2015.
Espirito Santo downgraded RBS and 40% taxpayer-owned Lloyds to "sell" from "neutral".
"With Lloyds most impacted if regulators moved to standardised credit risks and RBS, the weakest capitalised UK bank, dilutive equity raisings cannot be ruled out," Espirito Santo analyst Shailesh Raikundlia said.
A different, but equally bearish, case was made by Ian Gordon, analyst at Investec. He believes that RBS is adequately capitalised but that investors are expecting too rapid a recovery in lending.
"As such, revenues, returns and, by extension, valuation may yet disappoint the bulls," Mr Gordon wrote in his note.
RBS is expected to follow Barclays and UBS in agreeing a deal with regulators in the UK and US over manipulation of key interest rates within the next two weeks.
It is thought that senior executives, including investment banking chief John Hourican, might resign after failing to spot the problems.
It is now thought that RBS is under pressure to settle the matter by pleading guilty to criminal charges in the US in addition to an anticipated £500m penalty.
Such an admission could leave the bank open to future legal action. UBS's £940m settlement included a guilty plea to felony wire fraud by one of its subsidiaries.
An RBS spokesman said: "Discussions with various authorities in relation to Libor setting are ongoing.
"We continue to co-operate fully with their investigations." RBS's position was made more precarious by reports that it is considering a £250m bonus pay-out to staff in its investment banking division which is heavily implicated in the scandal. This could see Mr Hourican given a £4m bonus.
Matt Basi, senior sales trader at CMC Markets UK, said that the bonus report "comes at an inconvenient time" for RBS, given the looming Libor fine.
Last year RBS paid out £390m and its pool is likely to be smaller than those of key rivals, with some reports suggesting that Barclays is preparing to give its investment bankers £2bn.