In evidence to MPs, Mr Richardson denied that Co-op Bank's woes stem from loans made by Britannia building society which he ran before their 2009 merger.
He told the Treasury committee that he left in 2011 after warning that attempts to buy 632 branches from Lloyds Banking Group would be "absolutely disastrous".
The deal collapsed in April but had it gone ahead it "could well have been a replay of Northern Rock" which was nationalised in 2008, Mr Richardson claimed.
In an unusual move, the Bank of England said it "strongly disagreed" with Mr Richardson's comments. Its deputy governor Andrew Bailey had previously told the committee that Co-op's "main issue" was the Britannia loan book.
Treasury Committee chairman Andrew Tyrie said there was a "yawning gulf" between the two accounts.
"The committee will be investigating this a good deal further," he added.
Mr Richardson told MPs he had kept notes of meetings he held with senior Co-op Group figures, including former chief executive Peter Marks about his opposition to buying Lloyds's Verde portfolio.
But after being approached by investment bank Credit Suisse about buying Verde, they were keen to proceed with a bid in part due to a desire to expand in Scotland where Co-op would have acquired the 185 Lloyds TSB Scotland estate.
He said he viewed the decision as "dangerous for the bank". His opposition was "pragmatic" because Co-op Bank managers were already overloaded coping with the financial crisis, integrating Britannia, information technology reforms and the sale of its life and savings business as well as a management overhaul, he claimed.
The decision to go ahead with the bid led Mr Richardson to "bring things to a head" and engineer his own departure.
"For a chief executive to take the steps I took, I think it is incredibly serious," he told MPs, because it ended his executive career.
But he said that he had not spoken out publicly at the time about his worries because it could have had "serious consequences" for Co-op.
Mr Richardson said he was "surprised at the quantum" of the £1.5 bn capital increase demanded of Co-op Bank by regulators after the unravelling of the Verde deal.
Lloyds will rebrand that business as TSB next week with a view to floating it on the stock market.
The regulator, he said, "has had a significant influence on that £1.5bn black hole" and should have phased in rule changes which had led to Co-op taking large provisions on loans.
He said to claim that loans made up to a decade ago were now turning sour, despite repeated audits "simply does not make sense".
Rather, the move to buy the Verde business had led also the bank to "take its eye off the ball" in its management of loans.
Co-op had also been hit by writedowns on technology and mis-selling compensation, he said.
Mr Richardson also criticised an overhaul before he left that had him answering to Mr Marks instead of the Co-op Bank board.
"For the financial people in a bank to report to people who do not have financial services experience is, in my mind, dangerous," he said.
Some Co-op investors object to a restructuring unveiled by new group chief executive Euan Sutherland designed to close the capital hole by imposing losses on bondholders and listing the bank on the stock market.
Mr Richardson denied that he was "ducking and diving".
"One of my personal values is taking personal responsibility," he said.