Sir Jon Cunliffe said that while progress had been made towards resolving the problem, further reforms were needed to guard against the risks posed by a "failing global giant".
The former Treasury mandarin said G20 leaders needed to overcome any divisions to be able to reach agreement on the issue later this year - and that public mistrust over the sector would not abate until it was resolved.
Sir Jon who was at the heart of international crisis talks at the height of the financial crisis in November 2008, made the remarks at a conference in London.
He said: "Nothing has so incensed public opinion and damaged societal support for the financial sector than the apparent 'heads I win, tails you lose' experience of private profits and pay when things went well and public losses when they did not.
"We will not fully restore public confidence until we can show that we can resolve failing banks - no matter how large - without public support.
"We have made a lot of progress on too big to fail. But we are not yet there. I do not think we can say with confidence now that we could resolve a failing global giant."
When Lehman Brothers collapsed in 2008, it sent shock waves throughout the financial system. Later the US government decided it must step in to save insurance giant AIG.
In the UK, Royal Bank of Scotland and Lloyds Banking Group both had to be rescued by taxpayers.
Sir Jon, speaking at the Chatham House City Series conference, outlined progress that had been made since G20 leaders agreed at that time that no institution could be "too big to be allowed to fail".
He said the UK, EU and US would soon have legal powers to break up failing banks with far less threat to the economy as a whole, with the bank's creditors rather than taxpayers made responsible for losses.
In addition, legislation in the UK and US separated key banking activities in the domestic economy from riskier business, while the EU was also considering how to address the issue.
Meanwhile, international agreement on further reforms on how major international banks fund themselves and how their winding up could be made less disruptive was close to being reached, he added.
This would require authorities in the countries where they operated to reach a deal.
Sir Jon said: "Getting agreement on international standards to end Too Big to Fail is perhaps the most important regulatory priority for the G20 summit in Brisbane in November this year."
But he warned: "Unless we can maintain and foster mutual trust as we implement the reforms, we will not succeed in, on the one hand, maintaining an integrated global financial sector, and, on the other, ensuring it is proof to frequent destabilising crises."