Former Tory chancellor Norman Lamont said Mr Osborne had "overplayed his hand" with the accusation.

Earlier one Tory MP, who sits on the Treasury Select Committee, called on Mr Osborne to say sorry for what Labour have condemned as a "smear".

Andrea Leadsom said: "Obviously he made a mistake and I think he should apologise."

Mr Osborne had spoken out after publication of a memo last week from former Barclays chief executive Bob Diamond that Whitehall figures had raised concerns about the rate of Libor in October 2008.

It was suggested the document was interpreted by another Barclays banker as encouraging the bank to cut its Libor submissions. The Chancellor told The Spectator magazine that allies of then prime minister Gordon Brown were "clearly involved" in attempts to manipulate the rate.

Prime Minister David Cameron has backed the Treasury's stance that there would be no apology.