The political blame game over the banking scandal threatened to descend into open warfare after George Osborne accused Gordon Brown's circle of being "clearly involved" in the Libor rate-rigging affair.

The Chancellor's comments signalled a significant stepping- up in the Conservatives' attack on Labour over the actions of bankers. It came after Prime Minister David Cameron accused the Opposition of fearing its "dirty washing" was about to be exposed in public.

Labour hit back last night, accusing Mr Osborne of desperation in the face of an escalating crisis and of "lashing out in a frenzy" that demeaned his office. The party said the Chancellor was scoring political points instead of getting to the bottom of the scandal.

The parties are embroiled in a game of brinkmanship over the Prime Minister's decision to order a parliamentary inquiry into the rate-rigging affair.

Labour has threatened to boycott the investigation amid fears it will be little more than a witch-hunt by Tory MPs.

Attempts to find a way through appeared to falter as Mr Osborne singled out the last Labour administration, saying: "They were clearly involved and we just haven't heard the full facts, I don't think, of who knew what, when."

His comments followed the release of a Barclays memo from October 2008 which appeared to suggest "Whitehall" sources and the Bank of England had tried to reduce a key interest rate.

Mr Osborne said it was clear the central bank had not instructed Barclays to get its rate down. He added: "As for the role of the Labour government and the people around Gordon Brown, well, I think there are questions to be asked of them."

The Conservatives also seized on comments by former Barclays boss Bob Diamond. He told the House of Commons Treasury Committee yesterday that in 2008 he felt the Bank of England was passing on concerns from "ministers in Whitehall" over the high levels of its rate.

Tory MP Jo Johnson called for "clarity" on who these ministers were and what they said.

Mr Balls, Labour's shadow chancellor, Baroness Vadera, a Treasury minister under the former prime minister, and Paul Myners, a former City minister, all denied speaking to anyone at the Bank of England about the Libor rate-setting process.

Chris Leslie, Labour's shadow treasury minister, said it was "increasingly clear" Mr Osborne was interested only in playing party politics. He added: "We need an independent, forensic, public and judge-led inquiry that can rise above party politics. David Cameron and George Osborne need to explain why they have spent the past week desperately trying to resist one."

Labour has threatened to refuse to participate in an inquiry, which would be dominated by Coalition MPs. It has called for an independent judge-led investigation instead, but the Government is threatening to vote that down when MPs debate the issue today.

Yesterday Labour leader Ed Miliband urged David Cameron to hold a two-part judicial inquiry. This could make recommendations in time for the Coalition's deadline of next January and also look at wider issues. Ministers want the review to report by the end of the year so they have the chance to make changes to the banking reform Bill early in 2013.

However, Mr Cameron appeared to reject that suggestion and reiterated his threat to vote down Labour's call for a public inquiry.

The Prime Minister said: "If he wants a quick resolution, he must accept the outcome of the vote in the House of Commons.

"I am prepared to do that, why isn't he?

"People will take a very dim view of an opposition party that stands in the way of an inquiry because it doesn't want its dirty washing done in public."

The row centres on the interbank lending rate (Libor). This shows how easy or difficult banks like Barclays find borrowing money from other institutions.