Mr Posen, who stepped down last summer, told MPs: "We probably should break up at least the banks that are currently in Government/public control."
His comments came a day after Mr Posen, now president of the Peterson Institute for International Economics, mooted the splitting of RBS in part because it would make the 82% state-owned lender easier to sell.
Mr Posen told the Treasury Select Committee that small business funding in the UK is a "market failure" and "just paltry" compared with what is available in countries such as France, Germany, the US and Japan.
"We need more new entrants into the banking system," he said.
But he said this did not necessarily mean detaching Halifax Bank of Scotland from 40% taxpayer-owned Lloyds Banking Group, where it has sat since its rescue takeover in 2009, or splitting NatWest and RBS.
Instead, it could mean forcing them to sell bank branches to emerging rivals, he said, adding: "The idea is to get some new players over the barriers to entry."
Lloyds and RBS are both in the process of selling portfolios of branches to rivals. But the Independent Commission on Banking's 2011 report, which is guiding banking reform, did not recommend splitting the two banks further.
Mr Posen said incoming Bank Governor Mark Carney could face a big culture shock when he moves from his current post as Governor of the Bank of Canada because of the leeway for dissent allowed to policy-makers.
Nevertheless he warned that incumbent Governor Sir Mervyn King is too powerful and by the time Mr Carney succeeds him the position will be even stronger.
After Sir Mervyn blocked calls for the Bank to buy corporate bonds to boost the economy, Mr Posen said he was "quite furious" at people in the Bank and the Treasury "who let him get away with that".
Mr Posen said two ideas associated with Mr Carney, signalling where interest rates will be long into the future and targeting the size of the economy rather than inflation, "would be a grievous mistake".
He also restated his view that Chancellor George Osborne was cutting Britain's budget deficit too rapidly. He said the idea that business confidence is improving is "false on any market evidence we have".