The Chancellor wants to cut billions of pounds more from the benefits bill after the next general election, but faces opposition from other Tory ministers.
Yesterday he hailed a new report calling for greater investment in infrastructure. However, he added: "I think the real choice in our country is (between)... spending money on this big economic infrastructure... and spending money on, for example, welfare payments which are not generating either a real economic return and at the same time, are trapping people in poverty."
He also said that he wanted to rebalance the economy so that the UK was not "wholly dependent" on the "global city that is London".
And he said that the whole of the UK could benefit from growing the GDP of the North of England to the UK average.
A number of politicians, including the First Minister Alex Salmond, have warned that London and the South East of England attract too much investment and infrastructure, as private cash follows public expenditure.
Mr Osborne's comments came ahead of the publication of a £15 billion plan to make northern cities better connected.
Shadow treasury minister Shabana Mahmood said that Labour welcomed the report.
But she added: "Only Labour will properly back our city and county regions with ambitious plans to devolve more funding and economic power to them.
"Mr Osborne will be judged on his actions, not his words. He is failing to back the Heseltine report or Labour's plans to devolve billions of pounds of funding.
"Infrastructure output has fallen by over 12% since 2010. And the Chancellor has refused to back our proposal for an independent infrastructure commission to end the dither and delay on long-term decisions."