He hit out at the UK Government's "welfare reform juggernaut" which he claimed was "undermining" work done by the Scottish Government to boost the economy and get more people into work.
The "prolonged recession" is impacting on the lives of ordinary Scots, Mr Swinney added.
The Finance Secretary spoke out as Stewart Hosie, the SNP's treasury spokesman at Westminster, warned there was "much, much more austerity to come".
During a debate on the economy at the SNP spring conference in Aberdeen, Mr Swinney said Chancellor George Osborne and Chief Secretary to the Treasury Danny Alexander had adopted the "wrong approach".
The Finance Secretary said: "We said it was the wrong approach, we said they were cutting too far and too fast, and they told us we were wrong.
"George Osborne is today going to be borrowing £190 billion more than he anticipated and predicted in 2010. Not to invest in our economy, but to deal with the price of his economic failure.
"Osborne was wrong in 2010 to take the steps he took to reduce public expenditure in the fashion he did.
"The economy today is 5% smaller than George Osborne predicted it would be as a consequence of the measures he set out in 2010. That may sound like a dry statistical piece of information. But what it means is 5% less opportunities for people to get into employment, for people to improve their living standards, for people to improve their security.
"These may be dry statistics but they affect the lives of the ordinary people of Scotland whose quality of life has been undermined by prolonged recession under the Tory and Liberal Government."
He added: "Just as fast as we invest in the third sector, in helping people into employability, the welfare reform juggernaut comes hurtling towards us undermining the good investment that we are making.
"A Scottish Government should be able to undertake a joined-up approach to employability that focuses on supporting people into work, not stigmatising people and terrorising people as the welfare reform process is doing in Scotland today."
Mr Hosie said economic recovery in Scotland was "taking place in spite of the austerity actions of George Osborne and Danny Alexander, not because of them".
He warned: "What we need to understand is there is much, much more austerity to come. Next year the UK Government plans to take £126 billion out of the economy in cuts and tax rises. With a ratio of four to one cuts to taxes, not only will they stifle recovery further, they intend to continue to balance the books on the back of the poor."
The SNP MP said the UK Government's "austerity medicine" was not working, adding: "In 2010 George Osborne told us debt would begin to fall as a share of GDP this year. That will now not happen for at least another two years.
"He told us the current account would be in balance next year. That will not now happen until 2017-18. He told us public sector borrowing next year would be barely £20 billion. This year he told us it would more than three times that at almost £68 billion.
"The austerity medicine isn't working. George Osborne and this Tory Government have failed on every single measure they have set themselves."
Mr Hosie said: "Mitigating Westminster's decisions can never and will never be enough. We need to to be independent to ensure the damaging decisions we have seen taken in London are never taken here."