We were warned there might be a post-Olympic hangover, but the speed with which the news agenda has turned negative has been, as sports commentators put it, simply incredible.

The train operators have broken all records in the race to increase rail fares, high street sales are sinking faster than a relegated team of synchronised swimmers, the eurozone has sunk into the pit of recession again, inflation has bounced back, and even the Dandy is threatened with closure. And of course we have the continuing saga of under-employment, a headline reduction in unemployment concealing a record number of workers – 1.4 million – forced to accept part-time work.

It was always facile to believe that something as ephemeral as the Olympics could influence economic performance, promote world peace or benefit anyone apart from multi-millionaire athletes and their sponsors. The Games were an emotional spasm, like the funeral of Diana, Princess of Wales, which gave everyone a moment of shared experience that cannot be sustained for long.

The Games gave thousands of unpaid volunteers the chance to be nice to people for a change in London. You can't help thinking that we could all do with more of that in times of financial hardship. But context here is everything. If someone in a purple shell suit came up to me and asked me if I was having fun, my first reaction would probably be to run a mile because the only people who normally approach me in the street are drunks, charity muggers and opinion researchers wanting to know my views on the Olympics.

National psychology does, however, have an impact on the economy, as the economist John Maynard Keynes noted half a century ago. He talked of the "animal spirits" of entrepreneurs being depressed by a lack of business confidence. The very phrase "economic depression" encompasses this psychological dimension. Banks stop lending because they don't think it's worth the risk; businesses give up and sack their workers; and people looking for work lose hope after a couple of years.

We shouldn't over-estimate the psychological downturn. In Scotland there is evidence that people have become less depressed during the economic crisis, at least according to Dr Katherine Trebeck of Oxfam, who has compiled a "humankind index" of personal wellbeing. After intensively interviewing 3000 Scots, she says that in terms of "collective prosperity", human relationships, health, community and social activity, Scots have actually become a bit more positive during the recession. Indeed, it may be that hard-pressed consumers are relieved at having a break from the relentless pressure to spend that had come to dominate ordinary peoples' lives before the great "correction".

I'm not saying the Government should just tell people to buck up because they've never had it so good. Of course many people are experiencing real hardship, and all those part-time workers are living day to day. But we shouldn't ignore evidence just because it doesn't fit into preconceived notions of economic rationality. There's further evidence, indeed, that we went into this economic downturn with a rather more positive attitude than in previous recessions.

Those million-and-a-half workers accepted short-time hours because they thought it would be temporary, and firms tried to hold on to skilled workers to prepare for the upturn. We really did try to keep calm and carry on. But with Europe now slipping back into recession, attitudes will harden, and a real shake out of those jobs is now imminent.

Recessions are a huge waste of resources. Machine tools that have been idle for four years will be scrapped, idle employees will lose their skills, product innovation will slump, markets will wither. Only the drug dealers, loan sharks and pawnshops are looking optimistic.

Critics of the Government's austerity programme say this depressive psychology can be mitigated. The Nobel-prize winning economist Joseph Stiglitz, who is speaking at the Edinburgh Book Festival next week, says that the role of government should be to replace the confidence that the depressed private sector has lost. He is horrified by the waste of productive resources that arise from people and factories being left to rust, while British firms sit on a cash mountain of £700 billion. Governments need to compensate, through spending, for the lack of effective demand in the economy. If they don't, the depression will just go on and on.

Other schools of economists disagree. They point out that the Government can't go on spending much more borrowed money without plunging Britain into a sovereign debt crisis, like Spain or Italy. And they are right, because public and private debt in the UK is currently running at around five times GDP. We are on an economic knife edge here. However, it is equally the case that by doing nothing, and allowing the recession to continue, Government debt will increase anyway because tax revenues will fall and more unemployed and underemployed families will have to be supported by taxpayers' money. It's the vicious cycle of debt deflation.

Keynes famously said that the Government should pay people to "dig holes and fill them in again" and that, I suppose, is one way of looking at the £13bn spent on the Olympics. It has certainly boosted parts of the economy other Government spending cannot reach. And since tangible assets have been created, such as velodromes and parks, this is not all wasted cash. But there is a limit to what governments can do by going around spending money they don't have. There needs to be a positive response from the public. But the truth is that, burdened with debt, consumers are spent out.

We need to reflect on what kind of economic recovery we should be looking for. It can't be built simply on importing more trashy consumer goods from China. Nor can it be done by feeding the debt monkey and handing more public money to bankers through quantitative easing. Governments need to focus on worthwhile investment in areas such as renewable energy, education, sustainable infrastructure, health and elderly care. There isn't any choice. And intriguingly, that's where people are going of their own accord – away from empty consumerism. Perhaps we need a few thousand economy makers to get this show on the road.