It's a good job the new book Tory Modernisation 2.0: The Future Of The Conservative Party wasn't published earlier, or someone might have given it to me for Christmas, which would certainly have put a dampener on the festive season.

In it, a group of Tory modernisers, including the Cabinet Office minister Francis Maude, David Willets, minister for higher education, and the journalist Matthew d'Ancona argue, in the words of the last, that "the shelving of the modernisation campaign was the worst strategic error made by the Conservative Party since the poll tax". There are many things – almost everything, in fact – in contemporary political discourse which make me throw my hands up in despair, but this may constitute a new low.

If this section of the Conservative Party believes the primary failing of David Cameron's government is an insufficient emphasis on same-sex marriage, the plight of the polar bear, international aid or building wind farms, or that the public is more interested in such policies than – to pick a random example – getting out from under the avalanche of debt brought crashing down by Gordon Brown, we may as well send for the men in white coats.

It's not that there's nothing in modernisation. Mr Maude's discovery that polling suggested support for any policy halved when people were told it was a Tory proposal was convincing evidence that the party had an image problem. Its electoral standing in Scotland conclusively illustrates that; even if they are not a majority, there are many more people here with basically conservative views than are prepared even to countenance voting Tory.

The problem is in what the Cameron wing of the party chose to modernise. Their priorities have been those of the metropolitan chattering classes rather than the aspirational low-to-middle income voters who ought to be their natural constituency, not least because they are the section of the population which decides elections. Though this group is certainly more socially liberal, and racially mixed, than it once was, many of these people hold decidedly right-wing opinions – not all of which I share – on many issues. The toxic aspect of the modern Tory party is not so much that it is too right-wing, but that its leadership is too rich, too complacent and too out of touch.

The same charge could just as easily be made of the other parties, of course, which is probably the reason for the general dissatisfaction with politicians, but it is the Tories to whom the label sticks most easily. Nobody may be very enthusiastic about the alternatives, but it's the Conservatives they hate.

Tory governments before John Major's had, however, at least one thing going for them: a reputation for economic competence. That may not always have been deserved, but they compared favourably with Labour administrations, every one of which left office with the country in worse financial shape than it was before. The one thing which might have won the electorate round to Mr Cameron, then, would have been to sort out the economy. Instead, the Chancellor has persisted in the borrowing of the Brown years. Indeed, it's worse. By the time of the next election, the Coalition Government will have borrowed more than the last Labour Government.

So the Tories have managed to saddle themselves with the rhetoric and consequent unpopularity of austerity, cuts, "attacks" on the welfare-dependent and so on, with none of the benefits which might have accrued if they were, you know, actually doing any of that. Because they aren't. There ought to be a ticker-tape running across the bottom of the screen during every news programme about Austerity Britain which repeats the words: public spending is still going up.

Paul Goodman, editor of the ConservativeHome website, former MP for High Wycombe and my old boss on the leader-writers' desk of The Daily Telegraph, stated bluntly yesterday that Mr Cameron cannot win a majority. He gave a number of reasons: the Tories' failure to gain support from ethnic minorities; the built-in advantage the Labour Party has from current constituency boundaries; the defection of many Conservatives to UKIP and, by contrast, of many former LibDem supporters to Labour (thereby splitting the right and uniting the left).

It was never within the Prime Minister's power to overcome all these obstacles, but one thing is obvious. His "modernising" policies are part of the problem, not the solution. Concentrating on the green agenda or same-sex marriage or reform of the voting system does plenty to alienate some natural Tories (the ones flocking to UKIP). But I can't see that it increases the party's appeal to ethnic minority voters or gays or environmentalists or anyone else since, being people, they are chiefly interested in what all other people are interested in at the moment: how on earth we'll ever get out of this mess.

What Mr Goodman didn't list as a reason for his conviction that Mr Cameron can't win in 2015 is the most obvious point of all. And it is this: whether you'd prefer more conservative Tories, or more liberal Tories, or greener Tories, or bluer Tories, you have no reason to vote for them if their economic policy doesn't improve matters. And it can't do that if it doesn't differ from the economic policy of the last government.

If, in two years, the electorate were to be convinced things were fundamentally better – even if recovery and strong growth had not yet materialised – they might overlook any given aspect of the Tory party which didn't chime with their own views. Former LibDem voters might not actually vote Tory, but at least they wouldn't vote Labour. Diehard Conservatives might stop banging on about immigration, crime and the EU and feel the party had been vindicated, and ought to be rewarded. And the aspirational swing voters who gave Margaret Thatcher and Tony Blair three election victories each might feel that Mr Cameron deserved their gratitude, and possibly their support.

That would all have been very nice but, unfortunately for the Tories, it can't possibly happen. That's because, as with anything, it doesn't matter whether something is ancient or modern if it doesn't bloody work in the first place. No number of gay weddings or wind turbines can disguise the fact, by any rational estimation, the Tories' greatest strategic error since the poll tax is continuing to spend and borrow like there's no tomorrow.