HOW many people do you know who live in houses worth more than £2m?

Thought so. The prospect of a mansion tax on properties above this level may be spreading fear and loathing in the mean streets of Chelsea and Kensington, but in Scotland there was only a tiny handful of houses worth this much. Registers of Scotland couldn't even put a figure on it when I asked them yesterday. How many people do you know who make more than £150k a year, about 0.6% of UK earners? Even the First Minister of Scotland only earns £135,000, so he won't be worrying over much about paying 50p tax on earnings above £150k.

Nothing better illustrates the gulf that divides Westminster from the rest of the nation than the current row over whether or not to keep the 50p tax band on earnings or replace it with a mansion tax.

It's like a spat between plutocrats over who should pay the restaurant bill in La Gavroche, while the rest of us miserable taxpayers gaze with horrified fascination, noses pressed against the glass. Chancellor George Osborne says he's minded to scrap the 50p band because business organisations say it damages the economy. Business Secretary Vince Cable says he wants it replaced with a 1% tax on properties worth more than £2m.I'll tell you what I'd do: keep them both. We live in a grossly unequal society where the top 1% have cornered a disproportionate share of the national wealth. In the interests of equity and social solidarity, they need to contribute and be seen to contribute.

In Germany, Europe's leading industrial power, 50p tax doesn't appear to be doing too much damage. There, the highest rate of income tax is 45% – but on top of that all citizens pay a 5.5% "solidarity surcharge" on all earnings over 1000 euros. The wealthy pay very much more in Germany than in Britain because here the 50% is only levied on income above £150,000. But British businessmen still complain that they will be paying a marginal tax rate of 58%. Well, tough. Middle-class families earning £42,000 face a 70% marginal tax rate because of benefit changes this spring, according to the Institute for Fiscal Studies. And the really poor of course face a far more serious prospect: the effective loss of entire benefits if they don't find work.

And the mansion tax? Well, it seems we are to feel concern for people who have the misfortune to live in extremely expensive houses. Grant Shapps, the Coalition's housing minister, said on BBC yesterday that the mansion tax would be a "granny tax" that would hit elderly people who are property rich but cash poor. Poor dears. There they are sitting in their mansions, living off tins of cat food, worrying about how to get the Rolls through its MOT. I don't know any grannies who live in houses worth £2million, but I do know about the near two million elderly people who are living in poverty, mostly because they fail to get the benefits to which they are entitled.

And what about the many thousands who are about to lose child benefit, or be evicted from their homes through the welfare cap, or lose working family tax credits because they work part time ? The London media seems utterly preoccupied with the 50p/mansion tax row, which I'm sure has nothing to do with the fact that many UK newspaper editors, columnists and television presenters live in mansions and earn more than £150,000. The London Mayor, Boris Johnson, said recently that the £250,000 he gets for writing a column for the Daily Telegraph is "chickenfeed". These people live in a world of unreality, consoled by myths about taxation stifling initiative and drive – but only among People Like Us.

Here are a few reminders for the Mayor and his chums about what chickenfeed looks like. Median annual full-time gross earnings in Scotland currently amount to £25,600. That's the median, note, not the average. It means that half of Scottish full-time workers earn less than that sum before tax. Retired people are paying tax on incomes of little more than £10,000, even as their savings are destroyed by inflation and near-zero interest rates. Why is no one arguing about that?

But that's the politics of envy –say Tory MPs. Tax isn't a question of fairness but of efficiency. The 50p tax band hasn't brought in the £2.8bn it was supposed to yield because wealthy people just find ways of avoiding it. So, they say, why keep a tax that discourages wealth creators to come to Britain? Even the Liberal Democrats seem to have swallowed this, which is why they are looking at taxing property rather than income, on the grounds that houses can't be relocated to tax havens like Monaco or Liechtenstein.

But imagine if this principle were applied to people further down the income scale? The builder, for example, who offers to do your bathroom cash in hand, no questions asked. "Well, you know", he'd say "I'm sorry, but VAT is a very inefficient tax because it's so easy to avoid. So I'm going to do the Revenue a favour by not paying it." And I wonder what the Revenue would do if we miserable PAYE payers said: "Look, income tax is economically harmful because it lowers my incentive to work, so I've decided, in the interests of the broader economy, to withhold payment until further notice."

No – the problem with the 50p tax isn't inefficiency but fiscal defeatism –an unwillingness to combat tax avoidance. It is an insult to millions of responsible taxpayers who don't have the option of outsourcing themselves to a tax haven, or registering themselves as non-domiciled, or setting up front companies to buy their houses for them in order to avoid paying stamp duty. Leona Helmsley, the property speculator, famously said that "only little people pay tax". Well, it's time for the little people to bite back.