A chair is still a chair, even when there's no-one sitting there, as the lyricist Hal David put it.

Whether Bishop Berkeley would have subscribed to this view we can only guess, but most of us, I imagine, feel Hal is on solid ground with his subsequent assertion: "But a chair is not a house."

Yet why is a chair not a house? I ask, not as an enquiry into Platonic forms, but rather to wonder why we persist in regarding houses as quite different from other material goods. We do not pay stamp duty on chairs, or pay annually for having them, in graded bands distinguishing three-legged stools from Chesterfield sofas. Vince Cable is not calling for a Hepplewhite tax. Not all the Sheratons and Chippendales are clustered together in half a dozen London postal zones.

Now that I come to think of it, that last statement may not be true. But even so, the recent discussion of a mansion tax emphasises (as if we needed any reminding) the British obsession with property and property prices.

The thinking behind it is connected with the dominant mood of the times, which is that a very small number of people ("the 1%") are very, very rich and that – not to put to fine a point on it – the rest of us should take more of their money from them. Of course, we already do; that top 1% earn 13% of the salaries paid in Britain and pay 28% of the income tax collected.

The 50p tax rate has very little to do with this, and may actually cost more to administer than it brings in. Public sentiment, however, is quite clear (if quite wrong) about the fact these high earners are all evil bankers who landed us in this mess, and that they must therefore be made to pay.

The trouble for the Government is that even if cutting the top rate of tax – one of the world's highest, and 5% higher than Germany's, 10% higher than France's and 15% higher than the United States's – brought in more money, it would be politically difficult, because "we're all in it together", which means, again, that the rich must be made to pay, and seen to be made to pay.

The mansion tax is Mr Cable's solution, because it will tax "wealth" rather than income, which the rich seem always to be able to minimise for tax purposes, the better to maximise it for all other purposes. That is why they are rich.

Mr Cable sets his sights on a property tax because, in this country, wealth is tied up in housing. The puzzling thing is why that should be so. At other times and in other places, wealth has been measured in all sorts of ways – gold, cattle, agricultural land, stocks and shares, art. Even, indeed, chairs: three years ago, a leather armchair designed by Eileen Gray and owned by Yves St Laurent sold for €21.9m.

For most homeowners, however, what wealth they have is tied up in their house, which is for almost everyone the most expensive thing they ever buy. One plausible reason for the British belief that you should put your money in bricks and mortar is historical. It is a long time since we were invaded. On the Continent, far more people are happy to rent, perhaps because if your home was in the path of tanks several times during the last century, real estate looks a less certain investment than it does in these islands.

But the other reasons are to do with the inflation in house prices over the past few decades, to the point where many people have come to expect that they will make a killing by moving every few years. In turn, this creates the absurdity that in some areas people who are very well paid cannot expect to be able to buy property until they are in their late thirties, if ever.

The very phrase "property ladder" shows up the oddity of this. The primary purpose of housing is to provide a place to live, not a lump sum which can be cashed in. The mansion tax's great flaw is not to discriminate. Someone in a £2m house which is mortgaged to the hilt would have to pay, no matter their income, while someone with a portfolio of rental houses worth £1.5m each would not.

While it may be difficult to work up much sympathy for them, many people on middling incomes bought relatively cheap houses in areas which were run-down 30 years ago, and have not moved since. Though their homes may now fall into the mansion tax bracket, it doesn't mean they have the money to pay it. Indeed, they are only rich if they sell the house, or borrow against it. And these are precisely the people who haven't played the property market – they just bought somewhere to live.

Anyone who has seen daytime television knows, of course, that some people do buy houses to make money. If you are a landlord, or a property developer who renovates houses with an eye to profit, or a Russian oligarch who thinks a £20m house in Mayfair is a solid investment, you're in it as a commercial player, and the Exchequer has a point in seeing you as fair game. Closing the loophole which allows companies to buy residential property to avoid tax, for example, seems a reasonable step.

But the anomaly arises because most homeowners pay too much attention to house prices. We should hope they gradually fall over the next decade or so. It shouldn't matter, after all, if your house is worth less than it used to be, because the ones you might move to will be cheaper, too. It only matters if you think your house is your pension.

I've never really seen the logic of council tax being based on property values, since the cost of a house has nothing to do with the number of local services its occupants call upon. Since people buy houses out of income which has already been taxed, they are doubly penalised, simply because their priority is their accommodation rather than, say, an expensive motor car or a collection of Sevres vases. But, leaving aside the fact everyone wants a roof over his head, and that the planning system unecessarily restricts the stock available, the ever-increasing price of property is largely created by those who expect their homes also to present them with the equivalent of a lottery win when they cash them in, rather than merely holding their value.

If we stopped thinking like that, we might have a more sensible property market, young people might be able to buy a home and conversation at dinner parties might even become bearable.