During last week's charity tax-grab row, I began to wonder if capping relief on charitable donations was a stunt dreamed up by a misguided Tory spin doctor as a way of justifying the abolition of the 50p tax band.

How better to show that rich people aren't all selfish plutocrats than by focusing attention on their philanthropy? Suddenly, everyone from the Scottish Council for Voluntary Organisations to Universities UK is saying how much they depend on the rich.

Universities alone hoover up £560 million a year from high-value donors and say the cap will damage research. Sir Nicholas Hytner of the National Theatre in London warned arts funding may dry up if the Government limits tax relief to £50,000 a year or 25% of income. Charities like Oxfam and Macmillan have taken to collecting signatures for a "Give It Back George" petition. It's all a little embarrassing for David Cameron as he prepares to relaunch his Big Society next month with a "Giving Summit". Not giving, more like. After the wave of unforced errors over the granny tax and the fuel scare, the man with a plan is looking more like the man without a clue.

But listening to the voluntary sector howl, you might think if it weren't for rich people there wouldn't be any charities at all. Children would starve in Africa, culture would die, cancer research would cease, and the streets would be littered with homeless people. I'm not sure all this is entirely justified. Certainly, high-value donations are important – the top 10% of donors account for 45% of charitable giving according to the Charities Aid Foundation. But there is a moral problem in allowing wealthy people the right to set their donations against tax.

When you or I donate to charities, we don't use it to reduce our tax bills. But we do have the option of donating the income tax that we have paid on it to the charity – if we sign the appropriate Gift Aid forms. You might think the sensible thing would be to get wealthy charity-givers to do this too, pushing up the value of their donations by 40%, instead of appearing to give them a cash-back that can reduce their tax bill to zero.

But that wouldn't solve the problem. Scottish philanthropists like Willie Haughey and Sir Tom Hunter are adamant the cap will make them rethink their charitable giving. Others say if they have to pay tax on the part of their earnings they give away, they just won't give it away. Since the top 100 donors in Britain gave a total of £1.6 billion to charities last year, this could be a problem. But maybe not as great as supposed.

Much of this big-donor charity is in the form of gifts to set up grant-making trusts or foundations. This shelters wealth from inheritance tax and is a powerful incentive to donate not directly affected by the £50,000 cap on tax relief. Some of these trusts can continue to pay a salary to the philanthropist, or to his children and grandchildren, which is why many well-heeled gap-year students are called "trustafarians". This kind of tax planning is a huge industry.

The Chancellor was "shocked' to learn many multi-millionaires are reducing their tax bills to zero. He shouldn't have been. You only need to look up "tax planning" on the internet to find heaps of accountancy firms selling avoidance schemes. Many promise to reduce capital gains tax to zero and seriously reduce income tax using charitable giving at home and abroad. Wealthy people can also take out mortgages on strings of flats, then set the interest against their tax bill, which means the taxpayer subsidises their property portfolio. Surely a scandal in the middle of a housing crisis.

Charitable giving is probably the least objectionable form of tax avoidance, albeit some of the charities are hardly good causes. Why should wealthy people get tax relief, for example, for donating to private schools? The Chancellor said a lot of charities didn't seem to do much charitable work. And having seen the donation game played for high stakes in universities I'm not entirely sure it is healthy for the recipients.

THE way senior academics fawn over wealthy individuals in the hope of getting cash for their universities is demeaning. The scramble to attract high-value donors is pregnant with reputational risk too as university bureaucrats rub shoulders with the super-rich. This is big money. To get a business school named after you could easily set you back £20m.

Philanthropists like giving to universities because they believe it means their names will live on after their deaths. Fair enough. There's nothing wrong with vanity giving if it boosts medical research. But you would think people rich enough to hand millions to universities to endow professorial chairs wouldn't be too bothered about paying their taxes, especially now the 50p band has gone. But they are.

Many wealthy people have a pathological aversion to tax, as if giving money to the Government was theft, even though most tax revenue is used for things like hospitals, schools, or tackling poverty. Why is this seen as less legitimate than giving money to charities? It's partly the Government's fault for failing to make clear that taxation is not legal robbery but the entry fee for a civilised society.

Maybe the Government should just set itself up as a charity and allow people to claim relief on the amount they pay in income tax. Let us all become tax avoiders. Now, there's a thought for the next Budget.