IT seems only yesterday everyone was talking about a "Grexit" – the forecast by the UK press that Greece was about to leave the European Union because of the onerous bailout terms imposed by the EU and IMF.

Now suddenly we are talking about the "Brixit" – the possibility, indeed probability, of a British exit from Europe when the current Lisbon Treaty comes up for negotiation in 2015.

Last week three Tory grandees – former chancellor Lord Lawson, former defence secretary Michael Portillo, and Boris Johnson, the Mayor of London – called for British withdrawal from the EU. The former Tory Scottish Secretary Sir Malcolm Rifkind said they'd "hurled a hand-grenade into a small building".

It certainly put a bomb under David Cameron's policy of promising to renegotiate British membership, and then putting the result to a referendum after the next general election. Tory backbenchers, emboldened by Lawson and co, are demanding a firm commitment right away. Labour opposes a referendum.

Viewed from Scotland, where opposition to Europe is muted and we have another referendum on our minds, this seems more than a little surprising. Polls show most Scots want to stay in the EU. But in England – especially the south – there has been a growing frustration with Europe that finally erupted two weeks ago in the English local elections, when UKIP – the party seeking withdrawal from the EU – won up to 25% of the vote.

There is no doubt many in England feel the EU is a "bureaucratic monstrosity", to use Lord Lawson's words, and their democracy is being subverted by Brussels.

But what exactly do they mean by this? When you ask Eurosceptic Tory MPs, they give relatively trivial examples – compulsory seatbelts for children under 12, regulations on food standards, health and hygiene, those infamous straight bananas. But most of these relate to the terms of UK membership of the Single European Market, where standardisation is needed to ensure a level playing field for trading nations.

Similarly, the social protections of the EU, like the working time directive, are meant to make the single market work fairly and stop some countries seeking advantage by forcing their workers to spend longer at work. The "social Europe", as it is called, is hardly onerous, and Britain anyway has an opt-out from the 48-hour working week.

Eurosceptics also talk of the Human Rights Act and claim the failure to deport people such as Abu Qatada is something to do with the EU. This is completely wrong. The Human Rights Act is based on the European Convention on Human Rights set up by Winston Churchill after the Second World War.

None of these, it seems to me, are reasons to go to war with Europe, and deny the benefits of the single market which has undoubtedly boosted prosperity. Trade within Europe has doubled since 1992, thanks to the abolition of tariffs and barriers to the free movement of goods and services in Europe.

The reason Lawson has come out now and demanded withdrawal appears to be to do with the future not of Britain as a whole but of the City of London. The EU is in the process of constructing a fiscal and banking union because of the eurozone crisis. Europe's central bankers have concluded the only way to stop any recurrence of financial turmoil is to create a central European treasury with powers to regulate banks and to issue bonds backed by the EU rather than simply member states. This would stop borrowing costs rising unsustainably as they did in Greece and Spain last year.

To finance all this and police the banks, the EU wants to cap bankers' bonuses and introduce a financial transactions tax on banks' activities. This money would go into an insurance fund to guarantee a future bailout is paid for by banks, not taxpayers. Many regard these measures as the very least needed to get banking back onto a stable and socially-responsible footing.

But the City of London's bankers are fiercely opposed to the tax. They don't want to lose their bonuses either. They say financial services is one of Britain's biggest earners and Europe simply wants to cut British finance down to size.

But a lot of the earnings of the City come from Europe and pulling out of the EU now would hand much of that over to Frankfurt and Paris. Anyway, do we really want to leave Europe just to make life easier for institutions such as Barclays and Royal Bank of Scotland?

Lawson argues Britain could become like Norway or Switzerland. But Norway is a member of the single market and pays a lot for the privilege. It has to abide by the rules about bananas and working times with no say in how they are framed. Switzerland is similarly bound by many conditions of membership in order to stay in the free trade area.

America is opposed to British withdrawal from Europe, and President Obama made his views fully known to David Cameron before the PM's speech on Europe in January. This is partly to do with a lingering fear of Russia. But it is also because a lot of American firms have come to Britain to service European markets, and they don't want to find they are excluded from trade with Europe.

Cameron says he wants to negotiate a change in Britain's relations with Europe rather than simply withdraw. But there seems little chance negotiation will succeed. Europe is determined to introduce major financial reforms to strengthen the EU at the centre, and cut the fiscal liberties of member states. This will not be acceptable to Britain, which leaves a very serious possibility Britain will leave the EU as we understand it.

Scotland has long benefited from Europe, not least from the EU structural fund. Many electronics firms came to Scotland in the 1990s because of the EU. The Scottish Government says Europe has been responsible for more than 60,000 jobs since 2003. Scots also benefit from social chapter rules on maternity pay, working hours and the like.

The big question in the next year is what impact the impending Europe referendum will have on the Scottish independence referendum. Unionists have attacked the SNP for risking Scotland's membership of the EU by becoming independent. But it looks just as likely Scotland could find itself out of the EU by staying in the UK.