THE shooting of the Labour MP Jo Cox, on the day that Ukip’s Nigel Farage unveiled one of the most racist posters in British electoral history, caused the referendum campaign to slam on the brakes. The killing of British MP by a middle-aged white man alleged to have had far right leanings is a wild card. I don't intend to dishonour Jo Cox's memory by drawing the obvious political conclusions from it, or speculate about its impact on the result.

It was an unspeakably tragic conclusion to this dismal referendum campaign, which has been fought with threat, fear and often a wilful disregard for the truth. The refugees depicted on the Brexit “Breaking Point” poster were not EU citizens trying to enter Britain, but Syrian refugees in Croatia. They have no right of free movement in the EU, and Britain already has control of the numbers of such refugees that can come here. The image had nothing to do with the EU and everything to do with race, and fomenting fear of it.

This lapse into the politics of the BNP makes it easy for those of us who have been critical of the EU to swallow our reservations and vote for Remain. The official Leave campaign has disowned the poster, but they cannot avoid guilt by association. Immigration has been central to the case for withdrawing from Europe made by Michael Gove, Boris Johnson and Liam Fox. The Brexit campaign, as I have said before, simply cannot avoid returning to the issue, like a dog to its own vomit.

This is not to deny that Remain has been guilty of its own version of the politics of fear. Last week's “punishment budget” from Chancellor George Osborne, threatening to slash public services and increase taxes if Britain votes to leave the EU, put the tin hat on an intellectually disreputable campaign of economic alarmism. Project Fear has alternated with the naïve Pollyanna Europeanise of politicians like Kezia Dugdale, gushing about our membership of “this wonderful European family”.

The EU is not a family – or if it is, it is one currently headed by an austere and domineering patriarch who cares little about the welfare of its weaker members. The way the EU Troika has imposed a 1930s style depression on the people of Europe – leading to unemployment of up to 50 per cent among young people in Spain and Greece – is economically illiterate and socially divisive. The European Union is like America in the Depression, but without a Roosevelt .

So why am I still intending to vote to remain in this bankers' club, this playground for corporate lobbyists, with its wasteful and protectionist Common Agricultural Policy and its nonsense bureaucracy? Well, I'm afraid I find myself agreeing with Jeremy Clarkson: because my gut tells me to. I can't help thinking there is still something to be gained from being in an economic alliance with the rest of Europe. This is bolstered by lingering left-wing notions of international solidarity, multiculturalism and brother/sisterhood that perhaps bear little relation to the reality of Europe, but have a huge emotional power over people of my generation.

Perhaps it is easier to put it the other way: what would life look like if we were out? I don't subscribe the forecasts of economic apocalypse from the Remain campaign. Britain would get along fine on its own, just as Scotland would have had we left the UK. Eventually. But in the immediate aftermath of Brexit there will be a degree of instability, both economic and political. The departure of one of the pillars of the European Union would send shock waves across the continent that would be heard as far away as the Kremlin. David Cameron's warning of another war after Brexit was over the top, and a little silly. We really don't matter that much. But Europe is in a fragile state at the moment and its chaotic disintegration if other nations followed our lead could lead to a revival of destructive nationalisms.

Of course, Leavers say Europe is destroying itself pretty well on its own, and they're probably right. Britain’s departure may not make as much difference today as it would have a decade ago. But my principal worry is domestic, not geo-political. It is about the way the Brexit campaign has summoned the genie of immigration, south of the Border at least. Racial antagonism has never been far below the surface in UK politics, but we always believed that Labour and the more responsible wing of the Conservative party would hold it in check. Not any more.

The Guardian's John Harris in his video reports from the north of England has been tracing how the EU referendum has turned into a revolt of the English working classes against immigration. It is a rebellion of the new precariat – people without secure jobs who see their public services crumbling, their living standards eroded, their homes taken away. This has coincided with a decade of unprecedented immigration, and people have understandably seen a connection between the two. Mass immigration has changed their neighbourhoods and their political culture and white working-class people resent it.

Middle-class metropolitan liberals deplore this proletarian revolt, and have tried to ignore it, but it is a reality that will not go away – especially now that prominent Labour MPs such as Tom Watson are endorsing the politics of immigration control. Race is now central to political debate in England, which is why Nigel Farage was emboldened to exploit it in an advertising campaign. A departure from the European Union will, I believe, intensify this animosity at the same time as aggravating the circumstances that created it.

The Big Lie of the Brexit campaign is that the queues of immigrants will stop if Britain leaves the EU and “takes back control of our borders”. They won't. Half the migrants come from elsewhere in the world and the economic factors that have drawn migrants from Europe will continue to attract them here. If business needs them, they will come. And when France stops trying to hold up illegal migrants at Sangatte – saying, in effect, “It’s not our problem any more” – border controls will move from Calais to Dover, greatly escalating the visibility of immigration. This will be nasty.

Remaining in Europe will not end racial antagonism of course. This toxic passion is here to stay. But it will keep Britain connected to a multi-national and multi-ethnic Europe, where free movement and migration and breaking down borders is still seen as something positive. I fear that if we vote to leave, Britain would end up as the racially-divided, neo-liberal Hong Kong of Europe. It would perhaps be a land capable of economic dynamism of a sort, but it wouldn't be a place I would want to live. A kind of little England super-colony in which Scotland and the Scots would be regarded with suspicion as a nation with divided loyalties to Britain. It might hasten Scottish independence, but at what cost?

Nigel Farage's “Breaking Point” poster was certainly the breaking point for me. It was a kind of homage to Margaret Thatcher's “Britain Isn't Working” of 1979, which showed lines of unemployed. But this was black and brown faces, and the message was: they're coming to get you. We've never seen anything like this before. This is no time for sitting on the fence. As the US economist Paul Krugman wrote: vote Remain and despair.