ED Miliband has achieved an insight.

He has realised that one clue to the hopes and fears of the working class is in the designation: work, getting by, providing, gaining a fair crack of the whip, and being granted decent treatment in exchange for honest labour. It's uncomplicated, but fundamental. The politician who throws any of that in doubt earns a world of trouble.

Tories with a superstitious faith in the market pay lip service to the contract, then try to ignore it. When Iain Duncan Smith was attempting to "redefine" poverty recently he forget to mention that half of poor families contain a working adult. When his colleagues meanwhile talk of making work pay, they somehow neglect to say where the paying work might be.

At least the Tories don't blame excessive wage demands for Britain's slide back into recession. That old favourite would be one joke too many. The working class – if in work – has more than done its bit these past three years with wage cuts, pay freezes, reduced hours, and salary increases far below the rate of inflation. If nothing else, the idea that people don't need or want employment has been killed stone dead.

For millions, life remains precarious. Financially, a great many are close to the edge, without even an offshore tax-avoidance strategy for comfort. They feel powerless, taken for granted – mostly because they are taken for granted – and fearful. Immigration bothers them.

Xenophobes always hoped it would. Nothing cheers a demagogue more than a fear of incomers. But in hard times immigration ceases to be a test of tolerance alone. It becomes an issue of fair pay, housing, resources in education and health, and of social justice. The problems that could be handled, more or less, during the long Blair-Brown boom have become harder to manage. Mr Miliband has chosen to recognise as much.

On one level, it's a tactical choice. Where immigration is concerned Labour has long been outflanked, in the parlance, by the Tory appeal to prejudice. Gordon Brown's inept promise to secure "British jobs for British workers" – impossible within the EU, as he must have known – was a symptom of confusion. It added to the impression that the Brownites and Blairites were entirely out of touch with the core Labour vote.

Too often, that electorate has been patronised. To complain about increased competition for scarce housing does not make a person a racist. To worry over crowded schools and doctors' surgeries is not the mark of a bigot. Above all, when immigration becomes an excuse to drive down wages in what Mr Miliband calls a "nasty, brutish and short-term" labour market, mere discrimination, with us in good times and bad, is not the problem.

The Labour leader wants to stop employment agencies from favouring those famously reliable foreign workers. That's fair enough: the practice is probably illegal, in any case. Mr Miliband also wants to double fines on employers who cheat on the minimum wage. That's certainly fair: the law allows no exemptions.

Labour's leader admits meanwhile that it was probably a mistake to allow instant entry to Britain for migrants from EU accession countries when other European states were raising obstacles. Here the ground becomes shaky. All EU citizens gain the right to free movement sooner or later. Merely postponing the entry of people from, say, Croatia, solves no long-term problems.

Equally, Mr Miliband's promise to "review" immigrants' access to benefits and council house waiting lists is liable to run foul of EU law. He admits himself that, proportionately, migrants claim benefits less often than British-born workers. Overwhelmingly, they come seeking employment, and they pay their taxes. If they enter the country legally, which part of their rights as EU citizens would be up for review?

Mr Miliband has a point, nevertheless, when he says that "It is the short-term, fast-buck culture that is at the root of this..." He argues, rightly, that "you cannot address people's concerns about immigration unless you change the way the economy works". Then he fails to tell us what he has in mind.

Clearly, he isn't too worried about professionals and other skilled workers coming to Britain from elsewhere in Europe. The benefits they bring are well-attested. Nor is Labour's leader in the business of doubting the migrant work ethic: that phenomenon has survived down the generations. Instead, he invokes a concept his party has all but forgotten: the "relevance of class". Immigration matters most when people are most vulnerable.

Ours is fast becoming a low-wage economy shorn of job security. This is a world in which Tory ministers promote the right to sack at will, in which recession and unemployment have left people defenceless, and in which the exploitation of migrants has helped to entrap the least well-off. Put crudely, immigrant labour is too often being used to shackle the unskilled working class.

Why won't "our people" work for less than the minimum wage? Why can't they be hired and fired at will? Why can't they be driven off the dole and into the arms of a gangmaster who mocks the idea of employment rights? Mr Miliband's analysis of what has been going on behind the arguments over intolerance is precise. Yet he fails to say what should be done to prevent any workers from being treated as migrants are treated, and set in desperate competition, one against the other.

Labour's leader has another problem: he could soon find himself out of date. If the eurozone implodes, as some hope and others fear, Europe will swarm with migrant workers. Those from the poorest countries will head, rationally enough, to the places they deem most prosperous. Desperate – and how many Greeks are already desperate? – they will take whatever work is going, no questions asked. Such will be their right, as citizens of the EU.

Mr Miliband could be risking a paradox. Let's say he restores decency to Britain's labour market. Let's say he enforces the minimum wage and employment rights for all. That could make even Britain seem attractive to those fleeing the aftermath of the euro's collapse. So how many penurious folk from southern Europe could this country hope to shelter when its own workers are caught in the vice of free-market failure?

Like it or not, economic incompetence and the crimes of the banking classes have altered the terms of the immigration debate. Our native bigots will exploit the fact soon enough. Ruthless employers, intent on rendering the concessions of the past three years permanent, will be quick to demand still more. For some, there is a profit in every catastrophe.

Mr Miliband has described a situation that is already dire. He has failed to wonder how much worse it could become. Some in Europe have already begun to speculate over changes to the Schengen policing agreement and the possibility that border controls might be reintroduced. Those won't halt a human tide. They will put an end, though, to the principle that free people must have the right to free movement.

Mr Miliband says that it is time his party began to talk about immigration and class. He's right about that: Britain is institutionalising its inequalities. The Labour leader forgets that there might be a lot more talking still to come.