How clean are your clothes?

Let's not allow the dust to settle on Rana Plaza without doing something for the garment workers of Bangladesh. After this eight-storey death trap in Savar near Dhaka collapsed last week, entombing hundreds of poor seamstresses, an angry crowd gathered briefly outside Primark on Oxford Street. But as the desperate cries for help from the collapsed building faded to an ominous silence and the bodies mutilated beyond recognition were piled into a mass grave, Britain's high streets returned to business as usual.

Instead fury focused on the building's owner, a local politician who used his influence to flout building regulations and on the Bangladeshi government, which freely admits that 90% of Dhaka's multi-storeys would fail even local construction standards.

Responsibility must go wider. The retailers who sell the dirt cheap garments made in Asian sweatshops have learned to speak the lingo of ethical consumerism following the campaigns of the 1990s. Most shelter under the umbrella of the Ethical Trading Initiative, which has a charter offering vague commitments on minimum pay and conditions that have done little to either protect or enrich the 3,600,000 people (mostly women) who work in the Bangladeshi garment industry. The government-approved minimum wage is currently about £28 a month.

Picking on Primark is arbitrary. Rana Plaza workers produced some of its garments but this tragedy could have happened anywhere. It's the third deadly accident in six months in the country that relies on cheap clothing for 80% of its exports. Two years ago in a report from the charity ActionAid on efforts to improve the pay and conditions of garment workers, Asda was bottom of the league.

The crux of the issue is that much of today's fashion is too fast and too cheap. Buyers pit one impoverished, poorly regulated country against another, so that contractors win orders by shaving pennies from their quotes, then make profits by cutting corners on safety and holding down wages. Rising commodity prices make the whole equation worse. Workers were under pressure to return to the Rana Plaza despite cracks appearing in the walls, partly because contractors risked fines from retailers if work was delivered late.

So our clothes come from women we will never see, slaving for pennies in conditions that were outlawed in Britain by the Victorian factory acts. Some retailers, including Nike and Gap, have made real improvements. M&S has not only led the way on fairtrade cotton but has pledged to pay a living wage to all its south Asian garment workers by next year. Yet their clothing sales are sluggish, while Primark's half-year profits are up 56%.

This isn't so much about poor families in Glasgow living off the backs of unimaginably poorer ones in Dhaka because paying the Asia Floor Wage, agreed by trade unions (and currently around $100 a month), would add only pennies to a garment. No, it's about feeding our fantasies with throwaway fashion. It's about the British chucking out a million tonnes of waste textiles a year, much of it unworn. It's about a kid's T-shirt so cheap that it's not worth washing the ketchup out.

Boycotting the Primarks of this world is not the answer. Without their breadline jobs, the women of Rana Plaza could be reduced to something worse: domestic service tantamount to slavery or the sex industry. Rather, we should pummel away at these companies until they start taking their moral responsibilities more seriously. Primark could start by signing up to the Bangladesh Fire and Safety Agreement, drawn up by labour rights groups following the deaths of 112 Bangladeshi workers in a fire last year. Political action is needed too. David Cameron should use his membership of the panel drawing up new UN global development goals to press for a common reporting network for businesses. The EU should threaten action on Bangladesh's duty-free and quota-free access to its markets unless it takes verifiable action on garment workers' pay and conditions. And the rest of us need to get a whole lot more curious about where our clothes come from. They may be dirtier than you think.