The Work and Pensions Secretary has tabled emergency regulations to deal with a Court of Appeal ruling that the rules of the work experience scheme were unclear.
And he issued a direct rebuke to university graduate Cait Reilly, 24, from Birmingham, who challenged having to work for free at a local Poundland discount store or face losing jobless benefits..
"You have to learn early that if you commit to something you stay by and do it," he told BBC1's Andrew Marr Show.
Denying claims of "slave labour", he said: "She was paid jobseeker's allowance by the taxpayer to do this.
"Most young people love this programme and I am sorry but there are a group of people out there who think they are too good for this kind of stuff."
Former Tesco boss Sir Terry Leahy started his career at the retailer stacking shelves, he pointed out.
"The next time these smart people who say there's something wrong with this go into their supermarket, ask themselves this simple question: when they can't find the food on the shelves, who is more important: them, the geologist or the person who's stacked the shelves."
Lord Justice Pill, Lady Justice Black and Sir Stanley Burnton unanimously agreed the 2011 regulations failed to give the unemployed enough information, especially about the sanctions for refusing jobs under the schemes.
Mr Duncan Smith dismissed the court ruling as "rubbish".
"I have already put emergency regulations down. That has ended it.
"I am not going to give way on this. I absolutely clearly tell you this: people who think it is their right to take benefit and do nothing for it - those days are over."