The Department for Work and Pensions said the ruling was disappointing but added emergency regulations were being drawn up to ensure jobless people could continue to take part in workplace schemes.
It is also considering an appeal to the UK Supreme Court.
However, if it were to fail, then the department could face a bill running into millions of pounds to reimburse an estimated 130,000 people who have carried out work for no pay.
The Government made clear it had no intention of making retrospective benefit payments.
Cait Reilly, 24, from Birmingham, and Jamieson Wilson, 40, an unemployed HGV driver from Nottingham, succeeded in claiming the Government's unpaid schemes were legally flawed.
Their solicitors, Public Interest Lawyers, said it meant "all those people who have been sanctioned by having their Jobseeker's allowance withdrawn for non-compliance with the back-to-work schemes affected will be entitled to reclaim their benefits".
They added: "The Government has unlawfully required tens of thousands of unemployed people to work without pay and unlawfully stripped thousands more of their subsistence benefits."
Employment Minister Mark Hobson expressed surprise at the Court of Appeal's ruling but said regulations would be introduced to avoid any uncertainty.
He said the court had "backed our right to require people to take part in programmes which will help get them into work".
In November 2011, Ms Reilly had to leave her voluntary work at a museum and work unpaid for two weeks at a Poundland store in Birmingham, stacking shelves and cleaning floors, under a scheme known as the "sector-based work academy".
She was told she would lose her jobseeker's allowance if she did not do the placement.
Mr Wilson, a qualified mechanic, was told he had to work unpaid cleaning furniture for 30 hours a week under a scheme known as the Community Action Programme.
He objected to doing unpaid work that was unrelated to his qualifications and would not help him re-enter the jobs market and refused to participate. He was stripped of Jobseeker's allowance for six months.
The court found that although the Work Programme did not breach human rights law, the regulations around the scheme did not properly describe what was involved.
After the ruling Ms Reilly said the two weeks of unpaid work at Poundland were a "complete waste of my time as the experience did not help me get a job".
She added: "I don't think I am above working in shops like Poundland. I now work part-time in a supermarket. It is just I expect to get paid for working."
Frances O'Grady, the TUC general secretary, said: "This blows a big hole through the workfare policies."
She added: "Voluntary work experience can help the jobless and it is right to expect the unemployed to seek work but it is pointless to force people to work for no pay in jobs that do nothing to help them while putting others at risk of unemployment."
Gerard Eadie, chairman of Scots double glazing firm CR Smith, said: "If we are to give people, and particularly young people, a meaningful work experience, it should be through a proper paid job.
"The sense of reward that comes, in part, from being paid for your efforts is fundamentally important to anyone's motivation to strive to do more."
Mr Eadie said a growing number of firms were participating in a scheme he had set up called Hand Picked, in which young people are given three months' paid work "and a chance to prove themselves".