Citizens Advice Scotland (CAS) said rogue employers were taking advantage of many workers.
It said some employees were going for long periods with little or no pay due to being offered few hours of work or none at all.
Others were unable to budget or plan their lives due to unpredictable fluctuations in the amount of work on offer, with changes often imposed at short notice.
Other employers failed to tell staff that their job would be on a zero hours contract when they were taken on, CAS said.
Workers on such contracts often cannot claim benefits or tax credits, leaving them reliant on debt or food banks during fallow periods. In some cases, employers have used drastic cuts to an employees' hours to force them to resign, CAS claimed.
Publishing the report, CAS manager Keith Dryburgh said: "Zero hours contracts are meant to provide flexibility for employers and workers alike.
"They can be a useful option for some people - as long as the system is applied fairly. However, we see growing evidence that the system is in fact being abused by some employers, who are mis-using it to exploit their workers."
He said advisers in Citizens Advice Bureaux around Scotland were seeing the benefits of the system accruing mainly to the employer not the worker.
The majority of zero hours workers are aged under 25 or over 65 and the majority are women, working in areas like catering, tourism, food and care.
Mr Dryburgh added: "Too often workers are left with no hours, no pay, no security and no chance.
"We hope to persuade employers that they should do right by their staff, and we want to engage with ministers, unions and others to discuss ways to ensure a fair deal for all workers."
Workers on zero hours contracts already have rights and staff in local bureaux can help people to demand them, Mr Dryburgh added.
Recommendations in the CAS report include:
l The introduction of a right for workers to request guaranteed hours without fear of dismissal;
l Guidance for JobCentre Plus staff clarifying that workers on zero hours contracts should not be penalised in relation to benefit claims;
l That sanctions should not be applied to those on Universal Credit if they fail to apply for a zero hours job if it does not suit their needs.
In one case, a 19-year-old woman was interviewed for a job advertised as full-time by a clothes shop. When she accepted the job, she was told during training that it would be a zero hours post.
She was often not told of her hours until 10pm the night before, and was told she would be sacked if she did not turn up for shifts. She left and is now a student.
The woman, identified only as Marion by CAS, said: "Some weeks I would have no shifts at all, then the next week I would work every day or just odd days.
"I moved to another job three months later, and I know many other staff on the same contract also left as it was a nightmare with childcare and travel. A lot of people can't move to another job and are stuck in situations like that."
In April, in an interim report on zero hours contracts, the Scottish Affairs Select Committee said the Government should act to tighten up the rules. Committee chair Ian Davidson MP said: "The overwhelming majority of zero hours contracts are abusive and exploitative and should be abolished."