However the group says even members of the public are often afraid to raise concerns. Kathleen White, of Patients First, said: "I've seen patients' relatives who don't want to complain -because they know their relative might be back in (hospital) again."
The group was commenting after Professor June Andrews, of Stirling University Dementia Services Development Centre, claimed NHS staff and politicians were colluding to present an unrealistic picture of hospital care.
She said the public should acknowledge that hospitals are dangerous, and called for a debate about the implications of understaffing and underfunding on patient care, particularly for those with dementia.
Chairwoman, Fiona van der Meer, spokeswoman for Patients First, said the organisation agreed with the analysis of Ms Andrews and added: "There is still not enough protection for people who want to speak out about poor care, and the NHS Scotland staff survey showed that most staff do not feel confident to do so."
Kate Fearnley, deputy chief executive of Alzheimer Scotland, said improvements in hospital care would not happen overnight.
"While I wouldn't be complacent about standards of care in hospitals, and the Mental Welfare Commission report highlighted the problems in continuing care for some of the most vulnerable people with dementia, I think it would be a pity to label hospitals as 'unsafe places'," she said.
The Scottish Government is holding an international dementia conference in Edinburgh today, at which experts from around the world will hear about Scotland's approach to improving the quality of care for people with dementia.
It is being held partly in response to the G8 Dementia Summit in London in December last year.