An Alzheimer's Society report states 80% of people in residential care homes have either dementia or severe memory problems, a rise from previous estimates of 62%.
However, the charity said that while excellent care exists, less than 50% of the 322,000 sufferers in care homes across the UK have a good quality of life. It said there are such low expectations of care homes that people are "settling for average".
A spokesman said 41% of more than 1000 relatives and carers surveyed by the charity believe their loved ones enjoyed a good quality of life, but 28% said it was poor.
A separate poll of 2000 UK adults found two-thirds did not feel the care sector was doing enough to tackle abuse in care homes, and many admitted they would be "scared" about moving into a care home in later life.
The charity called on ministers and the care sector to work together to strengthen existing minimum standards to boost quality of life for sufferers.
It said they should also improve public understanding about the quality of care sufferers can be offered.
Jeremy Hughes, chief executive of the Alzheimer's Society, said: "There is a forgotten scandal of people with dementia who are failed and left living a life that can only be described as 'OK'."