Councils have raised the amount they charge people who need care services by more than £11 million in the last three years, the research by the Learning Disability Alliance Scotland (LDAS) suggested.
While local authorities have not changed the criteria by which they assess people needing care, the report claims frontline social workers must be reducing the number of people given services, in order to meet budgets.
LDAS looked at the proportion of Scots who needed help such as care at home and residential care in 2007 and calculated how many people would be receiving care today, assuming needs were the same now as then.
It found that despite a growing elderly population and rising demand for care from people with disabilities, the numbers had fallen in real terms.
About 23,000 people who would have been expected to need care at home or residential care are not receiving it, the report says. Even if about 2500 of these are paying for care using personalised budgets, this still leaves 20,000 who do not receive a service.
LDAS, which represents a number of learning disability charities, said many vulnerable people were no longer getting the help they needed because of tougher eligibility criteria and care charging.
The report says government figures show charges made to social work clients for 2010 totalled £40.3 million but had risen to £51.6m by last year, a 21.9% increase.
Ian Hood, coordinator of LDAS, described increased charges for social care support as a "care tax". He said: "Over the last three years, councils across Scotland have made up real shortfalls in their incomes by rapidly increasing the amount they receive in 'care tax' from vulnerable adults. This increasing burden on people who get support from councils leads to more people exiting the system when they are told what their client contribution will be."
The report criticised a lack of transparency and over-complication in the system for deciding who qualifies for services.
Michael McMahon MSP, chairman of Holyrood's cross-party group on disability, said: "We have been worried about the changes in eligibility criteria for a while. This report helps to show that there are changes taking place that reshape social care but unfortunately most of it has taken place below the radar."
Jim Elder Woodward, convener of Independent Living In Scotland, said many people who previously received services were looking for informal care from family and friends or not receiving help at all, because they could not afford charges.
He said those with disabilities faced a policy double-whammy: "There is a huge disconnect here because the people who are most able to find work are those who do not qualify for help - because they are not in the greatest need. But without some help, they can't go out to look for work."
A spokesman for Cosla said: "We recognise there are challenges in respect of managing access to care. However councils are doing a very good job in demanding circumstances, as we face rising demand and changes in the structure of our population, while having to operate social work services in the context of diminishing resource. We want to work with all communities and stakeholders as to how these difficult decisions should be made."