The official figures suggest a lack of progress and a decline in the opportunities they have and their chances of getting a job.
Hundreds fewer are getting jobs, or taking part in training or purposeful activities such as attending day centres.
The annual eSAY data is commissioned by the Scottish Government from councils to demonstrate progress made under The Same as You, a strategy aimed at improving the lives of adults with learning disabilities.
But the new statistics found a 16% fall in the number of people with learning disabilities known to be in employment or training. Only 3393 adults were known to be in a job or training for a job, compared with 4046 in 2011.
Jobs specifically for people with disabilities, so called "non-open" employment, also saw a decline with 318 fewer people in such roles than there were in 2011.
There was also a drop in the number in further education, with 289 fewer adults enrolled in 2012 than in 2011, a 10.7% fall.
Meanwhile, other activities for adults with learning disabilities are in decline. A drop from 6164 people attending a day centre in 2012 to 5483 in 2011 represents a 12.4% decrease. Alternatives to day care slumped by 13.8%.
The figures also found a slight increase in the number of adults known to local authorities, and showed a large proportion of people with learning disabilities are living independently in their own accommodation.
However the new data, compiled for the Government by the Scottish Consortium on Learning Disability (SCLD), has prompted questions about what is really on offer for people with learning disabilities and the impact of public spending cuts.
The figures were described as concerning and disappointing by SCLD.
Chris Creegan, chief executive, said the research added greater urgency to implementation of the Scottish Government's new Keys To Life strategy.
He said: "People with learning disabilities should expect to have access to a range of opportunities. The drops in figures for adults with learning disabilities gaining access to further education and employment opportunities are particularly worrying.
"And while the figures around access to day centres and other alternative opportunities have remained relatively stable across the past five years, we need to find out more about what people are doing with their days and ensure they are able to play an active part in their communities."
A spokeswoman for SCLD added: "There has been a lot of effort to try to improve levels of employment for people with learning disabilities, so these figures are a concern.
"If fewer people with learning disabilities are in work, or education, or attending day centres or alternatives, you wonder what people are doing with their days."
Peter Scott, chairman of SCLD, added: "The trend in these statistics towards more independent living is welcome, but there's also an underlying concern that whether in relation to education, employment or day services, opportunities for people may be diminishing."
A Scottish Government spokesman said findings from the survey would help determine how the Keys To Life strategy is taken forward, adding: "The Scottish Government is committed to ensuring students with a learning disability receive an improved educational experience, which is tailored to their needs and ambitions.
"The Scottish Funding Council is working closely with groups that help those with a learning disability to ensure colleges are delivering for such students.
"We have funded the employment of dedicated support officers and development of a new Good Practice Guide using an additional £250,000 which will assist 200 students to progress from learning to employment over the next 12 months."