Even a moderate deficiency was found to increase the chances of developing the disease by 69 per cent.
Scientists working for the Cardiovascular Health Study, a US investigation of heart disease and stroke risk factors, were surprised by the dramatic findings, which suggest upping vitamin D intake might delay or prevent Alzheimer's.
But more research has to be done before experts can know if there is a causal link between levels of the vitamin and dementia risk.
The study looked at 1,658 American adults aged 65 and over who were free from dementia, heart disease and stroke at the outset.
Lead researcher Dr David Llewellyn, from Exeter University, said: "We expected to find an association between low Vitamin D levels and the risk of dementia and Alzheimer's disease, but the results were surprising - we actually found that the association was twice as strong as we anticipated.
"Clinical trials are now needed to establish whether eating foods such as oily fish or taking vitamin D supplements can delay or even prevent the onset of Alzheimer's disease and dementia.
"We need to be cautious, our latest results do not demonstrate that low vitamin D levels cause dementia. That said, our findings are very encouraging, and even if a small number of people benefit, this would have enormous public health implications."
The research showed that over six years, study participants moderately deficient in vitamin D had a 53% increased risk of developing any kind of dementia. For the severely deficient, the risk increased to 125%.