At their peak there were around 29 deaths per 100,000 men from the disease, but this figure has fallen to around 24 deaths per 100,000, according to a report issued by Cancer Research UK.
The downward trend is largely as a result of new approaches to treating prostate cancer such as earlier, more widespread, use of hormone therapy, radical surgery and radiotherapy.
The earlier diagnosis of some cancers linked to the use of the PSA (prostate specific antigen) blood tests in GP surgeries is also thought to be behind the fall.
However, Professor Hing Leung, prostate cancer expert at the Beatson Institute in Glasgow, said more research was required to find out why some people cannot be cured.
Mr Leung said: "This new report shows we've come a long way in improving the treatment of prostate cancer in the last couple of decades.
"Improvements in how we treat prostate cancer have been key to reducing deaths of men diagnosed with the disease. But a lot more work still needs to be done.
"We still don't understand why some prostate cancers turn out be harmless while others are aggressive and resistant to treatment. Better tests and improved treatment are necessary for us to build on what we have achieved so far."
Each year around 2700 men in Scotland are diagnosed with prostate cancer. There were 900 deaths from the disease in 2011, the latest official figures show.
It emerged last month that Billy Connolly had undergone surgery for early-stage prostate cancer. Connolly, who had also been diagnosed with the initial symptoms of Parkinson's disease, said later that he had been given the all-clear from the cancer.
Retired GP Rob Lester, who was diagnosed with advanced prostate cancer 18 months ago, welcomed the news that death rates from the disease have fallen.
The 57-year-old, of Dalgety Bay, is taking part in the Cancer Research UK Stampede Trial which aims to find better treatments for men with advanced prostate cancer.
Dr Lester, who is receiving treatment at the Western General Hospital in Edinburgh, said: "It is really good news that research is already producing better treatments and the outlook is much more promising with the new drugs that are becoming available.
"These advances mean a great deal to men who are living with a prostate cancer diagnosis."
Cancer Research UK's chief clinician Professor Peter Johnson said there was still a need for further research to find a "better screening test, better diagnostics, better treatments".
Martin Ledwick, Cancer Research UK's head information nurse, said: "The symptoms are similar to a number of benign and harmless conditions but it's worth being aware and getting anything unusual checked out with your GP. Things such as having to rush to the toilet to pass urine and difficulty urinating should be checked out, especially if it's getting you up several times during the night.
"It's also worth remembering it is more common in men over 60, anyone who has had a relative diagnosed with it and men of African-Caribbean descent."