Sir Harry Burns, who is quitting to join a university institute, said worldwide research shows a lack of physical activity is a major factor for early death.
Doctors across the country are being urged to help patients increase their fitness levels in a bid to improve overall health.
And Dr Burns said GPs and other medical staff should be telling patients what exercises are best and what foods to avoid. He said however doctors cannot force individuals to up their exercise and people to choose to do it themselves.
A meeting of experts will be held this week at Edinburgh's Royal College of Physicians to discuss exercise and sports medicine.
Dr Burns said: "For people who find it hard to engage in physical activity, we need to be finding out what the impediments are and helping them to deal with that.
"We need to help them get to a place in their minds where they actively choose to be physically active."
One way to do this would be for doctors to record physical activity and fitness levels, Dr Burns said, similar to the way in which smoking levels are recorded during health checks.
He added the combined risk of death from smoking, obesity and diabetes was similar to being physically unfit.
Dr Burns is to become professor of Global Public Health in Strathclyde University's Institute for Global Public Health at the International Prevention Research Institute.