Paul Gray, who is currently the Scottish Government's director general for governance and communities, will take over the position at the start of December.
He is filling the shoes of Derek Feeley, who left in July after a relatively short and rocky period running the health service.
Mr Gray has previous experience of the health field, having worked as director of primary and community care between 2005 and 2008.
Mr Gray said: "It is an outstanding privilege to be appointed to serve as director general for health and social care, and as chief executive NHS Scotland.
"It is a crucial time as we drive forward greater health and social care integration, and improvements to the quality of care, while maintaining high standards of performance and delivery.
"I'm also looking forward to working with a range of people across the public sector and beyond in tackling Scotland's public health challenges, with a particular focus on tackling health inequalities."
Health Secretary Alex Neil said he was delighted Mr Gray had been appointed. He added: "We have seen impressive improvement across NHS Scotland in a number of areas, and benefit from a world-leading patient safety programme - but there is still more to do."
John Connaghan has been working as acting chief executive for NHS Scotland since Mr Feeley left, and will resume his role as director of health workforce and performance from December 2.