Mr King has been credited with reviving the fortunes of the supermarket and more recently helping it fight off growing pressure on the sector from discount retailers.
Chairman David Tyler said: "Justin is a truly exceptional leader who has reshaped Sainsbury's during his 10 years as CEO, as well as playing a leading role in the sector and wider business world.
"He leaves a lasting legacy with the company stronger than ever."
Mr Tyler said Mr King's successor was a man of "unique talent and experience".
"He has worked hand-in-hand with Justin over the past decade and has a proven track record of success, making him the natural choice to take the company forward."
Mr King, who will stand down at the annual general meeting on July 9, said: "This was not an easy decision for me to make, and in truth it will never feel like the right time to leave a company like Sainsbury's.
"It has been a privilege to have led the company for the past 10 years and I am incredibly proud of our achievements in that time. I am confident that under Mike's leadership the business will go from strength to strength."
Mr Coupe said: "It's an absolute honour to be appointed as the new CEO of Sainsbury's in this, the company's 145th year, at a time when thanks to Justin's leadership, we have been consistently outperforming the market."