The group, which is seeing its worst sales performance in four decades, announced Mr Clarke's departure as it issued a fresh warning on profits.
He will step down from the board on October 1 to be replaced by Unilever executive Dave Lewis, though he will continue in a support role until the end of January.
Tesco chairman Sir Richard Broadbent said: "Having guided Tesco through a substantial re-positioning in challenging markets, Philip Clarke agreed with the board that this is the appropriate moment to hand over to a new leader with fresh perspectives and a new profile."
Mr Clarke said: "Having taken the business through the huge challenges of the last few years, I think this is the right moment to hand over responsibility and I am delighted that Dave Lewis has agreed to join us.
"Dave has worked with Tesco directly or indirectly over many years and is well-known within the business. I will do everything in my power to support him in taking the company forward through the next stage of its journey."
Mr Clarke, who had worked his way up from the shop floor to lead Tesco, admitted last month that the store's quarterly like-for-like sales, which posted a 3.7% drop, were the worst he had known in 40 years.
But today's statement said conditions had become even worse than had been thought at that time.
"The overall market is weaker and, combined with increasing investments we are making to improve the customer offer and to build long-term loyalty, this means that sales and trading profit in the first half of the year are somewhat below expectations."
Tesco has been battling with intense competition from discount rivals amid a continuing upheaval in customers' shopping habits and a squeeze on household budgets.
It said a better picture for the full year would be reliant on seeing improvements as a result of its investments and in an upturn in the market, as well as "any steps that may be taken during the remainder of the year to improve our customer offer further".
The group said: "Philip has done a huge amount to set a clear direction and re-position Tesco to meet the rapid changes taking place in the retail market.
"He has achieved a great deal across all areas of the business in the face of considerable pressures.
"The board are deeply grateful to Philip for his contribution to Tesco, over the last four decades, as well as more recently as chief executive. His has been an outstanding achievement.
"Dave Lewis brings a wealth of international consumer experience and expertise in change management, business strategy, brand management and customer development.
"He is already known to many people inside Tesco, having worked with the business over many years in his roles at Unilever.
"The board believes that, with Dave's leadership, Tesco will sustain and improve its leading position in the retail market."