The Prime Minister told MPs: "We have had the same Education Secretary (for four years) and we have 250,000 fewer children in failing schools. If you have a strong team with a strong plan, stick with them, and keep on putting it in the back of the net."
Now, however, the Scottish politician has been substituted and moved to be Chief Whip; the one who keeps Tory MPs in line. While an important role, it is, despite No 10's insistence, a demotion from running a big department of state.
Remarkably perhaps, given Mr Gove's clear passion for the education brief, Coalition sources suggested the Surrey MP had, in fact, been on the Downing Street sofa with Mr Cameron and George Osborne to draw up the Cabinet changes.
Last night, the Tory leader praised his friend as "one of my big-hitters, one of my real stars" and insisted his new parallel role as Minister for TV was "one of the most important jobs in government."
In Westminster terms, this sounded, faced with criticism about the switch, as protesting too much.
Conservative MP Sir Greg Knight tweeted: "If spin is true and we are to have a Chief Whip who regularly addresses the nation, this will prove to be a mistake; hope I'm wrong.
Certainly, Mr Gove's "iron fist in a velvet glove" approach alienated teaching unions
During his four-year stint at education, there were repeated clashes as he attempted to push through far-reaching reforms to England's schools system, from exams and the curriculum to teachers' pay and league tables. He also spearheaded the introduction of the policy on free schools.
Mr Gove famously described the unions and academics who opposed his plans as "the blob" and "enemies of promise".
There was also the spat with Home Secretary Theresa May over the alleged infiltration of schools by hardline Muslims, for which he publicly apologised.
It seems Mr Cameron adjudged his colleague had simply made too many enemies and, in the run-up to an election, it is best to try to keep these to a minimum.