But the UK Government's new Chief Whip, in charge of party discipline, who will attend Cabinet when necessary, played up his move from the Education portfolio, which he had shown a passion for in promoting the policy of free schools and stringent academic standards in England.
He declared he was "happy" with his new role, which, he insisted, would allow him to "shape the agenda".
Mr Gove, born in Edinburgh and raised in Aberdeen, denied it was a demotion, adding: "Demotion, emotion, promotion, locomotion, I don't know how you would describe this move, though move it is."
In contrast, the teaching unions in England and Wales, which had fiercely opposed his reforms and which Mr Gove had derided as being part of the educational establishment "blob", were delighted to see him go.
Christine Blower, from the National Union of Teachers, stressed that Mr Gove had lost the confidence of the profession and said: "I would imagine there are lots of faces wreathed in smiles in staff rooms today."
Meantime, Ofsted chief Sir Michael Wilshaw said he was "surprised and shocked" by Mr Gove's departure from Education, adding: "I'm a great admirer of the Secretary of State; he's been a transformative and radical minister of education."
Downing Street insisted the 47-year-old politician's new job was "absolutely not" a downward push while the Prime Minister also talked up his new role; already dubbed "Minister for TV".
He said his friend and colleague would "have an enhanced role in campaigning and doing broadcast media interviews".
Responding to the negative coverage, Mr Cameron added: "I wanted one of my big-hitters, one of my real stars, one of my great political brains - someone who has done extraordinary things for education - to do that job, to deliver the Government's programme"
The biggest surprise in the Cabinet shake-up was the appointment of Nicky Morgan as Mr Gove's replacement as Education Secretary.
It has taken the 41-year-old mother-of-two, who worked as a corporate lawyer before entering Parliament, just nine months as a minister to claim a top job.
She is one of a number of women and younger MPs being promoted by Mr Cameron to counter perceptions that his Cabinet is too "male, pale and stale".
Paisley-educated Liz Truss, 38, becomes the youngest member of the Cabinet after being appointed Environment Secretary to replace rightwinger Owen Paterson.
Downing Street emphasised how 30 per cent of Tory full Cabinet ministers were now women.
In the most dramatic reshuffle of his premiership, Mr Cameron promoted Defence Secretary Philip Hammond to replace William Hague as Foreign Secretary, after the surprise announcement on Monday night that the former Tory leader was to become Leader of the Commons before quitting as an MP next year.
Scots-born Energy Minister Michael Fallon was promoted to Cabinet as Defence Secretary.
Key among the departees was former Chancellor Kenneth Clarke, 74, who issued a parting shot at the Conservative eurosceptics, insisting no British government would want to quit the EU and claiming those pressing for an exit were "crackers".
Elsewhere, Labour branded the reshuffle as the "massacre of the moderates", which marked a lurch to the right by Mr Cameron less than a year ahead of the 2015 General Election.
Unison leader Dave Prentis dismissed the promotion of a number of women as "window dressing" while Len McCluskey of Unite said the reshuffle had confirmed that the Tories would be going into the next election with an "anti-worker, Thatcherite agenda".