In the most dramatic reshuffle of his premiership, Mr Cameron also promoted Defence Secretary Philip Hammond to replace William Hague as Foreign Secretary after the surprise announcement last night that the former Tory leader was moving to become Leader of the Commons before quitting as an MP next year.
Treasury minister Nicky Morgan, 41, who replaces Mr Gove as Education Secretary, is expected to be one of a number of women and younger MPs being promoted by Mr Cameron in an attempt to counter perceptions that his Cabinet is too "male, pale and stale".
Liz Truss becomes the youngest member of Cabinet at 38 after being appointed Environment Secretary to replace Owen Paterson, who ran into trouble over the failed badger cull and his handling of the winter floods, as well as antagonising green groups with his scepticism about man-made climate change.
Energy minister Michael Fallon was promoted to join the Cabinet as Defence Secretary, replacing Mr Hammond.
Mr Gove's move to Chief Whip will be seen in Westminster as a demotion from a job in which he has shown a personal passion for free schools and stringent academic standards but has met fierce opposition from teaching unions.
Gove, who was born in Edinburgh and raised in Aberdeen,is a former journalist who has worked for the Press and Journal and The Times.
Announcing his move, Mr Cameron said: "Michael Gove is Commons Chief Whip. He'll have an enhanced role in campaigning and doing broadcast media interviews."
Mr Gove's removal was welcomed by teaching unions - part of the educational establishment which he reportedly derided in private as "the blob".
General secretary of the Association of Teachers & Lecturers (ATL) Mary Bousted said: "David Cameron has belatedly realised that Michael Gove's ideological drive is no substitute for measured, pragmatic reform of the education system.Time after time he has chased newspaper headlines rather than engage with teachers.
"The dismantling of the structures which support schools, the antagonism which he displayed to the teaching profession and the increasing evidence of chaos in the bodies he established, in particular the Education Funding Agency, has led Cameron to one conclusion - Gove is more of a liability than an asset.
"Successful education systems value the views of the teaching profession, which Gove insulted when he called them 'the blob'. ATL looks forward to a more constructive relationship with his successor, Nicky Morgan."
Labour described the reshuffle as a "massacre of the moderates" which marked a shift to the right by Mr Cameron less than a year ahead of the 2015 general election.
Among casualties of the reshuffle were David Jones, who was removed as Welsh Secretary and is expected to be replaced by his deputy Stephen Crabb, while Mr Cameron is expected to confirm that Environment Secretary Owen Paterson - whose allies mounted a desperate rearguard effort to save him - and Attorney General Dominic Grieve have been sacked.
Downing Street said the Prime Minister also accepted the resignations of universities minister David Willetts and energy and climate change minister Greg Barker, who will both stand down as MPs next year.
Andrew Robathan quit as a minister in the Northern Ireland Office and Alan Duncan left his post at international development, while news of Hugh Robertson's resignation from the Foreign Office filtered through while he was on an overseas trip in Beirut.
Nick Hurd said he was leaving his post as minister for civil society, while reports suggested policing minister Damian Green, rail minister Stephen Hammond and solicitor general Oliver Heald were all being sacked.
Mr Cameron also needs to find a senior figure to be the UK's next European commissioner, but Mr Hague indicated that he wanted to concentrate on his writing career.
"After the general election I will return to my writing, while still giving very active support to the Conservative Party and campaigning on international causes I believe in," he said.
"I wish to thank my constituents in Richmond, Yorkshire, one of the greatest places on earth, for their emphatic support through thick and thin over such a long period. I will serve them with unabated energy between now and the general election, and I look forward to supporting my eventual successor."
The Prime Minister said: "William Hague has been one of the leading lights of the Conservative Party for a generation, leading the party and serving in two Cabinets.
"Not only has he been a first-class Foreign Secretary, he has also been a close confidant, a wise counsellor and a great friend. He will remain as First Secretary of State and my de facto political deputy in the run-up to the election - and it is great to know that he will be a core part of the team working to ensure an outright Conservative victory at the next election."
Departing Kenneth Clarke, 74, issued a parting shot at eurosceptics within the Tory party, insisting that no British government would want to quit the EU and saying that those pressing it to do so were "crackers".
Mr Clarke said he regretted the departure of Mr Grieve, which has sparked speculation that Mr Cameron may be preparing the way for changes to Britain's membership of the European Convention of Human Rights.
"I personally think it's unthinkable we should leave the European Convention on Human Rights . It was drafted by British lawyers after the Second World War in order to protect the values which we fought the war for. It is the way in which we uphold the kind of values we strive for," Mr Clarke told BBC Radio 4's Today programme.
"Any government of this country is going to want Britain to stay in the European Union. It is the best way of effecting global political events and boosting our economy and the best way of protecting the interests of our citizens in a globalised world."
Mr Cameron paid tribute to veteran Mr Clarke, who first entered government in 1972, served as health secretary, education secretary, home secretary, chancellor of the Exchequer and lord chancellor, and spent the last few years as Minister without Portfolio in a front-bench career lasting more than four decades.
"Ken Clarke has been a political Titan for more than a generation," said the Prime Minister. "His wise and trenchant views will be missed around the Cabinet table."
Labour highlighted Mr Hammond's Euroscepticism.
Shadow Cabinet Office minister Michael Dugher said: "This speaks volumes about David Cameron's leadership.
"Four years of failure to promote women and now we have the massacre of the moderates.
"Britain's foreign policy is now set to be led by a man who has talked about taking us out of the EU. The Tories are now retreating out of Europe with all the threat that poses to jobs and business in Britain.
"This reshuffle shows how weak David Cameron is, running scared of his own right wing. That's why he cannot focus on the big challenges facing families up and down the country."
Mr Cameron added another female face around the Cabinet table in the shape of former TV presenter Esther McVey, who keeps her job as minister for employment and disabilities, but will now attend Cabinet. Ms McVey is not a full member of Cabinet.
Ms McVey smiled as she left Downing Street after being told she will begin attending Cabinet.
Cities minister Greg Clark confirmed he is to become universities minister and will also begin attending Cabinet.
Greg Clark will take the universities and science ministerial role vacated by Mr Willetts and will also continue to be a Cabinet Office minister.
Mr Cameron said he will attend all cabinets.
Stephen Crabb said he is "very pleased" and "looking forward to working hard" after being appointed Secretary of State for Wales.
Baroness Stowell has been appointed Leader of the House of Lords and Chancellor of the Duchy of Lancaster - replacing Lord Hill of Oureford whom Mr Cameron has picked as his candidate for the UK's place on the European Commission.
Matt Hancock, a key ally of Chancellor George Osborne, takes the role of Minister of State for Business, Enterprise and Energy vacated by Mr Fallon, as well as being Minister for Portsmouth, and will attend Cabinet.