In a day of high drama at Westminster, the Prime Minister sought to justify why, after six days of bad headlines and a draining away of backbench support for the Hampshire MP, he had not sacked her.
"If people clear themselves of a serious offence, you let them get on with their job; you let them try to do their job," he told MPs. "That is actually the right thing to do. Firing someone at the first sign of trouble; that is not actually leadership, that is weakness."
But Mr Cameron admitted public anger at MPs' expenses was still "raw" despite the scandal breaking five years ago.
He acknowledged that the furious public reaction to the disclosure Ms Miller had wrongly claimed thousands of pounds in mortgage interest payments showed further reform of the expenses system was essential.
Labour leader Ed Miliband focused on what he branded the PM's "terrible error of judgment" in failing to sack her immediately after the publication of last week's Commons Standards Committee report. He accused the Tory leader of being an "apologist for unacceptable behaviour".
The announcement by Downing Street of Ms Miller's resignation came just after 7am yesterday. It followed a late-night telephone conversation between the Secretary of State and the Prime Minister as he was returning to London after attending a state banquet at Windsor Castle in honour of Irish President Michael Higgins.
Earlier, an emotional Ms Miller insisted she took "full responsibility" for her decision to go, saying she had become a "distraction" from the work of government.
"This has been a really difficult 16 months. Because I was cleared of the central allegation made about me by a Labour MP I hoped I could stay, but it has become clear to me it has become an enormous distraction," she said.
Although Ms Miller was cleared by the Parliamentary Commissioner for Standards of using taxpayer-funded expenses to provide a home for her elderly parents, she was told she should repay £45,000 wrongly claimed towards mortgage payments.
The sum was reduced to £5800 by the Commons Standards Committee. Nonetheless, she was ordered to apologise to MPs for failing to co-operate with the commissioner's inquiry.
The brevity and tone of that apology further fuelled criticism of the Basingstoke MP, and a growing number of Tory MPs joined a chorus of disapproval.
No 10 announced Ms Miller's replacement would be Treasury Minister Sajid Javid, a former millionaire investment banker.
However, a row broke out over the issue of the women's minister. Ms Miller had overseen the women's and equalities brief, and the former went to current Treasury Minister Nicky Morgan, who voted against gay marriage.
Downing Street stressed Ms Morgan would be able to attend Cabinet. But it was later announced she would answer not to Mr Javid, who has overall responsibility for the women's and equalities portfolio, but to Mr Cameron himself.
Labour accused the PM of having a "blind spot" when it came to women and called the reshuffle a "complete mess".
Meanwhile, Ms Miller said she would donate her ministerial payoff, which will be about £17,000, to a local charity,