It was only the monarch's third visit to Chequers and was arranged by Mr Cameron - who has weekly audiences at Buckingham Palace - so that he could "host her for a change", Number 10 said.
A menu of Scottish smoked salmon followed by lamb and then bread and butter pudding with ice cream was laid on for the royal couple and other invited guests.
As well as Samantha Cameron, those around the table included former cabinet minister Lord Carrington - who was defence secretary when the Queen first visited the grace-and-favour mansion. That was in 1970 when she joined US president Richard Nixon for lunch at the invitation of then premier Edward Heath in what was seen as a special salute to the US first family.
It remains a very rare event at the Chilterns house gifted to the nation for use by prime ministers in 1921, with a 1996 visit as the guest of John Major believed by the Palace to be the only other.
Other guests were ex-head of MI5 Baroness Manningham-Buller, Lord Guthrie, the former head of the armed forces and writer William Shawcross, the Queen Mother's official biographer and his wife, the designer and hotelier Olga Polizzi.
Number 10 said Mr Cameron had been inspired to make the invitation after coming across archive photographs of the previous royal occasions. The No 10 spokeswoman said: "The Prime Minister and the Queen see each other regularly. The Queen hosts the Prime Minister at the Palace once a week.
"The Prime Minister thought it would be a good opportunity to invite the Queen to have lunch, to host her for a change."