The threat came at an EU summit in Brussels where he acknowledged that "enlargement" had been one of the union's greatest strengths and a spur for peace and prosperity.
Days after announcing tighter UK controls on migrants' financial benefits ahead of an expected wave of Romanians and Bulgarians coming to Britain from January 1, the Prime Minister was looking for support from other EU leaders to rewrite the rules before any more membership hopefuls are given the green light.
The EU accession process starts with Serbia next month, and plans are under way to grant Albania "candidate status" as soon as June.
Mr Cameron said he had opposed immediate status for Albania, and wanted to see the start of a serious debate on a rules shake-up - including consideration of the Prime Minister's own idea of only granting full cross-border work and residency rights to citizens of newly added member states once their national prosperity has reached a certain level.
Mr Cameron said: "We must get back to the principle of granting free movement to workers ready to work hard, not those who are after the best benefit deal."
"It is not supposed to be about free movement for benefit tourism for those who don't have the means to support themselves: it is about people in work looking for work elsewhere."
The Prime Minister said one possible answer he was putting forward was to restrict access depending on a new member state's average GDP or domestic wage rates.