The coalition agreement of 2010 included a promise of early legislation to introduce a "power of recall", allowing voters to force their MP to face a by-election by raising a petition of 10% of constituents.
If no bill is included there is virtually no chance of it becoming law before next year's general election.
But Tory backbencher Zac Goldsmith, who has campaigned for recall, said: "Clegg told me he couldn't back a proper Recall Bill (despite his manifesto promise) because MPs might actually be sacked. Now in typical LibDem fashion, Clegg is briefing that the Tories have ditched recall..."
He added: "Even by the s****y standards of dishonest UK politics, the LibDems really are revolting. ."
LibDem president Tim Farron said in a radio interview: "We understand that the Prime Minister has blocked it.
"Nick Clegg wanted it in the Queen's Speech, which would have meant that there was a bill there that we could have discussed."
Shadow justice secretary Sadiq Khan said: "This is another broken promise from David Cameron. He's now backtracking from his own manifesto commitment to introduce recall of MPs."
Downing Street refused to comment on "speculation" about the Queen's Speech.