His comments came as Labour's Ian Davidson suggested a "Tory toff from the home counties" campaigning for Scotland to remain part of the UK would prove counter-productive for those fighting independence.
The Prime Minister made the surprise admission as he again ruled out facing Scottish First Minister Alex Salmond in a live television debate ahead of September's vote.
But Mr Davidson, the MP for Glasgow South West, said it was "genuinely absurd" the leader of the No campaign against independence, Labour former chancellor Alistair Darling, could not secure a debate against Mr Salmond as the SNP leader was still hoping to face the Prime Minister in a head-to-head debate on television instead.
Turning to Mr Cameron, the MP also teased the Prime Minister about his decision to award his hairdresser Lino Carbosiero an MBE in the New Year's honours list.
Mr Davidson said: "There is more. Without seeking to give offence to you, can I tell you that the last person Scots who support the No campaign want to have as their representative is a Tory toff from the home counties, even one with a fine haircut?"
Mr Cameron replied: "I humbly accept that while I am sure there are many people in Scotland who would like to hear me talk about this issue, my appeal doesn't stretch to every single part.
"But the key point you are making is absolutely right. The reason the Yes campaign head and the No campaign head can't seem to get a debate is because those who want to break up the United Kingdom, they know they are losing the argument and so they want to change the question. It's the oldest trick in the book and we can all see it coming."
In an earlier exchange during Prime Minister's questions in the Commons, Mr Cameron was criticised by Angus Robertson, the SNP's leader in Westminster, for repeatedly refusing to hold a live television debate with Mr Salmond.
Turning to the Prime Minister, Mr Robertson said: "Your anti-independence campaign launched a campaign this week encouraging people from outside Scotland to take part in the debate.
"Given that initiative, why won't you debate with the First Minister on television?"
Mr Cameron replied: "The calls for this debate show a mounting frustration amongst those calling for Scotland's separation from the rest of the United Kingdom because they know they are losing the argument.
"They know they are losing the argument about jobs, they know they are losing the argument about investment, they completely lost the argument about the future of the pound sterling. They are losing the argument about Europe.
"Yes, of course there should be a debate. But it is a debate between people in Scotland. The leader of the In campaign should debate with the leader of the Out campaign. Of course, you as the lackey of Alex Salmond want to change the terms of the debate but I am not falling for that one."
Outside of the chamber a Downing Street source was asked if it did not matter how many times Mr Cameron was challenged by the SNP for a live television debate with Mr Salmond, he would still refuse.
The source replied: "That is correct."