The former chancellor and home secretary - currently the Coalition's Minister without Portfolio - made the comment at a dinner of Scottish Conservative activists last week, where he was introduced by Scots Tory leader Ruth Davidson.
The SNP said the remark was "offensive" and showed it was Westminster, not Scotland, that was stuck in the past.
Clarke, 73, also referred repeatedly to the independence ballot as the "devolution vote" and said it was in May rather than September.
"Well, whenever it is," he quipped when members of his audience corrected him.
Clarke also predicted a No vote would be vital for Tory momentum, helping the Conservatives win more seats in Scotland at the 2015 General Election.
Well-known for his colourful language and flamboyant character, Clarke was the keynote dinner speaker at the Scottish Conservative conference in Edinburgh.
Although the event was a private one the Conservatives were happy to grant the Sunday Herald access.
Clarke said the pro-UK parties deserved praise for working together for "the national interest in all four nations of the United Kingdom" before his controversial remark.
He said Scots were engaging seriously with the "devolution vote" and thinking what it would mean "to revive the medieval state of Scotland and start disentangling the closely intertwined peoples and economies of Britain and Scotland in some as yet unplanned and unknown way".
Referring to the Tories' one MP out of 59 in Scotland, he added: "We need to come back in Scotland because political momentum here would make a huge difference to the task we face.
"If - when - we win this devolution vote, I trust we will not only have saved the national interest of the population of all parts of the United Kingdom, we should give a political momentum to fresh efforts to get the seats we want in Scotland.
"It should take us forward to another victory in May 2015."
In other remarks, Clarke praised the work of the Tory-LibDem coalition at Westminster, and said it had worked better than he had expected.
"At the time I was horrified by a hung parliament, but actually I think the Coalition has achieved more than the Conservative Party could have achieved on its own in a minority."
But he also wanted a Tory majority in 2015. One of the most ardently pro-European voices in the party, he said the alternative could be yet another coalition, and he even raised the spectre of negotiating with europhobic Ukip.
"It is a very real possibility, another hung parliament. It may not be the same hung parliament. If we have another hung parliament, are there are any Liberal Democrats left?
"Might we find ourselves negotiating with Nationalists - Welsh Nationalists, Scottish Nationalists - trying to form a majority?
"Democratic Ulster Unionists nowadays, we might be negotiating with. We might - heaven knows - [with] Ukip, if the public vote for it?"
SNP Westminster leader Angus Robertson said: "This is a silly and offensive intervention by Ken Clarke, whose own words reveal he doesn't know what the referendum is about, or even when it is.
"A Yes vote will pave the way for a modern, 21st-century system of government in Scotland.
"It is the Westminster system that is stuck in the distant past, no written constitution, and a majority of Westminster lawmakers still unelected and swathed in ermine."
Patrick Harvie, co-convener of the Scottish Greens, added: "It sounds like if anybody is stuck in the past it's Ken Clarke. Independence is about opening up new possibilities for the future of Scotland.
"Those decisions should be in the hands of Scottish voters, not dinosaurs like Mr Clarke."
Clarke, MP for Rushcliffe in Nottinghamshire since 1970, was first appointed a minister under Margaret Thatcher in 1982, then served as health secretary, education secretary, home secretary and chancellor of the exchequer.
After the Tories returned to power in 2010, he was made Justice Secretary, and in 2012 became Minister without Portfolio, advising on economic strategy and exports.
A Scottish Conservative spokesman said: "This was an off-the-cuff remark made at a private dinner for party members and should be taken as such. Such mock outrage from the SNP is nothing new in this referendum campaign."