Professor Bernard Ryan, of Leicester University, said the UK could remove British citizenship from Scots with "no personal connection to the rest of the United Kingdom" following a Yes vote in next year's referendum.
However, he also warned the government of a newly independent Scotland might "object" if the UK allowed too many Scottish residents to remain British nationals.
In a paper for the Royal Society Of Edinburgh, he said: "The Scottish Government has indicated its acceptance of the possibility of dual nationality and it is to be presumed a post-referendum United Kingdom would do so too.
"It does not follow, however, that dual nationality would be available to all persons who were British citizens prior to independence, and who then became Scottish citizens.
"The United Kingdom's intentions, and their implications for Scotland, appear to be matters on which post-referendum agreement between Scottish Government and the United Kingdom Government would probably prove necessary."
Mr Ryan, a professor of migration law, was among a panel of experts who addressed a Royal Society Of Edinburgh seminar on how independence could affect citizenship and immigration last night.
He also argued Scotland and the rest of the UK might establish "partial" immigration controls between the two countries, based on existing arrangements between Britain and Ireland.
The UK does not apply immigration controls to any arrivals from the Republic Of Ireland. However, the Republic Of Ireland applies immigration controls to UK nationals arriving in the country by sea or air.
Mr Ryan wrote: "This precedent shows that, in the event of Scottish independence, the absence of immigration control might be partial with, for example, full or selective controls upon journeys by air and sea, and that differences in the approach of Scotland and the United Kingdom may arise."
A Scottish Government spokeswoman said: "Citizenship in an independent Scotland will be based upon an inclusive model.
"Many people in Scotland have ties to the rest of the UK, and elsewhere, including familial, social and economic connections. An independent Scotland will offer dual citizenship, reflecting the continuing social union amongst the peoples of these islands and our links with countries further afield."
But Drew Smith, Scottish Labour's constitutional spokesman, said: "This is another area that the SNP has failed to give any indication to us what would happen in an independent Scotland.
"It is hard to believe Alex Salmond has been campaigning for separation for decades yet he still does not have answers to these basic questions."
An independent Scotland would seek to join the UK and Ireland's common travel area, the SNP has said, rather than the border-free Schengen zone shared by most other EU countries.
Mr Ryan said the common travel area would have to be made more formal if an independent Scotland were to join.
He said: "It cannot be ruled out that something closer to the formalised 'Schengen' arrangements applicable elsewhere in Europe might eventually emerge."