Professor Malcolm Chalmers, director of the Royal United Services Institute (RUSI), says a Yes vote would leave the rest of the UK "in a state of humiliation and existential crisis" which could sour negotiations over the split of assets with Scotland.
In an article for the RUSI Journal titled "Dissolution and Defence: Scotland's Armed Forces after a Yes vote", Prof Chalmers says: "In the aftermath of what would, arguably, be the country's greatest humiliation since the loss of the American colonies, some senior officials believe that the resignation of the current Coalition government would be a strong possibility. The peoples of England, Wales and Northern Ireland are likely to have little sympathy for any policies seen as too accommodating to their Scottish cousins".
He also says the UK and Scotland would have different approaches to post-Yes talks, dragging them beyond the SNP's 18-month timetable.
Crucially, he says the UK would want to keep the Trident nuclear deterrent based on the Clyde for up to 15 years as part of a "grand bargain", whereas the SNP would like to agree the key terms in outline, and then finesse Trident. But he also says Trident - for which there is no other base in the UK - would be "a bargaining chip of considerable value" to Scotland and could be "the key to open the way to a smooth transition to independence".
He says the SNP's broad defence plans are credible, but warns the future of the Clyde shipyards could be grim, as the UK would switch its warship purchases to English yards like Southampton.
An SNP government spokesman said: "Independence will give Scotland the opportunity to develop specific defence capabilities that better meet Scotland's needs and circumstances."