The publication from Citi also looks at the possible currency and debt arrangements in the event of a yes vote, and considers the strength of Scotland's credit rating.
It found that an independent Scotland's fiscal position would be "relatively weak and risky" while a monetary union between Scotland and the rest of the UK is "unlikely".
The report has been compiled jointly by Citi's economics, rate strategy and political analysis teams ahead of the September referendum.
The Scottish Government backs a formal monetary union, in which an independent Scotland and the rest of the UK would share the pound, but this option has been ruled out by the unionist parties.
Citi's report says: "We regard a sterling monetary union as unlikely but we are genuinely unsure what currency and monetary policy would be adopted by an independent Scotland.
"In our view, it is astonishing that the Scottish Government, in seeking independence, has reached this stage: seeking a currency union without agreement with the rest of the UK and without a clear alternative plan."
It continued: "Overall, we believe an independent Scotland would have a relatively weak and risky fiscal position. This might well produce a sizeable borrowing premium.
"With this backdrop, we do not consider the Scottish Government's threat - that if a currency union is refused, Scotland will walk away from its share of UK national debt - to be wholly a bluff."
The report also notes concerns about the effects of drops in oil revenues and the size of Scotland's banking sector, which it said creates "additional fiscal risks".
It goes on to consider future credit ratings for an independent Scotland.
"It is unlikely, in our view, that Scotland would be given the top AAA rating, but highly likely that it would be investment grade," the report said.
"The exact rating would depend on many factors, including the fiscal position - which might well be worse than that of the UK.
"Our best guess, weighing up the uncertainties, is that an independent Scotland would be rated in the high 'single A' region with risks on either side."
It warned that any suggestions that Scotland would walk away from its share of national debt could "negatively impact" on credit ratings.
Citi also highlights the "referendum risks" associated with a proposed vote on Britain's European Union membership.
Even a failed vote might "set the stage for market-moving surprise and generate heightened uncertainty", it said.
"Brexit (Britain's exit from the EU) would have significant policy impact that should not be taken lightly."
Scottish Secretary Alistair Carmichael said: "The Citi report identifies a number of further risks, such as falling oil revenues and a rising Scottish deficit as well as the uncertainty for business created by losing the UK pound.
"It's absolutely clear independence would be a big problem for Scotland rather than the solution - and not having any sort of currency plan makes it an even bigger problem."
A Scottish Government spokeswoman said: "As leading international credit rating agency Standard and Poor's last week concluded, Scotland's wealth levels 'are comparable' to those of AAA-listed nations, and that as an independent country - even without North Sea oil - Scotland would qualify for S&P's 'highest economic assessment'.
"The Fiscal Commission Working Group, which comprises economic experts including two Nobel Laureates, has already considered a range of currency options in its detailed report published a year ago, and concluded that it is in the interests of both Scotland and the UK to continue to retain Sterling in a formal monetary union, and that is the policy the Scottish Government proposes.
"The pound is as much Scotland's as it is the rest of the UK's, and the Scottish Government has put forward sensible proposals for a formal monetary union that would ensure both governments had full flexibility over their fiscal policies such as taxation, within an overall sustainable framework."
Better Together campaign director Blair McDougall said the Citi report had left the SNP's economic credibility at "a new low".
"All the money we get from the North Sea is used to fund our public services and pay for benefits and pensions," he said.
"Being part of the larger UK means we can take advantage of this vital resource without gambling the budget for our schools and hospitals on a volatile, declining resource. It's the best of both worlds.
"As Citi argues, it is astonishing that the nationalists haven't set out their Plan B on what would replace the pound. Would we rush to adopt the Euro or set up a separate unproven currency? Alex Salmond cannot spend the next six months denying economic realities."