The Royal Society of Edinburgh, one of the country's most prestigious academic bodies, has suggested the Scottish Government proposal is unlikely to be legal.
The SNP's White Paper on the issue states that universities could continue to charge students from England, Wales and Northern Ireland (rUK) fees of up to £9000, while at the same time offering free tuition to those from Scotland and the rest of the EU.
However, a working group from the Royal Society has raised significant doubts about whether that could happen in a submission to the Scottish Parliament's education committee, which is looking at the impact of independence.
The society paper states: "The working group is unsure as to what the legal basis for Scottish universities charging rUK students' fees might be in the event of Scotland becoming independent.
"There is.... neither unanimity nor consensus among legal authorities on the issue of tuition fees. We are, on balance, more persuaded by the opinion that suggests that it would be very difficult for Scottish universities to justify charging rUK students in the event of independence."
The working group goes on to advise the Scottish Government to exercise prudence in planning how, and to whom, it intends to charge tuition fees.
Liam McArthur, education spokesman for the Scottish Liberal Democrats, said: "This reasoned opinion from the Royal Society of Edinburgh raises further huge question marks over the viability of the SNP's plan.
"It seems the SNP are making it up as they go along in an attempt to make the sums add up, but as is made clear by this evidence, the only people buying into the SNP's assertions are the SNP."
Kezia Dugdale, education spokeswoman for the Scottish Labour Party, said the submission was a "hammer-blow" to the SNP's position.
She said: "Instead of the SNP pretending that everything will be fine, they need to provide certainty and clear, evidenced legal advice which substantiates their position. Until then, our universities will rightly be extremely concerned about the dramatic financial black-hole which will emerge in their funding if the SNP succeed in breaking up Britain."
Liz Smith, education spokeswoman for the Scottish Conservatives, added: "It has been very clear for almost two years now that, as a result of EU laws, the Scottish Government in an independent Scotland could not treat rUK students differently from other EU students."
A Scottish Government spokeswoman insisted there was a clear case for universities to continue to charge fees.
She said: "The requirements of the EU allow for objective justification - that is clear evidence of exceptional circumstances.
"This is explicitly acknowledged on page 199 of Scotland's Future, where it expressly identifies objective justification as the basis of our approach. This is a point also made in independent legal advice sought by Universities Scotland."
The body, which represents university principals, has demanded "robust legally defensible certainty' over whether institutions could continue to charge fees in the event of a Yes vote in September's poll..