Jose Manuel Barroso said it was "obvious" a newly independent country would not automatically be a member.
Opposition parties seized on his comments, which they said raised the prospect that Scotland could be forced to join the euro and Schengen travel area – leading to passport controls at the border with England – if it left the UK.
But Mr Barroso's analysis was disputed by Deputy First Minister Nicola Sturgeon, who called for talks as soon as possible.
Last night, as the row intensified, the Conservatives also demanded Scottish ministers make a statement to Holyrood.
John Lamont, the party's chief whip, said MSPs had repeatedly been told a separate Scotland would automatically become part of the EU, adding: "Either the SNP has to admit it was wrong and misled Parliament on the issue, or state that it categorically disagrees with the view of Mr Barroso."
Intense argument has raged over the future of Scotland's EU membership since Scottish ministers admitted they had not sought legal advice on the issue.
The Scottish Government has long argued Scotland would inherit membership. It has said negotiations would have to take place but insisted these would happen alongside independence talks, while Scotland was still part of the UK.
In his strongest intervention yet, Mr Barroso said it was "clear since 2004 in legal terms, if one part of a country – I am not referring now to any specific one – wants to become an independent state, of course as an independent state it has to apply to the European membership according to the rules – that is obvious".
Asked if such a country would have to renegotiate its terms, he said: "Yes."
The rest of the UK would remain a member state, he added.
But Ms Sturgeon rejected that stance. An independent Scotland would not have to reapply "because there is no provision for removing EU treaties from any part of EU territory, or for removing European citizenship from the people of a country which has been in the EU for 40 years", she said.
She added: "We are now seeking early talks with the European Commission to discuss the specific process of Scotland becoming independent."
Mr Barroso's intervention was described as significant by Prime Minister David Cameron, who accused First Minister Alex Salmond of wanting to "have his cake and eat it".
"He says 'I want to separate myself from the UK, I want this new future for Scotland' but, on the other hand, 'no I don't want all the consequences from that'," he said.
Patricia Ferguson, Scottish Labour's constitutional spokeswoman, said: "For months, Alex Salmond and Nicola Sturgeon have put their heads in the sand, ignored the warnings about Scotland's future in the EU and pretended that everything will be fine. That is no longer an option."
Speaking in Washington at the start of a three-day visit to the US and Canada, Scottish Secretary Michael Moore said Mr Barroso's comments were "not surprising".
"We have said for some time that if Scotland were to leave the UK the most likely outcome is that it would need to seek EU membership on newly negotiated terms while the rest of the UK
would continue as a member state," he said.
He warned that in such a scenario Scotland would be forced to negotiate "from a position of weakness".
And he said that the issue revealed a "bigger lesson".
"Our view has been based on legal and academic evidence while their position has been based on nothing. That is why we are where we are."
The row comes ahead of today's appearance by Finance Secretary John Swinney before the Lords Economic Affairs Committee, which is investigating the implications of Scottish independence.
Last night, in a letter to the same committee, Mr Barroso reiterated his stance on EU membership.
He said: "A new independent state would, by the fact of its independence, become a third country with respect to the EU and the treaties would no longer apply on its territory."
Any European state could apply to become an EU member, he said, adding: "If the application is accepted by the council acting unanimously, an agreement is then negotiated between the applicant state and the member states on the conditions of admission and the adjustments to the treaties which such admission entails".