Jose Manuel Barroso said an independent Scotland would have to apply for EU membership and get the approval of all current member states, in the wake of a Yes vote in September's referendum.
Mr Barroso told the BBC he respected the ongoing democratic processes surrounding the independence debate and said it was for the Scottish people to decide on the country's future.
But he told the Andrew Marr Show: "In case there is a new country, a new state, coming out of a current member state, it will have to apply and... the application and the accession to the European Union will have to be approved by all the other member states of the European Union."
He went on: "I don't want to interfere on your referendum here, your democratic discussion here, but of course it will be extremely difficult to get the approval of all the other member states to have a new member coming from one member state.
"We have seen Spain has been opposing even the recognition of Kosovo, for instance. So it is to some extent a similar case because it's a new country and so I believe it's going to be extremely difficult, if not impossible, a new member state coming out of one of our countries getting the agreement of the others."
The Scottish Government has previously said it will negotiate Scotland's EU membership in the 18 months after the referendum.
Scottish ministers want the negotiations to be hammered out from within the Union while Scotland remains part of the UK, according to the Government's White Paper on independence.
The independence referendum will take place on September 18, with voters being asked: "Should Scotland be an independent country?"
Mr Barroso's comments come just days after the main parties at Westminster ruled out an independent Scotland sharing the pound with the rest of the UK under a currency union - the Scottish Government's preferred option.
And First Minister Alex Salmond has today written to Prime Minister David Cameron alleging "bullying" behaviour by Westminster ministers.
He claimed that recent interventions in the independence debate were contrary to both the letter and the spirit of the Edinburgh Agreement, which paved the way for a legally-binding referendum on Scottish independence to take place.
Scotland's Deputy First Minister Nicola Sturgeon branded Mr Barroso's stance "preposterous".
Ms Sturgeon said: "This is a preposterous assertion - as the ridiculous comparison with Kosovo illustrates. Scotland is already in the EU and has been for 40 years.
"As Mr Barroso rightly says, the question of Scotland's independent membership of the EU is a matter for the democratic wishes of the people of Scotland and the views of other member states - not the European Commission.
"The fact of the matter is that no member state has said that it would veto Scotland's continuing membership.
"We have put forward a suggested mechanism by which Scotland can make the transition from being in the European Union as part of the UK to being part of the European Union as an independent country.
"This can be done with continuity of effect and no detriment to other European states.
"And as a number of experts, including former European Court judge Sir David Edwards and honorary vice president of the European Commission Graham Avery, have said this is a process that will take place in the period between a vote for independence and Scotland becoming independent.
"The question Mr Barroso was asked was would Scotland be welcome. Every indication we have is that the answer to that question is yes.
"Indeed, the only threat to Scotland's membership of the EU is if we don't become independent and face the prospect of David Cameron's in/out referendum in 2017."
Alistair Darling, leader of the Better Together campaign to keep the Union, said Mr Salmond was "a man without a plan on currency and Europe".
"On the two biggest issues for jobs and businesses in Scotland, currency and Europe, the nationalists are all over the place. We are being asked to take a huge leap into the unknown," he said.
"When Alex Salmond is told Scotland won't keep the pound, he says everybody is bluffing and only he is right. When the president of the European Commission says there would be huge difficulties with EU membership, nationalists say he is talking nonsense. It isn't credible.
"People in Scotland are entitled to know what would replace the pound and the difficulties we would face on Europe. Instead of more bluff and bluster it's time for Alex Salmond to face up to reality."
Both Scottish Labour and the Scottish Liberal Democrats said Mr Salmond now needs a "Plan B" on the EU. There have already been calls for the First Minister to outline his fall-back position on currency after plans for a currency union were dealt a blow last week.
Scottish Labour leader Johann Lamont said: "The reality is that a separate Scotland won't use the pound. The reality is that a separate Scotland would find it extremely difficult to join the EU.
"His assertion-riddled White Paper lies in shreds. Instead of complaining about being told some home truths, Alex Salmond needs to tell Scotland what are his alternatives."
She added: "Tomorrow, Alex Salmond needs to come clean and tell us what his Plan B is; not just on the currency, but now on Scotland's extreme difficulty in joining the EU."
Scottish Liberal Democrat leader Willie Rennie said: "The European Commission President knows more than most how other countries will react to an independent Scotland's application to join the EU.
"He can't be aggressively dismissed with Alex Salmond's characteristic bluster. It's not up to Alex Salmond if Scotland joins the EU. If even just one country says no it does not matter how loud Alex Salmond shouts, Scotland will not be in."
He added: "Not only does the SNP need a plan B on the pound but now they need a plan B for the EU."
But responding to Mr Barroso's comments, SNP MP Angus MacNeil told Pienaar's Politics on BBC Radio 5 Live: "I thought it was absolutely fascinating. It'll make tremendous news in Oslo, Reykjavik and Kiev of course that Barroso's effectively saying no new members coming into the EU. It's a very, very surprising step from somebody in an influential position."
And Holyrood's Finance Secretary John Swinney told the BBC's Sunday Politics Scotland: "As far as I'm aware, not a single state in the European Union has made any remark about the fact that it would be in any way likely to veto Scottish membership of the European Union.
"And I can't see why they would want to do that, given that the whole history of the European Union has been about expansion, about growth, about involving new members, about bringing more countries into the fold of the European Union."