MPs were also told that the rest of the UK was likely to end up bailing out Scotland in the event of a crisis if there was a currency union even if there were explicit guarantees written into an agreement ruling out such an event.
European laws that require banks to have their head offices in the same member state as their registered offices would be likely to hit Scotland, the Treasury select committee heard.
Asked if RBS would have to move to the remaining UK if an independent Scotland joined the EU, Mr Carney said: "It's a distinct possibility but I shouldn't prejudge it.
"It depends on their arrangements as well, if they were to adjust more into Scotland the minor management of the institution."
Committee chairman Andrew Tyrie said: "Distinct possibility is a central bank phrase with a sort of layer of lawyer on top."
Chancellor George Osborne has already ruled out a currency union between an independent Scotland and the rest of the UK.
First Minister Alex Salmond's Scottish Government wants to create a "sterling zone" with the rest of the UK if there is a Yes vote in the break-away referendum.
Mr Carney said in a speech in January that an effective currency union would force a newly-independent Scotland to hand over some national sovereignty in a similar way to the eurozone.
He told MPs today that the Bank of England did not have to be the lender of last resort to Scottish banks if there was no formal currency union and would be "very conscious of the exposure to the public balance sheet".
Mr Carney said there would be "advantages" to an independent Scotland if it became part of a EU banking union but it would not have a common deposit guarantee scheme and lender of last resort negotiations would have to be made.
"Does it make it more viable?" he added. "It's a mechanism, there's a wide range of factors which would determine viability and viability in a currency union is a bit like being pregnant. You can't be half viable in a currency union.
"You need all the components. It would advance it but the other components would need to be put in place so you have to look at the package."
Mr Carney suggested the rest of the UK could end up bailing out Scotland if it was hit by a financial crisis.
He told MPs: "There's famously a no-bailout clause in the Maastricht Treaty but when push came to shove with risks in the periphery that was not fully credible because the spillover effects of not bailing out, or not assisting in some sort of arrangement, were judged, in the teeth of a global crisis, to be too great to risk and so there were extraordinary efforts made, and appropriate efforts, made by European officials to avoid that.
"The issue for the rest of the UK would be would it be credible, I'm not predicting this but one has to plan for contingencies, would it be credible to stand by if an independent Scotland were to be in fiscal difficulties?
"If it is not credible then the structure of the currency union obviously is less than perfect but it is also taking on a contingent liability or responsibility to the rest of the UK balance sheet.
"So, these are all issues that have to be thought through in advance."
Pressed on whether he was going "a long way down the road of saying" that experience showed any system of rules would be vulnerable to the risk of bailouts, he replied: "My view is that the eurozone needs to build that form of fiscal federalism to be viable."
Mr Carney dismissed claims that he thought a currency union was a bad idea for the UK and something he did not support.
He said: "I have been absolutely clear and I will continue to be clear that what I have tried to do, what the Bank has tried to do, and what we will continue to do is to draw attention to economic issues which the interested parties can address and then others can make the judgments.
"At no time have I said, or will I say, that I do not support or that I advocate a currency union, either side of the debate."
When it was suggested that he had created a "less than optimistic picture", he told MPs his intention was to draw attention to the underlying economic issues.
A Better Together spokesman said: "This is an important contribution from the Bank of England governor. What people in Scotland need is clarity from Alex Salmond about his Plan B for what would replace the pound. Would we have to sign up to the euro, as Mark Carney said, or would we set up an unproven separate currency?
"No doubt Alex Salmond's response will the same as every other time an expert has questioned his plans - Mark Carney is wrong and only the First Minister is right. It simply isn't credible."
Stewart Hosie, SNP Treasury spokesman and a Treasury committee member, said: "I am pleased that the governor took this opportunity to confirm the Bank of England's neutrality on the issue of Scottish independence, and that his Edinburgh speech was a technical assessment of currency unions, not a judgment on independence.
"Mark Carney was clear that the issue he wanted to get across was the nature of the stability arrangements which are required for the formation of a successful currency union. I am pleased that the Scottish Government's Fiscal Commission Working Group have described in detail a blueprint for such a successful currency union.
"He made the point - in terms of stability arrangements - that what would matter is the aggregate position of the economy and of public spending. That is extremely helpful given Scotland's fiscal position has been stronger on average than the UK's for over three decades.
"I was also pleased that Dr Carney confirmed that judgments on stability arrangements would be based on aggregate numbers such as debt and deficit. This completely contradicts the No campaign assertion that a 'foreign country would be running Scotland's economy'."
Liberal Democrat treasury select committee member John Thurso said: "When a sensible person says sensible things, SNP ministers would be wise to listen. What we heard today was a reasoned, technical and sobering analysis from Mark Carney on the many questions around the SNP's plans for currency, EU entry and our financial services.
"Being in Europe and in Britain is good for Scotland's jobs and growth. We benefit from a single UK-wide market and a single currency which is backed up by a lender of last resort in the Bank of England. Our membership of the EU gives us access to billions of pounds worth of trade. What the SNP are proposing is to leave these key economic and currency unions and seek to renegotiate their way back in, on inevitably poorer terms to Scotland.
"The only way to guarantee these positive unions is to vote to stay in the UK family of nations. But I doubt the SNP's devil may care attitude towards these sensible questions, from people such as Mark Carney, will please even the most ardent of independence supporters."