In either case, a rejection of the status quo would mean the pragmatic renegotiation of relationships between institutions, Gerard Lyons told the final day of the National Association of Pension Funds (NAPF) investment conference.
Mr Lyons, 25 years in the City and now economic adviser to the London mayor Boris Johnson, said: "The relationship between the UK and the rest of Europe is as significant an issue as that between Scotland and the rest of the UK."
But while power was already devolving to Edinburgh, it was centralising on Brussels.
In the financial sector, a Yes vote in the Scottish referendum could destabilise the balance between London and Edinburgh if firms were forced to move south, Mr Lyons said.
He added: "It doesn't do the UK any good to have everything centred on London."
The economist noted that while Berlin accounted for 4% of the German economy and Paris 9% of France's, London was now 25% of the UK gross value-added economy generating around £309 billion.
That compares with Manchester on £51bn, Glasgow on £18bn and Edinburgh at £17bn.
While speaking at the Edinburgh International Conference Centre yesterday, Mr Lyons said if the UK had a referendum on EU membership, "in some respects it would be like the debate in Scotland - it is about the subsequent relationships of the institutions we set up and how would they work".
He added: "Even if Britain were to leave, it is difficult to imagine London not remaining the wholesale financial centre of Europe - but it is opening up the whole retail sector that is still the big upside."
Mr Lyons said his high 2% growth forecast for the UK last year had proved about right, and he now expected growth of 3.5% to 4% this year.