Bob Dudley told the BBC there were "quite big uncertainties" over currency, European links and tax regimes if Scotland becomes independent.
However, he emphasised the firm was continuing to invest in Scotland, saying: "We have a lot of people in Scotland. We have a lot of investments in Scotland. My personal view is that Great Britain is great and it ought to stay together."
BP plans to invest £10bn in the North Sea between 2011 and 2016, its highest ever investment in the region.
Mr Dudley said: "I'm not concerned but there's enough uncertainty and talk about it and questions raised.
"It would create extra costs for our business. We have to duplicate the centres and do things, and again the currency question I don't know the answer to."
BP's future in the North Sea in the event of independence "would depend on what it really led to", he said.
"These investments are big, they are under way, we want to see them developed. It depends on what tax regimes are there and it depends on currency."
He added: "We have got a lot of people in Scotland, we have got a lot of investments in Scotland.
"There's much debate about what would happen with the currency, and of course whether there are connections with Europe or not.
"These are quite big uncertainties for us, and at the moment we are continuing to invest at a pace because these projects are under way.
"But it's a question mark. I think all businesses have a concern."
A Yes Scotland spokesman said: "A shared currency is in the overwhelming economic interests of both Scotland and the rest of the UK.
"This was the view of the (Scottish Government's) Fiscal Commission Working Group last year.
"With independence, the continued use of sterling has the overwhelming support of the people of Scotland and the public in the rest of the UK."
Better Together chief Alistair Darling said: "This is perhaps the biggest intervention by a major business so far in the referendum debate.
"I hope that more companies and business leaders speak out over the coming weeks and months. This debate is far too important to be left to politicians alone.
"Bob Dudley is quite right to express concern about the issue of currency. It is far from certain what currency we would use if we vote to leave the UK."
Labour's shadow energy minister Tom Greatrex said: "The warning of uncertainty, instability and impact on activity from BP today is one that echoes concerns expressed by others in the energy sector, including SSE, Scottish Power and Infinis.
"As Sir Ian Wood's influential interim report on the future of the North Sea made it clear that we need greater collaboration and co-operation to get the most from this diminishing resource. In oil and gas, as in other sectors, it makes no sense for us to be talking about putting up barriers and making business more difficult."
Mr Dudley's remarks can only be interpreted as a clear warning of what might happen if Scotland goes independent and will do nothing to help Alex Salmond's insistence that everything will be fine if Scots say Yes this September.
Coming just a few days after Mark Carney, the Govenor of the Bank of England, noted how currency union would mean an independent Scotland ceding sovereignty to a foreign institution based in London, Mr Dudley saying there was a "question mark" over whether sterling would continued to be used north of the border will again do little to ease any anxieties people have about Scotland breaking away from the UK.
Mr Dudley's remarks come just ahead of David Cameron's big speech on Scotland this Friday and a couple of weeks ahead of the UK Cabinet's visit to Aberdeen, Britain's oil capital.