A total of 36% of firms would consider moving following a Yes vote on September 18, while 40% would not, according to a Survation poll of 100 Scottish businesses for the Scottish Mail on Sunday.
Only 15% of those surveyed said an independent Scotland would be "beneficial" to their business, while 45% said separation would be "harmful".
Three quarters (75%) said it was "essential" or "important" for their business that an independent Scotland remain in a currency union with the rest of the UK.
That option has been ruled out by Chancellor George Osborne, backed by Labour and Liberal Democrat leaders.
The companies, said to have combined sales worth more than £15 billion, took part in the poll on condition of anonymity.
Concerns about independence have been raised by a string of Scottish businesses in recent weeks, including Lloyds Banking Group, Barclays, Standard Life, Royal Bank of Scotland, Alliance Trust and Aggreko.
The heads of major oil companies Shell and BP have also stated they would like to see Scotland remain in the UK.
Blair McDougall, director of the pro-Union Better Together campaign, said: "The fact that so many businesses would consider relocating from Scotland if we left the UK is a big concern.
"This poll confirms what companies like Standard Life and Alliance Trust have already made clear. Independence would cost Scottish jobs and put up costs for families here. It's a risk we don't have to take.
"Being part of the UK single market is vital for Scottish businesses.
"Today they have the strength, stability and security of the UK pound, with no barriers between them and their customers elsewhere in the UK. The only thing putting this at risk is Alex Salmond's obsession with independence."
Kenneth Gibson, SNP MSP and convener of the Scottish Parliament's Finance Committee, said: "Business in Scotland clearly wants a currency union with the rest of the UK if there is a Yes vote, and that is exactly what the Scottish Government is proposing.
"This survey puts further pressure on the UK Government to drop their bluff and bluster on the pound, which will be the currency of an independent Scotland within a sterling area."
He highlighted the 1,500 member pro-independence organisation Business for Scotland, as well as pointing to high-profile figures who have voiced support, including British Airways boss Willie Walsh, Clyde Blowers founder Jim McColl and Klin Group Chief Executive Marie Macklin.
A Business for Scotland spokeswoman said: "In line with the majority of businesses surveyed for the Mail on Sunday Survation Poll it clearly makes sense to have a sterling area after a Yes vote in September. This is in the interests of both Scotland and the rest of the UK."
Billionaire Jim Ratcliffe, whose company Ineos operates the Grangemouth oil refinery, became the latest business figure to indicate that Scotland could be economically successful under independence.
He said: "It will work whether Scotland is part of the UK or independent. It will work as an independent country in the same way that Switzerland works.
"You don't have to be big to be successful. Switzerland is small and it is very efficient and it has got one of the highest gross domestic products on the planet."
Scottish Labour's Iain Gray MSP said: "What is now becoming clear is that the SNP's ill-thought through proposals are increasing uncertainty and damaging business confidence in Scotland. Put simply, Alex Salmond's constitutional obsession is putting Scottish jobs at risk."
Mr Gray said that an intervention by Larry Fink, chairman and chief executive of the world's largest asset management company BlackRock, was "another blow" to First Minister Alex Salmond.
Commenting on independence in an interview with The Sunday Times, Mr Fink said: "I assume there would be a different currency. One should assume that as a certainty - that would be my observation.
"How would you have an independent country and share a currency? That generally doesn't work - that's called the euro."