The latest study by polling firm YouGov found 53% of those surveyed wanted Scotland to continue as part of the UK.
Meanwhile 35% of people questioned said that Scotland should be an independent country, according to the research for the Scottish Sun newspaper.
More than one in 10 voters (12%) were undecided about how to vote in the September 18 referendum.
The poll found 56% of women questioned backed the union, while 27% of females favoured independence, with the remaining 17% undecided.
Among men support for Scotland remaining in the UK was 48% - only four points above support for a Yes vote (44%). The other 8% of men said they did not know how they would cast their vote.
The poll findings were welcomed by those campaigning for independence, as well as those who want Scotland to remain in the UK
Better Together leader and former Chancellor Alistair Darling said the poll showed that First Minister Alex Salmond's "refusal to tell us what currency we would use if we left the UK is taking its toll" on the Yes campaign.
Mr Darling added: "While it is great that we speak for the majority of people in Scotland, we know that the only poll that matters is the one that takes place in September this year. We will fight for every vote."
Blair Jenkins, chief executive of the pro-independence campaign group Yes Scotland, said: "All recent polls have shown that Yes support is going up while No is on the way down.
"Compared to last August, Yes is up seven points and No is down seven. Our positive message that Scotland is a rich and prosperous nation and has got what it takes to be a successful and fairer country, too, is convincing more and more people than we can and should and, indeed, must be an independent country."
He added: "With more than six months to go until the vote, we are confident that with continued hard work we can persuade a majority that Scotland's future in Scotland's hands is the right thing to do."
The YouGov poll of 1,200 adults in Scotland was conducted between February 24 and 28 and found after undecided were removed that Yes voters sat at 40% and No voters at 60%.
The SNP also said that support for a Yes vote had therefore increased by 1% since YouGov's last poll while backing for a No vote had fallen by the same amount.
SNP Business Convener Derek Mackay MSP said: "This is a welcome poll, which confirms the progress that is being made towards securing a Yes vote in September.
"This is the first time that YouGov have put support for a Yes vote above 40% since the start of the referendum campaign.
"Just as with other recent polls, the direction of travel is firmly in favour of a Yes vote, and this is the sixth poll in a row to put Yes support in the 40s."