The Public and Commercial Services (PCS) union, which represents 30,000 civil servants and other workers north of the Border, supported neutrality over independence by a margin of more than three to one.
But the No side received a drubbing - failing to win any support at the conference yesterday.
Following a consultation with its branch members, PCS delegates debated three options in Glasgow on the constitutional question: for, against, or take "no campaign side" in the referendum.
Deputy First Minister Nicola Sturgeon urged the union to back a Yes, left-wing Labour MSP Neil Findlay spoke in favour of a No, while PCS Scottish secretary Lynn Henderson backed neutrality.
Sturgeon said Scotland could be a "progressive beacon" under independence. Findlay countered: "Do we accept the council of despair view that Britain will never change? And do we say to our brothers and sisters in England and Wales that we're withdrawing politically, socially and economically from the relationship we share?"
Henderson argued that the official campaigns, Yes Scotland and Better Together, had "misjudged the people of Scotland", adding that voters do not want a "slagging match".
"We have 50 shades of tartan in this debate, with many people still to make up their minds," she said.
The Sunday Herald's sister newspaper, The Herald, last week reported that PCS was set to back independence. In the end, this proved not to be the case.
Senior figures inside PCS were alarmed at the claims that the union was on the verge of backing independence.
In an email to members, Henderson wrote: "The report quotes from a 'well placed source'. I can categorically assure you that this did not emanate from within the PCS Scotland office."
The Yes proposition received 5775 votes, neutrality 18,025 and the No option zero.
The neutrality position means the Yes campaign is trailing the No side in trade union endorsements.
Several unions - including Community, the GMB and Aslef - have backed a No vote, while the Prison Officers' Association Scotland supports independence.
Janice Godrich, PCS national president, said: "Today does not mean sitting on the fence. 'PCS informs - you decide' will mean taking our arguments - for an alternative to austerity, for properly resourced public services, for jobs, for fairness - to all sides of the constitutional debate in the run up to the referendum and beyond."
Blair Jenkins, chief executive of Yes Scotland, said: "The decision adds weight to the growing number of trade union members who say they are 'more attracted' to Yes."