A meeting of PCS branch delegates in Glasgow voted overwhelmingly in favour of staying neutral.
Of the 30,000 Scottish members represented, some 18,000 endorsed that position, against only 5000 who wanted to support a Yes vote.
However, none of the civil service union members, many of whom work for UK Government departments or agencies, were prepared to back a No vote.
There had been speculation that the PCS would become the first major Scottish union to back independence officially, so the rejection of the move will disappoint the Yes campaign.
They will be heartened, though, that no delegate votes were cast in favour of the No position.
The union's Scottish branches heard the case for each option from PCS Scottish secretary Lynn Henderson, Deputy First Minister Nicola Sturgeon and Labour MSP Neil Findlay.
The branches voted to recommend that the union should take no side in the Scottish referendum, but instead adopt an active engagement campaign under the banner "PCS informs - you decide".
The outcome of the vote will be considered by the PCS National Executive Committee, with any final decision to be taken by the Annual Delegate Conference in May.
Janice Godrich, PCS national president and chair of today's conference, said "Today we've seen PCS members present thoughtful, balanced arguments weighing up the pros and cons of the way forward for the union and its members.
"Today does not mean sitting on the fence.
"PCS informs - you decide" will mean taking our arguments - for an alternative to austerity, for properly resourced public services, for jobs, for fairness - to all sides of the constitutional debate in the run up to the referendum and beyond."
Ms Henderson spoke for a neutral position. She said the official campaigns, Yes Scotland and Better Together, had "misjudged the people of Scotland", adding that voters do not want a "slagging match".
"We have 50 shades of tartan in this debate, with many people still to make up their minds," she said.
"It is our electoral challenge to engage with both sides, yes and no, and also the argument for more powers."
She added: "Whatever the outcome of the vote today, in May, or in September, our industrial demands will remain."
Ms Sturgeon made the bid for a yes vote, stating that Scotland could be a "progressive beacon" under independence.
"Right now the UK is one of the most unequal societies in the developed world," she said.
"That's not something that has happened overnight, or even since the Tories got back in in 2010.
"It's the result of decades of Westminster government decisions. Very often these are governments which the vast majority of people here in Scotland didn't even vote for in the first place."
Ms Sturgeon also offered guarantees to union members who are currently working for the UK civil service, and stated her party's support for public services.
"I am not neutral about who provides public services. Public services must always be provided in the public interest by the public sector."
She made "explicit commitments" that there would be continuity of employment for all staff, by transferring them from UK Government departments to new Scottish ones.
Ms Sturgeon said: "I can guarantee you that we will protect your terms and conditions and your pensions.
"We will extend the no compulsory redundancy guarantee to all staff who transfer from Westminster departments.
"I know there will be people from the opposite side of the debate who want to scare people with unfounded assertions about threats to employment. What they don't want to talk about are the job cuts that are happening right now."
Mr Findlay followed Ms Sturgeon on stage with an address in support of a campaigning for a No vote.
He said: "Do we accept the council of despair view that Britain will never change?
"And do we say to our brothers and sisters in England and Wales that we're withdrawing politically, socially and economically from the relationship we share?
"And do we withdraw 58 non-Tory MPs from Westminster, leaving workers in the UK with the possibility of even more Tory futures?
"I want political change, but I want political change to improve the lives of people from Shetland to Dorset, from Cardiff to Belfast.
"That is the solidarity the movement was founded on, and should continue to build."
Blair Jenkins, Chief Executive of Yes Scotland, has welcomed the result of the vote.
He said: "The fact that a trade union representing 30,000 civil servants across Scotland has refused to support the pro-union campaign backed by Labour is highly significant.
"The decision adds weight to the growing number of trade union members who say they are 'more attracted' to Yes.
"We take great encouragement from the knowledge that trade unionists remain open to independence and it is our job and intention between now and September 18 to persuade them of the opportunities of Yes to build a fairer and more socially just Scotland, to protect and enhance workers' rights and to create new jobs by growing our economy in a responsible and sustainable way."
Mr Jenkins said PCS had set the highest standards of democracy and debate among trade unions.
Branches across the country had voted on their position - and the gap was closing between Yes and a neutral stance, with a number of branches - including HMRC East Kilbride, DWP Glasgow, MoJ Scotland and Driving Instructors - backing Yes. A number of branches voted for PCS not to take a position but to provide information to members..
Mr Jenkins added: "A neutral stance allows a fair and open comparison of the consequences of No and Yes for civil servants and public services, are we are confident this in turn will to lead more and more people backing Yes in September."