Members of the National Union of Rail, Maritime and Transport Workers (RMT) voted for the union to come out in support of Yes, a move unlikely to calm the nerves of the Better Together campaign after its lead shrank to just six percentage points.
Members voted by a margin of 1051 to 968, with 365 undecided.
It is only the second union in Scotland to come out in favour of Yes after the Prison Officers' Association.
An RMT spokesman said: "RMT has conducted a referendum ballot of our members in Scotland and they have narrowly recorded a majority in support of the "yes" position.
"That ballot result has been reported to the union's executive and we will be sending out a formal letter to our members in Scotland in due course."The RMT, led until earlier this year by the charismatic Bob Crow, is the largest of the rail unions and organises over 10,000 members in Scotland.
It is also heavly involved in the North Sea oil and gas industry, representing many offshore workers, and with a significant base and presence in Aberdeen.
Mr Miliband had been due to visit Aberdeen today but switched his plans to Blantyre, in Lanarkshire.
RMT sources said the decision came about after over a year of internal discussions and debates.
The Labour Party disaffiliated the union in early 2004 after some Scottish branches voted to donate some of their funds to the Scottish Socialist Party in 2003.
It is however, affiliated to the Left pressure group the Labour Representation Committee which aims to promote socialist principles within the Labour Party, while some local branches are still affiliated to their local Constituency Labour Party.
In June of this year it personally criticised First Minister Alex Salmond for failing to confront offshore drilling bosses over the issue helicopter ditchings in the North Sea.
Yes Scotland said the timing of the RMT decision "reflects the journey of more and more trade union members in believing Yes is the best choice for working people, and the only way to protect our public services from the damaging impact of decisions taken at Westminster".
Labour Party member and former Chairman of the Scottish Labour Party Bob Thomson, who recently came out as Yes supporter, said: "On the day Ed Miliband is in Scotland, it demonstrates that trade union members are rejecting the Westminster elite and backing Yes in increasing numbers because they want the powers here in Scotland to build social justice.
"As this ballot shows, Mr Miliband doesn't speak for trade unionists and Labour voters in Scotland on the referendum he speaks on behalf a No campaign, which wants to deny the people of Scotland the opportunity of a lifetime to build a fairer society."
A Scottish Labour Spokesperson said: "Six trade unions, representing 140,000 people across the public and private sector in Scotland, are in favour of a 'no' vote on September 18. The vast majority of trade unionists in Scotland agree that we are stronger together and want to side with the politics of unity and solidarity, and not the politics of division.
"Even this ballot did not show a majority in favour of independence - 59% of people who voted didn't support a Yes vote."
Kevin Lindsay, Scottish Secretary of train drivers' union ASLEF, said: "ASLEF, together with five other trade unions, is supporting a 'No' vote on September 18. I am voting against Scottish independence because I am a trade unionist. My aim is to see all working people - regardless of a line on a map - united to strive for more equality and higher living standards.
"As a trade unionist I have three overriding priorities: to build solidarity among all working people; to extend fairly remunerated career opportunities; and to establish a democratic framework with control over local affairs. All of these lead me to one conclusion. I will vote to retain Scotland within the UK."