Members of the National Union of Rail, Maritime and Transport Workers (RMT) voted for the union to come out in support of Yes, a move unlikely to calm the nerves of the Better Together campaign after its lead shrank to just six percentage points.
Members voted by a margin of 1051 to 968, with 365 undecided.
It is only the second union in Scotland to come out in favour of Yes after the Prison Officers' Association.
But the timing of the announcement underlined the challenge facing the No campaign as it attempts to shore up support for the union in Scotland's traditional Labour heartlands.
Yesterday, Mr Miliband urged Labour voters considering backing independence in the referendum to remain true to the party's traditions and vote No, so he can build a fairer Scotland within the United Kingdom.
He travelled to Lanarkshire, where party founder Keir Hardie was born, to make a passionate plea ahead of the September 18 referendum.
He said: "I think lots of Labour voters and others are trying to make up their minds about how to get rid of this Tory government. That is front and centre.
"I know the way to get rid of this Tory government, that is to vote No and to elect a Labour government, and that is what I believe is going to happen.
"As people confront that decision, I just want them to be clear about who really represents social justice and what the prospects are of a Labour government. Our opponents would try to say they are better for social justice and they are not.
"When that argument fails, as it does, they then say 'you can't elect a Labour government, you're going to have a Tory government' when all of that flies in the face of the evidence."
Mr Miliband said Labour supporters were right to "absolutely loathe" David Cameron's government at Westminster.
But he also argued an SNP administration in an independent Scotland would represent a continuation of Tory policies, citing proposals such as plans for a 3p cut in corporation tax after a Yes vote, and the nationalists' failure to support an energy price freeze and the reintroduction of a the 50p income tax rate for higher earners.
The Labour leader said: "Take their position on corporation tax, or on energy, or on tax or on higher taxes for the wealthiest. They are a continuation of the things the Tories have been putting forward, not a change.
"If you want real change, if you want a change from this Tory Government, the right answer is to vote No and then elect a Labour government, which I believe is going to happen."
In response, Alex Salmond was scathing of the Labour leader's intervention in the debate.
He said: "Labour people in Scotland are turning their back on the Westminster elite and that includes Ed Miliband."
The First Minister defended SNP plans to cut corporation tax, saying the move was designed to create 27,000 jobs and increase economic growth, and attacked Mr Miliband's credibility on the issue.
On the RMT ballot, a spokesman said: "RMT has conducted a referendum ballot of our members in Scotland and they have narrowly recorded a majority in support of the 'yes' position.
"That ballot result has been reported to the union's executive and we will be sending out a formal letter to our members in Scotland in due course."
A Scottish Labour spokesman said: "Six trade unions, representing 140,000 people across the public and private sector in Scotland, are in favour of a No vote on September 18. The vast majority of trade unionists in Scotland agree that we are stronger together and want to side with the politics of unity and solidarity, and not the politics of division.
"Even this ballot did not show a majority in favour of independence - 59% of people who voted didn't support a Yes vote."
Kevin Lindsay, Scottish Secretary of train drivers' union Aslef, said: "Aslef, together with five other trade unions, is supporting a No vote. I am voting against Scottish independence because I am a trade unionist. My aim is to see all working people - regardless of a line on a map - united to strive for more equality and higher living standards."