Len McCluskey, general secretary of Unite, also suggested his union could break its links with Labour, which would cost the party millions.
His remarks follow a difficult few days for the pro-union Better Together campaign and the Labour leadership.
In Scotland Unite has said that it will stay neutral in the independence campaign.
But Mr McCluskey, who last year described the SNP's argument for independence as "seductive", told journalists at Westminster: "I know at the moment that the opinion polls are showing that there would be a solid No vote on independence.
"But I would suggest that everybody just looks at what happened in Canada back in 1995 when there was an argument on the separation of Quebec."
Mr McCluskey said that the anti-independence campaign, which won by a margin of less than 1% of the vote, was accused of containing "too much negativity and the Quebecers felt they were being threatened.
"And we are seeing that now with the No campaign, just threats, threats, threats."
He added there was a challenge in the Scottish Labour Party "to demonstrate that they are on the side of ordinary Scottish working people".
He said: "Alex Salmond at the moment is charging up Labour's left wing and seems in many ways more radical than the Labour Party."
The union boss had a similar warning for Mr Miliband, who last year reported Unite to police over allegations of vote-rigging in Falkirk.
Labour later had to withdraw the allegation after key witnesses changed their statements.
If Labour's offer to voters at the next General Election is "a pale shade of austerity", Mr McCluskey said, "I believe that Labour will be defeated."
Unless there was a real alternative the electorate would think "we best stick with the devil we know", he predicted.
"So Ed's challenge ... and I hope he will pick this up sooner rather than later, is to simply demonstrate that he is on the side of ordinary people."
He added that he could envisage a time when Unite disaffiliated from Labour: "I can do. And that's a challenge to Ed Miliband. I believe the Labour Party are at a crossroads."
Angus Robertson, the SNP's Westminster leader, said: "Len McCluskey's comments today add to the No campaign's woes.
"When he says the 'threats, threats and more threats' tactics of Project Fear are in danger of backfiring, he is clearly taking his own soundings in Scotland."
Conservative Party Chairman Grant Shapps accused Mr McCluskey of threatening to cut off the money if he does not get his way.