The Liberal Democrat Scottish Secretary admitted it was "not impossible" that the Yes campaign could triumph in September's referendum and called on Labour to play a greater role in the contest.
Considering the prospect of a vote for independence Mr Carmichael said: "It is not impossible. I am not expecting to lose, but it is eminently possible that they will be able to buy momentum with the advertising and campaign resource they have.
"If they do, it could all get very difficult."
Mr Carmichael said the "volume" of the No campaign needed to be increased. "The basic messages are the right ones. They are delivered in a fairly professional way. We just need to do more of it," he said.
Warning of the risk of complacency among unionists, he told The Observer: "The danger is that by the time they realise it could happen, it could be too late.
"Everybody needs to know that this is a serious contest, and one which it is not impossible that the nationalists could win."
He said the No campaign needed to match the nationalists' "hunger" for victory.
"We're never going to match them for the spend, but in terms of the hunger I think we have to match them for just how badly we want this," he said.
"That is always going to be a challenge, because for nationalists this is the issue that defines them, whereas for a Labour supporter, a Liberal or a Conservative, this can be an issue you care about but is not one that defines you.
"So that is where we need to work harder at motivating our people in a way that their people come ready motivated."
The cross-party Better Together campaign is being led by Labour former chancellor Alistair Darling, but Mr Carmichael called for a wider range of voices from Ed Miliband's party to join the debate.
"You have in Alistair Darling a first-rate campaign head, but I would want to hear and would expect to hear a wider range of Labour voices coming into the debate as it gets closer to polling day," he said.
"The interventions of Gordon Brown have all been good and positive and helpful, and I'd like to see more of that."