Alistair Carmichael, Secretary of State for Scotland, said the Scottish National Party's immigration plans could make such a move necessary, even if Scotland did not join the European Union's Schengen free travel area.
His comments are the strongest warning yet by a Coalition minister that independence could lead to border posts.
Last night the Scottish Government accused him of scaremongering.
London and Edinburgh are increasingly moving in opposite directions on immigration.
Last week SNP MSP Christina McKelvie accused the Tory end of the Coalition of an "anti-EU, anti-migrant, right-wing [and] dangerous discourse" after David Cameron urged other European leaders to limit free movement of travel between EU member states, particularly poorer countries.
Mr Carmichael was responding to proposals for a more liberalised immigration policy set out in the Scottish Government's independence White Paper last month.
The document said that the laws did not serve Scotland's interests. After independence, it suggested, there would be a new points-based system and a cut in the minimum salary levels required to come to Scotland. Reports yesterday also suggest that asylum seekers would be quickly given the right to work in an independent Scotland, under plans to boost the economy
Mr Carmichael, a Liberal Democrat, warned of the dangers of having separate immigration policies on either side of the border.
"It is just not possible, it won't work", he said.
"At the moment [the system]works very well. We are one country with a single labour market and a single immigration policy. If you draw a line on a map and you have one immigration policy on one and a different immigration policy on the other then inevitably one side or the other is going to want to protect the integrity of their systems."
Asked if he was talking about border posts, he said: "That is the real danger... that one side or the other will want to protect whichever is the stricter of the two policies. And you do that by putting in place border controls, whatever shape or form that takes."
He also pointed to the example of Ireland, a member of the Common Travel Area with the UK and not the EU's Schengen area. Because of its desire to remain part of the Common Travel Area Ireland had a broadly similar immigration system to the UK's, Mr Carmichael said.
"That's one reason why I think the Republic of Ireland has never joined Schengen," he added.
He agreed that the signals were that the UK was likely to have a stricter immigration policy than an independent Scotland.
"As things stand at the moment that is the case," he said.
But he also accused the Scottish Government of confusion over its plans. Scottish ministers talked of the need to bring more migrant labour into Scotland but had yet to set out any figures on how many people they had in mind, he said.
And he disputed the analysis that the system was letting down Scotland. He said Scotland's immigration needs were "broadly .. the same as the rest of the UK's".
A Scottish Government spokesman said: "These claims are simply scaremongering.
"Our controlled immigration system will be robust and secure, enabling the Scottish Government to ensure that those granted entry are here for the right reasons.
"An independent Scotland will remain in the Common Travel Area.
"Just as there are no border posts between the Republic of Ireland and the UK, there will be no border controls between an Independent Scotland and the rest of the UK and Ireland."