Scotland would "discover the failures of socialism pretty quickly" if solely responsible for its own economy, the Cato Institute argues.
It also questions Alex Salmond's eagerness to challenge the Washington Post newspaper's assertion that Scotland would be "unable to contribute meaningfully to global security".
Executive vice-president of the institute David Boaz said: "You'd think the opportunity to extricate your country from quagmires like Iraq would be a great benefit to the Scots.
"But Salmond denies that an independent Scotland would mind its own business and live in peace."
The Cato Institute was founded in 1974 and promotes the principles of "liberty, limited government, free markets and peace".
A recent University of Pennsylvania study ranked it as the 14th most influential think-tank in the world, sixth in the US, third for US economic policy and eighth for international economic policy.
Mr Boaz said: "Alas, the land of Adam Smith has become one of the poorest and most socialist parts of Great Britain. So maybe a libertarian shouldn't look forward to Scottish independence.
"On the contrary, I think it's easy for Scotland to whine and demand more money from the British central government. An independent Scotland would have to create its own prosperity, and surely the people who produced the Enlightenment are smart enough to discover the failures of socialism pretty quickly if they become free, independent and responsible for their own future."