Humza Yousaf said the UK Government should be "extraordinarily careful" of the language it used and not "kowtow" to an agenda driven by Ukip.
He made the comments after claims that the lifting of border restrictions from January 1 will see hundreds of thousands of Bulgarian and Romanian immigrants come to Britain.
Yousaf - whose parents came to the UK in the 1960s from Pakistan and Kenya - pointed out immigration policy is currently reserved to the UK Government, and said: "Scotland's needs are different to those in the rest of the UK, and we welcome the contribution new Scots can make to our economy and society.
"Rhetoric and hyperbole describing floods or invasions of immigrants coming to the UK in the last week does nothing but potentially heighten any tensions that may exist. Politicians and governments have a duty to act responsibly in respect to immigration.
"An independent Scotland will operate a controlled immigration system that meets our own social, economic and demographic needs."
Yousaf said there were challenges and opportunities around immigration, adding: "The simple point is that [these challenges] are always inflamed if the rhetoric from the top - that can be government, but also certain media outlets - is being ramped up, then tensions could potentially also ramp up. I don't mean specifically with Romanians or Bulgarians in Scotland - with any immigration there is tension."
Temporary barriers withholding full working rights from Bulgarians and Romanians were lifted in the UK - and in eight other EU countries - on New Year's Day.
Bulgarians and Romanians could travel visa-free to the UK from 2007 when the countries joined the EU, but until now could only work under conditions such as being self-employed or taking a specialist post a UK worker could not fill.
Ukip claimed in leaflets that the UK would struggle to cope with an "influx" when the restrictions came to an end, with 29 million Bulgarian and Romanians having the right to "live, work and draw benefits". The combined populations of Romania and Bulgaria totals 28 million.
Ukip suggested Romanian/Bulgarian immigration would result in "fewer jobs available, pressure on hospitals, longer queues for care, wages going down, less money to go around and longer waiting lists for council housing".
In a speech on immigration last year, Prime Minister David Cameron spoke of ending the "something-for nothing culture" and has tightened the rules on benefit claims by EU citizens who come to the UK.
Ninety senior Conservatives last week urged Cameron not to lift controls on Bulgarian and Romanian migrants for seven years to stop a "hugely disruptive and destabilising wave of mass immigration".
Yousaf said: "Politicians and governments and the Westminster government have to be careful of the language that is used.
"We don't want at all to be kowtowing to an agenda which is being driven by Ukip … The politicians of any hue, of any government on these isles, have to be extraordinarily careful of the language they use and make sure they use it responsibly."
He added: "Backbenchers as well should be extraordinarily careful about the language that is being used around immigration." Yousaf pointed out that there were hundreds of thousands of UK citizens living in EU countries.
He said: "We have to accept if UK citizens can live and benefit from the services in Spain and France and across Europe, then of course Europeans will be able to benefit from services and so on in the UK.
"People should be judged on the contribution they make and new Scots of Polish, Pakistani, Irish or any descent should be welcomed on that basis."
Yesterday it emerged that Scotland is experiencing a new wave of Italian immigration, with educated youngsters from the impoverished Bari area of the country viewing jobs such as dish-washing as an improvement on their employment prospects back home.
Last week Bulgaria's former foreign affairs minister, who is now the United Nations Special Representative to Iraq, criticised the "mass hysteria" surrounding the immigration debate he said was driven by the far right.
Nikolay Mladenov, who was Bulgaria's foreign affairs minister until last spring, said claims of a sudden influx of Bulgarian and Romanian immigrants to Britain in 2014 were "purely politically motivated".
He said there was "no reason to believe the UK will be swarmed by waves of immigrants from Bulgaria".
How many Romanians and Bulgarians have arrived in the past few days is not clear. But according to Labour MP Keith Vaz, who greeted the first flight from Romania at Luton airport on January 1, most of the 140 passengers on board already lived and worked in the UK.
The Home Office said it could not give data yet on how many Romanians and Bulgarians had come to the UK to work since January 1.
The Office for National Statistics is expected to give an indication in May, based on national insurance number data.
Atul Hatwal, director of campaign group The Migration Matters Trust, said: "Ukip had a projection that we were going to have 300,000 net new arrivals this year from Romania and Bulgaria, so that is basically 25,000 new people a month.
"To achieve that, every flight, every ferry, every coach has got to be full. That is just not the case.
"In fact, travel operators are reporting that demand for travel from Romania and Bulgaria to the UK is slightly down on last year."
Hatwal said the debate about immigration was "dominated by hysteria and hyperbole".
He added: "It actually sends a very disturbing message to our European partners and to business, that Britain is closed for business.
"When companies are looking at foreign investment in this country and they want to judge whether they will bring their employees in, and whether this country will provide a hospitable environment for their investment, it raises doubts. That is the sort of thing that threatens our recovery."