The SNP have rejected the claim saying that leaving the UK would enhance Scotland's reputation by breaking an association with "illegal wars and weapons of mass destruction".
Peers said a 'Yes' vote could raise question marks over the UK's membership of the UN's Security Council.
The UK Government's immigration crackdown was also leading to a "nasty Britain" image, the Soft Power Committee's chairman Lord Howell, Chancellor George Osborne's father-in-law, warned.
The report by the committee, whose members include Scottish peer Lord Foulkes of Cumnock, says that independence could be bad for both for Scotland and for the remaining parts of the UK.
It reads: "The UK's aim and claim to continue to play a major role in world affairs would be undermined by Scottish separation, because even a debate about whether the UK should continue to be a member of the UN Security Council, for example, would do damage to its reputation.
"Dismembering the UK is not consistent with promoting the country abroad as a strong, stable and successful state; nor is it consistent with promoting the sense of internal social cohesion that is so important to presenting a positive view of the UK on the international stage. This damage would be to the disadvantage of the Scottish people, as much as to the UK as a whole."
SNP Westminster leader and foreign affairs spokesman Angus Robertson said: "Scotland already has a positive global reputation and independence offers huge potential for the Scottish 'brand'.
"Instead of being associated with illegal wars and weapons of mass destruction, our global reputation would be a reflection of the will and votes of the people of Scotland, enabling us to effectively harness our 'soft power'."
In an interview, Lord Howell said that the UK had to be "open for business" and let in skilled staff and students. He added: "This requires very careful handling.
"We're not getting it right. All our witnesses... and a vast amount of written evidence said the visa policy and some of the handling of immigration policy was creating rather a block, creating a nasty Britain feeling."
He said business people who want to move into Britain find it "very difficult to get through the tangle of regulations".