The Royal Incorporation of Architects in Scotland (RIAS) has written to Perth and Kinross Council to oppose the proposed demolition of Perth City Hall, which was approved last December.
Councillors, having rejected plans to turn it into an indoor market, are currently looking at plans to tear down the listed building, which is more than 100 years old, and replace it with a public square.
However, Neil Baxter, RIAS Secretary & Treasurer, said: "The city hall is part of the densely-built fabric of the city which, like so many wind-swept Scottish places, has a medieval streetscape that lends unique character.
"The council's proposal, to replace the building with a gap site, or, in their terms, a 'piazza', indicates a serious misunderstanding of the character of the city which they were elected to protect.
"Remove this key building and you do fundamental damage, civic vandalism, which would be regretted for decades to come."
He added that the proposed £4 million cost of demolishing the building would be better spent on a restoration project.
Mr Baxter said: "Saving something good, as a legacy for the future, requires vision and imagination.
"How about, instead of spending £4m of public money on destroying something irreplaceable, spend £1m as a bounty to support a new viable commercial enterprise within the Concert Hall's beautiful historic walls?
"Perth Council should stop acting like a town council and become a city council. All of Scotland would benefit."
A Perth City Council spokeswoman said councillors would make a decision within 28 days, and that any plans to demolish the building would have to be agreed by Historic Scotland.