Adam, who is also chairman of Elgin-based housebuilders Springfield Properties, praised the economy-boosting effects of the Help to Buy scheme and the work of the Scottish Government, some councils, and the grassroots industry.
But he said the continuing "absolutely shocking" shortage of new housing supply was due to the attitude of "bods with the bums on the seat in the nice warm offices who put the words on the paper and the lines on the plans".
He added: "They are the problem, they are the blockage in this - we should be able to get this sorted."
Deputy First Minister Nicola Sturgeon, who followed Adam on the platform, said she would convey the message to Planning Minister Derek Mackay. She also announced an extra £40 million for the Scottish Government's Help to Buy Scotland initiative.
In an impassioned short address, Adam detailed what he depicted as the chronic, economy-inhibiting dearth of new public and private housing in Scotland, citing the difference between the shortfall of 35,000 houses a year identified in 2007 and the 14,000 delivered annually in the past few years. He cited a Swiss counterpart who expressed surprise that Scotland, where housing is a devolved matter, should not deliver the number of houses that demographic changes required.
He said: "I don't know about you, but I am heartily sick of listening to people going on about how our planning system is not fit for purpose, year after year after year. It's not getting any better."