Statistics from the Scottish Government revealed work began on fewer than 3780 houses between January and March, the smallest number for that quarter since 1980.
Overall, the number of homes built in Scotland dropped by 14% in the past year, with only 14,629 houses hitting the market.
This includes all new builds, refurbishments and conversions, and was driven by a downturn in work carried out by private homebuilders, housing associations and councils.
Council housing stock has fallen by 1224 units in that time, and the number of homes being built under the Scottish Govermnent-funded Affordable Housing Supply Programme (AHSP) has dropped for the third year in a row.
Philip Hogg, chief executive of the housebuilders' umbrella group, Homes for Scotland, warned of severe long-term social and economic consequences unless more homes are built.
He said: "Today's figures reinforce the fact that Scotland is mired in a housing crisis.
"Less than 14,000 new homes were built in 2012-13, yet our population is at a record level and households are growing by more than 21,000 a year.
"It is estimated that 465,000 new homes are needed in Scotland by 2035 to meet demand. However, the build rate announced today points to a shortfall of around 160,000 by this time."
Mr Hogg said: "We need to see the Scottish Government's promised £120 million shared equity scheme launched as soon as possible and proposals to increase regulation and cost ... reconsidered.
"The cautious optimism which was apparent in the first half of the year could easily disappear, with investment channelled elsewhere."
Graeme Brown, director of the homeless charity Shelter Scotland, said: "There is no good news in these figures for anyone in Scotland who is waiting for a home of their own. The figures are stark and show even with recent additional cash injections Scotland isn't building enough new homes."
Overall, local authorities started building 1189 homes, bringing the number of new council houses under construction in the last four years to more than 4000.
Housing Minister Margaret Burgess said the Government was committed to delivering 30,000 affordable homes this Parliament.
She added: "We are still experiencing challenging economic conditions which are continuing to have an impact on house-building activity. This is due to world economic conditions exacerbated by Westminster's drastic cuts to our capital budget.
"We are doing all we can to facilitate the industry's recovery, boost supply and protect and create jobs through our National Housing Trust initiative and House-building Infrastructure Loan Fund.
"We are also helping households get onto, or move up the property ladder through our shared equity schemes and our support for the MI New Home scheme."