Calls to the premium-rate line are "a hideous waste of money" at a time of tight budgets and a proliferation of clocks on phones and computers, according to the Conservative Party, which obtained the figures.
Councils are the biggest spenders, making £12,947 of calls since 2008/09, health boards have amassed £1551, and the Scottish Government spent £30.
Total speaking clock call charges have increased sixfold in the last five years from £852.79 in 2008/09 to £5436 in 2012/13, despite the growing availability of accurate digital sources.
North Ayrshire was the biggest spending council (£2206), while NHS Lanarkshire spent the most among health boards.
The Tories have called for the 31 pence-per-call number to be blocked on public-sector phones, amid suggestions staff may be using it as "a distraction tactic when the boss is around".
Conservative chief whip John Lamont said: "The total amount may be small in comparison to overall public budgets, but it's still a hideous waste of money.
"It beggars belief more councils and health boards have not put a block on this.
"If staff are calling 123 as a distraction tactic when the boss is around, that's even more inexcusable."
He added: "This waste of taxpayers' money is the sort of thing that infuriates the public."