The annual report of the company, which runs Argyll Ferries and Caledonian MacBrayne, said total passenger numbers declined from 5.2 million to 4.8 million with poor weather cited as having an adverse impact.
The financial statement details a £40,000 payment to former chief executive Archie Robertson for loss of office after he stepped down in August last year.
Mr Robertson also received a salary of £136,000, a performance payment of £28,000, pension contributions of £28,000 and benefits with a value of £12,000.
The accounts for the 12 months to March 31 show David MacBrayne posting pre-tax profits of £20.4m, down from £4.1m in the prior year. The document also shows the company brought in £70.8m from fares and other income while it also got more than £86m of grants from the Scottish Government.
That compares to fares of £85.6m and net grants of around £116m in the previous financial year.
The Northlink service, which goes to Orkney and Shetland and is now run by Serco after it secured a six-year contract worth £243m in May last year, brought in only £20.2m of revenue and £325,000 of operating profit compared to almost £70m and £1.3m of operating profit.
The David MacBrayne workforce fell from 1667 to 1374, according to the annual report.
Writing in the accounts, chairman David McGibbon said: "The Gourock and Dunoon service, operated by Argyll Ferries, continues to be challenging, with campaigners seeking the return of a car ferry to the routes. Following a very tough winter, the reliability of the service was hit hard."
A spokesman for the group said trading since the year-end had been mixed, with extremely busy periods at the height of the summer weather along the Clyde contrasting with a relatively static performance on more northerly ferries.
Mr McGibbon confirmed that several new initiatives are being launched in the coming months.
He said: "Work is progressing to improve our customers' experience of our services, including the revamp of our ticketing and online booking system and introduction of wi-fi across the fleet. Our rolling programme of boat replacement also continues, with the MV Loch Seaforth on schedule for the Ullapool-Stornoway route next year and our two new innovative diesel hybrids MV Hallaig and MV Lochinvar also due to enter service in the coming months."