Mr Anderson, addressing a London audience packed with fellow fund managers, also lambasted short-termism in investment management and the wider business world.
He warned his audience that their fees and pay will have to fall.
"I think we have attracted into the industry people who are there because of these rewards," he told the Association of British Insurers' annual investment conference.
He said that a cut in fees would improve returns to savers and also help in "attracting the right sort of people to the industry in the first place", those who were interested in overseeing the companies they invest in.
Mr Anderson has a strong long-term record as manager of the £2.4 billion Scottish Mortgage investment trust.
Baillie Gifford does not disclose how much its fund managers are paid, although its partners are widely regarded as well-rewarded through stakes in the business worth millions of pounds.
Industry surveys show that an average fund manager with 10 years of experience can expect a salary of £110,000 to £150,000, plus discretionary bonuses that can be double that in a good year.
Mr Anderson was scathing about the way the investment industry has performed.
"We have lost sight that we exist for the good of companies and savers not for our own bank balances," he said.
"That has allowed us to make profound structural mistakes in the way our companies and savers have been served or not served over the last decade."
He added: "I am profoundly depressed that we have not done a better job."
Mr Anderson linked the perceived failings of the funds industry to a wider corporate malaise, claiming there were only five companies in the UK – which he did not name – that he regarded as well run.
He attacked the short-termism of incentives that led to errors by "my wonderful colleagues in the Scottish banking industry".
Mr Anderson, former chief investment officer at Baillie Gifford, also criticised the lack of focus in the pharmaceutical industry as to whether its products were useful, arguing that the industry judged the impact of its research output in time-frames that were frivolous.
He said of the energy industry: "Do I believe any of our major oil companies has been well run? I do not believe that.
"I believe it is fundamentally connected to the dominance of short-term investment and pursuit of incentives within them."
But Mr Anderson insisted that executives in the technology sector were profoundly serious about taking a long-term view.
Elizabeth Corley, chief executive of Allianz Global Investors, agreed that the funds industry must shake up fees and pay.
"You cannot live in a low-return world, a single-digit world - without scrutiny of remuneration," she said. Ms Corley also called for a sensible discussion about pay and said she had long been expecting a fall in fees that could lead to the disappearance of some firms.
"If we do not get a shake-up and survival of the best, we will not get a more healthy industry," she said.
Ms Corley said there was a real need for humility in the financial services after the credit crisis.
She added: "Global capital markets failed and the taxpayer paid.
"One has to accept a political and policy consequence to that. We have to assume there is still an awful lot of blame out there."
Contextual targeting label: