Michael Matheson claimed there had "obviously been very serious failings" in the way Mortonhall Crematorium in Edinburgh has been run.
He spoke out after a report yesterday said hundreds of parents whose babies had been cremated there faced a ''lifetime of uncertainty'' over what happened to their child's remains.
Over decades, staff at Edinburgh's Mortonhall Crematorium secretly disposed of the remains of stillborn and dead newborn babies without their families' knowledge.
Parents were instead told there would be no ashes left to scatter following cremation.
A year-long inquiry into the crematorium's practices from 1967 to 2011 concluded that the situation was a ''great tragedy'' which left many parents facing uncertainty about their baby's final resting place.
A 600-page report by former Lord Advocate Dame Elish Angiolini contains 22 recommendations for Edinburgh City Council, which operates Mortonhall Crematorium, and other agencies.
Mr Matheson said he hoped the "thoroughness" of the report would provide bereaved parents with some comfort.
He told BBC Radio Scotland's Good Morning Scotland programme: "For any parent it's difficult to imagine how it must feel to lose a child, but to then be put through the trauma this reports highlights is very challenging and it is very distressing.
"One element I can say that I hope some parents would take some reassurance from is the detail and the very thoroughness of this report that Dame Elish Angiolini has completed, which goes into the whole issue in great detail and illustrates what have been systemic failings at Mortonhall Crematorium for several decades now."
He added: "At Mortonhall Crematorium there appears to have been a culture which has been based upon practices established by certain members of staff over many years on the basis of how they wanted to run the crematorium in their own way.
"There have obviously been very serious failings in the way in which this crematorium has been managed over a long period of time and that's why it is important that the recommendations that Dame Elish outlined in her report are considered in great detail, so we can address the failings."
Mr Matheson stressed the need to "take action to prevent this type of thing of thing from ever happening again and ensure no parent is ever put through the trauma".
After the practices at Mortonhall came to light, some other local authorities, including Glasgow and Aberdeen City Councils, faced questions about their crematoriums.
A separate independent commission, led by former High Court judge Lord Bonomy, was set up to review policies and practice across Scotland in relation to the handling of ashes following the cremation of babies and infants. His report is expected to be released in the coming weeks.
Mr Matheson also said that the Scottish Government would introduce new legislation, if this was needed.
"We already have a legislative slot identified, if necessary, to take forward any legislative changes, which is highly likely," he said.
He also said that Dame Elish's report was being considered by both the Crown Office and the police, adding that it was "important they are given time to consider if there are any issues of criminality that should be prosecuted".
Mr Matheson said the Scottish Government had never "ruled out the possibility of a pubic inquiry" into the issue but added: "What is extremely important is that we have to make sure if there is to be any further investigation it works in a way that is able to offer some comfort to the families in being able to investigate individual cases, and a public inquiry would not actually do that.
"What we will do is consider what further measures are required once we have received Lord Bonomy's report in the coming weeks and look at addressing any concerns the parents may have."
Meanwhile Edinburgh City Council is to set up a special working group to produce an action plan in the wake of Dame Elish's report.
It is likely this will include representatives from both the council and the Scottish Government, NHS Lothian and the child bereavement charity Sands Lothians.
Council leader Andrew Burns said: "On behalf of the council, I want to offer profound apologies to parents for the distress and pain caused by the previous practices at Mortonhall. I also want to thank Dame Elish Angiolini and her team for their work on this investigation and all parents and witnesses who have contributed to the report."
He said he had asked council chief executive Sue Bruce to "put together a multi-agency working group, including parents' groups and other relevant agencies, to consider the report's recommendations".
He added: "The group will be tasked with a detailed action plan for consideration at next month's special meeting of council. We must act on the recommendations to ensure that the highest possible standards are adhered to at Mortonhall and that nothing like this can happen again."