Alexander Mortimer, 29, was jailed for eight years after he admitted sexual assault and child porn charges. He had 17,967 images and 582 videos of child porn on his computer.
A Significant Case Review has revealed that bosses at the nursery in Rutherglen, South Lanarkshire, were alerted to "issues of discomfort" raised by several staff to internal managers about Mortimer's behaviour before the abuse emerged.
But they had never been discussed further with external managers or with Mortimer. There is no evidence of any abuse or filming having happened at the nursery.
A report into the review said there were concerns Mortimer had closed toilet doors when with a child, and that he did not use gloves when changing nappies. There was also a report of him blowing on a child's body.
But it was felt the response to the reports were "confused with assumptions that others were being discriminatory about male workers".
The report by an independent local Child Protection Committee said: "As noted (in other relevant reports), overall laxity in organisational cultures can mean abusers are aware they have more freedom.
"As these issues were not raised with external managers by the internal management team, it is not known what response would have ensued, nor what response would have been given if the workers who had expressed concern had gone over the heads of their own managers."
The report authors suggested national bodies should consider how difficult it is to 'whistle blow', that they should take account of "power dynamics" in these situations and of "fears of receiving a 'shoot the messenger' response".
The nursery worker began looking after the brothers - a two-year-old and a three-year-old - after befriending their mother and grandmother. He began to help looking after the boys and sometimes took them on trips.
Mortimer was caught after "intelligence" led police to raid his home and seize computer and phone equipment.
The review also found Mortimer's job interview was "very brief" at 10 minutes and was scored 'low' by the two interviewers, who nevertheless initially recommended appointment "with reservations".
Review contributors reflected that the time taken in the interview was "not sufficient to enable dialogue to take place on motivation in any depth".
The report, which refers to Mortimer as M, states: "During the police enquiries, and a series of support sessions set up for relatives and staff in the period following M's arrest, it emerged M had made regular efforts during the time of his employment there and in previous settings to befriend relatives and colleagues and thereby to gain access to children outside the nursery via activities such as babysitting.
"A number of parents had resisted these efforts. It was also noted by the police that around the time of his arrest he had made an enquiry to the council about the possibility of adoption."
South Lanarkshire Council welcomed the findings and noted that the great majority of parents continued to use the nursery after Mortimer's crimes came to light. The report noted that staff "continue to provide care which is valued and trusted".
Jim Gilhooly, the council's executive director of education resources, said: "We were keen that any lessons that could be learned were learned, not just locally but nationally. That is the best way for all of us as a society to protect our children.
"I am grateful the child protection committee has identified key strengths in our approach to this case. We have already moved to address the issues raised in the review's recommendations."