Laura McGilp's explanation of the benefits of the flagship Triple P parenting programme is exactly the one health chiefs want to hear.

The often-traumatic trip to the shops with a young child is no longer a challenge, she says.

"I no longer come out of the supermarket dripping with sweat, and wanting to wring someone's neck or run them over later in the car park," the single mum explains.

Loading article content

Her son Spencer, five, is 16 years younger than his older sister. When her daughter was younger, Laura had no idea about the importance of routines, rules and boundaries, she says. Equally, she didn't know how to listen to her child or engage her in play. With Spencer, since she began taking part in Triple P classes two years ago, everything has been different, she says. "I love Triple P, it works for me every day. If you stick to the rules, it is foolproof," she says.

The Triple P approach – it stands for Positive Parenting Programme – is one NHS Greater Glasgow and Clyde, and its partners, are tremendously keen to see work.

City planning chiefs have invested more than £2 million to buy in the system, developed in Australia, and to gather evidence of its impact on parenting, education, inequality and other social problems.

However that evidence-gathering has so far proved challenging. A report earlier this month on the first year's progress revealed that researchers based at the University of Glasgow were having significant problems showing whether or not it is having an impact.

The most intensive support offered through the programme is described as level four, and involves group sessions like those Laura has taken part in – with parents attending eight sessions where they learn about their child's needs, strategies for dealing with problem behaviour and other issues like how to form a loving attachment with their child or how to play with them, if parents struggle with that.

Although parents were "largely satisfied" with the intervention, lead author Louise Marryat reported, the completion of questionnaires designed to measure the parents' situations before and after the group interventions were poor. In the first year fewer than half (46.8%) of families taking part provided "pre-intervention" data and even fewer (37%) gave researchers "post-intervention" evidence.

The result, simply put, is that Glasgow can't say how many people completed the courses, how many benefited, and whether the lack of completion of questionnaires is because they just didn't finish courses, because they didn't gain from them, or because they couldn't really engage with the booklets. (There are five, asking for assessments of any change in areas of their life including their child's behaviour, their personal relationships and how they felt about themselves).

Meanwhile some existing staff (quietly) and former staff, less discreetly, have been critical of aspects of the approach.

They question whether it is reaching the right people and whether a programme developed in Australia, which talks about cubbyhouses (wendy houses) and keeping children away from the pool is appropriate for Glasgow.

Dr Jackie Kirkham, a former health visitor in the city who now works for Edinburgh University, responded to the Marryat report with a blog outlining a range of criticisms based on her experience of being asked to implement Triple P. "I felt that I had no choice but to offer families a Triple P intervention, regardless of whether I thought it was what they needed, because of the pressure (and there really was pressure) to meet targets and figures," she said. "When we complained about pressure to deliver Triple P we were basically told that as they had spent so much money bringing it in, tough, we had to do it."

She, and others, have expressed doubts about the evidence base behind Triple P, which boasts of success in many countries, but often citing academic papers co-authored by one of the creators of the programme, Dr Matt Sanders.

These criticisms are met head on by Glasgow's director of public health Dr Linda De Caestecker. "We know this works in a research setting," she insists. "I'm confident in that research but there's a lot of learning involved in how to implement these programmes at a population level."

She points out that this is the first of five annual reports as the scheme moves towards a target of reaching some 57,000 families across the city. To date it has worked with around 21,000, at the time data for the report was gathered the figure was 12,818, with only 995 having received the more intensive group work. Some 9000 of those registered as receiving Triple P had merely attended a seminar when enrolling a child into primary one. "This is a phased report, and it is less about saying is it working or is it not working," Dr De Caestecker says, although she points out that those for whom data is known did benefit. "But we do know we need improve data collection."

She believes staff need time to buy into a change in styles of working to gather the information necessary to demonstrate the worth of the programme. A whole field of research "implementation science" is looking at how best to make such initiatives work in the field, she says. "What the implementation science people would say is 'get started, then get better'. Health visitors are taking on a different way of working and that takes time."

Criticisms from Dr Kirkham's blog included the rigidity with which Triple P is implemented, with research focusing only on parents' own evaluations of any change in their child's behaviour and the programme's owner company refusing to sanction translations of the materials .

Dr De Caestecker says the tip sheets are now being translated into Urdu, but Triple P's parent company are right to insist that this is faithful to the original programme. "Flexibility is important, but so is fidelity," she says, adding: "It is important, if a parent says 'I'm better able to manage and my child's behaviour has improved'

"But we are also looking at teacher evaluations and other routinely collected data." This includes numbers of children taken into care or put on child protection measures, she adds. "If health visitors or other staff are trained in another evidence-based programme, they should use it, but otherwise we are saying that in this part of their work, we want them to use Triple P as the main approach."

Assembling results from the thin year one data made the first report overdue. The second will be out in a matter of months, allowing a fresh assessment of progress.

In the meantime, renewed efforts are being made to find the best way for parents to engage with the programme, such as by telephone and in a new initiative via an online course.

"I'm not surprised we are where we are," says Dr De Caestecker. "But the answer can't be to lose confidence in our parenting support, because that's what we believe parents need."