IT was the 11-year-old daughter of a prisoner who pinpointed the Holy Grail for criminologists.

She explained that the critical factor in ending her parent's cycle of reoffending is quite simple: hope. She is one of hundreds affected by crime in Scotland who are taking part in innovative new research trying to unravel how to stop our most persistent offenders from reoffending.

The interim findings of the project, which will eventually help define Scottish Government policy, say that the way to stop people committing crime lies in giving offenders hope, helping them contribute to society, making better use of ex-offender mentors and supporting relatives who are trying to help them reform.

The results to date argue for greater consistency in supervision services across the country and an end to prison sentences of less than one year. It suggests the rehabilitation of offenders needs to be significantly changed to focus less on 'fixing offenders' and more on repairing their relationships with families and communities.

As part of the project, academics worked with ex-offenders, social workers and professional film makers from across the UK and the US to produce a documentary exploring how and why some people stop offending, and how criminal justice can get better at supporting the process.

Presented by Allan Weaver, a Scottish ex-prisoner turned social worker, the documentary echoes Ken Loach's recent box office hit, The Angel's Share, which shows how offenders can escape a life of crime and violence.

The film has been used as a platform for workshop discussions across Scotland, England and Northern Ireland with offenders, ex-offenders, prisoners' families, social workers and prison governors to look at what will effectively address recidivism. It will be shown in other countries including Australia and previewed to the public tomorrow. Justice Secretary Kenny MacAskill will be introducing it.

The film starts in the west of Scotland with Weaver, author of So You Think You Know Me, an autobiography examining his own pathway into crime and punishment – and discovering the varying fates of some of his former co-offenders. It shows how individuals get caught up in cycles of crime, as well as how the "revolving door at the prison gate" can be avoided so that people move on with their lives and make a positive contribution to their families and communities.

Through the interviews in the film, Weaver learns that real change involves processes of self-discovery and support. Ex-offenders talk in detail about people who helped them find hope or gave them the confidence to change.

Weaver started offending at 12. He has served five prison sentences – the last for a charge of assault to severe injury. He was released 30 years ago. In the film he refers to jail as a "crime factory" where he learnt how to break into cars and make a weapon from a razor blade and a toothbrush. He also highlights the fact that for many the real punishment begins on release.

"Even though I was getting into trouble on a regular basis when I was young and was in and out of prison and care homes, part of me never gave myself over to that fully," he told the Sunday Herald. "I was born into that life and felt I didn't have many options. I grew up in a gang and got caught up in that west coast of Scotland hardman image.

"During my last sentence I worked in the gardens with an older guy serving a life sentence. He talked about his wasted life. He'd been involved in organised crime – something which appealed to me when I was younger. But he showed me a different side of it.

"The film addresses the impact of hope. When I got out that last time I had hope I could change. My social worker had a lot of faith in me and that helped me build a sense of self-belief. She got me involved in voluntary work and then helped me to get into a social work course. If we want to stop people offending we have to give them hope. I'm proud to say I still meet up with my social worker."

He added: "Labelling and stigmatising offenders operates as a barrier to desistance from crime. A key message that came through from all the people I interviewed in this film was that we need to encourage a more inclusive and collaborative approach to supporting change, and to achieve this we need to actively involve and listen to those who have travelled this path. To encourage and support the sometimes difficult process of change, people need to be given both the opportunities to change the direction of their lives and the hope that they can be something or someone different."

Despite pledges by successive governments to reduce the escalating prison population, Scotland now has record numbers behind bars: more than 8000. Of the 7000 prisoners released in 2008-09, almost half had been reconvicted within a year.

Official statistics show that those given a community sentence are less likely to be reconvicted within two years than those given a prison sentence.

A Scottish Government pledge to reduce reoffending by 2% by 2011 was met, but the country still has one of the highest reconviction rates in Europe. particularly for shoplifting, housebreaking and prostitution.

Fergus McNeill, professor of criminology and social work at Glasgow University and one of the academics on the project, said: "The film makes it obvious that, unless we learn from those that have travelled 'the road from crime', we won't be able to help others along it. It is about how people help people seize opportunities for change and whether or not we believe they can change. At the end of the project we'll be making recommendations to ministers, especially about how supervision should be constituted. The film comes out ahead of the whole process being completed. The research continues and will form the basis of the recommendations.

"The project indicates that we need to reconfigure criminal justice social work supervision so that it challenges inequality and is consistent across the country, is holistic, includes a skills, volunteering or educational component to help people develop their abilities to make a positive contribution in communities, and makes better use of mentors who are themselves reformed offenders."

Professor McNeill added: "For decades, criminological researchers have been engaged in their equivalent of the alchemist's quest – searching for the elusive answer to the question 'what works to reduce reoffending?'. This film starts with a different and broader question –'How and why do people go straight?' – and shows that if we were better at listening to people involved in this challenging journey, and to those that have successfully navigated it, we could get much better at helping people along it."

The film is the brainchild of Fergus McNeill, Steve Farrall of the University of Sheffield, Claire Lightowler of the Institute for Research and Innovation in Social Services and Shadd Maruna of Queen's University Belfast who worked with ex-offenders, practitioners, policymakers and film-makers, to produce the documentary.

'I came out of prison to a strong family ... most people don't even have anywhere to stay'

PHIL Bradley, 45, says he drifted into crime. In 2003 he was sent to prison for nine years for armed robbery and possession of firearms. While inside, he kept his head down. He trained as a literacy tutor and a Samaritan.

Bradley (pictured above) is one of a number of ex-offenders involved in the research project and workshops that have looked at the film Road From Crime, and the best way forward for practitioners and policy-makers.

Released more than a year ago, he is now working to help support prisoners' families and mentor inmates.

"When I was arrested I knew I was going to plead guilty," he says. "In prison I learnt from helping other people. It was the only thing that made sense. My kids and family were so deeply affected by me being in prison. I swore I would never reoffend.

"Prison is not physically hard but the confinement is really hard and it's insanely boring. I read a lot and wrote a lot. I was so shocked by the illiteracy rates in prison. I wanted to help people to learn to read. Crime rarely happens in a vacuum. Not being able to read or write bars people from employment.

"I came out to a strong family, structure, voluntary work and the chance to change. Most people don't have those supports and don't even have anywhere to stay.

"I think the single most important thing is to ensure there is a mentor there to meet them at the gate. That first 24 hours is critical. The system, as it is, is just not coping."

The steps to a life free from crime ... why are we not taking them?

By Professor Fergus McNeill

ONE of the major drivers behind the government's "Rehabilitation Revolution" is the cost of reoffending by ex-prisoners, estimated at £7-10 billion per year. Behind these numbers lie the human and social costs of both crime and punishment.

Why is there so much reoffending by ex-prisoners? Imagine yourself trying to shift the established pattern of your life (think, for example, of making the transition to parenthood) in the context of insecure housing, poor health, substance use problems, inadequate benefits and no obvious means of making a living.

Add into the mix your own accumulated anger, despair and frustration, and the stigma and rejection that come with the label 'criminal'. Suddenly, reoffending starts to look not just understandable but inevitable.

And yet, reoffending isn't inevitable. Most persistent offenders do stop offending, but how and why do they eventually stop? The research points to three inter-related influences. Firstly, they get older and become more mature. Secondly, they develop the sorts of social "ties that bind", both in the sense of constraining and of healing, giving them an investment in and some of the resources for going straight. Thirdly, they develop a new and more positive form of identity – leaving the label "offender" behind and replacing it with something better; perhaps worker, parent or volunteer.

Contrast this with the kind of punishment for which we Scots seem to have an unhealthy predilection: imprisonment. Prisons are not places where maturation is enabled; by their nature they take responsibility away. They are not places where positive social ties are formed; they tend to sever or weaken such ties and sometimes to develop anti-social ones. They are not often places where new identities are forged; by its nature imprisonment tends to confirm and cement negative identities. If we want to reduce reoffending, we might have to moderate our appetite for imprisonment.

Equally, we need to rethink rehabilitation. One of the most disturbing moments in our film features an ex-prisoner who has done everything conceivable to rehabilitate himself, even acquiring a degree, but who hasn't been able to secure a job in the last eight years and who was homeless when we filmed him.

A lot of money has been invested over the years in interventions that aim to sort offenders out. But though these programmes can and do support change, they can never be enough, because supporting desistance from crime requires other sorts of change too – changes that extend far beyond the reach even of the criminal justice system.

Every arm of Government, every aspect of civil society, every community needs to play a part in making reintegration real.