If faces tell stories, there are many tales to be told in Cornton Vale women's prison.

It is not just the narratives themselves that are shocking, but the ease with which they can be read without a word being spoken. The addicts, with their glassy stares and grey skin. The mentally ill, who sit curled, in a semi-foetal position, their heads in their hands, or staring into space as if there is something of deep significance in the void. The self-harmers, whose arms are decorated with an almost symmetrical, feathery pattern of scars, from wrist to shoulder, silently proclaiming their deep self-loathing to the world. It's obvious immediately that many of them shouldn't be here. Service provision for society's most vulnerable? It's called "prison".

The number of women in Scotland's jails has doubled in the last decade. Last June, Justice Minister Kenny MacAskill appointed an Independent Commission on Women Offenders, headed by the ex-Lord Advocate, Dame Elish Angiolini, to look at ways of reducing numbers and providing alternatives to custody. But the Sunday Herald understands the Commission has concluded that failures in the penal system are so systemic, a radical overhaul of Criminal Justice Services in Scotland, affecting men as well as women, is needed.

The Commission, which will report this week, will recommend establishing a new national agency to oversee criminal justice arrangements, and creating a network of local justice centres to provide community alternatives to prison. It will also suggest ending the £4.6 million payment the prison service currently makes to local authorities for social work services, and putting the money into a more integrated criminal justice social work service which addresses different needs such as housing. In a bid to get swift action, the report will recommend that MacAskill outlines his plans to parliament within six months of the report's publication.

Community alternatives are particularly important for female offenders who still make up just 5% of the prison population. While a small number are guilty of serious crime, many are there because society doesn't know where else to put them. Alternatives to prison for women would pose little public threat: while male prisoners are often among the most violent in society, female prisoners are among the most vulnerable.

Their back stories are remarkably similar and usually involve abuse, sexual violence, exploitation and addiction. An astonishing 80% of women in Cornton Vale have mental health problems with the majority serving short sentences for minor offences connected to drug and alcohol addiction. "We often think there's some nice, clean dichotomy between victims and offenders," says Angiolini. "Well I've worked with victims of crime for over 25 years – and they are the same people that I'm seeing in prisons."

Most people imagine prison to be like a scene from Porridge: traditional cell blocks with bare rooms. Cornton Vale looks more like a housing scheme – though with barred windows. Bruce House is the scheme's slum end. Dingy and depressing, it has two dilapidated old showers with specially adapted curtains to prevent suicide attempts, and bare rooms.

This is where the most chaotic prisoners live. Angie emerges instantly when she hears a visitor, launching into a tirade. "I have been living like a homeless person for three years. I have no wardrobe, no drawers and all my things are in a plastic bag. It's not right." She is shrill, insistent, confrontational. Angie might be alienating if you didn't feel a wave of despair, that is nothing to do with the absence of wardrobes, hit you. There are strange ringmarks around her neck, evidence of attempted self-harm. Angie is a young woman but her face tells a tale of addiction, her discoloured teeth falling together like collapsed tombstones.

Silently, women appear at doorways. They stand watching then join in the shouts, a cacophony of complaint. It is poignant the way, almost subconsciously, these "rebels" conform to a hierarchy that they lie at the bottom of. "Miss! Miss!" they shout instinctively, as if anyone who is free to walk in and out of the door, must be an authority figure who can "solve" their problems. They seem desperate for a voice. "This is the only place you will see these women," explains Paul, a prison officer with 18 years' experience in both male and female prisons. "They don't go to the same pubs or restaurants you or I would go to - they're a subculture." It's not judgment – just literal fact.

But what makes women offenders so different from men? "Women are much more difficult to deal with. More complex and demanding and more chaotic, particularly with regard to drink and drugs. I have never worked harder than I have here." Regimes in prison are very clearly geared to male behaviour and psychology, and prisons struggle to cope with the more time-consuming emotional support women require. Paul will instruct male prisoners to change cell without thinking about it. "I'll say, 'you're moving son'. Men tend to be career criminals and moving is just part of the deal. But women make friendships that are very important to them and when I've moved a female prisoner, I find myself driving home thinking, did I do the right thing?"

Women verbalise problems more but are also more likely to self-harm. "The feeling of worthlessness and self-loathing is much more intense in women than you get in men," notes Teresa Medhurst, Cornton Vale's governor. "When I walk round a male jail, prisoners come up sporadically. But women constantly want to tell you their problems. They are constantly looking for reassurance, affirmation, and reinforcement. If somebody has had their hair done or put make up on I'll say, oh you're looking lovely today. I wouldn't do that in a male prison."

Cornton Vale has been criticised for its physical conditions, its overcrowding, and the resulting limits on rehabilitation work. Angiolini's Commission will recommend a new prison. But this isn't just about buildings. Women's short sentences create a catch 22. They are usually addicts. But addiction cannot be addressed in a two-month period. They have mental-health problems. But prison officers are not psychiatric nurses. By the time women come out of prison, their problems have increased: any local authority housing has been lost and employment prospects are reduced. The vicious circle of offences, short sentences, more offences, continues at huge cost to the taxpayer, with 70% returning to prison within two years. "Some people think prison is a short, sharp shock," says Angiolini. "I think it has the opposite effect. I think it's inoculation. Once you have been there, even for a short period, you begin to accommodate it."

So why have prisons become baby-sitters for women who should be in detox units, training centres or psychiatric hospitals? Partly because we now have "care in the community" in a community which doesn't particularly care. "We used to live in communities," says Medhurst, "and they dealt with some of the behaviours we now see in prison. People are less tolerant now. They don't have time to spend and they don't want to. A lot of the support women used to have - close-knit families - neighbours popping in and out of the house -that doesn't happen now. Where do these women get support? As soon as you're labelled an offender, or you've been to prison, people don't want to know."

Here's the irony: women are society's carers except when it comes to themselves. Prison officers talk of young mothers, clutching toddlers, coming in daily to see their male partners. But women on society's margins are less acceptable than men. "Male prisoners usually have a mum, sister or partner who will take them in," says Medhurst. "For women, it's not the same."

In Wallace House, Mel says until recently, her mum didn't speak to her for three years. Wallace is a calmer, more modern, unit and Mel is sitting writing quietly in her room. She is tiny, with a lived-in face and dyed black hair with two vivid red streaks at the front. She smiles welcomingly. How long until her release? "Twelve years." She is in for murdering her partner.

Serious female crimes are rare but often have the same underlying themes. Mel's life was chaotic. She took drugs at 13, graduating later to heroin. Her relationships with men were abusive. When things became violent with the man she killed, it flicked a switch. "I just thought no way. I'm not going down that road again." When she discovered he was sleeping with other women, it sparked a two-day drug and vodka-fuelled argument before she snapped and stabbed him. She didn't, she says, mean to kill him.

Mel has worked hard, says Paul. She has been trained by the Samaritans to be a prison "listener". "If I can help somebody, give them advice, it feels good." Mel says prison programmes have taught her that actions have consequences – but few women stay long enough for prison programmes. How does she feel now about her crime? "I am disgusted with myself," she says quietly. "I've taken a father away from his kids. I have to be punished and I accept that."

But the most telling thing is when Mel is asked what she misses most in prison about her old life. She looks startled, then blank. "Nothing," she says finally. "It was shit."

Prison services for women have flashes of excellence: caring staff, well-respected initiatives like Edinburgh's Willow Project and Glasgow's 218 Centre which address the physical, emotional and practical needs of small numbers of women. But overall, provision is as chaotic as the women themselves. There is no national, strategic, coherent voice overseeing services, pulling together housing, social work, health and criminal justice needs. Scotland has a population of 5 million – smaller than some English regions, Dame Angiolini point out. Yet we have 32 local authorities, eight criminal justice boards, eight police forces, six sheriffdoms and 150 charities with an input into criminal justice services. "It is utterly fragmented," she admits. "There is a vacuum of leadership and it needs a strong driven voice."

The new community justice centres the Commission will propose would work closely with local sheriffs and provide a significant increase in support for women offenders. At present, men are usually housed in prisons near their homes; women are not. All female prisoners go through Cornton Vale, with a few being transferred to Saughton and Greenock. Women from outlying areas have little family support in prison, and are released without housing, social work, psychiatric or employment services set up at home. They are doomed to failure.

Two small re-integration projects set up at Aberdeen and Inverness prisons, catering for eight and six women respectively, illustrate the difference local initiatives make. Women nearing the end of their sentences work during the day and return in the evening but the units also address housing, health and social skill needs.

For Gordon Morrice, assistant governor at Inverness, working with women for the first time has been an eye-opener. Some arrive in a terrible physical state: dirty, covered in scabies, with bad teeth and nits in their hair. Some make themselves as sexually unattractive as possible to repel abuse.

"Often, their crimes are orchestrated by men," he explains. "A lot of it comes down to mental health and abuse issues – and again there are men in the background." Only in small units can these complex issues be addressed.

The unit operates like a house, where six women each have a room. Staff and prisoner relationships are critical. "Jenny, who runs this unit, is like family to me," says Rosie, a 34-year-old who has spent eight years of her adult life in prison on constant, short sentences. For Jenny, that's the reward of her job. "In the main prison you're just a key. You can't get into the guts of anybody to sort out their problems. It's a totally different relationship here."

Rosie is friendly and articulate but like many female offenders, exudes anxiety. She isn't violent. She isn't a threat. She's an ex-addict with 40 sentences for shoplifting. The system kept desperately prescribing prison for an ailment it never seemed to cure. Rosie first stole at nine when her father died and her mother was left with three children and no money. "I remember someone saying, 'you have a pencil case from the 99p shop'. That just stuck in my head and I stole a pencil case." Her father's death changed her family. "Everything dived. Dad was the love of my mum's life. I don't ever remember them arguing – not once. Mum didn't like talking about him and stealing was my way of dealing with things."

At 11, she was taking drugs. At 14, she was sent away to a secure unit in Glasgow. At 16, she was involved with an older, abusive man and serving her first sentence. When she got a new, supportive partner, she got clean. "People ask why but I think I just got exhausted. I had lived a thousand lives. I got to the point where I knew if I didn't, I would be dead." She stayed out of prison for eight years and completed two years of a sociology degree. What caused her to steal again? "Depression. I knew I had a lowness in me but I didn't recognise it."

Shoplifting became a release, like cutting for self-harmers. "The minute I do it, it's like a big deep sigh, a relief. It's not the actual taking, not what it is - it's just relief. At first, it was to feed my habit. Then it became a behaviour." Afterwards, the relief changes. "I hate myself for doing it. I wish I could stop." Rosie gets tearful with frustration. She has been told about a pill that can help obsessive compulsive behaviour. "I would take it in a heartbeat. Jail is no life."

Rosie longs for release but fears it, too. She knows she is lucky. She has a house, a supportive partner, and her job has been left open. Her Highland employer knows she's not a bad person. Until now, the penal system has been incapable of responding to her with that level of insight. But this unit has saved her. For the first time, she has been given access to a forensic psychologist who has diagnosed kleptomania. With 40 shoplifting convictions it's hardly surprising. Yet Rosie's gratitude for that label is overwhelming. Now she has a word. "People are so judgmental. I tried to explain before but people said what a load of baloney. Now I have a diagnosis and I can work hard with that."

Locks and bars aren't always an answer, argues Medhurst – for men or women. "As a society, to be locking up the numbers that we are is unacceptable. 8400 individuals every year? I'm sorry, I don't find that acceptable."

There are those who think more humane – and creative – treatment of offenders is just bleeding heart liberalism. But, says Angiolini, there's a business case too. The annual cost of keeping an adult in mainstream prison is around £32,000. Inverness's unit costs around £100,000 a year for six women and the normal re-offending rate of 70% has dropped to around 14%. That's the real bottom line.

"At the moment," says Angiolini, "we're locking people up only for them to come back out with the problems completely unsolved and unaddressed. There are better ways forward."