he first time I saw Nick Cave in the flesh, it almost knocked me sideways.

There he was – a tall, ghostly figure with a hangdog face and a handlebar moustache. He was in Venice to collect the inaugural Gucci prize for writing the script for 2005's Australian western The Proposition. Surrounded by models, in an impossibly plush palazzo by the Grand Canal, the mercurial singer-songwriter-poet seemed entirely at ease in the midst of the balmy Italian chaos.

Six years on, and he's still surrounded by chaos – this time of the French sort. He's in Cannes, accompanying Lawless, his latest screenwriting collaboration with The Proposition's director John Hillcoat. Herded in front of photographers, in a ground-floor hotel ballroom, Cave poses for the camera with his director and several of the Lawless cast members, like brothers in arms. He's even more arresting than I remember him, with his caterpillar-thick eyebrows and receding hair, dyed jet-black, dominating that pale, elongated face.

Dressed in a pale-blue shirt and a navy suit with a distinct yellow pinstripe, he even has a touch of bling – a gold wrist watch. While it would be wrong to call him ageless – he turns 55 this month – when he removes his sunglasses to reveal blue eyes there's something oddly vampiric about him.

Right back to his breakthrough band The Birthday Party, on whose 1981 single Release The Bats he sang of "vampire sex", Cave has always been interested in the imagery of American gothic along with the fire and brimstone of the Old Testament.

Influenced by such disparate artists as the Sex Pistols, John Lee Hooker, the Velvet Underground and Leonard Cohen, throughout the 14 albums with his follow-up band the Bad Seeds – their most recent LP, Dig, Lazarus, Dig!!!, came out in 2008 – Cave's lyrics have obsessed on such themes as religion, death, violence and love. (It's a delicious thought that one such outpouring made it on to the soundtrack of Harry Potter And The Deathly Hallows: Part 1, the melancholy of O Children accompanying a dance between Harry and Hermione.)

Lawless too feels like a melting pot for Cave's interests. A 1920s Prohibition drama about a trio of real-life Virginia bootlegging brothers, the Bondurants, it has been freely adapted from the book by their descendant Matt Bondurant into a quasi-mythical blood-splattered fable. "It's not that violent a film," Cave protests. "It's not endless. It's not like The Expendables, which is way more violent. I think what affects people is you actually feel something. When you see someone get punched, you feel something."

This is true. Featuring Shia LaBeouf, Tom Hardy and Jason Clarke as the Bondurant siblings, the violence takes off when a psychotic FBI agent (Guy Pearce) is brought in from outside their county to put a stop to their illegal moonshine trade. While the violence isn't "endless" as Cave puts it, it's most certainly sickening – one character is castrated, another tarred and feathered. You might not see much, but the images conjured by Cave and Hillcoat fester in your mind long after you leave the cinema.

"I'm interested in our capacity for violence," Cave says. "We live in civilised societies where we live civilised lives, all of us. We can all sit down and communicate with each other but inside each one of us is the potential for untold violence and evil, given the right or the wrong situation."

It's not all violence – what with LaBeouf's wooing of a young Mennonite choir girl (Mia Wasikowska). Cave loves the "calm" of the film, "these moments of tenderness and civilised relationships". But what really gets him is this notion that "raging inside of everybody is this infinite capacity for violence". Is it inside him? He nods, slowly. "I don't have any problem with it. It's there. I understand it's there. I understand it's in everybody. You learn how to deal with that or not."

A look back at his childhood shows just where those fires were first lit. Like the characters in Lawless, Cave was born in a rural environment – Warracknabeal, a country town nearly 200 miles from Melbourne. With two older brothers, Tim and Peter, and one younger sister, Julie, his was a cultured upbringing; his father Colin was an English teacher at the local school and his mother Dawn its librarian. The young Cave began singing in a boys' choir at Wangaratta Cathedral but as he grew older he became disruptive and antisocial. Packed off to boarding school, Cave became a ringleader for a small gang of misfits. One story that has taken on mythic status has it that his band of outsiders dressed up in drag, attacking anyone that came near with their handbags – loaded with bricks. By the time Cave's father died in a car accident, when his son was 19, he was even more lost. He discovered his father had died as his mother was bailing him out of a Melbourne police station for burglary.

Shortly afterwards he went to art college but dropped out a year later to concentrate on music. It was around this time he met Hillcoat. Still a teenager, Hillcoat was making short films while Cave, some four years older, was in The Boys Next Door. As their friendship evolved, they began working together – on videos, documentaries and, crucially, Hillcoat's feature films. "It's been a very close relationship," says Cave. "We're very, very good friends. We socialise. Our children play together."

Their first collaboration was Hillcoat's debut, the 1988 hard-hitting prison film Ghosts - Of The Civil Dead. Cave, who contributed to the soundtrack and makes an appearance in the film as a psychotic inmate, was one of five credited writers.

It was a fertile if fraught time for Cave, who had formed his partnership with the Bad Seeds in 1984 (recording their first four albums in just two years) and even appearing with them in Wim Wenders's 1987 film Wings Of Desire. At the same time, he published King Ink, a collection of lyrics and plays, and was working on his debut novel, And The Ass Saw The Angel, a slab of southern gothic about an outcast mute boy for which Cave "plagiarised" the Bible – the one book he had by his side at all times.

Finally published in 1989, And The Ass - came after years of marathon drug-fuelled writing sessions in his dank Berlin apartment. Overblown urban myth had it that he wrote it using his own blood when he couldn't afford ink. The truth was he was strung out on heroin for much of this time. A user since the late 1970s, he was arrested in 1988 for possession, and it was the catalyst he needed. Agreeing to clean up his act to avoid prison, he was good to his word; shortly afterwards, he began a relationship with Brazilian journalist Viviane Carneiro, who gave birth to Cave's son, Luke, in 1991. He has another grown-up son, Jethro, in Australia.

By the time he reunited with Hillcoat to score his second film, 1996's To Have & To Hold, Cave had managed a further five albums with the Bad Seeds, including Murder Ballads, which sent shockwaves through the pop world when he teamed up with Kylie Minogue for Where The Wild Roses Grow (recounting the story of a man courting a woman then killing her). The song gave him his most recognisable hit to date – cementing his transition from alt-rock junkie to elder statesman.

All the while, Cave was ruminating on The Proposition. Ever since Hillcoat had left film school, he'd been talking about getting an Australian western off the ground. "He just wasn't getting the script together," says Cave. "I would read them – because I was his friend – and I would say, 'These scripts are s***.' In the end he goes, 'Well, you write one then.' And I said, 'All right.'" Cave finally put pen to paper around the time he was recording No More Shall We Part, the 11th Bad Seeds album, in 2001.

"I'd explored different territories and then I put it to him to give it a go," remembers Hillcoat. "He was scared that the format of scriptwriting would be too foreign to him. I knew he was good with characters, and his songs are narrative based. But I gave him all the research I'd done, and asked him to come up with the story and characters, and the rest we'd hand over to a professional screenwriter. But once he started, he took off like a runaway train."

The resulting film, an Outback western set in the 1880s, feels like a forerunner to Lawless, with its story of three murderous brothers – something Cave calls "largely coincidental", even though he comes from a family with the same male dynamic. Winning numerous prizes at the Australian Film Institute awards, The Proposition remains a highpoint for Cave, who also composed the soundtrack with his long-time collaborator and fellow Bad Seed Warren Ellis (they returned to the genre two years later, scoring The Assassination Of Jesse James By The Coward Robert Ford).

Cave sees songwriting as "much more abstract and obscure" than screenwriting. "Finding the song," he says, "is like running around after a tiny little butterfly with a net with big holes in it. You never feel you can catch hold of the thing. That's the beauty to me of music; it's so easily broken. It's really difficult to write a song. A script is much more a craft. You're just telling a story, in as entertaining a way as possible. It's not much different from making up a story to your kids at night."

The talk turns to Australia, a country he seems ambivalent about. He left Melbourne in the early 1980s, ostensibly heading for the music scenes of London and Berlin, but there were other reasons. "Australia is a massively repressed society – ultra-conservative," he says. "[There is] a general fear of becoming anything different from anybody else. There's a beautiful aspect to the Australian character – outgoing, fun-loving – but there's also a dark side."

Is he disappointed by his homeland? "I'm not disappointed by Australia. I'd just had enough of a kind of attitude there that didn't like anyone getting too big. I love Australia – I just don't want to live there." It matters not. Along with Kylie, he was named Australian of the Century by one magazine, while he was inducted into the Australian record industry hall of fame (though controversially without the Bad Seeds), the chairman claiming he was "an Australian artist beyond comparison, beyond genre, beyond dispute".

If he still speaks with a mild Australian twang, there is something cosmopolitan about Cave. Take his romantic life. After a brief romance with singer PJ Harvey – a love affair and break-up that inspired him to write his 10th Bad Seeds album, 1997's The Boatman's Call – Cave met Susie Bick. A former Vivienne Westwood model (and cover star of The Damned's album Phantasmagoria), she caught Cave's eye at a fashion show at the Natural History Museum in London – while standing under the tail of the brachiosaurus.

"I thought she was the most beautiful woman I had ever seen in my life," he later said, "and she continues to be. Even elbow-deep in baby s*** she looks pretty good to me." Bick gave birth to their twin sons Arthur and Earl in 2000, a year after she and Cave married. For a while, they lived on a houseboat in Hove, on the East Sussex coast. Now they live close by in Brighton, near Hillcoat. All of a sudden the picture of domesticity, Cave's well aware of the effect this has had on his output – though it's not what you might think.

"The calm records usually come out of the most turbulent periods," he says. "With music, you tend to write the songs you need. And now I'm a happily married man with children, I just sit there and write about bloody violence the whole time. Maybe it's just your imagination goes that way." Does he worry about it getting boring? "I don't necessarily think that that's boring, I just see it as doing my work – and I've always seen it like that, no matter what's been going on in my life. I've always sat down at a desk and worked. I'm a working stiff."

Not everything has worked out, however. He was asked by Russell Crowe to write a sequel to Gladiator. The studio rejected it for being too surreal. And he and Hillcoat are still struggling with Death Of A Ladies' Man, a script Cave wrote that morphed into a three-part TV series – with Ray Winstone lined up for the lead – but was rejected by Channel 4.

"John doesn't want to do it for some reason," Cave notes, a little sarcastically. "He just wants to make big Hollywood movies, and this is a little English film."

It's why, when asked if he plans to write a script about the horrors of the music industry, he says he'd prefer to turn his attention towards the film business. "That's where the real cannibals are. The music industry is relatively tame – you just go into the studio, get f***** up, make a record and put it out. Why would you want to make a movie about that? There are much more interesting dynamics going on in the film world. There are more people involved, more interesting characters; more drama and heartache and heartbreak and triumph and failure."

Having ultimately developed his Ladies' Man script into the plot for his second novel, 2009's The Death Of Bunny Monro, with its story about a womanising door-to-door salesman, such rejection doesn't seem to bother Cave.

"The funny thing about it is that you're usually on to something else anyway," he says. "By the time someone sends something back and says they don't like it, it's like, 'So what?' You always feel engaged in the present thing that you're working on. And for me personally – I'm sure it's the same with most artists – it's that thing that is of prime importance and nothing else really matters."

Even now, in Cannes, the fate of Lawless is fading into insignificance for him as he looks forward. "I'm making a new record and now I'm thinking I'm making the greatest record ever made," he says, "but the fate of this record will become a secondary thing in six months' time because you're working on something else." While his recent side project Grinderman is no more – the band lasted two albums, breaking up in 2009 – Cave is rather coy about his next musical endeavour. What can he say? "Nothing," he grins. "I'm just writing it. It's great, though." You can bet on that. n

Lawless (18) opens on Friday.