It's safe to say that British-born, New York-based novelist Patrick McGrath is dark in his concerns – safe because the description was applied to him by Hilary Mantel, who knows a thing or two about writing, and because the man himself agrees with every word of it.

"It's not a thing one can do anything about really," he tells me while sitting in his Manhattan home. "You write the novels that you have to write and, yes, my concerns have always been dark. I'm interested in the disturbed mind, in people who have been negatively affected by their experience – by trauma or by their family environment – and I'm interested in putting them into situations of crisis and seeing what happens."

The 63-year-old will also cop to "gothic" as a description of his work, though not in what he calls the "classical" sense.

"Gothic literature has always foregrounded ideas of transgression and decay and psychological phenomena, like the doubling of individuals. These have all crept into my work, and I haven't really resisted them. So I think thematically I'm gothic, even if I don't deal in a lot of thunderstorms and crumbling castles in the dead of night."

Castles, no, but in new novel Constance there is the next best thing: an Addams Family-style pile in upstate New York. McGrath based this on Wilderstein, a noted Queen Anne-style country house in the same state which dates from the mid-19th century and has interiors by a member of the Tiffany jewellery and design clan. In Constance, the house is renamed Ravenswood and is inhabited by a cold, patrician doctor who is father to two motherless girls – the titular Constance, emotionally neglected as a child and one of the novel's two narrators, and her fiery younger sister, Iris, clearly the old man's favourite.

The novel's setting is split between Ravenswood and New York City, the year is 1963 or thereabouts. McGrath never actually states it, instead leaving the reader to work it out from the fact that work has just begun on the demolition of New York's iconic Penn Station. It's the era of Mad Men, in other words, so the Big Apple is both exciting and gloomily run-down. The people who live in it – or the characters McGrath puts in it – are brittle and troubled, their relationships tinged with hate and regret. And they drink too much.

"I began with a picture of a difficult woman, a rather unsympathetic and unlikeable woman, a woman of some bitterness," McGrath says, "and yet with a certain fascination, a certain beauty, a certain aloof manner that to some men is terribly attractive. So I thought I would have somebody hone in on her, not knowing that they were stepping into a minefield."

That somebody is English-born poetry professor Sidney Klein, the novel's other narrator. Two decades older than Constance, he meets her at a party, woos her, marries her and moves her into his dark but roomy book-lined apartment. As she is buffeted by subsequent events – an earth-shattering revelation from her father, her sister's slide into dissolution and depression – he puts his side of the story and offers his take on events. It's classic McGrath territory: couples, siblings and families in crisis, on the edge of it, or dealing with its aftermath, their actions fuelled by the psychological breakdown of one or more of their number. Dark concerns, indeed.

For those who like to find a causal link between a writer's background and their literary preoccupations, Patrick McGrath is among the finest case studies going. His father, Dr Pat McGrath, was a Glaswegian forensic psychiatrist whose job from 1957 was as medical superintendent at Broadmoor Lunatic Asylum, later Broadmoor Hospital. His son was seven when the family moved to a red brick Victorian villa in the grounds. He was eight when, in the summer of 1958, Kray gang member Frank "The Mad Axeman" Mitchell escaped over the wall, and 12 in 1962 when serial killer Graham Young, the "Teacup Poisoner", arrived after his conviction for murdering his stepmother.

"I can't underestimate the influence my father had on me," McGrath admits. "When I began to write, I found myself very rapidly moving towards the sorts of thoughts and themes that intrigued and fascinated him. Because he had talked about his work when I was really very small, my imagination, when I began to employ it as a fiction writer, was, to my astonishment, very well stocked with stories, ideas and characters. They then went straight into my writing."

One Broadmoor visitor who has only recently become notorious is Jimmy Savile. Among the many allegations made against the late TV personality is that he abused inmates at the secure hospital which he visited often as a voluntary worker.

McGrath met Savile a few times, but says: "I just thought he was a bit of a twit. I was too old to be in any way dazzled or charmed by him. He was just weird. His personality was identical to what it was on television. So there he was with a big cigar and this weird hair, shouting his head off in this most unlikely setting. It was just slightly grotesque."

His father, however, had many more dealings with Savile. "He disliked him intensely," says McGrath. "He thought he was an awful man and I do think he might also have suspected he was a less than wholesome man. But he tolerated him because he was good for the hospital, which needed all the good press it could get."

Predictably, McGrath has revisited these memories in the months since Savile's actions first came to light. "The first feeling was, 'Did anything go amiss with Savile with Broadmoor, and if so what and was it during my father's time there?'" That said, the writer still finds it difficult to believe there could have been abuse at the secure hospital, given the nature of the place. "It is a top security institution - It was very hard to imagine that you could get away with anything that Savile was getting away with in other institutions, in Broadmoor."

McGrath's unusual childhood environment and the imaginative fuel he gathered there found its most blatant expression in Spider, published in 1990, and Asylum, from 1996. In the first, a man – Spider – is released from a psychiatric institution into the care of a half-way house just a few streets away from where he grew up. As the novel unfolds, Spider tells his story. Asylum, meanwhile, turns on the affair between the wife of a prison psychiatrist and a sculptor who has been declared insane after murdering his wife.

Both works have been filmed, as has McGrath's first novel, 1989's The Grotesque. ("Sting was in it," he says. "It kind of disappeared without trace"). Spider didn't. It was shot by Canadian horror maestro David Cronenberg in 2002 with a screenplay by McGrath and with Ralph Fiennes in the title role. It premiered at the Cannes Film Festival and also screened at the Edinburgh International Film Festival.

Asylum, by Scottish director David Mackenzie and with a script by Patrick Marber, had its UK premiere at the EIFF in 2005. It stars Natasha Richardson and Marton Csokas and also features actress Maria Aitken, mother of actor Jack Davenport and McGrath's wife since 1981.

"The outstanding one for me was Spider," says McGrath. "To watch David Cronenberg work was a joy and I think he made a great film out of what was, to my mind, unpromising material – this shabby fellow wandering around East London mumbling to himself."

What McGrath found particularly interesting was the Canadian's reasons for wanting to make Spider in the first place.

"We were doing an interview together and he said, 'I'm not interested in the madness at all. In fact I don't think Spider is mad', which astonished me. I said, 'What do you think he is then?' And he said 'I think he's an artist. Spider, c'est moi. What you have written is a picture of the artist – the lonely man with an individual, eccentric vision of the world. That, to me is, what I am.'"

McGrath, dark concerns laid aside for the moment, laughs heartily at the memory. "I was delighted that he had an utterly different interpretation of the story from me."