I've been feeling so melancholy today," confides Edna O'Brien brewing tea - the "acquired taste" of rooibos for her, Earl Grey for me - in the kitchen of the tall, quirky house in London's Knightsbridge that she's rented for almost 30 years.

"But, oh, how you've lifted my spirits!"

I am, of course, enchanted since I fell under O'Brien's lyrical literary spell decades ago; then I met her more recently at a party in Edinburgh and was bewitched by her famous beauty, the crackling glamour of her red-gold hair and her mesmerising brogue which sounds as if she gargles daily with Irish cream liqueur. So I'd agonised over what to bring as a small gift, finally settling on an M&S bouquet of roses.

"Before you arrived, I was thinking that I felt sad because there were no flowers in the house - an orchid I'd had for months died only yesterday," she says, deputing me to arrange the roses in a large vase and carry them up the dark-red, perilously narrow, twisting staircase to her drawing-room cum study where we talk about writing, love, sex and religion, mothers, the death of her friend Seamus Heaney (whose funeral she attended), her 50-year exile from "mad" Ireland ("which is always, always in my heart and on my mind") and her latest book.

The Love Object is a 533-page selection of her finest, most celebrated short stories, many of them prize-winning - in 2011, she was awarded the £25,000 Frank O'Connor international short story award for Saints And Sinners, her first collection in two decades. It's just one of a long list of prizes she's won, including the Irish PEN Lifetime Achievement Award and the American National Arts Gold Medal.

The new collection is her 22nd work of fiction - she's also written five plays and four works of non-fiction, including the memoir, Country Girl (2012), in which she recalls her childhood when she would run out into the fields to write. She wrote her first novel at the age of eight, although she grew up in a house where there were no books, apart from the Bible and a cookery book.

She tells of the loss of her virginity to a handsome newspaperman while she was training as a pharmacist in Dublin after escaping from Drewsboro, the once-grand, crumbling County Clare house where she grew up with her alcoholic father, an inveterate gambler, and her anxious, hard-working, gifted mother, whose hundreds of letters she's kept. They are "small masterpieces," she says. Yet her mother hated and mistrusted the written word "as if it were redolent of sin".

O'Brien's only marriage was to the late writer Ernest Gebler, the father of her two sons - Carlo, a writer, and Sasha, an architect. Like Nora in Ibsen's A Doll's House she slammed the door on the misery of the marriage, then fought and won a long, bitter custody battle for her children. She also writes of the parties she threw - Richard Burton, Judy Garland, Marlon Brando, Marianne Faithfull came. Paul McCartney walked her home from another party and wrote a song for her.

There was a one-night stand with Robert Mitchum, although she tells me, contrary to received opinion, she's never been a legendary sex siren. She's had only two long love affairs, one with a high-profile politician - "Lochinvar" in her memoir - whose identity she'll take to the grave. In the Swinging Sixties, Sean Connery introduced her to the psychiatrist RD Laing, with whom she dropped acid, a bad trip she took years to recover from.

And yet when O'Brien speaks about those fabled parties today, she reveals that she was often lonely in the middle of them. Maybe, she thinks, all these famous people were lonely, too.

Today, she lives alone, freely admitting to suffering from loneliness for which the great solace is to "read, read, read" when she's not writing and constantly rewriting, which she does every day. She reads everything she writes out loud again and again and again. "It's a sort of madness," she sighs. She feels "possessed" by the need to write, although she tires more easily nowadays and claims that her memory sometimes takes a lurch.

"I wouldn't be a writer if I were not lonely. Most writers are - you just need to read their lives or their letters. To remain faithful to writing one has to live much of one's life alone. I was born lonely so I can always tell lonely people when I see them," she says. "I'm often drawn to them because I feel they might have some secret to tell me." Before she begins work she prays, tells the Rosary, then reads a few pages of Shakespeare because he's endlessly "invigorating".

The room where she tells me this is exactly the sort of feminine lair you'd expect O'Brien to inhabit. Incredibly, she's 82 years old, creamy-skinned, clear-eyed, flawlessly made up and elegantly dressed in pleated cream silk and several shades of grey. A fire blazes in the hearth - smirr falls outside - and hundreds of books ­wallpaper the room.

There are portraits of O'Brien's mentors - James Joyce and Samuel Beckett - on the wooden mantel and a glass bookcase dedicated to the works of Chekhov, to whom she's often been compared. Indeed, in his introduction to The Love Object, John Banville writes that Chekhov has been her "true teacher" while Henry James "surely would acknowledge her as a fellow traveller". He also describes her as "the poet of vulnerability".

Selecting the stories for The Love Object, which she's dedicated to Philip Roth ("in long friendship"), who has called her "the most gifted woman now writing in English", and which spans five decades of her mastery of the genre, revived memories of the first stories she wrote.

"I began writing short stories as I finished my first book, The Country Girls, which poured out of me in just three weeks," she recalls. Her critically acclaimed debut novel, about two young girls discovering their sexuality, which came out in 1960, was banned by the Irish censorship board. The O'Brien parish priest took copies to the village of Tuamgraney, where she was born, and publicly burned them. Overnight she became Ireland's scarlet woman of letters - "a Jezebel", she says. Banned, burned, blamed. Writing well has been the best revenge, however.

After the death of her mother, Lena, she discovered her copy of the book hidden in a bolster case, with "offending words daubed out with black ink". Read it again today and you are struck by the innocence of her rebellious heroines, Baba and Kate, as well as the freshness and gorgeous immediacy of O'Brien's prose.

And so it is with the The Love Object stories. "What I love about short stories as a writer and, indeed, as a reader is the duration of time [in one] is less than the doing or reading of a novel," she confesses. "A good short story is done in three or four months, regardless of the length, whereas a novel is three or four years. The momentum and energy is containable, it doesn't get dispersed by life. I don't sit down and say, 'I'll write a short story now,' but some subjects decide their own form. They ask to be what they are.

"The greatest master of the short story is Chekhov," she continues. "His stories are perfect. They are like little gunshots, perfect in form, full of suspense. I know they are not crime stories - why, oh why does everyone want to read crime thrillers now? - but he's such a natural, pure writer. And, of course, Joyce's Dubliners … The Dead, well, it has all the story and the breath of a complete novel." We talk at length about Joyce, "an atheist but a Jesuit at heart".

Like Colette - O'Brien has been called "the Irish Colette" although she's a darker, more profound writer - she constantly returns to her childhood and her mother who remains "within" her. "I was lucky; I grew up surrounded by stories. Story is story is story. Our house was full of stories, some painful, some not. I had a rich cache to begin with. Less so now, because I live in the city and I live alone. Also to my great disappointment, people don't tell each other stories any more. Narrative has gone. Everybody is looking at television or their phones. I mourn the loss of language."

Before I leave, O'Brien shows me her study - she's run out of work space upstairs, where she's always written, "in a kind of trance", at the arbutus wood desk. Now she writes, always in longhand, at her dining-room table, blanketed beneath an avalanche of papers and books. "We bleed, Jackie ... we bleed," she says, referring to the act of writing, showing me several pages that she's forgotten to number of her new novel, about the hidden, night-time lives of migrant cleaners, which she hopes to finish in 18 months. (She's also working on two plays.)

"Oh dear, do I sound precious? I look at what is happening in Syria and I ask myself what I'm saying about 'bleeding' over words."

But surely she's always said writing is purgatorial?

"Purgatory? I'd opt for hell now. I think I'm moving into the inferno - into the seven circles! It gets harder and harder the older I get," she says, preparing to return to work - "if I can ever find the pages where I left off. I keep losing the best bits I've written. Perhaps I ought to start using different coloured inks. What do you think?" she asks, then embraces me warmly for cheering her up on a day that seemed filled with Thomas Hardy's "the sadness of things" but which, she insists, is now "gorgeously lit up" by a few white roses on her windowsill.

The Love Object: Selected Stories by Edna O'Brien is published by Faber and Faber, £20