The man who sold his world

D'Offay's gallery also provided an inspiring meeting space where art stars and wannabes could mingle. In 1986, two decades before he would win the Turner Prize, a 20-year-old Jeremy Deller met Warhol there at a private viewing. "It was the single most formative experience of my life," Deller would later say. Two years on, another meeting, this time between Lucien Freud and performance artist Leigh Bowery, who was performing a week-long spectacular at D'Offay's invitation. Bowery would go on to sit for some of Freud's most affecting portraits.

As for employees, many came and went but one in particular sticks in D'Offay's mind - a snotty young art student from Leeds called Damien Hirst ...

D'Offay was in his early 60s when he closed the gallery, but he had no thoughts of retiring. Instead he threw himself almost immediately into another project. It begins to flower in London tomorrow and will soon bloom magnificently across the rest of the UK. Edinburgh, Glasgow, Orkney, Aberdeen and Inverness will all catch the scent this spring. Next year, other towns and cities will benefit. The year after, others still. And so on.

The project is Artist Rooms, an ever-rotating series of exhibitions drawn from D'Offay's 725-piece collection, which is now owned jointly by the National Galleries of Scotland and the Tate. A graduate of Edinburgh University, D'Offay originally offered it to Scotland alone but its cost - £26.5 million - meant a combined purchasing fund was needed. It might not look it on paper, but it is a mighty gesture on D'Offay's part. The real value of the collection is five times the asking price, and climbing. The dealer wanted only what he had originally paid for the works.

"It was the nearest we could do to making a gift of it. We bought the things over 30-odd years, but if we bought it for $5000 and it was worth $500,000, what we got back was $5000," he explains. "It never occurred to us to try to get more money for it. What we had to do was do something that was possible for the nation and which covered the money we had borrowed to buy it."

D'Offay's former gallery, a warehouse-style building just off New Bond Street, is now being redeveloped but he still occupies a magnificent late-17th century building just a few metres away. We meet in his first-floor study, a wood-panelled room lined with books and illuminated - the only word for it - by a series of Warhol self-portraits. "I knew him pretty well in the short time we worked together," he tells me. Propped up on the fireplace is another Warhol, a screenprint of Beuys.

The room also contains a desk, at which D'Offay's assistant sits, and a long, burnished wooden table by the windows. It looks like the sort of thing the Mayflower Pilgrims would have studied tide times on before sailing to America. Now it holds piles of art books and our two cups of tea.

I start by asking D'Offay if he sees himself as a philanthropist. "Good Lord," he laughs, "I couldn't even spell that word let alone know what it means."

Still, he does view his "gift" as an educational resource meant for the greater good.

"Don't you think that those people who are interested in culture can point to certain moments when they saw a show which started a new way of thinking? That's what it's about," he says. "The idea is to get children involved in culture at an early age, to get them involved in looking at paintings, sculpture and photography. If you get a child involved in culture early, you've got them for life."

There is a reason why he is so taken by this idea: it happened to him. Although his French father gives him an exotic art dealer surname, D'Offay was born in Sheffield and grew up in a semi-detached house in Leicester.

At an early age he was taken to the local museum, an event which had a profound effect on him.

"It did play a big part in my life," he says. "There was a painting by LS Lowry which was a matter of wonder and delight to me as a child. Later on, there was a painting by Francis Bacon and there were some terrific 20th-century German paintings. But the idea that you could be alive and be in a museum seemed to me to be a colossal idea when I was nine."

D'Offay's love affair with Scotland, meanwhile, is deep and long standing. His wife, the former Tate curator Anne Seymour, has a bothy near Loch Rannoch, but the connection began as a student in Edinburgh years earlier. D'Offay went to Edinburgh University in the late 1950s to study Spanish Literature, but his professor died in the first term and he switched to art history. He studied under the eminent art critic Professor David Talbot Rice, whose injunction in the first minute of the first lecture was this: "Your textbook is the National Galleries of Scotland."

Accordingly, D'Offay would walk through the gallery on the Mound every day, on his way to or from the university from his flat on Jamaica Street in the New Town.

"Seeing all these great Old Masters and Impressionist and post-Impressionist paintings, aged 17, was what made the most impression on me," he recalls. "You were conscious of living in a capital city which wasn't London. You could feel the greatness of Scotland, you could feel that this was a place with royal palaces and extraordinary history and these wonderful works of art."

Happily, there were new masters around as well. Art impresario Richard Demarco was just starting out, the Traverse Theatre was in its early days, and there was something called The Paperback Bookshop, where you would be given a cup of tea as you browsed. This seemed like a thrilling innovation at the time. Best of all was D'Offay's friendship with artist Ian Hamilton Finlay, who lived in nearby Fettes Row, a street of poets and booksellers that exuded an "intellectual, avant-garde feeling". D'Offay lapped it up.

"Ian was an inspiration," he says. "In those days he had agoraphobia. He couldn't really leave his apartment. He didn't go out. It was a big occasion if he sat on the step outside."

At the time, Hamilton Finlay was predominantly a poet but a few years later he moved out of Edinburgh to Dunsyre, where he established his now-celebrated garden, Little Sparta - "a great gift to the world," says D'Offay - and began making works that combined sculpture and text. "He was a great artist and has left behind a great legacy."

It is appropriate, then, that the first exhibition in the Artist Rooms series features Hamilton Finlay's work. The Scot's installation piece, Sailing Dinghy 1996, goes on show at Tate Britain tomorrow.

Scotland's first taste of Artist Rooms comes a week later, in a group show at the Scottish National Gallery of Modern Art in Edinburgh, featuring work by Warhol and Hirst among others. Hirst's piece is Away From The Flock, a sheep in formaldehyde. "I thought it would be nice to show it in Scotland, which is a country where you see plenty sheep," says D'Offay, grinning.

In April, shows featuring work by Bruce Nauman and photographer Robert Mapplethorpe open in Glasgow and Inverness respectively, and in June, video works by Bill Viola go on show in Stromness. The Scottish National Gallery of Modern Art's festival show is minimalist painter Agnes Martin, and in the early autumn, Aberdeen has the chance to experience Ron Mueck's crowd-pleasing sculptures.

National Galleries of Scotland director John Leighton can hardly wait. "To describe it simply as a collection is an understatement,"

he says. "It is a new way of thinking about how public collections can be used to reach audiences right across the country. It is, in many senses, an art gallery without walls or boundaries."

But while its worth as an educational tool can't be underestimated, neither can its importance to the economic health of Scotland's town and cities. It is a fact that cultural tourism is now a prime economic driver. Edinburgh has known it for some time and benefited from it with blockbuster art shows like 2007's Warhol exhibition and 2006's Mapplethorpe retrospective, both mounted with significant help from D'Offay. Now the rest of the country will get a share of the spoils. The point isn't lost on the dealer.

"Why does one travel? There are very few reasons," he says. "You travel for business, holidays, to see your grandmother and for culture. Cultural tourism is a phenomenon of the last 15 or 20 years, and I think it will become more important."

Although D'Offay's father was French, he came originally from the Seychelles, a predominantly black country. D'Offay found out recently that he has a black half-brother, a fact that adds a personal dimension to his newest project: a photographic history of the black American experience in the 20th century.

"I think it's important that that big question - the question of being black in the 20th century - is addressed, and it can be addressed in photography in very important ways," he says.

He has spent three years on the scheme to date, and still has a little further to travel. With a black president now in office, his timing is impeccable.

Meanwhile, the work on Artist Rooms continues. He still has a significant personal holding of modern art and is currently making plans for it to be gifted to the collection when he dies. He is still ferreting around for new acquisitions too, and hopes that in 20 years' time there will be new funds available to continue that work. "So that it can grow and stay young and be abreast of the times," says D'Offay. "That's one of the reasons it's called Artist Rooms, and why it hasn't got our name above the door."

Are there any Scottish artists D'Offay would like to see added to the collection in the first wave of new acquisitions? "It would be a long list," he says, "but let's get Douglas Gordon first and then turn to the others."

At 69, D'Offay doesn't look like he's about to slow down. Like the Mayflower Pilgrims, he, too, is about to set off for America's east coast, though by plane rather than boat.

He has to meet with members of the Warhol Institute, attend a few New York gallery openings, and slot a few more pieces of the "black in the 20th century" jigsaw puzzle into place.

In fact, his flight leaves in a few hours. But, even as I make to leave, he is intent on showing me other new things: a beautiful set of photographic prints made from original Alexander Rodchenko negatives, for instance, and - almost as I head out the door - an extraordinary book of photographs by EJ Bellocq.

Taken in the very early 1900s, they show prostitutes in brothels in New Orleans's infamous Storyville district. The women are mostly naked and many of the faces have been scratched out by some unknown hand.

The images are beautiful and sad and grotesque, and D'Offay is rapt as he leafs through the pages. After a moment's contemplation, he snaps the book shut and returns again to his grand theme - that art is nourishment, an energising fuel for the imagination. "If you were a 15-year-old boy, wouldn't you just love to see those in an art gallery?" he asks with a grin.

Looking at them, I have to admit I would. Anthony D'Offay may be unable to spell "philanthropist", but I don't think he'd struggle with "mischief".

Artist Rooms opens at the Scottish National Gallery of Modern Art, Edinburgh on March 14, with a show consisting of work by Damien Hirst, Andy Warhol, Francesca Woodman, Alex Katz, Ellen Gallagher and Vija Celmins Comment on this story here