It was living in Morningside that made Richard Williams start reading Freud.

The professor of contemporary visual cultures at the University of Edinburgh moved to the suburb 13 years ago and found perhaps the best preserved Victorian cityscape in Britain. But as the years went by he started to be bothered by a persistent question. And the question? Was anyone having sex here?

In the midst of his own mildife crisis the parallels between Edinburgh and Vienna a century before were striking, he thought. "You've got a relatively small and very well ordered built environment with a very well developed sense of what's public and what's private," he says. "Anxieties about what is appropriate in public behaviour. And lots of anxieties about sex actually. How do you do the right thing? How do you behave correctly?"

The distance between what was public and what was private was written into the very buildings, he decided. "All the domestic architecture is basically the nuclear family built in stone. Every dwelling has a big public room, but it keeps private functions well hidden. Anywhere that sex might happen is kept well out of public view."

Is architecture always so uninterested in the erotic, he began to wonder. So much so, that he's now written a book on the subject. Sex and Buildings starts with Morningside propriety but then travels to the Playboy Mansion, the top of skyscrapers and Manchester's Canal Street to look at how architecture has reflected changing ideas of the body and sexuality over the last century.

In contrast to Morningside, Californian architecture of the first half of the 20th century was much more transparent, full of open spaces and glass walls. Architects like John Lautner would say they were inspired by nature. "But it got taken up by people who were interested in making 'sexy' houses," says Williams. "He acquired a bit of a reputation for building what were essentially very sophisticated bachelor pads."

As a result Hugh Hefner's Playboy magazine became very interested in architecture like Lautner's in the middle of the last century. "It saw architecture as part of its programme of liberation - its own very particular form of sexual liberation - involving pipe smoking, martinis and playing lots of golf," explains Williams.

The association was still strong enough as late as the 1990s for Hollywood to use Californian modernism as a signifer for pornography. In The Big Lebowski the pornographer Jackie Treehorn lives in Lautner's Sheats House. The building was even the location for a porn movie shoot in 1997 too.

Architecture is a very masculine profession, Williams points out. There is some truth in the cliche about skyscrapers being phallic symbols. Williams points to Norman Foster. "Foster's a great architect, but his work is extremely phallic in lots of ways and he's obsessed with bathrooms. There's a huge amount of attention that goes into the male bathroom. They're beautifully done. There's one at the top of the Commerz bank in Frankfurt. The urinals are right against a glass window so you literally pee over the city.

"Foster likes fast planes and cars and helicopters. It did occur to me - I can't believe I'm saying this - that he himself is this tough looking, bald robust figure. He's quite phallic himself."

In the end, of course, it's how people live in buildings rather than the architects' intentions that decide how erotic or not they are. Many of the Californian houses Williams visited for the book have a history of unhappy and divorced couples. And it was the decision of Britain's gay community to become more visible in the last quarter of the last century in places like Manchester's Canal Street that began to transform the image of British cities. "There was a very close association between the rise of British city centres and the rise of gay culture. One wouldn't have happened without the other really," says Williams.

And, as it turned out, Morningside isn't quite so buttoned up as Williams initially thought. Not every house contains a nuclear family, he realised once he began to know the area he lived in. Indeed, recent Edinburgh City Council figures have revealed that only 3% of Edinburgh households conform to the nuclear family model. 41% of households are occupied by single adults. "I think Edinburgh has a huge problem that it doesn't know how to deal with yet. It's an emerging problem that is well beyond anything that the city council can possibly deal with at the moment. They have a very unconventional city demographically; mostly single people, mostly adults. Yet it imagines itself as this bastion of propriety and family."

As yet Edinburgh's architecture doesn't reflect those demographics. But Williams thinks he may have an alternative model. "I've just been on holiday to Lanzarote fairly recently and what struck me is you've got this neo-communal arrangement. Lots of apartments of different sizes built around a pool. Lots of space to swim, sunbathe and show yourself off and lots of stuff orientated around pleasure. And okay, it's a bit warmer there, and everyone is on holiday and they're going expecting something different. But I did come away thinking 'why don't we do this all the time? Why don't we organise our lives in a more pleasurable way?'"

Maybe Butlins, via Shakespeare, had it right all along then. What was it Billy Butlin used to say? "Our true intent is all for your delight".

Sex and Buildings by Richard J Williams is published by Reaktion Books, priced £25.