We are now leaving the 20th century.

All the systems and structures we built back then – political, cultural, financial (especially financial) – are increasingly no longer fit for purpose. How we live and how we consume are changing. Then was analogue, now is digital. As the years pass by, the music we listened to, the films we watched, the books we read will seem more and more old-fashioned, the stuff of nostalgia. And when that happens, what will we be left with? Memories. Memories and buildings.

Phaidon's 20th Century World Architecture deals with both. It is a catalogue raisonné of a century of architecture around the world: 757 buildings in 97 countries designed by 699 architects (some of them even – shock, horror – by women). And while in form it is minimalist – a few photographs, architectural drawings and short, functional essays about the buildings and their builders – it is full of stories. Stories we know and others we've forgotten. Stories about modernism (you could argue, with some merit, that the book's hero is Le Corbusier), about colonialism, about politics, about style and about ambition (case in point, the late Oscar Niemeyer's design for Brasilia). Stories, too, about how we embody our belief systems, our national outlooks and our leisure pursuits in bricks and mortar (and concrete and steel and glass).

It's the unfamiliar stories that catch the eye and the imagination. As its editors say, the book is an attempt to "reorient our architectural knowledge". To this end it begins in Perth, Western Australia and sweeps through Oceania and Asia before it gets to Europe and America, in a bid to remind us that great buildings don't only happen in London and New York. Who knew that Italian futurism had found its way to Eritrea, in the shape of Giuseppe Pettazzi's Fiat Tagliero Service Station, a building that looks like it's about to take off? Or that Le Corbusier had helped design a sports stadium in Baghdad?

What else do we learn? That concrete was the true material of the 20th century and that for all the conservatism of the Catholic Church, it has always been positively avant garde in its architectural taste – check out Etienne Gabourg's design for the Precious Blood Church in Winnipeg, which looks as if someone has taken the building and twisted the top of it like a piece of paper.

Scotland features briefly. Gillespie, Kidd and Coia's St Bride's Church in East Kilbride is here (but not their St Peter's Seminary in Cardross, surprisingly). And so is Charles Rennie Mackintosh's Glasgow School of Art (there's no sign of the Scottish Parliament building though). But it's the novelty of Scottish-Japanese practice Ushida-Findlay that catches the eye. Their 1993 Truss Wall House in Tokyo is a gloopy 1970s sci-fi dream of a building.

In the end, that may be the real fun of this book; it offers a bucket list for architectural tourists. Reading it I wanted to travel to Japan just to see Tadao Ando's Church of the Light in Ibaraki. And then swoop over to Noumea in New Caledonia to see Renzo Piano's Jean-Marie Tjibaou Cultural Centre, a set of buildings that all look like native headwear writ gigantic.

Two niggles; one small, one big. The small is the picture choice. Maybe it's a budgetary thing, but some of the images here are a little bland. I thought it was pretty much impossible to take a picture of the Barbican in London and not come away with a striking image. It seems I was wrong. (At times, too, I could have done with a little more information. How did the Swedish experiment in new towns work in comparison to our own?)

And the big? The book's bigness. It's a bloody monster of a thing. And so heavy you can't browse through it sat on the sofa. That is a recipe for backache. You need a solid table and lots of room. It's less coffee table, more dinner table.

There is one other story being told here too, of course: a story of decay and disrepair. Some of the buildings featured here – like the Nakagin Capsule Tower in Tokyo, which looks like a collection of washing machines piled on top of each other – are falling down. Some were never built, or half-built and never completed. They are a reminder that sometimes our greatest dreams don't – maybe can't – become reality. Or that, even when they do, they are, just like flesh and blood, prone to failing and fading. Everything we are and everything we do is at the mercy of time and other thieves.

In a way then, this book is just one long goodbye. We are leaving. We are always leaving.