So yes, you may have heard, or seen, that there is an inflatable Stonehenge in the middle of Glasgow now.

And, because of the same art festival, the city's modern art gallery has its enormous main room full of sawdust and cellophane. Elsewhere, there is a diamond made from the remnants of the London riots, and scattered and displayed Turner Prize winners, nominees, international stars, art celebrities and names who will bloom in the future. There may be the odd piece here and there that courts or may cause controversy or alarm. But there is also much beauty, as well as mere baffling loveliness, at the heart of the Glasgow International Festival of Visual Art, which opened this weekend.

Contemporary art can, even to frequent observers, sometimes seem distant, even slight, lost in its own discourses and references. The better work, however, breaks out of the strings of its own theories and portrays itself easily and honestly to any public which takes the time to view it. Where better to start a brief tour of this year's GI, then, than at the least contemporary venue in the city, the Kelvingrove Art Gallery and Museum? This is no astringent white space, but the comforting and inviting free museum for the city. Richard Wright said recently that he has wondered whether his strict conceptual rigour, with much of his art almost always confined to wall-based painting, has limited him in some way. For his first (and probably last) show on paper, at the Kelvingrove, he and curator Katrina Brown, also in charge of the whole festival, have collated a beguiling collection of his rare paper works from the last 12 years. Slightly quirkily hung in the Burns Gallery, there are austere, linear images, there are detached architectural drawings and gorgeous Rorschach-like butterflies of pattern and colour. There is, most lovely of all, some spectacular, luscious works in gold and silver. In particular the silvery, heavenly-clouds of one untitled work, currently owned by a collector in Oslo, is worth a visit alone, its nebulous majesty redolent of Gustav Dore's mesmeric illustrations for the Divine Comedy.

A short walk from there is the Common Guild at Woodlands Terrace, and a quietly powerful collection of photographs by Wolfgang Tillmans. On one floor, there are photographs shown primarily as objects in themselves. But on the first floor, there are photographs as images, with carefully chosen, although seemingly random shots from the photographer's wanderings in Ethiopia, Rio, Munich, Berlin and Tasmania. There is a series of evil looking close-ups of car headlights, animalistic and violent, and a fascinating shot of a cancer operation in Germany, a patient's guts calmly held to one side like a string of bulbous sausages. There is a still of a waterfall, wonderfully shot and framed, which turns the water into a vast grotto, or a series of intertwined Gothic stalactites.

If you amble from there to the silvery mass of the imposing Skypark block, in its basement is Petrosphere, an intriguing group show of five artists from Glasgow and five from Athens. Neatly set up, there is a good amount to see here, from Ruth Barker's spooky Gilgamesh Song, an incantation set in darkness in a revolving video, to Helen de Main's elegant work with materials and prints. In the centre of the room is a sculpture by James McLardy which seems to throb with some kind of malignant power. An altar, it seems, to some kind of Cthonic spirit, it uses wax as both ornament and blood-substitute, and is adorned with a gold leafed hand or gauntlet, and a bent medallion at its base, perhaps tributes to its nameless god, a disc of wax and bronze at its apex: a slab of scary HP Lovecraft in a group show that is well worth finding. Less easy to discern is the main show at the Mitchell Library, where Nairy Baghramian has installed Spanner (Strecher/Loiterer) in the main hall. In an otherwise empty room, a flex has been strung, insulated with long pieces of piping and other adornments. It seems like a barrier. It may hint at some kind of obscure utility. Its starkness left me a little perplexed, even though something made me want to duck under, or leap over it. Not so hard to work out is, next door, the Art Lending Library, where you can borrow works from more than 50 artists for three days, a scheme from the Market Gallery and put into effect by Walker & Bromwich. It is a neat idea, but it will be interesting to learn of how many people loan works in the next 18 days.

A more satisfying sculpture looms in the city centre, at GOMA. Here Karla Black has installed an enormous example of her work, two in fact, with one made from the sawdust of pine, spruce, teak, maple, yew, and oak wood. It sits massively, like an gigantic slab of striated rock, or, if you are being even more literal, a very big cake. It looms in Goma's main hall, solemn under gently dancing streamers and ringlets of polythene, paint, body wash, shampoo, gel bronzers, lipgloss, and nail varnish. Like a lot of Black's work, both works are very serious and very child like. You want to jump into and on top of Empty Now, perhaps to find out whether it has meaning beyond its sheer physical shape and form. It may not have.

Although GI does not have a lead gallery of any kind, it does have a city centre base near Goma, in Miller Street. Here is where you can see Rosalind Nashashibi's film of Scottish Ballet dancers rehearsing, and the closely miked reactions and comments of local observers – ordinary men and women, and a pair of policemen. Nashashibi doesn't quite document and she doesn't quite tell a story. The film glides by like a sunlit memory, the voices of the observers clear and sharp with the artist's first use of digital technology, although the film is still her sun-soaking 35mm. The dancers can be seen in their eerie, athletic grace, but you can also hear the extraordinary exertion and expense of energy. Outside the Tramway windows, Glasgow seems grey and rather ugly, while in the rehearsal room, the ordinary mortals seem stunned and bewitched by the ballet dancers extreme grace, beauty and athletic precision.

There is beauty, too, in the tiny diamond made by Teresa Margolles, the Mexican artist who has been resident in Glasgow for five months and whose show at the new Glasgow Sculpture Studios base is a memorial to the lost lives of Ciudad Suaraz and the debris and ashes of the 2011 London riots. She has transformed the carbon of the Croydon riot into the small diamond, a jewel that cannot be destroyed. And in Recovery Process... she is showing the archive of more than 6000 negatives from the work of Luis Alvarado, a local photographer in Juarez. There are wrestlers spattered with blood, hopeful weddings, naked women posing on beds, stolid men smiling and unsmiling, all from the 1960s to the 1980s. Juarez is now known for its history of extreme violence, especially the murders of women. You see these very human images and wonder how many are still breathing and celebrating, smiling and posing, today.

GI is enormous, and you need at least a week to see it all. But one item cannot be missed, especially if you have children, and that is Sacrilege, Jeremy Deller's fairly magnificent inflatable Stonehenge on Glasgow Green. No one really knows what Stonehenge was used for, or even how it was built. The ancient architecture is a potent symbol of the British Isles, but it stands impotent, its utility lost in time. Deller takes the icon and makes it work: it has, literally, been given the breath of life. It is inflatable, bouncy, fun, and joyous. Maybe back in the pits of time, ancient children danced around its stones. Yesterday, as dozens of children crawled all over this bendy simulacrum, its effects were funny, bizarre, and, in its outlandish way, beautiful too.