On a quiet Wednesday afternoon in mid February, the only sign of the new contemporary art gallery in Temple - a village meandering halfway down the road that winds through the idyllic Esk Valley - is a large white balloon, jibbing like a truculent child in the air above a cluster of artists who look as if it might be getting the better of them.

Lust & The Apple is the latest project from art curator, dealer and sometime artist Paul Robertson, the former curator of visual art at Edinburgh's highly successful Summerhall venue. If this sudden displacement to a former primary school building in a rural community some 10 miles from the capital might seem unexpected, there is precedent, perhaps, in the rolling acres of Jupiter Artland or the remote HICA (Highland Institute of Contemporary Art).

And Temple (a once bustling agricultural village named, so they say, for the Knights Templar who farmed the fertile lands of the Esk some 700 years ago) can in any case lay its own claim to artistic heritage. Sir William Gillies, who died in 1973, lived and painted here for much of his working life. The current villagers have been very welcoming, says Robertson, "although possibly a bit annoyed with me as some of them wanted to turn this place into a pub".

Robertson, whose three-year tenure as curator of Summerhall's visual arts programme ended at the beginning of the Edinburgh Art Festival in 2014 when he was abruptly "excluded", is currently in the process of court action against his former employer, due to begin in March this year. Despite this cloud, Robertson is proud of his achievements at Summerhall. "Two hundred shows in three years is not bad. During the Art Festival in August, we often had 30 exhibitions running. People said, 'Isn't it impossible putting on so many exhibitions?' To be honest, it's not. It's just work."

Lust & The Apple is, perhaps as a result, a private venture. "I've got no financial help. It's not a gallery like a normal commercial gallery. It's somewhere people can come and have a chat, look at art and have a coffee with me," he says, cheerfully. He does not intend to go looking for funding from state coffers, and has little time for "creatively stifling bureaucracy".

If popular myth about the so-called lost millions of the Knights Templar, buried in Temple "'twixt the oak and the elm tree", are true, of course, he might not have to. "I've got an oak and an elm tree in the garden," he jokes, although he is more likely to find his funds, as he has always done, from sales of his vast archive, ranging from the letters of Andre Breton to ephemera by Damien Hirst.

Artistic plans for the weekends-only venue, which will show 12 artists a year in three-month blocks, include barbeques in summer and an emphasis on land art which he hopes, neighbours willing, may spread into the surrounding fields during the Art Festival. "Edinburgh has a tendency to show safe work. Abstracts that match your coat, things that appeal to the corporate body. I like things that make you think about the world, things that are socially engaging."

Certainly, a closed-down rural primary is a somewhat apt place to socially engage with the public. There is nothing of the white-box gallery about this former Victorian primary - no doubt a rather idyllic place to have begun one's education - its unheated rooms chilly in the gloam of a late winter's day. Installation debris litters the corridors, from which a door opens onto Robertson's archive, once on display at Summerhall. Another door, and a room dominated by the artist Kenny Watson's large billboard painting, originally designed for the former Odeon cinema on Edinburgh's Nicolson Street. "I quite like the rough nature of it," says Watson, of his assorted 'works in progress' that form one of the gallery's three inaugural exhibitions.

In the car park out front, Tim Sandys, currently studying on the MFA at Glasgow School of Art, is installing his work Damocles, "a terrifying work to walk under," says Robertson. Here, then, the balloons which bob jauntily in the wind eddies above our heads, each anchored by a razor sharp hanging rapier, referencing the legendary Sword of Damocles. Sandys tells me he tripped while constructing the work in his studio, effectively throwing himself on one of his trompe l'oeil swords and ending up in hospital. "I'm interested in the idea of randomness and danger at the moment," he tells me, somewhat aptly, although visitors might feel somewhat reassured at his excellent powers of fakery. Recent projects have included leaving realistic forged banknotes in £1000 bundles on the Glasgow Metro.

In the garden out back, amidst some of Sandys's Hard Tack fibreglass sculptures, Watson's rough-hewn Electric Chair is another exercise in random danger. Fascinated by the anachronism of electric chairs in museums, he designed a full-scale replica that would be charged by a lightning strike. Visitors sit on it, quite literally, at their peril. "I put it up during a storm once," Watson tells me. "I really thought I might be on my way out." Around the corner, Alexander and Susan Maris are mid-installation of their 'Potters Field', a series of paupers graves for the nine Classical Muses.

The installation is running a day late, but Robertson says he's not worried. "I've been sweeping the floors within 15 minutes of opening before," he grins. And after all, he's only got three installations to worry about this time, not 30.

Alexander and Susan Maris/Tim Sandys/Kenny Watson, Lust & The Apple, Old Primary School, Temple, Gorebridge, Midlothian (01875 830788 or 07973 988471, www.lustandtheapple.com) until April 19, Fri-Sun, noon-6pm or by appointment