There's a strange kind of magic happening at the Royal Lyceum Theatre in Edinburgh.

While not directly related to The Infamous Brothers Davenport, the biggest show to date from Jamie Harrison and Candice Edmunds's visually inventive Vox Motus company, the two incidents of unintentional jiggery-pokery are by-products worthy of Derren Brown.

First of all, an old-school dictaphone resolutely refuses to record an interview with Harrison and Edmunds. Even when fresh batteries are hastily located, the admittedly ancient micro-cassettes fail to whirr into action. The moment the interview, recorded ad hoc on a mobile phone, is over, the dictaphone starts working again, original batteries and all. This is apparently the second such incident to happen over the previous week as the company pull together a technically audacious concoction of old-time Victorian hokum and sibling rivalry scripted by Peter Arnott.

Earlier at rehearsals, something infinitely less spooky but just as telling about how Vox Motus operate occurred. Real-life acting brothers Ryan and Scott Fletcher, who play the Davenport double act who so spectacularly conned audiences in need of salvation by seemingly conjuring up dead spirits, are going through their paces in a scene depicting something which these days might well end up on one of the numerous haunted house style shows that populate multi-channel TV.

Scott Fletcher is playing Willie Davenport, as in real life the younger of the brothers. Sporting period tail-coat, he is seated at a round table, while his brother Ira, as played by Ryan Fletcher, watches over him. Seemingly possessed by spirits unknown, Willie seemingly acquires the soft-spoken demeanour of a southern belle, only to deviate from the Davenport's script, taunting Ira like a ventriloquist's dummy who's gone off message.

As Edmunds and Harrison stop and start the action, fine-tuning it as they go, there's a split-second pause when something happens that only Ryan and Scott can see, but which shifts the whole mood of the moment.

It's the sort of unspoken alchemy that only those with a chemistry born of long-term intimacy can acquire. If they wanted to, as with the Davenports, the Fletchers could probably pull the wool over everybody's eyes. As it is, all crack up laughing as Ryan tells Scott to stop whatever mischief he was signalling.

"It's like the way your mum looks at you," says Ryan, "and she doesn't need to say anything, but you know you're in for a hiding. When you're onstage as well, your senses are heightened, so you do see things going on with someone that nobody else can see."

Scott points out: "Because we're playing brothers, you can have fun with that sort of thing in rehearsals, although you should probably keep it to your down time. Working together, you're much more settled straight away, because we know each other so well already. There's a confidence and an understanding there before you've even started."

The same might well be said of the characters the Fletchers are playing. The Infamous Brothers Davenport is inspired by the real-life yarn of two American siblings who somehow connived their way into taking the world by storm. The sons of a New York policeman who later became their manager, Ira and William Davenport clung onto the coat-tails of the spiritualist movement which had become all the rage in the mid-19th century.

Using only the power of suggestion and a bag of theatrical tricks that conjured up the apparent image of dark forces from the spirit world, the Davenports became a sensation. They travelled the country for a decade with their trademark "spirit box", and later travelled to England before being exposed as frauds. Even legendary showman PT Barnum took notice, going so far as to include them in his 1865 book, Humbugs of the World.

While such jiggery-pokery forms a major part of The Infamous Brothers Davenport, the truths or otherwise of the Davenports' routines aren't the play's driving force.

"We've been interested in doing something on the séance for some time," says Harrison, "so we started looking at spiritualist churches which still exist today. Out of my own experiences as a magician, we started looking at the Davenport Brothers and found out that they started doing tricks from a very young age, and eventually their father realised they could make money out of it. They grew up in this really tough frontier town, and had a very hard life."

"They were violent and naughty and mischievous," Edmunds continues. "We thought at the beginning that the spirit cabinet was going to be the main part of the show, but once we realised how interesting the lives of the Davenports were, that became our angle, and it actually moves away from that pretty quickly."

Vox Motus have come a long way since Edmunds and Harrison formed the company while students at what is now the Royal Conservatoire of Scotland in Glasgow. Their 2007 debut, How To Steal A Diamond, gave little hints of the technical complexities that would become increasingly ambitious in the broad comedy of Slick, the darkness of Bright Black and the musical audacity of 2010's The Not So Fatal Death of Grandpa Fredo.

One novel aspect of the company's biggest show to date is bringing members of the audience onstage to sit alongside the Davenports as they conduct their séance, just as might happen at such an occasion in real life. As Edmunds and Harrison acknowledge, such close-up involvement has the potential to disrupt things unintentionally.

"An interesting thing as directors is trying to foresee every possible reaction these members of the audience might have," Harrison observes.

"Everyone will have a slightly different response, so we have to bed the performers down, so they can continue the narrative arc no matter what the people who come up onstage might do. The look of horror on the actors' faces when we unveiled these plans was a picture, but the thing to emphasise is that they're not random members of the audience who do this. It's a choice they make."

This choice is made in the foyer as the audience enter into a mock-up of a Victorian world in which they go as willingly as real-life believers in the spirit world.

"It's been fascinating to look back at the start of spiritualism," says Edmunds, "which dates back to the Fox sisters, who predated the Davenports. People just wouldn't believe that these two children were lying."

Suspension of disbelief, then, is everything in The Infamous Brothers Davenport.

"At the top of the show we want people to experience all the things they associate with seances," Edmunds explains, "but we also want to take it further so people have an emotional investment in the narrative and so it's not just all about tricks."

As Harrison puts it: "The thing we've been holding dear all through the creative process is the power of the human imagination, and the dark places the imagination can take you. This belief in things like spiritualism, I think it's directly proportional with what's going on in terms of a decline in belief in organised religion. The less that people engage with organised religion, the more of a void there is, and the more we try to fill it. That can be through scientific rational beliefs, or something like spiritualism. I've put my foot in it a few times talking about this show in presuming that everybody has the same beliefs, and finding out that they really don't.

"As theatre-makers it's difficult to get a balance, because we don't believe in anything beyond death as human beings, but we don't want to criticise anyone's belief system in anyway, because for a lot of people it provides hope."

The Infamous Brothers Davenport, Royal Lyceum Theatre, Edinburgh, January 19-February 11, then tours. Visit www.voxmotus.co.uk, www.lyceum.org.uk.