LIFE is up and down for Martin Creed. The most tangible manifestations of the Turner Prize winner's seemingly structured world-view can be seen in his public restoration of the Scotsman Steps in Edinburgh in 104 different types of marble.

It's there too just across the road from the Steps in the lift of the Fruitmarket Gallery, who commissioned the restoration. In 2010, the gallery showed Down Over Up, an exhibition in which the gallery stairs were transformed into a synthesiser, with each step playing a different musical note. The lift did something similar, as a whooshing chorale moved up and down the scale depending on which way you were going.

Creed released albums of spindly minimalist ditties whose words went back and forth as they reduced an idea to its bare bones. He did something similar with ballet when he appeared alongside dancers from Sadlers Wells, who performed the most basic of steps. The programme also featured Creed singing songs and screening films featuring people vomiting, as well as one charting the rise and fall of his own penis.

This year, on the back of digitally released single, What the F*** Am I Doing?, Creed has joined Edinburgh International Festival's theatre programme, with a three week late night run of a show called Martin Creed's Words and Music. The show forms part of EIF and the British Council's Spirit of '47 season to commemorate the Festival's 70th anniversary. For what sounds a mixture of art cabaret and show and tell, Creed's approach remains singularly contrary.

“Part of the point of it is to try and think out loud,” he says, sitting in a neat little office space in the Fruitmarket, and dressed like a technicolour Victorian hipster. “Whenever I've done things like talks, where I've prepared stuff beforehand, as soon as I get up there, it all suddenly doesn't feel relevant. It's the same as well for exhibiting works in galleries. So I'm bringing songs that are pre-written, and I suppose I'm bringing ideas that I've been working on to talk about as well, but it's not a show in the sense of, erm...”

Creed checks himself like this a lot, his speech patterns going back and forth as he considers every word. A sing-song Glasgow accent acquired when his Quaker parents moved to the city from Wakefield when he was three years old sounds consistently surprised by what comes out of his mouth. It's the perfect illustration too of his resistance to apply any clear structure to his Words and Music show.

“In a way, the point of it isn't clear either,” he says. “If there is any point to it, it's a matter of trying to get through the day. Trying to live your life. The other idea behind it is that onstage is the same as offstage. So when I'm onstage, I might be just as disorganised as I am when I'm offstage.”

This willingness to fly blind in both life and what he's been consistently reluctant to call "art"is telling in everything he does.

“I've been trying to work on words,” he says, “to work on talking as much as working on other things. I try to work on the noises I make in my life just as much as I try and work on the movements I make. I'm just trying to live my life, and that includes making noises, because I find myself here in this world, with other people, and I feel lonely and want to talk to them.”

Creed did a similar show in New York last year, where a “terrible” thing happened.

“The first night, I felt on quite a high afterwards,” he says, “and then the next night, I started talking about the same thing that I started talking about the night before, and immediately I felt that it wasn't alive, because I was just trying to repeat myself.”

There is an obsessiveness to Creed's work that suggests a serious level of control freakery.

“I think that's what's wrong with my work,” he says, laughing, as he does throughout the conversation, “but I look at it and it's too controlled. That's what I'm always fighting against, the tendency to try and control everything. I'll end up with everything just all being neat and clean and nice, nice colours in a little box, or the song equivalent of that. You pare it down and pare it down till you're in danger of it being too controlled, and you take the life out of it.

“But I think I try and control things because I'm scared of losing control, and if I think about that, I think what I need to do is get the fact that I'm scared into the work, so people going to see the bit where I'm controlling things also get to see the bit where I'm scared.”

Is Creed trying to make chaos out of order, then?

“Well, I dunno,” he says. “I just think I'm trying to fight against my inclination to try and control things, and kill things. I want to feel better, so I feel safer if things are under control, but then that'll lead to killing things. Doing a show like I'm doing here, it's one hour, so it's a little microcosm of life. Getting through the show is like getting through the day.”

Why push himself in this way? Wouldn't it be easier to hone a few routines and bluff his way through the gig?

“Aye,” he says, “except I just think I can't do that, and I feel shit if I'm repeating myself. It doesn't feel alive, and it's not exciting. Or maybe I just think I'm not a good enough actor, for want of a better way of putting it. On the other hand, I don't know how actors do it, but I don't think that they maybe do it in the way that I think that they do it. Maybe what I'm talking about is not that different from the way actors try to be fresh every time.”

Being in the moment?

“Aye. Exactly, aye.”

Where did the desire to do that come from?

“I think it's come from frustration,” he says, “and not being happy with my work, thinking that, although I might like some of the things I've done, maybe I'm just not really happy with them.”

Creed talks about his work as “a sort of soup."

"Everything's joined together, and there are some things in the soup, floating around, but it's mostly a purée. But if you're having an exhibition or making an album, then what you're doing is picking things out of that soup and then displaying them, so these are selected from the soup. But there's always something missing, and that's the soup. It's always artificial that you've taken bits out, and that's why the individual works are never good enough.

“But in a live show like this, it's possibly more possible to show different bits and pieces, so there's the possibility as well to display the soup. The point of that to me is that it's more like life, so then I don't feel that it's fake, or some weird, artificial, tidied up version of life. To me, that's more exciting. I think one of the worst feelings in life I find is trying to keep up a false pretence about something.”

The purpose of theatres and galleries, he says, is that they are “a safe place to be chaotic."

"Because there's a framework, it's like a place where you can hopefully enjoy either the difficulty of life, or the kind of crazy mixed-up-ness of a life. It's like having a fence with a rigid structure in front of a garden that's got wild animals in it. The fence basically allows you to enjoy it, and you're not in danger from it.”

Creed may not know what his new show is yet, but he knows what it's not.

“It's definitely not a display of what I've achieved,” says Creed.

What does he think he has achieved?

“That's the point,” he says. “I don't think I've achieved anything. The idea of achievement just seems really pompous, anyway. Basically, if you think you've achieved something, you're a dick. Because it means you haven't.”

Is that a fear of success?

“Aye, definitely,” he says, before he checks himself again. “I think so. Well, it could be something like that. It depends what you call success.”

Martin Creed's Words and Music, The Studio, Edinburgh International Festival, August 4-27, 10.30pm

www.eif.co.uk