There's a lovely line in James Yorkston's Woozy with Cider, a lucid 2006 post-party psalm.

"The world is not exactly going to be leaping out of its bed to make me rich," lulls the East Neuk raconteur, over a down-time disco beat. These modest, existential words are typical of our self-proclaimed purveyor of "slow songs for a select audience", but the music is significant too. It is a warm electro hymn from a man who has long been pegged as a folk bard, despite being equally (if not more) indebted to krautrock, Malagasy guitars and punk.

Yorkston's new album, I Was a Cat From a Book, is his fifth solo outing for Domino Records, home to Franz Ferdinand, Arctic Monkeys and enduring Fife ally King Creosote. It is 10 years since he released his Domino debut, the recently re-issued Moving Up Country, and longer since he found champions in John Peel, Bert Jansch and John Martyn, but the new record resonates with the Yorkston traits that struck a chord back then, and have done ever since, from his enthralling, wandering psalms to the recurring cat motifs.

But this album also turns over new ground. For starters, his regular house band, The Athletes has been supplanted (for "various unsavoury reasons"), by a jazz trio. "Making this record has been a completely different process for me," says Yorkston over peppermint tea in a Cellardyke harbour pub. "I was keen to break out a bit – I wanted to make a live-sounding album. I love percussion, but I love it if it's buried, and delicate, and driving, and interesting. If it's just meat and two veg drums, I cannot stand it. That's why we got the jazz band in. The jazz band," he chuckles.

The striking, intuitive troupe in question comprises Lamb's Jon Thorne on double bass, the Cinematic Orchestra's Luke Flowers and John Ellis on percussion and piano respectively, with exquisite string arrangements courtesy of violin diviners Geese. "Emma [Smith, of Geese] has played with me since 2006, and I've kind of been leaning on her a bit more as a musical accomplice recently," says Yorkston. There are also gorgeous vocal cameos from Kathryn Williams and Sparrow and the Workshop's Jill O'Sullivan.

A version of the balmy O'Sullivan- Yorkston duet Just as Scared featured on last year's excellent Fruit Tree Foundation compilation, which also included contributions from Karine Polwart, Emma Pollock and Alasdair Roberts, plus members of The Twilight Sad and Frightened Rabbit. Did Yorkston always plan to revisit Just as Scared on the new record? "My dad always said he liked that song, so I thought, 'Well, why not put it on?' It's nice and poppy. And it's a bit lighter than some of the other songs on the album," he laughs quietly. "There's no harm in having a little bit of light."

This is the part that is hard to write. Yorkston's new album, though frequently beautiful, and often uplifting, is shot through with darkness. The Fire and the Flames in particular is devastating, and no parent should ever have to sing lines like, "All I want is for you to be well, my love" or, "The look in your eyes that says, 'why do you let them hurt me so?'" let alone discuss them in a pop interview. Suffice to say, at the time of our speaking things are looking brighter for Yorkston's young daughter, who has been seriously ill for the past two years.

Along with a sense of heartbreak, and relief, there is also anger on the album – and catharsis – but it was ever thus. Yorkston has long made incendiary songs along with those he calls the quiet ones, to the extent that you might be tempted to align his (admittedly largely acoustic) music with punk. Is that a reasonable charge? "It sounds ridiculous saying it, because sonically it doesn't sound punk-rock, but it was definitely always an influence," he nods. "Bands like Dead Kennedys, that first Dag Nasty album, I absolutely loved all that. But it was more about the attitude."

Yorkston's ardour for sub-cultural pop is rooted in a musical youth shared with his childhood best friend, the BBC presenter Vic Galloway. They grew up together in the Fife village of Kingsbarns and performed in 1990s combos Miraclehead and Huckleberry. "Vic and I wrote our first song when we were about six," he says. "It was called Baa Baa Baa." He croons said farmyard ode into the tape recorder. "We wrote that on a one-stringed guitar and a banjo." It sounds like a proto-Fence masterwork. "Yeah, we were laying down the boundaries for that," he laughs.

Later, the comrades would turn to punk. "The first band I ever saw was The Damned, on their [1985] Grimly Fiendish tour. Vic's mum drove us to Edinburgh for it. The ethos I took from punk was that whole, 'Do your own thing, follow your own will', and I still consider that part of my music. I'm doing what I want."

Yorkston's long-term affiliation with King Creosote's Fence Collective underscores his DIY credo (Fence have released some of his recordings, and Yorkston performs as The Three Craws in cahoots with KC and The Pictish Trail), and there is something of the punk aesthetic in his take on traditional folk songs too – most notably his carnal 2008 account of Lal Waterson and Oliver Knight's glorious Midnight Feast, which re-casts it as a lusty (or lustier) shanty, all ravenous cravings, moonlit climaxes and wanton kosmische swells. "Sometimes when I'm playing with the band it can break into a krautrock thing," he says.

Although he's now based in Fife, Yorkston spent 17 years in Edinburgh, and credits the city's music library with his love for folk singer Anne Briggs and krautrock alchemists Can. "There was also this Madagascan guitar player called D'Gary," he adds. "It was actually my brother who got D'Gary out of the record library, but he wrote his name down wrong. We thought he was called M'Gary for years."

Yorkston is an intricate, fierce acoustic guitarist – did he learn to play by imitating D'Gary? "Yeah, absolutely, I tried to replicate him, I tried to copy him, and I couldn't because he's absolutely extraordinary. Him and Mississippi John Hurt, they were the ones."

Last year, Yorkston published a terrific memoir, It's Lovely to Be Here: The Touring Diaries of a Scottish Gent. At the time, we discussed how writing a book had impacted on his songwriting, and he said he was trying to write simpler songs. How did that pan out across the new album? "I was thinking about this yesterday," he offers. "The thing is, I tend to write more complicated songs like [the new album's] Catch, or Border Song, or Shipwreckers [from 2004's Just Beyond The River], where you really just fill them full of lyrics, and I love that.

"So if I'm writing other things I have to pull myself back. On the new album there's Kath with Rhodes, A Short Blues and Two – they've got a lot less lyrics, but they're not naturally where I'm going. I'm happy that they're there, they're poppy, and they're easy to understand, but as a general rule I tend to prefer the ebb and flow of the more lyrical pieces."

Ever the salesman, Yorkston hails his new album as "not an embarrassing duffer". And there are caveats. "There's one word on it I'm not so happy with. Actually, there's a guitar part that's slightly too loud as well." He laughs quietly. "I don't take myself too seriously, but I do take my music seriously, and I put a lot of effort into it. There is humour in there, but they're not jokey songs."

Yorkston's lyrical humour is a lot like he is: gentle and droll. You can hear it in wistful lines like "I miss your short-skirted dignity," from 2008's chamber-ballad Summer's Not The Same Without You, or "Finally my Catholic brain kicked in on I Was a Cat's motorik-folk aria, Spanish Ants. You can hear it in Woozy with Cider's warm humility: "It'll be interesting to see if anyone ever bought those songs of mine; if anyone heard those words that I never got quite right," he sings.

Some of us have frequently leapt out of bed for Yorkston's non-duffers; to buy those songs; to hear those words. We always will.

I Was a Cat From a Book is out on August 13 via Domino. James Yorkston plays the Queen's Hall, Edinburgh, on September 21.