WE should probably start with a list because that’s how Sir Malcolm Rifkind would do it. The long-serving Scottish politician was defence secretary, foreign secretary and Scottish secretary under Margaret Thatcher and John Major, but for several years he was also an advocate in Edinburgh, which is obvious whenever he talks. His arguments come with footnotes and lists; we talk about Thatcherism, independence and the future of Scotland, and every one of the answers seems to be broken down into 1s, 2s and 3s, and then further broken down into a chain of clauses and sub-clauses. Sir Robin Day once said, in his famously dismissive way, that Malcolm Rifkind was “just a smart Edinburgh lawyer” and, in a way, it’s still close to the truth. Sir Malcolm is a politician, but he’s a politician built from brain and synapse, not heart and sinew.

So here, for Sir Malcolm’s benefit and ours, is a list of what we’re going to talk about today. 1) The prospects of the SNP – there’s a bomb under the party, says Sir Malcolm, and it’s ticking. 2) The chances of another independence referendum – the Nationalists have an even higher barrier to climb than they did two years ago and if it happens, they’d lose again. 3) Sir Malcolm’s role in putting Thatcherism into practice in Scotland – he has few regrets and some pride, especially on council house sales. And 4) that Channel 4 sting in which Sir Malcolm was accused of taking cash for access – he was angry and upset and did nothing wrong. We talk about other things too – the “extraordinary” idea of making Boris Johnson foreign secretary and more personal matters such as his wife’s multiple sclerosis – but the main focus today is Scotland as it was, is, and will be.

It just so happens we’re talking about the subject 250 miles south of Scotland, in the Cotswolds where Sir Malcolm, who’s 70, is on holiday with his children and grandchildren. His wife Edith, who’s in a wheelchair, is inside watching the Olympics; his son Hugo, a journalist, is upstairs writing, and we are sitting on the terrace with a cup of tea and a superb view out over the sunny hills of this most Conservative part of England.

Sir Malcolm has come here for a few days after the publication of his memoirs Power and Pragmatism, which cover, in great detail, his political life and 18-year ministerial career. The book has been called plodding and dull by the critics, but Sir Malcolm insists he doesn’t mind and admits to being a bit of a square. He was educated in Edinburgh in the 1960s and Princes Street, he says, wasn’t exactly Carnaby Street. “And if I didn’t have drugs,” he says, “it was partly because nobody ever offered me any.”

This earnest and serious streak makes for an earnest and serious book but it does include interesting detail about his career: the fact, for example, that he has always been a strong supporter of devolution and resigned from the opposition front bench over devolution when Margaret Thatcher insisted her MPs vote against a Scottish Assembly. However, he then appeared to do a volte-face in government and did absolutely nothing for devolution for 18 years. Shouldn’t he have pushed more for it while he was a minister?

“It’s a fair question and it’s something I gave considerable thought to,” he says. “It was not simply a question of persuading Margaret Thatcher or the cabinet to look again at devolution, I had to accept a substantial majority of Scottish Conservatives were against it. And it wasn’t dominating Scottish politics – if it was, why weren’t 100,000 people marching down Princes Street saying ‘We want a Scottish parliament’?”

Twenty five years on, everything has changed and Scottish politics is about devolution, independence and little else but Sir Malcolm is much less directly involved than he was. He doesn’t have a home north of the Border any longer (Edith’s MS means they had to sell their place in Edinburgh); he has also been caricatured as an unScottish figure and you can see why. There’s his voice for a start, which is marinated plum, and the fact he still looks like one of those Thatcherite viceroy toffs of old – the clothes are weekend-in-the-country, and the face is where the cartoonist Gerald Scarfe gets all his ideas from. But don’t be fooled – the former Scottish secretary keeps a close watch on Scotland’s affairs and has an interesting take on where we are going next.

So, to the list, and item one: the prospects for the SNP. Sir Malcolm says they don’t look so good because of a very particular problem. The Nationalists, he says, have been espousing left-wing policies for a long time, but there will come a point when large numbers of Scots get fed up with it – in fact, he thinks the process has already started.

“In order to break the Labour vote, they have become the most left-wing party in Scotland,” he says. “That cannot continue indefinitely. They’re anti-austerity, they’re anti-Trident and pretty much on every other trendy issue they go for the left-wing view. But in Perthshire, in Galloway, in the north-east of Scotland and elsewhere, people are not prepared to see themselves, not just as left wing, but pretty hard left.”

He says it amounts to a political bomb. “And it’s already begun to tick. What we saw at the last election wasn’t just that the Tories won votes, they won back votes they’d lost to the SNP and that process has begun. I’m not saying it’s 100 per cent certain it will continue, but what the last Scottish election showed is the public, if the Tories get their act right, no longer say: ‘We will never vote Conservative.'”

Which leads to item two: the possibility of another independence referendum. Sir Malcolm is bullish – he believes the chances of another referendum, and the SNP winning it, are low. “Not only is it a complete try on but I don’t think Nicola Sturgeon wants one because she knows she would lose it,” he says.

There are three factors, he says, mitigating against her calling a referendum. “The first,” he says, “is the collapse of the oil price and the £10-12 billion hole in the Scottish budget which would have to be funded by substantially higher taxation. They have nothing to say on that, absolutely nothing.

“The second is the currency. If the whole assumption is that Scotland would be outside the UK but in the EU, the idea that we would share a currency with the UK is bizarre. It can’t possibly function – they would either have to have a Scottish currency or join the euro.

“The third factor is the hard border. Two years ago, the Nationalists were able to say: ‘If we become independent, you won’t notice anything, we’re all part of the EU, the border will be invisible.' Now, if the whole assumption is we want to be in the EU, that means a hard border.”

Sir Malcom also believes the quasi-federal UK we’ve ended up with will probably convince Scots to stay. He points out in his book that he has supported some kind of federal structure for the UK since his days as a member of a group of young progressive Conservatives called the Thistle Group. When Ted Heath appointed a commission to consider an assembly, the Thistle Group was asked to contribute and Malcolm Rifkind was one of its most radical voices.

Of course, it was this radicalism that made Sir Malcolm’s relationship with Margaret Thatcher difficult – she says in her memoirs that she was suspicious of his pro-devolution views and I ask Sir Malcolm whether he liked her. “Yes,” he says. Then he thinks again. “Did I like her? I admired her.” The problem, he says, came during his time as Scottish secretary. “She saw my job as being the cabinet’s man in Scotland, which I was,” he says, “but there were two parts to the job – I was also Scotland’s man in the cabinet.”

Which takes us to item three: Sir Malcolm’s role in implementing Thatcherism in Scotland. He tells me he has never been a Thatcherite but at the same time he is happy to say that, when he thought the policies were right, he pursued them. He also believes Scotland is not as anti-Thatcherism as it thinks it is.

“The right to buy your council house – hugely popular. Privatisation and buying shares in privatised industries – the Scots queued up as much as the English and Welsh to buy their shares. Reducing income tax – just as popular in Scotland as down south. But they felt guilty and said, ‘Don’t worry, we’re still Labour.’”

Sir Malcolm says he is particularly proud of the right to buy, which he introduced as a junior minister in 1980. At the time, he says, Scotland was lumbered with a paternalistic culture that meant levels of home ownership were lower than in Communist Hungary. He also remembers what some of the big estates were like in the 1970s: huge, monocultural places that were run with authoritarian zeal. Afterwards, the council estates were still there, but they were socially better and healthier than what had gone before, he says.

Sir Malcolm dismisses the Scottish Government’s decision to end the policy. “I think it’s symbolic and pretty silly and doesn’t make much difference as the vast majority of people who want to buy their council home have already done so. But what they can’t escape from is that Scottish home ownership is no longer 30 per cent – it’s about 70 per cent. And many of the people who vote SNP – they or their parents bought their council house.”

There are other parts of his legacy he is less pleased about – the poll tax, for example – and I ask whether he feels responsible for the decline in industry in the 1980s. “No, because the process had begun before we arrived – it was happening all over western Europe. The idea that you would still have coal mining, the idea that Scotland would still be producing steel when south Wales is facing the end of steel making – the most you could argue is the timing might have been slightly different.”

Basically, Sir Malcom believes Thatcherism made Scotland a better place, but, sitting here in the Cotswolds, it’s starting to feel like a long, long time ago. Sir Malcolm left government when he lost his seat in the Tory wipeout of 1997, and became the member for Kensington and Chelsea in 2005. He then stood for the Conservative leadership when Michael Howard stood down, but never had much of a chance. So instead of becoming leader, Sir Malcolm went where old Tories go to die: he became a grandee.

That might have been how the story ended, with Sir Malcolm as the chairman of the Intelligence and Security Committee, but there’s the little matter of item four on our list: last year’s Channel 4 and Daily Telegraph sting against him and Labour’s Jack Straw. The MPs thought they were speaking to a communications company about setting up an advisory council, but they were really speaking to undercover reporters who alleged the MPs were willing to accept “cash for access”. Sir Malcolm found himself vilified when he appeared to dismiss the money he earned as an MP. “Nobody pays me a salary,” he said. “I have to earn my income.”

Sir Malcolm’s response to the claims at the time was that any fees that were discussed were for his position on the council and as such were perfectly proper, and in due course he was cleared by the standards commissioner, but does he understand that people did not necessarily see it that way? “All people saw was the way Channel 4 and the Telegraph presented it, which was hidden cameras, secret revelations, all the hyperbole,” he says. “I’m not against investigate journalism, but there has never been a sniff of a suggestion that either Jack or I had acted in any improper way. And there was nothing in the conversations we had that implied any impropriety.”

I wonder, though, about the comment about “not having a salary” – that was a mistake, wasn’t it? “If you look at the transcript, I did say that but that’s when we were talking about my business interests. I knew perfectly well I had a salary – it would be absurd for me to say I didn’t. At that stage, I was saying that in my business interests, I would have been a non-executive director getting a fee, not a salary, or I would have been doing consultancy work in which case I’m paid in relation to the particular work that I do, but I don’t get a salary. They took a couple of sentences which I said and presented it in a context that made it sound absurd. I’ve been in public life for 45 years and it’s the only occasion when my integrity and reputation has been attacked.”

Sir Malcolm stood down from his constituency of Kensington at last year's election and says he isn’t necessarily bitter and in many ways he’s quite happy to have moved on from the Commons. These days, his time is taken up with a few business interests and family. There is a carer who helps look after Edith, who has been in a wheelchair for two years, having been diagnosed with MS 20 years ago. She has coped with the condition fantastically, says Sir Malcolm, although their way of life now has to centre of what she is capable of doing. “I remember when she was first diagnosed, the consultant said: ‘Edith will have the greatest challenge,’ but he turned to me and said, ‘Don’t underestimate the way it will change your life.’ And he was totally right.”

Edith’s condition means the couple, who have two children, Hugo and his sister Caroline, live permanently in London, which also makes it easier for Sir Malcolm to keep in touch with his business and political interests as well as what’s going on in the Tory party. He’s keen on Ruth Davidson but less so on Boris Johnson. It was an extraordinary and ridiculous decision to appoint him as foreign secretary, wasn’t it? “It’s extraordinary but it’s not ridiculous. It depends on Boris. He’s very cosmopolitan and has an internationalist view but that doesn’t necessarily make him a good foreign secretary. He’s made his reputation as a celebrity and you can’t be a good foreign secretary if that’s your priority. He has to reinvent himself.”

As for Sir Malcolm, reinvention is no longer necessary. Here he is, in the Conservative Cotswolds, being himself: an old-fashioned, pragmatic Conservative. In her memoirs, Margaret Thatcher called him sensitive and highly strung. Is he? “Sensitive yes but I’m not highly strung.” Mrs Thatcher also said he was unpredictable and he’s much happier to accept that description. Ideologues are always predictable, he says, because you know what they will think on any issue and he likes to think of himself as different. He did what he did not because he thought it was right, but because he thought it would work.

Power and Pragmatism: The Memoirs of Malcolm Rifkind is published by Biteback, priced £25.

Sir Malcolm Rifkind

Born June 21, 1946 in Morningside, Edinburgh, to Jewish parents

Studied law at Edinburgh University from 1963 to 1967 where one of his contemporaries was Robin Cook. During his time there, he appeared on University Challenge.

He was elected MP for Edinburgh Pentlands in 1974 and was appointed junior minister in the Scottish Office in 1979. He was then promoted in 1986 to secretary of state for Scotland. He was later defence secretary and foreign secretary.

He lost his Pentlands seat in 1997 but returned to the Commons in 2005 as MP for Kensington and Chelsea. He did not stand in 2015.