So Margaret Thatcher's lobbying for Nelson Mandela was not, despite recent claims, something on which a prisoner would pin his hopes.

So the Tories planned, three decades ago, to cut Scotland's funding on the sly. So Arthur Scargill was dead right in 1984 about a secret list of pits marked for closure.

The release of papers under the (now reformed) 30-year rule is one of the fun parts of any new year for journalists and historians. Without fail, several pictures of the world are readjusted or confirmed. Without fail, you are given a reminder of the difference between what politicians say and what they get up to behind closed doors. Then you are left with a puzzle.

Loading article content

Governments keep secrets: that's inevitable. Sometimes they have no choice in the matter. Issues of national security, privacy and confidentiality arise. On occasion, honesty might suit one department while another prefers to dissemble. "Reputation", the abiding shibboleth for politicians, is always treated as a reasonable justification for all manner of secrets.

That doesn't explain, though, why those in government are so secretive about their actions when issues of principle and conviction are at stake. Margaret Thatcher believed absolutely the National Union of Mineworkers must be broken and their industry rendered "lean". By 1984, pit closures had become frequent and familiar. So why not lay the arguments for rationalisation before the public?

Instead, while telling the country that only 20 mines were at risk, the government of the day had reached a decision before the strike began to shut 75 pits over three years. In other words, Mr Scargill's claim that a "hit-list" existed was entirely correct. Mrs Thatcher, telling mining families that her fight was not with them even as she contemplated 64,000 job losses, was dishonest.

The then prime minister did not lack media support. In Mr Scargill she faced an opponent who was not overburdened with charm and charisma. Some pits - though far fewer than 75 - were certainly uneconomic. This, too, was the Mrs Thatcher who proclaimed herself a conviction politician on a mission to save the country. But the existence of the authentic closure plan was denied and denied repeatedly.

Clearly, this was one of those little problems of democracy that politicians find so frustrating. Mrs Thatcher believed with absolute certainty that she was right, but there was a risk that the public might not be persuaded. Was there really an economic case for spending billions to defeat a trade union while tearing Britain apart? Were the majority really prepared to regard the mining communities as "the enemy within"? The evidence says that Mrs Thatcher did not want to hear the answers to these questions.

You can find a parallel (not for the first time) in the career of Tony Blair, not least in the saga of Iraq. Having started work back in 2009, Sir John Chilcot is due to publish the findings of his inquiry into the handling of that conflict later this year. Much of the delay, extraordinary even by British standards, has been due to arguments over the release of correspondence between Mr Blair and President George Bush. This might tell us how much support for the American position was offered by the prime minister and when was it offered?

But why should that cause difficulties? Mr Blair, like Mrs Thatcher, has often used his convictions to explain his actions. Time and again he has said that he took Britain into the Iraq war on America's heels because "it was the right thing to do". So why has something so right depended so often on so many half-truths and outright deceits? Shouldn't Mr Blair be eager now for vindication from Sir John?

As things stand, the full correspondence between a president and a prime minister will not be published for many years to come. A "compromise" has been reached, reportedly, between Sir John Chilcot and Sir Jeremy Heywood, the Cabinet Secretary. Whitehall's wish to have the material suppressed entirely has been overcome, but full documentation of Mr Blair busy following his conscience, as he would have it, will not be provided. And this is not, as was once suggested, because of any great reluctance on the part of the US and Mr Bush.

What do virtue's claims amount to when they depend on years of secrecy and obfuscation? Mr Blair is often given to portraying himself as misunderstood. Mrs Thatcher had a similar taste. The public has been told, in effect, to suspend judgement of circumstances and acts it cannot understand by politicians ensuring that understanding would remain impossible. Such is the real argument over secrecy and transparency.

It has not been resolved by freedom of information (FoI) legislation, not with the system of exemptions maintained in Britain. The more rapid release of "30-year" documents with a move towards a 15-year rule and the inclusion of material such as Cabinet minutes counts as progress, but something less than a revolution. Politicians up to no good will still find ways to keep their discussions secret: FoI doesn't cover backstairs chat. The seeming puzzle of conviction and conscience remains.

In effect, a Blair or a Thatcher says simply this: "I'm right and the public is wrong." When it matters, the public does not count. Nor, for that matter, do most of the people elected by the public. The publication of the Chilcot findings is predicted to lead to demands for greater parliamentary oversight of the executive's affairs. A fine idea - if the legislature knows what it is supposed to be overseeing.

Recent pathetic attempts in Westminster to get some answers from the security services over the extent and purpose of domestic surveillance are not encouraging. No doubt the men who run MI5, MI6 and GCHQ regard themselves as dedicated public servants whose only desire is to protect us from the worst the world can offer. They, too, act from conviction. That does not mean we can or should take them at their word in every circumstance.

Mrs Thatcher's unbending self-belief in her battle with the miners inflicted vast social and economic damage. She was wrong. There was no excuse for turning an argument over industrial reform, however bitter, into a vendetta. That she refused even to admit the existence of the real ambition at the heart of her plans for coal mining is telling. Her conscience did not run to honesty.

Few now believe that Mr Blair was right about Iraq. Scarcely any believe that he told the truth while preparing, in secret, to take Britain to war.

Yet the cause was just, so he still maintains. In short, he subscribed to the old, shabby doctrine that ends justify means. The fact that the ends were catastrophic is something that the former prime minister simply "regrets".

The irony in all of this is that there is only one alternative to the conviction politician. There are plenty of examples of the alternative to hand.

Would we rather have the politician who believes in nothing very much? Or are we better off always mistrusting those who never tire of flaunting their beliefs even as they conceal their true purpose?