OPINION polling, in particular political polling, is all about credibility, which is why the major companies take themselves so seriously.
It is also why, when they make a comprehensive backside of it as they did last week, this is so amusing to the rest of us who carry around such normal human qualities as wariness and self-doubt.
Pollsters can afford themselves none of this. They appear to have become so convinced that they make the weather, rather than simply report it, that when they do as badly as they did last week they feel they have to institute an independently chaired inquiry, as they did last Friday.
The problem for the pollsters is that they are always playing catch-up. There was a crisis of confidence when they got it wrong in 1992, completely failing to spot that John Major was about to overtake Neil Kinnock at the tape.There will no doubt be a time when they get things wrong in the future.
Whenever they get it wrong they tweak their methodology and usually get it about right at the next big set-piece, only for these pesky humans to change - their working lives, their home lives, their internet use, their voting habits - and it then goes belly-up again.
Last week was spectacular in the amazing consistency of the inaccuracy of the polling, with 10 of the final 11 polls released putting the gap between Conservative and Labour at a single percentage point. The final gap was actually 6.4% or more than double the margin of error which might have been expected from a single poll.
But this was after all the full flowering of the British tendency to move towards multi-party politics. Were we in the old two-party days the result would have had little impact on voter behaviour, as at neck-and-neck supporters of neither contender would have an incentive or disincentive to vote.
But the truth is that in the more fluid position involving Ukip and the Greens in England and Nationalists in Scotland and Wales, it is almost impossible to say what the impact of the deadlocked poll findings were.
For the polling companies the embarrassment was profound. "A terrible night for us pollsters," tweeted Stephan Shakespeare, chief executive of YouGov. "I apologise for a poor performance. We need to find out why."
Populus concurred: "Election results raise serious issues for all pollsters. We will look at our methods and have urged the British Polling Council to set up a review."
Three weeks before the election Strathclyde University's excellent, ubiquitous John Curtice, president of the British Polling Council, forewarned in a BPC blog: "Polling is far from being a straightforward enterprise. Those who undertake polls are attempting to provide an accurate measure of the nation's political pulse at a time when people have busier lives than ever, when many are increasingly reluctant to answer any kind of survey, and when no less than three insurgent political parties are enjoying unprecedented levels of support."
That same blog admitted that polls clearly have influence, as voters look at issues such as coalition prospects, and explains why in the face of some calls for a total ban on polls during the General Election campaign period in the wake of last Thursday's debacle the BPC announced that Patrick Sturgis, professor of methodology at the National Centre for Research Methods would be overseeing an inquiry.
US pollster Jim Messina has told the Spectator that he things a key was pollsters' reliance on an either or approach to telephone versus internet polling. That seems a fine point on which to dismiss almost a dozen polls, but he also highlighted the impact of Ukip in the North of England, something everyone is going to have to factor in from now on.
Perhaps the most brutal mea culpa came from Damian Lyons Lowe of Survation, who said his company had conducted a final poll last week to tweak methods - naming constituency candidates, revisiting demographics and using both mobile and landline phones - and got a breakthrough result.
As he put it: "The results seemed so 'out of line' with all the polling conducted by ourselves and our peers - what poll commentators would term an 'outlier' - that I 'chickened out' of publishing the figures, something I'm sure I'll always regret."
I'm sure you will, Damian. I only hope that given you chose not to publish this astounding intelligence you had the good sense to use the info to stiff the bookies, the only folk in our society whose reputation is down there with pollsters and journalists.
Why are you making commenting on The Herald only available to subscribers?
It should have been a safe space for informed debate, somewhere for readers to discuss issues around the biggest stories of the day, but all too often the below the line comments on most websites have become bogged down by off-topic discussions and abuse.
heraldscotland.com is tackling this problem by allowing only subscribers to comment.
We are doing this to improve the experience for our loyal readers and we believe it will reduce the ability of trolls and troublemakers, who occasionally find their way onto our site, to abuse our journalists and readers. We also hope it will help the comments section fulfil its promise as a part of Scotland's conversation with itself.
We are lucky at The Herald. We are read by an informed, educated readership who can add their knowledge and insights to our stories.
That is invaluable.
We are making the subscriber-only change to support our valued readers, who tell us they don't want the site cluttered up with irrelevant comments, untruths and abuse.
In the past, the journalist’s job was to collect and distribute information to the audience. Technology means that readers can shape a discussion. We look forward to hearing from you on heraldscotland.com
Comments & Moderation
Readers’ comments: You are personally liable for the content of any comments you upload to this website, so please act responsibly. We do not pre-moderate or monitor readers’ comments appearing on our websites, but we do post-moderate in response to complaints we receive or otherwise when a potential problem comes to our attention. You can make a complaint by using the ‘report this post’ link . We may then apply our discretion under the user terms to amend or delete comments.
Post moderation is undertaken full-time 9am-6pm on weekdays, and on a part-time basis outwith those hours.
Read the rules hereComments are closed on this article