THE polite and contentedly nerdy world of opinion polling was ruffled this week by a row between two companies that have been monitoring the referendum race:
YouGov and Survation. The spat followed publication of YouGov's latest poll which put support for Yes at 39 per cent and for No at 61 per cent after the don't-knows were excluded. The findings were in line with YouGov's recent polling but they did highlight the wide differences between its results and some of its rivals, inlcuding Survation.
To put it in context, Survation's most recent poll, three weeks ago, put Yes on 47 per cent (a record high) and No on 53 per cent. Since the start of the campaign it's been clear that, of the half dozen pollsters following the action, three (YouGov, TNS and Ipsos Mori) have consistently found a big lead for No, and three (ICM, Panelbase and Survation) a much closer race.
Looking into the differences, YouGov president Peter Kellner said there was a "technical flaw" in Survation's methodology and, in a robust blog, also questioned the other polls suggesting the race is tight. "They are wrong," he wrote. "It isn't. The No campaign is well ahead." Survation defended its methods and accused YouGov of skulduggery, of which more later, but to understand how differences arise at all it's important to consider how the pollsters operate.
Whether they are focused mainly on politics or the commercial world (most polls are actually about who prefers Frosties to Cocoa Pops) all the pollsters have panels of several thousand people to whom they turn to gather about 1000 responses to any given question. Some poll online, others face-to-face or over the phone but, however they obtain their raw data, they then "weight" the results to make them reflect the population at large.
If, for example, relatively few older people reply to an online survey, the responses of those who do will be multiplied to help create a more representative sample. It is fairly straightforward to adjust for age, gender or regional variations but the process becomes more complex where political affiliation - which has a bearing on people's support for independence - is concerned. In referendum polling, raw data is weighted to reflect how people voted in the 2011 Holyrood election. Most companies ask people how they voted three years ago then adjust their raw figures to match the proportion of people who backed the SNP, Labour and so on.
Mr Kellner claimed Survation had failed to weight its results to take account of the large number of "passing Nats", non-traditional SNP supporters who handed Alex Salmond his landslide victory in 2011 because they wanted him as First Minister, not because they supported independence. He claimed Survation was basing its findings on "the wrong mix of SNP supporters" and exaggerating support for independence as a result.
Survation's director of research, Patrick Brione, defended the company's methodology and suggested YouGov may have a surfeit of Labour voters in its panel (which, though he did not make the point, might explain other YouGov findings this week putting Scottish Labour ahead of the SNP in 2016 Holyrood voting intentions). He also voiced his disappointment that YouGov conducted special research to test and criticise Survation.
How much does all this matter? Quite a lot, judging by the two lead campaigns. "Salmond's campaign in crisis," yelled the headline on the Better Together press release about this week's YouGov poll. "The momentum is with Yes," declared Deputy First Minister Nicola Sturgeon after the last Survation survey. It was, she said, a "fantastic poll showing we are closer than ever to achieving a Yes vote".
We'll know which set of pollsters has been nearer the mark on September 19. In the meantime, confused voters will be tempted to consult Professor John Curtice's regularly updated poll of polls, which averages out all the most recent findings, alleged methodological warts and all, to plot a middle course. It puts Yes on 43 per cent and No on 57 per cent.
"The polls will fluctuate from time to time," a Yes Scotland spokesman shrugged in response to YouGov's findings this week. With even the pollsters starting to fall out, there is no arguing with that.
Why are you making commenting on The Herald only available to subscribers?
It should have been a safe space for informed debate, somewhere for readers to discuss issues around the biggest stories of the day, but all too often the below the line comments on most websites have become bogged down by off-topic discussions and abuse.
heraldscotland.com is tackling this problem by allowing only subscribers to comment.
We are doing this to improve the experience for our loyal readers and we believe it will reduce the ability of trolls and troublemakers, who occasionally find their way onto our site, to abuse our journalists and readers. We also hope it will help the comments section fulfil its promise as a part of Scotland's conversation with itself.
We are lucky at The Herald. We are read by an informed, educated readership who can add their knowledge and insights to our stories.
That is invaluable.
We are making the subscriber-only change to support our valued readers, who tell us they don't want the site cluttered up with irrelevant comments, untruths and abuse.
In the past, the journalist’s job was to collect and distribute information to the audience. Technology means that readers can shape a discussion. We look forward to hearing from you on heraldscotland.com
Comments & Moderation
Readers’ comments: You are personally liable for the content of any comments you upload to this website, so please act responsibly. We do not pre-moderate or monitor readers’ comments appearing on our websites, but we do post-moderate in response to complaints we receive or otherwise when a potential problem comes to our attention. You can make a complaint by using the ‘report this post’ link . We may then apply our discretion under the user terms to amend or delete comments.
Post moderation is undertaken full-time 9am-6pm on weekdays, and on a part-time basis outwith those hours.
Read the rules hereComments are closed on this article