How polls turn a lead for Yes into a lead for No
Speymouth
You may have seen that a poll on support for Scottish independence was published this week. The headline figures on the poll, carried out by Survation, put No ahead on 44%, with Yes on 43% and undecided on 12%. When the undecideds were stripped out, this gave a result of 50.57% (rounded up to 51%) No, and 49.42% Yes (rounded to 49%), cue the usual jeering on social media by the staunch brigade and gnashing of teeth by pro-independence opponents of the SNP that John Swinney is throwing away Scotland’s chances of independence.
However Survation is one of those polling companies which maintains the increasingly indefensible methodological practice of weighting its results according to the outcome of the 2014 independence referendum. As I explained in a previous blog piece, this methodology has the effect of artificially inflating the result for No while depressing the result for Yes. You can read that piece here. I explain why poll weighting according to the outcome of a vote held eleven years ago effectively undoes the demographic changes which have occurred in Scotland over the past eleven years, and produces a misleading result.
Scotland is on the threshold of real change when Yes has a double digit lead
Now the person behind the Independent Voices BlueSky account @indyvoices.bsky.social has very helpfully calculated the effects of weighting according to the outcome of the 2014 referendum on Survation’s results and showed how in so doing the polling company turned a poll which had a Yes majority into a poll with a No majority. Independent Voices produced the following graph, which I publish here.
Thankyou to @indyvoices.bsky.social for the graph
As you can see from the graph, Survation’s raw data, its unweighted sample in the first set of columns, actually showed a Yes majority of 7%. However when the numbers were crunched and the 2014 weighting was applied, this 7% Yes majority vanished and was replaced by a 1% lead for No, the second set of columns. Those are the figures which were splashed all over the newspapers, figures which give the misleading impression that opposition to independence maintains a narrow lead.
This skewed result is what happens when you employ the fiction of pretending that 550,000 predominantly older people who disproportionately would have voted No hadn’t passed away and been replaced in the electorate by younger people who are more likely to support independence. That’s effectively what a polling company is doing when it weighs its results according to the 2014 referendum. But back in the real world those 550,000 predominantly older people who would disproportionately have supported No really have passed away.
The third set of columns in Independent Voices’ very useful graph shows the raw sample weighted by the current demographics of Scotland, This too produces a majority for independence, this time of 4%.
Polling for Scottish independence that is weighted by the outcome of the 2014 referendum produces results which inflate the headline figures for No and which are ever less a true reflection of the real state of Scottish public opinion as the years go by. The Grim Reaper has a big effect on the oldest age cohorts, a demographic group which numerous polls have told us are far more likely to oppose independence. Further studies have demonstrated that the younger independence supporters who replace older No voters in the electorate do not tend to morph into No supporters as they age. A very real change is taking place in Scotland, as the years go by, Scots are increasingly supporting independence. Polls which weight by the outcome of the 2014 referendum discard these real changes and present a fictitious picture of a Scotland that’s eleven years out of date.
We are now at the stage where independence polls which continue to deploy this methodology should be dismissed out of hand. This includes polls by Survation and polls by YouGov, both of which use this by now indefensible methodology.
Why do polling companies keep doing this, you might reasonably ask. It’s not some Unionist conspiracy designed to keep independence down and burst the bubble of the independence campaign’s morale, although it certainly has that effect. It has more to do with inertia and the natural reluctance of people to change their habits. Weighting by the results of the 2014 referendum made good methodological sense in the months and first couple of years after the independence referendum, but as the years have passed it produces results which are increasingly out of kilter with the true demographic make up of Scotland in 2025. Approximately 14% of the total electorate in 2014 has passed away, overwhelmingly older people who predominantly voted No.
There is also the not inconsequential matter of the deeply distorted nature of Scotland’s media landscape. As is well known by any independence supporter, only one of Scotland’s newspapers supports Scottish independence, the rest are universally hostile to the idea, sometimes hysterically so. Opinion polls on the independence question are typically commissioned by a newspaper editor. Do you honestly think that when they commission a poll on independence, the editor of the viscerally anti-independence Scotsman or the Herald whose comments section is stuffed to the gills with outright British nationalist fascists, is going to tell YouGov or Survation not to use a methodology which is more likely to produce a result which panders to the anti-independence prejudices of their readers? Of course they won’t. It suits them to have a poll to publish which puts No in the lead, even if, as is the case with this latest Survation poll, that apparent No lead is entirely an artefact of the methodology used by the polling company and with different and arguably more accurate and up to date weighting, the exact same poll would have shown a majority for Yes.
Given our media landscape, Scotland is going to keep getting polling on the independence question which artificially wrests Scotland back to a country which no longer exists, undoing the demographic changes of the past eleven years, and which give the false impression that support for independence is lower that it really is.
The first thing any independence supporter needs to do when confronted with a poll on independence, irrespective of whether it puts Yes or No ahead, is to check its methodology. If the poll weights by the outcome of the 2014 referendum then you can be certain that it is underreporting the Yes figures and they are most likely to be higher, it will also be overstating the No figures, which in reality will be lower. If the poll shows a narrow lead for No then in reality this will probably reflect an actual lead for Yes.
___________________________________________________
A reminder that I am currently running the annual fundraiser for the blog. You can donate to my GoFundMe fundraiser, the link is here. Please consider making a donation.
I’ve added a new method to make donating easier, you can now donate via Ko-fi
Or click the following link https://ko-fi.com/weegingerdug
https://www.gofundme.com/f/wee-ginger-crowdfunder-2025
Alternatively you can help to support this blog with a PayPal donation. I hope that you will consider making a regular monthly donation, there is an option on PayPal which permits that. It’s an immense help to me when it comes to budgeting. Please log into Paypal.com and send a payment to the email address [email protected]. Or click the Pay Pal donate button below. If you don’t have a PayPal account, just select “donate with card” after clicking the button. You can also donate by PayPal by using my PayPal.me link PayPal.Me/weegingerdug
https://www.paypal.com/paypalme/weegingerdug
If you would like an alternative method of donation, please email me at [email protected] for details of how to donate by direct bank transfer or cheque.
Thank you all very much, whether you support this blog by donating, or by reading and sharing my work. It means more than you know.
