Archive for the 'Pollsters/polling' Category


How old men being available on Friday nights to do online polls might be skewing results

Tuesday, May 24th, 2016


Very early responders to poll invites might not be representative

After YouGov’s methodology changes last week ICM have announced their own measures as we approach the big day.

This is the firm’s Martin Boon he explains it on the pollster’s website:

“..Interviews tend to build up quickly on each Friday night, probably because certain types of people are more readily available and willing to participate. Indeed, there is a remarkable consistency across our online polls, with big Leave leads being built up in each hour from 4pm to 9pm on a Friday, partially mitigated by big Remain In leads every hour thereafter until the survey closes, ostensibly by Monday morning for data delivery to clients.

We believe it likely that the weight of interviews generated before 9pm on a Friday has the effect of consolidating a Leave lead as a result of the survey process itself – demographic quota cells fill up and ‘close’ once the target number has been hit. If a specific cell, such as 65+ men, is filled early with people disproportionately likely to support Leave, no additional 65+ men will subsequently be allowed on the survey. As a result, interviews with 65+ men are unlikely to be politically or attitudinally representative of all such 65+ men even though in demographic terms they are identical. But they are not, and their presence possibly introduces a small skew into in favour of Leave (or UKIP, depending on the question looked at).”

As a result the pollster is to stagger the release of invites to take part in its political polls and also to introduce a new weighting.

“..However, it is unlikely that process change outlined above will solve the problem other than partially. Respondents more inclined to Brexit may be equally fast to respond to their invite at other times during the weekend, thus still affecting the data but less overtly. As a consequence we are overlaying a new weighting scheme to reflect the profile of response by quickness to participate.

We will not publish full technical details of this weighting scheme, for fear of conditioning its power. However, we will be applying a “time of response weight” to reflect disparity in response between early responders and late responders. The net effect of this weight, so far, has been to reduce the Leave share by up to 2-points, with a corresponding increase in the Remain share by up to 2-points. It is entirely possible that the strength and direction of this weighting effect will change, if the pattern of response changes on any individual survey.”

The latest poll sees IN and OUT level pegging following a 4% OUT lead in last week’s poll.

Mike Smithson


Perhaps Leave really are going to win this referendum

Sunday, May 22nd, 2016

Vote leave badge

What if the phone polls are wrong?

Earlier on this year I spoke to someone who works in the polling industry, they ruefully observed that the 2015 general election opinion polls accurately predicted the Tory victory, so long as you ignored the headline voting intention figures and focussed on the supplementaries, and I’m starting to wonder if we might be seeing a similar situation with this EU referendum, where whatever the headline voting intention figure is, the supplementaries are generally consistent.

So if we assume that all the polling problems of 2015 haven’t been ironed out (and the modal difference bears that out) we look at some of the other polling findings (which appears to be consistent across all pollsters, whatever the mode of polling)

  1. David Cameron’s personal ratings especially with Tory voters, since he announced his EU deal have fallen off a cliff at at a rate that would leave lemmings panting for breath
  2. David Cameron who is fronting the Remain campaign is now seen as the most distrusted UK politician on EU matters
  3. Whilst Boris Johnson, the front man of the Leave campaign, is seen as the least distrusted UK politician on EU matters
  4. Leave voters in a range of polls are shown to be more motivated and, and thus more likely to turn out to vote than Remain voters, and also least likely to change their mind
  5. Immigration is the most important issue facing the country according to the voters, and that’s not an issue Remain can win on as Leave is seen as the best option for reducing immigration to the UK according to the voters
  6. The polls show the over 65s are in favour of Leave and younger voters are in favour of Remain. The Old v Young split was felt by many to be crucial to the Tories winning the 2015 general election.
  7. The online polls were accurate in the London Mayoral election
  8. The local election results did not reflect the significant leads the Tories have in most opinion polls.

So we have on one side a relatively popular & trusted leader versus an unpopular & untrusted leader, whilst older voters are backing the popular & trusted leader, and the younger voters backing the unpopular & untrusted leader, and the online polls generally show it neck and neck, sound familiar? Is this this referendum the 2015 general election all over again?

If the opinion polls continue to show that economy is the most important decision influencing voters, and Brexit is seen as being bad financially and economically for individual voters and the country, I expect Remain to win, with class not age being a predictor of election outcomes, and among classes that vote Remain is winning comfortably.

But were Leave were to win, some if not all of the reasons I listed to above might explain why Leave won. At the time of writing, Leave winning were a 21% implied chance of winning the referendum, it might be prudent to back Leave if you think the phone polls are calling this referendum wrong.



The extraordinary public battle between two of the men behind the creation and success of YouGov

Friday, May 20th, 2016


If it was just online polls the referendum narrative and the betting would look very different

Friday, May 20th, 2016

EU Ref polling   Google Sheets

This week being the third of the month has seen a glut of phone polls. We’ve had ICM, ORB, Ipsos MORI and last night ComRes all showing substantial REMAIN leads.

Inevitably this has had an impact on the huge EUref betting markets where on Betfair alone getting on for £0.5m is being wagered each day. As I write REMAIN’s chances are rated by the market at 79% just about the highest it has ever been.

But what if there were no phone polls and we just had to rely on the cheaper internet ones. The table above shows how the last few weeks would have looked. The race would have appeared to be neck and neck and the betting would have had it much much closer.

I get asked many times a day in Tweets like this which mode I think is best and I’ve now got a standard response.

The big thing, of course, is confirmation bias. People, as we see so often on PB and I am as guilty as anyone, trust most the evidence that supports their position.

The very successful performance of the online polls in the May 5th London Mayoral race reinforces the point that this form of polling can be very accurate indeed. The only phone poll had a Sadiq margin much bigger than the actual result.

At the moment I’m not betting on the referendum. I don’t regard the REMAIN price as value.

Mike Smithson


Zac Goldsmith led between 22% to 30% with the over 65s and still lost

Sunday, May 8th, 2016

A pointer to the EU Referendum?

One of the patterns that have emerged with a lot of the EU referendum polling is that the over 65s favour Brexit by a substantial margin, which gives the Leaver side a lot of hope that they may win the referendum on June 23rd, because older voters have had a higher turnout than younger voters in past elections.

But look at the charts above from the final London Mayoral polling by YouGov and Opinium, Zac had leads ranging between 22% and 30% with the overs 65s, yet still lost by nearly 14%, which these polls accurately predicted. Whilst London might not be representative of the United Kingdom, it is interesting nonetheless that even in a low turnout election, the power of the oldies was negated.

The betting on Betfair seems to be maintaining its consistent pattern, with Remain an implied circa 70% chance of winning.





Super Thursday – what have we learned so far?

Friday, May 6th, 2016

We now have a good bank of results, which while incomplete allow us to draw some conclusions.  So what conclusions can we draw?

  1. The pollsters have either made the right corrections from last year or have been lucky

The local election results so far in are very much in line with the broad thrust of the opinion polls we’ve been seeing in the last month or so.  The Welsh assembly results are very close to what we could expect from the polls.  The Scottish results are consistent with the last YouGov poll taken (meaning that the 8/1 that I tipped yesterday morning on No Overall Majority in Scotland came home).

Political observers have gone from one extreme to the other with their treatment of opinion polls.  From poring over them obsessively last year, they are now ignoring them completely.  The correct treatment lies somewhere in between.  They should be taken note of, but not treated as tablets of stone.

In turn, that means we should take their findings about the EU referendum seriously.  There is no reason to assume that this isn’t very close at present.

  1. There is no evidence that Jeremy Corbyn is particularly vote-repellent

Before the local elections, as in the Oldham by-election, there were noises that Labour were having major problems on the doorstep.  But that hasn’t materialised in a voter strike.  Indeed, in the past Labour have underperformed their polling at local elections and the Welsh assembly elections while this time they performed to polling expectations.  Whether that’s because the pollsters are doing better or Jeremy Corbyn has unlocked more assiduous Labour support, we can’t tell.

But the idea that Jeremy Corbyn makes traditional Labour voters reach for the smelling salts or to change allegiance has no substance as yet.  Those Labour supporters, so noisy in the media and on Twitter, who are appalled with him and have sat on their hands or changed sides seem to be a small minority.

This is the single most important point to be deduced so far.

  1. Jeremy Corbyn is safe for now

These results are far too good to get a broad consensus of Labour MPs to march with pitchforks on their leader to depose him.  It was never going to happen anyway because Jeremy Corbyn remains far too strongly supported by the membership but it isn’t even going to be fantasised about now.

This has certain real world consequences.  Sadiq Khan, if elected as Mayor of London, has no reason to backtrack on his promise to give up his seat.  Andy Burnham has no reason not to pursue his idea of becoming Mayor of Manchester if he thinks Labour are a busted flush at national level under their current management.  Neither therefore look like good bets for next Labour leader (though either might yet get the job at some point later on).

This in turn depletes the pool of experienced ministers from whom Labour could select a leader to take over as and when Jeremy Corbyn is ultimately replaced.  The options for unity candidates are diminishing and Tom Watson is becoming ever more unchallengeable in that role.

  1. Life just got a bit more complicated for the SNP

While the SNP will form the next administration, they are going to have to play nice with someone.  They could align with the Greens, who support independence but who come with other problems.  Or they could align with the Lib Dems, who have long experience of being a junior partner in coalitions and who will no doubt be very enthusiastic to repeat the experience.

When they were last the largest party without a majority they formed a minority government.  They could do this again, but at the risk of being subject to the kind of guerrilla ambush by opposition parties that the SNP enjoy carrying out so much in Westminster.

And threatening Westminster with the idea of holding a referendum every time Nicola Sturgeon has a bad hair day is now out of the window.  While a referendum in this Scottish Parliament is still possible, it has become rather less likely.

  1. There is no end for Scottish Labour’s misery

Every time that you think Scottish Labour has hit rock bottom, they find another trapdoor.  They are now some way adrift in third place, behind the hated Conservatives.  Right now they lack purpose, with the Conservatives having grabbed the mantle of unionism and the SNP having become the Scottish progressive party.  This polarisation seems set to continue, given that both of those parties have highly capable leaders.

Is this rock bottom?  I doubt it.  Scottish Labour are in severe danger of becoming irrelevant, if they have not already got there.  If that happens, the only way is down.

Alastair Meeks


The numbers that the pollsters hope will help restore faith in their industry

Thursday, May 5th, 2016

London Mayoral race   Google Sheets
For the past year it has not been much fun being a political pollster. Whenever any new survey has been published it has been greeted with “Well we all know what happened at the General Election”.

So today’s elections where there’s been polling, the Welsh Assembly, the Scottish Parliament and most of all London could play a big part in the renewing faith in what they do.

The reason this might sound London-centric is because the General Election Scottish polling was, in the main, pretty good and only one firm, YouGov, does regular Welsh polls. In the two national regions, as well, there is the complication of the list voting system on top of those for individual members of the national parliaments.

Of course there are a large range of contenders fighting for the capital’s mayoralty but voters do have what is known as a “supplementary vote” so this is a sort of AV system.

    If there is a Zac victory or the Khan lead is in low single figures then the doubts over polling will remain – something that’s particularly important in the count down to the June 23rd referendum.

In the list above I have only included those where fieldwork took place during the final week. All the surveys are online.

Mike Smithson


Sadiq Khan 20% ahead of Goldsmith according to Survation phone poll

Friday, April 29th, 2016


So on the face of it a great poll for Sadiq Khan, Labour, Corbyn and those PBers who got on the Labour man at 33/1 in March 2103 when Henry G Manson tipped him.

But the survey took place before the blow up involving the last LAB mayor of London, Ken Livingston and phone polls have not had a good record on London Mayoral contests.

Of the 2012 final polls the least accurate was the only phone survey.

There’s also an issue that Labour voters might be less likely to turnout given it looks like a foregone conclusion.

Just in case I’ve been laying off large parts of my 2013 Sadiq bet at odds of up to 17/1 on Betfair so I’m certain of a substantial pay day next Friday whatever happens.

Mike Smithson