The extraordinary public battle between two of the men behind the creation and success of YouGov

May 20th, 2016


If it was just online polls the referendum narrative and the betting would look very different

May 20th, 2016

EU Ref polling   Google Sheets

This week being the third of the month has seen a glut of phone polls. We’ve had ICM, ORB, Ipsos MORI and last night ComRes all showing substantial REMAIN leads.

Inevitably this has had an impact on the huge EUref betting markets where on Betfair alone getting on for £0.5m is being wagered each day. As I write REMAIN’s chances are rated by the market at 79% just about the highest it has ever been.

But what if there were no phone polls and we just had to rely on the cheaper internet ones. The table above shows how the last few weeks would have looked. The race would have appeared to be neck and neck and the betting would have had it much much closer.

I get asked many times a day in Tweets like this which mode I think is best and I’ve now got a standard response.

The big thing, of course, is confirmation bias. People, as we see so often on PB and I am as guilty as anyone, trust most the evidence that supports their position.

The very successful performance of the online polls in the May 5th London Mayoral race reinforces the point that this form of polling can be very accurate indeed. The only phone poll had a Sadiq margin much bigger than the actual result.

At the moment I’m not betting on the referendum. I don’t regard the REMAIN price as value.

Mike Smithson


Against the trend of other recent phone polls ComRes for the Mail has the REMAIN lead getting smaller

May 19th, 2016


Age versus social class: ComRes assess the first test for their turnout model and its implications for EURef

May 19th, 2016

EU flag

A special post by ComRes’s Adam Ludlow

The polls for the London mayoral election performed strongly across the board, perhaps bringing pollsters some respite following last year’s General Election. But as pollsters, we must be careful never to rest on our laurels and make sure to review our methods and seek to improve when things go well, as well as when bigger problems occur.

Indeed, at ComRes, the London result was of particular interest to us as it was the first outing in live action for our probabilistic electorate modelling.

For those unfamiliar with the ComRes Likely Electorate Model, this was the innovation we brought in following the General Election and uses a demographics-based approach to simulate what those who turn out to vote are likely to look like at an election, based on historic turnout patterns between different social groups. Respondents are now assigned a probability of voting based on their demographic characteristics, rather than just their own self-identified likelihood to vote.

As a number of pollwatchers have commented, this has tended to lead to our polls showing slightly better results for the Conservatives than other polling firms have. This is primarily because the model boosts the importance of older and more affluent voters, who data show are by far the most likely to vote and (at the current time) lean heavily towards the Conservatives.

On the face of the evidence from London though, these polling reforms appear to have worked. While all polling companies got the picture at the two party run-off right, there were a range of outcomes predicted for the first round where all parties were involved. Using the Likely Electorate Model, the final ComRes poll got all top five parties to within a point of their actual vote share on both rounds of voting – the only poll to do this.


It was also the only poll not to overstate the lead between Labour and the Conservatives on the first past the post share of the vote (nine points) – the exact problem the polls faced at the General Election.

Of course, there could be range of explanations for getting the winning margin correct, but reviewing the findings suggests the electorate modelling made a key difference. Rerunning our final London mayoral poll using the pre-GE2015 self-reported likelihood to vote filter, it shows that the poll would have seen an overstatement of the Labour lead on both the first and second round. Rather than correctly having a nine point lead on the first round, it would have had Sadiq overstated on 48% and Zac understated on 33%. On the second round, it would have had 60% to 40% – a result outside the margin of error. Positively though, the methodological changes ComRes made since the General Election successfully corrected for the error and accurately reflected the result. There is obviously good news across the industry too that the methods used by other pollsters worked in London as well.


So what does this mean for the upcoming referendum? That we can sit back and wait for the results to come in just in line with our polling? Unfortunately, polling is rarely so easy.

Firstly, while the Likely Electorate Model helped us produce the right result in London, this was using an online survey methodology (as it is more appropriate for contacting London’s young and transient population). We are yet to see how the model interplays with telephone polling, which we are using for polling the EU referendum (for reasons explained here).

Secondly, our polling for the London election used a different output from the Likely Electorate Model than what is used for our national polling. While the computations are the same, they are applied to different data: the London model makes projections based on historic data from mayoral elections, the national model uses data based on General Elections.

As we explored in depth in the run-up to Sadiq Khan’s election victory, the relationship between demographics and likelihood to vote is far weaker at mayoral elections in London than it is nationally at General Elections. In other words, at General Elections, there is a big difference between how likely young people and older people are to vote. At London elections, the difference is far smaller.

Due to this weaker relationship between age and turnout in London, the effect of the modelling ended up being weaker (where it explained 38% of the variance in someone’s likelihood to vote) than in our national polling (where it explains nearly 70% of the variance). Another way of putting this is that whereas we can be around 70% sure of someone’s likelihood to vote at a General Election based on their demographic characteristics, we can only be 38% sure of it at London mayoral elections (there are instead other “out of model” factors, such as non-demographic ones which determine whether they vote or not). The weight given to demographic factors was therefore less in London than in national polling.

Another effect of there being almost no relationship between age and likelihood to vote in London, was that the model was particularly reliant on social grade as a predictor of turnout. The strong performance therefore is somewhat reassuring for us in the lead up to the EU referendum vote.

This is because, despite there being much talk about the effect of turnout helping the anti-EU side thanks to older voters leaning towards Leave, when we applied our Likely Electorate Model to our last referendum poll, it actually increased the Remain lead. The significance of AB social grades (who lean heavily towards Remain) actually slightly outweighed that of older voters. It is early days, but the success of the model in London suggests that the inclusion of affluence in the model – and the importance attributed to it – is justified.

Of course, every election presents its own challenges, and EU referendum will do so more than most. But for the time being, the electorate modelling passed its first hurdle, and gives us some indications about how we should take the second.


REMAIN now move to 78% chance

May 19th, 2016


Why REMAIN, even at the very tight odds currently available, is a value bet

May 19th, 2016


A guide by Alastair Meeks on betting on the overall EURef outcome

Given the amount of heat that has been generated by the EU referendum, there has been surprisingly little discussion about the actual chances of the main event.  It’s time to put that right.

At the time of writing, Remain is 1.31 on Betfair (a 76% chance)   I understand that the conventional bookies have seen Remain overwhelmingly favoured at all stages despite the short odds offered.  Should you be doing the same?

The polling

The polling is utterly confusing.  If you follow the online polls, the two sides are neck and neck with Leave having a small and perhaps growing lead.  If you follow the telephone polls, Remain has a substantial lead, perhaps growing.  They can’t all be right (though they could all be wrong).

Do we have any clues as to which to place most weight in?  The polls did pretty well in the election round in early May, suggesting that the pollsters have made great progress in addressing the problems they ran into this time last year.  But choosing which party to vote for is very different from choosing sides for an idea, especially one that cuts across party lines.

Online polls are composed from panels, so we can expect their panellists to be disproportionately interested in politics.  That implies that Leave would probably be too high, given that opposition to the EU energises a particular type of politically-engaged voter.  Set against that, telephone polls have sampling problems of their own – the response rate is very poor nowadays and so those who actually respond are unlikely to be particularly typical either (though it is harder to predict in what ways they might be atypical).

Might telephone polls be finding too few rightwing voters?  What are we to read into the implausibly high turnout estimates that the polls are producing?  Might “respectable” Leavers be more reluctant to reveal their views to telephone pollsters than online?

Anyone sensible is proceeding with great caution here because the pollsters themselves are pretty open that they aren’t sure what is going on.  Personally, I am working on the theory that the disproportionately-engaged online panels are overstating the Leave vote making them more inaccurate, but I am doing so warily.

The BES commissioned one of their studies and this suggested that Remain have a very small lead.  While it is a little old, I take this very seriously indeed.  So should you.

The dynamics of the campaign

Remain are rigidly sticking with their theme of the risk of voting Leave, issuing daily blood-curdling warnings about the possible effect on the economy and the risks to security that it would entail.  They are evidently going to carry on doing so for the next month. This strategy will have one of three outcomes:

  1. The public will be too terrified to go to the polling booth for fear of being hit by an asteroid.
  2. The public will regard the claims as wholly laughable.
  3. The public will decide that Leave is risky (even if it isn’t as risky as David Cameron and the economic glitterati are suggesting).

My expectation is outcome three.

Leave is less predictable.  They have been trying to focus on the ability to control immigration if we leave, but wandering off onto historical disquisitions of doubtful historicity and still more doubtful political utility.  Because their campaign is less disciplined, they risk squandering the advantage they have on immigration by being diverted onto less fertile territory.

The public aren’t suddenly going to have a Damascene conversion either way.  The polls have been pretty static all year (once you subdivide them into phone polls and online polls), with movements being quite marginal.  To the extent that they break one way or the other, I expect them to break towards safety.  Unless Leave can talk up the risks of immigration as effectively as Remain have talked up the economic risks, I expect any movement to be towards Remain.

The mood music of the camps

It is fair to say that neither camp has exuded an image of confidence.  Remain’s warnings have been shrill, while Leave have issued wild attacks on the integrity of those voicing inconvenient views.  Both Nigel Farage and Boris Johnson have been lured into discussing a rerun of the referendum if Remain secures a narrow victory, which suggests that they feel that the campaign is ebbing away from them.

The lack of confidence in both camps and the impression that Leave thinks it is behind can be explained by the two sides’ different aims.  Leave wants to get to 50%+1.  Remain wants to settle the debate for a generation.  It is entirely possible that both will be thwarted in their aim and that both believe that on their own terms they will fail.

If you agree, the 5/6 offered by Ladbrokes that Leave will secure over 44.5% of the vote looks fair value, and I’ve backed this bet.  If you’re betting on the main event, anything like 1/4 offered on Remain looks an excellent price and you are unlikely to get much better value in the absence of an unexpected development.  Take it.

Alastair Meeks


The polling that suggests Corbyn is unassailable discussed in the latest PB/Polling Matters podcast

May 18th, 2016


What is the future for Labour?

On this week’s show, Keiran and Rob analyse YouGov’s recent polling of Labour members and what’s behind Jeremy Corbyn’s seemingly unassailable position as Labour leader. They also address the apparent limitations of such polling as described by pollster James Morris in today’s Times and what this poll tells us (and doesn’t tell us) about the future of the Labour Party. Meanwhile, the Polling Matters team also look at the continued difference between what online and telephone polls say about the upcoming EU referendum and debate how Leave can win.

(The podcast remains audio only this week whilst TipTV finalise their studio move. We expect to return to the TV studio next week).

You can follow Keiran on twitter at @keiranpedley and Rob and @robvance.


The EU Dog that hasn’t barked. Yet

May 18th, 2016


A guest slot by Cyclefree

It was Socrates who said that the “unexamined life is not worth living”.  By the same token, one might also say that the unexamined EU is not worth being a member of.

And – despite all the claims and counter-claims, exaggerations, half-truths, figures plucked out of obscure studies or the air and celebrity or heavy weight endorsements – it often feels that all the referendum campaign has amounted to is little more than “a tale told by an idiot, full of sound and fury signifying nothing.”

Well, the referendum will signify something.  One side will win.  There will be an answer to the referendum question.  To the question – yes.  But if Remain win, will there be an answer to the more important – and so far largely ignored – question of what Britain’s role in the EU will be, should be, what it can be and, even more critically, what the EU is and is likely to become?  And if Leave win, what will Britain’s role within Europe and with a different EU be?

Judging by the campaigns so far: no.

There are two principal reasons for this.  The first is that both sides have concentrated on why people should not do what the other side wants them to do.  So Remain is all about the reasons for not Leaving – the costs, the uncertainty, the risks, the isolation, the effects on government here, the apparent complete inability of any British government to legislate in the future on matters currently subject to EU law.  Leave is all about how ineffably ghastly the EU has been, how condescendingly supercilious Remainers are, the general ghastliness of Cameron & co., the appalling interference by foreigners in offering a view coupled by a vague statement that Britain will be free to do what it wants.  There are very few grown-ups visible in this debate.

The second is that the entire debate has been about the economics of the project, what it will cost us if we go or stay.  But this is to miss fundamentally what the EU project has been about ever since its start.  This has been Britain’s original mistake: its failure to understand what has always been clear to and desired by the other states.  And those who have understood it have compounded this original mistake by not being honest with the voters about this.  It’s always the cover up which gets you.  And this dishonesty or fear of the consequences of being honest is poisoning the current debate, making it largely meaningless.

So rather than rage at outsiders telling us their view, rather than declaiming that other states should not opine on domestic matters, rather than telling other EU states and the EU itself to keep quiet until June 24, here’s my modest proposal.

Let’s invite the EU Commissioners to tell us what they think the EU should be, how it should develop in their areas of competence.  Let’s hear from Francesca Mogherini on her view of how the EU should deal with the instability on its borders, how it should handle the migration issue.  Let’s hear more about the EU Army and the proposed budget.

Let’s hear what a common Justice policy means in an organization with states with no habeas corpus and the practice of trials of people in their absence and states with trials by jury.  Let’s hear from Hollande about what the French view of Europe should be, let’s hear from the 5 Presidents (whoever they are), let’s hear from the Poles and the Latvians and Germans and Italians and Spanish and Irish.  Let them come over and tell us frankly what sort of EU they want and why.  Let us ask them: what is this organization we’ve been a member of for the last 40 or more years?  How do you want it developing and why?  Do you think it should be reformed?  And what do you mean by “reform”?

A risk: yes – a big one for Remain.  We might not like what we hear and vote to go.  But if Remain wins after all this then the issue really will have been put to bed for a generation.  A risk and an opportunity for Leave: risk because the EU may not turn out to the bogey man too many in that camp think; opportunity – because it will force us to think intelligently about life outside the EU.  Migration is not going to go away even if we leave the EU.

Britain’s approach to and relationship with a large and powerful Continental neighbour still needs thinking about.   And if we go after an honest conversation with our neighbours we may be more likely to get a worthwhile relationship with them after we’ve gone.  The current petulant “You’re all like Hitler” approach is not grown up and is not likely to endear us to countries who have had rather more experience of us of what it’s like to live under genocidal maniacs.

Let’s hear from the EU.  Let’s stop talking to ourselves.  Let the European dog bark.