Archive for the 'Pollsters/polling' Category


Do 1 in 5 British Muslims really ‘sympathise with Jihadis’?

Monday, November 23rd, 2015

Keiran Pedley looks at this morning’s front page of The Sun and argues that we should always check the small print when reading opinion polls.

As someone that has spent most of his professional life reading opinion polls I have always enjoyed this scene from Yes Minister where Sir Humphrey explains to Bernard how to rig an opinion poll. It’s a funny scene but does demonstrate a pretty important point that all pollsters know – opinion poll results are often as much about how the question is asked as what the question actually is.

This feels particularly relevant today as the front page of The Sun screams ‘1 in 5 Brit Muslims sympathy with Jihadis’. On face value, this is a very worrying finding for obvious reasons.


However, when you look at the data behind the headline things start to unravel a bit. It should be said first and foremost that polling a representative sample of a religious group is very difficult. Tom Mludzinski of ComRes and Maria Sobolewska of the University of Manchester explain why in more detail on last week’s PB / Polling Matters podcast here and Matt Singh is good on this today here too.

However, my real complaint about this poll is the complete disconnect between the wording of the question and the way the result has been displayed in this morning’s paper. The actual question wording can be found below. Keep in mind that this is the question that has led to the headline on the front page of The Sun claiming that one in five British Muslims have sympathy with Jihadis.

So what is wrong with this question? Firstly, it sets a very low bar for support. The one in five figure that The Sun quotes includes anyone that expresses at least ‘some’ sympathy with young Muslims that join fighters in Syria. However, I think that the words ‘sympathy’ and ‘fighters in Syria’ are the most important here. ‘Sympathy’ does not mean support. It can do but the link is not certain. It could just mean that they understand why a young person might go to Syria even if they disagree with the decision. Even more importantly, what should we suppose that ‘fighters in Syria’ actually means? Again, it ‘could’ mean ISIS or perhaps it doesn’t. Notice how the words ‘ISIS’ and ‘Jihadis’ are not mentioned in the poll question but are used in the headline and in this opening line of the supporting article.

This might sound very picky and pedantic but it is important. Let’s consider how an alternative question wording might have been answered.

Do you support or oppose young British Muslims leaving the UK to fight for ISIS in Syria?

1) Strongly support

2) Somewhat support

3) Neither support or oppose

4) Somewhat oppose

5) Strongly oppose

6) Don’t know

Not a perfect question by any means but you can see how it might have produced very different results to the one above. It makes the ‘fighting for ISIS’ point much more explicit.

Perhaps the question was not designed to elicit the headline that it did. This is a common problem for pollsters. We often have no control over how the results of our polls are presented in the public domain. However, in instances such as today – on such a sensitive topic and in the aftermath of the Paris terror attacks – the media has a real responsibility to be careful with how it presents poll findings. I think The Sun has got it wrong this morning.

The average person on the street is not going to go to the trouble of scrutinising sampling techniques or question wording. What they will see is a headline on the front page of one of the most popular newspapers in the country that nudges to an ‘enemy within’ – with a giant picture of a knife-wielding ‘Jihadi John’ just in case you didn’t get the message. It leaves a sour taste to behonest.

In this piece I do not seek to play down the scale of the threat posed to our national security from Islamist terrorism. It is real and needs to be dealt with at home and abroad. However, the media has a real responsibility not to make things worse and today’s Sun splash was unhelpful in that regard and unjustified based on the data it was based on. After all, using the same data, it could just as easily have said ‘Just 1 in 20 British Muslims sympathise with those travelling to Syria’. I will leave others to judge why it did not.

Keiran Pedley tweets on polling and politics at @keiranpedley and presents the podcast ‘Polling Matters’


Corbyn slumps to new YouGov leader ratings low while latest Survation #EUREF polling has REMAIN back in the lead

Wednesday, November 18th, 2015


Even EdM had net positives at this stage

It used to be that YouGov published its well/badly party leader ratings on a weekly basis. Now in this post-GE2015 polling disaster era we only get them once a month. The latest, for November, is just out for the Times and the figures continue to decline for Mr. Corbyn. Of those polled 52% said he was doing badly with 30% saying well.

To put that into context on November 19th 2010 Ed Miliband was still in positive territory with the online firm – a net plus 1%.

There are are several different polling approaches to leader ratings. Ipsos asks about “satisfaction/dissatisfaction“; ComRes “favourable/unfavourable“, Opinium “approval/disapproval” and YouGov “well/badly“.

    In every single poll of whatever form where leader rating questions have been put since Corbyn got the job he has been in negative territory.

Meanwhile there’s a new Survation EUREF poll which has REMAIN in lead – almost a reversal of the figures that the firm had last week.

Mike Smithson


Pre “adjustment” ICM has LAB and CON level-pegging – after it the Tories are 6% ahead

Tuesday, November 17th, 2015

After “adjustment” Con 39 (+1) Lab 33 (-1) UKIP 12 (+1) LD 7 (nc)

In a note of explanation the pollster observes:

It should be noted, however, that the raw data shows substantive change which our newly strengthened adjustment process disguises. Based on (pre-adjusted) turnout weighted data, the parties are neck and neck, which the manual adjustment converts into a 6-point Conservative lead.

This is due to unusual combination of three factors. Firstly, the sample recalls voting in a Labour government for the fourth time out of six occasions since the last election (which is frustrating, but not the unusual part) but secondly, the level of partial refusal (respondents who told us what they did in 2015 but don’t know/refuse to tell us what they would do next time) this month has cut into the Conservative share significantly. In previous polls subsequent to the General Election, partial refusers have been fairly evenly balanced between the two parties.

Thirdly, the Conservatives are rounded up from 38.5% to 39%, and Labour rounded down from 34.4% to 34%.

In short, the adjustment has offset potential sampling imbalance and has worked to correct data outcomes in exactly the way we intended them to in light of the General Election polling miss.”

We’ll get the first details of inquiry into the GE2015 polling failure on January 19th.

Mike Smithson


Corbyn’s LAB closes the gap once again in the October ComRes/Mail phone poll

Friday, October 30th, 2015

This may ease some jitters within the red team

After the GE2015 polling disaster ComRes was the first firm to announce radical changes in its weightings to deal with the apparent problem of the Conservatives being understated. The result is that its polls have broadly shown bigger CON leads than those from other firms. So today’s numbers will give the red team even more comfort.

Con 38% (-1)
Lab 33% (+3)
LD 8% (-1
UKIP 10% (-2)
Green 3% (-1)

The tax credits’ findings from the firm are not good for Mr. Osborne and show, I’d suggest, how he failed to make a substantive case when announcing them. The Tory position has not been helped by analysts suggesting that the impact on many families will not be alleviated that much by his new National Living Wage.

Mike Smithson


Farage and UKIP the big gainers in the October Ipsos phone poll

Monday, October 26th, 2015


Ipsos MORI referendum REMAIN lead drops by an astonishing 24 points

Thursday, October 22nd, 2015

Looking at the the June and October Ipsos polls the former looks increasingly like an outlier but even so the current gap is substantially larger than anything we have seen in recent online polls.

There does seem to be something very odd when the two main communication polling methodologies are producing too very different pictures of opinion in the country.

It is very hard to come to a conclusion. The previous phone poll to today’s one from ipsos was buy ComRes at the end of September and that had a remain main lead of 19%.

I can’t explain it. Maybe the people who want to leave the EU are more reluctant to admit it to a live interviewer on the phone. Do we have the shy noes?

Mike Smithson


Doing “Best PM” comparisons between Corbyn & Dave is like asking US voters to choose between Obama and Trump

Tuesday, October 20th, 2015

We all know that David Cameron is not planning to remain as prime minister after the next general election. So the choice will between Corbyn, unless he’s replaced in the meantime, and AN Other.

So why is it that pollsters, and presumably their media clients who agree to the form of questioning, continue with best p.m. ratings that include the current prime minister? The findings really don’t have any relevance to the next big election battle in the UK.

Thankfully, in its latest poll, Opinium has chosen to put some other options in as well although they still have included Mr Cameron.

This is very much on the American model that we are seeing at the moment as the two main parties go through the process of choosing their nominees for next year’s White House Race. Many pollsters are putting forward to those sampled a huge range of possible options to try and test the water as to which of the leading contenders would be a better choice.

The findings can have a significant effect on the nomination process itself. Those that are doing well are, understandably, going to do their best to try and highlight this the ahead of the primaries.

The Opinium poll tested three choices as CON leader in its latest survey as well as Cameron. The findings are in the chart above.

    The most surprising one, I’d suggest, is how poorly George Osborne is doing against Jeremy Corbyn. Given the very negative reaction that there’s been to the new Labour leader Osborne, surely, should have been polling substantially better than Mr Corbyn as best PM.

Clearly, at the moment, Osborne is being hit by much of the response to his tax credit move which is proving to be unpopular within his party as well as in the country.

Things have to change for George. He cannot go into a conservative leadership contest with him still just level pegging with the Labour leader.

Mike Smithson


ComRes: Osborne 6% behind Boris as “best PM” and just a third say that cutting tax credits is necessary

Saturday, October 17th, 2015


39% say Boris would make a better PM than Osborne who is at 33%.

But CON voters prefer Osborne to Boris by 48% to 34%

CON lead now 13% in latest ComRes online poll for IoS/S Mirror
CON 42%
LAB 29-1
LD 7=   
UKIP 13=
No Corbyn bonus there

ComRes Cameron & Osborne have and 19 point lead over Corbyn & McDonnell on who is trusted more on the economy

Mike Smithson