Post on 26-Dec-2015
© 2010 GfK NOP
Exit Polls – the Whys and Wherefores (and more importantly the Hows)
Nick Moon, GfK NOP
2
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
My own experience
Extensive research to develop an exit polling technique for the BBC
Ten Parliamentary by-elections
1990 European elections
Four GB general elections – 1992 – 2005
One Greek general election
On-site observation of US Presidential Election 2000
The most recent Ukrainian Presidential Election
3
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
Opinion Polls and Exit Polls
Methods for conducting opinion polls can differ enormously from one country to another
•Face to face/telephone/online
•Random/quota
Methods for conducting exit polls are much the same the world over
•Ukraine is a vast country (with even worse weather) but there exit poll was almost exactly the same as ours
4
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
Cut to the Chase, Nick
Don’t just make excuses about Ukraine being a big country – tell us how you got on
5
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
While the Triumph is Still Fresh
Result Poll ErrorYanukovych 35.3 34.5 -0.8Tymoshenko 25.1 25.6 +0.5Tigipko 13.1 13.8 +0.7Yatsenyuk 7.0 7.0 0Yuschenko 5.5 5.6 +0.1
6
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
What is an Exit Poll?
A very expensive way of getting an idea of an election result a few hours before you know for sure
A major contribution to understanding of the electoral process
A way to tell if the election was conducted fairly
• (or is it?)
7
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
Types of Exit Poll
Prediction Poll
•Who is going to win and by how much?
Analysis Poll
•Why did voters vote the way they did?
8
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
An exit poll may be more than a poll
Interviewing voters
•This is all we do in the UK
Collection of precinct-level results as soon as they are announced
•A representative sample can predict the whole, quickly
– In some European countries this is all that is done
– In the US this is done alongside a voter survey
Analysis of partially aggregated actual results
•Used to fine-tune projections made using the above
9
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
Exit Poll Problems - issues
Interviewing has to be clustered by polling station
•Cost is a function of number of stations, not number of interviews
Voting behaviour tends to be very clustered at the polling station level
•Several cases in the past of >90% support for one party
– And several of 100% in Ukraine
•Far more interviews are conducted than are actually needed, just because the interviewer might as well do as many interviews as are easily achievable
Actual results at polling station level not always available, which makes sampling less reliable
•US and Greece yes, UK and Ukraine no
10
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
Exit Poll Problems – issues (UK)
We are really predicting the result of over 600 separate elections, and can’t cover each one
The only local level information available on actual voting is at ward level, and only for local elections
•Local level turnout is far lower and behaviour often very different
•This makes stratification of the sample by voting is virtually impossible
11
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
Exit Poll Problems – UK solutions
Use as many polling stations as possible
Use the same polling stations as the previous election
•Can look at change rather than simple totals
•Eliminates effect of bias IF bias is constant (Big IF)
12
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
The 2005 GB Exit Poll
Conducted jointly by NOP and MORI
Conducted on behalf of both BBC and ITN
120 polling stations
•Most used previously by NOP or MORI
Approx 16,500 interviews in all
Just measuring claimed current vote
13
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
The Practicalities (1)
Polling runs from 7am to 10pm
Split into two equal shifts
Each has an interviewer and a “clicker”
“Clicker” uses preordained sampling interval to select respondent
Interviewer estimates voting behaviour, then asks voter to fill in an exit poll ballot
No replacement for refusals
14
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
The Practicalities (2)
Ballot designed to replicate actual one as closely as possible
•Has candidate names and parties
•In alpha order as on real ballot
Ballots collected every hour
Individual results phoned back to team of telephone interviewers
Results weighted to take into account any discrepancy between intended and actual number of interviews in each hour
15
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
The Practicalities (3)
Aggregate data at level of time by polling station produced at regular intervals
•Along with individual level data file
Collected by the BBC/ITN analysts
Run through various algorithms
Initial projection of result at 10pm
•Final projection at 11pm
16
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
And How Good Was It?
17
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
In case you’ve forgotten
NOP Actual result
36 36
33 33
23 23
18
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
The BBC/ITN exit poll
Forecast majority 66 seats
Actual majority 66 seats
19
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
The BBC/ITN exit poll
Jeremy Paxman (paraphrased)
“I don’t know why we spend all this money on the exit poll, we should just do without it”
Tony King (paraphrased)
“These results suggest our exit poll is overestimating Labour”
20
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
The BBC/ITN exit poll
Predicted majority based on exit poll
66 seats
Predicted majority based on early actual results
80 seats
Final majority
66 seats
21
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
The BBC/ITN exit poll
Sorry Jeremy
The BBC should carry on paying for the exit poll, and stop paying for everything else
22
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
The 2010 General Election Exit Poll
Conducted jointly by GfK NOP and Ipsos MORI
BBC and ITN now joined by Sky
As close as possible to the 2005 design
•Changed polling places where the constituent voters or the place itself had changed significantly
•Replaced these with an eye to filling gaps in current design and also looking to the next election
23
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
Are exit polls measures of election fixing?
Ukraine 2004 Round 2
•But many other indicators of fraud
Venezuela
•Questions raised about exit poll
US 2000
•There almost certainly was some fraud but the exit poll is not a smoking gun
24
GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010
© 2010 GfK NOP
To conclude
Exit polls are a good way to give a good advance idea of who has won, and why, and properly conducted exit polls can act as some sort of check on electoral fraud, but as President Carter said, they should not be used as sole arbiters of the fairness of an election