Timing Isn’t Everything, But Money Talks

28
Timing Isn’t Everything, But Money Talks How to encourage a face-to- face household panel to go online? University of Essex, July 2013 Gerry Nicolaas Carl Cullinane

description

University of Essex, July 2013. Timing Isn’t Everything, But Money Talks. How to encourage a face-to-face household panel to go online?. Gerry Nicolaas Carl Cullinane. Contents. Background Design of Experiment Results Summary of Results and Conclusions. Background. 1. Background. - PowerPoint PPT Presentation

Transcript of Timing Isn’t Everything, But Money Talks

Page 1: Timing Isn’t Everything, But Money Talks

Timing Isn’t Everything, But Money TalksHow to encourage a face-to-face household panel to go online?

University of Essex, July 2013

Gerry NicolaasCarl Cullinane

Page 2: Timing Isn’t Everything, But Money Talks

2

ContentsBackgroundDesign of ExperimentResults Summary of Results and Conclusions

Page 3: Timing Isn’t Everything, But Money Talks

Background

1.

Page 4: Timing Isn’t Everything, But Money Talks

4

BackgroundSwitch to mixed mode data collection at wave 7 of Understanding SocietyPotential for cost savingsPotential for reducing attrition

Page 5: Timing Isn’t Everything, But Money Talks

5

ContextLongitudinal household panelFace-to-face interviewing of all household members at waves 1 to 6Greatest potential for reducing data collection costs when an interviewer does not have to visit the householdPrevious experiment mixing telephone & face-to-face:

Costs can be reduced BUT response rates suffer

Page 6: Timing Isn’t Everything, But Money Talks

6

Innovation Panel – wave 5Vehicle for methodological development & testing About 2,500 individuals in 1,500 householdsMain objective of IP5 =

Determine whether it is possible to reduce costs by mixing web questionnaires and face-to-face interviews without sacrificing data quality

Sequential mixed mode design starting with web

Page 7: Timing Isn’t Everything, But Money Talks

77

IP5 Design

Experimental group

F2F phase, web open

Web only phase

Control group F2F only phaseNo web

Page 8: Timing Isn’t Everything, But Money Talks

8

Research question Is it possible to boost the proportion of whole households completing web questionnaires by

Timing the arrival of the invitation to go online?

Offering a web bonus?

Page 9: Timing Isn’t Everything, But Money Talks

Design of experiment within mixed-mode sample

2.

Page 10: Timing Isn’t Everything, But Money Talks

10

Design of experiment within mixed-mode sample(1) Timing of invitation to go online Random allocation of households to:

Letter (+ email) to arrive on Friday Letter (+ email) to arrive on Monday

Reminder letters (+ emails) sent 2 and 4 days later

(2) Web bonus Random allocation of households to:

No web bonus £5 per household member conditional on all completing online questionnaire

Page 11: Timing Isn’t Everything, But Money Talks

11

Design of experiment within mixed-mode sample

270 household

s

265 household

s266

households

276 household

s

No web bonus

Cond. £5 per hhld

member

Invite arrival on

Friday

Invite arrival on Monday

Page 12: Timing Isn’t Everything, But Money Talks

Results

3.

Page 13: Timing Isn’t Everything, But Money Talks

13

Experimental effects on Web Response

Base: IP5 WEB Sample (n=1077)

21.4 23.7

0

20

40

60

80

100

Monday Friday

Full Household Web Response (%)

Page 14: Timing Isn’t Everything, But Money Talks

14

Experimental effects on Web Response

Base: IP5 WEB Sample (n=1077)

21.4 23.7 20.4 24.7

0

20

40

60

80

100

Monday Friday No Bonus Bonus

Full Household Web Response (%)

Page 15: Timing Isn’t Everything, But Money Talks

15

Effect of Bonus by Sample Type

Base: IP5 WEB Sample (n=1077)

188.6

3430.5

9.4

33

0

20

40

60

80

100

W4 Responding W4 Non responding Refreshment

No bonus Bonus

Page 16: Timing Isn’t Everything, But Money Talks

16

Effect of Bonus by Upfront Incentives

Base: IP5 WEB Sample (n=1077)

13.5 17.9

37.4

18.8 22.6

40.2

0

20

40

60

80

100

£5 £10 £20+

No bonus Bonus

Page 17: Timing Isn’t Everything, But Money Talks

1717

Unconditional Incentives

Original Sample

Refreshment Sample

£5

£10

£10

£30£20

Page 18: Timing Isn’t Everything, But Money Talks

18

Effect of Bonus by Children in Household

Base: IP5 WEB Sample (n=1077)

22.8 20.6 16.43.7

22.833.3 35.4

18.2

0

20

40

60

80

100

None One Two Three +

No bonus Bonus

Page 19: Timing Isn’t Everything, But Money Talks

19

Effect of Bonus by Advance Mailing

Base: IP5 WEB Sample (n=1077)

5.1

36.3

6.5

43.8

0

20

40

60

80

100

Postal Valid email

No bonus Bonus

Page 20: Timing Isn’t Everything, But Money Talks

20

Effect of timing by Advance Mailing

Base: IP5 WEB Sample (n=1077)

4.8

38.3

6.8

41.9

0

20

40

60

80

100

Postal Valid email

Monday Friday

Page 21: Timing Isn’t Everything, But Money Talks

21

Experimental effects on Final Response

Base: IP5 WEB Sample (n=1077)

51.9 54.7

23.7 18.1

0

20

40

60

80

100

Monday Friday

Full household Partial household

75.6 72.8

Page 22: Timing Isn’t Everything, But Money Talks

22

Experimental effects on Final Response

Base: IP5 WEB Sample (n=1077)

51.9 54.7 53.6 53

23.7 18.1 20.4 21.4

0

20

40

60

80

100

Monday Friday No Bonus Bonus

Full household Partial household

75.6 72.8 74 74.4

Page 23: Timing Isn’t Everything, But Money Talks

23

Estimation of cost savingsWeb bonus of £5 compared to no web bonus:Small reduction in travel and mileage costs15% reduction in interviewer fees Offsetting the cost of web bonuses reduces the saving in interviewer fees to less than 5%

Page 24: Timing Isn’t Everything, But Money Talks

24

Summary of results and conclusions 4

.

Page 25: Timing Isn’t Everything, But Money Talks

25

Summary of resultsFriday mailing had a small but diluted effectWeb bonus increased full household web responseEffect of bonus varied by a number of factors, e.g.

Diminishing returns with larger upfront incentives Greater effect for households with children Greater effect when household contacted by email

Web bonus did not increase final response rateBut potential for cost savings

Page 26: Timing Isn’t Everything, But Money Talks

26

LimitationsSmall sample sizesConfounding of sample type and value of unconditional incentives sent with advance letterEstimation of costs

Page 27: Timing Isn’t Everything, But Money Talks

27

Points for discussionTiming to be looked at further?Potential for greater cost savings through targetingLarge upfront incentives vs conditional web bonus- costs

Page 28: Timing Isn’t Everything, But Money Talks

If you want further information or would like to contact the author,Carl CullinaneT. 020 7549 7158E. [email protected] us online, natcen.ac.uk

Thank you