Timing Isn’t Everything, But Money Talks

Click here to load reader

download Timing Isn’t Everything, But Money Talks

of 28

  • date post

  • Category


  • view

  • download


Embed Size (px)


University of Essex, July 2013. Timing Isn’t Everything, But Money Talks. How to encourage a face-to-face household panel to go online?. Gerry Nicolaas Carl Cullinane. Contents. Background Design of Experiment Results Summary of Results and Conclusions. Background. 1. Background. - PowerPoint PPT Presentation

Transcript of Timing Isn’t Everything, But Money Talks

How to encourage a face-to-face household panel to go online?

Timing Isnt Everything, But Money Talks

How to encourage a face-to-face household panel to go online? University of Essex, July 2013

Gerry NicolaasCarl Cullinane1ContentsBackgroundDesign of ExperimentResults Summary of Results and Conclusions

Background1.3BackgroundSwitch to mixed mode data collection at wave 7 of Understanding SocietyPotential for cost savingsPotential for reducing attrition

4ContextLongitudinal household panelFace-to-face interviewing of all household members at waves 1 to 6Greatest potential for reducing data collection costs when an interviewer does not have to visit the householdPrevious experiment mixing telephone & face-to-face:Costs can be reducedBUT response rates suffer

5Innovation Panel wave 5Vehicle for methodological development & testing About 2,500 individuals in 1,500 householdsMain objective of IP5 =Determine whether it is possible to reduce costs by mixing web questionnaires and face-to-face interviews without sacrificing data qualitySequential mixed mode design starting with web66IP5 DesignExperimental groupF2F phase, web openWeb only phaseControl groupF2F only phaseNo web7Research question Is it possible to boost the proportion of whole households completing web questionnaires byTiming the arrival of the invitation to go online?Offering a web bonus?Design of experiment within mixed-mode sample2.9Design of experiment within mixed-mode sample(1) Timing of invitation to go onlineRandom allocation of households to:Letter (+ email) to arrive on FridayLetter (+ email) to arrive on MondayReminder letters (+ emails) sent 2 and 4 days later(2) Web bonusRandom allocation of households to:No web bonus5 per household member conditional on all completing online questionnaire

10Design of experiment within mixed-mode sample270 households265 households266 households276 householdsNo web bonusCond. 5 per hhld memberInvite arrival on FridayInvite arrival on MondayResults3.12Experimental effects on Web ResponseBase: IP5 WEB Sample (n=1077)

13Experimental effects on Web ResponseBase: IP5 WEB Sample (n=1077)

14Effect of Bonus by Sample TypeBase: IP5 WEB Sample (n=1077)

15Effect of Bonus by Upfront IncentivesBase: IP5 WEB Sample (n=1077)

1616Unconditional IncentivesOriginal SampleRefreshment Sample510103020

17Effect of Bonus by Children in HouseholdBase: IP5 WEB Sample (n=1077)

18Effect of Bonus by Advance MailingBase: IP5 WEB Sample (n=1077)

19Effect of timing by Advance MailingBase: IP5 WEB Sample (n=1077)

20Experimental effects on Final ResponseBase: IP5 WEB Sample (n=1077)

75.672.821Experimental effects on Final ResponseBase: IP5 WEB Sample (n=1077)

75.672.87474.422Estimation of cost savingsWeb bonus of 5 compared to no web bonus:Small reduction in travel and mileage costs15% reduction in interviewer fees Offsetting the cost of web bonuses reduces the saving in interviewer fees to less than 5%

23Summary of results and conclusions4.24Summary of resultsFriday mailing had a small but diluted effectWeb bonus increased full household web responseEffect of bonus varied by a number of factors, e.g.Diminishing returns with larger upfront incentivesGreater effect for households with childrenGreater effect when household contacted by emailWeb bonus did not increase final response rateBut potential for cost savings

25LimitationsSmall sample sizesConfounding of sample type and value of unconditional incentives sent with advance letterEstimation of costs

26Points for discussionTiming to be looked at further?Potential for greater cost savings through targetingLarge upfront incentives vs conditional web bonus- costs

If you want further information or would like to contact the author,Carl CullinaneT. 020 7549 7158E. carl.cullinane@natcen.ac.ukVisit us online, natcen.ac.ukThank you