Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results
description
Transcript of Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results
![Page 1: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/1.jpg)
Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results
Mario Callegaro, Tim Macer
![Page 2: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/2.jpg)
Mobile Phone Penetration Up
![Page 3: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/3.jpg)
Rules of Thumb
No horizontal scrollingVertical scrolling OK
Avoid long listsEspecially in check all that apply
Situation FluidAs Tablet become Popular
Depends on the Platform
![Page 4: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/4.jpg)
Platform Considerations
Need to Test on Multiple Platforms
Apple (iPhone/iPad) can’t implement Flash
Default on all phones is to not enable Java
![Page 5: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/5.jpg)
Useful Paradata
UserAgentStringDevice\Model
Operating System
Screen Resolution
Fonts
![Page 6: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/6.jpg)
“Can You See It Now? Good”Usability Testing of a Mobile Health Application
Sarah Cook, Rita Sembajwe, Emily Geisen, Barbara Massoudi
![Page 7: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/7.jpg)
New Way to Do a Diary
![Page 8: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/8.jpg)
Benefits
Immediate Results
Cost Effective
Create Easy to Use Dashboard
![Page 9: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/9.jpg)
Usability Suggestions
Don’t scroll vertically on Select All
Make it easy to trace any sliding
Hard to video what they do
![Page 10: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/10.jpg)
Mobile Phone Effects at Event-Based Sampling
Dan Williams
![Page 11: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/11.jpg)
Case Study
![Page 12: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/12.jpg)
Three Modes of Collection
WebMost Popular
Not all on Mobile Device
IVRCapture Older Population
SMSImmediate Response
Younger Respondents
![Page 13: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/13.jpg)
Are you who you say you are? Using a Multisource Cross-validation Methodology for
Panel Membership Information.
Kumar Rao
![Page 14: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/14.jpg)
Real, Unique, and Engaged
3rd Party Database ValidationInclude Demographics
Use Multiple Databases
![Page 15: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/15.jpg)
Results
Cost could be worth the extra
All more likely to be established households
False Positives Too High
Still Important Part of Process
![Page 16: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/16.jpg)
Differential Sampling Based on Historical Individual-Level Data in Online Panels
Richard Kelly
![Page 17: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/17.jpg)
Quota Sampling
Way to Deal with Non-ResponseDidn’t Know Demographics
More Efficient to Screen Out
Just Transferred Over
to Online
![Page 18: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/18.jpg)
Differential Sampling
Know the Demographics
Know the Response Rates
Oversample those Hard to Reach
More Efficient and Cost Effective
![Page 19: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/19.jpg)
Designing Questions for Web Surveys: Effects of Check-List, Check-All, and Stand-Alone Response
Formats on Survey Reports and Data Quality
Jennifer Dykema, Nora Cate Schaeffer, Jeremy Beach, Vicki Lein, and Brendan Day
![Page 20: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/20.jpg)
Three Types Web Designs
Check-ListMore Items Selected
Check-AllLower Break-offs
Stand AloneLess Primacy Effect
![Page 21: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/21.jpg)
Category Selection Probing in Online Access Panels
Dorothée Behr, Lars Kaczmirek,
Michael Braun, Wolfgang Bandilla
![Page 22: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/22.jpg)
Cognitive Testing OE
Face-to-Face too Expensive
Online TestingProbing Open Ends
Community vs PanelMore chatty?
![Page 23: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/23.jpg)
Results
Topic Trumps Source
Use Communities Built Around the Topic
Face-to-Face More Involved
![Page 24: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/24.jpg)
Response Quantity, Response Quality, and Costs of Building an Online Panel via Social Contacts
Vera Toepoel
![Page 25: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/25.jpg)
Snowball Recruiting
No Online Panel in NL RepresentativeRequires More Commitment
Try Refer a Friend Program
Use Network Theory
![Page 26: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/26.jpg)
Results
Snow Never RolledOnly got 120 recruits
Don’t Use Students
Incentives not Worth the Cost
![Page 27: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/27.jpg)
Representativeness
![Page 28: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/28.jpg)
The Use of Web Panels to Characterize Rare Conditions
John Boyle
![Page 29: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/29.jpg)
Hard to Reach Population
Only 23 in a sample of 10,000 HH
Costs High
Variance Too High
Important Diseases
![Page 30: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/30.jpg)
Clean the Online Data
Certain Improbable Conditions
Speeders
Straightliners
![Page 31: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/31.jpg)
Results In Line
Prevalence In Line
Treatments Numbers Good
Got Much More Sample Size
Cost Less
![Page 32: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/32.jpg)
Measuring Intent to Participate and Participation in the 2010 Census and Their Correlates and Trends:
Comparisons of RDD Telephone and Non-probability Sample Internet Survey Data
Josh Pasek and Jon Krosnick
![Page 33: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/33.jpg)
Intent to Complete Census
Better Demographics Compositions
Intent Numbers Varied
Predictors for Intent to Complete Different
Trends Also Different
![Page 34: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/34.jpg)
Can a Non-Probability Sample Ever be Useful for Representing a Population?: Comparing Probability and Non-Probability Samples of Recent College Graduates
Cliff Zukin, Jessica Godofsky, Carl Van Horn, Wendy Mansfield, and J. Micheal Dennis
![Page 35: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/35.jpg)
Comparing Sampling
Probability Samples have a Prob Theory
Can’t Intelligently Trade Off Error
Compare KN Panel to volunteer Panel
Recent Graduates
![Page 36: Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results](https://reader035.fdocuments.net/reader035/viewer/2022062314/56813d58550346895da71e62/html5/thumbnails/36.jpg)
Results
Differences between probability and non-probability panel
No mode effects or questionnaire effects
Differences mitigated a lot when weighting for other non-quota variables