Results of Quality Control Surveys of Radioimmunological ...
Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile...
Transcript of Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile...
![Page 1: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/1.jpg)
GOR 2018 | 2nd March 2018
Effects of mobile devices on data quality in web-based surveys
![Page 2: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/2.jpg)
2
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
Device Ownership
90% 43% 58%
2017 2017 2016
Laufende Wirtschaftsrechnung, Statistisches Bundesamt
![Page 3: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/3.jpg)
3
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
Mobile Devices
>15‘‘ 7-12‘‘ <6‘‘
External Keyboard
+ Mouse
Internal Keyboard
+ Touch
Internal Keyboard
+ Touch
Cable or WiFiWiFi or mobile
network
WiFi or mobile
network
![Page 4: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/4.jpg)
4
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
Difficulties when Assessing the Effects
SELECTION EFFECTS
MEASUREMENT EFFECTS
![Page 5: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/5.jpg)
5
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
Propensity Score Matching
AGE
GENDER
HOUSEHOLD INCOME
EDUCATION
EMPLOYMENT STATUS
EXPERIENCE WITH SURVEYS
AFFINITY FOR TECHNOLOGY
INTERNET USAGE
EXPERIENCE WITH DEVICE
Comparison of “similar” respondents within treatment and control group
QUASI EXPERIMENT
![Page 6: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/6.jpg)
6
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
Data
Gesis Panel
10th wave (August 2015)
Not optimized for mobile
![Page 7: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/7.jpg)
7
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
Indicators for Data Quality
LENGTH OF OPEN
ANSWERS
PRIMACY
EFFECTS
SURVEY
DURATION
![Page 8: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/8.jpg)
8
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
Results
LENGTH OF OPEN ANSWERS
![Page 9: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/9.jpg)
9
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
* p < 0,10 ** p < 0.05 *** p < 0.01
Results
- 23** Characters - 9 Characters + 3 Characters
LENGTH OF OPEN ANSWERS
![Page 10: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/10.jpg)
10
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
Results
PRIMACY EFFECTS
![Page 11: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/11.jpg)
11
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
+ 22 PP***more 1./2. answers
+ 1 PPmore 1./2. answers
+/- 0 PPmore 1./2. answers
* p < 0,10 ** p < 0.05 *** p < 0.01
Results
PRIMACY EFFECTS
![Page 12: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/12.jpg)
12
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
Results
SURVEY DURATION
![Page 13: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/13.jpg)
13
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
* p < 0,10 ** p < 0.05 *** p < 0.01
Results
+ 6.5** minutes + 4.6* minutes + 4.7** minutes
SURVEY DURATION
![Page 14: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/14.jpg)
14
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
Lower Evaluation of the Study
Higher Item Nonresponse
Shorter Open Answers
Higher Primacy Effects
Stronger Left-Alignment
Longer Duration
Conclusion
![Page 15: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/15.jpg)
15
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
Responsive
survey design
Item nonresponse:
Make answers
mandatory
Multiple choice
questions:
Randomize answer
options
Exclusion of
mobile devices or
asking the
respondents to
switch
Possible Solutions
Optimization for
and testing on a
variety of devices
![Page 16: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/16.jpg)
16
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
Data
30 SKOPOS surveys
2017
Optimized for mobile devices
![Page 17: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/17.jpg)
17
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
Participation by Device
Single survey
76%
24%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
![Page 18: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/18.jpg)
18
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
Data Quality
Closed Questions
(Average std
in item set)
Open-Ended Questions
(Number of characters written)
0,860,96
60,5352,51
![Page 19: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/19.jpg)
19
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
Completion Rate
Single survey
82%
77%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
![Page 20: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/20.jpg)
20
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
When do respondents drop out? (Average seconds)
363 sec.333 sec.
![Page 21: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/21.jpg)
21
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
But is survey length and drop-out rate correlated?
R² = 0,057
0%
25%
50%
75%
100%
0 5 10 15 20
Average interview duration (minutes)
Com
plet
ion
Rat
e R² = 0,0657
0%
25%
50%
75%
100%
0 5 10 15 20 25
Single survey
Average interview duration (minutes)
![Page 22: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/22.jpg)
22
GOR 2018 | Effects of mobile devices on data quality in web-based surveys
Conclusion
Survey
Duration
(Sweetspot: 5 min.)
Mobile ExperienceResponsive Design
![Page 23: Effects of mobile devices on data quality in web-based surveys · GOR 2018 | Effects of mobile devices on data quality in web-based surveys Data Quality Closed Questions (Average](https://reader031.fdocuments.net/reader031/viewer/2022041309/5e14f789f55bfc5346511992/html5/thumbnails/23.jpg)
23
GOR 2018 | Learning From All Answers | March 1, 2018
Further questions? Let‘s talk!
Thank You For Your Attention!
Patrick Schlickmann
Project Manager
Sebastian Schmidt
Director Research & Development