Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th...
Transcript of Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th...
![Page 1: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/1.jpg)
Mixed mode in LFS: questionnaire design and mode-effects
Liisa Larja, Statistics Finland
9th Workshop on Labour Force Survey Methodology
Rome, 15th of May 2014
![Page 2: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/2.jpg)
StatFin/EssNet web-project:
1. Designing LFS-CAWI questionnaire + iterative testing with cognitive interviews (N = 22)
2. Pilot survey (N = 8 000), experimental design (”split-plot”) with DK-option, working time and job search questions. Comparison to CAWI data from the same period. Paradata, respondent feedback.
![Page 3: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/3.jpg)
Content of the presentation:
1. Questionnaire design: results & recommendations
2. Data collection process
3. Mode-effects: WSTATOR (employment status)
4. Mode-effects: HWACTUAL (actual working hours)
5. Qualitative feedback
![Page 4: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/4.jpg)
1. Questionnaire design: layout
BlaiseIS default layout was not perceived as attractive Number of question / page Navigation bars Instructions Error messages Don’t know –option (DK)
![Page 5: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/5.jpg)
Error message, DK
Instructions, info-button
Status bar
Number of questions, updating the remaining questions
Scrolling down for navigation bars
![Page 6: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/6.jpg)
When the cursor points the info-symbol, the instructions appear in a box below.
![Page 7: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/7.jpg)
Navigation bars:Takaisin = Back Jatka = ContinueJatka myöhemmin = Continue later/pause
Kyllä = YesEn = No(Don’t know not visible)
![Page 8: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/8.jpg)
Kyllä = yesEi = noEn osaa sanoa = Don’t know
An error message: Please respond to this question or choose the “Don’t know” option.
DK-option, version 1
![Page 9: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/9.jpg)
Kyllä = yesEi = noEn osaa sanoa = Don’t know
DK-option, version 2
![Page 10: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/10.jpg)
2. Data collection process
Response rate 30 % potential for savings in data collection costs when the sample is large enough
CAWI-data more ”distorted” ? CAPI non-response more evenly distriputed with regard to educational background, gender and age than CAWI-data.
Speed: 40 % responses were received withing 3 days after that switch to CAPI?
![Page 11: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/11.jpg)
3. Mode-effects: WSTATOR (employment status)
![Page 12: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/12.jpg)
3. Mode-effects: WSTATOR (employment status)
No significant mode-effects, but….• media attention remains if unemployment rate 8,2 % instead of 7,4 %• sample for pilot study was small• CAWI respondents answer more often that they have been looking for job• = mode effect or selection bias?
![Page 13: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/13.jpg)
4. Mode-effects: HWUSUAL (actual working hours)
”One question” –format for HWACTUAL:
“How many hours did you work in your main job during the week 7.10.-13.10.2013?”
___ hours ___ minutes
![Page 14: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/14.jpg)
”Grid” –format for HWACTUAL:“How many hours did you work in your main job during the week 7.10.-13.10.2013?”
Date Hours Minutes
Monday 7.10.2013 ___ ___
Tuesday 8.10.2013 ___ ___
Wednesday 9.10.2013 ___ ___
Thursday 10.10.2013 ___ ___
Friday 11.10.2013 ___ ___
Saturday 12.10.2103 ___ ___
Sunday 13.10.29013 ___ ___
Total ___ ___
![Page 15: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/15.jpg)
Average number of hours worked (CATI-data and two different CAWI-datas)
![Page 16: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/16.jpg)
5. Qualitative feedback
Too intimate information on employer
Questions do not fit for people with multiple jobs without main job or with scholarship
All kind of entrepreneurial activity is not covered
Working time items do not fit for people in atypical employment
Pensioners do not understand point of reference week
![Page 17: Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of.](https://reader036.fdocuments.net/reader036/viewer/2022062718/56649e7f5503460f94b8374c/html5/thumbnails/17.jpg)
Conclusions and lessons learnt
Response rate of 30 % and speed of data collection encourage to continue towards mixed-mode LFS.
Visual design is important: balanced layout, intuitive navigation bars, informative error messages, a few questions per page, hidden instructions, hidden DK.
CAWI data more ”distorted”
In working time questions different format for CATI and CAWI may be needed to reduce mode-effects (”functional equivalency”). Differences in WSTATOR need to be further analysed.