Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF...
Transcript of Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF...
![Page 1: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/1.jpg)
Differential item functioning analysis of
the COPSOQ across ten countries:
the NEXT Study
Jian LI, MD, MPH, PhD, Dr.rer.sec.
and the NEXT Study Group
Institute of Occupational and Social Medicine
University of Düsseldorf, Germany
The 4th International Workshop on COPSOQ
Gent, Belgium, 19-20 September 2013
![Page 2: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/2.jpg)
World Happiness Report 2013
http://unsdsn.org/files/2013/09/WorldHappinessReport2013_online.pdf
![Page 3: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/3.jpg)
![Page 4: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/4.jpg)
![Page 5: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/5.jpg)
![Page 6: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/6.jpg)
![Page 7: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/7.jpg)
Question:
Can the scores be used to
compare across countries?
![Page 8: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/8.jpg)
Whether questionnaire scales capture the same
concepts across countries?
Whether respondents interpret and respond to
items in the same way?
Differential Item Functioning (DIF) analysis is
traditionally used in Psychology, Education, and
Business/Management disciplines. Recently, it
has been applied to mental health/quality of life
measurement.
International Comparison based on questionnaire scores
![Page 9: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/9.jpg)
Translation inequivalence
Confounding effect
Cultural differences
Main sources of DIF:
![Page 10: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/10.jpg)
Orhede E, et al. Item bias in indices measuring psychosocial work
environment and health. Scand J Work Environ Health, 2000, 26 (3): 263-
272.
Pelfrene E,et al. Scale reliability and validity of the Karasek 'Job Demand-
Control-Support' model in the Belstress study. Work & Stress, 2001, 15 (4):
297-313.
Kristensen TS, et al. The distinction between work pace and working hours
in the measurement of quantitative demands at work. Work & Stress, 2004,
18 (4): 305-322.
Pejtersen JH, et al. The second version of the Copenhagen Psychosocial
Questionnaire. Scand J Public Health, 2010, 38 (3 Suppl): 8-24.
DIF analysis in research of work stress
questionnaire within one country
![Page 11: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/11.jpg)
Questionnaires
JCQ
ERI
COPSOQ
DIF methods
• Three-way contingency tables
• Item response theory
• Logistic regression
Choi B, et al. A cross-national study on the multidimensional characteristics of the
five-item psychological demands scale of the Job Content Questionnaire. Int J Behav
Med, 2008, 15 (2): 120-132.
Tsutsumi A, et al. Application of item response theory to achieve cross-cultural
comparability of occupational stress measurement. Int J Methods Psychiatr Res,
2009, 18 (1): 58-67.
DIF analysis in research of work stress
questionnaire across countries
![Page 12: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/12.jpg)
10 participating EU countries
United Kingdom
Finland
Norway
The NEXT Study
Germany
Italy
France
The Netherlands
Belgium
Poland
Slovakia
1 participating Asian country
China
![Page 13: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/13.jpg)
COPSOQ used in the NEXT Study
Quantitative Demands (QD)
QD1: Do you have to work very fast?
QD2: Is your workload unevenly distributed so it piles up?
QD3: How often do you not have time to complete all your work tasks?
Hardly/ever (1), Seldom (2), Sometimes (3), Often (4), Always (5)
Possibilities for Development (PD)
Quality of Leadership (QL)
Meaning of Work (MW)
Insecurity at Work (IW)
![Page 14: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/14.jpg)
Independent judgmental review on the
translation comparability of the COPSOQ items
England
English
Germany
German
Italy
Italian
France
French
Netherlan
ds
Dutch
Belgium
French/Flemi
sh
Finland
Finnish
Poland
Polish
Slovakia
Slovak
China
Chinese
QD
QD1 Reference A A A A A A A A A
QD2 Reference A A B A B B A A A
QD3 Reference A A A A A A A A A
Translation equivalence:
A: no difference;
B: negligible difference (no conceptual difference, but some wording is not perfect);
C: moderate difference;
D: large difference.
![Page 15: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/15.jpg)
Characteristics of the NEXT Study subjects
Note: * p < 0.05, difference among countries by Analysis of Variance or Chi-Square Test
England
(N=1694)
Belgium
(N=1954)
Germany
(N=2145)
Finland
(N=1869)
France
(N=3657)
Italy
(N=3452)
Netherlan
ds
(N=2186)
Poland
(N=3171)
Slovakia
(N=2725)
China
(N=3088)
Age
(mean ± SD) *
39.70 ±
9.69
37.48 ±
9.21
37.21 ±
9.28
41.18 ±
9.97
38.81 ±
9.36
37.18 ±
7.50
36.98 ±
9.48
37.63 ±
7.22
38.40 ±
8.65
30.94 ±
8.29
Marital status (N, (%)) *
Single 198
(11.69)
288
(14.74)
512
(23.87)
254
(13.59)
558
(15.26)
447
(12.95)
318
(14.55)
156
(4.92)
165
(6.06)
1193
(38.63)
Cohabitated 1496
(88.31)
1666
(85.26)
1633
(76.13)
1615
(86.41)
3099
(84.74)
3005
(87.05)
1868
(85.45)
3015
(95.08)
2560
(93.94)
1895
(61.37)
Position rank (N, (%)) *
Low 1176
(69.42)
1750
(89.56)
1769
(82.47)
1624
(86.89)
3460
(94.61)
3118
(90.32)
2172
(99.36)
2829
(89.21)
2441
(89.58)
2446
(79.21)
High 518
(30.58)
204
(10.44)
376
(17.53)
245
(13.11)
197
(5.39)
334
(9.68) 14 (0.64)
342
(10.79)
284
(10.42)
642
(20.79)
Shift work (N, (%)) *
Day shift 919
(54.25)
868
(44.42)
835
(38.93)
623
(33.33)
2416
(66.07)
1355
(39.25)
600
(27.45)
614
(19.36)
929
(34.09)
1036
(33.55)
Rotating shift 775
(45.75)
1086
(55.58)
1310
(61.07)
1246
(66.67)
1241
(33.93)
2097
(60.75)
1586
(72.55)
2557
(80.64)
1796
(65.91)
2052
(66.45)
![Page 16: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/16.jpg)
DIF analysis of COPSOQ
Belgium Germany Finland France Italy Netherlan
ds Poland Slovakia China
% of
sample
s with
DIF
Quantitative Demands
QD1 0 0 0 0 0 0 0 0 + 11%
QD2 0 0 + 0 0 0 + - 0 33%
QD3 0 0 - 0 - 0 - + - 56%
Note: England as reference (adjusting for age, marital status, position rank, and shift work)
0: non-DIF item
+: DIF item with higher item score for country than for England at a given score
-: DIF item with lower item score for country than for England at a given score
![Page 17: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/17.jpg)
Measurement of quantitative demands
QD1: Do you have to work very fast?
QD2: Is your workload unevenly distributed so it piles up?
QD3: How often do you not have time to complete all your work tasks?
Kristensen TS, et al. The distinction between work pace and
working hours in the measurement of quantitative demands
at work. Work & Stress, 2004, 18 (4): 305-322.
Work pace
Work time
![Page 18: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/18.jpg)
Means of Quantitative Demands
Note: ANCOVA adjusting for age, marital status, position rank, and shift work
![Page 19: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/19.jpg)
Means of Quantitative Demands
Note: ANCOVA adjusting for age, marital status, position rank, and shift work
![Page 20: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/20.jpg)
Attention should be paid for the international comparison of psychosocial work characteristics.
DIF analysis is useful for such comparison study: more sensitive than routine methods.
Conclusion
![Page 21: Differential item functioning analysis of the COPSOQ ...€¦ · Questionnaires JCQ ERI COPSOQ DIF methods • Three-way contingency tables • Item response theory • Logistic regression](https://reader033.fdocuments.net/reader033/viewer/2022053101/605d7eee12cfd527615b9619/html5/thumbnails/21.jpg)
Thank you
for your attention!