NSSE 2013 How to Use Results (or “Why you should care about NSSE”)
Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?
description
Transcript of Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?
![Page 1: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/1.jpg)
Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?
Presented by Chris Maxwell
Purdue University
AIR 2009
![Page 2: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/2.jpg)
Introduction
Purdue NSSE 2004: Max Item Nonresponse: 10%
Purdue NSSE 2007: Max Item Nonresponse: 18%
Purdue NSSE 2007:• Web Only• Targeted Oversample: Underrepresented• 24% response rate• 1,388 total respondents (703 FY and 685 SN)
![Page 3: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/3.jpg)
All Students (FY and SN)
70%
75%
80%
85%
90%
95%
100%
1 11 21 31 41 51 61 71 81
NSSE 2007 Question Order
Pe
rce
nt
Re
sp
on
din
g
N=1,388
![Page 4: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/4.jpg)
Investigation
Question order is an apparent factor, but is anything else at play?
Model the response probability with logistic regression:
• Separate models for students grouped by class, ethnicity, gender, and STEM status (18 total).
• Each model includes the explanatory variables question order, type, length, and first page status.
![Page 5: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/5.jpg)
Model Details
Ln(P/(1-P)) =A+B1(order)+B2(page1)+B3(likert4_scale)
+B4(numerical_scale)+B5(length)
Use SAS proc Logistic (binary modeling, stepwise selection)
Input data set
![Page 6: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/6.jpg)
Model Results
Stepwise selection rejected question types and length as useful predictors in every case
15 models use question order and page one status, with 3 models only using question order
Ln(P/(1-P)) = A + B1(order) + B2(page1) -or- P = exp(A + B1(order) + B2(page1)) 1 + exp(A + B1(order) + B2(page1))
![Page 7: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/7.jpg)
70%
75%
80%
85%
90%
95%
100%
1 11 21 31 41 51 61 71 81
NSSE 2007 Question Order
Per
cen
t R
esp
on
din
g
SN
FY
FY and SN Results
N=685
N=703
![Page 8: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/8.jpg)
FY and SN Results
70%
75%
80%
85%
90%
95%
100%
1 11 21 31 41 51 61 71 81
NSSE 2007 Question Order
Per
cen
t Res
po
nd
ing
SN
FYN=685
N=703
![Page 9: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/9.jpg)
Model Parameters
FY SN
A 2.39 2.70
B1 -0.0141 -0.0130
B2 4.69 4.58
Percent Concordant
62.7 61.6
Thus, for FY students the model predicts a 9% decline in response after the first page, and in the middle of the survey a 1% drop for every 6 questions.
![Page 10: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/10.jpg)
FY: STEM and Non STEM Majors
70%
75%
80%
85%
90%
95%
100%
1 11 21 31 41 51 61 71 81
NSSE 2007 Question Order
Pe
rce
nt
Re
sp
on
din
g
STEM
Non STEMN=301
N=399
![Page 11: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/11.jpg)
70%
75%
80%
85%
90%
95%
100%
1 11 21 31 41 51 61 71 81
NSSE 2007 Question Order
Pe
rce
nt
Re
sp
on
din
g
STEM
Non STEM
SN: STEM and Non STEM Majors
N=459
N=219
![Page 12: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/12.jpg)
FY: Male and Female
70%
75%
80%
85%
90%
95%
100%
1 11 21 31 41 51 61 71 81
NSSE 2007 Question Order
Pe
rce
nt
Re
sp
on
din
g
Female
Male
N=347
N=356
![Page 13: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/13.jpg)
SN: Male and Female
70%
75%
80%
85%
90%
95%
100%
1 11 21 31 41 51 61 71 81
NSSE 2007 Question Order
Pe
rce
nt
Re
sp
on
din
g
Female
MaleN=339
N=346
![Page 14: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/14.jpg)
FY: All Underrepresented
70%
75%
80%
85%
90%
95%
100%
1 11 21 31 41 51 61 71 81
NSSE 2007 Question Order
Pe
rce
nt
Re
sp
on
din
g
White/Other/Asian
Underrepresented
N=99
N=564
![Page 15: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/15.jpg)
70%
75%
80%
85%
90%
95%
100%
1 11 21 31 41 51 61 71 81
NSSE 2007 Question Order
Pe
rce
nt
Re
sp
on
din
g
White/Other/Asian
African American
FY: African American
N=564
N=45
![Page 16: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/16.jpg)
70%
75%
80%
85%
90%
95%
100%
1 11 21 31 41 51 61 71 81
NSSE 2007 Question Order
Pe
rce
nt
Re
sp
on
din
g
White/Other/Asian
Hispanic
FY: Hispanic
N=564
N=44
![Page 17: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/17.jpg)
SN: All Underrepresented
70%
75%
80%
85%
90%
95%
100%
1 11 21 31 41 51 61 71 81
NSSE 2007 Question Order
Pe
rce
nt
Re
sp
on
din
g
White/Other/Asian
UnderrepresentedN=592
N=76
![Page 18: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/18.jpg)
70%
75%
80%
85%
90%
95%
100%
1 11 21 31 41 51 61 71 81
NSSE 2007 Question Order
Pe
rce
nt
Re
sp
on
din
g
White/Other/Asian
African American
SN: African American
N=592
N=36
![Page 19: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/19.jpg)
70%
75%
80%
85%
90%
95%
100%
1 11 21 31 41 51 61 71 81
NSSE 2007 Question Order
Pe
rce
nt
Re
sp
on
din
g
White/Other/Asian
Hispanic
SN: Hispanic
N=592
N=36
![Page 20: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/20.jpg)
All
Mo
del
Par
amet
ers
Model A (Intercept) B1 (order) B2 (page1)
sn_stem 3.36 -0.0197
sn_female 3.02 -0.0137 4.26
sn_white 2.74 -0.0131 4.68
sn_all 2.70 -0.0130 4.58
fy_female 2.67 -0.0153 5.35
sn_hispanic 2.65 -0.0123 3.07
sn_not_stem 2.58 -0.0127 4.30
fy_stem 2.50 -0.0126 4.25
fy_white 2.46 -0.0141 4.62
sn_male 2.46 -0.0128 4.83
fy_hispanic 2.42 -0.0184
fy_all 2.39 -0.0141 4.69
fy_not_stem 2.31 -0.0152 5.13
sn_black 2.28 -0.0183
sn_underrep 2.27 -0.0118 4.19
fy_male 2.16 -0.0134 4.44
fy_black 1.98 -0.0158 3.98
fy_underrep 1.92 -0.0133 4.81
![Page 21: Factors Influencing Item Nonresponse in the NSSE Survey: Why do Students Skip so many Questions?](https://reader036.fdocuments.net/reader036/viewer/2022062518/568145cb550346895db2d410/html5/thumbnails/21.jpg)
Summary
Study Limitations
Conclusions
Discussion
Available at: www.purdue.edu/OIR/irvba/AIR2009/nsse.ppt