Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of...
Transcript of Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of...
![Page 1: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/1.jpg)
Measuring the Correlates of Intent to Participate and Participation in the Census and Trends
in These Correlates:
Comparisons of RDD Telephone and Non-probability Sample Internet Survey Data
Josh PasekStanford & University of Michigan
Jon A. KrosnickStanford & U.S. Census Bureau
![Page 2: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/2.jpg)
Monitoring Threats toCensus Compliance
U.S. Census Bureau commissioned surveys in late 2009 and early 2010 to determine factors that might cause people to complete or not
complete the form
![Page 3: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/3.jpg)
To Assess This . . .
• Examine what proportion of individuals hold particular beliefs about the Census
• Test whether those beliefs relate to intent to complete or completion of Census form
• Regularly examine prevalence of beliefs to assess emergent threats to Census completion
![Page 4: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/4.jpg)
Two Synchronous Data Streams
• 2 data streams collected for Census Bureau
• RDD telephone (Gallup) and non-probability Internet (E-Rewards)
• 13 simultaneous weeks of data collection with > 900 interviews per data stream per week
• Very similar measures
![Page 5: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/5.jpg)
Two Synchronous Data Streams• Intent to complete Census (before and after forms mailed)
• Reported completion of Census (after forms mailed)
• Census will help/hurt respondent
• Locate illegal immigrants
• Trust confidentiality
• Time to fill out
• Importance of counting everyone
• Respondent's participation does not matter
![Page 6: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/6.jpg)
Today's Question
Would These Two Data Streams Lead To The Same
Conclusions?
![Page 7: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/7.jpg)
Three Comparisons
• Proportions
•
• Relations between variables
•
• Trends over time
•
![Page 8: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/8.jpg)
Three Comparisons
• Proportions
• Who holds particular beliefs about Census?
• Relations between variables
•
• Trends over time
•
![Page 9: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/9.jpg)
Three Comparisons
• Proportions
• Who holds particular beliefs about Census?
• Relations between variables
• Which beliefs relate to expected and actual completion?
• Trends over time
•
![Page 10: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/10.jpg)
Three Comparisons
• Proportions
• Who holds particular beliefs about Census?
• Relations between variables
• Which beliefs relate to expected and actual completion?
• Trends over time
• Do the surveys indicate that purported predictors changed similarly over time?
![Page 11: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/11.jpg)
Weighting
• Unweighted
• Weights provided by both houses
• Weighted identically using anesrake (Pasek, 2010)
![Page 12: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/12.jpg)
Weighting
• Unweighted
• Weights provided by both houses
• Weighted identically using anesrake (Pasek, 2010)
Both with and without matching on dates
![Page 13: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/13.jpg)
Three Comparisons
• Proportions
• Relations between variables
• Trends over time
![Page 14: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/14.jpg)
Three Comparisons
• Proportions
• Relations between variables
• Trends over time
![Page 15: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/15.jpg)
Proportions
• Absolute difference between modal categories of non-demographic variables
• | proportion in RDD - proportion in Internet |
• Bootstrap to test significance
• Assessed within each week
![Page 16: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/16.jpg)
ProportionsDifferences Between Data Streams
Difference in Percentage Points
Freq
uenc
y of
Diff
eren
ces
![Page 17: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/17.jpg)
Difference in Percentage Points
Freq
uenc
y of
Diff
eren
ces
ProportionsDifferences Between Data Streams
Week 10Census can help you
RDD: 43.4%Internet: 44.2%Difference: 0.8%
![Page 18: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/18.jpg)
Difference in Percentage Points
Freq
uenc
y of
Diff
eren
ces
ProportionsDifferences Between Data Streams
Week 3Rs participation does not matter - agree
RDD: 54.4%Internet: 40.5%Difference: 13.9%
![Page 19: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/19.jpg)
Difference in Percentage Points
Freq
uenc
y of
Diff
eren
ces
ProportionsDifferences Between Data Streams
Week 12Important to CountEveryone - Agree
RDD: 65.7%Internet: 32.2%Difference: 33.5%
![Page 20: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/20.jpg)
ProportionsDifferences Between Data Streams
Difference in Percentage Points
Freq
uenc
y of
Diff
eren
ces
Diff. Data
> 5 -- 80%
> 10 -- 55%
> 15 -- 40%
> 20 -- 26%
> 25 -- 16%
![Page 21: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/21.jpg)
Three Comparisons
• Proportions
• Relations between variables
• Trends over time
![Page 22: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/22.jpg)
Relations Between Variables
• Regressions predicting relevant outcomes
• Intent to complete Census
• Reported completion of Census
• Correlations between pairs of variables
![Page 23: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/23.jpg)
Predicting Census Completion
Variable RDD Internet Difference
Numbers shown are from a logistic regression with all measures * p<.05 | ** p<.01 | *** p<.001
![Page 24: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/24.jpg)
Predicting Census Completion
Variable RDD Internet Difference
Married .31* .58** .27
Same story, same magnitude
Numbers shown are from a logistic regression with all predictors simultaneously * p<.05 | ** p<.01 | *** p<.001
![Page 25: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/25.jpg)
Variable RDD Internet Difference
Married .31* .58** .27
Age 25-34 -1.51*** -.68* .83*
Numbers shown are from a logistic regression with all predictors simultaneously * p<.05 | ** p<.01 | *** p<.001
Predicting Census Completion
Same story, different magnitude
![Page 26: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/26.jpg)
Variable RDD Internet Difference
Married .31* .58** .27
Age 25-34 -1.51*** -.68* .83*
Don't have time to fill out - Disagree .88** .68 .19
Predicting Census Completion
Different story, same magnitude
Numbers shown are from a logistic regression with all predictors simultaneously * p<.05 | ** p<.01 | *** p<.001
![Page 27: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/27.jpg)
Variable RDD Internet Difference
Married .31* .58** .27
Age 25-34 -1.51*** -.68* .83*
Don't have time to fill out - Disagree .88** .68 .19
Importance of counting everyone - Agree .98* -.37 1.27*
Numbers shown are from a logistic regression with all measures * p<.05 | ** p<.01 | *** p<.001
Predicting Census Completion
Different story, different magnitude
![Page 28: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/28.jpg)
Predicting Census Completion
TypeOne
Predictor at a Time
AllPredictors
Same storySame magnitude 57% 31%
Same storyDifferent magnitude 13 6
Different storySame magnitude 30 56
Different storyDifferent magnitude 0 6
Total(Number of Variables)
100%(23)
100%(16)
Numbers shown are for variables that were significant in at least one data stream
![Page 29: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/29.jpg)
Predicting Census Completion
TypeOne
Predictor at a Time
AllPredictors
Same storySame magnitude 57% 31%
Same storyDifferent magnitude 13 6
Different storySame magnitude 30 56
Different storyDifferent magnitude 0 6
Total(Number of Variables)
100%(23)
100%(16)
Numbers shown are for variables that were significant in at least one data stream
![Page 30: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/30.jpg)
Predicting Intent to Complete FormType
OnePredictor at a
Time
All Predictors
Same storySame magnitude 41% 48%
Same storyDifferent magnitude 29 20
Different storySame magnitude 15 20
Different storyDifferent magnitude 15 12
Total(Number of Variables)
100%(41)
100%(25)
Numbers shown are for variables that were significant in at least one data stream
![Page 31: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/31.jpg)
Predicting Intent to Complete FormType
OnePredictor at a
Time
All Predictors
Same storySame magnitude 41% 48%
Same storyDifferent magnitude 29 20
Different storySame magnitude 15 20
Different storyDifferent magnitude 15 12
Total(Number of Variables)
100%(41)
100%(25)
Numbers shown are for variables that were significant in at least one data stream
![Page 32: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/32.jpg)
Three Comparisons
• Proportions
• Relations between variables
• Trends over time
![Page 33: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/33.jpg)
Trends Over Time
• Correlations between variable categories over weeks
• Chi-squared tests comparing differences between data streams across weeks
Among variables with significant variations over time in at least one data stream
![Page 34: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/34.jpg)
Sometimes the trends match
![Page 35: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/35.jpg)
Trends Over Time
Date
Perc
ent
RDDInternet
![Page 36: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/36.jpg)
Trends Over Time
Date
Perc
ent RDD
Internet
![Page 37: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/37.jpg)
Sometimes the trends don't match
![Page 38: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/38.jpg)
Trends Over Time
Date
Perc
ent
RDD
Internet
![Page 39: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/39.jpg)
Trends Over Time
Date
Perc
ent RDD
Internet
![Page 40: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/40.jpg)
And sometimes, the trends are opposites
![Page 41: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/41.jpg)
Trends Over Time
Date
Perc
ent
RDDInternet
![Page 42: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/42.jpg)
Trends Over Time
Date
Perc
ent RDD
Internet
![Page 43: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/43.jpg)
Trends Over Time
Among variables with significant variations over time in at least one data stream
Correlations Between Data Streams
Freq
uenc
y
![Page 44: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/44.jpg)
Differences Between Streams vs. "Sampling Error"
0
20
40
60
80
100
Proportions Relations Trends
Chance
73%significantly
different
30%significantly
different
76%significantly
different
![Page 45: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/45.jpg)
Differences Between Streams vs. "Sampling Error"
0
20
40
60
80
100
Proportions Relations Trends
Chance
73%significantly
different
30%significantly
different
76%significantly
differentNone of the weighting strategies
changed these basic results
![Page 46: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/46.jpg)
Data from one of the top RDD firms and one of the most visible
Internet firms produced very different results
![Page 47: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/47.jpg)
Researchers need to choose, and that choice should depend
on the validity of the results
![Page 48: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/48.jpg)
But Which is More Valid?
• More accurate self-reports using Internet mode (Chang and Krosnick, 2009)
• Some theoretical reasons to prefer probability sampling (even with contemporary response rates)
We cannot know based on these data alone
![Page 49: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/49.jpg)
Measuring the Correlates of Intent to Participate in the Census and Trends
in These Correlates:
Comparisons of RDD Telephone and Non-probability Sample Internet Survey Data
Josh PasekStanford & University of Michigan
Jon A. KrosnickStanford & U.S. Census Bureau
![Page 50: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/50.jpg)
![Page 51: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/51.jpg)
![Page 52: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/52.jpg)
Not really the point . . .
If the methods reach different conclusions, they can't both be correct
![Page 53: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/53.jpg)
What Can We Make Of This?
• Mode effects
• Slight differences in question wording
• Neither survey is consistently within sampling error of accuracy benchmarks (though RDD is a little closer)
![Page 54: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/54.jpg)
The results are not equivalent!
![Page 55: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/55.jpg)
Benchmark Comparison
• Comparison of modal categories
• Variables not used in weighting or quotas• Primary household language (English)• Own or rent home (Own)• Children in household (Yes)
• Absolute difference between sample mean and benchmark (weighted)
![Page 56: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/56.jpg)
Benchmark ComparisonTelephone Internet
Freq
uenc
y
Percentage Point Difference Percentage Point Difference
11.4 Avg. Error 13.8
18.6 Max Error 21.1
![Page 57: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/57.jpg)
Benchmark ComparisonTelephone Internet
Freq
uenc
y
Percentage Point Difference Percentage Point Difference
11.4 Avg. Error 13.8
18.6 Max Error 21.1p<.001 difference
![Page 58: Josh Pasek Jon A. Krosnick - aapor.org€¦ · To Assess This . . . • Examine what proportion of individuals hold particular beliefs about the Census • Test whether those beliefs](https://reader035.fdocuments.net/reader035/viewer/2022070611/5b15f49d7f8b9a00708c1331/html5/thumbnails/58.jpg)
Benchmark Conclusions
• Neither survey is consistently within sampling error
• RDD sample is somewhat more accurate than Internet sample