Manifesto: Adrian Ely - Innovation, Sustainability, Development: A New Manifesto
Interaction models – the checklist manifesto(s) · Interaction models – the checklist...
Transcript of Interaction models – the checklist manifesto(s) · Interaction models – the checklist...
![Page 1: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/1.jpg)
Interaction models – the checklist manifesto(s)
Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder, 2006
Jason Windawi
![Page 2: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/2.jpg)
Interactions in linear models
1
A way of measuring the conditional effect of context on the relationship between a focal independent variable and an outcome
– How does the effect of treatment D on outcome Y vary given moderator X?
![Page 3: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/3.jpg)
“State of the art” – Brambor, Clark and Golder (2006) A checklist for empirical analysis using linear interaction models:
1. Include all constitutive terms
2. Don’t interpret constitutive terms/coefficients as unconditional marginal effects
3. Calculate and plot substantively meaningful marginal effects and standard errors
2
![Page 4: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/4.jpg)
Sociology?
3
• Breznau (2015) vs. Brooks & Manza (2006)
LiberalDemocracy?
![Page 5: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/5.jpg)
Hainmueller et al. (2016)
Two problems with the literature post-Brambor:
1. Failure to meet assumptions of a linear interaction effect (LIE)
2. Potential lack of common support (for both treatment D and moderator X) necessary
4
![Page 6: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/6.jpg)
New checklist!
Hainmueller et al. recommend adding the following diagnostics to the Brambor checklist:
1. Scatterplots
2. Binning estimator
3. Kernel estimator
5
![Page 7: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/7.jpg)
Simulated data
6
,
![Page 8: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/8.jpg)
Diagnostic 1: Binary Treatment (D)
7
SEPARATION/HETEROGENEITY• Dividedataintocasesbytreatment
LINEARITY• Doesthedistribu4onofresultsindicatealinearrela4onship?
LOESS(red)vs.regression(blue)
SUPPORT• Istheresufficientcommonsupport?
Boxplotofdistribu:onofX
![Page 9: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/9.jpg)
Diagnostic 1: Continuous Treatment (D)
8
SEPARATION/HETEROGENEITY• Dividedataintothreebinsbymoderator
LINEARITY?SUPPORT?
![Page 10: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/10.jpg)
Alternative Diagnostic 1: Generalized Additive Model
9
![Page 11: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/11.jpg)
Diagnostic 2: Binning Estimator
10
1 Separatecon4nuousmoderatorXintobins(recommend3)
2 Establishevalua4onpointsxj , j = 1, 2, 3
3 Es4matecoefficientsusingevalua4onpoints
4 Plotallthethings(newchecklist)
![Page 12: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/12.jpg)
Diagnostic 2: Plotting the Binning Estimator
11
Startwithoutputfromstandardlinearinterac4onmodelperBramboretal…
….addgraphictoshowsupportforX
![Page 13: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/13.jpg)
Diagnostic 2: Plotting the Binning Estimator
12
….addes4matesofαjatxjwith95%CIs
Why?Forbines4mator:X = xj => ME(xj) = αj
![Page 14: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/14.jpg)
Diagnostic 2: Simulation results
13
![Page 15: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/15.jpg)
Diagnostic 3: Kernel estimator
• Draws on Li and Racine (2010)’s semi-parametric, variable-coefficient model – Designed to accommodate both dichotomous and continuous
variables – Designed to capture variation in coefficient(s) of interest while
address shortcomings of approaches relying on separation/binning
14
![Page 16: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/16.jpg)
Diagnostic 3: Kernel estimator
15
AssumedModel Kernel
g(x0 ) = aD - βD(0) = aD = a(x0 )
⇒∂Y∂D(x0 ) = a
![Page 17: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/17.jpg)
Diagnostic 3: Kernel estimator
16
Graphing across support of X a(x0 )
![Page 18: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/18.jpg)
Problem replication: lack of common support*
17*Chapman(2009)
![Page 19: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/19.jpg)
Problem replication: lack of common support*
18*Chapman(2009)
![Page 20: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/20.jpg)
19
Widespread problems
![Page 21: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/21.jpg)
How widespread? A scoring system
Four possible points, one each for:
• Reject equality of marginal effects (αj) for low and high bins
• No severe interpolation or extrapolation (includes L-kurtosis hurdle)
• Monotonic
• Fail to reject linear model in Wald test vs. binned
20
![Page 22: Interaction models – the checklist manifesto(s) · Interaction models – the checklist manifesto(s) Papers discussed: Hainmueller, Mummolo and Xu, 2016 Brambor, Clark and Golder,](https://reader035.fdocuments.net/reader035/viewer/2022062603/5f4785afaec0af589735a9d3/html5/thumbnails/22.jpg)
Scoring results
21
Score: 4 3 2 1
Number 4 5 10 17
Share 8.7% 10.9% 21.7% 37%
Sample:55replica:onsfrom22papersinleadingPoli:csjournals