Introduction of Regression Discontinuity Design (RDD)
-
Upload
amya-herrin -
Category
Documents
-
view
242 -
download
0
Transcript of Introduction of Regression Discontinuity Design (RDD)
![Page 1: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/1.jpg)
Introduction of Regression Discontinuity Design (RDD)
![Page 2: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/2.jpg)
This Talk Will:
Introduce the history and logic of RDD, Consider conditions for its internal validity, Considers its sample size requirements, Consider its dependence on functional form, Illustrate some specification tests for it, Describe an application. Consider limits to its external validity, Consider how to deal with noncompliance,
![Page 3: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/3.jpg)
RDD History
In the beginning there was Thislethwaite and Campbell (1960)
This was followed by a flurry of applications to Title I (Trochim, 1984)
Only a few economists were involved initially (Goldberger, 1972)
Then RDD went into hibernation It recently experienced a renaissance among
economists (e.g. Hahn, Todd and van der Klaauw, 2001; Jacob and Lefgren, 2002)
Tom Cook has written about this story
![Page 4: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/4.jpg)
RDD Logic
Selection on an observable (a rating)
A tie-breaking experiment
Modeling close to the cut-point
Modeling the full distribution of ratings
![Page 5: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/5.jpg)
![Page 6: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/6.jpg)
![Page 7: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/7.jpg)
![Page 8: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/8.jpg)
![Page 9: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/9.jpg)
![Page 10: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/10.jpg)
![Page 11: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/11.jpg)
Many different rules work like this.Examples:
Whether you pass a test Whether you are eligible for a program Who wins an election Which school district you reside in Whether some punishment strategy is enacted Birth date for entering kindergarten
This last one should look pretty familiar-Angrist and Krueger’s quarter of birth was essentially a regression discontinuity idea
![Page 12: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/12.jpg)
The key insight is that right around the cutoff we can think of people slightly above as identical to people slightly belowFormally we can write it the model as:
if
is continuous then the model is identified (actually all you really need is that it is continuous at x = x*)
![Page 13: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/13.jpg)
To see it is identified not that
Thus
That it
![Page 14: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/14.jpg)
There is nothing special about the fact that Ti was binary as long as there is a jump in the value of Ti at x*
This is what is referred to as a “Sharp Regression Discontinuity”
There is also something called a “Fuzzy Regression Discontinuity”
This occurs when rules are not strictly enforced
![Page 15: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/15.jpg)
![Page 16: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/16.jpg)
![Page 17: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/17.jpg)
![Page 18: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/18.jpg)
![Page 19: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/19.jpg)
![Page 20: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/20.jpg)
The size of the discontinuity at the cutoff is the size of the effect.
![Page 21: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/21.jpg)
Conditions for Internal Validity The outcome-by-rating regression is a
continuous function (absent treatment). The cut-point is determined independently of
knowledge about ratings. Ratings are determined independently of
knowledge about the cut-point. The functional form of the outcome-by-rating
regression is specified properly.
![Page 22: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/22.jpg)
RDD Statistical Model
iiii eRTY 10 where:
Yi = outcome for subject i,
Ti = one for subjects in the treatment group
and zero otherwise,Ri = rating for subject i,ei = random error term for subject i,
which is independently and identically distributed
![Page 23: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/23.jpg)
Sample Size Implications Because of the substantial multi-collinearity that
exists between its rating variable and treatment indicator, an RDD requires 3 to 4 times as many sample members as a corresponding randomized experiment
![Page 24: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/24.jpg)
Specification Tests
Using the RDD to compare baseline characteristics of the treatment and comparison groups
Re-estimating impacts and sequentially deleting subjects with the highest and lowest ratings
Re-estimating impacts and adding: a treatment status/rating interaction a quadratic rating term interacting the quadratic with
treatment status Using non-parametric estimation
![Page 25: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/25.jpg)
Here we see a discontinuity between the regression lines at the cutoff, which would lead us to conclude that the treatment worked. But this conclusion would be wrong because we modeled these data with a linear model when the underlying relationship was nonlinear
![Page 26: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/26.jpg)
![Page 27: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/27.jpg)
Here we see a discontinuity that suggests a treatment effect. However, these data are again modeled incorrectly, with a linear model that contains no interaction terms, producing an artifactualdiscontinuity at the cutoff…
![Page 28: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/28.jpg)
![Page 29: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/29.jpg)
![Page 30: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/30.jpg)
![Page 31: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/31.jpg)
Example: State Pre-K
Pre-K available by birth date cutoff in 38 states, here scaled as 0 (zero)
5 chosen for study and summed here How does pre-K affect PPVT (vocabulary) and
print awareness (pre-reading)
![Page 32: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/32.jpg)
Correct specification of the regression line of assignment on outcome variable
![Page 33: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/33.jpg)
Best case scenario –regression line is linear and parallel (NJ Math)
![Page 34: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/34.jpg)
Sometimes, form is less clear
![Page 35: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/35.jpg)
![Page 36: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/36.jpg)
So, what to do?
![Page 37: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/37.jpg)
Graphical approaches
![Page 38: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/38.jpg)
![Page 39: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/39.jpg)
![Page 40: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/40.jpg)
Parametric approaches
Alternate specifications and samples Include interactions and higher order terms Linear, quadratic, & cubic models Look for statistical significance for higher order
terms When functional form is ambiguous, overfit the
model (Sween1971; Trochim1980) Truncate sample to observations closer to cutoff
Bias versus efficiency tradeoff
![Page 41: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/41.jpg)
Non-parametric approaches
Eliminates functional form assumptions Performs a series of regressions within an interval,
weighing observations closer to the boundary Use local linear regression because it performs
better at the boundaries What depends on selecting correct bandwidth? Key
tradeoff in NP estimates: bias vs precision–How do you select appropriate bandwidth?–Ocular/sensitivity tests
Cross-validation methods “Leave-one-out” method
![Page 42: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/42.jpg)
State-of-art is imperfect So we test for robustness and present multiple
estimates
![Page 43: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/43.jpg)
Example I
![Page 44: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/44.jpg)
![Page 45: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/45.jpg)
Example II
![Page 46: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/46.jpg)
![Page 47: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/47.jpg)
Do Better Schools Matter? Parental Valuation ofElementary Education
Sandra Black, QJE, 1999
In the Tiebout model parents can “buy” better schools for their children by living in a neighborhood with better public schools
How do we measure the willingness to pay?
Just looking in a cross section is difficult: Richer parents probably live in nicer houses in areas that are better for many reasons
![Page 48: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/48.jpg)
Black uses the school border as a regression discontinuity
We could take two families who live on opposite side of the same street, but are zoned to go to different schools
The difference in their house price gives the willingness to pay for school quality.
![Page 49: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/49.jpg)
![Page 50: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/50.jpg)
![Page 51: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/51.jpg)
![Page 52: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/52.jpg)
![Page 53: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/53.jpg)
Tie-breaker experiment?
![Page 54: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/54.jpg)
Show sample density at the cutoff
![Page 55: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/55.jpg)
Summary of To-Do List
Graphical analyses Alternative specification and sample choices in
parametric models Non-parametric estimates at the cutoff Present multiple estimates to check for
robustness Move to tie-breaker experiment around the cutoff Sample densely at the cutoff Use pretest measures
![Page 56: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/56.jpg)
Recommendations
Pray for parallel and linear relationships
![Page 57: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/57.jpg)
External Validity
Estimating impacts at the cut-point Extrapolating impacts beyond the cut-point
with a simple linear model Estimating varying impacts beyond the cut-
point with more complex functional forms
![Page 58: Introduction of Regression Discontinuity Design (RDD)](https://reader030.fdocuments.net/reader030/viewer/2022033022/56649cba5503460f94981e73/html5/thumbnails/58.jpg)
References Cook, T. D. (in press) “Waiting for Life to Arrive: A History of the
Regression-discontinuity Design in Psychology, Statistics and Economics” Journal of Econometrics.
Goldberger, A. S. (1972) “Selection Bias in Evaluating Treatment Effects: Some Formal Illustrations” (Discussion Paper 129-72, Madison WI: University of Wisconsin, Institute for Research on Poverty, June).
Hahn, H., P. Todd and W. van der Klaauw (2001) “Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design” Econometrica, 69(3): 201 – 209.
Jacob, B. and L. Lefgren (2004) “Remedial Education and Student Achievement: A Regression-Discontinuity Analysis” Review of Economics and Statistics, LXXXVI.1: 226 -244.
Thistlethwaite, D. L. and D. T. Campbell (1960) “Regression Discontinuity Analysis: An Alternative to the Ex Post Facto Experiment” Journal of Educational Psychology, 51(6): 309 – 317.
Trochim, W. M. K. (1984) Research Designs for Program Evaluation: The Regression-Discontinuity Approach (Newbury Park, CA: Sage Publications).