Regression Discontinuity

7/27/2019 Regression Discontinuity

1/60

EITM2011

Chris Berr


2/60

.

II. DiscussionofExemplaryPapers

III. Practitioner sGui e

IV. Stataexamples


3/60

RegressionDiscontinuity


4/60

Im r n R f r n RDmethodswereintroducedbyThistlewaite andCampbell

. .

Recentapplicationsinpoliticsincludeanalysesoftheincumbencyeffect(Lee,2008), electoralcompetitionon

, , ,onlegislatorbehavior(Rehavi nd),thevalueofaseatinthe

legisalture (EggersandHainmueller 2009),theeffectof

Recentimportanttheoreticalworkhasdealtwithidentificationissues(Hahn,Todd,andVanDer Klaauw,2001),

design(McCrary,2008),bandwidthselection(Imbens andKalyanaraman 2011).

,Klaauw(2008),andImbensandLemieux(2008). Todaysdiscussionborrowsfromallofthem


5/60

Re ressionDiscontinuit Basics ThebasicideabehindtheRDdesignisthatassignment

tothetreatmentisdetermined,eithercom letel or

partly,bythevalueofanassignment(orforcing)variable(thecovariateX)beingoneithersideofafixedthreshold. Assignmenttotreatmentbycovariatevalue,assignallunits

withXic totreatment

ofthetreatmentatX= c

RDislikearandomizedexperimentatthecutpoint X= c

TheRDdesignisgenerallyregardedashavingthegreatestinternalvalidityofallquasiexperimentalmethods.Itsexternalvalidityismorelimited,sincetheestimatedtreatmenteffectislocaltothediscontinuity.


6/60

RDScatterplot:PositiveTreatmentEffect

(Y)

Assignment Variable (T)Cutting Point


7/60

RDScatterplot:NoTreatmentEffect

(Y)



8/60

Startwiththeusualpotentialoutcomesframework

TheunitleveltreatmenteffectisYi(1) Yi(0),whereYi(1)istheoutcomethatwouldoccuriftheunitwereexposedtotreatment Y 0 if not. The familiar roblem is that we cannotobservethepairYi(0)andYi(1)simultaneously.

AssumeabinarytreatmentvariableTi andacontinuousi

Inasharpregressiondiscontinuitydesign:

=

Wherecisacutoffsuchthatallunitsabovereceivethetreatmentandallthosebelowdonot


9/60

Underfairlyweakassumptions,theeffectofthe

cXYEcXYE iiii |lim|lim00

Whichis: RD =E[Yi(1) Yi(0)|Xi=c].

cc iiii mm00

Thisistheaveragetreatmenteffectatthecutoffpoint,aparticularLATE.

andE[Y(0)|X]are(assumedtobe)continuousinx.

SomeextrapolationisrequiredbecausebydesigntherearenounitswithXi =cforwhomweobserveYi(0).


10/60

RDComparedtoOtherObservationalMethods

Remem ert etwoassumptionsnee e toi enti ytreatmente ectsfromobservationaldatausingregression/matching(Kosukes slide#31):overlap andunconfoundedness. .

Ingeneral,unconfoundedness isnotconsideredaparticularlycredibleassumption,andtheothermethodswerestudyingthisweekcanbethoughtofaswaysformakingitmoreplausible.

RDisspecialinthefollowingways: Unconfoundedness issatisfiedbydefinition.WhenXc,Tisalways1;when

X


11/60

RDComparedtoanExperiment RDisoftendescribedasaclosecousinofarandomizedexperimentorasa

localrandomizedexperiment.

Coughey andSekhon argueagainstthisconceptualization,forreasonswewillsee a er, u swor un ers an ngw y eana ogy sma e

Consideranexperimentinwhicheachparticipantisassignedarandomlygeneratednumber,v,fromauniformdistributionovertherange[0,1]. Unitswithv 0.5assi nedtotreatment unitswithv


12/60

ExperimentasRD

(Y)

Assignment Variable (T)

Cutting Point

0.50


13/60

EstimationBasics1

Wehavenowdefinedacausaleffectasthedifferenceoftwofunctionsatapoint.Howdowe

.

Approach#1:Comparemeans = , ,

arenounitsatthecutoffthatdontgetthetreatment,

butinprincipleitcanbeapproximatedarbitrarilywellby=

Thereforeweestimate:

cXYEcXYE ||

Thisisthedifferenceinmeansforthosejustaboveandbelowthecutoff.

Thisisanon arametrica roach.A reatvirtueisthatitdoesnotdependoncorrectspecificationoffunctional

forms.


14/60

NotethatIsaidinprinciplewecanestimatemeans

arbitraril closetothecutoff.In ractice,thisde endsonhavinglotsofdatawithin ofthecutoff.Supposeyoudont.

TheoriginalRDdesign(Thistlewaite andCampbell1960)was

implementedbyOLS.

where isthecausaleffectofinterestand isanerrorterm.

XTY

Thisregressiondistinguishesthenonlinearanddiscontinuousjumpfromthesmoothlinearfunction.

OLSwithonelinearterminXisseldomusedanymoreecauset e unctiona ormassumptionsareverystrong.

Whatarethey?


15/60

Supposetheunderlyingfunctionsarenonlinearandmaybeunknown.In

particular,supposeyouwanttoestimate

wheref(X)isasmoothnonlinearfunctionofX.

)(XfTY

.CommonpracticeistofitdifferentpolynomialfunctionsoneachsideofthecutoffbyincludinginteractionsbetweenTandX.

Modelingf(X)withapthorderpolynomialinthiswayleadsto

p

p

p

p

TXTXTXT

XXXY

...

...

221

02

0201

coefficientonTisthetreatmenteffect.

Commonpractice,forwhateverreason,seemstousea4th orderpolynomial,thoughyoushouldbesurethatyourresultsarerobusttootherspecificationsmoreont is e ow .


16/60

EstimationBasics4 OLSwithpolynomialsisaparticularlysimplewayofallowinga

flexiblefunctionalforminX.Adrawbackisthatitprovidesglobalestimatesoftheregressionfunctionthatusedatafarfromthe

cutoff. Theremanyareotherways,buttheRDsetupposesacoupleof

. Weareinterestedintheestimateofafunctionataboundarypoint.

(Forwhythisisaproblem,seeHTVorImbens andLemieux.)

Standardnon arametrickernelre ressiondoesnotworkwellhere

ThisleadstoApproach#3:LocalLinearRegression Insteadoflocallyfittingaconstantfunction(e.g.,themean),fitlinear

regressionstoobservationswithinsomebandwidthofthecutoff

Arectangularkernelseemstoworkbest(seeImbens andLemieux),butoptimalbandwidthselectionisanopenquestion

Aseriousdiscussionoflocallinearregressionisbeyondthescopeofthislecture.See forexam le FanandGi bels 1996

But,really,werejusttalkingaboutrunningregressionsondatanear

thecutoff.


17/60

RDPitfall:MistakingNonlinearity

forDiscontinuity

formarepotentiallymoresevereforRDthanforothermethodswearestudyingthisweek

Misspecificationofthefunctionalformmay

generateabiasinthetreatmenteffect emos commons u a o no s ype sw enanunaccountedfornonlinearityintheconditionalmeanfunctionismistakenfora

discontinuity Eachofthe3estimationmethodsdealswiththis

ssue na eren way


18/60

(Y)

False discontinuity



19/60

(Y)



20/60

LocalLinearRegression

(Y)



21/60

CompareMeans

(Y

)



22/60

CompareMeans:SmallerBandwidth

(Y

)



23/60

Manipulation Ifindividualshavecontrolovertheassignmentvariable,thenwe

shouldexpectthemtosortinto(outof)treatmentiftreatmentisdesirable(undesirable) Thinkofameanstestedincomesupportprogram,oranelection

Thosejustabovethethresholdwillbeamixtureofthosewhowouldpassedandthosewhobarelyfailedwithoutmanipulation.

I in ivi ua s aveprecisecontro overt eassignmentvaria e,wewouldexpectthedensityofXtobezerojustbelowthethresholdbutpositivejustabovethethreshold(assumingthetreatmentisdesirable . McCrary(2008)providesaformaltestformaniupulaiton ofthe

assignmentvariableinanRD.TheideaisthatthemarginaldensityofXshouldbecontinuouswithoutmanipulationandhencewelookfor

.

HowprecisemustthemanipulationmustbeinordertothreatentheRDdesign?SeeLeeandLemieux(2010).

Thismeansthatwhen ourunanRD oumustknowsomethin aboutthemechanismgeneratingtheassignmentvariableandhowsusceptibleitcouldbetomanipulation.


24/60

ExampleofManipulation

n ncomesupportprogram nw c t oseearn ngun er , qua y orsupport

SimulateddatafromMcCrary2008


25/60

Itisimpossibletotestthecontinuityassumptiondirectly,but

Namely,allobservedpredeterminedcharacteristicsshouldhaveidenticaldistributionsoneithersideofthecutoff,inthe, . ,

thereshouldbenodiscontinuitiesintheobservables.

Againthereisananalogytoanexperiment:wecannottest,testtheobservables.Rejectioncallstherandomizationintoquestion.

discontinuityinobservablecovariatesindicatesaviolationofthecontinuityassumption,notaviolationof

, .thisbelow)


26/60

In rinci le,covariatesarenotneededforidentificationinRD,buttheycanhelpreducesamplingvariabilityintheestimatorandimprove

Thisisastandardargumentwhichalsosupports

inclusionofcovariatesinanalysesofrandomizedtrials Addingcovariatesshouldnotaffectthepoint

estimateoftheeffect(verymuch). Ifitdoes,

Thewiderthebandwidththemoreimportantitmaybetoincludecovariates.


27/60

GraphicalinspectionisanintegralpartofanyRDanalysis.

3typesofgraphsshouldalwaysbeproduced,where

1:theoutcome

2:othercovariates : ens yo cases

1shouldshowadiscontinuity;2and3shouldshownodiscontinuity

Ifyoucan'tseethemainresultwithsuchasimplegraph,it'sprobablynotthere

,


28/60

BandwidthSelection

ForLocalLinearRegression Bandwidthselectionrepresentsthefamiliartradeoffbetweenbiasandprecision

Whenthelocalregressionfunctionismoreorlesslinear,thereisntmuchofatradeoffsobandwidthcanbelar er.

Therearetwogeneralmethodsforselectingbandwidth

Adhoc,orsubstantivelyderived(e.g.,electionsbetween4852%areclose) Datadriven

Optimalbandwidthmethods(Imbens andKalyanaraman)

Crossvalidationmethods(LudwigandMiller;Imbens andLemieux)

ForPolynomialRegression Choosingtheorderofthepolynomialisanalogtothechoiceofbandwidth

Twoapproaches UsetheAkaike informationcriterion(AIC)formodelselection:AIC=Nln(2)+2p,where 2 (shouldhavea

a s emeansquare erroro eregress onan p s enum ero mo e parame ers

Selectanaturalsetofbins(asyouwouldforanRDgraph)andaddbindummiestothemodelandtesttheirjointsignificance.Addhigherordertermstothepolynomialuntilthebindummiesarenolonger

jointlysignificant. Thisalsoturnsouttobeatestforthepresenceofdiscontinuitiesintheregressionfunctionatpointsotherthenthe

cutoff,whichyoullwanttodoanyway

Inbothcases Inpractice,youmaywanttofocusonresultsfortheoptimalbandwidth,butit'simportanttotestforlotsofdifferentbandwidths.Thinkoftheoptimalbandwidthonlyasastartingpoint.

Ifresultscriticallydependonaparticularbandwidth,theyarelesscredibleandchoiceof.

Inprinciple,theoptimalbandwidthfortestingdiscontinuitiesincovariatesmaynotbethesameastheoptimalbandwidthforthetreatment. Again,followthepracticeoftestingrobustnessto

variationsinbandwidth.


29/60

FuzzyRD Supposetheprobabilityoftreatmentchangesdiscontinuously

atathreshold,butnotfrom0to1.ThisisasituationforapplyingFRD. Notet att e uzziness inFRDcomes romt ec angein

probabilityoftreatment,notfuzzinessaboutthethreshold InsharpRDdesigns,thejumpintheoutcomeatthecutoffisthe

es ma eo ecausa mpac o e rea men . na es gn,thejumpintheoutcomeisdividedbythejumpinthe

probabilityoftreatmentatthecutofftoproducethelocalWald.(InSRD,thejumpisone,sothedivisionisinconsequential,butthisdemonstratestherelationshipbetweenSRDandFRD).

equivalent(andconceptuallysimilar)toIV(seeMostlyHarmlesssec.6.2)

treatmentbyassignmentvariableshouldshowadiscontinuous

probabilityatthethreshold


30/60



31/60

DoVotersAffectorElectPolicies?

byLee,Moretti,andButler(LMB)


32/60

Motivation:2fundamentallydifferentviewsoftheroleofelections

Convergence:Competitionforvotesdrivescandidatestoseekmiddlegroundpolicies,compromise(medianvotertheorem).

o ersa ec po cyc o ceso po c ans.

Divergence:Votersselectcandidates,whothenenacttheirownpreferredpolicies.Voterselectpolicies.

makecrediblepromisestoimplementpoliciesthatarenotattheirownblisspoint(crediblecommitmentsarefacilitatedbyrepeatinteractions)

ThegoalofthepaperistoexaminewhichphenomenonismoreempiricallyrelevantforUSpolitics,specificallyvotingintheHouse


33/60

Consider2parties,DandR

Rsblisspointis0,Dsblisspointisc(>0)

TheprobabilitythatDwinstheelectionisP

IfDwinselection,policyxisimplemented;ifRwins,yis

P*representstheunderlyingpopularityofpartyD,ortheprobabilitythatDwouldwinifx=candy=0.Anincreasein

*

Whendx*/dP*anddy*/dP*>0,wesaythatvotersaffectcandidatespolicychoices

*

Whendx*/dP*anddy*/dP*=0,wesaythatvotersmerelyelectpoliticianswithfixedpolicies.Thatis,anincreasein

* .


34/60

Therollcallvotingrecordoftherepresentativeinthe

RCt =(1 Dt)yt +Dtxt

t . ,

onlythewinningcandidatespolicyisobservable

Theex ressioncanbetransformedinto:

This simply parameterizes the derivatives from

prior slide as 0.

Italsoallowsanindependenteffectofparty,1.


35/60

EstimatingFrameworkContd

WecannotobserveP*soequation(2)cannot

ButsupposewecouldrandomizeDt.ThenDtwou e n epen en o t an t. en:

Everyt ngun er ne nre can eest mate

fromthedata


36/60


37/60

GraphicalEstimateofEquation4

20


38/60

5


39/60

0.50


40/60


41/60

atthediscontinuity

representativevotingrecords

Resu tsro usttoa ow ng orvar oussortso

districtheterogeneity

Results(smallaffectcomponent)stableovertime


42/60

RegressionDiscontinuityDesigns

ofProIncumbentBiasinCloseU.S.

HouseRaces

byCaughey andSekhon (CS)


43/60

Closeelectionsarenotlikeotherelections

Strategicpoliticalactorshavestrongincentivestotargettheir

resourceswheretheywillhavethegreatestmarginalimpact Thereisanincumbencyadvantageeveninveryclose

elections Theincumbentwinsdisproportionatelyandhasgreaterfinancial

resources Thisfinding,alongwithothercovariateimbalancesatthe

cutpoint,callsintoquestiontheLMBincumbencyadvantageresultsand,moregenerally,theassumptionthatou comes nc osee ec onsare asgoo asran om yassigned NotethatCScritiqueLee(2008),notLMB,buttheimplications

same


44/60

TheLee(&McCrary)TestsforManipulation

A graph like (A) led Lee, and separately McCrary, to conclude that there is no manipulation.

However, (B) and (C) begin to suggest another story. Remember, the concern is with theincumbent partys vote share, not the Democratic vote share.

Density of the Assignment Variable


45/60

DensityoftheAssignmentVariable

Key Takeaway: The candidate of the incumbent party is about three times more likely to winelection by half a percentage point or less than to lose by a similar margin. The density of thisvariable appears to diverge rather than converge in the neighborhood of the cut-point.


46/60

BasedoncorrectingsomeofLeesdataandaddingsome

newvariables,CSfindimbalanceatthecutoffinthe

following: Democraticmargininthepreviouselection

thepartiesrelativecampaignexpenditures

1stdimensionNOMINATEscoreofthecurrentincumbent

whethertheDemocratic(Republican)candidateisthecurrentincumbent

numberoftermstheDemocrat(Republican)hasservedintheU.S.HouseofRepresentatives

w e er e emocra epu can asmorepo caexperiencethantheRepublican(Democrat)

CongressionalQuarterlysOctoberpredictionofwhich

Covariate Imbalance Graph


47/60

CovariateImbalanceGraph

S iti it t B d idth S l ti


48/60

SensitivitytoBandwidthSelection

P t ti l M h i


49/60

PotentialMechanisms

Notlikelybeoutrightfraud,becausesignificanceoflaggedvoteshareisincreasingovertimeandwebelievepotential

Controloverrecountsdoesnotappeartobethekeybecausetheyrarelyhappenandevenmorerarelychange.

Butwedontneedanexplanationbasedonvotecounting.Differencesbetweenwinnersandlosersinincumbency,

, , resourcesareevidentfarbeforeanyvotesarecast,counted,ormanipulated.

closeexanteandinthosethatwereinfactdecidedbyanarrowmargin.

,expectations,andallelseshouldbebalancedintheclosest

elections.


50/60


51/60

Thisisacautionarytale

LMBareverygoodscholars.

Theydidalmosteverythingright Theydonotdoalottojustifyfunctionalformorshowrobustnesstodifferentbandwidths

Remember,theLMBresultsforDemocraticvote(eqn.5)arenotimplicatedinthiscritique.Thisisthebestpartofthepaperanyway,IMHO.

Whatcanyoulearnfromthisexchange: Trytofindproblemsinyourdesignbeforesomeoneelsedoesitforyou

Identifyandcollectaccuratedataontheobservablecovariatesmostlikelytoreveal .

yourdataset. Laggedvaluesofthetreatmentvariablearealwaysagoodidea.Inelections,thepartythat

currentlycontrolstheoffice.

Automatedbandwidthselectionalgorithmsdonotguaranteegoodresults.Theyareustastartingpoint.

ForRDpurposes,whatconstitutesacloseelectionappearstobecloserthanthe4852%bandwidthwidelyuseduptonow.CSgetmostoftheirresultsusing49.550.5%.

GiventhecurrentfetishwithRDinpoliticalscience,understandthatitisnotafactofnaturethatcloseelectionsarerandom.Rememberthiswhenyousee(orsetout

towrite)thenextRDpaperoncloseelections.


52/60



53/60

Thetreatmentisdeterminedatleastin artb theassignmentvariable

Thereisadiscontinuityintheleveloftreatmentatsomecutoffvalueoftheassignmentvariable(selectiononobservablesatthecutpoint)

Unitscannotpreciselymanipulatetheassignmentvariabletoinfluencewhethertheyreceivethe

Othervariablesthataffectthetreatmentdonot


54/60

The strength of the RD design is its internal validity,arguably the strongest of any quasi-experimental design

External validity may be limited Shar RD SRD rovides estimates for the sub o ulation

with X=c, that is those right at the cutoff of the assignmentvariable.

The discontinuit is a wei hted avera e treatment effectwhere weights are proportional to the ex ante likelihoodthat an individuals realization of X will be close to thethreshold.

Fuzzy RD (FRD) restricts the estimates further to compliersat the cutoff (more on this below)

(e.g., treatment homogeneity)


55/60

Therearethree eneralt esofthreatstoanRD

design

1. Othervariableschangediscontinuouslyatthe

Testforjumpsincovariates,includingpretreatment

valuesoftheoutcomeandthetreatment2. Therearediscontinuitiesatothervaluesofthe

assignmentvariable

. an pu a ono eass gnmen var a e Testforcontinuityinthedensityoftheassignment

variableatthecutoff

Steps for Sharp RD Analysis


56/60

StepsforSharpRDAnalysis

1. Graphthedatabycomputingtheaveragevalueoftheoutcomevariableoverasetofbins.

The bin width has to be lar e enou h to have a sufficient amount of recisionsothattheplotslookssmoothoneithersideofthecutoffvalue,butatthesametimesmallenoughtomakethejumparoundthecutoffvalueclear.

2. Estimatethetreatmenteffectbyrunninglinearregressionsonbothsidesof

Witharectangularkernel,thesearejuststandardregressionestimatedwithinabinofwidthhonbothsidesofthecutoffpoint.Notethat: Standarderrorscanbecomputedusingstandardleastsquaremethods(robuststandarderrors).The

o timalbandwidthcanbechosenusin crossvalidationorothermethods.3. Therobustnessoftheresultsshouldbeassessedbyemployingvarious

specificationtests. Lookingatpossiblejumpsinthevalueofothercovariatesatthecutoffpoint

variable Lookingwhethertheaverageoutcomeisdiscontinuousatothervaluesofthe

forcingvariable

v ou v u o w ,w w ou o ov maybeavailable.


57/60

1. Graphtheaverageoutcomesoverasetofbinsasinthe

caseofSRD,butalsographtheprobabilityoftreatment.

2. Estimatethetreatmenteffectusing2SLS,whichisnumericallyequivalenttocomputingtheratiointheestimateofthejump(atthecutoffpoint)intheoutcomevariableoverthejumpinthetreatmentvariable.

Standarderrorscanbecomputedusingtheusual(robust)stan ar errors

Theoptimalbandwidthcanagainbechosenusingoneofthemethodsdiscussedabove.

. ero us nesso eresu scan eassesse us ng evariousspecificationtestsmentionedinthecaseofSRDdesigns.

EvaluatinganRDPaper


58/60

g p

oss y our wn Doestheauthorshowconvincinglythat

Treatmentchan esdiscontinuousl atthecut oint

Outcomeschangediscontinuouslyatthecutpoint

Othercovariatesdonotchangediscontinuouslyatthecutpoint Pretreatmentoutcomesdonotchangeatthecutpoint

ere snoman pu a ono eass gnmen var a e unc ngnear ecutpoint)

Arethebasicresultsevidentfromasimplegraph?

Aretheresultsrobusttodifferentfunctionalformassumptionsabouttheassignmentvariable Forexample,parametricandnonparametricfits,differentbandwidths,etc.

Couldotherpossiblyunobservedtreatmentschangediscontinuouslyat

Forexample,18th

birthdaymarksadiscontinuouschangeineligibilitytovote,butalsoeligibilityfordraft,sentencingasanadult,andlotsofotherthings,whichmayormaynotberelevantdependingontheoutcomeinquestion

Arecasesnearthecutpoint differentfromcasesfarfromthecutpoint inotherways?Dothesedifferencesmakethemmoreorlessrelevantfroma

theoreticalorpolicyperspective?


59/60



60/60

EITM2011

Chris Berr

Regression Discontinuity

Documents

Transcript of Regression Discontinuity