ECO 7377 - faculty.smu.edufaculty.smu.edu/millimet/classes/eco7377/lectures/lectures.pdf · DL...

ECO 7377Microeconometrics

Daniel L. Millimet

Southern Methodist University

Spring 2020

DL Millimet (SMU) ECO 7377 Spring 2020 1 / 478

Introduction

Applied research in economics can be loosely classified into two types1 Predictive modeling (forecasting, associations)2 Causal estimation (cause-and-effect)

While the first is important and useful, the second is of primaryinterest (to me)


Causal analysis is needed to predict the impact of changingcircumstances or policies, or for the evaluation of existing policies orinterventions

I Predictive modeling addresses the following question: “If an agentarrives with attributes x , what is the minimum-MSE estimate ofhis/her y?”

F Answer: E[y |x ] = x β (assuming linearity)

I Causal estimation addresses the following question: “If an interventionexposes an agent to a treatment, ∂xj , what is the minimum-MSEestimate of his/her ∂y?”

F Answer: ∂ E[y |x ]/∂xj = β (assuming linearity)


NotesI In general, estimating a CEF is very different than estimating a partialderivative

I Origins of machine learning techniques lie with predictive modeling(with high dimensional data)

F See Mullainathan & Spiess (2017)

Prior to conducting, or when reviewing, empirical analysis, questionsthat need to be answered:

1 Is predictive modeling suffi cient or is causal estimation necessary?2 What is the causal relationship of interest? Is it economicallyinteresting?

3 What is the identification strategy? Is it reasonable?4 What is the method of statistical inference?


Several statistical issues are confronted when answering thesequestions in economic research:

Specification of the causal relationship of interest entails more thanjust defining x and y ... lots of parameters could be estimated

I Heterogenous vs. homogeneous effects

F To whom does it apply?F What question does it answer?

I Know what you are estimating

Statistical inference is often diffi cult and overlookedI Spherical vs. non-spherical errorsI Generated regressorsI Derivation/computation of estimated asymptotic variances ofestimators

I Finite sample properties of estimators


Identification of causal relationships frequently encountersI Selection issues

F Self-selection (endogeneity)F Sample selection (missing data, attrition)

I Measurement issues

F Classical vs. non-classical errorF Dependent vs. independent variableF Continuous vs. discrete variablesF Differential vs. non-differential

I Modeling issues

F Functional form (P, SNP, NP)F Role of space (spillovers, spatial correlation)F Structural vs. atheoretic (Keane 2010)


Outline

1 Inference: simulation methods2 Program Evaluation

1 Causation2 Randomization3 Selection on Observed Variables4 Selection on Unobserved Variables

3 Data Issues

1 Sample Selection2 Measurement Error

4 Spatial Models5 Effi ciency Models


Simulation-Based InferenceIntroduction

General structure of estimation

population ⇒ θ

↓random sample ⇒ θ

Problem: θ is an estimate; need to assess its dbn for proper inference

SolutionsI Asymptotic theoryI Simulation methods ⇒ bootstrap, jacknife, sub-sampling, ...

Stata: -bootstrap-, -bsample-


Bootstrap setup1 yi , xiNi=1 is a random sample from a population distribution, F2 θ is a statistic computed from the sample3 F ∗ is the empirical distribution of the data (the so-called resamplingdistribution)

4 y∗i , x∗i Ni=1 is a resample of the data with replacement of the samesize as the original sample

5 θ∗is the statistic computed from the resample


The bootstrap principle statesI F ∗ ≈ FI The variation in θ is wellapproximated by the variationin θ

∗

In practice, processI Results in a vector ofestimates, θ

∗b , b = 1, ...,B,

where B is the # of bootstraprepetitions

I Use this vector of estimates toconduct inference

population ⇒ θ

↓random sample ⇒ θ

↓bootstrap sample ⇒ θ

∗


Many different bootstrapmethods

I Parametric vs. nonparametricI Resampling algorithms

F iidF Wild bootstrapF Block/clusterF Subsampling (M/N)

I Estimand

F ParameterF Test statistic (notdiscussed here)


Simulation-Based InferenceImplementation

To fix ideas, consider the following estimation problemI Regression model

yi = xi β+ εi

I Problem: given sample estimates, β, need to obtain std errors orconfidence intervals


There are two common sampling methods

1 Resampling the data2 Resampling the errors

Resampling dataI Resample (with replacement) observations (yi , xi ) ⇒ y∗i , x∗i Ni=1I Estimate the original model (OLS) on the re-sampled data set ⇒ β

∗

I Repeat B times ⇒ β∗b , b = 1, ...,B


Resampling residualsI Given β from OLS on original sample, obtain residuals ⇒ εi ,i = 1, ...,N

I Resample (with replacement) a vector of N residuals ⇒ ε∗i , i = 1, ...,N

F This represents a random draw from the (nonparametric) empirical dbnof the residuals

I Generate y∗i = xi β+ ε∗i (which imposes β = β)I Regress y∗ on x by OLS ⇒ β

∗

I Repeat B times ⇒ β∗b , b = 1, ...,B

I Alternative (parametric) approach replaces step 2 above with thefollowing:

F Estimate

σ2 =1

N −K ∑i ε2i

F Draw N random numbers, ε∗i , i = 1, ...,N , from N(0, σ2)


Notes on resampling

Resampling data is typically preferred since it less model dependent

Whether resampling the data or the residuals, previous discussionassumes iid data since re-sampling occurs without regard to anydependence across observations

If there exists some sort of dependence in the data, then resampleblocks or clusters of data

Example #1: Time series data with serial correlationI Model

yt = xtβ+ εt , t = 1, ...,T

I Resample blocks of length l by drawing obs randomly fromt = 1, ...,T − l

I If obs t ′ is chosen for the bootstrap sample, also include obst = t ′ + 1, ..., t ′ + (l − 1)

I Draw T/l obs so final bootstrap sample size remains T


Example #2: Panel dataI For example, individuals within hhs, or employees within firms, orindividuals over time

I Modelyit = xitβ+ εit , i = 1, ...,N

I Generate bootstrap samples by resampling (with replacement)observations i

I If i is chosen for the bootstrap sample, include i for all t

Key: Blocks/clusters are chosen such that data are iid across blocks


What to do with β∗b , b = 1, ...,B? Several options ...

Obtain std error for original sample estimate, β, given by

se(β) =

√1

B − 1 ∑b

(β∗b − β

∗)Obtain symmetric CI using normal approximation

β ∈

β± t1− α2 ,B−1se(β)

Obtain asymmetric CI using percentile method

β ∈

β α2, β1− α

2

where subscript refers to the quantile of the empirical dbn of β

∗

Obtain symmetric CI using the centered percentile method

β ∈2β− β1− α

2, 2β− β α

2


Obtain asymmetric bias corrected and accelerated CIs (BCa)

I CI given by β ∈

βp1, βp2

for appropriately chosen p1, p2 quantiles

I Quantiles given by

p1 = Φ

[z0 +

z0 − z1− α2

1− a(z0 − z1− α2)

]; p2 = Φ

[z0 +

z0 + z1− α2

1− a(z0 + z1− α2)

]where z1− α

2is the (1− α/2)th quantile of the std normal distribution

and

z0 = Φ−1[1B ∑b I

(β∗b 6 β

)](median bias)

a =∑i

(βJ − β

J(i )

)36

[∑i

(βJ − β

J(i )

)2]3/2 (acceleration parameter)

where βJ(i ) is the jacknife estimate (omitting obs i from original

sample) and βJis the mean of the jacknife estimates


Notes:I BC CI obtained by setting a = 0I BCa requires B > 1000I z0 = 0 when β = median of β

∗

I a reflects the rate of change of the standard error of β with respect tothe true value, β

F The standard normal approximation assumes that the standard error isinvariant with respect to the true value

F The acceleration parameter corrects for deviations in practice


Example: x ∼N(0, 1), N = 1000, xa∼N(0, 0.001)

05

1015

20

.2 0 .2

Bootstrap Asymptotic

Reps = 50

05

1015

.2 0 .2


Reps = 500

05

1015

.2 0 .2


Reps = 1000

05

1015

.2 0 .2


Reps = 10000


Simulation-Based InferenceAlternative Resampling Methods

Jacknife estimation

Also known as leave-one-out estimation

AlgorithmI Estimate model using original sample ⇒ β (if OLS model, say)I Omit obs i and re-estimate model on sample of N − 1 obs ⇒ β(i )I Repeat omitting each i once (implies N estimations)I Standard error obtained as

se(β) =

√N − 1N ∑i

(β(i ) − β(i )

)2In some situations, delete-d jacknife achieves superior performance


Subsampling

AlgorithmI Estimate model using original sample ⇒ β (if OLS model, say)I Draw samples of size M << N without replacement

I Repeat B ≤(NM

)times ⇒ β

∗b , b = 1, ...,B

I Rule-of-thumb perhaps is M = N/4


Comparison of bootstrap and subsamplingI The dbn of the data in the population is given by FI The dbn of the data in the sample is given by F ∗I β is an estimate obtained from a random sample of size N from FI Objective: understand the dbn of β if one repeatedly samples data ofsize N from F

F This is not possibleF Bootstrap: repeatedly sample data of size N from F ∗F Subsampling: repeatedly sample data of size M from F (since randomsubsamples of the data are random samples from F )

I Thus, neither bootstrap or sub-sampling are exactly right

F Bootstrap generates resampled data of the right sample size, but fromthe wrong dbn (F ∗ vs. F )

F Subsampling generates resampled data of the wrong sample size(M << N), but from the right dbn

I In some cases, subsampling performs better


Simulation-Based InferenceFailure of the Bootstrap

Resampling methods are not guaranteed to work; theoreticaljustification is needed

Most common failures occur1 when parameter of interest is a non-smooth function of the data (e.g.,median vs. mean)

2 when parameter of interest lies on the edge of the parameter space(e.g., probability close to one or variance close to zero)


Example: x ∼N(0, 1), N = 1000, xmeda∼N(0, 0.00157)

05

1015

.2 0 .2


Reps = 50

05

1015

.2 0 .2


Reps = 500

05

1015

.2 0 .2


Reps = 1000

05

1015

.2 0 .2


Reps = 10000


CausationIntroduction

Many empirical questions in economics, business, medicine, etc.pertain to the causal effect of a program or policy or treatment

I Correlation, in contrast, is (typically) less interesting and informative

Statistical and econometric literature analyzing causation has seentremendous growth over the past several decades

Central problem concerns evaluation of the causal effect of exposureto a treatment or program by a set of units on some outcome

I These units are agents such as individuals, households, firms,geographical areas, etc.


Philosophy of causality...I Rich literature in analytic philosophy on causalityI Two main approaches to defining causality:

F Regularity approaches: Hume: “We may define a cause to be anobject followed by another, and where all the objects, similar to thefirst, are followed by objects similar to the second.” (from An EnquiryConcerning Human Understanding, section VII)

F Counterfactual approaches:—Hume: “Or, in other words, where, if the first object had not been,the second never had existed.” (from An Enquiry Concerning HumanUnderstanding, section VII)— JS Mill: “Thus, if a person eats of a particular dish, and dies inconsequence, that is, would not have died if he had not eaten of it,people would be apt to say that eating of that dish was the cause of hisdeath.”


Regularity approach: a minimal constant conjunction between thetwo objects

I Basic idea behind predictive modelingI May be spurious if there exists some factor B to explain the conjuction(or correlation) between the two objects

F B is known as a confounder or confounding variable

I Be wary: correlation does not imply causation


Counterfactual approach: differences across a range of possibleworlds

I Holland (1986, 2003): a treatment (cause) is a potential manipulationthat one can imagine

F “NO CAUSATION WITHOUT MANIPULATION”F Gender, race are not treatments?!? (see Greiner and Rubin 2011)

I Imbens and Wooldridge (2009):

F “A CRITICAL FEATURE IS THAT, IN PRINCIPLE, EACH UNIT CANBE EXPOSED TO MULTIPLE LEVELS OF THE TREATMENT.”

I Angrist and Pischke (2009): a treatment should be manipulatableconditional on other factors

F “NO FUNDAMENTALLY UNIDENTIFIED QUESTIONS”

I Microeconometrics today emphasizes the counterfactual approach


CausationRubin Causal Model

The dominant view of causation (in microeconomics) is based on thecounterfactual approach

Greiner & Rubin (2011):“For analysts from a variety of fields,the intensely practical goal of causalinference is to discover what wouldhappen if we changed the world insome way.”


Typically referred to as the Rubin Causal Model (Neyman 1923; Roy1951; Rubin 1974)Crucial underpinning is the notion of potential outcomes

Potential outcomes refer to the outcome that would be realized underdifferent states of nature

I Example: A sick individual may receive either Treatment 0 or 1. Theoutcome is either Recovery or Death. Thus, there are two possiblestates of nature (Treatment 0 or 1) and there is an outcome thatwould be realized in each state of nature.

Under the counterfactual approach, the causal effect of Treatment 1relative to Treatment 0 would be the difference in outcomes acrossthese two states of nature for a given individualDL Millimet (SMU) ECO 7377 Spring 2020 32 / 478

The counterfactual approach immediately leads to three salient points

1 Causal impacts of a treatment/intervention/program/policy are onlydefined with respect to a well-defined alternative

I Typically the alternative is the ‘absence of treatment’I Not always obvious and must be made explicit

2 Causal impacts are individual-specificI Each individual potentially has his or her own potential outcomes andhence treatment effect

I Referred to as constant vs. heterogeneous treatment effectsI Has important implications for interpreting the results

3 Only one state of nature is actually realized at a point in timeI We can observe at most one potential outcome for any individual,remainder are missing

I The causal effect of a treatment is not observable for any individualI Any estimator of causal effects must overcome this missing dataproblem

I To do so requires assumptions and the assumptions must be credible


Overcoming the missing data problem arising from the fact that onlyone state of nature is realized is referred to as the fundamentalproblem of causal inference (Holland 1986) and is diffi cult toovercome


Notation

Treatment AssignmentI Di = (binary) treatment indicator for observation i

Di =1 treated0 untreated

Potential OutcomesI Yi (1) = outcome of observation i with treatmentI Yi (0) = outcome of observation i without treatment


Implicit in this representation is the Stable Unit Treatment ValueAssumption (SUTVA, Rubin 1978)

1 SUTVA implies that potential outcomes of observation i areindependent of the treatment assignment of all other agents (rulesout general equilibrium or indirect effects via spillovers)

I Allows one to write potential outcomes solely as a function of owntreatment assignment

Yi (0) ≡ yi (D1,D2, ...,Di−1, 0,Di+1, ...,DN ) = yi (0)

Yi (1) ≡ yi (D1,D2, ...,Di−1, 1,Di+1, ...,DN ) = yi (1)

I See Imbens & Wooldridge (2009) for references that relax thisassumption

F Stata: -ntreatreg-

2 SUTVA implies that the treatment, D, is identical for all observations(i.e., no variation in treatment intensity)


Parameters of Interest

∆i = Yi (1)− Yi (0) = treatment effect for obs iI Either Yi (1) or Yi (0) is observed for each i , never both

F As a result, ∆i is never observed

I Missing potential outcome is the missing counterfactual

Can summarize the distribution of ∆i by focusing on different aspects

∆ATE = E[∆i ] = E[Y (1)− Y (0)]∆ATT = E[∆i |D = 1] = E[Y (1)− Y (0)|D = 1]∆ATU = E[∆i |D = 0] = E[Y (1)− Y (0)|D = 0]


Other Parameters of Interest

1. Local Average Treatment Effect (Imbens & Angrist 1994; Angrist etal. 1996)

I Defined as∆LATE = E[Y (1)− Y (0)|i ∈ Ω]

where Ω refers to some specified local or subpopulation

2. Marginal Treatment Effect (Heckman & Vytlacil 1999, 2001, 2005,2007)

I Defined later


3. Intent to Treat EffectI Defined as

∆ITT = E[Y (1)− Y (0)]where

D =1 if agents have the opportunity to undertake the treatment0 if agents do not have the opportunity to undertake the treatment


4. Average Direct, Indirect, and Total EffectsI Define potential outcomes as

Y (d) = Y (d ,M(d)), d = 0, 1

where M is referred to as a mediator as D affects M and M affectspotential outcomes

F Example: D is college completion, M is occupation, and Y is wages

I Parameters of potential interest defined as

Causal Mediation Effect (Indirect Effect): E[Y (d ,M(1))− Y (d ,M(0))]Direct Effect: E[Y (1,M(d))− Y (0,M(d))]Total Effect: E[Y (1,M(1))− Y (0,M(0))]

I See Robins and Greenland (1992), Pearl (2001), Imai & Yamamoto(2013), Acharya et al. (2016)


Relationship among the ATE, ATT, and ATUI Let

Yi (1) ≡ E[Y (1)] + υ1i

Yi (0) ≡ E[Y (0)] + υ0i

I This implies

∆i = Yi (1)− Yi (0)= E[Y (1)− Y (0)] + υ1i − υ0i

= ∆ATE + υ1i − υ0i

and

∆ATT = ∆ATE + E[υ1i − υ0i |D = 1]∆ATU = ∆ATE + E[υ1i − υ0i |D = 0]

where E[υ1i − υ0i |D = j ] is the average, obs-specific gain fromtreatment for group j


Can re-define any of the above parameters for sub-population definedon the basis of attributes, x

∆ATE (x) = E[Y (1)− Y (0)|x ]∆ATT (x) = E[Y (1)|x ,D = 1]− E[Y (0)|x ,D = 1]∆ATU (x) = E[Y (1)|x ,D = 0]− E[Y (0)|x ,D = 0]

where these are conditional average treatment effects

The previous unconditional parameters are obtained by integratingover the dbn of x in the relevant population

∆ATE =∫

∆ATE (x)f (x)dx

∆ATT =∫

∆ATT (x)f (x |D = 1)dx

∆ATU =∫

∆ATU (x)f (x |D = 0)dx


Evaluation Problem

Setup...

Attributes of i Observed for iYi (0),Yi (1),Di , xi yi ,Di , xi

whereI yi = DiYi (1) + (1−Di )Yi (0) = observed outcome for obs iI xi is a vector of attributes of obs i

Question: How does one circumvent the missing counterfactualproblem to estimate ∆ATE , ∆ATT , ∆ATU , or any other summarystatistic of the distribution of ∆?Any answer must address possible bias from self-selection intotreatment and control groups


Example #1... ATTI Consider ∆ATT = E[Y (1)|D = 1]− E[Y (0)|D = 1]I The sample counterpart to E[Y (1)|D = 1] is observed in the data, butone does not observe the counterpart to E[Y (0)|D = 1]

I If one uses outcomes of the untreated, we can define

∆ ≡ E[Y (1)|D = 1]− E[Y (0)|D = 0]

I Some algebra reveals

∆ATT = E[Y (1)|D = 1]− E[Y (0)|D = 1]= E[Y (1)|D = 1]− E[Y (0)|D = 0]

+ E[Y (0)|D = 0]− E[Y (0)|D = 1]⇒ ∆− ∆ATT = E[Y (0)|D = 1]− E[Y (0)|D = 0]︸︷︷︸

selection bias

I Generally, ∆ 6= ∆ATT


Example #2... ATEI Consider estimating ∆ATE = E[Y (1)]− E[Y (0)]I The sample counterpart of neither unconditional expectation isobserved in the data

I If one uses conditional expectations, we can again define

∆ ≡ E[Y (1)|D = 1]− E[Y (0)|D = 0]

I Some algebra reveals

∆− ∆ATE = (E[Y (1)|D = 1]− E[Y (0)|D = 0])− (E[Y (1)]− E[Y (0)])= (E[Y (1)|D = 1]− E[Y (1)|D = 0])[1− Pr(D = 1)]

+ (E[Y (0)|D = 1]− E[Y (0)|D = 0])Pr(D = 1)=

(∆− ∆ATT

)Pr(D = 1)

+(

∆− ∆ATU)[1− Pr(D = 1)]

which is a weighted average of the selection bias for the ATT and ATU


Decomposition of the selection biasesI Biases given by

E[Y (0)|D = 1]− E[Y (0)|D = 0], andE[Y (1)|D = 1]− E[Y (1)|D = 0]

I Terms are decomposed into 3 or 4 components in

F Heckman et al. (1998)F King & Zeng (2006)


Question: How does one circumvent the missing counterfactualproblem to estimate ∆ATE , ∆ATT , ∆ATU , or any other summarystatistic of the distribution of ∆?Answer: Typically by estimating the missing counterfactual, but suchestimates are only as valid as the assumptions that underlie them andthe data used to derive the estimates

I The central issue in the RCM is the relationship between treatmentassignment and potential outcomes

F Typically referred to as the treatment assignment rule or mechanismF Growing literature on assignment rules (Manski 2000, 2004; Pepper2002, 2003; Dehejia 2005; Lechner & Smith 2007; Kitagawa & Tetenov2018)

I Estimation proceeds under different assumptions concerning theassignment of or selection into treatment

I Three different categories of assumptions

1 Random assignment2 Selection on observables (observed variables)3 Selection on unobservables (unobserved variables)


Early Example of Potential Outcomes: Roy Model (Roy 1951)

At the center of the RCM is the interplay between treatmentassignment, potential outcomes, and observed outcomes

Problem is one of self-selection; highlighted in a very clever fashion inRoy (1951)

Specific issue in Roy (1951) was occupational choiceI Individuals have potential earnings associated with differentoccupations

I Realized earnings reflect the chosen occuption

Example

Suppose (Y (0)Y (1)

)∼N

(01,∑)


Unconditional outcome distributions look like

0.1

.2.3

.4

6 4 2 0 2 4

Y(0) Y(1)N=100,000; rho = 0.7

Unconditional Distributions of Potential Outcomes


Conditional distributions

Depends onI Who selects into treatment or control group, andI Correlation of potential outcomes

Positive correlation in above example (ρ ≈ 0.7)


Positive selection: Assume those with Y (1) > 1 select into treatment

0.2

.4.6

.8

6 4 2 0 2 4

Y(0) Y(1)N=100,000; rho = 0.7

Conditional Distributions of Potential Outcomes


Negative selection: Assume those with Y (0) < 0 select into treatment

0.2

.4.6

.8

4 2 0 2 4

Y(0) Y(1)N=100,000; rho = 0.7



Random assignment:

0.1

.2.3

.4

6 4 2 0 2 4

Y(0) Y(1)N=100,000; rho = 0.7


Lesson to be learned: observed distributions are not the unconditionaldistributions; hence, difference in observed dbns tells us nothingabout treatment effects in the absence of further assumptionsDL Millimet (SMU) ECO 7377 Spring 2020 53 / 478

Roy Model

Two occupations: hunter, fisherPotential incomes

Y (d) = µd (x) + υd , d = 0 (h), 1 (f)

Decision rule: maximize income

D = I(Y (1)− Y (0) > 0)= I(µ1(x)− µ0(x) + υ1 − υ0 > 0)

Observed income

y = DY (1) + (1−D)Y (0)

Treatment assignment depends on observables, x , and unobservables,υ1 − υ0Notes:

1 Cov(D, υ1 − υ0) 6= 0 referred to as essential heterogeneity (Heckmanet al. 2006)

2 Cov(D, υ1 − υ0) 6= 0⇒ Cov(D,D(υ1 − υ0)) 6= 0DL Millimet (SMU) ECO 7377 Spring 2020 54 / 478

Generalized Roy Model

Replace income maximization decision rule with a more general rule

Decision rule

Y (d) = µd (x) + υd , d = 0, 1

D = I(h(z)− u > 0)

When D is a voluntary program (e.g., job training), u may reflect (i)costs of participation and (ii) foregone earnings (opportunity costs)

Implies that treatment assignment depends on observables, z , andunobservables, u

I Random Assignment: x ∩ z = ∅ and Corr(u, υd ) = 0 ∀dI Selection on Observables: x ∩ z 6= ∅ and Corr(u, υd ) = 0 ∀dI Selection on Unobservables: x ∩ z 6= ∅ and Corr(u, υd ) 6= 0 ∀d


Directed Acyclic Graphs (DAGs)

Random Assignment

D Y

Xv

Z =coin toss

u =empty


Selection on Observables

D Y

Xv

Z =observables

u =unobservables


Selection on Unobservables

D Y

Xv

Z =observables

u =unobservables


Moving Forward

Guided by the potential outcomes framework, figure out conditionsunder which different estimators may provide consistent estimates ofthe ATE, ATT, ATU, etc.

Key points:I Given the missing counterfactual problem, any estimator of the causaleffects of a treatment must rely on some assumptions

F Thus, no estimator is guaranteed to ‘always’work or (perhaps) ‘always’fail

F Performance of every estimator is application-specific

I Different estimators rely on different assumptions and thus should notbe expected to yield similar estimates

I Not all assumptions can be testedI Different estimators may estimate different aspects of the dbn of ∆ andthus answer different questions


Random Assignment

First solution is to randomize treatment assignment via RandomizedControl Trials (RCTs)

I In Generalized Roy Model, z could reflect the outcome of a coin tossand u = 0 ∀i

D Y

Xv

Z =coin toss

u =empty


Generally speaking, randomization is the preferred solution; oftencalled the “gold standard”


Reason: randomization ensures that treatment assignment isindependent of potential outcomes in expectation

Freedman (2006): “Experiments offer more reliable evidence on causation thanobservational studies.”

Imbens (2009): “More generally, and this is the key point, in a situation where onehas control over the assignment mechanism, there is little to gain, and much to lose, by

giving that up through allowing individuals to choose their own treatment regime.

Randomization ensures exogeneity of key variables, where in a corresponding

observational study one would have to worry about their endogeneity.”


That said, not everyone is convinced by experiments (without doingsome more mental work)

“Much of the criticism about experiments is about the diffi culty ofgeneralizing fom the evaluation of one particular program to predictingwhat would happen to this program in a different context. Clearly,without theory to guide us on why a result extends from a context toanother, it is diffi cult to jump directly to a policy conclusion. However,when experiments are motivated by a theory, the results of experiments(not only on the final outcomes, but on the entire chain of intermediateoutcomes that led to the endpoint of interest) serve as a test of someof the implications of that theory. The combination of data points theneventually provides suffi cient evidence to make policyrecommendations.”

Duflo (2010),http://www.aeaweb.org/econwhitepapers/white_papers/Esther_Duflo.pdf


“From an ex post evaluation standpoint, a carefully plannedexperiment using random assignment of program status represents theideal scenario, delivering highly credible causal inference. But from anex ante evaluation standpoint, the causal inferences from a randomizedexperiment may be a poor forecast of what were to happen if theprogram were to be ‘scaled up’.”

DiNardo & Lee (2011)

Ex post evaluation answers the question: “What happened?”(descriptive)

Ex ante evaluation answers the question: “What would happen?”(predictive)


At issue is the distinction between internal and external validityI Proper RCTs yield high internal validity, but external validity is notguaranteed

I See recent work on external validity and problems of scale-up (e.g.,Kline & Tamer 2018; Bisbee et al. 2017; Andrews & Oster 2017 NBERWP; Davis et al. 2017 NBER WP)

I Stata: -extbounds-


Randomization may occur at different stages1 Population-level: randomize among agents in the population; typicallynot feasible since it would entail ‘compelling’treatment by some

2 Eligibility-level: randomize among the population of eligibles byrandomly denying eligibility to a subset

3 Application-level: randomize among the population of programapplicants by randomly accepting/rejecting a subset

NotesI Stage at which randomization occurs generally affects what can belearned unless additional assumptions are made

I Lab experiments are generally type 3 since randomization occurs withinthe subpopulation of experiment applicants


Assumptions (with population-level randomization)

(A.i) y ,D is iid sample from the population(A.ii) Y (0),Y (1) ⊥ D(A.iii) Pr(D = 1) ∈ (0, 1)

NotesI (A.i) implies SUTVAI (A.ii) implies E[Y (1)|D = 1] = E[Y (1)|D = 0] = E[Y (1)]; similarlyfor E[Y (0)]

I (A.ii) also implies ∆ATE = ∆ATT = ∆ATU since

E[Y (1)− Y (0)]︸︷︷︸ATE

= E[Y (1)− Y (0)|D = 1]︸︷︷︸ATT

= E[Y (1)− Y (0)|D = 0]︸︷︷︸ATU

I (A.ii) relies on perfect complianceI (A.iii) ensures all agents have some probability of receiving and notreceiving the treatment


Imperfect compliance may invalidate (A.ii) if such non-compliance isrelated to potential outcomes

Two options in this case:1 Difference in average outcomes based on initial assignment estimatesthe intent to treat effect, ∆ITT , under imperfect compliance; mayactually be more policy relevant

2 Use initial assignment as an instrument for actual assignment(discussed later)


PropertiesI UnbiasedI ConsistentI Asymptotically normalI Nonparametrically identified: no parametric or functional formassumptions needed


Notes

Randomization succeeds by balancing (in expectation) both observedand unobserved attributes of participants in the treatment andcontrol group

Balance can be assessed by testing for differences in the joint dbn ofpredetermined attributes across the treatment and control groups


Randomization at the eligibility or application stage only yield anestimate of the ATT, which does not equal the ATE unless (i)treatment effects are homogeneous or (ii) agents do not becomeeligible or apply due to unobserved, observation-specific gains to thetreatment, υ1 − υ0

I Implies level of randomization is important for interpreting resultsF Example: Project on Incentives in Teaching (POINT) experiment

“The Project on Incentives in Teaching (POINT) was a three-year study conducted in the Metropolitan

Nashville School System from 2006-07 through 2008-09, in which middle school mathematics teachers

voluntarily participated in a controlled experiment to assess the effect of financial rewards for teachers whose

students showed unusually large gains on standardized tests. The experiment was intended to test the notion

that rewarding teachers for improved scores would cause scores to rise. It was up to participating teachers to

decide what, if anything, they needed to do to raise student performance... Thus, POINT was focused on the

notion that a significant problem in American education is the absence of appropriate incentives... By and large,

results did not confirm this hypothesis. [S]tudents of teachers randomly assigned to the treatment group

(eligible for bonuses) did not outperform students whose teachers were assigned to the control group (not

eligible for bonuses).”

https://my.vanderbilt.edu/performanceincentives/files/2012/09/Full-Report-Teacher-Pay-for-Performance-

Experimental-Evidence-from-the-Project-on-Incentives-in-Teaching-20104.pdfDL Millimet (SMU) ECO 7377 Spring 2020 72 / 478

Estimation via regression is also possibleI Specification

yi = DiYi (1) + (1−Di )Yi (0)= Yi (0) +Di [Yi (1)− Yi (1)]= Yi (0) + ∆iDi= E [Y (0)] + E [∆]Di + [(Yi (0)− E [Y (0)]) + (∆i − E [∆])= α+ βDi + εi

where β = ∆ATEI Augmenting regression model with covariates can improve precisionsince it reduces the variance of ε

yi = α+ βDi + xi δ+ εi


Example: Krueger (1999), “Experimental Estimates of EducationProduction Functions,”Quarterly Journal of Economics, 114, 497-532.

Student/Teacher Achievement Ratio (STAR) RCT in Tennessee in1980s

Students assigned to1 Small kindergarten class: 13-17 students2 Large kindergarten class + teacher’s aide: 22-25 students3 Large kindergarten class + no teacher’s aide: 22-25 students

Randomization occurred within schools



Randomization is often not feasible in economics

Applied economists typically must rely on observational (ornon-experimental) data

D Y

Xv

Z =observables

u =unobservables


Data structure is now given by ...

Attributes of i Observed for iYi (1),Yi (0),Di , xi , υ1i , υ0i yi ,Di , xi

where xi is the full vector of observable attributes of i



Assumptions

(A.i) iid sample: y ,D, x is iid sample from the population

(A.ii) Conditional independence (CIA) or unconfoundedness:Yi (0),Yi (1) ⊥ D |x

(A.iii) Common support (CS) or overlap: Pr(D = 1|x) ∈ (0, 1)Notes

(A.i) implies SUTVA

(A.ii) implies D is randomly assigned conditional on x

Pr(Di = 1|xi ,Yi (1),Yi (0)) = Pr(Di = 1|xi )

(A.iii) ensures one observes agents with a particular x in both thetreatment and control groups

(A.ii), (A.iii) ⇒ strong ignorability (Rosenbaum & Rubin 1983)


Comments on Strong Ignorability

Is a ‘strong’assumption (no pun intended)

x’s must be pre-determined (i.e., unaffected by treatment assignment)I If some x’s are affected by D or the anticipation of D, then inclusionwill mask (at least) some of the treatment effect (Lechner 2008)

There may not exist any vector x in a particular data set for aparticular treatment such that strong ignorability holds

I There is some tension between CIA and CS; CIA takes precedenceI Strong ignorability requires an instrument to exist, but it need not beobserved (or even known) such that D is random conditional on x

Imbens & Rubin (2015) argue that CIA is a reasonable approximationin many applications and alternative assumptions may be even lesscredible

I I do not totally agree, ...I but they are smarter than I am!


Nonparametric Identification

Estimation

∆ATE (x) = E[yi |xi = x ,D = 1]− E[yi |xi = x ,D = 0]

=∑Ni=1 yi I[xi = x ,Di = 1]

∑Ni=1 I[xi = x ,Di = 1]

− ∑Ni=1 yi I[xi = x ,Di = 0]

∑Ni=1 I[xi = x ,Di = 0]

p−→ E[yi |xi = x ,D = 1]− E[yi |xi = x ,D = 0]= E[Yi (1)|xi = x ,D = 1]− E[Yi (0)|xi = x ,D = 0]= E[Yi (1)|xi = x ]− E[Yi (0)|xi = x ]

and then

∆ATE = E[∆ATE (x)

]=∫

∆ATE (x)f (x)dx =1N ∑i ∆ATE (xi )

Similar story for other parameters, except final step uses f (x |D = 1)or f (x |D = 0)


Final Notes

If x is continuous and/or high dimensional, then this estimator cannotbe used since the probability of observing more than one obs with thesame value of x is zero

I Possible solutions:

1 Functional form assumptions ⇒ regression2 Dimensionality reduction ⇒ matching, weighting


CIA is not testableI One common ‘test’employed entails testing for differences inpre-treatment outcomes conditional on x between the to-be-treatedand the controls

I Intuition: if D is uncorrelated with unobservables related to theoutcome conditional on x , then pre-treatment outcomes should beunrelated to (future) D conditional on x

I Heckman et al. (1999) refers to this as the alignment fallacy

F Test based on outcomes more than one period in the past is misleadingif shocks are serially correlated and agents self-select into the treatmentgroup due to an adverse shock in the period directly before treatment

F In general, test is useful if it rejects the independence of D and yconditional on x in periods prior to treatment; if it fails to reject, thenthe test is ambiguous

Several simulation studies comparing the performance of variousestimators

I Frölich (2004), Busso et al. (2014), Frölich et al. (2017)


Assumptions

(A.iv) Separability:

Yi (0) = µ0(xi ) + υ0i

Yi (1) = µ1(xi ) + υ1i

where E[υ1 |x ] = E[υ0 |x ] = E[υ1 − υ0 |x ] = 0(A.v) Functional forms:

(A.va) Constant treatment effect

µ0(xi ) = α0 + xi β

µ1(xi ) = α1 + xi β

(A.vb) Heterogeneous treatment effects

µ0(xi ) = α0 + xi β0µ1(xi ) = α1 + xi β1


EstimationI Via OLS

yi ≡ Yi (0) +Di (Yi (1)− Yi (0))= α0 + xi β+ υ0i +Di (α1 + xi β+ υ1i − α0 − xi β− υ0i )

= α0 + xi β+ (α1 − α0)Di + [υ0i +Di (υ1i − υ0i )]

= α0 + xi β+ ∆ATEDi + υi

I Coeffi cient on D is an unbiased estimate of the causal parameter, and

∆ATE = ∆ATT = ∆ATU


EstimationI Via OLS

yi = α0 + xi β0 + (α1 − α0)Di + xiDi (β1 − β0)

+ [υ0i +Di (υ1i − υ0i )]

= α0 + xi β0 + α1Di + xiDi β1 + υi

I Estimates given by

∆ATE (x) = α1 + x β1∆ATE = α1 + x β1∆ATT = α1 + x1 β1∆ATU = α1 + x0 β1

where x j = ∑i xi I[Di = j ]/ ∑i I[Di = j ], j = 0, 1

Stata: -teffects, ra-


Example: Millimet (2000), “The Impact of Children on Wages, JobTenure, and the Division of Household Labor,”Economic Journal, 110,C139-C157.

N = 1485 married couples from 1976 PSID

VariablesI y = log(hourly wage) (husband)I D = union status (husband)I x = educ, race, age

Specifications

yi = α0 + xi β+ ∆Di + εi

yi = α0 + xi β+ ∆0Di + xiDi∆1 + υi


Selection on ObservablesStrong Ignorability: Matching (Intro)

PreliminariesI Matching methods were quite popular, but (perhaps) less so now

F Pearl (2010, p. 114): “The method of propensity score (Rosenbaumand Rubin 1983), or propensity score matching (PSM), is the mostdeveloped and popular strategy for causal analysis in observationalstudies...”

I (Incorrectly) viewed by many as a ‘magic bullet’and/or solution to‘endogeneity’

I In practice, only as good as the underlying assumptions

Assumptions required: (A.i), (A.ii), and (A.iii)

CS assumption (A.iii) is needed in place of functional formassumptions (A.iv and A.v) used in the regression approach


Selection on ObservablesStrong Ignorability: Matching vs. Regression

RegressionI Uses the full sample and gives equal weightto all controls

I Uses extrapolation based on assumedfunctional forms for the potential outcomesif the distribution of the x’s differs in thetreatment and control groups

F Extrapolation is particularly prominent ifthe x’s are unbalanced in the full sample

MatchingI Matching weights observations differently, giving more weight to thosedeemed most ‘similar’and potentially giving many controls zero weight

I No functional form assumptions

F Instead, matching requires the presence of reasonably close matches


Matching requires, and thus highlights problems due to, failure of CSwhereas regression extrapolates to overcome a failure of CS

12

34

5

.2 .4 .6 .8 1x

Untreated Units Untreated, Regression LineTreated Units Treated, Regression Line

E[y|x,D=0]=1+1x; E[y|x,D=1]=1.5+2.5x; sigma = 0.25

I CS is violated, but OLS simply extrapolates from each group toestimate the missing counterfactual at a particular value of x

I If linear regression specification is not globally accurate, thenregression may yield severe bias


Prior to implementing regression approach,it is useful to examine the standardizeddifferences in x across the treatment andcontrol groups

I Standardized difference for a particular x isgiven by

norm− diff = |x1 − x0 |√12

(σ2x1 + σ2x0

)I If norm− diff > 0.25, regression results aresensitive to functional form assumptions

I See Imbens & Wooldridge (2009)


Selection on ObservablesStrong Ignorability: Matching (Basics)

To proceed, recall our parameters of interest

∆ATE = E[Y (1)− Y (0)]∆ATT = E[Y (1)− Y (0)|D = 1]∆ATU = E[Y (1)− Y (0)|D = 0]

Sample counterparts

∆ATE =1N ∑i [Yi (1)− Yi (0)]

∆ATT =1N1

∑i [Yi (1)− Yi (0)] I[Di = 1]

∆ATU =1N0

∑i [Yi (1)− Yi (0)] I[Di = 0]

These are infeasible estimators since each is a function of missing data


Feasible estimators

∆ATT =1N1

∑i

[Yi (1)− Yi (0)

]I[Di = 1]

∆ATU =1N0

∑i

[Yi (1)− Yi (0)

]I[Di = 0]

∆ATE =N1N

∆ATT +N0N

∆ATU

whereI Yi (0), Yi (1) are estimates of the missing counterfactuals, obtained as

Yi (0) =1

∑j∈Dj=0

ωij∑

j∈Dj=0ωijYj (0)

Yi (1) =1

∑j∈Dj=1

ωij∑

j∈Dj=1ωijYj (1)

I ωij = weight given to observation j by observation iI and

Nd = ∑i I[Di = d ], d = 0, 1


NotesI Estimated missing counterfactual is a weighted average of outcomesfrom the control group

I All matching estimators take this formI Various matching estimators differ only in terms of how the weights arespecified


Example

ID D Y Y (0) Y (1) x1 0 10 10 ? 122 0 15 15 ? 163 1 14 ? 14 124 1 18 ? 18 16


Example (cont.)

ID D Y Y (0) Y (1) x1 0 10 10 14 122 0 15 15 18 163 1 14 10 14 124 1 18 15 18 16

⇒ ∆3 = 14− 10 = 4∆4 = 18− 15 = 3

∆ATT = 3.5

⇒ ∆1 = 14− 10 = 4∆2 = 18− 15 = 3

∆ATU = 3.5

⇒ ∆ATE = 0.5(3.5) + 0.5(3.5) = 3.5


Selection on ObservablesStrong Ignorability: Matching (Weighting Schemes)

Three primary classes of weighting schemes

1. Exact matchingI Positive weight to observations with identical x , zero otherwiseI Implies weights satisfy

ωij

> 0 if xj = xi= 0 if xj 6= xi

I With multiple matches, typically just use the average (i.e., ωij = 1/Ni ,where Ni is the number of matches for obs i)

I Estimator is subject to ‘curse of dimensionality’I R package: -MatchIt-I Stata: -kmatch-


2. Coarsened exact matchingI Intuition: ‘round’x to fewer distinct values, then match exactly on thecoarsened data

I Developed in Iacus et al. (2011)I Implies weights satisfy

ωij

> 0 if xj = xi= 0 if xj 6= xi

where x is a vector of coarsened attributesI With multiple matches, typically just use the average (i.e., ωij = 1/Ni ,where Ni is the number of matches for obs i)

I R package: -cem-I Stata: -cem-


3. Inexact matchingI Positive weight given to observations deemed to be suffi ciently ‘close’I If multiple matches are used, weights are increasing in ‘closeness’I Implies weights satisfy

ωij

> 0 if xj ≈ xi= 0 otherwise

I Requires specification of a metric to measure the ‘distance’between xjand xi and then a choice of weights as a function of distances

I R packages: -MatchIt-, -Matching-I Stata: -teffects-, -kmatch-


Inexact Matching in Detail

Requires a measure of distance between any two observations, i and j1 Euclidian-type distance metrics are of the form

dij = (xi − xj )′W (xi − xj )where common choices for W are

F W = I (identity matrix)F W = Σ−1, where Σ is the average sample variance-covariance matrix ofx across the treated and control groups (Mahalanobis metric)

F W = diag(Σ−1

), which replaces the off-diagonal terms in the above

version with zeros (Euclidean metric)

2 Propensity score methods compute the distance based on differences inthe probability of being in the treatment group given x

p(x) = Pr(D = 1|x) ∈ [0, 1]where distance between two observations is

dij = |p(xi )− p(xj )|, ordij = |`(xi )− `(xj )|

where `(xi ) = log[p(xi )/(1− p(xi ))]DL Millimet (SMU) ECO 7377 Spring 2020 114 / 478

Both distance measures have some meritI Both circumvent dimensionality as dij is a scalarI Goal is to balance the x’s; i.e., xj ≈ xi if dij ≈ 0

F Ho et al. (2007, p. 209): “[T]he goal of matching is to achieve thebest balance for a large number of observations, using any method ofmatching that is a function of X , so long as we do not consult Y .”

Both potentially entail some estimation in order to compute dijI Euclidian-type distance metrics potentially requires estimation of WI Propensity score methods most likely requires estimation of p(x)

F Will return to this later ... for now assume we know or have anestimate of dij

F And most of the following discussion focuses on estimating ∆ATT


Identification using the propensity scoreI Rosenbaum & Rubin (1983, Theorem 3) prove that

Yi (0),Yi (1) ⊥ D |x ⇒ Yi (0),Yi (1) ⊥ D |p(x)

I Estimation

∆ATE (p(x)) = E[yi |p(xi ) = p,D = 1]− E[yi |p(xi ) = p,D = 0]p−→ E[Yi (1)|p(xi ) = p]− E[Yi (0)|p(xi ) = p]

and then

∆ATE = E[∆ATE (p(x))

]=∫

∆ATE (p(x))f (p)dx =1N ∑i ∆ATE (p(xi ))

I Similar story for other parameters, except final step uses f (p|D = 1) orf (p|D = 0)

I Exact matching is still problematic since p(x) is continuous on the unitinterval

I Inexact matching assigns ωij > 0 when dij ≈ 0 ⇒ estimators arebiased


Given dij several weighting schemes are frequently used1 Single nearest neighbor (SNN)2 k−nearest neighbor (k−NN)3 Caliper (or radius)4 Kernel5 Local-linear6 Stratification (or interval or subclassification or blocking)

Biggest difference is whether all controls are utilized or only a subsetwhen estimating the missing counterfactual for obs i

I (1) — (3) use a subsetI (4) — (5) is ambiguous; depends on kernelI (6) uses all

Asymptotically, all inexact matching estimators are equivalent sincethe ‘inexactness’disappears as N → ∞Finite sample performance can vary dramatically


Single Nearest Neighbor Matching

Setsj∗ = arg min

j :Dj=0|dij |

⇒ωij =

1 if j = j∗

0 otherwise

Intuition: j∗ is ‘closest’to i , but with different treatment assignment

Estimated missing counterfactual given by

Yi (0) = yj ∗ = Yj ∗(0)


k-Nearest Neighbor Matching

Setsj∗ = k- arg min

j :Dj=0|dij |

⇒ωij =

1/k if j ∈ j∗0 otherwise

Intuition: compute the average of the k ‘closest’to i , but withdifferent treatment assignment than i


Yi (0) =1k ∑j ∗ yj ∗ =

1k ∑j ∗ Yj ∗(0)


Caliper or Radius Matching (Cochran & Rubin 1973)

Setsj∗ = j : |dij | < ε

for a specified value of ε⇒

ωij =

1/ki if j ∈ j∗0 otherwise

Intuition: compute the average over all ki obs that are ‘closer’to ithan ε, but with different treatment assignment than i


Yi (0) =1ki

∑j ∗ yj ∗ =1ki

∑j ∗ Yj ∗(0)


Kernel Matching (Smith & Todd 2005)

Sets

j∗ =j :

∣∣∣∣ dijaN∣∣∣∣ 6 ε

⇒

ωij =

G( dijaN

)∑

j ′∈Dj ′=0G(dij ′aN

) if j ∈ j∗

0 otherwise

where G (·) is the kernel function and aN is the bandwidthIntuition: compute the weighted average over all ki obs that receivepositive weight given the choice of G (·) and aN , but with differenttreatment assignment than i

Some kernel functions imply ε = 1 and thus all controls receivepositive weight for obs i


Local Linear Matching (Smith & Todd 2005)

Sets

j∗ =j :

∣∣∣∣ dijaN∣∣∣∣ 6 ε

⇒

ωij =

Gij ∑

j ′∈Dj ′=0Gij ′d

2ij−(Gijdij )

∑j ′∈Dj ′=0

Gij ′dij ′

∑

j∈Dj=0Gij ∑

j ′∈Dj ′=0Gijd 2ij ′−

∑j ′∈Dj ′=0

Gijdij ′

2 if j ∈ j∗

0 otherwise

where Gij = G(dijaN

)Intuition: similar to kernel matching, but differs in handling of weightsassigned to obs when obs are distributed asymmetrically around i orwhen there are gaps in the distribution of the propensity score


Stratification

Differs from above schemes (although it can be written as a matchingestimator)

AlgorithmI Unit interval is divided into k intervals based on the propensity scoreI The average outcome of treated and untreated is computed withineach interval

I ∆ATE (k) = Y 1 − Y 0 is computed within each interval

Finally

∆ATT = ∑k

N1kN1

∆ATE (k)

where N1k is the number of treated within strata k


Example

ID D y Y (0) Y (1) p(x)

1 0 10 10∑

l∈Dl=1ω1l yl ≡

ω13y3 +ω14y40.3

2 0 15 15∑

l∈Dl=1ω2l yl ≡

ω23y3 +ω24y40.4

3 1 14∑

l∈Dl=0ω3l yl ≡

ω31y1 +ω32y214 0.5

4 1 18∑

l∈Dl=0ω4l yl ≡

ω41y1 +ω42y218 0.6


Selection on ObservablesStrong Ignorability: Matching (Implementation)

Several practical issues are confronted when implementing matchingestimators

1 Distance metric is unknown2 With or without replacement3 Trimming4 Balance of the covariates, x5 Covariate adjustment6 Variable selection7 Weighting scheme8 Failure of CIA9 Non-binary treatments10 Inference


1. Distance metric is unknownI Euclidian-type distance metrics are of the form

dij = (xi − xj )′W (xi − xj )

F Requires estimation of W if W depends on variances and/orcovariances of x’s

F Just use plug-in estimator with sample values

I Propensity score distance metrics depend on p(x)F Estimation typically based on logit or probitF Some discussion of more complex estimators such as semi- ornon-parametric estimators or machine learning algorithms

F Not clear it matters much given the choice of x (discussed below)F Recommendation: Logit or probit is suffi cientF Newer alternatives⇒ Sant’Anna et al. (2018); R package: -ips- (integrated propensityscore)⇒ Imai & Ratkovic (2014); R package: -cbps- (covariate balancingpropensity score)⇒ Hainmueller (2012); Stata: -ebalance- (entropy balancing)

F King & Nielsen (2018) strongly advocate against propensity scorematching


2. With or without replacementI After a control is matched to a treated observation under SNN,k−NN, or caliper matching, should it be removed as a possible matchfor subsequent treated observations?

F Removed ⇒ matching without replacementF Not Removed ⇒ matching with replacement

I Choice has three implications:

1 Match Order: Without replacement implies that estimates are notinvariant to match order

2 Bias vs. Effi ciency: Without replacement leads to worse matches onaverage (↑ bias), but utilization of more controls (↑ effi ciency)

3 Inference: Without replacement makes inference (standard errors)easier since matched controls are independent

I Recommendation: Without replacement if N0 >> N1 and matchfrom highest p(xi ) to lowest (since presumably highest is most diffi cultto match)


3. TrimmingI May want to exclude control and/or treated observations that aredeemed ‘too different’than remainder of the sample

F Entails simply discarding some data prior to matchingF Changes the interpretation of the parameter being estimated from, e.g.,

∆ATE = E[∆i ]

to∆ATE = E[∆i |i ∈ C]

where C is the trimmed subpopulationI Why trim?

F CS region is defined as

Sp = p(x) : f (p|D = 1) > 0 and f (p|D = 0) > 0

F Matching estimates are only defined at values of p(x) ∈ SpF In practice, may want to exclude obs outside SpF To do so requires an estimate, Sp


3. Trimming (cont.)


3. Trimming (cont.)I Implemented two ways in practice

1 Discard obs outside of Sp , defined as

Sp = p(x) : p ∈

max

mini∈Di=0

p(xi ), mini∈Di=1

p(xi ),

min

maxi∈Di=0

p(xi ), maxi∈Di=1

p(xi )

2 Drop if p(xi ) /∈ [α, 1− α], where α is chosen based on criteriadeveloped in Crump et al. (2009) and may be equal to zero; see alsoImbens & Rubin (2015)

I Recommendation: Method 1 is very common, but Method 2 ispreferred

I Stata: -teffects overlap-


05

1015

20de

nsity

.2 .3 .4 .5 .6 .7Propensity score, unionh=1

unionh=0 unionh=1


4. Balance of the covariates, xI Matching provides unbiased estimates of average treatment effectsbecause the data mimic a randomized experiment conditional on x orp(x)

F By mimicing a random experiment, the x’s should be balanced (i.e.,have the same distribution in expectation) across the matched treatedand controls

F In fact, Rosenbaum & Rubin (1983, Theorem 1) prove that thepropensity score is a balancing score

x ⊥ D |p(x)

which holds regardless of validity of CIAF This is an asymptotic property; balancing tests gauge finite sampleperformance

F See Lee (2013), Imbens & Rubin (2015)


4. Balance of the covariates, x (cont.)I Most common approach is compare normalized differences in each xbefore and after matching

norm− diff = |x1 − x0 |√12

(σ2x1 + σ2x0

)where xd is the sample mean of either the original data or weighteddepending on how observations factor into the matching process

F A norm − diff > 0.2 is considered ‘large’(Rosenbaum & Rubin 1985)I Hoteling T 2 test

F Test joint null of equal (weighted) means across treatment and controlgroup

T 2 = (x1 − x0)′∑−1(x1 − x0)I Regression-based test

F Regress each x on a polynomial of p(x), D , and D interacted with thesame polynomial of p(x)...

xi = φ0 +∑Ss=1 φsp(xi )

s + π0Di +∑Ss=1 πsDip(xi )

s + ηi

and test Ho : π0 = π1 = · · · = πS = 0F Regression may be unweighted or weighted


4. Balance of the covariates, x (cont.)I Failure of matching to suffi ciently balance some x’s ⇒

1 Choose a different distance metric (e.g., alter the propensity scorespecification)

2 Perform covariate adjustment within the matched sample (next)

I Stata: -tebalance-


05

10

.2 .4 .6 .8 .2 .4 .6 .8

Raw Matched

control treated

Den

sity

Propensity Score

Balance plot


5. Covariate adjustmentI Inexact matching entails some bias due to this inexactnessI Imbalance in the x’s exacerbates this biasI Adjusting for differences in x’s after matching can reduce this biasI How? Example for ATT:

1 Regress Y on x using matched controls ⇒ βc2 Adjust estimated missing counterfactual for treated obs i to

Yi (0) =

1

∑j :Dj=0

ωij

∑j :Dj=0

ωij

[Yj + (xi − xj )βc

]

which leads to following adjusted estimator for ATT

∆ATTadj = Y 1 − Y 0 − (x1 − x0)βc = ∆ATT − (x1 − x0)βcwhere means are computed over the matched treated and control obs


5. Covariate adjustment (cont.)I Stata: -teffects nnmatch, biasadj-I Reference: Abadie & Imbens (2011)I Recommendation: Covariate adjustment is preferred to remove anyresidual differences in x’s


6. Variable selection (choice of x)I Three questions to ask:

1 Which (unique) variables to include in x?2 Which higher order and interaction terms of the x’s should be included?3 Are some x’s more important than other x’s?

I Answers:

1 Any variable that is pre-determined (i.e., unaffected by the treatment)and necessary for CIA, Y (0),Y (1) ⊥ D |x—Goal is not to predict treatment assignment—Do not include instrumental variables (Wooldridge 2016)

2 Higher order and interaction terms can be added to improve covariatebalance in the matched sample— It is fine to iterate; estimate distance, match, check balance, adjustand repeat ...

3 If so, can manipulate the distance metric to match exactly on some x’s—Can adjust W in Euclidean-type distance metrics—Can adjust p(x) so matches will be constrained to have identicalvalues of certain x’s


6. Variable selection (choice of x)I References: Rubin and Thomas (1996), HIT (1997), HIST (1998),Heckman and Smith (1999), Lechner (2002), Smith & Todd (2005),Hirano et al. (2003), Brookhart et al. (2006), Zhao (2007),Wooldridge (2009), Pearl (2009), Shaikh et al. (2009), Millimet &Tchernis (2009), Imbens & Rubin (2015)


7. Weighting schemeI SNN minimizes bias, but is less effi cientI Multiple matches per treated obs (via k−NN or caliper) may bebeneficial if N0 >> N1

I Kernel matching may be optimal in a mean-squared error senseI Local linear matching may be optimal if many obs have p(x) close tothe boundary of 0 or 1

I Stratification is intuitive, but requires potentially ad hoc choice ofstrata

I References: Huber et al. (2013), Busso et al. (2014)I Recommendation: Try several and compare


8. Failure of CIAI Presence of unobserved attributes correlated with both treatmentassignment and potential outcomes invalidates CIA

F Implies the treatment is not random conditional on observed x aloneF This implies estimated average treatment effects are (more) biased(than just due to inexactness of matches)

I Four options:

1 Gather additional data such that CIA holds—This may entail use of panel data— Leads to difference-in-differences matching based on changes inoutcomes

2 Assess how important such unobserved attributes would have to be inorder to explain estimated treatment effects obtained by matching

3 Alter the estimand4 Different estimator (selection on unobservables approach)


Difference-in-Differences Matching

Consider ∆ATT

∆ATT (p(x)) =

E[Y (1)|p(x),D = 1]− E[Y (0)|p(x),D = 0]+ E[Y (0)|p(x),D = 0]− E[Y (0)|p(x),D = 1]

where matching estimators are based on

∆ATT (p(x)) = E[Y (1)|p(x),D = 1]− E[Y (0)|p(x),D = 0]

which implies

bias = ∆ATT (p(x))− ∆ATT (p(x))= E[Y (0)|p(x),D = 1]︸︷︷︸

Counterfactual

− E[Y (0)|p(x),D = 0]︸︷︷︸Observed

which is zero under CIA


Rearranging terms yields

∆ATT (p(x)) = ∆ATT (p(x))− bias

This suggests a bias-corrected estimator is feasible if the bias can beconsistently estimated

Might assume the bias equals the difference in mean outcomes priorto treatment

bias = E[Yt (0)|p(x),D = 1]− E[Yt (0)|p(x),D = 0]?= E[Yt ′(0)|p(x),D = 1]− E[Yt ′(0)|p(x),D = 0]

where t ′ precedes the treatment, t is post-treatment


Implementation: difference the data ∀i , then match to estimate theATE , ATT , or ATU

DID matching requires the original CIA be replaced with

∆Y (0),∆Y (1) ⊥ D |p(x)

Intuition:I DID matching requires the change in potential outcomes to beindependent of treatment assignment given the PS

I Equivalently, there are no time varying unobservables correlated withboth the changes in outcomes and treatment assignment given x


Rosenbaum Bounds

Method of assessing sensitivity of matching estimator to anunobserved confounder (Rosenbaum 2002)Assume

p(xi ) = F (xi β+ ui ) =exp(xi β+ ui )

1+ exp(xi β+ ui )

where u is an unobserved binary variable and F is the logistic CDFImplications

I Odds ratio for obs i is

p(xi )1− p(xi )

= exp(xi β+ ui )

I Odds ratio for obs i relative to obs i ′

p(xi )1−p(xi )p(xi ′ )1−p(xi ′ )

=exp(xi β+ ui )exp(xi ′β+ ui ′)

= expγ(ui − ui ′) if xi = xi ′

I Thus, two observationally identical obs have different probabilities ofbeing treated if γ 6= 0 and ui 6= ui ′


How does inference regarding the treatment effect parameters changeas γ and ui − ui ′ change?

I Since u is binary, ui − ui ′ ∈ −1, 0, 1I Implies

1expγ 6

p(xi )1−p(xi )p(xi ′ )1−p(xi ′ )

6 expγ

where

F expγ = 1⇒ no selection biasF expγ → ∞⇒ greater selection bias

I Rosenbaum bounds compute bounds on the significance level of thematching estimate as expγ changes values

F If matching estimate is statistically insignificant even whenexpγ ≈ 1, then treatment effect is not robust

F If matching estimate is statistically significant even when expγ is‘large’, then treatment effect is “not sensitive to hidden bias”

Stata: -rbounds-, -mhbounds-


Simulation Approach

Nannicini (2007) and Ichino et al. (2008) propose an alternativemethod of assessing the robustness of ATT estimates obtained underCIA

Intuition:I If there is a relevant unobservable that invalidates the CIA, but suchthat CIA would hold if one observed this variable, then estimation isstraightforward if one can simulate/impute this unobserved variable

I The sensitivity analysis proceeds by comparing the baseline matchingestimate to estimates obtained after additionally conditioning upon thesimulated confounder

I The unobserved variable can be simulated in different ways to capturedifferent hypotheses regarding the nature of potential confounders


SetupI The parameter of interest is the ∆ATTI Accordingly, Y (0) ⊥ D |x denotes the required CIAI Suppose that this condition is not met, but if an unobservable, U, isadded then a stronger CIA holds

Y (0) ⊥ D |x ,U

I Implies

E[Y (0)|D = 1, x ] 6= E[Y (0)|D = 0, x ]E[Y (0)|D = 1, x ,U ] = E[Y (0)|D = 0, x ,U ]


SolutionI Simulate the potential confounder and use it as a matching covariate

F For simplicity, the potential outcomes and the confounding variable areassumed to be binary

F Conditional independence of U and x is also assumedF Hence, the distribution of U is fully characterized by the choice of thefollowing four parameters

pij ≡ Pr(U = 1|D = i , y = j) = Pr(U = 1|D = i , y = j , x)

with i , j ∈ 0, 1F Given the parameters pij , a value of U is simulated for each observationdepending on D , y

I ∆ATT is then estimated with U as an additional matching covariate

For a given set of the parameters pij , many simulations are performed,∆ATT computed for each simulation, and the mean/sd of theestimates reported


Choosing pij ...I It is essential to consider useful potential confoundersI Calibrated confounders: choose pij to make the distribution of Usimilar to the empirical distribution of observable binary covariates

I Killer confounders: search over different pij for the existence of a Uwhich makes ∆ATT = 0

I One can also simulate other meaningful confounders by setting theparameters pij and pi ·, where pi · can be computed as

pi · ≡ Pr(U = 1|D = i) =1∑j=0

pij · Pr(y = j |D = i)

with i ∈ 0, 1

Stata: -sensatt-


Partial Conditional Independence

See Masten & Poirier (2018)

Intuition: Allow for small deviations from CIA

|Pr(D = 1|Y (0),Y (1), x)− Pr(D = 1|x)| ≤ c

I CIA implies that c = 0I Partial CIA implies c > 0, but ‘small’

Estimation proceeds by bounding average treatment effect parametersconditional on choice of c

Stata coding coming


Minimum Bias Approach

See Millimet & Tchernis (2013), McCarthy et al. (2014)

Intuition: Restrict the region of analysis to minimize the bias ofmatching estimators arising from the failure of CIA

Disadvantages:I Requires a lot of structure to estimate this regionI Interpretation of estimated treatment effect changes to the averagetreatment effect in this region

Stata: -bmte-


9. Non-binary treatmentsI In some instances, there may be a vector of treatments,D ∈ 0, 1, 2, ...,T

I This alters the analysis in three ways

1 Many possible parameters of interest; e.g.,

∆d′′dd ′ = E

[Y (d )− Y (d ′)|D = d ′′

]∀d , d ′, d ′′

(E.g., What is the expected effect of job training (d) relative to jobsearch assistance (d ′) for those who did nothing (d ′′)?)

2 Altered identification assumptions: conditional independence of allpotential outcomes given x or only some?

3 Now, must estimate generalized propensity score (GPS) given by

e(d , x) = Pr(D = d |x)

which may be estimated using multinomial or ordered models

I With D representing variable treatment intensity or D continuous,treatment effects referred to as the dose-response function

I Recent work in Lee (2018)I Stata: -teffects multivalued-, -poparms-


10. InferenceI Usual t−test for diff in mean outcomes across matched treated anduntreated group ignores estimation of propensity score and nature ofmatching

I Smooth matching estimators (e.g., kernel matching) rely on bootstrapI Standard bootstrap fails in the case of non-smooth matching estimatorsI Abadie & Imbens (2006) provide asymptotic standard errors forEuclidean matching estimators

I See also Abadie & Imbens (2008, 2011, 2012, 2016), Otsu & Rai(2017), Rothe (2017), Bodory et al. (2018), Yang & Ding (2018)


Selection on ObservablesStrong Ignorability: Inverse Propensity Score Weighting (IPW) Estimators

Alternative to matching estimators, but still rely on estimating thepropensity score

Identities

E[Dyp(x)

]= E

[DY (1)p(x)

]= E

[E[DY (1)p(x)

]| x]

= E[1p(x)

E [DY (1)] | x]CIA= E

[1p(x)

E[D | x ]E[Y (1) | x ]]

= E[p(x)p(x)

E[Y (1) | x ]]= E [E[Y (1) | x ]] = E[Y (1)]

and, similarly,

E[(1−D)y1− p(x)

]= E[Y (0)]


Parameters of interest (Horvitz & Thompson 1952)

∆ATE = E[Dyp(x)

− (1−D)y1− p(x)

]= E

D − p(x)

p(x)[1− p(x)]y

∆ATT = E

D − p(x)Pr(D = 1) [1− p(x)]y

∆ATU = E

D − p(x)

Pr(D = 0) [1− p(x)]y

Proof: Wooldridge (2002, p. 615)

Estimation: Replace p(x) with p(x) and expectations andprobabilities with sample means


Normalized estimators (Hirano & Imbens 2001)

I ∆ATE is the difference in two weighted averages, where weights are

Di

Np(xi )and

1−DiN[1− p(xi )

]I Problem: weights may not sum to unityI HI assign weights normalized by the sum of propensity scores fortreated and untreated groups

I Unnormalized estimator assigns equal weights of 1/N to eachobservation

I Normalized estimator (e.g., ∆ATE )

∆ATE =

[∑i

Di yi

p(xi )

/∑i

Di

p(xi )

]−[∑i

(1−Di )yi1− p(xi )

/∑i

(1−Di )1− p(xi )

]

I Tends to be more stable in practice as it restricts weights to ≤ 1;Millimet & Tchernis (2009), Busso et al. (2011) find it performs better

Stata: -teffects ipw-


Easily extended to multi-valued treatmentsI Vector of treatments given by D ∈ 0, 1, 2, ...,TI Treatment parameters defined as

∆d′′dd ′ = E

[Y (d)− Y (d ′)|D = d ′′

]∀d , d ′, d ′′

I Generalized propensity score given by

e(d , x) = Pr(D = d |x)

I Estimators utilize sample counterparts to the following equalities

E[Dd ye(d , x)

]= E[Y (d)]

E[Dd y

e(d ′, x)e(d , x)

]= E[Y (d)|D = d ′]

I See Uysal (2015)


Weighting estimators can be extremely sensitive if obs have p(x) ≈ 0or 1

I Typically necessary to trim sampleI Fairly ad hoc

Standard errors obtained via bootstrap

Other weighting schemes considered in Li et al. (2018)


Selection on ObservablesStrong Ignorability: Propensity Score Residual Estimation

Lee (2017) proposes a computationally simple estimator relying onthe propensity score

Estimator obtained via OLS estimation of

yi − y = ∆[Di − p(xi )

]+ εi

where p(xi ) = Φ(xi γ) obtained via probit model and

plim ∆→ E [ω(x)E [Y (1)− Y (0)|x ]] 6= ∆ATE

and

ω(x) ≡ p(x)[1− p(x)]E p(x)[1− p(x)]

However, finite sample performance of the estimator is good

Standard errors obtained via bootstrap or asymptotic formula


Two alternative estimators that may perform better1 OLS estimation of

yi − Γi = ∆[Di − p(xi )

]+ εi

where Γi = ∑qj=0 δj (xi γ)j

F I.e., yi − Γi = residual from the OLS regression of yi on a constant anda polynomial in the linear index from the propensity score model , xi γ

F Motivation is to improve performance if p(x) is mis-specifiedF Lee (2017) suggests q = 2 or 3

2 OLS estimation of

yi − y√p(xi )

[1− p(xi )

] = ∆

Di − p(xi )√

p(xi )[1− p(xi )

]+ εi

which is consistent for ∆ATE , but may do poorly in practice ifp(xi ) ≈ 0, 1 for some observations


Method extends easily to non-binary treatmentsI Let D ∈ 0, 1, 2, ...,T and Dd = I(D = d)I Let p(x) = p1(x), p2(x), ..., pT (x) be the set of propensity scoresestimated via multinomial or ordered probit

I Estimator obtained via OLS estimation of

yi − y = ∑d

∆d[Ddi − pd (xi )

]+ εi

oryi − Γi = ∑

d∆d[Ddi − pd (xi )

]+ εi

where Γi = ∑qj=0 δj (xi γ)j if an ordered probit is used and

∆dd ′ = ∆d′′dd ′ = ∆d − ∆d ′


Selection on ObservablesStrong Ignorability: Double-Robust Estimators

Robins and Rotnizky (1995), Hirano & Imbens (2001), Lunceford andDavidian (2004), and others discuss DR estimators

DR estimators combine regression and weighting estimators and aredouble robust because they are consistent as long as

I The regression specification for the outcome is correctly specified, orI The propensity score specification is correctly specified

DR is a class of estimators that possess this property

Intuition: Control for x twice, once via linear regression and once viapropensity score, as a means of overkill

Several DR estimators exist in the literature


Estimation

OLS estimation

yi = α0 + xi β+ α1Di + θ0Di

p(xi )+ θ1

1−Di1− p(xi )

+ εi

∆ATE = α1 + 1N ∑i

[θ0

Di

p(xi )− θ1

1−Di1− p(xi )

]

∆ATT = α1 + 1N1

∑i :Di=1

[θ0

Di

p(xi )− θ1

1−Di1− p(xi )

]

∆ATU = α1 + 1N0

∑i :Di=0

[θ0

Di

p(xi )− θ1

1−Di1− p(xi )

]


WLS estimation: ATE or ATT

yi = α0 + xi β+ α1Di + υi

where weights are

λATEi =

√Di

p(xi )+

1−Di1− p(xi )

λATTi =

√√√√Di + (1−Di ) p(xi )1− p(xi )

and similarly for ATU


Augmented IPW: ATE (Lunceford and Davidian 2004; Glynn andQuinn 2010)

∆ATE =1N ∑i

[Di yi − (Di − p(xi ))µ1(xi )

p(xi )− (1−Di )yi + (Di − p(xi ))µ0(xi )

1− p(xi )

]

where µ0(xi ) and µ1(xi ) are estimated via separate OLS regressionsof y on x

Stata: -dr-, -teffects aipw-


Uysal (2015) discusses doubly robust estimators of multi-valuedtreatment effects

SetupI Vector of treatments given by D ∈ 0, 1, 2, ...,TI Treatment parameters defined as

∆d′′dd ′ = E

[Y (d)− Y (d ′)|D = d ′′

]∀d , d ′, d ′′

I Generalized propensity score given by

e(d , x) = Pr(D = d |x)

which may be estimated using multinomial or ordered logit/probitmodels


EstimationI WLS estimation: ∆dd ′ or ∆d

′′dd ′

yi = ∑d

∆dDdi +∑dDdi (xi − x)βd + εi

where∆dd ′ = ∆d

′′dd ′ = ∆d − ∆d ′

and weights are

λ∆dd ′i =

√∑d

Ddie(d , xi )

λ∆d′′dd ′i =

√√√√∑dDdi

e(d ′′, xi )e(d , xi )

I Standard errors obtained via bootstrap


Waernbaum & Pazzagli (2017, WP)I Assess the bias of DR estimators when the propensity score andoutcome equations are both mis-specified

I Assess situations when the bias of DR estimators is smaller than thebias of IPW estimators (normalized and non-normalized)


Selection on ObservablesRegression Discontinuity

First introduced in Thistlethwaite & Campbell (1960)

Two classes of models: sharp, fuzzyI Sharp RD is a selection on observables estimator, but is not based onstrong ignorability (in fact, it precludes it)

I Fuzzy RD is a selection on unobservable estimators (discussed later)

Note: Recent work also on Regression Kinked Design (Card et al.2009)


RD setupI Agents self-select into treatment groupI Selection done at least in part on the basis of an observed continuousvariable, s

F s is referred to as the “score”, “running” variable, or “forcing” variable

I s may directly impact potential outcomes as wellI There exists a discrete jump in Pr(D = 1) at a known value, s

Thus, s and s are both known to the econometrician

Applications: eligiblity rules for social programs, voting outcomes,financial aid, GED, Clean Air Act attainment status, geographicborders, time


Examples: Sharp vs. Fuzzy RD


Sharp RD model

(SRD.i) Treatment assignment is a deterministic function of s (with a knownthreshhold, s)

Di = D(si ) =1 if si > s0 otherwise

(SRD.ii) Positive density at the threshold: fS (s) > 0(SRD.iii) Potential outcomes are continuous in s at least around s(SRD.iv) For each agent, the dbn of s is continuous at least around s

NotesI (SRD.ii) implies we see agents near sI (SRD.iii) precludes discontinuities in y at s due to other reasonsbesides changes in D

I (SRD.iv) implies that agents cannot manipulate s to ensure s ≷ sF This is crucial to give the setup the interpretation of a randomexperiment in the neighborhood of s

I Eggers et al. (2017) discuss failure of (SRD.iii) and (SRD.iv)


Example: Manipulation of the running variable


Notes (cont.)I Y (0),Y (1) ⊥ D |s follows from (SRD.i)I All RD estimators require existence of following limits

D+ = lims↓sPr(D = 1|s)

D− = lims↑sPr(D = 1|s)

and D+ 6= D−F (SRD.i) implies D+ = 1 and D− = 0

I Common support condition is necessarily violated since

Pr(D = 1|s) =1 if si > s0 otherwise

which implies that Pr(D = 1|s) /∈ (0, 1) ∀s


Parameter of interest(LATE)

∆ATE (s) = E[Y (1)− Y (0)|s ]= lim

s↓sE[y |s ]

− lims↑s

E[y |s ]


EstimationUse only sub-sample with si ∈ s − δ, s + δ for small δ

I Similar s ⇒ similar observationsI Compute mean difference in outcomes across treatment groups

∆ATE (s) = E[yi |si ∈ s, s + δ,D = 1]− E[yi |si ∈ s − δ, s,D = 0]

=∑Ni=1 yi I[si ∈ s, s + δ,Di = 1]

∑Ni=1 I[si ∈ s, s + δ,Di = 1]

− ∑Ni=1 yi I[si ∈ s − δ, s,Di = 0]

∑Ni=1 I[si ∈ s − δ, s,Di = 0]

p−→

E[yi |si ∈ s, s + δ,D = 1]− E[yi |si ∈ s − δ, s,D = 0]

=

E[Yi (1)|si ∈ s, s + δ,D = 1]− E[Yi (0)|si ∈ s − δ, s,D = 0]

= E[Yi (1)|si ∈ s, s + δ]− E[Yi (0)|si ∈ s − δ, s]6= lim

s↓sE[y |s ]− lim

s↑sE[y |s ] for fixed δ > 0


This is essentially a kernel estimator with a uniform kernel over theinterval s, s + δ or s − δ, s, which entails a non-negligible biasfor δ > 0

Example: If y is increasing in s, then

I E[yi |si ∈ s, s + δ,D = 1] will overestimate lims↓s E[y |s ]I E[yi |si ∈ s − δ, s,D = 0] will underestimate lims↑s E[y |s ]⇒ ∆ATE (s) will be biased up


Regression approachI Model

yi = ∆Di + εi

where D = treatment indicator, ∆ = parameter of interestI OLS is biased since Cov(D, ε) 6= 0I However, E[ε|D, s ] = E[ε|s ]I Implies ∆ is estimable if the model is augmented with a suffi cientlyflexible function of s to proxy for E[ε|s ]

yi = ∆Di + k(si ) + ηi

where Cov(D, η) = 0I What is k(s)?

F Linear: k(s) = s (Goldberger 1972; Cain 1975)F Quadratic: k(s) = θ1s + θ2s2 (Berk & Rauma 1983; van der Klaauw2000)

F Semiparametric: k(s) = ∑Mm=1 θmsm , with M choosen bycross-validation (Trochim 1984; van der Klaauw 2000)

F Common to allow k(s) to be differ across s (ia interactions with Di )F See Gelman & Imbens (2017)


Example: Common linear k(s) on each side of s

100

010

020

030

0O

utco

me

(y)

50 0 50Score (s)


Example: Different linear k(s) on each side of s


200

020

040

060

0O

utco

me

(y)

50 0 50Score (s)


Example: Same DGP, different cutoff, different LATE

200

020

040

060

0O

utco

me

(y)

50 0 50Score (s)


NotesI Testing of some of the underlying assumptions is feasible

F Examine the density of s to look for evidence of discontinuity at s ,suggesting manipulation by agents (McCrary 2008)

F Look for existence of discontinuities in predetermined variables at s(similar to assessing balancing of predetermined variables in randomizedexperiments)

I If treatment effect is heterogeneous, then RD estimates a uniqueparameter (discussed above) that may be uninteresting

F This is an example of a local average treatment effect (LATE)F May be a policy relevant parameter if the question is the impact of amarginal change in an ‘eligibility’cut-off, s

F Angrist & Rokkanen (2015) discuss extrapolating away from s

Stata: -rd-, -rdcv-, -rdrobust-


Selection on ObservablesDistributional Approaches

Analysis to this point has focused on mean effects of treatments

Averages may mask a lot of heterogeneity

Distributional methods seeks to assess the effects of treatments onother quantities

Traditional approach is quantile regression (QR)

More recent approaches have been couched in the potential outcomesframework and focus on quantile treatment effects (QTE)


Selection on ObservablesDistributional Approaches: Quantile Regression

MotivationI QR provides a convenient linear framework for assessing the impact ofchanges in a vector of covariates on the quantiles of the dependentvariable

I Equivalently, QR allows estimation of linear conditional quantilefunctions

I Analogous to linear regression, which estimates the conditional meanfunction

I Common applicationsF Studies of wage determinationF Studies of student achievement

NotationI F (y) = CDF of yI Qθ(y) = θth quantile of the random variable, y , given by

Qθ(y) = infy : F (y) > θ


(Unconditional) quantiles as a minimization problemI Prior to discussing QR, it is useful to view unconditional quantiles as asolution to a minimization problem

I Example: median

Q0.5(y) = argminb

∑i |yi − b|

F Solution depends on the sign of the residuals, not the magnitudeF y = 99, 100, 101 ⇒ Q0.5(y ) = 100;y = 99, 100, 150 ⇒ Q0.5(y ) = 100 as increasing b closer to 150reduces that residual, but increases the sum of the other two residualsby twice as much

F Implies median is less sensitive to outliers than the meanI General formula for any quantile θ ∈ (0, 1)

Qθ(y) = argminb

∑i :yi>b

θ|yi − b|+ ∑i :yi<b

(1− θ)|yi − b|

F Quantiles other than the median are defined as the arg min of aweighted sum of the absolute residuals

F Intuition: say θ = 0.75 and b = median, then problem puts moreweight on residuals above b, which pushes the solution to theminimization problem above the median


QR model (Koeneker & Bassett 1978)I Replace b in previous problem with a linear function of covariates

βθ = argminβ

1N

∑

i :yi>xi βθ|yi − xi β|+ ∑

i :yi<xi β(1− θ)|yi − xi β|

which may be rewritten as

βθ = argminβ

1N∑i ρθ(εθi )

where ρθ(εθi ) is known as the check function, defined as

ρθ(εθi ) = [θ − I(εθi < 0)]εθi

and εθi is the residual for i and θI Preceding objective fn is equivalent (after some algebra) to

βθ = argminβ

1N

∑i

[θ − 1

2+12

sgn(yi − xi β)](yi − xi β)

I Error distribution

F Key assumption: Qθ(εθ |x) = 0F No other assumption about the distribution


Estimation

The objective fn is not differentiable ⇒ standard optimizationmethods are not viable

Solved using linear programming methods

GMM estimation is also feasible (Buchinsky 1998)

Special case: median regressionI Corresponds to QR model with θ = 0.5; β obtained from

β0.5 = argminβ

1N∑i |yi − xi β|

I Analogous to OLS, but β minimizes the sum of absolute errors insteadof sum of squared errors

I Also known as LAD (Least Absolute Deviations) estimatorI Useful alternative to OLS, particularly when the distribution of theerror term is symmetric (so the conditional mean and median areequal), yet outliers are a concern

Bootstrap methods used for inference (Buchinsky 1998)


Results

Parameters of interest are the partial derivatives of the conditionalquantile fn w.r.t. x

∂Qθ(y |x)∂xk

which equals βθk if x enters linearly

Presentation of resultsI Diffi cult as there are a large number of results that are possible toobtain (i.e., βθk , k = 1, ...,K and θ ∈ (0, 1))

I Possibilities

F Typical table of coeffi cient estimates at several quantiles (typically θ =0.10, 0.25, 0.50, 0.75, and 0.90)

F Graph the conditional quantile fns against xk if there is one x that isthe focus of the paper (again, typically for a few quantiles)

F Graph βθk vs. θ for several different x’s on one graph (only works if xkenters linearly)


Sequential estimationI In practice, one typically wishes to estimate βθ for multiple values of θI Estimates are not independent since they are obtained from the samedata

I Estimation one equation at a time, however, is effi cient unless there arecross-equation restrictions

Stata: -qreg-, -bsqreg-, -sqreg-, -grqreg- (for graphing), -qcount- (forcount data models), -lqreg- (for logistic models)


Test of equal slopes across quantiles


Selection on ObservablesDistributional Approaches: Distribution Regression

Chernozhukov et al. (2013) propose an alternative to QR calleddistribution regression

Algorithm1 Define yi (k) = I(yi ≥ k)2 Estimate

yi (k) = F (xi β(k))

where F is some link function for probit, logit, LPM, etc.


Example: Dynarski (2008)


Selection on ObservablesDistributional Approaches: Quantile Treatment Effects

Traditional quantile regression estimates conditional quantiles and isnot developed using the potential outcomes framework

Literature on QTEs estimates unconditional quantiles and isembedded in the potential outcomes framework

Bedoya et al. (2018) provide a nice introduction


NotationI Yi (1), Yi (0) = potential outcomes for iI ∆i = treatment effect for iI Di = binary indicator of treatment assignmentI Fj (y) ≡ Pr[Yi (j) < y ], j = 0, 1 = CDFs of potential outcomesI Y θ(j) = infy : Fj (y) > θ = quantiles of potential outcome dbnsI ∆θ = infδ : F∆(δ) > θ = quantiles of treatment effect dbn


Parameters of interest fall into 2 classes1 Attributes of the dbn of ∆, such as quantiles or measures of dispersion2 Attributes of the dbns of Y (1), Y (0), such as quantiles

For now, focus on (2) and define the following parameters of interest

4QTEθ = E[Y θ(1)− Y θ(0)], θ ∈ (0, 1)4QTTθ = E[Y θ(1)− Y θ(0)|D = 1], θ ∈ (0, 1)4QTUθ = E[Y θ(1)− Y θ(0)|D = 0], θ ∈ (0, 1)


Interpretation

Constant treatment effect assumptionI Yi (1) = Yi (0) + ∆ ∀iI Implies F−11 (θ) = F−10 (θ) + ∆

0.2

.4.6

.81

4 2 0 2 4

y1 y0

F(y)

NOTE: y0~N(0,1); y1=y0+1

I 4QTEθ = 4QTTθ = 4QTUθ = ∆ ∀θ ∈ (0, 1)


Heterogeneous treatment effectsI Yi (1) = Yi (0) + ∆iI Perfect rank correlation (Heckman et al. 1997)

F Definition: F1(Yi (1)) = F0(Yi (0)) ∀iF Intuition: each observation lies in the identical quantile in bothpotential outcome dbns, which implies that Y (1) is a monotonetransformation of Y (0)

F Implication: 4QTEθ = E[Y θ(1)− Y θ(0)] = Qθ(∆), which is the θth

quantile of the dbn of ∆, which implies that QTEs identify thedistribution of the treatment effect, BUT this requires a strongassumption about the joint dbn of potential outcomes

I No perfect rank correlationF No assumption about the joint dbn of potential outcomesF Implication: 4QTEθ = E[Y θ(1)− Y θ(0)] 6= Qθ(∆), which implies thatQTEs identify the difference in the two marginal dbns of the potentialoutcomes, BUT say nothing about the dbn of actual treatment effects... QTEs reflect the effect of D on quantiles of the potential outcomedbns, NOT on observations at particular quantiles.

I Tests of rank preservation developed in Dong & Shen (2018), Frandsen& Lefgren (2018)


Example #1...

ID Y (0) Y (1) ∆1 1 2 12 2 4 23 3 6 34 4 8 45 5 10 5

Rank preservation holds; ∆ivaries

4QTEθ varies with θ

4QTEθ = Qθ(∆)


Example #2...

ID Y (0) Y (1) ∆1 1 1 02 2 4 23 3 3 04 4 2 -25 5 5 0

Rank preservation is violated; ∆ivaries

CDF of Y (0), Y (1) areidentical ⇒ 4QTEθ = 0 ∀θ

4QTEθ 6= Qθ(∆)


Quantiles of potential outcomes versus quantiles of treatment effectsin more detail

I To see the difference more clearly and understand the importance ofrank preservation, we can use our prior notation and write potentialoutcomes as

Yi (1) ≡ Y θ1i (1)

Yi (0) ≡ Y θ0i (0)

where θdi = percentile of obs i in the dbn of potential outcome dI The individual-level treatment effect is

∆i = Yi (1)− Yi (0) ≡ Y θ1i (1)− Y θ0i (0)

= Y θ1i (1)− Y θ1i (0)︸︷︷︸∆QTEθ1i

+ Y θ1i (0)− Y θ0i (0)︸︷︷︸mobility effect

where the mobility effect reflects the movement of obs i across the twopotential outcome dbns

F Under rank preservation, θ0i = θ1i and the mobility effect is zeroF Hence, the QTEs will identify the quantiles of the dbn of the treatmenteffects


Estimation

Identification assumptions: strong ignorability (CIA, CS)

yi = DiYi (1) + (1−Di )Yi (0) = observed outcome∆θ obtained using sample analogues of y θ

1 and yθ0

I Obtain Fd (y), d = 0, 1

Fd (y) =∑i∈d ωi I(yi ≤ y)

∑i∈d ωi

where

ωi =Dip(xi )

+1−Di1− p(xi )

(QTE)

ωi = Di +p(xi )(1−Di )1− p(xi )

(QTT)

ωi =[1− p(xi )]Di

p(xi )+ (1−Di ) (QTU)

I ∆QT ·θ = Y θ(1)− Y θ(0), where Y θ(d) = infy : Fd (y) > θ


NotesI The estimator follows from the fact that

E[Dg(y)p(x)

]= E[g(Y (1))]

E[(1−D)g(y)1− p(x)

]= E[g(Y (0))]

F The IPW estimator of mean potential outcomes sets g (y ) = yF The IPW estimator of quantiles of potential outcomes setsg (y ) = I(yi ≤ y )

I Alternative estimator obtained via (Firpo 2007)

β0θ, ∆QT ·θ = argmin

β,∆

1N

∑i

ωi ρθ(yi − β0 − ∆θDi )

where ωi is given above

F Here, x enters only through the weights to avoid estimating conditionalquantiles

Stata: -ivqte-, -dbn- (my code)


Selection on ObservablesDistributional Approaches: Inequality Treatment Effects

Firpo & Pinto (2016) define a new set of inequality treatment effectparameters

∆Oυ = υ[FY (1)

]− υ

[FY (0)

]∆TTυ = υ

[FY (1)|D=1

]− υ

[FY (0)|D=1

]∆CITυ = υ [FY ]− υ

[FY (0)

]where υ(·) is a measure of inequality, F· is the dbn of a particularoutcome (potential or observed), and

I ∆Oυ = overall inequality treatment effectI ∆TTυ = inequality treatment effect on the treatedI ∆CITυ = current inequality treatment effect

Possible choice of υ1 Gini index2 Interquartile range3 Theil index


Estimation

Identification assumptions: strong ignorability (CIA, CS)yi = DiYi (1) + (1−Di )Yi (0) = observed outcome∆υ obtained using sample analogues of the F sObtain Fd (y), d = 0, 1

Fd (y) =∑i∈d ωi I(yi ≤ y)

N

where

ω0i =1−Di1− p(xi )

(Y (0))

ω1i =Dip(xi )

(Y (1))

ω01i =

[1−Di1− p(xi )

]p(xi )p

(Y (0)|D = 1)

ω11i =Dip

(Y (1)|D = 1)


Firpo & Pinto (2016) provide explicit formulas using different v(·)Standard errors obtained via asymptotic formula or bootstrap


Selection on Unobservables

Failure of CIA ⇒ selection on unobservables

Implies unobserved attributes of obs i are correlated with bothpotential outcomes and treatment assignment of obs i

D Y

Xv

Z =observables

u =unobservables


In general, this implies

E[Y (d)|x ,D = d ] 6= E[Y (d)|x ,D = d ′], d , d ′ = 0, 1

In a regression framework, with functional form assumptions, thisimplies

yi = DiYi (1) + (1−Di )Yi (0)= α0 + xi β0 + (α1 − α0)Di + xiDi (β1 − β0)

+ [υ0i +Di (υ1i − υ0i )]

where SOU results ifI Cov(D, υ0) 6= 0 ⇒ selection on unobservables impacting outcome inuntreated state, or

I Cov(D, υ1 − υ0) 6= 0 ⇒ presence of and selection on unobserved,obs-specific gains from treatment


Possible solutions1 Bound treatment effects (set or partial identification as opposed topoint identification) under minimal assumptions

2 Utilize panel data3 Utilize exclusion restrictions (i.e., instrumental variables)4 Model dependence between treatment and unobservables ⇒ controlfunction approach

5 Other methods that ‘find’identification elsewhere


Selection on UnobservablesBounding Treatment Effects

Recall, the ATE

∆ATE (x) = E[Y (1)− Y (0)|x ] = E[Y (1)|x ]− E[Y (0)|x ]= E[Y (1)|x ,D = 1]Pr(D = 1|x)

+ E[Y (1)|x ,D = 0]Pr(D = 0|x)− E[Y (0)|x ,D = 1]Pr(D = 1|x)

+ E[Y (0)|x ,D = 0]Pr(D = 0|x)= g1(x)− E[Y (0)|x ,D = 1]p(x)

+ E[Y (1)|x ,D = 0]− g0(x)[1− p(x)]

where p(x), the propensity score, and gd (x), d = 0, 1, are allobservable from the data


Similar derivation for other two primary mean treatment effectparameters

∆ATT (x) = g1(x)− E[Y (0)|x ,D = 1]∆ATU (x) = E[Y (1)|x ,D = 0]− g0(x)

Thus, without additional information, no parameter is identified

Objective of bounding is to replace the unknown terms with feasiblevalues that minimize and maximize ∆ATE , ∆ATT , or ∆ATU

Thus, the bounds represent the range of possible values of theaverage treatment effect parameter that are consistent with the data

The bounds are not like a confidence interval since, absenceadditional information, the probability of the parameter being equal toany value in the bounds is constant

Additional assumptions/information can be added to tighten thebounds


Early bounding approach outlined in Smith and Welch (1986)I Objective was to estimate the average wage for blacks accounting forselection into employment

E[w ] = E[w |E = 1]Pr(E = 1) + E[w |E = 0]Pr(E = 0)

where E[w |E = 0] is not observedI Solution: E[w |E = 0] = γ E[w |E = 1], γ ∈ [0.5, 1]I In treatment effects context, can specify

E[Y (d)|D = d ′] = γ E[Y (d)|D = d ] for different values of γ, whered 6= d ′

Rosenbaum (2002) summarizes other papers that bound causaleffects by varying the unobserved parameters


More recent approaches focus on adding assumptions to tighten thebounds on the parameter of interest

Notation (Lechner 1999; Manski 1990)I L1, L0 = lower bounds of the support of Y (1), Y (0), respectivelyI U1, U0 = upper bounds of the support of Y (1), Y (0), respectivelyI BLk , B

Uk = lower, upper bounds, respectively, of treatment effect k

(k = ATE ,ATT , or ATU)I wk = BUk − BLk = width of bounds for treatment effect k


Trivial caseI No additional information

BLk = L1 − U0BUk = U1 − L0wk = (U1 − L0)− (L1 − U0)

= (U1 − L1) + (U0 − L0)

I Example: y is binary (e.g., employment after job training program)

L1 = L0 = 0

U1 = U0 = 1

BLk = −1BUk = 1

wk = 2


Tightening Bounds With Data

Use sample dataI p(x), g0(x), g1(x) may be consistently estimated from the data by

F Sample meansF Nonparametric smoothing methodsF Parametric methods

I Recall

∆ATE (x) = E[Y (1)− Y (0)|x ]= g1(x)− E[Y (0)|x ,D = 1]p(x)

+ E[Y (1)|x ,D = 0]− g0(x)[1− p(x)]


New bounds with sample data (worst case bounds)I ∆ATE (x)

BLATE = g1(x)− U0p(x) + L1 − g0(x)[1− p(x)]

BUATE = g1(x)− L0p(x) + U1 − g0(x)[1− p(x)]wATE = (U1 − L1)[1− p(x)] + (U0 − L0)p(x)

I ∆ATT (x)

BLATT = g1(x)− U0BUATT = g1(x)− L0wATT = U0 − L0

I ∆ATU (x)

BLATU = L1 − g0(x)

BUATU = U1 − g0(x)wATU = U1 − L1


Example: y is binary ⇒ wk = 1 ∀k (sample data cuts width in half)Note: Bounds necessarily include zero

I Cannot rule out zero average treatment effectI Can exclude some extreme valuesI Kreider, Pepper, and co-authors incorporate measurement error in Dinto the bounds (discussed later)


Tightening Bounds With Assumptions

Assume ∆ATT (x) = ∆ATU (x)I Calculate bounds for ∆ATT (x) and ∆ATU (x)I New bounds include only the intersection of the two boundsI Example

∆ATT (x) ∈ [−0.25, 0.75]∆ATU (x) ∈ [−0.75, 0.25]

then new bounds are [−0.25, 0.25]I Note: always includes zero since bounds on ∆ATT (x), ∆ATU (x) bothinclude zero


Level-set restrictions: treatment effects are constant ∀x ∈ X0 ⊆ X(the support of x)

I Calculate bounds for ∆k (x) ∀x ∈ X0I New bounds include only the intersection of these boundsI Example (∆ATE )

∆ATE (xa) ∈ [−0.25, 0.75]∆ATE (xb) ∈ [−0.75, 0.25]

where xa, xb ∈ X0, then new bounds are [−0.25, 0.25]I Note: always includes zero since bounds on ∆k (x) include zero ∀xI Formally

BLk (X0) = supx∈X0

BLk (x)

BUk (X0) = infx∈X0

BUk (x)

wk (X0) = BUk (X0)− BLk (X0)


Level-set restrictions: expected outcomes are constant∀x ∈ X0,1 ⊆ X (for Y (1)) and ∀x ∈ X0,0 ⊆ X (for Y (0))

I Implies

E[Y (1)|x ] is constant ∀x ∈ X0,1E[Y (0)|x ] is constant ∀x ∈ X0,0


⇒ Bounds become

BLATE (x0) = supx∈X0,1

g1(x)p(x) + L1[1− p(x)]

− infx∈X0,0

g0(x)[1− p(x)] + U0p(x)

BUATE (x0) = infx∈X0,1

g1(x)p(x) + U1[1− p(x)]

− supx∈X0,0

g0(x)[1− p(x)] + L0p(x)

BLATT (x0) = supx∈X0,1

g1(x) − infx∈X0,0

U0

BUATT (x0) = infx∈X0,1

g1(x) − supx∈X0,0

L0

BLATU (x0) = supx∈X0,1

L1 − infx∈X0,0

g0(x)

BUATU (x0) = infx∈X0,1

U1 − supx∈X0,0

g0(x)

where x0 ∈ X0,1 ∩ X0,0DL Millimet (SMU) ECO 7377 Spring 2020 251 / 478

Assumption: positive selectionI Implies

E[Y (1)|x ,D = 1] > E[Y (0)|x ,D = 1]which means that the treated only join the treatment group if there arenon-negative gains on average

I Bounds become

BLATE = L1 − g0(x)[1− p(x)]

BUATE = g1(x)− L0p(x) + U1 − g0(x)[1− p(x)]BLATT = 0

BUATT = g1(x)− L0

I Does not affect bounds on ∆ATU (x)


Combining assumptions, restrictions

BLk ,combine = maxp∈ΨBLk ,p

BUk ,combine = minp∈ΨBUk ,p

where Ψ is the set of restrictions being combined


Manski (1990), Manski & Pepper (2000) consider additionalassumptions

1 InstrumentE[Y (d)|z ] = E[Y (d)], d = 0, 1

2 Monotone Instrument

z1 ≤ z ≤ z2 ⇒ E[Y (d)|Z = z1 ] ≤ E[Y (d)|Z = z ] ≤ E[Y (d)|Z = z2 ], d = 0, 1

3 Monotone Treatment Selection

E[Y (d)|D = 1] ≥ E[Y (d)|D = 0], d = 0, 1

4 Monotone Treatment Response

Y (0) ≤ Y (1)⇒ E[Y (0)] ≤ E[Y (1)]

where x is omitted for notational convenience


Use of an instrumentI E[yd |z ] = E[yd ], d = 0, 1, implies

E[Y (d )] ∈[supzE[y |D = d ,Z = z ]Pr(D = d |Z = z ) + Ld Pr(D 6= d |Z = z ),

infzE[y |D = d ,Z = z ]Pr(D = d |Z = z ) + Ud Pr(D 6= d |Z = z )

]

I Bounds for ∆ATE become

BLATE = supzg1(z)p(z) + L1 [1− p(z)]

− infzg0(z)[1− p(z)] + U0p(z)

BUATE = infzg1(z)p(z) + U1 [1− p(z)]

− supzg0(z)[1− p(z)] + L0p(z)

I Bounds are tighter than worst case bounds if p(z) 6= Pr(D = 1); i.e., zis correlated with treatment assignment


Use of a monotone instrument (MIV)I z1 ≤ z ≤ z2 ⇒ E[Y (d)|Z = z1 ] ≤ E[Y (d)|Z = z ] ≤ E[Y (d)|Z =z2 ], d = 0, 1

F Weaker assumption than the prior, mean independence assumptionF Implies that potential outcomes are non-decreasing in z

I Implies

E[Y (d)] ∈[

∑z∈Z

Pr(Z = z)

supz1≤zE[y |D = d ,Z = z1 ]Pr(D = d |Z = z1)

+ Ld Pr(D 6= d |Z = z1)

,

∑z∈Z

Pr(Z = z)

infz2≥zE[y |D = d ,Z = z2 ]Pr(D = d |Z = z2)

+ Ud Pr(D 6= d |Z = z2)

]I Bounds derived based on this


Monotone treatment selection (MTS)I E[Y (d)|D = 1] ≥ E[Y (d)|D = 0], d = 0, 1, implies that the treatedgroup has weakly higher potential outcomes in all treatment states

I Plausible in certain cases when one does not condition on x and x iscorrelated with both D and Y (d) in the same direction

I Implies

E[Y (d)] ∈ [E[y |D = d ]Pr(D ≥ d) + Lj Pr(D < d),E[y |D = d ]Pr(D ≤ d) + Uj Pr(D > d)]

Monotone treatment response (MTR)I Y (0) ≤ Y (1)⇒ E[Y (0)] ≤ E[Y (1)] implies we know the sign of thetreatment effect (inclusive of zero)

I Implies ∆ATE ≥ 0I Stronger than the positive selection assumption previously as that onlyapplied to the sub-sample with D = 1

MIV can be combined with MTS, MTR


Notes

Finite sample propertiesI Plug-in estimators involving min,max, inf, sup are can be severelybiased

I Bootstrap bias-correction proposed in Kreider & Pepper (2008)I Alternative approach suggested in Chernozhukov et al. (2013), but notobvious

InferenceI Typical bootstrap yields confidence intervals for the bounds, not thetreatment effect

F For example, a 90% CI implies that the probability that the ‘true’bounds lie in the CI is 90%; the probability that the ‘true’treatmenteffect lies in the CI is even higher

I Bootstrap based on Imbens & Manski (2004) is usual approachI Much recent work in this area

Stata: -tebounds-, -bpbounds-, -clrbounds-


Selection on UnobservablesSensitivity to SOU

Altonji et al. (2005) offer two approaches to assess the sensitivity ofestimates obtained under SOO assumption when this assumption isfalse

I Approach #1 is applicable to the case of a binary outcomeI Approach #2 is applicable regardless of type of outcome

Oster (2017) builds on Altonji et al. (2005)

Krauth (2016) offers a related method


AET Approach #1: Bivariate probit model

Model

y ∗i = xi β+ ∆Di + εi

D∗i = xiγ+ µi

where ε, µ ∼N(0, 0, 1, 1, ρ) and

y =

1 if y ∗ > 00 otherwise

D =

1 if D∗ > 00 otherwise


Estimation by ML

lnL = ∑i :y=1,D=1 ln[Φ2(xi β+ ∆, xiγ, ρ)]+∑i :y=1,D=0 ln[Φ2(xi β,−xiγ,−ρ)]

+∑i :y=0,D=1 ln[Φ2(−xi β− ∆, xiγ,−ρ)]

+∑i :y=0,D=0 ln[Φ2(−xi β,−xiγ, ρ)]

Model is technically identified with no exclusion restriction, but treatρ as unidentified

Assessing treatment effect as ρ varies provides evidence of sensitivityto selection on unobservables

Constrain ρ > 0⇒ positive selection; ρ < 0⇒ negative selection;ρ = 0⇒ no selection on unobservables

Stata: -biprobit- (with -constraints-)


Approach #2: SOU relative to SOO

Intuition is to assess how much SOU, relative to the amount of SOO,is needed to fully explain the observed positive association between Dand y

If

(AET.i) Random observables: x is a random subset of all factors, w , influencingy

(AET.ii) Equally important factors: the number of elements in w is large and nosingle variable factor has an undue influence on y

(AET.iii) Relationship between x and unobservables: slightly weaker technicalassumption than independence between x and remaining elements of w

then one should expect the amount of selection controlled for by x toequal the amount of selection on unobservables

Implies that if the amount of SOU needed to explain the observedassociation is less than amount of SOO, the estimated treatmenteffect should not be viewed as robust


Model for outcomeyi = xi β+ ∆Di + εi

The (normalized) amount of SOU is given by

E[ε|D = 1]− E[ε|D = 0]Var(ε)

The (normalized) amount of SOO — ignoring the impact of D — isgiven by

E[xβ|D = 1]− E[xβ|D = 0]Var(xβ)

The goal is to assess how large SOU must be relative to SOO to fullyaccount for the positive treatment effect estimated under exogeneity


Express actual treatment participation as

Di = xiγ+ µi

plim of OLS estimator of ∆ is

plim ∆ = ∆+Cov(µ, ε)

Var(µ)

= ∆+Var(D)Var(µ)

E[ε|D = 1]− E[ε|D = 0]

Under the assumption that SOO = SOU, the asymptotic bias term is

Cov(µ, ε)Var(µ)

=Var(D)Var(µ)

E[xβ|D = 1]− E[xβ|D = 0]

Var(xβ)Var(ε)


This bias can be consistently estimated under Ho : ∆ = 0The ratio ∆/bias indicates how much larger SOU needs to be relativeto SOO to entirely explain the treatment effect

A small ratio ⇒ treatment effect is highly sensitive to selection onunobservables; a ratio >> 1 implies treatment effect is robust

Algorithm:1 Estimate Var(D) from sample2 Estimate treatment eqtn via LPM ⇒ Var(µ)

3 Estimate outcome eqtn via OLS restricting ∆ = 0 ⇒ x β, Var(x β),

Var(ε)4 Obtain sample means of x β in treatment and control groups ⇒

E[x β|D = 1], E[x β|D = 0]5 Estimate outcome eqtn via OLS ⇒ ∆6 Compute ratio of ∆/bias


Notes:I If y is binary, estimate treatment eqtn via probit perhaps in step 3 ⇒

Var(ε) = 1I AET methods have relatively little to say about economic significanceof treatment effect unless one makes assumptions about actual amountof SOU


Oster Approach

Limitations of AET1 AET’s test in linear model (Approach 2) is only valid under the null ofzero treatment effect. Thus, a bias-corrected estimator is not provided.

2 AET assume that observation of all unobservables produces an R2 = 1.May want to allow for R2 < 1 if some variation in y is due tomeasurement error.

Goal: Obtain an unbiased (bias-adjusted) treatment effect estimate inthe presence of observed and unobserved confounders


SetupI DGP

y = ∆D +W1 +W2 + ε,

where

F x = observed covariatesF W1 = xo β = index of observed covariatesF W2 = xuγ = index of unobserved covariates

I Cov(W1,W2) = 0I Proportional selection implies

δCov(W1,D)

σ2W1

=Cov(W2,D)

σ2W2


Notation

I Short regression:∆,R is the OLS estimate and R2 from the regression

of y on DI Intermediate regression: ∆, β, R is the OLS estimate and R2 from theregression of y on D, xo

I Rmax is the R2 from the regression of y on D, xo ,W2I µ is the vector of OLS coeffi cients from the regression of D on xoI ψ is the vector of OLS coeffi cients from the regression of y on D, xo

Oster considers a restricted and unrestricted bias-corrected approach


Restricted EstimatorI Assumptions

(O.i) δ = 1, soCov(W1,D)

σ2W1

=Cov(W2,D)

σ2W2

(O.ii) ψk/ψk ′ = µk/µk ′ ∀k , k ′F (O.i) implies equal selection on observables and unobservablesF (O.ii) implies that the OLS estimate of ∆ is identical in the regression ofy on D , xo as of y on D ,W1. It requires that the relative contributionof each x on D is the same as its relative contribution on y .

I Under (O.i)-(O.ii), consistent bias-corrected estimator given by

plim ∆BC = ∆−(

∆− ∆)[

Rmax − R

R −R

]→ ∆

I Under (O.ii), an approximate bias-corrected estimator given by

plim ∆BC = ∆− δ

( ∆− ∆

)[Rmax − R

R −R

]→ ∆


Unrestricted EstimatorI Two approaches

1 With the results from the short and intermediate regression andimposing a value for Rmax, can solve for the unique value of δ s.t.∆ = ∆o (e.g., ∆o = 0).

2 Under (O.i), imposing a value for Rmax, and adding the followingassumption

(O.iii) sgn(Cov(D , W1)) = sgn(Cov(D ,W1))

can compute a consistent, bias-corrected estimate, ∆BC , where (O.iii)implies that the selection on unobservables is not so severe so as tobias the direction of the covariance between the observed index and D

I Under Approach 1, δ is the degree of SOU relative to SOO necessaryto explain the intermediate regression estimate of ∆

I Under Approach 2, without (O.i), can bound ∆ based on possiblevalues of δ and Rmax

Stata: -psacalc-


Krauth Approach

SetupI DGP

y = ∆D + xβ+ ε

I Relative correlation restriction implies

Corr(D, ε) = λ Corr(D, xβ),

where λ ∈ [λL,λH ]I Interpretation of λ:

F λ = 0⇒ Corr(D , ε) = 0 (OLS is unbiased)F λ = 1⇒ Proportional selection

I Confidence intervals on bounds use Imbens-Manski (2004)


Stata: -rcr

⇒ bounds with λ ∈ [0, 1] are [0.24, 0.33] with CI [0.21, 0.39]⇒ λ = −2.69 is needed for ∆ ≈ 0 as there is negative selection into D


Selection on UnobservablesPanel Data

Panel data is used in to address selection on unobservables in twocommon ways

1 Replacing covariates with their lag

F Not recommended, not discussed hereF See Reed (2015), Bellemare et al. (2017)

2 Incorporating fixed effects

F Controls for unobservables that are invariant along a certain dimension

Thus, at best, panel data methods provide a solution to selection onunobservables in only certain situations


Standard panel model

yit = ci + λt + xitβ+ εit

I Individual effect is time invariantI Time effect is individual invariant

Estimation techniquesI Least squares dummy variable model (LSDV)I Fixed effects (FE)I First-differencing (FD)I Long-differencing (LD)

Stata: -xtreg-


Selection on UnobservablesPanel Data: Treatment Effects Models

Structural model

yit = ci + λt + xitβ+ τDit + εit , i = 1, ...,N; t = 1, ...,T

where τ is now used for the treatment effect to avoid confusion withthe first-difference operator

Estimating equation arises by assuming the following functional formsfor potential outcomes

yit = DitYit (1) + (1−Dit )Yit (0)Yit (1) = ci + λt + xitβ+ τ + υ1it

Yit (0) = ci + λt + xitβ+ υ0it


Special caseI Setup

F T = 2F Di1 = 0 ∀iF Di2 ∈ 0, 1 ∀iF Assume no x’s

I FE or FD estimation ⇒

τ = E[∆y |D2 = 1]− E[∆y |D2 = 0]

I Known as difference-in-differences estimator


Visual representation of special case

yit = ci + λt + τDit + εit

I Expected outcomes by period and treatment status

t = 1 t = 2D = 0 c0 + λ1 c0 + λ2D = 1 c1 + λ1 c1 + λ2 + τ

I Implies

E[∆y |D2 = 1] = (c1 + λ2 + τ)− (c1 + λ1) = τ + λ2 − λ1

E[∆y |D2 = 0] = (c0 + λ2)− (c0 + λ1) = λ2 − λ1

which implies

τ = E[∆y |D2 = 1]− E[∆y |D2 = 0]


Illustration #1


Illustration #2


01

23

Y

1 2Time

Controls Est. MC: BeforeAfterTreated Est. MC: CrossSection

Est. MC: DID


Identifying assumptions

(DID.i) No anticipation

E[Y1(0)|D2 = 1] = E[Y1(1)|D2 = 1]

where Y1(d) is the potential outcome associated with d in thepre-treatment period

F Implies that the existence of the program does not change thepre-treatment behavior of the to-be-treated

(DID.ii) Parallel trends

E[Y2(0)− Y1(0)|D2 = 1] = E[Y2(0)− Y1(0)|D2 = 0]

F Implies that the average outcomes for the treated should have the sameevolution over time as the average outcomes for the controls in theansence of the program

Under these assumptions, DID identifies the ATT if treatment effectsare heterogeneous


Beyond the special caseI Special case is useful to gain the intuition, not requiredI In general, as long as Dit is time-varying for some units i , then τ canbe estimated by any panel data method given the required assumptionsare met

I Important: FE/FD is not a magic bullet (Duflo et al. 2004)F FE and FD require strict exogeneity ; rules out Ashenfelter’s Dip ⇒

Cov(Dit , εit−1) 6= 0F Rules out selection on contemporaneous shocks ⇒ Cov(Dit , εit ) 6= 0F Diff-in-diff-in-diff may be an option


DID has been the subject of much research in the past few yearsI DID matching: enables relaxation of functional form assumptions(discussed previously)

I DID with multiple treatments: Fricke (2017)I DID in LDV models: Puhani (2012)I DID with variation in treatment intensity across groups: deChaisemartin and D’HaultfŒ uille (2018) ... “fuzzy DID”

I DID with multiple periods and variation in treatment timing: Abrahamand Sun (2018), Callaway & Sant’Anna (2018), de Chaisemartin andD’HaultfŒ uille (2018), Goodman-Bacon (2018)

F Stata: -net install ddtiming, from(https://tgoldring.com/code/)-

I DID with pooled cross-section data only: Botosaru & Gutierrez (2018)I DID with double robustness property: Sant’Anna & Zhao (2018)I DID sensitivity to time-varying confounders (Keele et al. 2019)I Synthetic DID (Arkhangelsky et al. 2019)


Timing issues (LaPorte & Windmeijer 2005)

Previous model restricts D to a one-time intercept shift, τ

In certain applications, agent may anticipate treatment and alterbehavior prior to actual treatment; or, response may occur with a lag;or, some combination of both

I Examples: policy changes announced, but not implemented until futuredate; or, lags in adjustment to policy changes

General structural model

yit = ci + λt + xitβ+∑L0l=1 δ−lD

−lit + δ0Dit +∑L1

l=1 δlDlit + εit

where

D−lit = Dit+l (treatment assignment l periods in future)

D lit = Dit−l (treatment assignment l periods in past)

I δ−l reflects anticipatory effects of treatmentI δl reflects lagged effects of treatmentI δ0 reflects instantaneous effects of treatment


Similar model to those used in event studies

Also related to literature on dynamic treatment effects (e.g., Lee andHuang 2011)


Comparative Case Study Approach (Synthetic Control Approach)

Provides an alternative to DID whenI Treatment occurs at an aggregate levelI Typically only a single observation is treated and lengthy history ofpre-treatment data are availble for the treated and the pool of controls

Examples:I Mariel Cuban Boat Lift (Card 1990; Peri & Vasil Yasenov 2018)I State minimum wage (Card & Krueger 1994)

SolutionI Construct a synthetic control which is a weighted average of availablecontrols to estimate the missing counterfactual in post-treatmentperiod(s)

I Weights are chosen by matching pre-treatment covariates and outcomesI Allows for differential time trends in treatment and control observations

F By matching pre-treatment outcomes, one is implicitly matching on thetime-invariant unobserved effect

F Thus, does not matter if unobserved effect has differential effects overtime if the time-specific effect is a common factor


ModelI yit is observed outcome for obs i , i = 1, ..., J + 1, in periodt = 1, ...,To , ...,T

I Obs 1 is treated; remaining 2, ..., J + 1 are never treatedI Timing of treatment effects

1 No Anticipatory Effects: To is period prior to obs 1 being treated2 Anticipatory Effects: To is period prior to any anticipatory effects forobs 1 begining

I Outcomes in the absence of treatment

yit = yNit = δt + θtZi + λtui + εit

I Outcomes with treatment

yit = yIit = y

Nit + αit


Synthetic control is defined as

∑J+1j=2 ωjyjt = ∑J+1

j=2 ωj (δt + θtZi + λtui + εit )

where ωj is the weight given to control j and

I ∑J+1j=2 ωj = 1I ωj ≥ 0 ∀j

Conditional on choice of weights, ω∗j , period-specific treatment effectis estimated as

αit = y1t −∑J+1j=2 ω∗j yjt , t > To

Requires a SUTVA-type assumption that the treatment does notimpact outcomes in the controls


Weights are chosen to match moments of the data in periods t ≤ ToI Define

yKi = ∑Tos=1 ksyis

where K = (k1, ..., kTo ) is a vector of weights and thus yKi represents

a particular linear combination of pre-treatment outcomes for obs iI Given M unique linear combinations, define the vector of pre-treatmentoutcomes for obs 1 as

X1 = (Z′1, y

K11 , ..., y

KM1 )

with dimension R × 1I Define the R × J matrix of variables for the remaining obs i ,i = 2, ..., J + 1 as X0, where column j is given by

(Z ′j−1, yK1j−1, ..., y

KMj−1)

I Weights, W , are chosen to minimize some distance function

||X1 − X0W ||V =√(X1 − X0W )′V (X1 − X0W )

where V is a R × R symmetric, positive semidefinite matrixDL Millimet (SMU) ECO 7377 Spring 2020 297 / 478

IntuitionI X1 and X0 are pre-treatment attributes — that may include covariatesand/or outcomes —of the to-be-treated unit and the pool of controls

I V allows the attributes to be weighted differently

F In practice, V is chosen to minimize the MSE of the pre-interventionpredictions

I Weights are chosen to minimize the difference between thepre-treatment attributes for the to-be-treated unit and a weightedaverage of the controls

I Weighted average of the controls constitutes the synthetic control


Example: Abadie et al. (2010)


Inference is handled byI Re-doing the analysis, treating obs i , i = 2, ..., J + 1, as ‘treated’afterperiod To and the remaining obs as the pool of potential controls

I This yields a dbn of treatment effect estimates under Ho of notreatment effect

I If actual estimates of α1t look very different, this is evidence of astatistically meaningful treatment effect


Doudchenko & Imbens (NBER WP 2016) propose an extensionwhereby the weights for the synthetic control are not restricted to benon-negative or sum to one

I Negative weights reduce bias (more exact match with the syntheticcontrol) but requires extrapolation outside the convex hull created bythe controls

Hsiao et al. (2012) offers a similar approach to the synthetic controlmethod

I Essentially chooses synthetic control only using pre-treatmentoutcomes, yNit , t = 1, ...,T0, and sets V to the identity matrix

I Gardeazabal & Vega-Bayo (2017) compare the two approaches

Xu (2017) proposes a generalized synthetic control method

Ben-Michael et al. (2018) propose an augmented synthetic controlmethod that bias corrects for the inexact match of the syntheticcontrol in the pre-treatment period

Stata: -synth-


Selection on UnobservablesInstrumental Variables

Refer to ECO 6374 for refresher on basics...

TerminologyI ‘Structural’model

yi = β0 + β1xi + εi

I First-stage modelxi = π0 + π1zi + ui

I Reduced form model

yi = (β0 + β1π0) + β1π1zi + (εi + β1ui )

= π0 + π1zi + εi


Goal: devise alternative estimation technique to obtain consistentestimates when E[ε|x ] 6= 0

I Solution: identify β from exogenous variation in x isolated usinginstruments, z

I z is a valid IV for x iff

(IV.i) First-stage: E[z ′x ] 6= 0(IV.ii) Exogeneity: E[z ′ε] = 0(IV.iii) Exclusion: E[y |x , z ] = E[y |x ]

where z and x are both N ×K matricesI Exogenous x’s serve as instruments for themselvesI Need unique instrument for each endogenous var

Stata: -ivregress-, ivreg2-, -xtivreg2-


Several issues remain under scrutiny in the literature1 Choice of estimation technique2 Properties and inference with weak IVs ⇒ E[z ′x ] ≈ 03 Properties and inference with endogenous IVs ⇒ E[z ′ε] 6= 04 Interpretation

Estimators1 IV2 Two-Stage Least Squares (TSLS or 2SLS)3 GMM4 LIML5 etc., etc.


Selection on UnobservablesEstimators

IV estimatorI Given by

βIV = (z′x)−1z ′y

TSLS estimatorI Allows for overidentified modelsI Given by

βTSLS = (x′x)−1 x ′y = [x ′z(z ′z)−1z ′x ]−1x ′z(z ′z)−1z ′y

where x is fitted value from first-stage

GMM estimatorI Allows for overidentified modelsI Based on moment conditions of the form E[z ′i εi ] = 0


Notes

Estimators are CAN, but biasedI Moments of the estimator only exist up to the degree ofoveridentification

F Implies mean and variance only exist if model is overidentifed by two

I Finite sample performance can be very poor

F Standard errors based on asymptotic variance can be very misleadingF Young (2018 WP), “Consistency without Inference: InstrumentalVariables in Practical Application”

Incorrectly treating other covariates in the model as exogenous ⇒inconsistent estimates if instrument(s) are correlated with thesecovariates


Selection on UnobservablesIV: Specification Tests

Much specification testing is required when utilizing IV in appliedresearch

Types of tests available

I Tests of endogeneity: E[x ′ε]?= 0

I Tests of overidentification: E[z ′ε]?= 0 (partial test only)

I Tests of instrument relevance: E[z ′x ]?= 0

I Tests for weak instruments:E[z ′x ] ≈ 0

Covered in ECO 6374


Selection on UnobservablesIV: Imperfect Instruments

Recent work has explored what can be learned if z is an imperfectinstrumental variable (IIV)Two possible imperfections:

1 z is also endogenous2 z is not excludable from the second-stage

Nevo & Rosen (2010), Ashley (2009), Ashley and Parmeter (2015),Kiviet and Niemczyk (2015) address endogeneityConley et al. (2010), van Kippersluis & Rietveld (2018) addressexcludabilitySmall (2007) explores endogeneity with multiple endogenouscovariates

Note: These are intimately related since if z is incorrectly treated asexcludable, then it will be correlated with the second-stage compositeerror that now includes the error and zDL Millimet (SMU) ECO 7377 Spring 2020 310 / 478

Conley et al. (2010)Setup

yi = xi β+ ziγ+ εi

xi = ziπ + ui

where x is a kx -dimensional vector of endogenous regressors, z is akz -dimensional vector of instruments, kz ≥ kx , and E[z ′ε] = 0Classical IV requires the assumption that γ = 0

I With kx = kz = 1, we have

plim βIV = β+σz ε

σxz= β+

γσ2zπσ2z

= β+γ

π

where ε = ziγ+ εi is the composite errorI Thus, IV is asymptotically biased when γ 6= 0 and the bias isdecreasing in π and increasing in γ

I Authors refer to deviations from γ = 0 as plausible exogeneity

Approach

I Track estimates β(γ) = βIV − γ/π for different values of γ

I Estimates will be more sensitive to γ the weaker the instrument


Authors present several possible methods of inference, only somepresented here

Method #1. Union of CIs with γ Support AssumptionI Suppose the true value of γ = γ0 ⊂ Gkz , with known boundsI If γ0 were known, then IV/TSLS applied to

yi − ziγ0 = xi β+ εi

using z as instruments is consistent for βI With γ0 unknown, but contained in Gkz , one can

F Apply IV/TSLS to a grid of values for γ from Gkz

F For each value, γs , s = 1, ...,S , obtain the (1− α)% CI for βF Compute a final CI as the union of these S CIs

CI (1− α) = ∪γ∈Gkz CI (1− α,γ)

which has an asymptotic coverage probability ≥ 1− αF If some prior info, may want to weight different γ’s differently


Method #2. γ Local-to-Zero ApproximationI γ is treated as unknown, but coming from a known dbnI Assuming γ ∼N(µγ,Ωγ) leads to the following approximate dbn

β ∼N(β+ Aµγ,VIV + AΩγA′)

where A = (x ′z(z ′z)−1z ′x)−1x ′zI If µγ = 0, then this approach simply leads to a revised variance for theIV/TSLS estimator

Stata: -plausexog-


Nevo & Rosen (2012)

SetupI Model given by

yi = βxi + wi δ+ εi

where x is a single endogenous regressor, w is exogenous (oralternatively are endogenous with valid instruments), and z is 1× kzvector of imperfect instruments for x

I z is an imperfect IV (IIV) in the sense that it is also correlated with εI Assumptions:

(IIV.i) Sign of correlation: ρx ερzj ε ≥ 0, j = 1, ..., kz(IIV.ii) Degree of endogeneity: |ρx ε| ≥ |ρzj ε|, j = 1, ..., kz(IIV.iii) True model: yi = βxi + wi δ+ εi

(IIV.ii) contrasts with the classical IV assumption that ρzj ε = 0


Define

λ∗j =ρzj ε

ρx ε

which is in the unit interval under (IIV.i), (IIV.ii)

If λ∗j were known, then a valid IV for x is

Vj (λj ) = σx zj − λ∗j σzj x

However, Λ∗ = [λ∗1 · · · λ∗kz ] is unknown, but lies in the unit cube inRkz -space

Intuitively, searching over feasible values of Λ∗, one may bound β


Consider kz = 1I Partial out the effects of w by defining (recall FWL Theorem)

yi = yi − wi [(w ′w)−1w ′y ]xi = xi − wi [(w ′w)−1w ′x ]

I Under (IIV.i) — (IIV.iii) and assuming without loss of generality thatρx ε ≥ 0, obtain the following bounds:

F Case I. (σz xσx − σx xσz )σz x > 0

β ∈

[

βIVV (1), βIVz

]if σz x < 0[

βIVz , βIVV (1)

]if σz x > 0

F Case II. (σz xσx − σx xσz )σz x ≤ 0

β ∈

[max

βIVz , βIVV (1)

,∞) if σz x < 0

(−∞,min

βIVz , βIVV (1)

] if σz x > 0

I Additional work to bound δ is also possible


Extension to kz > 1I Bounds can be tightened by obtaining bounds for each z individuallyand then computing the final bounds as the intersection of the kzbounds

I Formally

F For each zj , obtain B∗j =[

βlj , βuj

]F Final bounds given by

β ∈[maxj

βlj

,minj

βuj

]F In Case II, these bounds are one-sided; one trick may be to try anddefine a new IV that is a weighted average of two of the IVs such that(σqxσx − σx xσq )σqx > 0, where qi = γzji + (1− γ)zj ′ i

I Need to be careful, though, and make sure different z’s estimate thesame parameter (discussed later)

Stata: -imperfectiv-


ncorr option because I assume Cov(unionh, ε) < 0


Selection on UnobservablesIV: Heterogenous Treatment Effects

Assume a binary endogenous regressor, D, and a binary instrument, z

Allow for the possibility that the treatment effect may vary across byi and agents may act on observation-specific gains, υ1i − υ0i , whenmaking treatment decision

Admitting this possibility implies that one must think more carefullyabout what parameter one is estimating

Stata: -etregress-


Linear Model

Setup

Yi (1) = α1 + xi β+ υ1i

Yi (0) = α0 + xi β+ υ0i

yi ≡ Yi (0) +Di (Yi (1)− Yi (0))

Treatment effects

∆i = Yi (1)− Yi (0)= (α1 − α0) + (υ1i − υ0i ) heteogeneous

= (α1 − α0) homogeneous (iff υ1i = υ0i )


Substitution yields

yi ≡ Yi (0) +Di (Yi (1)− Yi (0))= α0 + xi β+ υ0i +Di (α1 + xi β+ υ1i − α0 − xi β− υ0i )

= α0 + xi β+ (α1 − α0 + υ1i − υ0i )Di + υ0i

≡ xi β+ ∆iDi + εi

which is a random coeffi cients model

Convert to something potentially estimableI Define ∆i = (α1 − α0) + (υ1i − υ0i ) ≡ ∆+ ∆∗iI Substitution implies

yi = xi β+ ∆Di + (∆∗i Di + εi )

where ∆∗i Di + εi is the composite error term


A valid IV in the homogeneous treatment effects setup requires

E[εi |xi ,Di , zi ] = E[εi |xi ,Di ]

and in the heterogeneous setup requires

E[∆∗i Di + εi |xi ,Di , zi ] = E[∆∗i Di + εi |xi ,Di ]

is required

Thus, z must beI Correlated with Di (as usual)I Uncorrelated with the error term from the structural model andindividual-specific gains (or losses) from treatment

F Not possible unless (i) ∆∗i = 0 ∀i (implying a constant treatmenteffect) or (ii) ∆∗i ⊥ Di |xi (implying that agents either do not know ordo not act on specific gains ... no essential heterogeneity)

F Model with ∆∗i and Di correlated known as Correlated RandomCoeffi cients (CRC) model


Much more restrictive requirementI Example: if z is an exogenous variable representing the cost ofparticipation in the treatment (e.g., distance to job training center),then high z will lead to no participation unless the benefit fromparticipation, ∆∗i , is very high; if z is low, one will participate if ∆∗i islow or high ⇒ positive correlation between z and ∆∗i conditional on Di

If z is uncorrelated with ε, but correlated with ∆∗i , then IV estimatesare still useful, but identify a different parameter

Parameter known as local average treatment effect (LATE)


Formally, given the model (ignoring x)

yi = α+ ∆Di + (∆∗i Di + εi )

and an instrument, z , we have

plim ∆OLS =Cov(y ,D)

Var(D)= ∆+

Cov(ε,D) +Cov(∆∗D,D)Var(D)

6= ∆

plim ∆IV =Cov(y , z)Cov(D, z)

= ∆+Cov(ε, z) +Cov(∆∗D, z)

Cov(D, z)

= ∆+Cov(∆∗D, z)

Cov(D, z)6= ∆

where the last inequality holds unless (i) ∆∗i = 0 ∀i or (ii) ∆∗i ⊥ Di |xi(as stated above)

How do we interpret ∆IV ?


LATE

Assume a binary endogenous regressor, D, and a binary instrument,z , and no other covariates (for simplicity)

Four potential subpopulations

z = 0 z = 1Never Takers (NT) D = 0 D = 0Defiers (DF) D = 1 D = 0Compliers (C) D = 0 D = 1Always Takers (AT) D = 1 D = 1

Compliers and defiers are the key, as their treatment status varieswith the instrument


Assumptions

(LATE.i) Independence: Y (0),Y (1),D(0),D(1) ⊥ z , where D(j), j = 0, 1,are potential treatment assignments

(LATE.ii) Exclusion: E[Y (0)|z ] = E[Y (0)]; E[Y (1)|z ] = E[Y (1)](LATE.iii) First-Stage/Compliers: Pr(C ) > 0⇒ Pr(D = 1|z) is a non-trivial

function of z(LATE.iv) Monotonicity: Pr(Di = 1|zi = 1) > Pr(Di = 1|zi = 0) ∀i ⇒

Pr(DF ) = 0

Imposing these assumptions ⇒

∆IV = ∆LATE = E[Y (1)− Y (0)|C ]

which is interpretable


Notes

LATE is a well-defined economic parameter that applies to a specificsubpopulation

I The relevant subpopulation is unique to the instrument being used

F Different instruments potentially identify different parameters!

I Whether LATE is an interesting parameter is debateable

Interpretation is similar, but derivation more complex, if D or z iscontinuous

Using multiple IVs potentially yields a weighted average of differentLATEs (Mogstad et al. 2019)

Recent literatureI Bolzern & Huber (2017) provide a straighforward test of instrumentvalidity in this context

I Mourifié & Wan (2017) discuss tests of monotonicity and instrumentvalidity

I Machado et al. (2018), Swanson et al. (2018) analyze what can belearned concerning the ATE based on the LATE


Not possible to know who are the compliers in the data, but cancount them

I Under (LATE.iv) we have

Pr(C ) = Pr[D(1) > D(0)] = E[D(1)−D(0)]= Pr(D = 1|z = 1)− Pr(D = 1|z = 0)

which is given by the first-stage relationshipI Proportion of compliers among the treated in the sample is

Pr(C |D = 1) = Pr(Z = 1)Pr(D = 1)

[Pr(D = 1|z = 1)− Pr(D = 1|z = 0)]

and among the controls is

Pr(C |D = 0) = Pr(Z = 0)Pr(D = 0)

[Pr(D = 1|z = 1)− Pr(D = 1|z = 0)]

using Bayes’Rule


Not possible to know who are the compliers in the data, but cancharacterize them

I Example: Are compliers more or less likely to be female?

Pr[x = F |C ]Pr[x = F ]

=Pr[C |x = F ]Pr[x = F ]/Pr(C )

Pr[x = F ]=Pr[C |x = F ]Pr(C )

=Pr(D = 1|z = 1, x = F )− Pr(D = 1|z = 0, x = F )

Pr(D = 1|z = 1)− Pr(D = 1|z = 0)

again using Bayes’RuleI Final line is the ratio of the first-stage for x = F to the full samplefirst-stage


Selection on UnobservablesIV: Fuzzy RD

Recall from sharp RD case that we require the existence of thefollowing limits

D+ = lims↓sPr(D = 1|s)

D− = lims↑sPr(D = 1|s)

and D+ 6= D−I Sharp RD setup implies D+ = 1 and D− = 0I Fuzzy RD setup implies 1 ≥ D+ > D− ≥ 0


Formally

(FRD.i) Treatment assignment is a discontinuous function of s (with a knownthreshhold, s)

Di = D(si , υi )

wherelims↑sPr(D = 1|s) 6= lim

s↓sPr(D = 1|s)

(FRD.ii) Positive density at the threshold: fS (s) > 0(FRD.iii) Outcomes are continuous in s at least around s and do not depend on

whether s ≷ s(FRD.iv) For each agent, the dbn of s is continuous at least around s


NotesI Endogenous treatment variable, D, depends on observed score variable,s, and stochastic element

I Discrete jump in Pr(D = 1) at sI Example: Pr(D = 1) = max0, 0.5s + 0.25 I(s > 0.5) + υ

0.2

.4.6

.81

0 .2 .4 .6 .8 1x

Pr(D

=1)

Implies Di = E[D |si ] + υi , where Cov(ε, υ) 6= 0DL Millimet (SMU) ECO 7377 Spring 2020 337 / 478

OLS estimation of

yi = xi β+ ∆Di + f (si ) + εi

where x is a vector of exogenous controls, is biased, even with aflexible function of s included

SolutionI Estimate propensity score, where f (s) is included along with the

indicator I(s > s) ⇒ p(D)I Estimate by OLS

yi = xi β+ ∆p(Di ) + f (si ) + εi

I Equivalent to TSLS, with I(s > s) as the instrument, when f (s) ischosen parametrically

Typical interpretation: RD identifies the LATE at s


Selection on UnobservablesIV: Heteroskedasticity-Based Instruments

Originally proposed as a solution to measurement error, butpotentially applicable to more general dependence between x and ε(Lewbel 1997, 2010)

SetupI ‘Structural’model

yi = β1Di + xi β2 + εi

I First-stage modelDi = xiπ + ui

where

F x includes the interceptF Cov(ε, u) 6= 0

D may be discrete or continuous


Potential instruments for D include (zi − z)ui , where z ⊆ xEstimation requires consistently estimating the first-stage andreplacing u with u

Validity of the IVs requires

(HM.i) E[z ′u2 ] 6= 0(HM.ii) E[z ′εu] = 0

Restrictions are satisfied if, say,

εi = θi + εi

ui = θi + ui

where θi is a homoskedastic common factor and the sole source ofcorrelation between ε and u, and u is heteroskedastic with variancedepending on z

Stata: -ivreg2h-


Selection on UnobservablesIV: Control Function Approaches

Recall from ECO 6374, TSLS can be recast as a control functionapproachSetup

I ‘Structural’modelyi = β1Di + xi β2 + εi

I First-stage modelDi = ziγ+ ui

Estimate by OLS

yi = β1Di + xi β2 + β3ui + εi

where u is the first-stage OLS residual ⇒ βOLS = βTSLS (butstandard errors are incorrect)

I Referred to as a control function approach since the inclusion of u‘controls’for the bias

I See Hausman (1978)


With no instrument, the first-stage is

Di = xiγ+ ui

implying that ui = Di − xi γThe second-stage now suffers from perfect multicollinearity

yi = β1Di + xi β2 + β3ui + εi


Two alternative control function approaches exist, both of whichtechnically do not require an exclusion restriction

They circumvent the perfect multicollinearity problem in differentways

Approaches1 Heckman Bivariate Normal Selection model

F Alters the estimator so that the control function is now a nonlinearfunction of D , x

2 Klein & Vella approach

F Alters the estimator so that the coeffi cient on the control function isnow observation-specific


Heckman Bivariate Normal Selection model

Places more structure on the problem to derive the control function

Addresses the binary nature of D

Two cases:1 Common effect2 Heterogenous effects

Difference is in whether there are unobserved gains to treatment,υ1 − υ0


Case #1: Treatment effects model with common effect

Setup

Yi (0) = xi β0 + υ0i

Yi (1) = xi β1 + υ1i

yi = DiYi (1) + (1−Di )Yi (0)D∗i = ziγ+ ui

Di =

1 if D∗i > 00 if D∗i 6 0

Common effect: εi ≡ υ1i = υ0i


NotesI εi = common error component (or common effect) in both potentialoutcome equations

I β’s allowed to differ across outcome equationsI D∗i = latent indicator of treatment statusI Model rules out selection on observables assumption sinceunobservables associated with treatment status, u, are correlated withunobservables affecting outcomes conditional on x

Assumptions

(BVN.i) ε, u ∼ N2(0, 0, σ2ε , σ2u , ρ)(BVN.ii) ε, u ⊥ x , z(BVN.iii) σ2u = 1


Parameters of interestI Given the setup, individual-specific treatment effect is given by

∆i = Yi (1)− Yi (0) = xi (β1 − β0)

I Average treatment effects are

∆ATE = E[∆i ] = E[xi ](β1 − β0)

∆ATT = E[∆i |Di = 1] = E[xi |Di = 1](β1 − β0)

∆ATU = E[∆i |Di = 0] = E[xi |Di = 0](β1 − β0)

I Implies consistent estimates of all three parameters require consistentestimates of β0, β1

I Two naïve options:

F Split sample into D = 1 and D = 0, and regress y on x via OLS ineach sub-sample

F Pool sample, regress y on x ,Dx

I Under selection on unobservables, ρ 6= 0, neither option producesconsistent estimates


Conditional expectations (following from the properties of conditionalnormal random variables)

I Of the outcome in the treated state for the treated

E[yi |Di = 1, xi , zi ] = xi β1 + E[εi |ui > −ziγ]

= xi β1 + ρσε

[φ(ziγ)Φ(ziγ)

]= xi β1 + ρσε [λ(−ziγ)]

where λ(·) is known as the Inverse Mills’RatioI Of the outcome in the untreated state for the untreated

E[yi |Di = 0, xi , zi ] = xi β0 + E[εi |ui 6 −ziγ]

= xi β0 + ρσε

[ −φ(ziγ)1−Φ(ziγ)

]I Given ρ 6= 0, error term is no longer well-behaved


EstimationI Several possible estimators, such as OLS estimation of

yi = xi β0 + xiDi (β1 − β0) + βλ1Di

[φ(ziγ)Φ(ziγ)

]+ βλ0(1−Di )


]+ ηi

or the restricted version

yi = xi β0 + xiDi (β1 − β0)

+ βλ

Di

[φ(ziγ)Φ(ziγ)

]+ (1−Di )


]+ ηi

where γ is replaced by estimates obtained via probitI The term, Di

[φ(ziγ)Φ(ziγ)

]+ (1−Di )

[−φ(ziγ)1−Φ(ziγ)

], is the generalized

residual of the probit model (Gourieroux et al. 1987)

Simultaneous estimation via ML is also possible


Upon estimation of β0, β1 ...

I Can predict Yi (0), Yi (1) ∀iI Estimate treatment effect parameters

∆ATE = x(β1 − β0)

∆ATT = x1(β1 − β0)

∆ATU = x0(β1 − β0)

where x is the sample mean, and xd , d = 0, 1, is the sample mean inthe sub-sample with D = d


Shortcut for ATE


CommentsI Technically no instrument, z , is required for identification

F Identification arises from the non-linearity of the selection correctionterms, which in turn arises from the assumption of bivariate normality

F Performance in such cases depends crtically on the model beingcorrectly specified and is therefore not recommended

I Semi-parametric versions exist

F Relaxes dependence on bivariate normalityF Require exclusion restrictionsF One version includes a polynomial of the propensity score in theregression model; motivation is to include a flexible functional form tocapture the selection terms without reliance on bivariate normality

See Vella & Verbeek (1999)


Case #2: Treatment effects model without a common effect

Relaxation of common effect assumption allows for heterogeneouseffects of the treatment even conditional on x

Setup

Yi (0) = xi β0 + υ0i

Yi (1) = xi β1 + υ1i

= xi β1 + [(υ1i − υ0i ) + υ0i ]

= xi β1 + [δi + υ0i ]

yi = DiYi (1) + (1−Di )Yi (0)D∗i = ziγ+ ui

Di =

1 if D∗i > 00 if D∗i 6 0


NotesI δi = obs-specific gain to treatment (conditional on x)I ∆i = Yi (1)− Yi (0) = xi (β1 − β0) + δi (heterogeneous treatmenteffects given x)

I Selection into treatment may depend on either υ0i (untreated outcomelevel given x) or δi (obs-specific gains given x)

I Otherwise, intuition is identical to common effect version

Assumptions (replaces (BVN.i))

(BVN.i’) υ0, υ1, u ∼ N(0,Σ), where

Σ =

σ2υ0 ρ01 ρ0uσ2υ1 ρ1u

1


Conditional expectations

E[υ0i |Di = 0, xi , zi ] = ρ0uσυ0


]E[υ0i |Di = 1, xi , zi ] = ρ0uσυ0

[φ(ziγ)Φ(ziγ)

]E[δi |Di = 1, xi , zi ] = ρδuσδ

[φ(ziγ)Φ(ziγ)

]


EstimationI Generalization of the previous two-step approach in the common effectmodel

I OLS estimating equation

yi = xi β0 + xiDi (β1 − β0) + βλ1Di

[φ(ziγ)Φ(ziγ)

]+ βλ0(1−Di )


]+ ζ i

where

βλ1 = ρ0uσυ0 + ρδuσδ

βλ0 = ρ0uσυ0

I ML estimation of entire model is feasible, but it requires estimation ofa trivariate normal dbn (computationally diffi cult)

I ρ01 is not identified since never observe Y (1) and Y (0) for same i


Upon estimation of β0, β1 ...

I Can predict Yi (0), Yi (1) ∀iI Estimate ∆ATE

∆ATE = x(

β1 − β0

)∆ATT = x1

(β1 − β0

)+(βλ1 − βλ0

) [ φ(z1γ)

Φ(z1γ)

]∆ATU = x0

(β1 − β0

)+(βλ1 − βλ0

) [ −φ(z0γ)

1−Φ(z0γ)

]where the final terms for the ATT and ATU correspond toE[δi |Di = d , xi , zi ], d = 0, 1

F If there is no selection on unobservable gains, then ρδu = 0 ⇒ commoneffect model

F βλ1 − βλ0 = ρδuσδ ⇒ ρδuσδ =βλ1 − βλ0, which gives the sign of the

selection on gains (which one expects to be positive if obs know theirunobservable gains)

See Wooldridge (2015)


Klein & Vella approach

See Klein & Vella (2009, 2010); Farré et al. (2010)

Setup as in the prior modelI ‘Structural’model

yi = β1Di + xi β2 + εi

I First-stage modelDi = xiγ+ ui

where

F x includes the interceptF Cov(ε, u) 6= 0

D may be discrete or continuous


Identification assumptions

(KV.i) εi = Sε(wi )ε∗i and/or ui = Su(wi )u∗i , where w ⊆ x such that

Sε(wi )/Su(wi ) varies across i and ε∗, u∗ have unit variances(KV.ii) E[ε∗i u

∗i ] = ρ, which is constant

Under (KV.i) and (KV.ii), the structural model may be re-written as

yi = β1Di + xi β2 + ρiui + εi

where εi is now a well-behaved error term

This is a random coeffi cients model, but the form of the coeffi cient isknown


The structural model is given by

yi = β1Di + xi β2 + ρ

[Sε(wi )Su(wi )

ui

]+ εi

I The term in brackets acts is the control functionI Due to the variation in Sε(wi )/Su(wi ), no need for an exclusionrestriction since

ui ≡Sε(wi )Su(wi )

ui

is not perfectly collinear with the remaining covariates

Klein & Vella (2009) propose a semiparametric estimator of the model

Farré et al. (2010) outline a parametric estimator


Parametric EstimationI Assuming

Sε(wi ) =√exp(wi θε)

Su(wi ) =√exp(wi θu)

the structural model becomes


[√exp(wi θε)√exp(wi θu)

ui

]+ εi

I Estimate the first-stage by OLS ⇒ uI Estimate by OLS

ln(u2i ) = wi θu + ui

and form Su(wi ) =√exp(wi θu)


Parametric Estimation (cont.)

I Substitute u and Su(wi ) into the structural model and estimate theremaining parameters by NLS


[√exp(wi θε)

Su(wi )ui

]+ εi

I While one could stop, performance is perhaps improved by addingadditional steps

F Given NLS estimates of β1 and β2 ⇒ εF Estimate by OLS

ln(ε2i ) = wi θε + ˜εiand form Sε(wi ) =

√exp(wi θε)

F Estimate by OLS


[Sε(wi )

Su (wi )ui

]+ εi

I Obtain std errors via bootstrap


Selection on UnobservablesDistributional Approaches

Many interesting questions require addressing endogeneity in thecontext of distributional models

Here, we present two approaches1 Quantile regression2 Nonlinear difference-in-differences

Other estimators not discussed here1 Fixed effect QR models (Koenker 2004, Canay 2011, Machado &Santos Silva 2019)

2 Nonparametric bounds applied to QR models (Giustinelli 2011)


Selection on UnobservablesDistributional Approaches: FE Quantile Regression

Machado & Santos Silva (2019)

Stata: -xtqreg-


Selection on UnobservablesDistributional Approaches: IV Quantile Regression

Recall, QR model (Koenker & Bassett 1978)I Assuming linear conditional quantiles, estimation is

βθ,∆θ= argminβ,∆

1N

∑

i :yi>xi βθ|yi − ∆Di − xi β|+ ∑

i :yi<xi β(1− θ)|yi − ∆Di − xi β|

I May be rewritten as

βθ, ∆θ = argminβ,∆

1N

∑i

ρθ(εθi )

where ρθ(εθi ) is check function, defined as


and εθi is the residual for i and θ


QR model is biased and inconsistent if D is endogenousI Conditional quantile effect estimators

F Abadie et al. (2002) present an IV-QTE modelF Chernozhukov & Hansen (2005, 2006, 2008) present an IV-QR modelF Lee (2007) presents a control function version

I Unconditional quantile effect estimators

F Frölich & Melly (2013)


Abadie et al. (2002)Linear (in parameters) model of conditional quantile fn

E[Qθ(y |x ,D)] = ∆θDi + xi βθ

with a binary instrument, ZEstimation

I The objective fn

βθ, ∆θ = argminβ,∆

1N

∑i

ωi ρθ(εθi )

where


εθi = yi − ∆θDi − xi βθ

ωi = 1− Di (1− Zi )1− Pr(zi |xi )

− (1−Di )ZiPr(zi |xi )

although in practice Zi is replaced with an estimated conditionalexpectation

Stata: -ivqte-DL Millimet (SMU) ECO 7377 Spring 2020 372 / 478

Chernozhukov & Hansen (2005, 2006, 2008)

Linear (in parameters) model of conditional quantile fn


EstimationI Consider the objective fn

βθ, ∆θ, γθ = arg minβ,∆,γ

1N

∑i

ρθ(εθi )

where


εθi = yi − ∆θDi − xi βθ − Diγθ

and Di is the predicted value from the first-stage regression of D onx , z

I Given correctly specified ‘structural’model, γθ should equal zero


Algorithm1 Define a grid of possible values of ∆, ∆j , j = 1, ..., J2 For each θ, estimate a QR model with yi − ∆Di as the dependentvariable and x , Di as covariates

3 Obtain estimates βθj , γθj , j = 1, ..., J4 Choose ∆θ = ∆θj and βθ = βθj to minimize |γθj |

Inference via sub-sampling or typical, nonparametric iid bootstrap, asin QR model

Easily extendable to multiple endogenous variables, but grid searchincreases exponentially

Estimator works regardless of whether D, z are binary or continuous

Stata: -ivqreg2-


Lee (2007)

Linear (in parameters) model of conditional quantile fn


EstimationI Consider the objective fn

βθ, ∆θ, γθ = arg minβ,∆,γ

1N

∑i

ρθ(εθi )

where


εθi = yi − ∆θDi − xi βθ − Viγθ

and Vi is the first-stage residual obtain via OLS or a QR at someα−quantile

Stata: -cqiv-


Frölich & Melly (2013)

With a binary IV, Z , the potential distributions of the outcomevariable are identified for the subpopulation of compliers

Zi satisfies the following three assumptions:I Independence: Yi (0),Yi (1),Di (0),Di (0) ⊥ Zi |xiI Correlation: Pr(Zi = 1|xi ) ∈ (0, 1) andPr(Di (0) = 1) < Pr(Di (1) = 1)

I Monotonicity: Pr(Di (0) ≤ Di (1)) = 1

Weights for QTE defined as

ωi =[zi − Pr(zi = 1|x)]

Pr(zi = 1|x) [1− Pr(zi = 1|x)](2Di − 1)


∆θ obtained using sample analogues of y θ1 and y

θ0

I Obtain Fd (y |C ), d = 0, 1

Fd (y |C ) =∑i∈d ωi I(yi ≤ y)

∑i∈d ωi

where

ωi =[zi − Pr(zi = 1|x)]

Pr(zi = 1|x) [1− Pr(zi = 1|x)](2Di − 1)

and C denotes the subpopulation of compliersI ∆QTEθ = Y θ(1)− Y θ(0), where Y θ(d) = infy : Fd (y) > θ

Alternative estimator given by

β0θ, ∆θ = argminβ,∆

1N

∑i

ωiρθ(yi − β0 − ∆θDi )

where x only enters via the weights to estimate unconditional quantileeffects

Stata: -ivqte-


Selection on UnobservablesDistributional Approaches: Nonlinear Difference-in-Differences

Recall, standard DID strategyI Treatment group observed pre- and post-interventionI Control group observed in same time periodsI Assume treatment and control groups follow same time trend absenttreatment

I Estimate treatment effect by the additional change over time in thetreatment group relative to the control group

Idea is extendable beyond just average treatment effects via twoapproaches

1 Quantile DD estimator2 Changes-in-Changes estimator

Model requires panel data or repeated cross-sections


Setup (Athey & Imbens 2006)

Notation: standard difference-in-differencesI Individual i belongs to a group Di ∈ 0, 1, where D = 1 is treatmentgroup

I Individual i observed at time T ∈ 0, 1I Yit (0),Yit (1) = potential outcomesI yit = observed outcomeI All agents are untreated when T = 0 and treatment group is treatedprior to T = 1

I Conditional dbns

Ydt (0) ∼ Y (0)|D = d ,T = tYdt (1) ∼ Y (1)|D = d ,T = t

ydt ∼ y |D = d ,T = t

I Inverse CDFsF−1y (θ) = infy : FY (y) > θ


GoalI Estimate

∆QTTθ = F−1Y (1),11(θ)− F−1Y (0),11(θ)

I Observable dbns include: FY (0),00, FY (0),01, FY (0),10, and FY (1),11I FY (0),11 is the missing counterfactual

Assumptions

(CIC.i) Model: Y (0) = h(U,T ), where U are unobserved attributes(CIC.ii) Strict monotonicity: h(u, t) is strictly increasing in u for t ∈ 0, 1(CIC.iii) Time invariance within groups: U ⊥ T |D(CIC.iv) Support: U1 ⊆ U0


EstimatorCounterfactual CDF

FY (0),11(y) = Fy ,01(F−1y ,00(Fy ,10(y)))

which is estimable using empirical CDFsEquivalently,

F−1Y (0),11(θ) = F−1y ,01(Fy ,00(F

−1y ,10(θ)))

Treatment effect estimate

∆CICθ = F−1Y (1),11(θ)− F−1Y (0),11(θ)

Note, ∆CICθ is the difference in two QTE estimates

∆CICθ = ∆QTEθ,1 − ∆QTEθ′,0

whereI ∆QTTθ,1 is change over time in y at quantile θ for D = 1 groupI ∆QTU

θ′,0is change over time in y at quantile θ′ for D = 0 group, where

q′ is the quantile in the D = 0,T = 0 dbn corresponding to the valueof y associated with quantile θ in the D = 1,T = 0 dbn


Alternative estimatorI QDID treatment effect estimator

∆QDIDθ = F−1Y (1),11(θ)− F−1Y (0),11(θ)

whereF−1Y (0),11(θ) = F

−1y ,10(θ) + [F

−1y ,01(θ)− F

−1y ,00(θ)]

which corresponds to

∆QDIDθ = ∆QTTθ,1 − ∆QTUθ,0

where ∆QTTθ,1 , ∆QTUθ,0 is change over time in y at quantile θ forD = 1, 0, respectively

I Relies on (perhaps) unrealistic assumptions


Athey & Imbens (2006) discuss extensions toI Discrete outcomesI Multiple groups and multiple time periodsI Incorporating covariates (see also Melly & Santangelo 2015)

F Semiparametric specification of potential outcomes

Y (0) = h0(u, t) + xβ

Y (1) = h1(u, t) + xβ

where U ⊥ T ,X |DF OLS estimation of outcomes

yi = Wi δ+ xi β+ εi

where W = [DT (1−D)T D(1− T ) (1−D)(1− T )]F Perform CIC estimation on

yi = yi − xi β = Wi δ+ εi

I InferenceF Athey & Imbens (2006) prove asymptotic normality, and deviseasymptotic variance

F Bootstrap alternative?

Stata: -cic-DL Millimet (SMU) ECO 7377 Spring 2020 384 / 478

Selection on UnobservablesMarginal Treatment Effects

All of the estimators discussed to this point —ATE, ATT, ATU,LATE, etc. — can be obtained as different weighted averages ofso-called marginal treatment effects (MTE)

Recall the set-up from the Generalized Roy model

Y (d) = µd (x) + υd , d = 0, 1

D = I(µD (x , z)− υD > 0)

where z is an instrument

NotesI The propensity score is given by p(x , z)I D = 1⇔ p(x , z) = FυD (µD (x , z)) > FυD (υD ) = U, where U ∈ [0, 1]I Key assumptions:

1 υ0, υ1, υD ⊥ x , z2 µD (·) is a non-trivial function of z


Observation-specific treatment effect is given by

∆i = µ1(xi )− µ0(xi ) + υ1i − υ0i

Marginal treatment effect is the expected treatment effect forobservations indifferent between being treated or not when U = p

∆MTE (p, x) = E[∆i |x ,U = p]= µ1(x)− µ0(x) + E[υ1 − υ0|x ,U = p]= µ1(x)− µ0(x) + k(p, x)

In contrast, recall that ∆ATE (x) = E[∆i |x ]For estimation

I Independence assumption ⇒ k(p, x) = k(p)I See Brinch et al. (2017)


Estimation Algorithm: Local IV

1 Estimate the propensity score, p(x , z), via probit or logit2 Estimate via OLS

yi = xi β0 + pxi (β1 − β0) +K (p)

where K (·) is a polynomial in p and p replaces D to permit SOO3 MTE is given by

∂ E[y |x , p]∂p

= x(β1 − β0) +∂K (p)

∂p


ExampleI Estimate

yi = xi β0 + pi xi β1 + α1pi + α2p2i2+ α3

p3i3+ εi

I ∆MTE (p) = x β1 + α1 + α2pi + α3p2i


Estimation Algorithm: Separate Estimation

1 Estimate the propensity score, p(x , z), via probit or logit2 Estimate via OLS

yi = xi β0 +K0(p) if D = 0

yi = xi β1 +K1(p) if D = 1

where Kd (·) is a polynomial in p3 MTE is given by

E[∆i |x , υD = p] = x(β1 − β0) +K1(p)−K0(p)


ExampleI Estimate

yi = xi β0 + α011− p2i2(1− pi )

+ α021− p3i3(1− pi )

+ εi if D = 0

yi = xi β1 + α11pi2+ α12

p2i3+ εi if D = 1

I ∆MTE (p) = x(β1 − β0) +(α11p + α12p2

)−(α01p + α02p2

), where

F α11p + α12p2 = K1(p) + p · ∂K1(p)/∂pF α01p + α02p2 = K0(p)− (1− p) · ∂K0(p)/∂p


Final notesI Results should be displayed graphically, usually at x (but could be atspecific values of x)

I Standard errors for ∆MTE (p) obtained via bootstrapI Weights, ω(p), defined such that∫

sω(s)∆MTE (s)ds

yields various average treatment effect parameters are provided in theliterature

F Simple case: ω(p) = 1 for ATEF See Carniero et al. (2011)

Stata: -margte-, -mtefe-, -mtebinary-


10

010

2030

40

0 .2 .4 .6 .8 1U_D

MTE ATE = 5.9195% norm CI

Estimated Marginal Treatment Effects


Data Issues

Data issues are a fact of life

Frequently encountered are problems pertaining to missing orcontaminated data

I Sample selection concerns missing data on the dependent variableI Contaminated data refers to a scenarious where one is interested in themarginal distribution of a potentially mismeasured variable

I Measurement error more generally refers to mismeasured dependent orindependent variables


Data IssuesSample Selection

Population model

yi = xi β+ εi , εi ∼N(0, σ2)

Given a random sample, yi , xiNi=1, then OLS is consistent andeffi cient if the usual assumptions are satisfied

Problem arises when data on y is only available for a non-randomsample

I Let Si = 1 if yi is observed; Si = 0 if yi is unobserved

Note: While exposition is using cross-section, a common source of(non-random) selection is attrition in panel data; particularlyimportant in firm-level studies where attrition may be due to firmsexiting the market


Example: Certain subpopulations may not be representative of thepopulation


Implies following data structureI Have data on a random sample, yi , xi ,SiNi=1, but yi = . if Si = 0I Can only use M ≡ ∑i Si observations to estimate any modelI Examples

F Wages only observed for workersF Firm profits only observed for firms that remain in businessF Test scores only observed for test takersF House prices only observed for houses on the market (sold?)

IssueI Is OLS still unbiased and consistent?I Answer: depends


Data IssuesSample Selection: Heckman (1979)

Setup

yi = xi β+ εi

S∗i = ziγ+ ui

Si =

1 if S∗i > 00 if S∗i 6 0

yi = . if Si = 0

εi , ui ∼ N2(0, 0, σ2ε , 1, ρ)

x , z are exogenous


ProblemI E[y |x ] = xβ, but

E[y |x , S = 1] = E[y |x , z , u] = xβ+ E[ε|x , z , u]= xβ+ E[ε|u > −ziγ]

= xβ+ ρσεφ(zγ)

Φ(zγ)

where ρσεφ(zγ)/Φ(zγ) is the Inverse Mills’Ratio from beforeI Implies that E[y |x , S = 1] = xβ iff ρ = 0I OLS estimation of

yi = xi β+ εi

using only M observations omits the IMR term, which implies that

εi = ρσεφ(zγ)/Φ(zγ) + εi

which is not mean zero, and is not independent of x , unless ρ = 0


Solution: Heckman two-step method1 Estimate IMR (using i = 1, ...,N)

F Estimate probit model, where S is dependent variable and z are thecovariates ⇒ γ

F Obtain

IMRi =φ(zi γ)Φ(zi γ)

2 Regress yi on xi , IMRi via OLS (using i = 1, ...,M)

Test of endogenous selection

Ho : βλ = 0

Ha : βλ 6= 0

where βλ is the coeffi cient on the IMR


NotesI Usual OLS standard errors are incorrect since IMR is predicted; mustaccount for additional uncertainty due to estimation of γ

I Other complications in derivation of standard errorsI Need an exclusion restriction(s)

F A variable in z not in xF Otherwise model is identified from non-linearity of IMR, which arisessolely from the assumption of joint normality

F However, even though technically identified from the non-linearity,substantial collinearity in practice makes identification questionable

I Model can be estimated in one-step by MLF More effi cient if model assumptions are validF Less robust in general since more dependent on functional formassumptions

I Model may be expanded to the case of a single sample selectioncriterium taking on multiple values, multiple sample selection criteria,and cases where y is a LDV

Stata: -heckman-, -heckman2-, -oheckman-, heckprobit-,-heckoprobit-


Data IssuesSample Selection: QR Alternative

Assume the latent outcome is

y ∗i = xi β+ ui

y ∗ is unobserved; instead observe

yi =y ∗i if observed. otherwise

QR model estimated using data on yi , xi, where

yi =yi if observedminyi otherwise

yields

βθ = argminβ

1N

∑i

ρθ(yi − xi β)

which is consistent as long as all missing values of y ∗i 6 Qθ(y ∗|x)DL Millimet (SMU) ECO 7377 Spring 2020 402 / 478

More generally, QR model estimated using data on yi , xi, where

yi =yi if observedimputed value otherwise

yields

βθ = argminβ

1N

∑i

ρθ(yi − xi β)

which is consistent as long as imputed values lie on the correct side ofQθ(y ∗|x)


Example:

.50

.51

1.5

0 .2 .4 .6 .8 1x

ystar 'true' OLS fitted line'true' LAD fitted line OLS fitted line, y>0 onlyLAD fitted line

NOTE: x~U[0,1]; ystar=0.25+x+e; e~N(0,0.25^2); y=ystar if ystar>0.LAD fitted line obtained by first replacing y=10 if ystar>true LAD line, 10 otherwise.


Data IssuesSample Selection: Bounding Distributions

Based on Blundell et al. (2007)

NotationI W ∗ = latent outcome variableI E = selection indicatorI W = outcome variable, where

W =

W ∗ if E = 1. otherwise

I X = covariate vector

Goal: bound CDF F (w |x) given observable CDF F (w |x ,E = 1)Examples:

I Dbn of wages under full employmentI Dbn of child health under full HI coverageI Dbn of test scores on college entrance exams with full participation


Can be rewritten in terms of bounds on quantiles

w θ,l (x) 6 w θ(x) 6 w θ,u(x)

whereI w θ(x) = θth quantile of F (w |x)I w θ,l (x) is the value of w that solves

θ = F (w |x ,E = 1)p(x) + [1− p(x)]

⇔ w = F−1(

θ − [1− p(x)]p(x)

|x ,E = 1)

I w θ,u(x) is the value of w that solves

θ = F (w |x ,E = 1)p(x)

⇔ w = F−1(

θ

p(x)|x ,E = 1

)


ExampleI θ = 0.5, p(x) = 0.9I w θ,l (x) = F−1(θ′′|x ,E = 1), where

θ′′ = (0.5− 0.1)/0.9 = 0.4/0.9 ≈ 0.44I w θ,u(x) = F−1(θ′|x ,E = 1), where θ′ = 0.5/0.9 ≈ 0.55⇒ bounds on the median are given by the values of the observedconditional dbn at the 44th and 55th quantiles

NotesI Bounds cannot be used to determine if selection is non-random; onlyassess the possible consequences

I Bounds only estimable for θ ∈ [1− p(x), p(x)]I Bounds converge to point estimates as p(x)→ 1


Median restrictionI Weaker characterization is to assume (conditional on x) thatw0.5(E=1) > w0.5(E=0)

I Equivalent toPr(E = 1|W 6 w0.5(E=1), x) 6 Pr(E = 1|W > w0.5(E=1), x)

I Bounds on F (w |x) become

F (w |x ,E = 1)p(x) 6 F (w |x) 6 F (w |x ,E = 1)p(x) + [1− p(x)]if w < w0.5(E=1)

F (w |x ,E = 1)p(x)+ 0.5[1− p(x)] 6 F (w |x) 6 F (w |x ,E = 1)p(x) + [1− p(x)]

if w > w0.5(E=1)

I Bounds are tightened (relative to worst case) only above the mediansince the missing term, F (w |x ,E = 0), is now bounded from below at0.5 for w > w0.5(E=1) (instead of zero)


Tightening the bounds: Exclusion restriction

Conditional independenceI Assume z satisfies

F (w |x , z) = F (w |x) ∀w , x , z

I Bounds on F (w |x) become

maxzF (w |x , z ,E = 1)p(x , z)

6 F (w |x)6 min

zF (w |x , z ,E = 1)p(x , z) + [1− p(x , z)]

I If conditional independence is not true, bounds may cross; failure ofbounds to cross does not prove conditional independence holds


MonotonicityI Higher values of z improve the dbn in a FSD sense

F (w |x , z ′) 6 F (w |x , z ′′) ∀w , x , z ′, z ′′ s.t. z ′ > z ′′

I Bounds on F (w |x , z1) become

maxz>z1F (w |x , z ,E = 1)p(x , z)

6 F (w |x , z1)6 min

z6z1F (w |x , z ,E = 1)p(x , z) + [1− p(x , z)]

I Bounds on F (w |x) obtained by integrating over the dbn of z ; entailscomputing the weighted average of the upper and lower bounds acrossthe different values, z1, where the weights are sample proportion,Pr(z = z1 |x)


Bounding differences in QTEs across groups accounting fornon-random selection

NotationI D ∈ 0, 1 indexes groupsI T ∈ 0, 1 indexes time period

Bounds on QTEs across groups in a given time period

w θ,l (1,T )− w θ,u(0,T ) 6 w θ(1,T )− w θ(0,T )

6 w θ,u(1,T )− w θ,l (0,T )

Bounds on QTEs across time for a given group

w θ,l (D, 1)− w θ,u(D, 0) 6 w θ(D, 1)− w θ(D, 0)

6 w θ,u(D, 1)− w θ,l (D, 0)


Bounds on diff-QTEs across groups

[w θ(1, 1)− w θ(0, 1)]− [w θ(1, 0)− w θ(0, 0)] ∈ [LB,UB ]

where

LB = [w θ,l (1, 1)− w θ,u(0, 1)]− [w θ,u(1, 0)− w θ,l (0, 0)]

UB = [w θ,u(1, 1)− w θ,l (0, 1)]− [w θ,l (1, 0)− w θ,u(0, 0)]

I Example: Change in median wage gap across males and females overperiod T = 0 to T = 1


Level set restrictionsI Assume diff-QTE, [w θ(1, 1)− w θ(0, 1)]− [w θ(1, 0)− w θ(0, 0)], isconstant across different values of some covariate x ∈ X

I Calculate LB(x),UB(x) ∀x ∈ XI New LB,UB given by

LB = maxx∈X

LB(x)

UB = minx∈X

UB(x)

Test statistics derived in Blundell et al. for bounds crossings, whetherobserved conditional distribution, F (w |x ,E = 1) lies in the boundsInference via bootstrap


Data IssuesSample Selection: Average Treatment Effects

Literature discusses bounding ATEs accounting for non-randomselection

I Lechner and Melly (2007)I Imai (2008)I Lee (2009)I Huber and Mellace (2015)

Stata: -leebounds-


Huber (2015) discusses estimates of the ATE under non-randomsample selection and strong ignorability of treatment assignment

I Assumes the presence of an exclusion restriction

F z impacts selection into the sample, denoted by S = 1F p(x , z) ≡ Pr(S = 1|x , z)

I Treatment assignment depends on x

F Propensity score still given by p(x) = Pr(D = 1|x)I Strong ignorability holdsI Treatment effects obtained by weighting based on propensity score andsample inclusion probability

E[Y (1)] = E[

SDyp(x , z)p(x)

]E[Y (0)] = E

[S(1−D)y

p(x , z)[1− p(x)]

]I Similar strategy for ATT, ATU, and QTEs


Dong (2019) addresses nonrandom sample slection in the context ofthe fuzzy RD estimator

I Selection into the sample denoted by S = 1I Expected outcomes among the selected sample at the cut-off given by

E[Y (1)|S = 1,R = r0,C ] =limr↓r0 E [SDy |R = r ]− limr↑r0 E [SDy |R = r ]limr↓r0 E [SD |R = r ]− limr↑r0 E [SD |R = r ]

E[Y (0)|S = 1,R = r0,C ] =limr↓r0 E [S(1−D)y |R = r ]− limr↑r0 E [S(1−D)y |R = r ]limr↓r0 E [S(1−D)|R = r ]− limr↑r0 E [S(1−D)|R = r ]

where R is the running variable, r0 is the threshold, and C denotescompliers (treated if R > r0, untreated otherwise)

F Difference yields the intensive margin effectF If selection is not differentially random across the threshold, r0, thenomitting S is inconsequential


Dong (2019) cont.I The extensive margin effect (and a test for differential selection acrossthe threshold) is given by

E[S(1)− S(0)|R = r0,C ] =limr↓r0 E [S |R = r ]− limr↑r0 E [S |R = r ]limr↓r0 E [D |R = r ]− limr↑r0 E [D |R = r ]

where S(r) are potential selection indicators, denoting the value of Sif R > r0 and R < r0


Data IssuesMeasurement Error

Refer to ECO 6374 for refresher on basics...

Problem: sometimes (often!) data are measured imprecisely; seeBound et al. (2001), Millimet (2011)Focus here is on measurement in DVery little work on the effects of measurement in y , x in the contextof causal estimators discussed to this point

I Battistin & Chesher (2014) consider case of strong ignorability holdsgiven x∗, but only observe x

I Davezies & Le Barbanchon (2017) and others consider measurement ins in RD designs


Data IssuesME: Classical Errors-in-Variables (CEV) model

Recall, for a continuous independent variable

xi︸︷︷︸observed

= x∗i︸︷︷︸actual

+ µi︸︷︷︸ME

I Implications under CEV assumptions

F OLS biased, inconsistent unless β = 0F βOLS suffers from attenuation bias


Data IssuesME: Binary Independent Variable

True modely ∗i = α+ βD∗i + εi , εi ∼N(0, σεε)

where ‘∗’on a variable indicates correctly measured

Given a random sample y ∗i ,D∗i Ni=1, assume OLS is consistent andeffi cient

With measurement error, do not observe D∗iInstead one observes Di where

Di︸︷︷︸observed

= D∗i︸︷︷︸true

+ µi︸︷︷︸ME

which implies that µ ∈ 0, 1 if D∗ = 0, and µ ∈ 0,−1 if D∗ = 1Thus, measurement error is

I Not normally distributed (violates CEV.ii)I Is negatively correlated with D∗ (violates CEV.iii)


Assumptions

(BME.i) Non-differential classification errors: E[y |D∗] = E[y |D,D∗](BME.ii) D∗ ⊥ ε(BME.iii) Cov(D,D∗) > 0(BME.iv) Cov(D∗, µ) < 0

Given (BME.i) — (BME.iv), asymptotic bias given by

plim(

βOLS

)=

(σD ∗D ∗ + σD ∗µ

σD ∗D ∗ + 2σD ∗µ + σµµ

)β

Results in attenuation bias for β if σD ∗µ + σµµ > 0

Likely true for any mismeasured bounded variable


Millimet (2011) conducts MC study comparing common treatmenteffect estimators (∆ = 1)


Partial solution: Frisch bounds (Aigner 1973; Bollinger 1996; Blacket al. 2000)

Reverse regressionI Estimate via OLS

Di = π0 + π1y∗i + υi

I plim given by

plim(

π−11,OLS

)=

β2σ2D ∗ + σ2ε

β(

σ2D ∗ + σD ∗µ

) = β+σ2ε

β(

σ2D ∗ + σD ∗µ

)which is biased up in absolute value if σ2D ∗ + σD ∗µ > 0

I Implies∣∣∣βD ∗,OLS ∣∣∣ ∈ (∣∣∣βOLS ∣∣∣ , ∣∣∣π−11,OLS ∣∣∣), where βD ∗,OLS is the OLS

estimate if D∗ were observed (Frisch bounds)I If R2 is low, then bounds obtained using reverse regression may beuninformative


Improved upper bound may be obtained via IV (not a consistentestimate!)

I Inconsistency of IV results from fact that any instrument correlatedwith D∗ will most likely be correlated µ since Cov(D∗, µ) 6= 0

Improved lower bound obtained by estimating

y ∗i = α+ β0 I[Di = 0,D ′i = 1]+ β1 I[Di = 1,D ′i = 0] + β2 I[Di = 1,D ′i = 1] + ηi

where D ′i is a second mis-measured indicatorI If the measurement errors are independent conditional on actualtreatment assignment, D∗i , then

0 <∣∣∣E[βOLS ]∣∣∣ < ∣∣∣E[β2,OLS ]∣∣∣ < |β|


Partial solutions (Kreider & Pepper 2007)

Utilize a non-regression approach to bound the effect of amis-measured binary treatment

Authors do not wish to invoke (BME.i), which implies thatmis-reporting is independent of outcomes conditional on the truth

NotationI y ∈ 0, 1 is a binary outcome (correctly measured)I D∗ ∈ 0, 1 is the true binary treatmentI D ∈ 0, 1 is the reported binary treatmentI Z ∈ 0, 1, where Z = 1 if D = D∗ and 0 otherwise

Estimand of interest: ∆ = Pr(y = 1|D∗ = 1)− Pr(y = 1|D∗ = 0)Data provides an estimate of Pr(y = 1|D)


Manipulation yields

Pr(y = 1|D∗ = 1) =Pr(y = 1,D∗ = 1)Pr(D∗ = 1)

=

Pr(y = 1,D = 1)+Pr(y = 1,D = 0,Z = 0)−Pr(y = 1,D = 1,Z = 0)

(Pr(D = 1) + Pr(D = 0,Z = 0)

−Pr(D = 1,Z = 0)

)where Pr(D = 1,Z = 0) is a false positive and Pr(D = 0,Z = 0) isa false negative

Data provide estimates of Pr(y = 1,D = 1), Pr(D = 1)

Other elements are unknown, but bounded by the unit interval


Lower-Bound Accurate Reporting RateI Assume Pr(Z = 1) ≥ vI Can show that

Pr(y = 1|D∗ = 1) ∈[

Pr(y = 1,D = 1)− δ

Pr(D = 1)− 2δ+ (1− v ) ,Pr(y = 1,D = 1) + γ

Pr(D = 1) + 2γ− (1− v )

]

where

δ =

min(1− v ),Pr(y = 1,D = 1) if Pr(y = 1,D = 1)− Pr(y = 0,D = 1)− (1− v ) ≤ 0max0, (1− v )− Pr(y = 0,D = 0) otherwise

γ =

min(1− v ),Pr(y = 1,D = 0) if Pr(y = 1,D = 1)− Pr(y = 0,D = 1) + (1− v ) ≤ 0max0, (1− v )− Pr(y = 0,D = 1) otherwise

I Bounds for Pr(y = 1|D∗ = 0) are obtained by replacing D with 1−DI Bounds for each term obtained by replacing elements with sampleanalogs

I Bounds for ∆ obtained using relevant upper and lower bounds for eachterm

I When v = 1, bounds collapse to a point estimate


Partial VerificationI Might assume a lower bound for accuracy among some sub-groupwhose status is more certain, W = 1

I Assume Pr(Z = 1|W = 1) ≥ vwI Can show that

Pr(y = 1|D∗ = 1) ∈

Pr(y = 1,D = 1,W = 1)− δ Pr(D = 1,W = 1)

+Pr(y = 0,W = 0)−2δ+ (1− vw )Pr(W = 1)

,(

Pr(y = 1,D = 1,W = 1)+Pr(y = 1,W = 0) + γ

)(

Pr(D = 1,W = 1) + Pr(y = 1,W = 0)+2γ− (1− vw )Pr(W = 1)

)

where

δ =

min(1− vw )Pr(W = 1),Pr(y = 1,D = 1) if α ≤ 0max0, (1− vw )Pr(W = 1)− Pr(y = 0,D = 0,W = 1) otherwise

γ =

min(1− vw )Pr(W = 1),Pr(y = 1,D = 0) if α′ ≤ 0max0, (1− vw )Pr(W = 1)− Pr(y = 0,D = 1,W = 1) otherwise

α = Pr(y = 1,D = 1,W = 1)− Pr(y = 0,D = 1,W = 1)

− Pr(y = 0,W = 0)− (1− vy )Pr(W = 1) ≤ 0

α′ = Pr(y = 1,D = 1,W = 1)− Pr(y = 0,D = 1,W = 1)

+ Pr(y = 1,W = 0) + (1− vy )Pr(W = 1) ≤ 0

I If vw = 1, then one has full verification for an observed sub-sample →bounds are tightened


Combine the prior assumptions with a Monotone IV assumption topossibly further tighten the bounds

MIV AssumptionI ∃ x s.t.

x0 ∈ [x1, x2 ]⇒ Pr(y = 1|D∗, x0) ∈ [Pr(y = 1|D∗, x1),Pr(y = 1|D∗, x2)]

I Implies that Pr(y = 1|D∗, x) is weakly monotonically increasing in xI Proceed by

F Computing bounds conditional on different values of xF Obtaining unconditional bounds by ‘integrating’over the dbn of x

Kreider & Hill (2009), Kreider et al. (2011) combine thismethodology on reporting errors with prior methods on boundingtreatment effects under SOU

Imai & Yamamoto (2010) offer a similar analysis in poli sci


Partial solutions (Battistin & Sianesi 2009)

Consider ME of a binary or multi-valued treatment in the context ofpropensity score estimatorsSetup

(MPS.i) CIA given no MEY (0),Y (1) ⊥ D∗|x

(MPS.ii) CS given no ME

p∗(x) = Pr(D∗ = 1|x) ∈ (0, 1) ∀x

I D∗ is not observed, instead D is, where Di 6= D∗i for at least some iEstimation based on D yields

∆ATE = EE[y |D = 1, x ]− E[y |D = 0, x ]

where the outer expectation is over S , where

S = x : p(x) = Pr(D = 1|x) ∈ (0, 1)

In contrast, estimation based on D∗ ⇒ ∆ATE∗


NotationI (Mis)classification probabilites given by

λjj ′(x) = Pr(D∗ = j |D = j ′, x), j , j ′ ∈ 0, 1

F λ10 = proportion of incorrect reported zerosF λ01 = proportion of incorrect reported ones

I Condensed notation for correct reporting rates

λ00(x) = λ0(x) = Pr(D∗ = 0|D = 0, x)

λ11(x) = λ1(x) = Pr(D∗ = 1|D = 1, x)

I Matrix of (mis)classification probabilities can be written in terms ofλ0,λ1

Λ(x) =[

λ0(x) 1− λ0(x)1− λ1(x) λ1(x)

]


Assumptions

(MPS.iii) Non-differential classification errors: E[y |D∗, x ] = E[y |D,D∗, x ](MPS.iv) Informative reported treatment status: λ0(x) + λ1(x)− 1 6= 0

Outcomes condition on D can be written as a weighted average ofoutcomes conditional on D∗[

E[y |D = 0, x ]E[y |D = 1, x ]

]= Λ(x)

[E[y |D∗ = 0, x ]E[y |D∗ = 1, x ]

]⇒[

E[y |D∗ = 0, x ]E[y |D∗ = 1, x ]

]= Λ−1(x)

[E[y |D = 0, x ]E[y |D = 1, x ]

]provided det[Λ(x)] = λ0(x) + λ1(x)− 1 6= 0Two cases satisfy (MPS.iv)

I Minimal classification errors: λ0(x) + λ1(x) > 1I Severe classification errors: λ0(x) + λ1(x) < 1


The bias when using D is

∆ATE (x) = [λ0(x) + λ1(x)− 1] · ∆ATE∗(x)

Implications:I ∆ATE (x) is unbiased if λ0 = λ1 = 1I ∆ATE (x) suffers from attenuation bias if λ0(x) + λ1(x) > 1I ∆ATE (x) suffers from attenuation bias AND

sgn[∆ATE (x)

]6= sgn

[∆ATE

∗(x)]if λ0(x) + λ1(x) < 1

I ∆ATE (x) = −∆ATE∗(x) if λ0 = λ1 = 0


The bias of the unconditional ATE, ∆ATE , also depends on theerroneous determination of the CS

I Can show that

p(x) =p∗(x)− [1− λ0(x)]λ0(x) + λ1(x)− 1

I This implies that boundary values of p(x) can be obtained even ifp∗(x) ∈ (0, 1) if

p(x) = 0⇔ λ0(x) = 1− p∗(x)p(x) = 1⇔ λ1(x) = p

∗(x)

To ensure one does not utilize a different CS based on D, mustassume

(MPS.v) λ0(x) 6= 1− p∗(x) and λ1(x) 6= p∗(x)


EstimationI Under (MPS.i) — (MPS.v)

∆ATE∗=

∫Sω(x)∆ATE (x)f (x)dx

= ∆ATE +∫

S[ω(x)− 1]∆ATE (x)f (x)dx

where

ω(x) =Pr(D = 1)Pr(D∗ = 1)

[1+

1p(x)

1− λ0(x)λ0(x) + λ1(x)− 1

]Pr(D∗ = 1) =

∫S[1− λ0(x)]f (x)dx

+∫

S[λ0(x) + λ1(x)− 1]p(x)f (x)dx

I Shows that ∆ATE∗can be obtained from an appropriately weighted

average of ∆ATE (x)I Weights depend on λ0(x), λ1(x)


NotesI Bounds obtained by computing ∆ATE

∗(λ0,λ1) over a grid of values

and obtaining the lower and upper bounds

F Restrictions on possible values of λ’s can be imposed based on prior infoF ∆ATE

∗(λ0,λ1) can be obtained using any propensity-score based

estimatorF In their paper, they use a (5 strata) stratification estimator and assume(λ0,λ1) are stratum-specific

I Extension to multi-valued treatments provided as well


Full Solutions

Point estimates possible using method-of-moments framework (Blacket al. 2000)

Brachet (2008) proposes following algorithm1 Estimate Hausman et al. (1998) misclassification probit (discussed inECO 6375), including an instrument z in the first-stage

2 Replace D with Pr(D∗i = 1|x , z) in second-stage

Almada et al. (2016), Nguimkeu et al. (2019) consider a similarapproach in a Bayesian framework

Calvi et al. (2017), DiTraglia & García-Jimeno is work in progress


Concluding Thoughts on Causal Inference

“Natural, quasi-, and computational experiments, as well as regression discontinuity design (RDD), can

all, when well applied, be useful, but none are panaceas... Because we are not an experimental science, we face

diffi cult problems of inference. The same data generally are subject to multiple interpretations. It is not that we

learn nothing from data, but that we have at best the ability to use data to narrow the range of substantive

disagreement. We are always combining the objective information in the data with judgment, opinion and/or

prejudice to reach conclusions...

Natural experiments, difference-in-difference, and regression discontinuity design are good ideas. They

have not taken the con out of econometrics – in fact, as with any popular econometric technique, they in some

cases have become the vector by which ‘con’is introduced into applied studies. Furthermore, over-enthusiasm

about these methods, when it leads to claims that single-equation linear model with sandwiched errors are all

we ever really need, can lead to our training applied economists who do not understand how to fully model a

dataset.” — Sims (2010)


In light of these sentiments, recall the points made at the start of thiscourse:

Prior to conducting, or when reviewing, causal analyses, questions thatneed to be answered:

1 What is the causal relationship of interest? [Is it economicallyinteresting?]

2 What is the identification strategy?3 What parameter are you actually estimating?4 To whom does the parameter apply?5 What question does the analysis answer?6 Are the data representative of the population of interest?7 Are the data reliable?8 What is the method of statistical inference?

While applied work is open to “multiple interpretations,” theseinterpretations and objections to research are lessened when one is precisein answering these questions.


Social InteractionsModels of social interactions have become very popular over the pasttwo decades

I Popularity stems from an increasing recognition of the importance ofagent interactions in the determination of many behaviors andoutcomes

I Examples include models ofF Peer effectsF Neighborhood effectsF Policy spillovers

See Manski (2000), Blume et al. (2011, 2015)


Identifying interactions are important for two main reasons1 Improved understanding of the determinants of behavioral choices andoutcomes

2 Identifying social multipliers to better understand effects of treatments

However, identifying social interactions is incredibly diffi cult and hasgiven rise to a rich literature

Fundamental identification challenge: Do the behaviors or attributesof agents have a causal effect on a other agents’behaviors oroutcomes within a reference group?

Attributing causation faces three econometric challenges:1 Possible non-random self-selection into reference groups2 Possible reference group-specific unobserved confounders3 Simultaneity that arises in an equilibrium set of agent choices


Setup

NotationI yig = behavior of agent i in reference group gI xi = vector of observed attributes of iI wg = vector of observed attributes of the group gI εi = unobserved attributes of iI αg = unobserved attributes of the group gI µe (y−ig ) = the beliefs of agent i about the behaviors of all agents ingroup g except i

To be clear

Observed by i Observed by Econometricianxjj∈g ,wg , εi , αg yj , xjj∈g ,wg


Agents choose y to maximize some objective fn

yig = arg maxυ∈Ωig

V (υ, xi ,wg , εi , αg , µe (y−ig ))

where Ωig is the agent’s choice setBeliefs

I Solving the model requires assumptions about how beliefs aredetermined

I Standard assumption imposes an equilibrium condition: self-consistencybetween subjective beliefs, µe (y−ig ), and the objective conditionalprobabilities of others given the information set, Fi , of agent i

µe (y−ig ) = µ(y−ig |Fi )where Fi = xjj∈g ,wg , εi , αg

SolutionI This is an incomplete information game in that agent i does notobserve ε−ig

I It is assumed to be a simultaneous move gameI The Bayes-Nash equilibrium yields behaviors of the form

yig = ψ(xi ,wg , εi , αg , µ(y−ig |Fi ))


NotesI Existence of an equilibrium is established using a fixed point theoremI It is possible that the objective fn, V , depends on wg and/or

µ(y−ig |Fi ), but yig does notF This implies that other agents’behaviors have externalities, but do notdirectly influence behavior

I ψ(·) is typically assumed to be supermodular s.t. ‘increases’inµ(y−ig |Fi ) increase yig

Social interactions are of three types in the model (Manski 1993)1 Contextual (or exogenous) effects: ∂ψ/∂wg 6= 02 Correlated effects: ∂ψ/∂αg 6= 03 Endogenous effects: ∂ψ/∂µ(y−ig |Fi ) 6= 0

F Contextual and correlated effects allow for similarity in the behaviors ofagents within a reference group due to predetermined (from theperspective of the model) observed and unobserved (from theperspective of the econometrician) attributes

F Endogenous effects allow for the possibility of simultaneity in the choiceof agent behaviors


Social InteractionsLinear-in-Means Model

Much empirical work uses the linear-in-means model

yig = β0 + xi β1 + wg β2 + β3meig + αg + εig

wheremeig =

1ng

∑j∈g

E[yj |Fi ],

ng is the reference group size, and εig is assumed to be mean zeroand uncorrelated across agentsNotes

I Blume et al. (2011) provide a justification for this functional formusing a Bayes-Nash equilibrium with a particular assumption on theform of agent preferences, V

I Manski (1993) assumes

wg = xg =1ng

∑j∈g

xj


Assuming ng is large enough s.t. each agent is small in the group(impliying that εi can be ignored), then each agent’s expectationregarding average behavior in the group will be identical and given by

meig = mg =β0 + xg β1 + wg β2 + αg

1− β3

implying that β3 < 1 must hold

The final equation is now given by

yig = β0 + xi β1 + wg β2 + β3

[β0 + xg β1 + wg β2 + αg

1− β3

]+ αg + εig

where social interactions are indicated by1 Contextual (or exogenous) effects: β2 6= 02 Correlated effects: Var(αg ) 6= 0 ∀g3 Endogenous effects: β3 6= 0


Estimation

Approach #1. Reduced form. wg = xg . αg = 0.I Estimating equation given by

yig =β0

1− β3+ xi β1 + xg

[β2 + β1β31− β3

]+ εig

= π0 + xiπ1 + xgπ2 + εig

I Typically estimated via OLS

F π2 6= 0⇔ β2 6= 0 and β2 6= −β1β3F However, cannot disentangle contextual vs endogenous effects


Approach #2. Instrumental variables. αg = 0.I Structural equation given by

yig = β0 + xi β1 + wg β2 + β3mg + εig

where

mg =β0 + xg β1 + wg β2

1− β3I If wg = xg , then mg is a linear fn of the intercept and xg , hence mgand xg are perfectly collinear and the model is not identified

yig = β0 + xi β1 + xg β2 + β3

[β0 + xg (β1 + β2)

1− β3

]+ εig

F Manski (1993) refers to this as the reflection problem

I Identification then requires an exclusion restriction; a variable mustimpact mg that does not impact yig conditional on mg


Approach #2 (cont.). Instrumental variables. αg = 0.I Identification is not obviousI Common (and perhaps ad hoc) solution is to assume wg ⊂ xg ⇒

yig = β0 + xi β1 + wg β2 + β3

[β0 + xg β1 + wg β2

1− β3

]+ εig

and the variables in xg but not wg identify the modelI Reduced form in this case becomes

yig =β0

1− β3+ xi β1 + wg

[β2

1− β3

]+ xg

[β1β31− β3

]+ εig

and yields identification of the various social interactions


Approach #3. Dynamic models. αg = 0.I Lagged model

yigt = β0 + xitβ1 + wgtβ2 + β3mgt−1 + εigt

where

mgt =β0 + xgtβ1 + wgtβ2

1− β3L

and L is the lag operatorI Recalling the mathematics of the lag operator, this implies that

mgt = m(xgt ,wgt , xgt−1,wgt−1...)

I Thus, mgt−1 is not perfectly collinear with the covariates in the modeland the model is identified if εigt is not serially correlated


Approach #4. Instrumental variables. αg 6= 0.I Structural equation given by

yig = β0 + xi β1 + wg β2 + β3mg + αg + εig

where

mg =β0 + xg β1 + wg β2 + αg

1− β3I Identification is not obvious and typically relies on untestableassumptions

I Option 1. Treat αg as a random effect since fixed effects estimationeliminates wg and mg as well

F But, Cov(mg , αg ) 6= 0, so need to instrument for mgF Assumption wg ⊂ xg may again work, but diffi cult to justify

I Option 2. Utilize panel data and assume αg is invariant along somedimension.

F Multiple time periods

yigt = β0 + xit β1 + wgt β2 + β3mgt + αg + εigt

F Multiple sub-reference groups

yisg = β0 + xi β1 + ws β2 + β3ms + αg + εisg

(e.g., s = neighborhoods, g = counties)DL Millimet (SMU) ECO 7377 Spring 2020 453 / 478

Social InteractionsSocial Network Models

Linear-in-means model presumes that interactions are generated bygroup-specific averages

Social network models provide further focus on the microstructure ofinteractions among agents and allow for heterogeneity of interactionsacross pairs of agents

Estimation methods borrow from the well-developed spatialeconometrics literature

See Jackson (2008), Blume et al. (2011)


NotationI P(i) denotes the set of peers of agent i

F Unlike Manski (1993), agent i is not in this setF Peer sets need not be symmetric (i.e., j ∈ P(i); i ∈ P(j))

I Social interactions are described by a weighted adjacency matrix

ωij =

1|P (i )| if j ∈ P(i)0 otherwise


Structural equation given by

yi = β0 + xi β1 +

(∑j 6=i

ωijxj

)β2 + β3

(∑j 6=i

ωijyj

)+ εi

which can be written in matrix form as

y = β0 ι+ xβ1 +Wxβ2 + β3Wy + ε

where ι is a column vector of 1s and W is the N ×N weight matrixReduced form is given by

y = β0(I−β3W )−1 + (I−β3W )

−1(β1 I+β2W )x + (I−β3W )−1ε

Model is identified if I, W , and W 2 are linearly independent(Bramoullé et al. 2009)

I Estimation via MLI Estimation via IV using ∑k 6=j ωjk xk to instrument for ∑j 6=i ωijyj(i.e., peers of agent i’s peers)


Social InteractionsSpatial Econometrics Models

General Nesting Spatial Model (GNS) is given by

yi= β0+x i β1+

(∑j 6=i

ωijxj

)β2+β3

(∑j 6=i

ωijyj

)+ui , ui= λ

(∑j 6=i

ωijuj

)+εi

whereI(∑j 6=i ωijxj

)= spatial lag of x

I(∑j 6=i ωijyj

)= spatial lag of the dependent variable

I(∑j 6=i ωijuj

)= spatial lag of the error term

Special casesI Spatial Lagged-X (SLX) model: β3 = λ = 0I Spatial Autoregressive (SAR) model: β2 = λ = 0I Spatial Error model (SEM): β2 = β3 = 0I Spatial Durbin (SD) model: λ = 0


Relationship among the models relies on similar math used in timeseries; here, the lags occur across space rather than time

SEM (in matrix form) is

y = xβ1 + u, u = λWu + ε

= xβ1 + λWy − λWxβ1 + ε

which is a restricted SDM (β2 = λβ1)

SAR (in matrix form) is

y = xβ1 + β3Wy + ε

= xβ1 + β3Wxβ1 + β23W2xβ1 + · · ·+ ε+ β3W ε+ β23W

2ε+ · · ·

which is an ‘infinite’SLX-SEM

The GNS is analogous to the complete model of social interactionswith contextual, endogenous, and correlated effects

Thus, the GNS is at best weakly identified if I, W , and W 2 arelinearly independent


Social InteractionsFinal Comments

Weight matrixI Is not identified; N2 elements, N observationsI May be assumed to be a function of a small set of unknownparameters, W (θ)

F Analogous to GLS

I Is typically assumed based on partitions in the data (e.g.,neighborhoods, schools, racial groups, etc.)

I May allow for ‘strong’and ‘weak’ties by allowing for weights to varyacross agents within the reference group for a given i (e.g., distancedecay)

I May be symmetric or asymmetricI Elements are typically row-standardized s.t. rows sum to oneI Mis-specifying the weight matrix should be viewed as measurementerror problem


Results can be interpreted in different ways based on direct, indirect,and total effects

I Analogous to short- and long-run effects in dynamic models in timeseries and panel

I Direct effect captures the marginal effect of a unit change in own xI (Average) indirect effect captures the (average) marginal effect of aunit change in peer x

I (Average) total effect is the sum of the prior two∂y1∂x1k

· · · ∂y1∂xNk

.... . .

...∂yN∂xk

· · · ∂yN∂x1k

=

(I−β3W )−1

β1k ω12β2k · · · ω1N β2k

ω21β2k. . .

. . ....

.... . .

. . . ωN−1,N β2kωN1β2k · · · ωN ,N−1β2k β1k


Additional literatures onI Endogenous network formation; identifying determinants of networkformation (e.g., Graham 2017)

Dij = I(wij β+ αi + αj − uij ≥ 0)

given data on network connections (Dij = 1 if ij connected, 0otherwise)

I Identifying network structure when this is unobserved (e.g., de Paulo etal. 2018)

I Spatial econometric models with discrete outcomes (e.g., Shang & Lee2011; Lee et al. 2014), endogenous W , measurement error (e.g.,Advani & Madle 2018)

I Identification based on variance restrictions (e.g., Graham 2008)I Treatment effects

F Related to addressing a failure of SUTVAF Manski (2013); Ferracci et al. (2014); Arpino et al. (2017)F Delgado & Florax (2015) → spatial DID model

Stata: -sp-


Effi ciency

Research on estimating of (in)effi ciency is quite prolificI Assessing extent of heterogeneity in productivity across firmsI Understanding determinants of differences in productivity

F Effects of firm-level factorsF Effects of policy factors

Econometric tools are also useful in unrelated contexts whereempirical specifications include a one-sided error term

I Non-classical measurement errorI Bargaining power


Historically, fundamental interest among economists in estimatingproduction functions given by

yi = f (xi , β)

Estimation proceeds in the usual way

β = argminβ

∑ (yi − f (xi , β))2

However, since it is a production function, one might additionallyimpose the constraint

yi ≤ f (xi , β)Issues

1 How to understand differences in y across firms with identical x?2 How to understand firms producing below the frontier, f (x , β)?


Effi ciencyStochastic Frontier

Early answers focused on measurement error as an explanation, butthis is viewed as inadequate

Farrell (1957) introduced the notion that production is stochastic innature in theoretical work

Aigner et al. (1977) and Meeusen and Van den Broeck (1977) soughtto incorporate the ideas of Farrell into empirical models

Here, empirical models allow for firms to be operating below thefrontier due to

1 Technical ineffi ciency2 Statistical noise (measurement error)


Technical ineffi ciency can be measured in two ways1 Output distance: Amount by which a firm can (proportionally) increaseits output maintaining its existing input levels

2 Input distance: Amount by which a firm can (proportionally) decreaseall its inputs maintaining its existing output level


Given this notion of technical ineffi ciency, empirical model is nowspecified as

yi = f (xi , β) + εi

where εi ≡ υi − ui andI υ = symmetric (two-sided) error term accounting for statistical noiseI u = asymmetric (one-sided) non—negative error term accounting fortechnical ineffi ciency

Firms must produce on or below their (stochastic) frontier,f (xi , β) + υi


Initial test for technical ineffi ciencyIf ui = 0 ∀i , then εi = υi which is symmetricIf ui > 0 for some i , then εi = υi − ui which is negatively skewedTest of skewness of OLS residuals, ε, should be performedStata: -sktest-

0.1

.2.3

.4

15 10 5 0 5

kdensity e kdensity vNote: v~N(0,1). u~abs(N(0,3)).


Estimation

Proceeds by MLI DGP given by

υiid∼ N(0, σ2υ)

uiid∼ N+(0, σ2u)

where N+ denotes the half-normal dbnI Other common choices for the dbn of u include

1 Truncated normal (identical to half-normal if mean zero)2 Exponential3 Gamma


Properties of the half-normal dbnI Denoted as N+(0, σ2u), where 0 and σ2u are not the mean and varianceof u

I PDF of u is obtained by setting Pr(u < 0) = 0 and re-scaling positiveprobabilities to sum to one

I Equivalent to defining the dbn of u as∣∣N(0, σ2u)∣∣

I Implies

E[u] =

√2π

σu

Var(u) = σ2u

(1− 2

π

)


Likelihood fnI Define γ = σ2u/σ2ε ∈ [0, 1], where σ2ε ≡ σ2υ + σ2uI Battese and Corra (1977) derive

lnL = −N2ln(π

2

)− N2ln(σ2ε ) +∑i ln (1−Φ(zi ))−

12σ2ε

∑i (yi − xi β)2

where

zi =yi − xi β

σε

√γ

1− γ

and yi = ln(qi ), where q is output, since typically C-D or translogproduction fn is used

Stata: -frontier-, -xtfrontier-


Estimating techical effi ciencyI Technical effi ciency for a firm is given by

TEi =qi

f (xi , β) + υi= · · · = exp(−ui )

I Estimation is not trivialI Estimator in Battese and Coelli (1988) given by

TE i = E[exp(−ui )|qi ] =

Φ(u∗iσ∗− σ∗

)(u∗iσ∗

) exp(σ2∗

2− u∗i

)

where

F u∗i = −(yi − xi β)γF σ2∗ = γ(1− γ)σ2ε


Extensions

Alternative estimation techniquesI Non-linear least squaresI Corrected OLSI GMMI Bayesian

Heteroskedastic ineffi ency

uiid∼ N+(0, σ2ui )

which implies

E[u] =

√2π

σui

Var(u) = σ2ui

(1− 2

π

)Battese and Coelli (1995) consider observable determinants, w , ofineffi ciency

ui = wi δ+ ηi


Effi ciencyAlternative Estimators

Other estimators of ineffi ciency aside from stochastic frontier models1 Data Envelopment Analysis2 Panel data approach


Data Envelopment Analysis

Nonparametric alternative approach to stochastic frontier modelsbased on linear programmingOriginates in Charnes et al. (1978)Fits a piecewise linear approximation to the production fn

Particularly susceptible to outliers, measurement errorDL Millimet (SMU) ECO 7377 Spring 2020 475 / 478

Panel data approach

Schmidt and Sickles (1984) introduce a panel data model with timeinvariant ineffi ciency

yit = α+ xitβ+ υit − ui

I Note, there is no firm-specific unobserved effect in the DGPI Re-write the model as

yit = αi + xitβ+ υit

where αi = α− uiI Algorithm

1 Estimate the model by FE/RE2 Obtain estimates, αi3 Assume the firm with largest value of αi is fully effi cient (i.e. ui = 0).Implies that α = αi for this firm.

4 Technical ineffi ciency of all remaining firms given by

ui = α− αi = maxj

αj− αi


Advantages1 No distributional assumption for u is needed2 FE estimation allows for u and x to be correlated

Disadvantages1 Assumes technical ineffi ciency is time invariant

F Note: Could instead have ‘panel’data at a single point in time, butwith multiple outputs

2 Assumes no other firm-level unobserved heterogeneity; αi solelycaptures technical ineffi ciency

F Extensions in Greene (2005a,b), Belotti and Ilardi (2018)


The End


ECO 7377 - faculty.smu.edufaculty.smu.edu/millimet/classes/eco7377/lectures/lectures.pdf · DL...

Documents

Transcript of ECO 7377 - faculty.smu.edufaculty.smu.edu/millimet/classes/eco7377/lectures/lectures.pdf · DL...