Propensity Score Matching Lava Timsina Kristina Rabarison CPH 786-001 Doctoral Seminar Fall 2012.

Propensity Score Matching

Lava TimsinaKristina Rabarison

CPH 786-001 Doctoral SeminarFall 2012

Introduction

• Program evaluation• Counterfactual outcome– what would have happened to the participants in

absence of treatment• Statistical techniques• Propensity score

Concept of PSM

• Identify neighborhoods that are as similar as possible to each other with respect to the probability of receiving the treatment (Chiavegatto Filho, Kawachi, & Gotlieb, 2012).

• The average treatment effects is then measured based on the mean difference in outcomes across these comparison and treatment groups.

Experimental Vs Non-experimental

• Experimental evaluation – Random assignment to treatment and control

control group can be regarded as counterfactual.

Motivation to Propensity Score Matching

• Non-experimental evaluation– Random assignment may not be possible in

nonexperimental evaluation methods (Heinrich et al., 2010).

– Assignment to treatment is often nonrandom and hence may bias participation and treatment outcomes.

– Treatment units are matched with their “similar” counterparts that differ only in the treatment under study.

– Extent of matching is challenging

• Propensity score matching allows this matching problem to be reduced to a single dimension.

• Let me restate that propensity score is defined as the probability that a unit in the combined sample of treated and control units receive the treatment, given a set of observed covariates.

Assumptions for PSM

• PSM holds under two assumptions (Khandker, 2010; Rosenbaum & Rubin, 1983):– Conditional Independence or Unconfounded

Assumption – Common Support or Overlap Condition

• Conditional Independence or Unconfounded Assumption:– conditional on observable covariates, the outcomes

are independent of treatment• In absence of randomization, the groups may differ not

only in the treatment status, but also in other covariates. Thus it is necessary to control for these covariates to avoid potential biases. There is a set of covariates observable to the researcher, such that after controlling for these covariates, the potential outcomes (r1,r0) are independent of the treatment status:

(r1,r0) Z|X⊥

• Common Support Condition Assumption– This condition ensures that treatment

observations have comparison observations “nearby” in the propensity score distribution. For each value of X, there is positive probability of being both treated and untreated:

0<P(Z=1|X)<1– Also called overlap condition

Steps of PSM (European Commission, 2009; Khandker, 2010):

1. Estimating a model of program participation2. Defining the region of common support and

balancing tests3. Matching participants to nonparticipants4. Estimating the Average Effect and its

Standard Error


1. Estimating a model of program participationi. Samples of participants and nonparticipants

should be pooled, ii. Participation Z should be estimated on all the

observed covariates X in the data that are likely to determine participation.

Probit or logit model of program participation This predicted outcome represents the estimated

probability of participation or propensity score for every sampled participants and non-participants.


2. Defining the region of common support and balancing tests

– The region of common support needs to be defined where distributions of the propensity score for treatment and comparison group overlap.


• Some of the participant and nonparticipant observation falling outside the region of common support may have to be dropped.


• Balancing tests can be conducted to check whether, within each quantile of the propensity score distribution, the treatment and comparison groups have similar average propensity scores and the mean of X. That is, the distributions of the treated group and the control group must be similar:

(X|Z=1) = (X|Z=0)p̂� p̂�


3. Matching participants to nonparticipants– Matching participants to nonparticipants on the

basis of propensity score can be done using different matching technique.

Techniques of matching

1. Nearest-neighbor (NN) matching– Each treatment unit is matched to the comparison unit

with the closest propensity score. Matching can be done with or without replacement.

– In this method, both treatment and control groups are first randomly sorted. Then the first treatment unit is selected to find its closest control match based on the absolute value of the difference between the propensity score (or logit of the propensity score) of the selected treatment and that of the control under consideration. The closest control unit is selected as a match.


2. Caliper or radius matching– This method is similar to NN matching except it

adds restriction. Both treatment and control units are randomly sorted and then the first treated unit is selected to find its closest control match in terms of the propensity score but only if the control’s propensity score is within a certain radius (caliper).

– This avoids bad matching and ensures that the matched pairs are within a certain range of propensity scores.


3. Stratification or interval matching– Partitions the common support into different strata

(or intervals) and calculates the program’s impact within each interval.

– The treated and control groups are ranked on the basis of their propensity scores, and then grouped into K intervals (strata).

– Then the impact for each kth stratum is evaluated.– The overall impact is the weighted average of the

strata effects, with weights proportional to the number of treated units in each stratum.


4. Kernel and local linear matching– Kernel matching uses weighted averages of all

individuals in the control group to construct the counterfactual outcome.

– Weights depend on the distance between each individual from the control group and the participant observation for which the counterfactual is estimated.

– The kernel function assigns higher weight to observations close in terms of propensity score to a treated individual and lower weight on more distant observations.


4. Estimating the Average Effect and its Standard Error– compute the sample averages of the two groups and

calculate the difference– Standard errors can be computed using bootstrapping

methods.

Example

• Research question: Does homelessness affect physical health, as measured by pcs score from the SF-36?

A glimpse at the datasetproc print data=ref.help (obs=10);var id pcs mcs age female i1 homeless;run;

Obs ID PCS MCS AGE FEMALE I1 HOMELESS1 1 58.4137 25.1120 37 0 13 0

2 2 36.0369 26.6703 37 0 56 1

3 3 74.8063 6.7629 26 0 0 0

4 4 61.9317 43.9679 39 1 5 0

5 5 37.3456 21.6758 32 0 10 1

6 6 46.4752 55.5090 47 1 4 0

7 7 24.5150 21.7930 49 1 13 0

8 8 65.1380 9.1605 28 0 12 1

9 9 38.2709 22.0297 50 1 71 1

10 10 22.6106 36.1438 39 0 20 1

Modeling as linear regressionproc reg data=ref.help;model pcs=homeless;run;

Number of Observations Read 453

Number of Observations Used 453

Parameter Estimates

Variable DFParameter

EstimateStandard

Error t Value Pr > |t|Intercept 1 49.00083 0.68802 71.22 <.0001

HOMELESS 1 -2.06405 1.01292 -2.04 0.0422

Parameter Estimates


EstimateStandard


HOMELESS 1 -1.14707 0.99794 -1.15 0.2510

AGE 1 -0.26593 0.06410 -4.15 <.0001

FEMALE 1 -3.95519 1.15142 -3.44 0.0006

I1 1 -0.08079 0.02538 -3.18 0.0016

MCS 1 0.07032 0.03807 1.85 0.0654

proc reg data=ref.help;model pcs = homeless age female i1 mcs;run;quit;

Creating propensity scoresproc logistic data=ref.help desc;model homeless = age female i1 mcs;output out=propen pred=propensity;run;

Association of Predicted Probabilities and Observed Responses

Percent Concordant 64.9 Somers' D 0.302

Percent Discordant 34.7 Gamma 0.304

Percent Tied 0.4 Tau-a 0.151

Pairs 50996 c 0.651

Looking for the assumption of common support

proc means data=propen;class homeless;var propensity;run;

Analysis Variable : propensity Estimated Probability

HOMELESSN

Obs N Mean Std Dev Minimum Maximum0 244 244 0.4296704 0.1166290 0.2136791 0.78760001 209 209 0.4983750 0.1382013 0.2635031 0.9642827

proc univariate data=propen;class homeless;var propensity;histogram propensity;run;

Looking for the assumption of common support

Restricting analysis to common support region

Parameter Estimates

Variable Label DFParameter

EstimateStandard

Error t Value Pr > |t|Intercept Intercept 1 54.19945 1.98265 27.34 <.0001

HOMELESS 1 -1.19613 1.03893 -1.15 0.2502

propensity Estimated Probability 1 -12.09909 4.33385 -2.79 0.0055

proc reg data=propen;where propensity<0.8;model pcs=homeless propensity;run;quit;

Macrosdata prop2;set propen;if propensity<0.8;run;

%include "C:\Documents and Settings\LRTI222\Desktop\vmatch.sas";%include "C:\Documents and Settings\LRTI222\Desktop\dist.sas";%include "C:\Documents and Settings\LRTI222\Desktop\nobs.sas";

%dist(data=prop2, group=homeless, id=id, mvars=propensity, wts=1, vmatch=Y, a=1, b=1, lilm=201, dmax=0.1, outm=mp1_b, summatch=n, printm=N, mergeout=mpropen);

Before & After Matchingproc means data=propen mean;class homeless;var age female i1 mcs;run;

HOMELESSN

Obs Variable Mean0 244 AGE

FEMALEI1MCS

35.04098360.2745902

13.512295132.4868303

1 209 AGEFEMALEI1MCS

36.36842110.1913876

23.038277530.7308549

proc means data=mpropen mean;where matched;class homeless;var age female i1 mcs;run;

HOMELESSN

Obs Variable Mean0 201 AGE

FEMALEI1MCS

35.62189050.1791045

15.915422931.4815123

1 201 AGEFEMALEI1MCS

36.14925370.1990050

19.945273630.9176772

Propensity score matched analysisproc reg data=mpropen;where matched;model pcs=homeless;run;quit;

Parameter Estimates


EstimateStandard


HOMELESS 1 -1.79386 1.07763 -1.66 0.0968

Propensity Score Matching Lava Timsina Kristina Rabarison CPH 786-001 Doctoral Seminar Fall 2012.

Documents

Transcript of Propensity Score Matching Lava Timsina Kristina Rabarison CPH 786-001 Doctoral Seminar Fall 2012.