Seminar 9 - Forsiden - Universitetet i Oslo€¦ · high 1 =high quality resume call back 1...

Seminar 9

ECON4150

University of Oslo

April 25, 2014

ECON4150 (Dep. of Economics) Seminar 9 April 25, 2014 1 / 16

Names dataset

Randomly assigned ”white-sounding” names and ”African americansounding” names to resumes and sent out as job application

Recored the number of ”call backs” from employers

Data used in article published in AER, 2004, Vol. 94, no. 4: ”AreEmily and Greg More Employable that Lakisha and Jamal? A FieldExperiment on Labor Market Discrimination”


Key variables

Variable name Descriptionfirst name applicant’s first namefemale 1 = femaleblack 1 = blackhigh 1 =high quality resumecall back 1 =applicant was called backchicago 1 = data from Chigaco


Call back rate by race

The call back rate for whites is 0.097 while for blacks is 0.064.


Confidence interval

YW − Y B + /− 1.96 ∗ se(YW − Y B)

0.097 − 0.064 + /− 1.96 ∗√

(s2Wnw

+s2BnB

)

0.033 + /− 1.96 ∗√

(0.2952

2435+

0.2462

2435)

[0.018, 0.048]

Using equation 3.19 and s2w is sample variance for white and is 0.2952 andnw number of observation white = 2435. The difference is statisticallysignificant at a 5% level. The difference implies that 9,7% of resumes ofwhites receive call back while only 6,5% of resumes of black receive callback which is a large difference.


Call back rate by race for men

White men have a call back rate of 8,8% while black men have a call backrate of 5,8% giving a difference of: 3%


Call back rate by race for women

White women have a call-back rate of 9,9% while black women have a callback rate of 6,6% giving a difference of 3,3%.


Significance of gender difference

The coefficient femaleblack = female*black and measure the differentialeffect of being black for female relative to male. The effect is insignificantindicating that the racial gap is equal for men and women.


High versus low quality CV

High quality resumes have a call back rate of 8,7% while low qualityresumes have a call back rate of 7,3%. (difference 1,4%)


High versus low quality CV - whites

High quality resumes have a call back rate of 10,8% while low qualityresumes have a call back rate of 8,5%. (difference (2,3%)


High versus low quality CV - blacks

High quality resumes have a call back rate of 6,7% while low qualityresumes have a call back rate of 6,2%. (difference (0,5%)


High vs low quality

(1) (2)call back call back

high 0.0141 0.0229(0.00779) (0.0120)

black -0.0231∗

(0.0106)

high black -0.0178(0.0156)

cons 0.0734∗∗∗ 0.0850∗∗∗

(0.00530) (0.00801)

N 4870 4870

Standard errors in parentheses∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001


High vs low quality

Black is significant, thus there is a significant difference betweenblack and whites

High is insignificant, thus there is no significant difference betweenhigh quality resumes and low quality resumes

Interaction high black is insignificant, thus there is no significantdifference in call back between blacks and whites with high qualityresumes.

Thus racial difference stems from a difference in call back ratesbetween whites with low quality resumes and blacks with low qualityresumes. Blacks with low quality resumes receive fewer call backsthan whites with low quality resumes (8,5%-6,2% = 2,3%)


Failure to randomize


Failure to randomize - variables we use

None of the coefficients are individually significant

Neither is the coefficients jointly significant as the p-value of theF-statistic of the full regression is 0.8385. (Joint test that all includedvariables are zero)

We cannot reject the null that treatment was randomly assigned.


Failure to randomize - all variables

If you run the regression with all the control variables the p-value of theF-statistic of the full regression is still insignificant. However, if you lookindividually at computer skills it is statistically significant and indicatesthat the blacks have slightly higher computer skills. However, statistically,5% of the control variables will be significant at the 5% level even if thetrue effect is zero as the probability of falsely rejecting the null is 5%.


Seminar 9 - regular exercises

ECON4150

University of Oslo

April 25, 2014

ECON4150 (Dep. of Economics) Seminar 9 - regular exercises April 25, 2014 1 / 9

Exercise 13.3 a - Average treatment effect

Average treatment effect is given by the average SAT score in thetreatment group less the average SAT score in the control group:

X̄treatment − X̄control − 1241− 1201 = 40

The estimated effect of being assigned to a preparatory course is 40 pointson the SAT test.


Exercise 13.3 b - Non-random assignment

Non-random assignment occurs if the treatment is not assigned randomlybut instead based (fully or partly) on characteristics or preferences of thesubjects. Example:

All participants were offered the preparatory course but only thosethat wanted took it

The information about the preparatory course was given over theschool e-mail so only those that check their e-mail could attend

Only those that struggled on the mid-term was offered the course

etc, etc.

If there is non-random assignment the outcome will reflect both the effectof the treatment and the effect of the non-random assignment and wecannot separate out the effect of the treatment as E (ui |Xi ) 6= 0



To estimate the pure treatment effect the treatment and control must beidentical on observables and non-observables.Test for non random assignment:

Are treatment and control identical on observables (W). Here:Gender distribution

Run a regression of treatment variable (Xi ) on the observables andcompute the F-statistics of whether the coefficients of the W’s arezero.

Randomization has failed of we can reject the joint hypothesis that allcoefficients are zero.



Manually when we have only one control variable:

Let p̂ be the probability that a male is assigned to treatment group.Which if there is complete randomization should be 0.5 thus:

H0 : p̂men = 0.5

H1 : p̂men 6= 0.5

tmen =p̂men − H0,1

se(p̂men)=

p̂men − H0,1√1n p̂men(1− p̂men)

=0.55− 0.50√1

1000.55(1− 0.55)= 1

From e.q. 2.7. σX =√p(1− p) when X is binary and

eq 2.48 st.dev(X̄ ) = σX√n

t90,10%two−sided = 1.66. Cannot reject the null. No evidence ofnon-random assignment


Exercise 13.5a

Question: Is drop out of male ahtletes a threat to internal validity?

Potential problem Attrition - subjects drop out after they have beenrandomly assigned to treatment and control.Cause a bias in the OLS estimator if the attrition isrelated to the treatment itself because treatment andcontrol will differ on observables or unobservables. Thusthe results may be due to differences in who remained ineach group rather than the effect of the treatment.If attrition is related to the treatment we get a sampleselection bias and the treatment and the error term willbe correlated

Solution The attrition is here unrelated to the treatment and thus wedo not get a bias. However, the results are the averagetreatment effect on the population excluding male athleteswhich may be different from the average treatment effect ofthe full population.

No internal validity problem as the attrition is unrelated to the treatment.ECON4150 (Dep. of Economics) Seminar 9 - regular exercises April 25, 2014 6 / 9

Exercise 13.5b

Question: Engineers in the control set up a LAN to share?

Problem: Failure to follow treatment protocol: When theparticipants in the experiment do not, or only partiallycomply with the treatment protocol.Subjects assigned to the control group receive treatmentor subjects assigned to treatment do not receivetreatment which bias the estimate of the treatmenteffect

Solution: If the experimenter knows who actually received thetreatment, the treatment effect can be estimated using theinitial random assignment as instrument for treatment.

Thus the LAN is a threat to internal validity as it creates a bias due tofailure to follow treatment protocol, however this bias can be corrected forby using IV.


Exercise 13.5c

Question: The art majors never learn how to access their internetaccounts

Potential problem Failure to follow treatment protocol

Solution The treatment is defined as ”availability of internet in dormrooms”

The art majors have received the treatment and thus there is no threat tointernal validity. (Treatment is not defined as ”use of internet in dormrooms”.


Exercise 13.5d

Question: Economics students sell access to their internet

Potential problem: Failure to follow treatment protocol, students thatwere not supposed to receive treatment do recieve treatment

Solution: Use the initial random assignment as instrument fortreatment if you have information about who actuallyreceived treatment

Answer: Again failure to follow treatment protocol as in b. and the OLS isbiased but can be corrected for using IV if the experimenter knows whoreceived access and who was initially assigned access.


Seminar 9 - Forsiden - Universitetet i Oslo€¦ · high 1 =high quality resume call back 1...

Documents

Transcript of Seminar 9 - Forsiden - Universitetet i Oslo€¦ · high 1 =high quality resume call back 1...