Econometrics Appendices B and C

8/11/2019 Econometrics Appendices B and C

1/133

1

ECO3021S 2013Weeks 1 and 2

Econometrics

Wooldridge, Appendix B and C

Fundamentals o !ro"a"ilit#$

%at&ematical Statistics

'at&erine E#al


2/133

2

C&anges

(nlike pre)ious #ears, *e *on+t "e doing t&e la*

o iterated expectations, or t&e proo t&ereo, t&ust&ese slides taken out o t&e originals posted on)ula


3/133

3

&ttp--***.lickr.com-p&otos-doug/////-


4/133

4

Some uestions o t&e Week

W&at is an estimator

o* do *e measure "ias in an estimator

W&at is t&e dierence "et*een a cd and a pd

oes sample si4e matter or an estimator

W&at is a consistent estimator

?W&at are t&e ormulae or E567, Co)56,87, 9ar567


5/133

5

What is an experiment?

:t can "e repeated infinitely,:t &as well defined outcomes,

;&ink coin toss, t&ro* o t&e die, gender o a "a"#

at "irt&, outcomes o a randomised controlleddrug trial etc etc...

Eac& time *e perorm an experiment

5i.e. perorm anot&er trialo t&e experiment7,*e ma# get a dierent outcome


6/133

6

W&at is a


7/133

7

But#ou said numerical )alues, &o* is a coin toss *it&

&eads and tails numeric

We must assign )alues to eac& o our outcomes,like 1 or &eads, 0 or tails

Note

Capitals 6,8, used to denote random )aria"les

Smalls denote particular outcomes x 2, # ?


8/133

8

9er# common


9/133

9

Discrete !"

#initeor counta"l# inite num"er o )alues

6 &as k possi"le )alues >x1, x2, I, xkD

Wit& !ro"a"ilit#56xi7 pior eac& xi>p1, p

2, I, p

kD

NB" Eac& 0 J piJ 1, Kp

i 1

:s a Bernoulli


10/133

10

$robability Density #unctiono 6

5xL7 !56 xL7 pL, L 1,2, I, k

5x7 is t&e pro"a"ilit# t&at 6 takes on t&e )alue x.

For


11/133

11


12/133

12

% continuous !is a


13/133

13


14/133

14

;&e CF means *e can talk a"outa ran'e ofvalues, and t&e pro"a"ilit# t&at 6 lies in t&em.

!5a J 6 M "7 F5x7 N 5x7 dxarea under t&e cur)e "et*een a and "

;&e cd is t&e integral o t&e pd 5continuous


15/133

15

E.G.6 is num"er o ree t&ro*s made "# a "asket"allpla#er out o 2 attempts. 6 can take on >0,1,2D

oing "ack to t&e pd on slide 11,W&at is !56 P 17

i.e. *&at is pro"a"ilit# t&at a pla#er makes at leastone ree t&ro*


16/133

16

!roperties o CFs

For an# num"er c, !56 P c7 1 Q F5c7 1 = !56 J c7

For an# num"ers aJ",

!5a J 6 J "7 F5"7 = F5a7 !56 J "7 = !5x J a7

We use cds onl# or continuous


17/133

17

)oint Distribution

or 6,8 2 discrete


18/133

18

*ndependence

: 6, 8 are independent, R so are 567 and 58761, 62, I, 6ndiscrete


19/133

19

Binomial istri"ution

i)en 6 81T 8

2T I T 8

n

W&ere eac& 8i&as pro"a"ilit# o success H, and

t&e 8iare independent, t&en 6 G"inomial5n,H7


20/133

20

&onditionalistri"utions

Bayes ule

!58#U6x7 8U65#Ux7 6,85x,#7

65x7

i.e.

&onditional +- /oint -,+ mar'inal -

W& t i 6 8 i d d t W& t


21/133

21

W&at i 6, 8 are independent W&at can *e sa#a"out

8U65#Ux7 or

6U85xU#7

!58#U6x7 8U65#Ux7

6,85x,#7

65x7 65x7

85#7

85#7

65x7

oes t&is &old or all t#pes o


22/133

22

Expected !alue

E567 *eig&ted a)erage o)er outcomes o 6

6 a discrete


23/133

23

Vote

We can get an ans*er or t&e expected )alue*&ic& is not one o t&e outcomes. :s t&at odd


24/133

24

$roperties of Expected !alues

g567 some unction o 6, 6 a discrete


25/133

25

!roperties o Expected !alues

6, 8 discrete


26/133

26

E5c7 c

For constants a," E5a6 T "7 aE567 T "

Expected )alue o a sum is t&e sum o t&eexpected )alues 5i.e. eac& a 17

: 6 81T 8

2T I T 8

k, i.e. 6 is BinomialG5n,H7

and eac& 8iGBernoulli5H7

E56 87X Kk Kk 5 7 5 7


27/133

27

E56,87X KkL1Kk&15x&,#L76,85x&,#L7

: 6, 8 independent, s&o* E5687 E567E587

E5687 KkL1

Kk&1

686,8

5x&,#

L7

KkL1

Kk&1

6865x

&7

85#

L7

(by defn of independence)

KkL1

665x

&7 Kk

&18

85#

L7

(take Xs out of sum of Y as they are constants in

that sum) E567E587

(by definition)


28/133

28

0edian

6 >Q?, 0, 2, /, 10, 13, 1D

%edian o 6 /;&e middle )alue or odd sets.

Evensets &a)e t*o median )alues, or t&ea)erage.

6 >Q@,3,,1D%ed567 3 or , or t&e a)erage o t&e t*o


29/133

29

If X is continuous, then the median of X, say,m, is the value such that one-half of the area

under the pdf is to the left of m, and one-half of the area is to the right of m.

Vote%ed567 E567 *&en 6 is s#mmetric


30/133

30

V t t t k ti lik


31/133

31

Vo* *e start to ask 1uestionslike

o* ar is 6 rom E567,or as *e *ill call it, 2, or2x

W&at does t&e distributionlook like

Fat, skinn# and tall, ske*ed to one side or t&eot&er...

We *ant to kno* t&e )ariance o 6= &o* muc& it )aries around t&e mean


32/133

32

! i 3 4t d d D i ti


33/133

33

!ariance 3 4tandard Deviation

a)erage distance o 6 rom its mean 2

Q 9ar567 denoted 56

or5x6

Q W do *e suare t&e distance to t&e mean

Q 9ar567 E56QY72X E5627 = Y2


34/133

34

!ariance !roperties

9ar567 0i t&ere is a const c, s.t. !56 c7 1,

and i so, t&en E567 c.

9ar 5a6 T "7 a29ar567

9ar567 H51QH7 or 6GBernoulli5H7


35/133

35

4tandard Deviation5 or5x

Z sd567 T [9ar567sd5c7 0

sd5a6 T "7 UaUsd567

4t d di i


36/133

36

4tandardisin'a


37/133

37

4(ewness and 7urtosis


38/133

38

&ovariance

o* do 6 and 8 )ar# *it& eac& ot&er

Q : E567 0 or E5870,R Co)56,87 E5687


39/133

39

&ovariance

&8!9"6, 8 independent R Co)56,87 0

: pro)ing, remem"er E5687 E567E587 i 6,8independent

NB"Co)56,87 0 does VO; R t&at 6,8 independent

E.g. *it& )aria"les 6 and 8 62, t&e# &a)eCo)56,87 0 5C&eck7 "ut are clearl# related.

&8!6" For constants a " a "


40/133

40

&8!6"For constants a1, "

1, a

2, "

2

%ake sure #ou can s&o* t&e a"o)e. Noteto alterco)56,87, multipl# 6 "# some constant. Vot good\

&8!:" Cauc Sc&*art4 :neualit#

UCo)56,87U J ZxZ

#

Note" &ov;-,-< E556QYx756QYx77 !ar;-


41/133

41

So Co)56,87 measure o linear relationship

NB Co)56,87 depends on units of measurement


42/133

42

&89" 1 Corr56,87 1.

So Corr56,87 1 R a perect positi)e linear

relations&ip "et*een 6 and 8, *e can *rite8 aT"6

Corr56,67 Co)56,67-Zx2 - 9ar567-9ar567 1

&86"

Correlation is in)ariant to units o measurementCorr5a16 T "1, a28 T "27 Corr56,87 i a1, a2P 0Corr5a16 T "1, a28 T "27 QCorr56,87 i a1, a2J 0


43/133

43

!roperties

Co)56,87 measures linear dependence

Corr56,87 1 R 6,8 perfectly linearly relatedi.e. 8 a T "6 or some constants a and "P0

Corr56,87 0 R no linearrelations&ip


44/133

44

9ariance o Sums

!%:"

9ar5a6 T "87 a29ar567 T "29ar587 T 2a"Co)56,87

: Co)56,87 0, t&en

9ar56 T 87 9ar567 T 9ar5879ar56Q87 9ar567 T 9ar587

9ariance o Sums


45/133

45

9ariance o Sums

!%="$airwise uncorrelated variables

i)en >61, 62, ..., 6nD, *it& Co)56i, 6L7 0 or all i, LR

9ar5a161 T...T an6n7 a12

9ar5617 T...T an2

9ar56n7

;&e variance of the sumis eual to t&e sum ofthe variances i all t&e ai 1.

For 6 G Binomial5n H7


46/133

46

For 6 Binomial5n,H7,i.e. 6 81 T 82 T I T 8n,

and eac& 8 is an independent Bernoulli5H7


47/133

47

p

E8U6xX

e.g. W&at is expected income, or t&ose *it&education 12 #ears

We still do a *eig&ted sum, "ut the wei'hts arenow different= t&e# are t&e conditional pd.

We can s&o* E8U6X grap&icall# or in a ta"le.


48/133

48

Conditional Expectation


49/133

49


50/133

50

!roperties o CE


51/133

51

!roperties o CE&E9"

Ec567U6X c567i.e. i *e kno* 6, *e also kno* c567. Functions o6 "e&a)e as constants *&en *e condition on 6.

&E6"Ea5678 T "567U6X a567E8U6X T "567

&E:"

6,8 are independent R E8U6X E8X

E.g. E68 T 262U6X 6E8U6X T 262.

: (,6 independent, $ E5(7 0,E5(U67 E5(7 0 5"# deinition o independence7


52/133

52

E5(U67 E5(7 0 5"# deinition o independence7

J^ea)e out CE.?QCE.PConditional 9ariance

9ar58U6x7 E82U6X = 5E8U6X72

E.g. 9ar5Sa)ingsU:ncome7 ?00T0.2@:ncome

&!9": 6 and 8 are independent,

R9ar58U67 9ar587

Vormal and ot&er distri"utions...


53/133

53

We rel# &ea)il# on normal>Gaussiandistri"utions

Q to simplifypro"a"ilit# calculations, "#assuming our 6s are normall# distri"utedQ to conduct inference

6GVormal5Y,Z27

Will "e gi)en on a ormula s&eet.


54/133

54

;&e normal distri"ution is s#mmetric,so median567 E567


55/133

55

so median567 E567

Birt& *eig&ts, test scores, unemplo#ment ratestend to ollo* a normal distri"ution.

:ncome, price )aria"les tend not to.

We can transorm )aria"les to "e normalised, e.g."# using log567 instead o 6. ;&en 6 is lo'

normal.

?

Standard Vormal GVormal50,17


56/133

56

as pd

CF denoted _547 = ;@< A;@


57/133

57

!roperties


58/133

58

;@< A;@


59/133

59

$N9": 6 V l5 27 t& 56 7- V l50 17


60/133

60

: 6GVormal5Y,Z27, t&en 56QY7-ZG Vormal50,17

So i 6GVormal53,?7, *&at is !56J 17

!56J 17 !556 = 37-2J 51Q37-27 !5 J Q17 0.1@/ !5P 17 1 = !5J 17

1 = 0./?13 0.1@/^ook at ex B. on ! ?0.

$N6" 6 G Vormal5Y,Z27R a6 T " G Vormal5aY T " a2Z27


61/133

61

R a6 " Vormal5aY ",a Z 7.

$N:"i)en 6 and 8 Lointl# normall# distri"uted,6,8 independent i Co)56,8 7 0.

$N="An# linear com"ination o independent,

identicall# distri"uted normal


62/133

62

: 6GVormal51,7,*&at is t&e distri"ution o 8 26 T 3 !?0.

Ot&er istri"utions&hi 41 ared


63/133

63

&hi 41uared"^et

i, i 1,2,...n,

"e independent


64/133

64

tdistri"ution


65/133

65

^et GVormal50,17, 6G62n, , 6 independent

;Gtn

; is similar to normal distn, "ut more spread out,as t gets larger, it approac&es t&e std normal.

Given E5;7 0 or nP19ar5;7 n-5nQ27 or nP2


66/133

66

#distri"ution


67/133

67

61G62

k1, 62G62k2, assume 61, 62independent

FGFk1,k2

Order o t&e degrees o reedom is important

5numerator is irst7.

Note 8ou &a)e t&e tools to calc t&e exp )alueand )ariance o t&e t and c&i s distri"utions


68/133

68

Appendix C !opulations,!arameters $


69/133

69

!arameters $


70/133

70

Appendix C !opulations,

!arameters $


71/133

71

1. :denti# populationo interest2. Speci# a modelor relations&ip o interest

3. %odels in)ol)e probability distributions

*&ic& depend on unkno*n parameterso t&emodel

!arameters determine direction$ stren'thorelations&ips = e.g. return to education, eect oneig&"our&ood *atc& programs on crime

Sampling


72/133

72

8 = random )aria"le representing a population, *it&

probability density function5#,H7 H is unkno*n

!ro"581 #7 !ro"582 #7 I !ro"58n #7 5#,H7,

81, 82, I, 8nare independent random )aria"les *it&common pd 5#,H7 t&en >81, 82, I, 8nD is a randomsamplerom 5#,H7

E.. Famil# income rom n100 amilies

>20000,2/000,@?0000,1.@m, 20m, @000, I, 1000D

Sampling


73/133

73

;&is stu is trick#. :.e.

Q Bernoulli)aria"le = does passenger iarri)eor t&eir lig&t !ro"a"ilit# t&e# do is Bernoullidistri"ution.

!58i 17 H, !58i 07 1QH !assengers independent, t&ereore 81, 82etc

are independent, identically distributed

5i.i.d7, and >81, 82, I, 8nD are a random samplerom densit# 5#, H7, *&ere &ere 5#,H7 is aBernoulli distn, and popnis all passengers

E & d l t il t&


74/133

74

Eac& random sample not necessaril# t&e same

5t&e luck o t&e dra*7:s it appropriate to assume random samplin'E.g. rom a (C; class, or gender 5ma#"e7, race,

academic a"ilit#, income 5pro"a"l# not7 And romall matric graduates

F can "e any prob distn= normal, Bernoulli,

Binomial, t, F, 62

Finite Sample !roperties oEstimators


75/133

75

Estimators#inite t&ese sample properties &old or an# si4eo sample, "ig or small 5"ut *e don+t let n R `7.

i)en a random sample >81, 82, I, 8nD,

dra*n rom a population t&at depends on H,an estimatoro H is a rule*&ic& assigns eac&possi"le outcome o t&e sample a )alue o H

e.g. W &581, 82, I, 8n7

;&e rule s&ouldn+t c&ange *it& dierent samples.


76/133

76

Example

>81, 8

2, I, 8

nD a random sample *it& mean Y

An estimator o Y could "e

enoted t&e sample avera'e + bar


77/133

77

An estimatorW o a parameter H can "eexpressed W &581, 82, I, 8n7


78/133

78

1 2 n

his Lust an# old unction.Wis a


79/133

79

W&en talking a"out E5W7 or 9ar5W7

W&ere W is an estimator, t&is isv differentto*&en W is a


80/133

80

W is an unbiasedestimator o H i

E5W7 Hor all )alues o H

So or a gi)en sample, *ill W H Or close to H

NB" Hnbiased definition" : *e take infinitelymany samples, and o"tain W in eac& sample,t&en t&e a)erage o all t&e Ws *ould eual H

: W is aB*%4EDestimator o H, t&en

Bias5W7 E5W7 = H \ 0


81/133

81

Bias5W7 E5W7 H \ 0

Q ;&e "ias depends on h, and on t&epdf of +.

Q For e.g. :s t&e sample avera'eun"iased

^ooks like no. W Because E58 "ar7 Y.


82/133

82


83/133

Sample 9ariance


84/133

84

;&is estimator is unbiased= E5S27 Z2. ;&is

result #ou deinitel# can s&o*=ask me i #ou can+t.

We divide by nI9 and not n"ecause Y isestimated, and not kno*n. : *e kne* n, *e *ould

calculate t&e a"o)e *it& Y replacing 8 "ar, anddi)iding t&roug& "# n and not nQ1.

(n"iased estimator good estimator?


85/133

85

(n"iased estimator good estimator?%a#"e, "ut )ariance is also important.

+ou can 'et an unbiased estimator which ispretty bad.E.g. n 100, (se rule W 81581is

t&e irst o"ser)ation o t&e sample7 to estimate Y,and t&us E5W7 Y 5take enoug& samples,calculate W, and t&us E5W7 *ill Y7. So W isun"iased, "ut a prett# dum" t&ing to do 5t&ro*sout all ino7.

We care a"out t&e samplin' distributiontoo.

Sampling 9ariance


86/133

86

9ariance o t&e Sample A)erageo* spread out is t&e estimator


87/133

87

p

W&ere did *e use t&e independence o t&e 8is

;o summarise

For random sample >81, 8

2, I, 8

nD *it& 8

iG 5Y,Z27,

t&en E58 "ar7 Y, "ut 9ar58 "ar7 Z2-n

So no* *&at


88/133

88

9ar58 "ar7 Z

2

-n:s 9ar58 "ar7 a


89/133

89

Eicienc#: W and W are un"iased estimators o H


90/133

90

: W1and W

2are un"iased estimators o H,

W1is efficientrelati)e to W2i

9ar5W17 J 9ar5W27

For all HCan *e al*a#s c&oose an estimator usingeicienc# Sometimes not, sometimes t&e# &a)e

dierent )ariances or dierent )alues o H.S&ould *e compare t&e eicienc# o "iasedestimators Vope\ See page @ or e.g.%SE p@

As#mptotic !roperties o Estimators

9ar58 7 Z2 9ar58 "ar7 Z2 -n


91/133

91

9ar5817 Z , 9ar58 "ar7 Z -n

W&at &appens to 81and 8 "ar as n R `:s adding more data al*a#s good

Wnan estimator o H rom sample >81, 82, I, 8nDo si4e n. Wnis a consistent estimatoro H i

!5 U Wn= H U P 7 R 0 as n R `, or e)er# P 0.

: Wnis consistent, plim;Wn< L


92/133

92

VB, re B:AS

o *e actuall# take lots and lots o samples ;&at


93/133

93

# p

*ould cost a lot o mone# rig&t:t+s a thou'ht experiment= *e don+t actuall# doit, except ma#"e in a computer simulation = "ut

*e t&ink a"out it to ans*er uestions a"out whatvaluest&e estimator *ould gi)e in eac& o t&osesamples, and whethert&e a)erage o t&ose

)alues *ould eual t&e population )alue

o #ou kno* t&e popn )alue Vo, "ut *e s&o*alge"raicall# t&at E5W7 popn )alue

VB unbiased= &as to do *it& *&at &appens i*e take lots o samples, calculated W in eac&

sample, and take t&e expected )alue o t&e Ws,


94/133

94

p , p ,

and see i t&e# eual H&onsistency= *&at &appens to Wnas n R `

;&e euation is a s&ocker, "ut t&e intuition s&ouldmake sense

:ntuiti)e explanation : Wnis consistent, W

n

"ecomes more and more concentrated around H5Wn R H7 as n R `


95/133

95

Consistenc# is a minimal re1uirement.i.e. i an estimator doesn+t e)en tend to H as

n R `, t&en it+s not muc& use to us.

For Wn an un"iased estimator o H, another way

of sayin' that Wnis consistent, "esides

plim5Wn7 H, is to sa# !ar;Wn< J M as n J K.

^ t >8 8 8 D " i d d t id ti ll

^a* o ^arge Vum"ers


96/133

96

^et >81

, 82

, I, 8n

D "e independent, identicall#distri"uted random )aria"les *it& mean Y. ;&en

plim 58n "ar7 Y

;o estimate Y, *e Lust need to pick a large enoug&sample to get ar"itraril# close to Y

:n englis& : #ou pick a "ig enoug& randomsample, #our sample a)erage *ill tend to*ards#our popn a)g.


97/133

97

!roperties o pro"a"ilit# limitsi)en H a parameter,

deine b g5H7 or a continuous unction g5H7


98/133

98

deine b g5H7 or a continuousunction g5H7.

Suppose plim5Wn7 H.

eine an estimator o b "# n g5Wn7.

;&en $F*0 9plim ';W

n< ';plim W

n


99/133

99

plim ';Wn< ';plim Wn


100/133

100

p

:t is consistent or Z25s&o*ing t&is is complex "utpossi"le, "ut not reuired or t&is course7.

;&e sample standard deviationS srt5S27 is

not un"iased, "ut it is consistent or Z.

Not covered in lectures"


101/133

101

As#mptotic Vormalit#!age 0 mid*a#, !1

eneral Approac&es to !arameter Estimation!2, !3, &al o !?

:nter)al Estimation, Conidence:nter)als


102/133

102

Estimated rate return to education


103/133

103

let >81, 8

2, I, 8

nD "e random sample rom popn

;&en+ bar Normal;2, 9>n


132/133

132

;&en+ bar Normal;2, 9>n

Econometrics Appendices B and C

Documents

Transcript of Econometrics Appendices B and C