1
CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE
Suppose that you have alternative estimators of a population characteristic , one unbiased, the other biased but with a smaller variance. How do you choose between them?
0
0
pro
bab
ilit
y d
ensi
ty
estimator B
estimator A
One way is to define a loss function which reflects the cost to you of making errors, positive or negative, of different sizes.
2
CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE
0
-3 0 3
loss
error (negative) error (positive)
04
pro
bab
ilit
y d
ensi
ty
estimator B
3
A widely-used loss function is the mean square error of the estimator, defined as the expected value of the square of the deviation of the estimator about the true value of the population characteristic.
222MSE ZZZEZ
CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE
Z
bias
04
pro
bab
ilit
y d
ensi
ty
estimator B
4
222MSE ZZZEZ
CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE
Z
bias
The mean square error involves a trade-off between the variance of the estimator and its bias. Suppose you have a biased estimator like estimator B above, with expected value Z.
04
pro
bab
ilit
y d
ensi
ty
estimator B
5
222MSE ZZZEZ
CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE
Z
bias
The mean square error can be shown to be equal to the sum of the variance of the estimator and the square of the bias.
6
CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE
Mean square error = variance + bias squared
To demonstrate this, we start by subtracting and adding Z .
22
22
22
22
22
2
2
2
2
2
2
MSE
ZZ
ZZZZZ
ZZZZ
ZZZZ
ZZZZ
ZZ
ZE
ZEEZE
ZZE
ZE
ZEZ
7
CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE
Mean square error = variance + bias squared
We expand the quadratic using the rule (a + b)2 = a2 + b2 + 2ab, where a = Z – Z and b = Z – .
22
22
22
22
22
2
2
2
2
2
2
MSE
ZZ
ZZZZZ
ZZZZ
ZZZZ
ZZZZ
ZZ
ZE
ZEEZE
ZZE
ZE
ZEZ
8
CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE
Mean square error = variance + bias squared
We use the first expected value rule to break up the expectation into its three components.
22
22
22
22
22
2
2
2
2
2
2
MSE
ZZ
ZZZZZ
ZZZZ
ZZZZ
ZZZZ
ZZ
ZE
ZEEZE
ZZE
ZE
ZEZ
9
CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE
Mean square error = variance + bias squared
The first term in the expression is by definition the variance of Z.
22
22
22
22
22
2
2
2
2
2
2
MSE
ZZ
ZZZZZ
ZZZZ
ZZZZ
ZZZZ
ZZ
ZE
ZEEZE
ZZE
ZE
ZEZ
10
CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE
Mean square error = variance + bias squared
(Z – ) is a constant, so the second term is a constant.
22
22
22
22
22
2
2
2
2
2
2
MSE
ZZ
ZZZZZ
ZZZZ
ZZZZ
ZZZZ
ZZ
ZE
ZEEZE
ZZE
ZE
ZEZ
11
CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE
Mean square error = variance + bias squared
In the third term, (Z – ) may be brought out of the expectation, again because it is a constant, using the second expected value rule.
22
22
22
22
22
2
2
2
2
2
2
MSE
ZZ
ZZZZZ
ZZZZ
ZZZZ
ZZZZ
ZZ
ZE
ZEEZE
ZZE
ZE
ZEZ
12
CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE
Mean square error = variance + bias squared
Now E(Z) is Z, and E(–Z) is –Z.
22
22
22
22
22
2
2
2
2
2
2
MSE
ZZ
ZZZZZ
ZZZZ
ZZZZ
ZZZZ
ZZ
ZE
ZEEZE
ZZE
ZE
ZEZ
13
Hence the third term is zero and the mean square error of Z is shown be the sum of the variance of Z and the bias squared.
22
22
22
22
22
2
2
2
2
2
2
MSE
ZZ
ZZZZZ
ZZZZ
ZZZZ
ZZZZ
ZZ
ZE
ZEEZE
ZZE
ZE
ZEZ
CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE
Mean square error = variance + bias squared
14
In the case of the estimators shown, estimator B is probably a little better than estimator A according to the MSE criterion.
CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE
0
0
pro
bab
ilit
y d
ensi
ty
estimator B
estimator A
Copyright Christopher Dougherty 2012.
These slideshows may be downloaded by anyone, anywhere for personal use.
Subject to respect for copyright and, where appropriate, attribution, they may be
used as a resource for teaching an econometrics course. There is no need to
refer to the author.
The content of this slideshow comes from Section R.6 of C. Dougherty,
Introduction to Econometrics, fourth edition 2011, Oxford University Press.
Additional (free) resources for both students and instructors may be
downloaded from the OUP Online Resource Centre
http://www.oup.com/uk/orc/bin/9780199567089/.
Individuals studying econometrics on their own who feel that they might benefit
from participation in a formal course should consider the London School of
Economics summer school course
EC212 Introduction to Econometrics
http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx
or the University of London International Programmes distance learning course
EC2020 Elements of Econometrics
www.londoninternational.ac.uk/lse.
2012.10.31
Top Related