Chapter2. Minimum variance unbiased

estimation

Student: Işfan Geraldina ,1st year of Master in Communication Networks

01.11.2010

Coordinating professor: Sl. Dr. Eng. Corina Naforniţă

Summary

Minimum variance criterion

Existence of the minimum variance unbiased estimator

Finding the minimum variance unbiased estimator

Extension to a vector parameter

Conclusions

References

3

In searching for optimal estimators we need to

adopt some optimality criterion. A natural one is the

mean square error (MSE):

(*)

This measures the average mean squared deviation of

the estimator from the true value.

Minimum variance criterion

4

Adoption of this natural criterion leads to

unrealizable estimators, ones that cannot be written

solely as a function of the data.

Dem:

That shows that the MSE is composed of errors due

to the variance of the estimator as well as the bias.

5

Generally, any criterion which depends on the

bias will lead to an unrealizable estimator.

From a practical viewpoint the minimum MSE estimator

needs to be abandoned.

An alternative approach is to constrain the bias to be zero

and find the estimator which minimizes the variance.

Such an estimator is called the minimum variance

unbiased (MVU) estimator.

Note: The MSE of an unbiased estimator is just the

variance.

6

Existence of the Minimum Variance

Unbiased Estimator

Minimum variance unbiased (MVU) estimator : is an

unbiased estimator that has lower variance than any

other unbiased estimator for all possible values of the

parameter.

The question arises as to whether a MVU estimator

exists, i.e., an unbiased estimator with minimum

variance for all θ.

Two possible situations are described in the next figure.

7

If there are three unbiased estimators that exist and

whose variances are shown in a), θ3 is the MVU

estimator.

In the other figure, b), there is no MVU estimator

since for θ<θ0 , θ2 is better, while for θ>θ0 , θ3 is

better.

Fig 1.1 Possible dependence of estimator variance with θ

8

Finding the Minimum Variance

Unbiased Estimator

Despite the fact that the MVUE doesn't always exist, in

many cases of interest it does exist, and we need

methods for finding it. Unfortunately, there is no 'turn

the crank' algorithm for finding MVUE's.

There are, instead, a variety of techniques that can

sometimes be applied to find the MVUE.

9

These methods include:

1. Compute the Cramer-Rao Lower Bound, and check the

condition for equality.

2. Apply the Rao-Blackwell Theorem.

3. Restrict the class of estimators to be not only unbiased

but also linear. Then find the minimum variance

estimator within this restricted class.

1. and 2. may produce the MVU estimator, while 3. will yield it only

if the MVU estimator is linear in the data.

10

If θ = [θ1 θ2 …θp]T is a vector of unknown parameters,

then we say that an estimator

is unbiased if:

(**)

for i= 1,2,…p.

Extension to a Vector Parameter

11

By defining we can equivalently

define an unbiased estimator to have the property

for every θ conteined within the space defined in (**).

A MVU estimator has the additional property that

is minimum among all unbiased estimators.

12

Conclusions

MSE of an estimator is one of many ways to quantify

the difference between an estimator and the true

value of the quantity being estimated. MSE is an

expectation.

MVUE is an unbiased estimator that has lower

variance than any other unbiased estimator for all

possible values of the parameter.

13

An efficient estimator doesn’t need to exist, but if it

does, it’s the MVUE.

Since the mean square error (MSE) of an

estimator is

MSE(θ) = var(θ) + bias(θ)2

the MVUE minimizes MSE among unbiased

estimators .

In some cases biased estimators have lower MSE

because they have a smaller variance than does any

unbiased estimator.

14

References

Fundamentals of Statistical Signal Processing :

Estimation Theory, Steven M.Key, University of Rhode

Island.

Prediction and Improved Estimation in Linear Models, J.

Wiley, New York, 1977.

Unbiased estimators and Their Applications,

V.G Voinov, M.S. Nikulin, Kluwer Academic Publishers.

15

Thank you for your attention !