7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
1/21
Chapter 3Cramer-Rao Lower Bound
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
2/21
Abbreviated: CRLB or sometimes just CRB
CRLB is a lower bound on the variance of any unbiased
estimator:
The CRLB tells us the best we can ever expect to be able to do
(w/ an unbiased estimator)
then,ofestimatorunbiasedanisIf
)()()()( 2
CRLBCRLB
What is the Cramer-Rao Lower Bound
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
3/21
1. Feasibility studies ( e.g. Sensor usefulness, etc.) Can we meet our specifications?
2. Judgment of proposed estimators Estimators that dont achieve CRLB are looked
down upon in the technical literature
3. Can sometimes provide form for MVU est.
4. Demonstrates importance of physical and/or signal
parameters to the estimation problem
e.g. Well see that a signals BW determines delay est. accuracy
Radars should use wide BW signals
Some Uses of the CRLB
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
4/21
Q: What determines how well you can estimate ?
Recall: Data vector is x
3.3 Est. Accuracy Consideration
samples from a random
process that depends on an
the PDF describes thatdependence:p(x;)
Clearly ifp(x;) depends strongly/weakly on we should be able to estimate well/poorly.
See surface plots vs. x & for 2 cases:
1. Strong dependence on 2. Weak dependence on
Should look atp(x;) as a function of forfixed value of observed data x
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
5/21
Surface Plot Examples ofp(x; )
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
6/21
( )
= 2
2
2 2)]0[(exp
21];0[
AxAxp
x[0] =A + w[0]
Ex. 3.1: PDF Dependence for DC Level in Noise
w[0] ~N(0,2
)
Then the parameter-dependent PDF of the data pointx[0] is:
A x[0]
A3
p(x[0]=3;)
Say we observe x[0] = 3So Slice atx[0] = 3
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
7/21
The LF = the PDFp(x;)
but as a function of parameter w/ the data vectorx fixed
Define: Likelihood Function (LF)
We will also often need the Log Likelihood Function (LLF):
LLF = ln{LF} = ln{p(x;)}
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
8/21
LF Characteristics that Affect Accuracy
Intuitively: sharpness of the LF sets accuracy But How???
Sharpness is measured using curvature: ( )
valuetruedatagiven
2
2 ;ln
==
x
xp
Curvature PDF concentration Accuracy
But this is for a particular set of data we want in general:
SoAverage over random vector to give the average curvature:
( )
valuetrue
2
2 ;ln
=
xpE
Expected sharpness
of LF
E{} is w.r.t p(x;)
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
9/21
Theorem 3.1 CRLB for Scalar Parameter
Assume regularity condition is met:
=
0
);(ln xpE
E{} is w.r.t p(x;)
Then
( )
valuetrue
2
2
2
;ln
1
=
xpE
3.4 Cramer-Rao Lower Bound
( ) ( )dxxp
ppE );(
;ln;ln2
2
2
2
=
xx
Right-Hand
Side is
CRLB
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
10/21
1. Write log 1ikelihood function as a function of:
ln p(x;)
2. Fix x and take 2ndpartial of LLF:
2
lnp(x;)/2
3. If result still depends on x:
Fix and take expected value w.r.t. x
Otherwise skip this step
4. Result may still depend on :
Evaluate at each specific value ofdesired.
5. Negate and form reciprocal
Steps to Find the CRLB
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
11/21
Need likelihood function:
( )
[ ]( )
( )
[ ]( )
=
=
=
=
2
21
0
22
1
02
2
2
2exp
2
1
2exp2
1
;
Anx
Anx
Ap
N
nN
N
nx
Example 3.3 CRLB for DC in AWGN
x[n] =A + w[n], n = 0, 1, ,N 1w[n] ~N(0,2)
& white
Due to
whiteness
Property
of exp
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
12/21
( ) [ ]( )!!! "!!! #$
!! "!! #$
?(~~)
1
0
22
0(~~)
22
212ln);(ln
=
=
=
=
A
N
n
A
N
AnxAp
x
Now take ln to get LLF:
[ ]( ) ( )AxNAnxApA
N
n
==
=2
1
02
1);(ln
x
Now take first partial w.r.t.A:sample
mean
(!)
Now take partial again:
22
2
);(ln
NAp
A
=
x
Doesnt dependon x so we dont
need to do E{}
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
13/21
Since the result doesnt depend on x orA all we do is negate
and form reciprocal to get CRLB:
( ) NpE
CRLB2
valuetrue
2
2 ;ln
1
=
=
=
xN
A2
}var{
Doesnt depend on A
Increases linearly with 2
Decreases inversely with N
CRLB
For fixedN
2
A
CRLB
For fixedN& 2
CRLB Doubling Data
Halves CRLB!
N
For fixed 2
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
14/21
Continuation of Theorem 3.1 on CRLB
There exists an unbiased estimator that attains the CRLB iff:
[ ]
=
)()(
);(lnx
xgI
p
for some functionsI() andg(x)
Furthermore, the estimator that achieves the CRLB is then given
by:
)( xg=
(!)
Since no unbiased estimator can do better this
is the MVU estimate!!
This gives a possible way to find the MVU: Compute ln p(x;)/ (need to anyway) Check to see if it can be put in
form like (!)
If so theng(x) is the MVU esimator
with CRLBI
==)(
1}var{
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
15/21
Revisit Example 3.3 to Find MVU Estimate
( )AxN
ApA =
2);(ln x
For DC Level in AWGN we found in (!) that:
Has form of
I(A)[g(x) A]
[ ]
=
===1
0
1)(N
n
nxN
xg xCRLBN
ANAI === 22
}var{)(
So for the DC Level in AWGN:
the sample mean is the MVUE!!
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
16/21
Definition: Efficient Estimator
An estimator that is:
unbiased and
attains the CRLBis said to be an Efficient Estimator
Notes:
Not all estimators are efficient (see next example: Phase Est.)
Not even all MVU estimators are efficient
So there are times when our
1st partial test wont work!!!!
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
17/21
Example 3.4: CRLB for Phase Estimation
This is related to the DSB carrier estimation problem we used
for motivation in the notes for Ch. 1
Except here we have a pure sinusoid and we only wish to
estimate only its phase
Signal Model: ][)2cos(][
];[
nwnfAnx
ons
oo ++=!!! "!!! #$
AWGN w/ zeromean & 2
Assumptions:
1. 0
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
18/21
Problem: Find the CRLB for estimating the phase.
We need the PDF:
( )
( )
[ ]( )
+
=
=2
21
0
22 2
)2cos(
exp
2
1;
nfAnx
po
N
nN
x
Exploit
Whiteness
and Exp.
Form
Now taking the log gets rid of the exponential, then taking
partial derivative gives (see book for details):
( )[ ]
21
02
)24sin(2
)2sin(;ln
++
=
=
nf
Anfnx
Apoo
N
n
x
Taking partial derivative again:
( )[ ]( ))24cos()2cos(
;ln 1
0
22
2
++
=
=
nfAnfnxAp
oo
N
n
x
Still depends on random vectorx so need E{}
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
19/21
Taking the expected value:
( )
[ ]( )
[ ]{ }( ))24cos()2cos(
)24cos()2cos(
;ln
1
02
1
022
2
++=
++=
=
=
nfAnfnxEA
nfAnfnx
A
E
p
E
oo
N
n
oo
N
n
x
E{x[n]} = A cos(2 fon + )
So plug that in, get a cos2
term, use trig identity, and get
( )SNRN
NAnf
ApE
N
n
o
N
n
=
+=
=2
21
0
1
02
2
2
2
2)24cos(1
2
;ln
x
=N
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
20/21
Now invert to get CRLB:SNRN
1
}var{
CRLB Doubling Data
Halves CRLB!
N
For fixed SNR
Non-dB
CRLB Doubling SNR
Halves CRLB!
SNR (non-dB)
For fixedN Halve CRLBfor every 3B
in SNR
7/30/2019 EECE 522 Notes_04 Ch_3A CRLB Definition
21/21
Does an efficient estimator exist for this problem? The CRLB
theorem says there is only if
[ ]
=
)()(
);(lnx
xgI
p
( )
[ ]
21
02 )24sin(2)2sin(
;ln
++
=
=
nf
A
nfnx
Ap
oo
N
n
x
Our earlier result was:
Efficient Estimator does NOT exist!!!
Well see later though, an estimator for which CRLB}var{asN or as SNR
Such an estimator is called an asymptotically efficient estimator
(Well see such a phase estimator in Ch. 7 on MLE)
CRBN
}var{
Top Related