Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a...
Transcript of Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a...
![Page 1: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/1.jpg)
Chapter 7: Point Estimation
MATH 450
September 19th, 2017
MATH 450 Chapter 7: Point Estimation
![Page 2: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/2.jpg)
Where are we?
Week 1 · · · · · ·• Chapter 1: Descriptive statistics
Week 2 · · · · · ·• Chapter 6: Statistics and SamplingDistributions
Week 4 · · · · · ·• Chapter 7: Point Estimation
Week 7 · · · · · ·• Chapter 8: Confidence Intervals
Week 10 · · · · · ·• Chapter 9: Test of Hypothesis
Week 13 · · · · · ·• Two-sample inference, ANOVA, regression
MATH 450 Chapter 7: Point Estimation
![Page 3: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/3.jpg)
Overview
7.1 Point estimate
unbiased estimatormean squared errorbootstrap
7.2 Methods of point estimation
method of momentsmethod of maximum likelihood.
7.3 Sufficient statistic
7.4 Information and Efficiency
Large sample properties of the maximum likelihood estimator
MATH 450 Chapter 7: Point Estimation
![Page 4: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/4.jpg)
Questions for this chapter
given population parameter θ (e.g., population mean µ,population variance σ2)
use a random sample sample X1,X2, . . . ,Xn to find a goodestimate θ̂ of θ
MATH 450 Chapter 7: Point Estimation
![Page 5: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/5.jpg)
Point estimate
Definition
A point estimate θ̂ of a parameter θ is a single number that can beregarded as a sensible value for θ.
⇒ a point estimate are computed from a sample⇒ a point estimate is a statistic⇒ a point estimate is random
MATH 450 Chapter 7: Point Estimation
![Page 6: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/6.jpg)
Example 1: population mean and standard deviation
Last week, we saw a lot of:“Let X1,X2, . . . ,Xn be a random sample from a distributionwith population mean µ and standard deviation σ”
What are the population parameters?
What is a good estimate of µ?
What is a good estimate of σ?
MATH 450 Chapter 7: Point Estimation
![Page 7: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/7.jpg)
Example 2: Linear regression
Model:Yi = β0 + β1Xi + εi , εi ∼ N (0, σ2)
MATH 450 Chapter 7: Point Estimation
![Page 8: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/8.jpg)
Example 3: Phylogenetics
MATH 450 Chapter 7: Point Estimation
![Page 9: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/9.jpg)
How should the next vaccines be designed?
MATH 450 Chapter 7: Point Estimation
![Page 10: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/10.jpg)
Point estimate
population parameter =⇒ sample =⇒ estimate
θ =⇒ X1,X2, . . . ,Xn =⇒ θ̂
MATH 450 Chapter 7: Point Estimation
![Page 11: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/11.jpg)
Mean Squared Error
Measuring error of estimation
|θ̂ − θ| or (θ̂ − θ)2
The error of estimation is random
Definition
The mean squared error of an estimator θ̂ is
E [(θ̂ − θ)2]
MATH 450 Chapter 7: Point Estimation
![Page 12: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/12.jpg)
Bias-variance decomposition
Problem
Recall that the variance of random variable X is computed by
V (X ) = E [X 2]− (EX )2
Prove that
MSE (θ̂) = E [(θ̂ − θ)2] = V (θ̂) +(E (θ̂)− θ
)2Bias-variance decomposition
Mean squared error = variance of estimator + (bias)2
MATH 450 Chapter 7: Point Estimation
![Page 13: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/13.jpg)
Unbiased estimators
Definition
A point estimator θ̂ is said to be an unbiased estimator of θ if
E (θ̂) = θ
for every possible value of θ.
Unbiased estimator
⇔ Bias = 0
⇔ Mean squared error = variance of estimator
MATH 450 Chapter 7: Point Estimation
![Page 14: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/14.jpg)
Sample mean as an unbiased estimator
Proposition
If X1,X2, . . . ,Xn is a random sample from a distribution with meanµ, then X̄ is an unbiased estimator of µ.
Proof: E (X̄ ) = µ.
LetT = a1X1 + a2X2 + . . .+ anXn,
then the mean and of T can be computed by
E (T ) = a1E (X1) + a2E (X2) + . . .+ anE (Xn)
Let a1 = a2 = . . . an = 1/n
MATH 450 Chapter 7: Point Estimation
![Page 15: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/15.jpg)
Sample variance
Problem
Let
S2 =1
n − 1
[(∑X 2i
)− 1
n
(∑Xi
)2]Compute E (S2)
Step 1:
S2 =1
n − 1
[(∑X 2i
)− n(X̄ )2
]Step 2:
E [S2] =1
n − 1
[(∑E [X 2
i ])− nE [(X̄ )2]
]
MATH 450 Chapter 7: Point Estimation
![Page 16: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/16.jpg)
Sample variance
Step 1:
S2 =1
n − 1
[(∑X 2i
)− n(X̄ )2
]Step 2:
E [S2] =1
n − 1
[(∑E [X 2
i ])− nE [(X̄ )2]
]Recall:
Var [Y ] = E (Y 2)− (EY )2
MATH 450 Chapter 7: Point Estimation
![Page 17: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/17.jpg)
Sample variance
Step 2:
E [S2] =1
n − 1
[(∑E [X 2
i ])− nE [(X̄ )2]
]Recall:
Var [Y ] = E (Y 2)− (EY )2
Step 3:
E [S2] =1
n − 1
[(∑Var [Xi ] + (E [Xi ])
2)− n
(E [(X̄ )2]
)]=
1
n − 1
[(∑σ2 + µ2
)− n
(Var [X̄ ] + (E [X̄ ])2
)]
MATH 450 Chapter 7: Point Estimation
![Page 18: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/18.jpg)
Sample variance
Step 3:
E [S2] =1
n − 1
[(∑Var [Xi ] + (E [Xi ])
2)− n
(E [(X̄ )2]
)]=
1
n − 1
[(∑σ2 + µ2
)− n
(Var [X̄ ] + (E [X̄ ])2
)]Recall:
Var [X̄ ] =σ2
n, E [X̄ ] = µ
Step 4:
E [S2] =1
n − 1
[nσ2 + nµ2 − n
(σ2
n+ µ2
)]=
1
n − 1
[(n − 1)σ2
]= σ2
MATH 450 Chapter 7: Point Estimation
![Page 19: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/19.jpg)
Sample variance as an unbiased estimator
Theorem
The sample variance
S2 =1
n − 1
[(∑X 2i
)− 1
n
(∑Xi
)2]is an unbiased estimator of the population variance σ2.
MATH 450 Chapter 7: Point Estimation
![Page 20: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/20.jpg)
Minimum variance unbiased estimator (MVUE)
Definition
Among all estimators of θ that are unbiased, choose the one thathas minimum variance. The resulting θ̂ is called the minimumvariance unbiased estimator (MVUE) of θ.
Recall:
Mean squared error = variance of estimator + (bias)2
unbiased estimator ⇒ bias =0
⇒ MVUE has minimum mean squared error among unbiasedestimators
MATH 450 Chapter 7: Point Estimation
![Page 21: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/21.jpg)
Example 7.1 and 7.4
A test is done with probability of success p
n independent tests are done, denote by Y the number ofsuccesses
What is a good estimate of p?
MATH 450 Chapter 7: Point Estimation
![Page 22: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/22.jpg)
Sample proportion
A test is done with probability of success p
n independent tests are done, denote by Y the number ofsuccesses
Denote by Xi the result of test i th, where Xi = 1 when thetest success and Xi = 0 if not, then
Each Xi is distributed by
x 0 1p(x) 1-p p
E [X ] =?Moreover,
Y =n∑
i=1
Xi
E [Y ] =?
MATH 450 Chapter 7: Point Estimation
![Page 23: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/23.jpg)
Sample proportion
A test is done with probability of success p
n independent tests are done, denote by Y the number ofsuccesses
Let
p̂ =Y
n
the E [p̂] = p, i.e., p̂ is an unbiased estimator
MATH 450 Chapter 7: Point Estimation
![Page 24: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/24.jpg)
Sample proportion
A test is done with probability of success p
n independent tests are done, denote by Y the number ofsuccesses
Let
p̂ =Y
n
the E [p̂] = p, i.e., p̂ is an unbiased estimator
Crazy idea: How about using
p̃ =Y + 2
n + 4
What is the bias of p̃?
MATH 450 Chapter 7: Point Estimation
![Page 25: Chapter 7: Point Estimation · Point estimate De nition A point estimate ^ of a parameter is a single number that can be regarded as a sensible value for .)a point estimate are computed](https://reader036.fdocuments.net/reader036/viewer/2022070806/5f04c89b7e708231d40faf8f/html5/thumbnails/25.jpg)
Example 7.1 and 7.4
MATH 450 Chapter 7: Point Estimation