P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive...
-
Upload
lillian-parker -
Category
Documents
-
view
216 -
download
1
Transcript of P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive...
![Page 1: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/1.jpg)
PROBABLITY
STATICS&
![Page 2: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/2.jpg)
PROJECT.1Assuming that the error terms are distributed as:
Please derive the maximum likelihood estimator for the simple linear regression model,assuming the regressor X is given (that is, not random -- this is also commonly referred to as conditioning on X = x). One must check whether the MLE of the model parameters ( and ) are the same as their LSE’s.
![Page 3: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/3.jpg)
Solution
The p.d.f. is
We derive the Maximum Likelihood Estimator for the simple linear regression model:
![Page 4: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/4.jpg)
Derive the total probability function L
Derive lnL
![Page 5: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/5.jpg)
We compute the and , namely that
Note:
![Page 6: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/6.jpg)
We also set up
We will still use them in next projects.
![Page 7: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/7.jpg)
It's easy to get the result:
As we can see, they are the same as their LSE's
![Page 8: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/8.jpg)
PROJECT.2Errors in Variable (EIV) regression.
In this case, let's derive its distribution.
![Page 9: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/9.jpg)
![Page 10: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/10.jpg)
X and Y follow a bivariate normal distribution:
And we need to derive the MLE of the regression slope :
![Page 11: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/11.jpg)
In order to simplify the calculation,we also bulit a simpleEIV model.
In this model, are not ramdon any longer.It means that Xand Y are independent.And we can also derive the MLE of the regression slope as same expression .
![Page 12: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/12.jpg)
SolutionLet's derive the MLE for simple EIV model:
![Page 13: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/13.jpg)
Let 、 and
then we have
Here, stands for .
![Page 14: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/14.jpg)
So we can get
Now,we will pin-point which special cases correspond to the OR and the GMR(they are 2 kinds of way to derive the linear regression).
![Page 15: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/15.jpg)
the Orthogonal Regression
So,we have
![Page 16: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/16.jpg)
the Geometric Mean Regression
So,we have
![Page 17: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/17.jpg)
PROJECT.3Our third project is to derive a class of non-parametricestimators of the EIV model for simple linear regressionbased on minimizing the sum of the following distance fromeach point to the line as illustrate din the figure below:
![Page 18: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/18.jpg)
Please also show whether there is a 1-1 relationship between this class of estimators and those in Project 2(A/B). That is, try to ascertain whether there is a 1-1 relationship between c and .
![Page 19: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/19.jpg)
We know that
Therefore ,we need to minimize the sum
and
Here, stands for
![Page 20: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/20.jpg)
Let and ,we can get that
![Page 21: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/21.jpg)
Solving these two equations, we can get s.t.
When we have the examples, we can put them in this founction,then we can compute the .
![Page 22: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/22.jpg)
PROJECT.4In the project 3,we have found that and are 1-1 relationship.
In this case, we can easily derive that and are 1-1 relationship, so and are 1-1 relationship.
![Page 23: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/23.jpg)
PROJECT.6For those who have finished Projects 2 & 3 & 4 above, youmay also examine how to estimate the error variance ratio , when we have two repeated measures on each sample.
![Page 24: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/24.jpg)
Let be the material and measurement.Therefore, we have
It’s easy to know these elements are independent.
![Page 25: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/25.jpg)
In order to simplify the calculation,we also bulit a simpleEIV model.
![Page 26: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/26.jpg)
Let and ,we can get
and
Let and ,we can get
![Page 27: P ROBABLITY S TATICS &. PROJECT. 1 Assuming that the error terms are distributed as: Please derive the maximum likelihood estimator for the simple linear.](https://reader031.fdocuments.net/reader031/viewer/2022032805/56649eeb5503460f94bfd066/html5/thumbnails/27.jpg)
Therefore