Founded 1348
-
Upload
rebekah-spence -
Category
Documents
-
view
37 -
download
0
description
Transcript of Founded 1348
Founded 1348Charles University
http://www.fsv.cuni.cz
Institute of Information Theory and Automation
Academy of Sciencesof the Czech Republ
Institute of Information Theory Institute of Economic Studies Faculty of Social Sciences
Charles UniversityPrague
Institute of Economic Studies Faculty of Social Sciences
Charles UniversityPrague
ROBUSTIFYING INSTRUMENTAL VARIABLES
and AutomationAcademy of Sciencesof the Czech Republ
Jan Ámos VíšekJan Ámos Víšek
COMPSTAT 2004 Prague
23.-27.8. 2004
http://samba.fsv.cuni.cz/~visek/compstat
Prague 23.-27.8. 2004 COMPSTAT 2004
ROBUSTIFYING INSTRUMENTAL VARIABLES
http://samba.fsv.cuni.cz/~visek/compstat
Topic of presentation
● Recalling definition of the Least Weighted Squares
● Proposing an instrumental version of the Least Weighted Squares
● Recalling the “classical” Instrumental Variables and their robust version
● Conditions for their consistency and asymptotic normality
● An algorithm for their evaluation
● A heretic question at the end
● Why another robust version of the Instrumental Variables ?
Classical regression ( by the Ordinary Least Squares )
the Ordinary Least Squares
are not consistent, hence ....
If
,
Notice that the true value of the vector of regression coefficients is
i.i.d. r.v.’s Model
The Instrumental Variables
The Instrumental Variables
are consistent.
as “close” as possible to
Notice that the Instrumental Variables are solution of the normal equations
BUT
.
.
continued
The Instrumental Variables
the Instrumental Variables are vulnerable to the influential observations.
Víšek, J.Á. (2000): Robust instrumental variables and specification test. Proc. PRASTAN 2000, ISBN 80-227-1486-0, 133 - 164..
Víšek, J.Á. (1998): Robust instruments. Proc. Robust'98 (ed. J. Antoch & G. Dohnal) Union of Czechoslovak Mathematicians and Physicists, 195 - 224.
As ,
What about M-version of the Instrumental Variables,
i.e.
? or
Bickel, P.J. (1975): One-step Huber estimates in the linear model.
Jurečková J., P. K. Sen (1984): On adaptive scale-equivariant
JASA 70, 428-433.
Statistics and Decisions, vol. 2 (1984), Suppl. Issue No.1.
continued
The Instrumental Variables
for discussion see
scale- and regression-equivariant, Since M-estimators are not
M-estimators in linear models.
the M-version of Instrumental Variables is not
scale- and regression-equivariant, too !
continued
The Instrumental Variables
Jurečková J., P. K. Sen (1984): On adaptive scale-equivariant
Statistics and Decisions, vol. 2 (1984), Suppl. Issue No.1.
● Studentization of residuals by an estimator of scale which has to be scale-equivariant and regression-invariant
M-estimators in linear models.
Víšek, J.Á. (1998): Robust estimation of regression model. Bulletin of the Czech Econometric Society,
Let’s employ the Least Weighted Squares .....
There are basically two possibilities:
see again
● To start with a robust, scale- and regresion-equivariant estimator
Not very easy to evaluate. Vol.6, No 9/1999, 57 - 79.
Much easier to carry out.
non-increasing, absolutely continuous
If interested in, ask me for sending by e-mail.
Víšek, J.Á. (2000): Regression with high breakdown point. ROBUST 2000, 324 – 356, ISBN 80-7015-792-5.
The Least Weighted Squares
Let us agree, for a while, that the majority of data determines the “true” model.
Then a small change even of one observation can cause a large change of estimate.
High breakdown point (assuming deletion of some observations) may be sometimes self-destructive !!
The method too much relies on selected “true” points ! What is the problem ?
Hence, it may be preferable to reject observations “smoothly”. Moreover, ...
Why the Least Weighted Squares?
Hampel, F. R., E. M. Ronchetti, P. J. Rousseeuw, W. A. Stahel (1986):
New York: J.Wiley & Sons.Robust Statistics - The Approach Based on Influence Functions.
Consistency Asymptotic normality
Controllable level of efficiency Scale- and regression-equivariance
Controllable gross-error sensitivity Controllable local shift sensitivity
Possibly finite rejection point Controllable breakdown point
Hampel’s paradigm of robust estimation
Requirements on the estimator of regression coefficients naturally inherited from the classical statistics
General discussion
The Least Weighted Squares ....
Víšek, J.Á. (2000): A new paradigm of point estimation. Proc. of Data Analysis 2000/II, Modern Statistical Methods - Modeling, Regression, Classification and Data Mining, ISBN 80-238-6590-0, 195 - 230.
An efficient and acceptable heuristics
Requirements inevitable for meaningful, competent and liable application
Available diagnostics, sensitivity studies and accompanying procedures
Extremely important, hence discussed in details below
Existence of an implementation of the algorithm with acceptable complexity
and tested reliability of evaluation
Evidently geometric, similar to the Least Squares
Under progress,something already available
continued General discussion
The Least Weighted Squares ....
Mašíček, L. (2003): Consistency of the least weighted squares estimator. To appear in Kybernetika.
Plát, P. (2003): Nejmenší vážené čtverce. (The Least Weighted Squares, in Czech.) Diploma thesis on the Faculty of Nuclear and Physical Engineering , he Czech Technical University, Prague
Mašíček,, L. (2003): Diagnostika a sensitivita robustního odhadu. (Diagnostics and sensitivity of robust estimators, in Czech) Dissertation on the Faculty of Mathematics, Charles University.
Both, in the framework of random carriers
as well as for deterministic ones
we have consistency, asymptotic normality and Bahadur representation of the Least weighted Squares.
There are also some optimality results
Mašíček,, L. (2003): Optimality of the least weighted squares estimator. To appear in the Proceedings of ICORS'2003.
Already available
The Least Weighted Squares ...
The instrumental version of the Least Weighted Squares
Recalling
,
let’s put ranks of the squared residuals
.
Hence define
.
The instrumental version ...
Notice that
can be written as
It is nearly equivalent to
which can be interpreted as empirical counterpart of
.
.
Conclussion: The instrumental version of the Least Weighted Squares
can be interpreted as a Weighted GMM estimation – see Víšek, J.Á. (2004):. Weighted GMM estimation.
Submitted to ROBUST 2004.
Assumptions
The instrumental version of the Least Weighted Squares ...
compact support
bounded
i.i.d. r.v.’s with absolutely continuous d.f.
bounded from below
absolutely continuous, non-increasing
positive definite
independent
continued
Assumptions
The instrumental version of the Least Weighted Squares ...
for all
only for
Existence of an implementation of the algorithm with acceptable complexity
and tested reliability of evaluation
Hettmansperger, T.P., S. J. Sheather (1992): A Cautionary Note on the Method of Least Median Squares. The American Statistician 46, 79-83.
- the timing of sparks - air / fuel ratio - intake temperature - exhaust temperature
Explanatory variables:
Response variable: Number of knocks of an engine
Number of observations: 16
Engine knock data - treated by the Least Median of Squares
The results were due to bad algorithm, they used. They are on the next page.
A small change (7.2%) of one value in data caused a large change of the estimates.
Requirements inevitable for the meaningful and competent application
An example
Existence of an implementation of the algorithm with ....
Requirements inevitable for meaningful and competent application
Data Intrc. spark air intake exhaust 11th res.
Correct 30.08 0.21 2.90 0.56 0.93 0.570
Wrong -86.5 4.59 1.21 1.47 .069 0.328
Engine knock data - results by Hettmansperger and Sheather
Data Intrc. spark air intake exhaust 11th res.
Correct 30.04 0.14 3.08 0.46 -.007 0.450
Wrong 48.38 -.73 3.39 0.19 -.011 0.203
Boček, P., P. Lachout (1995): Linear programming approach to LMS-estimation. Mem. vol. Comput. Statist. & Data Analysis 19 (1995), 129 - 134..
A new algorithm, based on simplex method, was nearly immediately available, although published a bit later.
It indicates that the reliability of algorithm and its implementation is crucial.
Minimized squared residual
continued
An efficient and acceptable heuristics (?)
hints that, in the case of sufficient “demand for data-processing”, we may “cope” without any heuristics.
- it seems quite acceptable heuristics, unfortunately it does not work,
- for the example of data for which the min-max-estimator failed see
- maximum was taken over some set of underlying d.f.’s and minimum over possible estimators,
Víšek, J.Á. (2000): On the diversity of estimates. CSDA 34, (2000) 67 - 89.
But papers like
-the problem is that the method implicitly takes maximum over “unexpected” set of d.f.’s.
Hansen, L. P. (1982): Large sample properties of generalized method of moments estimators. Econometrica, 50, no 4, 1029 - 1054.
In 1989 Martin et al. studied estimators minimizing maximal bias of them
Martin, R.. D., V. J. Yohai, R. H. Zamar (1989): Min-max bias robust regression. Ann Statist. 17, 1608 - 1630.
Requirements inevitable for meaningful and competent application
Another example
The Least Weighted Squares ...
It is a modification of the algorithm for the LEAST TRIMMED SQUARES which was described and tested in:
Víšek, J.Á. (1996): On high breakdown point estimation. Computational Statistics (1996) 11:137-146.
Víšek, J.Á. (2000): On the diversity of estimates. CSDA 34, (2000) 67 - 89.
Čížek, P., J. Á. Víšek (2000): The least trimmed squares. User Guide of Explore, Humboldt University.
(Of course, the algorithm for LTS is available in the package EXPLORE.)
There is also algorithm for evaluating the LEAST WEIGHTED SQUARES.
The algorithm for the instrumental version of the Least Weighted Squares is a straightforward slight generalization of the algorithm for the Least Weighted Squares.
The Least Weighted Squares - algorithm
Select randomly p + 1 observations and find regression plane through them.A
Put
Evaluate squared residuals for all observations, order these squared residuals from the smallest one to the largest, multiply them by the weights
and evaluate the sum of these products.
Is this sum of weighted squared residuals smaller than the sum from the previous step?
BNo
Order observations in the same order as the squared residuals and apply the classical weighted least squares on them with weights
Yes
, i.e.and so find new regression plane.
This step will be modified
for ILWS
Return to
Have we found already 20 identical models or have we exhausted a priori given number of repetitions ?
End of evaluation
Yes No
A
B
The algorithm is available in MATLAB.
In the case when we were able to pass all n! orders of observations ( less than 18 observations), i.e. when we were able to find the LEAST WEIGHTED SQUARES estimator precisely, the algorithm returned the same value.
An arbitrary reasonable number
continued
The Least Weighted Squares - algorithm
A stopping
r
ule
The instrumental version of the Least Weighted Squares - algorithm
The only modification of the previous algorithm:
Instead of employing
,
we utilize
.
A heretic question ...
as “close” as possible to
BUT
THANKS for A
TTENTION