Springer Series in Statistics

Advisors: P. Bickel, P. Diggle, s. Fienberg, K. Krickeberg, 1. Olkin, N. Wermuth, s. Zeger

Springer Science+Business Media, LLC

Springer Series in Statistics

Andersen/Borgan/Gill/Keiding: Statistical Models Based on Counting Processes. Berger: Statistical Decision Theory and Bayesian Analysis, 2nd edition. Bolfarine/Zacks: Prediction Theory for Finite Populations. Borg/Groenen: Modern Multidimensional Scaling: Theory and Applications Brockwell/Davis: Time Series: Theory and Methods, 2nd edition. Chen/Shao/Ibrahim: Monte Carlo Methods in Bayesian Computation. Efromovich: Nonparametric Curve Estimation: Methods, Theory, and Applications. FahrmeirlTutz: Multivariate Statistical Modelling Based on Generalized Linear

Models. Farebrother: Fitting Linear Relationships: A History of the Calculus of Observations

1750-1900. Federer: Statistical Design and Analysis for Intercropping Experiments, Volume I:

Two Crops. Federer: Statistical Design and Analysis for Intercropping Experiments, Volume II:

Three or More Crops. Fienberg/Hoaglin/KruskallTanur (Eds.): A Statistical Model: Frederick Mosteller's

Contributions to Statistics, Science and Public Policy. Fisher/Sen: The Collected Works ofWassily Hoeffding. Good: Permutation Tests: A Practical Guide to Resampling Methods for Testing

Hypotheses, 2nd edition. Gourieroux: ARCH Models and Financial Applications. Grandell: Aspects of Risk Theory. Haberman: Advanced Statistics, Volume I: Description of Populations. Hall: The Bootstrap and Edgeworth Expansion. Hardie: Smoothing Techniques: With Implementation in S. Hart: Nonparametric Smoothing and Lack-of-Fit Tests. Hartigan: Bayes Theory. Hedayat/Sloane/Stufken: Orthogonal Arrays: Theory and Applications. Heyde: Quasi-Likelihood and its Application: A General Approach to Optimal

Parameter Estimation. Huet/Bouvier/Gruet/Jolivet: Statistical Tools for Nonlinear Regression: A Practical

Guide with S-PLUS Examples. Kolen/Brennan: Test Equating: Methods and Practices. Kotz/Johnson (Eds.): Breakthroughs in Statistics Volume 1. Kotz/Johnson (Eds.): Breakthroughs in Statistics Volume II. Kotz/Johnson (Eds.): Breakthroughs in Statistics Volume III. Kiichler/Serensen: Exponential Families of Stochastic Processes. Le Cam: Asymptotic Methods in Statistical Decision Theory. Le CamlYang: Asymptotics in Statistics: Some Basic Concepts, 2nd edition. Longford: Models for Uncertainty in Educational Testing. Miller, Jr.: Simultaneous Statistical Inference, 2nd edition. Mosteller/Wallace: Applied Bayesian and Classical Inference: The Case of the

Federalist Papers. ParzenITanabe/Kitagawa: Selected Papers of Hirotugu Akaike. Politis/RomanolWolf: Subsampling.

(continued after index)

Masanobu Taniguchi Yoshihide Kakizawa

Asymptotic Theory of Statistica! Inference for Time Series

With 10 Illustrations

, Springer

Masanobu Taniguchi Department of Mathematical Science Faculty of Engineering Science Osaka University Toyonaka 560-8531 Japan taniguti@sigmath.es.osaka-u.ac.jp

Library of Congress Cataloging-in-Publication Data Taniguchi, Masanobu.

Yoshihide Kakizawa Faculty of Economics Hokkaido University Sapporo 060-0809 Japan kakizawa@econ.hokudai.ac.jp

Asymptotic theory of statistical inference for time series/Masanobu Taniguchi, Yoshihide Kakizawa.

p. cm. - (Springer series in statistics) Includes bibliographical references and index. ISBN 978-1-4612-7028-7 ISBN 978-1-4612-1162-4 (eBook) DOI 10.1007/978-1-4612-1162-4 1. Time-series analysis. 2. Stochastic processes. 1. Kakizawa,

Yoshihide. II. Title. III. Series. QA280.T355 2000 5 19.5'5-dc2 l 00-040043

Printed on acid-free paper.

© 2000 Springer Science+BusÎness Media New York Originally published by Springer-Verlag New York, Inc. in 2000 Softcover reprint ofthe hardcover Ist edition 2000 AII rights reserved. This work may not be translated or copied in whole or in part without written permis sion of the publisher Springer Science+Business Media, LLC, except for brief excerpts in connection with reviews Of scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former are not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.

Production managed by Frank MCGuckin; manufacturing supervised by Jerome Basma. Photocomposed copy prepared from the authors' LaTeX files using Springer's svsing.sty macro.

987654321

ISBN 978-1-4612-7028-7

To our families

Preface

There has been much demand for the statistical analysis of dependent ob­servations in many fields, for example, economics, engineering and the nat­ural sciences. A model that describes the probability structure of a se­ries of dependent observations is called a stochastic process. The primary aim of this book is to provide modern statistical techniques and theory for stochastic processes. The stochastic processes mentioned here are not restricted to the usual autoregressive (AR), moving average (MA), and autoregressive moving average (ARMA) processes. We deal with a wide variety of stochastic processes, for example, non-Gaussian linear processes, long-memory processes, nonlinear processes, orthogonal increment process­es, and continuous time processes. For them we develop not only the usual estimation and testing theory but also many other statistical methods and techniques, such as discriminant analysis, cluster analysis, nonparametric methods, higher order asymptotic theory in view of differential geometry, large deviation principle, and saddlepoint approximation. Because it is d­ifficult to use the exact distribution theory, the discussion is based on the asymptotic theory. Optimality of various procedures is often shown by use of local asymptotic normality (LAN), which is due to LeCam.

This book is suitable as a professional reference book on statistical anal­ysis of stochastic processes or as a textbook for students who specialize in statistics. It will also be useful to researchers, including those in econo­metrics, mathematics, and seismology, who utilize statistical methods for stochastic processes.

Chapter 1 reviews the elements of stochastic processes because we do not assume that the reader knows stochastic processes well. We also give

viii Preface

some limit theorems that are useful in the asymptotic theory for stochastic processes. Chapter 2 describes a general LAN theorem and a few of its versions, which are applied to a class of non-Gaussian linear processes. The LAN property is very important in the asymptotic theory of estimation, test and discriminant analysis, and so on. Chapter 3 discusses the asymptotic efficiency of estimators and tests for various stochastic processes, including that for nonlinear models, non-ergodic models, and diffusion processes. In Chapter 4, higher order asymptotics of estimators and tests for stochastic processes are illuminated in terms of differential geometry. This will devel­op a new direction in the usual asymptotic theory. We also discuss higher order asymptotics for transformations of statistics and iterative methods, for example, Newton-Raphson method. Recently much attention has been paid to long-memory processes, which appear in many fields (e.g., hydrol­ogy and economics). Hence, in Chapter 5 we study a modern asymptotic estimation and testing theory for them. This chapter includes the asymp­totic theory for regression models with long-memory disturbances, which shows different features from that for short-memory processes.

For a stationary process many important quantities are often expressed as integral functionals of spectral density. We consider statistical analyses based on functionals in Chapter 6. Some numerical examples to real data are given as applications of several asymptotic results in this book. We also study robust estimation and testing in the sense that the statistical proce­dures are not influenced by the presence of the decaying trend or periodic components. Discriminant analysis for stationary time series is treated in Chapter 7. Starting with the standard methods for Gaussian processes, we introduce frequency domain approach for the discriminant analysis. We mainly discuss the problem for discriminating spectrum of linear process­es, not only by parametric approach but also by non parametric approach, including the discussion related to LAN. Chapter 8 deals with a classical large deviation theory and a modern saddlepoint approximation method. These techniques are applied to some problems for stochastic processes. We put some fundamental results of mathematics, probability, and statistics in the Appendix.

We have many people to thank. First, we thank the late Professors E.J. Hannan and P.R. Krishnaiah for enlightening us to multivariate time series analysis. We have collaborated with Professors Bob Shumway, Madan L. Puri, and Marc Hallin in the discriminant analysis of time series, nonpara­metrics for time series, and the LAN approach for long-memory processes, respectively. Special thanks to Bob, who kindly offered us some seismic da­ta which were used in Chapter 6. Also Professors Kei Takeuchi and Minoru Siotani gave us much impact in higher order asymptotic theory and mul­tivariate analysis, respectively. Doctors An Hongzhi and N akahiro Yoshida provided us with high-level information and instruction in Sections 3.2 and 4.4, respectively. Takayuki Shiohama helped us with the art of figures in Chapters 3-5. Executive editor John Kimmel patiently encouraged us to

Preface ix

complete this book. We deeply thank all of these people.

Toyonaka, Japan Sapporo, Japan February 2000

Masanobu Taniguchi Yoshihide Kakizawa

Contents

Preface

Notation

1 Elements of Stochastic Processes 1.1 Introduction..... 1.2 Stochastic Processes 1.3 Limit Theorems.

Problems ..... .

vii

xv

1 1 1

17 28

2 Local Asymptotic Normality for Stochastic Processes 31 2.1 General Results for Local Asymptotic Normality . 31 2.2 Local Asymptotic Normality for Linear Processes. 37

Problems ....................... 48

3 Asymptotic Theory of Estimation and Testing for Stochastic Processes 51 3.1 Asymptotic Theory of Estimation and Testing for Linear

Processes . . . . . . . . . . . . . . . . . . . . . . . . . . .. 52 3.1.1 Asymptotic Theory Based on a Gaussian Likelihood 52 3.1.2 Asymptotic Theory of Estimation and Testing Based

on LAN Results ................ 63 3.2 Asymptotic Theory for Nonlinear Stochastic Models 79

3.2.1 Nonlinear Models. . . . . . . . . . . . . . . . 79

xii Contents

3.2.2 3.2.3

3.2.4 3.2.5

Probability Structure of Nonlinear Models . . Statistical Testing and Estimation Theory for Nonlinear Models ............... . Asymptotic Theory Based on the LAN Property Model Selection Problems ......... .

85

92 103 109

3.2.6 Nonergodic Models . . . . . . . . . . . . . . 115 3.3 Asymptotic Theory for Continuous Time Processes 121

3.3.1 Stochastic Integrals and Diffusion Processes 122 3.3.2 Asymptotic Theory for Diffusion Processes 129 3.3.3 Diffusion Processes and Autoregressions with Roots

Near Unity . . . . . . . . . . . . . . . . 140 3.3.4 Continuous Time ARMA Processes. . . 149 3.3.5 Asymptotic Theory for Point Processes 152 Problems ..................... 163

4 Higher Order Asymptotic Theory for Stochastic Processes 167 4.1 Introduction to Higher Order Asymptotic Theory. . . . .. 168 4.2 Valid Asymptotic Expansions . . . . . . . . . . . . . . . . . 187 4.3 Higher Order Asymptotic Estimation Theory for Discrete

Time Processes in View of Statistical Differential Geometry 212 4.4 Higher Order Asymptotic Theory for Continuous Time

Processes .... . . . . . . . . . . . . . . . . . . . . . . 234 4.5 Higher Order Asymptotic Theory for Testing Problems. 249 4.6 Higher Order Asymptotic Theory for Normalizing

Transformations ...................... 266 4.7 Generalization of LeCam's Third Lemma and Higher Order

Asymptotics of Iterative Methods . 277 Problems ....................... 303

5 Asymptotic Theory for Long-Memory Processes 307 5.1 Some Elements of Long-Memory Processes. . . . . . . . . . 309 5.2 Limit Theorems for Fundamental Statistics . . . . . . . . . 317 5.3 Estimation and Testing Theory for Long-Memory Processes 333 5.4 Regression Models with Long-Memory Disturbances 345 5.5 Semiparametric Analysis and the LAN Approach 371

Problems ........................ 381

6 Statistical Analysis Based on Functionals of Spectra 385 6.1 Estimation of Nonlinear Functionals of Spectra . . . 386 6.2 Application to Parameter Estimation for Stationary

Processes ......... . . . . . . . . . . . . . . . 399 6.3 Asymptotically Efficient Nonparametric Estimation of

Functionals of Spectra in Gaussian Stationary Processes 415 6.4 Robustness in the Frequency Domain Approach. . . . . 424

Contents xiii

6.4.1 Robustness to Small Trends of Linear Functionals of a Periodogram . . . . . . . . . . . . .

6.4.2 Peak-Insensitive Spectrum Estimation 6.5 Numerical Examples

Problems ................... .

424 430 455 472

7 Discriminant Analysis for Stationary Time Series 477 7.1 Basic Formulation ................... 478 7.2 Standard Methods for Gaussian Stationary Processes. 480

7.2.1 Time Domain Methods ............. 480 7.2.2 Frequency Domain Methods. . . . . . . . . . . 490 7.2.3 Admissible Linear Procedure: Case of Unequal Mean

Vectors and Covariance Matrices . . . . . . . . .. 494 7.3 Discriminant Analysis for Non-Gaussian Linear Processes 500 7.4 Nonparametric Approach for Discriminant Analysis. . .. 505 7.5 Parametric Approach for Discriminant Analysis. . . . .. 514 7.6 Derivation of Spectral Expressions to Divergence Measures

Between Gaussian Stationary Processes 526 7.7 Miscellany. 531

Problems ................. 534

8 Large Deviation Theory and Saddlepoint Approximation for Stochastic Processes 537 8.1 Large Deviation Theorem . . . . . . . . . . . . . . . . . . . 538 8.2 Asymptotic Efficiency for Gaussian Stationary Processes:

Large Deviation Approach . . . . . . . . . . . . . . . . 546 8.2.1 Asymptotic Theory of Neyman-Pearson Tests. 546 8.2.2 Bahadur Efficiency of Estimator .. . . . . . . 551 8.2.3 Stochastic Comparison of Tests . . . . . . . . . 561

8.3 Large Deviation Results for an Ornstein-Uhlenbeck Process 570 8.4 Saddlepoint Approximations for Stochastic Processes. 578

Problems ......................... 601

Appendix 605 A.l Mathematics 605 A.2 Probability 609 A.3 Statistics 617

Bibliography 619

Author Index 649

Subject Index 657

Notation

N : the set of all positive integers

Z : the set of all integers

Rm : the m-dimensional Euclidean space

Em : the Borel a-algebra of Rm

C : the set of all complex numbers

a.e. : almost everywhere

a.s. : almost surely

CLT : central limit theorem

i.i.d. : independent and identically distributed

LAN : local asymptotic normality or locally asymptotically normal

ML : maximum likelihood estimator

LS or OLS : (ordinary) least squares estimator

o : end of a proof

[xl: the integer part of a real number x Re(z) : the real part of a complex number z Im(z) : the imaginary part of a complex number z #(5) : the cardinality of a set 5

Be : the complement of a set B

XB : the indicator function of a set B Ip : the p x p identity matrix

XVI Notation

det(A) or IAI : the determinant of a matrix A

A-I : the inverse of a matrix A

tr(A) : the trace of a matrix A

A' : the transpose of a matrix A A * : the conjugate transpose of a matrix A A ~ B : the matrix A - B is positive semidefinite

Iiall or lIaliE : the Euclidean norm of a vector a IIAIIE : the Euclidean norm of a matrix A defined by Jtr(A* A)

IIAII : the square root of the greatest eigenvalue of A* A

I A I : the sum of the absolute values of the entries of A 6(·,·) : Kronecker's delta

(n, F, P) : probability space

E(X) : the expectation of a random variable X

Var(X) : the variance of a random variable X Cov(X, Y) : the covariance of X and Y

£P(n) : the family of all random variables Y satisfying EIYIP < 00

l.i.m. n --+ oo : limit in the mean

Xn ~ X : Xn converges almost surely to X

Xn ~ X : Xn converges in probability to X

Xn ~ X or Xn ==> X : Xn converges in distribution to X LP

Xn ~ X: Xn converges in pth mean to X e[O,l] : the space of continuous, real-valued functions on [0,1] D[O, 1] : the space of real-valued functions on [0,1] that are right continuous

and have finite left limits

E(·I F) : the conditional expectation of . given a a-algebra F

Var(·1 F) : the conditional variance of . given a a-algebra F

£(·1 P) : the probability distribution of . under P

F(Xl, ... ,Xn) : the a-algebra generated by Xl, ... ,Xn (X, Y) : the inner product defined by E(XY)

Np(/-L,~) or N(/-L,~) : p-variate normal with mean vector /-L and covarinace matrix ~

</>(x) : standard normal probability density function on R

<p(x) : standard normal probability distribution function on R

X} : central chi-squared distribution with f degrees of freedom

X}(82 ) or x2 (f,62 ) : non-central chi-squared distribution with f degrees of freedom and non centrality parameter 82

AR(p) : autoregressive process of order p

MA(q) : moving average process of order q

Notation xvii

ARMA(p, q) : autoregressive moving average process of order (p, q)

FARIMA(p, d, q) : fractional autoregressive integrated moving average pro-cess of order (p, d, q)