Impact Bootsrap Time Series

8/7/2019 Impact Bootsrap Time Series

1/13


2/13

StatisticalScience2003, Vol. 18, No. 2, 219-2300 Instituteof MathematicalStatistics,2003

T h e I m p a c t o f Bootstrap Methods o nT i m e S e r i e s AnalysisDimitris N. Politis

Abstract. Sparkedby Efron's seminalpaper,the decadeof the 1980s wasa period of active researchon bootstrapmethods for independentdata-mainly i.i.d. or regressionset-ups. By contrast,in the 1990s muchresearchwas directedtowardsresamplingdependentdata, for example, time seriesand random fields. Consequently,the availabilityof valid nonparametricinference proceduresbased on resamplingand/or subsamplinghas freedpractitionersfrom the necessityof resortingto simplifying assumptionssuchas normalityor linearitythatmaybe misleading.Key words and phrases: Block bootstrap, confidence intervals, linearmodels, resampling, large sample inference, nonparametricestimation,subsampling.

1. INTRODUCTION:THESAMPLEMEANOFA TIMESERIESLet X, ..., Xn be an observed stretch from a strictlystationarytime series {Xt, t E Z}; the assumptionofstationarityimplies that the joint probabilitylaw of

(Xt, Xt+l, ..., Xt+k) does not depend on t for anyk > 0. Assume also that the time series is weaklydependent;that is, the collection of random variables{Xt,t < 0} is approximately independent of {Xt,t > k) when k is large enough.An exampleof a weakdependencestructureis givenby m-dependenceunderwhich {Xt,t < 0} is (exactly) independentof {Xt,t > k} whenever k > m; independence is just thespecialcase of 0-dependence.Due to the dependence between the observations,even the most basic methods involved in appliedsta-tistical work suddenly become challenging; an ele-mentarysuch example has to do with estimatingtheunknownmean ,u= EXt of the time series. The sam-ple mean X, = n-~1E1 Xt is the obvious estima-tor; however-and here immediately the difficultiescrop up-it is not the most efficient. To see why, con-sider the regression Xt = i + Et, which also servesas a definition for the ?t process; it is apparentthatDimitrisN. Politis is ProfessorofMathematicsand Ad-junct Professor ofEconomics,Universityof California,San Diego, La Jolla, California92093-0112 (e-mail:dpolitis @ucsd.edu).

Xn is the ordinaryleast squaresestimatorof Ltin thismodel. However, because of the dependence in theerrors Et, the best linear unbiased estimator (BLUE)of ,t is instead obtainedby a generalizedleast squaresargument;consequently, BLUE = (l'rn 1 n)-1 'rnX,whereX = (X1,... , Xn), 1 = (1,..., 1)'andrn is the(unknown)covariancematrixof the vectorX with i, jelement given by y(i - j) = Cov(Xi, Xj).It is immediate that ,BLUE is a weighted average ofthe Xt data, (i.e., ^BLUE= Ein= wiXi); the weightswi are a function of the (unknown)covariancesy(s),s = 0, 1,..., n- 1. For example, under the simpleAR(p), thatis, autoregressiveof orderp, model,

(1) (X, - it)= i(Xt-_ - iL)+ - - -+ qp(Xt-p - .It)+ Zt,where Zt ' i.i.d. (0, T2), it is easy to see that wj =wn-j+l for j = 1, .... p, Wp+l = Wp+2 = ..= Wn-pand 1=l wi = 1; calculating wi,..., p in termsof y (.) is feasiblebutcumbersome-see Fuller(1996).For instance, in the special case of p = 1, we cancalculate that wl = w2/(l - p(l)), where p(s)=y (s)/y(0) is the autocorrelationsequence.Nevertheless, it is apparentthat, at least in theAR(p) model above, the difference of the wi weightsfrom the constant weights in Xn is due mostly tothe end effects, that is, the first and last p weights.

219


3/13

D. N. POLITISThus, it may be reasonable to conjecture that, fora large sample size n, those end effects will have anegligiblecontribution.This is, in fact, true;as shownin Grenanderand Rosenblatt(1957) for a large classof weakly dependent processes [not limited to theAR(p) models],the samplemeanXn is asymptoticallyefficient.Therefore,estimatingu by ourfamiliar,easy-to-use, samplemeanXn is fortuitouslyjustified.The problem,however,is furthercomplicatedif onewants to constructan estimate of the standarderrorof Xn. To see this, let an2= Var(x/Xn) and note thatnan isjust thesum of all the elements of the matrixFn,thatis, a2 = En=_n(1 -Isl/n)y(s). Under the usualregularityconditions,

o00a02: lim 2 y(s)= 2rf(0),n-->oo

where f (w) = (27r)-1 E eiwsy(s), for w E[-r, 7r], is the spectral density function of the {Xt}process.It is apparentnow thatthe simpletaskof stan-darderrorestimationis highlynontrivialunderdepen-dence,as it amountsto nonparametricestimationof thespectraldensityatthe origin.One may think that estimating y and plugginginto the above formula would give a consistent es-timator of the infinite sum y=- (s). Estimatorsof y (s) for IsI< n cancertainlybe constructed;y (s) =(n - s)-1 ES(Xt - Xn)(Xt+s - Xn) and y(s) =n-1 Et=n (Xt - Xn)(Xt+s - Xn) are the usual candi-dates. Note that P(s) is unbiasedbut becomes quiteunreliablefor large s because of increasingvariabil-ity; at the extreme case of s = n - 1, there is no aver-aging at all in y(s). Thus, y(s) is preferablefor tworeasons:(a) it tradesin some bias for a huge variancereductionby usingtheassumptionof weakdependence[which implies that y(s) -- 0 as s --oo] to shrinkthe y(s) toward0 for large s; and (b) y(s) is a non-negativedefinite sequence, that is, it guaranteesthatLsl 0. To understand the last claim, notethat

Elsl


4/13


5/13

D. N.POLITISThe possibility of drawing an arbitraryresamplesize was noted as early on as in Bickel and Freed-man(1981); nevertheless,the researchcommunityfo-cused for some time on the case of resamplesize nsince that choice leads to improvedperformance,that

is, higher-orderaccuracy,in the sample mean case[cf. Singh (1981)]. Nevertheless, in instances wherethe i.i.d. bootstrapfails, it was observedby many re-searchers,most notably K. Athreya, J. Bretagnolle,M.Arcones andE. Gine,thatbootstrapconsistencycanbe restoredby taking a smaller resamplesize b sat-isfying (2); see Politis, Romano and Wolf (1999) forreferencesbut also note the earlypaperby Swanepoel(1986).As a matterof fact, the i.i.d. bootstrapwith resam-ple size b immediatelyinherits the general validityofsubsamplingin the case where b = o(fnr) in whichcase the questionwith/withoutreplacementis imma-terial;see Politis andRomano(1992c, 1993)orPolitis,RomanoandWolf(1999, Corollary2.3.1). However,torelaxthe conditionb = o(/ni) to b = o(n), some extrastructureis requiredthathas to be checkedon a case-by-casebasis;see Bickel, Gotze and vanZwet (1997).Finally, note that in the case of the sample meanof i.i.d. data (with finite variance),the bootstrapwithresample size n outperformsboth subsamplingandthe bootstrapwith smaller resample size, as well asthe asymptoticnormal approximation.Although theperformanceof subsamplingcan be boosted by theuse of extrapolationand interpolationtechniques-see, for example, Booth and Hall (1993), Politis andRomano(1995), Bickel, Gbtze and van Zwet (1997)and BertailandPolitis (2001)-the understandinghasbeen that subsamplingsacrifices some accuracy forits extremelygeneral applicability;this was also theviewpoint adoptedin Politis,RomanoandWolf (1999,Chapter10). Nevertheless,some veryrecentresultsofSakovand Bickel (2000) and Arcones(2001) indicatethatthis is not alwaysthe case: for the samplemedian,the i.i.d. bootstrapwith resample size b = o(n) hasimprovedaccuracyas comparedto the bootstrapwithresamplesize n; ananalogousresultis expectedto holdfor subsampling.2.2 Block Resampling

In the previous section, we saw the connection ofsubsamplingin the i.i.d. case to the delete-d jack-knife. Recall that the standarddelete-i jackknife iscommonlyattributedto Tukey(1958) and Quenouille(1949, 1956). Interestingly,Quenouille'sworkwas fo-cusedon blockingmethodsin atimeseriescontext,and

thus it was the precursorof (block) subsampling;ofcourse,blockingmethods for estimationin time seriesgo backto Bartlett(1946) as previouslymentioned.After Carlstein's(1986) subseriesvarianceestimator[as well as Hall's (1985) retilingideasfor spatialdata]the time was ripe for Ktinsch's (1989) introductionof the block bootstrapwhere, insteadof recomputingthe statistic 0b on the smaller blocks of the typeBi = (Xi, ..., Xi+b-1), a new bootstrap pseudo-seriesX, ...., Xl is created (with I = kb - n) by joiningtogether k (= [n/b]) blocks chosen randomly (andwith replacement) from the set B1, ..., Bn-b+ }; thenthe statistic can be recomputedfromthe new, full-sizepseudo-series X, ..., X.Thus,10yearsafterEfron's(1979) pioneeringpaper,Kiinsch's (1989) breakthroughindicated the startingpointforthe focus of researchactivityon thisnew areaof block resampling.The closely relatedworkby LiuandSingh(1992), which was availableas a preprintinthe late 1980's, was also quite influentialat thatpointbecause of its transparentand easily generalizableapproach.Examplesof some early work on block resamplingfor stationarytime series includethe blocks-of-blocksbootstrapof Politis and Romano (1992a), which ex-tendedthe applicabilityof the block bootstrapto para-meters associatedwith the whole infinite-dimensionalprobabilitylaw of the series;the circularbootstrapofPolitis and Romano (1992b), in which the bootstrapdistributionis automaticallycenteredcorrectly;andthestationarybootstrapof Politis and Romano (1994a),whichjoins togetherblocksof randomlength-havinga geometricdistributionwith mean b-and thus gen-eratesbootstrapsample pathsthatarestationaryseriesthemselves.

As in the i.i.d. case, the validity of the block-bootstrapmethodsmust be checkedon a case-by-casebasis, and this is usually facilitatedby an underlyingasymptoticnormality;typically,a conditionsuch as (2)is required.For instance, the block-bootstrapestima-tor of the distributionof /'n(Xn - ,L) is essentiallyconsistentonly if the latter tends to a Gaussianlimit,a result thatparallelsthe i.i.d. situation;see Gine andZinn (1990) and Radulovic(1996). To see why, notethatin the samplemean case the block-bootstrapdis-tributionis really a (rescaled) k-fold convolutionofthe subsamplingdistributionJb,SUBwith itself; sincek = [n/b] - oo under (2), it is intuitive that the block-bootstrap distribution has a strong tendency towardGaussianity.

222


6/13


7/13

D. N. POLITISblock resamplingand subsamplingwill generallyre-main valid if b is chosen to be an integer multipleof d. In addition,resampling/subsamplingshould notbe done from the set {B1, B2, ..., Bn-b+l} but ratherfrom a set of the type {B1, Bd+l, B2d+l, .. .}; that is,full overlapof the blocks is no longerrecommended-see Politis (2001a).(ii) Local stationarity. Suppose that the time se-ries has a slowly changingstochastic structurein thesense that the joint probabilitylaw of (Xt, Xt+, ... ,Xt+m) changes smoothly (and slowly) with t foranym; see, for example,Dahlhaus(1997). Thena "lo-cal" blockbootstrapmaybe advisable;hereabootstrappseudo-series is constructedby a concatenationofk blocks of size b (such that kb _ n), where the jthblockof theresampledseriesis chosenrandomlyfroma distribution(say, uniform)on all the size-b blocksof consecutivedata whose time indices are "close" tothose in the originalblock. A rigorous descriptionofthe local block bootstrapis given in PaparoditisandPolitis(2002c).(iii) Unit root-integrated time series. A usefulmodel in econometrics is a generalization of therandom walk notion given by the unit root modelunder which Xt is not stationarybut Dt is, whereDt = Xt - Xt_- is the series of first differences-see,for example, Hamilton(1994). In this case, a blockbootstrap of the differences Dt can be performedyielding Dt, Dy,..., and a bootstrap pseudo-seriesfor Xt can be constructedby "integrating"the Dt, thatis, by letting Xt = Ei= D*. This idea is effectivelyimposinga sample path"continuity"on the bootstraprealizationXt; see PaparoditisandPolitis (2001c). Onthe otherhand,if a unit root is just suspectedto exist,a bootstraptest of the unit root hypothesis (vs. thealternativeof stationarity)may be constructedusingsimilarideas;see PaparoditisandPolitis (2003).2.4 WhyHave Bootstrap Methods Been soPopular?

Finally, it is interesting to recall the two mainreasonsfor the immensesuccess andpopularityof thei.i.d. bootstrapof Efron(1979):(a) Thebootstrapwas shownto give valid estimatesof distributionand standarderrorin "difficult"situa-

tions;a primeexampleis the medianof i.i.d. dataforwhich the (delete-1)jackknifewas known to fail, andtheasymptoticdistributionis quitecumbersome.(b) In "easy" cases where there exist easy-to-constructalternativedistributionestimators,for exam-ple, the regular sample mean with its normal

large-sampledistribution,the (Studentized)bootstrapwas shown to outperformthose alternatives,thatis, topossess "second-orderaccuracy"-see Hall (1992) fordetails.In the case of dependent stationarydata, undersomeregularityconditions,theappropriatelystandard-ized/Studentizedblock bootstrap(in all its variations,including the stationaryand circularbootstrap)alsopossesses a "higher-orderaccuracy"propertywhenes-timatingthe distributionof thesamplemean;see Lahiri(1991, 1999)andGotze andKiinsch(1996). [Notethatit is not advisableto use a simpleblock-bootstrapesti-mate of standarderrorfor the Studentizationas it doesnotconvergesufficientlyfast; however,one mayuse anestimatorsuch as ab-see Davison andHall (1993) orBertail and Politis (2001).] Nevertheless,the gains in

accuracyare not as spectacularas in the i.i.d. case; inaddition,estimatingthe standarderroraccuratelyis thedominantfactor so thatit is somewhatof a luxurytothink of capturinghigher-ordermoments.The bottom line is that practicallyall time seriesproblemsfall under the "difficult"category,and thusthe aforementioned block resampling/subsamplingmethods become invaluable.To give a simple exam-ple, considertheproblemof estimatingthedistributionof thelag-p sampleautocorrelationp(k) = y (k)/y (0),for some fixed k, with the purposeof constructingaconfidenceintervalorhypothesistestconcerningp(k).Itis quitestraightforwardto show,underusualmomentandmixing conditions,that the large-sampledistribu-tion of ,/I(p(k)- p(k)) is N(0, Vk). Unfortunately,the varianceV2 is, in general,intractableinvolvingin-finite sums of fourth-ordercumulants and so on; see,for example,Romano and Thombs(1996). Neverthe-less, block resamplingand subsamplingeffortlesslyyield consistentestimatorsof V2 and the distributionof Vn(p3(k)- p(k)).One may ask how it had been possible in the pre-bootstrap days to practically address such a basictime series inference question regardingthe autocor-relations;the answer is that people had to resort tosome more restrictivemodel assumptions.For exam-ple, a celebratedformulafor V2 exists involving in-finite sums of the (unknown)p(-) coefficientsbut nohigher cumulants;this formulais due to Bartlettbutis validonly underthe assumptionthatthe time series{Xt} is linear;thatis, it satisfiesanequationof thetype(4)

00Xt = + -E fiZt-i,i=-00oo

224


8/13

IMPACT OF BOOTSTRAP METHODS ON TIME SERIES ANALYSISwhere Zt ~ i.i.d. (0, r2) and the /j coefficientsare absolutely summable-see Brockwell and Davis(1991). Nonetheless,the class of nonlinear time seriesis vast;for example,the simple time series definedbyYt= ZtZt-1 for all t is nonlinearand 1-dependentalthough uncorrelated.Other examples of nonlineartime series models include the ARCH/GARCHandbilinear models that are of particularprominenceineconometricanalysis; see, for example, GrangerandAndersen(1978), Subba Rao and Gabr(1984), Tong(1990), Bollerslev,Chou and Kroner(1992) or Engle(1995).To give a concrete example, the ?1.96/J/n bandsthat most statisticalprogramslike S+ automaticallyoverlay on the correlogram[the plot of p(k) vs. k]are based on Bartlett's formula. These bands areusually interpretedas giving an implied hypothesistest of the first few p(k) being 0 or not, p(l) inparticular;however, this interpretationis true onlyif one is willing to assume (4). If the time series{Xt} is not linear, then Bartlett's formula and the?1.96/1/n bands are totally uninformativeand maybe misleading;resamplingand subsamplingmay wellgive the only practicalsolution in this case-see, forexample,Romanoand Thombs(1996).As an illustration,consider a realization X1,...,X200fromthe simpleARCH(1)model:

Xt = ZtO. + 0.9X ,where Zt ~ i.i.d. N(0, 1); the samplepathis plottedinFigure l(a), while the sampleautocorrelationfunctionis given in Figure1(b).Note that the time series {Xt} is uncorrelatedal-thoughnot independent.Consequently,the +1.96/fibandsaremisleadingas they indicatethatp(1) is sig-nificantlydifferentfrom0. Because of the nonlinearitypresent,the correct95% confidence limits should beapproximately?4.75/v/-n, thatis, morethantwo timeswider than the ones given by Bartlett'sformula.Us-ing the correctconfidence limits ?0.34, the estimatep/(1)= -0.33 is seen to not be significantlydifferentfrom0 (if only barely).Figure l(c) depictsthe subsamplingestimatorof thestandarderrorof / (1) as a functionof the subsamplingblock size b. It is apparentthatthe block size choicediscussed in the previoussectionis an importantissue:forgood results,b must be largebutsmallwithrespectto n = 200. Withthis caveat,the subsamplingmethodseems to be successful in capturing/estimatingthecorrectstandarderrorinthis nonlinearsetting.Evenfora wide rangeof block size values going from b = 50

cli

x0

a,

13: T -

0~LL0

C4

9 -

!2 .

rla

EloX '

I I I I I0 50 100 150 200

t

(a)

I I5 II2

I I I I I0 5 10 15 20Lag

(b)

0 50 100 150 200block size

(c)FIG. 1. (a)Samplepath of an ARCH(1)timeseries withn = 200;(b) sample autocorrelation of the series with the customary?1.96/,/n bands;(c) subsampling-basedestimatesof thestandarderrorof p(1) as afunction of the subsamplingblocksize b.

to 100, the subsamplingestimator is 0.14 or above,which is areasonableapproximationof thetargetvalueof 0.17.

225

qo


9/13

D. N. POLITIS

3. MODEL-BASED RESAMPLINGFOR TIMESERIES

3.1 Linear AutoregressionsHistorically speaking, block resampling/sub-

samplingwas not the first approachto bootstrappingnon-i.i.d. data.Almost immediatelyfollowing Efron's(1979) paper, the residual-basedbootstrap for lin-earregressionand autoregressionwas introducedandstudied; cf. Freedman (1981, 1984) and Efron andTibshirani(1986, 1993). In the context of the semi-parametricAR(p) model of (1), the residual-basedbootstrapamountsto an i.i.d. bootstrapof (a centeredversionof) the estimatedresidualsZt definedby

Zt = (Xt - Xn)- (l(Xt-1 - Xn) -* -p(Xt-p

- Xn),where yj are some consistent estimators of jy,for example, the Yule-Walker estimators. Havingconstructed bootstrap pseudo-residuals Z*, Z *...,a bootstrap series X... X* can be generated via therecursion(1) with /j in place of Oj; finally,our sta-tistic of interestcan be recomputedfrom the bootstrapseriesX, ..., X.If the assumed AR(p) model holds true, then theabove residual-basedbootstrap works well for thesample mean and other more complicated statistics,and even enjoys a higher-orderaccuracy propertysimilar to Efron's i.i.d. bootstrap;see Bose (1988).Nevertheless, one never really knows whether theunderlying stochastic structureconforms exactly toanAR(p) model,andfurthermoreone would not knowthe value of p. A morerealisticapproachfollowed bythe vastmajorityof practitionersis to treat(1) only asanapproximationto theunderlyingstochasticstructureand to estimate the order p from the databy criteriasuch as the AIC; see, for example, Shibata (1976).Doing so, one is implicitly assuming that {Xt} is alinear time seriespossessinganAR(oo) representationof the form:

00(5) (Xt - t) = E Xi (Xt-i - L) + Zt,

i=1where Zt - i.i.d. (0, T2) and the /j coefficients areabsolutelysummable.Of course, in fitting model (5) to the data, a fi-nite order p must be used which, however, is al-lowed to increase as the sample size n increases.Thus, the residual-basedbootstrap applies verbatim

to the AR(oo) model of (5) with the understandingthat p increases as a function of n; this is the so-called AR(oo) "sieve" bootstrap employed bySwanepoeland vanWyk(1986) andrigorouslyinvesti-gated by Kreiss (1988, 1992), Paparoditis(1992) andBuhlmann(1997), among others. Conditions for thevalidityof theAR(oo) bootstraptypically requirethat(6) p - oo as n - oo butwith pk/n - 0,where k is a small integer,usually3 or 4. Consistencystill holds when the orderp is chosenvia theAIC;thatis, it is datadependent.Underassumption(5), Choi and Hall (2000) proveda higher-orderaccuracypropertyof the AR(oo) boot-strapin the samplemeancase, while Buhlmann(1997,2002) showed that the AR(oo) bootstrapestimatorR2 f 2 Var(/-Xn) hasa performancecompa-A'fiR(p) a -hsrableto the aforementionedoptimalestimator 2. BothA2R(p)andb2 exhibitan adaptivity to the strengthofdependencepresentin the series {Xt , thatis, theybe-come moreaccuratewhen thedependenceis weak,andthey both achieve a close to an rateof convergencefor series withexponentiallydecayingcovariancey (s),for example,in the case of stationaryARMA models.Not surprisingly,the best performancefor ^R() isachievedwhen {Xt satisfiesa finite-orderAR model,thatis, a linear Markovianstructure,whereasthe bestperformancefor a2 is achieved when {XIt has anm-dependentstructure,for example,a finite-orderMAmodel.Nevertheless,ab is moregenerallyvalid.Evenamong the class of linear time series (4), there aremany series that do not satisfy (5), for example, thesimple MA(1) model Xt = Zt + Zt-1, where Zt -i.i.d. (0, T2).3.2 Nonlinear Autoregressions andMarkov Processes

Although the class of linear time series is quiterich including,for example, all Gaussiantime series,as mentioned before the class of nonlineartime se-ries is vast. It is intuitive that the AR(oo) bootstrapwill generallynot give good results when appliedtodata X1, ..., Xn from a nonlinear time series. [Unless,of course, it so happensthat the statistic to be boot-strappedhas a large-sampledistributionthatis totallydeterminedby the firsttwo momentsof {Xt}, i.e., by/u and y (); this is actually what happens in the samplemean case.] See, for example,Buhlmann(2002) for adiscussion and some illustrativeexamples.The linearityof model (1) is manifestedin the factthat the conditional expectation m(Xt-1 ,..., Xt-p) '=

226


10/13

IMPACTOF BOOTSTRAPMETHODSON TIME SERIESANALYSISE(Xt Xt-, ..., Xt-p) is a linear function of the vari-ables (Xt-, ..., Xt-p) just as in the Gaussian case.If we do not know and/or assume that m(.) is lin-ear, it may be of interest to estimate its functionalform for the purposes of prediction of X,n+ giventhe recent observed past X, ...., Xn-p+l. Under somesmoothnessassumptionson m(.), this estimationcanbe carriedout by differentnonparametricmethods,forexample, kernel smoothing;see, for example, Masryand Tj0stheim (1995). The bootstrapis then calledto providea measureof accuracyof the resultinges-timatorm(.), typically by way of constructingcon-fidence bands. With this objective, many differentresamplingmethodsoriginallydesigned for nonpara-metric regression successfully carry over to this au-toregressiveframework.We mention some prominentexamples:A. Residual-based bootstrap. Assuming the struc-ture of a nonlinear (and heteroscedastic)autore-

gression:Xt = m(Xt-l,..., Xt-p)(7) + v(Xt-, .. ., Xt-p)Zt,

where Zt ~ i.i.d. (0, 1) and m(.) and v(.) aretwo (unknown) smooth functions, estimates ofthe residuals can be computed by Zt = (Xt -tm(Xt_l ..., Xt-p))/v(Xt-,..., Xt-p), wheremh(.)and v(-) are preliminary [oversmoothed as inHardle and Bowman (1988)] nonparametricesti-matorsof m(.) andv(.). Thenani.i.d. bootstrapcanbe performedon the (centered)residuals Zt, andrecursion (7)-with m(.) and v(.) in place of m(-)and v(.)-can be used to generatea bootstrapse-ries X, ..., X* from which ourtargetstatistic,forexample, m*(.), can be recomputed.The consis-tency of this procedureundersome conditions hasbeen recentlyshownin a highly technicalpaperbyFranke,Kreiss andMammen(2002).B. Wildand local bootstrap. Let Xt- = (Xt-l,.. ,Xt-p) and consider the scatterplot of Xt vs. Xt-1;smoothing this scatterplotwill give our estima-tor m(.) as well as estimators for higher-order con-ditional moments if so desired. Interestingly,thewild bootstrapof Wu (1986) and the local boot-strapof Shi (1991) both have a direct applicationin this dependentcontext; their respective imple-mentationcan be performedexactlyas in the inde-pendent(nonparametric)regressionsettingwithouteven havingto assume the specialstructureimpliedby the nonlinearautoregression(7) which is quite

remarkable.Neumannand Kreiss (1998) show thevalidityof the wildbootstrapin this time series con-text, while Paparoditisand Politis (2000) show thevalidityof thelocal bootstrapunderveryweakcon-ditions.

Comparing the above two methods, that is, theresidual-basedbootstrapvs. the wild/local bootstrap,we note the following: the wild/local bootstrapmeth-ods arevalid underweakerassumptions,as, for exam-ple, when (7) is not assumed;on the otherhand, theresidual-basedbootstrapis able to generatenew boot-strapstationarypseudo-seriesX1,..., X* fromwhicha varietyof statisticsmaybe recomputedas opposedtojust generatingnew scatterplotsto be smoothed.Inorderto haveour cake and eat it, too, theMarkov-ian local bootstrap(MLB for short)was recently in-troducedby PaparoditisandPolitis (200 b, 2002a). Todescribe it, assume for the sake of argumentthat thetime series {Xt is Markov of order p, with p = 1 forsimplicity. Given the data X1, ..., Xn, the MLB proce-dure constructs a bootstrap sample path X1, ..., Xn asfollows:

(a) Assign XT a value chosen randomlyfrom theset {Xi,..., Xn}. Suppose this chosen value is equalto Xt, (forsome tl).(b) Assign X2 a valuechosenrandomlyfrom the set{Xs+1: Xt -Xsl


11/13


12/13

IMPACTOF BOOTSTRAPMETHODSON TIME SERIESANALYSISEFRON,B. (1979). Bootstrapmethods:Another look at the jack-knife.Ann.Statist.7 1-26.EFRON,B. and TIBSHIRANI,R. J. (1986). Bootstrapmethodsforstandarderrors,confidence intervals, and other measuresofstatisticalaccuracy(withdiscussion).Statist. Sci. 1 54-77.EFRON,B. and TIBSHIRANI,R. J. (1993). An Introductionto the

Bootstrap.ChapmanandHall, New York.ENGLE,R., ed. (1995). ARCH: Selected Readings. OxfordUniv.Press.FRANKE,J., KREISS,J.-P.andMAMMEN,E. (2002). Bootstrapofkernelsmoothingin nonlineartime series. Bernoulli 8 1-37.FREEDMAN,D. A. (1981). Bootstrappingregressionmodels. Ann.Statist.9 1218-1228.FREEDMAN,D. A. (1984). On bootstrapping two-stage least-

squaresestimatesin stationarylinear models.Ann. Statist. 12827-842.FULLER,W. A. (1996). Introductionto Statistical Time Series,2nded. Wiley,New York.GINi, E. and ZINN, J. (1990). Necessary conditions for the

bootstrapof the mean.Ann.Statist.17 684-691.GOTZE,F. and KUNSCH,H. (1996). Second-order correctnessof the blockwise bootstrapfor stationaryobservations.Ann.Statist.24 1914-1933.GRANGER,C. and ANDERSEN,A. (1978). An IntroductiontoBilinear Time Series Models. Vandenhoeckund Ruprecht,

Gottingen.GRENANDER,U. and ROSENBLATT,M. (1957). Statistical Analy-sis of StationaryTimeSeries. Wiley,New York.HALL, P. (1985). Resampling a coverage pattern. StochasticProcess.Appl.20 231-246.HALL, P. (1992). The Bootstrap and Edgeworth Expansion.

Springer,New York.HALL, P., DICICCIO,T. J. and ROMANO,J. P. (1989). Onsmoothingand the bootstrap.Ann.Statist.17 692-704.HALL,P., HOROWITZ,J. L. and JING,B.-Y. (1995). On blockingrules for the bootstrapwith dependentdata. Biometrika 82561-574.

HAMILTON,J. D. (1994). Time Series Analysis. Princeton Univ.Press.HARDLE,W. and BOWMAN,A. (1988). Bootstrapping in nonpara-metric regression:Local adaptive smoothing and confidencebands. J. Amer.Statist.Assoc. 83 102-110.HOROWITZ,J. L. (2003). BootstrapmethodsforMarkovprocesses.Econometrica71 1049-1082.KREISS,J.-P. (1988). Asymptotic statistical inference for a classof stochasticprocesses. Habilitationsschrift,Facultyof Math-ematics,Univ. Hamburg,Germany.KREISS,J.-P.(1992). Bootstrapproceduresfor AR(oo) processes.In Bootstrapping and Related Techniques (K. H. Jockel,G. Rothe andW. Sendler,eds.) 107-113. Springer,Berlin.KUNSCH,H. R. (1989). Thejackknifeand thebootstrapforgeneral

stationaryobservations.Ann. Statist.17 1217-1241.LAHIRI,S. N. (1991). Second order optimality of stationarybootstrap.Statist.Probab.Lett.11 335-341.LAHIRI,S. N. (1999). Theoreticalcomparisonsof block bootstrapmethods.Ann.Statist.27 386-404.

LIu, R. Y. and SINGH,K. (1992). Moving blocks jackknife andbootstrapcaptureweak dependence.In Exploring the Limitsof Bootstrap(R. LePageand L. Billard,eds.) 225-248. Wiley,New York.

MASRY,E. andTJOSTHEIM,D. (1995). Nonparametricestimationand identificationof nonlinearARCH time series. EconometricTheory11 258-289.

NEUMANN, M. and KREISS, J.-P. (1998). Regression-type in-ference in nonparametricautoregression.Ann. Statist. 261570-1613.PAPARODITIS,E. (1992). Bootstrapping some statistics useful inidentifying ARMA models. In Bootstrapping and Related

Techniques(K. H. Jockel, G. Rothe and W. Sendler, eds.)115-119. Springer,Berlin.PAPARODITIS,E. and POLITIS,D. N. (2000). The local bootstrapfor kernel estimators undergeneral dependence conditions.Ann.Inst.Statist.Math.52 139-159.PAPARODITIS,E. and POLITIS, D. N. (2001a). Tapered block

bootstrap.Biometrika88 1105-1119.PAPARODITIS,E. and POLITIS,D. N. (2001b). A Markovian local

resamplingscheme fornonparametricestimatorsin time seriesanalysis.EconometricTheory17 540-566.

PAPARODITIS,E. and POLITIS, D. N. (2001c). The continuous-path block-bootstrap.In Asymptoticsin Statistics and Prob-ability (M. Puri, ed.) 305-320. VSP Publications,Zeist, TheNetherlands.

PAPARODITIS,E. and POLITIS, D. N. (2002a). The local boot-strapfor Markovprocesses. J. Statist. Plann. Inference 108301-328.

PAPARODITIS,E. and POLITIS,D. N. (2002b). The tapered blockbootstrap for general statistics from stationary sequences.Econom.J. 5 131-148.

PAPARODITIS,E. and POLITIS, D. N. (2002c). Local blockbootstrap.C. R.Math.Acad. Sci. Paris. 335 959-962.

PAPARODITIS,E. and POLITIS,D. N. (2003). Residual-based blockbootstrapfor unitroottesting.Econometrica71 813-855.

POLITIS,D. N. (2001a). Resampling time series with seasonalcomponents.In Frontiersin Data Miningand Bioinformatics:Proceedings of the 33rd Symposium on the Interface ofComputingScience and Statistics.

POLITIS,D. N. (2001b). Adaptivebandwidthchoice. J. Nonpara-metr.Statist.To appear.POLITIS,D. N. andROMANO,J. P. (1992a). A generalresamplingscheme for triangulararraysof a-mixing randomvariableswith applicationto the problemof spectraldensityestimation.Ann.Statist.20 1985-2007.POLITIS, D. N. and ROMANO, J. P. (1992b). A circular block-

resampling procedurefor stationarydata. In Exploring theLimitsof Bootstrap(R. LePageandL. Billard,eds.) 263-270.Wiley,New York.

POLITIS,D. N. and ROMANO,J. P. (1992c). A general theoryforlarge sample confidence regions based on subsamplesunderminimalassumptions.TechnicalReport399, Dept. Statistics,StanfordUniv.

POLITIS,D. N. and ROMANO,J. P. (1993). Estimating thedistributionof a Studentized statistic by subsampling.Bull.Internat.Statist.Inst.2 315-316.POLITIS, D. N. and ROMANO, J. P. (1994a). The stationarybootstrap.J. Amer Statist.Assoc. 89 1303-1313.POLITIS, D. N. and ROMANO, J. P. (1994b). Large sample con-fidenceregions based on subsamplesunder minimalassump-tions. Ann.Statist.22 2031-2050.

229


13/13

D. N. POLITISPOLITIS,D. N. andROMANO,J.P.(1995). Bias-correctednonpara-metricspectralestimation.J. TimeSer Anal. 16 67-103.POLITIS,D. N., ROMANO,J. P. and WOLF,M. (1999). Subsam-

pling. Springer,New York.POLITIS,D. N. and WHITE,H. (2001). Automaticblock-lengthselection for the dependent bootstrap.Econometric Rev. To

appear.QUENOUILLE,M. (1949). Approximate tests of correlation intime-series.J. Roy.Statist. Soc. Ser B 11 68-84.QUENOUILLE,M. (1956). Notes on bias in estimation.Biometrika43 353-360.RADULOVIC,D. (1996). The bootstrapof the mean for strong

mixing sequencesunderminimal conditions. Statist. Probab.Lett.28 65-72.RAJARSHI,M. B. (1990). Bootstrapin Markovsequences basedon estimates of transitiondensity.Ann.Inst. Statist. Math.42253-268.ROMANO,J. P. and THOMBS,L. (1996). Inference for autocor-relations under weak assumptions.J. Amer.Statist.Assoc. 91590-600.SAKOV,A. and BICKEL,P. (2000). An Edgeworthexpansionforthe m out of n bootstrappedmedian. Statist. Probab. Lett.49217-223.SHAO,J. and Wu, C.-F.J. (1989). A generaltheoryforjackknifevarianceestimation.Ann. Statist.17 1176-1197.

SHERMAN,M. andCARLSTEIN,E. (1996). Replicate histograms.J. Amer Statist.Assoc. 91 566-576.SHI, S. G. (1991). Local bootstrap.Ann. Inst. Statist. Math. 43667-676.SHIBATA,R. (1976). Selection of the order of an autoregres-sive model by Akaike's informationcriterion.Biometrika63

117-126.SINGH,K. (1981). On the asymptotic accuracyof Efron's boot-strap.Ann. Statist. 9 1187-1195.

SUBBA RAO, T. and GABR, M. (1984). An Introduction toBispectralAnalysisand Bilinear TimeSeries Models. LectureNotes in Statist.24. Springer,New York.

SWANEPOEL, J. W. H. (1986). A note on proving that the(modified)bootstrapworks. Comm.Statist. TheoryMethods.15 3193-3203.SWANEPOEL,J. W. H. and VAN WYK, J. W. J. (1986). The boot-

strap applied to power spectraldensity function estimation.Biometrika73 135-141.TONG,H. (1990). Non-linear Time Series:A DynamicalSystems

Approach.Oxford Univ. Press.TUKEY,J. W. (1958). Bias and confidence in not-quite largesamples(abstract).Ann. Math.Statist.29 614.

Wu, C.-F. J. (1986). Jackknife,bootstrapand other resamplingmethods in regressionanalysis (with discussion).Ann. Statist.14 1261-1350.

230

Impact Bootsrap Time Series

Documents

Transcript of Impact Bootsrap Time Series