Statistical Inference for Nonstationary Time Series...

IntroductionStatistical inference for APC models

Resampling in the time series contextFunctional data analysis (FDA) approach to APC signals

Statistical Inference for Nonstationary Time Series:Resampling and Functional Data Perspectives

Jacek Leśkow

Cracow Technical UniversityPoland

ESTE, Sao Carlos, August 2017

Jacek Leśkow Resampling, APC, FDA



Plan of the talk I1 Introduction

Motivating applicationsTime series approachThe concept of APC stochastic models

2 Statistical inference for APC modelsStationary case - spectral analysisHarmonizable processesHarmonizable almost periodically correlated (HAPC) modelsTime domain, discrete timeFrequency domain

3 Resampling in the time series contextResampling methods for APC modelsConsistency of resampling proceduresResampling in practice




Plan of the talk II4 Functional data analysis (FDA) approach to APC signals

MotivationFDA for stationary signalsReducing the dimensionality with FDAEmpirical basis





Motivating example no 1.

Time series of riverflows (Leśkow et al (2008), Journal of TimeSeries Analysis.)





Motivating example no 2.Nordspool energy market





Motivating example no 3.

Engine signal (Lafon, Antoni, Sidahmed, Polac, Journal of Soundand Vibration (2011))





Motivating example no 4.Wheel bearing signal - normally operating and inner race default





Motivating example no 5.Wind turbines





Stochastic processes setup

Let X = X (t); t ∈ R be a real (or complex) valued stochasticprocess. The process X is called strictly stationary if

for each t1, . . . , tk , each k ∈ Z and each h ∈ R we have

(Xt1 , . . . ,Xtk )d= (Xt1+h, . . . ,Xtk+h)

The process X is called weakly stationary iffor each t ∈ R we have EX (t) = EX (0)

for each t, τ ∈ R we have EX (t)X (t + τ) = EX (0)X (τ)





Traditional work on time series and signals

Identify some linear operation (filtering, differencing) makingyour data stationaryGo ahead with analysis for stationarized dataInterpret the obtained results

Problem: Many real life phenomena can not be stationarized in aneasy way.





Spectral theory for stationary stoch. proc

Stationary stochastic process X (t) ; t ∈ R has a spectralrepresentation:

X (t) =

∫e iλtZ (dλ).

Spectral measure is defined as

R ((a, b]× (c , d ]) = E [(Z (b)− Z (a)) (Z (d)− Z (c))] .

The spectral process Z has orthogonal increments.If the spectral measure R has a density f with respect to theLebesgue measure then f plays the role of the spectral density.





Initial remarksStationarity not sufficient, but a good starting point (spectraltheory)Modelling of the law - quite difficult in that approachFocus on moment behaviour





We want to find a suitably general prob/stat model for phenomenaethat exibit some repeatability in the first and the second order.

Definition of APCWe say that X (t) ; t ∈ Z - APC, when µX (t) = E (Xt) and theautocovariance function

BX (t, τ) = cov (Xt ,Xt+τ )

are almost periodic function at t for every τ ∈ Z.Function f is almost periodic in the norm ‖·‖ iffor each ε there exists an almost period Pε such that

‖f (·+ Pε)− f (·)‖ < ε

.





Every function f almost periodic (AP) in the norm sup can beapproximated UNIFORMLY on the real line via the Fouriertrygonometric polynomials. Let us apply it to autocovariancefunction:

BX (t, τ) =∑λ∈Λ

a (λ, τ) e iλt

Exercise: show that

Every weakly stationary process has an almost periodic covariance

AM signal X (t) = f (t) · Z (t) is APC if f (·) is AP and Z (·)stationary

If the function B(t, τ) is periodic in t then it is also AP in t.




Stationary case - spectral analysisHarmonizable processesHarmonizable almost periodically correlated (HAPC) modelsTime domain, discrete timeFrequency domain

"Repeatable" mean and covarianceAs noted earlier

the stochastic model should represent first and second order"repeatable" behaviorif the period is known then we should use cyclostationary/PCapproachif the period is not known,then APC approach is moreconvenient





Formalization - APC modelsLet X , as before, denote the stochastic model corresponding tosignals (time series) that are "repeatable" in the first and secondorder behaviour. This means that

EX (t) =∑

γ∈Γ bγ exp(iγt)

Cov(X (t),X (t + τ) =∑

λ∈Λ a(λ, τ) exp(iλt)

If X is APC then a(λ, τ) = limT→∞1T

∫ T/2−T/2 B(t, τ) exp(−iλt)

exists - is sometimes called cyclic spectrum.





Goals of time domain approachTherefore, the time domain approach to modelling C, PC and APCsignals is focused on

first order analysis : characterizing Γ and bγsecond order analysis: characterizing a(λ, τ) and Λ

Important: The frequency signature sets Γ and Λ do notnecessarily coincide.





Special role of Gaussian PC models

The stochastic process X (t) ; t ∈ R is called Gaussian if forevery k and every t1, . . . , tk ∈ R we have

(X (t1), . . . ,X (tk))d= Nk(µ(t1, . . . , tk),Σ(t1, . . . , tk))

Note: Gaussian C, PC yields periodicity of the law, sinceµ(t1, . . . , tk) and Σ(t1, . . . , tk) periodic.





Why knowing moments is sometimes not enough ?

In practice, you want to know the ’measure of precision’ ofyour estimateSpecial role of Gaussian models comes throughcharacterization of first and second order momentsFor Gaussian models, moments of the order α, where α > 2are completely characterized by first and second ordermoments





Spectral measure -stationary case

Take the stochastic process (or time series)X = X (t); t ∈ R (or Z) . Assume that X is weakly stationary(fixed mean, autocovariance depending only on time lag). Then wehave

Cov(X (t),X (t + τ)) =

∫exp(iλt)dZ (λ).

Here the spectral process has orthogonal increments. If Z is regularthan we can also write

Cov(X (t),X (t + τ)) =

∫exp(iλt)f (λ)dλ

where f (·) is the spectral density (power spectrum ,spectrum) of X .





Spectral density - stationary caseConvenient facts in stationary case

Spectral density f completely characterizes second orderbehavior of XVarX (t) =

∫f (λ)dλ (power spectrum)

X white noise - f flat.

Support of the spectral measure EZ (dλ)Z (dλ′) - main diagonal.





Figure: Wide-sensestationarity

Figure:Cyclostationarity Figure: APC





Spectral measure - harmonizable case

Harmonizable time series X (t) ; t ∈ Z.

X (t) =

∫ 2π

0e iλtZ (dλ).

Spectral bimeasure is defined as

R ((a, b]× (c , d ]) = E [(Z (b)− Z (a)) (Z (d)− Z (c))] ,





Spectral density - harmonizable caseIf the harmonizable, nonstationary X is regular then the spectralbimeasure R is fully characterized by the bispectrum f (λ, λ′) where

R(dλ, dλ′) =

∫ ∫f (λ, λ′)dλdλ′.

Therefore, for harmonizable, nonstationary X we have

Cov(X (t),X (t ′)) =

∫ ∫exp(iλt) exp(iλ′t ′)f (λ, λ′)dλdλ′.

The bispectrum completely characterizes the second-orderbehaviour of harmonizable, nonstationary X .





Spectral bimeasureSpectral bimeasure for harmonizable, APC stochastic processes(time series) is concentrated on lines parallel to the main diagonal.Formally, we have that the support set S has the form

S =⋃λk∈Λ

(λ, λ′

)∈ (0, 2π]2 : λ′ = λ± λk

.





Therefore, the second order behaviour of harmonizable APCstochastic process (time series) is completely characterized by

the bispectrum f (λ, λ′)

spectral support set S

Figure: spectral support set S for APC





Estimation for APC modelX (t) ; t ∈ Z - APC, when

µX (t) = E (Xt)

and the autocovariance function

BX (t, τ) =∑λ∈Λ

a (λ, τ) e iλt .

Time domain approach (µX (t) ≡ 0):

an (λ, τ) =1

n − τ

n−τ∑t=1

X (t + τ) X (t) e−iλt





Asymptotic normality (Dehay, Leśkow (1996))√

n (an(λ, τ)− a(λ, τ))d→N2(0,Σ(λ, τ)),

where

Σ(λ, τ) =

(σ11 σ12σ21 σ22

),

σ11 = 1T

T∑s=1

∞∑k=−∞

BZ (s, k) cos (λs) cos (λk),

σ22 = 1T

T∑s=1

∞∑k=−∞

BZ (s, k) sin (λs) sin (λk),

σ12 = σ21 = 1T

T∑s=1

∞∑k=−∞

BZ (s, k) cos (λs) sin (λk),

and Z (t, τ) = X (t)X (t + τ)− BX (t, τ),BZ (s, k) = Cov(Z (s, τ),Z (s + k , τ))





Conclusions from the previous work

Nice to have asymptotic results :)How to use them ? :(





Continuous time approach (Dehay, Dudek, Leśkow (2014))Consider the following conditions:

(A1) supt E|X (t)|4+δ

<∞ for some δ > 0, the fourth moment is almost periodic

in the following sense : the functionv 7→ cov

X (u + v + τ)X (u + v),X (v + τ)X (v)

is almost periodic for each u.

Moreover the process X is α-mixing and the mixing coefficient satisfies∫∞0 αX (t)δ/(4+δ) dt <∞,

(A2) For each λ ∈ Λ the following separability property is fulfilled

∑λ′∈Λ\λ

∣∣∣∣a(λ′, τ)

λ′ − λ

∣∣∣∣ <∞.Under the following conditions we have

√TaT (λ, τ)− a(λ, τ) L−→ N2(0,V (λ, τ)).

where





V (λ, τ) = limT→∞

T VaraT (λ, τ)

=

12

∫R

Bc(2λ, u + τ, τ, u)S1(λu) + Bs(2λ, u + τ, τ, u)S2(λu)

+Bc(0, u + τ, τ, u)S3(λu)

du,

with

Bc(λ, u, v ,w

)= lim

T→∞

1T

∫ T

0covX (s)X (s + u),X (s + v)X (s + w)

cos(λs) ds

Bs(λ, u, v ,w

)= lim

T→∞

1T

∫ T

0covX (s)X (s + u),X (s + v)X (s + w)

sin(λs) ds

S1(θ) =

[cos(θ) sin(θ)sin(θ) − cos(θ)

], S2(θ) =

[− sin(θ) cos(θ)cos(θ) sin(θ)

],

S3(θ) =

[cos(θ) sin(θ)− sin(θ) cos(θ)

]





Estimator at workTypical cyclostationary signal.





Autocovariance surface.





Properly working ball bearing

autocovariance structure





Ball bearing with rolling element damaged

autocovariance structure





Frequency domain analysis

Important in engineering applications

How to test for the presence of (modulating) frequency ?How to produce algorithms easy to use for practitioners ?

GOAL: To show utility of resampling.





Spectral estimation

Spectral density estimator in the APC context

Gn (f1, f2) =1

2πn

n∑t=1

n∑s=1

Kn (s − t) XtXse−if1te if2s .

Support lines

Figure: Wide-sensestationarity

Figure:Cyclostationarity Figure: APC





Frequency domain estimation

If(i) there exists δ > 0 such that supt∈Z‖Xt‖6+3δ ≤ ∆ <∞

(ii) wn = O (nκ) for some κ ∈(0, δ

4+4δ

)(iii)

∞∑h=1

h2rα(h)δ

2(r+1)+δ <∞ where r is the integer such that

r > max1 + δ,

1− κ4κ

,κ (1 + δ)

δ − 2κ (1 + δ)

then √

nwn

(Gn (ν, ω)− P (ν, ω)

)→ N (0,Σ (ν, ω))




Resampling methods for APC modelsConsistency of resampling proceduresResampling in practice

Resampling - simplest case - mean of i.i.d. sampleLet X1, . . . ,Xn - i.i.d from the distribution F . Let θ be theparameter describing the cdf F . Statistical inference has threefundamental goals:

building optimal estimators θ of θconstructing the confidence intervals for θtesting procedures for the unknown parameter θ

Example: Inference about µ = EXF . Optimal estimateµn = 1

n∑n

i=1 Xi . Confidence interval for µ built using the centrallimit theorem

µn ± zα/2 · SE (µn)





Nonparametric bootstrapLet X1, . . . ,Xn - i.i.d from the distribution F . We want to build aC.I around µ = EFX . Instead of using the formulaµn ± zα/2 · SE (µn) we can use nonparametric bootstrap.

Step 1: Draw with replacement X ∗11 , . . . ,X ∗1n fromX1, . . . ,Xn.Step 2: Calculate µ∗1n = 1

n∑n

j=1 X ∗1j .

Step 3: Repeat Step 1 and 2 B times to get µ∗1n , . . . , µ∗Bn .

Step 4: Calculate C.I. from Step 3.





Resampling for time series

Quick facts:Nonparametric bootstrap - useless, as it destroys thelongitudinal structureStationary time series: moving block bootstrap (MBB)Cyclostationary time series: periodic block bootstrap (PBB),seasonal block bootstrap (SBB), generalized seasonal blockbootstrap, subsampling.





Moving block bootstrap (MBB)

Let (X1, ...,Xn) - sample from the time series andB(j , b) = (Xj , ...,Xj+b−1) b−block of the data. The length of theb−block is b = bn. Assume, without a loss of generality, thatk = n/b ∈ N.

The MBB Algorithm

Let the i.i.d. random variables i1, i2, ..., ik come from thedistribution

P(ij = t) =1

n − b + 1for t = 1, ..., n − b + 1.

To obtain the MBB resample

(X ∗1 ,X∗2 , ...,X

∗n )

we join the blocks (B(i1, b),B(i2, b), ...,B(ik , b)) together.





Generalized Seasonal Block Bootstrap (GSBB)

Let d - the (unknown) period length, b - the length of the block, n- sample size. For simplicity, let n = lb. Let also n = dω. GSBB isdefined in the following way.

Step 1: Choose the length of the block bStep 2: Let t = 1. Create a bootstrapped sample(X ∗1 , . . . ,X

∗1+b−1) sampling from (Xk1 ,Xk1+1, . . . ,Xk1+b−1),

where k1 is a discrete uniform random variable on the discreteset 1 + vd , v = 0, 1, . . . , ω − 1.Step 3:. Repeat Step 2 taking t = b + 1, 2b + 1, . . . , lb + 1and get a bootstrapped sample (X ∗1 , . . . ,X

∗n ).





Subsampling

1 Create the blocks Bi = (Xi , . . . ,Xi+b−1) of the length b for asample X1, . . . ,Xn from the original time series x(t).

2 Recalculate the subsampled versionan,b,t(λ, τ) = 1

b∑t+b−1

j=t X (j + τ)X (j)e−iλj for t < n-τ -b+1of the original estimate an(λ, τ)

Subsampling procedure is consistent for APC time series (seeLeśkow (2008)).





Subsampling for Fourier coefficient of autocovariance function(Leśkow et al (2008))

Let X (t) : t ∈ be APC time series. Assume that(i) b →∞ but b/n→ 0(ii) suptE |X (t)|4+4δ <∞

(iii)∞∑

k=0(k + 1)2α(k)

δ4+δ <∞

(iv) the functionV (t, τ1, τ2, τ3) = E (X (t) X (t + τ1) X (t + τ2) X (t + τ3)) isalmost periodic.

Then subsampling is consistent, which means that

supx

∣∣Jn (x ,P)− Ln,b (x)∣∣ P→ 0.





What does that all mean ??

You can use subsampling as a statistically valid technique tobuild your C.I and testYou can even have fun running some Matlab programsFor PhD students: many unresolved research problems to workwith





Subsampling in practice

Typical cyclostationary signal analysis.

Underlying residual signal





Subsampling - wheel bearing signal

Wheel bearing data - frequency signature. Healthy structure (left) and damagedstructure (right).





Big data, FDA and APC

Motivation to use the FDA comes from massive data sets (curves)available while doing the analysis of

GRF signalenergy datawheel bearing data, spectrogram





Motivating example.Spectrogram for cyclostationary signal





FDA can provideSignal representation taking into account large number ofobservations (20kHz)Relatively simple model (principal component, dimensionalityreduction, autoregressive structures)Estimating procedures using operator theory and Hilbert spaceapproach





Introduction to FDATo start, we will see any signal X (t) ; t ∈ Z as a collection ofindependent curves yi (u), i = 1, . . . ,N; u ∈ A belonging to aHilbert space H. For simplicity, assume that H = L2[A] andA = [0, 1]. This is for example the spectrogram perspective, wherefor the function yi (u) the index i is the window number and u isthe frequency argument.

Now, let us see the fundamental steps of the FDA approach tosignal analysis.Step 1 The stochastic model for the signal is the random elementX from (Ω,F ,P) to L2[0, 1].





FDA - cont.

Step 2 Expectation of the random elementIf X is integrable, there is a unique function µ ∈ L2 such thatE〈y ,X 〉 = 〈y , µ〉 ∀ y ∈ L2. It follows that µ(t) = E[X (t)] fort ∈ [0, 1].Step 3 Covariance operatorFor X intergrable and EX = 0, the covariance operator of X isdefined by

C (y) = E[〈X , y〉X ], y ∈ L2.

Notice that

C (y)(t) =E[〈X , y〉X (t)] = E∫

X (s)y(s)dsX (t) =

=

∫E[X (s)X (t)]︸︷︷︸

=c(s,t)

y(s)ds =

∫c(s, t)y(s)ds.





FDA - covariance

Step 4. Eigenvalues and eigenfunctions of the covarianceoperatorLet vj , λj , j ≥ 1 be the eigenfunctions and the eigenvalues of thecovariance operator C . The relation C (vj) = λjvj implies that

λj = 〈C (vj), vj〉 = 〈E[〈X , vj〉X ], vj〉 = E〈X , vj〉2.

Having defined the mean and the covariance of the randomelement, we will proceed to the usual statistical questions, that is:

What is the approximate distribution of the linear statistics forsamples generated by our random element ?How to introduce the estimator of the covariance ?Is there any chance for the dimensionality reduction ?





FDA - CLT

Suppose Xn, n ≥ 1 is a sequence of iid mean zero randomelements in a separable Hilbert space such that E‖Xi‖2 <∞. Then

1√N

N∑n=1

Xnd→ Z

where Z is a Gaussian random element with the covariance operator

C (x) = E[〈Z , x〉Z ] = E[〈X1, x〉X1].

Notice that a normally distributed function Z with a covarianceoperator C admits the expansion (Karhunen-Lòeve representation)

Z d=∞∑j=1

√λjNjvj

where Njiid∼ N (0, 1), λj , vj - eigenvalues, eigenfunctions of the

covariance operator C . Jacek Leśkow Resampling, APC, FDA




Functional parameters ...

µ(t) =E[X (t)] (mean function);c(t, s) =E[(X (t)− µ(t))(X (s)− µ(s))] (covariance function);

C =E[〈(X − µ), ·〉(X − µ)] (covariance operator).

and estimators:

µ(t) =1N

N∑i=1

Xi (t);

c(t, s) =1N

N∑i=1

(Xi (t)− µ(t))(Xi (s)− µ(s));

C (x) =1N

N∑i=1

〈Xi − µ, x〉(Xi − µ), x ∈ L2.





FDA estimation - cont.

Assume that the observations have mean zero. We therefore have

c(t, s) =1N

N∑i=1

Xi (t)Xi (s); C (x) =1N

N∑i=1

〈Xi , x〉Xi , x ∈ L2

thereforeC (x)(t) =

∫c(t, s)x(s)ds, x ∈ L2.

Introduce the random functions

ZN(t, s) =√

N(c(s, t)− c(s, t))

where c(s, t), c(s, t) are centered with the (sample) mean function.





FDA CLT for covariance

If the observations X1,X2, . . . ,XN are iid in L2, and have the samedistribution as X , which is assumed to be square integrable withEX (t) = 0 and E‖X‖4 <∞, then ZN(t, s) converges weakly inL2([0, 1]× [0, 1]) to a Gaussian process Γ(t, s) with EΓ(t, s) = 0and

E[Γ(t, s)Γ(t ′, s ′)] = E[X (t)X (s)X (t ′)X (s ′)]− c(t, s)c(t ′, s ′).

For the users of spectrogram

If X1,X2, . . . ,XN represent functions of the frequency (verticalstripes), then FDA approach provides a simple description of thewhole energy of the signal.





FDA and reduction of dimensionality

Let λ1 > λ2 > . . . be the eigenvalues of operator C . Theeigenfunctions vj are defined by Cvj = λjvj . The vj are typicallynormalized, so that‖vj‖ = 1.

cj = sign(〈vj , vj〉)∫c(s, t)vj(s)ds = λj vj(t), j = 1, 2, . . . ,N.

Using the above ideas we will construct optimal empiricalorthonormal basis for our signal X (t) ; t ∈ Z represented byrandom elements X1, . . . ,XN .





Suppose we observe functions x1, x2, . . . , xN . Fix an integerZ 3 p < N(p N). We want to find an orthonormal basisu1, u2, . . . , up such that

S2 =N∑

i=1

∥∥∥xi −p∑

k=1

〈xi , uk〉uk

∥∥∥2is minimal.

Empirical basis

xixixi = [〈xi , u1〉, 〈xi , u2〉, . . . , 〈xi , up〉]T .

The functions uj are called collectively the optimal empiricalorthonormal basis or natural orthonormal components.





Empirical basis and covariance

The functions u1, u2, . . . , up minimizing S2 are equal (up to a sign)to the normalized eigenfunctions of the corresponding samplecovariance operator.

We have

S2 =N∑

i=1

(‖xi‖2 −

p∑k=1

〈xi , uk〉2)

S2 is minimum, whenN∑

i=1

p∑k=1〈xi , uk〉2 is maximum.

N∑i=1

p∑k=1〈xi , uk〉2 = N

p∑k=1〈C (uk), uk〉

= Np∑

k=1

∞∑j=1

λj〈uk , vj〉2 ≤ Np∑

k=1λk

maximum is attained if u1 = v1, u2 = v2, . . . , up = vp.Jacek Leśkow Resampling, APC, FDA




Dimensionality reduction can be achieved byConstructing the empirical basisChoosing the number of components p such that the modelwill exhaust the most important part of the energy(variance/covariance) of the signalWorking with eigenvalues instead of many functions

Choosing pTo this end we can consider the function

CPV (p) =

p∑i=1

λi

N∑i=1

λi





F-AR(1) model

Our starting point is again a sequence of Hilbert space valuedrandom elements X1, . . . ,XN that no longer are assumedindependent. In the spectrogram representation, it is NOT realisticto assume that vertical stripes are independent.Consider the model

F-AR(1)

Xn = Ψ(Xn−1) + εn

where Ψ ∈ L while L is the space of bounded continuous linearoperators on L2 equipped with the sup norm. Moreover, εn is asequence of iid mean zero elements in L2.





F-AR(1) model

It is known that under appropriate conditions (see Horvath,Kokoszka (2012)) we have that F-AR(1) is causal and strictlystationary.

Example of Ψ

ConsiderΨ(x)(t)

def=

∫ψ(t, s)x(s)ds

where x ∈ L2 and∫ ∫

ψ2(t, s)dtds < 1.





Estimation in F-AR(1)

Define the lag 1 autocovariance operator

C1(x) = E[〈Xn, x〉 Xn+1], x ∈ L2

Like in the scalar case, we have the relationship

C1 = ΨC

where C is the covariance operator. Thus, to estimate Ψ we coulddefine

Ψ = C1C−1.

Warning: getting C−1 may be difficult. However, we will use theempirical basis principle and take only p first components.





Estimation in F-AR(1)

Instead, we use

ICp(x) =p∑

j=1λ−1j 〈x , vj〉vj .

We get: C1(x) =1

N − 1

N−1∑k=1〈Xk , x〉Xk+1

For any x ∈ L2 obtain

C1 ICp(x) =1

N − 1

N−1∑k=1

p∑j=1

λ−1j 〈x , vj〉〈Xk , vj〉Xk+1.

The estimate

Ψp(x) =1

N − 1

N−1∑k=1

p∑j=1

p∑i=1

λ−1j 〈x , vj〉〈Xk , vj〉〈Xk+1, vi 〉vi .





PFAR(1) model

The functional time series Zi fulfill the PFAR(1) model if for eachi = (n − 1)T + ν with ν = 1, . . .T we have

Zi = Φν(Zi−1) + εi . (1)

In the above model the operators Φν , ν = 1, . . . ,T areHilbert-Schmidt integral operators in L2 with corresponding kernelsφν fulfilling the assumption

Φν(x)(t) =

∫φν(t, s)x(s)ds.

The sequence εi is a sequence of iid mean zero elements in H.





Causal representation for PFAR(1)

In order to have a causal representation of PFAR(1) model we needthe followingAssumption A1. There exists an integer j0 such that for eachν, ν = 1, . . . ,T we have ||Φj0

ν ||L < 1.

We get the followingTheorem. If condition A1 holds then we get the causalrepresentation for our periodic time series Zi in the following form

Zi =∞∑j=0

Φjν(εi−j), ν = 1, . . . ,T ; i = (n − 1)T + ν.





Covariance operators

Introduce the symbol ⊗ to denote the linear operator defined forx , y and z from H as: x ⊗ y(z) =< x , z > y .For a Hilbert space valued random element Z define the covarianceoperator Γ as

Γ = E (Z ⊗ Z ).

Let now Zn be the time series with values in Hilbert space. Thelag-p autocovariance operator Cp as

Ci ,p = E (Zi ⊗ Zi+p).

For Zn - strictly stationary we get that Γ does not depend on nand Ci ,p does not depend on i . Recall that i is the current index ofan observation (curve) and in the periodic case we havei = nT + ν, where T is the period and ν = 1, . . . ,T .





More on PFAR(1) model

Corollary 1. Let Γi be the covariance operators corresponding tothe time series Zi from PFAR(1) model with period T ,i = (n − 1)T + ν. Then for each ν = 1, . . . ,T the operatorsΓ(i−1)T+ν are identical for all i ∈ N. This means that the model(1) generates the periodic structure of covariance operatorsΓν ; ν = 1, . . . ,T with the period equal to T .

Corollary 2. Assume that the time series Zi fulfills the PFAR(1)model defined in (1), where i = (n − 1)T + ν with ν = 1, . . . ,T .Let also C1,ν be the lag-1 autocovariance operator for eachν = 1, . . . ,T . Then

C1,ν = ΦνΓν .

This means that the model (1) generates the periodic structure oflag-1 covariance operators with the period equal to T .





PFAR(1) operator parameters and estimators

PFAR(1) generates the operator parameters: Cν ,Φν and Γν . Formean zero functional time series Zi , 1 ≤ i ≤ (n − 1)T + ν andν = 1, . . . ,T i the estimators Γν and C1,ν have the form:

Γν,n =1n

n∑i=1

Z(i−1)T+ν ⊗ Z(i−1)T+ν

and

C1,ν,n =1

n − 1

n−1∑i=1

Z(i−1)T+ν ⊗ Z(i−1)T+ν+1.

Naively, one could therefore think that

Φν = C1,ν,nΓ−1ν,n .





Unbouded inverse!

Γν is positive definite therefore the inverse Γ−1ν is not bounded!Γν has a representation:

Γν(x) =∞∑j=1

λν,j < x , ην,j > ην,j , , ν = 1, . . . ,T

so

Γ−1ν (x) =∞∑j=1

λ−1ν,j < x , ην,j > ην,j , , ν = 1, . . . ,T





First q components

To avoid estimating an unbounded operator we will introduce q firstfunctional principal components of the operator Γ, that is consider

Γqν(x) =

q∑j=1

λν,j < x , ην,j > ην,j , ν = 1, . . . ,T .

The above formula allows to introduce an estimator

Γqν,n(x) =

q∑j=1

λν,j < x , ην,j > ην,j , ν = 1, . . . ,T . (2)





Estimating Φν

The estimator of the operators Φν will be defined in the followingway:

Φν,n = C1,ν,n(Γqν,n)−1.

We need to have a more exact (and, alas, more complicated) formof the estimator Φν,n.Recall that C1,ν,n = 1

n∑n−1

k=0 ZkT+ν ⊗ ZkT+ν+1. Moreover,

(Γqν)−1(x) =

q∑j=1

(λν,j)−1 < x , ην,j > ην,j

where ν = 1, . . . ,T .





The estimator of the periodic operator

Therefore, we have the following equation for the estimator Φν,n ofthe periodic operator Φν,n in the PFAR(1) model

ZnT+ν = Φν(ZnT+(ν−1)) + εnT+ν

The estimator

Φν,N(x) =1

N − 1

N−1∑i=1

p∑j=1

p∑k=1

λ−1ν,j 〈x , ην,j〉〈Zi , ην,j〉〈Zi+1, ην,k〉ην,k .

where N = nT + ν, ν = 1, . . . ,T .





Conditions for consistency

Lp-m-approximability

A sequence Zn of Hilbert space valued random elements is Lp-mapproximable if for each n Zn = f (εn, εn−1, ...), where εi - i.i.d.Moreover, there exists an m-dependent sequence Z (m)

n such that

∞∑m=1

||Zm − Z (m)m ||p <∞.

A periodic time series Zi fulfilling the PFAR(1) model is L4-mapproximable.





Consistency and CLT

Theorem

Assume that our PFAR(1) model has fourth moments and is L4 mapproximable. Then(F1) E ||Γν,n − Γν ||2 = O(n−1).

(F2) E |λν,j − λν,j |2 = O(n−1).

(F3) E ||cj ηj − ηj ||2 = O(n−1) where cj is a random sign.

Conditions (F1), (F2) and (F3) imply consistency of theestimator Φν,N .





Applications of FDA to PC signals

Clustering the STFTs





Some open questions:

APC models from FDA perspectiveSolid limit theory approach for the estimatorsValidity of bootstrap





Acknowledgement

The results represent the joint work of: Dominique Dehay, ElżbietaGajecka, Oskar Knapik, Jacek Leskow, Sofiane Maiz and AntonioNapolitano.

It was supported by: Cracow Technical University, Cracow, LASPIRoanne, (France), Universite Rennes II (France), ParthenopeUniversity, Napoli (Italy) and the Polish National Center for Scienceunder the grant 2013/10/M/ST1/00096.





References- selected

Cioch, Knapik, Leśkow (2013), Finding a frequency signature forcyclostationary signals with applications to wheel bearing diagnostics,Mechanical Systems and Signal processing, to appear.Dehay,Dudek, Leśkow (2012), Subsampling for continuous-timenonstationary stochastic processes, to appear in Journal of StatisticalPlanning and Inference.Gardner, Napolitano, Paura (2006) Cyclostationarity : half a centuryof research, Sig. Proc., 86 639–697.Hurd (1991) Correlation theory of almost periodically correlatedprocesses, J. Multivariate Anal. 30 24–45.Horvath, L. and Kokoszka, P. (2012), Inference for Functional Datawith Applications, Springer Series in Statistics.Lenart,Leśkow, Synowiecki (2008), Subampling in estimation ofautocovariance for PC time series, J. Time Ser. Anal. 29, 995–1018.K.S. Lii, M. Rosenblatt (2002), Spectral analysis for harmonizableprocesses, Ann. Statist. 30 (2002) 258–297.





Thank you!


Statistical Inference for Nonstationary Time Series...

Documents

Transcript of Statistical Inference for Nonstationary Time Series...