Identification of stochastictime-varying systemsK.A.F. Moustafa, B.Sc. (Eng.), M.Sc, Ph.D.

Indexing terms: Control theory, Control systems, Process control

Abstract: The parameter identification of a class of time-varying stochastic systems is considered in this paper.Online schemes which track the time-varying parameters in real time are proposed. Conditions which guaranteeeither almost sure convergence of the estimation error to the null or bounded mean-square error are obtained.The analysis is based on stochastic Lyapunov functions. This allows the convergence conditions to beweakened, and makes investigation of the stability problem of the proposed identification schemes possible.

1 Introduction

In recent years, there has been a rapid development ofonline process control techniques. The use of process com-puters in the control and optimisation of dynamic systemsof many industrial applications is steadily increasing. Thishas naturally attracted the attention of many researcherstoward online or recursive identification schemes. Muchwork has been done in the area of recursively identifyingtime-invariant or constant-parameter systems [1, 2, 3, 4].

An important application of on-line schemes is whenthe system parameters are time varying, where it is absolu-tely necessary to track the parameters in real time. Anexample of such application is the high-performance aero-space adaptive control system [4]. The problem of iden-tifying time-varying systems is rather difficult, and muchwork is still to be done. It is known that no rigoroustheory exists for the identification of time-varying systemsif no structural information about the parameter variationis available [4]. Therefore, to make the problem tractable,it is usually assumed that some a priori information aboutthe parameter variations is available.

Schemes for identifying constant parameter systems canbe extended, under appropriate conditions, to the time-varying case, but only when deterministic variations areconsidered [3, 4]. The problem, however, becomes moreinvolved when the parameters vary in a stochastic way.These stochastic variations result from the presence of astochastic deriving term in the equation error system of theidentification scheme. As a result, estimates with boundederror can only be obtained, unless certain conditionsregarding the noise sequences are imposed [4, 5].

Kalman filtering techniques are generalised to obtainparameter identifiers for systems with stochastically time-varying parameters [6]. In this approach, the parametersequence must be a stationary process with rational spec-tral density [1, 6]. Also, knowledge of the covariancematrices of the noise sequences characterising the varia-tions in the model parameters is required. Such assump-tions limit the practical applicability of this approach.

Identification and stochastic adaptive control aspects ofsystems, whose parameters vary in a random fashion, andrelated problems are recently studied by Caines [7].

In this work, we propose general identification schemesfor identifying a class of time-varying systems. The analysisis based on stochastic Lyapunov functions. This consider-

Paper 2535D, first received 1st March 1982 and in revised form 7th March 1983The author is with the Department of Mechanical Engineering, College of Engi-neering, Mosul University, Mosul, Iraq, on leave from the Department of Designand Production Engineering, Faculty of Engineering, Ain-Shams University, Cairo,Egypt

ably simplifies the convergence proof. Conditions usuallyimposed when treating time-varying systems [4, 5, 8] arerelaxed. Neither stationarity of the observations nor stabil-ity of the system to be identified is needed. Theboundedness of the input-output and noise sequences isnot assumed. In previous analyses of time-varying systemsconvergence is obtained with probability 1 (wp 1) not withrespect to the whole space, but with respect to the setwhich contains all sample paths which have bounded error[4, 8]. In the present work, the use of stochastic Lyapunovfunctions enables us to obtain almost sure convergence ofthe estimates or bounded mean-square error depending onthe noise properties. Further, the investigation of the sto-chastic stability problem of online identification schemes,which is important in practice, is made possible by thisapproach.

The organisation of the paper is as follows. In Section 2,we give the problem formulation. The proposed identifica-tion schemes and convergence analysis are given in Section3. An upper bound on the mean square error is obtained inSection 4. Computer simulations to illustrate the resultsare given in Section 5.

2 Problem formulation

Let us consider the following linear discrete-time system:m n — m

y(k) = £ ai{k)y{k - 0 + I b&Mk - i) + w(k) (1)i = i i = i

where u(k) and y(k) are scalar system input and output attime k, respectively; [w(/e)] is an unmeasurable randomsequence representing the system noise and satisfies

(Al) £w(/c) = Ew(k)w(j) ^ c2w(k)3kj

(A2) E[u{k)/w{k + 1), w(k + 2), ...] = 0

where Skj is the Kronecker delta function; a^k) and bare the unknown system parameters to be identified. Weassume that variations of the parameters with time can bemodelled by the stochastic dynamical system

0(k + 1) = f(k)0(k) + v(k)

f(k) = 1 - f*(k), f*(k) > 0 (2)

where 0(k) is an n-dimensional parameter vector defined as

8(k) = Ib.ik), ..., bn_m(k), ai(k), ..., am(k)Jf

In eqn. 2,f(k) is a known deterministic function containingthe a priori information about 6{k); [v(/c)] derives the noise

t ' Denotes the transpose of a matrix or a vector.

1EE PROCEEDINGS, Vol. 130, Pt. D, No. 4, JULY 1983 137

sequence reflecting the uncertainities in this information.This type of model was reported in References 4 and 5.The deriving noise sequence is assumed to satisfy

(A3) Ev(k) = 0, Ev'{k)v(j) ^ c2v(k)SkjI

(A4) Ev(k)w(j) = 0, V k, j

In this paper f(k) is assumed to be deterministic andknown a priori. However, two important classes of par-ameter identification problems, which are widely con-sidered in the literature, are contained as special cases inthe present formulation; namely, systems with constantparameters and systems whose parameters contain anadditive stochastic noise term.

Our problem is to develop online identification schemesto estimate the unknown time-varying parameter vector0(k). These identification schemes are intended to utilisethe current measurements of input and output, u(k) andy(k), respectively, and to produce estimates 9(k). Sufficientconditions, which guarantee either almost sure con-vergence of the estimates to the true values or boundedmean-square error, are to be sought. The analysis will bebased on stochastic Lyapunov functions. This approachhas recently been applied successfully to developing onlineschemes for identifying constant-parameter stochasticsystems [9, 10, 11].

3 Identification schemes and convergence analysis

Using the parameter vector 0(k), eqn. 1 is written as

y(k) = 0'(k)z(k - 1) + w(k)

z(k - 1) = [u{k - 1), . . . , u(k - n + m), y(k - 1), . . . ,

y{k-mj] (3)

An estimate 6{k) of 0(k) is obtained from the followingrecursion:

^Ofc+i) = fUkWUk) ~ eUk)Q(Jk ~ l)eUk)

O(j) = O(jk) j e Jk

where the sequence of integers jk satisfies

1 = h < h • • •with

•'fc = Ok > A ~l~ IJ ' " J i + l 1)

nk = jk+ 1 ~ Jk ^ 1

In the above, the following definitions are used:

&(jk) = [zOfcX z(jk + 1)» - -, z(Jk + i — 1)] (««k)

</*) = [</*), </* + 1). • • •, e(jk+i - 1)]' (nk 1)

</*) = ^Ok) - y(jk% y(jk) = Q'UMk - i)

^,N _ POk)

Using eqns. 2, 3 and 4, we can write

(4)

/Ok - 1) = tr Q(jk - 1)

Qtik - i) = n'(/k

here, p{jk) is a non-negative design parameter, and trdenotes the trace operation. Define the estimation errorvector

(5)

where

Consider the candidate Lyapunov function

v(jk) = ^'OJ^Ok) (6)

The above scheme processes the data in batches, each ofsize nk, and the parameters are updated accordingly. Apartfrom the usefulness of batch processing schemes in manyapplications [11, 12], they are of particular interest in thepresent work; some technical details are given in the proofof theorem 3 in the next Section. In the analysis of theo-rems 1 and 2 below, the usual sequential processing ofscheme 4 with nk = 1 is used for proving almost sure con-vergence of the parameter estimates. Upper bounds on themean-square error of the estimates is obtained in theorem3 of the next Section, using the batch processing schemewith nk ^ n. Therefore, we will first consider the sequentialprocessing scheme by taking nk = 1 in the recursion of eqn.4. For this case, it is obvious that jk = k, and V(k) usingeqn. 5 can be shown to satisfy

V(k - V(k) = g(k)V(k) + £2{k)X{k - l)w\k)

+ v'(k)v(k) + ai

+ a3(k)W(k)

where

g(k) = f*2(k) - 2f*(k)

ai(k) = s2(k)X(k - 1) - 2e(k)f(k)

a2(k) = -2e(k)z(k - l)w(/c)

+ 2e(k)Z{k)z(k - 1)

a3(k) = [2e(/c)/(/c) - 2e\k)X{k -

a'2(k)v(k)

(7)

Denote the Borel field generated by [w(0), w(l), . . . ,w(/c - 1), v(0), v(l), . . . , v(k - 1), u(0), u(l), . . . , u(k - 1)] byF(k). The Borel fields, thus defined, are increasing. Since£(/c), s(k) and \p{k) are continuous functions of [w(0), . . . ,w(k - 1), v(0), ...,v(k- 1), M(0), . . . , u(k - 1)], they belongto F(k). Taking the conditional expectation of both sides ofeqn. 7 with respect to the Borel field F(k), and usingassumptions Al to A4, we obtain

ElV(k - V(k) ^ g(k)V(k)

e2(k)l(k - \)c2jk) + c2v(k) (8)

We prove in theorem 1 that, if the design parameter p(k) isproperly chosen, the almost sure convergence of the esti-mation error vector \J/(k) to the null is guaranteed for aclass of systems of eqns. 1 and 2. We should note that noassumptions regarding the input sequence, such as enoughgenerality or persistent excitation, which are usually usedwhen proving convergence of online identification schemes[2, 11], are used. Also, the boundness assumption of theinput-output and noise sequences is dispensed with.

We have the following lemma which is useful for laterdevelopments. It will be used to prove that certain sets areeither ultimately reached or returned to, before going toinfinity, almost surely, by the sample path of

138 IEE PROCEEDINGS, Vol. 130, Pt. D, No. 4, JULY 1983

LemmaConsider the stochastic candidate Lyapunov functionV(k) = ij/'(k)ilf(k), and the open sets Pk. Let

E[V{k - V(k) = AV(k) ^ -b(k) in /*

where P° is the complement of P. Let the indicator setfunction of the (k, co) set, where ij/(k) is in Pk and not in Pk

(in PI), be I{k, co) and T{k, co), respectively. Assume that

Z£[AK(fc)/(/c, co)] < oo

then(a) limfc F(k, co) = 0 wp 1, if b(k) ^ b > 0(£>) There exists a subsequence /(&,•, co) such that

lim F(kj, co) = 0 wp 1, if b(k) ^ 0, and

(9)

= oo

(c) With Pk = P and condition 9 not necessarily satis-fied, the trajectory of [^(/c)] always returns to P wp 1before going to infinity.

ProofWe can write

EV(0) - EV{N + 1) = - X £ AF(/c)fc = 0

E[b{k)T{k, co)]fc = 0

- £ £[AK(/e)/(/e,fc = 0

Rearranging and taking limits, we have

fc=0

which implies that

EV(0) + ffc=0

, co)]

(10)£ b{k)T(k, co) < co wp 1fc = 0

If 6(/c) ^ fc > 0, we have

lim T(k, co) = 0 wp 1

This proves (a) of the lemma.Now we prove (b) of the lemma. Since Hb{k) = oo and

b(k) ^ 0, eqn. 10 implies that there exists a subsequence\T(kj, co)] such that

lim T(kj, co) = 0 wp 1j

Proof of part (c) is very similar to that of theorem 8 inKushner [13] which is given for the continuous time case.

Theorem 1Consider the time-varying system described by eqns. 2 and3. Let the unknown parameter vector 0(k) be identified bythe recursion eqn. 4 with nk = 1. Let assumptions A1-A4be satisfied. Then the estimation error vector ij/(k) con-verges to the null wp 1 if

(Cl) Zp2(k) < oo and Xp(k) = oo

(C2) max supcj(k)

Ak)\ ~= M < co

(C3) f*(k) = O(p(/c)), i.e., f*(k) is, at most, of the sameorder as p(k), and inf [f*(k)/p(k)~] > 1

are satisfied.

IEE PROCEEDINGS, Vol. 130, Pi. D, No. 4, JULY 1983

ProofRewrite eqn. 8:

E[V(k + l)/F(k)] - V(k) = AV(k)

^ g{k)V(k) + ai(kK2(k) + t\k)X{k - \)c2w(k) + c]{k)

Which can be bounded, for large k, as

AV(k) ^ g(k)V(k) + 2p2(k)M

since a^k) ^ 0, for large k, by conditions Cl and C3. Con-dition C2 is used to obtain the second term of the right-hand side. Consider the set Pk defined by

where V(k) is the candidate Lyapunov function previouslydefined. Consider also a v-neighbourhood Nv(Pk) => Pk

such that

V(k) ^ lp{k) + v]M in Nv(Pk)

We have for large k, by conditions Cl and C3, that

AV(k) ^ -hp(k)lp(k) + v]M, h > 0

^ -hMvp(k) for ${k) e ^(P^ (12)

We also have

AV(k)^2p2{k)M, V k

which implies that

S AV(k)I(k, co) < oo (13)

where I(k, co) is the indicator set function of the (k, co) setwhere if/(k) is in Nv(Pk). Since hMv > 0 and HhMp{k) = oo,the lemma can be applied to eqns. 12 and 13, to obtain

lim I(kj, co) = 0 (14)

where I(k, co) is the indicator set function of the (k, co) setwhere \j/(k) is in Nc(Pk). Note that p(k) = 0 implies thattj/(k) = if/ (a constant), which, together with eqn. 14, imply

- * G NJLPJ

Since v is arbitrary, we have

This completes the proof.

We should note that theorem 1 provides sufficient con-ditions for the identification scheme to track the time-varying parameters in real time with error that, almostsurely, converges to zero. These conditions are weakerthan those usually imposed when treating time-varyingsystems [2, 4, 5, 8]. In particular, the conditions on theinput sequence are relaxed and no boundedness assump-tions regarding the input, output and noise sequences areimposed.

However, the time-varying parameters are required intheorem 1 to vary in such a way that they approach zeroas time increases, because of condition C3. Such a class ofsystems appears in some practical applications where onlythe transient period of time is of greatest interest. For suchcases, parameter variations in the transient period arerequired to be tracked online. Consider the case where adynamical system is removed from a stable equilibriumstate by a random disturbance whose intensity eventuallydecreases to zero as time goes to infinity. Examples areearthquakes and echoing series of shocks [13].

t yv is the complement of /V

139

The tracking of time-varying systems with parametersthat do not tend to zero is considered in theorem 2. Con-vergence wp 1 is obtained, provided that the data sequenceis stationary ergodic and general enough (see definition ofgeneral enough sequence below). The more general case,where a lower bound is assumed on the noise variance, istreated in theorem 3 of the next Section. For this case, anupper bound on the mean-square error of the estimates isobtained.

Definition (general enough sequence)The sequence [z(/c)] is said to be general enough if thereexists a sequence of integers 1 = jx < j 2 , . . . , such thatw i t h n k = ; k + 1 - ; f c , w

q < oo(/c = 1, 2, 3, ...)

inf-AmJnk \_j

j - l)z'U -

e Jk ']-= T > 0

where Jk is the set index (jk, jk + 1, ..., jk + l — 1), andAmin( •) is the minimum eigenvalue of (•).

Theorem 2Consider the time-varying system described by eqns. 2 and3 with the data sequence [z(/c)] being general enough. Letthe unknown parameter vector 0(k) be identified by theonline recursion of eqn. 4 with nk = 1. Let assumptionsA1-A4 be satisfied. Then the estimation error vector ij/(k)converges to the null wp 1 if the following conditions aresatisfied:

(Cl) Sp2(/c) < oo and = oo

(C2) max sup[•cj(k)

, supx(k-\y^ P\k)= M < oo

(C3) lim f*(k) = 0

(C4) [z( •)] is an ergodic stationary sequence.

ProofInequality 8 can be bounded, for large k, as

E{V(k + 1) /F(fc)] - V(k) ^ -he (k)£2(k) + 2p\k)M, h>0

where conditions (Cl to C3) are used.Consider the set

Proceeding in the same way as in theorem 1, and using thelemma, we obtain

<K/e)^«M[>:£2(/c) = 0] w p l

If W ) ] is general enough, and satisfies condition C4, itcan be shown that the set [^: £2(k) = 0] contains thesingle point i]/ = 0 [Reference 11, theorem 3].

We should note that the stability of the proposed onlineidentification schemes can be investigated in the same wayas in Reference 11. In fact, it can be proved that the identi-fication schemes of theorems 1 and 2 are stable withrespect to the triple (Qr, Qm, p), in the sense that \J/(0) in Qr

implies that

prob [^(/c) e Qm, for all k < oo] ^ p

where Qx is the open set \jj/: \\J/\2 < x]. For more details,the reader is referred to References 3 and 11.

4 Conditions for bounded mean-square error

We have proved in Section 3 the almost sure convergenceof the estimation error of the time-varying parameters tothe null, provided that the variance of the noise derivingthe dynamic model of the parameters, C2(/c), is at most ofthe same order as p(k). In many practical applications,however, satisfactory parameters models can be obtainedby assuming a lower bound on C2(k). We will consider thiscase in the present Section. We will show that an upperbound on the mean-square error of the estimates can beobtained, provided that p(k) = S and the data sequence[z(')] is general enough. In the present analysis, weassume that m = 0, i.e. the data vector z( •) contains onlyprevious values of input. The parameters will be updatedaccording to the batch processing recursion of eqn. 4.

Theorem 3Consider the time-varying system described by eqns. 2 and3. Let the data sequence [z(-)3 be general enough andsatisfy

i n f ^ ™ ^max j e Jk \z(j)\-

Let the unknown parameter vector 0(k) be identified by therecursion of eqn. 4 with nk^ n and p(k) = S. Let assump-tions A1-A4 and the following conditions be satisfied:

(Cl) 0 < 8 < 2

(C2) sup- 1)

= Mw < oo and

sup c2(k) = Mv < oo

(C3) lim f*(k) = 0

then(a) the trajectory [^(/c)] always returns to the setP A fy. yty) ^ M*} M* < oo] w p l

where T is defined as in the definition of general-enoughsequence given above.

ProofSimilar to the case of sequential processing considered inthe previous Section, we can write, for large k, using condi-tions Cl , C2 and C3,

+ 52MW + Mv (15)

where Ami-n(-) is the minimum eigenvalue ofand h = 2 - 5 - 3" > 0.

The conditions onz(-) imply that [11]

and accordingly

A^O'fc) < -S5*hV{jk) + 32MW + Mv

Consider the set

<52M + M + vS3*h

then, for ijf(jk) e Ncv, we have

140 IEE PROCEEDINGS, Vol. 130, Pt. D, No. 4, JULY 1983

The lemma can be applied to conclude that [^(fc)] alwaysreturns to the set Nv wp 1. As v is arbitrary, (a) of thetheorem is proved. To prove (b), take expectations of bothsides of inequality 15 and iterate back to j t and, takinglimits, we obtain

82M, Mv

<5<5*(2 - 3)

Since £|\J/(yk)|2 is bounded, the Weirstrass-Bolzanotheorem implies the existence of the limit superior of

/fc) I2- Hence

lim sup E\ij/{jk)\:

k

32M Mv

85*(2 - 3)

Note that at the limit 8" = 0.

Remarks(i) The bound given above is simple as compared to thatgiven in Reference 5. Further, the existence of the limit ofthe mean-square error was assumed in the analysis of Ref-erence 5. In our work, this assumption is not needed aswell as the boundness of the input-output and noisesequences.(ii) The proof of theorem 3 is still valid if 8 is replaced byp(k) > 0 with lim p(k) = 8. This obviously will increase therate of convergence.

5 Computer simulations

Using the scheme of theorem 1, computer simulations werecarried out to track, in real time, the two time-varyingparameters of the following single-input/single-output first-order system:

y(k) + a(k)y(k - 1) = b(k)u(k - w(k)

where w(/c) is chosen to be white Gaussian noise sequencewith zero mean and variance of 0.7. The dynamics of theparameters variations are

B(k + 1) = f(k)6(k) + v(k)

where B'(k) = [b{k), a(/c)], and v(/c) is a zero-mean 2-dimensional white Gaussian noise sequence with covari-ance

£[v(/>'(/c)] = (0.1/k2)I3jk

We assume that b{0) = 2.0, a(0) = 0.80 and y(0) = 0.0.The simulations were carried out with data length of

750 and the initial values for b(k) and a(k) were taken aszero. The input u(k) was chosen to be a sinusoidal function

u(k) = 0.7 sin k

12

0 8

b(k)

200 400 600

Fig. 1 Real-time tracking ofa(k) and b(k) with sinusoidal input

IEE PROCEEDINGS, Vol. 130, Pt. D, No. 4, JULY 1983

and the deterministic function/(/c) is chosen to be

f(k) =1 - T I for 1 300

for k > 300

The identification scheme was carried out in this examplewith p(k) = l/k.

The simulations are repeated for the above system butwith a(k) = 0.0. The input sequence in this case was chosento be constant, i.e. u(k) = 0.7.

200 400 600

Fig. 2 Real-time tracking ofb(k) with constant input

The identification result is illustrated in Figs. 1 and 2,respectively. Satisfactory convergence and a good trackingproperty of the time-varying parameters are obtained.

6 Conclusions

We present online schemes to identify time-varyingsystems operating in a stochastic environment with par-ameters that vary in a stochastic way. Using stochasticLyapunov functions, the almost sure convergence of theidentification error to the null is proved, for the case wherethe noise deriving the parameters dynamics has summablevariance.

We show that convergence can be achieved withoutimposing the boundedness assumption on the input-outputand noise sequences. We also show that identification ofmodels of some practical systems that are obtained byassuming a lower bound on the parameters' noise covari-ance can only be obtained with bounded mean-squareerror. The bound obtained does not need the boundednessof the data. What is needed are finite second moments.

Work is now under way to minimise the bound on themean-square error of the time-varying parameters bychoosing an optimum time-varying design parameter.Also, research work to modify the proposed identificationschemes to include systems with coloured noise is currentlybeing carried out.

7 Acknowledgment

The author would like to thank Mr. T.G. Abou-El-Yazidof the Department of Design and Production Engineering,Ain-Shams University, Egypt, for carrying out the com-puter work of the numerical examples of the present paper.

J References

1 ASTROM, K.J. and EYKHOFF, P.: 'System identification a survey',Automatics, 1971, 7, pp. 123-162

141

2 EYKHOFF, P.: 'System Identification, parameter and state estima-tion' (Wiley, New York, 1974)

3 MOUSTAFA, K.A.: 'Studies on stochastic systems identifications'Ph.D. dissertation, Kyoto University, Kyoto, Japan, 1978

4 MENDEL, J.M.: 'Discrete techniques of parameter estimation'(Marcel Dekker, New York, 1973)

5 STANKOVIC, S.S.: 'On asymptotic properties of real time identifica-tion algorithms based on dynamic stochastic approximation', IEEETrans., 1978, AC-23, pp. 58-61

6 BOHLIN, T.: 'Information pattern for linear discrete time modelswith stochastic coefficients', ibid., 1970, AC-15, pp. 104-106

7 CAINES, P.: 'Stochastic adaptive control: Randomly varying par-ameters and continually disturbed control'. IFAC World Congress,Kyoto, Japan, 1981, Paper 32.2

8 SOEDA, T., YOSHIMURA, T., and KONISHI, K.: 'Parameter iden-tification of bilinear systems by using model reference adaptive tech-

niques', Trans. Soc. Instrum. and Control Eng. (Japan), 1979, 15, pp.205-212

9 AKASHI, H., MOUSTAFA, K.A., and NOSE, K.: 'On stochasticsystem identification through Liapunov functions', Int. J. Control,1977, 26, pp. 573-582

10 AKASHI, H., and MOUSTAFA, K.A.: 'Comments on and extensionsto "On stochastic system identification through Liapunov functions"',ibid., 1978, 27, pp. 313-317

11 AKASHI, H., and MOUSTAFA, K.A.: 'A stable identification struc-ture for a class of stochastic systems', IEEE Trans., 1981, AC-26, pp.717-721

12 ALBERT, A.E., and GARDNER, L.A.: 'Stochastic approximationand nonlinear regression' (MIT Press, 1967)

13 KUSHNER, H.H.: 'Stochastic stability and control' (Academic Press,New York, 1967)

142 IEE PROCEEDINGS, Vol. 130, Pt. D, No. 4, JULY 1983