Shock model in Markovian environment
Transcript of Shock model in Markovian environment
Shock Model in Markovian Environment*
Gang Li,1 Jiaowan Luo2
1 School of Mathematics and Statistics, Carleton University, 1125 Colonel By Drive, Ottawa, Ontario K1S 5B6, Canada
2 School of Mathematics, Central South University, Changsha, Hunan 410075, People’s Republic of China
Received 31 March 2004; accepted 19 December 2004DOI 10.1002/nav.20068
Published online 3 February 2005 in Wiley InterScience (www.interscience.wiley.com).
Abstract: A Markov modulated shock models is studied in this paper. In this model, both the interarrival time and the magnitudeof the shock are determined by a Markov process. The system fails whenever a shock magnitude exceeds a pre-specified level �.Nonexponential bounds of the reliability are given when the interarrival time has heavy-tailed distribution. The exponential decayof the reliability function and the asymptotic failure rate are also considered for the light-tailed case. © 2005 Wiley Periodicals, Inc.Naval Research Logistics 52: 253–260, 2005.
Keywords: shock model; semi-Markov; reliability; martingale; bounds; asymptotic
1. INTRODUCTION
A common assumption in the classical shock models isthat the shock arrival is a renewal process, the magnitude ofshocks are i.i.d. and independent of the shock arrival pro-cess. However, in many practical problems, the interarrivaltime and the magnitude are correlated to each other and notnecessarily have the same distribution. To provide morerealistic models for this kind of problems, recently, someshock models with correlated element are introduced [5, 6,8, 12, 15]. In these papers, for shock models with somecorrelation structure, the optimal replacement policies (seeFeldman [5, 6] and Zucherman [15]) and system life distri-bution (see Igaki, Sumita, and Kowada [8], and Gut andJurg [7]) are studied. Here, we are going to study a shockmodel in Markovian environment, which incorporate theinfluence of external system states on the shock magnitudesand shock interarrival times by introducing in a Markovchain. In this model, both the interarrival time and themagnitude of the shock are governed by a Markov process.The shock arrives according to a Markov renewal process,the magnitude is determined by the state of a Markovprocess, the system fails whenever the magnitude of one
shock exceeds a prespecified level. Of interest is the lifedistribution of the system. A similar model has been studiedby Igaki, Sumita, and Kowada [8], where the Laplace trans-form of the system life distribution is given under theassumption that the joint distribution of interarrival time andshock magnitude is absolutely continuous and has partialderivative. In this paper, without those conditions, we con-sider the bounds and asymptotic property of the system lifedistribution by another approach.
In studying the life distribution of an extreme shocksystem, the interarrival time distribution plays a critical role.To some extent, it dominates the property of the extremeshock system’s life distribution. While there are two typesof interarrival distributions: light-tail and heavy-tail distri-bution, most existing works merely assume light tailedinterarrival distribution, such as exponential, normal distri-bution. That doesn’t mean that heavy-tail interarrival distri-bution is not necessary in reliability. In fact, many shockarrival processes exhibit heavy-tail behavior. For example, arepairable system may have a Pareto, log-normal, or heavy-tailed Weibull interfailure distribution [3, 10, 11], andheavy-tail distribution has properties that are qualitativelydifferent from light-tailed distribution. Conclusions reachedunder light-tail assumptions may be misleading or incorrectin the presence of heavy-tailed distributions. Consequently,it is of particular significance to analysis a shock model withheavy-tailed interarrival distribution. In present paper, wewill study both light-tail and heavy-tail cases. For the light-
* This research is supported in part by NSFC under Grant No.10301036.
Correspondence to: G. Li ([email protected]); J. Luo([email protected])
© 2005 Wiley Periodicals, Inc.
tail case, some asymptotic results of the reliability functionis proved. For the heavy-tail case, some bounds of thereliability function are provided.
As application, this model can be applied to the reliabilityevaluation of repairable system, networking traffic, powersystem, dam and so on.
The rest of this paper is organized as follows. In Section2, we will describe the shock model in detail. Some upperbounds of the reliability function are derived in Section 3.Section 4 is devoted to study the exponential decay of thereliability function. The asymptotic failure rate of the sys-tem is yielded in Section 5.
2. MODEL
Consider a shock model {( Jn, Zn, Yn), n � 0, 1, . . . },where { Jn} denotes the state of the environment, Zn � 0denotes the interarrival time between the (n � 1)th and nthshock, Yn � 0 is the magnitude of the nth shock, especially,Y0 � Z0 � 0.
Denote
Sn � �k�0
n
Zn.
Assume {( Jn, Sn), n � 0, 1, . . .} is a Markov renewalprocess, where { Jn} is an irreducible Markov chain withstate space � � {1, 2, . . . , M} and transition matrix
P � �pij�M�M;
the semi-Markov kernel is
Fij�x� � Pr�Sn�1 � Sn � x, Jn�1 � j�Jn � i�.
Here, we assume Fij( x) is nonlattice. Let
pij � �Fij� – Fij���.
and
F� i�t� � 1 � Fi�t� � �j�1
M
Fij�t�, i � 1, 2, . . . , M.
The nth shock comes at time Sn, which causes a damageYn � 0. Yn depends on Jn. Given Jn � i, Yn has adistribution
PrYn � x�Jn � i � Gi�x�,
and Yn is conditionally independent of Zn given Jn.The system fails whenever the magnitude of a shock
exceeds a prespecified deterministic level � � 0.Let
p�i� � 1 � q�i� � Pr� � Y1�J1 � i,
q � min1�i�M
q�i�, q � max1�i�M
q�i�, (1)
and
T � minSn�� � Yn.
Given the initial state i, the reliability function is denoted as
Ri�t� � PrT � t�J0 � i.
Let � � (�1, �2, . . . , �M) be the steady state distributionof the semi-Markov process which is associated with {( Jn,Sn), n � 0, 1, . . . }; denote
R�t� � PrT � t � �i�1
M
�iRi�t�
as the reliability function under stationary environment. Inthe following, let Pr� denote the conditional probabilitygiven the initial distribution �, Pri denote the conditionalprobability given the initial state i. Ei and E� are thecorresponding conditional expectations.
The number of the shocks which arrive before time t isdenoted as
N�t� � maxn�Sn � t.
3. SOME BOUNDS FOR THE RELIABILITYFUNCTION
In this section, By constructing a super-Martingale, wewill give some bounds for the life distribution of the shockmodel.
First, we give a lemma about the reliability function. Inthe following, we let �k�1
f p(k) � 1, if f 1.
LEMMA 1: For the model described in previous section,the reliability function given J0 � i is
Ri�t� � Ei��k�1
N�t�
q�Jk�� (2)
254 Naval Research Logistics, Vol. 52 (2005)
and the reliability function is
R�t� � E���k�1
N�t�
q�Jk��. (3)
PROOF: Since
Ri�t� � PrT � t�J0 � i
� �n�0
� �lk�1
k�1,2. . .n
M
PrT � t, N�t� � n, Jk � lk, k
� 1, 2, . . . , n � J0 � i}
� �n�0
� �lk�1
k�1,2,. . .,n
M �k�1
n
q�lk�PrN�t� � n, Jk � lk, k
� 1, 2, . . . , n�J0 � i}
� Ei��k�1
N�t�
q�Jk��,
by the same way, (2) is proved. �
To prove the main result in this section, we need thefollowing lemma. It is shown that under some conditions,N(t), t � 0 is finite with probability 1.
LEMMA 2: Suppose Fij(0) pij, i, j, � 1, 2, . . . , M,then Pr(N(t) �) � 1.
PROOF: Denote
Gij�x� – PrZ1 � x�J0 � i, J1 � j,
then
Gij�x� �Fij�x�
pij,
so, Fij(0) pij, i, j, � 1, 2, . . . , M, this just meansGij(0) 1. Choose bij � 0 and 1 � �ij � 0 such that
1 � Gij�bij� � �ij.
Let
� � min1�i,j�M
�ij, b � min1�i,j�M
bij;
thus
1 � Gij�b� � �.
Construct an independent renewal process N(t) with inter-arrival distribution
G�x� � � 1 � �, x � b,1, x � b.
Obviously
G�x� � Gij�x�, i, j, � 1, 2, . . . , M.
Hence, there exist a version of N(t) and N(t) [13] such that
PrN�t� � N�t� � 1, t � 0.
But
PrN�t� � n � � 0, n � �t�,��t��1�1 � ��n��t�, n � �t�,
where
�t� � t
b.
Since EN(t) � (see [13]), it is easy to see that
PrN�t� � � � PrN�t� � � � 1.
This completes the proof. �
REMARK 3: Similar to the proof in last lemma, we canconstruct a renewal process to bound the Markov renewalprocess when the probability measure is changed to Pri orPr�; therefore, we also have Pri(N(t) �) � 1, i � 1,2, . . . , M and Pr�(N(t) �) � 1.
To construct a super-Martingale, we need introduce aclass of function first. This kind of function is widelyapplied in reliability study, many distribution functions be-long to this class.
DEFINITION 4: A function L : [0, �) 3 [0, �) iscalled an NBU (NWU) function if L is nonincreasing,positive, with L(0) � 1, L(�) � 0, and
255Li and Luo: Shock Model in Markovian Environment
B�x y� � ���B�x�B�y�.
The following theorem is one of our main results in thepaper. It gives the upper bound of the reliability function.
THEOREM 5: Assume that there exists an NWU func-tion L such that
�0
�
L�1�x� dFi�x� �1
q�i�
for any 1 � i � M and Fij(0) pij, i, j, � 1, 2, . . . , M,then
R�t� �L�t�
q. (4)
PROOF: First, we construct a super-martingale
U0 � 1, Un � �k�1
n
�L�1�Zk�q�Jk�1��, n � 1, 2, . . . .
Since
E�Un�1�J0, Z0, J1, Z1, . . . , Jn, Zn
� �k�1
n
�L�1�Zk�q�Jk�1��E��L�1�Zn�1�q�Jn� � Jn�
� �k�1
n
�L�1�Zk�q�Jk�1�� � Un, (5)
Un is a super-martingale with respect to {( Ji, Zi), i � 0, 1,2, . . . }.
Let
��t� � N�t� 1, t � 0.
Note that for any n � �
� ��t, � � n � � Sn�1� � � t, Sn� � � t,
�(t) is a stop time with respect to {( Ji, Zi), i � 0, 1,2, . . . ,}.
Thus by the optimal stop theory (see [4, 9], Lemma 2, and(2)),
1 � E�U0 � E�U� � E� �k�1
��t�
�L�1�Zk�q�Jk�1��
� qE��L�1�S��t�� �k�2
��t�
q�Jk�1�� � qR�t�L�1�t�,
where the first inequality follows from the NWU property offunction L and (1), the second inequality follows fromLemma (2). Then (4) follows. �
Correspondingly, we have the bounds for conditionalreliability given the initial state of the Markov chain.
THEOREM 6: Under the condition of Theorem 5, wehave
Ri�t� �q�i�Ei�L
�1�Z1��
qL�t�.
PROOF: The same notations as in the proof of Theorem5 are used here. Similarly, we can prove that (5) also holdwhen E� is changed to Ei. So Un is a super-martingale withrespect to {( Ji, Zi), i � 0, 1, 2, . . . }. By the optimal stoptheory, we have
q�i�Ei�L�1�Z1�� � EiU1 � J0 � i � EU�
� Ei��k�1
��t�
�L�1�Zk�q�Jk�1��� qEi�L�1�S��t�� �
k�2
��t�
q�Jk�1�� � qRi�t�L�1�t�.
This complete the proof. �
By condition on Z�(t), we can refine the bounds in The-orems 5 and 6.
THEOREM 7: Under the condition of Theorem 5, wehave
Ri�t� �q�i�Ei�L
�1�Z1��
q��t� (6)
and
R�t� ���t�
q, (7)
256 Naval Research Logistics, Vol. 52 (2005)
where
��1�t� � inf0�w�t1�i�M
�L�t � w���1 �w� �L�y���1dFi�y�
F� i�w�.
PROOF: Similar to the proof of Theorem 6, by theoptimal stop theory, we have
q�i�Ei�L�1�Z1�� � EiU1 � EiU�
� Ei��k�1
��t�
�L�1�Zk�q�Jk�1��� qEi��L�SN�t��L�Z��t���
�1 �k�2
��t�
q�Jk�1�� qRi�t��
�1�t�; (8)
denote W(t) � t � SN(t) and
� � J0 � i, Jm � jm, ��t� � k, W�t� � w,
m � 1, 2, . . . , k � 1.
The last inequality in (8) holds because
Ei��L�SN�t��L�Z��t����1 �
k�2
��t�
q�Jk�1� �� Ei��L�t � w�L�Zk��
�1 �m�2
k
q�jm�1� �� L�1�t � w� �
m�2
k
q�jm�1�Ei�L�Zk���1 � �
� ��1�t� �m�2
k
q�jm�1�.
This completes the proof of (6). (7) can be proved in thesame way. �
REMARK 8: For all of the three bounds in this section,the bound in Theorem 7 is the tightest; and the bound inTheorem 6 is better than that in Theorem 5. That meansmore information about the initial state or the interarrivaldistribution leads to a much tighter bound.
EXAMPLE 9: Consider the lifetime of a dam, which isaffected by earthquakes. Suppose the magnitude of an earth-
quake can be classed into two types, denoted as {1, 2}.Suppose the magnitude of the earthquake and its interarrivaltime constitute a Markov renewal process [1] with Markovkernel:
Fij�x� � pij�1 � �1 x
���1��i� , � � 0, �i � 0.
[Here, assuming the intershock time has Pareto distributionmeans that a earthquake may not happen for a long timewith a higher probability (comparing with light-tailed dis-tribution).] And, assume the dam will break down withpossibility 0 1 � qi 1 when an earthquake withmagnitude i � {1, 2} happen. Thus, this is a Markov-modulated shock model.
Suppose pi� �i; let
L1�x� � �1 x
����1���
,
where
� � sup�� 0 �1 �i
�i � ��
1
qi, i � 1, 2,
then
�0
� �1 x
��1��
dFi�x� �1 �i
�i � ��
1
qi.
It is easy to prove that L1(t) is an NWU function; then byTheorem 5
R�t� �L1�t�
minq1, q2,
by Theorem 6
Ri�t� �qi�1 �i�
��i � �� minq1, q2L1�t�, i � 1, 2,
and by Theorem 7
Ri�t� �qi�1 �i�
��i � �� minq1, q2
� �
1 �1 t
����1���
,
i � 1, 2,
where
257Li and Luo: Shock Model in Markovian Environment
� max�1, �2.
REMARK 10: Since the negative exponential function isan NWU function, all of the bounds can be a negativeexponential function as long as the moment generatingfunction exist.
3. ASYMPTOTIC PROPERTY OF THERELIABILITY
In this section, we will study the exponential decay of thereliability Ri(t). First, we construct a Markov renewal equa-tion.
LEMMA 11: The conditional reliability of the systemRi(t), i � 1, 2, . . . , M, satisfies the following equation:
Ri�t� � F� i�t� �j�1
M �0
t
Rj�t � x�q�j� dFij�x�. (9)
PROOF: Since
Pr�Z1 � t, T � t, J1 � j � J0 � i�
� �0
t
Pr�Z1 � dx, T � t, J1 � j � J0 � i�
� �0
t
Pr�T � t � J0 � i, J1 � j, Z1 � dx,�Pr�Z1 � dx,
J1 � j � J0 � i�
� �0
t
Pr�T � t � x � J0 � j�q�j�Pr�Z1 � dx, J1 � j � J0 � i�,
then
Ri�t� � Pr�Z1 � t � J0 � i� Pr�Z1 � t, T � t � J0 � i�
� �j�1
M
Pr�Z1 � t, J1 � j � J0 � i� �j�1
M
Pr�Z1 � t, T � t, J1 � j � J0 � i�
� F� i�t� �j�1
M �0
t
Rj�t � x�q�j� dFij�x�.
This complete the proof. �
Based on the renewal equation, we have following theo-rem.
THEOREM 12: Suppose there exist a � 0 such thatA � (aij)M�M has a spectral radius 1, where
aij � �0
�
exq�j� dFij�x�; (10)
then, there exist vectors v � (v1, v2, . . . , vM) and h �(h1, h2, . . . , hM)� such that
vA � v, Ah � h
and vi � 0, hi � 0, i � 1, 2, . . . , M; thus
Ri�t� hi ¥k�1
M vk �0� exF� k�x� dx
¥k,j�1M vkhj �0
� xexq�j� dFkj�x�e�t, t3 �.
PROOF: Obviously, F� i is direct Riemann integrable.Based on the renewal equation (9), this theorem is a directconclusion of the renewal theory in [2], the result is easy toget. �
EXAMPLE 13: Consider the reliability of a power sys-tem. Suppose the system has two states {1, 2} and theevolution of the states can be modeled as a semi-Markovprocess with a semi-Markov kernel [14]:
� 0 1 � �1 0.12t�e�0.12t
1 � e�0.2t 0 � .
When the state changes, a power surge will happen. Withprobability 0.22 (0.08), this may result in a failure of thesystem if the state following the change is 1 (2). Apparently,this is a Markov modulated shock model. Therefore, byTheorem 12, � 0.01446, and
A � � 0 1.18940.8408 0 � .
Choose
h � �0.7654 0.6435��, v � �0.6435 0.7654�
such that Ah � h and vA � v; then
R1�t� 1.0953e�0.01446t,
258 Naval Research Logistics, Vol. 52 (2005)
R2�t� 0.9209e�0.01446t.
4. ASYMPTOTIC FAILURE RATE OF THESYSTEM
In this section, by constructing a Markov renewal equa-tion, the asymptotic failure rate of the system is given undercertain conditions.
LEMMA 14: Suppose Fij, i, j � 1, 2, . . . , M, isabsolutely continuous and its density function is fij(t), thenthe density function ri(t) of the system lifetime T given theinitial state i is the solution of the following equation:
ri�t� � �j�1
M
p�j�fij�t� �0
t �j�1
M
rj�t � x�q�j� dFij�x�. (11)
PROOF: First we have
ri�t� � lim�30
Prt � T � t � � J0 � i
�. (12)
Since
Prt � Z1 � t �, Y1 � � � J0 � i
� �j�1
M
Prt � Z1 � t �, Y1 � �, J1 � j � J0 � i
� �j�1
M
Prt � Z1 � t � � Y1 � �, J1 � j, J0 � iPrY1
� �, J1 � j � J0 � i
� �j�1
M
Prt � Z1 � t � � J1 � j,
J0 � i}PrY1 � � � J1 � jpij
� �j�1
M Prt � Z1 � t �, J1 � j � J0 � i
pijp�j�pij
� �j�1
M
Prt � Z1 � t �, J1 � j � J0 � ip�j�
we have
Prt � T � t � � J0 � i
� Prt � Z1 � t �, Y1 � � � J0 � i �0
t
Prt � T � t �, Z1 � dx � J0 � i
� �j�1
M
Prt � Z1 � t �, J1 � j � J0 � ip�j� �0
t �j�1
M
Prt � T � t �, Z1 � dx, J1 � j � J0 � i
� �j�1
M
Prt � Z1 � t �, J1 � j � J0 � ip�j� �0
t �j�1
M
Prt � T � t � � J0 � i, J1 � j, Z1 � dxPrZ1 � dx, J1 � j � J0 � i
� �j�1
M
Prt � Z1 � t �, J1 � j � J0 � ip�j� �0
t �j�1
M
Prt � x � T � t � x � � J0 � jq�j�PrZ1 � dx, J1 � j � J0 � i
� �j�1
M
Prt � Z1 � t �, J1 � j � J0 � ip�j� �0
t �j�1
M
Prt � x � T � t � x � � J0 � jq�j� dFij�x�. (13)
259Li and Luo: Shock Model in Markovian Environment
Combining (12) and (13) completes the proof. �
The Markov renewal equation is proved in the lastlemma, then the limit property of the density function is adirect result of the renewal theory.
THEOREM 15: Suppose there exists a � 0 such thatA � (aij)M�M has a spectral radius 1, where
aij � �0
�
exq�j� dFij�x�; (14)
then there exist vectors v � (v1, v2, . . . , vM) and h � (h1,h2, . . . , hM)� such that
vA � v, Ah � h
and vi � 0, hi � 0, i � 1, 2, . . . , M. Furthermore,suppose exfij( x), i, j � 1, 2, . . . , M, is directly Riemannintegrable; then
ri�t� hi ¥k�1
M vk �0� exfk�x� dx
¥k,j�1M vkhj �0
� xexq�j� dFkj�x�e�t, t 3 �,
where
fi�x� � �j�1
M
p�j�fij�x�.
PROOF: Obviously, fi is directly Riemann integrable.Based on the renewal equation (11) and the renewal theory[2], we get the result. �
Combining Theorem 11 and 15, we have the followingcorollary:
COROLLARY 16: Under the condition of Theorem 15,we have
limt3�
ri�t�
Rt�t��
¥k�1M vk ¥j�1
M �p�j�lkj���
¥k�1M vk�¥j�1
M lkj�� � 1�, (15)
where
lkj�� � �0
�
exfkj�x� dx.
PROOF: Since
�0
�
exF� k�x� dx �¥j�1
M lkj�� � 1
,
from Theorems 11 and 15,
limt3�
ri�t�
Ri�t��
hi ¥k�1M vk �0
� exfk�x� dx
hi ¥k�1M vk �0
� exF� k�x� dx
� ¥k�1
M vk ¥j�1M �p�j�lkj���
¥k�1M vk �¥j�1
M lkj�� � 1�.
This complete the proof. �
EXAMPLE 17: Consider Example 13. By Corollary 16,we can get the asymptotic failure rate 0.0145.
REFERENCES
[1] Y. Altinok and D. Kolcak, An application of the semi-Markov model for earthquake occurrences in North Anatolia,Turkey, J Balkan Geophys Soc 2(4) (1999), 90–99.
[2] S. Asmussen, Applied probability and queues, Springer, NewYork, 2003.
[3] W.R. Blischke and D.N.P. Murthy, Reliability: Modeling,prediction, and optimization, Wiley, New York, 2000.
[4] R.J. Elliott, Stochastic calculus and applications, Springer,New York, 1982.
[5] R.M. Feldman, Optimal replacement with semi-Markovshock models, J Appl Probab 13 (1976), 108–117.
[6] R.M. Feldman, Optimal replacement with semi-Markovshock models using discounted cost, Math Oper Res 2(1977), 78–90.
[7] A. Gut and H. Jurg, Extreme shock models, Extremes 2(3)(1999), 295–307.
[8] N. Igaki, U. Sumita, and M. Kowada, Analysis of Markovrenewal shock models, J Appl Probab 32 (1995), 821–831.
[9] S. Karlin and H.M. Taylor, A first course in stochasticprocesses, Academic, San Diego, 1975.
[10] G. Last and R. Szekli, Asymptotic and monotonicity proper-ties of some repairable systems, Adv Appl Probab 30 (1998),1089–1110.
[11] N.R. Mann, R.E. Schafer, and N.D. Singpurwalla, Methodsfor statistical analysis of reliability and life data, Wiley,Toronto, 1974.
[12] M.J.M. Posner and D. Zuckerman, Semi-Markov shock mod-els with additive damage, Adv Appl Probab 18 (1986), 772–790.
[13] S. Ross, Stochastic processes, Wiley, New York, 1983.[14] J.F.L. van Casteren, M.H.J. Bollen, and M.E. Schmieg, Re-
liability assessment in electrical power systems: TheWeibull-Markov stochastic model, IEEE Trans Ind Appl 36(2000), 911–915.
[15] D. Zucherman, Optimal stopping in a semi-Markov shockmodels, J Appl Probab 15 (1978), 629–634.
260 Naval Research Logistics, Vol. 52 (2005)