logarithmic poisson

8/7/2019 logarithmic poisson

1/9

A Logarithm ic Poisson Execution Time Model for Softwa re Reliability Measurem ent

J. D. Musa and K. Okumoto

Bell Laboratories, Whippany, N. J. 07981

ABSTRACTA new software reliability model is developed that predicts

expected fai lures (and hence related reliability quantit ies) as well orbetter than exist ing software reliability models, and is simpler thanany of the models that approach it in predictive validity. The modelincorporates both execution time and calendar time components,each of which is derived. The model is evaluated, using actual data,and compared with other models.

2. EXECUTION TIME COMPONENT OF MODELA software reliability model m ay be defined in terms of a random

process {M(r),r>~O} representing the number of failures experiencedby execution** time r. Such a counting process is characterized byspecifying the distribution of M(r ), including the mean valuefunction

u(r) = E[M( r)] (1)

1. INTRODUCTIONSoftware reliability is defined as the probability of failure-free

operation of a computer program in a specified environment for aspecified time. A fa ilure is a departure of program operation fromprogram requirements. A software reliability model provides ageneral form , in terms of a random process describing failures, forcharacte rizin g software reliability or a related quanti ty as a functionof failures experienced or time. The param eter s of this function aredependent on repai r activity and possibly propert ies of the softwa reproduct or development process. The new model is the result of aneffort to find a simple model of high predictive validity.

It was clear that the new model should be based on executiontime; its superiority over calendar time has been demonstrated [1,2].Hence, the genera l form of the model, including division intoexecution and calendar time components, has been based on Musa'sexecution time model [3].

It was also noted that the f ailure is indeed the observable uponwhich the model mus t be based and not the fault or code defect; it isimpracti cal to count fault s until tim e of execution becomes verylarge.* Since the numb er of failures occurring in infinite time isdependent on th e specific history of execution of a progr am, it is bestviewed as a random variable (the number of faults could beconsidered deterministic).

The new model has been chosen as one of the Poisson type [4][the failure process is a nonhomogeneous Poisson process (NHP P)]since that type has the above property and is easy to work with.Goel and Okumoto [5] applied the NHP P in softwa re reliabilitymodeling. The execution time model of Musa [3] can be interp retedin terms of an NHPP.

Finally, it h as been observed th at reductions in failure rateresulting from repair action following early failures are often greaterbecause they tend to be the most frequently-occurring ones, and thisproperty has been incorporated in the model.* F a u l t s c a n o n l y b e d e fi n e d b y t h e i r d i s c o v e ry , e . g . , t h r o u g h f a i l u r e s ; t h e re a r e s o

m a n y p o s s i b l e i m p l e m e n t a t i o n s o f a p r o g r a m t h a t f a u l t s c a n n o t b e d e f i n e d w i t hr e s p e c t t o a s t a n d a r d p r o g r a m . B u t t h i s m e a n s t h a t t h e f a u l t a s a u n i t o f m e a s u r ei s n o t c o n s t a n t a s e x e c u t i o n p r o c e e d s , b e c a u s e t h e e x t e n t o f c o m p r e h e n s i o n o f t h eu n d e r l y i n g d e f e c t a t a f a i l u re c a n v a r y . T h e a c t u a l n u m b e r o f f a u l t s c a n o n l y b ec o u n t e d a c c u r a t e l y a s t i m e o f e x e c u ti o n b e c o m e s v e r y l a r g e a n d u n d e r s t a n d i n g o fw h a t h a p p e n e d i s re a s o n a b l y c o m p l e t e .

or the failure intensity function

M r ) = - -du(r )d r (2 )

In this section we will describe basic assum ptio ns for theexecution time component of the proposed model and derive some ofthe important quantities related to the model.2.1 Assumptions

The following assumptions will be made to specify the executiontime component of the model:Assumption 1: There is no failure observed at time r = 0, i.e.,M(0) = 0 with probability one.Assumption 2: The failure intensity will decrease exponentiallywith the expected num ber of failures experienced. If we denote by?'o and 0 the initial failure intensity and the rat e of reduction in thenormalize d failure intensity per failure, respectively, then theassumption can be written as

Mr) = Xo e -e~r) (3)

It should be pointed out that many models postulate equalreductions in the failure intensi ty as each failure is experienced andrepair attempted (e.g., Musa [3], Goel and Okumoto [5], Jelinskiand Moranda [6], Shooman [7], Moranda [8], and Schneidewind[9]). In this model, the repair of early failures reduces the failureintensity more than later ones, as expressed by Assumption 2. Itaccomplish es a similar purpose to a differential model proposed byLittlewood [10], but is clearly a different model. It is more closelyrelated to the geometric de-eutrophication model proposed byMoranda [8], in which the hazard rate of the failure interval

* * I t s h o u l d b e n o t e d t h a t r e l a t i o n s h i p s a n d t h e d e r i v a t i o n s i n S e c t i o n s 2 a n d 3 o f th i sp a p e r a p p l y t o c a l e n d a r t i m e m o d e l s a s w e l l .

0270-5257/84/0000/0230501.00 1984 IEEE230


2/9

d e c r e a s e s in a g e o m e t r i c f a s h io n . T h e l o g a r i t h m i c P o i ss o n m o d e lm a y b e v i e w e d a s a c o n t i n u o u s v e rs i o n o f t h e g e o m e t r i c m o d e l. T h ed i f f e r e n c e b e t w e e n t h e t w o m o d e l s i s i l l u s t r a t e d i n F i g . 1 .

Xo

Dk

Dk zDk 3Dk 4

/ L O G A R f T H M I C M O DE L = ~ '0 -ON

/ MORANDA MODEL = Ok

. . . . - t - - - - - - - ii II Ii II 2

i - I

t II I I _

3 4 SN U M B E R O F F A IL U R E S E X P E R I E N C E D

F i g . 1 F a i l u r e i n t e n s i t y v s. n u m b e r o f f a i l u r e s e x p e r i e n c e d f o rt h e l o g a r i t h m i c m o d e l a n d t h e M o r a n d a g e o m e t r i c D e -E u t r o p h i c a t i o n m o d e l .

Assumption 3 : F o r a s m a l l t i m e i n t e r v a l A r t h e p r o b a b i l i t i e s o f o n ea n d m o r e t h a n o n e f a i lu r e d u r i n g ( r , r + A t ] a r e

o ( A t )X ( r ) A r + o ( A t ) a n d o ( A t ) , r e sp e c ti v e ly , w h e r e - - ~ 0 a sA rA r ~ 0 . N o t e t h a t t h e p r o b a b i l i t y o f n o fa i l u r e d u r i n g (r,r + A t ] i sg i v e n b y 1 - ~ , ( r) A r + o ( A r ) . T h i s a s s u m p t i o n w i l l b e u s e d inS e c t i o n 2 . 2 t o s h o w t h a t o u r p r o p o s e d m o d e l i s a P o i s s o n - t y p e m o d e l .2 . 2 D e r i v a t io n o f I m p o r t a n t M o d e l Q u a n t i ti e s2.2.1 Mean Value Function and Failure Inten sity Function

W e w i l l fi r s t u s e a s s u m p t i o n s 1 a n d 2 t o d e r i v e f u n c t i o n a l f o r m sf o r t h e m e a n v a l u e f u n c t i o n a n d f a i l u re i n t e n s i t y f u n c ti o n .

I f w e s u b s t i t u t e ( 2 ) i n t o ( 3 ) , w e g e t t h e d i f f e r e n t i a l e q u a t i o n :

# ' ( r ) = X o e -u(~) (4 )

or

u ' ( r ) e u ( ~ ) = X o . ( 5 )

N o t i n g t h a td[e0U( ~)]

dr Oz'(r) e ~(~) , (6)

w e g e t f r o m ( 5 )

d [ e 0 U ( ~ ) ]dr XoO. (7 )

I n t e g r a t i n g ( 7 ) y i e l d s

e au(~) = 3,oOr + C , ( 8 )

w h e r e C i s th e c o n s t a n t o f i n t e g ra t i o n . S i n c e u ( 0 ) = 0 f r o mA s s u m p t i o n 1 , w e g e t C = I . H e n c e , f ro m ( 8 ) w e o b t a i n t h e m e a nv a l u e f u n c t i o n a s

1z ( r ) = - 7- I n (hoOt + 1) , (9)O

w h i c h i s a l o g a r i t h m i c f u n c t i o n o f r.F u r t h e r m o r e , f r o m t h e d e f i n i t i o n g i v e n i n ( 2 ) w e g e t th e f a i l u r e

i n t e n s i t y f u n c t i o n a s

X( r ) = X0 / (Xo0 r + 1 ) , ( 1 0 )

w h i c h i s a n i n v e r s e l i n e a r f u n c t i o n o f z .

2.2.2 Failures ExperiencedT h e f a i lu r e s ex p e r i e n c e d b y t i m e z, M ( r ) , i s a r a n d o m q u a n t i t y .

U s i n g a s s u m p t i o n s 1 a n d 3 , i t c a n b e e a s i l y s h o w n t h a t t h ep r o b a b i l i t y t h a t M ( r ) h a s t h e v a l u e m i s g i v e n b y

P r { M ( r ) = m } = { t t( r )} m e - u ( ' ). ( 1 1 )m !

T h i s i s a P o i s s o n d i st r i b u t i o n w i t h a m e a n a n d v a r i a n c e o f # ( r ) ,w h i c h i s g i v e n i n ( 9 ) .

S u p p o s e t h a t m e f a i l u r e s h a v e b e e n o b s e r v e d d u r i n g ( 0 , r e ] .S i n c e t h e P o i s s o n p r o c es s { M ( r ) , r > ~ 0} h a s i n d e p e n d e n t i n c r e m e n t s ,t h e c o n d i t i o n a l d i s t ri b u t i o n o f M ( r ) g i v e n M ( r e) = me f or r > r e i st h e d i s t r i b u t i o n o f t h e n u m b e r o f f a i l u r e s d u r i n g ( r e , r] , i . e ., fo rm > ~ m e ,

P r { M ( r ) = m IM ( r ~ ) = m ~ } = Pr {M (r)-M ('re) = m-me}

{#(r)-#(re))m-'n" e - lu ( r ) - u ( f ') } . ( 1 2 )(m-me)!

2.2.3 Failure Time and Time Between FailuresT h e e x p r e s s i o n s d e r i v e d i n S e c t i o n 2 . 2 . 2 w i l l b e u s e d i n t h i s

s e c t io n t o s t u d y t h e b e h a v i o r o f r a n d o m q u a n t i t i e s s u c h a s f a i l u r et i m e s a n d i n t e r v a l s o f t i m e b e t w e e n f a i l u re s f o r th e m o d e l . T h e s eq u a n t i t i e s w i l l h e l p a p r o j e c t m a n a g e r t o p r e d i c t t h e t i m e i t t a k e s toe x p e r i e n c e a c e r t a i n n u m b e r o f f a i l u r e s a n d t h e p r o b a b i l i t y o ff a i l u r e -f r e e o p e r a ti o n d u r i n g a c e r t a i n a m o u n t o f t i m e ( i . e. ,r e l i a b i l i t y ) .

L e t Ti(i = 1 ,2 , - - . ) b e a r a n d o m v a r i a b l e r e p r e s e n t i n g t h e i - t hf a i l u r e i n t e r v a l a n d d e f i n e Ti(i = 1 , 2 , ' ) a s a r a n d o m v a r i a b l er e p r e s e n t i n g t i m e t o t h e i - t h f a i l u r e , i . e . ,

j--1

w h e r e T o = 0 .

2 3 1


3/9

O b s e r v e t h a t t h e e v e n ts E l . " T h e r e a r e a t l e a s t i f a i l u r e se x p e r i e n c e d b y t i m e r , a n d E 2 : T i m e t o t h e i - t h f a i l u r e i s a t le a s tr , a r e e q u i v a l e n t , i . e . ,

{M (r ) >/ i} ~=' {7",- ~< r} . (14 )

H e n c e , t h e c . d . f , o f T i c a n b e o b t a i n e d f r o m ( 1 1 ) a n d ( 1 4 ) a s

P r { T i ~ < r } = P r { M ( r ) > / i } = ~ P r { M ( r ) = j }j - - i

= ~ {u( r ) J J _ . ( ~ >j. i j----T--~ " ( 1 5 )

N o t e t h a t t h i s i s t h e d i s t r i b u t i o n o f t i m e t o r e m o v e t h e f i r st if a i l u r e s .

S i m i l a r l y , f r o m ( 1 2 ) a n d ( 1 4 ) t h e c o n d i t i o n a l c . d . f , o f T i g i v e nM ( r e ) = m e, w h e r e i > m e , i s d e r iv e d a s

P r { T / ~ < r i M ( r e ) = m e} = P r { M ( r ) >~ i l M ( r e ) = m e }

ffi ~ Pr {M ( r ) - M ( r e ) = j - - m e }j - - i

= ~ { # ( r ) - - # ( r e ) } J - m " e -{z(r)-z(r')} r >1 r e. ( 1 6 )j = i ( j - m e ) !

2 . 2 . 4 R e l i a b i l it y a n d H a z a r d R a t eT h e c ond i t i ona l r e l i a b i l i t y o f 7",-' on

T i - 1 ~ r i _ 1 c a n b e o b t a i n e d , u s i n g ( 1 6 ) , a s

R ( r ; I r i - , ) = P r {Ti' > - ; I r , = r , _ , )

t h e l a s t f a i l u r e t i m e

= 1 - P r { T i ~< rAM(ri-,) = i - 1 }f f i 1 - ~ {# (ri) -- #( ri- O} J- i+ l -{u(, ,)-u(, ,_ ,))

j= i ( j - / + l ) ! - e ( 1 7 )

N o t e t h a t t h e s e c o n d t e r m o f ( 1 7 ) i s t h e s u m o f P o i s s o n p r o b a b i li t ie se x c e p t f o r o n e t e rm . H e n c e , w e g e t

R(r~lri-O ffi e - 1 " % - ' + * ) - u ( " - ') } (18)

F u r t h e r , s u b s t i t u t i n g ( 9 ) i n t o ( 1 8 ) y i e l d s

t r i+ r - l ) + loOri-i + 1 1 }11oR ( r ; l r , - , ) = _ X o 0 ; - - r T - _ _ - . ( 1 9 )

T h e r e f o r e , t h e r e l ia b i li t y f o r t h e m o d e l d e p e n d s o n t h e l a s t f a i l u ret i m e r i - 1.

I f w e ta k e t h e n e g a t i v e o f t h e d e r i v a ti v e o f ( 1 8 ) w i t h r e s p e c t t or i , w e g e t t h e c o n d i t i o n a l d e n s i t y f u n c t i o n o f ri , i .e.,

f ( r ~ l r ; - 0 = x (r ~ + r i - 1 ) e - { " ( r ; + r , _ , ) - ~(r,_.)}, (20)

a n d h e n c e , t h e h a z a r d r a t e i s g i v e n b yR ( r ; l r , - 0z ( r ; I r , - , ) X ( r ; + r i - O . ( 2 1 )f(r~lr,-,)

N o t e t h a t t h e h a z a r d r a t e f o r t h e m o d e l i s t h e s a m e a s t h e f a i l u r ei n t e n s i t y f u n c t i o n . F u r t h e r , s u b s t i t u t i n g ( 1 0 ) i n t o ( 2 1 ) y i e l d s

z ( r i l r i _ O = X ( 2 2 )Xo0(r i + r i - O + 1

3 . M A X I M U M L I K E L I H O O D E S T I M A T I O N O F P A R A M E T E R SI n t h i s s e c t io n w e w i ll d e v e lo p t h e m e t h o d o f m a x i m u m l i ke l ih o o d

f o r e s t i m a t i n g t h e u n k n o w n p a r a m e t e r s X 0 a n d 0. W e w i ll t a k e a na p p r o a c h t h a t e s t i m a t e s t h e p r o d u c t ~ b = X 00 b y u s i n g a c o n d i t i o n a lj o i n t d e n s i t y f u n c t i o n a s t h e li k e l i h o o d f u n c t i o n . T h e n 0 i sd e t e r m i n e d f r o m t h e m e a n v a l u e f u n c t io n . I t c a n b e e as i ly s h o w nt h a t t h e f o r e g o i n g a p p r o a c h i s e q u i v a l e n t to t h e m e t h o d o f m a x i m u ml i k e l i h o o d e s t i m a t i o n b a s e d o n a n u n c o n d i t i o n a l j o i n t d e n s i t yf u n c t i o n . T h e a p p r o a c h s i m p l if i es t h e e s t im a t i o n p r o c e s s ( o n l y o n ep a r a m e t e r i n v o l v e d ) a n d h e n c e i t i s m o r e e ff i c ie n t c o m p u t a t i o n a l l y .

W e w i l l c o n s i d e r t w o t y p e s o f f a i l u r e d a t a ; f a i l u r e i n t e r v a l s( S e c t i o n 3 . 1 ) o r n u m b e r s o f f a i lu r e s p e r i n t e rv a l ( S e c t i o n 3 . 2 ) .3 . 1 E s t i m a t i o n B a s e d o n F a i lu r e I n t er v a l s

S u p p o s e t h a t e s t i m a t i o n i s p e r f o r m e d a t a s p e c i fi e d t i m e re .T h e n , t h e n u m b e r o f f a i l u re s e x p e r i e n ce d i n ( 0 , r e ] w i l l b e a r a n d o mv a r i a b l e . I n t h i s c a s e , w e c a n u s e a c o n d i t i o n a l j o i n t d e n s i t y f u n c t i o na s t h e l ik e l ih o o d f u n c t i o n . A s s u m i n g m f a i l u re s h a v e b e e n o b s e r v e db y e x e c u t i o n t i m e r e a n d n o t i n g t h a t T , ,, + l is d e p e n d e n t o n l y o n T ms i n c e t h e { T i , i = 1 , 2 , . . } f o r m a P o i s s o n p r o c e s s , w e g e t t h e j o i n td e n s i t y f u n c t i o n o f { T b " " ,Tin} c o n d i t i o n a l o n M ( r e ) = m e a s

f ( r l , " " " , r m ) P r { T m + l > r e l T m = r m } ( 2 3 )g (r 1 , " " " , T in I ra ) = ~ Pr {M (re) m }

w h e r e f ( r b ' ' " , r m ) r e p r e s e n t s t h e u n c o n d i t i o n a l j o i n t d e n s i t yf u n c t i o n o f { T b . . . , T m } .

U s i n g ( 2 0 ) , w e o b t a i n t h e u n c o n d i t i o n a l j o in t d e n s i t y f u n c t i o n a s

f ( r x . . . . . r m ) = H f ( r i l r i - 1 )i- 1

ffi ~I X (r i) e -(u(r')-u(r'-')}i- I

m= e - " ( T ' ) I I X ( r / ) . ( 2 4 )i - I

F r o m ( 1 8 ) w e a l s o g e t

P r { T . + l > r . l T m ffi r , .} = e -{~(~' )-u(T ')}. ( 2 5 )

2 3 2


4/9

T h e r e f o r e , i f w e s u b s t i t u t e ( 1 l ) , ( 2 4 ) a n d ( 2 5 ) i n t o ( 2 3 ) , w e g e tt h e c o n d i t i o n a l j o i n t d e n s i t y f u n c t i o n a s

g ( r l , ' ' ' , r m Im ) = m ! i I l X ( T i ) h . t ( T e ) , ( 2 6 )

N o t e t h a t ( 2 6 ) i s a p p l i c a b l e t o a n y o t h e r P o i s s o n p r o c e s s e s . A l s o ,n o t e t h a t w h e n a j o i n t d e n s i ty f u n c t i o n o f r a n d o m v a r i a b l esT ~ , - . . , T , , h a s t he f o r m o f ( 2 6 ) , th e r a n d o m v a r ia b le s a r e o r d e rs t a t is t i c s f r o m t h e p . d .f , h ( r ) / # ( r e ) . I n o t h e r w o r d s , r a n d o m l yo r d e r e d f a i l u r e t i m e s a r e i .i .d , f r o m t h e a b o v e p . d . f . .

F o r t h e p r o p o s e d m o d e l w e s u b s t i t u t e ( 9 ) a n d ( 1 0 ) i n t o ( 2 6 ) :

g(r , , "" ' ,*mlm)= m ! H X o0i - I ( h o 0 r i + l ) l n ( ~ , 0 0 r e + l ) ' ( 2 7 )w h i c h m a y b e u s e d a s t h e l i k el ih o o d f u n c t i o n f o r e s t im a t i n g t h ep a r a m e t e r ( = ~ ,o 0 ) .

A n e s t i m a t e o f ~b c a n b e f o u n d b y m a x i m i z i n g t h e l o g - l i k e l i h o o d( l o g a r i t h m o f t h e l i k e l ih o o d ) , i .e .,

L = I n g ( r b ' " , r m l m )

= I n ( m ! ) + m ln q ~ - ~ l n ( q b r i + l ) - m l n [ l n ( ~ b r e + l ) ] . ( 2 8 )i - I

T a k i n g t h e d e r i v a t i v e o f L w i t h r e s p e c t t o 4~ a n d s e t t i n g i t e q u a l t oz e r o , w e g e t

O L m ~ T m TeO~b ~b i -I ~b~"7 +l ( ~ b r e + l ) l n ( O r e + l ) = O . (29 )

S i n c e t h e a b o v e e q u a t i o n i s n o n l i n e a r , w e c a n n o t f i n d a n a n a l y t i c a ls o l u t i o n b u t m u s t o b t a i n i t n u m e r i c a l l y .

N o t e t h a t ( 2 9 ) d o e s n o t g i v e e s t i m a t e s o f h o a n d 0 s e p a r at e l y . I no r d e r t o o b t a i n t h e m w e u s e t h e c o n d i t io n t h a t m f a i lu r e s h a v e b e e no b s e r v e d b y ti m e r e H e n c e , t h e m e a n v a l u e f u n c ti o n a t "r m a y b ec h o s e n t o b e m , i . e . ,

U ( r e ) = m . ( 3 0 )

S u b s t i t u t i n g ( 9 ) i n t o ( 3 0 ) , w e g e t

1 l n ( X o O r e + l ) = m . ( 3 1 )0T h e r e f o r e , t h e e s t i m a t e 0 c a n b e f o u n d b y s u b s t i t u t i n g ~ i n to ( 3 1 ) a s

S i n c e q~ = boO , we g e t

= ~ I n ( a r e + l ) . ( 3 2 )m

~ o = $ / ' 0 . ( 3 3 )T h e e s t i m a t i o n m e t h o d o f t h i s s e c t i o n c a n b e a p p l i e d t o t h e c a s e

w h e n e s t i m a t i o n i s m a d e a t t h e t i m e o f t h e m - t h f a i lu r e b y s e t t i n gT e ~ T m .

3 . 2 E s t i m a t i o n B a s e d o n N u m b e r s o f F a i l u re s p e r I n t e rv a lS u p p o s e t h a t a n o b s e r v a t i o n i n t e r v a l ( 0 , X p ] i s p a r t i t i o n e d i n t o a

s e t o f p d i s j o i n t s u b i n t e r v a l s ( O , x l ] , ( x l , x 2 ] , . . ., ( x p - l , x p ] a n d t h en u m b e r o f f a i l u r e s in e a c h s u b i n t e r v a l i s r e c o r d e d . L e ty t ( l = l , 2 , , p ) b e t h e n u m b e r o f f ai l u r es i n ( 0 , x t ] . W e w i l l u s et h e c o n d i t i o n a l j o i n t d e n s i t y f u n c t i o n t o d e v e l o p t h e m e t h o d o fm a x i m u m l ik e l ih o o d f o r e s ti m a t i n g t h e u n k n o w n p a r a m e t e r s ~ o a n d 0f r o m t h e a v a i l a b l e d a t a Y l , Y 2 , " " " , Yp .

T h e j o i n t d e n s i t y f u n c t i o n o f Y / ' s c a n b e d e r iv e d a s f o l l o w s ,n o t i n g t h a t t h e Y e ' s f o r m a P o i s s o n p r o c e s s :

Pf ( Y l , ' ' ' , Y p ) = I I P r { M ( x t ) = Y t }1 - - 1

p= H P r { M ( x t ) = y l lM ( x t - l ) = Y H } P r { M ( O ) = O } , ( 3 4 )I - - 1

w h e r e x o = 0 , Y o = 0 , a n d P r { M ( 0 ) = 0 } = 1 . S u b s t i t u t i n g ( 1 2 ) i n to( 3 4 ) y i e l d s

P { u ( x / ) - # ( x / - 0 } y ' e-{U


5/9

L = In g ( Y t , " " " , Y p [ Y p )

= In(yp!) - ~ In yl + ~ y t In {ln(~bx/+l)I--1 l--1

- I n ( ~ b X l _ l - F 1 ) } - - y p I n { l n ( ~ b x p + l ) } , ( 4 1 )

wher e (38) was applied to the last term. Taking the derivative of Lwith respect to and setting it equal to zero, we get

O~ l - -1

X I X I - I( p x t + l c ~ x t + l

In (q~xt+l) - In (~X /- l+l )

y p X p(~bxp+ ) ln (~bxp+ 1) = 0 . ( 4 2 )

Since the above equation is nonlinear, we cannot find an analyticsolution but must obtain it numerically.

Using the same approach employed in Section 3.1, estimates of #and Xo for given ~ can be found as

= 1-L ln(~xp+l) ( 4 3 )Y pand

~ o = ~ 1 0 , ( 4 4 )respectively. It can be easily shown that the above approach isequivalent to the method of maxim um likelihood based on theunconditional joint density function.4. EVALUATION OF MODEL USING ACTUAL DATA

In this section we will use actual failure data to evaluate thecapability of the model to predict future failure behavior. Thepredictive validity of the model will be compare d with that of otherpublished models.

The failure data used is composed of 15 sets of data on a varietyof software systems (such as real time command and control, realtime commercial, military, and space systems) with system sizesranging from small (5.7 K object instructions) to large (2.4 Mobject instructions). The data sets were all taken during system test(except for one taken during subsystem test). Consult [1] fordetailed descriptions of the data source and syste m characteri stics.

Predictive validity is the capability of the model to predict futurefailure behavior during either the test or the operational phases frompresent and past failure behavior in the respective phase. Altho ughvarious met hods of evaluating predictive validity may be employed,we will take a relative error app roach based on the num ber offailures experienced, since it is especially practical to use. If amodel is found to have the best predictive validity based on failuresexperienced, it will also yield the bes t predictions of other reliabilityquantities.

Assume that q failures have been observed by the time rq at theend of test. The failure data up to t i m e "re(~7"q) is used to makemaximum likelihood est imates of the model paramet ers. The

number of failures by rq can be predicted by substituting theestimates of the parameters in the mean value function to obtain/~(7-q), which is compared with the actually observed nu mber q. Thiswill be repeated for various values of re.

The predictive validity can be checked visually by plotting therelative error { ( ~ ( ' r q ) - q ) / q } against the normalized execution time7"e/ 'r q, The error will approach zero as re approaches r q . If thepoints are positive (negative), the model tends to overes timate(underestimate). Numbe rs closer to zero imply more accurateprediction and hence the better model.The use of normalizat ion enables one to overlay relative erro rcurves obtained from different failure data sets. For an overallconclusion as to the relative predictive validity of models, we maycompare plots of the medians (taken with respect to the various datasets). The model which yields the curve that is the closest to zerowill be considered superior.

The above procedure for evaluating predictive validity of thelogari thmic Poisson model was applied to the 15 sets of failure data.Estimates of the model param eter s Xo and 0 were based on thefailure data up to execution time values of re that are 20(5)100% ofr q . The estimates Xo and 0 were then substituted into the meanvalue function given in (9) to predict the num ber of failures by rq.The relative error curves for all of the 15 failure data sets wereoverlaid and shown in Fig. 2. As can be seen, the model seems topredict the future behavior very well; the error curves are, ingeneral, with in + 10% when prediction is made after 50% of rq.Furthermore, there is no specific pattern such as overestimation orunderestimation (this can be better seen in the median plot shown inFig. 3).

Logarithmic Poisson Hodel

1

0.5

Y

0

-0,5 20 4~ 60 100NermaF~zed Execution T~me (%)

Fig. 2 Relative error curves for logarithmic Poisson model basedon 15 failure data sets.

The logarithmic Poisson model will now be compared with othermodels. In order to provide an efficient basis for comparison,softw are reliability models have been classified in term s of fivedifferent attributes (see [4]):

a. time domain - calendar tim e or execution (CPU or processor)time,

23 4


6/9

b . c a t e g o r y - t h e n u m b e r o f f a il u r e s t h a t c a n b e e x p e r i e n c e d ini n f i n i te t i m e is f ini te o r infinite,

c . t y p e - t h e f a i l u r e q u a n t i t y d i s t r i b u t i o n ,d . c l a s s ( f i n i t e f a i l u r e s c a t e g o r y o n l y ) - f u n c t i o n a l f o r m o f t h ef a i l u r e i n t e n s it y i n t e r m s o f t i m e ,e . f a m i l y ( i n f i n i te f a i lu r e s c a t e g o r y o n l y ) - f u n c t i o n a l f o r m o f t h ef a i l u r e i n t e n s i t y i n t e r m s o f t h e e x p e c t e d v a l u e o f f a i l u r e s

e x p e r i e n c e d .T a b l e I i l l u s t r a t e s t h e c l a s s i f i c a t i o n s c h e m e w i t h r e s p e c t t o t h e l a s tf o u r a t t r i b u t e s ( i t is id e n t i c a l f o r b o t h k i n d s o f t i m e ) a n d n o t e sw h e r e m o s t o f t h e p u b l is h e d m o d e l s f it in i t . T a b l e I 1 s u m m a r i z e st h e f u n c t i o n a l r e l a t io n s h i p s o f t h e f a i l u re i n t e n s i t y o f v a r io u s m o d e l sw i t h r e s p e c t t o ( e x e c u t i on ) t i m e a n d t h e e x p e c t e d n u m b e r o f f a i l u r e se x p e r i e n c e d ( s e e [ 1 ] f o r d e t a i l e d d e r i v a t i o n s ) .

W e w i ll m a k e c o m p a r i s o n s u s i n g th e f o l l o w in g s e v e n m o d e lg r o u p s ( c l a s s e s o r f a m i l i e s ) , w h i c h i n c l u d e m o s t p u b l i s h e d m o d e l s :e x p o n e n t i a l c l a s s , W e i b u l l c l a s s , P a r e t o c l a s s , g e o m e t r i c f a m i l y ,i n v e r s e l i n e a r f a m i l y , i n v e r s e p o l y n o m i a l ( 2 n d d e g r e e o n l y ) f a m i l y ,a n d p o w e r f a m i l y . T h e l o g a r i t h m i c P o i s s o n m o d e l i s a m e m b e r o ft h e g e o m e t r i c f a m i l y . W e d o n o t c o n s i d e r d i f f e r e n t t y p e s b e c a u s et h e m e a n v a l u e f u n c t i o n s o f t h e m o d e l s a r e i n d e p e n d e n t o f t yp e , a n dt h e m e a n v a l u e f u n c t i o n i s t h e p r i m a r y d e t e r m i n a n t o f t h e m o d e lc h a r a c t e r i s t i c s .

T h e r e l a t i v e e r r o r a p p r o a c h f o r e v a l u a t i n g p r e d i c t i v e v a l i d i t y w a sa p p l ie d f o r e a c h o f t h e m o d e l g r o u p s , u s i n g th e s a m e s e t s o f f a i lu r ed a t a . N o t e t h a t t h e e s t i m a t i o n m e t h o d d i s c u s s e d i n S e c t i o n 3 w a sd e s c r i b e d in t e r m s o f t h e g e n e r a l f o rm s o f ~ ,( r) a n d # ( r ) . T h e r e f o r e ,i t c a n b e e a s i l y p a r t i c u l a r i z e d f o r e a c h m o d e l g r o u p . T h e f o r e g o i n ga p p r o a c h r e p r e s e n t s a n e x a c t c o m p a r i s o n f o r m o s t o f t h e m o d e l s . I t

T a b l e 1 . S o f t w a r e r e l i a b i l i t y m o d e l c la s s i f i c a t i o n s c h e m e .

F i n i te F a i l u r e s C a t e g o r y M o d e l s

T y p eC l a s s P o i ss o n B i n o m i a l O t h e r T y p e s

M u s a e x e c u t i o n t i m e [ 3 ] J e l i n s k i - M o r a n d a [ 6 ]x p o n e n t i a l

G o e I - O k um o t o N H P P [ 5]M o r a n d a g e o m e t r ic

P o i s s o n [ 8 ]S c h n e i d e w i n d [ 9 ]

S h o o m a n [ 7 ]

L i t t l e w o o d - V e r r a l lg e n e r a l w i t hr a t i o n a l i f ( i )s u g g e s t e d b y M u s a [ 1 2 ]

K e i l l e r - L i t t l e w o o d [ 1 3 ]G o e l - O k u m o t o

i m p e r f e c t d e b u g g i n g [ 1 7 ]

W e i b u l l W a g o n e r [ 1 5]S c h i c k - W o l v e r t o n [ 1 6 ]P a r e t o L i t t l e w o o d

d i f fere n t ia l [ 10]

I n f i n it e F a i l u r e s C a t e g o r y M o d e l s

T y p eF a m i l y T 1 T 2 T 3 P o i s so nG e o m e t r i c M o r a n d a g e o m e t ri c M u s a - O k u m o t o

D e - e u t r o p h i c a t i o n [ 81 ( t h i s p a p e r )I n v e r s e L i n e a r L i t t le w o o d - V e r r a l l

g e n e r a l w i t h i f ( i )l inea r [ 11 ]I n v e r s e P o l y n o m i a l L i t t l e w o o d - V e r r a l l

g e n e r a l w i t h i f ( i )p o l y n o m i a l [ 1 1 ]( 2 n d d e g r e e )P o w e r C r o w [ 1 4 ]

2 3 5


7/9

T a b l e I I . F u n c t i o n a l r e l a t i o n s h p s f o r f a i l u r e in t e n s i t y w i t h r e s p e c t t o t i m e t a n d e x p e c t e dnu mb er o f f a i lu re s ex per ie nce d #(~Oo:Ol, to2 ,co3 ,@o,@h2 re rea l ) .

F i n i te F a i l u r e s C a t e g o r y ( A l l T y p e s )C la ss X( t ) X(#)

E x p o n e n t i a l Oo - ' ' ' 0 o ( 0 1 - # )- - - - %W ei bu ll o~ot,~, i e o,,t o{_ln(l_ /~/~bt)}o~(~bl_/~)

P a r e t o C O O ( W l+ t ) -~2 $ O ( ~ b l - - i . t) ,

I n f i n i t e F a i l u r e s C a t e g o r y ( A l l T y p e s )F a m i l y X ( t )

G eo m et ri c Oo(tOl+t) I 4,o~b~'Inv ers e Li ne ar wo(Wl+t) - t /2 1/(4~0+(;b1#)

{ 3 }6% 3~//t + , , / t 2 + w l + -~/t - - ~/t2+c~, I/(~bo+~b,/fl)I n v e r s e P o l y n o m i a l( 2 n d D e g r e e )Po w er ~OotO ''-I 0o~,

r e p r e s e n t s a n a p p r o x i m a t e c o m p a r i s o n f o r t h e L i t t l e w o o d - V e r r a l lm o d e l , i n t h a t a d i f f e r e n t i n f e r e n c e p r o c e d u r e i s u s e d ( m a x i m u ml i k el i ho o d r a t h e r t h a n B a y e s i a n ) .

P l o t s o f t h e m e d i a n e r r o r c u r v e s f o r t h e m o d e l g r o u p s a r e s h o w ni n F i g . 3 . I t c a n b e o b s e r v e d t h a t e x p o n e n t i a l , P a r e t o , a n d W e i b u l lc l a s s e s t e n d t o u n d e r e s t i m a t e w h e r e a s i n v e r s e li n e a r a n d p o w e rf a m i l i e s t e n d t o o v e r e s t i m a t e . T h e g e o m e t r i c a n d i n v e r s e p o l y n o m i a lf a m i l i e s o n t h e w h o l e y i e l d t h e b e s t p r e d i c t i o n . H o w e v e r , t h e i n v e r s ep o l y n o m i a l f a m i l y t e n d s t o b e b i a s e d t o t h e o v e r e s t i m a t i o n s i d e ,

e s p e c i a l l y w h e n p r e d i c t i o n i s m a d e a f t e r 6 0 % o f ~ - q. T h i s p a t t e r n f o rt h e i n v e r s e p o l y n o m i a l f a m i l y w a s a l s o c o n f i r m e d b y e x a m i n i n g t h eu p p e r a n d l o w e r q u a r t i l e c u rv e s . N o t e t h a t t h e i n v e r s e p o l y n o m i a lf a m i l y h a s v e r y c o m p l i c a t e d e x p r e s s i o n s fo r X ( r ) a n d t ~ (r ) a n d h e n c ei t is m u c h l e s s p r a c t i c a l i n u s e . T h e r e f o r e , i t i s c o n c l u d e d t h a t t h eg e o m e t r i c f a m i l y i s s u p e r i o r t o t h e o t h e r s o f t w a r e r e l i a b i l i t y m o d e lg r o u p s in p r e d i c t i v e v a l i d i t y a n d p r a c t i c a b i l i t y . S i n c e t h el o g a r i t h m i c P o i s s o n e x e c u t i o n t i m e m o d e l i s a m e m b e r o f t h i s g r o u p ,i t a p p e a r s t o b e t h e m o d e l o f c h oi c e .

Predictive Validity

02

-g

/ ~ ~ / " , . / / " o : E ~ ,o a?etoz ~ : W e i b u l I" / ~ / ~ x : _ I n v e r s e Linear/ voo=p:nOv~:~ee.: nmi 1 ,oo

- O ' 3 z O 6 ' ONormalized xecution ime X)F i g . 3 M e d i a n c u r v e s o f r e l a t i v e e r r o r f o r s e v e n P o i s s o n - t y p e m o d e l s .

2 3 6


8/9

5. CALENDAR TIME COMPONENT OF MODELWe shall now relate execution t ime r to calendar time t. The

calendar t ime component of the model is most practically applied tothe system test phase of a project. We will use the approach takenby Musa [3].5 . 1 A s s u m p t i o n s

The following assumptions will be made to specify the calendartime component of the model:As s u m p t io n 4: The pace of testing at any time is constrained byone of three limiting resources: failure-identification (tes t team )personnel (1), failure-correction (original designer) personnel (F ), orcomputer time (C).

In most projects during test, there wi ll be from one to threeperiods, each characte rized by a different limiting resource.Typically, one identifies a large n umber of failures s eparated byshort time intervals at the sta rt of test, and testing must be stoppedfrom time to time in order to let the people who are fixing thefailures keep up with the load. As testing progresses, the intervalsbetween failures become longer and longer and the debuggers are nolonger fully loaded, but the tes t team becomes the bottleneck.Finally, at even longer failure intervals, the capacity of thecomputing facilities is limiting.

Let d t t / d r , d t r / d r , and d t c / d r represent the instantaneouscalendar time to execution t ime ratios that result from the effects ofeach of the resource constraints taken alone, respectively.. Then, theassumption can be written as

d t t d t r d t c td_t_t = ma xl -~ r 'd r d r ' d r (45)

As s u m p t io n 5: The rate of resource expenditures with respect todXkexecution time ~ can be approximated by:

du(r)dx k --__ Ok + #k k = I ,F ,C , (46)d r ~ '

where Ok is an execution time coefficient of resource expenditure anduk is a failure coefficient of resource expenditure. For specificresources (k = I , F , or C ), eit her 0k or #k can be zero. Note tha t a

REFERENCES[1] J. D. Musa, K. Okumoto, "A comparison of time domains for

software reliability models," scheduled for publication inJo u r n a l o f S ys t em s a n d S o f tw a r e .[2] H. Hecht, "Allocation of resources for software reliability,"P r o c . C O M P C O N , Fall 1981, pp. 74-82.[3] J. D. Musa, "A theory of software reliability and its

application," 1E EE Transact ions on Sof tw are Engineer ing ,SE- I (3), Septembe r 1975, pp. 312-327.[4] J. D. Musa, K. Okumoto, "Software reliability models:

concepts, classification, comparisons , and practice", Proc.E lec t r o n i c S ys t em s E f f e c t i ven es s a n d L i f e C yc le C o s t in gConference, Norwich, U. K., July 19-31, 1982, N A T O A S ISer ies , Vol. F3, (Ed: J. W. Skwirzynski) Springer-Verlag,Heidelberg, 1983, pp. 395-424.

[5] A. L. Goel, K. Okumoto, "Time-dependent error detectionrate model for softw are reliability and other perform ancemeasures", I E E E Tr a n s . R e l . , R-28(3), August 1979, pp.206-211.

resource expenditure represen ts a resource applied for a time period(e.g., person hours).As s u m p t io n 6: The quantities of the available resources areconstant for the remainder of the test period. The maximumutilization of each of the available resources is also constant.Therefore, if we denote by Pk and p k ( k = I , F , C ) the fixed availablequant ity and the utilization factor, respectively, of resource k, thenthe effective available quanti ty of resource k is p k P k ,5 . 2 D e r i v a t i o n

Using the above assumptions, we will derive a relationship ofcalendar time and execution time for the logarithmic model.

Substituting (10) into (46), we obtain the rate of resourceexpenditure as

~kdX k ~ Ok + #k , (47)d z ~oOrh"1where k = I , F , or C. Since the effective available re source is P k P k(from Assumption 6), the rate of the calendar time with respect toexecution time associated with each resou rce is given by

d t k d x k / P k P kd r d r

{ ? 0 0 + + , )1 , O k + u kP kP k (48)Therefore, from assumption 4 we can obtain the instantaneous

calendar t ime to execution time ratio as

d t ~ d t k ]d--; - m axlT f , (49)

where k may be /, F, or C. Consult M usa [3] for detaileddiscussions on how to determine the parameter values.Ackn o w led g em en t

The auth ors are indebted to A. Iannino for his helpful comm entsand suggestions.

[6] Z. Jelinski, P. B. Mora nda, "Software reliability research",S ta t i s t i ca l C o m p u ter P er fo r m a n ce E va lu a t io n , W .Freiberge r, Ed., New York: Academic, 1972, pp. 465-484.

[7] M. Sbooman, "Probabilistic models for software reliabilityprediction", S ta t i s t i ca l C o m p u ter P er fo r m a n ce E va lu a t io n ,see [6], pp. 485-502.

[8] P. Mor anda, "Predictions o f softw are reliability duringdebugging," P r o c . An n . R e l ia b i l i t y a n d M a in ta in a b i l i t yS y m p o s i u m , Washington, D. C., Janua ry 1975, pp. 327-332.

[9] N. F. Schneidewind, "Analysis of erro r processes in comput ersoftware", Proc. 1975 In ternat ional Conference Rel iableS o f tw a r e , Los Angeles, April 21-23, 1975, pp. 337-346.

[10] B. Littlewood, "Sof tware reliability-growth: a model forfault-removal in computer- programs and hardware-design",I E E E Tr a n s . R e l ia b i l i t y , R-30(4), Oct. 1981, pp. 313-320.

[11] B. Littlewood, J. L. Verrall, "A Bayesian reliability grow thmodel for computer software," 1 9 73 I E E E S yr u p . C o m p u terS o f tw a r e R e l ia b i l i t y . New York, N.Y., Apr. 30 - May 2,1973, pp. 70-77.

237


9/9

[ 1 2 ] J . D . M u s a , " T h e m e a s u r e m e n t a n d m a n a g e m e n t o f s o f t w a r ere l iab i l i ty , " P r oc . I E E E , 68(9 ) , 1980 , pp . 1131-1143.

[ 1 3 ] P . A . K e i l l e r , e t a l . , "O n t h e q u a l i t y o f s o f t w a r e r e l i a b i l i typ r e d i c t i o n , " P r o ceed in g s o f N AT O Ad va n c ed S tu d y I n s t i tu t eon Electronic Sys tems Ef fect iveness and Li fe Cycle Cos t ing ,N o r w i c h , U . K . , J u l y 1 9 -3 1 , 1 9 8 2 , N A T O A S 1 S e r i e s , Vol.F 3 , ( E d : J . W . S k w i r z y n s k i ) S p r i n g e r - V e r l a g , H e i d e l b e r g ,1983, pp . 441-460 .

[ 1 4 ] L . H . C r o w , "R e l i a b i l it y a n a l y s i s f o r c o m p l e x , r e p a i r a b l es y s t e m s , " Reliabi l i ty and Biometry", E d i t e d b y F . P r o s h a na n d R . J . S e r f l i n g , S I A M , P h i l a d e l p h i a , P A , 1 9 7 4 , p p . 3 7 9 -410.

[ 1 5] W . L . W a g o n e r , "Th e F in a l R ep o r t o f S o f tw a r e R e l ia b i l i t yM ea s u r em en t S tu d y , " A e r o s p a c e R e p o r t N o . T O R -0 0 7 4 ( 4 1 1 2 - 1 ) , A u g u s t 1 9 7 3 .

[ 1 6 ] G . J . S c h i c k , R . W . W o l v e r t o n , "A s s e s s m e n t o f s o f t w a r ere l iab i l i ty , " Proc. Operations Research, P h y s i c a - V e r l a g ,W u r z b u r g - W i e n , 1 9 7 3 , p p . 3 9 5 - 4 2 2 .[ 1 7] A . L . G o e l , K . O k u m o t o , "A n A n a l y si s o f R e c u r r e n t S o f t w a r eE r r o r s i n a R e a l - T i m e C o n t r o l S y s t e m , " P r o c . AC MConference, 1978 , pp . 496-501 .

2 3 8

logarithmic poisson

Documents

Transcript of logarithmic poisson