Probabilistic Verification of Discrete Event Systems

Håkan L. S. Younes

The Problem Given a model of a discrete event

system, check if certain properties hold The model is a stochastic process

(GSMP) Properties are expressed using a logic

formalism (CSL)

Probabilistic Verification Verification of probabilistic properties

“The probability of reaching a failure state within 60 minutes is less than 0.1”

Probabilistic verification of properties “The probability of property P holding is

at least 0.95”

Discrete Event System (DES) Event-driven system Discrete state changes at the

occurrence of events Examples:

Manufacturing systems Queueing systems Communication protocols

Why Probabilistic Verification? The dynamics of a DES is too

complex for symbolic methods Use simulation to generate sample

paths Use acceptance sampling to verify

probabilistic properties

Stochastic Processes A stochastic process consists of

a set of transitions (or events)

a set of states

S1

S2

S3

S4

Markov Processes The Markov assumption

There is enough information in the current state to determine the future behavior

S1

S2

S3

S4

1.0

0.4

0.6 1.0

0.2

0.8

Holding Times The holding time is the time spent

in a state before an event occurs Holding times are positive random

variables Can be discrete or continuous

Holding times are governed by exponential distributions

0.4

0.6

1.0

1.0

0.20.8

S1

S2

S4

S3

4

3

2

1

Sta

teTime

Continuous-timeMarkov Chain (CTMC)

Semi-Markov Process Holding times are governed by

arbitrary (positive) distributions

0.4

0.6

1.0

1.0

0.20.8

S2

S3S1

S4

Generalized Semi-Markov Process (GSMP) Holding times can depend on the

history4

3

2

1

Sta

te

Time

0.7

0.6

1.0E2

E3

S2

S3E1

E2

S1

E1S4

0.4

1.0

0.50.5

0.3

E1

E2

E3

3.5

5.0

-

-

1.5

2.0

3.5

5.0

-

-

-

-

-

1.5

2.0

Properties Qualitative

“P will eventually hold on all future execution paths”

Quantitative “P will hold before time t with

probability at least on future execution paths”

Problem Space

GSMP

CTMC

Model

Qualitative Quantitative

[ACD91]

[ASSB96],[BKH99]

My Work

Properties

[EMSS89]*

*Discrete time

Continuous Stochastic Logic (CSL) State formulas: a, ¬, 1 2,

Pr() Truth value is determined in a single

state Path formulas: X , 1 Ut 2

Truth value is determined over an execution path

Execution Paths Current state + current clock

settings = internal state The internal state contains enough

information to determine the future behavior

A sequence of internal states is an execution path

S0,C0 S1,C1 S2,C2

t0 t2t1

CSL Semantics(State Formulas) Atomic proposition: a Negation: ¬

Holds iff does not hold in current state

Conjunction: 1 2

Holds iff both 1 and 2 hold in current state

CSL Semantics(More State Formulas) Probabilistic statement: Pr()

Holds iff is true over at most a proportion of execution paths starting in the current state

CSL Semantics(Path Formulas) Next state: X

Holds iff holds in the next state along the current execution path

Until: 1 Ut 2

Holds iff 2 becomes true in some state along the current execution path before time t, and 1 is true in all prior states

More on Until Consider the formula a U17 b

a,¬b a,¬b a,¬b b b2 3 4 11

a,¬b ¬a,¬b a,¬b b b2 3 4 11

a,¬b a,¬b a,¬b a,¬b b2 3 4 11

Verifying Probabilistic Statements Verify Pr()

Generate sample execution paths using discrete event simulation

Verify over each sample path If is true, then we have a positive sample If is false, then we have a negative

sample Based on the proportion of positive

samples, determine if Pr() holds

Sequential Hypothesis Testing

True, false,or anothersample?

Hypothesis: Pr()

Error Bounds Probability of false negative:

We say that Pr() is false when it is true

Probability of false positive: We say that Pr() is true when it is

false

False negatives

False positives

Indifference Region

– +

Indifference region

False negatives

False positives

Actual probability of holding

Pro

bab

ility

of

acc

ep

tin

gPr

(

) as

tru

e

1 –

Accept

Reject

Continue sampling

Accept

Reject

Continue sampling

Graphical Representation of Statistical Test We can find an acceptance line

and a rejection line given , , , and

Number of samples

Nu

mb

er

of

posi

tive s

am

ple

s

Verification of Nested Probabilistic Statements Suppose , in Pr(), contains

probabilistic statements

True, false,or anothersample?

Indirect Sampling Want samples from random

variable X Can only get samples from Y such

that Pr[Y=1|X=1] 1 – ’ Pr[Y=0|X=1] ’ Pr[Y=1|X=0] ’ Pr[Y=0|X=0] 1 – ’

Modified Test find an acceptance line and a

rejection line given , , , , ’, and ’:

Accept

Reject

Continue sampling

Number of samples

Nu

mb

er

of

posi

tive s

am

ple

s

Verification of Compound State Formulas To verify ¬ with error bounds

and Verify with error bounds and

To verify 1 2 … n with error bounds and Verify 1 though n with error bounds

/n and /n

Sequential Verification of Conjunction To verify 1 2 … n with error

bounds and 1. Verify each i with error bounds and ’

2. Return false as soon as any i is verified to be false

3. If all i are verified to be true, verify each i again with error bounds and /n

4. Return true iff all i are verified to be true

Verification of Path Formulas To verify X with error bounds and

Verify with error bounds and in the

next state To verify 1 Ut 2 with error bounds

and Convert to conjunction

1 Ut 2 holds if 2 holds in the first state, or if 2 holds in the second state and 1 holds in all prior state, …

More on Verifying Until Given 1 Ut 2, let n be the index of

the first state more than t time units away from the current state

Conjunction of n conjuncts c1 through cn, each of size i

Simplifies if 1 or 2, or both, do not contain any probabilistic statements

Example Verify Pr0.05(true U200 dead) in S1

SS11:: Safe, at restSS22: Under attack, at restSS33: Under attack, jumpingSS44: Safe, jumping

E1: Stork attackingE2: Frog killedE3: Start jumpingE4: Stork retreatsE5: Stop jumping

E1

S1

E2

E4

S3

E2

E3

S2

E5 S4

26.3----

0.0

-124.544.3

--

26.3

26.3----

0.0

-80.2

-95.2

-70.6

-124.544.3

--

26.3

-----

150.8

-80.2

-95.2

-70.6

89.9----

0.0

-155.848.3

--

89.9

89.9----

0.0

-107.5

-61.0

-138.2

-155.848.3

--

89.9

----

36.2199.2

-107.5

-61.0

-138.2

59.4----

235.4

----

36.2199.2

E1:E2:E3:E4:E5:t:

20 6040 80 100

2

4

6

Summary Algorithm for probabilistic

verification of discrete event systems

Sample execution paths generated using discrete event simulation

Probabilistic properties verified using acceptance sampling

Algorithm can be used in an anytime manner

Future Work Apply to hybrid dynamic systems Develop heuristics for formula

ordering and parameter selection Use verification to aid policy

generation for real-time stochastic domains