Conditional Probabilities over Probabilistic and Nondeterministic Systems M. E. Andrés and P. van...

Conditional Probabilities over Conditional Probabilities over Probabilistic and Nondeterministic Probabilistic and Nondeterministic

SystemsSystems

M. E. Andrés and

P. van Rossum

Radboud Universiteit Nijmegen, The Netherlands.

2TACAS - April 1TACAS - April 1stst

Budapest, HungaryBudapest, HungaryMiguel E. AndresRadboud University

OverviewOverview

Motivation Background

Markov Decision Processes and Schedulers Conditional Probabilities pCTL

Our Logic (cpCTL) Model Checking issues

Fully probabilistic case Probabilistic and Nondeterministic case

Comparison (pCTL vs cpCTL) cpCTL Complications

Model Checker Counterexamples Future work

OverviewOverview

Markov Decision Processes and Schedulers Conditional Probabilities pCTL

MotivationMotivation

Model Checking

Modelj=

Temporal Logics

P[§ DeadL]§ DeadL

P+[§DeadL]P+[§ DeadL j¤ SingU]

· 0:1

· 0:1(+ cond prob) cpCTL

(+ nondet) pCTL

(+ prob) pCTL

LTL – CTL

MotivationMotivation

Conditional ProbabilitiesAnonymity

Strong AnonymityProbable innocence

What we doDefine cpCTLModel Checker for cpCTLPresent a Notion of Counterexamples

Deterministic CaseNondeterministic Case

Risk assessmentP[dyke breaks| it rains heavily]

Diagnosability P[A failed|error message E]

OverviewOverview

Markov Decision Processes and Schedulers pCTL Conditional Probabilities

Probabilistic and Nondeterministic

Example

Background – MDPsBackground – MDPs

The Model (MDP)

®0®®®

TEXPH3:1:2®£ ±2

² S is the¯nitestatespaceof thesystem² s0 2 S is the initial state² L : S ! }(P ) is a labeling function² ¿: S ! }(Distr(S))

MDP =(S,s0;L ;¿), where:

Finite Paths Paths

s0s2s0s2s3...

s0s2(s3)!

s0(s1)!...

Background – SchedulersBackground – Schedulers

Schedulers resolve the Nondeterminism!

Schedulers: FinitePath ! Distr(S)

²P [s0s2s5]= 18

²P [s0s2s6]= 0

S2 ! ¼2

S2 ! ¼3

²P [s0s2s5]= 0²P [s0s2s6]= 1

S214! ¼2

S234! ¼3

Background – pCTLBackground – pCTL

SyntaxisState

Pathª :=©U©j §©j ¤© Semantic

¾j= ÁUÃ , Á holds until at somepoint Ã holds¾j= §Á , ¾j= trueUÁ¾j= ¤Á , ¾j= : § : Á

©:= P j ©^©j : ©j 8ª j 9ª j P ./ a[ª ]

a2 [0;1]

./ 2 f<;· ;>;¸ g

s j= var , var 2 L(S)s j= Á^Ã , s j= Á and s j= Ãs j= : Á , s 6j= Ás j= 8Á , ¾j= Á for all f. paths¾starting fromss j= 9Á , ¾j= Á for any f. path ¾starting fromss j= P · a[Á] , max´ P s;´ [Á] , P+

s [Á] · a

Example

Background – computing satisfactionBackground – computing satisfaction

0;775 34+14(12¡ ®) +

14®= 0;875

Background – Conditional ProbabilitiesBackground – Conditional Probabilities

Standard Conditional Probabilities

P (A j B) =P (A \ B)P (B)

Max and Min Conditional Probabilities

P +(¢ 1 j ¢ 2) = sup´2Sch> 0

P ´ (¢ 1 j ¢ 2) P ¡ (¢ 1 j ¢ 2) = inf´2Sch> 0

P ´ (¢ 1 j ¢ 2)

Conditional Probabilities over MDPs

P ´ (¢ 1j¢ 2) =P ´ (¢ 1 \ ¢ 2)P ´ (¢ 2)

² ( s;Bs;P ´ ) is theprobability space² ¢ 1;¢ 2 2 Bs are two sets of paths² P ´ (¢ 2) > 0

² ( ;F;P ) is a probability space² A;B 2 F are two events² P (B) > 0

OverviewOverview

s j= P · a[ÁjÃ]P s [Á^Ã]P s [Ã]

Our Logic – cpCTLOur Logic – cpCTL

pCTL cpCTL

ª :=©U©j §©j ¤©

©:= P j ©^©j : ©j 8ª j 9ª j P ./ a[ª ]

Interpretation

P+s [ÁjÃ]

s j= P · a[ÁjÃ] max´2Sch> 0

P s;´ [Á^Ã]P s;´ [Ã]

P [AjB]=P [A \ B]P [B]

max´P s0;´

[§ B ^¤ P ]

P s0;´[¤ P ]

· 0;99

cpCTL - ExamplecpCTL - Example

S0 j= P · 0;99[§ B j¤P ]

²P s0;´¼2[§ B j¤P ]= P [s0s1]+P [s0s2s3]

P [s0s1]+P [s0s2s3]+P [s0s2s4]= 1¡ 2®

max(1¡ 2®7; 3031) · 0;99

²P s0 ;´¼3[§ B j¤P ]= P [s0s1]

P [s0s1]+P [s0s2s6]= 30

OverviewOverview

Model Checking IssuesModel Checking Issues

Fully probabilistic case

Can be reduced to a pCTL* problem, using

P +s [ÁjÃ] 6=

P +s [Á^Ã]P +s [Ã]

Observation

Probabilistic and Nondeterministic case

pCTL cpCTLDeterministic Schedulers Deterministic Schedulers

History Independent Schedulers

Semi History Independent Schedulers

Bellman Equations NO Bellman Equations

P +s [ÁjÃ]=max

P s;´ [Á^Ã]

P s;´ [Ã]

Model Checking Issues – Model Checking Issues – Nondeterministic caseNondeterministic case

cpCTL case Deterministic Schedulers (Not trivial) Semi History Independent Schedulers No Bellman equations

Theorem: Deterministic Schedulers

P ´ [ÁjÃ]= P+[ÁjÃ] and P ´0[ÁjÃ]= P ¡ [ÁjÃ]

Thereexists Deterministic schedulers ´ and ´0 such that

Coming…

Semi History Independent Schedulers Why?If P +

s0[§ B j§ P ]= P s0 ;´

[§ B j§ P ]then ´ satis es

´(¾) =

¼3 if ¾= s0¼5 if ¾= s0s3¼1 if ¾= s0s3s0

Definition´ is ' -semi History Independent if

² ´ takes always the samedecision before the system reaches '² ´ takes always the samedecision after the system reaches '

P ´ [ÁjÃ]= P+[ÁjÃ] and P ´0[ÁjÃ]= P ¡ [ÁjÃ]Thereexists deterministic and sHI schedulers ´ and ´0 such that

Theorem: sHI Schedulers

Stopping condition

Local Bellman equation

P +s2[§ P ]=

P +s [Á]= max

¼2¿(s)

t2succ(s)

¼(t) ¢P +t [Á]

ABellman Equations

¢P+s6[§P ]+

¢P+s7[§P ]

(12¡ ®) ¢P +

s3[§ P ]+®¢P +

s4[§ P ]+

12¢P +

s5[§ P ]

P+s2[§P ]=max

(12¡ ®) ¢P+

s3[§P ]+®¢P+

s4[§P ]+ 1

2 ¢P+s5[§P ]

110 ¢P

+s6[§P ]+ 9

10 ¢P+s7[§P ]

M aximum over all outgoingdistributions ¼of s

RecursiveComputation

Why Not Bellman equations?

Bellman equation on cpCTL case… P +s0[ÁjÃ]= max

¼2¿(s)

t2 succ(s)

¼(t) ¢P +t [ÁjÃ]

P+s0 [§Bj¤P ] · 0;99

max(1¡ 2®7; 3031) · 0;99

P +s0[§Bj¤P ]= P s0;´¼3

[§Bj¤P ]

If ®¸ 762 then

…but P +s2[§Bj¤P ]= P s2;´¼2

[§Bj¤P ]= 1¡ 2¢®

OverviewOverview

Model Checker - IdeaModel Checker - Idea

P +s [ÁjÃ]=max

µP s;´ [Á^Ã]

P s;´ [Ã]

{By deterministic and sHI Theorem}

P +s [ÁjÃ]=max

µP s;´1

[Á^Ã]P s;´1

[Ã];¢¢¢;

P s;´k[Á^Ã]

P s;´k[Ã]

where f ´1;´2; : : : ;´kg is the set of all deterministic and sHI schedulers

What we actually computef (s;Á;Ã) =

©(P s;´1

[Á^Ã];P s;´1[Ã]);¢¢¢;(P s;´k

[Á^Ã];P s;´k[Ã])

P +s [ÁjÃ]=max

³ nabj (a;b) 2 f (s;Á;Ã) ^b6= 0

o[ f0g

Model Checker - ExampleModel Checker - Example

Optimizations Reusing information

Ussing pCTL algorithms after reaching the stopping condition

Example¡Case P+

s [Á1UÁ2jÃ1UÃ2]¢

f (s;Á1UÁ2;Ã1UÃ2) = f (P +s [Ã1UÃ2];P

+s [Ã1UÃ2])g if s j= Á2

f (s;Á1UÁ2;Ã1UÃ2) = f (P +s [Á1UÁ2];1)g if s j= : Á2^Ã2

f (s;Á1UÁ2;Ã1UÃ2) = f (0;P ¡s [Ã1UÃ2])g if s j= : Á1^: Á2^: Ã2

f (s;Á1UÁ2;Ã1UÃ2) = f (0;0)g if s j= Á1^: Á2^: Ã1^: Ã2f (s;Á1UÁ2;Ã1UÃ2) =S¼2¿(s)

³ Lt2succ(s)¼(t) ¯ f (t;Á1UÁ2;Ã1UÃ2)

´if s j= Á1^: Á2^Ã1^: Ã2

OverviewOverview

CounterexamplesCounterexamples

Counterexamples

Model'j=

Counterexamples for cpCTLA counterexample for P · a[ÁjÃ] is a pair (¢ 1;¢ 2) of measurable sets

of paths satisfying ¢ 1 µ ¢ Á^Ã , ¢ 2 µ ¢ : Ã , and a<P ´ (¢ 1)

1¡ P ´ (¢ 2), for some

scheduler ´.

s j= P · a[ÁjÃ] , for all ´ P s ;´ [Á^Ã]P s ;´ [Ã]

where¢ 1 µ ¢ Á^Ã , f ! 2 j ! j= Á^Ãgand ¢ 2 µ ¢ : Ã , f ! 2 j ! j= : Ãg

P ´ [Á^Ã]P ´ [Ã]

> a P ´ (¢ 1)1¡ P ´ (¢ 2)

OverviewOverview

Motivation Backgorund

Future WorkFuture Work

Implement our Algorithms in a probabilistic model checker.

Investigate features of cpCTL (expressivness –bisimulation issues).

Improve complexity.

Extend cpCTL to cpCTL*.

More research about counterexamples in cpCTL and cpCTL*.

Thanks for your attention!Thanks for your attention!

Why Deterministic Schedulers?Why Deterministic Schedulers?

Lema: Let v1;v2 2 [0;1 ) and w1;w2 2 (0;1 ). Then the functionf : R ! R de ned by f (®) , ®v1+(1¡ ®)v2

®w1+(1¡ ®)w2ismonotonous.

Á ÁÃ Ã

s1 s21¡ ®®

P s0[ÁjÃ]=

®P s 1[Á^Ã]+(1¡ ®)P s 2

[Á^Ã]®P s1

[Ã]+(1¡ ®)P s 2[Ã]

Conditional Probabilities over Probabilistic and Nondeterministic Systems M. E. Andrés and P. van...

Documents

Transcript of Conditional Probabilities over Probabilistic and Nondeterministic Systems M. E. Andrés and P. van...

Van rossum nieuwsbrief feestdagen

Paper: Nondeterministic Events in Business Processes

Lecture 6 Nondeterministic Finite Automata (NFA).

03a Nondeterministic Finite Accepters

Lecture Four: Nondeterministic Finite Automata

Nondeterministic Finite Automata

Purely functional lazy nondeterministic programminghomes.sice.indiana.edu/ccshan/rational/S0956796811000189... · 2011-09-14 · Purely functional lazy nondeterministic programming

1 Basics of automata theory Nondeterministic Finite Automata (NFA) Nondeterministic Finite Automata on infinite words (NFW)

Nondeterministic Finite State Automata

Java Card - Radboud Universiteit - Radboud Universiteit

Complementation Constructions for Nondeterministic ...

Nondeterministic Programming in Java with JSetL

Nondeterministic Hybrid Architectures for Vehicle Health ... · Nondeterministic Hybrid Architectures for Vehicle Health Management 41-3 41.2.1 Stochastic VEPHM Tools Development

Automata with Modulo Counters and Nondeterministic Counter ...

Introduction to Type Theory - Radboud Universiteit - Radboud

Nieuwsbrief Boekhandel van Rossum Boekenweek 2016

Nondeterministic Impact of CPU Multithreading on Training ...

Mifare Classic Troubles Peter van Rossum Digital Security Radboud University Nijmegen.

Learning in Networks of Nondeterministic Adaptive Logic ... · LEARNING IN NETWORKS OF NONDETERMINISTIC ADAPTIVE LOGIC ... number of logical input channels, ... Networks of Nondeterministic

23-August-1999© 1999 CNRI, Guido van Rossum 1. 23-August-1999© 1999 CNRI, Guido van Rossum 2 Python Track Opening Words Guido van Rossum guido@python.org.