Tutorial: Probabilistic ModelChecking ... Probabilisticmodels purely probabilistic probabilisticand...

Click here to load reader

  • date post

    15-Aug-2020
  • Category

    Documents

  • view

    2
  • download

    0

Embed Size (px)

Transcript of Tutorial: Probabilistic ModelChecking ... Probabilisticmodels purely probabilistic probabilisticand...

  • Tutorial:

    Probabilistic Model Checking

    Christel Baier Technische Universität Dresden

    1 / 373

  • Probability elsewhere

    2/373

  • Probability elsewhere

    • randomized algorithms [Rabin 1960] symmetry breaking, fingerprint techniques, random choice of waiting times or IP addresses, ...

    • stochastic control theory [Bellman 1957] operations research

    • performance modeling [Markov, Erlang, Kolm., ∼∼∼ 1900] • biological systems • resilient systems

    ...

    ...

    ...

    3 / 373

  • Probability elsewhere

    • randomized algorithms [Rabin 1960] symmetry breaking, fingerprint techniques, random choice of waiting times or IP addresses, ...

    • stochastic control theory [Bellman 1957] operations research

    • performance modeling [Markov, Erlang, Kolm., ∼∼∼ 1900] • biological systems • resilient systems

    discrete or continuous-time Markovian models

    memoryless property: future system behavior depends only on the current state, but not on the past

    4 / 373

  • Probabilistic models

    5/373

  • Probabilistic models

    purely probabilistic

    probabilistic and nondeterministic

    discrete time

    continuous time

    6 / 373

  • Probabilistic models

    purely probabilistic

    probabilistic and nondeterministic

    discrete time

    discrete-time Markov chain (DTMC)

    Markov decision process (MDP)

    continuous time

    7 / 373

  • Probabilistic models

    purely probabilistic

    probabilistic and nondeterministic

    discrete time

    discrete-time Markov chain (DTMC)

    Markov decision process (MDP)

    continuous time

    continuous-time Markov chain (CTMC)

    continuous-time MDP

    interactive Markov chains

    probabilistic timed automata

    stochastic automata ... ... ...

    8 / 373

  • Probabilistic models

    purely probabilistic

    probabilistic and nondeterministic

    discrete time

    discrete-time Markov chain (DTMC)

    Markov decision process (MDP)

    continuous time

    continuous-time Markov chain (CTMC)

    continuous-time MDP

    interactive Markov chains

    probabilistic timed automata

    stochastic automata ... ... ...

    9 / 373

  • Model checking

    functional requirements

    specification ΦΦΦ e.g., temporal formula

    reactive system

    operational modelMMM

    model checker:

    doesM |= ΦM |= ΦM |= Φ hold ?

    no +++ counterexample yes +++ witness 10 / 373

  • Probabilistic model checking

    quantitative requirements

    specification ΦΦΦ e.g., temporal formula

    reactive system

    probabilistic modelMMM

    probabilistic model checker:

    doesM |= ΦM |= ΦM |= Φ hold ?

    no +++ counterexample yes +++ witness 11 / 373

  • Probabilistic model checking

    quantitative requirements

    specification ΦΦΦ e.g., temporal formula

    reactive system

    probabilistic modelMMM

    probabilistic model checker:

    quantitative analysis ofMMM against ΦΦΦ

    probability for “bad behaviors” is < 10−6< 10−6< 10−6

    probability for “good behaviors” is 111 expected costs for ....

    12 / 373

  • Probabilistic model checking

    • termination of probabilistic programs [Hart/Sharir/Pnueli’83]

    • qualitative linear time properties [Vardi/Wolper’86] for discrete-time Markov models [Courcoubetis/Yannak.’88]

    13 / 373

  • Probabilistic model checking

    • termination of probabilistic programs [Hart/Sharir/Pnueli’83]

    • qualitative linear time properties [Vardi/Wolper’86] for discrete-time Markov models [Courcoubetis/Yannak.’88]

    • probabilistic computation tree logic [Hansson/Jonsson’94] for discrete-time Markov models [Bianco/de Alfaro’95]

    • continuous stochastic logic [Aziz et al’96] for continuous-time Markov chains [Baier et al’99]

    • probabilistic timed automata [Jensen’96] [Kwiatkowska et al’00]...

    ...

    ...

    tools: PRISM, MRMC, STORM, IscasMC, PASS, ProbDiVinE, MARCIE, YMER, . . .. . .. . .

    14 / 373

  • Tutorial: Probabilistic Model Checking

    Discrete-time Markov chains (DTMC)

    ∗ basic definitions ∗ probabilistic computation tree logic PCTL/PCTL* ∗ rewards, cost-utility ratios, weights ∗ conditional probabilities

    Markov decision processes (MDP)

    ∗ basic definitions ∗ PCTL/PCTL* model checking ∗ fairness ∗ conditional probabilities ∗ rewards, quantiles ∗ mean-payoff ∗ expected accumulated weights

    15 / 373

  • Tutorial: Probabilistic Model Checking

    Discrete-time Markov chains (DTMC)

    ∗ basic definitions ∗ probabilistic computation tree logic PCTL/PCTL* ∗ rewards, cost-utility ratios, weights ∗ conditional probabilities

    Markov decision processes (MDP)

    ∗ basic definitions ∗ PCTL/PCTL* model checking ∗ fairness ∗ conditional probabilities ∗ rewards, quantiles ∗ mean-payoff ∗ expected accumulated weights

    16 / 373

  • Markov chains

    ... transition systems with probabilistic distributions for the successor states

    17 / 373

  • Markov chains

    ... transition systems with probabilistic distributions for the successor states

    ααα βββ γ γγ

    transition system nondeterministic branching

    choice between action-labeled transitions

    1 3 1 3 1 3 1

    6 1 6 1 6

    1 2 1 2 1 2

    Markov chain probabilistic branching

    discrete-time

    18 / 373

  • Discrete-time Markov chain (DTMC)

    MMM === (S ,P, . . .)(S ,P, . . .)(S ,P, . . .)

    19 / 373

  • Discrete-time Markov chain (DTMC)

    MMM === (S ,P, . . .)(S ,P, . . .)(S ,P, . . .) • countable state space SSS

    20 / 373

  • Discrete-time Markov chain (DTMC)

    MMM === (S ,P, . . .)(S ,P, . . .)(S ,P, . . .) • countable state space SSS ←−←−←− here: finite

    21 / 373

  • Discrete-time Markov chain (DTMC)

    MMM === (S ,P, . . .)(S ,P, . . .)(S ,P, . . .) • countable state space SSS ←−←−←− here: finite • transition probability function P : S × S → [0, 1]P : S × S → [0, 1]P : S × S → [0, 1]

    s.t. ∑ s ′∈S

    P(s, s ′) = 1 ∑ s ′∈S

    P(s , s ′) = 1 ∑ s ′∈S

    P(s, s ′) = 1

    22 / 373

  • Discrete-time Markov chain (DTMC)

    MMM === (S ,P, . . .)(S ,P, . . .)(S ,P, . . .) • countable state space SSS ←−←−←− here: finite • transition probability function P : S × S → [0, 1]P : S × S → [0, 1]P : S × S → [0, 1]

    s.t. ∑ s ′∈S

    P(s, s ′) = 1 ∑ s ′∈S

    P(s , s ′) = 1 ∑ s ′∈S

    P(s, s ′) = 1� discrete-time or time-abstract:

    probability for the step s −→ s ′s −→ s ′s −→ s ′

    23 / 373

  • Discrete-time Markov chain (DTMC)

    MMM === (S ,P,AP, L, . . .)(S ,P,AP, L, . . .)(S ,P,AP, L, . . .) • countable state space SSS ←−←−←− here: finite • transition probability function P : S × S → [0, 1]P : S × S → [0, 1]P : S × S → [0, 1]

    s.t. ∑ s ′∈S

    P(s, s ′) = 1 ∑ s ′∈S

    P(s , s ′) = 1 ∑ s ′∈S

    P(s, s ′) = 1

    • APAPAP set of atomic propositions

    • labeling function L : S → 2APL : S → 2APL : S → 2AP

    24 / 373

  • Discrete-time Markov chain (DTMC)

    MMM === (S ,P,AP, L, . . .)(S ,P,AP, L, . . .)(S ,P,AP, L, . . .) • countable state space SSS ←−←−←− here: finite • transition probability function P : S × S → [0, 1]P : S × S → [0, 1]P : S × S → [0, 1]

    s.t. ∑ s ′∈S

    P(s, s ′) = 1 ∑ s ′∈S

    P(s , s ′) = 1 ∑ s ′∈S

    P(s, s ′) = 1

    • APAPAP set of atomic propositions

    • labeling function L : S → 2APL : S → 2APL : S → 2AP • µ : S → [0, 1]µ : S → [0, 1]µ : S → [0, 1] initial distribution • wgt : S → Zwgt : S → Zwgt : S → Z where wgt(s)wgt(s)wgt(s) is the reward (or weight)

    earned per visit of state sss 25 / 373

  • Example: DTMC for communication protocol

    MMM === (S ,P,AP, L, . . .)(S ,P,AP, L, . . .)(S ,P,AP, L, . . .) • countable state space SSS ←−←−←− here: finite • transition probability function P : S × S → [0, 1]P : S × S → [0, 1]P : S × S → [0, 1]

    s.t. ∑ s ′∈S

    P(s, s ′) = 1 ∑ s ′∈S

    P(s , s ′) = 1 ∑ s ′∈S

    P(s, s ′) = 1

    startstartstart trytrytry tototo sendsendsend

    delivereddelivereddelivered

    messagemessagemessage lostlostlost

    0.980.980.98

    0.020.020.02

    26 / 373

  • Example: DTMC for communication protocol

    MMM === (S ,P,AP, L, . . .)(S ,P,AP, L, . . .)(S ,P,AP, L, . . .) • countable state space SSS ←−←−←− here: finite • transition probability function P : S × S → [0, 1]P : S × S → [0, 1]P : S × S → [0, 1]

    s.t. ∑ s ′∈S

    P(s, s ′) = 1 ∑ s ′∈S

    P(s , s ′) = 1 ∑ s ′∈S

    P(s, s