Probabilistic Programming€¦ · Probabilistic Programming Probabilistic Guarded Command Language...

Probabilistic Programming

Probabilistic Programming

Lecture #4: Probabilistic GCL

Joost-Pieter Katoen

RWTH Lecture Series on Probabilistic Programming 2018

Joost-Pieter Katoen Probabilistic Programming 1/31

Probabilistic Programming Probabilistic Guarded Command Language

Overview

1 Probabilistic Guarded Command Language

2 Operational semantics

3 Recursion



Dijkstra’s guarded command language: Syntax

Z skip empty statementZ diverge divergenceZ x := E assignmentZ prog1 ; prog2 sequential compositionZ if (G) prog1 else prog2 choiceZ prog1 [] prog2 non-deterministic choiceZ while (G) prog iteration



Elementary pGCL ingredients

Z Program variables x " Vars whose values are fractional numbers

Z Arithmetic expressions E over the program variables

Z Boolean expressions G (guarding a choice or loop) over the programvariables

Z A distribution expression µ ⇥ � � Dist(Q)Z A probability expression p ⇥ � � [0, 1] = Q


2X fly

x -y

u Co ) =Dist C Q )

I

p- -

x


Probabilistic GCL: Syntax

Kozen McIver Morgan

Z skip empty statementZ diverge divergenceZ x := E assignmentZ x :r= mu random assignment (x ⇥ ⌅µ)Z prog1 ; prog2 sequential compositionZ if (G) prog1 else prog2 choiceZ prog1 [p] prog2 probabilistic choice

Z while (G) prog iteration

Conditioning in the form of observe-statements omitted for now.


/P - I

p -Iotx

4=2 X Eh

p= I p= I5

--


Let’s start simple

x := 0 [0.5] x := 1;y := -1 [0.5] y := 0

This program admits four runs and yields the outcome:

Pr[x =0, y =0] = Pr[x =0, y =�1] = Pr[x =1, y =0] = Pr[x =1, y =�1] = 1/4


D flip a fair coin

A flip a fair cosh

- - - -


A loopy program

For 0 < p < 1 an arbitrary probability:

bool c := true;int i := 0;while (c) {

i++;(c := false [p] c := true)

}

The loopy program models a geometric distribution with parameter p.

Pr[i = N] = (1�p)N�1 � p for N > 0


I flip a biased

- - coin

→


On termination

bool c := true;int i := 0;while (c) {

i++;(c := false [p] c := true)

}

This program does not always terminate. It almost surely terminates.


=Pr { one non - terminating run } = O


The good, the bad, and the ugly



Duelling cowboys

int cowboyDuel(float a, b) {int t := A [0.5] t := B;bool c := true;while (c) {

if (t = A) {(c := false [a] t := B);

} else {(c := false [b] t := A);

}}return t;

}


B

fhitting prob .

of cowboyA

Ll

I←

←

← turn

{a survivor


Random assignments

The random assignment x ⇥ ⌅µ works as follows:

1. evaluate distribution expression µ in the current program state s2. sample from the resulting probability distribution µ(s) yielding value v with

probability µ(s)(v )3. assign the value v to the variable x .

For denoting distribution expressions, we use the bra–ket notation.

12 � [aã + 1

3 � [bã + 16 � [cã

denotes the distribution µ with µ(a) = 1/2, µ(b) = 1/3, and µ(c) = 1/6. The supportset of µ equals { a, b, c }Examples on the black board.


X : = unit Et . . 2*54

① ye 2

unit G . . D

unit Er . . zy ) ( s ) ( s ) =I

t

gsafe .

'°

- - -

X in I E xti ) t 'z Ex - t )

equivalent

Xi -- xt ' Et ] Xi - x - s

-

× : = 1¥,

Ixtn ) t III. Ex - i >

-X = 2

scx ) -- 2 I → x .

- =xt7

3- → xi=x - s

Probabilistic Programming Operational semantics

Overview



3 Recursion



Why formal semantics matters

Z Unambiguous meaning to all programs

Z Basis for proving correctnessZ of programsZ of program transformationsZ of program equivalenceZ of static analysisZ of compilersZ . . . . . .



The inventors of semantics

Tony Hoare Robert W. Floyd Gordon Plotkin

Christopher Strachey Dana ScottJoost-Pieter Katoen Probabilistic Programming 14/31

ng 8N

rgb ?

ng 6g


Approaches to semantics

Z Operational semantics: (developed by Plotkin)Z The meaning of a program in terms of how it executes on an abstract

machine.Z Useful for modelling the execution behaviour of a program.

Z Axiomatic semantics: (developed by Floyd and Hoare)Z Provides correctness assertions for each program construct.Z Useful for verifying that a program’s computed results are correct with

respect to the specification.

Z Denotational semantics: (developed by Strachey and Scott)Z Provides a mapping of language constructs onto mathematical objects.Z Useful for obtaining an abstract insight into the working of a program.

Today: operational semantics of pGCL in terms of Markov chains.

Later: denotational semantics of pGCL in terms of weakest preconditions.












{ pre } p L post )











00

- -











- -


Structural operational semantics: ingredients

Z Variable valuation s ⇥ Vars � Q maps each program variable onto avalue (here: rational numbers)

Z Expression valuation1, let [[E ]] denote the valuation of expression E

Z Configuration (aka: state) ÖP, sã denotes thatZ program P is about to be executed (aka: program counter)Z and the current variable valuation equals s.

Z Transition rules for the execution of commands: ÖP, sã ∫ ÖP ¨, s ¨ãtransition rules are written as

premiseconclusion

where the premise is omitted if it is vacuously true.

1Here, we will not go into the details of this (simple) part.Joost-Pieter Katoen Probabilistic Programming 16/31

s C x ) = Is Cy) = n

p GCL- MC

program







premiseconclusion



0scx ) - Zz E=zx2tySly ) - I

FED ( s ) = 2. (E)"

+1







premiseconclusion



MC







premiseconclusion




Recall: Markov chains

A Markov chain (MC) is a triple (�, ‡I , P) with:Z � being a countable set of statesZ ‡I " � the initial state, andZ P ⇥ � � Dist(�) the transition probability function

where Dist(�) is a discrete probability measure on �.


- -


Operational semantics

Aim: Model the behaviour of a pGCL program P by the MC [[P ]].Approach:

Z Take states of the formZ ÖQ, sã with program Q or ⇤, and variable valuation s ⇥ Vars � QZ Ösinkã models program termination (successful or violated observation)

Z Take initial state ‡I = ÖP, sã where s fulfils the initial conditionsZ Transition relation � is the smallest relation satisfying the SOS rules

on the next slidesZ Where transition probabilities equal to one are omitted


O-

< t,

s > a

•( P

,s ) mum > I

,

s 7^-3 C sink >

thymus diverging behaviours





Z Take initial state ‡I = ÖP, sã where s fulfils the initial conditions

Z Transition relation � is the smallest relation satisfying the SOS ruleson the next slides

Z Where transition probabilities equal to one are omitted






Z Take initial state ‡I = ÖP, sã where s fulfils the initial conditionsZ Transition relation � is the smallest relation satisfying the SOS rules

on the next slidesZ Where transition probabilities equal to one are omitted


①


Transition rules (1)

Öskip, sã � Ö⇤, sã Ödiverge, sã � Ödiverge, sã

Ö⇤, sã � Ösinkã Ösinkã � ÖsinkãÖx ⇥= E , sã � Ö⇤, s[x ⇥= s([[E ]])]ã

µ(s)(v ) = a > 0Öx ⇥ ⌅µ, sã a�� Ö⇤, s[x ⇥= v ]ãÖP[ p]Q, sã � µ with µ(ÖP, sã) = p and µ(ÖQ, sã) = 1�p


a a

-



Öskip, sã � Ö⇤, sã Ödiverge, sã � Ödiverge, sãÖ⇤, sã � Ösinkã Ösinkã � Ösinkã

Öx ⇥= E , sã � Ö⇤, s[x ⇥= s([[E ]])]ãµ(s)(v ) = a > 0Öx ⇥ ⌅µ, sã a�� Ö⇤, s[x ⇥= v ]ã

ÖP[ p]Q, sã � µ with µ(ÖP, sã) = p and µ(ÖQ, sã) = 1�p


1 I



Öskip, sã � Ö⇤, sã Ödiverge, sã � Ödiverge, sãÖ⇤, sã � Ösinkã Ösinkã � ÖsinkãÖx ⇥= E , sã � Ö⇤, s[x ⇥= s([[E ]])]ã



P Falu ale E in s

2×2 t y-

I = {u ifEat

value so ×

s ( z ) if x I 2-




µ(s)(v ) = a > 0Öx ⇥ ⌅µ, sã a�� Ö⇤, s[x ⇥= v ]ã

ÖP[ p]Q, sã � µ with µ(ÖP, sã) = p and µ(ÖQ, sã) = 1�p


OO

f Sly ) = 2

X inn X : = unit Er . . ay ]-

V =3 a = Ig






to

C- Dist CE )



ÖP, sã � µÖP; Q, sã � ‹with ‹(ÖP ¨; Q ¨, s ¨ã) = µ(ÖP ¨, s ¨ã) where ⇤; Q = Q

s Ï GÖif (G){P} else {Q}, sã � ÖP, sã s /Ï GÖif (G){P} else {Q}, sã � ÖQ, sãs Ï GÖwhile(G){P}, sã � ÖP; while (G){P}, sã s /Ï GÖwhile(G){P}, sã � Ö⇤, sã


( X :=E,

s ) → A,

s'

✓E Dist CE) ✓

--

- -

i - -

O

I

( P,

s ) -4C P'

,s

' ) ( P, D → Chs ' )

- -

( P ;Q ,s ) ( P

'

;Q ,s

' ) ( P ;Q,s ) → ( I;nQ,s7=Q




s Ï GÖif (G){P} else {Q}, sã � ÖP, sã s /Ï GÖif (G){P} else {Q}, sã � ÖQ, sã

s Ï GÖwhile(G){P}, sã � ÖP; while (G){P}, sã s /Ï GÖwhile(G){P}, sã � Ö⇤, sã


-

SCD = } Gee X 72gS KG

Sly ) =3s G six ) > 2 Sb )

⇒ § > 2- I

s # G




s Ï GÖif (G){P} else {Q}, sã � ÖP, sã s /Ï GÖif (G){P} else {Q}, sã � ÖQ, sãs Ï GÖwhile(G){P}, sã � ÖP; while (G){P}, sã s /Ï GÖwhile(G){P}, sã � Ö⇤, sã



Example


Xiao C I ) x : - n ; yin unit Er . . no ]- -

P Q

( P,

s > s x : - o,

s )-

SP ; Q,

s ) s x .to ; Q,

s )

= S'

(x :-O,

s > Is at,

FEET>

-Sxieoso > Is Ct ; Q,

s Tx : -03 )-

= Q

unif En . . no ] ( s' ) ( 6) = It 7 O

-

( y in unif a . . no ] ,so > ¥-3 S I

,s

'

Ey .

. =D )-

S' Cx ) = O

Sky ) - b

to

+SP ; Q

,s > Zz

← →

SXi=o;Q,

s ) ( x 7510 ,'s )

it :.

.

( d ; Q ,s Ex : -03 )

. .

,

ditto

-

wx C t,

@Ex :-D ) Ey -

- no ] )

hit( sink )

On

what changes if

x : -0 C }3x ; yin wife ..

x ]

P :

:-X: -0 ; Cien ; while (c) { Xie a - x ;

Cin ft ) - Ot

⇒ . i )x :

# tr

SP ,s )

t( . Cien ; . . . .

,S Ex :-D )

t

swhileCd-.is

I s

::÷÷÷:← ¥

( while. . .

,x⇒

,⇐ o ) C while

,

^ f s×

?.

&,

Xen,

c- o ) → C sink ) !


Duelling cowboys


if (t = A) {(c := false [a] t := B);


}}return t;

}

This (parametric) MC is finite. Once we count the number of shots before one of thecowboys dies, the MC becomes countably infinite.



Duelling cowboys


if (t = A) {(c := false [a] t := B);


}}return t;

}

11 A 0

4 A 0

6 A 1

a

! 2 • •H--+13 B •

3 A * I 8

V.>--'-r-L....I

B I

1- b

4 A 1 8 B I

5A l \ 581

\ 6 A l 4 B I

1 - a

6 /\

b

8 B 1

4 B 0

I I B 0



±ft

2 § z← c

: O.OI O O

%OO

.


Duelling cowboys


if (t = A) {(c := false [a] t := B);


}}return t;

}

11 A 0

4 A 0

6 A 1

a

! 2 • •H--+13 B •

3 A * I 8

V.>--'-r-L....I

B I

1- b

4 A 1 8 B I

5A l \ 581

\ 6 A l 4 B I

1 - a

6 /\

b

8 B 1

4 B 0

I I B 0

This (parametric) MC is finite.

Once we count the number of shots before one of thecowboys dies, the MC becomes countably infinite.



Duelling cowboys


if (t = A) {(c := false [a] t := B);


}}return t;

}

11 A 0

4 A 0

6 A 1

a

! 2 • •H--+13 B •

3 A * I 8

V.>--'-r-L....I

B I

1- b

4 A 1 8 B I

5A l \ 581

\ 6 A l 4 B I

1 - a

6 /\

b

8 B 1

4 B 0

I I B 0



int EE O

itt

it


Playing with geometric distributions

Z X is a random variable, geometrically distributed with parameter pZ Y is a random variable, geometrically distributed with parameter q

Q: generate a sample x , say, according to the random variable X � Y

int XminY1(float p, q){ // 0 <= p, q <= 1int x := 0;bool flip := false;while (not flip) { // take a sample of X to increase x

(x +:= 1 [p] flip := true);}flip := false;while (not flip) { // take a sample of Y to decrease x

(x -:= 1 [q] flip := true);}return x; // a sample of X-Y

}




Z X is a random variable, geometrically distributed with parameter p

Z Y is a random variable, geometrically distributed with parameter qQ: generate a sample x , say, according to the random variable X � Y




}




Z X is a random variable, geometrically distributed with parameter pZ Y is a random variable, geometrically distributed with parameter q

Q: generate a sample x , say, according to the random variable X � Y




}



An alternative program

int XminY2(float p, q){int x := 0;bool flip := false;(flip := false [0.5] flip := true); // flip a fair coinif (not flip) {

while (not flip) { // sample X to increase x(x +:= 1 [p] flip := true);

}} else {

flip := false; // reset flipwhile (not flip) { // sample Y to decrease x

x -:= 1;(skip [q] flip := true);

}}

return x; // a sample of X-Y}



Program equivalence: X � Y

int XminY1(float p, q){int x, c := 0, 1;while (c) {

(x +:= 1 [p] c := 0);}c := 1;while (c) {

(x -:= 1 [q] c := 0);}return x;

}

int XminY2(float p, q){int x := 0;(c := 0 [0.5] c := 1);if (c) {

while (c) {(x +:= 1 [p] c := 0);

}} else {

c := 1;while (c) {

x -:= 1;(skip [q] c := 0);

}}

return x;}

The probability that x = k for some k " Z coincides for both programs if and only ifq = 1

2�p .



Program equivalence: X � Y

int XminY1(float p, q){int x, c := 0, 1;while (c) {

(x +:= 1 [p] c := 0);}c := 1;while (c) {

(x -:= 1 [q] c := 0);}return x;

}

int XminY2(float p, q){int x := 0;(c := 0 [0.5] c := 1);if (c) {

while (c) {(x +:= 1 [p] c := 0);

}} else {

c := 1;while (c) {

x -:= 1;(skip [q] c := 0);

}}

return x;}

The probability that x = k for some k " Z coincides for both programs if and only ifq = 1

2�p .Joost-Pieter Katoen Probabilistic Programming 25/31


The outcome of a pGCL program

Unlike a deterministic program, a pGCL program P has not an output for a giveninput. Instead, it yields a unique probability distribution over its final states.

In fact, this is a sub-distribution (probability mass at most one), as with a certainprobability P may diverge.

Let P be a pGCL program and s an input state. Then the distribution over finalstates obtained by running P starting in s is given by Pr(s Ï É Ö⇤, �ã).








- -

Stys>Stx ) = .

Sly ) = .

W







d. s >

fr : I → To , n ]

I MCD = s

TE Z

E meds a

re Z







-


Reachability probabilities

If the MC [[P ]] of pGCL program P has finitely many states, reachabilityprobabilities can be obtained in an automated manner. This applies to the

cowboy example for given probabilities a and b.

The same holds for expected rewards, e.g., the expected number of steps untiltermination of a finite-state program P.


± 's

Probabilistic Programming Recursion

Overview



3 Recursion



Probabilistic GCL with recursion: Syntax

Z skip empty statementZ diverge divergenceZ x := E assignmentZ x :r= mu random assignment (x ⇥ ⌅µ)Z prog1 ; prog2 sequential compositionZ if (G) prog1 else prog2 choiceZ prog1 [p] prog2 probabilistic choiceZ while (G) prog iterationZ P = prog process definition

Z call P process invocation

Recursion does not increase the expressive power, but is often convenient.


8


Pushdown Markov chains

Pushdown Markov chain

A pushdown Markov chain D is a tuple (�, ‡I , �, “0, �) where:Z � is a countable set of (control) statesZ ‡I " � is the initial (control) stateZ � is a finite stack alphabetZ “0 " � is the bottom-of-the-stack symbolZ � ⇥ � ✓ � � Dist(�) ✓ (� \ { “0 }ò is the probability transition relation


T t -

stackstack

symbol


Recursion: pushdown Markov chains


→← → → O %D.

Pie --

y

prog = call P

Probabilistic Programming€¦ · Probabilistic Programming Probabilistic Guarded Command Language...

Documents

Transcript of Probabilistic Programming€¦ · Probabilistic Programming Probabilistic Guarded Command Language...