ProbabilisticModelChecking withPRISM

MartaKwiatkowska Gethin NormanDaveParker

UniversityofBirmingham

UniversityofGlasgow

UniversityofOxford

HVC2016,Haifa,November2016

Past,Present&Future

1

Outline

• ProbabilisticmodelcheckingandPRISM

• Themesandtrends

• Advancesandapplications

• Currentresearchtopics

• Challenges&futuredirections

2

Probabilisticmodelchecking• Constructionandanalysisofprobabilisticmodels

– probability:failures,uncertainty,noise,randomisation,…– time:delays,time-outs,failurerates,…– costs:energy,resources,…

• Quantitativecorrectnesspropertiesexpressedintemporallogic,e.g.:– trigger→P≥0.999[F≤20 deploy ]– “theprobabilityoftheairbagdeployingwithin20millisecondsofbeingtriggeredisatleast0.999”

– reliability,timeliness,performance,efficiency,…

0.40.5

0.1

3

PRISM• A(brief)history

– late80s,early90s:firstunderlyingtheorydeveloped– 2001:firstofficialpublicreleaseofPRISM– 2011:version4.0- probabilisticrealtimesystems– 2013:PRISM-games– stochasticmulti-playergames

• PRISMtoday– usedin100+institutions;50,000+downloads– broadlyapplicable;manydiverseusecases– manynon-expert(andnon-CS)users– 300externalpapers(noinvolvementfromPRISMteam)– flawsfoundinreal-systems;industrialusage

4

WhatcanwedowithPRISM?• Identifyflawsinexistinganalyses

– e.g.reliabilityofNANDmultiplexing

• Investigateconjectures/models– e.g.Herman’sself-stabilisation– e.g.FireWirerootcontention– e.g.cellsignallingpathways(FGF)

1 2 3 4 5 6 70

0.2

0.4

0.6

0.8

1

Number of restorative stages

Prob

abilit

y

λ = 0.01 λ = 0.02 λ = 0.03 λ = 0.04

λ = 0.01 λ = 0.02 λ = 0.03 λ = 0.04

PRISM

Analytical

Herman FireWire FGF

NAND

5

Themesandtrends• ThemesinthedevelopmentofPRISM

– theory-to-practice (andpractice-to-theory)– widecollaboration (theory,algorithms,casestudies)– opensource software(anddata)– overlaps/interactswithmanyotherdisciplines

• Trends– improvementinscalability tolargermodels– increasinglyexpressive/powerfulclassesofmodel– fromverificationproblemstocontrol problems– everwideningrangeofapplicationdomains

6

Trends

• discrete-time Markov chains

• probabilistic automata

• continuous-time Markov chains

• Markov decision processes

• probabilistic timed automata

• stochastic multi-player games

• …

Models

• randomised distributed algorithms

• network/communication protocols

• computer security

• performance/reliability

• systems biology

• DNA computing

• robotics & autonomous vehicles

• wearable/implantable devices

• …

Applicationdomains

7

Enablingtechnologies• Symbolicmodelchecking

– [TACAS’00][TACAS’02][STTT’04][CAV’06] …

• Real-timeprobabilisticverification– [TCS’02][FMSD’06][Info&Comp’07][FORMATS’09] …

• Quantitativeabstractionrefinement– [QEST’06][VMCAI’09][FMSD’10][QEST’11]...

• Compositionalverification– [TACAS’10][QEST’10][FASE’11][Info&Comp’13]...

• Andmore...– statisticalmodelchecking,symmetryreduction,bisimulationminimisation,... M1 ||M2 ⊨ ⟨G⟩≥p

⟨A⟩≥q M2 ⟨G⟩≥pM1 ⊨ ⟨A⟩≥q

initx≤2

0.9

retry

donetrue

lostx≤5

failtrue

quitsend

x≥3

x:=0

0.1x≥1∧tries≤N

tries:=0

tries>Nx:=0,tries:=tries+1

8

Casestudy:Bluetooth• DevicediscoverybetweenapairofBluetoothdevices

– performanceessentialforthisphase

• Detailedmodelfromofficialspecification– twoasynchronous28-bitclocks– pseudo-randomhoppingbetween32frequencies– randomwaitingschemetoavoidcollisions– 32Markovchains,over3x1010 stateseach– 17,179,869,184initialconfigurations

• Symbolicprobabilisticmodelchecking– “worst-caseexpecteddiscoverytimeisatmost5.17s”

freq = [CLK16-12+k+ (CLK4-2,0-CLK16-12) mod 16] mod 32

9

Strategy/controllersynthesis• Verificationvs.control

– verify thatasystemis“correct”,foranyenvironment/adversary/…(counterexample yieldsflaw/attack/...)

– synthesise a"correct-by-construction”controllerfromformalspecification(witness yields strategy/controller)

• Applications– dynamicpowermanagement,robots/autonomousvehiclenavigation,task/networkscheduling,security,…

p1 p2 p3 p4

{se,usb} p1?

T

F

;

;

p2?T

F

p2?T

F

{ef}

;

;

;

Fig. 9. An attack decision tree for the optimal attacker strategy highlightedin the stochastic game shown in Figure 7.

If the state is an attacker state, s = s

A

2 S

A

, and the orderof the players is AD, we just move to the next probabilisticstate s

P

. The state s

P

is chosen nondeterministically. Note,that for any choice of s

P

further construction on the decisiontree is the same. If the order of the players is DA, this meansthat there is no defender action in the current phase. Thus, wecreate an empty set in the decision tree and move to the nextprobabilistic state s

P

.On probabilistic states, the function generateDD behaves

the same as the function generateAD, as described above.Finally, we note that the decision tree constructed as above

can subsequently be optimised by merging identical subtreesand removing decision nodes with identical then/else branches.Randomisation. As described in Sect. III, we also considerdecision trees that incorporate randomisation. This is becauseoptimal strategies for multi-objective properties may be ran-domised. Here, that means that strategy �

A

(or �

D

) mayselect a distribution over actions in a state s

A

(or sD

), ratherthan a single action. The decision tree synthesis algorithms inTables VI and V thus remain unchanged but the rules for s

A

and s

D

states, respectively generate random action nodes.

Example 7 The stochastic game in Figure 7 also shows anoptimal attacker strategy marked in bold. We show in Figure 9the (optimised) attacker decision tree corresponding to theoptimal attacker strategy.

V. IMPLEMENTATION AND RESULTS

We have developed a prototype implementation of ourtechniques, comprising a converter from attack-defence trees,specified in XML, into stochastic games modelled in the inputlanguage of PRISM-games [15], available from [21]. Theoutput of the tool can then be used to perform verificationand strategy synthesis as described earlier.

We applied our approach to a real-life scenario studiedin [5]: we consider part of a Radio-Frequency Identification(RFID) goods management system for a warehouse, modifiedby introducing temporal dependencies between actions.

The warehouse uses RFID tags to electronically identifyall goods. In the attack-defence scenario that we consider,

0 0.1 0.2 0.3 0.4

0

100

200

300

400

500

Attack success probability

Expe

cted

atta

ckco

st

Fig. 10. Pareto curve illustrating the trade-off between attack successprobability and expected attack cost over strategies for the RFID example.

the attacker aims to physically remove some RFID tags afterinfiltrating the building.

In order to achieve this goal, the attacker has to first getinto the premises and then into the warehouse. For gettinginto the premises the attacker can climb over the fence orenter through the main gate. The defender can protect againstclimbing by setting some barbed wire on the fence. To protectagainst the barbed wire the attacker can guard against barbseither by using a carpet over the barbs or by wearing protectivecloths. Once the attacker succeeds in accessing the premises,they have to get into the warehouse. The attacker can achievethis subgoal either by entering through the door or by enteringthrough the loading dock. The former action can be defendedagainst by monitoring the door with biometric sensors.

The defender can prevent the attacker from attaining themain goal by monitoring the premises with security cameras.In order to overcome the camera issue the attacker can disablethem either by shooting a strong laser at the cameras or byvideo looping the camera feed. The defender, in turn, canemploy guards in order to patrol the premises and counterthis kind of attack.

The corresponding attack-defence tree is given in Figure 11.The leaves (basic attack and defence actions) of the tree aredecorated with success probability and cost values. The attack-defence tree has three phases: the first phase corresponds to thesub-tree with the root “get into premises”, the second phaseis the “get into warehouse” sub-tree, and the last phase is thesub-tree on the right of the main goal with the defender actionon the root. The syntactic term corresponding to each phaseand to the full tree is:

t1 = _(^(ef,⇠ ^(bw,⇠ _(uc, pc))), tg)t2 = _(^(ed,⇠ bs), ld)t3 = ⇠ ^(sc,⇠ _(lc,^(vc,⇠ eg)))

t =�!̂(�!̂(t1, t2), t3)

The resulting stochastic game generated from the attack-defence by our approach has 1072 states and 2052 transitions.We verified a variety of properties, including the numericalproperty hhAiiP

max=?[F success]) that computes the maxi-mum success probability of an attack (equal to 0.41), and the

12

Attack-defence tree[CSF’16]

Taskschedule[FMSD’13]

0.1

s0 s1slow

fast

s3

off on

0.9

s4

0.1

s2slow

fast0.9

off on

MDP

10

Multiple objectives• Multi-objectivecontrollersynthesis[LMCS’08][TACAS’11]

– trade-offsbetweenconflictingobjectives

• Mixofoptimisationandguarantees– e.g.“whatstrategymaximisesprobability ofmessagetransmission,whilstguaranteeingexpectedbatterylife-timeis>10hrs?”

– Paretocurvegeneration/approximation

• Extensions– permissivecontrollersynthesisofmulti-strategiesforMDPs[LMCS’15]

– multipleobjectivesformulti-playergames(seelater)

obj1

obj 2

11

Robots&autonomoussystems• Navigationformobileservicerobots

– learntprobabilisticnavigationmaps– LTLtaskspecifications +controllersynthesis– ROS-basedruntimeplanningimplementation– multi-objectiveprobabilisticguaranteesontaskcompletion/duration[IROS’14/IJCAI’15/CDC’16]

• Autonomousunderwatervehiclenavigation– incremental/parametricverification+controllersynthesis– probabilisticprogramming+machinelearningtogeneraterealisticcomponent/environmentmodelsatruntime

12

Parametersynthesis• Synthesisingmodelsthatareguaranteedtosatisfyquantitativecorrectnesspropertiesisdifficult– butwecansynthesisecontrollers andparameters

• Parametersynthesis– givenaparametric modelandapropertyɸ…– findtheoptimalparametervalues,withrespecttoanobjectivefunctionO,suchthatthepropertyɸ issatisfied,ifsuchvaluesexist

• Quantitativeparametersynthesis– parameters:timingdelays,rates– objectives:optimiseprobability,reward/volume

13

Quantitativeparametersynthesis• Timed/hybridautomata

– findoptimaltimingdelays[EMSOFT2014][HSB’15][HSCC’16]

– constraintsolving,discretisation+sampling

• Probabilistictimedautomata– finddelaystooptimiseprobability [RP2014]

– parametricsymbolicabstraction-refinement

• Continuous-timeMarkovchains– findoptimalrates [CMSB’14][ActaInf'16], PRISM-PSY [TACAS’16]

– constraintsolving,uniformisation +sampling

• Focus:practicalimplementation,real-worldusage14

Applications

Pki kr

k r

ki

k

0

0.05

0.1

0.15

0.2

0.25

0.3

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2

r

EpidemicmodellingPacemakerverification

ɸ:infectionlastsatleast100timeunitsandendswithin120timeunits

Maximalvolumeobjective

15

Mobileautonomychallenge• Autonomoussystems

– interact withtheirenvironment,whichispossibly adversarial

– havegoals/objectives,whichmay conflict

– takedecisions

• Modelasstochasticgames– wellknownfrom,e.g.,decisionmakingineconomics– manyapplicationdomains:security,energygrid,etc

• ToolPRISM-games,extensionofPRISM[TACAS’16]16

Stochasticmulti-playergames• Probabilistictemporallogic withcoalitions

– probabilities,rewards(reachability,total,mean-payoffs/ratios,…)[FMSD’13][ICALP’16][ECC’16]

• Multi-objective strategysynthesis– Paretosetcomputationandoptimalachievabletrade-offs[MFCS’13][QEST’13][TACAS’15]

• Compositional strategysynthesis[CONCUR’14][Inf &Comp’16]

– assume-guarantee+multi-objectivestrategysynthesis– e.g.localstrategiesforG1 ⊨ φA ,G2 ⊨ φA⇒ φBcomposetoaglobalstrategyforG1||G2 ⊨ φB

17

Applications• UAVpathplanning[ICCPS’15]

– humanoperator+low-levelpiloting– quantitativemissionobjectives:minimisetime/fuel,restrictedzones,operatorfatigue/workload

– multi-objective MDPs,stoch.games

• Aircraftpowerdistribution [CONCUR’14]

– compositionalstrategysynthesisinstochasticgames(PRISM-games)

– specifycontrolobjectivesinLTLusingmeanpayoff

18

Aregamessufficient?• Complexdecisions!

– goals– perception– situationawareness– context(social,regulatory)

• Whataboutsocialsubtleties?• Whattodoinemergency?

– moral decisions,handover todriver,obeytrafficrules

• Needtomakerobotshuman-like…– needmulti-modalcommunication,cognitivereasoning,

trust,ethics,…19

Quantitativeverificationfortrust?• Socialtrust:fundamentalformobileautonomy[LK16b]

– influencedbyexternal factors,suchassocialnorms– alsointernal:personality,motivation,preferences– subjective:wouldyou trustanautonomoustaxitotakeyourchildtoschool?

• FormulateatemporallogictoexpressX’strustinYforG,basedonprobabilisticbelief[HK17]

• Admitsamodelcheckingprocedure,whichcan:– beusedindecision-making forrobots– explain decisions,i.e.whoisaccountablefortheaction

20

Perceptionsoftware

• Lidar, prior map, object recognition –relies on machine learning

Credits:OxfordRoboticsInstitute

21

Thingsthatcangowrong…• …inperceptionsoftware

– sensorfailure– objectdetectionfailure

• Machinelearningsoftware– notclearhowitworks– doesnotofferguarantees

- Verificationformachinelearning?- someprogresstowardssafetyverificationforneuralnetworks

22

Personalisationchallenge• Devicemustadapt tophysiologyofhumanwearer

– achievedthroughmodelparameterisation– parameterestimation,optimalparametersynthesis

• Multipleuses– automation ofpersonalisedmedicalintervention– devicesafetyassurance,fortesting– reproducetheuniquecharacteristicsforauthentication

• FocusonECGbaseddevices– pacemakermodels,heartmodels,syntheticECGs– futureworkonanxietymonitoringandcontrol

23

Pacemakerverification/optimisation• Hybridmodel-basedframework

– timedautomatamodelforpacemaker– hybridheartmodelsinSimulink(non-linearODEs)

• Properties– (basicsafety)maintain60-100beatsperminute

– optimisationofenergyusage &cardiacoutput[HSB’16][HSCC’16]

– in-silicoanalysisofrate-adaptivepacemakers[ICHI’14]

– hardwareintheloop[EMBC’15]24

DNAcomputationchallenge• Moore’slaw,henceneedtomakedevicessmaller…• DNAcomputation,directlyatthemolecularlevel

– DNAlogiccircuitdesigns&programmablenanorobotics– asynchronousDNAcircuitdesigns[DNA’16]

• Manyapplicationsenvisaged– e.g.bio-sensing,pointofcarediagnostics,…

• Applyquantitativeverificationandsynthesisto– finddesignflawsinDNAcomputingdevices[JRSI’12]– analyse reliabilityandperformanceofmolecularwalkers– automaticallysynthesise reactionrates toguarantee aspecifiedlevelofreliability

25

DNAnanostructures• DNAorigami [Rothemund,Nature 2006]

– DNAcanself-assembleintostructures– “molecularIKEA?”– programmableself-assembly(canformtiles,nanotubes,boxesthatcanopen,etc.)

26

DNAwalkercircuits• ComputingwithDNAwalkers[NatComp’14]

– branchingtrackslaidoutonDNAorigamitile

– startsat‘initial’,signalswhenreaches‘final’

– cancontrol‘left’/’right’decision

– anyBooleanfunction

• Parametersynthesisofrates– forguaranteedreliabilitylevel[CMSB’14]

27

DNAorigamitiles• DNAorigamitiles

• Aim:understandhowtocontrolthefoldingpathway– formulateanabstractMarkovchainmodel– yieldspredictions;performarangeofexperiments,consistentwithpredictions[Nature’15]

28

50nm

Conclusions• Probabilisticmodelchecking&PRISM

– 15yearssincefirstofficialtoolrelease– significantadvancesinunderlyingtheory&technologies– successfullydeployedinmanyapplicationdomains

• Manyresearchchallengesandapplicationsahead– verification,synthesis,learning,trust,cognitivemodels,…– autonomoussystems,DNAcomputing,personalisedwearable/implantabledevices,…

http://www.prismmodelchecker.org/

29

Acknowledgements• Contributors (toPRISM&itsunderlyingtheory)

– Aistis Simaitis,AlbertoPuggelli,AlessandroBruni,Alexandru Mereacre,AlistairJohnStrachan,AndrejTokarčík,AndrewHinton,AntonioPacheco,Archit Taneja,Ashutosh Trivedi,BenoitBarbot,BrunoLacerda,CarlosBederian,CharlesHarley,ChrisThachuk,ChristelBaier,ChristianDehnert,ChristianvonEssen,ChristopherZiegler,Chunyan Mu,ClemensWiltsche,DaveParker,ErnstMoritzHahn,FritsDannenberg,Fuzhi Wang,Ganindu Prabhashana,Gethin Norman,Håkan Younes,HolgerHermanns,Hongyang Qu,JanKřetínský,JensKatelaan,JeremySproston,JoachimKlein,JoachimMeyer-Kayser,Joost-PieterKatoen,KennethChan,KlausDraeger,Kousha Etessami,Lovejeet Singh,LuFeng,LucadeAlfaro,MarcinCopik,MarcoDiciolla,MariaSvorenova,MarkKattenbelt,MarkusSiegle,MartaKwiatkowska,MateuszUjma,MaximilianProbst,Mihalis Yannakakis,MikeArthur,MilanCeska,MosheVardi,MuhammadOmerSaeed,NickHawes,NicolaPaoletti,NicolasBasset,NicolasDelPiano,Nishan Kamaleson,PaoloBallarini,PedroD'Argenio,Qixia Yuan,Radu Calinescu,RashidMehmood,RobertoSegala,SebastianVermehren,SergioGiro,SteffenMärcker,StephenGilmore,Taolue Chen,Tingting Han,VincentNimal,Vojtěch Forejt,Xueyi Zou,YiZhang,ZakCohen,…

(andmanymorecollaboratorsoncasestudies&projects)

• Projectfunders

30