Model Checking - Simon Fraser University
Transcript of Model Checking - Simon Fraser University
![Page 2: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/2.jpg)
Technique limitations vs risks
● Most techniques have limited claims
![Page 3: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/3.jpg)
Technique limitations vs risks
● Most techniques have limited claims– Testing
● Show the program can behave correctly● Provide confidence for given criteria
![Page 4: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/4.jpg)
Technique limitations vs risks
● Most techniques have limited claims– Testing
● Show the program can behave correctly● Provide confidence for given criteria
– Dynamic Analysis● Helps find bugs● Collect information about programs
![Page 5: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/5.jpg)
Technique limitations vs risks
● Most techniques have limited claims– Testing
● Show the program can behave correctly● Provide confidence for given criteria
– Dynamic Analysis● Helps find bugs● Collect information about programs
– Code Review● Finds many surface level issues
![Page 6: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/6.jpg)
Technique limitations vs risks
● Most techniques have limited claims– Testing
● Show the program can behave correctly● Provide confidence for given criteria
– Dynamic Analysis● Helps find bugs● Collect information about programs
– Code Review● Finds many surface level issues
● Programs may exhibit subtle, hard to identify issues– Distributed file system integrity– Coordinating telephony
![Page 7: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/7.jpg)
Technique limitations vs risks
● Most techniques have limited claims– Testing
● Show the program can behave correctly● Provide confidence for given criteria
– Dynamic Analysis● Helps find bugs● Collect information about programs
– Code Review● Finds many surface level issues
● Programs may exhibit subtle, hard to identify issues– Distributed file system integrity– Coordinating telephony
● But what if these components are mission critical?!
![Page 8: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/8.jpg)
Proving properties of programs
● If a particular property is mission critical,it may be worth proving that the property holds
![Page 9: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/9.jpg)
Proving properties of programs
● If a particular property is mission critical,it may be worth proving that the property holds– This often has a higher cost than applying other tools
![Page 10: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/10.jpg)
Proving properties of programs
● If a particular property is mission critical,it may be worth proving that the property holds– This often has a higher cost than applying other tools– The ROI for mission critical infrastructure can pay off
![Page 11: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/11.jpg)
Proving properties of programs
● If a particular property is mission critical,it may be worth proving that the property holds– This often has a higher cost than applying other tools– The ROI for mission critical infrastructure can pay off
● What kinds of properties would be interesting?
![Page 12: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/12.jpg)
Proving properties of programs
● If a particular property is mission critical,it may be worth proving that the property holds– This often has a higher cost than applying other tools– The ROI for mission critical infrastructure can pay off
● What kinds of properties would be interesting?– Safety – Something bad never happens
![Page 13: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/13.jpg)
Proving properties of programs
● If a particular property is mission critical,it may be worth proving that the property holds– This often has a higher cost than applying other tools– The ROI for mission critical infrastructure can pay off
● What kinds of properties would be interesting?– Safety – Something bad never happens– Liveness – Something good eventually happens
![Page 14: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/14.jpg)
Proving properties of programs
● If a particular property is mission critical,it may be worth proving that the property holds– This often has a higher cost than applying other tools– The ROI for mission critical infrastructure can pay off
● What kinds of properties would be interesting?– Safety – Something bad never happens– Liveness – Something good eventually happens
Some things should always be true (invariants),while others should eventually be true.
![Page 15: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/15.jpg)
Proving properties of programs
● If a particular property is mission critical,it may be worth proving that the property holds– This often has a higher cost than applying other tools– The ROI for mission critical infrastructure can pay off
● What kinds of properties would be interesting?– Safety – Something bad never happens– Liveness – Something good eventually happens
Some things should always be true (invariants),while others should eventually be true.
● Model checking is one such tool for provingthese properties
![Page 16: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/16.jpg)
Model Checking Overview
● Model checking is an automated technique for proving properties of finite state systems
![Page 17: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/17.jpg)
Model Checking Overview
● Model checking is an automated technique for proving properties of finite state systems
SystemDesign
Implementation
![Page 18: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/18.jpg)
Model Checking Overview
● Model checking is an automated technique for proving properties of finite state systems
SystemDesign
Implementation
FormalAbstractModel
![Page 19: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/19.jpg)
Model Checking Overview
● Model checking is an automated technique for proving properties of finite state systems
SystemDesign
Implementation
FormalProperty
Specification
FormalAbstractModel
![Page 20: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/20.jpg)
Model Checking Overview
● Model checking is an automated technique for proving properties of finite state systems
SystemDesign
Implementation
FormalProperty
Specification
FormalAbstractModel
Verifier
![Page 21: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/21.jpg)
Model Checking Overview
● Model checking is an automated technique for proving properties of finite state systems
SystemDesign
Implementation
FormalProperty
Specification
FormalAbstractModel
Proof
Verifier
![Page 22: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/22.jpg)
Model Checking Overview
● Model checking is an automated technique for proving properties of finite state systems
SystemDesign
Implementation
FormalProperty
Specification
FormalAbstractModel
Proof Counterexample
Verifier
![Page 23: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/23.jpg)
Model Checking Overview
● Model checking is an automated technique for proving properties of finite state systems
SystemDesign
Implementation
FormalProperty
Specification
FormalAbstractModel
Proof Counterexample ?
Verifier
![Page 24: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/24.jpg)
How can we specify the model?
● Often written in a formal specification language– temporal logic (CTL, LTL), Alloy, TLA, ...
![Page 25: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/25.jpg)
How can we specify the model?
● Often written in a formal specification language– temporal logic (CTL, LTL), Alloy, TLA, ...
● Need to express the finite states & transitions
![Page 26: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/26.jpg)
How can we specify the model?
● Often written in a formal specification language– temporal logic (CTL, LTL), Alloy, TLA, ...
● Need to express the finite states & transitions– (Oven example from Edmund Clarke)
Start Close Heat Error
![Page 27: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/27.jpg)
How can we specify the model?
● Often written in a formal specification language– temporal logic (CTL, LTL), Alloy, TLA, ...
● Need to express the finite states & transitions– (Oven example from Edmund Clarke)
¬ Start¬ Close¬ Heat¬ Error
![Page 28: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/28.jpg)
How can we specify the model?
● Often written in a formal specification language– temporal logic (CTL, LTL), Alloy, TLA, ...
● Need to express the finite states & transitions– (Oven example from Edmund Clarke)
Start¬ Close¬ HeatError
¬ Start¬ Close¬ Heat¬ Error
¬ StartClose
¬ Heat¬ Error
![Page 29: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/29.jpg)
How can we specify the model?
● Often written in a formal specification language– temporal logic (CTL, LTL), Alloy, TLA, ...
● Need to express the finite states & transitions– (Oven example from Edmund Clarke)
Start¬ Close¬ HeatError
StartClose
¬ HeatError
¬ Start¬ Close¬ Heat¬ Error
¬ StartClose
¬ Heat¬ Error
StartClose
¬ Heat¬ Error
![Page 30: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/30.jpg)
How can we specify the model?
● Often written in a formal specification language– temporal logic (CTL, LTL), Alloy, TLA, ...
● Need to express the finite states & transitions– (Oven example from Edmund Clarke)
Start¬ Close¬ HeatError
StartClose
¬ HeatError
¬ Start¬ Close¬ Heat¬ Error
¬ StartClose
¬ Heat¬ Error
StartClose
¬ Heat¬ Error
StartCloseHeat
¬ Error
![Page 31: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/31.jpg)
How can we specify the model?
● Often written in a formal specification language– temporal logic (CTL, LTL), Alloy, TLA, ...
● Need to express the finite states & transitions– (Oven example from Edmund Clarke)
Start¬ Close¬ HeatError
StartClose
¬ HeatError
¬ Start¬ Close¬ Heat¬ Error
¬ StartClose
¬ Heat¬ Error
StartClose
¬ Heat¬ Error
¬ StartCloseHeat
¬ Error
StartCloseHeat
¬ Error
![Page 32: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/32.jpg)
How can we specify the model?
● Often written in a formal specification language– temporal logic (CTL, LTL), Alloy, TLA, ...
● Need to express the finite states & transitions– (Oven example from Edmund Clarke)
Start¬ Close¬ HeatError
StartClose
¬ HeatError
¬ Start¬ Close¬ Heat¬ Error
¬ StartClose
¬ Heat¬ Error
StartClose
¬ Heat¬ Error
¬ StartCloseHeat
¬ Error
StartCloseHeat
¬ Error
![Page 33: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/33.jpg)
How can we specify the model?
● Often written in a formal specification language– temporal logic (CTL, LTL), Alloy, TLA, ...
● Need to express the finite states & transitions– (Oven example from Edmund Clarke)
Start¬ Close¬ HeatError
StartClose
¬ HeatError
¬ Start¬ Close¬ Heat¬ Error
¬ StartClose
¬ Heat¬ Error
StartClose
¬ Heat¬ Error
¬ StartCloseHeat
¬ Error
StartCloseHeat
¬ Error
![Page 34: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/34.jpg)
● Often written in a formal specification language– temporal logic (CTL, LTL), Alloy, TLA, ...
● Need to express the finite states & transitions– (Oven example from Edmund Clarke)
How can we specify the model?
Start¬ Close¬ HeatError
StartClose
¬ HeatError
¬ Start¬ Close¬ Heat¬ Error
¬ StartClose
¬ Heat¬ Error
StartClose
¬ Heat¬ Error
¬ StartCloseHeat
¬ Error
StartCloseHeat
¬ Error
StartCloseHeatError
?
![Page 35: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/35.jpg)
● Often written in a formal specification language– temporal logic (CTL, LTL), Alloy, TLA, ...
● Need to express the finite states & transitions– (Oven example from Edmund Clarke)
How can we specify the model?
Start¬ Close¬ HeatError
StartClose
¬ HeatError
¬ Start¬ Close¬ Heat¬ Error
¬ StartClose
¬ Heat¬ Error
StartClose
¬ Heat¬ Error
¬ StartCloseHeat
¬ Error
StartCloseHeat
¬ Error
StartCloseHeatError
?
![Page 36: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/36.jpg)
How can we specify properties?
● Often in the same language, inspired by temporal logic
![Page 37: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/37.jpg)
How can we specify properties?
● Often in the same language, inspired by temporal logic
● Temporal constraints help express propertiesparticularly interesting to concurrent and distributed systems
![Page 38: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/38.jpg)
How can we specify properties?
● Often in the same language, inspired by temporal logic
● Temporal constraints help express propertiesparticularly interesting to concurrent and distributed systems– e.g. The oven doesn’t heat up until the door is closed
![Page 39: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/39.jpg)
How can we specify properties?
● Often in the same language, inspired by temporal logic
● Temporal constraints help express propertiesparticularly interesting to concurrent and distributed systems– e.g. The oven doesn’t heat up until the door is closed
● Temporal constraints for a proposition p:
![Page 40: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/40.jpg)
How can we specify properties?
● Often in the same language, inspired by temporal logic
● Temporal constraints help express propertiesparticularly interesting to concurrent and distributed systems– e.g. The oven doesn’t heat up until the door is closed
● Temporal constraints for a proposition p:– p will hold eventually in the future
![Page 41: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/41.jpg)
How can we specify properties?
● Often in the same language, inspired by temporal logic
● Temporal constraints help express propertiesparticularly interesting to concurrent and distributed systems– e.g. The oven doesn’t heat up until the door is closed
● Temporal constraints for a proposition p:– p will hold eventually in the future– p holds in all future states
![Page 42: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/42.jpg)
How can we specify properties?
● Often in the same language, inspired by temporal logic
● Temporal constraints help express propertiesparticularly interesting to concurrent and distributed systems– e.g. The oven doesn’t heat up until the door is closed
● Temporal constraints for a proposition p:– p will hold eventually in the future– p holds in all future states– p holds in the next state
![Page 43: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/43.jpg)
How can we specify properties?
● Often in the same language, inspired by temporal logic
● Temporal constraints help express propertiesparticularly interesting to concurrent and distributed systems– e.g. The oven doesn’t heat up until the door is closed
● Temporal constraints for a proposition p:– p will hold eventually in the future– p holds in all future states– p holds in the next state– p holds until another proposition q holds
![Page 44: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/44.jpg)
Traffic Lights
● Traffic lights are a common application of safety critical embedded systems
![Page 45: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/45.jpg)
Traffic Lights
● Traffic lights are a common application of safety critical embedded systems
Yellow
Red
Green
Waiting
![Page 46: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/46.jpg)
Traffic Lights
● Traffic lights are a common application of safety critical embedded systems
● Interesting properties?Yellow
Red
Green
Waiting
![Page 47: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/47.jpg)
Traffic Lights
● Traffic lights are a common application of safety critical embedded systems
● Interesting properties– The light is green infinitely often
Yellow
Red
Green
Waiting
![Page 48: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/48.jpg)
Traffic Lights
● Traffic lights are a common application of safety critical embedded systems
● Interesting properties– The light is green infinitely often
– A red light does not immediatelybecome green
– ...
Yellow
Red
Green
Waiting
![Page 49: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/49.jpg)
Traffic Lights
● Traffic lights are a common application of safety critical embedded systems
● Interesting properties– The light is green infinitely often
– A red light does not immediatelybecome green
– ...
● You can also specify lights at an intersection as a distributed system & check the consistency!
Yellow
Red
Green
Waiting
![Page 50: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/50.jpg)
Do people actually use it?
● Aerospace● Hardware● Critical infrastructure providers (including Amazon)● Microsoft holds internal (& external) lectures on it
![Page 51: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/51.jpg)
Do people actually use it?
● Aerospace● Hardware● Critical infrastructure providers (including Amazon)● Microsoft holds internal (& external) lectures on it
Amazon’s experience (Using TLA+)
● Now used by several teams within AWS
● Each system has a 1-2KLOC TLA+ specification
● Detected several internal issues before they struck
![Page 52: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/52.jpg)
Do people actually use it?
● Aerospace● Hardware● Critical infrastructure providers (including Amazon)● Microsoft holds internal (& external) lectures on it
Amazon’s experience (Using TLA+)
● Now used by several teams within AWS
● Each system has a 1-2KLOC TLA+ specification
● Detected several internal issues before they struck
It is increasingly desirable for platform providers
![Page 53: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/53.jpg)
What does TLA+ look like?
● Let’s walk through an example...
![Page 54: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/54.jpg)
Summary
● Model checking can be an excellent way of proving properties about programs.
![Page 55: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/55.jpg)
Summary
● Model checking can be an excellent way of proving properties about programs.
● While it requires more effort and cost, it can prevent critical issues.
![Page 56: Model Checking - Simon Fraser University](https://reader033.fdocuments.net/reader033/viewer/2022051909/6284a36d67348359694be945/html5/thumbnails/56.jpg)
Summary
● Model checking can be an excellent way of proving properties about programs.
● While it requires more effort and cost, it can prevent critical issues.
● One such platform for model checking is TLA+.