1 Formal Methods Quick Tutorial Ricky W. Butler Oct 22, 2003 .
-
Upload
nathaniel-bruce -
Category
Documents
-
view
217 -
download
0
Transcript of 1 Formal Methods Quick Tutorial Ricky W. Butler Oct 22, 2003 .
1
Formal MethodsQuick Tutorial
Ricky W. Butler
Oct 22, 2003
http://shemesh.larc.nasa.gov/fm
2
Outline
• Motivation Motivation (3 minutes)(3 minutes)
• Formal Methods by way of exampleFormal Methods by way of example
— Theorem Proving Theorem Proving (12 (12 minutes)minutes)
— Model Checking state machines Model Checking state machines (12 (12
minutes)minutes)
• A Few More Thoughts A Few More Thoughts (3 minutes)(3 minutes)
3
“There are no stupid questions”
4
Motivation
Say, I think I see where we went off. Isn’t eight times seven fifty-six?Say, I think I see where we went off. Isn’t eight times seven fifty-six?
5
• Increased Capabilities– Almost any breakthrough in any field will depend on advanced
computer systems– And software is where most of the complexity goes
• Software Costs Dominate– Development costs (More than half of the non-recurring development
of the Boeing 777)– Integration (only way of testing it is all-up simulation)– Certification (based on process, not product)
• Major Safety Concerns– Reliability (dealing with hardware failures)– Correctness (no faults in its own design)– Man-machine interaction (issue in majority of recent crashes)– Certification (complexity forces FAA to rely on DERs)
Software Underpins Everything In Aviation
6
x: VAR nat B: VAR bool f(x,B): nat = IF x > 30 AND B THEN x*x ELSE 90*x ENDIF
g(x): nat = IF x <= 30 THEN 2*x ELSE 0 ENDIF
h(B): nat = IF B THEN 60 ELSE 0 ENDIF
x
B
f(x,B)
g(x)
h(B)
System
safe_prop: THEOREM f(x,B) >= g(x) * h(B)
Example System
7
Design Verification
In the beginning, there was Simulation:
•Build a model•Feed IT inputs•Analyze results
8
x: VAR nat B: VAR bool f(x,B): nat = IF x > 30 AND B THEN x*x ELSE 90*x ENDIF
g(x): nat = IF x <= 30 THEN 2*x ELSE 0 ENDIF
h(B): nat = IF B THEN 60 ELSE 0 ENDIF
safe_prop: THEOREM f(x,B) >= g(x) * h(B)
Verification by Simulation
x B f g h safe_prop ----------- ---------------------------- -------- 0 T 0 0 60 TRUE 10 F 1200 20 0 TRUE 50 T 2500 0 60 TRUE 30 F 3600 60 0 TRUE 30 T 2700 60 60 FALSE
9
x: VAR nat B: VAR bool f(x,B): nat = IF x > 30 AND B THEN x*x ELSE 90*x ENDIF g(x): nat = IF x <= 30 THEN 2*x ELSE 0 ENDIF h(B): nat = IF B THEN 60 ELSE 0 ENDIF
Formal Verification
safe_prop: THEOREM f(x,B) >= g(x) * h(B)
IF x > 30 AND B THEN x*x ELSE 90*x ENDIF >= g(x) * h(B)
BNOT B
IF x > 30 THEN x*x ELSE 90*x ENDIF >= g(x) * h(B) 90*x >= g(x) * h(B)
90*x >= g(x) * 0
90*x >= 0
10
x: VAR nat B: VAR bool f(x,B): nat = IF x > 30 AND B THEN x*x ELSE 90*x ENDIF g(x): nat = IF x <= 30 THEN 2*x ELSE 0 ENDIF h(B): nat = IF B THEN 60 ELSE 0 ENDIF
Formal Verification (cont.)
B
IF x > 30 THEN x*x ELSE 90*x ENDIF >= g(x) * h(B)
x > 30 x <= 30
x*x >= g(x) * h(B) 90*x >= g(x) * h(B)
x*x >= 0 * h(B)
x*x >= 0
90*x >= 2*x * h(B)
90*x >= 2*x * 60
90*x >= 120*x
11
demo_incorrect: THEORYBEGIN
x: VAR nat B: VAR bool f(x,B): nat = IF x > 30 AND B THEN x*x ELSE 90*x ENDIF
g(x): nat = IF x <= 30 THEN 2*x ELSE 0 ENDIF
h(B): nat = IF B THEN 60 ELSE 0 ENDIF
safe_prop: THEOREM f(x,B) >= g(x) * h(B)
END demo_incorrect
safe_prop :
|-------{1} FORALL (B: bool, x: nat): f(x, B) >= g(x) * h(B)
Rule? (grind)
f rewrites f(x, B) to IF x > 30 AND B THEN x * x ELSE 90 * x ENDIFg rewrites g(x) to IF x <= 30 THEN 2 * x ELSE 0 ENDIFh rewrites h(B) to IF B THEN 60 ELSE 0 ENDIFTrying repeated skolemization, instantiation, and if-lifting,this simplifies to: safe_prop :
{-1} x!1 >= 0{-2} B!1 |-------{1} x!1 > 30{2} 90 * x!1 >= 120 * x!1
Rule?
PVS SPECIFICATION OUTPUT FROM PVS THEOREM PROVER
Using a “Mechanical” Proof Checker Using a “Mechanical” Proof Checker (PVS)(PVS)
12
demo_correct: THEORYBEGIN
x: VAR nat B: VAR bool f(x,B): nat = IF x > 30 AND B THEN x*x ELSE 90*x ENDIF
g(x): nat = IF x <= 30 THEN 2*x ELSE 0 ENDIF
h(B): nat = IF B THEN 40 ELSE 0 ENDIF
safe_prop: LEMMA f(x,B) >= g(x) * h(B)
END demo_correct
safe_prop :
|-------{1} FORALL (B: bool, x: nat): f(x, B) >= g(x) * h(B)
Rule? (grind)
f rewrites f(x, B) to IF x > 30 AND B THEN x * x ELSE 90 * x ENDIFg rewrites g(x) to IF x <= 30 THEN 2 * x ELSE 0 ENDIFh rewrites h(B) to IF B THEN 40 ELSE 0 ENDIFTrying repeated skolemization, instantiation, and if-lifting,Q.E.D.
Run time = 0.35 secs.Real time = 3.33 secs.
CORRECTED PVS SPECIFICATION
OUTPUT FROM PVS THEOREM PROVER
Using a “Mechanical” Proof Checker (PVS)Using a “Mechanical” Proof Checker (PVS)
13
The Key Difference
• Simulation/Testing only explores a small part of the state space
• Formal Verification explores the entire state space
The only way you can assure yourself that there are no safety-relevant bugs hidden in some dark corner of your software is to explore the entire state space.
14
Can We Automate Verification?
Program
SafetyProps
Verified!
Buggy!
The Big V or
Counter Example
15
Kurt Goedel (1906-1978)
Answer: NO!
16
• Algebra (in general) is undecidable(i.e. There is no terminating algorithm that can
determine whether a formula is true) But some sub-theories are decidable (e.g.
Presburger arithmetic): develop automated decision procedures
Why Can’t We Automate Verification?
• While loops in a program lead to induction proofs(Discovery of the induction invariant involves creativity: n: P(n) requires P(n) Q1(n) … Qj(n))
• Modeling the environment that the program interacts with often involves continuous mathematics (e.g. calculus, trig) hybrid models
• Finite-state models can be automatically analyzed via model-checking (execution times can be exorbitant)
17
Proof CheckersProof Checkers
But we do use Proof Checkers! But we do use Proof Checkers! (aka Theorem Provers)(aka Theorem Provers)
-- because we make too many -- because we make too many mistakesmistakes
Why then do you call it a Why then do you call it a “theorem prover” if it can only “theorem prover” if it can only check? check?
Because it does help, it just Because it does help, it just can’t do the whole job.can’t do the whole job.
18
State Machine Analysis
Event From Toex x A - 10 ( x+10 , y )ey y B - 10 ( x , y+10)em x > y AND y > 0 ( x – y , y )em y > x AND x > 0 ( x , y - x )es ( y , x )
State = (x: nat, y: nat)
Events = {ex, ey, em , es}
19
State Machine Analysis:
0, 0
ex
10,0 20,0 30,0
10,10 20,100,10 30,10
ex ex
exexex
ey ey ey ey
0,20
es
es
em em
PROVE: For all reachable states (x,y) from (0,0):
x A and y B
(Proving Invariants)
20
State Machine Analysis:
• Theorem proving– deduction– human directed
• Model checking– exhaustive search– automatic– state space must be finite
(Proving Invariants)
21
State Machine Analysis:
0, 0ex
10,0 20,0 30,0
10,10 20,100,10 30,10
ex ex
exexex
ey ey ey ey
0,20
es
es
em em
PROVE: for all reachable states (x,y) from (0,0):
x A and y B
Counter-example: A = 30, B = 20
(0,0) ----> (10,0) ----> (20,0) ----> (30,0) ----> (0,30)ex ex ex es
(Proving Invariants)
22
State Machine Analysis:
INVARIANT: For all reachable states (x,y) from (0,0) that
x max(A,B) and y max(A,B)
Event From Toex x A - 10 ( x+10 , y )ey y B - 10 ( x , y+10)em x > y AND y > 0 ( x – y , y )em y > x AND x > 0 ( x , y - x )es ( y , x )
Proof Approach: Assume invariant holds in current state. Show eachtransition preserves the invariant. By induction true for all reachable states.
(Proving Invariants)
23
MODULE mainVAR x : 0..50; y : 0..50;
event: {ex,ey,em};
ASSIGN
init(x) := 0; init(y) := 0;
next(x) := case event = ex & x <= AA - 10 : x + 10; event = ey & y <= BB - 10 : x; event = em & x > y & y > 0 : x - y; event = em & y > x & x > 0 : x; 1 : y; esac; next(y) := case event = ex & x <= AA - 10 : y; event = ey & y <= BB - 10 : y + 10; event = em & x > y & y > 0 : y; event = em & y > x & x > 0 : y - x; 1 : x; esac;
SMV Version of State Machine
24
0
50
100
150
200
250
300
0..50 0..100 0..150 0..200 0..250
SMV Execution Times
Range of State Space Variables
Exe
cutio
n T
imes
(se
cs)
25
26
DEFINE AA := 40; BB := 50;
invariant_1 := AG (x <= AA & y <= BB); invariant_2 := AG (x <= max_AB & y <= max_AB);
-- specification invariant_1 is false-- as demonstrated by the following execution sequence
state 1.1:x = 0y = 0event = em
state 1.2:event = ey
state 1.3:y = 10
state 1.4:y = 20
state 1.5:y = 30
state 1.6:y = 40
state 1.7:y = 50event = em
state 1.8:x = 50y = 0
-- specification invariant_2 is true
Output From SMV Model Checker
27
Different perspectives on the Joys of Theorem Proving
Ms. Miller, tell the Ms. Miller, tell the verification team I will be a verification team I will be a little late with that proof.little late with that proof.
Just two more lemmas and the proof of the KB3D CD&R algorithm will be complete!
28
What Does It Look Like In a Theorem Prover?
x: VAR posreal % positive real numbern: VAR posnat % positive natural number
Y = IF n > 15 THEN x ELSE n * x ENDIF
Z = IF x 10 AND n 10 THEN 2 * x / n ELSE 10 / x ENDIF
x
n
YSystem
safe_prop: THEOREM Y * Z <= 200
Z
29
Executable
Code
Low-level design
High-level design
Requirements
Test cases
Coverage
Reviews
Traditional V&V
FM in development process
sizesize
101066 statesstates
10101010 statesstates
1010100100 statesstates
1010100100 100100
statesstates (i.e. 10 million lines code)(i.e. 10 million lines code)
proofsproofs
deeper,conceptual
shallow,thousands of cases
Programming Programming language such as language such as C++ have C++ have complex, ill-complex, ill-defined semantics defined semantics
30
Formal Methods Does Not Bring Perfection!
Requirements
proof
proof
proof
High Level Design
Detailed Design
Implementation
intendedfunctionality
physical devices
• peer review• simulation• testing
• peer review• simulation• testing
Are the component models accurate?
31
You guys are both my witnesses…He insinuated that ZFC set theory is superior to type theory!
Warning: some of our team members are a little Warning: some of our team members are a little sensitivesensitive
32
The Research Challenge
• Formal Methods are not a silver bullet:– Must be applied early in life cycle---not a band aid– Involves a significant learning curve– Each new application requires the development of new
models and analysis techniques– Cost of using formal methods still high for some
applications.• The software engineering community is dominated by the
process management viewpoint (i.e., that careful management of software development process is the key to quality)
• Historically, disasters must occur before major changes are made.
33
EXTRA SLIDESEXTRA SLIDES
34
What Does It Look Like In a Theorem Prover?
sys_lem:
{-1} Y = IF n > 15 THEN x ELSE n * x ENDIF{-2} Z = IF x <= 10 AND n <= 10 THEN 2 * x / n ELSE 10 / x ENDIF |-------[1] Y * Z <= 200
Rule? (case "n > 15")
Case splitting on n > 15, this yields 2 subgoals
35
What Does It Look Like In a Theorem Prover?
sys_lem.1
{-1} n > 15[-2] Y = IF n > 15 THEN x ELSE n * x ENDIF[-3] Z = IF x <= 10 AND n <= 10 THEN 2 * x / n ELSE 10 / x ENDIF |-------[1] Y * Z <= 200
Rule? (assert)Simplifying, rewriting, and recording with decision procedures,
This completes the proof of sys_lem.1
What Happened? Well, assuming n > 15 we have:
Y = xZ = 10 / x
36
What Does It Look Like In a Theorem Prover?
sys_lem.2
[-1] Y = IF n > 15 THEN x ELSE n * x ENDIF[-2] Z = IF x <= 10 AND n <= 10 THEN 2 * x / n ELSE 10 / x ENDIF{-3} n <= 15 |-------[1] Y * Z <= 200
Rule? (assert)Simplifying, rewriting, and recording with decision procedures,
give us the other subgoal give us the other subgoal
37
What Does It Look Like In a Theorem Prover?
sys_lem.2 {-1} Y = n * x[-2] Z = IF x <= 10 AND n <= 10 THEN 2 * x / n ELSE 10 / x ENDIF[-3] n <=15 |-------[1] Y * Z <= 200
Rule? (replace -1) (hide -1)
sys_lem.2
[-1] Z = IF x <= 10 AND n <= 10 THEN 2 * x / n ELSE 10 / x ENDIF[-2] n <=15 |-------[1] n*x* Z <= 200
38
What Does It Look Like In a Theorem Prover?
Rule? (case "x <= 10 AND n <= 10")
Case splitting on x <= 10 AND n <= 10, this yields 2 subgoals
sys_lem.2.1
{-1} x <= 10 AND n <= 10[-2] Z = IF x <= 10 AND n <= 10 THEN 2 * x / n ELSE 10 / x ENDIF[-3] n <=15 |-------[1] n * x * Z <= 200
Rule? (replace -1) Replacing using formula -1,
this simplifies to
39
What Does It Look Like In a Theorem Prover?
sys_lem.2.1
{-1} Z = 2 * x / n[-2] n <= 10[-3] x <= 10|-------[1] n * x * Z <= 200
Rule? (replace -1) (hide -1)
[-1] n <= 10[-2] x <= 10 |-------{1} n * x * (2 * x / n) <= 200
Rule? (assert) hit it with the hammer
40
What Does It Look Like In a Theorem Prover?
sys_lem.2.1
[-1] n <= 15[-2] x <= 10 |-------{1} n * x * (2 * x / n) <= 200
Rule? (mult-ineq -2 -2)
{-1} x * x <= 10 * 10[-2] n <= 15[-3] x <= 10 |-------{1} n * x * (2 * x / n) <= 200
Rule? (assert)Simplifying, rewriting, and recording with decision procedures,This completes the proof of sys_lem.2.1.
But nothing happens!
hit it with the hammer again
41
What Does It Look Like In a Theorem Prover?
[-1] Z = IF x <= 10 AND n <= 10 THEN 2 * x / n ELSE 10 / x ENDIF{-2} NOT (x <= 10 AND n <= 10){-3} n <= 15 |-------[1] n * x * Z <= 200
Rule? (replace -1) (hide -1)
{-1} x >10 OR n > 10{-2} n <= 15 |-------[1] n * x * 10 / x <= 200
Rule? (assert)
This completes the proof of sys_lem2.2.This completes the proof of sys_lem.2.Q.E.D.
42
Different Verification Challenges
c(x,y) = y /[Sin2(x) + Cos2(x)]
x
yc(x,y)System
trigonometry
c(x,y) =
y
i
i
y
x
0
Need to formally reason about
series
c(x,y) = d/dx (x3y + y5)/xcalculus
Different Domains -->