Program Verification and Synthesis using Templates over Predicate Abstraction
description
Transcript of Program Verification and Synthesis using Templates over Predicate Abstraction
![Page 1: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/1.jpg)
Program Verification and Synthesisusing Templates
over Predicate Abstraction
Saurabh Srivastava #
Sumit Gulwani *Jeffrey S. Foster #
# University of Maryland, College Park* Microsoft Research, Redmond
On leave advisor: Mike Hicks
![Page 2: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/2.jpg)
Sorted
Verification: Proving Programs Correct
Selection Sort:
i jfind min
i jfind min
i jfind min
…
![Page 3: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/3.jpg)
Verification: Proving Programs Correct
Selection Sort:
Sorted
i jfind min
i jfind min
i jfind min
…
for i = 0..n{
for j = i..n{
find min index
}if (min != i) swap A[i] with
A[min]}
![Page 4: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/4.jpg)
Selection Sort:
Verification: Proving Programs Correct
for i = 0..n{
for j = i..n{
find min index
}if (min != i) swap A[i] with
A[min]}Prove the following:
• Sortedness• Output array is non-decreasing
• Permutation• Output array contains all elements of the input• Output array contains only elements from the
input• Number of copies of each element is the same
∀k1,k2 : 0≤k1<k2<n => A[k1] ≤A[k2]
∀k∃j : 0≤k<n => Aold[k]=A[j] & 0≤j<n
∀k∃j : 0≤k<n => A[k]=Aold[j] & 0≤j<n
∀n : n∊A => count(A,n) = count(Aold,n)
![Page 5: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/5.jpg)
Selection Sort:for i = 0..n{
for j = i..n{
find min index
}if (min != i) swap A[i] with
A[min]}
Verification: Proving Programs Correct
Outputarray
i<n
j<n
Find min index
if (min != i) swap A[i], A[min]
i:=0
j:=i
![Page 6: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/6.jpg)
Selection Sort:
Outputarray
i<n
j<n
Find min index
if (min != i) swap A[i], A[min]
i:=0
j:=i
Verification: Proving Programs Correct
Iouter
IinnerSortednessPermutatio
n
Output state
trueInput stateWe know from work in
the ’70s (Hoare, Floyd, Dijkstra) that it is easy
to reason logically about simple paths
The difficult task is program state
(invariant) inference:
Iinner, Iouter, Input state, Output state
![Page 7: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/7.jpg)
Selection Sort:
Outputarray
i<n
j<n
Find min index
if (min != i) swap A[i], A[min]
i:=0
j:=i
Verification: Proving Programs Correct
SortednessPermutatio
n
Output state
trueInput stateWe know from work in
the ’70s (Hoare, Floyd, Dijkstra) that it is easy
to reason logically about simple paths
The difficult task is program state
(invariant) inference:
Iinner, Iouter, Input state, Output state
∀k1,k2 : 0≤k1<k2<n ∧ k1<i => A[k1] ≤A[k2]
i < j i ≤ min < n
∀k : 0≤k ∧ i≤k<j => A[min] ≤A[k]
∀k1,k2 : 0≤k1<k2<n ∧ k1<i => A[k1] ≤A[k2]
∀k1,k2 : 0≤k1<k2<n => A[k1] ≤A[k2]
Iouter
Iinner
![Page 8: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/8.jpg)
User Input
Program properties
that we can verify/prove
Full functionalcorrectness
No input Fully specified invariants
Good
Bad
Tolerable
Inferring program state (invariants)
![Page 9: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/9.jpg)
User Input
Program properties
that we can verify/prove
Full functionalcorrectness
No input Fully specified invariants
Mind-blowing
Undesirable
Live with current te
chnology
Inferring program state (invariants)
![Page 10: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/10.jpg)
Arbitraryquantificatio
nand
arbitrary abstraction
Alternatingquantification (∀∃)
Quantified facts (∀)
Quantifier-free facts
User Input
Program properties
that we can verify/prove
Full functionalcorrectness
No input Fully specified invariants
Inferring program state (invariants)
![Page 11: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/11.jpg)
Arbitraryquantificatio
nand
arbitrary abstraction
Alternatingquantification (∀∃)
Quantified facts (∀)
Quantifier-free facts
User Input
Program properties
that we can verify/prove
Full functionalcorrectness
No input Fully specified invariants
Undecidable
Inferring program state (invariants)
![Page 12: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/12.jpg)
Arbitraryquantificatio
nand
arbitrary abstraction
Alternatingquantification (∀∃)
Quantified facts (∀)
Quantifier-free facts
User Input
Program properties
that we can verify/prove
Full functionalcorrectness
No input Fully specified invariants
Abstract interpretationQuantifier-free
domain Cousot etc
Quantifier-free domain lifted to ∀
Gulwani etc
Best-effort inference for alternating
quantification (∀∃)Kovacs etc
Validating program pathsusing non-interactive
theorem provers/SMT solversHAVOC, Spec#, etc
Validationusing interactive theorem ProvingJahob
Inferring program state (invariants)
Constraint-basedQuantifier-free
domain Gulwani, Beyer etc
Shape analysisSLAyer etc
Program propertiesInference Validation
![Page 13: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/13.jpg)
Arbitraryquantificatio
nand
arbitrary abstraction
Alternatingquantification (∀∃)
Quantified facts (∀)
Quantifier-free facts
User Input
Program properties
that we can verify/prove
Full functionalcorrectness
No input Fully specified invariants
Magicaltechnique !!✗
Inferring program state (invariants)Program propertiesInference Validation
![Page 14: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/14.jpg)
Arbitraryquantificatio
nand
arbitrary abstraction
Alternatingquantification (∀∃)
Quantified facts (∀)
Quantifier-free facts
User Input
Program properties
that we can verify/prove
Full functionalcorrectness
No input Fully specified invariants
Template-basedinvariant inference
Inferring program state (invariants)Program propertiesInference Validation
![Page 15: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/15.jpg)
User Input
Full functionalcorrectness
No input
Template-basedinvariant inference
Small; intuitive input
Inferring program state (invariants)
InvariantTemplates Predicates
My task in this talk:Convince you that these small hints go a long way in what we can achieve with verification€
∀k1,k2 : (−)⇒ (−)
If you wanted to prove sortedness, i.e. each element is smaller than some others:
€
∀k1∃k2 : (−)⇒ (−)
If you wanted to prove permutation, i.e. for each element there exists another:
Example: Example:Take all variables and array elements in the program.Enumerate all pairs related by less than, greater than, equality etc.
![Page 16: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/16.jpg)
Key facilitators: (1) Templates• Templates
– Intuitive• Depending on the property being proved; form straight-forward
– Simple• Limited to the quantification and disjunction• No complicated error reporting, iterative refining required• Very different from full invariants used by some techniques
– Helps: We don‘t have to define insane joins etc.• Previous approaches: Domain D for the holes, construct D∀
• This approach: Work with only D• Catch: Holes have different status:
€
∀k1∃k2 : (−)⇒ (−)
€
∀k1,k2 : (−)⇒ (−)Unknown
holes
€
∀k1,k2 : (−)⇒ (−)
![Page 17: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/17.jpg)
Key facilitators: (2) Predicates• Predicate Abstraction
– Simple• Small fact hypothesis
– Nested deep under, the facts for the holes are simple• E.g., ∀k1,k2 : 0≤k1<k2<n => A[k1] ≤A[k2]
– Predicates 0≤k1, k1<k2, k2<n, and A[k1] ≤A[k2] are individually very simple
– Intuitive• Only program variables, array elements and relational operators
€
x opr (y opa c)Relational operator
<, ≤, >, ≥ …Arithmetic operator
+, - …
Program variables,array elements
Limitedconstants
– E.g., {i<j, i>j, i≤j, i≥j, i<j-1, i>j+1… }
– Example: Enumerate predicates of the form:
![Page 18: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/18.jpg)
VS3
• VS3: Verification and Synthesis using SMT Solvers
• User input– Templates– Predicates
• Set of techniques and tool implementing them
• Infers – Loop invariants– Weakest preconditions– Strongest postconditions
![Page 19: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/19.jpg)
What VS3 will let you do!
A. Infer invariants with arbitrary quantification and boolean structure– ∀: E.g. Sortedness
– ∀∃: E.g. Permutation
..∀k1,k2 k1 k2
€
≤
j....
Aold
∀k∃jAnew
k
Outputarray
i<n
j<n
Find min index
if (min != i) swap A[i], A[min]
i:=0
j:=i
![Page 20: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/20.jpg)
What VS3 will let you do!
k1 k2
€
≤
<
B. Infer weakest preconditions
Weakest conditions on input?
…
No Swap!
Outputarray
i<n
j<n
Find min index
if (min != i) swap A[i], A[min]
i:=0
j:=i Worst case behavior: swap every time it can
![Page 21: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/21.jpg)
Improves the state-of-art• Can infer very expressive invariants
• Quantification– E.g. ∀k∃j : (k<n) => (A[k]=B[j] ∧ j<n)
• Disjunction– E.g. ∀k : (k<0 ∨ k>n ∨ A[k]=0)
• Can infer weakest preconditions• Good for debugging: can discover worst case inputs• Good for analysis: can infer missing precondition
facts
![Page 22: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/22.jpg)
Verification part of the talk• Three fixed-point inference algorithms
– Iterative fixed-point• Least Fixed-Point (LFP)• Greatest Fixed-Point (GFP)
– Constraint-based (CFP)
• Weakest Precondition Generation
• Experimental evaluation
(LFP)
(GFP)
(CFP)
![Page 23: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/23.jpg)
Verification part of the talk• Three fixed-point inference algorithms
– Iterative fixed-point• Least Fixed-Point (LFP)• Greatest Fixed-Point (GFP)
– Constraint-based (CFP)
• Weakest Precondition Generation
• Experimental evaluation
(LFP)
(GFP)
(CFP)
![Page 24: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/24.jpg)
Outputsortedarray
i<n
j<n
Find min index
if (min != i) swap A[i], A[min]
i:=0
j:=i
Analysis Setup
post
pre
I1
I2
Loop headers (with invariants) split program into simple paths. Simple paths induce program constraints (verification conditions)
vc(pre,I1)
vc(I1,post) vc(I1,I2)
vc(I2,I2)vc(I2,I1)
true ∧ i=0 => I1
I1 ∧ i≥n => sorted array
I1 ∧ i<n ∧ j=i => I2
I1
I2
E.g. Selection Sort:
![Page 25: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/25.jpg)
I1 ∧ i≥n => ∀k1,k2 : 0≤k1<k2<n => A[k1] ≤A[k2]
Analysis Setup
post
pre
I1
I2
Loop headers (with invariants) split program into simple paths. Simple paths induce program constraints (verification conditions)
vc(pre,I1)
vc(I1,post) vc(I1,I2)
vc(I2,I2)vc(I2,I1)
true ∧ i=0 => I1
I1 ∧ i<n ∧ j=i => I2
Simple FOL formulae over I1,
I2!We will exploit this.
![Page 26: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/26.jpg)
Candidate Solution
Iterative Fixed-point: Overview
post
pre
I1
I2
vc(pre,I1)
vc(I1,post) vc(I1,I2)
vc(I2,I2)vc(I2,I1)
{ vc(pre,I1),vc(I1,I2) }
VCs that are not satisfied
✗✓
✓ ✓
✗
Values for invariants
<x,y>
![Page 27: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/27.jpg)
Iterative Fixed-point: Overview
post
pre
I1
I2
vc(pre,I1)
vc(I1,post) vc(I1,I2)
vc(I2,I2)vc(I2,I1)
Improve candidate– Pick unsat VC– Use theorem prover to
compute optimal change– Results in new candidates
<x,y> { … }
Set of candidates
Improve candidate
Candidate satisfies all
VCs
![Page 28: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/28.jpg)
Forward Iterative (LFP)
Forward iteration:
• Pick the destination invariant of unsat constraint to improve
• Start with <⊥,…,⊥>
post
pre
⊥
⊥
✓✓✓
✗
✓
![Page 29: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/29.jpg)
Forward Iterative (LFP)
Forward iteration:
• Pick the destination invariant of unsat constraint to improve
• Start with <⊥,…,⊥>
post
pre
a
⊥
✓
✓
✓ ✗✓
![Page 30: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/30.jpg)
Forward Iterative (LFP)
Forward iteration:
• Pick the destination invariant of unsat constraint to improve
• Start with <⊥,…,⊥>
post
pre
a
b✓
✓
✓ ✗✓
How do we compute these improvements?
`Optimal change’procedure
![Page 31: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/31.jpg)
Positive and Negative unknowns• Given formula Φ• Unknowns in have polarities: Positive/Negative
– Value of positive unknown stronger => Φ stronger– Value of negative unknown stronger => Φ weaker
• Optimal Soln to Φ – Maximally strong positives, maximally weak negatives
¬negative
u1
positive∨ u2 ) ∧ u3(
positive∀x : ∃y : Φ =
![Page 32: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/32.jpg)
Optimal Change• One approach (old way):
∀k1,k2 : 0≤k1<k2<n ∧ k1<i => A[k1] ≤A[k2] Lifted
domain D∀x1:=e1
∀k1,k2 : ?? => ??Under-approximate Over-
approximate
Very difficult tx, join, widen functionsrequired for the defn of domain D∀
Domain D: quantifier-free facts
![Page 33: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/33.jpg)
Optimal Change• Our approach (new way):
∀k1,k2 : 0≤k1<k2<n ∧ k1<i => A[k1] ≤A[k2]
x1:=e1
∀k1,k2 : ?? => ??
![Page 34: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/34.jpg)
Optimal Change• Our approach (new way):
∀k1,k2 : 0≤k1<k2<n ∧ k1<i => A[k1] ≤A[k2]
x1:=e1
∀k1,k2 : ??1 => ??2
x2:=e2
x3:=e3
∀k1,k2 : ??3 => ??4
Suppose ei does not refer
to xi
![Page 35: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/35.jpg)
Optimal Change• Our approach (new way):
∀k1,k2 : 0≤k1<k2<n ∧ k1<i => A[k1] ≤A[k2]
∧ x1=e1
∀k1,k2 : ??1 => ??2
∧ x2=e2
∧ x3=e3 ∀k1,k2 : ??3 => ??4
Suppose ei does not refer
to xi=>
![Page 36: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/36.jpg)
Optimal Change• Our approach (new way):
∀k1,k2 : 0≤k1<k2<n ∧ k1<i => A[k1] ≤A[k2]∧ x1=e1 ∧ x2=e2 ∧ x3=e3
∀k1,k2 : ??1 => ??2∀k1,k2 : ??3 => ??4
Find optimal solutions to unknowns
??1, ??2, ??3, ??4
=>
Unknowns have polarities therefore
obey a monotonicity
property
Trivial search will involve k x 2|Predicates|
queries to the SMT solver
Algorithmic Novelty:This polynomial sized
information can be aggregated efficiently to compute optimal
soln
Key idea: Instantiate some unknowns and
query SMT solver polynomial number of
times
![Page 37: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/37.jpg)
SMT Solver Query Time
1 10 100 1000 More0
5000
10000
15000
20000
25000
30000
35000
Time in Milliseconds (log scale)
Num
ber
of Q
ueri
es
413 queries30 queries
![Page 38: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/38.jpg)
Optimal Change• Our approach (new way):
∀k1,k2 : 0≤k1<k2<n ∧ k1<i => A[k1] ≤A[k2]∧ x1=e1 ∧ x2=e2 ∧ x3=e3
∀k1,k2 : ??1 => ??2∀k1,k2 : ??3 => ??4
Find optimal solutions to unknowns
??1, ??2, ??3, ??4
=>
Unknowns have polarities therefore
obey a monotonicity
property
Trivial search will involve k x 2|Predicates|
queries to the SMT solver
Algorithmic Novelty:This polynomial sized
information can be aggregated efficiently to compute optimal
soln
Key idea: Instantiate some unknowns and
query SMT solver polynomial number of
times
Efficient in practice
![Page 39: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/39.jpg)
Backward Iterative Greatest Fixed-Point (GFP)
Backward: Same as LFP except
• Pick the source invariant of unsat constraint to improve
• Start with <⊤, …,⊤>
![Page 40: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/40.jpg)
Constraint-based over Predicate Abstraction
post
pre
I1
I2
vc(pre,I1)
vc(I1,post) vc(I1,I2)
vc(I2,I2)vc(I2,I1)
![Page 41: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/41.jpg)
Constraint-based over Predicate Abstraction
vc(pre,I1)
vc(I1,post)vc(I1,I2)
vc(I2,I2)
vc(I2,I1)
pred(I1)
€
∀k : (−)⇒ (−) template
p1,p2…pr p1,p2…pr
unknown 1 unknown 2
predicates
b11,b2
1…br
1b1
2,b22…br
2 boolean indicators
I1Remember: VCs are FOL
formulae over I1, I2
If we find assignments
bji = T/F
Then we are done!
![Page 42: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/42.jpg)
Constraint-based over Predicate Abstraction
vc(pre,I1)
vc(I1,post)vc(I1,I2)
vc(I2,I2)
vc(I2,I1)
pred(I1)
pred(A) : A to unknown predicate
indicator variables
Remember: VCs are FOL
formulae over I1, I2
![Page 43: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/43.jpg)
Remember: VCs are FOL
formulae over I1, I2
Constraint-based over Predicate Abstraction
vc(pre,I1)
vc(I1,post)vc(I1,I2)
vc(I2,I2)
vc(I2,I1)
boolc(pred(I1), pred(I2))
pred(A) : A to unknown predicate
indicator variables
boolc(pred(I1))
boolc(pred(I2), pred(I1))
boolc(pred(I2)) Boolean constraintto satisfying soln
(SAT Solver)
Invariant soln∧
boolc(pred(I1))
Program constraintto boolean constraint
SAT formulae over predicate
indicators
Local reasoning Fixed-Point Computation
![Page 44: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/44.jpg)
Verification part of the talk• Three fixed-point inference algorithms
– Iterative fixed-point• Greatest Fixed-Point (GFP)• Least Fixed-Point (LFP)
– Constraint-based (CFP)
• Weakest Precondition Generation
• Experimental evaluation
(LFP)
(GFP)
(CFP)
![Page 45: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/45.jpg)
Computing maximally weak preconditions
• Iterative:– Backwards algorithm:
• Computes maximally-weak invariants in each step– Precondition generation using GFP:
• Output disjuncts of entry fixed-point in all candidates
• Constraint-based:– Generate one solution for precondition– Assert constraint that ensures strictly weaker pre– Iterate till unsat—last precondition generated is
maximally weakest
![Page 46: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/46.jpg)
Verification part of the talk• Three fixed-point inference algorithms
– Iterative fixed-point• Greatest Fixed-Point (GFP)• Least Fixed-Point (LFP)
– Constraint-based (CFP)
• Weakest Precondition Generation
• Experimental evaluation– Why experiment?– Problem space is undecidable—if not the worst case, is
the average, `difficult‘ case efficiently solvable?– We use SAT/SMT solvers—hoping that our instances will
not incur their worst case exponential solving time
(LFP)
(GFP)
(CFP)
![Page 47: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/47.jpg)
Implementation
C Program
Templates
Predicate Set
CFG
InvariantsPreconditions
Microsoft’s Phoenix Compiler Verification
Conditions
IterativeFixed-point
GFP/LFP
Constraint-based
Fixed-Point
Candidate Solutions
Boolean Constraint
Z3 SMT Solver
![Page 48: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/48.jpg)
Verifying Sorting Algorithms• Benchmarks
– Considered difficult to verify– Require invariants with quantifiers– Sorting, ideal benchmark programs
• 5 major sorting algorithms– Insertion Sort, Bubble Sort (n2 version and termination checking
version), Selection Sort, Quick Sort and Merge Sort
• Properties:– Sort elements
• ∀k : 0≤k<n => A[k] ≤A[k+1] --- output array is non-decreasing– Maintain permutation, when elements distinct
• ∀k∃j : 0≤k<n => A[k]=Aold[j] & 0≤j<n --- no elements gained• ∀k∃j : 0≤k<n => Aold[k]=A[j] & 0≤j<n --- no elements lost
![Page 49: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/49.jpg)
Runtimes: Sortedness
Selection Sort Insertion Sort Bubble Sort (n2) Bubble Sort (flag) Quick Sort Merge Sort0
2
4
6
8
10
12
14
16
LFPGFPCFP
seco
nds
Tool can prove sortedness for all
sorting algorithms!
![Page 50: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/50.jpg)
Runtimes: Permutation
Selection Sort Insertion Sort Bubble Sort (n2) Bubble Sort (flag) Quick Sort Merge Sort0
5
10
15
20
25
30
35
40
LFPGFPCFP
∞ ∞
…
94.42
seco
nds
…Permutations too!
![Page 51: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/51.jpg)
Inferring Preconditions• Given a property (worst case runtime or functional
correctness) what is the input required for the property to hold
• Tool automatically infers non-trivial inputs/preconditions
• Worst case input (precondition) for sorting:– Selection Sort: sorted array except last element is the smallest– Insertion, Quick Sort, Bubble Sort (flag): Reverse sorted array
• Inputs (precondition) for functional correctness:– Binary search requires sorted input array– Merge in Merge sort requires sorted inputs– Missing initializers required in various other programs
![Page 52: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/52.jpg)
Runtimes (GFP): Inferring worst case inputs for sorting
seco
nds
Selection Sort Insertion Sort Bubble Sort (n2) Bubble Sort (flag) Quick Sort Merge Sort0
5
10
15
20
25
30
35
40
45
Tool infers worst case inputs for all
sorting algorithms!
Nothing to infer as all inputs yield the same behavior
![Page 53: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/53.jpg)
User Input
Full functionalcorrectness
No input
Template-basedinvariant inference
Small; intuitive input
Inferring program properties
InvariantTemplates Predicates
Enumerateall possible
predicate sets
More burden on theorem prover
Incrementally increase
sophistication
Loss of efficiency
![Page 54: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/54.jpg)
Verification using Templates: Summary• Powerful invariant inference over predicate abstraction
– Can infer quantifier invariants
• Three algorithms with different strengths– LFP, GFP, CFP
• Extend to maximally-weak precondition inference– Worst case inputs and preconditions for functional correctness
• Techniques builds on SMT Solvers, so exploit their power
• Successfully verified/inferred preconditions– All major sorting algorithms and other difficult benchmarks
![Page 55: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/55.jpg)
Verification
Strongest Postcondition
Inference (LFP)
Weakest Precondition
Inference (GFP)
Verification: Summary
Iouter
IinnerOutput state
Input state
![Page 56: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/56.jpg)
Synthesis: Problem Statement
Iouter
IinnerOutput state
Input state
??
![Page 57: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/57.jpg)
Looping structure; Resources
Iouter
IinnerOutput state
Input state
??
??
??
??
??
Resource Limits:• Stack space• Computational limits
![Page 58: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/58.jpg)
Constraint Setup
∀k1,k2 : 0≤k1<k2<n ∧ k1<i => A[k1] ≤A[k2]∧ x1=e1 ∧ x2=e2 ∧ x3=e3
∀k1,k2 : ??1 => ??2∀k1,k2 : ??3 => ??4
Find optimal solutions to unknowns
??1, ??2, ??3, ??4
=>Verification:
∀k1,k2 : 0≤k1<k2<n ∧ k1<i => A[k1] ≤A[k2] ∧ x1=??5 ∧ x2=??6 ∧ x3=??7
∀k1,k2 : ??1 => ??2∀k1,k2 : ??3 => ??4
Find optimal solutions to unknowns
??1, ??2, ??3, ??4
=>Synthesis:
and ??5, ??6, ??7, ??8
∀k1,k2 : ??9 => ??10
∧ x1=??5 ∧ x2=??6 ∧ x3=??7
∀k1,k2 : ??1 => ??2∀k1,k2 : ??3 => ??4
Find optimal solutions to unknowns
??1, ??2, ??3, ??4
=>Synthesis:
and ??5, ??6, ??7, ??8
and sometimes ??9, ??10
Stack space
Computation limits
![Page 59: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/59.jpg)
Constraint Solving
∀k1,k2 : ??9 => ??10
∧ x1=??5 ∧ x2=??6 ∧ x3=??7
∀k1,k2 : ??1 => ??2∀k1,k2 : ??3 => ??4
Find solutions to unknowns
??1, ??2, ??3, ??4??5, ??6, ??7, ??8
??9, ??10
=>Synthesis:
Precondition
Postcondition
GFP
LFP
Too inefficient
Ideal as it does piecewise reduction
Constraint-based
SAT solving fixed-point
uses information
from both bkwd and fwd
![Page 60: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/60.jpg)
Example: Strassen’s Matrix Mult.• Recursive Block matrix multiplication
– Trivial solution is O(n3)– Needs 8 multiplication; hence the O(nln8)
• There exists a way to compute the same eight– In 7 multiplications! Will give O(nln7) = O(n2.81)
Looping structure:• Acyclic
Resources:• 7 multiplications• 7 vars to hold the intermediate results
=
Strassen’s soln:v1 = (a1+a4)(b1+b4)v2 = (a3+a4)b1v3 = a1(b2-b4)v4 = a4(b3-b1) v5 = (a1+a2)b4v6 = (a3-a1)(b1+b2)v7 = (a2-a4)(b3+b4)
c1 = v1+v4-v5+v7c2 = v3+v5c3 = v2+v4c4 = v1+v3-v2+v6
=a1 a2
a3 a4
b1 b2
b3 b4
c1 c2
c3 c4
Our tool synthesizes strassen’s soln along with thousands others:
- 7 multiplications decides the exponent
- Constant factor decided by number of adds/subs
- Add another resource limitations: <19 adds/subs
- We get around 5-10 soln
Our tool can synthesize sorting algorithms too!!!
![Page 61: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/61.jpg)
Synthesis: Result
• If you can give me a verification tool that can handle multiple positive and negative unknowns
• And additionally, generates maximally-weak solutions to negative unknowns
Then
• It can be converted into a synthesis tool!
![Page 62: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/62.jpg)
Conclusion
??
Verification
Strongest Postcondition
Inference (LFP)
Weakest Precondition
Inference (GFP)
Iouter
IinnerOutput state
Input state
Program Synthesi
s
VS3: http://www.cs.umd.edu/~saurabhs/pacsDiscuss: [email protected]
Catch me after the talk! I’m here till Tuesday afternoon
![Page 63: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/63.jpg)
![Page 64: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/64.jpg)
Outline• Three fixed-point inference algorithms
– Iterative fixed-point• Greatest Fixed-Point (GFP)• Least Fixed-Point (LFP)
– Constraint-based (CFP)
• Optimal Solutions– Built over a clean theorem proving interface
• Weakest Precondition Generation
• Experimental evaluation
(LFP)
(GFP)
(CFP)
![Page 65: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/65.jpg)
Optimal Solutions• Key: Polarity of unknowns in formula Φ
– Positive or negative:• Value of positive unknown stronger => Φ stronger• Value of negative unknown stronger => Φ weaker
• Optimal Soln: Maximally strong positives, maximally weak negatives
• Assume theorem prover interface: OptNegSol– Optimal solutions for formula with only negative unknowns– Built using a lattice search by querying SMT Solver
¬negative
u1
positive∨ u2 ) ∧ u3(
positive∀x : ∃y : Φ =
![Page 66: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/66.jpg)
Optimal Solutions using OptNegSolformula Φ contains unknowns:
Repeat until set stable
Optimal Solutions for formula Φ
OptOpt’
Opt’’
Merge
…
Merge positive tuples
Φ[ ]Φ[ ]
Φ[ ]
u1…uP positive u1…uN negative
…
a1…aPa’1…a’P
a’’1…a’’P
P-tuple that assigns a single predicate
to each positive unknown
a1…aP,S1…SNa’1…a’P,S’1…S’N
…
Si soln for the negative unknowns
a’’1…a’’P,S’’1…S’’N
OptNegSol
…
P x
Size
of p
redi
cate
set
![Page 67: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/67.jpg)
Backwards Iterative (GFP)
post
pre
Backward: • Always pick the source invariant of unsat constraint to improve• Start with <⊤, …,⊤>
Candidate Sols <I1,I2> Unsat
constraints<⊤, ⊤> { vc(I1,post) }
⊤
⊤
✓✓
✓
✗✓
![Page 68: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/68.jpg)
Backwards Iterative (GFP)
post
pre
a1
⊤
✓ ✓
Backward: • Always pick the source invariant of unsat constraint to improve• Start with <⊤, …,⊤>
Candidate Sols <I1,I2> Unsat
constraints<⊤, ⊤> { vc(I1,post) }
<a1, ⊤> { vc(I2,I1) }
✓
Optimally strengthen so vc(pre,I1) ok unless no soln
✗✓
![Page 69: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/69.jpg)
Backwards Iterative (GFP)
post
pre
a1
b2
✓ ✓
Backward: • Always pick the source invariant of unsat constraint to improve• Start with <⊤, …,⊤>
Candidate Sols <I1,I2> Unsat
constraints<⊤, ⊤> { vc(I1,post) }
<a1, ⊤> { vc(I2,I1) }
<a1,b1> { vc(I2,I2) }<a1,b2> { vc(I2,I2),vc(I1,I2) }
Multiple orthogonal optimal sols
✓
Optimally strengthen so vc(pre,I1) ok unless no soln
✗✗
![Page 70: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/70.jpg)
Backwards Iterative (GFP)
post
pre
✓ ✓✓
Backward: • Always pick the source invariant of unsat constraint to improve• Start with <⊤, …,⊤>
Candidate Sols <I1,I2> Unsat
constraints<⊤, ⊤> { vc(I1,post) }
<a1, ⊤> { vc(I2,I1) }
<a1,b1> { vc(I2,I2) }<a1,b2> { vc(I2,I2),vc(I1,I2) }
<a1,b1> { vc(I2,I2) }<a1,b’2> { vc(I1,I2) }
Multiple orthogonal optimal sols
✓
Optimally strengthen so vc(pre,I1) ok unless no soln
✗b’2
a1
![Page 71: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/71.jpg)
Backwards Iterative (GFP)
post
pre
✓ ✓✓
✓
Backward: • Always pick the source invariant of unsat constraint to improve• Start with <⊤, …,⊤>
Candidate Sols <I1,I2> Unsat
constraints<⊤, ⊤> { vc(I1,post) }
<a1, ⊤> { vc(I2,I1) }
<a1,b1> { vc(I2,I2) }<a1,b2> { vc(I2,I2),vc(I1,I2) }
<a1,b1> { vc(I2,I2) }<a1,b’2> { vc(I1,I2) }
<a1,b1> { vc(I2,I2) }<a’1,b’2> none
Multiple orthogonal optimal sols
✓
Optimally strengthen so vc(pre,I1) ok unless no soln
b’2
a’1
<a’1,b’2> : GFP solution
![Page 72: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/72.jpg)
Forward Iterative (LFP)
post
pre
⊥
⊥
✓ ✓✓
✗
✓
Forward: • Pick the destination invariant of unsat constraint to improve• Start with <⊥,⊥>
Candidate Sols <I1,I2> Unsat
constraints<⊥,⊥> vc(pre,I1)
![Page 73: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/73.jpg)
Forward Iterative (LFP)
post
pre
a1
⊥
✓ ✓
Forward: • Pick the destination invariant of unsat constraint to improve• Start with <⊥,⊥>
Candidate Sols <I1,I2> Unsat
constraints<⊥,⊥> vc(pre,I1)<a1,⊥> vc(I1,I2)
✓
✗✓
Optimally weakened so vc(I1,post) ok unless no soln
![Page 74: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/74.jpg)
Forward Iterative (LFP)
post
pre
a1
b3
✓ ✓
Forward: • Pick the destination invariant of unsat constraint to improve• Start with <⊥,⊥>
Candidate Sols <I1,I2> Unsat
constraints<⊥,⊥> vc(pre,I1)<a1,⊥> vc(I1,I2)
<a1,b1> vc(I2,I2),<a1,b2> vc(I2,I1),
<a1,b3> vc(I2,I2),vc(I2,I1)
Multiple orthogonal optimal sols
✓
✗✗
Optimally weakened so vc(I1,post) ok unless no soln
![Page 75: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/75.jpg)
Forward Iterative (LFP)
post
pre
a1
b’3
✓
✓✓
Forward: • Pick the destination invariant of unsat constraint to improve• Start with <⊥,⊥>
Candidate Sols <I1,I2> Unsat
constraints<⊥,⊥> vc(pre,I1)<a1,⊥> vc(I1,I2)
<a1,b1> vc(I2,I2),<a1,b2> vc(I2,I1),
<a1,b3> vc(I2,I2),vc(I2,I1)
<a1,b2> vc(I2,I1),<a1,b’3> vc(I2,I1)
Multiple orthogonal optimal sols
<a1,b1> subsumed by <a1,b’3>
✓
✗
Optimally weakened so vc(I1,post) ok unless no soln
![Page 76: Program Verification and Synthesis using Templates over Predicate Abstraction](https://reader030.fdocuments.net/reader030/viewer/2022012911/56816230550346895dd26217/html5/thumbnails/76.jpg)
Forward Iterative (LFP)
post
pre
a’1
b’3
✓ ✓✓
✓
Forward: • Pick the destination invariant of unsat constraint to improve• Start with <⊥,⊥>
Candidate Sols <I1,I2> Unsat
constraints<⊥,⊥> vc(pre,I1)<a1,⊥> vc(I1,I2)
<a1,b1> vc(I2,I2),<a1,b2> vc(I2,I1),
<a1,b3> vc(I2,I2),vc(I2,I1)
<a1,b2> vc(I2,I1),<a1,b’3> vc(I2,I1)<a’1,b’3> none
Multiple orthogonal optimal sols
<a1,b1> subsumed by <a1,b’3>
✓
Optimally weakened so vc(I1,post) ok unless no soln