ESC Java

Static Analysis Spectrum

Power

CostType

checking

Data-flow

analysis

Modelchecking

Programverification

Automated Manual

ESC

Is This Program Correct?

int square(int n) {

int k = 0, r = 0, s = 1;

while(k != n) {

r = r + s; s = s + 2; k = k + 1;

}

return r;

}

• Type checking not enough to check this– Neither is data-flow analysis, nor model checking

Program Verification

• Program verification is the most powerful static analysis method– Can reason about all properties of programs

• Cannot fully automate

• But …– Can automate certain parts (ESC/Java)– Teaches how to reason about programs in a

systematic way

Specifying Programs

• Before we check a program we must specify what it does

• We need formal specifications– English comments are not enough

• We use logic notation– Theory of pre- and post-conditions

State Predicates

• A predicate is a boolean expression on the program state (e.g., variables, object fields)

• Examples:– x == 8– x < y– true– false– (8i. 0 <= i < a.length ) a[i] >= 0)

Using Predicates to Specify Programs

• We focus first on how to specify a statement

• Hoare triple for statement S

{ P } S { Q }

• Says that if S is started in a state that satisfies P, and S terminates, then it terminates in Q– This is the liberal version, which doesn’t care about

termination– Strict version: if S is started in a state that satisfies P

then S terminates in Q

precondition postcondition

Hoare Triples. Examples.

• { true } x = 12 { x == 12 }

• { y >= 0 } x = 12 { x == 12 }

• { true } x = 12 { x >= 0 } (Programs satisfy many possible specifications)

• { x < 10 } x = x + 1 { x < 11 }

• { n >= 0 } x = fact(n) { x == n ! }

• { true } a = 0; if(x != 0) { a = 2 * x; } { a == 2*x }

Computing Hoare Triples

• We compute the triples using rules– One rule for each statement kind

– Rules for composed statements

Assignment

• Assignment is the simplest operation and the trickiest one to reason about !

• { y >= 2 } x = 5 { ? }

• { x == y } x = x + 1 { ? }

• { ? } x = 5 { x == y }

• { ? } x = x + 1 { x == y }

• { ? } x = x + 1 { x2 + y2 == z2 }

• { x2 + y2 == z2 } x = x + 1 { ? }

Assignment Rule

• Rule for assignment

{ Q[x := E] } x = E { Q}

• Examples: – { 12 == 12 } x = 12 { x == 12 }

– { 12 >= 0 } x = 12 { x >= 0 }– { ? } x = x + 1 { x >= 0 }– { x >= 1 } x = x + 1 { ? }

Q with x replaced by E

x == 12 with x replaced by 12

Relaxing Specifications

• Consider { x >= 1 } x = x + 1 { x >= 2 }– It is very tight specification. We can relax it

• Example: { x >= 5 } x = x + 1 { x >= 2 } (since x >= 5 ) x + 1 >= 2)

x = E

{ P } if P ) Q[x:=E]

{ Q }

Assignments: forward and backward

• Two ways to look at the rules:– Backward: given post-condition, what is pre-

condition?

– Forward: given pre-condition, what is post-condition?

x = E

{ ??? }

{ Q }

x = E

{ P }

{ ??? }

Assignments: forward and backward

• Two ways to look at the rules:– Backward: given post-condition, what is pre-

condition?

– Forward: given pre-condition, what is post-condition?

x = E

{ Q [ x := E] }

{ Q }

x = E

{ P }

{ ??? }

Assignments: forward and backward

• Two ways to look at the rules:– Backward: given post-condition, what is pre-

condition?

– Forward: given pre-condition, what is post-condition?

x = E

{ Q [ x := E] }

{ Q }

x = E

{ P }

{ }

Example of running it forward

• { x == y } x = x + 1 { ? }

Example of running it forward

• { x == y } x = x + 1 { ? }

Forward or Backward

• Forward reasoning– Know the precondition– Want to know what postconditon the code establishes

• Backward reasoning– Know what we want to code to establish– Must find in what precondition this happens

• Backward is used most often– Start with what you want to verify– Instead of verifying everything the code does

Weakest precondition

• wp(S, Q) is the weakest P such that { P } S { Q } – Order on predicates: Strong ) Weak– wp returns the “best” possible predicate

• wp(x := E, Q) = Q[x := E]

• In general:

S

{ P } if P ) wp(S,Q)

{ Q }

Weakest precondition

• This points to a verification algorithm:– Given function body annotated with pre-condition P

and post-condition Q:• Compute wp of Q with respect to functon body• Ask a theorem prover to show that P implies the wp

• The wp function we will use is liberal (P does not guarantee termination)– If using both strict and liberal in the same context, the

usual notation is wlp the liberal version and wp for the strict one

Strongest postcondition

• sp(S, P) is the strongest Q such that { P } S { Q } – Recall: Strong ) Weak– sp returns the “best” possible predicate

• sp(x := E, P) = …

• In general:

S

{ P }

{ Q } if sp(S,P) ) Q

Strongest postcondition

• Strongest postcondition and weakest preconditions are symmetric

• This points to an equivalent verification algorithm:– Given function body annotated with pre-condition P

and post-condition Q:• Compute sp of P with respect to functon body• Ask a theorem prover to show that the sp implies Q

Composing Specifications

• If { P } S1 { R } and { R } S2 { Q}

then { P } S1; S2 { Q }

• Example:

x = x - 1;

y = y - 1

{ x >= y }

Composing Specifications

• If { P } S1 { R } and { R } S2 { Q}

then { P } S1; S2 { Q }

• Example:

x = x - 1;

y = y - 1

{ x >= y }

In terms of wp and sp

• wp(S1; S2 , Q) = wp(S1,wp(S2, Q))

• sp(S1; S2, P) = sp(S2,sp(S1, P))

Conditionals

• Rule for the conditional (flow graph)

• Example:

E

{ P }

{ P1 }if P && E ) P1

T F

{ P2 }if P && ! E ) P2

x == 0

{ x >= 0 }T F

{ x == 0 }since x >= 0 && x == 0 ) x == 0

{ x >= 1 }since x >= 0 && x != 0 ) x >= 1

Conditionals: Forward and Backward

• Recall: rule for the conditional

• Forward: given P, find P1 and P2

– pick P1 to be P && E, and P2 to be P && ! E

• Backward: given P1 and P2, find P

– pick P to be (P1 && E) || (P2 && ! E)

– Or pick P to be (E ) P1) && (! E ) P2)

E

{ P }

{ P1 }provided P && E ) P1

T F

{ P2 }provided P && ! E ) P2

Joins

• Rule for the join:

• Forward: pick P to be P1 || P2

• Backward: pick P1, P2 to be P

{ P1 } { P2 }

{ P } provided P1 ) P and P2 ) P

Review

E

{ P }

{ P1 }if P && E ) P1

T F

{ P2 }if P && ! E ) P2

{ P1 } { P2 }

{ P }

if P1 ) P and P2 ) P

x = E

{ P }

{ Q }

if P ) Q[x:=E]

Implication is always in the direction of the control flow

Review: forward

E

{ P }

{ P && E }

T F

{ P && ! E }

{ P1 } { P2 }

{ P1 || P2 }x = E

{ P }

{ \exists … }

Review: backward

E

{(E ) P1) && (! E ) P2) }

{ P1 }

T F

{ P2 }

{ P } { P }

{ P }x = E

{Q[x:=E] }

{ Q }

Example: Absolute value

static int abs(int x)//@ ensures \result >= 0{ if (x < 0) { x = -x; } if (c > 0) { c--; } return x;}

x < 0

x = -x

T F

c > 0

c--

T F

Example: Absolute value

x < 0

x = -x

T F

c > 0

c--

T F

Example: Absolute value

x < 0

x = -x

T F

c > 0

c--

T F

In Simplify

> (IMPLIES TRUE (AND (IMPLIES (< x 0) (AND (IMPLIES (> c 0) (>= (- 0 x) 0)) (IMPLIES (<= c 0) (>= (- 0 x) 0)))) (IMPLIES (>= x 0) (AND (IMPLIES (> c 0) (>= x 0)) (IMPLIES (<= c 0) (>= x 0))))))1: Valid.

>

So far…

• Framework for checking pre and post conditions of computations without loops

• Suppose we want to check that some condition holds inside the computation, rather than at the end

static int abs(int x) { if (x < 0) { x = -x; } if (c > 0) { c--; } return x;}

Say we want to check that x > 0 here

Asserts

• { Q && E } assert(E) { Q }

• Backward: wp(assert(E), Q) = Q && E

• Forward: sp(assert(E), P) = ???

assert(E)

Q

Q && E

assert(E)

???

P

Example: Absolute value with assert

static int abs(int x){ if (x < 0) { x = -x; assert(x > 0); } if (c > 0) { c--; } return x;}

x < 0

x = -x

assert(x > 0)

T F

c > 0

c--

T F

Example: Absolute value with assert

x < 0

x = -x

assert(x > 0)

T F

c > 0

c--

T F

Example: Absolute value with assert

x < 0

x = -x

assert(x > 0)

T F

c > 0

c--

T F

Adding the postcondition back in

x < 0

x = -x

assert(x > 0)

T F

c > 0

c--

T F

Adding the postcondition back in

x < 0

x = -x

assert(x > 0)

T F

c > 0

c--

T F

Another Example: Double Locking

“An attempt to re-acquire an acquired lock or release a released lock will cause a deadlock.”

Calls to lock and unlock must alternate.

lock

lock

unlock

unlock

Locking Rules

• We assume that the boolean predicate locked says if the lock is held or not

• { ! locked && P[locked := true] } lock { P }– lock behaves as assert(! locked); locked = true

• { locked && P[locked := false] } unlock { P }– unlock behaves as assert(locked); locked = false

Locking Example

…

lock

…

x==0

…

unlock

x==0

T

T

{ ! L && P[L := true] } lock { P }

{ L && P[L := false] } unlock { P }

{ ! L }

{ ! L }

Locking Example

…

lock

…

x==0

…

unlock

x==0

T

T

{ ! L && P[L := true] } lock { P }

{ L && P[L := false] } unlock { P }

{ ! L }

{ ! L }

Locking Example: forward direction

…

lock

…

x==0

…

unlock

{ ! locked }

{ ! locked && x == 0 }{ ! locked && x 0 }

x==0

T

T

{ ! locked && x == 0 }

{ locked && x == 0 }

{ locked = (x == 0) }

{ locked && x == 0 }

{ ! locked && (x == 0) }

{ ! locked && x 0 }

{ ! locked }

In Simplify

>

(DEFPRED (isLocked L) (EQ L locked))

(IMPLIES (NOT (isLocked L)) (AND (IMPLIES (EQ x 0) (AND (NOT (isLocked L)) (AND (IMPLIES (EQ x 0) (AND TRUE (NOT FALSE))) (IMPLIES (NEQ x 0) (NOT (AND TRUE (NOT FALSE))))))) (IMPLIES (NEQ x 0) (AND (IMPLIES (EQ x 0) (AND (isLocked L) (NOT FALSE))) (IMPLIES (NEQ x 0) (NOT (AND (isLocked L) (NOT FALSE))))))))

1: Valid.

>

Reasoning About Programs with Loops

• Loops can be handled using conditionals and joins

• Consider the while(E) S statement

E

{ P }

T FS { Q }

{ I }

{ I }

{ I && E }

if (1) P ) I (loop invariant holds initially)

and (2) I && ! E ) Q (loop establishes the postcondition)

and (3) { I && E } S { I } (loop invariant is preserved)

Loop invariant

Loops in the backward direction

• Given Q, want to find weakest invariant I that will establish (2) and (3), then pick P to be I

• Finding weakest I is:– Undecidable in theory– Hard in practice

E

{ P }

T FS { Q }

{ I }

{ I }

{ I && E }

Loop invariant

Loops in the forward direction

• Given P, want to find strongest invariant I that will establish (1) and (3), then pick Q to be I && E

• Again, finding I is hard

E

{ P }

T FS { Q }

{ I }

{ I }

{ I && E }

Loop invariant

Loop Example

• Let’s verify { x ==8 && y ==16 } while(x > 0) { x --; y -= 2; } { y==0 }– Is this true ?

– We must find an appropriate invariant I• Try one that holds initially x == 8 && y == 16• Try one that holds at the end y == 0

x > 0

{ x == 8 && y == 16 }

T Fx --;

y -= 2 { y == 0 }

{ I }

{ I }

{ I && x > 0 }

Loop Example (II)

• Guess the invariant y == 2*x

• Must check– Initial: x == 8 && y == 16 ) y == 2*x– Preservation: y == 2*x && x > 0 ) y – 2 = 2*(x – 1) – Final: y == 2*x && x <= 0 ) y == 0

x > 0

{ x == 8 && y == 16 }

T Fx --;

y -= 2 { y == 0 }

{ y == 2*x }

{ y == 2*x }

{ y == 2*x && x > 0 }

Loop Example (III)

• Guess the invariant y == 2*x && x >= 0

• Must check– Initial: x == 8 && y == 16 ) y == 2*x && x >= 0– Preservation: y == 2*x && x >= 0 && x > 0 ) y – 2 = 2*(x – 1) && x – 1 >= 0– Final: y == 2*x && x >= 0 && x <= 0 ) y == 0

x > 0

{ x == 8 && y == 16 }

T Fx --;y -= 2 { y == 0 }

{ y == 2*x && x >= 0}

{ y == 2*x && x >= 0}

{ y == 2*x && x >= 0 && x > 0 }

Functions

• Consider a binary search function bsearchint bsearch(int a[], int p) {

{ sorted(a) }

…

{ r == -1 || (r >= 0 && r < a.length && a[r] == p) }

return res;

}

• The precondition and postconditon are the function specification– Also called a contract

Postcondition

Precondition

Function Calls

• Consider a call to function F(int in)– With return variable out– With precondition Pre, postcondition Post

• Rule for function call:

y = F(E)

{ P } if P ) Pre[in:=E] }

{ Q } and Post[out := y, in := E] ) Q

Function Call Example

• Consider the call{ sorted(array) }

y = bsearch(array, 5)

if( y != -1) {

{ array[ y ] == 5 }

• Show Post[r := y, a := array, p := 5]

) array[y] == 5– Need to know y != -1 !

• Show sorted[array] ) Pre[a := array]

Function Calls: backward

y = F(E)

{ Q }

• Consider a call to function F(int in)– With return variable out– With precondition Pre, postcondition Post

Function Calls: backward

y = F(E)

{ Q }

• Consider a call to function F(int in)– With return variable out– With precondition Pre, postcondition Post

Pointers and aliasing

x = *y + 1

{ ??? }

{ x == 5 }

Pointers and aliasing

x = *y + 1

{ *y == 4 }

{ x == 5 }

Regular rule worked in this case!

Example where regular rule doesn’t work

x = *y + 1

Example where regular rule doesn’t work

x = *y + 1

{ ??? }

{ x == *y + 1 }

Example where regular rule doesn’t work

x = *y + 1

{ y != &x Æ x == *y + 1 }

{ x == *y + 1 }

Pointer stores

*x = y + 1

{ ??? }

{ y == 5 }

Pointer stores

*x = y + 1

{ (x == &y ) y + 1 == 5) Æ (x != &y ) y == 5) }

{ y == 5 }

One solution

• Perform case analysis based on all the possible alias relationships between the LHS of the assignment and part of the postcondition

• Can use a static pointer analysis to prune some cases out

• However, exponentially many cases in the pointer analysis, which leads to large formulas.

• eg, how many cases here: *x = *y + a

{ *z == *v + b }

Another solution

• Up until now the program state has been implicit. Let’s make the program state explicit...

• A predicate is a function from program states to booleans.

• So for wp(S, Q), we have:– Q() returns true if Q holds in – wp(S, Q)() returns true if wp(S, Q) holds in

New formulation of wp

• Suppose step(S, ) returns the program state resuling from executing S starting in program state .

• Then we can express wp as follows:

wp(S, Q)() =

New formulation of wp

• Suppose step(S, ) returns the program state resuling from executing S starting in program state .

• Then we can express wp as follows:

wp(S, Q)() = Q(step(S, ))

Example in Simplify syntax

From previous slide: wp(S, Q)() = Q(step(S, ))

*x = y + 1

{ y == 5 }

Q is:

step(S, ) is:

wp(S, Q) is:

Example in Simplify syntax

From previous slide: wp(S, Q)() = Q(step(S, ))

*x = y + 1

{ y == 5 }

Q is: (EQ (select s y) 5)

step(S, ) is: (store s (select s x) (+ (select s y) 1))

wp(S, Q) is: (EQ (select (store s (select s x) (+ (select s y) 1)) y) 5)

ESC/Java summary

• Very general verification framework– Based on pre- and post-conditions

• Generate VC from code– Instead of modelling the semantics of the code inside

the theorem prover

• Loops and procedures require user annotations– But can try to infer these