ClousotA static contract checker based on abstract interpretation

Francesco Logozzo

Microsoft Research, Redmond, WA

Demo!

Code ContractsIdea: Use the IL as contract representationUse static methods to a contract library

Language agnostic: same for C#, VB, F# …

Code Contracts toolsDocumentation generation (ccdoc)

Automatic generation of documentationRuntime checking (ccrewrite)

Binary rewritingStatic checking (Clousot)

Abstract InterpretationTheory of approximationsSemantics are order according to the precisionThe more the precise the semantics

The more the properties captured• A static analysis is a semantics• Precise enough to capture the properties

of interest• Rough enough to be computable

ClousotBased on Abstract Interpretation

≠ Usual approaches based on theorem proverAdvantages

Automatic Inference of loop invariants, pre, post, invariants

PredictableNo quantifier instantiationNo easy proofs by contradictory axioms

ScalableTune-up for the properties of interest

Clousot: The big picture

A.dll B.dll Z.dllC.dll …

Call Graph Construction

Contract Extraction

Analysis Inference

Assertion Checking

Method Analysis1. Analyze the method2. Collect the proof obligations

Explicit: Pre/Post, assertionsImplicit: Array bounds, non-null …

3. Discharge proof obligationsIf not, emit warning message

4. Propagate inferred contracts

Bytecode

Stack language

Why the bytecode???More faithful

Closer to what get executedClear semantics of the instructions

Exploit the work of the compilerName resolution, type inference, generics, LINQ…

Language agnosticBytecode does not change!

Languages yes : C# 2.0 → C# 3.0 → C# 4.0

DrawbacksExplicit stackProgram structure lostExpressions chunked out…Need a program normalization!

Clousot: Analysis structure

AnalysesBounds, nonnull, arrays…

Expression analysis

Heap analysis

Stack analysis

Source: z = x + y

Expression recoveryAssume x + y ≤ 4High level: easy!Low level: problem!

Eager expression reconstruction?

MDTransform9000 straight line instructions

Lazy expression recovery

Value AnalysesNonnull

Is a reference null?Bounds

Array bounds, numerical values …Arithmetic

Division by zero, negation of MinInt …Unsafe

Buffer overrun

Array content (with P. & R. Cousot)

StringsObject Invariants

(M. Monereau)Iterators

(S. Xia)

Pietro Ferrara, Francesco Logozzo and Manuel Fahndrich Safer Unsafe Code in .NET, in OOPSLA 2008

1. Numerical Abstract Domains

Abstract domains0 ≤ index < array.Length?

index

a.Length

IntervalsO(n)

a ≤ x ≤ bNo

index

a.Length

PentagonsO(n)

a≤ x ≤ b & x <yYes

index

a.Length

OctagonsO(n3)

± x ± y ≤ aYes

index

a.Length

PolyhedraO(2n)

Σ aixi ≤ bYes

index

a.Length

Numerical domains in Clousot

Basic Intervals, Pentagons, Leq, Karr, Octagons, Simple Disequalities, Stripes, Subpolyhedra …

Combination of thereofTree of domains

Incremental analysisFirst analyze with “cheap” domainsMove to more expensive if fails to prove

Domain D1

Domain D2

Domain D3

Why Subpolyhedra?Often proving a “easy” precondition requires a complex reasoning

From StringBuilder:

Subpolyhedra∑ai xi ≤ k ⇔ ∑ai xi = β ⋀ β ≤ kReduced product of

IntervalsScalable, fast…

Linear EqualitiesPrecise join, fast …

Challenge: Have a precise Join

Vincent Laviron and Francesco Logozzo, Subpolyhedra: A (more) scalable approach to the inference of linear inequalities, in VMCAI 2009

Naif Join

assume x <= y

x = 0; y = 1

assert x <= y

⟨x - y == β, β ∈ [-∞, 0]⟩

⟨T, x ∈ [0,0] ⋀ y ∈ [1,1]⟩

⟨T, T⟩

Join algorithm : SubPolyhedra1. Uniform slack variables2. Reduce the states3. Do the pair-wise join4. Recover precision using deleted

equalities5. Recover precision using hints• Templates, 2D Convex Hull, Annotations

…

Vincent Laviron and Francesco Logozzo, Refining Abstract Interpretation-based Static Analyses with Hints, in APLAS 2009

Example : Join Step 1Entry State:

s0 : ⟨x - y == β, β ∈ [-∞, 0] ⟩s1 : ⟨T, x ∈ [0,0] ⋀ y ∈ [1,1]⟩

Step 1 (uniform slack variables) s’0 : ⟨x - y == β, β ∈ [-∞, 0] ⟩s’1 : ⟨x - y == β, x ∈ [0,0] ⋀ y ∈ [1,1]⟩

Example: Join steps 2-3Step 2 (Reduction)

s’’0 : ⟨x - y == β, β ∈ [-∞, 0]⟩s’’1 : ⟨x - y == β, x ∈ [0,0] ⋀ y ∈ [1,1] ⋀ β ∈ [-1,-1]⟩

Step 3 (Pair-wise join)s2 : ⟨x - y == β, β ∈ [-∞, 0]⟩

Example: Join Step 4Recover lost relations

assume x == y x = 0; y = 1

assert x<= y

⟨x - y == 0, T⟩ ⟨T, x ∈ [0,0] ⋀ y ∈ [1,1]⟩

⟨T, T⟩⟨x - y == β, β ∈ [-1, 0]⟩

Critical operation: ReductionInfer tightest boundsInstance of a Linear programming problem

Solution in polynomial timeDrawbacks:

Numerical instability, Rounding errorsSimplex too slow for our purposes

Basis exploration (new)Based on static basis explorationLess concerned about numerical instability

Abstract when an error is detectedE.g. In a row operation, delete the row

To sum up on SubpolyhedraInfer arbitrary linear inequalities

Scales to hundreds of variablesPrecisely propagate linear inequalitiesGive up some of the inference power

Family of abstract domainsTwo precision axes

HintsTune the inference power at join points

ReductionInfer the tightest intervals

2. Abstract domain for array content inference

Inferring array contents…public void Init(int N){ Contract.Requires(N > 0);

int[] a = new int[N]; int i = 0;

while (i < N) { a[i] = 222; i = i + 1; }

Contract.Assert(∀ k ∈ [0, N). a[k] == 222);}

If i == 0 then a not initializedelse if i > 0 a[0] == … a[i] == 222else impossible

Challenge 1:Effective handling of disjunction

Challenge 2:No overapproximation (can be unsound)(no hole, all the elements are initialized)

Our ideaPrecise and very very fast!Basis: Array segments

[222, 222]0 i, k [0, 0] N

Segment bounds Uniform content abstraction

?

0 ≤ i, 0 ≤ k i == k i < N, k < N

Disjunction

ExampleContract.Requires(N > 0);int[] a = new int[N];

int i = 0;

assume i < N

a[i] = 222;

assume i ≥ N

j = i+1;

i -> _ j -> iN -> N

00 N

00,i N

00,i N

2220,i N01,i+1 ?

2220,i N01,i+1,j ?

2220 N01,i ?

Segment unification00,i N 2220 N01,i ?

00 N⊥ i ? 2220 N01,i ?

00 N⊥ i ? 2220 N0i ?

2220 N0i ??

Join

Can be empty segments! (Disjunction)

ExampleContract.Requires(N > 0);int[] a = new int[N];

int i = 0;

assume i < N

a[i] = 222;

assume i ≥ N

j = i+1;

i -> _ j -> iN -> N

2220 N0i ??

2220 N0i ?

And so on up to a fixpoint

…

2220 i, N

Remove doubts(i == N && N > 0)

We visited all the elements in [0, N)

Other…Intra-modular Inference

Pre/Post/Object invariantsReduce annotation burdenCan make the analysis bridleSerialize to C#

Backward analysis for disjunctionsSafe floating points in parametersSelective verificationRanking of warnings…

TODOCollections

Experimenting handling of arraysExtend to iterators, List<T> …

StringsNeed good domains to approximate strings

Modular overflow checkingCombine with automatic test generation

PEXMake Clousot parallel…

ConclusionsProgrammers are willing to write annotations

SAL, ESP … at Microsoft,CodeContracts Forum, PDC …

We should provide valuable toolsAutomatic, predictable, fast!!!!

Clousot is a step in that directionDownload it today at:

http://msdn.microsoft.com/en-us/devlabs/(Academic and Commercial license)

Thanks!!!!

⟨T, x ∈ [0,1] ⋀ y ∈ [0,+∞]⟩

⟨T, x ∈ [0,0] ⋀ y ∈ [0,+∞]⟩

Example : Join Step 5

assume y >= 0 ;x = 0;

while x < y

x++;assert x == y ;

⟨T, x ∈ [0,0] ⋀ y ∈ [0,+∞]⟩

⟨T, x ∈ [0,0] ⋀ y ∈ [1,+∞]⟩

⟨T, x ∈ [1,1] ⋀ y ∈ [1,+∞]⟩

⟨T, x ∈ [0,0] ⋀ y ∈ [0,0]⟩

⟨ x – y == β’, x ∈ [0,1] ⋀ y ∈ [0,1] ⋀

β’ ∈ [0,0]⟩

⟨ x – y == β, x ∈ [0,1] ⋀ y ∈ [0,1] ⋀

β ∈ [0,+∞]⟩

⟨x – y == β’, x ∈ [0,1] ⋀ y ∈ [0,+∞] ⋀ β’ ∈ [-

∞,0]⟩