Extensible Shape Analysis by Designing with the User in Mind Bor-Yuh Evan Chang Bor-Yuh Evan Chang,...

Extensible Shape AnalysisExtensible Shape Analysis

by Designing with the User in by Designing with the User in MindMind

Bor-Yuh Evan ChangBor-Yuh Evan Chang, Xavier Rival, and George Necula

University of California, Berkeley

OSQ RetreatMay 16, 2008

2

MotivationMotivation

Analyses find many kinds of bugsFor example,

– Reading from a closed file:

– Reacquiring a locked lock:

But often struggle when objects are put into data structures

read( );read( );

acquire( );acquire( );

Bor-Yuh Evan Chang - Extensible Shape Analysis by Designing with the User in Mind

…

Shape Analysis is Data Structure AnalysisShape Analysis is Data Structure Analysis

3

… code …// x now points to an unlocked lock in a linked listin a linked list

acquire(acquire(x));… code …

ideal analysis state analysis state

What’s hard about data structures?What’s hard about data structures?


x xx

or or or …

x

For decidability, must abstract abstract (e.g., merge objects)

For decidability, must abstract abstract (e.g., merge objects)

Abstraction too coarse or not precise not precise enough (e.g., lost x is always unlocked)

Abstraction too coarse or not precise not precise enough (e.g., lost x is always unlocked)

mislabels good code as buggy

4

To address the precision challengeTo address the precision challenge

TraditionalTraditional program analysis mentality:

“Why can’t developers write more specifications specifications for our analysisfor our analysis? Then, we could verify so much more.”

“Since developers won’t write specifications, we will use default abstractionsdefault abstractions (perhaps coarse) that work hopefully most of the time.”

Our approachOur approach:

“Can we design program analyses around the user? Developers write testing code. Can we adapt the analysisadapt the analysis to use those as specifications?”


5

Overview of contributionsOverview of contributions

Precise inference of data structure propertiesAble to check, for instance, the locking example

Targeted to software developersUses data structure checking code for guidanceTurns testing code into a specification for static Turns testing code into a specification for static

analysisanalysis

EfficientBuilds abstraction out of developer-supplied Builds abstraction out of developer-supplied

checking codechecking code


6

Shape analysis by example:Shape analysis by example:Removing duplicatesRemoving duplicates

// l is a sorted doubly-linked list

for each node cur in list l {remove cur if duplicate;

}assert l is sorted, doubly-

linked with no duplicates;

Example/TestingExample/Testing Code Review/Static AnalysisCode Review/Static Analysis

“no duplicates”l

“sorted dl list”l

program-specificprogram-specific

l 2 2 44

l 2 44

cur

l 2 4

“sorted dl list”l“segment withno duplicates”

cur

intermediate state more

complicated

intermediate state more

complicated


7

Shape analysis is not yet practicalShape analysis is not yet practical

Choosing the heap abstraction difficult for precision

Parametric in high-level, developer-oriented predicates++ Extensible

++ Targeted to developers

Xisa

Built-in high-level predicates

-- Hard to extend

++ No additional user effort (if precise enough)

Parametric in low-level, analyzer-oriented predicates++ Very general and expressive

-- Hard for non-expert

89


Traditional approachesTraditional approaches:

Our approachOur approach:

Space Invader [Distefano et

al.]

TVLA[Sagiv et al.]

8

Key insightKey insightfor being developer-friendly and for being developer-friendly and efficientefficientUtilize “run-time checking codechecking code” as specification for static analysis.

assert(sorted_dll(l,…));

for each node cur in list l {remove cur if duplicate;

}

assert(sorted_dll_nodup(l,…));

l

l

cur

l


dll(h, p) =if (h = null) then

trueelse

h!prev = p and dll(h!next, h)

checker

Contribution: Automatically generalize checkers for complicated intermediate states

Contribution: Automatically generalize checkers for complicated intermediate states

Contribution: Build the abstraction for analysis out of developer-specified checking code

Contribution: Build the abstraction for analysis out of developer-specified checking code

• p specifies where prev should point

9

Our framework is …Our framework is …

• Extensible and targeted for developers– Parametric in developer-supplied checkers

• Precise yet compact abstraction for efficiency– Data structure-specific based on properties of

interest to the developer

An automated shape analysisshape analysis with a precise memory abstraction based around invariant invariant checkerscheckers.

shape analyzer


trueelse

h!prev = prev and dll(h!next, h)

checkers


10

SplittingSplitting of summaries (materialization)

To reflect updates precisely (strong updates)

And summarizingsummarizing for termination (widening)

Shape analysis is an abstract Shape analysis is an abstract interpretation on memory descriptions interpretation on memory descriptions with …with …

cur

l

cur

l

cur

l

cur

l

cur

l

cur

l


11

OutlineOutline

shape analyzer

abstract interpretation

splitting andinterpreting update

summarizing

type“pre-analysis”

on checkerdefinitions


trueelse


checkers


Learn information about the checker to use it as an abstraction

Learn information about the checker to use it as an abstraction 11

22

12

Overview: Split summariesOverview: Split summariesto interpret updates preciselyto interpret updates precisely

l

cur

l

cur


Want abstract update to be “exact”, that is, to update one “concrete memory cell”.The example at a high-level: iterate using cur changing the doubly-linked list from purple to red.

l

cur

split at cur

update cur purple to red

l

cur

Challenge:How does the analysis “split” summaries and know where to “split”?

Challenge:How does the analysis “split” summaries and know where to “split”?

13

Split summaries by unfolding Split summaries by unfolding inductioninduction

Çdll(h, p) =

if (h = null) thentrue

elseh!prev = p and dll(h!next, h)


l

curget: cur!next

l

cur

null

p dll(cur, p)

l

cur

pdll(n, cur)

n

Analysis doesn’t forget the empty case

Analysis doesn’t forget the empty case

“dll segment l to cur”



Technical Details:What about unfolding segments?How are segments unfolded?(Key: Segments are also inductively defined)

[POPL’08]

How does the analysis know to do unfolding?How does the analysis know to do unfolding?

Technical Details:What about unfolding segments?How are segments unfolded?(Key: Segments are also inductively defined)

[POPL’08]

How does the analysis know to do unfolding?How does the analysis know to do unfolding?

14

OutlineOutline

shape analyzer

abstract interpretation

splitting andinterpreting update

summarizing

type“pre-analysis”

on checkerdefinitions


Contribution: Turns testing code into specification for static analysis

Contribution: Turns testing code into specification for static analysis

1122

How do we decide where to unfold?

How do we decide where to unfold?

Derives additional information to guide unfolding

Derives additional information to guide unfolding


trueelse


checkers

15

Types for deciding where to unfold Types for deciding where to unfold

InstanceInstance


trueelse


If it exists, where is:

cur!next ?

p!next ?

If it exists, where is:

cur!next ?

p!next ?

Checker DefinitionChecker Definition


SummarySummary

dll(l,null)

dll(p,l)

dll(cur,p)

dll(n,cur)

dll(null,n)

Checker “Run”Checker “Run” (call tree/derivation)

l p dll(cur, p)“dll segment l to cur”

cur

l

cur

p n

SaysSays:

For h!next/h!prev, unfold fromfrom h

For p!next/p!prev, unfold beforebefore h

SaysSays:

For h!next/h!prev, unfold fromfrom h

For p!next/p!prev, unfold beforebefore h

16

Types make the analysis robust with Types make the analysis robust with respect to how checkers are writtenrespect to how checkers are written

InstanceInstance dll(h, p) =if (h = null) then

trueelse



InstanceInstance

dll0(h) =if (h!next = null)

thentrue

elseh!next!prev = h

and dll0(h!next)

Alternative doubly-linked list checkerAlternative doubly-linked list checker

Doubly-linked list checker (as before)Doubly-linked list checker (as before)

Different types for different unfolding

Different types for different unfolding

cur

cur

17

Summary of checker parameter Summary of checker parameter typestypes

Tell wherewhere to unfold for whichwhich fields

Make analysis robustrobust with respect to how checkers are written

Learn where in summaries unfolding won’t help


Can be inferred automaticallyinferred automatically with a fixed-point computation on the checker definitions

Can be inferred automaticallyinferred automatically with a fixed-point computation on the checker definitions

18

Results: PerformanceResults: Performance

Benchmark

Max. Num.

Graphs at a

Program Pt

Analysis

Time (msms)

singly-linked list reverse 1 0.6

doubly-linked list reverse 1 1.4

doubly-linked list copy 2 5.3

doubly-linked list remove 5 6.5

doubly-linked list remove and back 5 6.8

search tree with parent insert 5 8.3

search tree with parent insert and back

5 47.0

two-level skip list rebalance 6 87.0

Linux scull driver (894 loc) (char arrays ignored, functions inlined)

4 9710.0

Times negligible for data structure operations (often in sec or 1/10 sec)

Times negligible for data structure operations (often in sec or 1/10 sec)ExpressivenessExpressiveness:

Different data structures

ExpressivenessExpressiveness: Different data structures

Verified shape invariant as given by the checker is preserved across the operation.

Verified shape invariant as given by the checker is preserved across the operation.Bor-Yuh Evan Chang - Extensible Shape Analysis by Designing with the User in Mind

TVLA: 850 msTVLA: 850 ms

TVLA: 290 msTVLA: 290 ms

Space Invaderonly analyzes lists (built-in)

Space Invaderonly analyzes lists (built-in)

19

ConclusionConclusion

Key Insight: Checkers as specificationsDeveloper View: Global, Expressed in a familiar

styleAnalysis View: Capture developer intent,

Not arbitrary inductive definitions

Constructing the program analysisIntermediate states: Generalized segment Generalized segment

predicates

Splitting: Checker parameter types with levelstypes with levels



What can inductiveshape analysis do for you?

http://xisa.cs.berkeley.edu

Extensible Shape Analysis by Designing with the User in Mind Bor-Yuh Evan Chang Bor-Yuh Evan Chang,...

Documents

Transcript of Extensible Shape Analysis by Designing with the User in Mind Bor-Yuh Evan Chang Bor-Yuh Evan Chang,...