Jinlin Yang and David Evans [jinlin, evans]@cs.virginia.edu Department of Computer Science...

Jinlin Yang and David Evans

[jinlin, evans]@cs.virginia.edu

Department of Computer Science

University of Virginia

PASTE 2004 June 7th 2004

http://www.cs.virginia.edu/ipa

Dynamically Inferring Temporal

Properties

PASTE 2004 Jinlin Yang 2

Temporal Properties

• Example: fopen/fclose

• Essential for program’s correctness

• Hard for humans to document correctly [Holzmann, FSE ’02 keynote]

• Can we infer them automatically?

Yes!


Related Work

• Value-based invariants– Daikon [Ernst, TSE, Feb. ‘01]– We focus on invariants in sequence of events

• Temporal properties– Recovering thread models [Cook, JSS ‘04]– Mining specification [Ammons, POPL ‘02]– Extracting component interfaces [Whaley, ISSTA ‘02]– We focus on relationship between 2 or 3 events:

greater automation, scalability


Our Approach

ProgramInstrumented

Program

Instrumentation

Test Suite

Execution Traces

Testing

Inferred Properties

CandidatePropertyPatterns

Infere

nc

e


Program Verification

Inferred Properties

Model

Report

DerivedPropertiesSatisfaction or

Counter-examples

Re

port G

en

eratio

n

Va

lida

tion

Program

Dyn

amic In

feren

ce


Program Evolution

Inferred Properties 1

DifferentProperties

Diffe

ren

ce

An

alyze

r

Inferred Properties 2

SharedPropertiesProgram

Version 1

Program Version 2

Dyn

amic In

feren

ce


Property Patterns

• Response pattern [Dwyer, ICSE ‘99]

[-P]* (P [-S]* S [-P]*)*

SPPSPS SPSP • Too weak

• 3 primitive patterns

• 4 derived patterns


Partial Order of Patterns

MultiEffectPSS

MultiCausePPS

EffectFirstSPS

AlternatingPSPS

OneCauseSPSS

OneEffectSPPS

CauseFirstPPSS

ResponseSPPSPS

Str

icte

r


Logical Relation of Patterns

MultiEffectPSS

MultiCausePPS

EffectFirstSPS

AlternatingPSPS

OneCauseSPSS

OneEffectSPPS

CauseFirstPPSS

• For each combination of two events

• Decide if they satisfy CauseFirst, OneCause, or OneEffect

• Find the strictest pattern

ResponseSPPSPS

Str

icte

r

Λ Λ Λ

Λ


Find Strictest Pattern

Trace 1: PSPS Trace 2: PPS All Traces

CauseFirst

OneCause

OneEffect

For any two events, determine the strictest pattern satisfied by all traces




CauseFirst

OneCause

OneEffect





CauseFirst

OneCause

OneEffect

CauseFirstΛOneEffect MultiCause



Implementation

• Automated testing and inference• Instrumentation and properties diff by hand• 900 lines of Perl code• 93 different events, 100 traces, each has 222 events on average:

less than an hour

ProgramInstrumented

Program

Instrumentation

Test Suite

Execution Traces

Testing

Inferred Properties

CandidatePropertyPatterns

Inferen

ce


Experiments• Hypotheses

– We can automatically extract interesting temporal properties.

– Differences of temporal properties among multiple versions can reveal interesting things.

• Target programs– Tour bus simulator (8 student submissions)– OpenSSL (0.9.6, 0.9.7-0.9.7d)– Simplified producer-consumer (in the paper)


Tour Bus Simulator

• Bus and each passenger is a separate thread

• Assignment in Fall 2003 graduate-level course (before we started this project)

• 8 submissions from the instructor


Testing

>cville_bus –N 2 –C 1 –T 2

Bus waiting for trip 1

Passenger 0 gets in

Bus drives around Charlottesville

Passenger 0 gets off

Bus waiting for trip 2

Passenger 1 gets in

Bus drives around Charlottesville

Passenger 1 gets off

Bus stops for the day

N, the number of people

C, the capacity of the bus

T, the number of trips

Executed each submission with 100 randomly generated inputs, where

20 < C ≤ 40

C+1 ≤ N ≤ 2C

1 ≤ T ≤ 10


Results: Tour Bus SimulatorPattern 7 Correct Versions 1 Faulty Version

Alternating waitdrives

MultiEffectdrivesgets offwaitgets offwaitgets in

waitdrives

waitgets in

MultiCause gets indrives gets indrives

CauseFirstgets ingets off gets ingets off

drivesgets offwaitgets off

• Differences reveal problems– Bus drives around before all passengers get off– Other problems


OpenSSL

• Widely used implementation of the Secure Socket Layer protocol

• 6 versions [0.9.6, 0.9.7, 0.9.7a-d]

• We focus on the handshake protocol


SR_CLNT_HELLO SW_SRVR_HELLO

SW_CERT SW_KEY_EXCH SW_CERT_REQ

SW_SRVR_DONE

SR_CERT SR_KEY_EXCH SR_CERT_VRFY

SR_FINISHED

SW_CHANGE SW_FINISHED

BEFORE+ACCEPT

SW_FLUSH OK

SW_FLUSH

Client Server


Testing

• Manually instrumented server

• Modified client: advance to a different state with 5% probability

• Executed each version of server with 1000 randomly generated clients


Inferred Alternating Patterns

0.9.6 0.9.7 0.9.7a 0.9.7b 0.9.7c 0.9.7d

SR_KEY_EXCHSR_CERT_VRFY

SW_CERTSW_KEY_EXCH

SW_SRVR_DONESR_CERT

Documented improvement

Fixed bug Race condition

7 alternating patterns same for all versions


Partitioning Traces

• Correct clients– Properties are exactly same as specified– All versions agree

• Faulty clients without server errors generated– Found one possible vulnerability of DOS attack

• Segmentation faults– A documented bug fixed since 0.9.7d


Summary of Experiments

• Useful in program evolution– Reveal interesting changes in OpenSSL– Identify unexpected differences

• Revealed bugs– Tour bus: identified faulty implementation

• Multiple implementations are rare, but multiple representations are common (i.e. design, model, code)

– OpenSSL: partitioning traces


Future Work/Research Questions• Automating instrumentation/validation

– Can we automatically identify interesting events?

• Identifying more useful patterns– Combine data-flow with events– More expressive, more events– Ed will talk more about this

• Prioritize results– How to identify interesting differences?

• Testing– How do the inferred properties vary with the test suite?


Conclusion

• Automatically inferring temporal properties is feasible

• Even very simple property patterns reveal interesting program properties


Thank You!http://www.cs.virginia.edu/ipa

This work is funded in part by the National Science Foundation.

We thank Marty Humphrey for providing the student submissions, Chengdu Huang for helping OpenSSL, Joel Winstead for discussion and comments on earlier version of this paper, anonymous PASTE reviewers for insightful comments, anonymous CS650 students who wrote the buggy code, and our research group members and CS graduate students at the UVa for helping improve the talk.

Jinlin Yang and David Evans [jinlin, evans]@cs.virginia.edu Department of Computer Science...

Documents

Transcript of Jinlin Yang and David Evans [jinlin, evans]@cs.virginia.edu Department of Computer Science...