Program Checking

Sampath Kannan

University of Pennsylvania

Talk Outline

• Traditional software reliability paradigms.• Program Checking• Examples• Extensions• Spot checking• Streams and stream checking• Monitoring JAVA programs• Future Directions

Traditional Approaches

Testing:

• ad hoc or random test cases.• Need to know answers to these cases.• No mathematical guarantees.• May look inside program.• Done during design phase.

Verification:

• Proves programs correct.• Hard to do.• Proofs not human-friendly.• Looks inside programs.• Done during design phase.

Program Checking [BK95]

I O

I1I2 . . .Ik

Ok . . .O2O1

Program

Checker

CoinTosser

Checker

“Correct” if P is correct on all inputs.

“Buggy” if P is wrong on input I.

Checkers outputs are as above with high probability.Notion of correctness of P assumed to be provided by some rigorous means.

Example – Matrix Multiplication

Checker

A, B C

CoinTosser

vector v

A(Bv) = Cv?

Program

Freivalds ’79.

Example – Graph Isomorphism

1

2

3

45

6

7

89

10

1

10

8

4

3

2

6

9

5

7

G H

ProgramG, H

“Yes”, here is isomorphism

“No”

• Pick G or H at random.• Permute to get graph K.• Ask the program (G,K).• If K obtained from G

Expect “Yes”.• If K obtained from H

Expect “No”.• Call program “Buggy” if it does not meet expectation.

If Program says “No”:

Checker[BK95]

Program Checking Paradigm• Checking done at run-time:

• Overhead to program.• Errors detected as they happen.• Environment errors also detected.• Actual implementation is checked.

• Checkers are problem specific:• Checker design requires ingenuity.• Not universal paradigm.• Works against all programs for a problem.• Don’t need to look at inside of program.

Checking Correctness of Memory

Unreliable Memory

UserChecker

Reliable memory

checkerrequests

userrequests

Can check using O(log N)-sizedreliable memory that an adversarialmemory (RAM, Stack, Queue) of size N is functioning correctly. Techniques used:Special classes of hash functions, cryptographic primitives.

BEGKN ’94.

Problems/Areas with Checkers

• Linear Algebra• Group Theory• Arithmetic• Number Theory• Polynomial Algebra• Computations with Real Numbers• Sorting• Data Structures• Graph Problems• Combinatorial Optimization

Summary of Part 1• Program checking works on mathematically “clean” problems... but it takes a fair amount of effort to come up with a good checker.

• The definition requires the checker itself to be “simpler” (faster) than the program being checked, but this does not count the cost of additional calls to the program.

Challenge: 1) Reduce Overhead2) Design checkers for more problems

So that the technique can be applied to systems w.low resources.

Spot-Checking

Question: Can we settle for an “approximate” notion of correctness in order to drastically reduce overhead?

Answer: We can in some cases. Using spot checks [EKKRV ’99]

Spot Checking Sorting

A program for sorting is approximately correct if(1 – ) fraction of the output is in ascending orderand (1 - ) fraction of the output elements are inputelements.

We want a spot-checker that:Says “okay” to correct outputSays “Buggy” to output that is not even approximately correct.

How do we design such a spot-checker?

Repeat O(1/) times• Pick random element x from input.• Binary search for x in output.• If (x not found) report “Buggy”

Report “okay”.Method works because:

All elements that are successfully found bybinary search form an ascending sequence.We are verifying that there is a large ascendingsequence and that most input elements are inoutput.

Checker takes O(log n) time!

Streams

• Deluge of time-dependent data... if we don’t process it soon, it will become irrelevant! or it will be too late!

• Need to change both• What we ask about the data.• How we find the answer to our questions.

• Need good theoretical models to model the constraints under which we operate.

Model

Processor

Memory

could be a

CISCO routerwith netflowsoftware.

(size is small relative tostream size.)

0 1 0 0 1 0 1 1 0 1 0 1 0 1 1 0 0 0 0 1 0 1 0 0 1 1 1 1 1 0 1 0 1 0 1 1 0 1 1 0 1 0 1 1 0 1 1 1 0

[FKSV ’99.]

Detecting Anomalies

Is this the output of a fair coin?

Given streams of data describing internet traffic flowis there an anomalous day?

Application to intrusion detection?

Can you stand on Times Square, watch the NYSE tickertape go by twice and decide whether there was a bigchange in the stock market in between?

0 1 0 0 1 0 1 1 0 1 0 1 0 1 1 0 0 0 0 1 0 1 0 0 1 1 1 1 1 0 1 0 1 0 1 1 0 1 1 0 1 0 1 1 0 1 1 1 0

Main Results on Streams

. . . (1, 3), (7, 12), (3, 6), (2, -1), (4, -5), (7, 6), (8, 0), (1, -4), (9, 5), (3, -2), (5, -10), (2, 6) . . .

What is the L1 distance between these two functions?(i.e., )

|)()(| xgxf

Our solution: Above distance can be approximatedarbitrarily closely using memory that is only logarithmicin length of stream.

Stream computing will become increasingly importantas will the problem of verifying properties of data streams.

Run-time Monitoring and Checking

Sampath Kannan

Moonjoo Kim

Insup Lee

Oleg Sokolsky

Mahesh Viswanathan

Objectives

• Static analysis – abstract model

• Dynamic behavior checking– consistency between abstract model

and implementation

• To provide a framework for automatic generation of monitors and checkers

Fundamental Issues

• How does a monitor gather information from a running system?

• How does the monitor relate to requirements?

• How do we integrate dynamic monitoring with static analysis?

• Can monitor be used to steer a system?• What mathematical guarantees do

monitors provide?

System SpecSystem

Spec

RequirementSpec

RequirementSpec

Formal verification

Design

SystemImplementation

SystemImplementation

MonitoringScript

MonitoringScript

Implementation

Checker/CorrectorChecker/Corrector

SystemSystem FilterCommunication

Run-time Check

MAC Architecture

EventHandler

EventHandler

CorrectorCorrector

CheckerChecker

Design Issues• Filter

– passive versus active– when to take snapshot

• Event Handler– mapping between concrete state and abstract

event

• Checker– inclusion based on trace, ready semantics,

bisimulation

• Corrector– how to provide feedback

Properties

• Safety– e.g.: The gate is always closed while a train

is in crossing.

• Security– e.g.: detecting denial of service attack.

• Performance, Real-time– e.g.: QoS - does my car accelerate from 0-

60 in 5 sec’s?

• Resource usage– e.g.: 10 MB of memory.

Future Directions• Run-time assurance of correctness is an attractive complement to static analysis.

• Traditional correctness paradigms are more able to deal with flow-of-control correctness, type checks and the like. The big gap is in ensuring correctness of mathematically sophisticated computation. Program checking fills this gap.

• As we begin to reason about large programs and large volumes of input and output we need non-traditional models (both in theory and practice) for talking about efficient computation.