Chris Wilson and David L. Dill

Computer Systems Laboratory

Stanford University

June, 2000

Reliable Verification Using Symbolic Simulation with Scalar Values

Reliable Verification Using Symbolic Simulation with Scalar Values

Verification BottleneckVerification Bottleneck

time

Bug rateMany

“easy”

fewer

“hard”

Directed testing

Random testing

“purgatory”

tapeout

Current ApproachCurrent Approach

time

Bug rate Directed testing

random testing

Model checking

emulation

semi-formal methods

Our ApproachOur Approach

time

Bug rate

Key issue:

Reliability!

Symbolic simulation

ReliabilityReliability

Definition: Always gives some coverage when resource limits

encountered. Gives coverage proportional to effort.

Ease of use predictable coverage useful feedback easy to debug

EfficiencyEfficiency

Efficiency = Coverage/Unit Effort

Coverage specified functionality “input space”

Effort manpower computer resource usage schedule

Reliability vs. EfficiencyReliability vs. Efficiency

Reliability

Efficiency

Directed testingRandom testing

Emulation

Model Checking% of bugs

found

GoalGoal

Have the reliability, ease of use of directed testing.

AND…

efficiency equal or greater than random testing.

Reliability vs. EfficiencyReliability vs. Efficiency

Reliability

Efficiency

Directed testingRandom testing

Emulation

Model Checking

Target area

Symbolic test = directed test with symbolic values23

408

05

0

<d1,d2,d3,d4>

<a1,a2,a3,a4>

<c1,c2,c3>

Symbolic SimulationSymbolic Simulation

datain

address

interrupt

dataout

DUT

req valid

request

=

counter

0

1

“read”

“write”

=0

pass/

fail

dly

Symbolic SimulationSymbolic Simulation

Efficiency 1 symbolic test <=> many directed tests.

Ease of use short tests => easy to write, debug.

Blow up? BDDs too unpredictable.

How to prevent blow up?

Quasi-symbolic simulationQuasi-symbolic simulation

Symbolic simulation externally

scalar values internally simulation run requires constant memory.

Key ideas Don’t compute exact value unless necessary.

many don’t cares in large designs. Trade time for memory.

Multiple runs to generate exact values.

Don’t care logic

Basic AlgorithmBasic Algorithm

&

&&

&Xaa

Xbb

Xcc

Symbolic variable

X-a-a

Xaa

0

Obeys law of excluded middle!

X

Conservative approximation

X

XX “traditional”

X value

0

Don’t care variables

Decision ProcedureDecision Procedure

Xaa

Xbb

X

X

X

&

O

O

Davis-Putnam AlgorithmDavis-Putnam Algorithm

Tree Search… Davis, Logemann, Loveland [DPLL62].

Xa=0 a=1

Xb=0 b=1

0 0

X

0

b=1

evaluate case split

unit propagate

Decision ProcedureDecision Procedure

X

?

a=0 a=1

Variable selection heuristic:

pick relevant variable by propagating from inputs.

&

&

O

Xaa

Xbb

X

X

X

00

0

Xbb0

1

0

Xbb

0 ?0

Test is Unsatisfiable!

ReactivityReactivity

Reactive Test test behavior depends on circuit.

Most tests require reactivity since goal is to find all bugs… must support reactivity efficiently.

Reactivity exampleReactivity example

Set ‘request’ = READ;

Set ‘reqv’ = “1”;

wait for ‘ack’;

check that ‘data’ = expected_data;

stop;

Reactivity exampleReactivity example

Set ‘request’ = READ;

Set ‘reqv’ = “1”;

wait for ‘ack’;

check that ‘data’ = expected_data;

stop;

What if ‘ack’ = “X”?

Wait StatementWait Statement

Set ‘request’ = READ;Set ‘reqv’ = “1”;wait for ‘ack’;

check that ‘data’ = expected_data;stop;

wait for ‘ack’ == “1”;

‘ack’ == F = “X”

‘ack’ == T = “X”

Virtual thread

wait for ‘ack’;

Cycle 2Cycle 2

check that ‘data’ = expected_data;stop;

‘ack’ == T = “X”

wait for ‘ack’;

‘ack’ == F = “X”

‘ack’ == F = “X”

StoppingStopping

check that ‘data’ = expected_data;stop;

Guard = “X”

Stop? or not stop?

Modify Davis-Putnam...

if guard condition = “X” when stopped… prove that test can really stop in this cycle. Case split on guard condition.

case split on fail/pass condition only if stop = “1”.

StoppingStopping

StoppingStopping

Modify Davis-Putnam...

if guard condition = “X” when stopped… prove that test can really stop in this cycle. Case split on guard condition.

Do not allow unit propagation.

case split on fail/pass condition only if stop = “1”. Unit propagation is allowed.

Disallowing unit propagation allows

method to be complete.

Related WorkRelated Work

BDD-based Symbolic Simulation STE [BryantSeger95], Innologic.

Sequential ATPG

SAT/ATPG-based Model Checking BMC [Biere99], [Boppana99]

Other SAT-based Semi-Formal Methods [Ganai99]

ExperimentsExperiments

Show that quasi-symbolic simulation can find bugs. Test case bugs do not cause bottlenecks.

Demonstrate graceful degradation get good coverage if simulation time limit hit.

Experiment 1Experiment 1

Write/debug testcase for “hard” bug. 140K gate industrial design. Not found in simulation or bringup!

Four possible results SAT - test case error. TIMEOUT - test case error (device timeout.) UNSAT - no bug found. BUG - bug found.

Experiment 1Experiment 1

SAT 19 3.8 31.4

TIMEOUT 22 1.6 49.0

UNSAT 9 52.3 445.9

BUG 1 78 863.0

cases evals time(sec.)

Experiment 2Experiment 2

Time limit hit!Highest

covered

sub-node

Experiment 2Experiment 2

0

5

10

15

20

25

30

10 13 16 19 22 25

highest node

highestcompletedsubnode

Number of dependent variables in the test

Maximum

tree size

ConclusionsConclusions

Want to find all bugs faster. Reliability is key.

Use quasi-symbolic simulation has the efficiency of random testing. And reliability of directed testing.

Experiments show it can be used as primary verification method.