Using a Formal Specification and a Model Checker to Monitor and Guide Simulation Verifying the...
-
Upload
anthony-chase -
Category
Documents
-
view
232 -
download
2
Transcript of Using a Formal Specification and a Model Checker to Monitor and Guide Simulation Verifying the...
Using a Formal Specification and a Model Checker
to Monitor and Guide Simulation
Verifying the Multiprocessing Hardware of the Alpha 21364 Microprocessor
Serdar TasiranKoç University, Istanbul, Turkey
(formerly Systems Research Center, Compaq/HP)
Yuan Yu (Microsoft Research, formerly Compaq)Brannon Batson (Intel, formerly Compaq)
The Problem
Given a formal specification:
An algorithm-level, executable description an implementation:
Hardware described at RT level
Verify that All executions of the implementation are
consistent with the specification
An implementation verification problem Design verification handled separately:
Verifying properties of the specification (absence of deadlock, …)
Earlier work:Validation Guided by Coverage
SimulatorInputs RT LevelDesign
Model CheckerMonitors, reference
model
Coverageanalysis
Validation Guided by Formal Spec Coverage
SimulatorInputs RT LevelDesign
Model Checker
Algorithm level Formal Spec
Abstraction map
Checks if spec is
violated Collects coverage data Generates traces tocoverage targets
Validation Guided by Formal Spec Coverage
SimulatorInputs RT LevelDesign
Model Checker
Algorithm level Formal Spec
Abstraction map
Alpha 21364 Multiprocessor System Block Diagram
1M
IO0
M
IO2
M
IO3
M
IO
5M
IO4
M
IO6
M
IO7
M
IO
9M
IO8
M
IO10
M
IO11
M
IO
Distributed shared memory (seamless SMP) Up to 256 processors, 32 GB per processor Each processor “owns” portion of memory Responsible for consistency of memory it owns: Cache coherence
Closer look: Chip Block Diagram
C
Z1
mem
systemdata
buffers
R
core&
cache
Z0
mem
IO
EV7
L2 Cachecontroller
Routingprotocolengine
Memorycontroller
s
EV7 Cache Coherence:
Hardware implementationof multiprocessing engine: ~20K lines of HDL code
C
Z1
mem
systemdata
buffers
R
core&
cache
Z0
mem
EV7
SVDB,FB0,FB1
Why is the problem difficult? Beyond the reach of automatic formal methods
Complex hardware, architecture :
Thousands of state variables per processor
Parallelism, several deep pipelines, speculation, redundancy
Complex system configuration: Need several processors to exercise certain scenarios
Decomposition methods difficult for non-specialists, large design teams
Complete verification of hardware against protocol not practical
Simulation only viable approach Even simulation is expensive
Must make judicious use of simulation resources
Validation Guided by Formal Spec Coverage
SimulatorInputs RT LevelDesign
Model Checker
Algorithm level Formal Spec
Abstraction map
The Spec: EV7 Cache Coherence Protocol
Distributed shared memory Each address belongs to a
“home node” but may be in other caches
Directory-based protocol Cache states: Modified (Dirty), Exclusive (Clean), Shared, Invalid Directory states: Local, Shared, Exclusive, Incoherent
Directory distributed, stored in memory at each node with data
CPU requests that miss in local caches are sent to home node
Home node may forward request to other nodes Directory In Flight Table (DIFT) keeps track of pending requests
1M
IO0
M
IO2
M
IO3
M
IO
5M
IO4
M
IO6
M
IO7
M
IO
9M
IO8
M
IO10
M
IO11
M
IO
Formal Specification of Protocol
Spec written in Temporal Logic of Actions [Leslie Lamport]
TLA: Formal language for writing high-level, executable specs of concurrent, reactive systems
Very expressive. Incorporates first-order logic, set theory, temporal operators sets, queues, records, tuples, …
Written by architects Some help from verification researchers Started from text documents at the same level of
abstraction Spec is a TLA formula, around 2000 lines, 60 pages
Formal Specification of Protocol
Architecture encapsulated in TLA spec at algorithm level
Spec state variables correspond to Contents of major data structures
E.g. DIFT linked list of transactions per address Messages in flight
Protocol transactions described by “TLA actions”
DIFT[addr0] DirectorystateCommand
SharedtoDirty {0,1,2}
Cachestate Response
SharedtoDirty
SharedtoDirty
Shared
Evicting
S2DSuccess
S2DFailureExclusive
Invalid
addr0
addr0
addr0
S1S2
HR
ReadMod
S3
BlkExclusiveCnt(3)
SharedInv
SharedInv
SharedInv
Action name
State variable updates
Messages sent
Preconditions
Macro definitions
……
…
The request is a “Read Modify”The caches and the victim buffer
do not have a recent
copy
The block is in the
“Shared” directory state
Tell the requestor how many “Invalidate
Acknowledge” messages to wait for before modifying the line
Send “Invalidate” messages
to the sharers
Update the directory
stateFree the memory controller state associated with this transaction
Validation Guided by Formal Spec Coverage
SimulatorInputs RT LevelDesign
Model Checker
Algorithm level Formal Spec
Abstraction map
The TLC Model Checker (Yu et. al.)
Explicit-state model checker for TLA descriptions Stores set of states reached during exploration For large state spaces, can store a “signature” of a
state instead e.g. projection onto a subset of state variables (a “view” )
Can generate error-trace to states violating correctness invariants
Validation Guided by Formal Spec
Simulator
Model Checker
Inputs
High level Formal Spec
RT LevelDesign
Abstraction map
Checks if spec is
violated
Formal Spec as Simulation Monitor
fabs : Abstraction
mapping
Model checker (TLC) checks if transition
is legal
Implementation State-Space
SpecState-Space
Model checker (TLC) checks if transition
is legal
Formal Spec + Model Checker as Monitor
Benefits Spec can be analyzed formally
Not true of some popular high-level description languages
More rigorous checking of each simulation run Discrepancy from spec detected as soon as it occurs
Before it causes observable data corruption
Model checker + formal spec: More modular: Specification and checking code separate
More reliable, easier to maintain than hand-written monitors More sophisticated properties can be checked than
automatically generated assertions
Formal Spec + Model Checker as Monitor
Price paid for benefit: Must write abstraction map
Unavoidable: Correctness checking code must reason at more abstract level Either informally
Existing checking code constructs objects from signals
Or formally, as in our approach
When model checker signals error Case 1: Error in implementation Case 2: Error in mapping Easy to distinguish which from the simulation run In both cases quality of validation improved
Iterative scheme to debug map and implementation
Validation Guided by Formal Spec Coverage
SimulatorInputs RT LevelDesign
Model Checker
Algorithm level Formal Spec
Abstraction map
Abstraction Map Issues
Protocol transactions appear atomic at spec level In the implementation they happen
over many clock cycles, interleaved with other transactions
In the hardware, a collection of lower level actions implement a protocol transaction.
Transaction 1
Transaction 2
Transaction 3
time
Low-level actions implementing a protocol transaction
time
Preconditions
Messages sent
Updates tostate
A protocol transaction
time
Commit point
Completion
CompletionCompleti
on
Commit point
Commit point
time
Abstract level
Concrete level
Delayed Aggregation of Actions
More implementation intricacies
time
Arbiter choosesthis DIFT entry
Memory read requestsent to Zbox middle end
Response arrives fromZbox middle end
Response written to DIFT
Response gets decoded
Map = composition of two simpler maps
Implementation
Intermediate level
Spec level
Response written to DIFT
Two-part recipe for map
Hardware signal transitions
Protocol events
Protocol transactions
Should be written by system architects
Should be written by
component implementers
Advantages of mapping technique
Modular description Clean division of responsibilities Distinguishes hw block integration errors from hw
block implementation errors
Easier to maintain the map Updates to the two components independent
Portions of map re-usable for next generation of design
Mapping technique applicable to other hardware implementing a complex protocol
Validation Guided by Formal Spec Coverage
Simulator
Model Checker
Inputs
High level Formal Spec
RT LevelDesign
Abstraction map
Collects coverage data Generates traces tocoverage targets
Formal Spec Coverage
Implementation State-Space
SpecState-Space
Model checker stores visited
spec states
Unexploredspec states
Model Checker Measures and Improves Coverage
SpecState-Space
Coveragehole
Path generatedby model checker
Identify parts of spec not exercised by simulation Path in spec state space = unexamined scenario Very useful starting point for generating simulation inputs
Scripts convert protocol message sequences to RT-level inputs Trial and error for getting the timing between messages right
Problem: Spec state space often too large
Coverage Metric on Formal Spec
Problem: Spec state space often too large
Solution: Record and target coverage of selected variables in TLA spec
Explore all combinations of Memory controller FSM state Result of cache + victim buffer lookup Directory state Type of message
Verification team had collected coverage data on prior simulations using this metric
Demonstration of concept Selected difficult bug from EV7 bug database
Discovered during prototype testing of 8-processor configuration.
Bug manifestation in the implementation: Protocol state-machine hits deadlock state Unexpected victim received at DIFT from victim buffer No next state defined under this condition Simulation doesn’t violate spec until deadlock state
Bug manifestation at spec level: Assertion violation during model checking run
– Model checking a 3-processor, 1-address configuration takes < 5 minutes, ~30 MB
Assertion says “no victim in the victim buffer at this state” Assertion was part of original spec
Advantages of Formal Spec Coverage
Formal spec encapsulates design intent, important architectural features Full coverage = All scenarios, important structures exercised
Model checker used to Measure coverage, detect gaps Generate spec-level traces to reach coverage holes
Spec at same level of abstraction as existing simulation coverage data “Is this a real coverage gap or an unreachable scenario?” Can be answered using model checker
Implementation Details
Abstraction map ~12K lines of C++ code Roughly the same size as other, “informal” checking code No extra price for being formal
Compiled together with compiled-code simulator Simulator has facility for extra modules being invoked at each
cycle Structure of map code much like composition of two
combinational circuits, simulated in event-driven way
~100% run-time overhead with rudimentary implementation Efficiency was not a consideration Model checker takes negligible time
Conclusions
Novel approach uses formal spec and model checkeri. to monitor simulation
ii. to identify coverage gaps
iii. to guide input generation
Many benefits to having all three be based on same formal spec
Abstraction map required Provided recipe to make map construction practical
Found valuable by architects and verification engineers
EV8 design started with formal specification first!