Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai...
-
Upload
rosanna-dickerson -
Category
Documents
-
view
225 -
download
0
Transcript of Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai...
![Page 1: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/1.jpg)
Using Model-Checking to Debug Device Firmware
Sanjeev KumarMicroprocessor Research Labs, Intel
Kai LiPrinceton University
![Page 2: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/2.jpg)
Using Model Checking to Debug Device Firmware 2
Programmable Devices
Network Card
Bus
CPU
D
MemDisk
Network
CPU
D
D
D
Mem
Main CPUMain CPU Main MemoryMain Memory
Move functionality from main CPUs to devices
Device firmware is more complex
![Page 3: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/3.jpg)
Using Model Checking to Debug Device Firmware 3
Firmware for Programmable Devices
• Difficult to write and debug– Use concurrency
• Inherently difficult to program correctly
– Optimized for high performance• Tradeoff program simplicity for performance
– Limited debugging support available
• Firmware reliability is important– Firmware is trusted by the OS– Bugs in firmware can crash the entire machine
• Model checking is a promising approach
![Page 4: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/4.jpg)
Using Model Checking to Debug Device Firmware 4
Model CheckingSystematically verify properties of concurrent systems• Using State-space exploration
– Try all possible scheduling options
• Advantages– Automatic – Produces counter example
• Disadvantages– Computationally expensive
• Exponential search
• Models cannot be too big
Model Test Code
Program
![Page 5: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/5.jpg)
Using Model Checking to Debug Device Firmware 5
Using Model Checking to Debug Firmware
• Extracting models from programs– Manually (by the programmer)– Automatic (using the compiler)
• Reduces programmer effort• Reduces mismatch between program and model
• Extracting smaller models models– Support for abstraction
• Discard irrelevant details in the program
![Page 6: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/6.jpg)
Using Model Checking to Debug Device Firmware 6
Our Work
• Extract abstract models using a compiler– General compiler techniques
• Programmer controls the abstraction process– Specifies what needs to be abstracted
• Compiler performs the abstractions conservatively
– Extract models for Spin model checker from programs written in the ESP language
– Make practical choices• Goal: Debugging and not Verification
• Used to debug VMMC firmware for a network card– Found 7 bugs that can cause the firmware to deadlock
• Could not find these bugs without support for abstraction
![Page 7: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/7.jpg)
Using Model Checking to Debug Device Firmware 7
Related Work
• Manual Model Extraction– Harmony, RUBIS, Plan 9, Fluke OS
• Used model checking to debug a subsystem
• Automatic Model Extraction– Teapot, Promela++, Esterel, Java Pathfinder
• Domain-specific and general-purpose languages
• Automatic Extraction + Support for abstraction– Feaver, Lie et. al., Bandera
![Page 8: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/8.jpg)
Using Model Checking to Debug Device Firmware 8
Outline
• Background• Extracting Abstract Models using a Compiler
– From ESP Language– For Spin Model Checker
• Evaluation: Debugging VMMC Firmware• Conclusion and Future Work
![Page 9: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/9.jpg)
Using Model Checking to Debug Device Firmware 9
ESP: A Language for Programmable Devices
pgm1.spin
pgmN.spin
pgm.C
ESP CompilerESP Compilerpgm.ESP
help.CGenerate
Firmware
Develop
and
Test
using
Model
Checker
test1.spin
testN.spin
Goals1) Easy to program2) Allow extensive testing3) Performance
![Page 10: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/10.jpg)
Using Model Checking to Debug Device Firmware 10
The ESP Language
• Concurrent language: Processes & Channels– Pure message-passing communication– in, out, alt operations on channels– Channels are synchronous or unbuffered– Processes and channels are static
• A number of interesting features– Explicit memory management scheme that uses
model-checking to ensure safety– Supports dispatch on channels– Efficient and powerful interface to C
![Page 11: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/11.jpg)
Using Model Checking to Debug Device Firmware 11
Extracting Models for Spin
• Step 1 : Detailed models [ PLDI’01 ]– Translate each language construct into Spin– Simple translation
• int, bool, records, arrays, unions• If-then-else, while-loops• process, channel
– Spin does not support• Dynamic memory allocation & Pointers• Additional bookkeeping necessary to support these
• Can be used to check local properties– Debug subsystems (1-2 processes) separately– Too big to debug the entire system
![Page 12: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/12.jpg)
Using Model Checking to Debug Device Firmware 12
Extracting Abstract Models Conservatively
• Step 2: Abstract models– Necessary to check global properties (like deadlocks)– Drop unnecessary details
• Depending on the property being verified
– Programmer controls the abstraction• Abstraction specified by the programmer
– Drop variables– Drop fields from records and unions
• Compiler used the abstraction specified conservatively– Could introduce fast-positive bugs– All bugs in the programs will be present in the extracted model– Involves dealing with a number of tricky cases
![Page 13: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/13.jpg)
Using Model Checking to Debug Device Firmware 13
Examples
$b2: boolean = true;
...
$b1: boolean = b2;
$b2: boolean = true;
...
$b1: boolean = b2;
type recT = #record of { int count; }type recT = #record of { int count; }
$r1: recT = {0};
if (b) { r2 = r1; }
...
r1.count = 5;
$r1: recT = {0};
if (b) { r2 = r1; }
...
r1.count = 5;
if
:: b1 = true
:: b1 = false
fi
if
:: b1 = true
:: b1 = false
fi
if
:: r2.count = 5
:: skip
fi
if
:: r2.count = 5
:: skip
fi
X
X
X
XX
Conservative: Use nondeterminism to broaden the state-space searched
![Page 14: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/14.jpg)
Using Model Checking to Debug Device Firmware 14
Outline
• Background• Extracting Abstract Models using a Compiler• Evaluation: Debugging VMMC Firmware• Conclusion and Future Work
![Page 15: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/15.jpg)
Using Model Checking to Debug Device Firmware 15
VMMC
• High-performance communication– Bypass OS for data transfers
• Used Myrinet network cards– Gigabit network– 33 MHz CPU, 1 MB memory
• Original VMMC firmware– Implemented in C
• Several man-years of debugging
• Still encounter bugs
• Some involve complex race conditions that are triggered only occasionally
Data
OS
Network Card
Application
Network
![Page 16: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/16.jpg)
Using Model Checking to Debug Device Firmware 16
Debugging VMMC Firmware
• Reimplemented VMMC firmware using ESP• Used model checking to debug
– Global property of program (deadlocks)– Hard-to-find bugs– Found 7 bugs using abstract models
• 4 Bugs would cause deadlock during normal operations• 3 Bugs would be triggered only by a malicious machine
– Could not find these bugs without abstractions
• No firmware bugs encountered on device– Microbenchmarks– SPLASH2 parallel application suite
• On a 16-processor SMP cluster
![Page 17: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/17.jpg)
Using Model Checking to Debug Device Firmware 17
Resource used for Model Checking
Spin ModeStates Searched
(in Millions)CPU Time
(in seconds)Memory
(in Mbytes)
Exhaustive 0.38 84.0 268.35*
Partial mode 99.7 14250.0* 167.92
• Only partial search was possible• Even partial searches were effective
* Limiting Resource
VMMC Firmware
![Page 18: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/18.jpg)
Using Model Checking to Debug Device Firmware 18
Model extracted from VMMC Firmware
File Lines of Code
ESP Program 453
Abstraction Specification 108
Abstract Model Extracted 2202
Test Code 128
• Programmer only write a small amount of Spin Code• Program can be rechecked with little effort
![Page 19: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/19.jpg)
Using Model Checking to Debug Device Firmware 19
Outline
• Background• Extracting Abstract Models using a Compiler• Evaluation: Debugging VMMC Firmware• Conclusion and Future Work
![Page 20: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/20.jpg)
Using Model Checking to Debug Device Firmware 20
Conclusions
• Use compiler to extract abstract models– Evaluation: Debugged VMMC firmware
• Using compiler to extract models is good– Significantly reduces effort required to model check
• Abstraction is required– To check global properties like deadlocks
• Programmer can control the abstraction– Compiler is conservative– Does not require the programmer to be correct
• Only partial search was possible– Still effective in finding bugs
![Page 21: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/21.jpg)
Using Model Checking to Debug Device Firmware 21
Future Work
• Optimizations to reduce size of state space– Eliminating more redundancies
• Quantify the effectiveness of a partial seach– Estimate the fraction of state-space searched
• Use type systems to reduce the size of state space that has to be searched
![Page 22: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/22.jpg)
Using Model Checking to Debug Device Firmware 22
To find out more, Visit http://www.cs.princeton.edu/~skumar
Questions?
![Page 23: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/23.jpg)
Using Model Checking to Debug Device Firmware 23
Debug not Verify
• Several sources of incompleteness and unsoundness remain– Programmer supplied Spin code– Partial model checking
• The goal is to isolate/reduce the unsound portions of the code
![Page 24: Using Model-Checking to Debug Device Firmware Sanjeev Kumar Microprocessor Research Labs, Intel Kai Li Princeton University.](https://reader035.fdocuments.net/reader035/viewer/2022062309/56649e625503460f94b5d7d9/html5/thumbnails/24.jpg)
Using Model Checking to Debug Device Firmware 24
ABCDEFABCDEF
ABCDEFABCDEF
ABCDEFABCDEF
Abcdef Ghijk
Abcdef Ghijk
Abcdef Ghijk
Abcdef Ghijk
Abcdef Ghijk
Abcdef Ghijk