A Framework for Heavyweight Dynamic Binary...
Transcript of A Framework for Heavyweight Dynamic Binary...
![Page 1: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/1.jpg)
1
ValgrindValgrind A Framework for Heavyweight A Framework for Heavyweight
Dynamic Binary InstrumentationDynamic Binary Instrumentation
Nicholas Nethercote — National ICT AustraliaJulian Seward — OpenWorks LLP
![Page 2: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/2.jpg)
2
FAQ #1FAQ #1
• How do you pronounce “Valgrind”?
• “Val-grinned”, not “Val-grined”
• Don’t feel bad: almost everyone gets it wrong at first
![Page 3: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/3.jpg)
3
DBA toolsDBA tools
• Program analysis tools are useful– Bug detectors– Profilers– Visualizers
• Dynamic binary analysis (DBA) tools– Analyse a program’s machine code at run-time– Augment original code with analysis code
![Page 4: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/4.jpg)
4
Building DBA toolsBuilding DBA tools
• Dynamic binary instrumentation (DBI)– Add analysis code to the original machine code at run-time– No preparation, 100% coverage
• DBI frameworks– Pin, DynamoRIO, Valgrind, etc.
Tool Framework Toolplug-in+ =
![Page 5: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/5.jpg)
5
Prior workPrior work
• Potential of DBI has not been fully exploited– Tools get less attention than frameworks– Complex tools are more interesting than simple tools
Complex toolsSimple toolsInstrumentation capabilitiesFramework performance
Not well-studiedWell-studied
![Page 6: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/6.jpg)
6
Shadow value toolsShadow value tools
![Page 7: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/7.jpg)
7
Shadow value tools (I)Shadow value tools (I)• Shadow every value with another value that describes it
– Tool stores and propagates shadow values in parallel
Leaked secrets“Secret tracker”
Array bounds violationsAnnelidRun-time type errorsHobbes
Uses of undefined valuesMemcheck
Dynamic dataflow graphsRedux
Shadow values help find...Tool(s)
InvariantsDynCompB
Uses of untrusted valuesTaintCheck, LIFT, TaintTracesecurity
bugs
properties
![Page 8: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/8.jpg)
8
MemcheckMemcheck
• Shadow values: defined or undefined
• 30 undefined value bugs found in OpenOffice
sh(R1) = addsh(R2, R3)R1 = R2 + R3
Shadow operationOriginal operation
complain if sh(R1) is undefinedif R1==0 then goto L
sh(R1) = sh(R2)R1 = R2sh(R1) = definedR1 = 0x12345678sh(p) = undefinedint* p = malloc(4)
![Page 9: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/9.jpg)
9
Shadow value tools (II)Shadow value tools (II)
• All shadow value tools work in the same basic way
• Shadow value tools are heavyweight tools– Tool’s data + ops are as complex as the original programs’s
• Shadow value tools are hard to implement– Multiplex real and shadow registers onto register file– Squeeze real and shadow memory into address space– Instrument most instructions and system calls
![Page 10: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/10.jpg)
10
Valgrind basicsValgrind basics
![Page 11: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/11.jpg)
11
ValgrindValgrind• Software
– Free software (GPL)– {x86, x86-64, PPC}/Linux, PPC/AIX
• Users– Development: Firefox, OpenOffice, KDE, GNOME, MySQL, Perl,
Python, PHP, Samba, RenderMan, Unreal Tournament, NASA, CERN– Research: Cambridge, MIT, Berkeley, CMU, Cornell, UNM, ANU,
Melbourne, TU Muenchen, TU Graz
• Design– Heavyweight tools are well supported– Lightweight tools are slow
![Page 12: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/12.jpg)
12
Two unusual features of ValgrindTwo unusual features of Valgrind
![Page 13: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/13.jpg)
13
#1: Code representation#1: Code representation
C&ACopy-and-annotate analysis
codeasmout
descriptionsasminannotate
interleaveinstrumentcopy
D&RDisassemble-and-resynthesize(Valgrind)
instrument
IR
asmout
asmindisassemble
resynthesize
![Page 14: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/14.jpg)
14
Pros and cons of D&RPros and cons of D&R• Cons: Lightweight tools
– Framework design and implementation effort– Code translation cost, code quality
• Pros: Heavyweight tools– Analysis code as expressive as original code– Tight interleaving of original code and analysis code– Obvious when things go wrong!
wrongbehaviour wrong
analysis
correctbehaviour
baddescriptionsbad IR
D&R C&A
![Page 15: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/15.jpg)
15
Other IR featuresOther IR features
• Writing complex inline analysis code is easy
BenefitFeature
As expressive as normal registersFirst-class shadow registers
Never have to find a spare registerInfinitely many temporaries
Fewer cases to handleRISC-like
Catches instrumentation errorsTyped, SSA
![Page 16: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/16.jpg)
16
#2: Thread serialisation#2: Thread serialisation
• Shadow memory: memory accesses no longer atomic– Uni-processors: thread switches may intervene– Multi-processors: real/shadow accesses may be reordered
• Simple solution: serialise thread execution!– Tools can ignore the issue– Great for uni-processors, slow for multi-processors...
![Page 17: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/17.jpg)
17
PerformancePerformance
![Page 18: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/18.jpg)
18
SPEC2000 PerformanceSPEC2000 Performance
(*) LIFT limitations:– No FP or SIMD programs– No multi-threaded programs– 32-bit x86 code on 64-bit x86 machines only
~1.5xPin/DynRIO, no-instrumentation 4.3xValgrind, no-instrumentation
10--180xMost other shadow value tools 3.6x (*)LIFT
22.1x (7--58x)Memcheck
![Page 19: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/19.jpg)
19
Post-performancePost-performance
• Only Valgrind allows robust shadow value tools– All robust ones built with Valgrind or from scratch
• Perception: “Valgrind is slow”– Too simplistic– Beware apples-to-oranges comparisons– Different frameworks have different strengths
![Page 20: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/20.jpg)
20
Future of DBIFuture of DBI
![Page 21: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/21.jpg)
21
The futureThe future
• Interesting tools!– Memcheck changed many C/C++ programmer’s lives– Tools don’t arise in a vacuum
• What do you want to know about program execution?– Think big!– Don’t worry about being practical at first
![Page 22: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/22.jpg)
22
If you remember nothing else...If you remember nothing else...
![Page 23: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/23.jpg)
23
Take-home messagesTake-home messages• Heavyweight tools are interesting• Each DBI framework has its pros and cons• Valgrind supports heavyweight tools well
www.valgrind.org
![Page 24: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/24.jpg)
24
(Extra slides)(Extra slides)
![Page 25: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/25.jpg)
25
The past: performanceThe past: performance
• Influenced by Dynamo: dynamic binary optimizer
• Everyone in research focuses on performance– No PLDI paper ever got rejected for focusing on
performance“The subjective issues are important — ease of use and robustness,but performance is the item which would be most interesting for theaudience.” (my italics)
• Slow tools are ok, if sufficiently useful
![Page 26: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/26.jpg)
26
Shadow value requirementsShadow value requirements
• Requirements:– (1) Shadow all sta te– (2) Instrument operations that involve state– (3) Produce extra output without disturbing execution
![Page 27: A Framework for Heavyweight Dynamic Binary Instrumentationnjn.valgrind.org/pubs/valgrind2007-talk.pdf · 2007-06-26 · •Dynamic binary instrumentation (DBI) –Add analysis code](https://reader034.fdocuments.net/reader034/viewer/2022042804/5f50f40e142a191ae527e7c2/html5/thumbnails/27.jpg)
27
RobustnessRobustness• Q. How many programs can Valgrind run?
– A. A lot
• Valgrind is robust, Valgrind tools can be
• SPEC2000 is not a good stress test!– Reviewer: “If the authors want to claim that their tool is to
be used in real projects, then they would need to evaluatetheir tools using the reference inputs for the SPEC CPU2Kbenchmarks.”