Please remove your earplugs :-)
description
Transcript of Please remove your earplugs :-)
Please remove your earplugs :-)
Program Analyses: A Consumer’s Perspective
Matthias FelleisenRice UniversityHouston, Texas
History: Successes, Failures, Lessons
• soft typing (Wright)
• synchronization of futures (Flanagan)
• static debugging (Flanagan)
• optimizations (Flatt and Steckler)
• theory of analyses (with Amr Sabry)
RICE PLT
The Guiding Ideas
• is it there a need?
• is it useful?
• is it sound?
• motivation & goal
• analysis
• implementation
• experiences
• problems
Soft Typing: Goals & Motivation
• infer types for Scheme programs
• insert checks where conflicts:– program must run – program must respect types
• use type information: – within compiler– as feedback for user
Soft Typing: Example
(define (foldr a-function e alist) (cond [(empty? alist) e] [else (a-function (first alist) (foldr a-function e (rest alist)))]))
is it a function? is it a list?
(foldr (lambda (x y) (printf "~s~n" x)) void '(1 2 3))
(foldr “this is not a function” void '(1 2 3))
Soft Typing: Another Example
;; form = boolean | (boolean -> form)
;; taut : form -> boolean;; to determine whether _a-form_ is a tautology(define (taut a-form) (cond [(boolean? a-form) a-form] [else (and (taut (a-form true)) (taut (a-form false)))]))
(taut true)(taut (lambda (x) (or (not x) x)))
(taut not) ;; re-use pre-existing functions as “form”s
(taut taut) ;; even use taut on itself
Soft Typing: The Analysis
• Hindley-Milner with recursive types, unions, and some subtyping
• type algebra of records a la D. Remy
• add “slack variables” to unions so that unification always succeeds -- produce run-time checks for non-empty slack variables
Soft Typing: Implementation
• Soft Scheme covers all of R4RS
• some 6,000 lines of code
• analyzes itself
• is reasonably fast
Soft Typing: Experience w/ Optimizations
• copes with entire GAMBIT suite• inserts few checks (down to 80% or less
of Scheme w/o soft typing)• caution: it leaves checks that are
dynamically critical• time savings for average program: 15%• but: in some large examples: less than
5%
Soft Typing: Experience w/ Programmers
• can’t analyze programs in an incremental or a modular fashion
• imprecise on “practical” parts of Scheme: apply, append, values, …
• understanding types (size!)• understanding casts --- as difficult as
understanding ML type errors – works well for very small programs – nearly unusable for programs with 100s loc– reverse flow of information!
Soft Typing: The Lesson
adapt and extend Hindley-Milner
get all the “good” and the “bad” and some “more bad” from the result
NO SURPRISE HERE
Part II: Optimizing Future Synchronization
Futures: Motivation
• applying soft typing to non-type problems while building on success of the work (optimization)
• exploring alternatives to Hindley-Milner: – Peter Lee and Nevin Heintze– Amr Sabry on Shiver’s dissertation
• futures: semantics, analysis, compilation– Bert Halstead
Futures: Goal
• slatex a preprocessor for type-setting Scheme, written in Scheme
• the little Schemer: 10 chapters, 2hrs
• code is mostly FP, with few set!
• ideal for Scheme with futures
Futures: Functional Parallelism
• functional programs provide too much parallelism
• add future annotations so compilers know where to start parallel threads (if resources are available)
• make strict primitive functions synchronize with “future values”
Futures: A Silly Example
;; fib : number -> number (define (fib n) (cond [(= n 0) 1] [(= n 1) 2] [else (+ (future (fib (- n 1))) (future (fib (- n 2))))]))
the + operation synchronizes: 1,000,000 times for (fib 25)
Futures: A Large Example
(future (process-file “chapter1.tex”))
(post-process x (size x))
(for-each integrate (list x … ))
Value flow across procedure & module boundaries etcetc
Control flow
Futures: Semantics and Analysis
• developed a series of equivalent reduction semantics for future until synchronization parts was exposed
• defined an optimizing transformation assuming an “oracle” about value flow and control flow information
• proved soundness wrt sound oracle
Futures: Semantics and Analysis
An oracle is a subset of future-strict program positions
An oracle is valid for an execution state if every future-value is associated with a program position in the oracle.
An oracle is always valid for a program if it is valid for all reachable execution states.
THEOREM: If P is a program, O is an always valid program for P, then eval(P) = eval(optimize(P,O))
PROOF: compare two reduction semantics
Futures: Analysis
• based on Heintze’s set-based analysis, derive constraints – syntax-directed manner– interpret program operations in a naïve set-based
manner
• future creates an abstract placeholder
• close constraints under “transitive closure through constructors”
Futures: Use, Soundness of Analysis
• solve constraints:
• soundness:
oracle(P) = { program-point | placeholder is in closed(SBA-constraints) of program-point }
Fix program-points in P and copy thru reduction.Consider a reduction sequence of a program: P -> P1 -> P2 -> … -> Pn At each stage, program-points are associated withvalues. The oracle correctly predicts placeholders.
Futures: Implementation
• implemented analysis and optimizer for purely functional Scheme without any extras
• extended Gambit Scheme (by Marc Feeley)
• benchmarked the Gambit suite on a BBN with 1, 4, and 16 processors
Futures: Experiences with FP Programs
• benchmarks with 100 to 1,000 loc• reasonably fast analysis • measurements produce great results
– reduce number of synchronization operations from ~90% to ~10%
– huge win for sequential execution– time savings of between 35% for 4
processors to 20% for 16 processors
Futures: … with mostly-FP Programs
• the benchmark suite (and slatex) contains – larger programs – programs with variable assignment and structure
mutation
• the analysis didn’t scale to these programs on our machines: – space (500MB)– time (a night)– precision (interpretation of set!)– feedback (why is a synchronization still here?)
Futures: The Lessons
• set-based analysis works really well for toy functional programs
• set-based analysis doesn’t seem to scale to real programs that needed optimizations of the synchronization operations
• but: not everything is lost …
Part III: On to Static Debugging
Static Debugging: Motivation
• what can SBA find out about mostly functional programs?
• can we turn SBA information into useful feedback for the programmer?
• does SBA scale to large programs?
Static Debugging: Goal
• Can we scale SBA to the full language so that it yields useful results?
• Can we improve the performance so that the analysis copes with the entire code?
• Can we provide feedback, find bugs?
DrScheme: a programming environment for Scheme written in an extension of Scheme
Static Debugging: Set-Based Analysis
• extend SBA to R4RS and DrScheme– variable number of arguments, apply– multiple values– exceptions– objects– first-class classes– first-class modules – threads (unsound)– staged computation (macros)
Static Debugging: Set-Based Analysis
• modify SBA to cope with – if (if-splitting)– control (flow sensitivity)– Scheme’s large constants (quote)– tracking individual constants– Scheme’s form of polymorphism – a modicum of arithmetic
Static Debugging: Set-Based Analysis
• enrich SBA for programmer feedback– check all primitive operations: acceptable vs
inferred sets of values– high-light mismatch – display analysis results (as types)– illustrate potentially flawed data flow (as
flow graph/path)
Static Debugging: Implementation
• two versions: browser-based and DrScheme-based
• runs efficiently on the sample programs
• provides decent feedback
Static Debugging: Feedback 1
structure mutation
higher-order functions
Static Debugging: Feedback 2
potential conflicts
Static Debugging: Feedback 3
void might flow here
Static Debugging: Feedback 4
the source of the problem
the potential data flow
Static Debugging: Experience 1
• easy to use for class-size programs: parsers, interpreters, type checkers
• student experiment: controlled experiment; MrSpidey wins
• the team members don’t use it
Static Debugging: Problems 1
• the analysis can’t analyze programs with more than 3,000 loc
• the analysis can’t cope with units (at that point)
• the analysis isn’t “incremental”
Static Debugging: Componential SBA
• analyzing units relative to– imports – exports
• determining smaller, observationally equivalent set constraints
• re-calculate with full sets on demand
Static Debugging: Componential Analysis
Focus Unit
constraints
Othr Unit
constraints
simplified
YA Unit
constraints
simplified
Solution
Static Debugging: Feedback 5
function is used externally
click and re-compute focus
Static Debugging: Feedback 6
MrSpidey shows source unit
Static Debugging: Implementation 2
• implemented componential analysis
• for all of DrScheme
• analyzed system on itself in a few hours (50,000 loc)
Static Debugging: Experience 2
• analyzed the run-time system: – found few problems, few bugs– noticed imprecision
• conducted experiment with course:– worked well on small multi-unit projects– worked badly for large multi-unit projects
that required several stages
Static Debugging: Problems 2
• comprehending static analyses across modules is difficult
• “real-world” features make analysis too imprecise
• imperative features demand more flow-sensitivity than SBA offers
• if-splitting is too weak
Static Debugging: Problems w/ Arity
• Scheme supports rest, default, list parameter specifications
• So: functions consume one argument
• applications package arguments as lists
• function bodies tease lists apart with selectors
Static Debugging: Problems w/ Arity
too few arguments
wrong kind of argument
Static Debugging: Problems with Arity
reports arity mismatch
… but computes data flow
… and thus pollutes rest of program with bad warnings
Static Debugging: The Lesson
• static debugging is worth pursuing
• we are not even close to a fully useful system
• we need – analyses tools for “real” languages– analyses that provide visual feedback– analyses for modular programs
Part IV: Optimizing Closure Allocation
Closures: Motivation
• Is information out of SBA good for back-end purposes? (back to static typing)
• Can we optimize heavily functional (closure intense) programs? – Steckler’s light-weight closure conversion
Closures: Goal
• modify SBA in support of light-weight closure conversion
• extend mzc compiler (mzscheme-c)
• apply to key modules in DrScheme– GUI front-end parser– mzc
Closures: An Example
(let* ([x (g 13)] [f (lambda (y) (+ x 20))]) (if (> (f 65) 0) (/ 1 (f 65)) 0))
free variable: x
calls to f are within lexical scope of x
(let* ([x (g 13)] [f (lambda (x y) (+ x 20))]) (if (> (f x 65) 0) (/ 1 (f x 65)) 0))
new call protocol for fno closure allocation
Closures: Avoid Allocation
• determine whether free variables are available at call site of closure
• transform all closures called there to accept additional arguments
• avoid closure allocation
• save > 50% on example [Wand & Steckler]
Closures: Analysis
• closure analysis -- which closures are called at a site
• invariance analysis -- which variables are available at call site with proper value
• protocol analysis -- which functions must share the extended protocol
Closures: Implementation 1
• extend to full Scheme – assignable variables– letrec
• separate analysis for units: prevent escape of procedures
• … based on Componential SBA
Closures: Implementation 2
• modified SBA consumed too much space and time (1 GB machine, 1 night) for benchmark programs
• re-implemented three specialized analyses
• extended mzc with analysis and transformation
Closures: Experiences 1
• benchmarked Gambit programs: travl, maze, mandelbrot, earley, …
• results are so-so: – closure conversion is hardly ever possible
even in closure-rich programs– closure conversion doesn’t save much time
-- in most cases < 5%– only rare programs benefit with > 10%
Closures: Experiences 2
• tested closure analysis/conversion on some key modules of the PLT Scheme suite
• none showed any improvement at all
Closures: The Lesson
• light-weight closure analysis and conversion works miracles on artificial example
• … does a bit of good in some of the standard benchmark programs
• … is a big disappointment for closure-intensive components
Part V: The Overall Lesson
General Guidelines
• is the analysis useful? – many dimensions
• is the analysis sound? – the core language needs a semantics [that
is, a machine-independent mathematical model]
– the predictions of the analysis about the set of values generated by an expression must hold at run-time
[note: ignored theory!]
Guidelines on Usefulness
• language: don’t do it for the core only
• size of programs: don’t do it for toy programs
• critical path: don’t do stand-alone analyses (even with optimizations)
• other constructs are interesting, too
• stress implementations with regularly used, “large” programs
• pick an “end-to-end” application of the analysis (a context)
On the “Critical Path”
“The User”
“The Program Run”
analysis
“Real Programs
On the “Critical Path”
“The User”
“The Program Run”
analysis
find the bottleneck of the entire set-up with respect to the static analysis:
• does the analysis deliver information that is presentable to the user? what kind of user?
• is the analysis precise on widely used frequently used constructs?
• does it pay off to produce this information? (code improvement)
“Real Programs
Challenge
• can we build an infrastructure for static analysis projects?– open programming environments – open compilers– benchmarks of all sizes– benchmarks of all kinds of programs– records of measurements– bottleneck problem statements
The Last Message:
• SA must do well “in context”
• set a concrete, reachable, ambitious goal
• … and work out all the problems
• others won’t do it for you
The End
Credits
• Cartwright• Cousot• Charter• Fagan• Findler• Flanagan• Flatt
• Krishnamurthi• Heintze• Lee • Sabry• Steckler• Wand• Wright