Composing Source-to-Source Data-Flow Transformations with Rewriting Strategies and Dependent Dynamic...

Composing Source-to-Source Data-FlowTransformations with Rewriting Strategies and

Dependent Dynamic Rewrite Rules

Karina Olmos & Eelco Visser

Institute of Information & Computing SciencesUtrecht UniversityThe Netherlands

April 5, 2005CC’05 Edinburgh

Source-to-Source

Data-Flow

Transformations

Goal: transformation tools for the working programmer

Transformations on various programming languages

I General-purpose languages

I (Embedded) domain-specific languages

Combine different types of transformations

I Program generation and meta-programming

I Simplification

I (Domain-specific) optimization

I Data-flow transformations

Source-to-source

I Transformations on abstract syntax trees

Concise and reusable

http://www.strategoxt.org Dependent Dynamic Rewrite Rules

Source-to-Source Data-Flow Transformations

Goal: transformation tools for the working programmer

Transformations on various programming languages

I General-purpose languages

I (Embedded) domain-specific languages

Combine different types of transformations

I Program generation and meta-programming

I Simplification

I (Domain-specific) optimization

I Data-flow transformations

Source-to-source

I Transformations on abstract syntax trees

Concise and reusable

Rewriting Strategies and Dynamic Rewrite Rules

Stratego/XT: language + tools for program transformation

I XT: infrastructure for transformation systems

I Stratego: high-level language for program transformation

I Not tied to one type of transformation or language

Stratego paradigm

I Rewrite rules for basic transformation steps

I Programmable rewriting strategies for controlling rules

I Dynamic rules for context-sensitive transformation

I Concrete syntax for patterns

Contributions

I Dependent dynamic rules

I Generic data-flow strategies

I Combination of transformations

Stratego paradigm

Contributions

Stratego paradigm

Contributions

Rewrite Rules and Strategies

Constant folding

y := x * (3 + 4) ⇒ y := x * 7

Constant folding rules

EvalAdd : |[ i + j ]| -> |[ k ]| where <add>(i, j) => k

EvalMul : |[ i * j ]| -> |[ k ]| where <mul>(i, j) => k

AddZero : |[ 0 + e ]| -> |[ e ]|

Constant folding strategy (bottom-up)

EvalBinOp = EvalAdd <+ AddZero <+ EvalMul <+ EvalOther

try(s) = s <+ id

constfold = all(constfold); try(EvalBinOp)

Constant folding

y := x * (3 + 4) ⇒ y := x * 7

AddZero : |[ 0 + e ]| -> |[ e ]|

try(s) = s <+ id

Constant folding

y := x * (3 + 4)

AddZero : |[ 0 + e ]| -> |[ e ]|

try(s) = s <+ id

Constant folding

y := x * (3 + 4)

AddZero : |[ 0 + e ]| -> |[ e ]|

try(s) = s <+ id

Constant folding

y := x * (3 + 4)

AddZero : |[ 0 + e ]| -> |[ e ]|

try(s) = s <+ id

Constant folding

y := x * (3 + 4)

AddZero : |[ 0 + e ]| -> |[ e ]|

try(s) = s <+ id

Constant folding

y := x * (3 + 4)

AddZero : |[ 0 + e ]| -> |[ e ]|

try(s) = s <+ id

Constant folding

y := x * (3 + 4)

AddZero : |[ 0 + e ]| -> |[ e ]|

try(s) = s <+ id

Constant folding

y := x * 7

AddZero : |[ 0 + e ]| -> |[ e ]|

try(s) = s <+ id

Constant folding

y := x * 7

AddZero : |[ 0 + e ]| -> |[ e ]|

try(s) = s <+ id

Constant folding

y := x * 7

AddZero : |[ 0 + e ]| -> |[ e ]|

try(s) = s <+ id

Context-Sensitive Transformations

Problem: Rewrite Rules are Context-free

Rewrite rules can only access information in term that is matched

Many Transformations are Context-Sensitive

I Constant propagation

I Copy propagation

I Common-subexpression elimination

I Partial evaluation

I Function inlining

I Dead code elimination

Solution: Dynamic Rewrite Rules

Define rewrite rules during transformation

I Copy propagation

I Function inlining

I Copy propagation

I Function inlining

Defining and Undefining Rules Dynamically

Constant Propagation and Folding in Straight-Line Code

b := 1;c := b + 3;b := foo();a := b + c

b -> 1b -> 1 & c -> 4b - & c -> 4b - & c -> 4 & a -

prop-const =PropConst <+ prop-const-assign<+ (all(prop-const); try(EvalBinOp))

prop-const-assign =|[ x := <prop-const => e> ]|; if <is-value> e then

rules( PropConst : |[ x ]| -> |[ e ]| )elserules( PropConst :- |[ x ]| )

b := 1;c := b + 3;b := foo();a := b + c

b -> 1b -> 1 & c -> 4b - & c -> 4b - & c -> 4 & a -

b := 1;c := b + 3;b := foo();a := b + c

b -> 1b -> 1 & c -> 4b - & c -> 4b - & c -> 4 & a -

b := 1;c := b + 3;b := foo();a := b + c

b -> 1

b -> 1 & c -> 4b - & c -> 4b - & c -> 4 & a -

b := 1;c := b + 3;b := foo();a := b + c

b -> 1

b -> 1 & c -> 4b - & c -> 4b - & c -> 4 & a -

b := 1;c := b + 3;b := foo();a := b + c

b -> 1

b -> 1 & c -> 4b - & c -> 4b - & c -> 4 & a -

b := 1;c := 1 + 3;b := foo();a := b + c

b -> 1

b -> 1 & c -> 4b - & c -> 4b - & c -> 4 & a -

b := 1;c := 1 + 3;b := foo();a := b + c

b -> 1

b -> 1 & c -> 4b - & c -> 4b - & c -> 4 & a -

b := 1;c := 4;b := foo();a := b + c

b -> 1

b -> 1 & c -> 4b - & c -> 4b - & c -> 4 & a -

b := 1;c := 4;b := foo();a := b + c

b -> 1

b -> 1 & c -> 4b - & c -> 4b - & c -> 4 & a -

b := 1;c := 4;b := foo();a := b + c

b -> 1

b -> 1 & c -> 4

b - & c -> 4b - & c -> 4 & a -

b := 1;c := 4;b := foo();a := b + c

b -> 1

b -> 1 & c -> 4

b - & c -> 4b - & c -> 4 & a -

b := 1;c := 4;b := foo();a := b + c

b -> 1b -> 1 & c -> 4

b - & c -> 4

b - & c -> 4 & a -

b := 1;c := 4;b := foo();a := b + c

b -> 1b -> 1 & c -> 4

b - & c -> 4

b - & c -> 4 & a -

b := 1;c := 4;b := foo();a := b + 4

b -> 1b -> 1 & c -> 4

b - & c -> 4

b - & c -> 4 & a -

b := 1;c := 4;b := foo();a := b + 4

b -> 1b -> 1 & c -> 4b - & c -> 4

b - & c -> 4 & a -

Properties of Dynamic Rules

I Rules are defined dynamically

I Carry context information

I Multiple rules with same name can be defined

I Rules can be undefined

I Rules with same left-hand side override old rules

b := 3;...b := 4;

b -> 3b -> 3b -> 4

Properties of Dynamic Rules

I Rules are defined dynamically

I Carry context information

I Multiple rules with same name can be defined

I Rules can be undefined

I Rules with same left-hand side override old rules

b := 3;...b := 4;

b -> 3b -> 3b -> 4

Flow-Sensitive Transformations

Flow-Sensitive Constant Propagation

(x := 3;y := x + 1;if foo(x) then(y := 2 * x;x := y - 2)

else(x := y;y := 23);

z := x + y)

(x := 3;y := 4;if foo(3) then(y := 6;x := 4)

else(x := 4;y := 23);

z := 4 + y)

fork rule sets and combine at merge point

x := 3

y := x + 1

y := 4

x -> 3

if foo(x)

if foo(3)

x -> 3 y -> 4

y := 2 * x

y := 6

x -> 3 y -> 4

x := y

x := 4

x -> 3 y -> 4

x := y - 2

x := 4

x -> 3 y -> 6

x -> 4 y -> 6

z := x + y

z := 4 + y

x -> 4 y -

y := 23

x -> 4 y -> 4

x -> 4 y -> 23

Flow-Sensitive Transformations

Flow-Sensitive Constant Propagation

(x := 3;y := x + 1;if foo(x) then(y := 2 * x;x := y - 2)

else(x := y;y := 23);

z := x + y)

(x := 3;y := 4;if foo(3) then(y := 6;x := 4)

else(x := 4;y := 23);

z := 4 + y)

fork rule sets and combine at merge point

x := 3

y := x + 1

y := 4

x -> 3

if foo(x)

if foo(3)

x -> 3 y -> 4

y := 2 * x

y := 6

x -> 3 y -> 4

x := y

x := 4

x -> 3 y -> 4

x := y - 2

x := 4

x -> 3 y -> 6

x -> 4 y -> 6

z := x + y

z := 4 + y

x -> 4 y -

y := 23

x -> 4 y -> 4

x -> 4 y -> 23

Constant propagation in abstract syntax tree

x := 3

y := x + 1

y := 4

if foo(x)

if foo(3)

x -> 3 y -> 4

y := 2 * x

y := 6

x := y - 2

x := 4

x := y

x := 4

y := 23

z := x + y

z := 4 + y

x -> 3

x -> 3y -> 4

x -> 3 y -> 4

x -> 4 y -

x -> 3y -> 4

x -> 3 y -> 6

x -> 3 y -> 4

x -> 4 y -> 4

Forking and Intersecting Dynamic Rulesets

Flow-sensitive Constant Propagation

prop-const-if =|[ if <prop-const> then <id> else <id> ]|; (|[if <id> then <prop-const> else <id>]|

/PropConst\ |[if <id> then <id> else <prop-const>]|)

s1 /R\ s2: fork and intersect

Propagation through Loops

(a := 1;i := 0;while i < m do (j := a;a := f();a := j;i := i + 1

);print(a, i, j))

(a := 1;i := 0;while i < m do (j := 1;a := f();a := 1;i := i + 1

);print(1, i, j))

Fixpoint Iteration

prop-const-while =?|[ while e1 do e2 ]|; (/PropConst\* |[while <prop-const> do <prop-const>]|)

/R\* s ≡ ((id /R\ s) /R\ s) /R\ ...)until fixedpoint of ruleset is reached

prop-const-while terminates:fewer rules defined each iteration

Fixpoint Iteration

prop-const-while =?|[ while e1 do e2 ]|; (/PropConst\* |[while <prop-const> do <prop-const>]|)

/R\* s ≡ ((id /R\ s) /R\ s) /R\ ...)until fixedpoint of ruleset is reached

prop-const-while terminates:fewer rules defined each iteration

Combining Analysis and Transformation

Unreachable code elimination

i := 1;j := 2;if j = 2then i := 3;else z := foo()

print(i)

⇒i := 1;j := 2;i := 3;print(3)

EvalIf : |[ if 0 then e1 else e2 ]| -> |[ e2 ]|EvalIf : |[ if i then e1 else e2 ]| -> |[ e1 ]|

where <not(eq)>(i,|[0]|)

prop-const-if =|[ if <prop-const> then <id> else <id> ]|;(EvalIf; prop-const<+ (|[if <id> then <prop-const> else <id>]| /PropConst\

|[if <id> then <id> else <prop-const>]|))

print(i)

⇒i := 1;j := 2;i := 3;print(3)

print(i)

⇒i := 1;j := 2;i := 3;print(3)

(x := 10;while A doif x = 10

then dosomething()else (dosomethingelse();

x := x + 1);y := x)

⇒(x := 10;while A dodosomething();

y := 10)

Conditional Constant Propagation [Wegman & Zadeck 1991]Graph analysis + transformation in Vortex [Lerner et al. 2002]

Dynamic Rule Scope

let var x := 17in let var y := x + 1

in let var x := y+1in () end

end; print(x)end

let var x := 17in let var y := 18

in let var x := 19in () end

end; print(17)end

Transformation in presence of local variables

I Dynamic rule scope restricts lifetime of dynamic rule

I See paper

Other Issues

Interprocedural transformation

[Olmos & Visser 2003]

I Type specialization for Octave

[Bravenboer, Van Dam, Olmos & Visser 2005]

I Poly-variant online specialization and unfolding

[Olmos 2005 forthcoming]

I Global variables

I Mono-variant specialization (summaries)

Aliasing

[Olmos 2005 forthcoming]

I Propagation with records and arrays

Putting it all together

Conditional Constant Propagation

prop-const =

PropConst <+ prop-const-assign <+ prop-const-declare

<+ prop-const-let <+ prop-const-if <+ prop-const-while

<+ (all(prop-const); try(EvalBinOp))

prop-const-assign =

|[ x := <prop-const => e> ]|

; if <is-value> e then rules( PropConst.x : |[ x ]| -> |[ e ]| )

else rules( PropConst.x :- |[ x ]| ) end

prop-const-declare =

|[ var x := <prop-const => e> ]|

; if <is-value> e then rules( PropConst+x : |[ x ]| -> |[ e ]| )

else rules( PropConst+x :- |[ x ]| ) end

prop-const-let =

?|[ let d* in e* end ]|; {| PropConst : all(prop-const) |}

prop-const-if =

|[ if <prop-const> then <id> else <id> ]|

; (EvalIf; prop-const

<+ (|[ if <id> then <prop-const> else <id> ]|

/PropConst\ |[ if <id> then <id> else <prop-const> ]|))

prop-const-while =

?|[ while e1 do e2 ]|

; (|[ while <prop-const> do <id> ]|; EvalWhile

<+ (/PropConst\* |[ while <prop-const> do <prop-const> ]|))

Recapitulation

I Rewrite rules for constant folding

I Strategies for (generic) traversal

I Dynamic rule propagates values

I Fork and intersection (union) for flow-sensitive transformation

I Dynamic rule scopes controls lifetime of rules

can this be applied to other data-flow transformations?

Recapitulation

I Rewrite rules for constant folding

I Strategies for (generic) traversal

I Dynamic rule propagates values

I Dynamic rule scopes controls lifetime of rules

can this be applied to other data-flow transformations?

Common-Subexpression Elimination

x := a + b;y := a * b;while y > a + b do (

a := a + 1;x := a + b

x := a + b;y := a * b;while y > x do (

a := a + 1;x := a + b

CSE with dynamic rule

rules( CSE : |[ e ]| -> |[ x ]| )end

This works

, kind of

a := a + 1;x := a + b

rules( CSE : |[ e ]| -> |[ x ]| )end

This works

, kind of

a := a + 1;x := a + b

rules( CSE : |[ e ]| -> |[ x ]| )end

This works

, kind of

a := a + 1;x := a + b

rules( CSE : |[ e ]| -> |[ x ]| )end

This works, kind of

Problem: Insufficient Dependency Information

x := a + b;a := foo();y := a + b

a + b -> x x := a + b;a := foo();y := x // wrong!

Analysis

Rule should be undefined when any variable changes value

Solution: Dependent Dynamic Rules

Record all dependencies of dynamic rules

rules( R : p1 -> p2 depends on [x1,...,xn] )

Undefine all rules depending on dep

undefine-R(|dep)

x := a + b;a := foo();y := a + b

Analysis

undefine-R(|dep)

x := a + b;a := foo();y := a + b

Analysis

undefine-R(|dep)

CSE with Dependent Dynamic Rules

cse-if =|[ if <cse> then <id> else <id> ]|; ( |[ if <id> then <cse> else <id> ]|

/CSE\ |[ if <id> then <id> else <cse> ]|)

cse-while =|[ while <id> do <id> ]|; (/CSE\* |[ while <cse> do <cse> ]|)

Hygienic Program Transformation

Respect variable bindings

Dependent rules avoid

I free variable capture

I escaping variables

See paper

Recapitulation

I Rewrite rules for basic transformations

I Strategies for control and (generic) traversal

I Dynamic rule propagates context information

I Dynamic rule scopes and dependent rules for control overlifetime of rules

Examples

I Constant propagation |[ x ]| -> |[ i ]|

I Copy propagation |[ x ]| -> |[ y ]|

I Common-subexpression elimination |[ e ]| -> |[ x ]|

I Forward substitution |[ x ]| -> |[ e ]|

I Partial redundancy elimination (down-safe, earliest)

Generic Data-Flow Strategies

I Observation: most of data-flow strategy is boilerplate

I Solution: generic data-flow strategy

I Generalized operators

I Intersection and union: /Rs1\Rs2/ and /Rs1\Rs2/*I Undefinition of multiple dynamic rules

Instantiation for common-subexpression elimination

cse = forward-prop(fail, id, cse-after | ["CSE"], [], [])

cse-assign =?|[ x := e ]|; where( <pure-and-not-trivial(|x)> |[ e ]| ); where( get-var-dependencies => xs ); rules( CSE : |[ e ]| -> |[ x ]| depends on xs )

cse-after = try(cse-assign <+ CSE)

Combining Transformations

super-opt =forward-prop(prop-const-transform, bvr-before, bvr-after; copy-prop-after; prop-const-after; cse-after

| ["PropConst", "CopyProp", "CSE"], [], ["RenameVar"]

Apply multiple data-flow transformations simultaneously

Experience with Dynamic Rules

I Tiger compiler: sandbox for transformation techniquesbound variable renaming, inlining, constant propagation, copy

propagation, common-subexpression elimination, dead assignment

elimination, partial redundancy elimination, online and offline partial

evaluation, loop normalization, loop vectorization, ...

I Octave compilertype specialization, partial evaluation, other data-flow

transformations, combined transformations, loop vectorization

I Stratego compilerinlining, specialization, bound-unbound variables analysis, ...

I LVM optimizer (functional)substitutions, inlining, (deforestation, warm fusion)

I Java Compilername disambiguation, type propagation, assimilation of embedded

domain-specific languages

Related Work

[Wegman & Zadeck 1991]

I SCC: special algorithm for conditional constant propagation

I propagation through SSA edges

[Lerner et al. 2002]

I integration of analysis and transformation for CFGs

I combination of multiple analyses/transformations

[Lacey & de Moor 2001]

I temporal logic : find context from occurrence

[Sittampalam, de Moor & Larsen 2004]

I regular path queries

I incremental analysis after applying transformation

Conclusion

I Abstract interpretation style of data-flow transformation

I combination of data-flow analysis and transformation

I Hygienic: correct treatment of variable binding constructs

I avoid free variable capture and escaping variablesI scoped transformation rules

I concise specification specification of data-flow transformationI combination of multiple transformations

I Combination of data-flow transformations with othertransformations

I reuse of (elements of) transformationsI alternative transformation strategies

Stratego/XT 0.14 (pre-release) from www.stratego-language.org

The End

Correctness of Transformations

I Invariant: rule set valid at current program point

I Check: each rule maintains invariant

Break and Continue

I Only structured control-flow supported

I Exit from loop: break-R

Composing Source-to-Source Data-Flow Transformations with Rewriting Strategies and Dependent Dynamic...

Technology

Transcript of Composing Source-to-Source Data-Flow Transformations with Rewriting Strategies and Dependent Dynamic...