F# Quotations:
Code as Data

Jack PappasDmitry Morozov

SkillsMatter Progressive F# Tutorials NYCSeptember 19th, 2013

Code as Data

Code & Data What is the difference?

Is there a difference?

Programs exist to process data

The F# compiler processes data (your source code)

Source code is no different than other kinds of data

This Session

What is a quoted expression?

A simple, immutable data structure

Represents a fragment of F# code

Syllabus

Background

Working with Quotations

Manipulating Quotations

Applications of Quotations

Background

The idea of treating code as data first appeared in Lisp

LispF#

Code(1 2 3)[1; 2; 3]

Code as Data(1 2 3)

Background

A language that has the natural ability to manipulate itself is extremely powerful

Amplify a developers productivity using metaprogramming techniques

Background

Other languages can manipulate their own code

clang: C++ program, manipulates C++

Roslyn: C# library, manipulates C#

f2c: C program, manipulates FORTRAN and C

What makes Lisp and F# different?

Reflective meta-programming

Built-in language feature vs. external functionality

Background

Now that we can manipulate code as data, what can we do with it?

Transform

modify, convert, translate

Analyze

Static analysis

Advanced testing techniques (e.g., verification)

Collect and record code metrics (e.g., complexity)

Background

Execute

Interpret the code/data

Translate to another lang., compile, execute

Specialized hardware (e.g., GPU, FPGA)

Specialized execution (e.g., auto-parallelization)

Execute remotely (e.g., web service)

Some combination of the above

(Rarely) execute in same context, e.g., by translating to IL and executing

Use Cases

LINQ support in F#

Extension of this idea:
Cheney, Lindley, Wadler.
The essence of language-integrated query

Use as an IL when developing type providers

Quotations much easier to work with

Create quotations, translate to IL

Use Cases

Testing

Moq/Foq/Unquote

GPU programming

Compute: FSCL, QuantAlea

Graphics: SharpShaders (F# -> HLSL)

Web Development (F# -> JavaScript)

WebSharper

FunScript

PitFw

Comparing Alternatives

What alternatives exist to quotations?

F# AST

LINQ Expression Trees

CIL (MSIL)

Imperative: Expression Trees, CIL

C# lambdas compiled into Expression Trees by the C# compiler

Functional: F# AST, Quotations

Comparing Alternatives

F# AST

Designed for the compilers internal use

Carries more information than quotations

Requires additional library

Unwieldy for simple use cases

Comparing Alternatives

LINQ Expression Trees

Introduced in .NET 3.5; initially somewhat limited

Expanded in .NET 4.0; now roughly 1:1 with CIL

Doesnt support certain functional features like closures

Comparing Alternatives

Common Intermediate Language (CIL)

Low-level; relatively difficult to work with

Supports all CLR features

F# Quotations dont support all CLR features

Notably, throw and rethrow these need to be wrapped in a function and compiled using the proto compiler

No fault/filter clauses

F# Quotations dont support open generics

Quotations: How-To

Open some namespaces

Define a module, then some functions within the module

open System.Reflectionopen Microsoft.FSharp.Reflectionopen Microsoft.FSharp.Quotations

module Foo = let addTwo x = x + 2

Quotations: How-To

Apply [] to your function

Or, apply it to your module as a shortcut for applying it to all functions in the module

module Foo = /// Dummy class used to retrieve /// the module type. type internal Dummy = class end

[] let addTwo x = x + 2

Quotations: How-To

Using Reflection, get the MethodInfo for your function

Call Expr.TryGetReflectedDefinition with the MethodInfo to get the Expr for the method

Quotations: How-To

[]let main argv = let fooType = typeof.DeclaringType let addTwoMethodInfo = match fooType.GetMethod "addTwo" with | null -> None | x -> Some x

addTwoMethodInfo |> Option.map Expr.TryGetReflectedDefinition |> Option.iter (printfn "addTwo: %A")

0 // return an integer exit code

Quotations: How-To

For each active pattern in Quotations.Patterns, there is a corresponding static member of the Expr type.

These are used to construct and destructure each quotation case.

Two types of quotations: typed and untyped

Typed: Expr

Untyped: Expr

Quotations: How-To

Expr is just a wrapper around Expr

Use the .Raw property for Expr -> Expr

Use the Coerce pattern for Expr -> Expr

Quoting an expression

Typed: x + 2 @>

Untyped: x + 2 @@>

Quotations: How-To

Splicing a way to combine quotations

Inserts one quotation into another

Typed:

let bar = let foo = x + %bar @>Untyped:

let bar = let foo = x + %%bar @@>

Quotations: How-To

Note: You dont always need [] to use quotations

Using our previous example, this is equivalent

let addTwo = x + 2 @>

Computations on Trees

When working with Quotations, its critical youre comfortable with trees and recursion

Tree computations in imperative languages usually built around the visitor pattern

Visitor class usually holds private, mutable state

Often implemented as an abstract base class per tree type, and one virtual method per node type

Mechanics of the traversal intermixed with computation happening at nodes

Computations on Trees

Possible to implement a functional alternative to the visitor pattern?

Knuth did decades ago!

Introducing attribute grammars

Initially used as a means of describing how LL/LR parsers annotate a parse tree

Later, used to describe how attributes are computed from on tree

Computations on Trees

Attribute grammars are declarative

They do not define any traversal order

Attribute grammars allow you to define purely functional, monadic computations over trees

For those familiar with Haskell attribute evaluation is actually co-monadic

AGs on ASTs (or Quotations) can be used to implement purely functional AOP

Computations on Trees

Three main kinds of attributes

Inherited (L-attributes)

Attribute inherited from parent node

Compare to the reader workflow

Synthesized (S-attributes)

Attribute synthesized by a child node, passed back up to parent node

Compare to the writer workflow

Computations on Trees

Threaded (SL-attributes)

Attribute value inherited from parent node, modified by child, then passed back up to parent

Combination of Inherited and Synthesized

Compare to the state workflow

Attribute values can be int, string, UDTs, etc.

They can also be tree nodes

We can use this to define tree transformations

Exercises

Implement a function(s) which, given an Expr:

Print the equiv. F# source code

Invert the mathematical operators (+)/(-), (*)/(/)

Compute the set of methods called by the Expr

Bonus: Compute this transitively

Determine if the Expr can ever raise an exn

Determine if the Expr is pure (side-effect free)

Rewrite it so all strings are passed through String.Intern

Click to edit the title text formatClick to edit Master title style

03/10/13

Click to edit the title text formatClick to edit Master title style

Click to edit the outline text formatSecond Outline LevelThird Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline Level

Seventh Outline LevelClick to edit Master text styles

Second level

Third level

Fourth level

Fifth level

03/10/13