Abstract Syntax

Different Levels of Syntax

• Lexical syntax– Basic symbols (names, values, operators, etc.)

• Concrete syntax– Rules for writing expressions, statements, programs– Input to compilers/interpreters

• Abstract syntax– “Internal” representation– Captures semantics

CS784(PM) 2

3

Overview

Parse-expression

Unparse-expression Interpreter

Concrete Syntax

Abstract Syntax

Results

CS784(PM)

Concrete vs. Abstract Syntax• Expressions with common meaning (should)

have the same abstract syntax.– C: a+b*c

• Assumes certain operator precedence (why?)– Forth: bc*a+ (reverse Polish)

This expression tree represents themeaning of expression

Not the same as parse tree (why?)Does the value depend on traversal order?abc*+ (or is this it?)

CS784(PM) 4

5

Parse tree vs. AST

+

expr

1 2 + 3

expr

expr

( ) ( )

expr

expr

1 2

+

3

+

exprexpr

CS784(PM)

6

Abstract Syntax Tree

• More useful representation of syntax tree– Less clutter– Actual level of detail depends on your design

• Basis for semantic analysis• Later annotated with various information

– Type information– Computed values

CS784(PM)

7

Compilation in a NutshellSource code(character stream)

Lexical analysis

Parsing

Token stream

Abstract syntax tree(AST)

Semantic Analysis

if (b == 0) a = b;

if ( b ) a = b ;0==

if==b 0

=a b

if==

int b int 0=

int alvalue

int bboolean

Decorated ASTint ;

;

CS784(PM)

8

λ-expressions

1. <exp> ::= 2. <identifier> 3. | (lambda (<identifier> ) <exp>) 4. | (<exp> <exp>)

• Compare with Scheme S-expressions.• EOPL3 p52: Lc-expCS784(PM)

Lc-exp ::=

• Identifier– var-exp (var)

• (lambda (Identifier) Lc-exp)– lambda-exp (bound-var body)

• (Lc-exp Lc-exp)– app-exp (rator rand)

• Abstract syntax

CS784(PM) 9

EOPL3 Scheme: define-datatype

• Syntax definition:(define-datatype type-name type-predicate-name

{(variant-name {(field-name predicate)}* )} + )• An Example:

(define-datatype environment environment? (empty-env-record) (extended-env-record (syms (list-of symbol?)) (vals (list-of scheme-value?)) (env environment?)))

• Data types built by define-datatype may be mutually recursive.

CS784(PM) 10

Syntax Driven Representation

(define-datatype expression expression? (var-exp (id symbol?)) (lambda-exp (id symbol?) (body expression?)) (app-exp (rator expression?) (rand expression?)))

11CS784(PM)

Figure 2.2: (lambda (x) (f (f x)))

CS784(PM) 12

Concrete to Abstract Syntax(define parse-expression (lambda (datum) (cond ((symbol? datum) (var-exp datum)) ((pair? datum) (if (eqv? (car datum) 'lambda)

(lambda-exp (caadr datum) (parse-expression (caddr datum))) (app-exp (parse-expression (car datum))

(parse-expression (cadr datum)))) ) (else (eopl:error

'parse-expression "Invalid concrete syntax ~s" datum)) )))

CS784(PM) 13

14

DrRacket

CS784(PM)

Unparse: Abstract to Concrete Syntax

(define unparse-expression (lambda (exp) (cases expression exp (var-exp (id) id) (lambda-exp (id body) (list 'lambda (list id) (unparse-expression body)) ) (app-exp (rator rand) (list (unparse-expression rator) (unparse-expression rand)) ))))

CS784(PM) 15

Role of Induction and Recursion

• Define data structures (infinite values) by induction.– Seed elements.– Closure operations.

• Define functions (operations) by recursion.– Boundary/Basis case.– Composite/Recursive case.

• Prove properties using structural induction.– Basis case.– Inductive step.

CS784(PM) 16

17

The Environment

• Environment:– table of variable-value pairs– Chronologically later var-value pair overrides

• Constructors– empty-env – extend-env

• Observer– apply-env

CS784(PM)

The Environment Spec

• (empty-env) = ∅• (apply-env f var) = f (var)• (extend-env var v f] ) = g,

– where g(var1 )= v, if var1 = var= f (var1). otherwise

• from EOPL3 p36

CS784(PM) 18

An Example Env e

• (define e(extend-env ’d 6(extend-env ’y 8

(extend-env ’x 7(extend-env ’y 14

(empty-env))))))

• e(d)=6, e(x)=7, e(y)=8CS784(PM) 19

20

Representing Environment

• Many representations are possible• Speedy access• Memory frugal• Change in the interface: Syms x Vals

(define extend-env (lambda (syms vals env) … ))

CS784(PM)

Alt-1: env is a List of Ribs

CS784(PM) 21

• left rib: list of variables• right rib: corresponding list of values.• Exercise 2.11 EOPL3 (Ribcage)

Alt-1: List of Ribs (Ribcage)(define empty-env (lambda () '()))

(define extend-env (lambda (syms vals env) (cons (list syms vals) env) ))

(define apply-env (lambda (env sym) (if (null? env) (eopl:error 'apply-env "No binding for ~s" sym) (let ((syms (car (car env))) (vals (cadr (car env))) (env (cdr env))) (let ((pos (rib-find-position sym syms))) (if (number? pos) (list-ref vals pos) (apply-env env sym)))))))

CS784(PM) 22

Alt-2: env is a Unary Function

(define empty-env (lambda () (lambda (sym) (eopl:error 'apply-env "No binding for ~s" sym)) ))

(define extend-env (lambda (syms vals env) (lambda (sym) (let ((pos (list-find-position sym syms))) (if (number? pos) (list-ref vals pos) (apply-env env sym)))) ))

(define apply-env (lambda (env sym) (env sym) ))

CS784(PM) 23

Alt-3: Tagged Records

(define-datatype environment environment? (empty-env-record) (extended-env-record (syms (list-of symbol?)) (vals (list-of scheme-value?)) (env environment?)))

(define scheme-value? (lambda (v) #t))(define empty-env (lambda () (empty-env-record) ))(define extend-env (lambda (syms vals env) (extended-env-record syms vals env)))

CS784(PM) 24

Alt-3: Tagged records

(define apply-env (lambda (env sym) (cases environment env (empty-env-record () (eopl:error 'apply-env "No binding for ~s" sym)) (extended-env-record (syms vals env) (let ((pos (list-find-position sym syms))) (if (number? pos) (list-ref vals pos) (apply-env env sym)))))))

CS784(PM) 25

26

Queue

(define reset (lambda (q) (vector-ref q 0)))(define empty? (lambda (q) (vector-ref q 1)))(define enqueue (lambda (q) (vector-ref q 2)))(define dequeue (lambda (q) (vector-ref q 3)))

(define Q (create-queue))

((enqueue Q) 55)

((empty? Q))

((dequeue Q))

((empty? Q))

((reset Q))

((dequeue Q))

CS784(PM)

27

1. (define create-queue2. (lambda ()3. (let ((q-in '()) (q-out '()))4. (letrec 5. ((reset-queue6. (lambda ()7. (set! q-in '()) (set! q-out '())) )8. (empty-queue?9. (lambda ()10. (and (null? q-in) (null? q-out))) )11. (enqueue12. (lambda (x)13. (set! q-in (cons x q-in))) )14. (dequeue15. (lambda ()16. (if (empty-queue?)17. (eopl:error 'dequeue "Not on an empty queue")18. (begin19. (if (null? q-out)20. (begin21. (set! q-out (reverse q-in)) (set! q-in '())))22. (let ((ans (car q-out)))23. (set! q-out (cdr q-out))24. ans))))) )25. (vector reset-queue empty-queue? enqueue dequeue))26.)))

CS784(PM)