Parsing & Language Architecture€¦ · !e evolution of a DSL? general-purpose domain-speci"c...
Transcript of Parsing & Language Architecture€¦ · !e evolution of a DSL? general-purpose domain-speci"c...
![Page 1: Parsing & Language Architecture€¦ · !e evolution of a DSL? general-purpose domain-speci"c library API internal external general-purpose? discover nouns & verbs add #uency remove](https://reader034.fdocuments.net/reader034/viewer/2022042122/5e9d108f4f3a966b7856cbdc/html5/thumbnails/1.jpg)
Parsing & Language Architecture
![Page 2: Parsing & Language Architecture€¦ · !e evolution of a DSL? general-purpose domain-speci"c library API internal external general-purpose? discover nouns & verbs add #uency remove](https://reader034.fdocuments.net/reader034/viewer/2022042122/5e9d108f4f3a966b7856cbdc/html5/thumbnails/2.jpg)
The evolution of a DSL?
domain-specificgeneral-purpose
library API externalinternal
general-purpose?
discover nouns & verbs
add fluency
remove host flavor
tools & environments
features
We are here
![Page 3: Parsing & Language Architecture€¦ · !e evolution of a DSL? general-purpose domain-speci"c library API internal external general-purpose? discover nouns & verbs add #uency remove](https://reader034.fdocuments.net/reader034/viewer/2022042122/5e9d108f4f3a966b7856cbdc/html5/thumbnails/3.jpg)
Towards a language architecture
![Page 4: Parsing & Language Architecture€¦ · !e evolution of a DSL? general-purpose domain-speci"c library API internal external general-purpose? discover nouns & verbs add #uency remove](https://reader034.fdocuments.net/reader034/viewer/2022042122/5e9d108f4f3a966b7856cbdc/html5/thumbnails/4.jpg)
A notation for defining all the syntactically valid programs of a language. (Whitespace usually ignored.)
Grammars
expr ::=
expr + expr
| expr - expr
| expr * expr
| expr / expr
| n
start symbol
terminal
alternative
production rule
non-terminal
n ∈ 𝒵
![Page 5: Parsing & Language Architecture€¦ · !e evolution of a DSL? general-purpose domain-speci"c library API internal external general-purpose? discover nouns & verbs add #uency remove](https://reader034.fdocuments.net/reader034/viewer/2022042122/5e9d108f4f3a966b7856cbdc/html5/thumbnails/5.jpg)
A notation for defining all the syntactically valid programs of a language. (Whitespace usually ignored.)
Grammars (Is this a DSL?)
expr ::=
expr + expr
| expr - expr
| expr * expr
| expr / expr
| n
![Page 6: Parsing & Language Architecture€¦ · !e evolution of a DSL? general-purpose domain-speci"c library API internal external general-purpose? discover nouns & verbs add #uency remove](https://reader034.fdocuments.net/reader034/viewer/2022042122/5e9d108f4f3a966b7856cbdc/html5/thumbnails/6.jpg)
An internal DSL for recursive-descent parsers
Parser combinators
def expr: Parser[String] =
( expr ~ "+" ~ expr
| expr ~ "-" ~ expr
| expr ~ "*" ~ expr
| expr ~ "/" ~ expr
| wholeNumber )
import scala.util.parsing.combinator._
object Parser extends JavaTokenParsers {
}
libraryDependencies+="org.scala-lang.modules"%%"scala-parser-combinators"%"1.0.4"
build.sbt
Warning: left-recursion
![Page 7: Parsing & Language Architecture€¦ · !e evolution of a DSL? general-purpose domain-speci"c library API internal external general-purpose? discover nouns & verbs add #uency remove](https://reader034.fdocuments.net/reader034/viewer/2022042122/5e9d108f4f3a966b7856cbdc/html5/thumbnails/7.jpg)
Allows left-recursion. Recursive-descent parsing with backtracking.
Packrat parsing
lazy val expr: PackratParser[AST] =
( expr ~ "+" ~ expr
| expr ~ "-" ~ expr
| expr ~ "*" ~ expr
| expr ~ "/" ~ expr
| wholeNumber )
import scala.util.parsing.combinator._
object Parser extends JavaTokenParsers with PackratParsers {
}
libraryDependencies+="org.scala-lang.modules"%%"scala-parser-combinators"%"1.0.4"
build.sbt
Warning: associativity / precedence
![Page 8: Parsing & Language Architecture€¦ · !e evolution of a DSL? general-purpose domain-speci"c library API internal external general-purpose? discover nouns & verbs add #uency remove](https://reader034.fdocuments.net/reader034/viewer/2022042122/5e9d108f4f3a966b7856cbdc/html5/thumbnails/8.jpg)
Describes the intermediate representation, i.e., the abstract syntax tree. An inductive data structure.
Abstract syntax
expr ::=
expr + expr
| expr - expr
| expr * expr
| expr / expr
| n
n ∈ 𝒵 sealed abstract class Expr
case class Plus(left: Expr, right: Expr) extends Expr
case class Sub(left: Expr, right: Expr) extends Expr
case class Mult(left: Expr, right: Expr) extends Expr
case class Div(left: Expr, right: Expr) extends Expr
case class Num(n: Int) extends Expr
![Page 9: Parsing & Language Architecture€¦ · !e evolution of a DSL? general-purpose domain-speci"c library API internal external general-purpose? discover nouns & verbs add #uency remove](https://reader034.fdocuments.net/reader034/viewer/2022042122/5e9d108f4f3a966b7856cbdc/html5/thumbnails/9.jpg)
Actions: transform strings to IR
lazy val expr: PackratParser[String] =
( expr ~ "+" ~ expr
| expr ~ "-" ~ expr
| expr ~ "*" ~ expr
| expr ~ "/" ~ expr
| wholeNumber )
import scala.util.parsing.combinator._
object Parser extends JavaTokenParsers with PackratParsers {
}
libraryDependencies+="org.scala-lang.modules"%%"scala-parser-combinators"%"1.0.4"
build.sbt
Warning: associativity / precedence
![Page 10: Parsing & Language Architecture€¦ · !e evolution of a DSL? general-purpose domain-speci"c library API internal external general-purpose? discover nouns & verbs add #uency remove](https://reader034.fdocuments.net/reader034/viewer/2022042122/5e9d108f4f3a966b7856cbdc/html5/thumbnails/10.jpg)
Actions: transform strings to IR
lazy val expr: PackratParser[AST] =
( expr ~ "+" ~ expr ^^ {case l~"+"~r ⇒ Plus(l,r) }
| expr ~ "-" ~ expr ^^ {case l~"-"~r ⇒ Minus(l,r) }
| expr ~ "*" ~ expr ^^ {case l~”*"~r ⇒ Times(l,r) }
| expr ~ "/" ~ expr ^^ {case l~”/"~r ⇒ Divide(l,r)}
| wholeNumber ^^ {s ⇒ Num(s.toInt)} )
import scala.util.parsing.combinator._
object Parser extends JavaTokenParsers with PackratParsers {
}
libraryDependencies+="org.scala-lang.modules"%%"scala-parser-combinators"%"1.0.4"
build.sbt
Warning: associativity / precedence
![Page 11: Parsing & Language Architecture€¦ · !e evolution of a DSL? general-purpose domain-speci"c library API internal external general-purpose? discover nouns & verbs add #uency remove](https://reader034.fdocuments.net/reader034/viewer/2022042122/5e9d108f4f3a966b7856cbdc/html5/thumbnails/11.jpg)
The “lower-down” the operation, the higher its precedence.
A less ambiguous grammar
expr ::=
expr + term
| expr - term
| fact
term ::=
term * fact
| term / fact
| fact
fact ::=
n | ( expr )
n ∈ 𝒵
sealed abstract class Expr
case class Plus(left: Expr, right: Expr) extends Expr
case class Sub(left: Expr, right: Expr) extends Expr
case class Mult(left: Expr, right: Expr) extends Expr
case class Div(left: Expr, right: Expr) extends Expr
case class Num(n: Int) extends Expr
![Page 12: Parsing & Language Architecture€¦ · !e evolution of a DSL? general-purpose domain-speci"c library API internal external general-purpose? discover nouns & verbs add #uency remove](https://reader034.fdocuments.net/reader034/viewer/2022042122/5e9d108f4f3a966b7856cbdc/html5/thumbnails/12.jpg)
A Scala architecture for languages
Read-Eval-Print-Loop (REPL) libraryDependencies+="org.scala-lang"%"scala-compiler"%scalaVersion.value
parser combinators
caseclasses
functions &pattern matching
tests libraryDependencies+="org.scalacheck"%%"scalacheck"%"1.13.0"%"test"libraryDependencies+="org.scalatest"%%"scalatest"%"2.2.6"%"test"
![Page 13: Parsing & Language Architecture€¦ · !e evolution of a DSL? general-purpose domain-speci"c library API internal external general-purpose? discover nouns & verbs add #uency remove](https://reader034.fdocuments.net/reader034/viewer/2022042122/5e9d108f4f3a966b7856cbdc/html5/thumbnails/13.jpg)
With a grammar that fixes the associativity / precedence problems
Let’s practice!