Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs
description
Transcript of Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs
![Page 1: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/1.jpg)
1
Lecture 12
Implementing Small Languagesinternal vs. external DSLs, hybrid small DSLs
Ras Bodik Ali and Mangpo
Hack Your Language!CS164: Introduction to Programming Languages and Compilers, Spring 2013UC Berkeley
![Page 2: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/2.jpg)
OverviewImplementation of DSLs
External vs. internal DSLsinternal == embedded in the host language
Deep embedding
Shallow embedding
Examples2
![Page 3: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/3.jpg)
Styles of DSL implementationsExternal DSL
- own syntax (and parser)- dedicated interpreter
Internal DSL: deep embedding in the host language
- deep embedding: program exists as data (eg AST)
- the host language implements an interpreterInternal DSL: shallow embedding in the host language
- shallow embedding: DSL constructs are composed purely of host language constructs
- we are using the interpreter of the host language
note: instead of interpreter, we can use a compiler
3
![Page 4: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/4.jpg)
External DSLs
4
![Page 5: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/5.jpg)
Architecture of external-DSL implementationsExternal DSL is a standalone language
- just like “big” languages, Java, Python, C, …- domain programming abstractions with own
syntax
Similar interpreter/compiler architecture parser, optional analysis of AST, interpretation/compilation
If compiled, DSL need not be translated into code
- instead, it can be translated to a data structure which will be used by some other program P
- yes, P can be viewed as an interpreter of the DSL, and the data structure is an “AST”
5
![Page 6: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/6.jpg)
ExamplesInterpreted external DSLs
External DSLs compiled to code
External DSLs compiled to a data structure
6
![Page 7: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/7.jpg)
Internal DSLsshallow embedding
7
![Page 8: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/8.jpg)
Embed your DSL into a host languageShallow embedding: the new language is a library written in the host language
sometimes called a “framework”
When DSL is implemented as a library, we often don’t think of it as a language
even though it defines own abstractions and operations
But the library implementation goes very far
8
![Page 9: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/9.jpg)
Example 1: socketsSockets are library with own abstractions and rules
Socket f = new Socket(mode)f.connect(ipAddress)f.write(buffer)f.close()
Programming abstractions:socket, address, buffer
Usage rules:- do not write into a closed socket- do not close a socket twice
9
![Page 10: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/10.jpg)
Usage rules (side note)How abstractions can be combined is part of the language.
With network sockets, these are enforced at runtime:
ex: sock.write() checks that socket has been open
This is analogous to dynamic (runtime) type checking
ex: in Python, 1 + “a” will fail at runtime, when types of 1 and “a” are checked in the + operation
10
![Page 11: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/11.jpg)
Usage rules (side note)Could we enforce these socket rules at compile time?
This would be analogous to static type checking.
ex: in Java, int i; String s; i+s; fails at compile time even knowing what specific values i and s will take at runtime.
11
![Page 12: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/12.jpg)
Usage rules (side note)
Could we enforce these socket rules at compile time?
Yes, use static types (as in say Java) and refine Socket into Socket, OpenSocket, ClosedSocket
How do we rewrite this code to use refined types?
Socket f = new Socket(mode)f.connect(ipAddress)f.write(buffer)f.close()
12
![Page 13: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/13.jpg)
Example 2: regex matchingThe library defines:
i) functions that DSL programmers use to define a pattern
ex: pattern ("abc"|"de")."x" can be defined as follows:
def patt = seq(alt(prim("abc"),prim("de")),prim("x"))
ii) a function matches of a string against a pattern:
match(string, patt)13
![Page 14: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/14.jpg)
Example 3:rfig: formatting DSL embedded into Ruby.
see slide 8 in http://cs164fa09.pbworks.com/f/01-rfig-tutorial.pdf
14…
![Page 15: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/15.jpg)
The animation in rfig, a Ruby-based language
slide!('Overlays',
'Using overlays, we can place things on top of each other.', 'The pivot specifies the relative positions', 'that should be used to align the objects in the overlay.',
overlay('0 = 1', hedge.color(red).thickness(2)).pivot(0, 0),
staggeredOverlay(true, # True means that old objects disappear 'the elements', 'in this', 'overlay should be centered',
nil).pivot(0, 0),
cr, pause, # pivot(x, y): -1 = left, 0 = center, +1 = right
staggeredOverlay(true, 'whereas the ones', 'here', 'should be right justified',
nil).pivot(1, 0), nil) { |slide| slide.label('overlay').signature(8) }
15
![Page 16: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/16.jpg)
DSL as a frameworkIt may be impossible to hide plumbing in a procedure
these are limits to procedural abstraction
We can use advanced language features, such as closures, coroutines
Framework, a library parameterized with client code• typically, you register a function with the library• library calls this client callback function at a
suitable point• ex: an action to perform when a user clicks on
DOM node
16
![Page 17: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/17.jpg)
Example 4: jQueryBefore jQuery
var nodes = document.getElementsByTagName('a'); for (var i = 0; i < nodes.length; i++) { var a = nodes[i]; a.addEventListener('mouseover', function(event)
{ event.target.style.backgroundColor=‘orange'; }, false ); a.addEventListener('mouseout', function(event)
{ event.target.style.backgroundColor=‘white'; }, false ); }
jQuery abstracts iteration and events
jQuery('a').hover( function() { jQuery(this).css('background-color', 'orange'); }, function() { jQuery(this).css('background-color', 'white'); } );
17
![Page 18: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/18.jpg)
Embedding DSL as a languageHard to say where a framework becomes a language
not too important to define the boundary precisely
Rules I propose: it’s a language if 1) its abstractions include compile- or run-time
checks --- prevents incorrect DSL programsex: write into a closed socket causes an error
2) we use syntax of host language to create (an illusion) of a dedicated syntax
ex: jQuery uses call chaining to pretend it modifes a single object: jQuery('a').hover( … ).css( …)
18
![Page 19: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/19.jpg)
rake rake: an internal DSL, embedded in RubyAuthor: Jim Weirich
functionality similar to make– has nice extensions, and flexibility, since it's
embedded– ie can use any ruby commands
even the syntax is close (perhaps better):– embedded in Ruby, so all syntax is legal Ruby
http://martinfowler.com/articles/rake.html19
![Page 20: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/20.jpg)
Example rake filetask :codeGen do # do the code generationend
task :compile => :codeGen do # do the compilationend
task :dataLoad => :codeGen do # load the test dataend
task :test => [:compile, :dataLoad] do # run the testsend 20
![Page 21: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/21.jpg)
Ruby syntax rulesRuby procedure call
21
![Page 22: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/22.jpg)
How is rake legal ruby?Deconstructing rake (teaches us a lot about
Ruby):
task :dataLoad => :codeGen do # load the test dataend
task :test => [:compile, :dataLoad] do # run the testsend
22
![Page 23: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/23.jpg)
Two kinds of rake tasksFile task: dependences between files (as in
make)file 'build/dev/rake.html' => 'dev/rake.xml' do |t|
require 'paper' maker = PaperMaker.new t.prerequisites[0], t.name
maker.runend
23
![Page 24: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/24.jpg)
Two kinds of tasksRake task: dependences between jobs
task :build_refact => [:clean] do target = SITE_DIR + 'refact/' mkdir_p target, QUIET require 'refactoringHome' OutputCapturer.new.run {run_refactoring}end
24
![Page 25: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/25.jpg)
Rake can orthogonalize dependences and rules
task :second do #second's bodyend
task :first do #first's bodyend
task :second => :first
25
![Page 26: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/26.jpg)
General rulesSort of like make's %.c : %.o
BLIKI = build('bliki/index.html')
FileList['bliki/*.xml'].each do |src|file BLIKI => src
end
file BLIKI do #code to build the blikiend
26
![Page 27: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/27.jpg)
Internal DSLsdeep embedding
27
![Page 28: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/28.jpg)
Deep embeddingRepresent the program as an AST
or with some other data structure
This AST can be interpreted or compiled
28
![Page 29: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/29.jpg)
Parsing involved: DSL in a GP languageGP: general purpose language
29
![Page 30: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/30.jpg)
Parsing involved: GP in a DSL languageGP: general purpose language
30
![Page 31: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/31.jpg)
Reading Read the article about the rake DSL
31
![Page 32: Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs](https://reader036.fdocuments.net/reader036/viewer/2022062410/5681619f550346895dd157d8/html5/thumbnails/32.jpg)
Acknowledgements This lecture is based in part on
Martin Fowler, “Using the Rake Build Language”
Jeff Friedl, “Mastering Regular Expressions”
32