Groovy On Trading Desk (2010)
-
Upload
jonathan-felch -
Category
Technology
-
view
1.785 -
download
0
description
Transcript of Groovy On Trading Desk (2010)
Groovy On The Trading Desk
Jonathan FelchQuantitative Strategies
Volant Trading LLC
Overview● Motivations for Polyglot Software with Groovy● Introducing Groovy: Scripting on the VM
● Skills To Take With You: Learning Groovy● Groovy DSLs: Creating the syntax you need● Unlearning Java, C++, C#● Groovy Performance and Concurrency● Relearning Java: The Future of the VM ● Questions
Motivations
● High Speed OO Tools + Dynamic / Functional Tools
● Re-use existing assets
● Lazy Evaluation
● Externalize Logic and Business Rules
● Dynamic Evaluation
Why Groovy ?● Meet Requirements
– Lazy & Dynamic Evaluation, – Concise Synatx, Rapid Development, Iterative Prototyping– Supported But Did Not Require Functional Style, Support for
Closures / Lamda Expressions– Tool Support, Attractive Adoption Pattern– Human Readable / Machine Executable Data Interchange– Natural Data Integration Platform
● Strong and Focused Community● Familiar Syntax / Ease of Adoption● Expensible Syntax : DSLs
Scripting on the VM
Evolution of the VM: Towards Dynamic Programming
– Features Popular in Most Used 'Frameworks' Exploited Specific Benefits and Uses of Dynamic Programming for a general-purpose or 'cross-cutting' concern
● Hibernate, Spring Framework, Jboss AOP, AspectJ
– Introduction of Dynamic Programming for Testing,Configuration and Business Rules
● BeanShell, Jython, JRuby
– Next Generation Scripting: Groovy, Scala, Clojure● Specific to VM, not a straight port of another language● Emphasis on Functional Programming● Tight Integration of FP / DP with a high performance OO Language
Introducing Groovy● The Blue Pill: Syntactically compatible with Java
– Easy Adoption for Java Programmers– Virtually seamless / virtually flawless integration with Java– Natural Support for Multiple Styles– Compatibility allows for remarkable performance tuning
● The Red Pill: A Language On Its Own– Type-system: Dynamically / Optionally Strong Typing– Closures: Lambda Expressions / Functions as Objects– GDK: Object and Collection Enhancements– Dynam Eval: Code is Data, Data is Code– Dynam Syntax: Write your own syntax– Enhancements: New operators, data literals, new basic types
Job Trends / Adoption Hints for new JVM Languages
Groovy Use Cases: ● 'Super Glue': I/O and Integration
● 'Code as Data': Configuration, Process Scripting, Business Rules
● 'Live Analysis': Ad-Hoc Queries & Expressions
● 'Language Building': New Syntax and DSLs
● 'Half-Baked Ideas'
● 'House Elf': Cleanup and Maintenance
Learning Groovy
Im proving o n O b je c tdef abstract class OptionPxModel {
def vol, carry, strike, expiry, pc def abstract px (def spot)
} OptionPxModel model = new BlackScholesModel(strike : 100, pc : 'C' ) model.vol = 0.15 model.expiry = Date.parse('MM/dd/yyyy', '11/18/2010') println “The Strike 100 Nov 2010 Call is worth ${model.px(102.0)}”
C lo s ure s : def r = new Random() def normRand = { r.nextGaussian() } def stockPrice = { px, vol, rate, time → px * exp((rate – vol**2) * time + normRand() * vol * time) }
Learning Groovy: CollectionsL is tsdef list = [5, 6, 7, 8]assert list.size == list.size() assert list[2] == list.getAt(2) == list.get(2) == 7 assert [1,2,3,4,5][-1] == 5 && [1,2,3,4,5][-2] == 4assert list.find { it > 6 } == 7 && list.findAll { it > 6 } == [7, 8]list.each { println “$it ${it * 2}” }['a', 'b', 'c'].eachWithIndex{ it, i -> println "$i: $it" }
Multiple A s s ig nm e ntdef (int a, int b) = [1, 2](a, b) = [b, a]R a ng e sassert ('a'..'d') == ['a','b','c','d']assert [ 1, *3..5, 7, *9..<12 ] == [1,3,4,5,7,9,10,11]def oneWeekForward = (new Date() .. (new Date() + 7))
Ma ps[ 'id': 'FX-17', name: 'Turnip', 99: 123, (-97): 987, "tail's": true ].each { key, value → println “$key => $value” }
Groovy One (or 2, 3, 4) Liners AppsExecuting A Process: ”processMyFile ./output.txt $Destination”.execute()
Adding A She-Bang To All Groovy Files: groovy -i .bak -pe "if (count == 1) println '#!/usr/bin/groovy'" *.groovy
Print the 1st column and 2nd-to-last column from filegroovy -a -pe "split[0] split[-2]" some.filename // or set of files
Create and Shuffle a Deck of Cardsdef deck = [ [ *2..10, *('J Q K A'.split()) ], 'C D H S'.split() ]
.combinations().collect { it.join('') }; Collections.shuffle(deck)
Generate 21-Character OSI Symbol for Option def osi = { symbol, year, month, day, cp, strike -> sprintf('%1$s%2$2d%3$2d%4$2d%5$s%6$08.3f', [ symbol.padLeft(6), year as Integer, month as Integer, day as Integer, cp as String, strike as Double ] as Object[] )}
Learning Groovy: Operators'A s O pe ra to r'
"3.14159" as Double def readable = { it.put("12 34".reverse()); 5 } as ReadableAs O pe ra to r O n C lo s ure s + Ma ps Ob je c ts ? →
def map; map = [ i: 10, hasNext: { map.i > 0 }, next: { map.i-- } ] as Iterator
while ( map.hasNext() ) println map.next()
E lvis , S a fe Re fe re nc e , S pre a d & S pa c e s hip
def lang = data?.favoriteLanguage ?: 'Groovy' assert ['cat', 'elephant']*.size() == [3, 8] assert (3 <=> 4) == -1 && (3 <=> 3) == 0 && (5 <=> 3) == 1
Re g e x
def list = ["foo", "bar", "moo"] def pattern = ~/foo/
assert "foo" =~ list.find { it =~ pattern }assert ["foo", "moo"] == list.findAll { it ==~ /.*oo/ }
One Slide Apps: RSS Reader
def url ='http://www.groovyblogs.org/feed/rss'def items = new XmlParser().parse(url).channel.itemdef cols = 'pubDate title description'.tokenize()groovy.swing.SwingBuilder.build { frame(id:'f', title: 'Groovy RSS', visible :true) {
scrollPane {table {
tableModel(list: items) {cols.each { col →
closureColumn header: col,read: { it[col].text() }
} } } } }f.pack()
}.show()
Credit: Dierk König, GR8 Conference 2009, Copenhagen
One Slide Apps: Monte Carlo
def px = 100, rate = 0.05, vol = 0.15, time = 1.0def strikes = [ 8..12 ].collect { it * 10 }def w = RandomNumbers.getNormDist(1000,1000)def S = px * exp((rate-vol ** 2)* time + sqrt(time)* vol * w)strikes.each { K → def optionValue = max(0, S – K) def df = exp(-rate * time) println “${K} : ${df * optionValue as Number}”})
One Slide Apps: RESTful Web Serverimport com.sun.net.httpserver.*;
def isPrime = { Long N -> if (N < 2) return false; for (long i = 2; i*i < N; i++) if (N % i == 0) return false return true }
def pws = { HttpExchange exch -> exch.sendResponseHeaders(200,0); def pe = exch.requestURI.path.split('/') if (pe.size() == 3 && pe[1] == 'prime') { exch.responseBody.write("${isPrime(pe[2] as Long)}".bytes) } else exch.responseBody.write("${exch.requestURI.path}".bytes) exch.responseBody.close();} as HttpHandlerdef run = { handler, minutes -> def server = HttpServer.create(new InetSocketAddress(8888),0) server.createContext('/', handler) server.start()}run(pws,1)
Unlearning Java: Taking the Red Pill● Meta-Programming
● Design Patterns
● Compile Time Meta-Programming
● Power of Lazy Evaluation
Dynamic Language: Evaluation or Syntax
“In Lisp, you don’t just write your program down toward the language, you also build the language up toward your program” - Paul Graham
“When [Smalltalk] is used to describe an application system, the developer extends Smalltalk, creating a domain-specific language by adding a new vocabulary of language elements ...” - Adele Goldberg DSLs
Meta-Programming● Understanding The Type System
– Groovy has an optionally dynamic strongly typed type system– What does that mean ?
● Dynamic Method Invocation and GroovyObject– invokeMethod, get/setProperty
– What is a MetaClass ?● Meta Object Protocol
– def methodMissing(String name, args) { }– def propertyMissing(String name)– def propertyMissing(String name, value)
● Overloading Operators
GORM as a DSL// These Class Definitions Generate SQL and Database Tables
class Book {String titleDate releaseDateAuthor author
}
class Author {String name
}
// These Operation Generate SQL and Database Queries
def book = Book.findByTitle("The Stand")book = Book.findByTitleLike("Harry Pot%")book = Book.findByReleaseDateBetween( firstDate, secondDate )book = Book.findByReleaseDateGreaterThan( someDate )
book = Book.findByTitleLikeOrReleaseDateLessThan( "%Something%", someDate )
GORM as a DSLclass GORM {
def dynamicMethods = [ ]
def methodMissing(String name, args) {def method = dynamicMethods.find { it.match(name) }if(method) { GORM.metaClass."$name" = { Object[] varArgs →
method.invoke(delegate, name, varArgs)}return method.invoke(delegate,name, args)
}// GORM actually writes or creates a new method hereelse throw new MissingMethodException(
name, delegate, args )}
}
DSL: Tool Kit● Meta-Methods
– String.metaClass.swapCase= {delegate.collect{ c →
c in 'A'..'Z'? c.toLowerCase() :c.toUpperCase()}.join()}
assert "Foo".swapCase()== "fOO"
Number.metaClass.multiply= { Grid g g.multiple(delegate) }→
● MOP– propertyMissing, methodMissing, respondsTo,
hasProperty, listMethods, selectMethod
● Categories, ObjectBuilder, MetaBuilder
Function Eval / Visualizationnew SwingBuilder().frame(title:'Quant Modeller', size : [1100,600], // .. parameters here not shown {
tableLayout() {tr {
td { widget(gui) } td { widget(editor, border : titledBorder('Model Script',
preferredSize :[900,600])) } td {
panel(border : titledBorder('Model Calibration'), preferredSize :[200,600]) {tableLayout(cellpadding :15) { tr {
td { label(text : 'Params:') }td { button(text : 'Add', \* Edited Out *\ }
tr { td(colspan : 2, align : 'center') {
button(text : 'Execute', actionPerformed : {
shell.setVariable 'f', [:]shell.eval1uate(editor.text)def f = shell.getVariable('f')def obj = [ f : f] as Model3dgui.model = objgui.redraw()
} ) } } }
New AST Transforms
• @Singleton — okay, not really a pattern :-)
• @Immutable, @Lazy, @Delegate
• @Newify
• More DSL and Meta-Magic! @Category and @Mixin
• @PackageScope
• Swing’s @Bindable and @Vetoable
• Grape’s @Grab
The Evil Java Singleton
public class Evil {public static final Evil instance = new Evil();
private Evil() {}static Evil getInstance() { return instance; }
}
• In Groovy now:–@Singleton class Evil {}
• A lazy version also:–@Singleton(lazy = true) class Evil {}
Credit: Guillaume Laforge, “To Infinity and Beyond” & GDK 1.7 Release notes
DSLs → They Start Out As APIs// closure maddness// Before refactoring
def phrase = "The quick brown fox jumps over the lazy dog"
def result = phrase.toLowerCase().toList().findAll{ it in "aeiou".toList() }. // like WHERE …groupBy{ it }. // like GROUP BY …findAll{ it.value.size() > 1 }. // like HAVING …sort{ it.key}.reverse(). // like ORDER BY …collect{ "$it.key:${it.value.size()}" }.join(", ")
println result
Credit: Paul King, “Groovy DSL”
DSL: APIs That Get Refactored// Refactored helper closuresdef lowercaseLetters= phrase.toLowerCase()def vowels = { it in "aeiou".toList() }def occursMoreThanOnce= { it.value.size() > 1 }def byReverseKey= { a, b -> b.key<=> a.key}def self = { it }def entriesAsPrettyString= { "$it.key:${it.value.size()}" }def withCommaDelimiter= ", "
// Refactored main closure println lowercaseLetters.
findAll(vowels). groupBy(self).
findAll(occursMoreThanOnce).sort(byReverseKey).collect(entriesAsPrettyString).
join(withCommaDelimiter)Credit: Paul King, “Groovy DSL”
Putting All Together I● Drop Parenthesis
● Named Parameters
● Objects Graphs → Tree of 'Nouns' and 'Verbs'
● Human Readable / Machine Executable
● Specific Syntax
Putting All Together IIorder to buy 200.shares of GOOG {
limitPrice 500 allOrNone false at the value of { qty * unitPrice - 100 }
}
take 2.pills of chloroquinine after 6.hours
def "length of Spock's & his friends' names"() { expect: name.size() == length where:
name | length "Spock" | 5 "Kirk" | 4 "Scotty" | 6
} Credit: Paul King, “Groovy DSL”
Config SlurperLog4j {
appender.stdout = "org.apache.log4j.ConsoleAppender"appender."stdout.layout"="org.apache.log4j.PatternLayout"rootLogger="error,stdout"
logger {org.springframework="info,stdout"
}
additivity {org.springframework=false
}}
databaseURL : //
development {databaseURL : //
}
testing {databaseURL : //
}
Performance ● Dynamic Typing is Expensive (But Worth It)
– Hot Spot Opportunity
● Dynamic Method Invocation is expensive (But Getting Better)
– Da Vinci Project and invokeDynamic bytecode– Hot Spot Opportunity
● BigDecimal & Collections → Insanely Expensive For Serious Math
– People need to better learn to use IEEE floating point types–
Performance II ● Getting Too Groovy ???
– Abusing .each { tuple } → // for loops versus anon closures
● Simplicity --> faster– Groovy reduces your dependency on Java Enterprise– Gpath vs Xpath (Gpath almost always wins)– Use Visual VM: Groovy Memory Use Can Be Mysterious
● Too Many Collections ? – Is The Idea to 'Cache' The World As A Map
● Use Groovy-Aware Java for Faster Code– People Need to learn IEEE Floating Point Types
Actors
class Player extends AbstractPooledActor {String namedef random = new Random()
void act() {loop {
react {// player replies with a random movereply Move.values()[
random.nextInt(Move.values().length)]
}}
}}
Scaling the VM: Immutable Data Message + Services
Are Java & C# Becoming Groovy ?● Google Guava (formerly Collections)● Lots of new Libraries for FP, Enhanced
Collections, Tuples, Monads, etc...● Doug Lea's Fork-Join + Predicate Libraries● Project Lambda : Closures for Java (JDK 8 ?)● Da Vinci Project : invokeDynamic (JDK 7 ?)● Is Gpath LINQ for Java ? (Now ?)● Universal Low Level VM● Will Groovy or Groovy++ Continue ?
A few bad months for Java
● JavaTM By Oracle , ● The Java Community Process, Doug Lea● The Java Community Process, Apache● JavaOne, The Conference● Oracle vs Google● Oracle vs Apache● IBM and Apache● OpenJDK
Where is Goovy Going ?● Some Important JDK Improvements will be in
JDK 7 (as planned): invokeDynamic
● More explicitly FP approach to concurrency and state
– AST MP / GoF Patterns– @Immutable / @Singleton ( lazy : true) – GPars Actor-like syntax
● Groovy Server for Fast Scripting– Difficult on a cluster
Where is Goovy Going ?● Closures with Memory def cl = {a, b →
sleep(3000) // create a noticeable pause a + b
}def mem = cl.memoize()
// memorize() variants are // memorizeAtMost(n), memorizeAtLeast(n), // memorizeBetween(m, n)
● Groovy++– Not a fully compatible fork– An interesting argument – But no.
Groovy Gotchas!● Subclassing Map, Unexpected Results
● Map.class● the dot operator,
getAt(...), getXXX(), [ name ] literals
● Private isn't Private ● Inconvenient when you really want to hide implementation data
● Unexpected Weird Effects – Combining Static and Dynamic Numeric Types Across Java Code
and Groovy Code ● Be careful and deliberate with typing
– Method selection surprises ● identical static closures and static functions do not respond to
the same way Invoked dynamically
Best Practices● Explicitly Signatures and Types Where You Can
● Better GroovyDoc documentation● Faster method selection● More shareable (with Java / Scala / Ioke / etc..) Libraries
● Write Groovy Aware (GroovySupport) Java● This is where to build your high-speed / high-performance code
● Use ConfigSlurper to externalize environment / context data
● Use CliBuilder and Groovy Server if it fits your production model
● Add Performance Tests To Your Suite / Learn to use Visual VM– Testing is more important with dynamic and functional programming– Small tests – small in terms of code coverage – on big data sets
● Best place to find surprising performance enhancements– Automating 'Just Got Lucky' Into Your Development Cycle
Acknowledgments & Suggestions
● Derik Kognig, Author Groovy In Action ● Peter Bell, Systems Forge● Paul King, Asset Australia ● Guillaume Laforge, Groovy Project Manager● PLEAC, Project Euler, Groovy Zone
Contact Info● [email protected] ● [email protected]● www.twitter.com/Jonathan.Felch
– Rare but regular contributions via Blog, GitHub, SlideShare
● 3 – 5 contributions a year, typically unfinished half-baked ideas● Really targeted at a dozen or so people that I have worked with
in the past or occasionally collaborate