Gr8conf EU 2013 Speed up your development: GroovyServ and Grails Improx Plugin
-
Upload
yasuharu-nakano -
Category
Technology
-
view
2.972 -
download
4
Transcript of Gr8conf EU 2013 Speed up your development: GroovyServ and Grails Improx Plugin
Speed Up Your Development:GroovyServ & Grails Improx Plugin
@nobeansYasuharu NAKANO
GroovyServ
In case of Developing
a Groovy Script
Write Run
Common Strategy:“Trial and Error”
$ time groovy -‐e “println ‘Hello, Groovy’”Hello, Groovy
real 0m0.823suser 0m1.016ssys 0m0.096s
But, It takes 1 sec!
N timesTrial and Error
takes N sec.
“Many a little makes a mickle”
Waiting timemakes me
Forget nextaction
Why so slow?• JVM initialization• Loading Many Many Classes• .... from Disk
http://en.wikipedia.org/wiki/Java_performance#Startup_time
“ It seems that much of the startup time is due to IO-bound operations rather than JVM initialization or
class loading (the rt.jar class data file alone is 40 MB and the JVM must seek a lot of data in this huge file).
Doom of JVM Languages?
Any solutions?
$ groovyConsole
DemogroovyConsole
groovyConsole• Handy for trying Java/Groovy APIs, etc.• But features as editor aren’t so rich:
• No code completion• No syntax checking• No auto import• etc.
• I want to use my favorite IDE/editor!
Other solutions?
GroovyServ“GroovyServ makes Groovy’s startup time much
faster, by pre-invoking Groovy as a server.”
http://kobo.github.io/groovyserv/
DemoGroovy and GroovyServ
$ time groovyclient -‐e “println ‘Hello, GroovyServ’”Hello, GroovyServ
real 0m0.031suser 0m0.001ssys 0m0.002s
Normal Groovy:real 0m0.823suser 0m1.016ssys 0m0.096s
Normal Groovy User Groovy
Script
Shell Environment Java VM
File System
CLASSPATH
cmd line args
stdin
stdou
t
stder
r
Ctrl-Cother ENVs
exit
statu
s
Syste
m.in
Syste
m.out
Syste
m.err
groovy
User GroovyScript
Shell Environment
Java VM
File System
User GroovyScriptCLASSPATH
cmd line args
stdin
stdou
t
stder
r
Ctrl-C
other ENVs
≪file≫Authtoken
exit
statu
s
TCP/IP
Syste
m.in
Syste
m.out
Syste
m.err
groovyservergroovyclient
GroovyServ
In case of Developing
a Groovy Script
GroovyServ Way• Write code with your favorite IDE/editor• Run the code by using GroovyServ as
External Tool• You can also see the result on a console
of IDE
DemoGroovyServ way
Available Editors/IDEs• IntelliJ IDEA• Eclipse/STS/GGTS• Sublime Text 2• Vim + QuickRun.vim• and more...
GroovyServ“GroovyServ makes Groovy’s startup time much
faster, by pre-invoking Groovy as a server.”
http://kobo.github.io/groovyserv/
How to Install
How to Install• Downloadable Binaries for
• Mac OS X (built at Mac OS X 10.8 (x86_64))• Windows
• bundled in Groovy Windows Installer• (built at WindowsXP (32bit) (x86))• (built at Windows7 (64bit) (AMD64))
• Linux (i386) (linked with glibc 2.9)• Self-build (required Gradle)• Homebrew (only Mac)
• $ brew install groovyserv
• http://kobo.github.io/groovyserv/howtoinstall.html
No setting required• If there is a binary set, you can use it:• $ /xxx/groovyserv/bin/groovyclient
• Adding to PATH, you can easily use it:• $ groovyclient
How to Use
Commands• groovyclient• groovyserver
groovyclient
• Invokes a groovy file or one-liner code, by sending a request to server
• Also invokes groovyserver process automatically if not exists
groovyserver
• Invokes a JVM process as a server.• Should be used only when you want to
specify special options.• e.g.• -‐v verbose option
• -‐-‐allow-‐from for remote feature
Available for Other JVM Languages
http://nobeans-en.blogspot.jp/2011/07/high-speed-start-up-jythonclojure-by.html
Jython$ alias gythonserver="env CLASSPATH=/your/jython.jar groovyserver"$ alias gython="groovyclient -‐e 'import org.python.util.jython; jython.main(args)' -‐-‐"
$ gythonserver -‐r...
$ time gython -‐c "print('Hello')"Hello
real 0m0.057suser 0m0.001ssys 0m0.004s
Clojure$ alias glojureserver="env CLASSPATH=/your/clojure.jar groovyserver"$ alias glojure="groovyclient -‐e 'import clojure.main;main.main(args)' -‐-‐"
$ glojureserver -‐r...
$ time glojure -‐e "(println 'Hello)"Hello
real 0m0.053suser 0m0.001ssys 0m0.004s main method of
closure.main class
• A script to get schedule data from groupware
• As editor’s macro• As mail filter• etc.
Using at Runtime
Other Features• Access Control• Propagation of CLASSPATH• Propagation of Environment Variables• Handling System#exit()• Dynamic CWD
Improx Plugincodenamed:GrailsServ
In case of Developing
Grails Application
Grails’ Test supports is Great.
How to run tests• All tests• grails test-‐app
While I’m writing a test,I want to run only it.
I want to run the test as fast as possible.
How to run a specified test fast• Run only a unit test• test-‐app unit: <FQCN of TestClass>
• Run only a integration test• test-‐app integration: <FQCN of TestClass>
• Run only a functional test• test-‐app functional: <FQCN of TestClass>
Why does it needa test phase?
If test phase not specified when run unit test, integration phase is also run.
$ cat test/unit/test/app/sample/FooTests.groovy....class FooTests { void testSomething() { // do nothing }}
$ time grails test-‐app FooTests| Packaging Grails application.....| Tests PASSED -‐ view reports in /xxx/test-‐reportsreal 0m15.375suser 0m35.119ssys 0m1.858s
Oops!����������� ������������������ Integration����������� ������������������ phase����������� ������������������ running!?
$ time grails test-‐app unit: FooTests| Completed 1 unit test, 0 failed in 546ms| Tests PASSED -‐ view reports in /xxx/test-‐reports
real 0m7.303suser 0m18.068ssys 0m0.909s
Less����������� ������������������ than����������� ������������������ a����������� ������������������ half����������� ������������������ of����������� ������������������ previous����������� ������������������ sample!
Specifying a phase can avoidrunning an extra phase
Without a phase:real 0m15.375suser 0m35.119ssys 0m1.858s
Required to run test-app• FQCN of target test class• Test phase of target test class
By the way
How do you run a test while writing it?• (A) By IDE support (GGTS, IntelliJ, etc.)• (B) By “grails test-app” as standalone• (C) By “grails test-app” via Interactive
Mode• (D) Others
Interactive Mode• Very very useful• Makes command execution faster because
the JVM doesn't have to be restarted for each command
• $ grails
• TAB completion is available• Since Grails 2.0
DEMOInteractiveMode/Pros
But still there is trouble• It needs frequent switching of windows
for use while writing code in IDE/editor• To run a test, you must prepare:
• FQCN of target test class• Test phase of target test class
DEMOInteractiveMode/Cons
I’m too tired from such Switching,
Copy & Paste, etc...
Improx PluginProvides the way of using interactive mode
from other process via TCP.
http://grails.org/plugin/improx
DEMOImprox Plugin
Improx Plugin• You can run a test
• Faster than general IDE supports because using interactive mode
• Directly from IDE/editor without complicated operations
Available Editors/IDEs• IntelliJ IDEA• Eclipse/STS/GGTS• Sublime Text 2• Vim + QuickRun.vim• and more...
InteractiveModePROXy
Grails
Improx
Shell Environment
≪script≫improxClient
TCP/IP
≪script≫improxSmartInvoker
≪class≫
InteractiveMode≪class≫
GrailsConsole
≪command≫
improx-start
≪command≫
improx-stop
≪command≫
improx-install-resourcesother clients
≪class≫
GrailsScriptRunner
echo back deleg
ate
run script
start
stop
grails command
expand client scripts
use
• As same as interactive mode• install-‐plugin
• uninstall-‐plugin
• Special built-in commands• create-‐app
• quit
• stop-‐app• exit
• !(shell command)
• Using threads complexly• run-‐app
Unsupported Commands
How to Install
BuildConfig.groovy:
plugins {//... compile ":improx:0.2"//...}
Install Plugin
Install Client Scripts$ grails improx-‐install-‐resources| Improx resources installed successfully: /xxx/improx-‐resources
$ chmod +x improx-‐resources/scripts/*.sh$ mv improx-‐resources $HOME/.improx-‐resources
If����������� ������������������ once����������� ������������������ you����������� ������������������ store����������� ������������������ them����������� ������������������ to����������� ������������������ global����������� ������������������ path,����������� ������������������ ����������� ������������������ you����������� ������������������ don't����������� ������������������ need����������� ������������������ the����������� ������������������ preparation����������� ������������������
for����������� ������������������ other����������� ������������������ projects.
How to Use
Grails Commands• improx-‐start
• improx-‐stop
• improx-‐install-‐resources
improx-‐start / improx-‐stop
• improx-‐start
• Starts an interactive mode proxy server• improx-‐stop
• Stops the interactive mode proxy server which is running
improx-‐install-‐resources
• Installs “improx-resources” directory which has some client scripts into your application project
improx-‐resources/ !"" scripts #"" improxClient.groovy #"" improxClient.sh #"" improxSmartInvoker.groovy !"" improxSmartInvoker.sh
Client Scripts• improxClient.sh• improxClient.groovy• improxSmartInvoker.sh• improxSmartInvoker.groovy
improxClient.(sh|groovy) <Command>
• Runs a command in interactive mode via TCP port of improx server
• Very simple
$ improxClient.sh helpExecuting 'help' via interactive mode proxy.........Usage (optionals marked with *):grails [environment]* [options]* [target] [arguments]*
...(snipped)...
test-‐apptomcatuninstall-‐pluginupgradewarwrapper$
improxSmartInvoker.(sh|groovy) FILE_PATH
• Invokes any *.groovy file in an appropriate way
• Very useful for IDE/editor
How SMART is it?
grails> test-‐app unit: sample.SampleUnitTests
$ improxSmartInvoker.sh /path/to/yourApp/test/unit/sample/SampleUnitTests.groovy
Runs the following commandon the interactive mode:
grails> test-‐app integration: sample.SampleIntegTests
$ improxSmartInvoker.sh /path/to/yourApp/test/integration/sample/SampleIntegTests.groovy
Runs the following commandon the interactive mode:
$ improxSmartInvoker.sh /path/to/yourApp/test/functional/sample/SampleFuncTests.groovy
grails test-‐app functional: sample.SampleFuncTests
Because����������� ������������������ functional����������� ������������������ test����������� ������������������ doesn’t����������� ������������������ work����������� ������������������ well����������� ������������������ on����������� ������������������ interactive����������� ������������������ mode����������� ������������������ at����������� ������������������ present.
Runs the following commandas new standalone Grails process:
$ improxSmartInvoker.sh /path/to/scriptDir/myTribialScript.groovy
groovy /path/to/scriptDir/myTribialScript.groovy
If����������� ������������������ you've����������� ������������������ installed����������� ������������������ GroovyServ,����������� ������������������ the����������� ������������������ groovyclient����������� ������������������ is����������� ������������������ automatically����������� ������������������ used����������� ������������������ instead����������� ������������������ of����������� ������������������ groovy����������� ������������������ command.
Runs the following commandas a normal Groovy script:
All you have to do isto set
improxSmartInvokeras External Tool
of IDE/editorhttp://kobo.github.io/grails-improx/guide/integrationWithEditors.html
SimpleProtocol
TCP• Request:
COMMAND
• Whole response is result of the command• You can also use as client
• e.g. telnet, nc
$ telnet localhost 8096Trying ::1...Connected to localhost.Escape character is '^]'.help......Usage (optionals marked with *):grails [environment]* [options]* [target] [arguments]*
...(snipped)...
uninstall-‐pluginupgradewarwrapper$
HTTP• Request
GET /COMMAND HTTP/[0-‐9.]+
• Whole response is result of the command• You can also use as client
• e.g. Web browser, wget, curl
• GroovyServ
• http://kobo.github.io/groovyserv/index.html
• https://github.com/kobo/groovyserv
• Improx - Interactive Mode Proxy
• http://kobo.github.io/grails-improx/guide/
• http://grails.org/plugin/improx
• https://github.com/kobo/grails-improx
• Demo code
• https://github.com/nobeans/gr8confeu2013-demo
Appendix
GroovyServUnder the hood
Under the hood• Client-Server Communication• Access Control• Propagation of CLASSPATH• Propagation of Environment Variables• Handling System#exit()• Dynamic CWD
Client-Server Communication
Transfer of Standard In/Out/Err Stream
$ groovyclient -‐e \"println ‘Hello, GroovyServ’"
groovyclient
ClientConnection
OutputStream
≪script≫
println “Hello, GroovyServ!”
≪System.out≫
DynamicDelegatedPrintStream
writes data converted based on a protocol into output stream of socket
Hello, GroovyServ!
if Channel header == ”err”To STDERR
if Channel header == ”out”To STDOUT
Client console
$ groovyclient -‐e \"System.in.eachLine{ println it*5 }"
ClientConnection
PipedOutputStream
PipedInputStream
groovyclient ≪thread≫
StreamRequestHandler
≪script≫
System.in.eachLine { ... }
Connected
rawData = Socket.inputStream.read()bodyData = convert(rawData)pipedOutputStream.write(bodyData)
≪System.in≫
DynamicDelegatedInputStream
Protocol over TCP
InvocationRequest'Cwd:' <cwd> LF'Arg:' <arg1> LF'Arg:' <arg2> LF :'Env:' <env1>=<value1> LF'Env:' <env2>=<value2> LF :'Cp:' <classpath> LF'AuthToken:' <authToken> LFLF
where: <cwd> is current working directory. <arg1>,<arg2>.. are commandline arguments which must be encoded by Base64. (optional) <env1>,<env2>.. are environment variable names which sent to the server. (optional) <value1>,<valeu2>.. are environment variable values which sent to the server. (optional) <classpath> is the value of environment variable CLASSPATH. (optional) <authToken> is authentication value which certify client is the user who invoked the server. LF is line feed (0x0a, '\n').
StreamRequest
'Size:' <size> LFLF<body from STDIN>
where: <size> is the size of body to send to server. <size>==-1 means client exited. <body from STDIN> is byte sequence from standard input.
InvocationResponse
'Status:' <status> LF
where: <status> is exit status of invoked groovy script.
StreamResponse
'Channel:' <id> LF'Size:' <size> LFLF<body for STDERR/STDOUT>
where: <id> is 'out' or 'err', where 'out' means standard output of the program. 'err' means standard error of the program. <size> is the size of chunk. <body from STDERR/STDOUT> is byte sequence from standard output/error.
Access Control
Against Evil Request
From Evil Client
Client Address Checking / Authtoken• Request only from loopback address is
allowed.• Authtoken is generated and stored into a
file when server process starts.• $HOME/.groovy/groovyserver/authtoken-‐<PORT>
• Authtoken always is used for checking that the request is from valid owner user.
-‐-‐allow-‐from
• By default, A request only from loop-back address is available.
• You can allow specified addresses.$ groovyserver -‐-‐allow-‐from 192.168.1.1
SECURITY RISK: Be careful when using this option
-‐-‐authtoken
• By default, authtoken is generated automatically.
• You can specify any string.
SECURITY RISK: Be careful when using this option
$ groovyserver -‐-‐authtoken MY_AUTHTOKEN
At your own risk
Propagation of CLASSPATH
Client and Server are Different
Process
≪process≫groovyserver
≪process≫groovyclient
CLASSPATH=/my/classpath/xxx.jar
CLASSPATH=/my/classpath/zzz.jar
Individual Env for each
If GroovyServ were too simple, ...
≪process≫groovyserver
≪process≫groovyclient
CLASSPATH=/my/classpath/foo.jar
≪process≫groovyserver
≪process≫groovyclient
Please execute this!
new foo.Foo().execute()
CLASSPATH=/my/classpath/foo.jar
≪process≫groovyserver
≪process≫groovyclient
CLASSPATH=/my/classpath/foo.jar
I don’t know its class!
Please execute this!
new foo.Foo().execute()
≪process≫groovyserver
≪process≫groovyclient
CLASSPATH=/my/classpath/foo.jar
I don’t know its class!
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:script_from_command_line: 1: unable to resolve class foo.Foo @ line 1, column 1. new foo.Foo().execute() ^
1 error
Please execute this!
new foo.Foo().execute()
But GroovyServ is enough smart
≪process≫groovyserver
≪process≫groovyclient
Please execute this!
new foo.Foo().execute()
CLASSPATH=/my/classpath/foo.jar
≪process≫groovyserver
≪process≫groovyclient
CLASSPATH=/my/classpath/foo.jar
Please execute this!
new foo.Foo().execute()
CLASSPATH=/my/classpath/foo.jar
new foo.Foo().execute()
≪process≫groovyserver
≪process≫groovyclient
CLASSPATH=/my/classpath/foo.jar
You got it!
Please execute this!
new foo.Foo().execute()
CLASSPATH=/my/classpath/foo.jar
CLASSPATHwhich is propagated
from client is Volatile
≪process≫groovyserver
≪process≫groovyserver
≪process≫groovyclient
CLASSPATH=/my/classpath/foo.jar
OK
Please execute this!
new foo.Foo().execute()
CLASSPATH=/my/classpath/foo.jar
≪process≫groovyserver
Done
≪process≫groovyserver
≪process≫groovyclient
CLASSPATH=/my/classpath/bar.jar
Please execute this!
new bar.Bar().execute()
CLASSPATH=/my/classpath/bar.jar
OK
≪process≫groovyserver
Done
How to specify CLASSPATH from
Client
-classpath / -cp / --classpath
$ groovyclient -‐cp /my/classpath/foo.jar -‐e “new foo.Foo().execute()”
CLASSPATH env$ export CLASSPATH=/my/classpath/foo.jar$ groovyclient -‐e “new foo.Foo().execute()”
Default CLASSPATHon Server
CLASSPATH env$ export CLASSPATH=/my/classpath/foo.jar$ groovyserver
≪process≫groovyserver
I feel some power...
CLASSPATH=/my/classpath/foo.jar
≪process≫groovyserver
≪process≫groovyclient
CLASSPATH=/my/classpath/foo.jar
Please execute this!
new foo.Foo().execute()
CLASSPATH=/my/classpath/foo.jar
I already know foo.jar!
≪process≫groovyserver
Done
CLASSPATH=/my/classpath/foo.jar
≪process≫groovyserver
≪process≫groovyclient
CLASSPATH=/my/classpath/bar.jar
Please execute this!
new bar.Bar().execute()
CLASSPATH=/my/classpath/bar.jar
OK
CLASSPATH=/my/classpath/foo.jar
≪process≫groovyserver
Done
CLASSPATH=/my/classpath/foo.jar
Propagation of Environment
Variables
Handywith IDE/editor
≪process≫groovyserver
≪process≫groovyclientCLASSPATH=
/my/classpath/xxx.jar
How to propagate env
KEY_A_1=VALUE_A_1
KEY_A_2=VALUE_A_2
KEY_B=VALUE_B
KEY_A_1=VALUE_A_1
KEY_B=VALUE_B
KEY_A_2=VALUE_A_2
groovyclient options• -‐Cenv <substr>
• Passes environment variables of which a name includes specified substr
• -‐Cenv-‐all
• pass all environment variables• -‐Cenv-‐exclude <substr>
• don't pass environment variables of which a name includes specified substr
≪process≫groovyserver
≪process≫groovyclientCLASSPATH=
/my/classpath/xxx.jar
-‐Cenv _A_
KEY_A_1=VALUE_A_1
KEY_A_2=VALUE_A_2
KEY_B=VALUE_B
KEY_A_1=VALUE_A_1
KEY_B=VALUE_B
KEY_A_2=VALUE_A_2
≪process≫groovyserver
≪process≫groovyclientCLASSPATH=
/my/classpath/xxx.jar
-‐Cenv-‐all
KEY_A_1=VALUE_A_1
KEY_A_2=VALUE_A_2
KEY_B=VALUE_B
KEY_A_1=VALUE_A_1
KEY_B=VALUE_B
KEY_A_2=VALUE_A_2
≪process≫groovyserver
≪process≫groovyclientCLASSPATH=
/my/classpath/xxx.jar
-‐Cenv-‐all -‐Cenv-‐exclude _2
KEY_A_1=VALUE_A_1
KEY_A_2=VALUE_A_2
KEY_B=VALUE_B
KEY_A_1=VALUE_A_1
KEY_B=VALUE_B
KEY_A_2=VALUE_A_2
Caution:ENV on server isn’t cleared after each
invocation
≪process≫groovyserver
≪process≫groovyclientCLASSPATH=
/my/classpath/xxx.jar
-‐Cenv KEY_A_1
KEY_A_1=VALUE_A_1
KEY_A_2=VALUE_A_2
KEY_B=VALUE_B
KEY_A_1=VALUE_A_1
≪process≫groovyserver
≪process≫groovyclientCLASSPATH=
/my/classpath/xxx.jar
-‐Cenv KEY_A_2
KEY_A_1=VALUE_A_1
KEY_A_2=VALUE_A_2
KEY_B=VALUE_B
KEY_A_1=VALUE_A_1
KEY_A_2=VALUE_A_2
≪process≫groovyserver
≪process≫groovyclientCLASSPATH=
/my/classpath/xxx.jar
-‐Cenv KEY_B
KEY_A_1=VALUE_A_1
KEY_A_2=VALUE_A_2
KEY_B=VALUE_B
KEY_A_1=VALUE_A_1
KEY_A_2=VALUE_A_2
KEY_B=VALUE_B
Handling System#exit()
Server processmust not be terminated,
even if a script executes System.exit()
SecurityManagerpublic class NoExitSecurityManager2 extends NoExitSecurityManager {
public void checkExit(final int code) { throw new SystemExitException(code, "called System.exit(" + code + ")"); }
}
GroovyMainprivate boolean run() { try { //... processArgs(...) //... } catch (SystemExitException e) throw e; } catch (Throwable e) { //... }}
static void processArgs(...) { //... if (!process(cmd, classpath)) { // Disabled because this causes a secondary disaster //System.exit(1); } //...}
Dynamic CWD
CWD =Current Working
Directory
$ cd /tmp$ groovyserver -r
$ cd /home/nobeans$ cat test.txtCan you read me?
$ groovyclient -e ‘println(new File(“test.txt”).text)’
Caught: java.io.FileNotFoundException: test.txt (No such file or directory) ...SNIP...
≪process≫groovyserver
CWD=/tmp
I wanna use “test.txt”of current directory.
Is it “/tmp/test.txt” ?Hmm, such a file not found.
CWD of client should be set to server
for each invocation.But, ...
In Java,CWD is thoroughly
hidden
Solution• Setting to “user.dir” system property
• It affects File#getAbsolutePath()• Changing CWD of native JVM process
• Many classes don’t use “user.dir” to complete an absolute path.
• But, JNI is troublesome...
JNA is easy way to use native API
UnixLibC.groovy:interface UnixLibC extends com.sun.jna.Library.Library { int chdir(String dir) int setenv(String envVarName, String envVarValue, int overwrite)}
pom.xml: <dependency> <groupId>net.java.dev.jna</groupId> <artifactId>jna</artifactId> <version>3.2.2</version> </dependency>
Usage:def libc = com.sun.jna.Native.loadLibrary("c", UnixLibC.class)libc.chdir(“/tmp/foo/bar”)
$ cd /tmp$ groovyserver -r
$ cd /home/nobeans$ cat test.txtCan you read me?
$ groovyclient -e ‘println(new File(“test.txt”).text)’
Can you read me?
≪process≫groovyserver
CWD=/tmp
I wanna use “test.txt”of current directory.
≪process≫groovyserver
CWD=/home/nobeans
Is it “/home/nobeans/test.txt” ?OK, I got it.
Yes!