Invoke dynamic your api to hotspot

InvokeDynamicYour API for HotSpot

Tony ArcieriBoundary

October 11th, 2012

Who am I?

Our story begins...

Java was slowOnce upon a time...

Anamorphic

AnamorphicStrongTalk

•Optional static typing

•Modern garbage collector

• JIT compiler



Sun MicrosystemsJava


Sun MicrosystemsJava

Stolen legacy!

HotSpot

Compile When Code Runs

Just-In-Time Compiler

JIT Spectrum

JIT SpectrumEager Lazy


CLRHiPE

V8HotSpot


CLRHiPE

V8HotSpot

Static Optimizations Runtime Optimizations

•C1: Client Compiler

•C2: Server Compiler

HotSpot

•C1: Client Compiler

•C2: Server Compiler

• -XX:+TieredCompilation

HotSpot

Mixed ModeProfiling JIT

Start by interpretingJVM “emulates” bytecodes in userspace

BYTECODE

Locate “hot spots”

BYTECODE

Using runtime profiling information

Generate machine codeCompile JVM bytecode to native ISA

BYTECODE1010

1010

1010

1010

1010

after 10,000 calls

• Inlining methods

•Unrolling loops

• Eliding locks

• Eliminating dead code

• Escape analysis

Optimizations

InliningCombine and optimize across calls

Method 1

CALL Method 2

InliningCombine and optimize across calls

Method 1

Method 2

Call SitesWhere the magic happens...

CALL

Call Sites

• Profile data

•Monomorphic inline cache

• Polymorphic inline cache

“Shapes”

MonomorphicOnly one type seen at call site

CALLA

BimorphicTwo types seen at call site

CALLA B

PolymorphicMany types seen at call site

CALLA CB

MegamorphicToo many types seen at call site

CALLA CB

D

E

F

G

•Measure invocations and branches

•Make an educated guess

• Inline the code in question

• Back out if we guessed wrong

Inlining

•Check invariants at a “safe point”

•Did we guess wrong?

• Revert optimization and try again

Deopt

JVM Limitations

Made for Java

• InvokeVirtual

• InvokeInterface

• InvokeStatic

• InvokeSpecial

Non-Java call sites are opaque to HotSpot

Can’t be inlined

Method 1

CALL Method 2

Teach HotSpot?See through non-Java call sites

Method 1

CALL Method 2

Inline just like Java

Method 1

Method 2

What if it worked for any language?

Even Ruby?

InvokeDynamic

InDy

• JSR-292: “Supporting Dynamically Typed Languages on the Java Platform”

• Initial version shipped in Java 7

• Feeds directly into HotSpot in Java 8

New JVM instruction

java.lang.invoke

InvokeDynamic

“User defined data endpoint”

JVM Data Endpoints

• Invoke operations

• Array element access

• Property lookup

• And more!

InDy Use Cases

•Custom invocation

• “Constant” values (e.g. globals, language intrinsics)

• Scoped values (e.g. instance variables)

InDy

How does it work?

Pieces of InDy

• Bootstrap methods

•Call sites

•Method handles

• Switch points

Bootstrap method

Bootstrap methodWire up the call site

Bootstrap methodYour own code!

Bootstrap Method

•Called the first time each InDy call site is used

• Find MethodHandle to dispatch

• Return CallSite object

Call SiteReplaces InDy instruction in bytecode

Call SiteHolds onto a chain of MethodHandles

java.lang.invoke.CallSite

•ConstantCallSite: unchangeable

• VolatileCallSite: seen coherently across cores

•MutableCallSite: may appear different across cores

Any of those can be subclassed

java.lang.invoke.MethodHandle

•Directly executable reference to a Java method (i.e. fast-as-Java)

• Inlineable by HotSpot

•MethodHandles all the way down

Methods are first-class objects

Project LambdaLambdas for the JVM

Coming in Java 8! (Summer 2013)

Guarded Invocation

MethodHandle.guardWithTest(! MethodHandle test, ! MethodHandle target,! MethodHandle fallback)

Guarded Invocation

http://docs.oracle.com/javase/7/docs/api/java/lang/invoke/MethodHandles.html#guardWithTest(java.lang.invoke.MethodHandle,%20java.lang.invoke.MethodHandle,%20java.lang.invoke.MethodHandle)


http://docs.oracle.com/javase/7/docs/api/java/lang/invoke/MethodHandle.html






fallback can rebind the call site

Handle new types as they’re seen

SwitchPoints

SwitchPoint.guardWithTest(! MethodHandle target,! MethodHandle fallback)

SwitchPoint.invalidateAll(! SwitchPoint[])

SwitchPoints







java.lang.invoke.SwitchPoint

• Publish events across threads (e.g. blow caches when classes change)

•Only event is valid -> invalid

•Hooks directly into the HotSpot deoptimizer (no additional branches)

Putting it together

CallSite: where the call is taking place

Invocation example

CallSite: where the call is taking placeSwitchPoint: did the class change?

Invocation example

CallSite: where the call is taking placeSwitchPoint: did the class change?GuardWithTest: is this the type we bound?

Invocation example

CallSite: where the call is taking placeSwitchPoint: did the class change?GuardWithTest: is this the type we bound?Target: invoke the target method

Invocation example


Rebind: lookup new method and rebuild call site- or -

Invocation example


Rebind: lookup new method and rebuild call site- or -

- or -Rebind: lookup potentially changed methods

Invocation example

InDy in the real world

Is it good?To the assembly!

java -XX:+UnlockDiagnosticVMOptions -XX:+PrintAssembly

Enabling ASM output

def foo; 1; enddef invoker; foo; endi = 0while i < 10000 invoker i+=1end

Contrived Ruby code

0x00000001060a1be0: mov %eax,-0x14000(%rsp) 0x00000001060a1be7: push %rbp 0x00000001060a1be8: sub $0x30,%rsp ;*synchronization entry ; - ruby.__dash_e__::method__1$RUBY$invoker@-1 (line 1) 0x00000001060a1bec: mov 0x8(%rcx),%r10d ; implicit exception: dispatches to 0x00000001060a1c55 0x00000001060a1bf0: cmp $0xfb7aedc9,%r10d ; {oop('org/jruby/RubyObject')} 0x00000001060a1bf7: jne 0x00000001060a1c39 0x00000001060a1bf9: mov %rcx,%r10 ;*checkcast ; - org.jruby.runtime.invokedynamic.InvocationLinker::testMetaclass@2 (line 633) ; - java.lang.invoke.MethodHandle::invokeExact@3 ; - java.lang.invoke.MethodHandle::invokeExact@5 ; - java.lang.invoke.MethodHandle::invokeExact@29 ; - ruby.__dash_e__::method__1$RUBY$invoker@3 (line 1) 0x00000001060a1bfc: mov 0x10(%r10),%ebp ;*getfield metaClass ; - org.jruby.RubyBasicObject::getMetaClass@1 (line 520) ; - org.jruby.runtime.invokedynamic.InvocationLinker::testMetaclass@5 (line 633) ; - java.lang.invoke.MethodHandle::invokeExact@3 ; - java.lang.invoke.MethodHandle::invokeExact@5 ; - java.lang.invoke.MethodHandle::invokeExact@29 ; - ruby.__dash_e__::method__1$RUBY$invoker@3 (line 1) 0x00000001060a1c00: cmp $0xfed77602,%ebp ; {oop(a 'org/jruby/MetaClass')} 0x00000001060a1c06: jne 0x00000001060a1c1e ;*if_acmpne ; - org.jruby.runtime.invokedynamic.InvocationLinker::testMetaclass@8 (line 633) ; - java.lang.invoke.MethodHandle::invokeExact@3 ; - java.lang.invoke.MethodHandle::invokeExact@5 ; - java.lang.invoke.MethodHandle::invokeExact@29 ; - ruby.__dash_e__::method__1$RUBY$invoker@3 (line 1) 0x00000001060a1c08: movabs $0x7f6bf4bb0,%rax ;*areturn ; - ruby.__dash_e__::method__0$RUBY$foo@6 (line 1) ; - java.lang.invoke.MethodHandle::invokeExact@6 ; - java.lang.invoke.MethodHandle::invokeExact@31 ; - java.lang.invoke.MethodHandle::invokeExact@29 ; - ruby.__dash_e__::method__1$RUBY$invoker@3 (line 1) ; {oop(a 'org/jruby/RubyFixnum')} 0x00000001060a1c12: add $0x30,%rsp 0x00000001060a1c16: pop %rbp 0x00000001060a1c17: test %eax,-0xec3c1d(%rip) # 0x00000001051de000 ; {poll_return} 0x00000001060a1c1d: retq


Stack juggling


Is “self” a Ruby object?


Same metaclass as before?


Store Fixnum “1” for return


Note: inside the “foo” method


Stack juggling


Safe point check


Done!

Benchmarks

That’s it!

Twitter:@bascule

Celluloid:celluloid.io

Blog:unlimitednovelty.com

Invoke dynamic your api to hotspot

Technology

Transcript of Invoke dynamic your api to hotspot