Download - Goto devoxx

@MSkarsaune#GotoJava

goto java;- Java compilation under the hood

Martin [email protected]

Kantega – Norwegian software craftsmanship since 2003

高馬丁

GOTO Statements in Java!


GOTO Statement –Objective

• Syntax goto identifier;• Runtime• Program control moves to target statement

• Other Semantics • Target (statement label) must be defined in

same scope, compilation error if not• Potential circular gotos should give

compilation warning.


GOTO Statement –Means

• OpenJDK• Open source Java implementation• Javac is implemented in plain Java

• Modify compiler to support GOTO


public class GotoSuccess {

public static void main(String[] args) {one: System.out.print("goto ");two: System.out.print(”DEV");goto four;three: System.out.print(”UK");goto five;four: System.out.print(”OXX ");goto three;five: System.out.print("!");

}}

Success Case


GOTOAHEAD

Dijkstra 1968:


What is a Compiler?

Compiler


What is a Compiler?

frontend

backend


Parse Enter Process Attribute Flow Desuga

rGenerat

e

processAnnotations(enterTrees(… parseFiles(…))),…)

…generate(desugar(flow(attribute(…))))



rGenerat

e

• Syntax: goto identifier;• First convert character stream to token

stream (Scanner.java)

GOTO IDENTIFIER SEMI

[g][o][t][o][ ][f][o][u][r][;]



rGenerat

e

• goto is already a reserved word in Java!• Lucky for us, goto is therefore defined as a

TokenKind. • Tokens.java:141: GOTO("goto")• The scanner therefore works out of the box!


final Scanner scanner=factory.newScanner(”goto identifier;”, false);


rGenerat

e

...import com.sun.tools.javac.parser.Scanner;...



scanner.nextToken();


rGenerat

e



scanner.nextToken();assertThat(”First token is GOTO”, scanner.token().kind, is(GOTO));


rGenerat

e






rGenerat

e




scanner.nextToken();assertThat("Second token is IDENTIFIER", scanner.token().kind,

is(IDENTIFIER));


rGenerat

e





is(IDENTIFIER));



rGenerat

e





is(IDENTIFIER));

scanner.nextToken(); assertThat("Third token is SEMI", scanner.token().kind, is(SEMI));


rGenerat

e


CompilationUnit

ClassDefinition

FieldDefintion

MethodDefinition

Signature

Body

IfStatement

Goto


rGenerat

e

visitClassDef(..)

visitMethodDef(..)

visitIf(..)

Abstract Syntax Tree

[g][o][t][o][ ][f][o][u][r][;]

GOTO IDENTIFIER SEMIWikipedia: “the visitor design pattern is a way of separating an algorithm from an

object structure on which it operates”



rGenerat

e

ClassInterface



rGenerat

e

Interface based visitors



rGenerat

e

Class based visitors

public void visitGoto(JCGoto tree) { try {

print("goto " + tree.label + ";"); } catch (IOException e) {

throw new UncheckedIOException(e); }

}

public void visitGoto(JCGoto tree) { //TODO implement

}



rGenerat

e

public static class JCLabeledStatement extends JCStatement implements LabeledStatementTree {…public GotoResolver handler;…}

public class GotoResolver { Map<GotoTree, Name> gotos; Map<Name, LabeledStatementTree> targets; List<StatementTree> statementsInSequence; ...}

Helper object



rGenerat

e

JavacParser.parseStatement()

case GOTO: { nextToken(); Name label = ident(); JCGoto t = to(F.at(pos).Goto(label, getGotoResolver())); accept(SEMI); return t; }

TreeMaker.java:

public JCGoto Goto(Name label, GotoResolver resolver) { JCGoto tree = new JCGoto(label, resolver); tree.pos = pos; return tree;}

@YourTwitterHandle@MSkarsaune#GotoJava

Dem

o



rGenerat

e



✔



rGenerat

e





rGenerat

e

• Basic sanity testing of compilation unit

• File name and folder location

• Duplicate class names• Corrections• Add default constructor if

no constructors are declared



rGenerat

e

Default Constructorpublic class SimpleClass {

}



rGenerat

e

Default Constructorpublic class SimpleClass { public SimpleClass() { super(); }}



rGenerat

e



✔



rGenerat

e





rGenerat

e

• Process Annitations• Annotation processing API• Part of ordinary javac process since Java 1.6

• Plugin API (see javac documentation)



rGenerat

e

• Output controlled by command line switches

@

- proc:only – only process annotations, do not compile

- proc:none – do not process annotations



rGenerat

e

Annotation processor Example• ensure that all fields with @Config

annotation are primitive or strings

@Configprivate String sender;//ok @Configprivate int retries;//ok @Configprivate String[] receivers;//not ok



rGenerat

e

META-INF/services/javax.annotation.processing.Processor:

org.kantega.reststop.apt.ConfigFieldProcessor



rGenerat

e

ConfigFieldProcessor.java :

@SupportedAnnotationTypes("org.kantega.reststop.api.Config")@SupportedSourceVersion(SourceVersion.RELEASE_7)public class ConfigFieldProcessor extends AbstractProcessor { //... public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) {

//... if(! isPrimitive(type) && ! isString(type) ) {

processingEnv.getMessager().printMessage(Diagnostic.Kind.ERROR, "@Config annotated field must primitive, boxed or String”, element)



rGenerat

e

Simply add to compilation path:

<dependency> <groupId>org.kantega.reststop</groupId> <artifactId>reststop-annotation-processor</artifactId> <version>${reststop.version}</version> <scope>compile</scope></dependency>



rGenerat

e

• Devoxx BE 2014 : “Plugging into the Java Compiler” , •https://www.parleys.com/tutorial/plugging-java-compiler-1

•Google the session title for additional resources.

https://www.parleys.com/tutorial/plugging-java-compiler-1






rGenerat

e



✔



rGenerat

e





rGenerat

e

• Attribution• Semantic checks

–Types–References

• Corrections–Add required calls to super constructor



rGenerat

e

Ensure target label exists in current scopepublic class GotoMissingLabel {

public static void main(String[] args) {one: System.out.print("goto ");two: System.out.print(”DEV");goto six;three: System.out.print(”UK");goto five;four: System.out.print(”OXX ");goto three;five: System.out.print("!");

}}



rGenerat

e

Attr.java: @Override public void visitGoto(JCGoto that) {

that.findTarget();if(that.target==null)

log.error(that.pos(), "undef.label", that.label);result = null;

}

class JCGoto: …public void findTarget() { this.target = (JCLabeledStatement)this.handler.findTarget(this);}


Dem

o



rGenerat

e



✔



rGenerat

e





rGenerat

e

Flow analysis• Detect unreachable code• Verify assignments• Ensure proper method return• Verify (effectively) final• Check exception flow



rGenerat

e

Detect circular gotospublic class GotoCircularWarning {

public static void main(String[] args) { one: System.out.print("goto "); two: System.out.print(”DEV"); goto four; three: System.out.print(”UK"); goto five; four: System.out.print(”OXX "); goto three; five: System.out.println("!"); goto one;//forms infinite loop }}



rGenerat

e

Flow.FlowAnalyzer class:

compiler.properties:

@Override public void visitGoto(JCGoto tree) { if (tree.handler.detectCircularGotoPosition(tree))

log.warning(tree.pos, "circular.goto"); }

...compiler.warn.circular.goto=circular goto...


Dem

o



rGenerat

e



✔



rGenerat

e





rGenerat

e

TransTypes Unlambda LowerErase generic types

public class Bridge implements Comparator {

}

public interface Comparator<T> {tint compare(T o1, T o2);

}or<T> {



rGenerat

e

TransTypes Unlambda LowerErase generic types

public class Bridge implements Comparator<Integer> {

public int compare(Integer first, Integer second) {return first - second;

}

}

public interface Comparator<T> {tint compare(T o1, T o2);

}or<T> {



rGenerat

e

TransTypes Unlambda LowerErasure - Runtime



}

}



rGenerat

e

TransTypes Unlambda LowerErasure - Bridge



} /*synthetic*/ public int compare(Object first, Object second) { return this.compare((Integer)first, (Integer)second); }

}



rGenerat

e

TransTypes Unlambda LowerExtract inner class

public class Outer {private void foo() { }

public Runnable fooRunner() {return new Runnable() {

public void run() {foo();

}};

}}



rGenerat

e



public Runnable fooRunner() {return new Outer$1(this);

}}

class Outer$1 implements Runnable {final Outer this$0;

Outer$1(final Outer this$0) { this.this$0 = this$0; super(); } public void run() {

this$0.foo(); }}



rGenerat

e



public Runnable fooRunner() {return new Outer$1(this);

}/*synthetic*/static void access$000(

Outer x0) { x0.foo(); }

}

class Outer$1 implements Runnable {final Outer this$0;

Outer$1(final Outer this$0) { this.this$0 = this$0; super(); } public void run() {

Outer.access$000(this$0); }}



rGenerat

e

TransTypes Unlambda LowerBoxing

List<Integer> list = Arrays.asList(1, 2);

for (Integer i : list) {System.out.println("Double: " + i * 2);

}



rGenerat

e


List<Integer> list = Arrays.asList(1, 2);


}

public static <T> List<T> asList(T... a) { return new ArrayList<>(a); }



rGenerat

e


List<Integer> list = Arrays.asList(Integer.valueOf(1), Integer.valueOf(2));


}



rGenerat

e

TransTypes Unlambda LowerUnboxing



}



rGenerat

e

TransTypes Unlambda LowerUnboxing


for (Integer i : list) {System.out.println("Double: " + i.intValue() * 2);

}



rGenerat

e

TransTypes Unlambda LowerVarargs



}

public static <T> List<T> asList(T... a) { return new ArrayList<>(a); }



rGenerat

e

TransTypes Unlambda LowerVarargs



}

public static <T> List<T> asList(T... a) { return new ArrayList<>(a);}



rGenerat

e

TransTypes Unlambda LowerVarargs - runtime

List<Integer> list = Arrays.asList(new Integer[]{Integer.valueOf(1), Integer.valueOf(2)});


}



rGenerat

e

TransTypes Unlambda LowerFor each loop



}



rGenerat

e




}

public interface Iterable<T> { Iterator<T> iterator();}



rGenerat

e



for (;;) {Integer i System.out.println("Double: " + i.intValue() * 2);

}



rGenerat

e



for (Iterator i$ = list.iterator();;) {Integer i System.out.println("Double: " + i.intValue() * 2);

}



rGenerat

e



for (Iterator i$ = list.iterator(); i$.hasNext();) {Integer i System.out.println("Double: " + i.intValue() * 2);

}



rGenerat

e



for (Iterator i$ = list.iterator(); i$.hasNext();) {Integer i = (Integer)i$.next(); System.out.println("Double: " + i.intValue() * 2);

}



rGenerat

e

TransTypes Unlambda LowerEnums

public enum Status {YES, NO, MAYBE

}



rGenerat

e

TransTypes Unlambda LowerEnums - constructor

public enum Status {YES, NO, MAYBEprivate Status(String $enum$name, int $enum$ordinal) {

super($enum$name, $enum$ordinal); }}

public static final Status TRUE = new Status("TRUE", 0);public static final Status FALSE = new Status("FALSE", 1);public static final Status MAYBE = new Status("MAYBE", 2);



rGenerat

e

TransTypes Unlambda LowerEnums - valueOf


super($enum$name, $enum$ordinal); } public static Status valueOf(String name) { return (Status)Enum.valueOf(Status.class, name); }}



rGenerat

e

TransTypes Unlambda LowerEnums - values


super($enum$name, $enum$ordinal); } public static Status valueOf(String name) { return (Status)Enum.valueOf(Status.class, name); } private static final Status[] $VALUES = new Status[]{ Status.YES, Status.NO, Status.MAYBE};

public static Status[] values() { return (Status[])$VALUES.clone(); }}



rGenerat

e

TransTypes Unlambda LowerEnum switch statement

public class SwitchStatus {

void switchStatus(Status status) {

switch (status) { case MAYBE: return;

default: break; } }}



rGenerat

e




switch (SwitchStatus$1.$SwitchMap$Status[(status).ordinal()]) { case 1: return;


class SwitchStatus$1 {

}



rGenerat

e






class SwitchStatus$1 {static final int[] $SwitchMap$Status = new

int[Status.values().length];

}



rGenerat

e







int[Status.values().length]; [0][0][0]

}



rGenerat

e







int[Status.values().length]; [0][0][1]static { try { SwitchStatus$1.$SwitchMap$Status[Status.MAYBE.ordinal()] = 1; } catch (NoSuchFieldError ex) { } }}



rGenerat

e



✔



rGenerat

e





rGenerat

e

• Organize initializers• String concatenation• Generate bytecodes



rGenerat

e

• Organize <init> (Constructor)public class InstanceInitialization { String key="key"; String value; public InstanceInitialization(String value) { this.value = value; }

}



rGenerat

e

• Organize <init> (Constructor)public class InstanceInitialization { String key="key"; String value; public InstanceInitialization(String value) {

super(); this.value = value; }

}



rGenerat

e

• Organize <init> (Constructor)public class InstanceInitialization { String key; String value; public InstanceInitialization(String value) {

super(); key = ”key”; this.value = value; }

}



rGenerat

e

• Organize <init> (Constructor)public class InstanceInitialization { String key; String value; public void <init> () {

super(); key = ”key”; this.value = value; }

}



rGenerat

e

• Organize <clinit> (static initialization)public class StaticInitialization { static String key="key"; static { init(); }}



rGenerat

e

• Organize <clinit> (static initialization)public class StaticInitialization { static String key; static { key="key"; init(); }}



rGenerat

e

• Organize <clinit> (static initialization)public class StaticInitialization { static String key; static void clinit() { key="key"; init(); }}



rGenerat

e

String concatenationSource code “Generated code”

”string” + value



rGenerat

e


”string” + value new StringBuilder()



rGenerat

e


”string” + value new StringBuilder() .append(”string”)



rGenerat

e


”string” + value new StringBuilder() .append(”string”) .append(value)



rGenerat

e


”string” + value new StringBuilder() .append(”string”) .append(value) .toString()



rGenerat

e

• Goto generation–Luckily for us there is a GOTO byte code–goto <addr>



rGenerat

e

if (<test>) { <ifblock>} else { <elseblock>}<codeafter>

Source code Byte codeCI

9 ifeq<elseblock>

goto<stackmap>

22 <ifblock>

22

<stackmap>

29 <codeafter>

29

Java >= 1.6:Stack map frames must be embedded at target of jump instruction

Code generator (Code.java) marked as not alive.Goto instruction added to list of pending jumps (Chain.java)

Pending jumps processed

Normal GOTO usage



rGenerat

e


…label: <somecode>…goto label;

…

20

… goto 20

<stackmap>

<somecode>

Used by goto?Must emit stackmap

Emit goto to label and turn code generation on again

GOTO scenario 1 : jump back



rGenerat

e


GOTO scenario 2 : jump forward

…goto label; …label: <somecode>

……

29 <somecode>

goto

<stackmap>

Label position not yet known?• emit goto• add to list of pending gotos• turn generation on again

29Label used? • emit stack frame • patch pending

gotos



rGenerat

e

• Goto generation–Gen.java , visitor for code generation–Modify for LabelledStatement–Add implementation for Goto



rGenerat

e

• Gen.java – Labelled Statement

public void visitLabelled(JCLabeledStatement tree) { // if the label is used from gotos, have to emit stack map if (tree.handler.isUsed(tree)) code.emitStackMap(); … }



rGenerat

e

• Gen.java – Goto

public void visitGoto(JCGoto tree) { tree.handler.addBranch(new Chain(code.emitJump(goto_), null, code.state.dup()), tree.target); //normally goto marks code as not alive, turn generation on code.entryPoint();}

Target position known?• Yes – patch immediately• No – add to list of pending

gotos


Dem

o


ambdaλ



rGenerat

e

TransTypes Unlambda Lower

• Lambda implementation in Java 8–Language change–Compilation–Runtime support

• Many interesting design considerations



rGenerat

e

TransTypes Unlambda LowerSimple Example

public Comparator<String> lambdaExample() { return (String a, String b) -> a.compareTo(b);}



rGenerat

e

TransTypes Unlambda LowerLambdaToMethod.java

public Comparator<String> lambdaExample() { return (String a, String b) -> a.compareTo(b);}

/*synthetic*/ private static int lambda$lambdaExample$0( , ) { return ;}

final String afinal String b

a.compareTo(b)



rGenerat

e

TransTypes Unlambda LowerRuntime

public Comparator<String> lambdaExample() { return <invokedynamic>LambdaMetafactory.metafactory(}

/*synthetic*/ private static int lambda$lambdaExample$0( final String a, final String b) { return a.compareTo(b);}

public interface Comparator {/*erased*/ int compare(Object o1, Object o2);}

Lookup(LambdaExample), /*caller*/ "compare", ()Comparator, /*MethodType*/ (Object,Object)int, /*MethodType*/ lambda$lambdaExample$0, /*MethodHandle*/ (String,String)int); /*MethodType*/

final class LambdaExample$$Lambda$1/1834188994 implements Comparator { private LambdaExample$$Lambda$1/1834188994() public int compare(Object,Object)}



rGenerat

e


Runtime implementation

LambdaMetaFactory• metaFactory(…)• altMetaFactory(…)

InnerClassLambdaMetafactory

ASM

Lambda Class



rGenerat

e


SerializationLambda Instance

Serialize

SerializedLambda

Deserialize

LambdaMeta

Factory



rGenerat

e


• Possible to back port ?–Capture generated class ?–Compile time generation of inner class!



rGenerat

e

TransTypes Unlambda LowerStep 1: Source.java

public boolean allowLambda() { return compareTo( ) >= 0;}

JDK1_8JDK1_5



rGenerat

e

TransTypes Unlambda LowerStep 2: Special handling

boolean mustBackportLambda() { return this.target.compareTo(Target.JDK1_8) < 0;}



rGenerat

e

TransTypes Unlambda LowerStep 3: Call backport

if(!this.attr.mustBackportLambda()) { result = makeMetafactoryIndyCall(...);}else{ result = new LambdaBackPorter(...).implementLambdaClass(...);}



rGenerat

e

TransTypes Unlambda LowerExample implementation

private static final class Lambda$$2 implements Comparator<String> {

Lambda$$2 { super(); } public int compare(String arg0, String arg1) { return LambdaExample.lambda$lambdaExample$0(arg0, arg1); } public int compare(Object o1, Object o2) { return this.compare((String)o1, (String)o2); }}



rGenerat

e

TransTypes Unlambda LowerExample invoking

public Comparator<String> lambdaExample() { }

return LambdaMetafactory.metafactory(...);return new Lambda$$289();


ambdaλ

@YourTwitterHandle#DVXFR14{session hashtag} @MSkarsaune#GotoJava

Sum

mar

y


Wrap Up

• The Java compiler is written in Java• Compilation is done in phases• Programming language advances (syntactic

sugar) require good compiler support• Lambdas are compiled in a forward compatible

manner


Resources• OpenJDK

• Source: http://hg.openjdk.java.net/• Compiler hacking tutorial

• http://www.ahristov.com/tutorial/java-compiler.html• Lambda design discussion

• http://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html

• Sample code:• https://github.com/skarsaune/goto

• Slide pack:• http://www.slideshare.net/MartinSkarsaune

http://hg.openjdk.java.net/

http://www.ahristov.com/tutorial/java-compiler.html

http://www.ahristov.com/tutorial/java-compiler.html

http://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html



https://github.com/skarsaune/goto

https://github.com/skarsaune/goto

http://www.slideshare.net/MartinSkarsaune

http://www.slideshare.net/MartinSkarsaune


Q &

A

https://wall-simple.sli.do/#/event/cmnxxfl0/section/18316/questions

https://wall-simple.sli.do/%23/event/cmnxxfl0/section/18316/questions

https://wall-simple.sli.do/%23/event/cmnxxfl0/section/18316/questions


Than

k yo

u

for yo

ur

time!

http://lanyrd.com/profile/martinskarsaune/