Groovy en 2014 et au-del

Martin SkarsauneJava Developer and Co-Owner

A peek into the OpenJDK compiler : goto java;

Were Hiring!

@MSkarsauneHello everyone.My name is Martin Skarsaune, developer working for Kantega.

Today we are going to have a peek into the compiler. I assume everyone here rely on a Java compiler in their everyday work. Just out of curiosity, have anyone had a look at the compiler source code ?I guess a lot of clever developers out here have read articles, have debugged a lot of Java applications and disassempled class files to see what your Java code turns into on the other end. So some of the stuff may be known to you, never the less I think it is worth while to look at the process step by step from the inside and see with out own eyes what goes on.1

GOTO Statements in Java!

@MSkarsauneGOTO Statement ObjectiveSyntax goto identifier;RuntimeProgram control moves to target statementOther Semantics Target (statement label) must be defined in same scope, compilation error if notPotential circular gotos should give compilation warning.

@MSkarsauneGOTO Statement MeansOpenJDKOpen source Java implementationJavac is implemented in plain Java Modify compiler to support GOTO

@MSkarsaune public class GotoSuccess {public static void main(String[] args) {one: System.out.print("goto ");two: System.out.print(J");goto four;three: System.out.print(2017");goto five;four: System.out.print(Fokus ");goto three;five: System.out.print("!");}}

Success Case

@MSkarsaune

GOTOAHEAD

Dijkstra 1968:

@MSkarsaune

6

What is a Compiler?

Compiler

@MSkarsauneA compiler is a tool that accepts input in some sort of programming language and produces output targeted for another language or instruction set. 7

What is a Compiler?

frontendbackend

@MSkarsauneA common compiler architecture is to try to decouple the input from the output. The front end deals with specifics handling, checks and optimizations for the given programming language, and the back end deals with generation, dependencies and optimizations for a given target platform. This separation is an obvious choice for compilers that generate native code.I was involved in a big project over several years, where we developed tools and routines for automated translation from Smalltalk to Java. In that project we did exactly this, we reused the parser and internal representation from the built in Smalltalk compiler, but instead of generating Smalltalk byte codes, we generated Java source code.

8

processAnnotations(enterTrees( parseFiles())),)generate(desugar(flow(attribute())))

@MSkarsauneprocessAnnotations(enterTrees( parseFiles())),)generate(desugar(flow(attribute())))

@MSkarsauneSyntax: goto identifier;First convert character stream to token stream (Scanner.java)

[g][o][t][o][ ][f][o][u][r][;]

@MSkarsaunegoto is already a reserved word in Java!Lucky for us, goto is therefore defined as a TokenKind. Tokens.java:141: GOTO(goto)The scanner therefore works out of the box!

@MSkarsaune

...import com.sun.tools.javac.parser.Scanner;...final Scanner scanner=factory.newScanner(goto identifier;, false);

@MSkarsaune

final Scanner scanner=factory.newScanner(goto identifier;, false);

scanner.nextToken();

@MSkarsaune

final Scanner scanner=factory.newScanner(goto identifier;, false);

scanner.nextToken();assertThat(First token is GOTO, scanner.token().kind, is(GOTO));

@MSkarsaune

final Scanner scanner=factory.newScanner(goto identifier;, false);

scanner.nextToken();assertThat(First token is GOTO, scanner.token().kind, is(GOTO));

scanner.nextToken();

@MSkarsaune

final Scanner scanner=factory.newScanner(goto identifier;, false);

scanner.nextToken();assertThat(First token is GOTO, scanner.token().kind, is(GOTO));

scanner.nextToken();assertThat("Second token is IDENTIFIER", scanner.token().kind,is(IDENTIFIER));

@MSkarsaune

final Scanner scanner=factory.newScanner(goto identifier;, false);

scanner.nextToken();assertThat(First token is GOTO, scanner.token().kind, is(GOTO));

scanner.nextToken();assertThat("Second token is IDENTIFIER", scanner.token().kind,is(IDENTIFIER));

scanner.nextToken();

@MSkarsaune

final Scanner scanner=factory.newScanner(goto identifier;, false);

scanner.nextToken();assertThat(First token is GOTO, scanner.token().kind, is(GOTO));

scanner.nextToken();assertThat("Second token is IDENTIFIER", scanner.token().kind,is(IDENTIFIER));

scanner.nextToken(); assertThat("Third token is SEMI", scanner.token().kind, is(SEMI));

@MSkarsaune

visitClassDef(..)visitMethodDef(..)visitIf(..)Abstract Syntax Tree[g][o][t][o][ ][f][o][u][r][;]Wikipedia: the visitor design pattern is a way of separating an algorithm from an object structure on which it operates

@MSkarsaune

ClassInterface

@MSkarsaune

Interface based visitors

@MSkarsauneClass based visitors

public void visitGoto(JCGoto tree) { try { print("goto " + tree.label + ";"); } catch (IOException e) { throw new UncheckedIOException(e); } }

public void visitGoto(JCGoto tree) { //TODOimplement}

@MSkarsaune

public static class JCLabeledStatement extends JCStatement implements LabeledStatementTree {public GotoResolver handler;}

public class GotoResolver { Map gotos; Map targets; List statementsInSequence; ...}Helper object

@MSkarsaune

JavacParser.parseStatement() case GOTO: { nextToken(); Name label = ident(); JCGoto t = to(F.at(pos).Goto(label, getGotoResolver()); accept(SEMI); return t; }

TreeMaker.java:

public JCGoto Goto(Name label, GotoResolver resolver) { JCGoto tree = new JCGoto(label, resolver); tree.pos = pos; return tree;}

@MSkarsaune

Demo

@MSkarsauneprocessAnnotations(enterTrees( parseFiles())),)generate(desugar(flow(attribute())))

@MSkarsauneprocessAnnotations(enterTrees( parseFiles())),)generate(desugar(flow(attribute())))

@MSkarsauneBasic sanity testing of compilation unit:File name and folder locationDuplicate class namesCorrectionsAdd default constructor if no constructors are declared

@MSkarsaune

Default Constructorpublic class SimpleClass {

}

@MSkarsaune

Default Constructorpublic class SimpleClass { public SimpleClass() { super(); }}

@MSkarsaune

processAnnotations(enterTrees( parseFiles())),)generate(desugar(flow(attribute())))

@MSkarsauneprocessAnnotations(enterTrees( parseFiles())),)generate(desugar(flow(attribute())))

@MSkarsauneProcess AnnitationsAnnotation processing APIPart of ordinary javac process since Java 1.6Plugin API (see javac documentation)

@MSkarsauneOutput controlled by command line switches

@- proc:only only process annotations, do not compile

- proc:none do not process annotations

@MSkarsauneprocessAnnotations(enterTrees( parseFiles())),)generate(desugar(flow(attribute())))

@MSkarsauneprocessAnnotations(enterTrees( parseFiles())),)generate(desugar(flow(attribute())))

@MSkarsauneAttributionSemantic checksTypesReferencesCorrectionsAdd required calls to super constructor

@MSkarsaune

Ensure target label exists in current scopepublic class GotoMissingLabel {public static void main(String[] args) {one: System.out.print("goto ");two: System.out.print(Java");goto six;three: System.out.print(2016");goto five;four: System.out.print(One ");goto three;five: System.out.print("!");}}

@MSkarsaune

Attr.java: @Override public void visitGoto(JCGoto that) {that.findTarget();if(that.target==null)log.error(that.pos(), "undef.label", that.label);result = null; }

class JCGoto: public void findTarget() { this.target = (JCLabeledStatement)this.handler.findTarget(this);}

@MSkarsaune

Demo

@MSkarsauneprocessAnnotations(enterTrees( parseFiles())),)generate(desugar(flow(attribute())))

@MSkarsaune

processAnnotations(enterTrees( parseFiles())),)generate(desugar(flow(attribute())))

@MSkarsauneFlow analysisDetect unreachable codeVerify assignmentsEnsure proper method returnVerify (effectively) finalCheck exception flow

@MSkarsaune

Detect circular gotospublic class GotoCircularWarning {public static void main(String[] args) { one: System.out.print("goto "); two: System.out.print(Java"); goto four; three: System.out.print(2016"); goto five; four: System.out.print(One "); goto three; five: System.out.println("!"); goto one;//forms infinite loop }}

@MSkarsaune

Flow.FlowAnalyzer class:

compiler.properties: @Override public void visitGoto(JCGoto tree) { if (tree.handler.detectCircularGotoPosition(tree)) log.warning(tree.pos, "circular.goto"); }

...compiler.warn.circular.goto=circular goto...

@MSkarsaune

Demo

@MSkarsaune

processAnnotations(enterTrees( parseFiles())),)generate(desugar(flow(attribute())))

@MSkarsauneprocessAnnotations(enterTrees( parseFiles())),)generate(desugar(flow(attribute())))

@MSkarsaune

Erase generic types

public class Bridge implements Comparator {

}public interface Comparator {int compare(T o1, T o1); }or {

@MSkarsaune

Erase generic types

public class Bridge implements Comparator {public int compare(Integer first, Integer second) {return first - second;}}public interface Comparator {int compare(T o1, T o1)