Post on 27-Apr-2020
m2e, an exercise in mixing oil and water
Key differences between Eclipse and Maven and what they mean for m2e
developers and users
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
PresentaIon overview
• m2e goals and approach • Look under the hood – Project import and configuraIon – Dependency resoluIon – Workspace build
• Q&A
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Prereqs
• This is an “advanced” session • Good m2e user experience is expected – m2e extension development experience is a plus – Solid understanding how to use Maven
• Maven plugin development experience is a plus • Understanding of Eclipse plugin development concepts – OSGi framework, bundles and classloading model – Workspace and incremental builders
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
m2e goals and approach
• Map Maven project configuraIon to Eclipse workspace
• Full support for incremental workspace build • Framework for extension execuIon, not a monolith – Integrate with “naIve” Eclipse tools, like jdt, wtp, ajdt, adt and so on
– Delegate to Maven plugins where desirable and feasible
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
m2e core features • Project registry – m2e maintains registry of all workspace Maven projects and their dependencies
– Highly opImized for speed, not for RAM – Workspace dependency resoluIon – AutomaIcally recalculates dependencies when Maven projects change
– Maven project change event listener • Build lifecycle mapping and extensions discovery • Project configurators and build parIcipants extension points
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
… and much more
• Full-‐blown JDT integraIon • Rich pom.xml editor with awesome Dependency Hierarchy view
• Extensive set of extension points • m2e extension discovery catalog with quick fixes and project import integraIon
• Maven launch configuraIons
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
LOOK UNDER THE HOOD Things that work and not so much
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Eclipse vs Maven
• Eclipse JVM is a long-‐lived process l JVM resources leaks become more prominent
l Eclipse workspace builds run aaer each resource l NoIon of ‘incremental’ build
l Eclipse workspace and underlying filesystem must be kept in sync – Elaborate resource change delta tracking
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
plugin goal execuIon
plugin goal execuIon
plugin goal execuIon
plugin goal execuIon
plugin goal execuIon
build lifecycle execuIon
Maven project build lifecycle • Build lifecycle is a sequence of plugin goal execuIons – Calculated upfront based on pom.xml
– Executed from start to finish • Plugin goal execuIon is the most basic unit of work in Maven – Executable goal, a.k.a “mojo”, and corresponding configuraIon
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
file system
Maven Project model
plugin goal execuIon
plugin goal execuIon
plugin goal execuIon
plugin goal execuIon
plugin goal execuIon
Mojo execuIon • A mojo is a Maven plain Old Java Object – Opaque to Maven core – Can access Maven Session, Project Model and filesystem
• Instances are discarded aaer execuIon
• There is no mechanism for direct mojo-‐to-‐mojo communicaIon
• No standard mechanism to track changes between builds
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Workspace project configuraIon • Maven project build lifecycle fully defines what happens
during project commend line build – No general mechanism to “know” what a Maven plugin goal does
• m2e provides metadata-‐driven build lifecycle mapping – Determine if project is compaIble with Eclipse installaIon – Discovery of addiIonal plugins required to work on the project
• Each plugin goal execuIon is mapped to <ignore>, <configurator> or <execute> – <configurator> allows Ight integraIon with other Eclipse tools
• m2e detects changes in project build lifecycle – … but workspace configuraIon update must be triggered manually
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Lifecycle mapping metadata sources
• pom.xml file of the project • parent, grand-‐parent and so on pom.xml files • [m2e 1.2+] workspace preferences • installed m2e extensions (in no parIcular order) • [m2e 1.1+] lifecycle mapping metadata provided by maven plugin
• default lifecycle mapping metadata shipped with m2e
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
m2e extension discovery
• Catalog of available m2e extensions – Uses P2 Eclipse Marketplace code, not necessary a good thing
• Includes m2e lifecycle mapping metadata to facilitate discovery
• Separate catalogs for m2e 1.0 and 1.1/1.2 – Due to P2 limitaIons
• Integrated in Maven project import wizard and mapping errors quick-‐fixes.
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
plugin goal execuIon
plugin goal execuIon
plugin goal execuIon
plugin goal execuIon
plugin goal execuIon
build lifecycle execuIon
<configurator> <configurator>
<ignore>
<execute>
<configurator>
<ignore>
m2e project configuraIon and build
m2e build lifecycle mapping
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Lifecycle mapping acIons • Lifecycle mapping affects both workspace project configuraIon and build
• <ignore> tells m2e to completely ignore the plugin goal execuIon
• <configurator> tells m2e to delegate to an extension – Can parIcipate in project configuraIon and/or workspace build
• <execute> tells m2e to execute the goal – During incremental or full workspace build – During project configuraIon
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
<execute> on configuraIon • Goals contribute addiIonal project model elements – Source roots – ProperIes
• Other goals or configurators require the model elements to be present – No mechanism to configure projects directly
• Must not generate any resources or honour empty BuildContext
<configurator>
<ignore>
<execute>
<execute>
<ignore>
Maven Project model
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Dependency resoluIon • Project dependencies are calculated from project pom.xml – No good support for this in Eclipse – m2e has two separate code paths for auto-‐build on and off
• m2e tracks dependencies of all workspace projects • When projects change, dependencies are recalculated and listeners are noIfied
• In Maven, dependencies are always resolved to packaged arIfacts
• In m2e, workspace dependencies are resolved to projects output folders – Some Maven plugins are not compaIble
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Akached arIfacts
• Maven project can produce mulIple arIfacts, eg., main, test, ejb-‐client, sources, etc – No arIfact contents informaIon in Maven project metadata
– m2e can guess content of main and test jar, but not other arIfacts
• JDT Java project dependency includes all output folders of the target project – Impossible to depend on just main or just test aspects of other workspace projects
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Dependency scopes
• Maven disInguishes main and test dependencies – EffecIvely each project has two classpaths
• JDT Java has single project classpath – Main classes are compiled against test classpath – JDT team proposed splilng projects in two, which would result in even harder to navigate workspace and likely won’t work
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
RunIme classpath and source lookup
• Maven runIme classpath calculaIon rules – No easy way to instrument exisIng launch configuraIons with custom classpath providers
• m2e hijacks JDT Java ApplicaIon and Junit launch configuraIon – ILaunchConfiguraIonListener injects m2e classpath providers, not preky but works
• No easy way to enable for other launch configuraIon types
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Dynamic source code lookup • ApplicaIons oaen load code not listed on iniIal JVM
classpath – OSGi bundles are loaded by framework – WARs are loaded by web container, like Jeky – Maven plugins are loaded by Maven core
• It is oaen possible to locate sources of arIfacts originaIng from Maven repositories – Embedded pom.xml – Match jar SHA1/MD5 to Maven repository index
• Generic javaagent/jsr45/jdwp soluIon possible, but… – No way to plug custom ISourceLocator to exisIng launch configuraIon
– JDT ignores custom jsr45 strata if default ‘java’ is present
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Resource change events • Eclipse workspace tracks state of all resources • Resource change event for all(!) changes
– Pre/post create, update, remove, markers, team, phantom… – Grouping of event delivery is not guaranteed
• Resource change listener – Workspace changes “may be disallowed”
• Incremental workspace builder – More elaborate, long running resource change listener – Workspace changes are allowed and other builders are noIfied
• Both listener and builder are hard to implement correctly
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Workspace build mapping
compiler :compile
antlr3 :antlr
resources :resources
resources :testResources
compiler :testCompile
build lifecycle execuIon Maven Build ParIcipant
Maven Build ParIcipant
Maven Build ParIcipant
Java Builder
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Workspace build • Different granularity of Maven plugin goals and Eclipse builders – Can’t order workspace compile/testCompile properly
• Single change triggers mulIple builder execuIon – All builders must keep workspace and filesystem in sync
– Required to propagate changes from Maven builder to Java and other Eclipse builders
• Resource deltas are not communicated among Maven build parIcipants – This is a bug in m2e
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Problem: endless build
l Aaer making a change to a file, workspace build runs forever
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Endless workspace build (1)
save
Best code change ever!
Workspace
codegen builder
jdt builder
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Endless workspace build (2)
save
Best code change ever!
Workspace
codegen builder
jdt builder
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Endless workspace build (3)
Workspace
codegen builder
jdt builder
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Endless workspace build (4 == 1)
Workspace
codegen builder
jdt builder
???
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Problem: missing generated resources
• Resources “disappear” from project output folder aaer git-‐pull or similar mulI-‐resource change
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Missing generated resources (1)
git-‐pull
Filesystem
resgen builder
jdt builder Workspace sync
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Missing generated resources (2)
git-‐pull
Filesystem
jdt builder Workspace no sync
change'
resgen builder
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Missing generated resources (3)
git-‐pull
Filesystem
resgen builder
jdt builder Workspace no sync
rebuild
???
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
m2e compaIble Maven plugins • Builders MUST ignore unrelated changes • Builders MUST keep workspace and filesystem in sync • Most mojos are incompaIble unless explicitly coded to support m2e workspace
• BuildContext API – Available since m2e 1.0 – Provides mojos access to resource delta, workspace refresh and error/warning markers
– Number of drawbacks and limitaIons • Build avoidance API – Significant rework – Planned but no ETA
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Wrapper project configurator
• Pseudo code if (mojo input changed) {! execute mojo;! refresh mojo output;!}!
• Must know details of the mojo configuraIon parameters and output – In many cases no easy way to determine
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Nested project support (lack of) Maven project nested modules Eclipse flat workspace
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Flat workspace • Hard to navigate workspaces, especially large – Modules are grouped together for a reason, flat workspace works against that reason
• Same/many resources appear in mulIple places – Confuses team providers and resources lookup dialogs
• Switching between git branches – Import/delete workspace project aaer each switch
• Cross-‐module resource references • Modules can have same names – Special requirement for workspace root projects is parIcularly annoying
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
Conclusions
• Maven and Eclipse provide significantly different runIme environments
• Fundamental Eclipse limitaIons – Lack of nested project support – Lack of classpath scope support
• Workspace build parIcipaIon is hard to implement correctly – Many mojos will require changes
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
We welcome quality patches
• m2e development environment setup instrucIons hkp://wiki.eclipse.org/M2E_Development_Environment
• m2e-‐dev mailing list hkps://dev.eclipse.org/mailman/lisInfo/m2e-‐dev
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
m2e overview
• From project homepage “The goal of the m2e project is to provide a first-‐class Apache Maven support in the Eclipse IDE”
• Tight integraIon with Eclipse workspace and tooling – Full support for incremental workspace build – IntegraIon with naIve Eclipse tools
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0
m2e history • m2e started as “m2eclipse” at codehaus in 2005 • Moved to Sonatype SVN in 2008 and to Github in 2010
• Proposed as Eclipse project in 2008 but actual move was delayed due to m2e dependency on prerelease version of Maven 3.0 (released on 2010-‐10-‐08)
• 3 releases since m2e moved to Eclipse • #1 most popular download on Eclipse Marketplace as of 2012-‐09-‐22
Copyright(c) 2012 Igor Fedorenko. Licensed under EPL v 1.0