www.eu-etics.org
ETICS All Hands meeting ETICS All Hands meeting BBolognaologna, , OctoberOctober 2 233--2525, 2006, 2006
WP4 Test and Metrics Plugin WP4 Test and Metrics Plugin FrameworkFramework
(WP(WP44))
Eva TAKACSEva TAKACS
2
Overview
• Why a plugin frameworkWhy a plugin framework• Advantages of a Plugin Framework in generalAdvantages of a Plugin Framework in general• Core functionality and first steps of the Test and Core functionality and first steps of the Test and
Metrics Plugin Framework Metrics Plugin Framework • Basic architecture of a plugin framework in generalBasic architecture of a plugin framework in general• Java Plugin Framework – implementation of a plugin Java Plugin Framework – implementation of a plugin
architecturearchitecture• User scenariosUser scenarios• Basic architecture of the Test and Metrics Plugin Basic architecture of the Test and Metrics Plugin
FrameworkFramework• ConclusionConclusion
3
Advantages of a plugin framework in general
Making an application extensible using the Making an application extensible using the plugin system has strong advantages over plugin system has strong advantages over other techniques:other techniques:
• It allows third-party developers to easily enhance It allows third-party developers to easily enhance software in a way that is loosely coupled: only the software in a way that is loosely coupled: only the plugin API is required to remain stable.plugin API is required to remain stable.
• It allows new plugins to be discovered flexibly. It allows new plugins to be discovered flexibly. More precisely,More precisely,
– plugins can be loaded and saved when a program is first run, or plugins can be loaded and saved when a program is first run, or re-discovered each time the program starts upre-discovered each time the program starts up
– allows the discovery of new plugins installed after the program allows the discovery of new plugins installed after the program has started.has started.
4
Why a Plugin Framework for ETICS?
The rational behind using „Eclipse style” plug-in architecture isThe rational behind using „Eclipse style” plug-in architecture is
• that mthat most of ost of present and/or future ETICSpresent and/or future ETICS users will with some users will with some tests and testing tools already in placetests and testing tools already in place and of course they would and of course they would like to use them when working with ETICSlike to use them when working with ETICS
On the other hand On the other hand
• they should be provided with „default” testing tools when they do they should be provided with „default” testing tools when they do not have their own but would like to use something not have their own but would like to use something
5
Core functionality of the Test and Metrics Plugin Framework
Core functionality of the framework is consisted ofCore functionality of the framework is consisted of
• Existing tools (default plug-ins)Existing tools (default plug-ins)– TestManager – a testdriver used for gLite testing already in useTestManager – a testdriver used for gLite testing already in use– Two coverage tools for Java testcasesTwo coverage tools for Java testcases
– EmmaEmma– JavaCovJavaCov– other (Cobertura ?)other (Cobertura ?)
• A plugin manager system able to handle the plugins and user A plugin manager system able to handle the plugins and user profile settings from the web interface and/or command-line profile settings from the web interface and/or command-line interfaceinterface
6
Plugin Framework – First steps 1
It was decided that as a first stepIt was decided that as a first step
– TestManager a testdriver developed in CERN and used for gLite TestManager a testdriver developed in CERN and used for gLite functional testingfunctional testing
– Two coverage tools Two coverage tools – EMMA, an open source tool requested by future ETICS usersEMMA, an open source tool requested by future ETICS users– JavaCov, a commercial tool providing a very strong MC/DC JavaCov, a commercial tool providing a very strong MC/DC
coverage criterion coverage criterion
will be taken into account. will be taken into account.
7
Plugin Framework – First steps 2
To examine how these tree tools can be integrated in a To examine how these tree tools can be integrated in a future plugin environmentfuture plugin environment
• to „modularise” TestManagerto „modularise” TestManager• to have a closer look to the running ways of coverage to have a closer look to the running ways of coverage
tools tools
8
EMMA
EMMA – is an open source coverage toolEMMA – is an open source coverage tool• Supported coverage types Supported coverage types
– class class – method method – line line – basic block (branch)basic block (branch)
• Supported Java version: 1.5.*Supported Java version: 1.5.*• Running way:Running way:
– configurable command line configurable command line – ANT integrationANT integration– no Eclipse pluginno Eclipse plugin
9
EMMA - JUnit
EMMA – Junit integrationEMMA – Junit integration
• EMMA does not support implicitely "JUnit integration„EMMA does not support implicitely "JUnit integration„• and not assume any particular testing methodology and not assume any particular testing methodology • But after doing some configuration can be used for But after doing some configuration can be used for
measuring the coverage of Junit test casesmeasuring the coverage of Junit test cases
10
JavaCov
• JavaCovJavaCov• Supported coverage types Supported coverage types
– class class – method method – line line – basic block (branch)basic block (branch)– MC/DC (refined version of branch, required in safety-critical MC/DC (refined version of branch, required in safety-critical
systems)systems)
• Supported Java version: 1.4.* ?????????Supported Java version: 1.4.* ?????????• Running way:Running way:
– ANT integrationANT integration– Eclipse pluginEclipse plugin– no separate command line no separate command line
11
JavaCov - JUnit
• JavaCov – JUnit integrationJavaCov – JUnit integration
• supports implicitely "JUnit integration„ without extra supports implicitely "JUnit integration„ without extra configurationconfiguration
• at the same time does not assume any particular test at the same time does not assume any particular test methodmethod
12
JavaCov ? Cobertura
Because of not supporting java 1.5 and the company is Because of not supporting java 1.5 and the company is not going to upgrade JavaCov this year it can be not going to upgrade JavaCov this year it can be replaced by an other coverage tool.replaced by an other coverage tool.
One candidate can be Cobertura – an other open source One candidate can be Cobertura – an other open source product but without supporting MC/DC criterion.product but without supporting MC/DC criterion.
13
Terms refering to a Plugin Architecture
Basic terms:Basic terms:
• a runtime engine that dynamically discovers and a runtime engine that dynamically discovers and loads "plug-ins„loads "plug-ins„
• A plug-in is a structured component that contributes A plug-in is a structured component that contributes code and resources to the „core” system and code and resources to the „core” system and describes itself to runtime engine using a "manifest describes itself to runtime engine using a "manifest file„ usually an xml. file„ usually an xml.
• Runtime engine maintains a Runtime engine maintains a registryregistry of available of available plug-ins and the functions they provide (via plug-ins and the functions they provide (via extension pointsextension points and and extensionsextensions))
• extension pointsextension points, well-defined methods that can be , well-defined methods that can be extended by other plug-ins.extended by other plug-ins.
• extensionextension to the system – when an implementatin is to the system – when an implementatin is added to an extension poit added to an extension poit
14
Requirements
• Important requirement of a plugin frameworkImportant requirement of a plugin framework
• Because of memory and performance issues the Because of memory and performance issues the installed plug-ins should not be loaded installed plug-ins should not be loaded (instantiated) until they are called. They are added (instantiated) until they are called. They are added to a central registry at application start-up and to a central registry at application start-up and are loaded when necessary.are loaded when necessary.
15
Basic Plug-in architecture
Plugin Manager
- loads and activates plugins- maintains a plug-in registry- finds the physical location of plugins
Plugin_1
extension point
Plugin_2
Plugin_n
API for definingManifest files
Manifest2.xml
Manifestn.xml
Manifest1.xml
•Plugin_11
16
Java Plugin Framework
JPF (Java Plugin Framework) is a general-purpose open JPF (Java Plugin Framework) is a general-purpose open source plug-in framework intended to build source plug-in framework intended to build extendable Java applicationsextendable Java applications
Main features:Main features:
• Open framework architecture but having a „default” Open framework architecture but having a „default” implementationimplementation
• Lazy plug-in activation (Plug-in classes are loaded Lazy plug-in activation (Plug-in classes are loaded into memory only when they are actually needed )into memory only when they are actually needed )
• "On the fly" plug-in registration and activation "On the fly" plug-in registration and activation • Xml based manifest files (Eclipse now uses OSGI)Xml based manifest files (Eclipse now uses OSGI)
17
OSGI
The OSGi framework specification forms the basis of the The OSGi framework specification forms the basis of the Eclipse Runtime. As of Eclipse 3.0, the Runtime is fully Eclipse Runtime. As of Eclipse 3.0, the Runtime is fully based on the OSGi notion of based on the OSGi notion of bundlebundle (equivalent to (equivalent to Eclipse plug-ins). The OSGi framework implementation Eclipse plug-ins). The OSGi framework implementation in Eclipse 3.0 is fully compliant with the OSGi in Eclipse 3.0 is fully compliant with the OSGi framework specification R3.0. The OSGi framework specification R3.0. The OSGi implementation in Eclipse 3.1.2 and the upcoming implementation in Eclipse 3.1.2 and the upcoming Eclipse 3.2 fully implement the OSGi R4.0 framework Eclipse 3.2 fully implement the OSGi R4.0 framework specification. specification.
18
User scenario in ETICS 1
Use case: : the user specifies in his/her the user specifies in his/her profile that the required coverage tool profile that the required coverage tool during the build is EMMAduring the build is EMMA
• Test and Metrics Plug-in Framework’s runtime Test and Metrics Plug-in Framework’s runtime engine discovers dynamically and loads the engine discovers dynamically and loads the „EMMA plug-in„„EMMA plug-in„
(For example scans a directory named (For example scans a directory named „plugins” looking for EMMA)„plugins” looking for EMMA)
• In „plugins” directory the engine discovers In „plugins” directory the engine discovers „emma.jar”„emma.jar”
• When the build process reaches the When the build process reaches the „coverage” target the ‘emma.jar’ will be run „coverage” target the ‘emma.jar’ will be run using the configuration described in the using the configuration described in the manifest (xml) file of the EMMA plugin manifest (xml) file of the EMMA plugin
19
User scenario in ETICS 2
Use caseUse case : : ETICS user wants to use TestManager to run gLite ETICS user wants to use TestManager to run gLite testcases outside the build process but using his/her own testcases outside the build process but using his/her own charting library to create reportscharting library to create reports
• In his profile (or somehow through the user interface) In his profile (or somehow through the user interface) or from the user interface of the Plugin Framework he or from the user interface of the Plugin Framework he can chose TestManager between the available test can chose TestManager between the available test driversdrivers
• The plugin manager observes that TestManager should The plugin manager observes that TestManager should be loadedbe loaded
• Also observes that when executing the reporting Also observes that when executing the reporting module in TestManager an extension of it e.g the module in TestManager an extension of it e.g the charting system of the user should be loaded charting system of the user should be loaded
20
User scenario in ETICS 3
Use caseUse case: : ETICS user (manager) wants to know how many ETICS user (manager) wants to know how many open bugs were fixed during a specific buildopen bugs were fixed during a specific build
• In his profile (or somehow through the user interface) In his profile (or somehow through the user interface) he specifies the required metriche specifies the required metric
• The Plug-in Frameworks runtime engine discovers the The Plug-in Frameworks runtime engine discovers the required metric calculation utilityrequired metric calculation utility
• When the build process reaches the target „metrics” When the build process reaches the target „metrics” the selected metric calculation utility will be run.the selected metric calculation utility will be run.
21
Test and Metrics Plug-in Architecture 1
Plugin Manager
- loads and activates plugins- maintains a plug-in registry- finds the physical location of plugins
EMMA
JavaCov/Cobertura
API for definingManifest files
Manifest2.xml
Manifestn.xml
Manifest1.xml
TestManager
JPF
22
Test and Metrics Plug-in Architecture 2
Plugin Manager
- loads and activates plugins- maintains a plug-in registry- finds the physical location of plugins
JavaCov/Cobertura
API for definingManifest files
Manifest2.xml
Manifestn.xml
Manifest1.xmlTestManager
extension point
Java Plugin Framework
Build system - (build.xml)
Charting system
EMMA
23
Conclusion
20062006• Specification on conceptual levelSpecification on conceptual level• Feasibility studyFeasibility study• Architectural specificationArchitectural specification• Integration of the plugin system in the ETICS Integration of the plugin system in the ETICS
framework framework
20072007• Implementation Implementation
Top Related