'Mixing Open And Commercial Tools' by Mauro Garofalo
30
Mixing Open and Commercial Tools Mauro Garofalo, Maveryx W22 – Automation
-
Upload
eurostar-software-testing-conference -
Category
Technology
-
view
111 -
download
0
description
Mixing open source and commercial software is a challenge we face today. The right combined solution offers advantages in flexibility, functionality, performance, and management that aren't available when either open source or commercial technologies are used alone. But one of the major issues is that they don’t always play well together. Some of them can’t be loaded together or they fail to integrate properly. In this presentation we will provide a case study of successful blending open source and commercial software for testing Java applications. The testing environment consisted of using Subversion as versioned repository for tests, JIRA for issue management and in mixing IBM Rational Functional Tester with an open framework for automated functional, regression and GUI testing. These tools provide a rich set of high-level Java API useful for integrating with each other, by minimizing the integration costs. During the presentation, we will explore the costs (open tools used have no fees, integration and training costs …), the advantage (innovation, products that are best-of-breed …), the risks (dedicated support, community feedbacks, ability to add extensions …) and the product’s quality achieved by our solution compared to a full open source or commercial approach. By our experience, we will provide some hints and tips to guide testers and managers to choose a good mix between open and commercial tools based on: budget, technology, know-how and requested quality.
Transcript of 'Mixing Open And Commercial Tools' by Mauro Garofalo
Mixing Open and Commercial Tools
Mauro Garofalo, Maveryx
W22 – Automation
Summary
Costs, Advantages and Risks of mixing open source and commercial software
A case study of blending open source and commercial tools for testing Java applications
Some practical rules to combine open and commercial test tools
2All trademarks referenced herein are the properties of their respective owners.
So, what is Open Source? First, no official definition exists (http://www.opensource.org/osd.html)
Open source doesn't just mean access to the source code
Open source rules are, simply enough:
Allow free redistribution
Allow source code access
Permit derived works
Protect the integrity of the author’s source code
….
3
Key Predictions
By 2012, at least 80% of all commercial software solutions will
include elements of open source technology (Gartner Highlights Key
Predictions for IT Organizations and Users in 2008 and Beyond)
By 2010, Global 2000 IT organizations will use open-source products
in 80% of infrastructure-focused software investments and 25% of
business software investments (Gartner's Positions on the Five Hottest IT
Topics and Trends in 2005)
…
4
Survey: Do you use open source software?
Do you use…
Firefox to browse the web?
OpenOffice to write documents?
Linux or Ubuntu as server or end user OS?
Subversion or CVS for the source code?
MySQL as database?
Eclipse to write code?
FileZilla for file transfer?
…
http://downloadpedia.org/Open_Source_Alternative_to_Commercial_Software
5
Companies use open source because…
COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research]
But cost is not the only reason for open source’s growing popularity: many firms now know that it offers more FLEXIBILITY than proprietary programs [Matthew Aslett, 451 Group]
Improving quality of products a/o processes
Achieving faster time-to-market
Higher level of security
…
6
OSS vs. Commercial Commonly it is framed as
Linux vs. Microsoft Security vs. Innovation
7
Rewards of OSSavailability of source codefreedom to customize, enhance or repairlower price, sometimes freedoes not depend on vendorsecurity…
Riskslack of support, training and docsfrequent updates & forksbackwards compatibility…
Rewards of Commercial softwarevendor professional serviceseasier to adopt in organizationscomply with industrial standardsmore user friendly mature products offer more features…
Riskscostsdependent upon vendordo not allow users to alter or customize the product.…
A ‘mixed source’ strategy The greatest chance for success with open source software involves a
strategy of mixing open with commercial software
A mixed-source strategy means identifying and integrating both open and commercial products that
are best-of-breed technologies
fits the requirements
minimizes integration ‘headaches’
rather than using “good enough” products
included in legacy solutions or
available (for free…) on SourceForge!
8
Integration
Mixing open & commercial tools requires integration and interoperability
Integrating any two programs is often challenging…
Integrating commercial software & OSS can be difficult because: variety of technologies (commercial) tools use proprietary ‘approaches’ data integration is not friendly (OpenOffice vs. Microsoft Office file formats) no design for interoperability different products have varying requirements for integration many products are intended for use with specific combinations of tools or designated platforms …
The result is that integration of OSS with commercial offerings is often partial or missing completely, or must be provided by a third-party vendor…
9
Test Tools
Application Test Tools for: Source testing Functional & GUI testing Load & Performance testing Embedded testing Database testing …
Supporting Tools for: Test Management Bug Tracking Requirements Management Configuration / version control …
10
Can they cooperate?
11
Open Source Commercial
Case Study
Testing a Java™ stand-alone, GUI-based application for the Telecom
market
The application-under-test has
‘legacy’ functionalities (taken from past project / application)
new features continuously changing (specially at early stage)
Reusing of many automated functional tests built with IBM Rational
Functional Tester for legacy functionality testing
12
The Test Environment
13
Subversion
Subversion is a free & open source version control system Industry standard Easy to use Local baseline Atomic commits One repository to maintain (single url for all projects) …
There are several plug-ins implementing SVN support into Eclipse (→ Subversive)
14
JIRA
JIRA is a commercial issue, bug and project tracking tool Easy to use Highly configurable (e.g. to manage a complex workflow) Flexible for different kind of issues (e.g. for test reporting) Accessible from tests scripts through Java API Extensible and customizable (source code is available to
customers) Eclipse integration Subversion integration Fully supported Quite cheap …
15
Eclipse
Eclipse is a free & open source development environment comprising an IDE and an extensible plug-in system Industry standard Fully supported Extensible and configurable more than 1200 plug-in available at the
Marketplace (most free!) Simple and powerful Workbench → Views, Editors and Perspectives …
16
IBM Rational Functional Tester
Rational Functional Tester is a commercial tool for automated functional and regression testing Built-in support for Java applications Java scripting language Eclipse rich client application ScriptAssure technology to accommodate UI changes Data-driven & Keyword-driven testing …
17
Maveryx
Maveryx is a free & open source tool for automated functional, regression, GUI and data-driven testing Built-in support for Java applications No ‘GUI Map’ needed to create and run tests Intelligent GUI objects recognition (including fuzzy matching) Available as Eclipse plug-in Extensible and configurable through plug-ins Integrated with all Java IDEs and testing frameworks (Eclipse,
NetBeans, JUnit, IBM Rational Functional Tester, …) …
18
Test Strategy
Legacy functionalities tested by reusing test scripts developed with IBM Rational Functional Tester Needed to update / re-capture the GUI Maps for the
current application
New functionalities tested by using Maveryx Maximizing the Rational Test Manager usage
Refactoring of some tests by mixing, where possible, Rational code with Maveryx code
19
Mixing open & commercial tools
You must decide which components be open vs. closed source
A possible strategy: closed source for “commodity” components open source for “differentiating” components
Following such a strategy: classify features as commodity or differentiating determine for each feature whether using open or closed SW based on:
budget
technology
know-how
requested quality
20
Evaluation Process
21
Identify candidates
Read existing reviews
Compare attributes to your needs
Analyze top candidates
Wrap-up
Identify Candidates
First step : find out your options
Search for open and commercial tools:
ask friends and co-workers
search using specialized sites
use search engines
thumb specialized magazines
…
22
Read existing reviews
After you've identified your options, read existing evaluations about the alternatives:
use search engines
search for specialized web sites
read blogs, forum
thumb specialized & independent magazines
look at social networks
…
23
Evaluation Criteria
24
TOOL #1 TOOL #2 TOOL #3
Functionality
Cost
Support
Market Share
Maintenance/ Longevity
Reliability
Performance
Scalability
Usability
Security
Flexibility / Customizability
Interoperability
…
Analyze top candidates
After the initial evaluation, pick the top candidates, and perform a more in-depth analysis
Try any contender in non-critical situations first, by evaluating the functionality that you’re interested in using (use a checklist)
Examine also the program’s documentation, source code (for OSS), and other related materials
If the tool had some but all the functions you need, examine what it would take to add those functions
25
Evaluating Software
In general, a mature platform will meet the following criteria: Project extensions are available The project has reached a one-year maturity mark Security patches, bug fixes and new features/enhancements are delivered
separately The software has reasonable automated unit and functional tests with code
coverage in the 30 percent - 80 percent range The software easily integrates with external tools The component’s bug database is kept up-to-date with revision numbers for each
product enhancement The solution has been ported across multiple platforms (Linux, Windows, Solaris
and Mac) Large-scale adoption, including both public and well-known large-scale
organizational deployments exists There is documentation purpose separation: User guide (or Getting Started
Guide), Installation guide, Admin guide and Development guide
26
Important factors influencing testing tools
Factors influencing the choice of Software Testing Tools Programming Language Support Long feedback Evaluation Period Selling is driven by Marketing Vendor locking Specialization & Generalization …
27
Wrap-up
Review and present to your manager the results of your evaluation
Identify the recommended solution
Identify the main alternatives, first of all the recommended alternative
Once a decision has been made, get the software & go!
28
Conclusion
A “mixed-source” approach combines open source & proprietary software, by taking the best of both worlds
The right mixed solution offers advantages in flexibility, functionality, performance, and management that aren't available when either open source or commercial technologies are used alone
But, integrating commercial software and OSS can be difficult…
For testing purposes, evaluate open source and commercial tools by making decisions based on cost, technical and integration factors
29
