QuESo: a Quality Model for Open Source Software Ecosystems
38
QuESo: a Quality Model for Open Source Software Ecosystems ICSOFT-EA 2014 Oscar Franco-Bedoya [email protected] David Ameller [email protected] Dolors Costal [email protected] Xavier Franch [email protected]
description
Open source software has witnessed an exponential growth in the last two decades and it is playing an increasingly important role in many companies and organizations leading to the formation of open source software ecosystems. In this paper we present a quality model that will allow the evaluation of those ecosystems in terms of their relevant quality characteristics such as health or activeness.
Transcript of QuESo: a Quality Model for Open Source Software Ecosystems
QuESo: a Quality Model for Open Source Software Ecosystems
ICSOFT-EA 2014
Oscar [email protected]
David [email protected]
Dolors [email protected]
Xavier [email protected]
2
Overview
Related work
Method
QuESoApplication
example
Discussion
Future work
Conclusions
Outline
Overview
3
0
500.000
1.000.000
1.500.000
2.000.000
2.500.000
2007 2009 2011 2013 2015
The number of open source software projects is growing
Source: Black Duck Management webinar 2014in IEEE OSS webcasts series
4
A challenge arises
How can we measure the quality of these OSS projects?
Way to prevent bad decisions Avoid problems Verify the compliance with requirements and
business goals Systematic monitoring to provide feedback
5
The ecosystem metaphor is used to understand software relationships
A system formed by the interaction of a community of
organisms with their environment
DevelopersAdopters
SoftwareProjects
Contributors Softwareproducts
6
Software ecosystems definitions(Messerschmitt and Szyperski, 2005)
Business perspective
“Set of actors functioning as a
unit and
interacting with a
shared market for
software and services.”
(Jansen and Cusumano, 2013)
Technical perspective
“ A collection of
software projects which are
developed and evolve together
in the same environment.”
(Lungu et al., 2009).
7
We use both definitions in our work
In order to assess software ecosystem quality in its broader sense
Technicalperspective
Businesspersepctive
OSSecosystem
8
From the challenge:
How can we measure the quality of these OSS projects?
To the research goal of this work:
How can we measure the quality of these OSS ecosystems?
9
Proposal: measure OSS ecosystem quality using a quality model
“ quality model is the set of characteristics and the relationships between them which provide the basis for specifying quality requirements and evaluating quality” (ISO/IEC 9126, 2001)
Our working principle:
Collect metrics proposed in the state of the art and structure them in a quality model
10
Overview
Related work
Method
QuESoApplication
example
Discussion
Future work
Conclusions
Outline
11
Related Work
12
Overview
Related work
Method
QuESoApplication
example
Discussion
Future work
Conclusions
Outline
13
Gathering measures
Systematic
literature
review
1Constructing Model
Bottom up
Top down
2
The methodology have two steps
14
Gathering measures from the available literature.
Systematic literature review(Kitchenham and Charters, 2007).
Measure inclusion criteria:(Hartigh et al., 2013; Neu et al., 2011)
User-friendly and operationalizable.
Non-redundant
68 measures!
BookManual search
Secondary studies
Digital libraries
35 11 1 6
Papers with measures of OSS ecosystems
17
15
Method for quality model construction.Radulovic and Garcia-Castro (2011)
Sub-characteristics
Relationships
Quality measures
Derived measures
Basic measures
Align quality sub-characteristics with a quality model
QuESo
SLR measures
QualOSSISO Quality model
16
Overview
Related work
Method
QuESoApplication
example
Discussion
Future work
Conclusions
Outline
17
Quality characteristics in QuESo have been organized into three dimensions
QuESoCommunity
quality
Ecosystemnetwork quality
Platform quality
Software platform in
which the projects
of the ecosystem
are built.
OSS communities
that grow inside the
ecosystem and
ecosystem’s projects
OSS ecosystem as a
network of interrelated
elements
#
11
54
8
20
18
Maintenance capacity :Ability to provide the resources necessary for maintaining community products
Dimension
Characteristic
Sub-characteristic
From QualOSS
Measure
QuESo
Maintenance capacity
Community quality
Size
Activeness
number authors
Gnome: 3.500 1997-2012
number of commitsGnome: 480.000
1997-2007
19
Network health: How well members are connected and their impact
Ability of the nodes to
establish connections
between them
Capacity of the
ecosystem nodes to be
classified around its
projects.
QuESo
Ecosystemnetwork quality
Network health
Clustering
Interrelatedness
number community
projectsGnome: 1.200
1997-2012
Contributor activity graphGnome:499 projects /15.000 changes1998-2011
20
Our proposal: QuESo
QuESo
Maintenance capacity
Sustainability
Process maturity
Community quality
Size
Cohesion
Activeness
Heterogeneity
Regenerationability
Effort balance
Expertisebalance Visibility
Ecosystemnetwork quality
Resource health
Network health
Information consistency
FinancialVitality
Clustering
Trustworthiness
Interrelatedness
Synergetic evolution
Platform quality
Dimension
Characteristic
Sub-characteristic
From QualOSS
4
7
15
1
2
11
311
2
3
7
4
3
1
21
Overview
Related work
Method
QuESoApplication
example
Discussion
Future work
Conclusions
Outline
22
Using QuESO for risk analysisKenett, R et al. (2014)
Managing Risk and Costs in OSS Adoption
23
Software ecosystem model
24
Relations between measures and quality characteristics
QuESomeasures
QuESocharacteristics
Mail per day
Activeness
25
SALMonOSS: Assessing Open Source Communities’ Health(Oriol , Franco-Bedoya, 2014)
26
: Ecosystem Quality Object
goals-tasks-resources-softgoals
: Ecosystem Quality Parameter
QuESo Quality characteristics & measures
: Ecosystem Measure Directive
Measures metrics process
: Ecosystem Quality Objectives
Logic expressions
: Ecosystem Quality Evaluation Event
Logic expressions
: Ecosystem Action Garantees
actions for OSS Actors
Has
Define
Trigger
EQLA: Ecosystem Quality Level Agreement
27
Overview
Related work
Method
QuESoApplication
example
Discussion
Future work
Conclusions
Outline
28
Discussion
Some observations
Completeness: To remark that QuESo model may not be complete
Quantitative vs. qualitative: QuESo measures are mostly quantitative.
Unbalanced distribution of measures: more research needed
Measure names: room for improvement
29
Overview
Related work
Method
QuESoApplication
example
Discussion
Futurework
Conclusions
Outline
30
Future work: Perform a complete quality assessment
Define
Quality assessment process
How are the values of each measure interpreted?
Good and bad values?
How can the measures be merged to
provide the assessment for a particular
sub- characteristic of the quality model?
What are the principles to perform the assessment with missing, incorrect,
and/or inconsistent measure data?
31
Overview
Related work
Method
QuESoApplication
example
Discussion
Futurework
Conclusions
Outline
32
Conclusions
We have presented QuESo, a quality model for assessing the quality of OSS ecosystems.
This quality model has been constructed following bottom-up –Top down strategies.
QuESo covers the platform, the community, and the ecosystem network dimensions.
This quality model can be used as a starting point for the quality assessment of an OSS ecosystem
Thanks for your attention
Comments and Questions
34
BibliographyLungu, M., Lanza, M., Gˆırba, T., and Robbes, R. (2010). The Small Project Observatory: Visualizing software ecosystems.
Science of Computer Programming, 75(4):264–275. Lungu, M., Malnati, J.
Jansen, S. and Cusumano, M. (2013). Software Ecosystems: Analyzing and Managing Business Networks in the
Software Industry, chapter Defining software ecosystems:a survey of software platforms and business network
governance, pages 13–28. Volume 1 of (Jansen et al., 2013).
ISO/IEC 9126 (2001). Product quality – Part 1: Quality model.
ISO/IEC 25010:2011 Systems and software engineering -- Systems and software Quality Requirements and Evaluation (SQuaRE)
-- System and software quality models.
Soto, M. and Ciolkowski, M. (2009). The QualOSS open source assessment model measuring the performance
of open source communities. In Proceedings of the 3rd ESEM, pages 498–501.
Jansen, S., Brinkkemper, S., and Michael Cusumano (2013). Software Ecosystems: Analyzing and ManagingBusiness Networks in
the Software Industry, volume 1. Edward Elgar Publishing.
Kitchenham, B. and Charters, S. (2007). Guidelines for performing Systematic Literature reviews in Software
Engineering Version 2.3. Technical report, Keele University EBSE.
Hartigh, E., Visscher, W., Tol, M., and Salas, A. J. (2013). Software Ecosystems: Analyzing and Managing Business
Networks in the Software Industry, chapter Measuring the health of a business ecosystem, pages 221– 245. Volume 1 of (Jansen
et al., 2013).
Radulovic, F. and Garcia-Castro, R. (2011). Extending Software Quality Models - A Sample In The Domain
of Semantic Technologies. In Proceedings of the 23rd SEKE, pages 25–30.
Kenett, R, Franch, X., Susi, A., Galanis, N. (2014). Adoption of Free Libre Open Source Software (FLOSS): A Risk Management
Perspective. In Proceedings of the 38th COMPSAC.
Oriol, M. Franco-Bedoya, O. Franch, X., Marco, J. Assessing Open Source Communities’ Health using Service Oriented
Computing Concepts. In proceedings of the 8th RCIS.
Messerschmitt, D.G., Szyperski, C.: Software ecosystem: understanding an indispensable technology and industry. The MIT Press,
Cambridge (2005)
35
Backup
36
Backup
37
Backup
38
