04.04.2007 Anna Ruokonen/TUT
Open source and empirical studies
An Empirical Study of Open-Source and Closed-Source Software Products
by Paulson, J.W.; Succi, G.; Eberlein, A.
04.04.2007 Anna Ruokonen/TUT
Empirical studies (1)
• Empirical research
– Empirical observations or experiments are run in order to answer particular research questions
• A research design: the logical sequence connecting the empirical data to initial research questions and conclusions
1. The research questions (who, what, why, how, where)
2. Propositions (->what should be studied)
3. Unit(s) of analysis
4. The logic linking the data to the propositions
5. The criteria for interpreting the findings[Jin2003]
04.04.2007 Anna Ruokonen/TUT
Empirical studies (2)
Construct validity
1. Select specific types of changes that are to be studied2. Demonstrate that selected measures of the changes do reflect the
selected types of changes
Internal validity
• Address rival explanations• Are the results due to circumstances only (the null hypotheses)?
[Jin2003]
04.04.2007 Anna Ruokonen/TUT
The Beliefs
• The 5 common beliefs of OS: – Defects are found and fixed more rapidly in OS projects
– Hypothesis of more prevalent creativity, faster project growth, modularity and simplicity
• The beliefs often lack empirical evidence
• Switch from CS to OS an attempt to win market share, expand products growth, improve market penetration
• This study investigates claims of OS SW community
• Chosen metrics compare OS and CS project characteristics
04.04.2007 Anna Ruokonen/TUT
Project characteristics
Some factors affecting a software growth (OS or CS):
• the application domain
• the SW development process
• the type of product
• the management
• marketing dynamics
• organizational dynamics
04.04.2007 Anna Ruokonen/TUT
Analyzed projects
The most well-known and succesful OS projects
• Linux
• Apache
• GCC, GNU Compailer Collection
Three CS projects
• Representative of particular CS domain
• SW protocol stacks of wireless telecommunications products
04.04.2007 Anna Ruokonen/TUT
GROWTH
HYPOTHESIS: OS development fosters faster system growth METRICS: • Overall project growth in functions over time• Overall project growth in LOC over time
FINDINGS:• Growth rates over time were similar and constant for all analyzed
projects• Does not support hypothesis
04.04.2007 Anna Ruokonen/TUT
SIMPLICITY
HYPOTHESIS: OS projects succeed because of their simplicity
METRICS:
• Overall project complexity
• Average complexity of all the functions
• Average complexity of the functions added
FINDINGS:
• All 3 metrics showed that OS projects are more complex
• Possible reasons: embedded SW, personal style, language
• Does not support hypothesis
04.04.2007 Anna Ruokonen/TUT
DEFECTS
HYPOTHESIS: OS projects have fewer defects as they are found and fixed more rapidly
METRICS: • Functions modified over time
• Modified functions as percentage of total functions
FINDINGS:
• In OS more function was initially changed
• In OS more defects are found and the product becomes more stable over time
• OS showed significantly higher percentage of modified functions
• Supports hypothesis
04.04.2007 Anna Ruokonen/TUT
CREATIVITY
HYPOTHESIS: OS foster more creativity
METRICS:
• Functions added over time
FINDINGS:
• No commercial project showed any positive growth rate over time
• The growing rate of OS projects was greater than the CS projects
• Supports hypothesis
04.04.2007 Anna Ruokonen/TUT
MODULARITY
HYPOTHESIS: OS projects are more modular
METRICS:
• Correlation between functions added and functions modified
FINDINGS:
• There was a strong corrrelation between the growing rate and changing rate in OS projects
• No correlation in CS projects
• Does not support hypothesis
04.04.2007 Anna Ruokonen/TUT
Summary
• OS foster more creativity
• OS projects have fewer defects as they are found and fixed more rapidly
• External global factors and feedback mechanisms have a greater impact on the SW evolution than the development metod
• What are the global factors shared by both OS and CS projects?
• Before move to use OS development method, all of the goals to be achieved should carefully considered
• Ensure that proper metrics are collected and monitored
Top Related