Quality Improvement in Volunteer

Free and Open Source Software Projects

Exploring the Impact of Release Management

A dissertation submitted to the University of Cambridge

for the Degree of Doctor of Philosophy

Martin Michlmayr Kings College

March 2007

Centre for Technology Management

Institute for Manufacturing

University of Cambridge

Quality Improvement in Volunteer

Free and Open Source Software Projects

Exploring the Impact of Release Management

Martin Michlmayr

Free and open source software has had a major impact on the computer

industry since the late 1990s and has changed the way software is perceived,

developed and deployed in many areas. Free and open source software, or

FOSS, is typically developed in a collaborative fashion and the majority of

contributors are volunteers. Even though this collaborative form of develop-

ment has produced a significant body of software, the development process

is often described as unstructured and unorganized. This dissertation studies

the FOSS phenomenon from a quality perspective and investigates where im-

provements to the development process are possible. In particular, the focus

is on release management since this is concerned with the delivery of a high

quality product to end-users.

This research has identified considerable interest amongst the FOSS commu-

nity in a novel release management strategy, time based releases. In contrast to

traditional development which is feature-driven, time based releases use time

rather than features as the criterion for the creation of a new release. Releases

are made after a specific interval, and new features that have been completed

and sufficiently tested since the last release are included in the new version.

This dissertation explores why, and under which circumstances, the time

based release strategy is a viable alternative to feature-driven development

and discusses factors that influence a successful implementation of this release

strategy. It is argued that this release strategy acts as a coordination mecha-

nism in large volunteer projects that are geographically dispersed. The time

based release strategy allows a more controlled development and release pro-

cess in projects which have little control of their contributors and therefore

contributes to the quality of the output.

Preface

Except for commonly understood and accepted ideas, or where specific refer-

ence is made, the work reported in this dissertation is my own and includes

nothing that is the outcome of work done in collaboration. No part of the

dissertation has been previously submitted to any university for any degree,

diploma or other qualification.

This dissertation consists of approximately 60,000 words, and therefore does

not exceed the 65,000 word limit put forth by the Degree Committee of the

Department of Engineering.

Martin Michlmayr

March 2007

iii

Contents

Abstract ii

Preface iii

1. Introduction 1

1.1. A Definition of FOSS . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2. Problems with the FOSS Process . . . . . . . . . . . . . . . . . 5

1.3. Contribution of this Dissertation . . . . . . . . . . . . . . . . . 7

1.4. Structure of the Dissertation . . . . . . . . . . . . . . . . . . . . 7

2. Quality Management 9

2.1. Quality Literature . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1.1. Definitions of Quality . . . . . . . . . . . . . . . . . . . . 9

2.1.2. Process Improvement . . . . . . . . . . . . . . . . . . . . 12

2.2. FOSS Literature . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.2.1. Quality Considerations . . . . . . . . . . . . . . . . . . . 13

2.2.2. Development Style . . . . . . . . . . . . . . . . . . . . . 15

2.2.3. Challenges and Problems . . . . . . . . . . . . . . . . . . 19

2.2.4. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.3. Exploratory Study . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.3.1. Methodology . . . . . . . . . . . . . . . . . . . . . . . . 22

2.3.2. Comparison of Proprietary Software and FOSS . . . . . 23

2.3.3. Development and Quality Practices . . . . . . . . . . . . 24

2.3.4. Quality Problems . . . . . . . . . . . . . . . . . . . . . . 27

2.3.5. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.4. Refining the Scope . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.5. Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . 34

iv

Contents v

3. Release Management 35

3.1. Literature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.1.1. Software Maintenance . . . . . . . . . . . . . . . . . . . 36

3.1.2. Release Management in FOSS . . . . . . . . . . . . . . . 38

3.2. Exploratory Study . . . . . . . . . . . . . . . . . . . . . . . . . 40

3.2.1. Methodology . . . . . . . . . . . . . . . . . . . . . . . . 40

3.2.2. Types of Release Management . . . . . . . . . . . . . . . 41

3.2.3. Preparation of Stable Releases . . . . . . . . . . . . . . . 42

3.2.4. Skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

3.2.5. Tools and Practices . . . . . . . . . . . . . . . . . . . . . 48

3.2.6. Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 50

3.2.7. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 52

3.3. Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

3.3.1. Modularity and Complexity . . . . . . . . . . . . . . . . 54

3.3.2. Coordination in FOSS . . . . . . . . . . . . . . . . . . . 55

3.3.3. Coordination Theory . . . . . . . . . . . . . . . . . . . . 56

3.4. Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . 59

4. Research Question and Methodology 61

4.1. Research Question . . . . . . . . . . . . . . . . . . . . . . . . . 61

4.2. The Case Study Approach . . . . . . . . . . . . . . . . . . . . . 62

4.3. Selection Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . 64

4.4. Case Study Projects . . . . . . . . . . . . . . . . . . . . . . . . 68

4.5. Sources of Evidence . . . . . . . . . . . . . . . . . . . . . . . . . 71

4.5.1. Interviews . . . . . . . . . . . . . . . . . . . . . . . . . . 73

4.5.2. Mailing Lists . . . . . . . . . . . . . . . . . . . . . . . . 73

4.5.3. Documents . . . . . . . . . . . . . . . . . . . . . . . . . 74

4.5.4. Observation . . . . . . . . . . . . . . . . . . . . . . . . . 75

4.6. Methodological Considerations . . . . . . . . . . . . . . . . . . . 75

4.6.1. Construct Validity . . . . . . . . . . . . . . . . . . . . . 75

4.6.2. Internal Validity . . . . . . . . . . . . . . . . . . . . . . 76

4.6.3. External Validity . . . . . . . . . . . . . . . . . . . . . . 76

4.6.4. Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . 77

4.6.5. Personal Involvement . . . . . . . . . . . . . . . . . . . . 77

Contents vi

4.6.6. Ethical Issues . . . . . . . . . . . . . . . . . . . . . . . . 78

4.7. Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . 79

5. Time Based Releases: Learning from 7 Case Studies 80

5.1. Debian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

5.2. GCC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

5.3. GNOME . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

5.4. Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

5.5. OpenOffice.org . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

5.6. Plone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

5.7. X.org . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

5.8. Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . 105

6. Release Strategy 106

6.1. Problems Prompting a Change . . . . . . . . . . . . . . . . . . . 106

6.1.1. Lack of Planning . . . . . . . . . . . . . . . . . . . . . . 107

6.1.2. Problems Caused by Lack of Planning . . . . . . . . . . 109

6.1.3. Long-term Issues . . . . . . . . . . . . . . . . . . . . . . 112

6.1.4. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . 113

6.2. Time Based Strategy as an Alternative . . . . . . . . . . . . . . 113

6.2.1. Conditions That Have to be Met . . . . . . . . . . . . . 113

6.2.2. Time Based Releases as a Coordination Mechanism . . . 120

6.2.3. Advantages of Time Based Releases . . . . . . . . . . . . 129

6.2.4. Open Questions . . . . . . . . . . . . . . . . . . . . . . . 133

6.3. Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . 136

7. Schedules 138

7.1. Choice of the Release Interval . . . . . . . . . . . . . . . . . . . 138

7.1.1. Regularity and Predictability . . . . . . . . . . . . . . . 138

7.1.2. User Requirements . . . . . . . . . . . . . . . . . . . . . 140

7.1.3. Commercial Interests . . . . . . . . . . . . . . . . . . . . 141

7.1.4. Cost Factors Related to Releasing . . . . . . . . . . . . . 143

7.1.5. Network Effects . . . . . . . . . . . . . . . . . . . . . . . 146

7.1.6. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . 148

7.2. Creation of a Schedule . . . . . . . . . . . . . . . . . . . . . . . 148

Contents vii

7.2.1. Identification of Dependencies . . . . . . . . . . . . . . . 149

7.2.