892019 Gartner (20140503) - Road Map for Opensource Succes Understanding Quality and Security

httpslidepdfcomreaderfullgartner-20140503-road-map-for-opensource-succes-understanding-quality 16

G00259749

Road Map for Open-Source Success

Understanding Quality and SecurityPublished 3 March 2014

Analyst(s) Mark Driver

Quality and security are major issues when comparing open- and closed-

source software IT leaders should follow these Dos and Donts to

minimize risk and maximize the value of open-source software in the

enterprise especially as it relates to quality and security

Key Challenges The open-source software model does not result in inherently higher or lower quality nor is it

more or less secure than closed-source alternatives

The transparent and collaborative nature of the OSS model affords several theoretical

advantages (eg a larger and more diverse developer base)

The real-world governance and engineering disciplines embraced by individual projects

ultimately determine the quality of the final product rather than any inherent advantage in the

model itself

Recommendations Dont assume OSS automatically has higher quality than closed-source alternatives

Dont assume OSS is automatically more secure than closed-source alternatives

Do evaluate the quality and security of OSS technologies on a case-by-case basis

Strategic Planning AssumptionsBy 2016 at least 95 of IT organizations will leverage nontrivial elements of open-source software

technology in their mission-critical IT portfolios including cases where they might not be aware of it

which represents an increase from 75 in 2010

892019 Gartner (20140503) - Road Map for Opensource Succes Understanding Quality and Security

httpslidepdfcomreaderfullgartner-20140503-road-map-for-opensource-succes-understanding-quality 26

Through 2020 security and quality defects publicly attributed to OSS projects will increase

significantly driven by a growing presence within high-profile mission-critical and mainstream IT

workloads

IntroductionIn this research series we address the Dos and Donts that IT practitioners should keep in mind

when addressing the role of open-source software (OSS) in their long-term technology road maps

Here we address key factors when dealing with quality and security issues versus closed-source

alternatives

Quality and security have long been topics of serious debate between open- and closed-source

software developers On one hand many argue that the transparent nature of the OSS model

inherently results in higher quality (and subsequently security) than closed-source solutions

Others argue that the traditional hobbyist nature of many OSS projects underplays the necessary

diligence and engineering best practices when trying to match closed-source efforts As our

research here shows the answer depends on a number of critical factors rather than any single

rule-of-thumb

As the use of OSS assets grows in mainstream IT portfolios IT leaders must set aside out-of-date

assumptions and generalities related to OSS security and quality factors (both pro and con) in favor

of detailed project-by-project measurements and assessments Specifically they must assess the

quality and security of each open-source solution in the enterprise portfolio on its individual merits

paying close attention to such factors as

Community effectiveness

Project maturity

Defect tracking and resolution discipline

Planned product road maps versus ad hoc coding

Robust IP governance

Security audits

Analysis

Dont Assume OSS Automatically Has Higher Quality Than Closed-Source Alternatives

Because of its transparent nature mdash and unlike closed-source efforts mdash it is more difficult to

maintain a state of mediocrity through obscurity in well-run and highly collaborative open-source

projects Consequently many users assume that OSS has inherently higher quality than closed-

source efforts Contributing to this assumption is the fact that among mature open-source projects

Page 2 of 6 Gartner Inc | G00259749

892019 Gartner (20140503) - Road Map for Opensource Succes Understanding Quality and Security

httpslidepdfcomreaderfullgartner-20140503-road-map-for-opensource-succes-understanding-quality 36

with effective and resilient communities overall quality is often high In 2012 a report by Coverity (a

company specializing in software quality measurement) showed that the average defect density

(number of defects per 1000 lines of code) for open-source projects was generally on a par with

closed-source code bases This report showed that although overall defect density for OSS was

lower (ie it demonstrated higher quality) than closed-source for projects of fewer than 1 million

lines of code beyond this threshold the defect density for OSS was higher (ie of lower quality)

than closed source (see Table 1)

Table 1 Defect Density by Project Size OSS Versus Proprietary Code

Lines of Code OSS Defects Proprietary Defects

Fewer Than 100000 040 051

100000 to 499999 060 066

500000 to 1 million 044 098

More than 1 million 075 066

Average Across Projects 069 068

Source Coverity ( wwwcoveritycom )

However the debate over open- versus closed-source quality continues Many OSS proponents

vehemently maintain that typical OSS solutions have fewer overall defects than closed-source

alternatives This assumption was captured in Eric Raymonds seminal discussion of the OSS

model The Cathedral and the Bazaar there he proposed that Given a large enough beta-testerand co-developer base almost every problem will be characterized quickly and the fix obvious to

someone In reality we find that the quality of open source is driven more by individual project

governance and best practices than any factor inherent in the model itself moreover open-source

quality is driven by virtually the same factors that are present in a closed-sourced project including

Community effectiveness

Project maturity

Defect tracking and resolution discipline

Planned product road maps versus ad hoc coding Robust IP governance

Security audits

The key differentiated benefit within the OSS model is its potential to draw a far greater number of

eyes on the code than a closed effort with a small team This creates the potential for wider and

more-in-depth peer review and investigation before the code is released however these

Gartner Inc | G00259749 Page 3 of 6

892019 Gartner (20140503) - Road Map for Opensource Succes Understanding Quality and Security

httpslidepdfcomreaderfullgartner-20140503-road-map-for-opensource-succes-understanding-quality 26

Through 2020 security and quality defects publicly attributed to OSS projects will increase

significantly driven by a growing presence within high-profile mission-critical and mainstream IT

workloads

IntroductionIn this research series we address the Dos and Donts that IT practitioners should keep in mind

when addressing the role of open-source software (OSS) in their long-term technology road maps

Here we address key factors when dealing with quality and security issues versus closed-source

alternatives

Quality and security have long been topics of serious debate between open- and closed-source

software developers On one hand many argue that the transparent nature of the OSS model

inherently results in higher quality (and subsequently security) than closed-source solutions

Others argue that the traditional hobbyist nature of many OSS projects underplays the necessary

diligence and engineering best practices when trying to match closed-source efforts As our

research here shows the answer depends on a number of critical factors rather than any single

rule-of-thumb

As the use of OSS assets grows in mainstream IT portfolios IT leaders must set aside out-of-date

assumptions and generalities related to OSS security and quality factors (both pro and con) in favor

of detailed project-by-project measurements and assessments Specifically they must assess the

quality and security of each open-source solution in the enterprise portfolio on its individual merits

paying close attention to such factors as

Community effectiveness

Project maturity

Defect tracking and resolution discipline

Planned product road maps versus ad hoc coding

Robust IP governance

Security audits

Analysis

Dont Assume OSS Automatically Has Higher Quality Than Closed-Source Alternatives

Because of its transparent nature mdash and unlike closed-source efforts mdash it is more difficult to

maintain a state of mediocrity through obscurity in well-run and highly collaborative open-source

projects Consequently many users assume that OSS has inherently higher quality than closed-

source efforts Contributing to this assumption is the fact that among mature open-source projects

Page 2 of 6 Gartner Inc | G00259749

892019 Gartner (20140503) - Road Map for Opensource Succes Understanding Quality and Security

httpslidepdfcomreaderfullgartner-20140503-road-map-for-opensource-succes-understanding-quality 36

with effective and resilient communities overall quality is often high In 2012 a report by Coverity (a

company specializing in software quality measurement) showed that the average defect density

(number of defects per 1000 lines of code) for open-source projects was generally on a par with

closed-source code bases This report showed that although overall defect density for OSS was

lower (ie it demonstrated higher quality) than closed-source for projects of fewer than 1 million

lines of code beyond this threshold the defect density for OSS was higher (ie of lower quality)

than closed source (see Table 1)

Table 1 Defect Density by Project Size OSS Versus Proprietary Code

Lines of Code OSS Defects Proprietary Defects

Fewer Than 100000 040 051

100000 to 499999 060 066

500000 to 1 million 044 098

More than 1 million 075 066

Average Across Projects 069 068

Source Coverity ( wwwcoveritycom )

However the debate over open- versus closed-source quality continues Many OSS proponents

vehemently maintain that typical OSS solutions have fewer overall defects than closed-source

alternatives This assumption was captured in Eric Raymonds seminal discussion of the OSS

model The Cathedral and the Bazaar there he proposed that Given a large enough beta-testerand co-developer base almost every problem will be characterized quickly and the fix obvious to

someone In reality we find that the quality of open source is driven more by individual project

governance and best practices than any factor inherent in the model itself moreover open-source

quality is driven by virtually the same factors that are present in a closed-sourced project including

Community effectiveness

Project maturity

Defect tracking and resolution discipline

Planned product road maps versus ad hoc coding Robust IP governance

Security audits

The key differentiated benefit within the OSS model is its potential to draw a far greater number of

eyes on the code than a closed effort with a small team This creates the potential for wider and

more-in-depth peer review and investigation before the code is released however these

Gartner Inc | G00259749 Page 3 of 6

892019 Gartner (20140503) - Road Map for Opensource Succes Understanding Quality and Security

httpslidepdfcomreaderfullgartner-20140503-road-map-for-opensource-succes-understanding-quality 36

with effective and resilient communities overall quality is often high In 2012 a report by Coverity (a

company specializing in software quality measurement) showed that the average defect density

(number of defects per 1000 lines of code) for open-source projects was generally on a par with

closed-source code bases This report showed that although overall defect density for OSS was

lower (ie it demonstrated higher quality) than closed-source for projects of fewer than 1 million

lines of code beyond this threshold the defect density for OSS was higher (ie of lower quality)

than closed source (see Table 1)

Table 1 Defect Density by Project Size OSS Versus Proprietary Code

Lines of Code OSS Defects Proprietary Defects

Fewer Than 100000 040 051

100000 to 499999 060 066

500000 to 1 million 044 098

More than 1 million 075 066

Average Across Projects 069 068

Source Coverity ( wwwcoveritycom )

However the debate over open- versus closed-source quality continues Many OSS proponents

vehemently maintain that typical OSS solutions have fewer overall defects than closed-source

alternatives This assumption was captured in Eric Raymonds seminal discussion of the OSS

model The Cathedral and the Bazaar there he proposed that Given a large enough beta-testerand co-developer base almost every problem will be characterized quickly and the fix obvious to

someone In reality we find that the quality of open source is driven more by individual project

governance and best practices than any factor inherent in the model itself moreover open-source

quality is driven by virtually the same factors that are present in a closed-sourced project including

Community effectiveness

Project maturity

Defect tracking and resolution discipline

Planned product road maps versus ad hoc coding Robust IP governance

Security audits

The key differentiated benefit within the OSS model is its potential to draw a far greater number of

eyes on the code than a closed effort with a small team This creates the potential for wider and

more-in-depth peer review and investigation before the code is released however these

Gartner Inc | G00259749 Page 3 of 6

892019 Gartner (20140503) - Road Map for Opensource Succes Understanding Quality and Security

httpslidepdfcomreaderfullgartner-20140503-road-map-for-opensource-succes-understanding-quality 46

advantages are entirely theoretical unless the project actually executes these practices In other

words there is a big difference between the dynamics of an open-source project with four

developers versus one with 40 or 400 Moreover there is a massive difference between projects in

which only some have the governance and discipline to continuously monitor and manage quality

through coding standards expert committer stewardship code reviews and security audits

Dont Assume OSS Is Automatically More Secure Than Closed Alternatives

In todays threat and vulnerability landscape designing secure IT solutions and discovering security

vulnerabilities in source code is a specialized and complex skill set You cant assume that any

arbitrary OSS community has these skills in every scenario Moreover one could argue with good

cause that the open nature of OSS projects may even create lower barriers to entry for security

vulnerabilities when their guard dog security best practices are insufficient The monetization

radically advanced market for and technology of vulnerability research now results in closed-source

products being subjected to extensive examination Fuzzing zero-day intrusion protection system

signatures and the regularization of vulnerability management (eg Microsoft Tuesday) have

brought increased visibility and pressure for closing vulnerabilities in closed-source products and

projects

Much like the issue of general code quality effective security results from specific project

governance and development best practices A robust and effective developer community that

includes specialized resources paying close attention to security-related issues can have a marked

impact on security that goes beyond the efforts of any single vendor team

Do Evaluate the Quality and Security of OSS Technologies on a Case-by-Case Basis

Dont just assume that OSS enjoys automatic advantages related to security concerns over closed-

source alternatives Rather view source code security as a complex topic that must be assessed ona case-by-case basis in specific projects

Any reputable open-source project will have transparent and automated issue (ie bug) tracking

(eg Bugzilla Jira ) accessible to the public Be very circumspect regarding any project that does

not Use these issue-tracking systems (virtually always accessed via a Web browser) to view the

number of outstanding issues and their status and criticality Use their reporting features to view

issue trends being tracked over time

In addition leverage an open-source project-tracking website such as Ohloh to examine project

trends (eg commits over time and contributors) that can provide insight into a projects overall

maturity and momentum (see Figure 1) You can also refer to individual project forums to finddiscussions on outstanding issues project direction and potential conflicts or stress points among

the community members

Page 4 of 6 Gartner Inc | G00259749

892019 Gartner (20140503) - Road Map for Opensource Succes Understanding Quality and Security

httpslidepdfcomreaderfullgartner-20140503-road-map-for-opensource-succes-understanding-quality 56

Figure 1 Example Summary Data From Ohloh for Firefox

Source Ohloh (March 2014)

Gartner Recommended Reading

Some documents may not be available as part of your current Gartner subscription

Toolkit How to Cut Through the Hype Surrounding Open-Source Software

Hype Cycle for Open-Source Software 2013

Predicts 2013 Open Computing Increases Influence on Product Selection and Application

Architecture

Pervasive Use of Open-Source Software Creates a Management Nightmare or Sweet Spot of

Opportunity

Understand the Challenges of Open-Source Software

Drivers and Incentives for the Wide Adoption of Open-Source Software

Gartner Inc | G00259749 Page 5 of 6

892019 Gartner (20140503) - Road Map for Opensource Succes Understanding Quality and Security

httpslidepdfcomreaderfullgartner-20140503-road-map-for-opensource-succes-understanding-quality 66

GARTNER HEADQUARTERS

Corporate Headquarters

56 Top Gallant RoadStamford CT 06902-7700

USA +1 203 964 0096

Regional Headquarters

AUSTRALIA BRAZILJAPANUNITED KINGDOM

For a complete list of worldwide locationsvisit httpwwwgartnercomtechnologyaboutjsp

copy 2014 Gartner Inc andor its affiliates All rights reserved Gartner is a registered trademark of Gartner Inc or its affiliates This

publication may not be reproduced or distributed in any form without Gartnerrsquos prior written permission If you are authorized to accessthis publication your use of it is subject to the Usage Guidelines for Gartner Services posted on gartnercom The information contained

in this publication has been obtained from sources believed to be reliable Gartner disclaims all warranties as to the accuracy

completeness or adequacy of such information and shall have no liability for errors omissions or inadequacies in such information Thispublication consists of the opinions of Gartnerrsquos research organization and should not be construed as statements of fact The opinions

expressed herein are subject to change without notice Although Gartner research may include a discussion of related legal issues

Gartner does not provide legal advice or services and its research should not be construed or used as such Gartner is a public companyand its shareholders may include firms and funds that have financial interests in entities covered in Gartner research Gartnerrsquos Board of

Directors may include senior managers of these firms or funds Gartner research is produced independently by its research organization

without input or influence from these firms funds or their managers For further information on the independence and integrity of Gartnerresearch see ldquoGuiding Principles on Independence and Objectivityrdquo

Page 6 of 6 Gartner Inc | G00259749

892019 Gartner (20140503) - Road Map for Opensource Succes Understanding Quality and Security

httpslidepdfcomreaderfullgartner-20140503-road-map-for-opensource-succes-understanding-quality 66

GARTNER HEADQUARTERS

Corporate Headquarters

56 Top Gallant RoadStamford CT 06902-7700

USA +1 203 964 0096

Regional Headquarters

AUSTRALIA BRAZILJAPANUNITED KINGDOM

For a complete list of worldwide locationsvisit httpwwwgartnercomtechnologyaboutjsp

copy 2014 Gartner Inc andor its affiliates All rights reserved Gartner is a registered trademark of Gartner Inc or its affiliates This

publication may not be reproduced or distributed in any form without Gartnerrsquos prior written permission If you are authorized to accessthis publication your use of it is subject to the Usage Guidelines for Gartner Services posted on gartnercom The information contained

in this publication has been obtained from sources believed to be reliable Gartner disclaims all warranties as to the accuracy

completeness or adequacy of such information and shall have no liability for errors omissions or inadequacies in such information Thispublication consists of the opinions of Gartnerrsquos research organization and should not be construed as statements of fact The opinions

expressed herein are subject to change without notice Although Gartner research may include a discussion of related legal issues

Gartner does not provide legal advice or services and its research should not be construed or used as such Gartner is a public companyand its shareholders may include firms and funds that have financial interests in entities covered in Gartner research Gartnerrsquos Board of

Directors may include senior managers of these firms or funds Gartner research is produced independently by its research organization

without input or influence from these firms funds or their managers For further information on the independence and integrity of Gartnerresearch see ldquoGuiding Principles on Independence and Objectivityrdquo

Page 6 of 6 Gartner Inc | G00259749