CMMI vs. ISO

11 April 2007

CMMI vs. ISODavid S. Craft CIRM, PMP

11 April 2007

CMMI vs. ISO, Sarbanes Oxley

Agenda

ProcessISOCMMI

11 April 2007


.

The Process Management PremiseThe quality of a system is influenced by the quality of the process used to acquire, develop, and maintain it, the analysis and forethought that goes into an architecture that supports business goals and requirements, and the training provided to teams involved in the project. Using proven methods for process and product quality, software success is predictable and achievable, and failure is avoidable.

Once coding starts, teams trained in mature software engineering processes can remove defects early, when defect removal is 10 to 100 times less costly than it is during test. This dramatically reduces test costs and only marginally increases costs upstream

11 April 2007


ProcessTo Develop Software and Systems You Need A Process

So what is a process:1. A systematic series of actions directed to some end2. A continuous action, operation or series of changes taking place

in a definite manner3. A series of actions, changes or functions bringing about a result4. A series of operations performed in the making or treatment of a

product5. Process or processing typically describes the action of taking

something through an established and usually routine set of procedures or steps to convert it from one form to another (such as processing paperwork to grant a loan, processing milk into cheese, converting computer data from one form to another, etc.

11 April 2007


Common MisconceptionsI don’t need defined processes I have:

– Really good people– Advanced Technology– An experienced manager

Defined Processes:– Interfere with creativity– Equals bureaucracy + regimentation– Isn’t needed when building prototypes– Is only useful on large projects– Hinders agility in fast moving projects– Costs too much

11 April 2007


Commitments consistently missed• Late deliveries• Last minutes crunches• Spiraling costs

No management visibility into progress• You’re always being surprised

Quality Problems• Too much rework• Functions do not work correctly• Customer complaints after delivery

Poor Moral• People frustrated• Is anyone in charge?

Symptoms of Process Failure

11 April 2007


Why We Need Structured Processes

Estimating (History)• Scope• Cost• Time • Tools

Deliver the Product to Estimate (Visibility)• Time• Cost• Quality

Handling/Controlling Changes• Planned• Unplanned• Scope Creep

11 April 2007


• Organizations and governments worldwide will spend about $1 trillion this year on IT projects. Recent data suggested only about 35 percent of those projects are likely to be completed on time and on budget with all their originally specified features and functions. Many projects, perhaps 20 percent, will be abandoned, often after multimillion-dollar investments—and the biggest projects will fail most often.

• One well-documented $170 million software failure was blamed on a lack of defined requirements in the original contract; a lack of software engineering, program, and contract management skills; and underestimates of the complexity of interfacing the new system with legacy systems, addressing security needs, and establishing an enterprise architecture.

• Other software-development failures have brought down entire companies, such as the $5 billion drug-distribution firm in Texas that declared bankruptcy as a result of a poorly implemented resource planning system.

Why We Need Standard Processes

11 April 2007


How to Achieve Quality Processes

ISO Standards

CMMI Models

11 April 2007


ISOInternational Standards Organization

11 April 2007


Meet The International Organization forStandardization (ISO)

• ISO – a nongovernmental organization – is a network of the national standards bodies of some 160 countries, one per country, from all regions of the world, including developed, developing and transitional economies.

• ISO is a global network that identifies what International Standards are required by business, government and society, develops them in partnership with the sectors that will put them to use, adopts them by transparent procedures based on national input and delivers them to be implemented worldwide.

11 April 2007


What are standards?

Standards are documented agreements containing technical specifications or other precise criteria to be used consistently as rules, guidelines, or definitions of characteristics, to ensure that materials, products, processes and services are fit for their purpose.

For example, the format of the credit cards, phone cards, and "smart" cards that have become commonplace is derived from an ISO International Standard. Adhering to the standard, which defines such features as an optimal thickness (0,76 mm), means that the cards can be used worldwide.

11 April 2007


• The ISO 9000 family addresses various aspects of quality management and contains some of ISO’s best known standards. The standards provide guidance and tools for companies and organizations who want to ensure that their products and services consistently meet customer’s requirements, and that quality is consistently improved.

• There are many standards in the ISO 9000 family, including:• ISO 9001:2008 - sets out the requirements of a quality management system• ISO 9000:2005 - covers the basic concepts and language• ISO 9004:2009 - focuses on how to make a quality management system more

efficient and effective• ISO 19011:2011 - sets out guidance on internal and external audits of quality

management systems

The ISO 9000:2008 standard has been implemented by over 1,000,000 organizations in 176 countries

ISO 9000 - Quality management

11 April 2007


• Customer Focus• Leadership• Involvement of People• Process Approach• System Approach to Management• Continual Improvement• Factual Approach to Decision Making• Mutually Beneficial Supplier Relationships

ISO 9000:2008 Key Principles

11 April 2007


Quality System Documentation

QualityManual

Level 1Defines

Approach andResponsibility

ProceduresLevel 2Defines

Who, What, When

Work/JobInstructions

Level 3Answers

How

Records/Documentation

Level 4Results: shows that

the system is operating

11 April 2007


ISO 9001:2000 Structure4. Quality Management System

4.1 General requirements4.2 Document requirements

5. Management Responsibility

5.1 Management commitment

5.2 Customer focus5.3 Quality policy5.4 Planning5.5 Responsibility, authority,

communication5.6 Management review

6. Resource Management6.1 Provision of resources6.2 Human resources6.3 Infrastructure6.4 Work environment

7. Product realization7.1 Planning of product realization7.2 Customer-related processes7.3 Design and development7.4 Purchasing7.5 Production and service provision7.6 Control of monitoring and

measuring devices

8. Measurement, Analysis & Improvement8.1 General8.2 Monitoring and measurement8.3 Control of nonconforming product8.4 Analysis of data8.5 Improvement

11 April 2007


5.2 Customer Focus“Top management shall ensure that customer requirements are determined

and are met with the aim of enhancing customer satisfaction.”

Standard Examples

11 April 2007


Advantages of ISO 9000• Increased marketability.• Reduced operational expenses• Better management control• Increased customer satisfaction• Improved internal communication• Improved customer service• Reduction of product-liability risks• Attractiveness to investors

11 April 2007


• Owners and managers do not have an adequate understanding of the ISO 9000 certification process or of the quality standards themselves

• Funding for establishing the quality system is inadequate• Heavy emphasis on documentation• Length of the process

Disadvantages of ISO Implementation

11 April 2007


ISO’s Impact In The Global EconomyISO 9001:2000 is now firmly established as the globally accepted standard for providing assurance about the quality of goods and services in supplier-customer relations.

The positive roles played in globalization by ISO’s standards for quality and environmental management systems include the following:

• a unifying base for global businesses and supply chains – such as the automotive and oil and gas sectors

• a technical support for regulation – as, for example, in the medical devices sector

• a tool for major new economic players to increase their participation in global supply chains, in export trade and in business process outsourcing;

• a tool for regional integration – as shown by their adoption by new or potential members of the European Union

In the rise of services in the global economy – nearly 33 % of ISO 9001:2000 certificates in 2005 went to organizations in the service sectors.

11 April 2007


CMMICapability Maturity Model Integrated

11 April 2007


• SEI is a federally funded research and development center sponsored by the U.S. Department of Defense and operated by Carnegie Mellon University in Pittsburgh, Pa.

• SEI helps advance software engineering principles and practices and serves as a national resource in software engineering, computer security, and process improvement.

• SEI works closely with defense and government organizations, industry, and academia to continually improve software-intensive systems.

• Its core purpose is to help organizations improve their software engineering capabilities and develop or acquire the right software, defect free, within budget and on time, every time.

• SEI transitions its technologies to the global software engineering community through its public courses, conferences, technical reports, and Partner Network.

Software Engineering Institute (SEI)

11 April 2007


Meet CMMICMMI® (Capability Maturity Model® Integration) models are collections of best practices that help organizations to improve their processes. These models provides a comprehensive integrated set of guidelines for developing products and services. The SEI’s body of work in technical and management practices is focused on developing software right the first time, which results not only in higher quality, but also predicable and improved schedule and cost

There are three CMMI models• CMMI-DEV – Systems and Software Engineering• CMMI -ACQ – Acquiring Products, Services or Outsourcing• CMMI -SVC – For service organizations

CMMI helps you to meet your organizations business objectives and improve performance.

11 April 2007


A structured collection of elements describing characteristics of effective processes

A maturity model provides:• A place to start• The benefits of companies prior experience• A common language and shared vision• A framework for prioritizing actions• A way to define what improvement means for your organization

The model can be used to assess where your organization is against other organizations

What is a Maturity Model

11 April 2007


CMMI is organized as a process framework clustering related practices into process areas that, when performed collectively, satisfy a set of goals. It requires that you define specific practices to meet specific goals but does not define how they are to be implemented.

The CMMI provides two representations – staged and continuous, each containing 25 Process Areas (PA).

CMMI Organization

11 April 2007


Process AreasRequirements Management Organizational Process Definition

Project Planning Organizational Training

Project Monitoring & Control Integrated Project Management

Supplier Agreement Management Risk Management

Measurement & Analysis Integrated Teaming

Process & Product Quality Assurance Integrated Supplier Management

Configuration Management Decision Analysis & Resolution

Requirements Development Organizational Environment for Integration

Technical Solution Organizational Process Performance

Product Integration Quantitative Project Management

Verification Organizational Innovation & DeploymentValidation Causal Analysis & Resolution

Organizational Process Focus

11 April 2007


CMMI Standard Example SP 1.2 Specify Measures Specify measures to address measurement objectives. Measurement objectives are refined into precise, quantifiable measures. Measurement of project and organizational work can typically be traced to one or more measurement information categories. These categories include the following:

• schedule and progress• effort and cost• size and stability • quality.

Measures can be either base or derived. Data for base measures are obtained by direct measurement. Data for derived measures come from other data, typically by combining two or more base measures. Examples of commonly used base measures include the following:

• Estimates and actual measures of work product size (e.g., number of pages) • Estimates and actual measures of effort and cost (e.g., number of person hours) • Quality measures (e.g., number of defects by severity) • Information security measures (e.g., number of system vulnerabilities identified) • Customer satisfaction survey scores

11 April 2007


CMMI Standard Example con’tExamples of commonly used derived measures include the following:

• Earned value • Schedule performance index • Defect density • Peer review coverage • Test or verification coverage • Reliability measures (e.g., mean time to failure) • Quality measures (e.g., number of defects by severity/total number of defects) • Information security measures (e.g., percentage of system vulnerabilities mitigated) • Customer satisfaction trends

Derived measures typically are expressed as ratios, composite indices, or other aggregate summary measures. They are often more quantitatively CMMI for Development, Version 1.3 Measurement and Analysis (MA) 180 reliable and meaningfully interpretable than the base measures used to generate them. There are direct relationships among information needs, measurement objectives, measurement categories, base measures, and derived measures. This direct relationship is depicted using some common examples in Table MA.1.

11 April 2007


Continuous ViewCapability Levels

Staged View Maturity Levels

A well defined evolutionary plateau describing the organization’s capability

relative to a particular process area

A well defined evolutionary plateau of process improvement

There are six capability levels There are five maturity levelsEach level is a layer in the foundation for continuous process improvement. Thus, capability levels are cumulative (i.e., a

higher capability level includes the attributes of the lower levels).

Each level is a layer in the foundation for continuous process improvement using a

proven sequence of improvements, beginning with basic management

practices and progressing through a predefined and proven path of successive

levelsEnables comparison across and among

organizations on a process-are-by process–area basis

Provides a single rating that summarizes appraisal results and permits comparisons

across and among organizations

Capability and Maturity Models

11 April 2007


11 April 2007


Level Continuous ViewCapability Levels

Staged ViewMaturity Levels

5 Focus on continuous process improvement

Optimizing Optimizing

4 Process measured and controled

Qualitatively Managed Quantitatively

3 Process characterized for the organization and is proactive

Defined Defined

2 Process characterized for projects and is often reactive

Managed Managed

1 Performed Initial0 Incomplete

Capability and Maturity Levels

11 April 2007


11 April 2007


Evaluation & Ratngs

• This is not a certification model, but ratings may be announced and published.

• The SEI publishes ratings provided the company gives it permission.

• Formal appraisals are typically 5 – 10 days and led by SEI-authorized internal or external lead appraisers, using trained teams and a formal methods. The method is named SCAMPI (Standard CMMI Appraisal Method for Process Improvement).

11 April 2007


Benefits of CMMI

• Decreased Costs• General Dynamics Advanced Information Suystems reduced maintenance staff

costs by 64% while doubling the size of the organization• TATA consultancy Service saved $4.6 million across all development centers

• Improved On-Time Delivery• Tufts Associated Health Plans achieved 100% on time delivery of major IT

projects in a full year• GM improved the percent of milestones met from 50% to 85%

• Improved Quality • Siemens Information Systems LTD. Reduced defect density an average of 71%

in three technical areas• IBM Australia Application Management Services closed 95% of problems

within the customer-specified time frame

11 April 2007


ISO – CMMI DifferencesISO9001:2008 CMMI-DEV

An audit standard A process modelA certification tool that certifies businesses whose processes conform to the laid down standards

A set of related “best practices” derived from industry leaders and relates to product engineering and software development

Flexible and applicable to all manufacturing industries

Rigid and only extends to businesses developing software intensive systems

Specific to conformance and remains oblivious as to whether conformance is of strategic value or not

Requires ingraining processes into business needs so that processes become part of the corporate culture and do not break down under the pressure of deadlines

Provides generic guidelines for risk management

Approaches risk management as an organized and technical discipline

Customer satisfaction is an important part of the requirements

Focuses on linkage of processes to business goals, customer satisfaction is not a factor in the ranking

11 April 2007


ISO – CMMI Differences

ISO9001:2008 CMMI-DEVCustomer satisfaction is and important part of ISO requirements

Focuses on linkage of processes to business goals, customer satisfaction is not a factor in ranking

Flexible, wider in scope and not directly linked to business objectives

More focused, complex and aligned with business objectives

Registration Document No documentation

Certification audit for a 50 employee organization will be executed by 1 -12 auditors during one day

Certification audit for a 50 employee organization will be executed by 4 auditors during 4-5 days

Netta Dotan, Quality Assurance & project management, Ronkal Office Technologies

11 April 2007


Steps to Implement ISO (and CMMI)• Decide to improve your internal processes• Determine method for improvement• Plan for ISO (CMMI) and gain commitment of people, particularly

upper management. • Assign the responsibility of the implementation process to

someone (internal or external). • Train all personal in ISO requirements• Perform assessment of current processes and find the gaps• Fill the gap by revising, adding or improving the current processes

and documentation to meet ISO requirements.• Perform internal audit(s)• External audit

11 April 2007


Both require the organization be explicit about what their processes and quality systems are

Say what you do; do what you sayThe organization records and tracks data for objective analysisRequire strong management support to succeedProvide a structured and measured approach to quality improvementRequire an outside audit for “certification”Both are refined/improved over time

ISO – CMMI Similarities

11 April 2007


So What

Why Should You Care

CMMI vs. ISO

Documents

Transcript of CMMI vs. ISO