482612main 2010 Tuesday 4 Cmmi Kulpa Margaret Executive Overview of CMMI v1.4b
-
Upload
arundhathinair -
Category
Documents
-
view
20 -
download
0
description
Transcript of 482612main 2010 Tuesday 4 Cmmi Kulpa Margaret Executive Overview of CMMI v1.4b
Empowering People through Process
© AgileDigm, Inc.
Executive Overview of CMMI®
18 Aug 2010NASA IT SummitMargaret Kulpa
Chief Operating Officer of AgileDigm, Inc.
www.agiledigm.com
® CMMI is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.
© 2010 Executive Overview of CMMI
2
Copyright NoticesThe presentation material in this seminar is copyrighted by AgileDigm, Incorporated, 2010. For further information, please contact:
AgileDigm, Inc.11 Twelve Oaks TrailOrmond Beach, Florida 32174 U.S.A.Phone: +1.386.673.1384Email: [email protected]
Terms like these are often used in the following material:CMMI
SCAMPIsm and SCAMPI Lead Appraisersm
CMMI is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.SM SCAMPI and SCAMPI Lead Appraiser are service marks of Carnegie Mellon University.
© 2010 Executive Overview of CMMI
3
Who is AgileDigm?AgileDigm, Incorporated is a US based corporation working with organizations around the world
– providing consulting, training, and appraisals for organizations that build software-intensive systems
– Software Engineering Institute (SEI) Partner– Scrum Training Institute (STI) Partner
Authors of Interpreting the CMMI, best selling book on using the CMMI. Unique experience working with one of the only CMMI Maturity Level 5 Agile companies.
CMMI is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.SM SCAMPI and SCAMPI Lead Appraiser are service marks of Carnegie Mellon University.
© 2010 Executive Overview of CMMI
4
What is the CMMI?• CMMI is an acronym for Capability Maturity Model
Integration. • Approach used by organizations to improve.• CMMI is a collection of best practices
– Over 100 organizations formally assisted in its development, including NASA
• There is a wide range of freedom in how an organization decides to embrace the CMMI. You can –– Go slow or fast– Choose the areas you need to focus on– Choose the appraisal mode you are comfortable with (SCAMPI -
A,B,C, High Maturity)
© 2010 Executive Overview of CMMI
5
Three Constellations• CMMI for Development
– Development and maintenance of products– Integrates software engineering, systems engineering,
collaborative teams, acquisition from the supplier’s side – Not just for software development! – Can be used for developing complex systems (healthcare,
bridges, valves, cleanroom technology)• CMMI for Services
– Organizations that provide services (from taxis to call centers)• CMMI for Acquisition
– Acquiring products and services • Common Process Areas are embedded within all three• This presentation focuses on CMMI for Development.
© 2010 Executive Overview of CMMI
6
CMMI Concepts
What the CMMI is not:–CMMI is not a process, standard, directive,
regulation, or description.–Not a list of “Shalls” that might require big
changes in how you do your work.You can make the CMMI in your organization as intrusive and painful as you want --- OR NOT! It’s up to you.
© 2010 Executive Overview of CMMI
7
Does the CMMI tell me how to do stuff?
• NO!• The CMMI documents best practices from many
organizations.• The CMMI describes WHAT to do, not HOW to
do it.• Organizations need to determine how to apply
the best practices of the CMMI to how the organization works in order to improve how stuff gets done.
© 2010 Executive Overview of CMMI
8
Why Should I Use the CMMI?• Structures the effort (Where do I start? How
do I know when I am done?)• Helps identify an organization’s improvement
objectives and priorities• Provides benefit from a larger community’s
experience• Has an an appraisal methodology to diagnose
the state of improvement efforts.
© 2010 Executive Overview of CMMI
9
Why Should I Use the CMMI?• CMMI focuses on “Process”• A process is a set of practices performed
to achieve a given purpose (from CMMI Glossary)
• Four major elements of process are tools, methods, materials, and people– Major determinants of product cost, schedule,
and quality are tools, methods, materials, and people.
© 2010 Executive Overview of CMMI
10
CMMI Performance Results Summary
Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006-TR-004
This is the summary of improvements from all sources collected regardless of maturity levels.
Improvement Nbr. of Data PointsCategory Median Lowest Highest
Cost 34% 3% 87% 29
Schedule 50% 2% 95% 22
Productivity 61% 11% 329% 20
Quality 48% 2% 132% 34
Customer Satisfaction
14% -4% 55% 7
Return on Investment
4.0 : 1 1.7 : 1 27.7 :1 22
© 2010 Executive Overview of CMMI
11
Cost – Benefits and Impact• Cost metrics (and reductions in cost) defined as:
– Cost of delivery – Cost of quality/Cost of poor quality
– Overhead rate – Costs of rework/Defect find & fix cost
– Software unit costs – Number/Cost of process staff– Variation in cost performance index (CPI)
• Organizations have reported improved– Budget estimation accuracy – Average in cost performance index (CPI)
Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006-TR-004
Cost example on next slide – cost per function reduced by 52%
© 2010 Executive Overview of CMMI
12
Cost Savings at DB Systems GmbH
Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006-TR-004
© 2010 Executive Overview of CMMI
13
Schedule – Benefits and Impact• Schedule metrics (and reductions) defined as:
– Variation in schedule – Schedule performance index (SPI)– Number of days late – Days variance from plan– Slippage of project delivery
• Organizations have reported improved or increased– Cycle time – Proportion of milestones met– Average SPI – Estimation accuracy
• Example from NCR Corporation (on next slide)– Average days schedule variance reduced from approximately
130 days to less than 20 days one year after reaching CMMI maturity level 2
Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006-TR-004
© 2010 Executive Overview of CMMI
14
Schedule Improvement at NCR
Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006-TR-004
© 2010 Executive Overview of CMMI
15
Productivity – Benefits and Impact• Productivity measures (and improvements)
defined as: – Time comparisons by build – Source statements per
month – Lines of code per labor hour – Number of releases per
year– Testing rates – Software production– Function points (FP) per full time equivalent (FTE) staff
• Example from IBM Australia (on next slide)– Over 20 percent improvement in account productivity as the
organization moved from SW-CMM maturity level 3 toward CMMI maturity level 5.
Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006-TR-004
© 2010 Executive Overview of CMMI
16
Productivity Improvement at IBM Australia
Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006-TR-004
Account Productivity (FP/FTE)
© 2010 Executive Overview of CMMI
17
Quality – Benefits and Impact
Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006-TR-004
• “Quality” most frequently defined as reductions in numbers of defects.
• Measurements include:– Defect counts by phase of injection– Defect counts by phase of discovery– Total defect density
• Example from IBM Australia (on next slide)– 40 percent reduction in overall production and over 80 percent
reduction in Severity 1 problems, as the organization moved from SW-CMM maturity level 3 toward CMMI maturity level 5
© 2010 Executive Overview of CMMI
18
Quality Improvement at IBM Australia
Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006-TR-004
© 2010 Executive Overview of CMMI
19
Customer Satisfaction – Benefits and Impact
• Customer satisfaction most frequently defined by the results of customer surveys.– Award fees are sometimes used as surrogate
measures.• An example from Lockheed Martin Management
and Data Systems– Increased award fees by 55 percent
Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006-TR-004
© 2010 Executive Overview of CMMI
20
ROI – Benefits and Impact
Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI-2006-TR-004
• ROI defined by cost avoidance measures and improvements including:– Rework avoided due to fewer defects– Improved productivity– Increased revenue due to shorter cycle times
• Example from Raytheon Corporation, anonymous site– 6:1 ROI in a CMMI maturity level 3
organization
© 2010 Executive Overview of CMMI
21
From personal experience …• Ensuring I am working on the latest (correct) version of
the code• Running tests that prove requirements are satisfied
(not that the system is just not blowing up)• Predicting the number of projects to start up and the
number of people I need ahead of time• Increasing communication throughout the organization
(and hearing things I sometimes did not want to hear)• Not making phone calls to my QA person while she
was in labor.
© 2010 Executive Overview of CMMI
22
CMMI StructureCMMI is a collection of best practices
– These “best practices” reside in 22 focus areas called “Process Areas.”
– Each process area has specifically stated goals that have to be met in order to have “maturity” or “capability” in the process area.
– Each process area has specific practices, subpractices, typical work products (outputs), examples, and other informative material
– Most of the goals are specific to the process area, but there are some generic or “global” goals that are common to all of the Process Areas like:
• Provide Resources for this area• Train People for this area• Identify Stakeholders for this area
Let’s look at the book …
© 2010 Executive Overview of CMMI
23
Structure of the CMMI• Maturity Level or Capability Level
– Specific and Generic Goals• Specific and Generic Practices
– Subpractices, Typical Work Products, Examples• Two types of Goals and Practices
– We will see these examples in the Example Process Area ☺
– Specific Goals pertain to one Process Area (SG1 in Project Planning is “Establish Estimates”)
– Specific Goals have associated Specific Practices (For SG1 above, SP 1.4 is Determine Estimates of Effort and Cost
– There are also Generic Goals and Generic Practices that pertain to all of the Process areas throughout the model. They are “Global.”
© 2010 Executive Overview of CMMI
24
Example Process Area (Maturity Level 3)
Decision Analysis and ResolutionThe purpose of Decision Analysis and Resolution (DAR) is to analyze possible decisions using a formal evaluation process that evaluates identified alternatives against established criteria.
The CMMI then continues with half a page of Introductory Notes that further describe what this Process Area is about.Note: Underlines and emphasis added by the presenter here and in the following slides. I have omitted some of the Informative Material.
Informative material is very useful in interpreting the intent of the process area/practice/subpractice and should not be ignored.
© 2010 Executive Overview of CMMI
25
Decision Analysis and Resolution• SG1: Evaluate Alternatives
– SP1.1 Establish Guidelines for Decision Analysis– SP1.2 Establish Evaluation Criteria– SP1.3 Identify Alternative Solutions– SP1.4 Select Evaluation Methods ***– SP1.5 Evaluate Alternatives– SP1.6 Select Solutions
Note: Other Process Areas may have more Specific Goals and more Specific Practices. There is more informative material presented in the book.
***This is the practice used in the following example.
SG – Specific GoalSP – Specific Practice
© 2010 Executive Overview of CMMI
26
Decision Analysis and ResolutionSP 1.4 Select the Evaluation MethodsTypical Work Products:1. Selected evaluation methods
Subpractices
1. Select the methods based on the purpose for analyzing a decision and on the availability of the information used to support the method.
Typical evaluation methods include the following: – Modeling and simulation– Engineering studies– Manufacturing studies– Cost studies– Business opportunity studies
© 2010 Executive Overview of CMMI
27
Decision Analysis and Resolution– Surveys– Extrapolations based on field experience and prototypes– User review and comment– Testing– Judgment provided by an expert or group of experts
2. Select evaluation methods based on their ability to focus on the issues at hand without being overly influenced by side issues.
3. Determine the measures needed to support the evaluation method. – Consider the impact on cost, schedule, performance, and risks.
© 2010 Executive Overview of CMMI
28
Generic “Common” Goals and Practices for All Process Area
Generic Goal 2 Institutionalize a Managed ProcessGP 2.1 Establish an Organizational PolicyGP 2.2 Plan the ProcessGP 2.3 Provide ResourcesGP 2.4 Assign ResponsibilityGP 2.5 Train PeopleGP 2.6 Manage ConfigurationsGP 2.7 Identify and Involve Relevant StakeholdersGP 2.8 Monitor and Control the ProcessGP 2.9 Objectively Evaluate AdherenceGP 2.10 Review Status with Higher-Level Management
Generic Goal 3 Institutionalize a Defined Process GP 3.1 Establish a Defined ProcessGP 3.2 Collect Improvement Information
GG – Generic Goal GP -Generic Practice
© 2010 Executive Overview of CMMI
29
Two representations of the model• Staged Representation (or “View”):
– Most like other currently accepted models– Focuses on all process areas contained within a level– Used for Maturity Level ratings for contract awards– Start with any Process Area within a level.
• Continuous Representation (or “View”):– Was designed to focus on specific, individual process areas– Process areas are arranged into functional categories– Start with any Process Area.
• Staged is the most “popular” CMMI representation.– When your customers ask for a level or when an organization
says “We are Level N”, they are referring to a Maturity Level of the staged representation.
© 2010 Executive Overview of CMMI
30
Not to Worry• The information contained is the same in
both!• The only difference is in your approach for
implementation and appraisal.
© 2010 Executive Overview of CMMI
31
Process Areas by Maturity Level (Staged)
1 Initial
Requirements Management Project PlanningProject Monitoring and ControlSupplier Agreement ManagementMeasurement and AnalysisProcess and Product Quality AssuranceConfiguration Management
2 Managed
Requirements Development Technical SolutionProduct Integration VerificationDecision Analysis and Resolution Validation Organizational Process FocusOrganizational Process Definition +IPPDOrganizational TrainingIntegrated Project Management +IPPDRisk Management
3 Defined
Organizational Process PerformanceQuantitative Project Management
4 Quantitatively Managed
Organizational Innovation and DeploymentCausal Analysis and Resolution
5 OptimizingProcess AreasMaturity Level
1 Initial
Requirements Management Project PlanningProject Monitoring and ControlSupplier Agreement ManagementMeasurement and AnalysisProcess and Product Quality AssuranceConfiguration Management
2 Managed
Requirements Development Technical SolutionProduct Integration VerificationDecision Analysis and Resolution Validation Organizational Process FocusOrganizational Process Definition +IPPDOrganizational TrainingIntegrated Project Management +IPPDRisk Management
3 Defined
Organizational Process PerformanceQuantitative Project Management
4 Quantitatively Managed
Organizational Innovation and DeploymentCausal Analysis and Resolution
5 OptimizingProcess AreasMaturity Level
© 2010 Executive Overview of CMMI
32
Process Areas by Category (Continuous)Process Management Project ManagementOrganizational Process FocusOrganizational Process Definition (with IPPD)
Organizational TrainingOrganizational Process PerformanceOrganizational Innovation & Deployment
Project PlanningProject Monitoring and ControlSupplier Agreement ManagementIntegrated Project Management (with IPPD)
Risk ManagementQuantitative Project Management
Engineering SupportRequirements ManagementRequirements DevelopmentTechnical SolutionProduct IntegrationVerificationValidation
Measurement & AnalysisProcess and Product Quality AssuranceConfiguration ManagementDecision Analysis & ResolutionCausal Analysis and Resolution
© 2010 Executive Overview of CMMI
33
CMMIMaturity Levels
Level 1 – InitialLevel 2 – ManagedLevel 3 – DefinedLevel 4 – Quantitatively ManagedLevel 5 - Optimizing
Capability LevelsLevel 0 - IncompleteLevel 1 – PerformedLevel 2 – ManagedLevel 3 – DefinedLevel 4 – Quantitatively ManagedLevel 5 - Optimizing
“Continuous”“Staged”
Process ManagementProject ManagementEngineeringSupport
Process Areas (22)
Process Areas (22) Active PA Subset
SpecificGoals
SpecificPractices
GenericGoals
GenericPractices
It’s a Model,not a specification
It can be hierarchical.
It’s highly detailed..
It’s complex, butnothing is
complicated.
There are no“shalls”.
Appraisals yield
Ratings Realmof
Appraisals
© 2010 Executive Overview of CMMI
34
What is a SCAMPIsm Appraisal?• SCAMPIsm – Standard CMMI Appraisal Method for
Process Improvement– Used for baselining current behaviors to begin
improvement activities or see into ongoing improvement activities
– To assign a formal rating for contract award or fees (if requested by appraisal sponsor)
– Several appraisal methods (class A, B, C, plus High Maturity)
– Uses the CMMI as reference model.• SCAMPI goal is to provide an accurate picture of your
organization relative to the reference model.
© 2010 Executive Overview of CMMI
35
Reported Appraisals
CMMI® For Development SCAMPIsm Class A Appraisal Results 2009 End-Year Update
Red indicates countries with reported SCAMPI Class A Appraisals
© 2010 Executive Overview of CMMI
36
Maturity Profile – End of Year 2009• The percentage of organizations at each maturity level
based on most recent appraisals of 2753 organizations.
Maturity Profile - CMMI/SCAMPI Results published March 2010
0.5%5.2%1.3%
25.2%
63.0%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
Level 1 Level 2 Level 3 Level 4 Level 5
© 2010 Executive Overview of CMMI
37
Appraisal Classes A, B, and C
© 2010 Executive Overview of CMMI
38
Appraisal Team (SCAMPI “A”)• Minimum team size is four - A Lead Appraiser and
three additional team members. • At least one team member must be from the
appraised organization.• All team members must be trained in the official
CMMI and SCAMPI courses• Conflicts of interest must be avoided
– Exclude process authors and individuals directly impacted by the outcome of the appraisal
• Team members cannot be managers of any of the selected projects or groups
• Appraisal sponsors cannot be team members.
© 2010 Executive Overview of CMMI
39
What does the SCAMPI Team do?
• Reviews an organization’s documentation– Processes, procedures, policies, status reports,
meeting minutes• Conducts interviews
– To ensure the documentation is actually used• Maps results to the CMMI
– Practices, Goals, Process Areas• Assigns ratings (if requested and allowed by
the type of SCAMPI).
© 2010 Executive Overview of CMMI
40
What does the Team NOT do?• Does not go looking for the documentation
– Easter egg hunts are not allowed!• Does not judge the “goodness” of the
process• Does not judge how well you are doing
your job.
© 2010 Executive Overview of CMMI
41
PPQA Example EvidenceID Goal/Practice Potential ArtifactsSG1 Objectively Evaluate Processes and Work ProductsSP1.1 Objectively Evaluate Processes QA audit report
SP1.2 Objectively Evaluate Work Products and Services
QA audit report
SG2 Provide Objective InsightSP2.1 Communicate and Ensure
Resolution of Noncompliance Issues
QA audit report
SP2.2 Establish Records QA Reports, Status of Corrective Actions, Report of Quality Trends
© 2010 Executive Overview of CMMI
42
Simplified Point of the SCAMPI:• Do you have a (documented) process?• Do you follow it?• How closely does it map to the CMMI?
© 2010 Executive Overview of CMMI
43
Conclusion (1of 2)
• Organizations can and have benefited from using the CMMI.
• Implement the CMMI to support your organization’s business objectives, goals, and culture.
• “Engage your brain.” Interpretation of the information must be used.
• Do not ignore the informative information and the subpractices. They help to understand the intent of the practice or Process Area.
• Use a SCAMPI to guide your implementation, measure your progress, and verify your achievements.
© 2010 Executive Overview of CMMI
44
Conclusion (2 of 2)
• My favorite websites for more information:– www.sei.cmu.edu (main SEI website)– seir.sei.cmu.edu (contains helpful
presentations and articles from the community)
© 2010 Executive Overview of CMMI
45
Questions?• Time for some questions?
© 2010 Executive Overview of CMMI
46
Contact Information
Margaret KulpaChief Operating OfficerAgileDigm, Inc.11 Twelve Oaks TrailOrmond Beach, FL 32174 USA+1 386 673 [email protected]
The presentation material in this seminar is copyrighted by AgileDigm, Incorporated, 2010. For further information, please contact:
© 2010 Executive Overview of CMMI
47
Backup Slides
© 2010 Executive Overview of CMMI
48
Combined View of Staged and Continuous
© 2010 Executive Overview of CMMI
49
Maturity Levels (MLs)• A maturity level signifies the level of
performance that can be expected from an organization. There are 5 maturity levels. Following are the key features of each maturity level (ML):– ML 1 (Initial) – ad hoc processes– ML 2 (Managed) – basic project management system – ML 3 (Defined) – process standardization – ML 4 (Quantitatively Managed) – quantitative
management “by the numbers”– ML 5 (Optimizing) – continuous process improvement
ML 2 provides a foundation for ML 3, ML 3 for ML4, and ML 4 for ML 5.
© 2010 Executive Overview of CMMI
50
Appraisal Sponsor• SCAMPI appraisals require an appraisal sponsor.
– Usually this is a senior manager.– The appraisal sponsor is responsible for
• Verifying that the appraisal team leader has the appropriate experience, knowledge, and skills to lead the appraisal.
• Ensuring that the appropriate organizational units (projects and support) participate in the appraisal.
• Supporting the required confidentiality and non-attribution rules.
– Some appraisals have multiple sponsors.
© 2010 Executive Overview of CMMI
51
“Typical” Appraisal Roadmap
Train Team• Train Appraisal Team• Begin PIID creation• Finalize Plan
Readiness Review
Plan• Plan SCAMPI activities• Monitor and adjust plan
Appraisal Onsite• Conduct Appraisal
• Organization Develops PIID documents
May 2010
August 2010 (1 – Week)
October 2010 (1 – Week)
December 2010 (1 – 3 Weeks)Reviewing PIIDs is a key focus of the required readiness review
© 2010 Executive Overview of CMMI
52
Presenter – Margaret Kulpa• Chief Operating Officer of AgileDigm,
Incorporated• She is the primary author of the book
Interpreting the CMMI: A Process Improvement Approach, Second Edition, Auerbach (2008) which details how to identify, define, and improve business, systems and software processes on both enterprise-wide and project levels.
• Ms. Kulpa has developed and taught courses in software quality assurance, project management, peer reviews, requirements development and management, process baselines and modeling, safety and security, and other software engineering-related subjects. She has developed courses in CMMI and has served as an external member of numerous SCAMPIsm Class A Appraisals.