CMMI Update V1.2 and Beyond...

8/6/2019 CMMI Update V1.2 and Beyond...

1/44

2007 Carnegie Mellon University

CMMI Update

V1.2 and Beyond

Mike Phillips

Software Engineering Institute

Carnegie Mellon University

Pittsburgh, PA 15213

August 1, 2007


2/44

2CMMI Update V1.2 and Beyond...Phillips, August 1, 2007


CMMI Transition StatusAs reported to the SEI as of 6-30-07 -1

TrainingIntroduction to CMMI 70,791

Intermediate CMMI 2,549

Introduction to CMMI Instructor 504

SCAMPI A Lead Appraiser 731

SCAMPI B&C-Only Team Lead 33

Understanding CMMI High Maturity Practices 120

Authorized

Introduction to CMMI V1.2 Instructors 400

SCAMPI V1.2 Lead Appraisers 417

SCAMPI B&C V1.2Team Leads 20


3/44



Transition from V1.1 to V1.2 StatusAs reported to the SEI as of 3-31-07 -2

Introduction to CMMI StudentsRegistered for Upgrade Training 2,387

Upgrades Complete 1,695

Lead Appraisers and Instructors

Registered for Upgrade training 637

Upgrade Complete 452

Authorized v1.2 Lead Appraisers 387

Certified High Maturity Lead Appraisers 54


4/44



CMMI Adoption, Web Views

421K views/month in Q4 2006; over 24Kviews on 27 Sep 2006

Most downloaded files in Q4 2006

CMMI-DEV, V1.2

CMMI V1.2 Overview Presentation

Extreme Programming (XP), Six

Sigma, & CMMI: How They Can

Work Together

CMMI V1.2 Model Changes

Presentation


5/44



6-30-07

Intro to the CMM and CMMI Attendees(Cumulative)


6/44



Reported as of 30 June 2007

50

75

100

125

150

175

200

225

250

275

300

Number of SCAMPI v1.1/v1.2 Class AAppraisals (Conducted by Quarter)


7/44



Based on organizations reporting size data

25 or fewer

10.8%

101 to 200

19.6%

201 to 300

10.2%

76 to 100

8.8%

51 to 75

12.9%

26 to 50

13.5%

301 to 500

9.6%

501 to 1000

7.4%

1001 to 20004.8%

2000+

2.5%

Organization SizeBased on the total number of employees within the area of the organization that was appraised

1 to 100

45.9%

201 to 2000+34.5%

1680


8/44



Countries where Appraisals have beenPerformed and Reported to the SEI

Argentina Australia Austria Bahrain Belarus Belgium Brazil Canada

C hile C hin a C olo mb ia C zec h R ep ub liD en mark D om in ica n R ep ub lE gyp t F in lan dFrance Germany Hong Kon India IndonesiaIreland Israel Italy

Ja pa n K orea , R ep ub lic L atv ia M ala ys ia Ma uritiu s Me xico M oro cco N eth erla nd s

New ZealandP akistan Peru Philippines Portuga l Russia S ingaporeSlovakia

S outh Africa S pain S weden S witz erland Taiw an T hailand T urkey U nited K ing do

United StatesVietnam

Red country name: New additions with this reporting


9/44

9

CMMI Update V1.2 and Beyond...Phillips, August 1, 2007


Number of Appraisals and Maturity LevelsReported to the SEI by Country

Co u n t r y

N u m b e r o f

Ap p r a i s a l s

M a t u r i t

L e v e l 1

Re p o r t e

Ma t u r i t y

L e v e l 2

Re p o r t e

M a t u r i t

L e v e l 3

Re p o r t e

Ma t u r i t y

L e v e l 4

Re p o r t e

Ma t u r i t y

L e v e l 5

R e p o r t e C o u n t ry

N u m b e r o

Ap p r a i s a l

Ma t u r i t

L e v e l 1

Re p o r t e

M a t u r i t

L e v e l 2

Re p o r t e

M a t u r i t

L e v e l 3

Re p o r t e

M a t u r i t

L e v e l 4

Re p o r t e

Ma t u r i t

L e v e l 5

Re p o r t e

A rge nt ina 19 N o Y e s Y e s Y e s Y es K orea , R epu b lic O f 7 8 Y es Y es Y e s Y e s Y es

A u s t ra l ia 23 Y es Y e s Y e s Y e s Y es La tvia 1 0 o r fe w er

A u s t ria 10 o r few er M a lay s ia 1 9 N o Y es Y e s N o Y es

B a hra in 10 o r few er M a urit ius 1 0 o r fe w er

B e la ru s 10 o r few er M e x ic o 1 5 N o Y es Y e s Y e s Y es

B e lg ium 10 or few er M o roc c o 1 0 o r fe w er

B raz il 48 N o Y e s Y e s Y e s Y es N etherlan ds 1 0 o r fe w er

C a nad a 26 N o Y e s Y e s Y e s Y es N ew Zea lan d 1 0 o r fe w er

C h ile 15 N o Y e s Y e s N o Y es P ak is tan 1 0 o r fe w er

C h ina 24 0 Y es Y e s Y e s Y e s Y es P eru 1 0 o r fe w er

C o lom bia 10 o r few er P h ilipp ine s 1 6 N o Y es Y e s N o Y es

C z ec h R ep ub lic 10 o r few er P ortu ga l 1 0 o r fe w er

D e nm a rk 10 o r few er R us s ia 1 0 o r fe w er

D o m in ic an R e p 10 o r few er S ing ap ore 1 0 o r fe w er E g y p t 17 N o Y e s Y e s Y e s Y es S lova k ia 1 0 o r fe w er

F in land 10 o r few er S outh A fric a 1 0 o r fe w er

F ra n c e 75 Y es Y e s Y e s Y e s Y es S pa in 3 1 N o Y es Y e s N o Y es

G erm an y 35 Y es Y e s Y e s Y e s Y es S w ede n 1 0 o r fe w er

H o ng K o ng 10 S w itz e rland 1 0 o r fe w er

In d ia 20 4 N o Y e s Y e s Y e s Y es Ta iw an 4 6 N o Y es Y e s N o Y es

In don es ia 10 o r few er Th a ilan d 1 0 o r fe w er

Ire lan d 10 o r few er Tu rk ey 1 0 o r fe w er

Is rae l 10 U n ite d K ing do m4 8 Y es Y es Y e s Y e s N o

Ita ly 10 o r few er U n ite d S tate s 7 18 Y es Y es Y e s Y e s Y es

Ja pa n 17 2 Y es Y e s Y e s Y e s Y es V ie t N am 1 0 or fe w er


10/44

10



0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Not Given Initial Managed Defined Quantitatively

Managed

Optimizing

USA: 100 % = 590

Non-USA: 100 % = 1122

Based on USA organizations and Non-USA organizations

%

of

Organization

s

Maturity Profile by All ReportingUSA and Non-USA Organizations

93

1122

189

9

70

60

16

346

439

62

199

218

11

590


11/44

11



Combined Appraisal Opportunities

Current ISO 9001

ISO 9001ISO 9001

IAIA

Current CMMI

SCAMPISCAMPI

AA

SCAMPI A

&

ISO 9001

SCAMPI A

Visit

Report

Rating letter

indicating level

achieved

continues to

demonstratecompliance with

ISO 9001:2000

no behaviours

inconsistent with

operating at level X

(Combined ISO Surveillance

using Cat C appraisal)

(Cat C appraisal)

Rating letter & or certificate

with scope indicating

in accordance with Level X


12/44

12



Improvements Median # of data

points

Low High

Cost 20% 21 3% 87%

Schedule 37% 19 2% 90%

Productivity 67% 16 11% 255%Quality 50% 18 29% 132%

Customer Satisfaction 14% 6 -4% 55%

Return on Investment 4.8 : 1 14 2 : 1 27.7 : 1

N = 25, as of 15 December 2005 Organizations with results expressed as change over time

Performance Results Summary


13/44

13



CMMI Books

A Guide to the CMMI: Second Edition

CMMI: A Framework

CMMI Assessments

CMMI Distilled: Second Edition

CMMI SCAMPI Distilled

CMMI Survival Guide

CMMI: Un Itinraire Flch: Second Edition

De kleine CMMI

Interpreting the CMMI

Making Process Improvement Work

Practical Insight into CMMI

Real Process Improvement Using the CMMI

Systematic Process Improvement Using ISO 9001:2000 and CMMI


14/44

14



How About SEI Publications?

Technical notes and special reports:

Interpreting CMMI:

for Operational Organizations

for COTS Based Systems

for Service Organizations

for Business Development

Using CMMI with:

Team Software Process (TSP)

Earned Value Management

Product Line Practices

Six Sigma

Supplementing CMMI for Safety Critical Development (+Safe)

Demonstrating the Impact and Benefits of CMMI (and Web pages http://www.sei.cmu.edu/cmmi/results)
http://www.sei.cmu.edu/cmmi/resultshttp://www.sei.cmu.edu/cmmi/results


15/44


CMMI V1.2 and Beyond...


16/44

16



CMMI Today

Version 1.1 CMMI Product Suite was released January 2002. CMMI Web site visits average over 20,000/day

Over 66,000 people have been trained

Over 1800 class A appraisals have been reported to the SEI

Annual rate now over 800/year

Now we want to continuously improve


17/44

17



Major Themes

Reduce complexity & sizeIncrease coverage

Increase confidence in appraisal results


18/44

18



CMMI Model Combinations

SERelated

Examples

Integrated Product andProcess Development

SupplierSourcing

SWRelated

Examples

V 1.1

SERelated

Examples

IPPD

SWRelated

Examples

HWRelated

Examples

V 1.2

Organizational Goal(OPD)

Project Goal (IPM)

CMMI Core (now includes SS)CMMI Core


19/44

19



Suppo

rt

P

As

Proce

ss

Mgt

PAs

Projec

tM

anage

IPM

IPPD Changes

SG1

SG2

SG3

SG4

IPM

SG1

SG2

SG3

IT

SG1

SG2

SG3 = Apply

IPPD principles

O

EI

SG1

SG2

OPD

SG1

SG2 SG2 = EnableIPPD principles

V1.1 V1.2


20/44

20



SCAMPI A Changes for V1.2

Method implementation clarifications

interviews in virtual organizations

practice characterization rules

organizational unit sampling options

Appraisal Disclosure Statement (ADS) improvements

reduce redundancy with other appraisal documents

improve usability for sponsor and government

require sponsors signature on the ADS

require all team members to show agreement on findings

both V1,1 and V1.2 ADS will reflect these as of Nov 2006

Lead Appraisers must be certified to lead high maturity appraisals

Maturity level and capability level shelf life 3 years, given 1 year of V1.2availability


21/44

21



Published Appraisal Results


22/44

22



CMMI V1.2 Schedules

Version 1.2 CMMI Product Suite release August 25, 2006

Update material available August 25, 2006

Upgrade course available on-line August 25, 2006

First Lead Appraiser face to face October, 2006

First high maturity oral exam October, 2006

V1.2 ADS required for all SCAMPIs November, 2006

Last V1.1 Intro training December, 2006

First expiration of V1.1 appraisals (3-year

validity)

August 25, 2007

Last V1.1 appraisal August 31, 2007


23/44

23

CMMI Update V1.2 and Beyond...

Phillips, August 1, 2007


Beyond V1.2

Improved architecture will allow post-V1.2 expansion. Extensions of the life cycle (Services, Outsourcing/Acquisition) could

expand use of a common organizational framework:

allows coverage of more of the enterprise or potential partnering

organizations

adapts model features to fit non-developmental efforts (e.g., CMMI

Services, CMMI Acquisition)


24/44

24




3 Complementary Constellations

CMMI-SVC

CMMI-DEV

CMMI-Servicesprovides guidancefor those providing

services withinorganizations and toexternal customers

CMMI-ACQ

CMMI-ACQprovides

guidance toenable

informed anddecisiveacquisitionleadership

CMMI-Devprovides

guidance formeasuring,

monitoring andmanaging

developmentprocesses

16 CoreProcess Areas,common to all


25/44

25




Acquirer/Supplier Mismatch

Acqu

irer

Supplier

Mismatch

Mismatch

mature acquirermentorslow maturity supplier

outcome not predictable

immatureacquirer

Customer

encourages shortcuts.

Matchedacquirer and supplierare both high maturity

highest probability ofsuccess

Disasterno disciplineno processno product

Technical &Management Skill

Low

Low

High

High

maturesupplier


26/44

26




Initial CMMI-ACQ Key Acquisition Processes

AcquisitionRequirements

Development

Solicitation &

SupplierAgreement

Development

Acquisition

Technical

Management

AgreementManagement

ProjectManagement

AcquisitionValidation

AcquisitionVerification

Core

Processes

*based on initial CMMI-ACQ model developed by General Motors/SEI


27/44

27




Acquisition Improvement is Needed.

Acquirers cannot ensure thatmature processes are applied to

their programs

Acquirers need more internalprocess focus

PMO

Contractor

Mismatch

Mismatch

mature acquirermentors

low maturity supplier

less mature acquirerderails mature supplier;encourages short cuts

Matchedacquirer and supplierare both high maturity

Disasterno disciplineno process

Technical & Management Skill

Low

Low

High

High

outcome notpredictable

highest probabilityof success

no product

supplier compromises

processes

XYZ Corp.

DivisionA

Project1

Project2

Project3

Division B Division N

Project5

Project6

Project7

Projectx

Projecty

Projectz

Project4

MLs usually

apply HERE

based upon

appraisals of

THESE

but your

project is

HERE orHERE


28/44

28




Guidebook Concept

Provide a process toolbox for the acquirer Include practical guidance on how to recognize the real practitioners

Encourage the use of capability and maturity profiles vice "single level"

approach

Improve acquisition organizations' understanding of the meaning of high

maturity (levels 4 and 5) and equivalent staging

Include multiple tools and guidance that may be used throughout the

acquisition lifecycle


29/44

29




Planned Sequence of Models

SA-CMMSA-CMM

GM IT SourcingGM IT Sourcing

CMMI-DEV V1.2CMMI-DEV V1.2

CMMI-ACQCMMI-ACQ

CMMI-SVCCMMI-SVC

CMMI V1.1CMMI V1.1

CMMI-AMCMMI-AM


30/44

30




Questions to Guide Discussion

Do we need something different or additional to define High Maturity (i.e. CMMI

Level 4 & 5)?

How can we "slim down" the CMMI models while still preserving integrity?

Can we likewise "slim down" the Appraisal method?

Can we eliminate the Staged representation?

Is the CMMI v1.2 Constellation Strategy the right approach?

Can we identify "next-generation" process improvement methodology?

Can CMMI be harmonized with other continuous process improvement efforts?

Can repeatability, consistency and overall model and appraisal methodology be

improved?

Are there "breakthrough" concepts that we can apply to overall process

improvement?


31/44


32/44

32




How can we "slim down" the CMMI modelswhile still preserving integrity?

How can we make this more user friendly?

Can we slim down for small projects? Can the model have some scalabilityaccording to various factors (e.g., project size, PoP, organization size)?

How do extensions fit in with the model?

Consider options for packaging (remove redundant stuff or repackage better)

Consider fundamental, intermediate and advanced volumes

Consider architectural views for appropriate for the different using communities

Consider streamlining the generic practices and look at measures forinstitutionalization

Consider folding the GPs into PAs [note risk of losing integrity]


33/44

33




Do we need something different or additionalto define High Maturity? -1

Focus on the best practices, not focus on the high maturity aspectsConsider combining level 4 and 5 into one level because of their close tie

4 and 5 are not adequately elaborated for implementation so these may

needs more detail to drive proper behavior

For example, if we tie level 4 and 5 to business objectives this may need tobe a practice

If there are additional requirements for the model these can be turned into

practices. E.g., High maturity body of knowledge and high maturity training

Risk: adds more to the model


34/44

34




Do we need something different or additionalto define High Maturity? -2

Consider redistributing practices across the levels to even out effort andexpectation

Atlas study items that impact the model and results

Consider maturity levels within PAs (e.g., project management PAs for

each level)

Consider better interfacing approaches with other methodologies (e.g., six

sigma for high maturity)


35/44

35




How can repeatability, coverage (scope) andconsistency of the model be improved?

Coverage (Scope)

Areas for consideration

Operations, Support, Transition to operations, Deployment, Disposal, Pre-project, Proposal,sustainment, transition to production, production/manufacturing, training

Better coverage of maintenance and technical reviews

Safety, security, dependability, systems assurance, environmental

Strategic planning, enterprise management, enterprise architectures, enterprise investment strategies,

finance Work force management and development

IR&D, Advanced technology, advanced technology test bed or lab environment

Small settings

Product lines

Business practices

Information management (both enterprise and project) Systems of systems and their acquisition


36/44

36




What are the "next-generation" process improvementmethodology? Are there "breakthrough" concepts that we canapply to overall process improvement?

Consider how CMMI could interface with other process improvementmethodologies (e.g. Lean, six sigma, PMBOK, theory of constraints, nextgeneration IDEAL)

Integration of how people use the various methodologies (same list as above)

Agile techniques (extreme programming), TSP/PSP

When everyone is level 5, then what? Consider optimizing measurements

Consider an emphasis on process performance effectiveness and efficiency,(e.g., effectiveness 6 sigma, efficiency LEAN)

How do we measure program health?

Need for leading indicators


37/44

37


Phillips, August 1, 2007 2007 Carnegie Mellon University

What representation should we have (e.g., Staged,Continuous)? -1

Is the question really level-mania? (root cause?) Level-mania is about doing the minimal work to achieve a level ignoring

what you did to achieve the grade

Levels are not bad, but we need to get the integrity of the level back up

Provides a roadmap for projects to break PI into pieces

Guidance for where improvement is needed

Risk: has 5 been around so long that getting rid of it will have unintended

consequences?

Maybe we have a 5 level model that only really has 3. Redesign the model

to address this


38/44


39/44

39



Is the CMMI v1.2 Constellation Strategy theright approach?

Alternative approach: Start with a CMMI Model Framework (CMF) and add

where you need to expand scope (+ concept)

Are there differentiators for constellations?

Instead of creating constellations, encourage projects to do what makes

sense with respect to what they are doing using the parent model

Consider looking at lifecycle and what is needed at each phase

Can the unsophisticated tailor the parent model for their perspective?

How multiple constellations can be used in an organization for

improvement and appraisal?


40/44

40



How can we "slim down" the CMMI appraisals while still preservingintegrity? How do we eliminate non-value add in appraisal and appraisalpreparation? How do we make appraisals more efficient and effective?

Consider making the appraisal be focused on goals

Add an appendix for application practices

Lead Appraiser and the Appraisal Team should have enough experience to review

company implementations

Make some assumptions that some processes are in place (e.g., assume project

planning has happened, but dont look at PP specifically unless you see somethingout of place in PMC; similarly, could start with IPM for a level 3, or QPM for a level 4)

Need more guidance on where and how you might be able to do this

More official or formal mechanisms for sampling coverage

Consider cost implications


41/44

41



Can we identify "next-generation" appraisal methodology? Arethere "breakthrough" concepts that we can apply to overallappraisals?

Delta appraisal, continuous, incremental, using measures to judge

satisfaction, leading indicators, process performance measures, program

health (versus process health), 15504 (SPICE), EIA 732 (percentage of

practices performed, effectiveness of generic attributes)

Data reuse from previous appraisals

Appraisal by parts

Example, OSP separate from projects

CMF separate from model components unique to constellations

Certify processes instead of model (e.g., EV or SEMP)

Sponsor commits to ongoing process improvement


42/44

42



How do we improve the trust and believabilityin the results of the appraisals?

Process enactment tools can help with approval (workflow management)

Need to define consistent process for OSP tailoring approval

Requiring the appraisal be redone every three years will help with

believability (already in place)

Consider notion of visits or interim steps (like ISO surveillance audits) Six month assessments focus on correlation between results and

performance (process reviews)

Doesnt interrupt the program and not as expensive

What if you could extend the life of your appraisal if you did interim review?

What is in it for the organization to provide incentive for the use of reviews?

Would the customer accept the results of the interim review?


43/44

43



How do we improve the trust and believabilityin the results of the appraisals?

What should the results of a bad review be? Should you lose your level?

Could you use the delta appraisal here?

How are organization changes that may impact the process capability

tracked?

Consider adding a practice to the model for these changes

Would interim reviews impact the capability profile

Might red flag the program or organization

What happens if you miss by one practice in the full appraisal?

Do I get rewarded if I fix it? Consider delta approach

Consider a more formal mechanism to track changes that may impact the

process capability or level


44/44


Next Steps

Send us your ideas

Form available on-line from SEI

Submit like a Change Request

Open Discussion.

CMMI Update V1.2 and Beyond...

Documents

Transcript of CMMI Update V1.2 and Beyond...