INTEGRATING RISK WITH EARNED VALUE

PIKES PEAK REGIONAL CHAPTER PROJECT MANAGEMENT PRODUCTIVITY TOOLS

SYMPOSIUM SATURDAY, FEBRUARY 28TH, 2009

GLEN B. ALLEMAN

The notion of integrating cost, schedule, technical performance, and risk is possible in theory. In practice care is needed to assure credible information is provided to the Program Manager.

Today's Learning Objectives Cost, Schedule, Technical

Performance, and Risk Are

Inseparable

All numeric values of cost,

schedule, technical

performance, and risk

assessment are random

variables drawn from some

underlying probability

distribution

Managing these variables

must be done through the

eyes of a Risk Manager 2

There are

usually two

phases to many

projects …

1. Too early to tell

2. Too late to stop

3

The Fundamental

Program Management Question

Can we plan with

sufficient accuracy, to

allocate resources to

reduce risk while

meeting requirements,

and obtain timely

feedback in order to

adjust plans and

performance goals?

4

Cost

Technical

Performance Schedule

Funding margin for

under performance

Schedule margin for over

target baseline (OTB)

Schedule margin for

underperformance or

schedule extension

Over cost or

under

performance

Over cost or

over

schedule

Over schedule

or under

performing

Dependencies Between The Inseparable Variables

5

Earned Value

Management

Establishes project

performance status

and extrapolates

that information to

understand future

trends and

allocation of

resources needed

to successfully

meet project

milestones

Programmatic and

Technical Risk

Management

Looks to the

unknown future to

identify risks and

recommend early

action to be taken

to limit the threat or

maximize the

exploitation of

opportunities

Technical

Performance

Measures

A time phased

progress plan for

the achievement of

the Technical

Performance

Parameters (TPP)

that indicate key

areas for risk

reduction and

program success

INCOSE G2SEBOK APM EV-Risk Working Group, 2007

Some Simple Definitions

6

Cost

Technical

Performance Schedule

Systems Engineering states

deliverables performance and

associated risks

Integrated Master Schedule &

Earned Value Management

tracks risk activities

Structure WBS contains risk

mitigations and retirement efforts

The Integration of Risk and Earned Value is a Systems

Engineering Paradigm

7

But Many Times, The Information from Cost,

Schedule, Techncial Performance, and Risk

Management Systems Gets Mixed Up When We

Try to Put Them Together

8

Every Program Manager Should Know…

When will this project finish?

What is the statistical confidence in this date?

Is the project’s schedule realistic and achievable?

What will it cost when we are done?

How is the project performing against its plan?

What deliverables are slipping?

How are we going to get the deliverables back on schedule?

What does past performance say about the future performance?

What is the impact on the schedule of any change requests?

Is there enough schedule slack and cost reserve to cover the risk?

Where is the risk in the schedule?

How can we get the work done sooner to reduce risk?

How can we recover from any foreseen delays?

How can we work around a problem?

9

SV

The First Step is to Start with Key Elements of Earned Value

B C W S

CV

The Work planned, performed, and the cost of

performing that work, is the common variable across

each Earned Value variable

A C W P

B C W P

10

EVM Analyst’s EAC Range

• Addresses Risk

• Based on CPI & SPI trend analysis not on

plugging numbers into a formula

• Incorporates technical performance, schedule

progress, CAIV and other program information

EAC Floor Performance Factor

EAC Ceiling Performance Factor

cum

cum cum

BAC BCWP BACEAC ACWP

CPI CPI

cum

cum cum cum

BAC BCWP BACEAC ACWP

CPI SPI CPI

Some Useful EV Information

http://www.daytonaero.com/Files/resource/31.pdf

11

ANSI 748B Says on Page 1

Plan all work scope for the program to completion.

Integrate program work scope, schedule, and cost objectives into a baseline plan against which accomplishments may be measured.

Objectively assess accomplishments at the work performance level.

Analyze significant variances from the plan and forecast impacts.

Provide data to higher levels for management decision making and implementation of management actions.

12

Some Criteria for Successful EVMS Beyond Full Compliance of the 32 Criteria

Define a Work

Breakdown Structure

Identify the

Organizations

doing the work

Integrate WBS and

OBS into a RAM

Schedule all

Planned Work

Indentify Products

and Milestones

Time Phase the

Budget

Record all Direct

Costs

Determine all

Variances

Sum These

Variances

Manage Action

Plans

Incorporate Changes 13

At The Same Time, Risk Management is

Commonly Misunderstood

It’s not about random chance, it’s about defining

mitigations and retirement plans in the presence of

uncertainty

14

1. Hope is not a strategy 2. No single point estimate of cost or schedule can be correct 3. Cost, Schedule, and Technical Performance are inseparable 4. Risk management requires adherence to a well defined process 5. Communication is the Number One success factor

Five Fundamental Principles of Risk Management

15

DoD’s Approach to Successful Risk Management

16

But ANSI 748B Also Says …

Identify physical products, milestones, technical

performance goals, or other indicators that will be

used to measure progress.

So why the disconnect between Risk and Earned

Value?

Isn’t the measure of risk, the rate of risk reduction

an “…other indicators…”

17

Define and

Organize the

Work

Establish MR

Issue Budget

Authorize Plan

Establish PMB

Authorize Work

Measure

Performance

Analyze Results

Plan Risk

Activities

Perform Risk

Assessment

Develop Risk

Handling Plans

Assign

Responsibilities

Monitor and

Communicate

Update Risk Register

Revisions & Change Control

I think you

should be more

explicit about

the steps here.

Integrating

Earned Value

and Risk

Management

18

SOME ACTIONABLE DETAILS FOR PUTTING THESE FOUR CONCEPTS TOGETHER

In Theory there is no difference between Theory and Practice.

In Practice there is — Yogi Berra

So much for our strategy of winning

through technical dominance

There are two types of “uncertainty” on any

sufficiently complex program

– Technical – uncertainty about the functional and

performance aspects of the program’s

technology that impacts the produceability of

the product or creates delays in the schedule

– Programmatic – uncertainty about the duration

and cost of the activities that deliver the

functional and performance elements of the

program, independent of the technical risk

We’re interested in connecting the two in the schedule and cost

model(s)

– When the technical uncertainty arises what is the impact on the schedule

and cost?

– When the schedule or cost uncertainty arises what is the impact on the

functional and performance aspects?

The Difference Between

Technical and Programmatic

Risk

20

Deterministic Versus

Probabilistic Deterministic

• Each activity has a planned value

• For the schedule each task has a

predecessor and a successor.

• The longest path through the

network is the critical path

• The total duration of the project is

a fixed value - it is deterministic

• The total cost is the sum of all the

activity costs

• Risks are defined and handled as

static entities

Probabilistic

• The program elements are not

random, but they are random

variables drawn from a probability

distribution.

• Three point estimates "can" be

used to describe task duration

random variables

• The total duration of the project is

a random number

• The total cost is a random number

• Risks are stochastic processes

that have probabilistic outcomes

for cost, schedule and technical

performance

21

Lack of predictive variance analysis

Untimely and unrealistic Latest Revised Estimates (LRE)

Progress not monitored in a regular and consistent manner

Lack of vertical and horizontal traceability cost and schedule data for corrective action

Lack of internal surveillance and controls

Managerial actions not demonstrated using Earned Value

Inattention to budgetary responsibilities

Work authorizations that are not always followed

Issues with Budget and data reconciliation

Lack of an integrated management system

Baseline fluctuations and frequent replanning

Current period and retroactive changes

Improper use of management reserve

EV techniques that do not reflect actual performance

Without these Principles The Program Train Wreck Starts When…

Mary K. Evans Picture Library

22

AN INTRODUCTION TO SOFTWARE ENGINEERING INSTITUTE’S CONTINUOUS RISK MANAGEMENT (CRM)

CRM is the Software Engineering Institute’s framework for managing risk in the context of system integration, COTS based product development, and the management of these activities.

Continuous Risk Management has Six Components

http://www.sei.cmu.edu/risk/index.html

24

Putting Continuous Risk Management Work

CRM Activity Project Representation

Identify Risk items with IMP/IMS #’s, CA/WP & resource

assignments

Analyze Risk management responsibilities assigned

Plan Mitigation plans with durations and resource

assignments

Track Status reported from Risk Management to IMS

Control Risk tasks reporting in weekly status process

Communicate Integrated Master Schedule (IMS) status reporting

as Physical Percent Complete

25

This All Comes Together in The Risk Registry

http://www.mitre.org/work/sepo/toolkits/risk/ToolsTechniques/RiskMatrix.html

26

PHYSICALLY CONNECTING EARNED VALUE WITH RISK MANAGEMENT

Some Historical Background†

Cost and schedule growth have been persistent

problems for decades

The use of Earned Value is common in DoD

environments

As is the use of Risk Management

It’s the integration that is missing

† “Integrating Risk Management with Earned Value Management,” NDIA

Report, www.ndia.org

28

The Core Problems Before We Start

Earned Value is a measure of quantity not quality

Risk measures the probability of an unfavorable

occurrence but rarely monetizes this outcome

Risk management focuses on dealing with future

events

Earned Value focuses on reporting past

performance

29

Starting with the Individual Elements

Identify Analyze Plan Track Control

Organization Identify risk

owners

Risk board

assessment of

impact

Mitigation and

retirement

owners

Connect PMB

with Risk registry

Vet members of

the leadership

Planning &

Budgeting

Identify

uncertainty and

response

Probability

impact in

schedule

PMB contains all

risk active

activities

Baseline the risk

registry

Transfer

mitigation to PMB

and close risk

Accounting

Assign risk to

CA/WP cost

baseline

Probability

impact on cost

Risk budgets

defined and

allocated

Risk Board

management of

cost impacts

Transfer through

EV change

control processes

Analysis

Asses impacts

and

dependencies

Risk registry

contains

probabilities of

cost and schedule

Risk Board and

IMS Board

impact analysis

Manage budget

assignment by

risk level

Revisions Keep historical

records

Update ETC from

risk management

Updated PMB

from risk

activities

Risk review Revise PMB

forecast

30

First, A (Notional) Big Picture

Risk ID Traceable to

work elements in the

IMS within the WBS

Risk reduction

waterfall metrics

connected to Earned

Value program

performance

Standard program

performance of TCPI

and IEAC

31

EV Data

EV Data

One Approach to Integrating Risk Management with Earned Value

Program Manager

Functional Managers

Work Package Managers

Individuals/Team Members

Identify

Analyze Review Prioritize

Evaluate Classify

Track

Plan Approve plans

Recommend actions Develop plans

Control Integrate across

functions Reprioritize Authorize project

resources

risks

Top N risks Decisions

Assign Responsibility

Required Indicators

Trends

Risk Status

Control Integrate Reprioritize Authorize Functional area

resources

EV Data

1

3 2

Top N risks

32

EXAMPLE SCHEDULE RISK ANALYSIS

Monte Carlo tools model the task durations with random variables for the durations and build “pictures” of the likelihood of a task completely on or before a specific date.

@Risk and Risk+ Sample Screens

34

Schedule for a Monte Carlo Risk analysis starts with a

credible schedule and defines the probabilistic behavior

of each activities and how it drives the deliverables

The Risk+ tool sets the upper and lower bounds of the possible durations 35

Date: 2/25/2009 3:34:12 PM

Samples: 300

Unique ID: 30

Name: Final Testing

Completion Std Deviation: 5.51d

95% Confidence Interval: 0.62d

Each bar represents 2d

Completion Date

Fre

qu

en

cy

Cu

mu

lative

Pro

ba

bili

ty

Tue 2/11/03Mon 1/20/03 Mon 3/3/03

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0.16 Completion Probability Table

Prob ProbDate Date

0.05 Wed 1/29/03

0.10 Fri 1/31/03

0.15 Tue 2/4/03

0.20 Wed 2/5/03

0.25 Wed 2/5/03

0.30 Thu 2/6/03

0.35 Fri 2/7/03

0.40 Mon 2/10/03

0.45 Mon 2/10/03

0.50 Tue 2/11/03

0.55 Wed 2/12/03

0.60 Thu 2/13/03

0.65 Fri 2/14/03

0.70 Fri 2/14/03

0.75 Mon 2/17/03

0.80 Tue 2/18/03

0.85 Wed 2/19/03

0.90 Thu 2/20/03

0.95 Tue 2/25/03

1.00 Mon 3/3/03

The output of Risk+ is a Probability Distribution Function and a

Cumulative Distribution of all the possible dates that “watched”

activity could take.

The result is a picture of the Confidence that the target date of

2/10/3 – can be met.

It shows 40% – which is not good

36

So with this information we can ask …

What cost impact will there be?

What resource impacts?

What technical dependencies?

What mitigation or retirement plans must be in

place to increase the probability of success to

something greater than 40%?

What other interdependencies are there in the

program?

37

Program Management is Risky Business

Traditional Approaches Are Seriously Flawed

Risk Management Is Not About Preventing Risk

Risk Management Is About Managing In The Presence Of

Risk

This means managing Cost, Schedule, and Technical

Performance

38

Source Materials

Interfacing Risk & Earned Value: Management Reserve,

“Technical Performance Measurement, Earned Value, and Risk Management: An Integrated

Diagnostic Tool for Program Management,” Commander N. D. Pisano, SC, USN, Program

Executive Office for Air ASW, Assault, and Special Mission Programs (PEO(A))

Practical Risk Management: The ATOM Method, David Hillson and Peter Simon, Management

Concepts.

Project Risk Analysis and Management Guide, APM Publishing.

“Formal Risk Management,” DACS Gold Practices

“Quantify Risk to Manage Cost and Schedule,” Fred Raymond, Acquisition Review Quarterly,

Spring 1999, pp. 147–154

Probabilistic Risk Assessment Procedures Guide for NASA Managers and Practitioners, Office of

Safety and Mission Assurance, April 2002

Development Of Risk Management Defense Extensions To The PMI Project Management Body

Of Knowledge – Tutorial,” Edmund Conrow, Acquisition Review Quarterly, Spring, 2003.

Effective Risk management: Some Keys to Success, Edmund Conrow, American Institute of

Aeronautics and Astronautics, 2000.

39