Risk Modeling and Analysis (Mitigating the Planning Fallacy)
-
Upload
bryan-ritch -
Category
Business
-
view
1.227 -
download
15
description
Transcript of Risk Modeling and Analysis (Mitigating the Planning Fallacy)
Mitigating the Planning Fallacy
Gui Ponce de Leon, PhD, PE, PMP, LEED AP
GPM Boot CampNewark, NJMarch 8th, 2013
(Risked Schedules─The New Normal)
©2012-2013 Permission is granted to PMA Technologies 1
2
There are lies, there are damned lies, and then there are deterministic schedules
Attributed to Dr. Vivek Puri, PMA’s resident simulation guru
©2012-2013 Permission is granted to PMA Technologies
3©2012-2013 Permission is granted to PMA Technologies
Mitigating the Planning Fallacy
CPM Risk Modeling & Analysis
GPM® Risk Modeling & Analysis
NetRiskTM Synopsis
Summary & Take-Aways
PRESENTATION OUTLINE
Just What Is the Planning Fallacy?
4©2012-2013 Permission is granted to PMA Technologies
Kahneman1 and his longtime colleague, Tversky, coined the term to describe plans that
Are unrealistically close to best-case scenarios
Could be improved by consulting the statistics of similar cases
Planning Fallacy aka Optimism Bias
5©2012-2013 Permission is granted to PMA Technologies
The problem of optimism bias arises when various factors combine to produce a systematic underreporting of the level of project uncertainty
Bent Flyvbjerg2, the renowned Danish planning expert, notes
Planning Fallacy aka Optimism Bias (cont’d)
6©2012-2013 Permission is granted to PMA Technologies
The failure to reflect the probabilistic nature of project planning, implementation and operation is a central cause of the poor track record for megaproject performance
Bent Flyvbjerg futher notes
Scheduling Strategies for Mitigating the Planning Fallacy
7©2012-2013 Permission is granted to PMA Technologies
The outside view does not try to forecast specific uncertain events that may affect the activities
Rely on an outside view as advocated by Kahneman 1
Scheduling Strategies for Mitigating the Planning Fallacy
8©2012-2013 Permission is granted to PMA Technologies
The schedule is dealt with as inherently stochastic in nature rather than being analyzed merely for risk (i.e., what-if exercise)
Work with risked schedules as the new normal2
Taking an Outside View Relative to Schedules
9©2012-2013 Permission is granted to PMA Technologies
At a minimum, the database captures ‘normal’ durations
The schedule is built using a database of historical activity durations by project type/context
Ideally, distributional information (mean, mode, low/high) is included
Physical work durations factor production rates, e.g., steel tons/day, concrete CY/day, large bore pipe LF/day, etc.
Working with Risked Schedules
Is the New Normal
©2012-2013 Permission is granted to PMA Technologies 10
A base‐case schedule portraying how the project would evolve with activities at their normal durations is the initial focus
The baseline schedule selected reserves schedule margin sufficient to support the targeted probability(ies) of completion
Following risk modeling, risk analysis trials are conducted to investigate alternate probabilistic and baseline scenarios
STEP 1 STEP 2 STEP 3
Risked schedules are generated in the following sequence:
©2012-2013 Permission is granted to PMA Technologies 13
The baseline schedule is risk assessed periodically and when revised to reflect scope of remaining work and current risks
Going from deterministic to probabilistic planning/scheduling and back is a seamless exercise throughout the project life cycle
RISKED SCHEDULES
©2012-2013 Permission is granted to PMA Technologies 14
1955 1960 1965 1970 1975 1980 1985 1990 1995 2000
1958 Development and implementation of PERT by the US Navy Special Projects Office
1962 Robert McNamara endorses use of PERT/COST (forerunner to earned value) throughout the US DOD3
1966 Pritsker develops GERT for NASA as a method to analyze stochastic activity networks5
1986-87 Risk management becomes a separate knowledge area in the PMBOK in the 1986-87 update6
1990s Simulation morphs into schedule risk analysis
1995 Primavera releases Monte Carlo version 3.0 offering “enhanced risk analysis software for project management”
Schedule Risk in the 20th Century
1963 First application of Monte Carlo simulation to network-based schedules by Van Slyke4
©2012-2013 Permission is granted to PMA Technologies 15
1998 2000 2002 2004 2006 2008 2010 2012 2014 2016
2000 Risk analysis is added to Pertmaster and Pertmaster supports full integration with Primavera
2004 The Third Edition of the PMBOK adds the ‘risk register’ as a primary output of the Identify Risks Process
2011 Schedule risk analysis is intrinsic to scheduling excellence in the Planning &
Scheduling Excellence Guide7
2012 Schedule risk analysis is codified as one of the nine scheduling best practices in
the GAO Schedule Assessment Guide8
2013 GPM® Risk is introduced at the NetPoint® User Conference in New Orleans
Schedule Risk in the 2000s
2008 Oracle acquires Primavera, and the Pertmaster software is renamed OPRA
2012 AACE International releases RP 64 on CPM Schedule Risk Modeling and Analysis9
Emerging Consensus on CPM Risk Analysis
16©2012-2013 Permission is granted to PMA Technologies
A schedule risk analysis (SRA) is conducted to determine
the likelihood of completion dates
schedule contingency needed for an acceptable level of certainty
Emerging Consensus on CPM Risk Analysis (cont’d)
17©2012-2013 Permission is granted to PMA Technologies
The baseline schedule includes contingency aka schedule margin to account for the occurrence of risks
Schedule margin supports the targeted likelihood of meeting completion dates
Emerging Consensus on CPM Risk Analysis (cont’d)
18©2012-2013 Permission is granted to PMA Technologies
An SRA is performed on the schedule periodically as the schedule is updated
CPM Risk Modeling & Analysis
19©2012-2013 Permission is granted to PMA Technologies
Minimal use of constraints on activities
PDM logic ties are used in only limited and well-understood circumstances
Modeling accepts ‘existence risks’ and ‘branching risks’
CPM Risk Modeling & Analysis
20©2012-2013 Permission is granted to PMA Technologies
Risk drivers occur with the same probability on impacted activities
In each realization, all activities are on early dates, but for perhaps SNE dates
Cruciality is combined with criticality
Weather risks are modeled
21©2012-2013 Permission is granted to PMA Technologies
GPM planned dates are favored over SNE constraint dates1Activities and activity nodes are encoded with stochastic rules2Any risk that, if occurring, impacts multiple activities, may occur with a different probability and impact on each activity
3
Risks resulting from common, contemporaneous decisions to start activities on dates later than early dates are modeled
4
Criticality and cruciality are combined to measure importance5
Adverse weather and weather-event risks are modeled6
GPM algorithms are extended for stochastic networks7
INTRODUCING GPM RISKMODELING & ANAYSIS
Planned Dates ILO SNE Constraints
22©2012-2013 Permission is granted to PMA Technologies
SNE constraints can be replaced in simulation with GPM planned dates
In CPM modeling, the consensus is to limit SNE dates to external dependencies10
Unlike a constraint date, a planned date may shift to an earlier date in a realization if predecessors on logic chains leading to the planned-date activity are sampled at the right mix of lower durations
Because GPM is not fixated on early dates
1
Planned Dates ILO SNE Constraints (cont’d)
23©2012-2013 Permission is granted to PMA Technologies
1
Pursue trial simulations with alternate, earlier SNE dates, or
Replace the SNE date with a variable-duration activity
In CPM risk, to mitigate SNE constraints, analysts may
Schedule Demonstrative, Base-Case Scenario
24©2012-2013 Permission is granted to PMA Technologies
Constraint Date
Simulation Trial, Planned Date ILO SNE Date11
25©2012-2013 Permission is granted to PMA Technologies
Planned Date
Not Risked
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
08/26/2012 09/09/2012 09/23/2012 10/07/2012 10/21/2012
Num
. of Iteratio
ns
Date
SNE Constraint Planned Date
Planned Dates vs. SNE Constraints Results
26©2012-2013 Permission is granted to PMA Technologies
10/03/2012
10/08/2012
01/09/2012
12/20/20
11
01/09/20
12If the SNE date could be moved up to 01/06/2012, the P80 date improves to 9/27/2012
Stochastic Activity Modeling in GPM Risk
27©2012-2013 Permission is granted to PMA Technologies
GPM activities are diagrammed with start and finish nodes
2
An ‘or’ node is realized when the sampled predecessor is realized
An ‘any’ node is realized when any merging predecessor is realized
Activity nodes are encoded with stochastic rules:
An ‘if’ node is realized if its predecessor is realized
Stochastic Activities in GPM Risk
28©2012-2013 Permission is granted to PMA Technologies
2
Delay risks
Branching risks
Stochastic activities occur based on a probability of occurrence and, if occurring, are of uncertain duration and may symbolize:
GPM Prime Risks vs. CPM Risk Drivers
29©2012-2013 Permission is granted to PMA Technologies
In GPM risk, an occurrence risk that may impact multiple activities may occur with a different probability and impact on each associated activity/group of activities
3
GPM Prime Risks vs. CPM Risk Drivers (cont’d)
30©2012-2013 Permission is granted to PMA Technologies
The risk driver approach restricts the risk to always occur/not occur in a realization for all of the associated activities/group of activities
This is not always true, to wit: if bad soil is hit when excavating the SW part of a building, the bad soil risk may not occur when excavating the NE part
In CPM, a risk driver is a particular case of a prime risk because, if occurring, occurs with the same probability of occurrence and the samepercentage impact for all associated activities
3
Float Consumption12
Risks in GPM Risk
31©2012-2013 Permission is granted to PMA Technologies
4
delaying the start of an eligible activity
within its float then existing when the activity is scheduled
Floating: event that occurs randomly and that involves
Float Consumption12
Risks in GPM Risk
32©2012-2013 Permission is granted to PMA Technologies
4
delaying the start of a pacing-eligible activity
within its float then existing when the activity is scheduled
Pacing: event that occurs randomly and that involves
provided the ratio then-existing float/deterministic float exceeds a threshold
Float Consumption12
Risks in GPM Risk (cont’d)
33©2012-2013 Permission is granted to PMA Technologies
4Modeling can control how often an eligible activity actually floats or paces during a realization by defining a likelihood factor
Float Consumption Risks
34©2012-2013 Permission is granted to PMA Technologies
A floating or pacing risk occurs whenever an activity that floated or paced and that falls on the longest path would not otherwise have been critical but for the floating or pacing decision
The floating or pacing decision in effect caused a critical path delay
Floating/pacing decisions rely on predicted vs. actual durations
Float Consumption Risks (cont’d)
35©2012-2013 Permission is granted to PMA Technologies
Floating/pacing risks cannot be modeled with CPM for two fundamental reasons:
The CPM scheduling algorithm defaults all activities to their earliest possible dates
No activity has (total) float in a CPM schedule during the forward pass calculations, which means that, unlike GPM, float does not exist in CPM when the activity is scheduled
Simulation Trial, Floating & Pacing Risks
36©2012-2013 Permission is granted to PMA Technologies
Floating
Pacing
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
08/11/2012 08/25/2012 09/08/2012 09/22/2012 10/06/2012 10/20/2012
Num
. of Iteratio
ns
Date
PERT Early Dates Floating & Pacing
Demonstrating the CPM ‘Optimism Bias’
37©2012-2013 Permission is granted to PMA Technologies
10/08/2012
09/28/2012
CPM Optimism BiasExclusion of floating & pacing events in risk modeling combines to produce a systemic overestimation of the true probability of accomplishing targeted completion dates
CPM Optimism BiasExclusion of floating & pacing events in risk modeling combines to produce a systemic overestimation of the true probability of accomplishing targeted completion dates
46%
09/23/2012
32%
Includes early‐dates and merge bias
Unbiased forecast
Includes early‐dates bias
Activity Criticality in GPM Risk
38©2012-2013 Permission is granted to PMA Technologies
5
A shorter, certain-duration activity and a longer, uncertain-duration activity may have equal criticality because they both fall on the stochastic longest path
Williams addressed this conundrum in 1992 when he developed his cruciality index13 that correlates sampled activity duration and realized project duration
Criticality index, while measuring the likelihood of activities falling on the stochastic longest path, fails to account for the correlation between activity duration and project duration
If an activity duration is certain, cruciality = zero
Activity Priority─A New Metric
39©2012-2013 Permission is granted to PMA Technologies
To solve the dilemma between criticality and cruciality, some CPM risk analyzers combine the two by multiplying criticality x cruciality
This formula tends to downplay the criticality index
Activity priority equals criticality index + criticality index x cruciality index
The comparable statistic in GPM risk is ‘activity priority’
Activity Priority─A New Metric (cont’d)
40©2012-2013 Permission is granted to PMA Technologies
GPM risk treats 66% confidence level
(2:1 odds of the value of cruciality being correct) as a default confidence threshold
Activity priority = criticality indexif cruciality has a significance confidence level below a:
Default Confidence Threshold
SYNOPSIS
NETRISK
©2012-2013 Permission is granted to PMA Technologies 42
NetPoint module that allows users to work with deterministic and probabilistic GPM &
CPM schedules seamlessly
Offers a full gamut of risk management processes, including:
43©2012-2013 Permission is granted to PMA Technologies
‘Risk Manager’ interface, which acts as a single streamlined window that works dynamically with the canvas rather than obscuring it
Interface for defining a fully-customizable risk breakdown structure
Fully-customizable probability & impact matrix with tolerance thresholds
Risk identification through a risk register
Full range of activity and risk correlations, including floating and pacing
Automated risk removal process for sensitivity tornado analysis
Full gamut of GPM (quantitative) schedule risk analysis
Wide range of simulation data mining that is fully customizable and that interface with MS Excel
Risked Trial Runs─P80 Date Comparisons
44©2012-2013 Permission is granted to PMA Technologies
33 33
15
30
24
38 38
15
37
31
43 43
33
43
37
0
5
10
15
20
25
30
35
40
45
Project Ready for Commissioning Elevator Install Complete Perm Power Available Start Process Installation
Uncertainty Only
Uncertainty + Planned Date
Uncertainty + Floating/Pacing
On the floating path
Very sensitive to floating
GPM & CPM Risk Software Comparison
45©2012-2013 Permission is granted to PMA Technologies
GPM Risk as Embodied in NetRisk CPM Risk as Embodied in OPRA
Schedule view relies on time-scaled LDM networks Minimalist schedule view that relies on logic GANTT charts
Planned dates can be used to model SNE constraint dates SNE constraints cannot reflect the network stochastic nature
A risk occurs with unique probability/impact per activity-risk pair A risk occurs with the same probability on all impacted activities
Longest path, sampled path & shortest path logic constructs Longest path & sampled path logic constructs
Floating and pacing risks are modeled as random risks Neither floating nor pacing risks can be modeled
Automated risk removal for risk sensitivity analysis Manual, one-by-one risk removal for risk sensitivity analysis
Multiple simulations within the same file for easy comparisons One simulation per file complicates comparison of results
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
08/11/2012 08/25/2012 09/08/2012 09/22/2012 10/06/2012 10/20/2012
Num
. of Iteratio
ns
Date
PERT Early DatesFloating & Pacing OPRA
NetRisk & OPRA Distribution Functions
46©2012-2013 Permission is granted to PMA Technologies
10/08/2012
09/28/2012
46%
09/23/2012
32%
Includes early‐dates and merge bias
NetRisk discretizes continuous distributions by dividing the range using proper (mathematical) rounding rules, which explains the slight difference in the distribution curves
Unbiased forecast
Includes early‐dates bias
NetRisk & OPRA Criticality Tornado Diagrams
47©2012-2013 Permission is granted to PMA Technologies
1%
1%
1%
1%
20%
21%
21%
20%
20%
27%
27%
27%
58%
58%
58%
58%
58%
77%
77%
100%
100%
0%
0%
0%
0%
20%
20%
20%
20%
20%
32%
32%
32%
52%
52%
52%
52%
52%
71%
71%
100%
100%
0% 20% 40% 60% 80% 100%
FDN Permit
Piping/HVAC/FS Rough‐In
Start Exc, FDN
Comp Exc, FDN
Shops, R & A, Delivery
SOG, Pour & Seal Decks
Power/Lighting/Low Voltage
Steel, Joists, Decking
Bid/Award Steel
BOD Process Equip
Equipment Procurement
Gather Equip Quotes
Substation Fab/Delivery
Subtation Installation
Permit Set
Comp. CD Set
Substation Shops
MEP Process Equip
DD Set
SD Set
Install/Connect Process Equipment
NetRiskOPRA
As a check, an OPRA simulation with 1000 iterations was run, which showed Criticality indices within 2% points of those calculated by NetRisk.
48©2012-2013 Permission is granted to PMA Technologies
TAKE-AWAYS1 Any project or contract schedule that is not risked through its
life cycle does not conform to scheduling best practices
2 Any schedule that does not expressly reserve reasonable schedule margin does not conform to best practices either
3 The CPM optimism bias impacts CPM risk analysis results in that ‘p dates’ are biased early/are optimistic
4 GPM planned dates are better suited to risk modeling than deterministic SNE constraint dates
5 Activity durations should be ranged using benchmarking
6 With schedule margin as critical path float, early completion schedules are the new normal
7 There is a new sheriff in town!
NetRisk Development Path 2013 - 2014
49©2012-2013 Permission is granted to PMA Technologies
1 Visual Risk
2 High-priority User Requests
3 Automated Risk Removal in Risk Sensitivity Analysis
4 Full Stochastic Network Modeling
6 Full Interoperability with Cost Risk Software
7 Integrated Resource Leveling During Simulation
5 Weather Risks
50©2012-2013 Permission is granted to PMA Technologies
REFERENCES
1) Kahneman, D. (2011). Thinking, Fast and Slow. New York: Farrar, Straus & Giroux.
2) Flyvbjerg, B. (2004). Procedures for dealing with optimism bias in transport planning and Flyvbjerg, B. (2008). Curbing optimism bias and strategic misrepresentation in planning: reference class forecasting in practice
3) NASA. (1962). PERT/COST Systems Design. DOD and NASA Guide
4) Van Slyke, R. (1963). Monte Carlo methods and the PERT problem.
5) Pritsker, A. (1966). GERT: Graphical evaluation and review technique.
6) Project Management Institute. (1996). Project management body of knowledge (1st ed.)
7) National Defense Industrial Association. (2011). Planning & scheduling excellence guide (PASEG)
8) United States Government Accountability Office. (2012). GAO schedule assessment guide
9) AACE International. (2012). CPM Schedule risk modeling and analysis: special considerations
10) AACE International. (2012). CPM Schedule risk modeling and analysis: special considerations (p. 7)
11) Kennedy, K. & Thrall, R. (1976). PLANET: A simulation approach to PERT (p. 324).
12) Ponce de Leon, G., Jentzen, G., Fredlund, D., Spittler, P. & Field, D. (2010). Guide to the forensic scheduling body of knowledge Part I
13) Williams, T. (1992). Criticality in stochastic networks
52©2012-2013 Permission is granted to PMA Technologies
A budget reserve is to contractors as red meat is to lions, and they will devour it!
Attributed to Bent Flyvbjerg by Kahneman in Thinking Fast and Slow
Photo source: http://www.vegansoapbox.com/we-are-not-lions/