#NoEstimates Project Planning using Monte Carlo
Transcript of #NoEstimates Project Planning using Monte Carlo
![Page 1: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/1.jpg)
Dimitar Bakardzhiev
Managing Partner
Taller Technologies Bulgaria @dimiterbak
#NoEstimates Project Planning using Monte Carlo
simulation
![Page 2: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/2.jpg)
Clients come to us with an idea for a new
product and they always ask the questions -
how long will it take and how much will it cost
us to deliver? They need a delivery date
and a budget estimate.
![Page 3: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/3.jpg)
WE CAN’T CONTROL THE WAVES OF
UNCERTAINTY, BUT WE CAN LEARN
HOW TO SURF!
![Page 4: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/4.jpg)
TO ME #NOESTIMATES MEANS
No effort estimates
Effortless estimates
No estimates of effort
![Page 5: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/5.jpg)
Deterministic planning used these days forces certainty on uncertain situations and
masks the uncertainty instead of highlighting it.
![Page 6: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/6.jpg)
How can we forecast project delivery time without a detailed schedule - that is assessing the
dependencies between the work, the cost of the work, and
the sequence of the work?
![Page 7: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/7.jpg)
We challenge the project management paradigm and suggest that for
planning purposes it is better to model projects as a flow of work items
through a system.
![Page 8: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/8.jpg)
A project is a batch of work items each one representing
independent customer value that must be delivered on or before
due date.
![Page 9: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/9.jpg)
We don’t try to estimate the size of the work items. There are only two "sizes" - “Small Enough" and
“Too Big". "Too big" should be split and not allowed to enter the
backlog.
![Page 10: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/10.jpg)
A Project in a Kanban System
Input QueueDEPLOYED!
Project Backlog
Development Test QA
WIP 5 WIP 4 NO WIPWIP 2
![Page 11: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/11.jpg)
High-level probabilistic planning
• The initial budget and the range of the time frame • Does not include detailed project plans • The plan is created with the appropriate buffers • Schedules are the execution of the high-level plan • Keep focus on the project intent
![Page 12: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/12.jpg)
Reference class forecasting
The forecast on a given project is based on knowledge about actual performance in a reference class of comparable projects.
Daniel Kahneman
![Page 13: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/13.jpg)
Reference class forecasting • Identification of a relevant reference class of past,
similar projects. The class must be broad enough to be statistically meaningful but narrow enough to be comparable with the specific project.
• Establishing a probability distribution for the selected reference class.
• Comparing the new project with the reference class distribution, in order to establish the most likely outcome for the new project.
![Page 14: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/14.jpg)
IDENTIFICATION OF A REFERENCE CLASS OF
SIMILAR PROJECTS
![Page 15: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/15.jpg)
Are the Team structures comparable?
![Page 16: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/16.jpg)
Are the Technologies used comparable?
![Page 17: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/17.jpg)
Are the Development processes comparable?
![Page 18: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/18.jpg)
Are the Client types comparable?
http://blog.7geese.com/2013/07/04/7-reasons-why-i-decided-to-work-for-a-startup/
![Page 19: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/19.jpg)
Are the Business domains comparable?
http://www.mindoceantech.com/
![Page 20: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/20.jpg)
ESTABLISHING A PROBABILITY
DISTRIBUTION FOR THE SELECTED
REFERENCE CLASS
![Page 21: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/21.jpg)
WHAT METRIC WILL BE USED IN THE FORECAST?
Takt Time!
![Page 22: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/22.jpg)
Takt Time is the time between two successive deliveries
![Page 23: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/23.jpg)
How manufacturing measure Takt Time?
![Page 24: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/24.jpg)
How knowledge workers measure Takt Time?
![Page 25: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/25.jpg)
Takt Time (TT) is the time between two successive deliveries
Start 5 days 7 days 2 days 2 days 1 day 5 days Finish
TT = 0 days
TT = 0 days
TT = 5 days TT = 7 days
Project delivery time (T) = 5 + 7 + 2 + 2 + 1 + 5 = 22 days
![Page 26: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/26.jpg)
![Page 27: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/27.jpg)
Average Takt Time
𝑇𝑇 =𝑇𝑁
• T is the time period over which the project was delivered • N is the number of items to be delivered in period [0,T] • 𝑇𝑇 is the Takt Time for period [0,T]
![Page 28: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/28.jpg)
Average TT calculation
𝑇𝑇 =𝑇
𝑁=22 𝑑𝑎𝑦𝑠
10 𝑠𝑡𝑜𝑟𝑖𝑒𝑠
= 2.2 𝑑𝑎𝑦𝑠/𝑠𝑡𝑜𝑟𝑦
![Page 29: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/29.jpg)
Average Project Delivery time
𝑇 = 𝑁𝑇𝑇 • T is the time period over which the project will be delivered
N is the number of items to be delivered in period [0,T] • 𝑇𝑇 is the Takt Time for period [0,T]
![Page 30: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/30.jpg)
Project Delivery time
𝑇 = 𝑁𝑇𝑇 =
45 𝑠𝑡𝑜𝑟𝑖𝑒𝑠 2.2 𝑑𝑎𝑦𝑠
𝑠𝑡𝑜𝑟𝑦= 99 𝑑𝑎𝑦𝑠
![Page 31: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/31.jpg)
We should NOT use the Average Takt Time as a single number but a
distribution of the average Takt Time instead!
![Page 32: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/32.jpg)
Bootstrapping • Introduced by Bradley Efron in 1979
• Based on the assumption that a random sample is a
good representation of the unknown population.
• Does not replace or add to the original data.
• Bootstrap distributions usually approximate the shape, spread, and bias of the actual sampling distribution.
• Bootstrap is based on the assumption of independence.
![Page 33: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/33.jpg)
1. Have Takt Time (TT) sample of size n 2. Have the number of work items delivered (N) 3. Draw the same number of observation 𝑻𝑻𝒊 as the
sample size n with replacement out of the sample from step 1
4. Calculate Project Delivery time (T) for the sample from step 2 using 𝑻 = 𝑻𝑻𝒊
5. Calculate Takt Time (TT) by 𝑻𝑻 = 𝑻/𝑵 using T from step 3 and N from step 2
6. Repeat many times 7. Prepare distribution for Takt Time (TT)
Bootstrapping the distribution of Takt Time
![Page 34: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/34.jpg)
Example: Monte Carlo simulation of Takt Time (TT)
Sampled Takt Time data 𝑻𝑻𝒊=(0,0,1,1,1,2,2,2,5,7)
𝑻 = 𝑻𝑻𝒊 = 𝟐𝟏 𝒅𝒂𝒚𝒔
𝑻𝑻 = 𝑻/𝑵 = 2.1 days/story
Another 998 draws with replacement
Historical Takt Time data 𝑻𝑻𝒊=(0,0,0,0,1,2,2,5,5,7)
𝑻 = 𝑻𝑻𝒊 = 𝟐𝟐 𝒅𝒂𝒚𝒔
𝑻𝑻 = 𝑻/𝑵 = 2.2 days/story Sampled Takt Time data 𝑻𝑻𝒊=(0,1,1,1,1,2,5,5,5,7)
𝑻 = 𝑻𝑻𝒊 = 𝟐𝟖 𝒅𝒂𝒚𝒔
𝑻𝑻 = 𝑻/𝑵 = 2.8 days/story
1st draw with replacement
1000th draw with replacement
![Page 35: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/35.jpg)
Result: Takt Time (TT) distribution
Median 2,2
STD 0,788833
Average T 2,1943
85 Perc 3
95 Perc 3,5
Mode(s) 2,4
SIP size 1000
![Page 36: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/36.jpg)
Stochastic Information Packet (SIP) • Comprised of a list of trials of some uncertain
parameter or metric generated from historical data using Monte Carlo simulation (resampling)
• Represents an uncertainty as an array of possible outcomes (distribution)
• It is unique per context (business domain, team, delivery process used etc.)
![Page 37: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/37.jpg)
COMPARING THE NEW PROJECT WITH
THE REFERENCE CLASS
DISTRIBUTION
![Page 38: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/38.jpg)
𝑇 = 𝑁𝑇𝑇 assumes linear delivery rate
Project Delivery Time (T)
Project Delivery Time (T)
Completed Work (N)
22 days
10 work items
![Page 39: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/39.jpg)
Most projects have non-linear delivery rate
![Page 40: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/40.jpg)
Z-curve
![Page 41: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/41.jpg)
Each leg of the Z-curve is characterized by:
• Different work type • Different level of variation • Different staffing in terms of headcount and level of
expertise
![Page 42: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/42.jpg)
1st leg – Setup time
• climbing the learning curve • conducting experiments to cover the riskiest work
items • Innovation! • setting up environments • adapting to client’s culture and procedures • understanding new business domain • mastering new technology
![Page 43: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/43.jpg)
2nd leg – Productivity period If the project is scheduled properly the system should be like a clockwork – sustainable pace, no stress, no surprises…
![Page 44: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/44.jpg)
3rd leg – Cleaning up • Clean up the battlefield • Fix some outstanding defects • Support the transition of the project deliverable into
operation
https://www.ocoos.com/me/professional-dog-training-in-home/
![Page 45: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/45.jpg)
Project delivery time T
𝑇 = 𝑇𝑧1 + 𝑇𝑧2 + 𝑇𝑧3 Where: 𝑇𝑧1 – is the duration of the 1st leg of the Z-curve 𝑇𝑧2 – is the duration of the 2nd leg of the Z-curve 𝑇𝑧3 – is the duration of the 3rd leg of the Z-curve
![Page 46: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/46.jpg)
Project delivery time T
𝑇 = 𝑁𝑧1𝑇𝑇𝑧1 + 𝑁𝑧2𝑇𝑇𝑧2 + 𝑁𝑧3𝑇𝑇𝑧3 Where: 𝑇𝑇𝑧1 is the Takt Time for the 1st leg of the Z-curve
𝑇𝑇𝑧2 is the Takt Time for the 2nd leg of the Z-curve
𝑇𝑇𝑧3 is the Takt Time for the 3rd leg of the Z-curve
𝑁𝑧1 is the number of items delivered during the 1st leg of the Z-curve
𝑁𝑧2 is the number of items delivered during the 2nd leg of the Z-curve
𝑁𝑧3 is the number of items delivered during the 3rd leg of the Z-curve
![Page 47: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/47.jpg)
Monte Carlo simulation of Project Delivery Time (T) based on Z-curve
1. Have three Takt Time SIPs (𝑇𝑇𝑧1, 𝑇𝑇𝑧2, 𝑇𝑇𝑧3) each one of size n for each of the three legs of the Z-curve
2. Have the number of work items to be delivered for each of the three legs of the Z-curve (𝑁𝑧1, 𝑁𝑧2, 𝑁𝑧3)
3. Draw one observation out of the n, with replacement (bootstrap) from each of (𝑇𝑇𝑧1, 𝑇𝑇𝑧2, 𝑇𝑇𝑧3)
4. Calculate Project Delivery time (T) for the sample from step 3 using 𝑇 = 𝑁𝑧1𝑇𝑇𝑧1 +𝑁𝑧2𝑇𝑇𝑧2 + 𝑁𝑧3𝑇𝑇𝑧3
5. Repeat many times 6. Prepare Delivery time (T) probability distribution
![Page 48: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/48.jpg)
EXAMPLE: MONTE CARLO SIMULATION OF PROJECT DELIVERY TIME (T)
![Page 49: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/49.jpg)
The New Project to be delivered
• THE SAME Fortune 500 Staffing company
• THE SAME development organization
• THE SAME technology – Java; Spring; Oracle;
• Delivery time TO BE PREDICTED
![Page 50: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/50.jpg)
Takt Time distributions for each of the three legs of Z-curve for the reference
class
![Page 51: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/51.jpg)
Project scope After some analysis the team have broken down the requirements into user stories, accounting for Cost of Delay, added work items for Dark matter and Failure load and decided that:
• 12 stories TO BE delivered in the 1st leg of Z-curve
• 70 stories TO BE delivered in the 2nd leg of Z-curve
• 18 stories TO BE delivered in the 3rd leg of Z-curve
![Page 52: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/52.jpg)
Monte Carlo simulated summation of…
…will give us the time needed to deliver the project!
12 work items 70 work items 18 work items
![Page 53: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/53.jpg)
Monte Carlo simulation of Project Delivery Time (T)
Simulated one Project Delivery Time value 𝑻 = 𝑁𝑧1𝑇𝑇𝑧1 + 𝑁𝑧2𝑇𝑇𝑧2 + 𝑁𝑧3𝑇𝑇𝑧3= 12 × 1.43 + 70 × 0.3 + 18× 1.11 = 58.14 𝑑𝑎𝑦𝑠
49998 draws with replacement from each of (𝑇𝑇𝑧1, 𝑇𝑇𝑧2, 𝑇𝑇𝑧3)
Takt Time SIPs: 𝑇𝑇𝑧1, 𝑇𝑇𝑧2, 𝑇𝑇𝑧3 Work items: 𝑁𝑧1, 𝑁𝑧2, 𝑁𝑧3
1st draw with replacement from each of (𝑇𝑇𝑧1, 𝑇𝑇𝑧2, 𝑇𝑇𝑧3)
50000th draw with replacement from each of (𝑇𝑇𝑧1, 𝑇𝑇𝑧2, 𝑇𝑇𝑧3)
Simulated one Project Delivery Time value 𝑻 = 𝑁𝑧1𝑇𝑇𝑧1 +𝑁𝑧2𝑇𝑇𝑧2 + 𝑁𝑧3𝑇𝑇𝑧3= 12 × 1.81 + 70 × 0.54 + 18× 0.64 = 71.04 𝑑𝑎𝑦𝑠
![Page 54: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/54.jpg)
Mode = 76 days; Median = 77 days; Mean = 78 days; 85th perc = 90 days
![Page 55: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/55.jpg)
By taking an outside view when forecasting a new project we will
produce more accurate results faster than using the deterministic inside
view.
![Page 56: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/56.jpg)
References Here are the distributions for the baseline project SIPs_MonteCarlo_FVR.xlsx Here is the planning simulation in Excel High_Level_Project_Planning.xlsx What is SIP?
![Page 57: #NoEstimates Project Planning using Monte Carlo](https://reader031.fdocuments.net/reader031/viewer/2022013015/61cffe784ac0375c6f451532/html5/thumbnails/57.jpg)
Dimitar Bakardzhiev is the Managing Director of Taller Technologies Bulgaria and an expert in driving successful and cost-effective technology development. As a Lean-Kanban University (LKU)-Accredited Kanban Trainer (AKT) and avid, expert Kanban practitioner, Dimitar puts lean principles to work every day when managing complex software projects with a special focus on building innovative, powerful mobile CRM solutions. Dimitar has been one of the leading proponents and evangelists of Kanban in his native Bulgaria and has published David Anderson’s Kanban book as well as books by Eli Goldratt and W. Edwards Deming in the local language. He is also a lecturer and frequent speaker at numerous conferences and his passion is to educate audiences on the benefits of lean principles and agile methodologies for software development.
@dimiterbak