ISMA 9 - van Heeringen - Using IFPUG and ISBSG to improve organization success
-
Upload
harold-van-heeringen -
Category
Business
-
view
276 -
download
0
description
Transcript of ISMA 9 - van Heeringen - Using IFPUG and ISBSG to improve organization success
Harold van Heeringen, ISBSG president
Harold van HeeringenPresident ISBSG International Software Benchmarking Standards Group (www.isbsg.org)
Board member NESMA Netherlands Software Metrics Association (www.nesma.nl)
IAC member COSMIC Common Software Measurement International Consortium (www.cosmicon.com)
Senior Cost Engineer, Sogeti Nederland B.V. (www.sogeti.com)
OverviewISBSG introductionIFPUG and ISBSG in practice: 3 cases
Reality check of an expert estimate;Competitiveness assessment of an IT
department;Selecting the right (sole) supplier to do all the
IT and manage the supplier based on productivity.
ISBSG: other useful purposesHow does Spain compare to the industry
We measured 350 FP, so... What is the estimate? Effort, duration, quality, risk? Price per FP?
Why are we losing most of the bids we do? Competitive advantage? Or not? What is our productivity compared to the industry?
We are going to send out a request for bid based on price/FP How can we compare the proposals? How can we exclude unrealistic (overoptimistic) ones? What should the market average bid approximately look like?
We did a project and the client thinks it was too expensive How did we perform against similar projects?
Size does matter, but….
Historical data of completed projects!!
The answer…Without software metrics based on an objective unit of measure, there is little we can do. To be able to analyze and to make decisions, we need:
International Software Benchmarking Standards GroupIndependent and not-for-profit;Full Members are non-profit organizations, like IFPUG, NESMA,
GUFPI-ISMA, FiSMA, QESP, DASMA, JFPUG, Swiss-ICT and CESI;Associate members: AEMES (Asociacion Espanola de Metricas de
Software), ASSEMI (France);Grows and exploits two repositories of software data:
New development projects and enhancements (> 6000 projects); Maintenance and support (> 1200 applications).
Everybody can submit project data DCQ on the site / on request (.xls) Anonymous Free benchmark report in return
ISBSG
Mission: “To improve the management of IT resources by both business and government, through the provision and exploitation of public repositories of software engineering knowledge that are standardized, verified, recent and representative of current technologies”.
All ISBSG data is validated and rated in accordance with its quality guidelines current representative of the industry independent and trusted captured from a range of organization sizes and industries
ISBSG
www.isbsg.org
ISBSG wishes to help restore the confidence in IT, by giving transparency of what is realistic, resulting inImproved project success rates;Better customer satisfaction;Shorter time to market;More efficient software development;Better alignment with the business/customer;Easier contracting, resulting in better
relationships between customer and supplier.
ISBSG IT Confidence
3 Cases from my experience
Case 1: Telecom project reality check on the expert estimate (project manager)
Case 2: Assessing the competitive position of an organization (Senior management)
Case 3: Supplier Performance Measurement (procurement)
Case 1: Reality Check
Case 1A telecom company wished to develop a new Java system
for the maintenance of subscription types;
A team of experts studied the requirements documents and filled in the WBS-based estimation calculation (bottom-up estimate);
They decide that an estimate of 5.500 hours and a duration of 6 months should be feasible;
The project manager decided not to believe the experts ‘on their blue eyes’ only and wished to carry out a reality check.
Why a realistic estimate?Non-linear extra costs
-Planning errors
-team enlargement more expensive, not faster
-Extra management attention / overhead
-Stress: More defects, lower maintainability !!
Linear extra costs
Extra hours will be used
ISBSG Reality Check: EffortAn estimated FPA comes up with the expected size:
Min: 550 FP, likely 850 FP, Max 1300 FPImplicit likely expert PDR: 5.500/850 = 6,5 h/FP
Selecting the most relevant projects in the ISBSG D&E repository show the next results:
PDR (h/FP)Min. 3,2Percentile 10% 4,3Percentile 25% 6,2Median 8,9Percentile 75% 12,9Percentile 90% 19,8Max. 34,2N 89
PDR (h/FP)Min. 3,2Percentile 10% 4,3Percentile 25% 6,2Median 8,9Percentile 75% 12,9Percentile 90% 19,8Max. 34,2N 89
550 850 1300
3.410 5.270 8.0604.895 7.565 11.5707.095 10.965 16.770
Functional Size550 850 1300
3.410 5.270 8.0604.895 7.565 11.5707.095 10.965 16.770
Functional Size
5.500 hours seems optimistic
ISBSG Reality Check: DurationSame analysis is possibleAlso, formulas have been published in the Practical
Project Estimation bookFor instance:table C-2.2 Project Duration, estimated from software size onlyFunctionele omvang 550 FPC uit tabel 0,507E1 uit tabel 0,429Duration = C * Size^E1Duration = 7,6 elapsed months
550 850 1300Duration 7,6 9,2 11,0
ResultExpert estimate was assessed optimisticAdjusted Estimate:
Effort: 8.000 hoursDuration: 10 months
This turned out to be quite accurate!
The project manager now always carries out reality checks and is ‘spreading the word’.
Case 2: Competitiveness
Case 2Senior management of a software company wondered
how competitive they were when it comes to productivity.
Many bids for projects were lost and they wished to improve, especially their Microsoft .Net department.
Analysis of the bids by department showed the next figures:Nr. of bids 23
Average PDR in bid 16,3 h/FPAverage Size (FP) 230 FPAverage teamsize 6 fte
PDR (h/FP)Min. 3,2Percentile 10% 3,8Percentile 25% 5,9Median 7,6Percentile 75% 12,9Percentile 90% 18,9Max. 34,2N 35
ISBSG data analysis
IssuesAnalysis of the bid phase showed a number of issues:
Estimates were extremely pessimistic due to severe penalties in case of overruns;
In a number of stages, risk surcharges were added;They wished to work in fixed team of 6 fte, but
ISBSG data shows that the project size was usually too small for this teams size to be efficient;
Because of the knowledge that the department bids were not market average (or better), the bid process was redesigned, making the company more successful!
Case 3: Supplier selection and management
Case 3An organization decided to outsource all of
their IT work to one supplier;Therefore, the internal competition has been
gone and potentially the chosen supplier could charge whatever they wish;
The ISBSG data was used to assess the productivity and competitiveness of the suppliers;
After the bidding procedure, the chosen supplier had to commit to productivity targets and quality targets.
Bidding process5 ICT suppliers bid for the 5-year contract;They had to submit data of 6 completed projects,
sized in IFPUG function points;The company metrics desk (MD) assessed the
productivity and competitiveness of the organizations;The MD constructed a model to do the assessment;The model was part of the overall bid process and an
important part of the selection of the one winner;The data submitted was also the basis for the baseline
performance that he had to live up to.
Assessment modelReality value; Compliancy value to data requirements;Productivity / Quality value.
Project Delivery Rate (PDR) = spent project effort related to function point (h/FP);
Productivity Index (PI) = metric from QSM, derived from size, duration and effort;
Quality: delivered defects per FP;
Benchmarks:• PI against the QSM Business trend line• PDR against ISBSG Repository• Adjusted = normalised to Construction+Test activities
Productivity/quality assessment
Supplier PI scoreRank
PI scorePoints
PI score
Supplier A 3,9 2 8
Supplier B 5,0 1 10
Supplier C 3,4 3 6
Supplier D 3,0 5 2
Supplier E 3,2 4 4
SupplierPDR score
Rank PDR score
Points PDR score
Supplier A -3,2 1 10
Supplier B -2,1 2 8
Supplier C 16,6 4 4
Supplier D 6,2 3 6
Supplier E 18,3 5 2
SupplierQuality Score
Rank Quality score
Points Quality score
Supplier A 3,1 1 10
Supplier B 13,9 2 8
Supplier C 52,6 3 6
Supplier D 1000,0 5 2
Supplier E 94,6 4 4
SupplierPoints
PI scorePoints
PDR scorePoints
Quality scoreProductivity/ Quality value
Supplier A 8 10 10 9,0
Supplier B 10 8 8 9,0
Supplier C 6 4 6 5,4
Supplier D 2 6 2 3,2
Supplier E 4 2 4 3,4
weight 50% 30% 20%
ResultsSupplier
Compliancy value
Reality value
Productivity/Quality value
Total Points Rank
Supplier A 0 6 9,0 7,2 2
Supplier B 0 8 9,0 7,8 1
Supplier C 4 10 5,4 6,6 3
Supplier D 0 10 3,2 4,9 4
Supplier E 0 8 3,4 4,4 5
weight 10% 30% 60%
Metrics Based Software Supplier Selection - Kuijpers & van Heeringen (IWSM2012)
ISBSG benchmark
Supplier targets and management
7
8
9
10
11
12
13
PDR
(h/F
P)
Target PDR PDR (h/FP)
500
600
700
800
900
1000
1100
1200
1300
EUR/
FP
Target PCR PCR (EUR/FP)
0
5
10
15
20
25
30
35
40
45
jan-13 feb-13 mrt-13 apr-13 mei-13 jun-13 jul-13 aug-13 sep-13 okt-13
Target Defects/1000 FP Defects/1000 FP
The supplier is measured continuously and still has to make his target for the first time!
The organization is happy that the trends show improvement and they feel in control.
Other useful purposes of ISBSG dataAnalyze the difference in productivity or
quality between two (or more) types of projects:Traditional vs. AgileOutsourced vs. In-houseGovernment vs. Non-governmentOne site, multi siteReuse vs. no reuse
ISBSG Special Analysis reports
Special reports, examplesImpact of Software Size on ProductivityGovernment and Non-Government Software Project
PerformanceISBSG Software Industry Performance reportISBSG/COSMIC The Performance of Business
Application, Real-Time and Component Software Projects
Estimates – How accurate are they?Planning Projects – Role PercentagesTeam size impact on productivity Manage your M&S environment – what to expect?Many more
2013 Country Analysis Report
USA
Thank you!
NEtherlands Software Metrics users Association
Historical landmarksHistorical landmarks
Modern landmarksModern landmarks
European Cultural Capital 2001European Cultural Capital 2001
The city of modern architectureThe city of modern architecture
Largest soccerstadium in the NetherlandsLargest soccerstadium in the Netherlands
Welcome toWelcome to
RotterdamRotterdam
Busiest port in EuropeBusiest port in Europe