presentation
-
Upload
softwarecentral -
Category
Documents
-
view
230 -
download
0
Transcript of presentation
University of Southern California
Center for Systems and Software Engineering
A Tractable Approach to Handling Software Productivity Domains
Thomas TanBrad Clark
24th International Forum on COCOMO and Systems/Software Cost Modeling
November 3, 2009
24th International Forum on COCOMO and Systems/Software Cost Modeling 2
University of Southern California
Center for Systems and Software Engineering
Table of Contents• Overview• Operational Context• Software Application Difficulties• Productivity Group Matrix
– Example Mapping of Systems– Aerospace Project Data Analysis
• COCOMO II Implications• Next Steps• Q&A
This work is sponsored by the Air Force Cost Analysis Agency
24th International Forum on COCOMO and Systems/Software Cost Modeling 3
University of Southern California
Center for Systems and Software Engineering
Overview• Conventional labels used in
Software Productivity Domains use system engineering terms
• Systems can be comprised of many different software applications, e.g. Command and Control
• Productivities are influenced by the difficulty of the application type and the operating environment
• Many applications in a system can exhibit different productivities
Command & Control System
Database
Controls & Displays
Simulator
Training System
Telemetry Processing
Sensor Processing
Communications
Information Assurance
Testbed
24th International Forum on COCOMO and Systems/Software Cost Modeling 4
University of Southern California
Center for Systems and Software Engineering
Motivation• Our motivation is to create a new productivity classification
system that can be– Understood by both cost estimators and engineers– Flexible enough to accommodate a wide range of software
applications
• This presentation discusses this new approach• As a work-in-progress, we welcome comments and suggestions• Please join us at the workshop to discuss this approach and its
usage
24th International Forum on COCOMO and Systems/Software Cost Modeling 5
University of Southern California
Center for Systems and Software Engineering
Overview• This approach employs general descriptions to characterize the
intended “operational context” (environment) and an application’s “degree of difficulty”:– Operational Context describes the intended environment, platform or
target host on which the software application will operate – The Application Difficulty is the degree of difficulty in specifying,
developing, and testing a software application
24th International Forum on COCOMO and Systems/Software Cost Modeling 6
University of Southern California
Center for Systems and Software Engineering
Operational Context
Very Unconstrained Virtually unlimited
resources Accessible physical
environment Stationary
conditions Stable platform
Very Constrained Limited physical
volume, power & weight
Unmanned Inaccessible
physical environment
Evolving platform Hard real-time
Un-Manned Space Bus
Manned Space Vehicle
Missile
Un-Manned Airborne
Airborne Shipboard
Mobile Ground
Land Site
Har
dwar
e P
latf
orm
s
Dimensions of constraints include electrical power, computing capacity, storage capacity, repair capability, platform volatility, physical environment accessibility, etc.
24th International Forum on COCOMO and Systems/Software Cost Modeling 7
University of Southern California
Center for Systems and Software Engineering
Operational Context• Very Unconstrained
– Unlimited electrical power with backup battery– Regular upgrades and maintenance– Heavily COTS-hardware and COTS-software dependent– Supports multiple users– Limited loss during system down time
• Unconstrained– Operating environment may move over land and water– Electrical power is limited by mobility (size dependent)– Computing and storage capability depends on ability to carry weight and size– Upgrades and maintenance are not frequent– Components are modular and cannot be disassembled
24th International Forum on COCOMO and Systems/Software Cost Modeling 8
University of Southern California
Center for Systems and Software Engineering
Operational Context• Constrained
– Similar to unconstrained in terms of upgrade, maintenance, and component structure
– Operating environment may move through the air– Electrical power, computing and storage capability are limited due to weight
constraints
• Very Constrained– Battery is the primary source of electrical power. – Components are miniaturized and custom-made with special tools to access– No ability to upgrade and maintain system after deployment
• Highly Constrained– Hardware is required to survive in adverse conditions– Limited software control– Real-time sensor inputs and commands– Upgrades and maintenance is not feasible after deployment– Unique system platform that is evolving in maturity as system is being
developed
24th International Forum on COCOMO and Systems/Software Cost Modeling 9
University of Southern California
Center for Systems and Software Engineering
Software Application Difficulties
Very Easy
Simple handling of events / inputs
Relies on O/S or middleware for control
Can be restarted with minor inconvenience
Very Challenging Autonomous
operation Extremely high
reliability Complex algorithms Micro-second
control loop Micro-code
programming
Firm
ware
(R
OM
)
Att
itude C
ontr
ol
Guid
ance /
Navig
ation
Radar
/ S
onar
/ T
ele
metr
y
pro
cessi
ng
Pro
cess
Contr
ol
Seis
mic
Pro
cess
ing
Fact
ory
Auto
mation
Dig
ital S
witc
hes
& P
BX
Case
Tools
/ C
om
pile
rs
Website
Auto
mation
Busin
ess
Syste
ms
Difficulty would be described in terms of required software reliability, database size, product complexity, integration complexity, information assurance, real-time requirements, different levels of developmental risks, etc.
24th International Forum on COCOMO and Systems/Software Cost Modeling 10
University of Southern California
Center for Systems and Software Engineering
Software Application Difficulties• Very Easy
– Risks are well understood with little loss from failure– Business or operational logic is straightforward– Limited interface to other software applications– Mostly stand-alone functionality– Simple tests
• Easy– Not a new type of application– Risks are understood and mitigation strategies exists– Business or operational logic is straightforward– Requires low reliability due to small or little loss when unavailable– Limited external interface and security requirements
• Nominal– Somewhat complicated business logic– Risks exists and may need additional study to find mitigation– May require distributed environment with additional security requirements– Moderate, easily recoverable loss for nominal reliability– Not a new type of application
24th International Forum on COCOMO and Systems/Software Cost Modeling 11
University of Southern California
Center for Systems and Software Engineering
Software Application Difficulties• Challenging
– High reliability due to greater impact of loss or high probability of risk– Risks are challenging to resolve– Very complicated business logic, external storage may be necessary due to distributed
environment– New type of application– Hard real-time control and security requirements– Additional communication interfaces necessary for external components or systems
• Very Challenging– Extremely complicated business logic– Risks are very challenging to resolve and loss is great (disastrous consequences)– Many automated controls with limited human control– New type of application– Hard real-time control and security requirements– Communication to external components through different interfaces
24th International Forum on COCOMO and Systems/Software Cost Modeling 12
University of Southern California
Center for Systems and Software Engineering
Productivity Group Matrix
Productivity Group Matrix
5,55,45,35,25,1Very
Challenging
4,54,44,34,24,1Challenging
3,53,43,33,23,1Nominal
2,52,42,32,22,1Easy
1,51,41,31,21,1Very Easy
Extremely Constrained
Very ConstrainedConstrained
Uncon-strained
Very Uncon-strained
Application Difficulty
Operational Context
• The intersection of an Operational Context rating and Application Difficulty rating represent a Productivity Group
– Each group includes the number of project data points, data range, simple cost estimating relationship (of the form: Y = aXb), standard error, percent bias, and the coefficient of determination, R2
24th International Forum on COCOMO and Systems/Software Cost Modeling 13
University of Southern California
Center for Systems and Software Engineering
Example Mapping to Systems• Very Unconstrained and Very Easy may include systems like:
– Business Analysis Systems– Data Warehousing Systems– Diagnostic Systems
• Very Unconstrained and Easy may include systems like:– Logistics Systems– Training Systems– Support Systems– Test Systems
• Constrained and Challenging may include systems like:– Flight Systems– Radar Systems– Signal Processing
• We welcome comments and suggestions on mappings of systems to the productivity groups.
24th International Forum on COCOMO and Systems/Software Cost Modeling 14
University of Southern California
Center for Systems and Software Engineering
Data Analysis Experiment• Extracted data from Aerospace Corp. report*
– 188 data records– 4 Operating Environments– 8 Application Domains
• Our experiment– Select a Challenging Application: Signal Processing– Select an Easy Application: Test– Calculate estimating relationship (in Y = aXb form), standard error, and R2
– Our hypothesis is that Easy Applications will have higher productivity than Challenging Applications
– Operational Context not considered in this experiment
* Gayek, Long, Bell and Larson, “Software Cost and Productivity Model,” Aerospace Report ATR-2004(8311)-1, The Aerospace Corporation, El Segundo, CA, Feb 04
24th International Forum on COCOMO and Systems/Software Cost Modeling 15
University of Southern California
Center for Systems and Software Engineering
Aerospace Project Data Analysis
• Signal Processing– 40 records– Estimating relationship for
effort• Effort = 11.82 x (Size) 0.93
• R2 = 0.7• Standard Error = 0.57
Size vs. Effort
0.0
200.0
400.0
600.0
800.0
1000.0
1200.0
1400.0
0.0 50.0 100.0 150.0 200.0
Size (KESLOC)E
ffor
t (P
M)
24th International Forum on COCOMO and Systems/Software Cost Modeling 16
University of Southern California
Center for Systems and Software Engineering
Aerospace Project Data Analysis
• Test:– 16 records– Estimating relationship for
effort• Effort = 7.46 x (Size) 0.97
• R2 = 0.68• Standard Error = 0.53
Size vs. Effort
0.0
100.0
200.0
300.0
400.0
500.0
600.0
0.0 10.0 20.0 30.0 40.0 50.0
Size (KESLOC)E
ffo
rt (
PM
)
24th International Forum on COCOMO and Systems/Software Cost Modeling 17
University of Southern California
Center for Systems and Software Engineering
Aerospace Project Data Analysis• The estimating relationships were used to compare Easy and
Challenging Application Difficulty– Size in KSLOC– Effort in PM– Productivity in SLOC/PM
13138251623096500
12316311571273200
117856154650100
11144915133250
10419214713620
991011447010
ProductivityEffort ProductivityEffortSize
Challenging: Signal ProcessingVery Easy: Test
24th International Forum on COCOMO and Systems/Software Cost Modeling 18
University of Southern California
Center for Systems and Software Engineering
Challenging Difficulty Plot
Challenging Application Difficulty
0
200
400
600
800
1,000
1,200
1,400
0 20 40 60 80 100 120 140 160
Size (ESLOC)
Eff
ort
(P
M)
Very Unconstrained Unconstrained Constrained
Very Constrained (Power (Very Constrained (Power (Unconstrained
(Power (Very Unconstrained
Very Unconstrained
UnconstrainedVery Constrained
24th International Forum on COCOMO and Systems/Software Cost Modeling 19
University of Southern California
Center for Systems and Software Engineering
Easy Difficulty Plot
Easy Application Difficulty
0
100
200
300
400
500
600
0 5 10 15 20 25 30 35 40 45 50
Size (KESLOC)
Eff
ort
(P
M)
Very Unconstrained Constrained Very Constrained
Unconstrained (Power (Very Unconstrained
Very Unconstrained
24th International Forum on COCOMO and Systems/Software Cost Modeling 20
University of Southern California
Center for Systems and Software Engineering
Productivity Group Matrix – 2nd Use• Productivity Group Matrix is “Cost Model” independent• The matrix could be used to “pre-set” parameters for different models, e.g.
– COCOMO II– SEER– True-S– SLIM
• Possibly, the matrix could be extended to have different distributions for effort and duration for lifecycle activities, i.e.
– Each cell has its own distribution
5,55,45,35,25,1Very
Challenging
4,54,44,34,24,1Challenging
3,53,43,33,23,1Nominal
2,52,42,32,22,1Easy
1,51,41,31,21,1Very Easy
Extremely Constrained
Very ConstrainedConstrained
Uncon-strained
Very Uncon-strained
Application Difficulty
Operational Context
24th International Forum on COCOMO and Systems/Software Cost Modeling 21
University of Southern California
Center for Systems and Software Engineering
COCOMO II Implications
Productivity Groups may be used to to pre-set COCOMO II Parameters. This is also a work-in-progress.
XHXHVHHNSTOR
XHXHVHHNTIME
VHVHHNLPVOL
XHVHHN, LVLCPLX-Device
VLVLLNHRESL
VLVLLNHPRED
Extremely Constrained
Very ConstrainedConstrainedUnconstrained
Very Unconstrained
COCOMO II Parameters
Pre-sets driven by Operational Context*
* These ratings have not been verified with data, they for illustration only
24th International Forum on COCOMO and Systems/Software Cost Modeling 22
University of Southern California
Center for Systems and Software Engineering
COCOMO II Implications
Pre-sets driven by Application Difficulty*
VHVHHNLDATA
XHVHHN, LVLCPLX-UI
XHVHHN, LVLCPLX-Data
XHVHHN, LVLCPLX-Computations
XHVHHN, LVLCPLX-Control
VHHNLVLRELY
VLVLLNHRESL
VLVLLNHPRED
Very ChallengingChallengingNominalEasyVery EasyFactors
* These ratings have not been verified with data, they for illustration only
24th International Forum on COCOMO and Systems/Software Cost Modeling 23
University of Southern California
Center for Systems and Software Engineering
COCOMO II Implications• Factors captured by local calibration of COCOMO constants A
and B:– Security and Information assurance (given complexity is at least high)
• Very Easy would have no security or information assurance requirements
• Very Challenging would require the software to authenticate access, handle denial of service attacks, ensure data integrity, etc.
– Components or Sub-system Interactions• Heterogeneous components require special interfaces to
communicate with each other• Homogeneous components are generally built with similar
communication interface
24th International Forum on COCOMO and Systems/Software Cost Modeling 24
University of Southern California
Center for Systems and Software Engineering
COCOMO II Implications• COCOMO Parameters not addressed:
– Development Flexibility– Team Cohesion– Process Maturity– Developed for Reuse– Analyst Capability– Programmer Capability– Personnel Continuity– Applications Experience– Platform Experience– Language and Tool Experience– Use of Software Tools– Multi-Site Development
• These are usually driven by business decisions
• May be indirectly driven by Difficulty or Context
24th International Forum on COCOMO and Systems/Software Cost Modeling 25
University of Southern California
Center for Systems and Software Engineering
Next Steps• The proposed approach to classifying software application
productivities by Operational Context and Application Difficulties needs to be verified with the government agencies and industrial communities that will use the approach for software project cost estimates.
• Solicit input through workshops and document reviews– Verify ratings for operational context and application difficulties.– Assistance in characterization of applications and determining where
they fit in the productivity group matrix
• Analyze project cost data for productivity: each project will be characterized by Operational Context and Application Difficulties
• If you would like to participate — please give us your contact information
24th International Forum on COCOMO and Systems/Software Cost Modeling 26
University of Southern California
Center for Systems and Software Engineering
Questions?
For more information, contact:Thomas [email protected]