Post on 18-Jan-2019
Requirements Quality Management How to deal with requirements quality all
along the life-cycle
REConf, 2016
29.February.2016
José Fuentes (The REUSE Company)
Martin Schleiermacher (XTRONIC)
2 © The REUSE Company, 2016
Content
Impact of low quality requirements
The Systems and Requirements Engineering life-cycles
Requirements authoring
The Requirements Quality Suite
Live demo
Q&A
Requirements Quality Management
3 © The REUSE Company, 2016
The impact of low quality requirements
Requirements Quality Management
The impact of low quality
requirements
6 © The REUSE Company, 2016
The impact of low quality requirements
Requirements Quality Management
In some cases:
But clearly, in every case:
7 © The REUSE Company, 2016
The impact of low quality requirements
Chaos report and some other surveys (e.g. PMI: Pulse of the
profession study):
[“40%-70% of defects in the projects are related to requirements. ”]
Capers Jones:
[“The average rework is over 40%-50%.”]
Requirements Quality Management
Project Success Factors % of Responses 1. User Involvement 15.9%
2. Executive Management Support 13.9%
3. Clear Statement of Requirements 13.0%
4. Proper Planning 9.6%
5. Realistic Expectations 8.2%
6. Smaller Project Milestones 7.7%
7. Competent Staff 7.2%
8. Ownership 5.3%
9. Clear Vision & Objectives 2.9%
10. Hard-Working, Focused Staff 2.4%
Other 13.9%
Source:
Chaos Report 2004
8 © The REUSE Company, 2016
The impact of low quality requirements
Requirements Quality Management
50%
29%
21%
30%
52%
18%
20%
22%
58%
Requirements Engineering Capability
Pro
ject P
erf
orm
an
ce
s
Correlation between Project Performances and
Requirement Engineering Capability
Source: Report on SE
Effectiveness Survey NDIA-IEEE-
SEI/CMU-INCOSE Nov 2012
9 © The REUSE Company, 2016
Requirements Quality Management
The Systems and Requirements Engineering
life-cycle: where should we put the focus?
10 © The REUSE Company, 2016
Systems and Requirements Engineering life-cycles
Where should we put the focus?:
Requirements Quality Management
95%
85%
70%
Time
Cu
mu
lati
ve p
erce
nta
ge
Life
cylc
e C
ost
Operations through Disposal
100% Production
and test
50%
8% Design
15%
20%
Concept
Commited Costs
3-6x
500-1000x
20-100x
Development Source:
INCOSE
Handbook
11 © The REUSE Company, 2016
Systems and Requirements Engineering life-cycles
Requirements Quality Management
CONOPS
Stakeholders
Requirements
System
Requirements
System
Design
Equipment
Requirements
Equipment
Design Equipment
Verification
System
Equipment
System Verification
Product
Product Verification
12 © The REUSE Company, 2016
Systems and Requirements Engineering life-cycles
Requirements Quality Management
Elicitation Analysis Specification Validation
close gaps clarify
rewrite
re-evaluate
confirm and correct
Source: Karl Wiegers
13 © The REUSE Company, 2016
Systems and Requirements Engineering life-cycles
Requirements Quality Management
Elicitation Analysis Specification Verification Validation
Management
close gaps clarify
rewrite
re-evaluate
fix errors
confirm and correct
create baseline
support
stakeholder
input
stakeholder
feedback
Adapted from: Karl Wiegers
14 © The REUSE Company, 2016
Systems and Requirements Engineering life-cycles
Requirements Quality Management
CONOPS
Stakeholders
Requirements
System
Requirements
System
Design
Equipment
Requirements
Equipment
Design Equipment
Verification
System
Equipment
System Verification
Product
Product Verification
Requirements
Verification
Requirements
Verification
Requirements
Verification
Requirements
Validation
16 © The REUSE Company, 2016
Requirements Authoring: quality checking
The cost of Requirements errors is very high
Fixing requirements after delivery may cost up to 100 times more than fixing in
the requirements definition stage
Training, best practices and verifying requirements by reviewers can
help to get SMART requirements:
But the process is still costly and time consuming
Introducing quality analysis during the authoring activity:
Reduce the number of iterations between System Engineers and sub-
contractors and leverage the verification activities
Requirements Quality Management
17 © The REUSE Company, 2016
Requirements Authoring: quality checking
Experiences showed that about 25% of system Requirements are critical
and can be grammatically improved
No Shall: 8 to 10%
Forbidden words: 10 to 15%
Subject, multiple objects, design: 15%
Incorrect grammar: 50%
…
Requirements Quality Management
Source:
AFIS & Gauthier Fanmuy
RAMP Project
18 © The REUSE Company, 2016
Requirements Authoring: quality checking
Well-known requirements quality characteristics
Requirements Quality Management
IEEE Std. 830:
Correct
Unambiguous
Complete
Consistent
Ranked
Verifiable
Modifiable
Traceable
ESA PSS-05,
ISO/IEC 29148, others:
Pretty much the same characteristics
SMART:
Specific
Measurable
Achievable
Relevant
Traceable
"I believe that this nation should commit
itself to achieving the goal, before this
decade is out, of landing a man on the
Moon and returning him safely to Earth"
19 © The REUSE Company, 2016
Requirements Authoring: quality checking
Good characteristics to check but…
…can we measure how correct, how complete, how consistent, how
measurable… a specification is??
Are those characteristics really SMART?
Are they specific?
Easy to measure? From a objective point of view?
Is it realistic to ask for those characteristics?
Requirements Quality Management
20 © The REUSE Company, 2016
Requirements Authoring: quality checking
Our approach:
Similar to INCOSE Guide for Writing Requirements
High-level quality characteristics
And Low-level set of metrics addressing these characteristics:
Easy to understand
Easy to measure
Easy even for a tool!!
Requirements Quality Management
Characteristic Cxx – Characteristic name
Rationale: xxxx
Strategy: xxxx
Rules that help establish this characteristic:
Rxx - /Section/Rule name
Avoid xxxx
Ryy - /Section/Rule name
Avoid yyy
21 © The REUSE Company, 2016
Requirements Authoring: on-the-fly metrics
Correctness verification:
Metrics based on information coming from the RMS:
Attributes, links, versions…
Metrics based on patterns:
Compliance with different types of requirements patterns
Detection of specific structures within the requirements
Metrics based on the conformance with models:
Concepts in your requirements coming from PBS, FBS…
Metrics based on linguistic algorithms:
Detection of passive voice, imperative tense…
Text length, misspelling….
Metrics based on lists of terms:
Forbidden: ambiguous…
Restricted: negations, pronouns…
Mandatory: ‘shall’
Requirements Quality Management
22 © The REUSE Company, 2016
Requirements Authoring: on-the-fly metrics
Consistency verification:
Use of consistent vocabulary
Consistency within a requirements document
Consistency among different levels of requirements
Consistency between requirements and models
Detection of duplicated requirements
…
Completeness verification:
Detection of the current gaps between:
Specifications and models
Different levels of requirements
Current project vs. other previous reference projects
…
Requirements Quality Management
23 © The REUSE Company, 2016
Requirements Authoring: ontologies
Ontologies as the driving element for requirements quality
Enable a set of tools to enhance performance and reduce defects
Requirements Quality Management
Terminology layer
Conceptual model layer
Patterns layer
Formalization layer
Inference layer
24 © The REUSE Company, 2016
Ontology : Example
Requirements Quality Management
A380 A350 System Operate Temperature Environment Pressure Controlled
Vocabulary
A380 A350
<<Aircraft>> “ Greater than (>) “
Operate Work
<<Operation>>
<<Aircraft>> Shall <Operation> <<Minimum>> At Environment Of [MEASUREMENT
UNIT] NUMBER
temperature “ Greater than (>) “
ºC -70
Patterns
Temperature Pressure
Environment
Temperature [-60ºC , +60ºC] “ Operation Range “
Inference
Rules NUMBER “ Lower than (<) “ -60º NUMBER “ Greater than (>) “ +60º ||
Thesaurus
Formalizations The aircraft shall be able to operate
at a minimum temperature of -70º C
If ºC ºC
“ Lower than (<) “
Shall
At a minimum
<<Minimum>>
At a minimum Of
25 © The REUSE Company, 2016
Requirements Authoring: requirements patterns
To provide consistent grammar
To provide consistent vocabulary
To allow completeness and consistency checking
To identify requirements in unstructured sources
Requirements Quality Management
27 © The REUSE Company, 2016
Requirements Authoring: semantic search engine
To promote reuse
To detect duplicates
To provide quick access to related requirements
Requirements Quality Management
28 © The REUSE Company, 2016
Requirements Authoring: the process
Requirements Quality Management
On-the-fly check of quality guidelines
Boilerplate based support
Communication author - V&V team
Centralized feedback repository
Centralized approval
Communication with ontology manager
29 © The REUSE Company, 2016
Requirements V&V: the process
Many other issues involved in this V&V process, still not covered in
the automatic verification during the authoring stage:
Proper allocation
Is the verification method adequate?
Is the specification conforming to standards, regulations…?
However:
Proven reduction of re-work
Reduced number of loops between OEM and contractors
Faster, accurate and objective V&V
Increased overall quality
Requirements Quality Management
30 © The REUSE Company, 2016
The Requirements Quality Suite
Requirements Quality Management
Introduction to RQS
31 © The REUSE Company, 2016
The Requirements Quality Suite
Requirements Quality Management
The Requirements Quality Suite (RQS) intends to tackle requirements quality management by offering a set of tools and processes
Automatic measurement of requirements quality metric
Support to Requirements Authoring
RQS models requirements quality metrics using the CCC approach (Correctness, Consistency and Completeness)
Requirements Quality Analyzer (RQA):
to setup, check and manage the quality of a
requirements specification.
Requirement Authoring Tool (RAT):
to assist authors while they are creating or
editing requirements.
Knowledge Manager (KM):
to manage knowledge around a requirements
specification: the ontology it is based on, the
structure of the requirements to be used in
the project, the communication between
authors and domain architects.
32 © The REUSE Company, 2016
1990
2000
2005
Present
The Requirements Quality Suite
The ages of Requirements Engineering
Requirements Quality Management
Requirements traceability
Requirements management
Requirements exchange (RIF, ReqIF)
Core of methods and best practices
Mandatory in the relationship OEM/Supplier
First professional tools
First methods
Semantic approaches
Requirements Quality Analysis
Requirements Authoring
Requirements Reuse
Requirements Product Lines
Requirements Interoperability
33 © The REUSE Company, 2016
The Requirements Quality Suite: connectors
Requirements Quality Management
IBM DOORS ©
PTC Integrity ©
CATIA Reqtify ©
VISURE Requirements ©
OSLC
Microsoft Excel ©
XML file
Near future:
Microsoft Word ©
Siemens Teamcenter ©
34 © The REUSE Company, 2016
The Requirements Quality Suite: natural language
Requirements Quality Management
RQS is highly dependent of the language of the requirements
Languages supported so far:
Other languages on the way:
35 © The REUSE Company, 2016
Requirements Quality Management
Live demo.
But if you want more?
Visit us: Booth PA2
37 © The REUSE Company, 2016
http://www.reusecompany.com
@ReuseCompany
contact@reusecompany.com
Margarita Salas, 16 2nd Floor
Innovation Center
LEGATEC Technology Park
28919 Leganés – Madrid
SPAIN – EU
Tel: (+34) 912 17 25 96
Fax: (+34) 916 80 98 26
Herrenberger Straße 56
71034 Böblingen
http://www.xtronic.de
Tel: +49 7031 20947 0
Fax: +49 7031 20947 101
reuse@xtronic.de