Best PracticeRequirements Engineering
i4DS Centre for Requirements Engineering
Gutes RE, schlechtes RE
04.01.2017i4Ds | Centre for Requirements Engineering 2
References
IREB Foundation Level Syllabus: https://www.ireb.org/en/cpre/foundation/
S. Fricker, R. Grau, A. Zwingli (2014): “Requirements Engineering: Best Practice” in S. Fricker. C. Thümmler, A. Gavras: Requirements Engineering for Digital Health. Springer. http://www.diva-portal.org/smash/get/diva2:834026/FULLTEXT01.pdf
SwissQ (2016): Software Development 2016 Trends& Benchmarks Report Schweiz. http://report.swissq.it/de/
Samuel Fricker
Professor for Requirements Engineering, FHNW
Assistant Professor for Software Engineering, BTH
Doctor in Informatics (Requirements Engineering) Uni Zürich
ABB, Fuchs-Informatik, Startups…
04.01.2017i4Ds | Centre for Requirements Engineering 4
About Us: i4Ds Centre for Requirements Engineering
Farnaz FotrousiPhD Student
Melanie StadePhD Student
Prof. Dr. Norbert SeyffProvessor
Prof. Dr. Samuel FrickerProfessor
04.01.2017i4Ds | Centre for Requirements Engineering 5
Centre for Requirements Engineering
Research and Innovation for Aligning Technology with Society’s Needs
i4DS Centre for Requirements Engineering
04.01.2017i4Ds | Centre for Requirements Engineering 6
What is ‘Best Practice’?
Google: commercial or professional procedures that are accepted or prescribed
as being correct or most effective.
04.01.2017i4Ds | Centre for Requirements Engineering 7
IREB Standard Practice
Elicitation: Creativity, Surveys, Document Analysis, and Observation
Analysis: System Interfaces and Context
Documentation: Documents, Shall-Templates, UML, SA, Goal Diagrams
Checking: Checklists, Review Techniques, Handling of Conflicts
Management: Attributes, Prioritization, Traceability, Baselining, Changes
Tools and Tooling
04.01.2017i4Ds | Centre for Requirements Engineering 8
Requirements Engineering
04.01.2017i4Ds | Centre for Requirements Engineering 9
What is ‘Best Practice’?
Idea: Best Practiceis Common Practice
04.01.2017i4Ds | Centre for Requirements Engineering 10
Survey Research in 2012
04.01.2017i4Ds | Centre for Requirements Engineering 11
Survey Research
State-of-art
Panel of Experts
Questionnaire
Statistical Analysis
04.01.2017i4Ds | Centre for Requirements Engineering 12
419 Projects
< 10 23 5% Banking, Finance 98 23% Research 32 8%10-49 39 9% Automotive, Transport 54 13% Product, Platform 87 21%
50-249 42 10% Software, IT 51 12% Bespoke 237 57%250-4499 144 34% Government, Military 40 10% Tender 42 10%
>= 4500 168 40% Healthcare, Medical 35 8% Other 21 5%n/a 3 1% Insurance 31 7%
Telecommunications 31 7% Prototyping 23 5%Europe 368 88% Manufacturing, Supply 22 5% Evolutionary 39 9%
Switzerland 248 59% Other 57 14% Incremental 113 27%Germany 69 16% Hybrid 104 25%
Other Europe 51 12% Information System 260 62% Sequential 135 32%Americas 26 6% Software-Intensive 62 15% Other 5 1%
Asia-Pacific 25 6% Engineering 28 7%Other 69 16% < 4 54 13%
4-9 107 26%Completely New 149 36% 10-19 89 21%
New Features or Use 107 26% 20-49 84 20%Changed Solution 79 19% >= 50 82 20%
Enhancement 84 20% n/a 3 1%
< 4.5 27 6%4.5-9 97 23%9-18 119 28%
>= 18 165 39%n/a 11 3%
Duration (calendar months)***
ProductCategory
Process (Lifecycle Model)
Size (number of staff)***
Location
ProjectIndustry (application domain)
Type
Innovation
CompanySize (number of employees)***
04.01.2017i4Ds | Centre for Requirements Engineering 13
Requirements Engineering
Product Planning Requirements Inquiry Requirements Management
Goals
Requirements
Design(Constraints)
Tests
code
Manuals
Inputs
Stakeholders
Project Team
Analyst
ProductManager
Software Project
04.01.2017i4Ds | Centre for Requirements Engineering 14
The Big Picture: Common Practice
Total 405 97% Total + 414 99% Total 384 92% Total 407 97% Total + 404 96%Reqs. Prioritizing 252 60% Workshops 328 78% Informal Modeling + 210 50% Functional 343 82% Natural Language 374 89%Release Planing 209 50% Feedback ± 183 44% Prototyping + 169 40% Scenarios ± 263 63% Use Cases 248 59%
Requirements Triage 206 49% Analysis ± 161 38% OOA ± 166 40% Quality 240 57% Informal Text 219 52%Business Case 202 48% Design 149 36% Quality Checks 107 26% User Interfaces 238 57% User Stories 111 26%Roadmapping 174 42% Creativity 142 34% SA 51 12% Processes ± 183 44% Shall Templates 94 22%
Vision ± 165 39% System Archeology 292 70% DDD 34 8% Rules 173 41% Other 37 9%Other 1 0% Requirements Reuse 270 64% Other 36 9% Software Interfaces 157 37% UML Diagrams 245 58%
Copy/Paste 159 38% Structure 140 33% Use Case Diagrams 188 45%Total 382 91% Delta Specification 121 29% Total + 391 93% Glossary ± 132 32% Activity Diagrams 128 31%
Reqs. Prioritizing 252 60% Standard Reqs. 81 19% Inspection + 266 63% Behavior 95 23% Class Diagrams 114 27%Handshaking 209 50% Variability Analysis 42 10% Walk-Through + 175 42% Agents 71 17% Sequence Diagrams 89 21%
Conflict Management 167 40% Modeling-based 3 1% Peer/Advisor Review 161 38% Formal Properties 24 6% State Machines 54 13%Strategy Alignment 125 30% Interviews + 265 63% Prototype Review 143 34% Other 26 6% Other 2 0%
Power Analysis 76 18% Document Analysis 211 50% Checklist ± 89 21% Graphical Processes 208 50%Win-Win Negotiation 45 11% Creativity 183 44% Simulation 33 8% Total 405 97% Activity Diagrams 128 31%
Variant Analysis 31 7% Workshops 142 34% Automated Checking ± 30 7% Document 265 63% DFD 111 26%Negotiation Analysis 29 7% Idea Castings 43 10% Other 4 1% Spreadsheet 149 36% BPMN; BPML 37 9%
Idea Databases 38 9% Database 146 35% Other 9 2%Total 341 81% Introspection 118 28% Modeling Tool 135 32% SA Diagrams 177 42%
Change Management 243 58% Observation + 87 21% Drawing Tool 61 15% DFD 111 26%Baselining 196 47% Surveys 50 12% Other 4 1% ERD + 94 22%
Traceability 167 40% Data Mining - 25 6% STD 62 15%Progress Tracking 106 25% Other 12 3% User Screens 151 36%
Report Generation 60 14% Informal Drawings 139 33%Process Measurement 55 13% Tables 67 16%
Other 3 1% Other 19 5%
Multiple answers were possible, + and -: significant changes compared with earlier surveys3, 4, ±: no change compared with earlier surveys according to Holm's step-down methodlight color: sub-categories, "Other": answers with less than 5% frequency
Requirements Management
ManagementProduct Planning Elicitation Requirement TypesAnalysis
Storage
Checking
Inquiry Specification
Stakeholder Negotiation
Notations
04.01.2017i4Ds | Centre for Requirements Engineering 15
Common Practice: Elicitation
Almost every project elicited requirements.
Projects tended to–use stakeholder collaboration and existing
systems or specifications…–…more often than creativity and knowledge
of project members.
Total + 414 99%Workshops 328 78%Feedback ± 183 44%
Analysis ± 161 38%Design 149 36%
Creativity 142 34%System Archeology 292 70%
Requirements Reuse 270 64%Copy/Paste 159 38%
Delta Specification 121 29%Standard Reqs. 81 19%
Variability Analysis 42 10%Modeling-based 3 1%
Interviews + 265 63%Document Analysis 211 50%
Creativity 183 44%Workshops 142 34%
Idea Castings 43 10%Idea Databases 38 9%
Introspection 118 28%Observation + 87 21%
Surveys 50 12%Data Mining - 25 6%
Other 12 3%
Elicitation
i4Ds | Centre for Requirements Engineering
Common Practice: Notations
Almost every project documented requirements–often with natural language.
Projects preferred pragmatic notation use over method compliance.
Total + 404 96%Natural Language 374 89%
Use Cases 248 59%Informal Text 219 52%
User Stories 111 26%Shall Templates 94 22%
Other 37 9%UML Diagrams 245 58%
Use Case Diagrams 188 45%Activity Diagrams 128 31%
Class Diagrams 114 27%Sequence Diagrams 89 21%
State Machines 54 13%Other 2 0%
Graphical Processes 208 50%Activity Diagrams 128 31%
DFD 111 26%BPMN; BPML 37 9%
Other 9 2%SA Diagrams 177 42%
DFD 111 26%ERD + 94 22%STD 62 15%
User Screens 151 36%Informal Drawings 139 33%
Tables 67 16%Other 19 5%
Notations
Total 384 92%Informal Modeling + 210 50%
Prototyping + 169 40%OOA ± 166 40%
Quality Checks 107 26%SA 51 12%
DDD 34 8%Other 36 9%
Analysis
04.01.2017i4Ds | Centre for Requirements Engineering 17
Common Practice: Checking
Almost every project checked requirements,–often by formal inspection.
Projects typically–preferred manual requirements checking…–…over simulation and automated checking
Total + 391 93%Inspection + 266 63%
Walk-Through + 175 42%Peer/Advisor Review 161 38%
Prototype Review 143 34%Checklist ± 89 21%
Simulation 33 8%Automated Checking ± 30 7%
Other 4 1%
Checking
04.01.2017i4Ds | Centre for Requirements Engineering 18
Common Practice: Tools
Almost every project stored requirements.
Projects tended to–prefer writing techniques (documents)…–…over list-oriented techniques
(spreadsheets, databases).
Uncommon was storage of requirements in drawing tools, wikis, and cards.
Total 405 97%Document 265 63%
Spreadsheet 149 36%Database 146 35%
Modeling Tool 135 32%Drawing Tool 61 15%
Other 4 1%
Storage
04.01.2017i4Ds | Centre for Requirements Engineering 19
What is ‘Best Practice’?
Idea: Best PracticeCorrelates with RE Success
04.01.2017i4Ds | Centre for Requirements Engineering 20
Success Criteria
Total 419 100%Productivity 228 54%
Effectiveness 156 37%Compliance 143 34%Satisfaction 137 33%
Flexibility 86 21%Safety 73 17%
Environment 7 2%Other 8 2%
Software Product GoalsTotal 419 100%
Shared Understanding 214 51%Specification Quality 197 47%
Clear Scope 160 38%Efficiency 155 37%
User Satisfaction 145 35%Timeliness 139 33%
Fit of Solution 94 22%Estimation Reliability 65 16%Architecture Quality 58 14%
Cost/Benefit Analysis 26 6%Other 4 1%
RE Constraints
04.01.2017i4Ds | Centre for Requirements Engineering 21
Success and Failure
Too Little 181 43%Just Enough 229 55%
Too Much 9 2%
Rather Yes 385 92%Yes 182 43%
Likely 203 48%Rather No 34 8%
No 18 4%Unlikely 16 4%
SuccessRequirements Engineering Amount
Product Goal AchievementANDOR
221 successes189 failures
04.01.2017i4Ds | Centre for Requirements Engineering 22
The Big Picture: Success-Correlating Practices
218 99% 220 100% 207 94% 219 99% 219 99%178 94% 185 98% 168 89% 179 95% 176 93%
135 61% 189 86% 101 46% 192 87% 199 90%111 59% 133 70% 102 54% 142 75% 168 89%
126 57% 114 52% 99 45% 160 72% 142 64%71 38% 66 35% 65 34% 100 53% 104 55%
118 53% 95 43% 99 45% 141 64% 113 51%84 44% 56 30% 66 35% 97 51% 102 54%
114 52% 89 40% 66 30% 138 62% 68 31%91 48% 62 33% 37 20% 95 50% 43 23%
98 44% 88 40% 32 14% 104 47% 46 21%72 38% 53 28% 18 10% 73 39% 44 23%
97 44% 152 69% 19 9% 98 44% 22 10%65 34% 133 70% 14 7% 72 38% 14 7%1 0% 148 67% 24 11% 94 43% 138 62%0 0% 115 61% 11 6% 61 32% 102 54%
78 35% 82 37% 106 48%76 40% 49 26% 79 42%
208 94% 61 28% 214 97% 81 37% 71 32%165 87% 58 31% 169 89% 56 30% 54 29%
135 61% 56 25% 148 67% 52 24% 68 31%111 59% 23 12% 112 59% 42 22% 44 23%
124 56% 29 13% 97 44% 46 21% 48 22%79 42% 13 7% 61 32% 25 13% 39 21%
96 43% 3 1% 91 41% 20 9% 31 14%66 35% 0 0% 50 26% 4 2% 21 11%
78 35% 147 67% 91 41% 15 7% 2 1%46 24% 113 60% 81 43% 12 6% 0 0%
52 24% 111 50% 51 23% 114 52%22 12% 71 38% 37 20% 88 47%
30 14% 88 40% 20 9% 217 98% 71 32%13 7% 53 28% 12 6% 179 95% 54 29%
20 9% 28 13% 18 8% 148 67% 68 31%10 5% 15 8% 11 6% 110 58% 40 21%
21 10% 22 10% 3 1% 84 38% 23 10%8 4% 16 8% 1 1% 64 34% 13 7%
104 47% 80 36% 4 2%101 53% 52 28% 3 2%
194 88% 57 26% 77 35% 101 46%139 74% 57 30% 65 34% 72 38%
142 64% 44 20% 39 18% 68 31%93 49% 41 22% 21 11% 40 21%
112 51% 31 14% 1 0% 56 25%79 42% 18 10% 3 2% 35 19%
104 47% 15 7% 33 15%59 31% 10 5% 28 15%
67 30% 7 3% 84 38%36 19% 3 2% 63 33%
37 17% 76 34%22 12% 61 32%
33 15% 36 16%21 11% 29 15%1 0% 15 7%2 1% 4 2%
Total
Requirements Management
Stakeholder Negotiation
Product Planning Elicitation Analysis
Checking
Requirement Types
Other
Simulation
Automated Checking
Other
Functional*
Scenarios*
Quality
User Interfaces
Processes
Rules
Software InterfacesOther
Inspection
Peer/Advisor Review
Prototype Review*
Walk-Through
Tables
Other
Total Total Total* Total*
Storage
Natural Language
UML Diagrams
Graphical Processes
SA Diagrams
User Screens
Informal Drawings
Document
Spreadsheet
Modeling Tool
Database
Drawing Tool
Other
Glossary
Structure
Behavior
Agents
Formal Properties*
Informal Modeling
OOA
Prototyping
Quality Checks
SA
DDD
Workshops*
Total*
Feedback*
Design*
Analysis
Creativity
Other
DFD
ERD
STD
System Archeology
Requirements Reuse
Interviews
Creativity
Document Analysis
Introspection
Sequence Diagrams
State Machines
Other
Activity Diagrams
DFD
BPMN; BPML
Idea Castings
Idea Databases
Variability Analysis
Modeling-based
Checklist
Observation
Surveys
Data Mining
User Stories
Shall Templates
Other
Use Case Diagrams
Activity Diagrams
Class Diagrams
Copy/Paste
Delta Specification
Standard Reqs.*
Win-Win Negotiation
Variant Analysis
Negotiation Analysis
Change Management*
Baselining
Traceability*
Progress Tracking*
Report Generation
Process Measurement
Other
Workshops
Total*
Other
Notations
Reqs. Prioritizing
Total
Reqs. Prioritizing
Handshaking*
Conflict Management
Strategy Alignment
Power Analysis*
Total
Business Case*
Requirements Triage
Release Planing
Roadmapping
Vision
Other
Use Cases
Informal Text
04.01.2017i4Ds | Centre for Requirements Engineering 23
3 practicescorrelated
with RE Success
Business Case
A business case predicts financial results and other business consequences.
126 57%71 38%
Business Case*
Source: M. Schmidt: The Business Case Guide.
Scenarios
Scenarios document exemplary sequences (stories) of system usage.
160 72%100 53%
Scenarios*
Buy Ticket:1) Traveller enters destination2) Ticket machine shows ticket options3) Traveller selects option4) Ticket machine shows price5) Traveller enters money6) Ticket machine issues ticketAlternatives…Exceptions….
Workshops
Workshops create an efficient, controlled, and dynamic setting for quickly eliciting, prioritizing, and agreeing on requirements.
189 86%133 70%
Workshops*
04.01.2017i4Ds | Centre for Requirements Engineering 27
Use of Success-Correlating Practices
92, 71%
38, 29%
Did Use Business Case, Workshops, and Scenarios
12, 31%
27, 69%
Did NOT use BC, WS, or SC
04.01.2017i4Ds | Centre for Requirements Engineering 28
What is ‘Best Practice’?
RefsQ 2015: “I heard it first at RefsQ”
Idea: Best Practice should beModern or Mature Practice
04.01.2017i4Ds | Centre for Requirements Engineering 29
The Winner was…
< 10% 2-Taken Up 3-Mature 4-Declining
ADO
PTIO
N
Maturity
04.01.2017i4Ds | Centre for Requirements Engineering 30
Practice Hype Cycle
Please Vote from a Practice Perspective: What Topics Go in Which Phase?
1-New
04.01.2017i4Ds | Centre for Requirements Engineering 31
Practice Hype Cycle
Most frequent answers from GI-RE meeting participants (Nov 24, 2016)
0
5
10
15
20
25
New Growing Mature Declining
Automated RE
Agile RE
User Feedback
Workshops
User Stories
SA
Goals
Interviews
Use Cases
UML
Workshops
SA
UserStories
Automated RE
04.01.2017i4Ds | Centre for Requirements Engineering 32
Modern Practices: A Swiss View
Consistent, longitudinal research of practice use:–2013–2014–2015–2016
04.01.2017i4Ds | Centre for Requirements Engineering 33
Modern Practices: A Swiss View
Consistent, longitudinal research of practice use:–2013–2014–2015–2016
I aggregated by looking at–average practice use
over all four years–difference between
average 2013/2014 andaverage 2015/2016
04.01.2017i4Ds | Centre for Requirements Engineering 34
Extent and Change of Practice: Elicitation
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
-50% -40% -30% -20% -10% 0% 10% 20% 30% 40% 50%
Exte
nt o
f Use
Growth
Interview
Introspection(Use of Experience)
ArchaeologyWorkshops
ReuseOn-site Customer
Observation
Surveys
Creativity
04.01.2017i4Ds | Centre for Requirements Engineering 35
Extent and Change of Practice: Specification
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
-50% -40% -30% -20% -10% 0% 10% 20% 30% 40% 50%
Exte
nt o
f Use
Growth
Informal Text
User StoriesUser ScreensUse Cases
Informal DrawingsUse Case Diagrams
BPMNActivity Diagrams
DFD Class DiagramsSequence Diagrams
04.01.2017i4Ds | Centre for Requirements Engineering 36
Extent and Change of Practice: Checking
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
-50% -40% -30% -20% -10% 0% 10% 20% 30% 40% 50%
Exte
nt o
f Use
Growth
Walkthrough
Reviews
Checklists Peer/Advisory ReviewSimulation
Prototype Review
Test Definition
04.01.2017i4Ds | Centre for Requirements Engineering 37
Extent and Change of Practice: Tools
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
-50% -40% -30% -20% -10% 0% 10% 20% 30% 40% 50%
Exte
nt o
f Use
Growth
Documents
JiraPaper and Pen
EnterpriseArchitect
VisioConfluence
Quality Center
TFS
Balsamiq
DoorsPolarion
i4DS Centre for Requirements EngineeringProf. Dr. Samuel Fricker
04.01.2017i4Ds | Centre for Requirements Engineering 38
Summary and Conclusions
Best Practice may be Standard Practice, Common Practice, Success-Correlating Practice, Modern Practice
Standard: requirements inquiry and management
Common: use of natural language, functional requirements, workshops
Success-correlating: business case, scenarios, and workshops
Modern practice: introspection, user screens, user stories, pen and paper, jira, confluence, Balsamiq
…how smart is practice, and what should we do about it?
04.01.2017i4Ds | Centre for Requirements Engineering 39
How Smart is RE Practice?
Total: 67 practices
Natural Language 374 89%Functional 343 82%Workshops 328 78%System Archeology 292 70%Requirements Reuse 270 64%Inspection + 266 63%Interviews + 265 63%Document 265 63%Scenarios ± 263 63%Reqs. Prioritizing 252 60%UML Diagrams 245 58%Change Management 243 58%Quality 240 57%User Interfaces 238 57%Document Analysis 211 50%Informal Modeling + 210 50%Release Planing 209 50%Handshaking 209 50%Graphical Processes 208 50%Requirements Triage 206 49%Business Case 202 48%
3
9
21
04.01.2017i4Ds | Centre for Requirements Engineering 40
Shall We Teach Requirements Engineering?
Successful projects used a wider variety of RE techniques and defined more requirement types than unsuccessful projects.
0
5
10
15
20
25
30
35
Avg. Number of RE Techniques
Successful RE Other RE
+25%
0
1
2
3
4
5
6
Number of Requirement Types
Successful RE Other RE
+26%
Top Related