Post on 24-Jul-2022
Slide 1
Inspection Tutorial© Fraunhofer IESE
Slide 3
Inspection Tutorial© Fraunhofer IESE
Small Bug, Great Disaster - Ariane 5 Flight 501 (1996)
Cause
Malfunction in control software reused from Ariane 4 (arithmetic overflow during number conversion)
Effect
Loss of spacecraft
Cost
>> $1 billion
In safety-critical domains, software defects may lead to great disasters
Inspection Tutorial© Fraunhofer IESESlide 4
Typical problems
Actual project costs exceed planned costs
Actual project schedule exceeds planned schedule
40-50% of the effort is avoidable rework
Results: reduced product quality and lowcustomer satisfaction
Slide 5
Inspection Tutorial© Fraunhofer IESE
Chaos Report 1994-2006
16
27
26
28
34
29
35
31
40
28
23
15
18
19
53
33
46
49
51
53
46
0% 20% 40% 60% 80% 100%
1994
1996
1998
2000
2002
2004
2006
Software projects completed on timeProjects cancelled before completionLate and over budget
Slide 6
Inspection Tutorial© Fraunhofer IESE
Which Quality Assurance (QA) Techniques Exist?
Dynamic QA techniques
Evaluation of a product/system with respect to fulfillment of requirements (specification)
Testing
Static QA techniques
Evaluation of documents that are created within the development process, in order to find defects
Inspections/Reviews
Analysis
Design
Implemen-tation
ModuleTest
IntegrationTest
SystemTest
Contract Software
Slide 7
Inspection Tutorial© Fraunhofer IESE
Typical Quality Assurance Strategy - Testing
Testing is often used as the only quality assurance activity
Executable code is needed
Testing is very expensive (up to 50% of the total costs)
Not all defects can be addressed by testing
Late detection can have destructive impact (erosion of software)
Defects in the early phases of development are hard to find and therefore remain in the product throughout later development phases
High concentration of defects at the end of product development
Chaos Zone
Design Code TestRequirements MaintenanceDefect detection
Defect injectionDesign Code TestRequirements Maintenance
Slide 8
Inspection Tutorial© Fraunhofer IESE
Costs for Late Defect Detection Increase Dramatically
0
10
20
30
40
50
60
70
80
90
100
Analysis Design Implementation Testing Maintenance
Costs for correcting one defect (in 1.000 €)Defects detected (in %)Defects introduced (in %)
Source: SQS AG, analysis of 5000 projects
Slide 9
Inspection Tutorial© Fraunhofer IESE
Solution: Use of Reviews / Inspections
Customized reviews / inspections in the early phases of softwaredevelopment in combination with testing
Goal: to find and remove defects in the phase in which they are injected
Focused use of inspections in order to guarantee the desired/required quality of the documents created within the development process
Design Code TestRequirements Maintenance
Defect injectionDesign Code Test MaintenanceRequirements
Defect detection
Slide 10
Inspection Tutorial© Fraunhofer IESE
Comparison of Inspections and Testing
Can be used in early development stages
Can address different aspects of quality
Usually manual performance
Identification of potential defects
Is usually used in late development stages
Requires an executable system/model
Identification of “wrong behavior” debugging is necessary
Very useful for dynamic aspects
TestingInspections
For effective QA, joint use of inspections and testing is required
Inspection Tutorial© Fraunhofer IESESlide 11
Definition of Formal Inspections
Definition
Formal inspections are methods of static analysis, used for evaluating quality attributes of software documents, with the goal of identifying defects
Characteristics
Follow a structured and defined process
Team is composed of professionally qualified people
Use of specific reading techniques for the detection of defects
Results are documented
Formal Inspections are the most formal static quality assurance method; most of the other methods can be derived from this one.
Inspection Tutorial© Fraunhofer IESESlide 12
The Basic Process of Inspections
Planning
Preparation Problem-list
Meeting
Correction
SoftwareArtifact
Inspections-package
Correctionlist
Organizer
Inspector
ModeratorInspector
Author
Author
Corrected SoftwareArtifactSoftware
Inspection
Defect-list
Overview
Third-hour
Follow-Up
Data Collection
Role
Activity
Document
Optional
Slide 13
Inspection Tutorial© Fraunhofer IESE
Overview: Static Quality Assurance Methods
Different approaches and definitions exist
Depending on the goals, the context of development, and the quality requirements, different methods can be used
Empirical studies show that formal inspections are the most effective method
Inspections typically allow for defect detection ratios of between 30% and 90%
Mostformal
Leastformal
FormalInspection
TeamReview
Walk-through
Peer Deskcheck,Passaround
Ad-hoc
Slide 14
Inspection Tutorial© Fraunhofer IESE
Ideen der verschiedenen Ansätze
Ad-hoc-Review:
Ziele: Fehlerreduzierung, Einholen einer zweiten Meinung
Ablauf: Eine weitere Person (z.B. Kollege) wird bei einem konkreten (häufig „lokalen“) Problem spontan eingebunden
Vorteile
einfach und überall einsetzbar
geringe Kosten
Nachteil
Ergebnis ist abhängig von der Erfahrung der eingebundenen Person
Slide 15
Inspection Tutorial© Fraunhofer IESE
Ideen der verschiedenen Ansätze
Peer Deskcheck:
Ziele: Fehlerfindung, konstruktive Hinweise, Mentoring / Coaching
Ablauf: Eine Person begutachtet ein Dokument und kommentiert entsprechend
Hinweise
Für nicht-kritische Systemteile zu empfehlen
Weder Prozess noch weitere Hilfsmittel sind vorgegeben, können aber eingesetzt werden
Vorteile
Geringer Aufwand, einfach einzusetzen
Verminderter Druck für den Autor, da nur ein Inspektor das Dokument inspiziert
Nachteile
Ergebnis abhängig von Wissen, Erfahrung und Disziplin der eingebundenen Person
„Gruppeneffekte“ werden nicht genutzt
Autor nicht anwesend
Anmerkung: Ein Deskcheck als solcher entspricht einem „Selbst-Review“
Slide 16
Inspection Tutorial© Fraunhofer IESE
Ideen der verschiedenen Ansätze
Passaround:
Ziele: Fehlerfindung, konstruktive Hinweise
Ablauf: Mehrere Personen begutachten ein Dokument und fügen Kommentare ein
Hinweis: Für nicht kritische Systemteile zu empfehlen
Vorteile
Geringer Aufwand, einfach einzusetzen
Schwächen des Deskcheck reduziert (Gefahr von keinem Feedback sowie geringwertigem Feedback)
Überbrückung geographischer Differenzen
Nachteile
Widersprüchliche oder identische Vorschläge
Kein oder verspätetes Feedback möglich
Keine Team-Inspiration
Autor nicht anwesend
Slide 17
Inspection Tutorial© Fraunhofer IESE
Walkthrough: Ziele: Präsentation einer Lösungsidee, gemeinsames Verständnis, Diskussionen zu Lösungsideen, Fehlerfindung
Ablauf: Der Autor eines Dokuments präsentiert es weiteren Person, regt eine entsprechende Diskussion an und sammelt Kommentare der Teilnehmer
Vorteile
Teilnehmer erlangen ein besseres Verständnis des Dokumentes
Teamaktivität (z.B. Diskussion, Feedback der Teilnehmer)
Autor ist aktive Rolle
Nachteile
Kein formaler Prozess, keine Rollenzuweisung, keine Datensammlung
keine Hilfestellung zur Bewertung der Qualität
Risiko: Autor stellt Arbeit extrem positiv dar
Risiko: Verständnislücken bei Teilnehmern werden nicht ausgeräumt und dadurch Fehler „maskiert“
Risiko: Kommentare der Teilnehmer werden nicht umgesetzt
Ideen der verschiedenen Ansätze
Slide 18
Inspection Tutorial© Fraunhofer IESE
Team-Review:
Ziel: Fehlerfindung, Diskussionen zu Lösungsideen
Ablauf: Verschiedene Kollegen begutachten ein Dokument, um potentielle Fehler zu finden
Hinweise
Sehr ähnlich zur Inspektion
Autor darf Moderatorrolle einnehmen
Mehr Spielraum für inhaltliche / konstruktive Diskussionen
Vorteile
Verständnis zum Dokument wird verbessert
Ergebnisse werden in einem Meeting mit dem Autor konsolidiert
Nachteil
Keine definierte Unterstützung bei der Fehlersuche
Metriken spielen eine untergeordnete Rolle
Ideen der verschiedenen Ansätze
Slide 19
Inspection Tutorial© Fraunhofer IESE
Reading Techniques – Overview (1/2)
Ad-hoc
No support, experience-based
Checklist-based reading
Checklist-based reading (CBR): “one checklist for all”
Focused checklist: Inspectors receive different checklists containing questions that focus on the needs of different perspectives
Guided checklist: Inspectors receive different checklists containing questions that focus on different quality aspects
Slide 20
Inspection Tutorial© Fraunhofer IESE
Reading Techniques – Overview (2/2)
Scenario-based reading
Perspective-based reading (PBR): The Inspector performs defect detection from one perspective. The Inspector receives a scenario that describes the activities that have to be performed in order to identify defects and which focuses on defects that are important for the respective perspective.
Defect-based reading (DBR): The Inspector receives a scenario that describes which defect classes the Inspector has to focus on during inspection as well as how to perform defect detection.
Usage-based reading (UBR): The Inspector receives a set of Use Cases. These will be executed “manually” during the inspection, in order to check if the design/the code implements the Use Cases.
Others
Stepwise abstraction
Slide 21
Inspection Tutorial© Fraunhofer IESE
Reading Techniques – Coverage and overlap
Better coverage of potential defect sources higher effectiveness
Reduction of detection overlapping higher efficiency
Reduction of the required experience and impact of experience
Artifact with defects
Single Checklist
PBR scenariosFocused checklist
Slide 22
Inspection Tutorial© Fraunhofer IESE
Example of a “tester” scenario for requirements inspection:
Introduction: Imagine you are a tester of the requirements. One of your main tasks is to derive acceptance test cases for the various requirements specified in the functional specification and the related user scenarios. To successfully perform your task, it is most important for you that the test cases are easy to derive from the requirements…
Instructions: For each assigned requirement of the functional specification in the document and for each related functional user scenario generate a test case or set of test cases that allow you to ensure that an implementation of the system satisfies the functional specification (set of acceptance cases)...Document all issues you identify while performing the task in the issue list.
Questions: While following these instructions answer the following questions:• Which parts of the requirements are difficult to understand and hard to derive the test cases from, i.e. where is it unclear what the correct behaviour should be?• Which necessary information is missing to identify the item being tested and the test criteria (e.g. where are valid inputs or outputs missing, are acceptance criteria missing)?
perspective task
quality focus
artifact unit activity
questions
Reading Techniques – PBR Example
Slide 23
Inspection Tutorial© Fraunhofer IESE
Literature
Peer Reviews in Software, K. Wiegers, 2002
Software Inspection, T. Gilb, D. Graham, 1993
http://inspection.iese.de Bibliographie