Introduction to White box and Black Box Software · PDF fileIntroduction to White box and...
Transcript of Introduction to White box and Black Box Software · PDF fileIntroduction to White box and...
![Page 1: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/1.jpg)
Introduction to
White box and Black Box
Software Testing
Henry MucciniComputer Science Department
University of L'Aquila - [email protected] –
[www.HenryMuccini.com]
Software Testing
![Page 2: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/2.jpg)
Copyright Notice
» The material in these slides may be freely reproduced
and distributed, partially or totally, as far as an explicit
reference or acknowledge to the material author is
2
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
reference or acknowledge to the material author is
preserved.
Henry Muccini
![Page 3: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/3.jpg)
Agenda
» Recap on Software Testing
» White box Testing
- JUnit
»
3
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
» Black box Testing
- Model-based Testing
![Page 4: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/4.jpg)
Recap on Software Testing
4
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
Recap on Software Testing
![Page 5: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/5.jpg)
Software testing consists of:
the dynamic verification of the behavior of a program
An all-inclusive definition
5
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
on a finite set of test cases
suitably selected from the (in practice infinite) input
domain
against the specified expected behavior
[A. Bertolino]
IMP
![Page 6: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/6.jpg)
Main Activities & Terminology
Main Activities:
– Test case Selection
– Test case Execution
– Regression Testing
Terminology:
– Test Coverage
– Oracle
– Adequacy criteria
– Test Objective
6
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
– Test Objective
– …
» Main principles:
- Repeatable
- Measurable
![Page 7: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/7.jpg)
Main Artefacts
7
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
» Any software artifact can be used for testing
purposes
![Page 8: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/8.jpg)
Main Type of Testing
» Unit, Integration, and System Testing
» White Box and Black box Testing
» Stress Testing
8
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
» Performance Testing
» …
![Page 9: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/9.jpg)
White Box Testing
9
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
White Box Testing
![Page 10: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/10.jpg)
Black Box vs. White Box Testing
SELECTED INPUTS
RESULTANT OUTPUTS
DESIRED OUTPUT
10
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
SELECTED
INPUTS
RESULTANT OUTPUTS
INTERNAL BEHAVIOR
DESIRED OUTPUT
SOFTWARE DESIGN
“BLACK BOX” TESTING
“WHITE BOX” TESTING
![Page 11: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/11.jpg)
Main principles
» The code is available
11
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
» Objective: Cover the software structure
- How?
![Page 12: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/12.jpg)
12
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
![Page 13: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/13.jpg)
Example: CFG
» Build a graph representation of the source code
» Cover the graph
13
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
![Page 14: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/14.jpg)
Coverage Criteria
» Based on Control-flow and Data-flow information
» Among the most famous:
- Statement coverage
- Branch coverage
14
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
- Branch coverage
- Path coverage
![Page 15: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/15.jpg)
Category Partition
» In the case of a branch, how to select the inputs?
15
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
![Page 16: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/16.jpg)
JUnit
» Unit Testing di Java classes
» White box Testing
16
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
» Intuizione:
- Data una classe, l’idea consiste nel:
> Creare una instanza di tale classe, con valori specificati (classe di
test)
> Controllare che tale classe di test si comporti come specificato
![Page 17: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/17.jpg)
JUnit
17
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
![Page 18: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/18.jpg)
Black Box Testing
18
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
Black Box Testing
![Page 19: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/19.jpg)
Black Box vs. White Box Testing
SELECTED INPUTS
RESULTANT OUTPUTS
DESIRED OUTPUT
19
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
SELECTED
INPUTS
RESULTANT OUTPUTS
INTERNAL BEHAVIOR
DESIRED OUTPUT
SOFTWARE DESIGN
“BLACK BOX” TESTING
“WHITE BOX” TESTING
![Page 20: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/20.jpg)
Main Principles
» The code is NOT available
- How to select inputs for test cases?
20
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
» Inputs can be selected based on specifications
» Model-based Testing
![Page 21: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/21.jpg)
Modeling: there is not just programming
» A model is a simplification of the reality
- The reality is abstracted, since it is too complex
- Everything is a model ?!?
» Examples
21
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
» Examples
![Page 22: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/22.jpg)
Examples
22
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
![Page 23: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/23.jpg)
Examples
23
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
![Page 24: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/24.jpg)
Examples
24
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
![Page 25: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/25.jpg)
Why to make models? Document and Analyze
25
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
![Page 26: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/26.jpg)
What is MBT?
» It consists in:
- Extracting Test Cases from the Model
- Run the test case over the system implementation
26
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
» Purpose:
- To validate the implementation conformance to the model
> The model itself is the oracle, i.e., it represents the expected
behavior
![Page 27: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/27.jpg)
Test Selection
» A test case, in MBT, is typically a possible “scenario”
- In scenario-based testing:
> A scenario itself is an abstract test case
27
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
- In state-based testing
> Execution flows are extraced from the state machine and represent a
test case
> MB coverage criteria are applied on this model
![Page 28: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/28.jpg)
In practice
28
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
![Page 29: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/29.jpg)
Test Execution
» It consists in running the test cases on the system
implementation
- So to compare the real execution with the expected
behavior
29
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
![Page 30: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/30.jpg)
MBT practical concerns
» Traceability, i.e. relating the abstract values of the specification to the concrete values of the implementation.
30
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
» Execution, i.e. forcing the Implementation Under Test (IUT) to execute the specific sequence of events that has been selected.
![Page 31: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/31.jpg)
Model-based Testing Approaches
TESTO
UMLAU
TOTEM
SOOT
COW-SU
SCENTO
SeDiTe
AGEDI
UMLTe
UCSC-Sy
AsmL
31
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
OR
UT
MTF
UITE
OR
eC IS
est
ystem
L
![Page 32: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/32.jpg)
MBT Tools
» There are some automated tools for MBT:
- Telelogic Tau and Tau Tester
- Leirios Test Designer
- Conformiq Qtronic
32
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
- Conformiq Qtronic
- Rhapsody Test Conductor and Automatic Test Generation
- AGEDIS
![Page 33: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/33.jpg)
MBT Tools
33
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
![Page 34: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/34.jpg)
MBT Tools
34
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
![Page 35: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/35.jpg)
Remarks
» Learn English
» GSEEM
35
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
» GSEEM
- http://informatica.di.univaq.it/gseem
» EURECA Project
- http://www.mrtc.mdh.se/eureca
![Page 36: Introduction to White box and Black Box Software · PDF fileIntroduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila](https://reader033.fdocuments.net/reader033/viewer/2022051507/5a6fa7b57f8b9aa2538b4df4/html5/thumbnails/36.jpg)
References
» H. Muccini. “Software Testing: Testing new Software Paradigms and new Artefacts”. In the Wiley Encyclopedia of Computer Science & Engineering –2008
» A. Bertolino. “Software Testing Research: Achievements, Challenges, Dreams”, ICSE 2007
36
SEA Group
© 2009 by H. Muccini
SEA GroupSEA Group
Achievements, Challenges, Dreams”, ICSE 2007 Future of Software Engineering Track
» J. Gao, H. Muccini, and X. Bai, “Software Test Automation from Components to Systems”, John Wiley & Sons, to appear in 2009