Integration testing(revisited)
-
Upload
softwarecentral -
Category
Documents
-
view
1.524 -
download
0
Transcript of Integration testing(revisited)
INTEGRATION AND SYSTEM TESTING CHAPTER 12 & 13Csci565
Spring 2009
H.R
eza
1
OBJECTIVES
Integration Testing Simple ATM (SATM) discuss integration testing strategies
Top-down Bottom-up Decomposition Based Integration Testing
(DBIT) Call Graph Based Integration Testing (CGBIT) Path-Based Integration Testing (PBIT)
2
H.R
eza
H.R
eza
3
A SOFTWARE TESTING STRATEGY: SPIRAL MODEL
System Testing
System eng.
validation Testing
Requirements
Integration Testing
Design
Unit Testing
Code
INTEGRATION TESTING:1
If all units/components work individually, do they work as a whole when we put them together? Not necessarily
The problem is “putting them together” or interfacing them
4
H.R
eza
PROBLEMS WITH INTERFACING
Integration faults often traceable to incomplete or misunderstood interface specifications mismatched assumptions about other components Individually acceptable imprecision may be
magnified to unacceptable levels Global data structures can present problems
Inconsistent interpretation of parameters or values
Mixed units (meters/yards) in Martian Lander Violations of value domains, capacity, or size
limits …
5
H.R
eza
INTEGRATION TESTING
Tests complete systems or subsystems composed of integrated components
Integration testing should be black-box testing when tests derived from the specification
Main difficulty is localising errors Incremental integration testing reduces this
problem
6
H.R
eza
APPROACHES TO INTEGRATION TESTING
Two major approachesIncremental approaches
The decomposition-based techniques or treeStubs/Drivers
Call Graph-based techniquesNo stubs or drivers
Non-incremental approaches 7
H.R
eza
INCREMENTAL INTEGRATION TESTING
8
H.R
eza
T3
T2
T1
T4
T5
A
B
C
D
T2
T1
T3
T4
A
B
C
T1
T2
T3
A
B
Test sequence1
Test sequence2
Test sequence3
INCREMENTAL APPROACHES: TOP-DOWN
Top-down testingStart with high-level system integrate from the top-down replacing
individual components by stubs where appropriateDepth-first Breadth-firstNo-best order
Critical sections Early skeletal version using I/O modules 9
H.R
eza
STUBS
Stubs Special module to simulate some functionality Its production is nontrivial task because the code
may simulate a very complicated tasks E.g.
Writing a stub performing a database table search routine
Creating multiple version of same stub for various reasons
10
H.R
eza
TOP-DOWN TESTING
11
H.R
eza
TOP-DOWN: COMPLICATIONS
The most common complication occurs when processing at low level hierarchy demands adequate testing of upper level
To overcome: Either, delay many tests until stubs are replaced
with actual modules (BAD) Or, develop stubs that perform limited functions
that simulate the actual module (GOOD) Or, Integrate the software using bottom up
approach Confusion about overlapping with design
12
H.R
eza
INCREMENTAL TESTING: BOTTOM UP
Bottom-up testing Integrate individual components in levels
until the complete system is created
13
H.R
eza
BOTTOM-UP APPROACH
Starts with construction and testing with atomic modules No need for stub Low level components are combined into cluster
(or builds) to perform a specific sub-function A driver (a control program for testing) is written
Contain hardcoded test input, calls the module being tested, and display the results
Cluster is tested Drivers are removed and clusters are combined
moving upward in the program structure
14
H.R
eza
BOTTOM-UP TESTING
15
H.R
eza
M
H.R
eza
16
A
C
E
B D
F
J
G H I
K L
H.R
eza
17
A
Stub C
Stub E
B Stub D
Stub F
second state in the top-down
H.R
eza
18
A
Stub C
Stub E
B D
F
J
Stub HStub H I
Intermediate state in the top-down
TOP-DOWN
Possible Sequences of modules A, B, C, D, E, F, G, H, I, J, K, L A, B, E, F, J, C, G, K, D, H, L, I A, D, H, I, K, L, C, G, B, F, J, E A, B, F, J, D, I, E, C, G,K, H, L
If parallel testing allowed, then other alternatives are possible After A has been tested, one programmer could
take A and test the combination A-B, Another programmer could test A-C
19
H.R
eza
GUIDELINE FOR INTEGRATION TESTING
Integrate the components that implement the most frequently used functionality
Perform regression testing for existing features
Perform progression testing for new features
20
H.R
eza
NON-INCREMENTAL
Big-bang Imposes no order (GOOD) Test all the units (Modules) at once Very easy, but difficult to localize the source of
errors
21
H.R
eza
INTEGRATION TEST DOCUMENT
Overall plan for integration of the system under construction must be documented in a Test Specification
The test plan should describe the overall strategy for integration Example of phases
User interaction (menu, button, forms, display presentation)
Data Manipulation and analysis Display processing and generation (RT-2D, RT-3D, etc) Database Mgt Logic??
22
H.R
eza
INTEGRATION: CRITERIA
The following criteria Interface integrity
(internal and external interfaces are tested as each module (or cluster) is incorporated
Functional validity Testes to uncover functional error
Information validity Tests to uncover error related to local/ global data
Performance (Quality) Tests designed to verify performance bounds during
software design
23
H.R
eza
TOP-DOWN VS. BOTTOM-UP
Architectural validation Top-down integration testing is better at
discovering errors in the system architecture System demonstration
Top-down integration testing allows a limited demonstration at an early stage in the development
Test implementation Often easier with bottom-up integration
testing24
H.R
eza
THE WATERFALL LIFE CYCLE
25
H.R
eza
Requirement specifications
Preliminary design
details design
coding
Unit testing
Integration testing
Systemtesting
INTEGRATION TESTING: SOFTWARE ARCHITECTURE
Integration testing How software architecture can be used to
conduct tests to uncover errors related to the interface?
26
H.R
eza
FIG. 12.2: PRIMARY DESIGN (OR INFORMAL SOFTWARE ARCHITECTURE) OF THE ATM USING TREE-BASED DECOMPOSITION
27
H.R
eza
Requirement specifications
Terminal I/OMange Session
Conduct Transactions
Card Entry PIN EntrySelect
Transaction
MORE ON PRIMARY DESIGN
How do perform Integration testing for non-tree based functional decomposition? E.g
integration testing for OO Integration testing for Client/server systems Integration testing for Layered systems ….
28
H.R
eza
SIMPLE ATM (SATM)
An ATM simple Provides 15 screens for interactions includes 3 function buttons Modeled in structural analysis
Data Model (ERD) Functional Model (DFD) Behavioral model (STD)
29
H.R
eza
FIGURE 12.7
30
H.R
eza
FIGURE 12.8
31
H.R
eza
FIGURE 12.9
32
H.R
eza
FIGURE 12.10
33
H.R
eza
FIGURE 12.11
34
H.R
eza
FIGURE 12.12
35
H.R
eza
FIGURE 12.13
36
H.R
eza
DECOMPOSITION BASED STRATEGIES
Decomposition based Top/down Bottom up Sandwich Big bang
37
H.R
eza
FIGURE 12.14
38
H.R
eza
FIGURE 13.1
39
H.R
eza
DECOMPOSITION BASED TESTING:1
Discussion revolves around the tree-based decomposition and the order by which units are tested and combined Top-to-bottom Bottom-to-top Sandwich Big bang
The focus is on the structural compatibility among interfaces
40
H.R
eza
TEST SESSIONS A test session refers to one set of tests for a
specific configuration of actual code and stubs
The number of integration test sessions using a decomposition tree can be computedSessions=nodes – leaves + edges
41
H.R
eza
DECOMPOSITION BASED TESTING: 2
For SATM system42 integration testing session (i.e.,
42 separate sets of integration test cases)top/down
(Nodes-1) stubs are needed 32 stub in SATM
bottom/up (Nodes-leaves) of drivers are needed 10 drivers in SATM
42
H.R
eza
DECOMPOSITION BASED STRATEGIES: PROS AND CON
Intuitively clear and understandable In case of faults, most recently added units are
suspected ones Can be tracked against decomposition
tree Suggests breadth-first or depth-first
traversals Units are merged using the
decomposition tree Correct behavior follows from individually
correct units and interfaces Stubs/Drives are major development Overhead
43
H.R
eza
CALL GRAPH BASED INTEGRATION TESTING
Call graph A directed graph Nodes corresponds to unit Edges corresponds to the call E.g.
AB (i.e., A is calling B)
Attempts to overcome the decomposition problem (structural)
Moves toward behavioral testing
44
H.R
eza
CALL GRAPH (CG): APPROACHES
Two main approaches based on Call Graph Pair-wise integration Neighborhood integration
45
H.R
eza
FIGURE 13.2: SATM CALL GRAPH
46
H.R
eza
TABLE 2: AM
47
H.R
eza
PAIR-WISE INTEGRATION
The main idea is to eliminate the overhead (i.e., stub/drive)
Uses actual code by restricting a session testing to a pair of units in the Call Graph One integration test for each edge in CG 40 edges means 40 integration tests for the
SATM
48
H.R
eza
PAIR-WISE INTEGRATION
49
H.R
eza
-Uses actual code
-one integration test session for each edge
-40 edges for SATM
NEIGHBORHOOD INTEGRATION The neighborhood of a node refers to the nodes that
are one edge away from the given nodes SATM Neighborhoods Number of neighborhoods can be computed
Neighborhoods = nodes – sink nodes Or the number of interior-nodes + X ( x=1 if
there exists leaf nodes connected directly to the root node otherwise X= 0)
Results a drastic reduction in the number of integration test session In case of SATM (11 vs. 40)
50
H.R
eza
NEIGHBORHOOD INTEGRATION
51
H.R
eza
TABLE 3: SATM NEIGHBORHOODS
52
H.R
eza
PROS AND CONS
Benefits (GOOD) Mostly behavioral than structural Eliminates sub/drive overhead Works well with incremental development
method such as Build and composition Liabilities (BAD)
The fault isolation E.g.,
Fault in one node appearing in several neighborhood
53
H.R
eza
PATH-BASED INTEGRATION TESTING
The hybrid approach (i.e., structural and behavioral) is an ideal one integration testing
The focus is on the interactions among the units Interfaces are structural Interactions are behavioral
With unit testing, some path of source statements is traversed What happens when there is a call to another unit?
Ignore the single-entry/single-exit Use exist follows by an entry Suppress the call statement
54
H.R
eza
NEW AND EXTENDED CONCEPTS
Source node (begin) A statement fragment at which program
execution begins or resumes E.g., BEGIN
Sink node (end) A statement fragment at which program
execution terminates E.g., Final END
55
H.R
eza
MORE ON CONCEPTS Module Execution Path (MEP)
A sequence of statements that begins with a source node and ends with a sink node, with no intervening sink nodes
The implication of this definition is that program graph (PG) may have multiple source/sink nodes
Message A programming language mechanism by which
one unit transfers control to another unit E.g.,
subroutine invocations Procedure calls Function references 56
H.R
eza
THE PATH BASED INTEGRATION TESTING (DEFINITION) MM-Path (definition)
An interleaved sequence of module execution paths (MEP) and messages
MM-Path can be used to describe sequences of module execution paths
including transfers of control among units using messages
To represents feasible execution paths that cross unit boundaries
To extent program graph Where
nodes = execution paths edges = messages
Atomic system function (ASF) An action that is observable at the system level in
terms of port input and output events 57
H.R
eza
FIGURE 13.3
58
H.R
eza
H.R
eza
59
There are seven module execution paths (MEP):
MM-PATH GRAPH (DEFINITION)
Given a set of units, their MM-Path graph is the directed graph in which nodes are module execution paths (MM-PATHS) and edges represents messages/returns from one unit to another
Supports composition of units
60
H.R
eza
FIGURE 13.4
61
H.R
eza
FIGURE 13.5
62
H.R
eza
PDL DESCRIPTION OF SATM
63
H.R
eza
H.R
eza
64
H.R
eza
65
MORE ON MM-PATH MM-Path issues
How long is an MM-Path? What is the endpoint?
The following observable behavior that can be used as endpoints Event quiescence ( event inactivity)
System level event Happens when system is ideal/waiting
Message quiescence (msg inactivity) Unit that sends no messages is reached
Data quiescence (data inactivity) Happens when a sequences of processing generate a
stored data that is not immediately used E.g. account balance that is not used immediately
66
H.R
eza
MM-PATH GUIDELINES
MM-Path Guiltiness Points of quiescence are natural endpoints for an
MM-path atomic system functions (system behavior) are
considered as an upper limit for MM-Paths MM-Paths should not cross ASF boundaries
67
H.R
eza
PROS AND CONS hybrid approach(GOOD) The approach works equally well for software
testing developed by waterfall model (GOOD) Testing closely coupled with actual system
behavior (GOOD) Identification of MM-Paths which can be
offset by elimination of sub/drivers(BAD)
68
H.R
eza
SYSTEM TESTING Closely related to everyday expertise The goal is to Test the quality (not specification) Works with notion of threads (or scenarios) A thread is a sequence of events (or ASFs)
Provides a unifying view of our three level of testing e.g.,
A scenario of normal usage A stimulus/response pair A sequence of machine instructions A sequences of ASFs ….
Identifying threads FSM (node/edge coverage metrics)
Top/down Bottom/up
69
H.R
eza
SOFTWARE ARCHITECTURE & TESTING
Using Traditional approach formalizing and automating the integration test stage is difficult the selection of the test cases for the subsystems stress structure over behavior the order in which the components are
incrementally combined is entirely dependent on the adopted system
How can we test if a design and implementation comply using SA?
How can we specify a SA such that one or more properties can “easily" be tested or verified?
How integration testing can be planned and controlled based on the SA?
70
H.R
eza
SA-BASED APPROACH
SA Used for prediction of the system-level quality
the approach would belong to the black box techniques of state transition testing, i.e., the system specification is modeled by an
automaton, the generation of test cases is aimed at covering
the arcs (i.e., the transitions) and the nodes (i.e., the states) of it
Create sub-graphs from SPEC representing specific views of a system
use these views as a base for the definition of coverage criteria and testing strategies.
Select test cases to cover these sub-graphs
71
H.R
eza
THE ADVANTAGES OF USING THE SATO DERIVE THE AUTOMATON
for state transition testing are evident: we have a formal description and the automaton
can be automatically derived; the SA is at a high level of abstraction, thus the
number of states in the automaton can be kept manageable;
we can trace changes in the SA to the corresponding modifications in the automaton, allowing for the testing of new systems obtained after insertion or modification of a component.
72
H.R
eza
ABOUT SURVEY PAPER
H.R
eza
73
Example of abstract for survey paper: In this report (or
paper), we survey a number of X-based representations/algorithms/tool support used to generate/identify/execute/etc test cases.
Many problems with references and citation
Examples of citation: