chap06 (méthodes de vérification)
Transcript of chap06 (méthodes de vérification)
-
7/28/2019 chap06 (mthodes de vrification)
1/28
Testing of CommunicationProtocols
These lectures are based on slides
developed over years with colleagues [Drs
Dssouli and En-Nouaary, ConcordiaUniversity]
-
7/28/2019 chap06 (mthodes de vrification)
2/28
What is the difference between
Verification and Testing ?
Verification: is related to the specification
Get the right specification/model/design
We use techniques such as reachability analysis and modelchecking ...
Testing: is related to the actual implementation
Is our design model correctly implemented?
Is our implementation correct wrt to our specification ?
-
7/28/2019 chap06 (mthodes de vrification)
3/28
Assumptions
Assumptions about FSM specifications
Minimized
Completely-specified
Deterministic
Strongly-connected (or initially-connected)
Initialized [an can always bring back to initial state with a
reset action]
-
7/28/2019 chap06 (mthodes de vrification)
4/28
Fault Models
Fault models for FSM
Output faults
Transfer faults
Transfer faults with additional states
Additional or missing transitions
Additional or missing states
-
7/28/2019 chap06 (mthodes de vrification)
5/28
Fault Models
Output faults
s1
a / 0
s2
a / 1
b / 1
s3
a / 0
b / 0
b / 1
s1
a / 0
s2
a / 0
b / 1
s3
a / 0
b / 0
b / 1
< specification > < implementation >
-
7/28/2019 chap06 (mthodes de vrification)
6/28
Fault Models
Transfer faults
s1
a / 0
s2
a / 1
b / 1
s3
a / 0
b / 0
b / 1
s1a / 0
s2
a / 1
b / 1
s3
a / 0
b / 0
b / 1
< specification > < implementation >
-
7/28/2019 chap06 (mthodes de vrification)
7/28
Fault Models
Transfer faults with additional states
s1
a / 0
s2
a / 1
b / 1
s3
a / 0
b / 0
b / 1
s1
a / 0
s2
a / 1
b / 1
s3
a / 0
b / 0
b / 1
< specification > < implementation >
s4
-
7/28/2019 chap06 (mthodes de vrification)
8/28
FSM-based Testing
Output faults
TT-method (transition tours)
Output faults + transfer faults
DS-method (distinguishing sequences) W-method (characterizing sets)
UIO-method (UIO sequences)
UIOv-method
Wp-method Harmonized Wp-method
Etc.
-
7/28/2019 chap06 (mthodes de vrification)
9/28
TT-method (1)
A transition tour of a FSM
A path starting at the initial state, traverses every transition
at least once, and returns to the initial state
s1
a / 0
s2
a / 1
b / 1
s3
a / 0
b / 0
b / 1
-
7/28/2019 chap06 (mthodes de vrification)
10/28
TT-method (2)
From a transition tour, we identify a test suite
consisting of an input sequence and its expected
output sequence
The input sequence ababab and Its expected output sequence 011100
s1
a / 0
s2
a / 1
b / 1
s3
a / 0
b / 0
b / 1
-
7/28/2019 chap06 (mthodes de vrification)
11/28
TT-method (3)
Transition tours can find all output faults
s1
a / 0
s2
a / 1
b / 1
s3
a / 0
b / 0
b / 1
s1
a / 1
s2
a / 1
b / 1
s3
a / 0
b / 0
b / 1
< specification >
Input sequence: ababab
Expected output sequence: 011100
< implementation >
Input sequence: ababab
Observed output sequence: 111100
-
7/28/2019 chap06 (mthodes de vrification)
12/28
TT-method (5)
Transition tours cannot find transfer faults
s1
a / 0
s2
a / 1
b / 1
s3
a / 0
b / 0
b / 1
s1a / 0
s2
a / 1
b / 1
s3
a / 0
b / 0
b / 1
< specification >
Input sequence: bababa
Expected output sequence: 111000
< implementation >
Input sequence: bababa
Observed output sequence: 111000
-
7/28/2019 chap06 (mthodes de vrification)
13/28
Detecting Output Faults and
Transfer Faults
DS-method, W-method, and UIO-method
The main idea
Generate a test suite such that, for every transition (s, i, o,
s), Step 1: Puts the implementation into state s (Setup)
Step 2: Applies input i and check whether the actual
output is o (Output fault)
Step 3: Determines whether the target state of theimplementation iss(Transfer fault)
-
7/28/2019 chap06 (mthodes de vrification)
14/28
DS-method (1)
An input sequence is a distinguishing sequence if
After applying the input sequence, we can determine the
source state by observing the output sequence
-
7/28/2019 chap06 (mthodes de vrification)
15/28
DS-method (2)
Show that a is not a distinguishing sequence
Initial
state
Input
seq
Output
seq
Final
state
S1 a 0 S1
S2 a 1 S2
S3 a 0 S3
s1
a / 0
s2
a / 1
b / 1
s3
a / 0
b / 0
b / 1
-
7/28/2019 chap06 (mthodes de vrification)
16/28
DS-method (3)
Show that ab is a distinguishing sequence
Initial
state
Input
seq
Output
seq
Final
state
S1 ab 01 S2
S2 ab 11 S3
S3 ab 00 S1
s1
a / 0
s2
a / 1
b / 1
s3
a / 0
b / 0
b / 1
-
7/28/2019 chap06 (mthodes de vrification)
17/28
DS-method (4)
For every transition (s, i, o, s)
Step 1: Put the implementation into state s (Setup)
Step 2: Apply input i and check whether the actual output is
o (Output fault) Step 3: Determine whether the target state of the
implementation issusing a distinguishing sequence
(Transfer fault)
-
7/28/2019 chap06 (mthodes de vrification)
18/28
DS-method (5)
s1
t1: a / 0
s2
t3: a / 1
t2: b / 1
s3
t5: a / 0
t6: b / 0
t4: b / 1
t1: reset/nulla/0a/0 b/1
t2: reset/null b/1a/1 b/1
t3: reset/null b/1a/1a/1 b/1
t4: reset/null b/1b/1a/0 b/0t5: reset/null b/1 b/1a/0a/0 b/0
t6: reset/null b/1 b/1b/0a/0 b/1
Setup Output Transfer
A test suite
-
7/28/2019 chap06 (mthodes de vrification)
19/28
DS-method (6)
Comparisons
Very few FSMs possess a distinguishing sequence
Even if an FSM has a distinguishing sequence, the
sequence is too long Example: there is no DS
A DS cannot start with a (s1, s2)
A DS cannot start with b (s2, s3)s1
t1: a / 0
s2
t3: b / 1
t2: a / 0
s3
t5: a / 1
t6: b / 0
t4: b / 1
-
7/28/2019 chap06 (mthodes de vrification)
20/28
W-method (1)
A set of input sequences is a characterizing set if
After applying all input sequences in the set, we candetermine the source state by observing the output
sequences
Compared to DS-method we need more than onesequence
-
7/28/2019 chap06 (mthodes de vrification)
21/28
W-method (2)
Show that {a,b} is a characterizing set
Initial
state
Input
seq
Output
seq
Final
state
S1 a 0 S1
S2 a 1 S2
S3 a 0 S3
Initialstate
Inputseq
Outputseq
Finalstate
S1 b 1 S3
S2 b 1 S2
S3 b 0 S2
s1
t1: a / 0
s2
t3: a / 1
t2: b / 1
s3
t5: a / 0
t6: b / 0
t4: b / 1
-
7/28/2019 chap06 (mthodes de vrification)
22/28
W-method (3)
s1
t1: a / 0
s2
t3: a / 1
t2: b / 1
s3
t5: a / 0
t6: b / 0
t4: b / 1
t1: reset/nulla/0a/0
t1: reset/nulla/0b/1
t2: reset/null b/1a/1
t2: reset/null b/1b/1t3: reset/null b/1a/1a/1
t3: reset/null b/1a/1b/1
t4: reset/null b/1b/1a/0
t4: reset/null b/1b/1b/0
t5: reset/null b/1 b/1a/0a/0
t5: reset/null b/1 b/1a/0b/0
t6: reset/null b/1 b/1b/0a/0
t6: reset/null b/1 b/1b/0b/1
A test suite
-
7/28/2019 chap06 (mthodes de vrification)
23/28
W-method (4)
Comparisons
Although every FSM has a characterizing set, the set may
have too many elements
-
7/28/2019 chap06 (mthodes de vrification)
24/28
UIO-method (1)
Lets be a state.
An input sequence is a UIO sequence fors if
After applying the input sequence, we can determine the
source state iss or not by observing the output sequence
-
7/28/2019 chap06 (mthodes de vrification)
25/28
UIO-method (2)
Show that a is a UIO sequence for s2
Show that b is a UIO sequence for s3
Initial
state
Input
seq
Output
seq
Final
state
S1 a 0 S1
S2 a 1 S2
S3 a 0 S3
Initial
state
Input
seq
Output
seq
Final
state
S1 b 1 S3
S2 b 1 S2
S3 b 0 S2
s1
t1: a / 0
s2
t3: a / 1
t2: b / 1
s3
t5: a / 0
t6: b / 0
t4: b / 1
-
7/28/2019 chap06 (mthodes de vrification)
26/28
UIO-method (3)
Show that ab is a UIO sequence for s1
Initial
state
Input
seq
Output
seq
Final
state
S1 ab 01 S2
S2 ab 11 S3
S3 ab 00 S1
s1
a / 0
s2
a / 1
b / 1
s3
a / 0
b / 0
b / 1
-
7/28/2019 chap06 (mthodes de vrification)
27/28
UIO-method (4)
s1
t1: a / 0
s2
t3: a / 1
t2: b / 1
s3
t5: a / 0
t6: b / 0
t4: b / 1
t1: reset/nulla/0a/0 b/1
t2: reset/null b/1a/1
t3: reset/null b/1a/1a/1
t4: reset/null b/1b/1b/0
t5: reset/null b/1 b/1a/0b/0
t6: reset/null b/1 b/1b/0a/0 b/1
A test suite
-
7/28/2019 chap06 (mthodes de vrification)
28/28
UIO-method (5)
Comparisons
Many FSMs have UIO sequences
UIO sequences are usually short
However, fault detection power less than DS, W, Wp