Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J....
-
Upload
barbara-morriss -
Category
Documents
-
view
218 -
download
1
Transcript of Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J....
![Page 1: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/1.jpg)
Data Flow Coverage
![Page 2: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/2.jpg)
Reading assignment• L. A. Clarke, A. Podgurski, D. J. Richardson
and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection Criteria,” IEEE Transactions on Software Engineering, 15 (11), November 1989, pp. 1318-1332.
• Background reading• "A Comparison of Data Flow Path Selection
Criteria ," by Lori A. Clarke et al.• "A Comparison of Some Structural Testing
Strategies ," by Simeon Ntafos, IEEE Transactions on Software Engineering, v.16, No. 6, 1988 • S. Rapps and E. J. Weyuker, "Data Flow
Analysis Techniques for Test Data Selection," Sixth International Conference of Software Engineering, Tokyo, Japan, September 1982, pp. 272-277.
![Page 3: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/3.jpg)
Control-Flow-Graph-Based Coverage Criteria
• Statement Coverage • Path Coverage• Branch Coverage• Hidden Paths• Loop Guidelines• General• Boundary - Interior
![Page 4: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/4.jpg)
2
1
43
5 6
7
8
Paths for Example
Boundary paths1,2,3,5,7 a1,2,3,6,7 b1,2,4,5,7 c1,2,4,6,7 d
Interior paths (for 2 executions of the loop)
a,a a,b a,c a,d b,a b,b ... x,y for x,y = a, b, c, d
![Page 5: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/5.jpg)
Need Control Flow AND Data Dependence
x:=
y := x
2
1
43
5 6
7
8
z := ...y...
![Page 6: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/6.jpg)
Non-looping Path Selection Problem
x : =1
2 3
4
5 6
7
:= x
All branches 1, 2, 4, 5, 7 1, 3, 4, 6, 7
does not exercise the relationship between the definition of X in statement 2 and the reference to X in statement 6.
![Page 7: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/7.jpg)
Definitions
• dn(x) denotes that variable x is assigned a value at node n (defined)
• um(y) denotes that variable y is used (referenced at node m)• a definition clear path p with
respect to (wrt) x is a subpath where x is not defined at any of the nodes in p
• a definition dm(x) reaches a use un(x) iff there is a subpath (m) p (n) such that p is definition clear wrt x
![Page 8: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/8.jpg)
Data Flow Path Selection
• Rapps and Weyuker• definition-clear subpaths from
definitions to uses
• Ntafos• chains of alternating definitions and
uses linked by definition-clear subpaths
• Laski and Korel• combinations of definitions that reach
uses at a node via a subpath
![Page 9: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/9.jpg)
Assumptions
• no edges of the form (n,ns) or (nf ,n)• no edges of the form (n,n)• there is at most one edge (m,n) for all
m,n• every control graph is well formed• Connected• Single start and single final node
• every loop has a single entry and a single exit
![Page 10: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/10.jpg)
More assumptions• at least one variable is associated with a
node representing a predicate• no variable definitions are associated with a
node representing a predicate• every definition of a variable reaches at least
one use of that variable• every use is reached by at least one definition• every control graph contains at least one
variable definition• no variable uses or definitions are associated
with ns and nf
![Page 11: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/11.jpg)
Rapps’ and Weyuker’s Data Flow Criteria
Foundation:• Definition-clear subpaths from each
definition to {some/all} use(s)
All-Defs• Some definition-clear subpath from
each definition to some use reached by that definition
x:= := x. . .def-clear
![Page 12: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/12.jpg)
Rapps’ and Weyuker’s Data Flow Criteria
All-Uses• Some definition-clear subpath from
each definition to each use reached by that definition and each successor node of the use
:= x. . .
:= x
x:= . . .. . .
:= xdef-clear
def-clear
def-clear
![Page 13: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/13.jpg)
Rapps’ and Weyuker’s Data Flow Criteria
C-use is a “computation use”P-use is a “predicate use”
All-C-Uses, Some-P-Uses
• either All-C-Uses for dm(x) or at least one P-Use
All-P-Uses, Some-C-Uses
• either All-P-Uses for dm(x) or at least one C-Use
![Page 14: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/14.jpg)
Rapps’ and Weyuker’s Data Flow Criteria
All-Du-Paths• All definition-clear subpaths that
are cycle-free or simple-cycles from each definition to each use reached by that definition and each successor node of the use
. . .x:=
. . .
. . .
def-clear
def-clear
def-clear
cycle-free or simple-cycles
cycle-free or simple-cycles
cycle-free or simple-cycles
:= x
![Page 15: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/15.jpg)
Example
1
2 3
4
5
6
d1(x)
u3(x)
u5(x)
u2(x)
![Page 16: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/16.jpg)
All-Defs
Requires:
d1(x) to a useSatisfactory Path:
1, 2, 4, 6
1
2 3
4
5
6
d1(x)
u3(x)
u5(x)
u2(x)
![Page 17: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/17.jpg)
All-Uses
Requires:d1(x) to u2(x)
d1(x) to u3(x)
d1(x) to u5(x)
Satisfactory Paths:1, 2, 4, 5, 6
1, 3, 4, 6
1
2 3
4
5
6
d1(x)
u3(x)
u5(x)
u2(x)
![Page 18: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/18.jpg)
All-Du-Paths
Requires:d1(x) to u2(x)
d1(x) to u3(x)
both paths for d1(x) to u5(x)
Satisfactory Paths:1, 2, 4, 5, 6
1, 3, 4, 5, 6
1
2 3
4
5
6
d1(x)
u3(x)
u5(x)
u2(x)
![Page 19: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/19.jpg)
Ntafos’ Data Flow Criteria
• Foundation:• Chains of alternating definitions and
uses linked by definition-clear subpaths (k-dr interactions)• ith definition reaches ith use,• which defines ith+1 definition• K is number of branches
![Page 20: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/20.jpg)
k-dr interactions
y := ..x..
. . .x:=
. . .. . .
:= x
def-cleardef-clear
def-clear
z:=..y..
x:=
x:= := ..y..y:= ..x..
1-dr
2-dr
. . .w:=..z..
. . .def-clear
. . .def-clear. . .def-clear
![Page 21: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/21.jpg)
Ntafos’ Data Flow Criteria
• Required K-tuplesSome subpath propagating each k-dr
interaction
+ if last use is a predicate, both branches
+ if first definition or last use is in a loop, minimal and some larger number of loop iterations
![Page 22: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/22.jpg)
Example
u4(x), d4(y)
d1(x) 1
2
3 4
5
6
u2(y)
u5(x)
u6(y), u6(x)
u3(y),d 3(x)
0 d0(y)
s
f
![Page 23: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/23.jpg)
1-DR interaction
d1(x) to u4(x)d1(x) to u5(x)d1(x) to u6(x)d0(y) to u2(y)d0(y) to u3(y)d0(y) to u6(y)
d3(x) to u5(x)d3(x) to u6(x)d3(x) to u4(x)d4(y) to u6(y)d4(y) to u2(y)d4(y) to u3(y)
a:
b:
c:
d:
e:
f:
g:
h:
i:
j:
k:
l:
u4(x), d4(y)
d1(x) 1
2
3 4
5
6
u2(y)
u5(x)
u6(y), u6(x)
u3(y),d 3(x)
0 d0(y)
PATHS
0,1, 2, 4, 5, 6; satisfies a-d,j
0,1, 2, 3, 5, 6: satisfies e-h
0,1, 2, 3, 5, 2, 4, 5, 2, 3, 5, 6
: satisfies i,k,l
![Page 24: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/24.jpg)
From 1-DR to 2-DR
d1(x) to u4(x)d1(x) to u5(x)d1(x) to u6(x)d0(y) to u2(y)d0(y) to u3(y)d0(y) to u6(y)
d3(x) to u5(x)d3(x) to u6(x)d3(x) to u4(x)d4(y) to u6(y)d4(y) to u2(y)d4(y) to u3(y)
a:
b:
c:
d:
e:
f:
g:
h:
i:
j:
k:
l:
Plus:d1(x) to u4(x) d4(y) to u6(y)d1(x) to u4(x) d4(y) to u2(y)d1(x) to u4(x) d4(y) to u3(y)d0(y) to u3(y) d3(x) to u5(x)d0(y) to u3(y) d3(x) to u6(x)d0(y) to u3(y) d3(x) to u4(x)
d3(x) to u4(x) d4(y) to u6(y)d3(x) to u4(x) d4(y) to u2(y)d3(x) to u4(x) d4(y) to u3(y)d4(y) to u3(y) d3(x) to u5(x)d4(y) to u3(y) d3(x) to u6(x)d4(y) to u3(y) d3(x) to u4(x)
u4(x), d4(y)
d1(x) 1
2
3 4
5
6
u2(y)
u5(x)
u6(y), u6(x)
u3(y),d 3(x)
0 d0(y)
![Page 25: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/25.jpg)
2-DR interactionsaj: d1(x), u4(x), d4(y), u6(y) ak: d1(x), u4(x), d4(y), u2(y)al: d1(x), u4(x), d4(y), u3(y)eg: d0(y), u3(y), d3(x), u5(x) eh: d0(y), u3(y), d3(x), u6(x)ei: d0(y), u3(y), d3(x), u4(x) ij: d3(x), u4(x), d4(y), u6(y)ik: d3(x), u4(x), d4(y), u2(y) il: d3(x), u4(x), d4(y), u3(y)lg: d4(y), u3(y), d3(x), u5(x) lh: d4(y), u3(y), d3(x), u6(x)li: d4(y), u3(y), d3(x), u4(x)Paths:
0, 1, 2, 4, 5, 6; satisfies aj0, 1, 2, 3, 5, 6: satisfies eg, eh
0, 1, 2, 3, 5, 2, 4, 5, 2, 3, 5, 6: satisfies ei, ij, ik, il, lh
0, 1, 2, 4, 5, 2, 3, 5, 6: satisfies ak, al, lg (but not li)
u4(x), d4(y)
d1(x) 1
2
3 4
5
6
u2(y)
u5(x)
u6(y), u6(x)
u3(y),d 3(x)
0 d0(y)
![Page 26: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/26.jpg)
Laski’s and Korel’s Criteria
• Foundation:Combinations of definitions that reach uses
at some node via a subpath
• Reach CoverageSome definition-clear subpath from each
definition to all uses reached by that definition
basically the same as all-uses
![Page 27: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/27.jpg)
Laski’s and Korel’s Criteria
• Context CoverageSome subpath along which each set of
definitions reach uses at each node
:=x..y..z
y:=. . .x:=
. . .def-clear wrt x
def-clear wrt x and z
z:=
. . .def-clear wrt x, y, and z
:=x..y..z
z:=
![Page 28: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/28.jpg)
Laski’s and Korel’s Criteria• Ordered Context Coverage
Some subpath along which each sequence of definitions reach uses at each node
y:=. . .
x:=. . .def-clear
def-clear
z:=
def-clear
def-clear
. . .:=x..y..z
. . .
![Page 29: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/29.jpg)
Context Coverage
DC(n6) = {d1(x), d4(y)},{d3(x), d0(y)}, {d3(x), d4(y)}
Paths
a: 1, 2, 4, 5, 6
b: 1, 2, 3, 5, 6
c: 1, 2, 3, 5, 2, 4, 5, 6
Note: must compute the sets for each node
u4(x), d4(y)
d1(x) 1
2
3 4
5
6
u2(y)
u5(x)
u6(y), u6(x)
u3(y),d 3(x)
0 d0(y)
abc
![Page 30: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/30.jpg)
Ordered Context Coverage
ODC(n6) = [d1(x), d4(y)], [d0(y), d3(x)], [d3(x), d4(y)],
[d4(y), d3(x)],
Paths
a: 1, 2, 4, 5, 6
b: 1, 2, 3, 5, 6
c: 1, 2, 3, 5, 2, 4, 5, 6
d: 1, 2, 4, 5, 2, 3, 5, 6
Note: must compute the sequences for each node
abcd
u4(x), d4(y)
d1(x) 1
2
3 4
5
6
u2(y)
u5(x)
u6(y), u6(x)
u3(y),d 3(x)
0 d0(y)
![Page 31: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/31.jpg)
How can we compare these criteria?
• all select a set of paths, so compare the paths that they selectset of paths that satisfy a criterion are not
necessarily unique
e.g., s1 or s2 satisfies criterion A
s1, s2, or s3 satisfy criterion B
![Page 32: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/32.jpg)
How can we compare these criteria?
• define a subsumption relationship• criterion A subsumes criterion B iff
for any flow graph P satisfies A ==> P satisfies B
• criterion A is equivalent to criterion B iff A subsumes B and B subsumes A
![Page 33: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/33.jpg)
Relationships among these criteria
All-Paths
All-DU-Paths
All-Uses
All-Defs
Required k-Tuples
All-P-Uses
All-Edges
All-Nodes
All-P-Uses/Some-C-Uses
ORDERED CONTEXT COVERAGE
CONTEXT COVERAGE
REACH COVERAGE
All-C-Uses/Some-P-Uses
![Page 34: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/34.jpg)
Should we define yet another criteria?
• could subsume all the others, (except all paths)?
“the NEW Winner”
All-DU-Paths
All-Uses
All-Defs
Required k-Tuples
All-P-Uses
All-Edges
All-Nodes
All-P-Uses/Some-C-Uses
ORDERED CONTEXT COVERAGE
CONTEXT COVERAGE
REACH COVERAGE
All-C-Uses/Some-P-Uses
![Page 35: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/35.jpg)
Problems with data flow coverage criteria
• infeasible paths• Don’t usually get 100% coverage
• Need to understand fault detection ability
• Artificially combines control with data flow• Considering p-uses or all predicate
alternatives, tacked on to incorporate control flow
![Page 36: Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649c775503460f9492bfb1/html5/thumbnails/36.jpg)
Conclusions
• An improvement over control flow techniques
• Provides a rational for how many times to iterate a loop or which combinations of subpaths to consider
• Most commonly used criterion is all-uses
• Need more empirical evidence to evaluate effectiveness