1

Graph Coverage Criteria

Structural Coverage Criteria : Defined on a graph just in terms of nodes and edgesData Flow Coverage Criteria : Requires a graph to be annotated with references to variables

2

Coverage Criterion

A coverage criterion is simply a recipe for generating test requirements in a systematicway:Coverage Criterion: A coverage criterion is a rule or collection of rules that impose test requirements on a test set.

3

Coverage Criterion

A coverage criterion is a rule or collection of rules that impose test requirements on a test set.That is, the criterion describes the test requirements in a complete and unambiguous manner.

The “flavor criterion” yields a simple strategy for selecting jellybeans. In this case, the set of test requirements, TR, can be formally written out as

TR = {flavor = Lemon, flavor = Pistachio, flavor = Cantaloupe,flavor = Pear, flavor = Tangerine, flavor = Apricot}

4

Coverage

Given a set of test requirements TRfor a coverage criterion C, a test set T satisfies C if and only if for every test requirement trin TR, at least one test t in T exists such that t satisfies tr .

5

Coverage Level

Given a set of test requirements TR and atest set T, the coverage level is simply the ratio of the number of test requirementssatisfied by T to the size of TR.

6

1

2

0

Node Coverage (Formal Definition): For each node n ∈ reachG(N0), TR contains the predicate “visit n.”

Node Coverage (NC): TR contains each reachable node in G.

Structural Coverage Criteria -I

Node Coverage (NC) (Standard Definition) Test set T satisfies node coverage on graph G if and only if for every syntactically reachable node n in N, there is some path p in path(T) such that p visits n.TR = { 0, 1, 2 }path(t1) = [0, 1 , 2] T1={t1} T1 satisfies node coverage on the graph

7

Structural Coverage Criteria - II Edge coverage is slightly stronger than node coverage Edge Coverage (EC) : TR contains each reachable

path of length up to 1, inclusive, in G.The “length up to 1” allows for graphs with one node and no

edges

Edge Coverage : TR = { (0,1), (0, 2), (1, 2) }path (t1 ) = [ 0,1,2]

path (t2 ) = [ 0, 2 ]

T2={t1, t2} Test Paths = [ 0, 1, 2 ] [ 0, 2 ]

T2 satisfies edge coverage on the graph1

2

0

8

Paths of Length 1 and 0

0

A graph with only one node will not have any edges

• It may be boring, but formally, Edge Coverage needs to require Node Coverage on this graph

Edge Coverage needs to require Node Coverage on this graph

Otherwise, Edge Coverage will not subsume Node CoverageSo we define “length up to 1” instead of simply “length 1”

We have the same issue with graphs that only have one edge for Edge Pair Coverage …

1

0

9

Structural Coverage Example

6

0

2

1

3 4

5

Node CoverageTR = { 0, 1, 2, 3, 4, 5, 6 }Test Paths: [ 0, 1, 2, 3, 6 ] [ 0, 1, 2, 4, 5, 4, 6 ]

Edge CoverageTR = { (0,1), (0,2), (1,2), (2,3), (2,4), (3,6), (4,5), (4,6), (5,4) }

Test Paths: [ 0, 1, 2, 3, 6 ] [ 0, 2, 4, 5, 4, 6 ]

path (t1 )= [ 0, 1, 2, 3, 6 ] path (t2 )=[ 0, 2, 4, 5, 4, 6 ]T={t1 , t2}

10

Structural Coverage Criteria – IIICovering Multiple Edges

Edge-pair coverage requires pairs of edges, or subpaths of length 2

Edge-Pair Coverage (EPC): TR contains each reachable path of length up to 2, inclusive, in G.

The “length up to 2” is used to include graphs that have less than 2 edgesThe logical extension is to require all paths …

11

Edge-Pair Coverage

6

0

2

1

3 4

5

Test Paths: [ 0, 1, 2, 3, 6 ] [ 0, 1, 2, 4, 6 ] [ 0, 2, 3, 6 ] [ 0, 2, 4, 5, 4, 5, 4, 6 ]

TR = { [0,1,2], [0,2,3], [0,2,4], [1,2,3], [1,2,4], [2,3,6], [2,4,5], [2,4,6], [4,5,4], [5,4,5], [5,4,6] }

path (t1 )= [ 0, 1, 2, 3, 6 ] path (t2 )= [ 0, 1, 2, 4, 6 ]path (t3 )=[ 0, 2, 3, 6 ]path (t4 )= [ 0, 2, 4, 5, 4, 5, 4, 6 ]

T={t1 , t2 , t3 , t4 }

12

Complete Path Coverage

Complete Path Coverage (CPC) : TR contains all paths in G. This is impossible if the graph has a loop, so a weak compromise is to make the tester decide which paths

Test Paths: [ 0, 1, 2, 3, 6 ] [ 0, 1, 2, 4, 6 ] [ 0, 1, 2, 4, 5, 4, 6 ] [ 0, 1, 2, 4, 5, 4, 5, 4, 6 ] [ 0, 1, 2, 4, 5, 4, 5, 4, 5, 4, 6 ] …

6

0

2

1

3 4

5

13

Simple Path : A path from node ni to nj is simple if no node appears more than once, except possibly the first and last nodes are the same No internal loops Includes all other subpaths A loop is a simple path

1 2

0

3

Simple Paths : [ 0, 1, 3, 0 ], [ 0, 2, 3, 0], [ 1, 3, 0, 1 ], [ 2, 3, 0, 2 ], [ 3, 0, 1, 3 ], [ 3, 0, 2, 3 ], [ 1, 3, 0, 2 ], [ 2, 3, 0, 1 ], [ 0, 1, 3 ],[ 0, 2, 3 ],[ 1, 3, 0 ],[ 2, 3, 0 ],[ 3, 0, 1], [3, 0, 2], [ 0, 1], [ 0, 2 ], [ 1, 3], [ 2, 3], [ 3, 0], [0], [1], [2], [3]

Simple Paths

14

Prime Paths and Prime Path Coverage

Prime path (PP) gives strong coverage of loops without requiring an infinite number of paths

In prime path coverage, tests must tour each prime pathin the graph G. Prime path coverage requires touring all subpaths of

length 0 (all nodes), of length 1 (all edges), length 2, 3, etc.

Thus it subsumes node coverage, edge coverage and edge-pair coverage

In prime path coverage, tests must tour each prime path in the graph G.

15

Prime Paths

Prime Paths : [ 0, 1, 3, 0 ], [ 0, 2, 3, 0], [ 1, 3, 0, 1 ], [ 2, 3, 0, 2 ], [ 3, 0, 1, 3 ], [ 3, 0, 2, 3 ], [ 1, 3, 0, 2 ], [ 2, 3, 0, 1 ]

1 2

0

3

Prime paths are maximal length simple paths

16

Another Example: Simple Paths

Len 0[0][1][2][3][4][5][6] !

5

0

2

1

3 4

6

Len 1[0, 1][0, 2][1, 2][2, 3][2, 4][3, 6] ![4, 6] ![4, 5][5, 4]

Len 2[0, 1, 2][0, 2, 3][0, 2, 4][1, 2, 3][1, 2, 4][2, 3, 6] ![2, 4, 6] ![2, 4, 5] ![4, 5, 4] *[5, 4, 6] ![5, 4, 5] *

! means path terminates * means path cycles

Len 3[0, 1, 2, 3][0, 1, 2, 4][0, 2, 3, 6] ![0, 2, 4, 6] ![0, 2, 4, 5] ![1, 2, 3, 6] ![1, 2, 4, 5] ![1, 2, 4, 6]

Len 4[0, 1, 2, 3, 6] ![0, 1, 2, 4, 6] ![0, 1, 2, 4, 5] !

17

Control Flow Graphs : The if Statement

if (x < y){ y = 0; x = x + 1;}else{ x = y;}

4

1

2 3

x >= yx < y

x = yy = 0

x = x + 1

if (x < y){ y = 0; x = x + 1;}

3

1

2x >= y

x < y

y = 0x = x + 1

18

CFG : The if-Return Statementif (x < y){ return;}print (x);return;

3

1

2x >= y

x < y

return

print (x)return

19

Control Flow Graphs : while and for Loops

x = 0;while (x < y){ y = f (x, y); x = x + 1;}

1x = 0

x >= yx < y

x >= yx < y 43

y =f(x,y)x = x + 1

2for (x = 0; x < y; x++){ y = f (x, y);}

2

3 5

x >= yx < y

y = f (x, y)

4

1

x = x + 1

20

Control Flow Graphs : do Loop, break and continue

x = 0;do{ y = f (x, y); x = x + 1;} while (x < y);println (y)

1x = 0

2x >= y

x < y

y = f (x, y)x = x+1

3

x = 0;while (x < y){ y = f (x, y); if (y == 0) { break; } else if y < 0) { y = y*2; continue; } x = x + 1;}print (y);

1 x = 0

8

3

x = x + 1

break

y < 0

2

4

5

6

7

y =f(x,y)

y == 0

y = y*2continue