Introduction to Software

Testing
(2nd edition)
Chapter 7.1, 7.2

Overview Graph Coverage

Criteria
Paul Ammann & Jeff Offutt

http://www.cs.gmu.edu/~offutt/softwaretest/

Update, January 2016

 Note

• This is the modified version of the original textbook slide

for SWE 202 course.

Introduction to Software Testing, Edition 2 (Ch 3) © Ammann & Offutt 2

 Ch. 7 : Graph Coverage
Four Structures for
Modeling Software

Input Space Graphs Logic Syntax

Applied Applied to
to
Applied
Source FSMs
to

Specs DNF

Source Specs Source Models

Design Use cases Integ Input

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 3
 Covering Graphs (7.1)
• Graphs are the most commonly used structure for testing
• Graphs can come from many sources
– Control flow graphs
– Design structure
– Finite state machines (FSMs) and statecharts
– Use cases
• Tests usually are intended to “cover” the graph in some
way
• Artifact: Anything associated with software.
• Main idea: Extract a graph from the artifact.
– Map test cases for the artifact to paths in the graph.

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 4

 Definition of a Graph
• A graph G formally is;
– A set N of nodes, N is not empty
– A set N0 of initial nodes, N0 is not empty, N0 ⊆ N
– A set Nf of final nodes, Nf is not empty, Nf ⊆ N
– A set E of edges, E ⊆ NxN

• Each edge from one node to another → (ni , nj)

– ni is predecessor, nj is successor

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 5

 Example Graphs
1 1 2 3 1

Not a
valid
graph
2 3 4 5 6 7 2 3

4 8 9 10 4

N0 = { 1} N0 = { 1, 2, 3 } N0 = { }
Nf = { 4 } Nf = { 8, 9, 10 } Nf = { 4 }
E = { (1,2), (1,3), E = { (1,4), (1,5), (2,5), (3,6), (3, 7), (4, E = { (1,2), (1,3),
(2,4), (3,4) } 8), (5,8), (5,9), (6,2), (6,10), (7,10) (2,4), (3,4) }
Introduction to Software Testing, Edition 2 (Ch 07)
(9,6) }© Ammann & Offutt 6
 Paths in Graphs
• Path: A sequence of nodes – [n1, n2, …, nM]
– Each pair of nodes is an edge
• Length (of a path): The number of edges
– A single node is a path of length 0
• Subpath: A subsequence of nodes in p is a subpath of p
• Cycle: A path that begins and ends in the same node.

1 2 3 A Few Paths A Few Invalid

Paths
[ 1, 4, 8 ]
[ 1, 8 ]
[ 2, 5, 9, 6, 2 ]
4 5 6 7 [ 4, 5 ]
[ 3, 7, 10 ]
[ 3, 7, 9 ]

8 9 10
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 7
 Test Path
• A location in a graph (node or edge) can be reached from
another location if there is a sequence of edges from the
first location to the second
– Syntactic reach : A subpath exists in the graph
– Semantic reach : A test exists that can execute that subpath
• Test Path: A path p, possibly of length zero, that starts at
some node in N0 and ends at some node in Nf.
– A path that starts at an initial node and ends at a final node
• Test paths represent execution of test cases
– Some test paths can be executed by many tests.
– Some test paths cannot be executed by any test.

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 8

 SESE Graph
• SESE Graph → Single Entry / Single Exit Graph
• SESE graph: All test paths start at a single node and end
at another node
• N0 has exactly one node. → n0
• Nf has exactly one node. → nf
• No edges start at nf, except when n0 and nf happen to be
the same node.
Double-diamond graph
2 5 Four test paths
[1, 2, 4, 5, 7]
1 4 7
[1, 2, 4, 6, 7]
3 6 [1, 3, 4, 5, 7]
[1, 3, 4, 6, 7]
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 9
 Visiting and Touring
• Visit: A test path p visits node n if n is in p
A test path p visits edge e if e is in p
• Tour: A test path p tours subpath q if q is a subpath of p

Test Path [ 1, 2, 4, 5, 7 ]
Visits nodes 1, 2, 4, 5, 7
Visits edges (1, 2), (2, 4), (4, 5), (5, 7)
Tours subpaths [1, 2, 4], [2, 4, 5], [4, 5, 7], [1, 2, 4, 5],
[2, 4, 5, 7], [1, 2, 4, 5, 7]
(Also, each edge is technically a subpath)

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 10

 Tests and Test Paths
• pathG(t): The test path in graph G that is executed by test t.
– If graph is obvious don’t need to use G. → path(t)
• path(T): The set of test paths executed by the set of tests
T.
• Each test executes one and only one test path
– Complete execution from a start node to a final node

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 11

 Tests and Test Paths
test 1 many-to-one
Test
test 2
Path
test 3
Deterministic software → A test always executes the same test path

many-to-many
test 1 Test Path 1

test 2 Test Path 2

test 3 Test Path 3

Non-deterministic software → The same test can execute different test paths

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 12

 Tests and Test Paths
• Test cases and corresponding test paths
• Test case t1: (a=0, b=1) maps to Test path p1=[1,2,4,3]
• Test case t2: (a=1, b=1) maps to Test path p2=[1,4,3]
• Test case t3: (a=2, b=1) maps to Test path p3=[1,3]

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 13

Testing and Covering Graphs (7.2)
• We use graphs in testing as follows:
– Develop a model of the software as a graph
– Require tests to visit or tour specific sets of nodes, edges or
subpaths
• Structural Coverage Criteria: Defined on a graph just in terms of
nodes and edges.
• Data Flow Coverage Criteria: Requires a graph to be annotated
with references to variables.
• Test Requirements (TR): Describe properties of test paths
• Test Criterion : Rules that define test requirements
• Graph Coverage : Given a set TR of test requirements for a criterion C,
a set of tests T satisfies C on a graph if and only if for every test
requirement in TR, there is a test path in path(T) that meets the test
requirement tr.
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 14
 Structural Coverage Criteria
• Node coverage (NC) criterion
• Edge coverage (EC) criterion
• Edge-Pair Coverage (EPC) criterion
• Complete Path Coverage (CPC) criterion
• Specified Path Coverage (SPC) criterion
• Prime Path Coverage (PPC) criterion
• Simple Round Trip Coverage (SRTC) criterion
• Complete Round Trip Coverage (CRTC) criterion

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 15

 Node and Edge Coverage
• The first (and simplest) two criteria require that each
node and edge in a graph be executed
– Node coverage
– Edge coverage

Node Coverage (NC): TR contains each reachable node

in G.

Edge Coverage (EC): TR contains each reachable path of

length up to 1, inclusive, in G.

• The phrase “length up to 1” allows for graphs with one

node and no edges
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 16
 Node and Edge Coverage
• NC and EC are only different when there is an edge and
another subpath between a pair of nodes (as in an “if-
else” statement)
• if-else structure without else

Test path -> path(t1) = [1, 2, 3]

Test path -> path(t2) = [1, 3]
1
x<y Node Coverage : TR = {1, 2, 3}
x>=y T1 = {t1}
2 T1 satisfies node coverage on the graph.

3 Edge Coverage :TR = {(1, 2), (1, 3), (2, 3)}

T2 = {t1, t2}
T2 satisfies edge coverage on the graph.

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 17

 Covering Multiple Edges
• Edge-pair coverage requires pairs of edges, or subpaths of
length 2
Edge-Pair Coverage (EPC): TR contains each reachable
path of length up to 2, inclusive, in G.

• The phrase “length up to 2” is used to include graphs that

have less than 2 edges
1 5
Edge-Pair Coverage :
2 4 TR = { [1,4,5], [1,4,6], [2,4,5],
[2,4,6], [3,4,5], [3,4,6] }
3 6

• The logical extension is to require all paths …

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 18
 Paths of Length 1 and 0
• A graph with only one node will not have any edges

• It may seem trivial, but formally, Edge Coverage needs to

require Node Coverage on this graph
• Otherwise, Edge Coverage will not subsume Node
Coverage
– So, we define “length up to 1” instead of simply “length 1”
• We have the same issue with graphs that
1
only have one edge – for Edge-Pair
Coverage …
2

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 19

 Covering Multiple Edges

Complete Path Coverage (CPC): TR contains all paths in G.

This is impossible if the graph has a loop, so a weak

compromise makes the tester decide which paths:
Specified Path Coverage (SPC): TR contains a set S of
test paths, where S is supplied as a parameter.

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 20

 Structural Coverage Example
Node Coverage
TR = { 1, 2, 3, 4, 5, 6, 7 }
Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 6, 5, 7 ]
1
Edge Coverage
TR = { (1,2), (1, 3), (2, 3), (3, 4), (3, 5), (4, 7), (5, 6), (5, 7),
2 (6, 5) }
Test Paths: [ 1, 2, 3, 4, 7 ] [1, 3, 5, 6, 5, 7 ]
3
Edge-Pair Coverage
TR = {[1,2,3], [1,3,4], [1,3,5], [2,3,4], [2,3,5], [3,4,7],
4 5 [3,5,6], [3,5,7], [5,6,5], [6,5,6], [6,5,7] }
Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 7 ] [ 1, 3, 4, 7 ]
[ 1, 3, 5, 6, 5, 6, 5, 7 ]
6
7
Complete Path Coverage
Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 7 ] [ 1, 2, 3, 5, 6, 5,7
] [ 1, 2, 3, 5, 6, 5, 6, 5, 7 ] [ 1, 2, 3, 5, 6, 5, 6, 5, 6, 5, 7 ] …
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 21
 Handling Loops in Graphs
• If a graph contains a loop, it has an infinite number of paths

• Thus, CPC is not feasible

• SPC is not satisfactory because the results are subjective

and vary with the tester

• Attempts to “deal with” loops:

– 1970s : Execute cycles once ([4, 5, 4] in previous example, informal)
– 1980s : Execute each loop, exactly once (formalized)
– 1990s : Execute loops 0 times, once, more than once (informal description)
– 2000s : Prime paths (touring, sidetrips, and detours)

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 22

 Simple Paths and Prime Paths
• Simple Path: A path from node ni to nj is simple if no node
appears more than once, except possibly the first and last
nodes are the same.
– No internal loops
– A loop is a simple path
• Prime Path: A simple path that does not appear as a
proper subpath of any other simple path.

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 23

 Simple Paths and Prime Paths
• [1,3,5,7] is a simple path
• [5,6,5] is a loop and simple path.
1 • [1,3,5,6,5,7] is not a simple path

4 5

6
7

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 24

 Simple & Prime Path Example
Simple Len 0 Len 1 Len 2 Len 3
paths [1] [1, 2] [1, 2, 3] [1, 2, 3, 4]
[2] [1, 3] [1, 3, 4] [1, 2, 3, 5]
[3] [2, 3] [1, 3, 5] [1, 3, 4, 7] !
1 [4] [3, 4] [2, 3, 4] [1, 3, 5, 7] !
[5] [3, 5] [2, 3, 5] [1, 3, 5, 6] !
2 [6] [4, 7] ! [3, 4, 7] ! [2, 3, 4, 7] !
[7] ! [5, 7] ! [3, 5, 7] ! [2, 3, 5, 6] !
[5, 6] [3, 5, 6] ! [2, 3, 5, 7] !
3
[6, 5] [5, 6, 5] *
[6, 5, 7] !
4 5 [6, 5, 6] *

6 Len 4
7 [1, 2, 3, 4, 7] !
[1, 2, 3, 5, 7] ! Prime Paths
[1, 2, 3, 5, 6] !
! means path terminates
* means path cycles
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 25
 Simple & Prime Path Example
Simple Paths
[1], [2], [3], [4],
[1,2], [1,3], [2,4], [3,4], [4,1],
[1,2,4], [1,3,4], [2,4,1], [3,4,1], [4,1,2], [4,1,3],
[1,2,4,1], [1,3,4,1], [2,4,1,2], [2,4,1,3], [3,4,1,2], [3,4,1,3], [4,1,2,4], [4,1,3,4]

Prime Paths
1
[1,2,4,1], [1,3,4,1],
2 3
[2,4,1,2], [2,4,1,3],
4 [3,4,1,2], [3,4,1,3],
[4,1,2,4], [4,1,3,4]

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 26

 Prime Path Coverage
• A simple, elegant and finite criterion that requires loops
to be executed as well as skipped

Prime Path Coverage (PPC): TR contains each prime path

in G.

• Will tour all paths of length 0, 1, …

• That is, it subsumes node and edge coverage
• PPC almost, but not quite, subsumes EPC …

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 27

 Round Trips
• Round-Trip Path: A prime path that starts and ends at the
same node.
Simple Round Trip Coverage (SRTC): TR contains at
least one round-trip path for each reachable node in G
that begins and ends a round-trip path.

Complete Round Trip Coverage (CRTC): TR contains all

round-trip paths for each reachable node in G.
• These criteria omit nodes and edges that are not in round
trips.
• Thus, they do not subsume edge-pair, edge, or node
coverage.
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 28
 Touring, Sidetrips, and Detours
• Visit and tour were previously defined.
• Using a path p to tour a subpath means that the subpath is
a subpath of p.
• Strict definition → Each node and edge in the subpath
must be visited exactly in the order that they appear in the
subpath.
• Need to relax this a bit to allow loops to be included in
the tour.

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 29

 Touring, Sidetrips, and Detours
• For example, we are required to tour subpath q=[2,3,5]
• Any path that contains 4 does not meet the requirement.
– Because we don’t visit in exactly the same order.
• P=[1,2,3,4,3,5,6] does not meet the requirement.
• Relax the tour definition in two ways: sidetrips, detours.

1 2 3 5 6

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 30

 Touring, Sidetrips, and Detours
• Prime paths do not have internal loops … test paths
might
• Tour: A test path p tours subpath q if q is a subpath of p
• Tour With Sidetrips: A test path p tours subpath q with
sidetrips if and only if every edge in q is also in p in the same
order.
• The tour can include a sidetrip, as long as it comes back to the
same node.
• Tour With Detours: A test path p tours subpath q with
detours if and only if every node in q is also in p in the same
order.
• The tour can include a detour from node ni, as long as it comes
back to the prime path at a successor of ni
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 31
 Sidetrips and Detours Example
1 2 3 4
1 2 3 5 6

Touring the prime path

[1, 2, 3, 5, 6] without 4
sidetrips or detours
1 2 5 6
1 2 3 5 6
3 4
Touring with a
sidetrip 4

1 2 5
1 2 3 5 6
3
Touring with a 4
4
detour

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 32

 Infeasible Test Requirements
• An infeasible test requirement cannot be satisfied
– Unreachable statement (dead code)
– Subpath that can only be executed with a contradiction (X > 0 and X < 0)
• Most test criteria have some infeasible test requirements
• It is usually undecidable whether all test requirements are
feasible
• When sidetrips are not allowed, many structural criteria
have more infeasible test requirements
• However, always allowing sidetrips weakens the test
criteria
Practical recommendation - Best Effort Touring
– Satisfy as many test requirements as possible without
sidetrips
– Allow sidetrips to try to satisfy remaining test requirements
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 33
 Graph Coverage Criteria Subsumption
Complete
Path Coverage
CPC

Prime Path Specified Path

Coverage Coverage
PPC SPC

Edge-Pair
Coverage
EPC
Complete Round
Trip Coverage
Edge CRTC
Coverage
EC
Simple Round
Trip Coverage
Node SRTC
Coverage
NC
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 34
 Summary 7.1-7.2
• Graphs are a very powerful abstraction for designing tests

• The various criteria allow lots of cost / benefit tradeoffs

• These two sections are entirely at the “design abstraction
level” from chapter 2
• Graphs appear in many situations in software
– As discussed in the rest of chapter 7

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 35