syllabus

• Traditional view of testing levels,

• Alternative life cycle models,
• the SATM systems,
• separating integration and system
testing, Guidelines and observations.

1
 Overview
• To enhance the quality of software testing, and
to produce a more unified testing methodology
applicable across several projects, the testing
process could be abstracted to different levels.

• This classification into different levels introduces

some parallelism in the testing process as
multiple tests could be performed simultaneously.

• Although these levels are best suited for the

waterfall model, (since the levels correspond
directly to the different stages in the waterfall
model) the level of abstraction still poses to be
useful across other software development models.
2
 Testing in the V-Model

Requirements Acceptance
Customer test
Developer
Architectural System test

s
t e st
Design
Wri

Run
Integration test
te te

Detailed
Design Functional
st s

(BB)
Module Structural
Unit test (WB)
implementation
3
 Forms of Integration Testing
• Integration testing can take many different forms. These
include:

“Big-bang” integration
Bottom-up integration
Top-down integration
Sandwich integration

• The above-named integration testing techniques are

decomposition-based integration testing
techniques.

• Decomposition-based integration testing techniques

involve breaking down the software system into smaller
components or modules and then integrating and testing
these components in a controlled manner.
4
• By this, it is meant that the system is functionally
 Forms of Integration Testing

• Starting from the main program, we break down each

subsystem using the list of units it calls.

• These units become the child nodes of the starting node

and they are further broken down into their own
constituent units.

• The units at the leaves of the tree are those that have no
calls to other procedures or functions.

5
 Forms of Integration Testing
• A partial functional decomposition of the simple
automated teller machine system can be found below.

• We start with the main system and check the possible function
calls from the system. These are calls to the Terminal I/O,
Manage Sessions and Conduct Transaction functions. The
decomposition goes on with each node till a leaf node is
reached.
• A partial functional decomposition of the SATM

6
 Integration Testing Strategy
• The entire system is viewed as a collection of
subsystems (sets of classes) determined during the
system and object design.

• The order in which the subsystems are selected for

testing and integration determines the testing
strategy
– Big bang integration (Nonincremental)
– Bottom-up integration
– Top-down integration
– Sandwich testing
– Variations of the above

• For the selection use the system decomposition

from the System Design
Example: Three Layer Call Hierarchy

A
Layer I

B C D Layer II

E F G Layer III
 Integration Testing: Big-Bang Approach
Unit Test
A
Don’t try this!
Unit Test
B
Unit Test
C
System Test
Unit Test
D
Unit Test
E
Unit Test
F

Big-Bang Integration: Integrate all the modules at

once.
 Bottom-up Testing Strategy
• The subsystem in the lowest layer of the call
hierarchy are tested individually

• Then the next subsystems are tested that call the

previously tested subsystems

• This is done repeatedly until all subsystems

are included in the testing

• Special program needed to do the testing, Test

Driver:
– A routine that calls a subsystem and
passes a test case to it.
 Bottom-up Integration
A
Layer I

B C D Layer II

Test E E F G
Layer III

Test B, E, F
Test F
Test
Test C
A, B, C, D,
E, F, G

Test D,G
Test G

Special program needed to do the testing, Test

Driver:
12
Pros and Cons of bottom up integration testing

• Useful for integrating the following systems

– Object-oriented systems
– real-time systems with strict performance
requirements
 Top-down Testing Strategy
• Test the top layer or the controlling subsystem first

• Then combine all the subsystems that are called by

the tested subsystems and test the resulting collection
of subsystems

• Do this until all subsystems are incorporated into

the test

• Special program is needed to do the testing, Test

stub :
– A program or a method that simulates the
activity of a missing subsystem by answering
to the calling sequence of the calling
subsystem and returning back fake data.
Top-down Integration
A
Testing Layer I

B C D Layer II

E F G
Layer III

Test
Test A Test A, B, C, D A, B, C, D,
E, F, G
Layer I
Layer I + II
All Layers

Special program is needed to do the testing, Test

stub :
16
 Pros and Cons of top-down integration
testing
• Test cases can be defined in terms of the
functionality of the system (functional
requirements)
• Writing stubs can be difficult: Stubs must allow all
possible conditions to be tested.
• Possibly a very large number of stubs may be
required, especially if the lowest level of the
system contains many methods.
• One solution to avoid too many stubs: Modified top-
down testing strategy
– Test each layer of the system decomposition
individually before merging the layers
– Disadvantage of modified top-down testing: Both,
stubs and drivers are needed
 Sandwich Testing Strategy

• Combines top-down strategy with bottom-up

strategy

• The system is view as having three layers

– A target layer in the middle
– A layer above the target
– A layer below the target
– Testing converges at the target layer

• How do you select the target layer if there are more

than 3 layers?
– Heuristic: Try to minimize the number of stubs and
drivers
 A
Sandwich Testing Strategy Layer I

B C D Layer II

E F G
Test E Layer III

up

Bottom Test B, E, F
Test F
Layer
Tests Test
A, B, C, D,
Test D,G E, F, G
Test G

Test A,B,C, D

Top Test A down

Layer
Tests
20
A Closer Look at the SATM System

21
A Closer Look at the SATM System

22
 A Closer Look at the SATM System
• The decomposition tree is the basis for the usual view
of Integration testing.

• As software design moves into more detail, the added

information lets us refine the functional decomposition
tree into a unit calling graph.

• The unit calling graph is the directed graph in

which nodes are program units and edges
correspond to program calls.

• Nodes with a high degree will be important to

integration testing, and paths from the Main program
(node 1) to the sink nodes can be used to identify
contents of builds for an incremental development. 23
24
 Call Graph Based Integration
• The basic idea is to use the call graph instead of
the decomposition tree

• The call graph is a directed, labeled graph

• Two types of call graph based integration

testing
– Pair-wise Integration Testing
– Neighborhood Integration Testing

25
 Pair-Wise Integration Testing

• The idea behind Pair-Wise integration testing is

to eliminate the need for developing
stubs/drivers

• The objective is to use actual code instead of

stubs/drivers

• The result is that we have one integration test

session for each edge in the call graph

26
27
 Neighborhood Integration
Testing
• We define the neighborhood of a node in a graph to
be the set of nodes that are one edge away from
the given node

• In a directed graph means all the immediate

predecessor nodes and all the immediate successor
nodes of a given node

• The number of neighborhoods for a given graph

can be computed as:

InteriorNodes = nodes – (SourceNodes + SinkNodes)

Neighborhoods = InteriorNodes + SourceNodes
Or
Neighborhoods = nodes – SinkNodes

• Neighborhood Integration Testing reduces the

28
number of test sessions.
29
 Advantages and Disadvantages of
Call-Graph Integration Testing

• Aim to eliminate / reduce the need for

drivers/stubs

• Neighborhoods can be combined to create

“villages”

• Suffer from the fault isolation problem

especially for large neighborhoods

• Nodes can appear in several neighborhoods

30
 Path Based Integration Testing

• The basic motivation is to combine structural

and behavioral type of testing for integration
testing as we did for unit testing

• The basic idea is to focus on interactions

among system units rather than merely to
test interfaces among separately developed
and tested units

• In this respect, interface-based testing is

structural while interaction-based is
behavioral

• Overall we want to express integration testing in

terms behavioral threads 31
 Extended Concepts (1)
• Source node:
– a program statement fragment at which
program execution begins or resumes.
– for example, the first “begin” statement in a
program.
– also, immediately after nodes that transfer
control to other units.
• Sink node:
– a statement fragment at which program
execution terminates.
– the final “end” in a program as well as
statements that transfer control to other units.
32
 Extended Concepts (2)
• Module execution path:
– a sequence of statements that begins with a
source node and ends with a sink node with no
intervening sink nodes.
• Message:
– a programming language mechanism by which
one unit transfers control to another unit.
– can be interpreted as subroutine invocations,
procedure calls and function references.
– convention: the unit which receives the
message always eventually returns control to
the message source.
– messages can pass data to other units.
33
 MM-Paths
• MM-Path :
– an interleaved sequence of module
execution paths and messages.

– We can describe sequences of module

execution paths that include transfers of
control among separate units.

– MM-paths always represent feasible execution

paths, and these paths cross unit boundaries.

34
 MM-Path Example

MODULE_A (input I) MODULE_B(Input I) MODULE_C(Input I)

1 X=2*I 1 A=2*I 1 IF I > 0
2 2 IF X > 0 2 B = Module_C(A) 3 J=2*I
4 Y = module_B(X) 3 C=B*B ELSE
5 Z = 2 * Y + 1 ELSE 4 RETURN C 2 J = -2 * I
6 Z=0 END MODULE END
END 4 K=J+1
6 RETURN Z 5 RETURN K
END MODULE END MODULE

This example has three modules:

MODULE_A, MODULE_B and MODULE_C

35
Ref: “Software Testing A Craftsman's Approach” 2nd edition, Paul C. Jorgensen
 MM-Path Example

1 C Figure 1
1
A 1 B
2
2 3
2
3 4
3 4

5
4 5
6

The Figure 1 above illustrates an MM-Path (the heavy

line) for three modules.
• For example, in module A nodes 1 and 5 are
source nodes while nodes 4 and 6 are sink nodes.
36
Ref: “Software Testing A Craftsman's Approach” 2nd edition, Paul C. Jorgensen
 Execution Paths Example
• In addition, the following are module execution
paths:

MEP(A,1) = <1,2,3,6> MEP(B,1) = <1,2>

MEP(A,2) = <1,2,4> MEP(B,2) = <3,4>
MEP(A,3) = <5,6> MEP(C,1) = <1,2,4,5>
MEP(C,2) = <1,3,4,5>

37
 MM-Path Graph
• Given a set of units,
their MM-Path graph is the directed graph in which
nodes are module execution paths and, edges
correspond to messages and returns from one unit
to another

• The definition is with respect to a set of units.

It directly supports composition of units and

composition-based integration testing.

It is possible to compose down to level of individual

module execution paths
• but it would be more detailed than necessary.
38
 MM-Path Graph Example
MEP(A,2)

MEP(B,1)

MEP(A,1) MEP(C,1)

Figure 2
MEP(B,2)
MEP(C,2)
MEP(A,3)

The Figure 2 above illustrates the MM-Path graph for

the example in Figure 1. The solid arrows indicate
messages and the corresponding returns are
indicated by dotted arrows.
39
Ref: “Software Testing A Craftsman's Approach” 2nd edition, Paul C. Jorgensen
 Alternative Life Cycle Models
• The traditional model of software development is
the waterfall model.

40
 Alternative Life Cycle Models
• It is sometimes drawn as a V to emphasize how
the basic levels of testing reflect the early
waterfall phases as V-Model.

41
 Alternative Life Cycle Models

• On the upper left side of the waterfall, the

tight what/how cycles are important.

• They underscore the fact that the predecessor

phase defines what is to be done in the
successor phase.

• When complete, the successor phase states

how it accomplishes “what” was to be done.

• These are also ideal points at which to

conduct software reviews

42
Alternative Life Cycle Models

43
Alternative Life Cycle Models

44
Alternative Life Cycle Models

45
The SATM system

46
The SATM system

47
 The SATM system
• The SATM system is described here with a
traditional, structured analysis approach.

• The technique is based on three complementary

models:
– Function, data and control.

• Here we use Data flow diagrams for functional

model, the entity/relationship model for data,
and finite state machine models for the control
aspect of the SATM system.

• The portion of the SATM system shown here pertain

generally to the personal identification
number(PIN) verification portion of the system.
48
49
50
51
52
53
54
55