Unit 2

Software Testing Strategies

Black Box Approaches
Based on the following textbooks:
lene Burnstein, Practical Software Testing: A Process-Oriented Approach,
Springer, 2003. (Chapter 4)
Outline
• Classification of Software Testing Strategies
• Definition of a Test Case
• Black box testing strategies:
• Exhaustive Strategy
• Random Testing Strategy
• Equivalence Class Partitioning
• Boundary Value Analysis
• Other black box testing strategies:
• Cause-and-Effect Graphing
• State Transition Testing
Classification of Software Testing
Strategies
• Black box (functional or specification) testing strategies:
• These strategies have no knowledge of the inner structure of the software-
under-test (i.e., how it works).
• White box (structural or glass box) testing strategies:
• These strategies take into account knowledge of the inner structure of the
software-under-test.
Classification of Software Testing
Strategies
Definition of a Test Case
• A test case is a test-related item which contains the following
information:
• A set of test inputs.
• Execution conditions. These are conditions required for running the test.
• Expected outputs.
Effective Test Cases
• It is important to develop and use effective test cases.
• An effective test case has a good possibility of revealing a defect.
• If test cases are effective, then:
• There is a greater probability of detecting defects.
• There is a more efficient use of organizational resources.
• There is a higher probability for test reuse.
• There is closer adherence to testing and project schedules and budgets.
• There is a greater possibility for delivery of a higher-quality software product.
Black Box and White Box Testing
Strategies
• To achieve the goal of providing users with low-defect, high-quality
software, both of these strategies should be used to design test cases.
• Neither approach by itself is guaranteed to reveal all defects types.
• The approaches complement each other
Using the Black Box Approach to
Test Case Design
• The tester has no knowledge of the inner structure of the software-
under-test’s. The tester only has knowledge of what it does.
• The size of the software-under-test can vary from a simple operation
to a subsystem or a complete software system.
• The description of functionality for the software-under-test may come
from a formal specification, a well-defined set of pre and post
conditions, or a requirements specification document
Back Box Testing Approaches
• Exhaustive Testing:
• This is prohibitively expensive e.g., testing a procedure that calculates the square root of a
number.
• Random Testing:
• A tester randomly selects inputs from the domain.
• Random testing has little chance of producing an effective set of test cases.
• Error Guessing:
• The tester makes an educated guess as to which types of defects may be present.
• Error guessing is still an ad hoc approach in most cases.
• Structured Approaches:
• Using Equivalence Class Partitioning and Boundary Value Analysis
• Using Cause-and-Effect Graphs
• Using State Transition Models
Equivalence Class Partitioning
• Equivalence class partitioning is a black box method aimed at increasing
the efficiency of testing, and at the same time, improving coverage of
potential error conditions.
• Equivalence class partitioning results in a partitioning of the input (or
output) variable values such that a given member of an equivalence class
can be selected as a representative of that class.
• All members of an equivalence class are processed in an equivalent way.
• Therefore, if a test case in a particular equivalence class reveals a defect, all the
other test cases based on that class would be expected to reveal the same
defect.
• Also, if a test case in a particular equivalence class does not reveal a defect, then
no other test cases based on that class would detect the defect.
Equivalence Class Partitioning
• Advantages:
1. It eliminates the need for exhaustive testing, which is not feasible.
2. It guides a tester in selecting a subset of test inputs with a high probability
of detecting a defect.
3. It allows a tester to cover a larger domain of inputs/outputs with a smaller
subset selected from an equivalence class.
Equivalence Class Partitioning
Notes
1. The tester must consider both valid and invalid equivalence classes.
2. Equivalence classes may also be selected from output conditions.
3. The derivation of inputs or outputs equivalence classes is a heuristic process.
There are no hard and fast rules; only guidelines are given for identifying the
partitions.
4. In some cases it is difficult for the tester to identify equivalence classes.
Equivalence Class Partitioning
Myers Guidelines
1. If an input condition for the software-under-test (SUT) is specified as a
range of values, select one valid equivalence class that covers the allowed
range and two invalid equivalence classes, one outside each end of the
range.
2. If an input condition for the SUT is specified as a number of values, select
one valid equivalence class that includes the allowed number of values and
two invalid equivalence classes that are outside each end of the allowed
number.
3. If an input condition for the SUT is specified as a set of valid input values,
select one valid equivalence class that contains all the members of the set
and one invalid equivalence class for any value outside the set.
Equivalence Class Partitioning
Myers Guidelines
4. If an input condition for the SUT is specified as a “must be” condition,
select one valid equivalence class to represent the “must be” condition and
one invalid class that does not include the “must be” condition.
5. If the input specification or any other information leads to the belief that an
element in an equivalence class is not handled in an identical way by the
SUT, then the class should be further partitioned into smaller equivalent
classes.
Equivalence Class Partitioning
Myers Guidelines
1. Each equivalence class should be assigned a unique identifier.
2. Develop test cases for all valid equivalence classes until all have
been covered (included in) a test case.
• A given test case may cover more than one valid equivalence class.
3. Develop test cases for all invalid equivalence classes until all have
been covered individually.
• This is to ensure that one invalid case does not mask the effect of another or
prevent the execution of another.
Equivalence Class Partitioning
Myers Guidelines - Example
The specification describes two
conditions for the input variable x:
i) x must be a real number.
ii) x must be ≥ 0.0

Therefore , the possible input

equivalence classes are:
EC1. The input variable x is real,
valid.
EC2. The input variable x is not real,
invalid.
EC3. The value of x is equal to or
greater than 0.0, valid.
EC4. The value of x is less than 0.0,
invalid.
Boundary Value Analysis
• The test cases developed based on equivalence class partitioning can
be strengthened by use of boundary value analysis.
• Many defects occur directly on, and above and below, the edges of
equivalence classes.
Boundary Value Analysis
1. If an input condition for the SUT is specified as a range of values, develop valid
test cases for the ends of the range, and invalid test cases for possibilities just
above and below the ends of the range.
• E.g., if a specification states that an input value for a module must be an integer that lies
between -1 and 1, Develop test cases for the input values of -2, -1, 1, and 2.
2. If an input condition for the SUT is specified as a number of values, develop valid
test cases for the minimum and maximum numbers as well as invalid test cases
that include one lesser and one greater than the maximum and minimum.
• E.g., if a specification states that a house can have one to four rooms, tests that include 0, 1, 4,
and 5 rooms need to be developed.
3. If the input or output of the SUT is an ordered set, develop tests that focus on the
first and last elements of the set
Notes on Equivalence Class
Partitioning and Boundary Value
Analysis
• Equivalence class partitioning and boundary value analysis apply to
testing both inputs and outputs of the SUT.
• Conditions are not combined for equivalence class partitioning or
boundary value analysis
• Each condition is considered separately, and test cases are developed to
insure coverage of all the individual conditions.
Example on Equivalence Class
Partitioning and Boundary Value
Analysis
• Suppose that the input specification for a module states that an input
value should consist of 3-15 alphanumeric characters of which the
first two must be letters.
• There are three separate conditions that apply to the input:
1. It must consist of alphanumeric characters,
2. The range of the total number of characters is between 3 and 15, and
3. The first two characters must be letters.
Example on Equivalence Class
Partitioning and Boundary Value
Analysis
Condition Equivalence Classes
The input must consist of alphanumeric characters.
(“Must” be condition)

The range of the total number of characters is

between 3 and 15.
(Range condition)

The first two characters must be letters.

(“Must be” condition)
Example on Equivalence Class
Partitioning and Boundary Value
Analysis
• The bound groups used in the boundary value analysis are
represented as follows:
Example on Equivalence Class
Partitioning and Boundary Value
Analysis
Other Black Box Test Design
Approaches
• Cause-and-Effect Graphing
• State Transition Testing
Cause-and-Effect Graphing
• A major weakness with equivalence class partitioning is that it does
not allow testers to combine conditions.
• Cause-and-effect graphing is a technique that can be used to combine
conditions and derive an effective set of test cases that may disclose
inconsistencies in a specification.
Cause-and-Effect Graphing
Notations
Cause-and-Effect Graphing
Example
Consider a module that allows a user to perform a search for a
character in an existing string:
• The user must input the string and the character to search for.
• If the string length is out-of-range, an error message will appear.
• If the character appears in the string, its position will be reported.
• If the character is not in the string, the message “not found” will be output.
Cause-and-Effect Graphing
Example
Cause-and-Effect Graphing
Example
For the test cases T1 and T2 below, use the input string “abcde”
Cause-and-Effect Graphing
Limitations
• The size of the cause-and-effect graphs (tables) can grow
exponentially with the number of conditions.
• In these conditions tool support becomes necessary.
State Transition Testing
• This method is based on the concepts of states and finite-state
machines.
• It allows the tester to view the software in term of its states,
transitions between states, and the inputs and events that trigger
state changes.
• This view gives the tester an additional opportunity to develop test
cases to detect defects that may not be revealed using the
input/output conditions as well as cause-and-effect views presented
by equivalence class partitioning and cause-and-effect graphing.
State Transition Testing
Definitions
• A state is an internal configuration of a system or component. It is
defined in terms of the values assumed at a particular time for the
variables that characterize the system or component.
• A finite-state machine (FSM) is an abstract machine that can be
represented by a state graph having a finite number of states and a
finite number of transitions between states.
State Transition Testing
Example

Simple State transition graph The corresponding state table

State Transition Testing
• The test cases can be designed:
• To cover all paths.
• To enter all states.
• To exercise every possible state transition (as in the example below):
State Transition Testing
• It is advisable to develop test cases that exercise many transitions,
especially those that look complex, may not have been tried before,
or that look ambiguous or unreachable.