1.

Implementation/Programming Guidelines
2. Software Testing Concepts
3. Unit Testing
4. Module Integration and Testing Strategies
5. Testing Approaches
1. White-Box Testing
2. Black-Box Testing
6. Other Types of Testing

Software Testing 2
1. Implementation/Programming Guidelines
2. Software Testing Concepts
3. Unit Testing
4. Module Integration and Testing Strategies
5. Testing Approaches
1. White-Box Testing
2. Black-Box Testing
6. Other Types of Testing

 9.1 
 Use of consistent and meaningful variable names
o “Meaningful” to future maintenance programmers
o “Consistent” to aid future maintenance programmers
 A code artifact includes the variable names freqAverage,
frequencyMaximum, minFr, frqncyTotl

 A maintenance programmer has to know if freq, frequency,

fr, frqncy all refer to the same thing
o If so, use the identical word, preferably frequency, perhaps freq or
frqncy, but not fr
o If not, use a different word (e.g., rate) for a different quantity
 We can use frequencyAverage, frequencyMaximum,
frequencyMinimum, frequencyTotal

 We can also use averageFrequency, maximumFrequency,

minimumFrequency, totalFrequency

 But all four names must come from the same set
Establish and use a fixed ordering for javadoc tags.
 In class and interface descriptions, use:
@author your name
@version a version number or date

 In method descriptions, use:

@param p A description of parameter p.
@return A description of the value returned
(unless it’s void).
@exception e Describe any thrown exception.

7
 Fully describe the signature of each method.
 The signature is what distinguishes one method from
another
o the signature includes the number, order, and types of the
parameters
 Use a @param tag to describe each parameter
o @param tags should be in the correct order
o Don’t mention the parameter type; javadoc does that
o Use a @return tag to describe the result (unless it’s void)

8
 There are almost no genuine constants

 One solution:
o Use const statements (C++), or
o Use public static final statements (Java)

 A better solution:
o Read the values of “constants” from a parameter file
 Document preconditions, postconditions,
and invariant conditions.
 A precondition is something that must be true
beforehand in order to use your method
o Example: The piece must be moveable
 A postcondition is something that your method makes
true
o Example: The piece is not against an edge
 An invariant is something that must always be true
about an object
o Example: The piece is in a valid row and column

10
 A combination of if-if and if-else-if statements is usually
difficult to read

 Simplify: The if-if combination

if <condition1>
if <condition2>

is frequently equivalent to the single condition

if <condition1> && <condition2>

 Rule of thumb
o if statements nested to a depth of greater than three should be
avoided as poor programming practice
 Standards can be both a blessing and a curse

 Modules of coincidental cohesion arise from rules like

o “Every module will consist of between 35 and 50 executable
statements”

 Better
o “Programmers should consult their managers before constructing a
module with fewer than 35 or more than 50 executable statements”
 “Nesting of if statements should not exceed a depth of 3,
except with prior approval from the team leader”

 “Modules should consist of between 35 and 50 statements,

except with prior approval from the team leader”

 These programming standards reduce the complexity of the

code and improve its readability and maintainability.
 Cyclomatic Complexity
 Cyclomatic Complexity V(G) is defined as the number of regions
in the flow graph.

V(G) = E – N + 2

E: number of edges in flow graph

N: number of nodes in flow graph

 Another method:
V(G) = P + 1

P: number of predicate nodes (simple decisions)

in flow graph

14
 Example-1
Flow chart Flow graph

1 Edge

1 2,3

2
6

3 7 8 R2 4,5
R3
R1
4
6
Region
9
5
7 8
R4
9
10
10
11

11
15
 Flow Graph Notation
• Each circle represents one or more nonbranching
source code statements.

if-else switch
Sequence

While
Until

16
 Compound Logic

a
If a OR b
b
then procedure x
x
y
else procedure y
ENDIF

17
Example-1 (Cyclomatic Complexity)

 V(G) = E – N + 2
=6–4+2=4
 Other method:
V(G) = P + 1
=3+1=4

18
 Example-2
Flow chart Flow graph

2
2
4 3
3
4 6
5 6 5

8
7 8

19
 Example-2 (Cyclomatic Complexity)

First, we calculate V(G):

V(G) = E – N + 2
= 10 – 8 + 2 = 4

Now, we derive the independent paths :

Since V(G) = 4, there are four paths

Path 1 : 1,2,3,6,7,8
Path 2 : 1,2,3,5,7,8
Path 3 : 1,2,4,7,8
Path 4 : 1,2,4,7,2,4,...7,8

20
1. Implementation/Programming Guidelines
2. Software Testing Concepts
3. Unit Testing
4. Module Integration and Testing Strategies
5. Testing Approaches
1. White-Box Testing
2. Black-Box Testing
6. Other Types of Testing

 9.2 
 Software Testing
• Testing is the process of executing a program to find errors.

• Testing is planned by defining “Test Cases” in a systematic

way.

• A test case is a collection of input data and expected

output.
•A good test case is one that has a high probability of finding an
as-yet-undiscovered error.

•A successful test is one that uncovers an as-yet-undiscovered

error.

22
 Most of the tests should be traceable to requirements.
o Both customer and system requirements
 Tests should be planned and implemented long before testing
begins.
o After requirements model is complete.
o Except unit testing, because units should be implemented before the tests
in a test-last approach.
 Exhaustive testing is impossible.
 Testing should be conducted by the developer of the software
and also by an independent test group.

Software Testing 1.23

 Verification and Validation
 Verification: Are we building the product right?
 Validation: Are we building the right product?

Figure from S. Easterbrook 2010

24
 Phases of Testing
1. Unit Testing
Does each module do what it supposed to do?

2. Integration Testing
Do you get the expected results when the parts
are put together?

3. System Testing
Does it work within the overall system?
Does the program satisfy the requirements ?
25
Levels of Testing

Figure from sei.cmu.edu

26
Figure from sei.cmu.edu

Software Testing 1.27

1. Implementation/Programming Guidelines
2. Software Testing Concepts
3. Unit Testing
4. Module Integration Testing
5. Testing Approaches
1. White-Box Testing
2. Black-Box Testing
6. Other Types of Testing

 9.3 
 Unit Testing
 Testing the smallest units of the code (methods, classes)
 Dynamic execution:
o Test the input/output behavior
o Also test the internal logic

 There are other static analysis techniques that are not

considered as testing.
o Hand execution/Code review (Reading the source code)
o Code inspection (also known as Walkthrough)
o Automated tools checking for syntactic and semantic
errors (FindBugs)

29
 Unit testing is done by programmers, for programmers.

 Unit tests are performed to prove that a piece of code does what
the developer thinks it should do.

 When there are dependencies between the units, it is the

programmers’ responsibility to reduce such dependencies as
much as possible, or to use additional strategies for testing the
module as a whole.

Software Testing 1.30

 Test Environment
• Stubs and drivers should be written in integration/unit testing.

Test Results
driver
cases

Module

stub stub

31
 Stub modules and driver modules
 A stub is an empty (dummy) module, which:
o Just prints a message (“Module X called"), or
o Returns pre-determined values from pre-planned test cases.
o Used in top-down strategy
 A driver is a caller module, which calls its stubs:
o Once or several times,
o Each time checking the value returned.
o Used in bottom-up strategy

32
Example: Stubs and driver (1)

void main() Driver

int f1()

bool f2() bool f3() Stubs

33
 Example: Stubs and driver (2)
#include <stdio.h> // We are testing f1
#include <math.h> int f1(int X)
#define TRUE 1 {
#define FALSE 0 if (f2(X) || f3(X))
printf("%f", sqrt(X));
// f2 is a stub else
bool f2(int X) printf("Invalid value for X“);
{ }
return TRUE;
} // This is the driver.
void main()
// f3 is a stub {
bool f3(int X) f1(49);
{ }
return TRUE;
}

34
1. Implementation/Programming Guidelines
2. Software Testing Concepts
3. Unit Testing
4. Module Integration Testing
5. Testing Approaches
1. White-Box Testing
2. Black-Box Testing
6. Other Types of Testing

 9.4 
 A strategic approach to Testing

 Testing begins at the module level and works outward

toward the integration of the entire system.

 Different testing techniques are appropriate at different

points in time.

36
 Strategies for Integration Testing
1) Big bang approach:
• All modules are fully implemented and combined as a whole,
then tested as a whole. It is not practical.

2) Incremental approach: (Top-down or Bottom-up)

• Program is constructed and tested in small clusters.
• Errors are easier to isolate and correct.
• Interfaces between modules are more likely to be tested
completely.
• After each integration step, a regression test is conducted.
37
 Big bang integration testing
Integration and
testing at once:
1. a, b, c, …, l, m

38
 If code artifact mAbove
sends a message to
artifact mBelow, then
mAbove is implemented
and integrated before
mBelow

 One possible top-down

ordering is
o a, b, c, d, e, f, g,
h, i, j, k, l ,m
 Another possible top-
down ordering is

a
[a] b, e, h
[a] c ,d, f, i
[a, d] g, j, k, l, m
 If code artifact mAbove
calls code artifact
mBelow, then mBelow is
implemented and
integrated before
mAbove

 One possible bottom-

up ordering is
l, m,
h, i, j, k,
e, f, g,
b, c, d,
a
 Another possible
bottom-up ordering is
h, e, b
i, f, c, d
l, m, j, k, g [d]
a [b, c, d]
 Logic artifacts are
integrated top-down

 Operational artifacts
are integrated
bottom-up

 Finally, the interfaces

between the two
groups are tested

Figure 15.7
1. Implementation/Programming Guidelines
2. Software Testing Concepts
3. Unit Testing
4. Module Integration and Testing Strategies
5. Testing Approaches
1. White-Box Testing
2. Black-Box Testing
6. Other Types of Testing

 9.5 
 Testing Approaches
 Black-box testing
o Testing of the software interfaces
o Used to demonstrate that
 functions are operational
 input is properly accepted and output is correctly produced

 White-box testing
o Close examination of procedural details
o Logical paths through the software is tested by test cases
that exercise specific sets of conditions and loops

45
 Exhaustive Testing
 There are 5 separate paths inside the following loop.
 Assuming loop <= 20, there are 520 ≈ 1014 possible paths.
 Exhaustive testing is not feasible.

loop

46
 Selective Testing
 Instead of exhaustive testing, a selective testing is more
feasible.

Selected path

loop

47
 There are two extremes to testing

 Test to code (also called glass-box, logic-driven, structured, or

path-oriented testing)
o Ignore the specifications — use the code to select test cases

 Test to specifications (also called black-box, data-driven,

functional, or input/output driven testing)
o Ignore the code — use the specifications to select test cases
 Example: Script for Unit Testing
 This is the format for a test plan to show what you’re planning to do.
 It should to be filled to show what happened when you run tests.

Scenario # : 1 Tester: AAA BBB Date of Test: 01/01/2024

Test Test Description / Expected
Actual Result Fix Action
Number Input Result

“Error: File
1 Invalid file name
does not exist”

Valid filename, but file “Error: File is

2
is binary not a text file”

“Average =
3 Valid filename
99.00”
4
5
49
1. Implementation/Programming Guidelines
2. Software Testing Concepts
3. Unit Testing
4. Module Integration and Testing Strategies
5. Testing Approaches
1. White-Box Testing
2. Black-Box Testing
6. Other Types of Testing

9.5.1
 White-Box Testing
 Our goal is to ensure that all statements and conditions have
been executed at least once.
 Code coverage: At a minimum, every line of code should be
executed by at least one test case.

51
 Flow graph for White box testing
 To help the programmer systematically test the code
o Each branch in the code (such as if and while statements)
creates a node in the graph

o The testing strategy has to reach a targeted coverage of

source code; the objective can be to:
 cover all possible paths (often infeasible)
 cover all possible nodes/statements (simpler)
 cover all possible edges/branches (most efficient)

52
 Example-1
Flow chart Flow graph

1 Edge

1 2,3

2
6

3 7 8 R2 4,5
R3
R1
4
6
Region
9
5
7 8
R4
9
10
10
11

11
53
 Although 100% coverage is an admirable goal, 100% of the
wrong type of coverage can lead to problems.

T,T,T would satisfy

%100 statement
coverage but does
not cover all
branches.
F,F,F and T,T,T

Software Testing 1.54

 Write test inputs for
%100 statement
coverage
 Now for %100 branch
coverage
 Are these inputs able to
catch the defects?

Software Testing 1.55

1. Implementation/Programming Guidelines
2. Software Testing Concepts
3. Unit Testing
4. Module Integration and Testing Strategies
5. Testing Approaches
1. White-Box Testing
2. Black-Box Testing
6. Other Types of Testing

9.5.2
 Black-Box Testing

requirements

output

input events

57
 Black Box Testing
 Black-box testing is conducted at the software interface to
demonstrate that correct inputs results in correct outputs.

 Testing software against a specification of its external behavior

without knowledge of internal implementation details

 It is not an alternative to White Box testing, but complements it.

 Focuses on the functional requirements.

 Attempts to find errors on

o Incorrect or missing functions
o Interface errors
o Errors in data structures or external database access
o Performance errors
o Initialization and termination errors
58
 Examples: Input Data
Valid data:
• user supplied commands
• responses to system prompts
• file names
• computational data
– physical parameters
– bounding values
– initiation values
• responses to error messages
• mouse picks

Invalid data:
• data outside bounds of the program
• physically impossible data
• proper value supplied in wrong place
59
 Examples of Input Domain
 Boolean value
o True or False

 Numeric value in a particular range

o 99 <= N <= 999
o Integer, Floating point

 One of a fixed set of enumerated values

o {Jan, Feb, Mar, …}
o {VisaCard, MasterCard, DiscoverCard, …}

 Formatted strings
o Phone numbers
o File names
o URLs
o Credit card numbers

60
 Equivalence Partitioning (1)
 First-level partitioning: Valid vs. Invalid input values

61
 Equivalence Partitioning (2)
 Partition valid and invalid values into equivalence classes
 Create a test case for at least one value from each equivalence
class

62
Equivalence Partitioning - Examples

63
Equivalence Partitioning - Examples

64
 Boundary Value Analysis (2)
 When choosing values to test, use the values that are most likely to
cause the program to fail

 If (200 < areaCode && areaCode < 999)

o Testing area codes 200 and 999 would catch the coding error

 In addition to testing center values, we should also test boundary values

o Right on a boundary
o Very close to a boundary on either side

65
 Boundary Value Analysis (3)
 Create test cases to test boundaries between equivalence
classes.

66
Boundary Value Analysis - Examples

67
Boundary Value Analysis - Examples

68
 Example:
Function Calculating Average

69
 Function Calculating Average
// The minimum value is excluded from average.
float Average (float scores [ ] , int length)
{
float min = 99999;
float total = 0;
for (int i = 0; i < length; i++)
{
if (scores[i] < min)
min = scores[i];

total += scores[i];
}
total = total – min;
return total / (length – 1);
}
70
Test Cases (Basis: Array Length)

71
Test Cases (Basis: Position of Minimum)

72
Test Cases (Basis: Number of Minima)

73
 Example:
Function Normalizing an Array

74
 Function Normalizing an Array
 The following C function is intended to normalize an array.

 Normalization Algorithm:
o First, the array of N elements is searched for the smallest and
largest non-negative elements.
o Secondly, the range is calculated as max - min.
o Finally, all non-negative elements that are bigger than the range are
normalized to 1, or else to 0.

 There are no syntax errors, but there may be logic errors in

the following implementation.

75
 C function
1. int normalize (int A[], int N) 19. range = max - min;
2. { 20. for (i = 0; i < N; i++)
3. int range, max, min, i, valid; 21. {
4. range = 0; 22. if (A[i] >= 0)
5. max = -1; 23. if (A[i] >= range)
6. min = -1; 24. A[i] = 1;
7. valid = 0; 25. else A[i] = 0;
8. for (i = 0; i < N; i++) 26. }
9. { 27. return valid;
10. if (A[i] >= 0) 28. }
11. {
12. if (A[i] > max)
13. max = A[i];
14. else if (A[i] < min)
15. min = A[i];
16. valid++;
17. }
18.}

76
 Tasks

 Draw the corresponding flow graph and calculate the

cyclomatic complexity V(g).

 Find linearly independent paths for basis path testing.

 Give test cases for boundary value analysis.

77
 Flow Graph

1-7 19

8-9
20-21
27-28

10-11 22

12 23

13 14 24 25

15 26

16-18

78
 Cyclomatic Complexity and Test Paths

Number of edges = E = 21
Number of nodes = N = 16
Cyclomatic complexity = V(g) = E – N + 2 = 21 – 16 + 2 = 7

Path1: 1-7, 8-9, 10-11, 12, 13, 16-18, 19, 20-21, 22, 23, 24, 26, 27-28
Path2: 1-7, 8-9, 10-11, 12, 14, 15, 16-18, 19, 20-21, 22, 23, 24, 26, 27-28
Path3: 1-7, 8-9, 10-11, 12, 14, 16-18, 19, 20-21, 22, 23, 24, 26, 27-28
Path4: 1-7, 8-9, 19, 20-21, 22, 23, 24, 26, 27-28
Path5: 1-7, 8-9, 10-11, 12, 13, 16-18, 19, 20-21, 22, 23, 25, 26, 27-28
Path6: 1-7, 8-9, 10-11, 12, 13, 16-18, 19, 20-21, 22, 26, 27-28
Path7: 1-7, 8-9, 10-11, 12, 13, 16-18, 19, 20-21, 27-28

79
 Test cases for boundary value analysis

Test Case-1)
Input Condition: N=0, A[ ] = {10, 20, 30, 40, 50}
Expected Output: Valid =0 (Due to invalid N value)

Test Case-2)
Input Condition: N=4, A[ ] = {10, 20, 30, 40, 50}
Expected Output: Valid =4 A[ ] = {0, 0, 1, 1, 50}

Test Case-3)
Input Condition: N=5, A[ ] = {10, 20, 30, 40, 50}
Expected Output: Valid =5, A[ ] = {0, 0, 0, 1, 1}
80
 Test cases for boundary value analysis
Test Case-4)
Input Condition: N=5, A[ ] = {-10, -20, -30, -40, -50}
Expected Output: Valid =0

Test Case-5)
Input Condition: N=5, A[ ] = {0, 0, 0, 0, 0}
Expected Output: Valid =5, A[ ] = {1, 1, 1, 1, 1}

Test Case-6)
Input Condition: N=5, A[ ] = {10, 10, 10, 10, 10}
Expected Output: Valid =5, A[ ] = {1, 1, 1, 1, 1}

81
1. Implementation/Programming Guidelines
2. Software Testing Concepts
3. Unit Testing
4. Module Integration and Testing Strategies
5. Testing Approaches
1. White-Box Testing
2. Black-Box Testing
6. Other Types of Testing

 9.6 
 Special Tests
• Regression testing
• Alpha test and beta test
• Performance testing
• Volume testing
• Stress testing
• Security testing
• Usability testing
• Recovery testing
• Testing web-based systems

83
 System Testing

1. Functional Testing
2. Performance Testing
3. Installation Testing

84
 An alternative form of black-box testing for classical software
o We base the test data on the functionality of the code artifacts

 Each item of functionality or function is identified

 Test data are devised to test each (lower-level) function

separately

 Then, higher-level functions composed of these lower-level

functions are tested
 Regression Testing
• Each time a new module is added as part of integration
testing, the software changes and needs to re-tested.

• Regression testing is the re-execution of some subset of

tests that have already been conducted to ensure that
changes have not propagated unintended side effects.

• In broader context, regression testing ensures that

changes (enhancement or correction) do not introduce
unintended new errors.
• When used?
1. After integration testing
2. Testing after module modification
86
 Performance Testing
 Measure the system's performance
o Running times of various tasks
o Memory usage, including memory leaks
o Network usage (Does it consume too much bandwidth? Does
it open too many connections?)
o Disk usage (Does it clean up temporary files properly?)
o Process/thread priorities (Does it run well with other
applications, or does it block the whole machine?)

87
 Volume Testing
 Volume testing: System behavior when handling large amounts of
data (e.g. large files)
• Test what happens if large amounts of data are handled

 Load testing: System behavior under increasing loads (e.g.

number of users)

 Test the system at the limits of normal use

o Test every limit on the program's behavior defined in the
requirements
o Maximum number of concurrent users or connections
o Maximum number of open files
o Maximum request or file size

88
 Stress Testing
 Stress testing: System behavior when it is overloaded
 Test the limits of system (maximum # of users, peak demands, extended
operation hours)

 Test the system under extreme conditions

o Create test cases that demand resources in abnormal quantity, frequency,
or volume
o Low memory
o Disk faults (read/write failures, full disk, file corruption, etc.)
o Network faults
o Power failure
o Unusually high number of requests
o Unusually large requests or files
o Unusually high data rates

 Even if the system doesn't need to work in such extreme conditions, stress
testing is an excellent way to find bugs

89
 Security Testing
 Try to violate security requirements
 Any system that manages sensitive information or performs
sensitive functions may become a target for illegal intrusion
 How easy is it to break into the system?
 Password usage
 Security against unauthorized access
 Protection of data
 Encryption

90
 The client determines whether the product satisfies its
specifications

 Acceptance testing is performed by

o The client organization, or
o The SQA team in the presence of client representatives, or
o An independent SQA team hired by the client

 The key difference between product testing and acceptance

testing is
o Acceptance testing is performed on actual data
o Product testing is preformed on test data, which can never be real, by
definition
 Alpha Test & Beta Test
Alpha Test
customer tests
software

developer site customer site

Beta Test

developer customer tests

reviews software

developer site
customer site

92
 Testing Web-based Systems
Concerns of Web-based Systems:
• Browser compatibility
• Functional correctness
• Usability
• Security
• Reliability
• Performance
• Recoverability

93
 Examples of Test Automation Tools

Vendor Tool
Hewlet Packard QuickTest
IBM Rational Functional Tester
Selenium Selenium
Microfocus SilkTest
AutomatedQA TestComplete

94
This week we present
 Good Software Implementation Principles
o Variable naming, commenting, debugging, etc.

 Low Level Testing

o Applying Unit Testing
o The concept of drivers and stubs
o How to integrate the system at overall

 Testing Strategies and Approaches

o White-box testing and black-box testing
o Input domain partitioning
o Higher Level Testing

1.95