Boundary Value Testing  ◾  85

5.5 Examples
Each of the three continuing examples is a function of three variables. Printing all the test cases
from all the methods for each problem is very space consuming, so we just have selected examples
for worst-case boundary value and robust worst-case boundary value testing.

5.5.1 Test Cases for the Triangle Problem

In the problem statement, no conditions are specified on the triangle sides, other than being
integers. Obviously, the lower bounds of the ranges are all 1. We arbitrarily take 200 as an upper
bound. For each side, the test values are {1, 2, 100, 199, 200}. Robust boundary value test cases
will add {0, 201}. Table 5.1 contains boundary value test cases using these ranges. Notice that test
cases 3, 8, and 13 are identical; two should be deleted. Further, there is no test case for scalene
triangles.
The cross-product of test values will have 125 test cases (some of which will be repeated)—too
many to list here. The full set is available as a spreadsheet in the set of student exercises. Table 5.2
only lists the first 25 worst-case boundary value test cases for the triangle problem. You can picture
them as a plane slice through the cube (actually it is a rectangular parallelepiped) in which a = 1
and the other two variables take on their full set of cross-product values.

Table 5.1  Normal Boundary Value Test Cases

Case a b c Expected Output

1 100 100 1 Isosceles

2 100 100 2 Isosceles

3 100 100 100 Equilateral

4 100 100 199 Isosceles

5 100 100 200 Not a triangle

6 100 1 100 Isosceles

7 100 2 100 Isosceles

8 100 100 100 Equilateral

9 100 199 100 Isosceles

10 100 200 100 Not a triangle

11 1 100 100 Isosceles

12 2 100 100 Isosceles

13 100 100 100 Equilateral

14 199 100 100 Isosceles

15 200 100 100 Not a triangle

© 2010 Taylor & Francis Group, LLC

86  ◾  Software Testing

Table 5.2  (Selected) Worst-Case Boundary Value Test Cases

Case a b c Expected Output

1 1 1 1 Equilateral

2 1 1 2 Not a triangle

3 1 1 100 Not a triangle

4 1 1 199 Not a triangle

5 1 1 200 Not a triangle

6 1 2 1 Not a triangle

7 1 2 2 Isosceles

8 1 2 100 Not a triangle

9 1 2 199 Not a triangle

10 1 2 200 Not a triangle

11 1 100 1 Not a triangle

12 1 100 2 Not a triangle

13 1 100 100 Isosceles

14 1 100 199 Not a triangle

15 1 100 200 Not a triangle

16 1 199 1 Not a triangle

17 1 199 2 Not a triangle

18 1 199 100 Not a triangle

19 1 199 199 Isosceles

20 1 199 200 Not a triangle

21 1 200 1 Not a triangle

22 1 200 2 Not a triangle

23 1 200 100 Not a triangle

24 1 200 199 Not a triangle

25 1 200 200 Isosceles

5.5.2 Test Cases for the NextDate Function

All 125 worst-case test cases for NextDate are listed in Table 5.3. Take some time to examine it
for gaps of untested functionality and for redundant testing. For example, would anyone actually
want to test January 1 in five different years? Is the end of February tested sufficiently?

© 2010 Taylor & Francis Group, LLC

 Boundary Value Testing  ◾  87

Table 5.3  Worst-Case Test Cases

Case Month Day Year Expected Output

1 1 1 1812 1, 2, 1812

2 1 1 1813 1, 2, 1813

3 1 1 1912 1, 2, 1912

4 1 1 2011 1, 2, 2011

5 1 1 2012 1, 2, 2012

6 1 2 1812 1, 3, 1812

7 1 2 1813 1, 3, 1813

8 1 2 1912 1, 3, 1912

9 1 2 2011 1, 3, 2011

10 1 2 2012 1, 3, 2012

11 1 15 1812 1, 16, 1812

12 1 15 1813 1, 16, 1813

13 1 15 1912 1, 16, 1912

14 1 15 2011 1, 16, 2011

15 1 15 2012 1, 16, 2012

16 1 30 1812 1, 31, 1812

17 1 30 1813 1, 31, 1813

18 1 30 1912 1, 31, 1912

19 1 30 2011 1, 31, 2011

20 1 30 2012 1, 31, 2012

21 1 31 1812 2, 1, 1812

22 1 31 1813 2, 1, 1813

23 1 31 1912 2, 1, 1912

24 1 31 2011 2, 1, 2011

25 1 31 2012 2, 1, 2012

26 2 1 1812 2, 2, 1812

27 2 1 1813 2, 2, 1813

28 2 1 1912 2, 2, 1912

(continued)

© 2010 Taylor & Francis Group, LLC

88  ◾  Software Testing

Table 5.3  Worst-Case Test Cases (Continued)

Case Month Day Year Expected Output

29 2 1 2011 2, 2, 2011

30 2 1 2012 2, 2, 2012

31 2 2 1812 2, 3, 1812

32 2 2 1813 2, 3, 1813

33 2 2 1912 2, 3, 1912

34 2 2 2011 2, 3, 2011

35 2 2 2012 2, 3, 2012

36 2 15 1812 2, 16, 1812

37 2 15 1813 2, 16, 1813

38 2 15 1912 2, 16, 1912

39 2 15 2011 2, 16, 2011

40 2 15 2012 2, 16, 2012

41 2 30 1812 Invalid date

42 2 30 1813 Invalid date

43 2 30 1912 Invalid date

44 2 30 2011 Invalid date

45 2 30 2012 Invalid date

46 2 31 1812 Invalid date

47 2 31 1813 Invalid date

48 2 31 1912 Invalid date

49 2 31 2011 Invalid date

50 2 31 2012 Invalid date

51 6 1 1812 6, 2, 1812

52 6 1 1813 6, 2, 1813

53 6 1 1912 6, 2, 1912

54 6 1 2011 6, 2, 2011

55 6 1 2012 6, 2, 2012

56 6 2 1812 6, 3, 1812

57 6 2 1813 6, 3, 1813
(continued)

© 2010 Taylor & Francis Group, LLC

 Boundary Value Testing  ◾  89

Table 5.3  Worst-Case Test Cases (Continued)

Case Month Day Year Expected Output

58 6 2 1912 6, 3, 1912

59 6 2 2011 6, 3, 2011

60 6 2 2012 6, 3, 2012

61 6 15 1812 6, 16, 1812

62 6 15 1813 6, 16, 1813

63 6 15 1912 6, 16, 1912

64 6 15 2011 6, 16, 2011

65 6 15 2012 6, 16, 2012

66 6 30 1812 7, 1, 1812

67 6 30 1813 7, 1, 1813

68 6 30 1912 7, 1, 1912

69 6 30 2011 7, 1, 2011

70 6 30 2012 7, 1, 2012

71 6 31 1812 Invalid date

72 6 31 1813 Invalid date

73 6 31 1912 Invalid date

74 6 31 2011 Invalid date

75 6 31 2012 Invalid date

76 11 1 1812 11, 2, 1812

77 11 1 1813 11, 2, 1813

78 11 1 1912 11, 2, 1912

79 11 1 2011 11, 2, 2011

80 11 1 2012 11, 2, 2012

81 11 2 1812 11, 3, 1812

82 11 2 1813 11, 3, 1813

83 11 2 1912 11, 3, 1912

84 11 2 2011 11, 3, 2011

85 11 2 2012 11, 3, 2012

86 11 15 1812 11, 16, 1812

(continued)

© 2010 Taylor & Francis Group, LLC

90  ◾  Software Testing

Table 5.3  Worst-Case Test Cases (Continued)

Case Month Day Year Expected Output

87 11 15 1813 11, 16, 1813

88 11 15 1912 11, 16, 1912

89 11 15 2011 11, 16, 2011

90 11 15 2012 11, 16, 2012

91 11 30 1812 12, 1, 1812

92 11 30 1813 12, 1, 1813

93 11 30 1912 12, 1, 1912

94 11 30 2011 12, 1, 2011

95 11 30 2012 12, 1, 2012

96 11 31 1812 Invalid date

97 11 31 1813 Invalid date

98 11 31 1912 Invalid date

99 11 31 2011 Invalid date

100 11 31 2012 Invalid date

101 12 1 1812 12, 2, 1812

102 12 1 1813 12, 2, 1813

103 12 1 1912 12, 2, 1912

104 12 1 2011 12, 2, 2011

105 12 1 2012 12, 2, 2012

106 12 2 1812 12, 3, 1812

107 12 2 1813 12, 3, 1813

108 12 2 1912 12, 3, 1912

109 12 2 2011 12, 3, 2011

110 12 2 2012 12, 3, 2012

111 12 15 1812 12, 16, 1812

112 12 15 1813 12, 16, 1813

113 12 15 1912 12, 16, 1912

114 12 15 2011 12, 16, 2011

115 12 15 2012 12, 16, 2012

(continued)

© 2010 Taylor & Francis Group, LLC

 Boundary Value Testing  ◾  91

Table 5.3  Worst-Case Test Cases (Continued)

Case Month Day Year Expected Output

116 12 30 1812 12, 31, 1812

117 12 30 1813 12, 31, 1813

118 12 30 1912 12, 31, 1912

119 12 30 2011 12, 31, 2011

120 12 30 2012 12, 31, 2012

121 12 31 1812 1, 1, 1813

122 12 31 1813 1, 1, 1814

123 12 31 1912 1, 1, 1913

124 12 31 2011 1, 1, 2012

125 12 31 2012 1, 1, 2013

5.5.3 Test Cases for the Commission Problem

Instead of going through 125 boring test cases again, we will look at some more interesting test
cases for the commission problem. This time, we will look at boundary values derived from the
output range, especially near the threshold points of $1000 and $1800 where the commission
percentage changes. The output space of the commission is shown in Figure 5.6. The intercepts of
these threshold planes with the axes are shown.

Barrels

90

72

40

22.2 40 70
Locks
33.3

60

80

Stocks

Figure 5.6  Input space of the commission problem.

© 2010 Taylor & Francis Group, LLC

92  ◾  Software Testing

Table 5.4  Output Boundary Value Analysis Test Cases

Case Locks Stocks Barrels Sales Comm Comment

1 1 1 1 100 10 Output minimum

2 1 1 2 125 12.5 Output minimum +

3 1 2 1 130 13 Output minimum +

4 2 1 1 145 14.5 Output minimum +

5 5 5 5 500 50 Midpoint

6 10 10 9 975 97.5 Border point –

7 10 9 10 970 97 Border point –

8 9 10 10 955 95.5 Border point –

9 10 10 10 1000 100 Border point

10 10 10 11 1025 103.75 Border point +

11 10 11 10 1030 104.5 Border point +

12 11 10 10 1045 106.75 Border point +

13 14 14 14 1400 160 Midpoint

14 18 18 17 1775 216.25 Border point –

15 18 17 18 1770 215.5 Border point –

16 17 18 18 1755 213.25 Border point –

17 18 18 18 1800 220 Border point

18 18 18 19 1825 225 Border point +

19 18 19 18 1830 226 Border point +

20 19 18 18 1845 229 Border point +

21 48 48 48 4800 820 Midpoint

22 70 80 89 7775 1415 Output maximum –

23 70 79 90 7770 1414 Output maximum –

24 69 80 90 7755 1411 Output maximum –

25 70 80 90 7800 1420 Output maximum

The volume between the origin and the lower plane corresponds to sales below the $1000
threshold. The volume between the two planes is the 15% commission range. Part of the reason
for using the output range to determine test cases is that cases from the input range are almost all
in the 20% zone. We want to find input variable combinations that stress the sales/commission
boundary values: $100, $1000, $1800, and $7800. The minimum and maximum were easy, and

© 2010 Taylor & Francis Group, LLC

 Boundary Value Testing  ◾  93

Table 5.5  Output Special Value Test Cases

Case Locks Stocks Barrels Sales Comm Comment

1 10 11 9 1005 100.75 Border point +

2 18 17 19 1795 219.25 Border point –

3 18 19 17 1805 221 Border point +

the numbers happen to work out so that the border points are easy to generate. Here is where it
gets interesting: test case 9 is the $1000 border point. If we tweak the input variables, we get values
just below and just above the border (cases 6–8 and 10–12). If we wanted to, we could pick values
near the borders such as (22, 1, 1). As we continue in this way, we have a sense that we are “exercis-
ing” interesting parts of the code. We might claim that this is really a form of special value testing
because we used our mathematical insight to generate test cases.
Table 5.4 contains test cases derived from boundary values on the output side of the commis-
sion function. Table 5.5 contains special value test cases.

5.6 Random Testing
At least two decades of discussion of random testing are included in the literature. Most of this
interest is among academics, and in a statistical sense, it is interesting. Our three sample problems
lend themselves nicely to random testing. The basic idea is that, rather than always choose the min,
min+, nom, max–, and max values of a bounded variable, use a random number generator to pick
test case values. This avoids any form of bias in testing. It also raises a serious question: how many
random test cases are sufficient? Later, when we discuss structural test coverage metrics, we will
have an elegant answer. For now, Tables 5.6 through 5.8 show the results of randomly generated
test cases. They are derived from a Visual Basic application that picks values for a bounded variable
a ≤ x ≤ b as follows:

Table 5.6 Random Test Cases for Triangle Program

Test Cases Nontriangles Scalene Isosceles Equilateral

1289 663 593 32 1

15,436 7696 7372 367 1

17,091 8556 8164 367 1

2603 1284 1252 66 1

6475 3197 3122 155 1

5978 2998 2850 129 1

9008 4447 4353 207 1

Percentage 49.83% 47.87% 2.29% 0.01%

© 2010 Taylor & Francis Group, LLC

94  ◾  Software Testing

Table 5.7 Random Test Cases for Commission

Program
Test Cases 10% 15% 20%

91 1 6 84

27 1 1 25

72 1 1 70

176 1 6 169

48 1 1 46

152 1 6 145

125 1 4 120

Percentage 1.01% 3.62% 95.37%

x = Int((b – a + 1) * Rnd + a)

where the function Int returns the integer part of a floating point number, and the function Rnd
generates random numbers in the interval [0, 1]. The program keeps generating random test cases
until at least one of each output occurs. In each table, the program went through seven “cycles”
that ended with the “hard-to-generate” test case. In Tables 5.6 and 5.7, the last line shows what
percentage of the random test cases was generated for each column. In the table for NextDate, the
percentages are very close to the computed probability given in the last line of Table 5.8.

5.7 Guidelines for Boundary Value Testing

With the exception of special value testing, the test methods based on the input domain of a function
(program) are the most rudimentary of all specification-based testing methods. They share the com-
mon assumption that the input variables are truly independent; and when this assumption is not war-
ranted, the methods generate unsatisfactory test cases (such as June 31, 1912, for NextDate). Each of
these methods can be applied to the output range of a program, as we did for the commission problem.
Another useful form of output-based test cases is for systems that generate error messages. The
tester should devise test cases to check that error messages are generated when they are appropriate,
and are not falsely generated. Boundary value analysis can also be used for internal variables, such
as loop control variables, indices, and pointers. Strictly speaking, these are not input variables;
however, errors in the use of these variables are quite common. Robustness testing is a good choice
for testing internal variables.
There is a discussion in Chapter 10 about “the testing pendulum”—it refers to the problem of
syntactic versus semantic approaches to developing test cases. Here is a short example given both
ways. Consider a function F of three variables, a, b, and c. The boundaries are 0 ≤ a < 10,000, 0 ≤
b < 10,000, and 0 ≤ c < 18.8. The function F is F = (a – b)/c; Table 5.9 shows the normal boundary
value test cases. Absent semantic knowledge, the first four test cases in Table 5.9 are what a boundary
value testing tool would generate (a tool would not generate the expected output values). Even just
the syntactic version is problematic—it does not avoid the division by zero possibility in test case 11.

© 2010 Taylor & Francis Group, LLC

 Boundary Value Testing  ◾  95

Table 5.8 Random Test Cases for NextDate Program

Days 1–30 of Day 31 of Days 1–29 of Day 30 of
31-Day 31-Day 30-Day 30-Day
Test Cases Months Months Months Months

913 542 17 274 10

1101 621 9 358 8

4201 2448 64 1242 46

1097 600 21 350 9

5853 3342 100 1804 82

3959 2195 73 1252 42

1436 786 22 456 13

Percentage 56.76% 1.65% 30.91% 1.13%

Probability 56.45% 1.88% 31.18% 1.88%

Feb. 28 of a
Days 1–27 Feb. 28 of a Non-Leap Feb. 29 of a Impossible
of Feb. Leap Year Year Leap Year Days

45 1 1 1 22

83 1 1 1 19

312 1 8 3 77

92 1 4 1 19

417 1 11 2 94

310 1 6 5 75

126 1 5 1 26

7.46% 0.04% 0.19% 0.08% 1.79%

7.26% 0.07% 0.20% 0.07% 1.01%

When we add the semantic information that F calculates the miles per gallon of an automo-
bile, where a and b are end and start trip odometer values, and c is the gas tank capacity, we see
more severe problems:

1. We must always have a ≥ b. This will avoid the negative values of F (test cases 1, 2, 9, and 10).
2. Test cases 3, 8, and 12–15 all refer to trips of length 0, so they could be collapsed into one
test case, probably test case 8.
3. Division by zero is an obvious problem, thereby eliminating test case 11. Applying the
semantic knowledge will result in the better set of case cases in Table 5.10.
4. Table 5.10 is still problematic—we never see the effect of boundary values on the tank capacity.

© 2010 Taylor & Francis Group, LLC

96  ◾  Software Testing

Table 5.9  Normal Boundary Value Test Cases for F = (a – b)/c

Test Case a b c F

1 0 5000 9.4 –531.9

2 1 5000 9.4 –531.8

3 5000 5000 9.4 0.0

4 9998 5000 9.4 531.7

5 9999 5000 9.4 531.8

6 5000 0 9.4 531.9

7 5000 1 9.4 531.8

8 5000 5000 9.4 0.0

9 5000 9998 9.4 –531.7

10 5000 9999 9.4 –531.8

11 5000 5000 0 Undefined

12 5000 5000 1 0.0

13 5000 5000 9.4 0.0

14 5000 5000 18.7 0.0

15 5000 5000 18.8 0.0

Table 5.10  Semantic Boundary Value Test Cases for F = (a – b)/c

Test Case End Odometer Start Odometer Tank Capacity Miles per Gallon

4 9998 5000 9.4 531.7

5 9999 5000 9.4 531.8

6 5000 0 9.4 531.9

7 5000 1 9.4 531.8

8 5000 5000 9.4 0.0

EXERCISES
1. Develop a formula for the number of robustness test cases for a function of n variables.
2. Develop a formula for the number of robust worst-case test cases for a function of n variables.
3. Make a Venn diagram showing the relationships among test cases from boundary value
analysis, robustness testing, worst-case testing, and robust worst-case testing.
4. What happens if we try to do output range robustness testing? Use the commission problem
as an example.

© 2010 Taylor & Francis Group, LLC

 Boundary Value Testing  ◾  97

5. If you did exercise 8 in Chapter 2, you are already familiar with the CRC Press website for
downloads (http://www.crcpress.com/product/isbn/9781466560680). There you will find
an Excel spreadsheet named specBasedTesting.xls. (It is an extended version of Naive.xls,
and it contains the same inserted faults.) Different sheets contain worst-case boundary value
test cases for the triangle, NextDate, and commission problems, respectively. Run these sets
of test cases and compare the results with your naive testing from Chapter 2.
6. Apply special value testing to the miles per gallon example in Tables 5.9 and 5.10. Provide
reasons for your chosen test cases.

© 2010 Taylor & Francis Group, LLC