CHAPTER 2

RANDOM NUMBER GENERATION

The material in this chapter presumes some knowledge of integer arithmetic and
elementary number theory; for background material see Appendix B.
Sections
2.1. Lehmer Random Number Generation: Introduction . . . . . . . . . . 38
2.2. Lehmer Random Number Generation: Implementation (library rng) . . 48
2.3. Monte Carlo Simulation (programs galileo and buffon) . . . . . . . 61
2.4. Monte Carlo Simulation Examples (programs det, craps, hat and san) . 74
2.5. Finite-State Sequences . . . . . . . . . . . . . . . . . . . . . . 88

Discrete-event and Monte Carlo simulation cannot be done correctly without access
to a good random number generator. Ideally, computer users should be able to assume the
existence of a good system-supplied generator that can be used as a “black box” like other
standard, reliable mathematical functions (sqrt, sin, exp, etc.). Unfortunately, history
suggests this is frequently a false assumption. For that reason, the first two sections of
this chapter provide a comprehensive discussion of an easily understood random number
generation algorithm that can be used with confidence.
Section 2.1 is introductory, beginning with a conceptual model of the two-parameter
algorithm as equivalent to drawing, at random, from an urn whose contents are determined
by the choice of parameters. Given an appropriate choice of parameter values, Section 2.2
is primarily concerned with the important issue of correct algorithm implementation. That
is, a software implementation of the random number generation algorithm is developed that
is correct, efficient, and portable to essentially any contemporary computing system. This
software implementation is then used in Section 2.3 as the basis for introducing Monte
Carlo simulation as it relates to the estimation of probabilities.
Section 2.4 provides Monte Carlo examples that are more involved and interesting
than those in the third section. Unlike the material in the first two sections, which is
specific to a particular random number generation algorithm, Section 2.5 provides a more
general discussion of random number generation algorithms.
 38 2.1 Lehmer Random Number Generators: Introduction
Programs ssq1 and sis1 both require input data from an external source. Because
of this, the usefulness of these programs is limited by the amount of available input data.
What if more input data is needed? Or, what if the model is changed; can the input data
be modified accordingly? Or, what can be done if only a small amount of input data is
available, or perhaps, none at all?
In each case the answer is to use a random number generator. By convention, each
call to the random number generator will produce a real-valued result between 0.0 and 1.0
that becomes the inherent source of randomness for a discrete-event simulation model. As
illustrated later in this chapter, consistent with user-defined stochastic models of arrival
times, service times, demand amounts, etc., the random number generator’s output can
be converted to a random variate via an appropriate mathematical transformation. The
random variates are used to approximate some probabilistic element (e.g., service times)
of a real-world system for a discrete-event simulation model.

2.1.1 RANDOM NUMBER GENERATION

Historically three types of random number generators have been advocated for com-
putational applications: (a) 1950’s-style table look-up generators like, for example, the
RAND corporation table of a million random digits; (b) hardware generators like, for ex-
ample, thermal “white noise” devices; and (c) algorithmic (software) generators. Of these
three types, only algorithmic generators have achieved widespread acceptance. The reason
for this is that only algorithmic generators have the potential to satisfy all of the following
generally well-accepted random number generation criteria. A generator should be:
• random — able to produce output that passes all reasonable statistical tests of ran-
domness;
• controllable — able to reproduce its output, if desired;
• portable — able to produce the same output on a wide variety of computer systems;
• efficient — fast, with minimal computer resource requirements;
• documented — theoretically analyzed and extensively tested.
In addition, as discussed and illustrated in Chapter 3, it should be easy to partition the
generator’s output into multiple “streams”.

Definition 2.1.1 An ideal random number generator is a function, say Random, with
the property that each assignment
u = Random();
will produce a real-valued (floating-point) number u between 0.0 and 1.0 in such a way
that any value in the interval 0.0 < u < 1.0 is equally likely to occur. A good random
number generator produces results that are statistically indistinguishable, for all practical
purposes, from those produced by an ideal generator.
 2.1 Lehmer Random Number Generators: Introduction 39

What we will do in this chapter is construct, from first principles, a good random
number generator that will satisfy all the criteria listed previously. We begin with the
following conceptual model.
• Choose a large positive integer m. This defines the set X m = {1, 2, . . . , m − 1}.
• Fill a (conceptual) urn with the elements of X m .
• Each time a random number u is needed, draw an integer x “at random” from the
urn and let u = x/m.
Each draw simulates a realization (observation, sample) of an independent identically dis-
tributed (iid ) sequence of so-called Uniform(0, 1) random variables. Because the possible
values of u are 1/m, 2/m, . . ., 1 − 1/m, it is important for m to be so large that the
possible values of u will be densely distributed between 0.0 and 1.0. Note that the values
u = 0.0 and u = 1.0 are impossible. As discussed in later chapters, excluding these two
extreme values is important for avoiding problems associated with certain random variate
generation algorithms.
Ideally we would like to draw from the urn independently and with replacement. If so
then each of the m − 1 possible values of u would be equally likely to be selected on each
draw. For practical reasons, however, we will use a random number generation algorithm
that simulates drawing from the urn without replacement. Fortunately, if m is large and
the number of draws is small relative to m then the distinction between drawing with
and without replacement is largely irrelevant. To turn the conceptual urn model into an
specification model we will use a time-tested algorithm suggested by Lehmer (1951).

2.1.2 LEHMER’S ALGORITHM

Definition 2.1.2 Lehmer’s algorithm for random number generation is defined in terms
of two fixed parameters
• modulus m, a fixed large prime integer
• multiplier a, a fixed integer in Xm
and the subsequent generation of the integer sequence x 0 , x1 , x2 , . . . via the iterative
equation
• xi+1 = g(xi ) i = 0, 1, 2, . . .
where the function g(·) is defined for all x ∈ X m = {1, 2, . . . , m − 1} as
• g(x) = ax mod m
and the initial seed x0 is chosen from the set Xm . The modulus function mod gives the
remainder when the first argument (ax in this case) is divided by the second argument (the
modulus m in this case). It is defined more carefully in Appendix B. A random number
generator based on Lehmer’s algorithm is called a Lehmer generator.
 40 2. Random Number Generation

Because of the mod (remainder) operator, the value of g(x) is always an integer
between 0 and m − 1; however, it is important to exclude 0 as a possible value. That is,
g(0) = 0 and so if 0 ever occurs in the sequence x 0 , x1 , x2 , . . . then all the terms in the
sequence will be 0 from that point on. Fortunately, if (a, m) is chosen consistent with
Definition 2.1.2, it can be shown that g(x) �= 0 for all x ∈ X m so that g : Xm → Xm .
This guaranteed avoidance of 0 is part of the reason for choosing m to be prime. Because
g : Xm → Xm , it follows (by induction) that if x0 ∈ Xm then xi ∈ Xm for all i = 0, 1, 2, . . .
Lehmer’s algorithm represents a good example of the elegance of simplicity. The
genius of Lehmer’s algorithm is that if the multiplier a and prime modulus m are properly
chosen, the resulting sequence x 0 , x1 , x2 , . . . , xm−2 will be statistically indistinguishable
from a sequence drawn at random, albeit without replacement, from X m . Indeed, it is only
in the sense of simulating this random draw that the algorithm is random; there is actually
nothing random about Lehmer’s algorithm, except possibly the choice of the initial seed.
For this reason Lehmer generators are sometimes called pseudo-random.*
An intuitive explanation of why Lehmer’s algorithm simulates randomness is based on
the observation that if two large integers, a and x, are multiplied and the product divided
by another large integer, m, then the remainder, g(x) = ax mod m, is “likely” to take on
any value between 0 and m − 1, as illustrated in Figure 2.1.1.
Figure 2.1.1. �ax/m� m g(x)
Lehmer |←−−−−−−−−−−−−−−−−−− −−−−−−−−−−−−−−−−−−→|←− −→|
generator |←−−−−−−−−−−−−−−−−−−−−− ax −−−−−−−−−−−−−−−−−−−−−→|
geometry. • • •
0 x a m 2m 3m 4m 5m ax
This is somewhat like going to the grocery store with no change, only dollars, and buying
many identical items. After you pay for the items, the change you will receive is likely to
be any value between 0/c and 99/ c. [That is, a is the price of the item in cents, x is the
number of items purchased, and m = 100. The analogy is not exact, however, because the
change is m − g(x), not g(x)].
Modulus and Multiplier Considerations
To construct a Lehmer generator, standard practice is to choose the large prime mod-
ulus m first, then choose the multiplier a. The choice of m is dictated, in part, by system
considerations. For example, on a computer system that supports 32-bit, 2’s complement
integer arithmetic, m = 231 − 1 is a natural choice because it is the largest possible positive
integer and it happens to be prime. (For 16-bit integer arithmetic we are not so fortunate,
however, because 215 − 1 is not prime. See Exercise 2.1.6. Similarly, for 64-bit integer
arithmetic 263 − 1 is not prime. See Exercise 2.1.10.) Given m, the subsequent choice of
a must be made with great care. The following example is an illustration.

* Lehmer generators are also known as (take a deep breath) prime modulus multiplica-
tive linear congruential pseudo-random number generators, abbreviated PMMLCG.
2.1 Lehmer Random Number Generators: Introduction 41

Example 2.1.1 As a tiny example, consider the prime modulus m = 13.*

• If the multiplier is a = 6 and the initial seed is x 0 = 1 then the resulting sequence of
x’s is
1, 6, 10, 8, 9, 2, 12, 7, 3, 5, 4, 11, 1, . . .
where, as the ellipses indicate, the sequence is actually periodic because it begins to
cycle (with a full period of length m − 1 = 12) when the initial seed reappears. The
point is that any 12 consecutive terms in this sequence appear to have been drawn at
random, without replacement, from the set X 13 = {1, 2, . . . , 12}.
• If the multiplier is a = 7 and the initial seed is x 0 = 1 then the resulting full-period
sequence of x’s is
1, 7, 10, 5, 9, 11, 12, 6, 3, 8, 4, 2, 1, . . .
Randomness is, like beauty, only in the eye of the beholder. Because of the 12, 6, 3
and 8, 4, 2, 1 patterns, however, most people would consider this second sequence to
be “less random” than the first.
• If the multiplier is a = 5 then either

1, 5, 12, 8, 1, . . . or 2, 10, 11, 3, 2, . . . or 4, 7, 9, 6, 4, . . .

will be produced, depending on the initial seed x 0 = 1, 2 or 4. This type of less-than-

full-period behavior is clearly undesirable because, in terms of our conceptual model of
random number generation, this behavior corresponds to first partitioning the set X m
into several disjoint subsets (urns), then selecting one subset and thereafter drawing
exclusively from it.
Example 2.1.1 illustrates two of the three central issues that must be resolved when
choosing (a, m).
• Does the function g(·) generate a full-period sequence?
• If a full-period sequence is generated, how random does the sequence appear to be?
The third central issue is implementation.
• Can the ax mod m operation be evaluated efficiently and correctly for all x ∈ X m ?
For Example 2.1.1 the issue of implementation is trivial. However, for realistically large
values of a and m, a portable, efficient implementation in a high-level language is a sub-
stantive issue because of potential integer overflow associated with the ax product. We will
return to this implementation issue in the next section. In the remainder of this section
we will concentrate on the full-period issue.

* For a hypothetical computer system with 5-bit 2’s complement integer arithmetic, the
largest positive integer would be 24 − 1 = 15. In this case m = 13 would be the largest
possible prime and, in that sense, a natural choice for the modulus.
42 2. Random Number Generation

Full Period Considerations

From the definition of the mod function (see Appendix B), if x i+1 = g(xi ) then there
exists a non-negative integer ci = �axi /m� such that

xi+1 = g(xi ) = axi mod m = axi − mci i = 0, 1, 2, . . .

Therefore (by induction)

x1 = ax0 − mc0
x2 = ax1 − mc1 = a2 x0 − m(ac0 + c1 )
x3 = ax2 − mc2 = a3 x0 − m(a2 c0 + ac1 + c2 )
..
.
xi = axi−1 − mci−1 = ai x0 − m(ai−1 c0 + ai−2 c1 + · · · + ci−1 ).

Because xi ∈ Xm we have xi = xi mod m. Moreover, (ai x0 − mc) mod m = ai x0 mod m

independent of the value of the integer c = a i−1 c0 + ai−2 c1 + · · · + ci−1 . Therefore, we
have proven the following theorem.

Theorem 2.1.1 If the sequence x0 , x1 , x2 , . . . is produced by a Lehmer generator with

multiplier a and modulus m then

xi = ai x0 mod m i = 0, 1, 2, . . .

Although it would be an eminently bad idea to compute x i by first computing ai ,

particularly if i is large, Theorem 2.1.1 has significant theoretical importance due, in part,
to the fact that xi can be written equivalently as

xi = ai x0 mod m = [(ai mod m) (x0 mod m)] mod m = [(ai mod m) x0 ] mod m

for i = 0, 1, 2, . . . via Theorem B.2 in Appendix B. In particular, this is true for i =

m − 1. Therefore, because m is prime and a mod m �= 0, from Fermat’s little theorem (see
Appendix B) it follows that am−1 mod m = 1 and so xm−1 = x0 . This observation is the
key to proving the following theorem. The details of the proof are left as an exercise.

Theorem 2.1.2 If x0 ∈ Xm and the sequence x0 , x1 , x2 , . . . is produced by a Lehmer

generator with multiplier a and (prime) modulus m then there is a positive integer p with
p ≤ m − 1 such that x0 , x1 , x2 , . . . , xp−1 are all different and

xi+p = xi i = 0, 1, 2, . . .

That is, the sequence x0 , x1 , x2 , . . . is periodic with fundamental period p. In addition,

(m − 1) mod p = 0.
2.1 Lehmer Random Number Generators: Introduction 43

The significance of Theorem 2.1.2 is profound. If we pick any initial seed x 0 ∈ Xm

and start to generate the sequence x 0 , x1 , x2 , . . . then we are guaranteed that the initial
seed will reappear. In addition, the first instance of reappearance is guaranteed to occur
at some index p that is either m − 1 or an integer divisor of m − 1 and from that index on
the sequence will cycle through the same p distinct values as illustrated

x0 , x1 , . . . , xp−1 , x0 , x1 , . . . , xp−1 , x0 , . . .
� �� � � �� �
period period

Consistent with Definition 2.1.3, we are interested in choosing multipliers for which the
fundamental period is p = m − 1.
Full Period Multipliers

Definition 2.1.3 The sequence x0 , x1 , x2 , . . . produced by a Lehmer generator with

modulus m and multiplier a has a full period if and only if the fundamental period p is
m − 1. If the sequence has a full period then a is said to be a full-period multiplier relative
to m.*
If the sequence x0 , x1 , x2 , . . . has a full period then any m − 1 consecutive terms in
the sequence have been drawn without replacement from the set X m . In effect a full-period
Lehmer generator creates a virtual circular list with m − 1 distinct elements. The initial
seed provides a starting list element; subsequent calls to the generator traverse the list.

Example 2.1.2 From Example 2.1.1, if m = 13 then a = 6 and a = 7 are full-period

multipliers (p = 12) and a = 5 is not (p = 4). The virtual circular lists (with traversal
in a clockwise direction) corresponding to these two full-period multipliers are shown in
Figure 2.1.2.
1 1
. . .•. . . . . . . . . .•. . . . .
...
11•. . . . . .. 6
..
•
2•. . . . . . 7
•. . Figure 2.1.2.
.
.. .. . .

.
.
.
. .
.
. .
.
. .
.
.
Two full
. . .
4 .•.. .
. 10
•. 4 •. .
.
. 10
•.
period
.
.
. .
.
. .
. .
. generators.
. . .
. . .
. .
. . . .
. . .
. .
5
.
•.
.
(a, m) = (6, 13) •.. 8 8 •.. (a, m) = (7, 13) •.. 5
. . . .
. . . .
. . . .
. . .
. . .
. . . .
. . . .
•.. . •.
.•
3 .
. .
. 9 3 •. . . .
. 9
. . . .
. . . .
. . . .
.. . . .
.
.•. ..
.•
.•. .
.. .. . . . .•
7 ...
. . . •. . . . . . 2 6 . . . .
•. . . .
. . 11
12 12
* A full-period multiplier a relative to m is also said to be a primitive root of m.
44 2. Random Number Generation

Algorithm 2.1.1 This algorithm, based upon Theorem 2.1.2, can be used to determine
if a is a full-period multiplier relative to the prime modulus m.
p = 1;
x = a;
while (x != 1) {
p++;
x = (a * x) % m; /* beware of a * x overflow */
}
if (p == m - 1)
/* a is a full-period multiplier */
else
/* a is not a full-period multiplier */
If m is not prime Algorithm 2.1.1 may not halt. It provides a slow-but-sure O(m)*
test for a full-period multiplier. The algorithm uses x 0 = 1 as an arbitrary initial seed
(note that x1 = a) and the recursive generation of new values of x until the initial seed
reappears. Before starting to search for full-period multipliers, however, it would be good
to know that they exist and with what frequency. That is, given a prime modulus m, how
many corresponding full-period multipliers are there? The following theorem provides the
answer to this question.

Theorem 2.1.3 If m is prime and p1 , p2 , . . . , pr are the (unique) prime factors of m − 1

then the number of full-period multipliers in X m is
(p1 − 1)(p2 − 1) . . . (pr − 1)
(m − 1).
p1 p2 . . . pr

Example 2.1.3 If m = 13 then m − 1 = 22 · 3. From the equation in Theorem 2.1.3 this

prime modulus has (2−1)(3−1)
2·3 (13 − 1) = 4 full-period multipliers: a = 2, 6, 7, and 11.

Example 2.1.4 The Lehmer generator used in this book has the (Mersenne, i.e., of the
form 2k − 1, where k is a positive integer) prime modulus m = 2 31 − 1 = 2 147 483 647.
Because the prime decomposition of m − 1 is
m − 1 = 231 − 2 = 2 · 32 · 7 · 11 · 31 · 151 · 331
from the equation in Theorem 2.1.3 the number of full-period multipliers is
� �
1 · 2 · 6 · 10 · 30 · 150 · 330
(2 · 32 · 7 · 11 · 31 · 151 · 331) = 534 600 000.
2 · 3 · 7 · 11 · 31 · 151 · 331
Therefore, for this prime modulus approximately 25% of the multipliers between 1 and
m − 1 are full-period multipliers.
* A function f is called “order m”, for example, written O(m), if there exist real positive
constants c1 and c2 independent of m such that c1 m ≤ f (m) ≤ c2 m.
2.1 Lehmer Random Number Generators: Introduction 45

Using Algorithm 2.1.1 (and a lot of computer time) it can be shown that if m = 2 31 −1
then a = 2, 3, 4, 5, 6 are not full-period multipliers but that a = 7 is. Remarkably, once one
full-period multiplier has been found, in this case a = 7, then all the others can be found
using the following O(m) algorithm. This algorithm presumes the availability of a function
gcd that returns the greatest common divisor of two positive integers (see Appendix B).

Algorithm 2.1.2 Given the prime modulus m and any full-period multiplier a, the
following algorithm generates all the full-period multipliers relative to m.*
i = 1;
x = a;
while (x != 1) {
if (gcd(i, m − 1) == 1)
/* x is a full-period multiplier equal to ai mod m */
i++;
x = (a * x) % m; /* beware of a * x overflow */
}
Algorithm 2.1.2 is based on Theorem 2.1.4 which establishes a one-to-one correspon-
dence between integers i ∈ Xm that are relatively prime to m−1 and full-period multipliers
ai mod m ∈ Xm . That is, the equation in Theorem 2.1.3 counts both the number of full-
period multipliers and the number of integers in X m that are relatively prime to m − 1.
The proof of Theorem 2.1.4 is left as an (advanced) exercise.

Theorem 2.1.4 If a is any full-period multiplier relative to the prime modulus m then
each of the integers
ai mod m ∈ Xm i = 1, 2, 3, . . . , m − 1
is also a full-period multiplier relative to m if and only if the integer i has no prime factors
in common with the prime factors of m − 1, i.e., i and m − 1 are relatively prime (see
Appendix B).

Example 2.1.5 If m = 13 then from Example 2.1.3 there are 4 integers between 1 and
12 that are relatively prime to 12. They are i = 1, 5, 7, 11. From Example 2.1.1, a = 6
is a full-period multiplier relative to 13; from Theorem 2.1.4 the 4 full-period multipliers
relative to 13 are therefore
61 mod 13 = 6, 65 mod 13 = 2, 67 mod 13 = 7, 611 mod 13 = 11.
Equivalently, if we had known that a = 2 is a full-period multiplier relative to 13 we could
have used Algorithm 2.1.2 with a = 2 to determine the full-period multipliers as
21 mod 13 = 2, 25 mod 13 = 6, 27 mod 13 = 11, 211 mod 13 = 7.

* See Exercises 2.1.4 and 2.1.5.

46 2. Random Number Generation

Example 2.1.6 If m = 231 − 1 then from Example 2.1.4 there are 534600000 integers
between 1 and m − 1 that are relatively prime to m − 1. The first few of these are
i = 1, 5, 13, 17, 19. As discussed previously, a = 7 is a full-period multiplier relative to this
modulus and so from Algorithm 2.1.2
71 mod 2147483647 = 7
75 mod 2147483647 = 16807
713 mod 2147483647 = 252246292
717 mod 2147483647 = 52958638
719 mod 2147483647 = 447489615
are full-period multipliers relative to 2 31 − 1 = 2147483647. Note that the full-period
multiplier 16807 is a logical choice; 7 is the smallest full-period multiplier and 5 is the
smallest integer (other than 1) that is relatively prime to m − 1. However, that the
multiplier 16807 is a logical choice does not mean that the resulting sequence it generates
is necessarily “random”. We will have more to say about this in the next section.
Standard Algorithmic Generators
Most of the standard algorithmic generators in use today are one of the following three
linear congruential types (Law and Kelton, 2000, pages 406–412):
• mixed : g(x) = (ax + c) mod m with m = 2b (b = 31 typically), a mod 4 = 1 and
c mod 2 = 1. All integer values of x ∈ {0} ∪ X m are possible and the period is m.
• multiplicative with m = 2b : g(x) = ax mod m with m = 2b (b = 31 typically) and
a mod 8 = 3 or a mod 8 = 5. To achieve the maximum period, which is only m/4, x
must be restricted to the odd integers in X m .
• multiplicative with m prime: g(x) = ax mod m with m prime and a a full-period
multiplier. All integer values of x ∈ X m = {1, 2, . . . , m − 1} are possible and the
period is m − 1.
Of these three, specialists generally consider the third generator, which has been presented
in this chapter, to be best. Random number generation remains an area of active research,
however, and reasonable people will probably always disagree about the best algorithm
for random number generation. Section 2.5 contains a more general discussion of random
number generators.
For now we will avoid the temptation to be drawn further into the “what is the best
possible random number generator” debate. Instead, as discussed in the next section, we
will use a Lehmer generator with modulus m = 2 31 − 1 and a corresponding full-period
multiplier a = 48271 carefully selected to provide generally acceptable randomness and
facilitate efficient software implementation. With this generator in hand, we can then turn
our attention to the main topic of this book — the modeling, simulation and analysis of
discrete-event stochastic systems.
 2.1 Lehmer Random Number Generators: Introduction 47

2.1.3 EXERCISES

Exercise 2.1.1 For the tiny Lehmer generator defined by g(x) = ax mod 127, find all
the full-period multipliers. (a) How many are there? (b) What is the smallest multiplier?

Exercise 2.1.2 Prove Theorem 2.1.2.

Exercise 2.1.3 Prove that if (a, m) are chosen consistent with Definition 2.1.2, then
g : Xm → Xm is a bijection (one-to-one and onto).

Exercise 2.1.4a (a) Prove that if (a, m) are chosen consistent with Definition 2.1.2, then
g(x) �= 0 for all x ∈ Xm and so g : Xm → Xm . (b) In addition, relative to this definition,
prove that there is nothing to be gained by considering integer multipliers outside of X m .
Consider the two cases a ≥ m and a ≤ 0.

Exercise 2.1.5 (a) Except for the special case m = 2, prove that a = 1 cannot be a
full-period multiplier. (b) What about a = m − 1?

Exercise 2.1.6 In ANSI C an int is guaranteed to hold all integer values between
−(215 − 1) and 215 − 1 inclusive. (a) What is the largest prime modulus in this range?
(b) How many corresponding full-period multipliers are there and what is the smallest one?

Exercise 2.1.7a Prove Theorem 2.1.4.

Exercise 2.1.8 (a) Evaluate 7i mod 13 and 11i mod 13 for i = 1, 5, 7, 11. (b) How does
this relate to Example 2.1.5?

Exercise 2.1.9 (a) Verify that the list of five full-period multipliers in Example 2.1.6 is
correct. (b) What are the next five elements in this list?

Exercise 2.1.10a (a) What is the largest prime modulus less than or equal to 2 63 − 1?
(b) How many corresponding full-period multipliers are there? (c) How does this relate to
the use of Lehmer random number generators on computer systems that support 64-bit
integer arithmetic?

Exercise 2.1.11 For the first few prime moduli, this table lists the number of full-period
multipliers and the smallest full-period multiplier. Add the next 10 rows to this table.
prime modulus m number of full-period multipliers smallest full-period multiplier a
2 1 1
3 1 2
5 2 2
7 2 3
11 4 2
13 4 2
 48 2.2 Lehmer Random Number Generators: Implementation
Recall that one good reason to choose m = 2 31 −1 as the modulus for a Lehmer random
number generator is that virtually all contemporary computer systems support 32-bit 2’s
complement integer arithmetic and on such systems 2 31 − 1 is the largest possible prime.
Consistent with that observation, a portable and efficient algorithmic implementation of
an m = 231 − 1 Lehmer generator that is valid (provided all integers between −m and m
can be represented exactly) is developed in this section.
We can use ANSI C for the implementation because 32-bit signed integer arithmetic
is supported in a natural way. That is, the ANSI C type long is required to be valid
for all integers between LONG MIN and LONG MAX inclusive and, although the values are
implementation dependent, LONG MAX and LONG MIN are required to be at least 2 31 − 1 and
at most −(231 − 1) respectively.*

2.2.1 IMPLEMENTATION
If there is no guarantee that integers larger than m can be represented exactly, then
the implementation issue of potential integer overflow must be addressed. That is, for any
full-period Lehmer generator, the product ax can be as large as a(m−1). Therefore, unless
values of t as large as a(m − 1) can be represented exactly, it is not possible to evaluate
g(x) = ax mod m in the “obvious” way by first computing the intermediate product t = ax
and then computing t mod m = t − �t/m�m.

Example 2.2.1 If (a, m) = (48271 , 231 − 1) then a(m − 1) ∼ = 1.47 × 246 . Therefore,
it would not be possible to implement this (a, m) Lehmer generator in the obvious way
without access to a register that is at least 47 bits wide to store the intermediate product t.
This is true even though t mod m is no more than 31 bits wide. (If a = 16807 then a 46 bit
register would be required to hold the intermediate product.)
Type Considerations
If we wish to implement a Lehmer generator with m = 2 31 − 1 in ANSI C, and do it
in the obvious way, then the type declaration of t will dictate the number of bits available
to store the intermediate t = ax product. Correspondingly, if t is (naturally) an integer
type then the integer division t/m can be used to evaluate �t/m�; or if t is a floating point
type then a floating point division t/m followed by a floating-point-to-integer cast can be
used to evaluate �t/m�.

Example 2.2.2 If the variable t is declared to be a long and m = 2 31 −1 then the obvious
implementation will be correct only if LONG MAX is a(m − 1) or larger. Most contemporary
computer systems do not support integers this large and thus for a Lehmer generator with
m = 231 − 1 the obvious implementation is not a viable algorithm option.
* The macros LONG MIN and LONG MAX are defined in the ANSI C library <limits.h>.
The type long is a shorthand representation for the type long int. LONG LONG is sup-
ported in C99, the new (maybe latest) ANSI/ISO Standard for C for 64-bit integers. This
option is only appropriate if the hardware supports 64-bit arithmetic (e.g., Apple, AMD).
2.2 Lehmer Random Number Generators: Implementation 49

Example 2.2.3 If the variable t is declared to be the ANSI C floating point type double
then the obvious implementation may be correct provided the multiplier a is not too large.
That is, double is generally consistent with the IEEE 754 64-bit floating point standard
which specifies a 53-bit mantissa (including the sign bit) and if m = 2 31 − 1 a mantissa
this large allows for t to be much larger than m. So it may be possible to implement a
m = 231 − 1 Lehmer generator with a sufficiently small multiplier in the obvious way by
doing the integer calculations in floating point arithmetic. However, when portability is
required and an efficient integer-based implementation is possible, only the unwise would
use a floating point implementation instead.
Algorithm Development
Consistent with the previous examples, it is desirable to have an integer-based imple-
mentation of Lehmer’s algorithm that will port to any system which supports the ANSI C
type long. This can be done provided no integer calculation produces an intermediate or
final result larger than m = 231 − 1 in magnitude. With this constraint in mind we must
be prepared to do some algorithm development.
If it were possible to factor the modulus as m = aq for some integer q then g(x)
could be written as g(x) = ax mod m = a(x mod q), enabling us to do the mod before the
multiply and thereby avoid the potential overflow problem. That is, in this case the largest
possible value of ax mod m would be a(q−1) = m−a and this is less than m. Of course if m
is prime no such factorization is possible. It is always possible, however, to “approximately
factor” m as m = aq + r with q = �m/a� and r = m mod a. As demonstrated in this
section, if the remainder r is small relative to the quotient q, specifically if r < q, then
this (q, r) decomposition of m provides the basis for an algorithm to evaluate g(x) in such
a way that integer overflow is eliminated.

Example 2.2.4 If (a, m) = (48271, 231 − 1) then the quotient is q = �m/a� = 44488
and the remainder is r = m mod a = 3399. Similarly, if a = 16807 then q = 127773 and
r = 2836. In both cases r < q.
For all x ∈ Xm = {1, 2, . . . , m − 1} define the two functions
γ(x) = a(x mod q) − r�x/q� and δ(x) = �x/q� − �ax/m�.
Then, for any x ∈ Xm
g(x) = ax mod m = ax − m�ax/m�
= ax − m�x/q� + m�x/q� − m�ax/m�
= ax − (aq + r)�x/q� + mδ(x)
� �
= a x − q�x/q� − r�x/q� + mδ(x)
= a(x mod q) − r�x/q� + mδ(x) = γ(x) + mδ(x).
An efficient, portable implementation of a Lehmer random number generator is based upon
this alternate representation of g(x) and the following theorem.
50 2. Random Number Generation

Theorem 2.2.1 If m = aq + r is prime, r < q, and x ∈ X m , then δ(x) = �x/q� − �ax/m�

is either 0 or 1. Moreover, with γ(x) = a(x mod q) − r�x/q�
• δ(x) = 0 if and only if γ(x) ∈ Xm ;
• δ(x) = 1 if and only if −γ(x) ∈ Xm .

Proof First observe that if u and v are real numbers with 0 < u − v < 1 then the integer
difference �u� − �v� is either 0 or 1. Therefore, because δ(x) = �x/q� − �ax/m�, the first
part of the theorem is true if we can show that
x ax
0< − < 1.
q m
Rewriting the center of the inequality as
� � � �
x ax 1 a m − aq xr
− =x − =x =
q m q m mq mq

and because r < q

xr x m−1
0< < ≤ <1
mq m m
which establishes that if x ∈ Xm then δ(x) is either 0 or 1. To prove the second part of this
theorem recall that if x ∈ Xm then g(x) = γ(x) + mδ(x) is in Xm . Therefore, if δ(x) = 0
then γ(x) = g(x) is in Xm ; conversely if γ(x) ∈ Xm then δ(x) cannot be 1 for if it were
then g(x) = γ(x) + m would be m + 1 or larger which contradicts g(x) ∈ X m . Similarly, if
δ(x) = 1 then −γ(x) = m − g(x) is in Xm ; conversely if −γ(x) ∈ Xm then δ(x) cannot be
0 for if it were then g(x) = γ(x) would not be in X m .
Algorithm
The key to avoiding overflow in the evaluation of g(x) is that the calculation with
the potential to cause overflow, the ax product, is “trapped” in δ(x). This is important
because, from Theorem 2.2.1, the value of δ(x) can be deduced from the value of γ(x) and
it can be shown that γ(x) can be computed without overflow — see Exercise 2.2.5. This
comment and Theorem 2.2.1 can be summarized with the following algorithm.

Algorithm 2.2.1 If m = aq + r is prime, r < q, and x ∈ X m , then g(x) = ax mod m

can be evaluated as follows without producing any intermediate or final values larger than
m − 1 in magnitude.
t = a * (x % q) - r * (x / q); /* t = γ(x) */
if (t > 0)
return (t); /* δ(x) = 0 */
else
return (t + m); /* δ(x) = 1 */
2.2 Lehmer Random Number Generators: Implementation 51

Modulus Compatibility

Definition 2.2.1 The multiplier a is modulus-compatible with the prime modulus m if

and only if the remainder r = m mod a is less than the quotient q = �m/a�.
If a is modulus-compatible with m then Algorithm 2.2.1 can be used as the basis for
an implementation of a Lehmer random number generator. In particular, if the multiplier
is modulus-compatible with m = 231 − 1 then Algorithm 2.2.1 can be used to implement
the corresponding Lehmer random number generator in such a way that it will port to
any system that supports 32-bit integer arithmetic. From Example 2.2.4, for example, the
full-period multiplier a = 48271 is modulus-compatible with m = 2 31 − 1.
In general, there are no modulus-compatible multipliers beyond (m − 1)/2. Moreover,
as the following example illustrates, modulus-compatible multipliers are much more densely
distributed on the low end of the 1 ≤ a ≤ (m − 1)/2 scale.

Example 2.2.5 The (tiny) modulus m = 401 is prime. Figure 2.2.1 illustrates, on the
first line, the 38 associated modulus-compatible multipliers. On the second line are the
160 full-period multipliers and the third line illustrates the ten multipliers (3, 6, 12, 13,
15, 17, 19, 21, 23, and 66) that are both modulus-compatible and full-period.

|||
||||||||||||||||| | | | | | | | | | Figure 2.2.1.
|| || ||||| |||| || ||| ||||| ||| | | | |||||| ||||||||||||||| || |||||| |||||| ||||||| | || ||| | ||||||||||||||| |||| ||| || | ||| ||||| || || |||| |||||||| || Modulus-
compatible
|| |||| ||| | full-period
multipliers
for m = 401.
0 50 100 150 200 250 300 350 400
Multipliers a between 1 and 400

If you have a very high resolution graphics device, a very long piece of paper, a magnifying
glass, and a few hours of CPU time, you are encouraged to make the corresponding figure
for m = 231 − 1. (For this modulus there are 92679 modulus-compatible multipliers. Of
these, 23093 are also full-period — see Example 2.2.6.)
Modulus-compatibility is closely related to the smallness of the multiplier. In partic-
ular, a is defined to be “small” if and only if a 2 < m. If a is small then a is modulus-
compatible with m — see Exercise 2.2.4. From this result it follows, for example, that all
the multipliers between 1 and 46340 inclusive are modulus-compatible with m = 2 31 − 1.
Thus smallness is sufficient to guarantee modulus-compatibility. Smallness is not a neces-
sary condition, however. In particular, a = 48271 is modulus-compatible with m = 2 31 − 1,
but it is not small. Similarly, of the 38 modulus-compatible multipliers for m = 401 in Ex-
ample 2.2.5, only about half (those between 1 and 20) satisfy the a 2 < m small multiplier
test.
52 2. Random Number Generation

As illustrated in Example 2.2.5, for a given prime modulus there are relatively few
associated full-period, modulus-compatible multipliers. A mechanical way to find one of
these is to start with the small (and thus modulus-compatible) multipliers a = 2, a = 3,
etc.; until the first full-period multiplier is found — see Algorithm 2.1.1. Given that we
have found one full-period, modulus-compatible multiplier, the following O(m) algorithm
can be used to generate all the others. This algorithm, an extension of Algorithm 2.1.2,
presumes the availability of a function gcd that returns the greatest common divisor of
two positive integers (see Appendix B).

Algorithm 2.2.2 Given the prime modulus m and any associated full-period, modulus-
compatible multiplier a the following algorithm generates all the full-period, modulus-
compatible multipliers relative to m.
i = 1;
x = a;
while (x != 1) {
if ((m % x < m / x) and (gcd(i, m - 1) == 1))
/* x is a full-period modulus-compatible multiplier */
i++;
x = g(x); /* use Algorithm 2.2.1 to evaluate g(x) = ax mod m */
}

Example 2.2.6 Using Algorithm 2.2.2 with m = 231 − 1 = 2147483647, a = 7 and a

lot of CPU cycles we find that there are a total of 23093 associated full-period, modulus-
compatible multipliers, the first few of which are

71 mod 2147483647 = 7
75 mod 2147483647 = 16807
7113039 mod 2147483647 = 41214
7188509 mod 2147483647 = 25697
7536035 mod 2147483647 = 63295.

Of these, the multiplier a = 16807 deserves special mention. It was first suggested by Lewis,
Goodman, and Miller (1969), largely because it was easy to prove (as we have done) that
it is a full-period multiplier. Since then it has become something of a “minimal” standard
(Park and Miller, 1988).
In retrospect a = 16807 was such an obvious choice that it seems unlikely to be the
best possible full-period multiplier relative to m = 2 31 − 1 and, indeed, subsequent testing
for randomness has verified that, at least in theory, other full-period multipliers generate
(slightly) more random sequences. Although several decades of generally favorable user
experience with a = 16807 is not easily ignored, we will use a = 48271 instead.
2.2 Lehmer Random Number Generators: Implementation 53

Randomness
For a given (prime) modulus m, from among all the full-period, modulus-compatible
multipliers relative to m we would like to choose the one that generates the “most ran-
dom” sequence. As suggested in Example 2.1.1, however, there is no simple and universal
definition of randomness and so it should come as no surprise to find that there is less than
complete agreement on what this metric should be.

Example 2.2.7 To the extent that there is agreement on a randomness metric, it is

based on a fundamental geometric characteristic of all Lehmer generators, first established
by Marsaglia (1968), that “random numbers fall mainly in the planes.” That is, in 2-space
for example, the points
(x0 , x1 ), (x1 , x2 ), (x2 , x3 ), . . .
all fall on a finite, and possibly small, number of parallel lines to form a lattice structure
as illustrated in Figure 2.2.2 for (a, m) = (23, 401) and (66, 401). If a simulation model of
a nuclear power plant, for example, required two consecutive small random numbers (e.g.,
both smaller than 0.08) for a particular rare event (e.g., melt down), this would never occur
for the (66, 401) generator due to the void in the southwest corner of the graph. [Using
the axiomatic approach to probability, the exact probability is (0.08)(0.08) = 0.0064.]
400 ·· ·· ·· ··· ·· ·· ··· ·· ·· ··· 400 ··· ··· ··· ··· ···
··· Figure 2.2.2.
··· ··· ··· ···
· ·· ·· ·· ·· ·· ·· ·· ·· ·· ··· ··· ··· · ··· ···
350 · ·· · · · · · · ·· · · · · · · ·· · · · · · · ·· 350 ···· ····
···
····
···
···
···
···
·· Random
· ·· ·· ·· ·· ·· ·· ·· ·· ·· ···
···
···
··· ··· ··· ··· numbers
300 · ·· · · · · · · ·· · · · · · · ·· · · · · · · ·· 300 ··· ···
···
···
·
···
·
···
· falling in
· ·· ·· ·· ·· ·· ·· ·· ·· ·· ···
··· ··· ···
·
···
·
···
·
250 · ·· · · · · · · ·· · · · · · · ·· · · · · · · ·· ···
250 ··· ···
···
···
···
····
···
····
···
planes.
· ·· ·· ·· ·· ·· ·· ·· ·· ·· ··· · ···· ···· ····
200 · ·· · · · · · · ·· · · · · · · ·· · · · · · · ·· 200 ···
···
···
···
··· ··· ··· ··
· ·· ·· ·· ·· ·· ·· ·· ·· ·· ··· ···· ···· · ··· ····
· ·· · · · · · · ·· · · · · · · ·· · · · · · · ·· ···
···
···
···
···
···
···
···
···
···
150 · ·· ·· ·· ·· ·· ·· ·· ·· ·· 150 ··· ··· ··· ··· ··
···
· ·· · · · · · · ·· · · · · · · ·· · · · · · · ·· ··· ···
···
· · ··· ·· ····
······
100 ·· ·· ·· ··· ·· ·· ··· ·· ·· ··· 100 ···
··· ··· ··· ··· ···
· · · ···
· ·· ·· ·· ·· ·· ·· ·· ·· ·· ··· ··· ···
· ··· ···
·
50 · ·· · · · · · · ·· · · · · · · ·· · · · · · · ·· 50 ···· · ···· · ····
···
·· ···
···
····
···
·· ·· · ··· ·· ·· ··· ·· ·· ··· ···
···
···
··· ··· ··· ···
0 · · · · · · · · · 0 · · · · ·
0 50 100 150 200 250 300 350 400 0 50 100 150 200 250 300 350 400
(a, m) = (23, 401) (a, m) = (66, 401)
In general, for any integer k ≥ 2 the points
(x0 , x1 , . . . , xk−1 ), (x1 , x2 , . . . , xk ), (x2 , x3 , . . . , xk+1 ), . . .
form a lattice structure. The best Lehmer generator randomness metrics are based on
analyzing (numerically, not visually) the uniformity of this lattice structure in k-space for
small values of k. Knuth (1998) gives one popular randomness metric of this type known
as the spectral test. Consistent with these metrics, for m = 401 the multiplier a = 23
would be a much better full-period, modulus-compatible multiplier than a = 66.
54 2. Random Number Generation

ANSI C Implementation
The kind of theoretical testing for randomness illustrated in Example 2.2.7 has been
done for all of the 23093 full-period, modulus-compatible multipliers relative to m = 2 31 −1.
Of these, the winner is a = 48271 with q = �m/a� = 44488 and r = m mod a = 3399.*

Example 2.2.8 A Lehmer random number generator with (a, m) = (48271, 2 31 − 1) can
be implemented correctly, efficiently and portably in ANSI C as follows
double Random(void)
{
const long A = 48271; /* multiplier */
const long M = 2147483647; /* modulus */
const long Q = M / A; /* quotient */
const long R = M % A; /* remainder */
static long state = 1;
long t = A * (state % Q) - R * (state / Q);
if (t > 0)
state = t;
else
state = t + M;
return ((double) state / M);
}

With minor implementation-dependent modifications, the random number generator

in Example 2.2.8 is the basis for all the simulated stochastic results presented in this book.
There are three important points relative to this particular implementation.
• The static variable state is used to hold the current state of the random number
generator, initialized to 1. There is nothing magic about 1 as an initial state (seed);
any value between 1 and 2 147 483 646 could have been used.
• Because state is a static variable, the state of the generator will be retained between
successive calls to Random, as must be the case for the generator to operate properly.
Moreover, because the scope of state is local to the function, the state of the generator
is protected — it cannot be changed in any way other than by a call to Random.
• If the implementation is correct, tests for randomness are redundant; if the implemen-
tation is incorrect, it should be discarded. A standard way of testing for a correct
implementation is based on the fact that if the initial value of state is 1, then after
10 000 calls to Random the value of state should be 399 268 537.
* In addition to “in theory” testing, the randomness associated with this (a, m) pair
has also been subjected to a significant amount of “in practice” empirical testing — see
Section 10.1.
2.2 Lehmer Random Number Generators: Implementation 55

Example 2.2.9 As a potential alternative to the generator in Example 2.2.8, the random
number generator in the ANSI C library <stdlib.h> is the function rand. The intent of
this function is to simulate drawing at random from the set {0, 1, 2, . . . , m − 1} with m
required to be at least 215 . That is, rand returns an int between 0 and RAND MAX inclusive
where the macro constant RAND MAX (defined in the same library) is required to be at
least 215 − 1 = 32767. To convert the integer value returned by rand to a floating point
number between 0.0 and 1.0 (consistent with Definition 2.1.1) it is conventional to use an
assignment like
u = (double) rand() / RAND MAX;
Note, however, that the ANSI C standard does not specify the details of the algorithm on
which this generator is based. Indeed, the standard does not even require the output to
be random! For scientific applications it is generally a good idea to avoid using rand, as
indicated in Section 13.15 of Summit (1995).
Random Number Generation Library
The random number generation library used in this course is based upon the imple-
mentation considerations developed in this section. This library is defined by the header file
"rng.h" and is recommended as a replacement for the standard ANSI C library functions
rand and srand, particularly in simulation applications where the statistical goodness of
the random number generator is important. The library provides the following capabilities.
• double Random(void) — This is the Lehmer random number generator in Exam-
ple 2.2.8. We recommended it as a replacement for the standard ANSI C library
function rand.
• void PutSeed(long seed) — This function can be used to initialize or reset the
current state of the random number generator. We recommended it as a replacement
for the standard ANSI C library function srand. If seed is positive then that value
becomes the current state of the generator. If seed is 0 then the user is prompted
to set the state of the generator interactively via keyboard input. If seed is negative
then the state of the generator is set by the system clock.*
• void GetSeed(long *seed) — This function can be used to get the current state of
the random number generator.∗
• void TestRandom(void) — This function can be used to test for a correct implemen-
tation of the library.
Although we recommend the use of the multiplier 48271, and that is the value used in
the library rng, as discussed in Example 2.2.6 the 16807 multiplier is something of a
minimal standard. Accordingly, the library is designed so that it is easy to use 16807 as
an alternative to 48271.

* See Section 2.3 for more discussion about the use of PutSeed and GetSeed.
 56 2. Random Number Generation

Example 2.2.10 As an example of the use of Random and PutSeed, this algorithm was
used to create the two scatterplots illustrated in Figure 2.2.3 for two different values of the
initial seed.
seed = 123456789; /* or 987654321 */
PutSeed(seed);
x0 = Random();
for (i = 0; i < 400; i++) {
xi+1 = Random();
Plot(xi , xi+1 ); /* a generic graphics function */
}
Unlike the lattice structure so obvious in Example 2.2.7, the (x i , xi+1 ) pairs in this case
appear to be random with no lattice structure evident, as desired. (For a more direct com-
parison all the integer-valued coordinates in Example 2.2.7 would need to be normalized
to 1.0 via division by m = 401. That would be a purely cosmetic change, however.)

1.00 ·· · · · · · · · · · 1.00 · ·· · · ·· ·· ·
· ·· · · · · ·· ··· ··· · · · · ·· · · · ·· · · ·· · · · · ·· · ··· · ····· ····· ·
Figure 2.2.3. · · · ·· · ··· ··· · ····· · · · · · · ·· · ·· · · ·
· · · ····· · ·· ····· ··· · ····· · ·
· · · · · · · · ·
Scatterplots · · · ·· · ·· ·
0.75 · ···· · · · · · ·· · · · · · · · · ·· · · 0.75 ·· ·· · ·· · ·· ··· ··· · ···· · ·· ·· ···
of 400 · · · ·· · · · · ·
···· ······ · · · ····· ·
· · · · · · · · ···
· ·
·
· ·· ·· · ·· · · ·· · · · ··· ···· · · ·· ·
random ·· · ·· · · ·· · · ·
· ··· · · · ·· · ·· · · · ··
· · · ·· · · · · · ·· · · · · · · · ··· ·· · ·· · · · · ·· ·
number ·· · · ·· · · · · · · ·· · · ·· ········ · ·· · 0.50 · · · · ··· · · · ···· · · · ··· ···· · · ·
pairs. 0.50 · ·· · · · · · ·· ·· · ·· ·· · · · ·
· · · · · ···· ·· · · · · · · ···· ··· · · · ·· · · · ·· · ··· · · · · ·· · ··· · ·· · ·· ·
·
···· · · · ··· · · · · · ·· · ·· · ·· · ·· · · · · · · ·· ·· · ·
· ·· · · ··· · ··· · ···· ·· ·· · · · · · · · ··· · · · ·· ·· · · ··
· ··· · ··
0.25 · · · · ·· · · · · ······ · · · · · · · · · 0.25 · · · ·· ··· ·· · · ·· · ··· · · ·· ·· ·· ·· ·
···· · · ·· · · · ·· ···· · ·· ·· · · · ···· · · · · ·· · ·
· · · · · · · · · ·· ··· · · · · ·· ··· · ···
· · · · ·
· · ···· · ·· · · · · · ·· · · · ·
· · ·· ·· · · · ··· ·· ·· · · ·· ··· · ·· · ····· · ·· ·· · · ·· · · · ·· · ··
0.00 · 0.00 ·
0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00
Initial seed x0 = 123456789 Initial seed x0 = 987654321

Consistent with the discussion in Example 2.2.7, it should be mentioned that in a

sense the appearance of randomness in Example 2.2.10 is an illusion. That is, if all of
the possible pairs of (xi , xi+1 ) points were generated (there are m − 1 = 2 31 − 2 of these)
and if it were somehow possible to plot all of these pairs of points at a fine enough scale
to avoid blackening the page, then like the figures in Example 2.2.7 a micro-scale lattice
structure would be evident.* This observation is one of the reasons why we distinguish
between ideal and good random number generators (see Definition 2.1.1).

* Contemplate what size graphics device and associated dpi (dots per inch) resolution
would be required to actually do this.
2.2 Lehmer Random Number Generators: Implementation 57

Example 2.2.11 Plotting consecutive, overlapping random number pairs (x i , xi+1 ) from
a full-period Lehmer generator with m = 2 31 − 1 would blacken the unit square, obscuring
the lattice structure. The fact that any tiny square contained in the unit square will
exhibit approximately the same appearance is exploited in the algorithm below, where
all of the random numbers are generated by Random(), but only those that fall in the
square with opposite corners (0, 0) and (0.001, 0.001) are plotted. One would expect that
approximately (0.001)(0.001)(231 − 2) ∼= 2147 of the points would fall in the tiny square.
seed = 123456789;
PutSeed(seed);
x0 = Random();
for (i = 0; i < 2147483646; i++) {
xi+1 = Random();
if ((xi < 0.001) and (xi+1 < 0.001)) Plot(xi , xi+1 );
}
The results of the implementation of the algorithm are displayed in Figure 2.2.4 for the
multipliers a = 16807 (on the left) and a = 48271 (on the right). The random numbers
produced by a = 16807 fall in just 17 nearly-vertical parallel lines, whereas the random
numbers produced by a = 48271 fall in 47 parallel lines. These scatterplots provide further
evidence for our choice of a = 48271 over a = 16807 in the random number generator
provided in the library rng.
0.001 ·· ··· ··· ·· ·· ··· ··· ··· ··· ··· ··· ·· ··· ··· ·· ··· ··· 0.001 ······························································································ Figure 2.2.4.
··· ·· ·· ·· ·· ·· ·· ·· ·· ·· · ··· ·· ·· ··· ·· ··
· · · · · · · · · · · · · · ··································································································
···· ··· ···· ···· ··· ···· ····· ···· ···· ···· ··· ····· ··· ··· ···· ···· ···· ·························································································································································· Scatterplots
· · · · · · · · · · · · · · · · · ······ ············ ············ ············
··· ···· ··· ···· ··· ··· ···· ···· ··· ···· ···· ··· ···· ···· ····· ···· ··· ·························································································································································· for a = 16807
··· ·· · ·· ·· ·· ··· ·· ··· ·· ·· ·· ·· ··· ·· ·· ··· ······················································································ ············ and a = 48271.
··· ····· ···· ····· ··· ···· ··· ···· ···· ····· ···· ···· ···· ···· ···· ···· ···· ···················································································································································
··· ·· ·· · ··· ·· ·· ··· ·· · · ·· ·· ·· ·· ·· ·· ··································································································
··· ···· ···· ···· ··· ···· ···· ··· ···· ····· ···· ··· ···· ··· ···· ····· ···· · ·································································································
······ ············ ············ ············
·· ·· ·· ·· ·· ·· ·· ··· ··· ·· ·· ·· ·· ··· ·· ·· ··
··· ·· ··· ··· ·· ·· ··· ·· ·· ·· ··· ·· ·· ·· ·· ··· ·· ·························································································································································
·
· ·· · · · ·· · ·· · ·· ·· ·· ·· · · · ·· ······ ············ ············ ············
····· ···· ···· ···· ····· ···· ····· ···· ···· ···· ···· ···· ···· ···· ···· ···· ··· ··························································································································································
··································································································
···· ···· ···· ···· ··· ··· ··· ··· ····· ···· ··· ···· ···· ··· ···· ··· ···· ······ ············ ············ ············
· · · ·
···· ··· ···· ···· ··· ···· ···· ···· ···· ···· ·· ·· ···· ···· ···· ···· ··· · ··························································································································································
······ ············ ············ ············
· · · ·
···· ···· ····· ····· ···· ···· ···· ···· ···· ···· ··· ···· ···· ··· ····· ···· ···
·
··························································································································································
· · · · · · ··································································································
···· ···· ···· ···· ···· ···· ···· ···· ···· ····· ···· ···· ····· ··· ···· ···· ···· ······ ············ ············ ············
·· ·· ··· ··· ··· ··· ··· ·· ··· ··· ··· ·· ··· ··· ·· ··· ··· ·
· ························································································································································
0.0 0.0 ······ ······ ······
0.0 0.001 0.0 0.001
Multiplier a = 16807 Multiplier a = 48271
We have shown that as we zoom in on the unit square the (unwanted) lattice structure
of the (xi , xi+1 ) pairs produced by our good random number generator becomes more
apparent. It is appropriate at this juncture to contemplate what would happen with an
ideal random number generator. An ideal generator exhibits no lattice behavior as you
zoom in on increasingly tiny squares. Like a fractal or a scale-free network, all scatterplots
will look like those in Figure 2.2.3, assuming more points are generated as you zoom in.
58 2. Random Number Generation

Other Multipliers
Recall that for the modulus m = 231 − 1 = 2 147 483 647, there are 534 600 000 mul-
tipliers a that result in a full-period random number generator. Of these, 23 093 are
modulus-compatible. Although the modulus-compatible, full-period multiplier a = 48 271
is used in library rng, are there other multipliers that are recommended? One rather te-
dious way of approaching this question is to execute the statistical tests for random number
generators outlined in Section 10.1 on all of the generators. A second, more efficient ap-
proach is to consult the experts who perform research in this area. Fishman (2001, pages
428–445) notes that a = 16 807, a = 630 360 016, and a = 742 938 285 are employed by
the discrete-event simulation languages SIMAN, SIMSCRIPT II.5, and GPSS/H, respec-
tively, ordered here by improving lattice structure. In addition to a = 742 938 285, he also
suggests four other multipliers with similar performance in terms of parallel hyperplanes
in 2, 3, 4, 5, and 6 dimensions: a = 950 706 376, a = 1 226 874 159, a = 62 089 911, and
a = 1 343 714 438. Determining whether these multipliers are modulus-compatible is left
as an exercise.
Good Generator — Non-representative Subsequences
In any simulation run, only a subset of the random numbers produced by a generator
are typically used. What if, for example, only 20 random numbers were needed for a
simulation. If you had the extreme misfortune of choosing the seed x 0 = 109 869 724 with
the generator in the library rng, the resulting 20 random numbers (to only 0.xx precision)
are:
0.64 0.72 0.77 0.93 0.82 0.88 0.67 0.76 0.84 0.84

0.74 0.76 0.80 0.75 0.63 0.94 0.86 0.63 0.78 0.67.

Is there something wrong with this generator to have produced 20 consecutive random
numbers exceeding 0.62? Certainly not. It will only occur once in a blue moon, but
this particular seed resulted in this rather unique sequence of random numbers.* This
sequence of 20 consecutive random numbers may initially seem analogous to an outlier
from statistics. Statisticians sometimes discard outliers as being unrepresentative, but it
is never appropriate to do so in simulation. The random number generator will have a
few sequences of unusually high random numbers as well as a few sequences of unusually
low random numbers, as it should. Analogously, if we were to perform the experiment of
tossing a fair coin 20 times for a large number of replications, there should be a few rare
cases when all 20 tosses come up heads (or tails).
This string of 20 large random numbers highlights the importance of replicating a
simulation many times so as to average out these unusual cases. Replication will be
considered in detail in Chapter 8.

* The probability of 20 consecutive random numbers exceeding 0.62 using the axiomatic
approach to probability is (0.38)20 ∼
= 4 · 10−9 .
 2.2 Lehmer Random Number Generators: Implementation 59

The Curse of Fast CPUs

When discrete-event simulation was in its infancy, the time required to cycle through
a full-period Lehmer generator with modulus m = 2 31 − 1 = 2 147 483 647 was measured
in days. This quickly shrank to hours, and now only takes a few minutes on a desktop
machine. Before long, the time to complete a cycle of such a generator will be measured
in seconds. The problem of cycling, that is, reusing the same random numbers in a single
simulation, must be avoided because of the resultant dependency in the simulation output.
Extending the period length of Lehmer generators has been an active research topic
within the simulation community for many years. One simple solution is to run all
simulations on 64-bit machines. Since the largest prime number less than 2 63 − 1 is
263 − 25 = 9 223 372 036 854 775 783, the period is lengthened from about 2.14 · 10 9 to
9.22 · 1018 . In this case, cycle times are measured in years. Obviously, more machines
supporting 64-bit integer arithmetic is beneficial to the simulation community. Since it is
too early to know if 64-bit machines will become the norm, other portable techniques on
32-bit machines for achieving longer periods have been developed.
One technique for achieving a longer period is to use a multiple recursive generator ,
where our usual xi+1 = axi mod m is replaced by

xi+1 = (a1 xi + a2 xi−1 + · · · + aq xi−q ) mod m.

These generators can produce periods as long as m q − 1 if the parameters are chosen
properly. A second technique for extending the period is to use a composite generator ,
where several Lehmer generators can be combined in a manner to extend the period and
improve statistical behavior. A third technique is to use a Tausworthe, or shift-register
generator, where the modulo function is applied to bits, rather than large integers. The
interested reader should consult Chapter 6 of Bratley, Fox, and Schrage (1987), Chapter 5
of Lewis and Orav (1989), Chapter 7 of Law and Kelton (2000), Chapter 9 of Fishman
(2001), Gentle (2003), and L’Ecuyer, Simard, Chen, and Kelton (2002) for more details.
Any of these generators that have been well-tested by multiple authors yield two
benefits: longer periods and better statistical behavior (e.g., fewer hyperplanes). The cost
is always the same: increased CPU time.

2.2.2 EXERCISES

Exercise 2.2.1 Prove that if u and v are real numbers with 0 < u − v < 1 then the
integer difference �u� − �v� is either 0 or 1.

Exercise 2.2.2a If g(·) is a Lehmer generator (full period or not) then there must exist
an integer x ∈ Xm such that g(x) = 1. (a) Why? (b) Use the m = aq + r decomposition of
m to derive a O(a) algorithm that will solve for this x. (c) If a = 48271 and m = 2 31 − 1
then what is x? (d) Same question if a = 16807.
60 2. Random Number Generation

Exercise 2.2.3a Derive a O(log(a)) algorithm to solve Exercise 2.2.2. Hint: a and m
are relatively prime.

Exercise 2.2.4 Prove that if a, m are positive integers and if a is “small” in the sense
that a2 < m, then r < q where r = m mod a and q = �m/a�.

Exercise 2.2.5 Write γ(x) = α(x) − β(x) where α(x) = a(x mod q) and β(x) = r�x/q�
with m = aq + r and r = m mod a. Prove that if r < q then for all x ∈ X m both α(x) and
β(x) are in {0, 1, 2, . . . , m − 1}.

Exercise 2.2.6 Is Algorithm 2.2.1 valid if m is not prime? If not, how should it be
modified?

Exercise 2.2.7 (a) Implement a correct version of Random using floating point arithmetic
and do a timing study. (b) Comment.

Exercise 2.2.8 Prove that if a, x, q are positive integers then ax mod aq = a(x mod q).

Exercise 2.2.9 You have been hired as a consultant by XYZ Inc to assess the market
potential of a relatively inexpensive hardware random number generator they may develop
for high-speed scientific computing applications. List all the technical reasons you can
think of to convince them this is a bad idea.

Exercise 2.2.10 There are exactly 400 points in each of the figures in Example 2.2.7.
(a) Why? (b) How many points would there be if a were not a full-period multiplier?

Exercise 2.2.11 Let m be the largest prime modulus less than or equal to 2 15 − 1
(see Exercise 2.1.6). (a) Compute all the corresponding modulus-compatible full-period
multipliers. (b) Comment on how this result relates to random number generation on
systems that support 16-bit integer arithmetic only.

Exercise 2.2.12a (a) Prove that if m is prime with m mod 4 = 1 then a is a full-period
multiplier if and only if m − a is also a full-period multiplier. (b) What if m mod 4 = 3?

Exercise 2.2.13a If m = 231 − 1 compute the x ∈ Xm for which 7x mod m = 48271.

Exercise 2.2.14 The lines on the scatterplot in Figure 2.2.4 associated with the multi-
plier a = 16807 appear to be vertical. Argue that the lines must not be vertical based on
the fact that (a, m) = (16807, 231 − 1) is a full-period generator.

Exercise 2.2.15 Determine whether the multipliers associated with m = 2 31 − 1 given

by Fishman (2001): a = 630 360 016, a = 742 938 285 a = 950 706 376, a = 1 226 874 159,
a = 62 089 911, and a = 1 343 714 438 are modulus-compatible.