Numer. Math.

34, 247-270 (1980) Numerische

Mathematik
9 by Springer-Verlag 1980

Global Optimization Using Interval Analysis-

The Multi-Dimensional Case

Eldon Hansen
Lockheed Missiles and Space Company
Sunnyvale, CA 94086, USA

Summary. We show how interval analysis can be used to compute the global
minimum of a twice -continuously differentiable function of n variables over
an n-dimensional parallelopiped with sides parallel to the coordinate axes.
Our method provides infallible bounds on both the globally minimum value
of the function and the point(s) at which the minimum occurs.
Subject Classification: AMS(MOS): 65K05, 90C30.

1. Introduction

Consider the function f(x) in C 2 of n variables x I .... , x,. We shall describe a

method for computing the minimum value f * o f f ( x ) over a box X (~ A box is
defined to be a closed rectangular parallelopiped with sides parallel to the
coordinate axes. We assume the number of points in X ~~ at which f(x) is
globally minimum is finite. Our method provides infallible bounds on f * and on
the point(s) x* for which f(x*)=f*. That is, our algorithm produces bounds on
x* and f * which are always correct despite the presence of rounding errors.
H o w sharp these bounds can be depends on the function f and the precision of
the computer used.
For a highly oscillatory function f, our algorithm could be prohibitively
slow. Presumably this wilt always be the case for any future global optimization
algorithm. However, our algorithm is sufficiently fast for 'reasonable' functions.
We assume that interval extensions (see [8]) of f and its derivatives are
known. This is the case if every function in terms of which f and its derivatives
are defined have known rational approximations with either uniform or rational
error bounds for the arguments of interest.
Since the initial box can be chosen as large as we please, our algorithm
actually solves the unconstrained minimization problem provided it is known
that the solution occurs in some finite region (which we enclose in the initial
box).

0029-599 X/80/0034/0247/$04.80
248 E. Hansen

There is a common misconception among researchers in optimization that it

is impossible to obtain infallible bounds on x* and f * computationally. The
argument is that we can only sample f ( x ) and a few derivatives o f f ( x ) at a finite
number of points. It is possible to interpolate a function having the necessary
values and derivatives values at these points and still have its global minimum
at any other arbitrary point. The fallacy of this argument is that interval analysis
can provide bounds on a function over an entire box; that is over a continuum
of points. It is only necessary to make the box sufficiently small in order to
make the bounds arbitrarily sharp. This is what our algorithm does. It narrows
the region of interest until the bound is as sharp as desired (subject to roundoff
restrictions).
In a previous paper [5], we gave a method of this type for the one-
dimensional case. The method never failed to converge provided f ' ( x ) and f"(x)
had only a finite number of isolated zeros. Our method for the n-dimensional
problem appears to always converge also; but we have not yet attempted to
prove it. When it does converge, there is never a question that x* and f * satisfy
the computed bounds.
Recently, R.E. Moore [9] published a method for computing the range of a
rational function of n variables over a bounded region. (See also [14].) Although
he does not note the fact, his method will serve to bound the global minimum
value f * of a rational function. However, our algorithm is more efficient.
Moreover, it is designed to bound x* as well as f * .
We suggest the reader read the previous paper [5] before the current one.
The one-dimensional case therein serves as an easier introduction. However, the
current paper is essentially self contained. It would be better if the reader had
some familiarity with the rudiments of interval analysis such as can be found in
the first three chapters of [8]. However, we shall review some of its relevant
properties.
Our method will find the global minimum (or minima). Because of computer
limitations of accuracy, it may also find near-global minima such that rounding
errors prevent determination of which is the true minimum. However, if the
termination criteria are sufficiently stringent, our algorithm will always elim-
inate a local minimum whose value is substantially larger than f * .
Our algorithm is composed of four separate parts. One part uses an interval
version of Newton's method to find stationary points. A second part eliminates
points of X t~ where f is greater than the smallest currently known value J~
A third part of our algorithm tests whether f is monotonic in a sub-box X of
X (~ If so, we delete part or all of X depending on whether X contains boundary
points of X~~
A fourth part checks for convexity of f in a sub-box X of X t~ If f is not
convex .anywhere in X, there cannot be a stationary minimum o f f in X.
The first part of the algorithm, if used alone, would find all stationary points
in X(~ The second part serves to eliminate stationary points where f>f*.
Usually they are eliminated before they are found with any great accuracy.
Hence computational effort is not wasted using the first part to accurately find
an unwanted stationary point. The second part also serves to eliminate bound-
ary points of X ~~ and to find a global minimum if it occurs on the boundary.
Global Optimization Using Interval Analysis 249

The second part of the algorithm used alone would find the global minimum (or
minima) but its asymptotic convergence is relatively slow compared to that of
the Newton method. Hence the latter is used also. The third and fourth parts of
the algorithm merely improve convergence.

2. Interval Analysis

The toot which allows us to be certain we have bounded the global minimum is
interval analysis. We bound rounding errors by using interval arithmetic. More
importantly, however, we use interval analysis to bound the range of a function
over a box.
Let g(x) be a rational function of n variables x 1.... , x,. On a computer, we
can evaluate g(x) for a given x by performing a sequence of arithmetic
operations involving only addition, subtraction, multiplication, and division.
Let X~ (i = 1.... , n) be closed intervals. If we use X~ in place of x~ and perform
the same sequence of operations using interval arithmetic (see [8]) rather than
ordinary real arithmetic, we obtain a closed interval g(X) containing the range

{g(x): xieXi(i = 1..... n)}

of g(x) over the box X. This result will not be sharp, in general, but if outward
rounding (see [8]) is used, then g(X) will always contain the range. The lack of
sharpness results from other causes besides roundoff. With exact interval arith-
metic, the lack of sharpness diappears as the widths of the intervals decrease to
zero.
If g(x) is not rational, we assume an algorithm is known for computing an
interval g(X) containing the range of g(x) for x~X. Methods for deriving such
algorithms are discussed in [8]).

3. Taylor's Theorem

We shall use interval analysis in conjunction with Taylor's theorem in two ways.
First, we expand f as

f(y) = f ( x ) + (y - x)r g (x) + 89(y - x)r H (x, y, 4)(Y - x) (3.1)

where g(x) is the gradient o f f ( x ) and has components gi(x)=Of(x)/Ox~. The

quantity H(x, y, ~) is the Hessian matrix to be defined presently. For reasons
related to the use of interval analysis, we shall express it as a lower triangular
matrix instead of a symmetric matrix so that there are fewer terms in the
quadratic form involving H(x, y, 4).
We define the element in position (i,j) of H(x, y, ~) as

[02f/Ox~ for j=i(i=l,...,n),

hq={2O2f/dx~Oxj for j < i ( i = l , . . . , n ; j = l .... , i - l ) , (3.2)
[0 otherwise.
250 E. Hansen

The arguments of hij depend on i and j. If we expand f sequentially in one of its

variables at a time, we can obtain the following results illustrating the case n = 3

-hi 1(~11, x2, x3) 0 0 ]

l
H ( x , y , ~)= hzl(~2x ,x2,x3)
[h31(~31, x2, x3)
hz2(Y1,~22, x3)
h3z(Yl, ~32, x3) h33(Yl, Y2, ~33)
0 .

Assume x i e X i and y~eX~ for i = 1.... , n. Then ~ij~Xj for each j = 1.... , i. For
general n, the arguments of H~j are (Yx, -.-, Yj- 1, ~j, xj+ 1.... , x,). Other arrange-
ments of arguments could be obtained by reordering the indices.
Let x be a fixed point in X. Then for any point y e X ,

H(x, y, ~)~H(x, X , X);

that is, for i>j,

hiJ(Yl .... , Y i- 1, ~i.i, x2+ 1. . . . . x,,)Ehl.i(X1 . . . . . X i , x j+ 1. . . . . x,,).

In the sequel, we shall shorten notation and use H(~) to denote H(x, y, ~) and
H ( X ) to denote H(x, X , X).
The purpose of this particular Taylor expansion is to obtain real (non-
interval) quantities for as many arguments of the elements of H ( X ) as possible.
The standard Taylor expansion would have intervals for all arguments of all
elements of H(X). This type of expansion was introduced in [3]. A more general
approach of this kind is discussed in I-4].
The other Taylor expansion we shall want is of the gradient g. Each element
gi(i= i , . . . , n) of g can be expanded as

gi(Y) -----gi(X) + (Yl -- x1) dil (t]l, x2, -", xn) + (Y2 - xz) Ji2(Y l, 1~2, x3 . . . . . Xn)
+(Y3 - x 3 ) Ji3(Yl, Y2, rl3, x4 . . . . . x,,)+ ... + ( y n - x n ) Jin(Yl . . . . , Y,- 1, rl,),
where 0.3)
Ji2 = c~2f/c3xi c3xj (i,j = 1.... , n).

This Jacobian matrix J and the Hessian H introduced above are, of course,
essentially the same. However, they will be evaluated with different arguments
depending on whether we are expanding f or g. Also, H is lower triangular while
d is a full matrix.
Let J(x, y, Yl) denote the Jacobian matrix with elements J~i(Yl . . . . , Y.i 1, ~lj,
x j+ 1.... , x,). Then
g(y)= g(x) + J (x, y, rl) ( y - x). (3.4)

If x e X and y e X , then theX ~ for all i-- 1, ..., n. Hence

g(y)eg(x) + J (x, X, X ) ( y - x). (3.5)

We shall again shorten notation and denote J(x, y, tl) by J(t/) and J(x, X , X) by
J(X).
Global Optimization Using Interval Analysis 251

Note that the elements of H(X) on and below the diagonal have the same
arguments as the corresponding elements of J(X). Thus we need only calculate
J(X); then H(X) follows easily.

4. The Approximate Value of the Global Minimum

As we proceed with our algorithm, we shall evaluate f(x) at various points in

X (~ Let f denote the currently smallest value o f f found so far. The very first
step is to evaluate f at the center of X ~~ This value serves as the first one for J~
One part of our algorithm deletes sub-boxes of X ~~ wherein f > f since this
implies i n f f > f * . (See Sect. 7.)
In practice we cannot generally evaluate f(x) exactly because of rounding
errors. Hence we do the evaluation using interval arithmetic. Suppose we obtain
the interval i f L,fR]. Then we know that f(x)<=fR and hence that f_<fR. Hence
when we evaluate f(x), we update f by replacing it by fR only i f f R is less than
the previous value o f f In this way, we assure that f is always an upper bound
for f * .

5. A Test for Convexity

As our algorithm proceeds, we dynamically subdivide X ~~ into sub-boxes. Let X

denote such a sub-box. We evaluate hii(X ~..... X,) for i = 1, ..., n, where hii is
the diagonal element of the Hessian. Note that every argument of hi~ is an
interval and hence the resulting interval contains the value of h~i(x) for every
xeX. That is, if [ui, v~] denotes the computed interval h~(X 1.... , X,), then

hii(x)E [[,li, vii

for all xeX.
Suppose we find v i < 0 for some value of i. Then h~i(x)<0 for every xeX.
Hence there is no point in X at which the real (non-interval) Hessian is positive
definite. Hence f is not convex and cannot have a minimum which is a
stationary point in X. Hence we can delete all of X except for any boundary
points of X ~~ which might lie in X.
When we evaluate h~(Xt . . . . . X,), we may find that the left endpoint u i > 0
for all i = 1,..., n. When this occurs, we know from inclusion monotonicity (see
[8]) that we will find each u~>0 for any sub-box of X. Hence we could save
some computational effort by noting when a box is a sub-box of one for which
u i > 0 for all i = 1. . . . . n. We would skip this test for such a box.
Note that an element h~ with arguments (X1 . . . . . X,) is not obtained when
we compute H(X) since the diagonal elements of H(X) have arguments different
from (X~, ..., X,) except for the element in position (n, n). Hence our test for
convexity requires recalculation of the diagonal of the Hessian.

6. The Interval Newton Method

For each sub-box X of X (~ that our algorithm generates, we can apply an

interval Newton method to the gradient g. Such methods seek the zeros of g and
252 E. Hansen

hence the stationary points off. Such a method produces from X a new box or
boxes N(X). Any points in X not in N(X) cannot contain a zero o f g and can be
discarded unless they are boundary points of X (~
These methods, in effect, solve (3.5) for points y where g(y)=0. The first such
method was derived by Moore [8]. Variants of Moore's method can be found in
[3, 8, 12, 13]. The most efficient variant is described below. Krawczyk's method
[8] is a suitable alternative to the method in [6]. Discussions of Krawczyk's
method can be found in [10] and [11].
We now give a brief synopsis of our method. We wish to solve the set of
equations
g(x) + J(~)(y-x)- 0 (6.1)

for the set of points y obtained by letting ~ range over X. We shall find a subset
Y of X containing this set.
Let Jc be the matrix whose element in position (i,j) is the midpoint of the
corresponding interval element J~j(X) of the Jacobian J(X). Let B be an
approximate inverse of Jc. As pointed out in [3], a useful first step in solving for
Y is to multiply (6.1) by B giving

Bg(x) + BJ(~.)(y- x) = 0. (6.2)

Note that the product BJ(~) approximates the identity matrix. However it may
be a very poor approximation when X is a large box.
We 'solve' (6.2) by a process similar to a single sweep of the Gauss-Seidel
method. Write
BJ(X)=L+D+U
where L, D, and U are the lower triangular, diagonal, and upper triangular part
of BJ(X), respectively. The interval matrix

D- 1 =diag [1/D 11, 1/D22 . . . . . 1/D,,] (6.3)

contains the inverse of every matrix in D. The box Y'solving' (6.2) is obtained as

Y= x ~ D - ~ [B g (x) + L ( Y - x) + U (X - x)]. (6.4)

When obtaining the component Y~ of Y, the components Y1, ..., Y~-1 appearing
in the right member of this equation have already been obtained.
This formulation presupposes that the intervals D~ (i=1 . . . . . n) do not
contain zero. When X is a small box, BJ(X) is closely approximated by the
identity matrix and hence D is also. However, for X large, it is possible to have
0eD u for one or more values of i. This case is easily handled. We simply use an
extended interval arithmetic which allows division by an interval containing
zero. A detailed discussion of this new method will be published elsewhere.
Note that we cannot allow the Newton procedure to delete boundary points
of X ~~ since the global minimum need not be a stationary point if it occurs on
the boundary. We discuss this point further in Sect. 10.
Global Optimization Using Interval Analysis 253

If we were to use this Newton method only, we would in general find

stationary points of f which were not minima. Moreover, we would find local
minima which were not global minima. To avoid this, we use an additional
procedure to delete points where f exceeds the smallest known value ~ This
procedure is described in the next section.
In some applications, it may be desirable to find all the stationary points o f f
in a given box. This can be done using the Newton method alone or in
conjunction with the monotonicity check of Sect. 9. If, in addition, the convexity
check of Sect. 5 were used, all stationary points except maximum would be
found.

7. Boundingf

We now consider how to delete points y~X where we know f ( y ) > f and hence
where f ( y ) is not a global minimum. We retain the complementary set which is a
sub-box (or sub-boxes) Y c X wherein f ( y ) may be <j~
As pointed out in [5], if we only wish to bound f * and not x*, we can delete
points where
f(y)>f -e I (7.1)

for some e 1 >0. We can allow e 1 to be nonzero only if we do not need to know
the point(s) x* at which f is globally minimum.
We want to retain points where (7.1) is not satisfied. F r o m (3.1), this is the
case for points y if

f(x) + ( y - x) T g(x) + 89(y - x) r H({)(y - x) < f - el

because the left member equals f(y). Denote

E = f - f ( x ) - e 1.
Then
~r g(x) + 89 H(~) ~ < E (7.2)

where ~ = y - x . We shall use this relation to reduce X in one dimension at a

time to yield the sub-box(es) Y resulting from deleting points where f ( y ) > f - ~ 1 .
We shall illustrate the process for the case n = 2. The higher dimensional case
follows in the same way. For n = 2, (7.2) becomes

~191(x)d-~Y292(x)-~89 hll(~)~-~l~Y2h21(~)-F~Y2h22(~)J~E. (7.3)

We first wish to reduce X in the xl-direction. Thus we solve this relation for
acceptable values of Yl. After collecting terms in y~, we replace Y2 by X 2. In the
higher dimensional case we would also replace Yi by Xi for all i = 3 .... , n. We
also replace ~ by X (since ~eX). We obtain

~1[gl(x)+89189 hll(X)+ X292(x)+89163h22(X)-E~O (7.4)

where X 2 = X 2 - x 2 .
254 E. Hansen

We solve this quadratic for the interval or intervals of points Yl as described

below. Call the resulting set Z s. Since we are only interested in points with
yleX~, we c o m p u t e the desired set Ys as I11 = X a c~Z~.
For the sake of argument, suppose II1 is a single interval. We can then try to
reduce X 2 the same way we (hopefully) reduced X1 to get Ys. W e again rewrite
(7.3). This time we replace Yl by Y~ and (as before) ~ by X. We could obtain
better results by replacing ~ by II1 rather than X 1 but this would require re-
evaluation of the elements of H. We obtain

-z h22(X) + f's gl (x) + 89f'? hls(X)-E<O=

.v2 [g2(x) + 89 J~s h2 ~(X)] +~Y2' (7.5)

where 91 = IIi - x l .
If the solution set Y2 is strictly contained in X 2, we could replace X 2 by Y2
in (7.4) and solve for a new Y1- We have not tried to do this in practice. Instead,
we start over with the box Y in place of X as soon as we have tried to reduce
each X i to Y~(i = 1.... , n). N o t e this means we re-evaluate H(X).
We now consider how to solve the quadratic equation (7.4) or (7.5). These
have the general form
A + Bt + Ct2 <O (7.6)

where A, B, and C are intervals and we seek values of t satisfying this inequality.
D e n o t e C = [c 1, c2] and let c be an arbitrary point in C. Similarly, let asA
and beB be arbitrary. Suppose t is such that (7.6) is violated; that is Q ( t ) > 0 ,
where
Q(t)=a+bt +ct 2.

If this is true for c=c~, then it is true for all c6C. Hence if we wish to find the
c o m p l e m e n t a r y values of t where (7.6) might hold we need only consider

A + B t +c 1 t2~0. (7.7)

If c~ = 0 , this relation is linear and the solution set T is as follows: D e n o t e A

= [as, a2] and B = [bl, b2]. Then the set of solution points t is

[-al/b2, ~ ] if a 1 _<0, b 2 < 0 ,

[-al/bs, ~] if as>O, bs <O, b2 <O,
[-~, ~] if as<O, bl<O<b2,
T= [-~,-aa/bz]w[-al/bs,~ ] if ax>O, bl<O<b2,
[-~, -al/bl] if a s<0, b s>0,
[-- ~ , -ax/b2] if a I > 0 , b s > 0 , b 2 > 0 ,
e m p t y set if a s > 0 , b 1=b 2 = 0 .

Recall that we will intersect T with X i for some value of i. Thus although T m a y
be unbounded, the intersection is bounded.
If c1:# 0, the quadratic (7.6) m a y have no solution or it m a y have a solution
set T c o m p o s e d of either one or two intervals. In the latter case, the intervals
m a y be semi-infinite. However, after intersecting T with Xi, the result is finite.
Global Optimization Using Interval Analysis 255

Denote
Ql(t)=a+bt + q t 2
where aeA, bEB, and c 1 is the left endpoint of C. We shall delete points t where
Q a ( t ) > 0 for all a~A and beB. Thus we retain a set T of points where Ql(t)=<0,
as desired. But we also retain (in T) points where, for fixed t, Q a ( t ) > 0 for some
aeA and beB and Qa(t)<O for other aEA and beB. This same criterion was
used to obtain T when c a = 0 . This assures that we shall always retain points in
Xi where f(x) is a minimum.
Denote
qa(t)={aa+b2t+clt2 if t < 0 ,
a,+bat+cat 2 if t > 0
and
qe(t)={a2+btt+Cat2 if t=<0,
a2+b2t+cit 2 if t>=0.

Then we can write the interval quadratic as

Q1 (t) = [aa, a2] q- [b I, be] t + C 1 t 2

= [q, (t), q2 (t)].

Thus for any finite t, qa(t) is a lower b o u n d for Qa(t) and q2(t) is an upper b o u n d
for Ql(t) for any aeA and any beB.
F o r a given value of t, if q l ( t ) > 0 , then 0 1 ( 0 > 0 for all aeA and beB. Hence
we need only to solve the real quadratic equation qa ( t ) = 0 in order to determine
intervals wherein, without question, Q1 (t)> 0. This is a straightforward problem.
The function qa(t) is continuous but its derivative is discontinuous at the
origin when b1=t= b 2 which will generally be the case in practice. Hence we must
consider the cases t < 0 and t =>0 separately.
If c a > 0 , the curve qa(t) is convex for t=<0 and convex for t=>0. Consider the
case t=<0. If q~(t) has real roots, then Q a ( t ) > 0 outside these roots, provided
t < 0 . Hence, we retain the interval between these roots. We need only examine
the discriminant of q~ (t) to determine whether the roots are real or not. Hence it
is a simple procedure to determine which part (if any) of the half line t <=0 can be
deleted. The same procedure can be used for t >_0.
F o r c a < 0 , qa(t) is concave for t__<0 and for t>=0. In this case we can delete
the interval (if any) between the roots of q~(t) in each half line. The set T is the
complement of this interval. It is c o m p o s e d of two semi-infinite intervals.
In determining T for either the case c 1 < 0 or in the case c I > 0 , it is necessary
to know whether the discriminant of qa (t) is non-negative or not. D e n o t e

A a=b2-4al ca, A2=b2-4aa C 1.

These are the discriminants of qa (t) when t >__0 and t _-<0, respectively.
W h e n we c o m p u t e A a or A2, we shall make rounding errors. Thus we should
c o m p u t e them using interval arithmetic to b o u n d these errors. W h e n computing
A i = ( i = 1, 2), suppose we obtain the interval

A[=[A],Aff] ( i = 1 , 2).
256 E. Hansen

We use the appropriate endpoint of A~ or A~2 to determine T which assures that

we never delete a point t where Q~(t) could be non-positive. Thus we use the
endpoint of A~ or A~ which yields the larger set T.
When we compute the roots of ql(t), we shall make rounding errors. Hence
we compute them using interval arithmetic and again use the endpoints which
yield the larger set T to assure we do not delete a point in X~ where f is a
minimum.
For i = 1 and 2, denote
R + = ( - b ~ +_A1/2)/(2Cl)
and
S+ = 2a( - b i 4- A~/2).

Note that R~+ = S/- and R i- = S~+. As is well known, the rounding error is less if
we compute a root in the form R~+ rather than in the form S 7 when b i <0. The
converse is true when bi > 0. Similarly, the rounding error is less when using R~-
rather than S~+ when bi>0. Hence we compute the roots of q~(t) as R~+ and S~
when b i < 0 and as R~- and S~- when bi>0.
Note that computing R~ or S~ involves taking the square root of the
interval A[. In exact arithmetic this would bc the real quantity Ai. We would
never be computing roots of q~(t) when A~ was negative. Hence if we find that
the computed result A[ contains zero, we can replace it by its non-negative part.
Thus we will never try to take the square root of an interval containing negative
numbers.
Given any interval 1, let I L and I R denote its left and right endpoint,
respectively. We use this notation below. Using the above prescriptions on how
to compute the set T, we obtain the following results:
For b~ __>0and c I >0,

[0 (the empty set) if A2R<0,

T= ~[(R;) L, ($2) R] if a l > 0 and A~>0, (7.8)
[[(R~)L, (Si-)R] if a~ <0.

For b 2 < 0 and c1>0,

if A~<0,
T = [ E ( S ~ ) , (R~-)n] if a l > 0 and A~>0, (7.9)
2 ) , ( R ~ ) R]
[ [ ( S+L if a 1 <0.

For b 1 <0=<b 2 and c 1 >0,

r if max(A~, A2g)<O,
[(R2)L, (S~-)R] if [bl[<b 2 and min(A~, A2g)__<O
=<max (A~, A2R),
T= [(S~-)L, (R~-)R] if [blI>b 2 and min(AIR, A2R)_--<O(7.10)
____max(A~, A2R),
[(R; )L, (S ~-)R] u [(S +)g, (R ~-)R] if a l > O and min(A~,A2R)>O
E(Ry )L, (R ~-)R] if a 1 <0.
Global Optimization Using Interval Analysis 257

For b ~ > 0 and c~<0,

[ - o0, (s2)R]w[(R~) L, oO] if a 1 >0,

T= [--oo,(s~)R]w[R~)L, oo] if a~ <O<A L, (7.11)
[--oe, oo] if AL<0.

For b z < 0 and c1<0,

{
[--oe,(Rf)g]w[(S-~) L, ~ ]
T= [--oQ,(R~)R]~[(S~)L, oO]
[ - - ~ , oe]
if a l > 0 ,
if al <O<A~,
if AL<0.
(7.12)

For b l < 0 < b 2 and c 1 <0,

T=~E--oo,(S[)R]m[(S~{)L,].[ o0] if al>O,

(7.13)
o0, o0] if a 1<0.

Note that if c1>0, then A 2 c a n be negative only if a l > 0 . Hence the

condition A z < 0 implies a 1 > 0. This, and similar cases, has been used to shorten
the conditional statements in the above expressions for T.
We have seen that the solution of the quadratic inequalities such as (7.4) or
(7.5) can be an interval Z i or two semi-infinite intervals, say ZI 1) and Z~z). The
desired solution set Y~is obtained by intersection with Xi. In the former case, Y~
= X~ c~Z~ can be empty or a single interval. In the latter case,

can be empty, a single interval, or two intervals. We now consider the logistics
of handling these cases.
The quadratic inequality to be solved for Z~ will have quadratic term
1"2hii(X ) so the interval C in (7.6) is 89
gY~ ) and the left endpoint is c]i)
= [89 L. If c(/)>0 the solution set is a single interval. But if c]~ it is two
semi-infinite intervals and it may be that Y~ will be two intervals. This would
complicate the process of finding Yi+ 1,-.., Y,. Thus we proceed as follows.
Let 11 denote the set of indices i for which c]~)>0 and I z denote the set of
indices i for which c]i~<0. We first find Y~for each i~11. We then begin to find Y~
for ieI 2. L e t j e I 2 be such that Y~is composed of two intervals, say y)l) and y)2).
Then Xj is the smallest interval containing both y)l~ and y)2). When finding Y~
for the remaining values of i, we use Xj in place of Yj.
After finding all Y~for i = t .... , n, we wish to use the fact that we can delete
the interval, say Yf, between y)n and y)2). We would like to do this for all the
values of j for which Y~ was two intervals. However, it could be that this
occurred for all the indices j = 1, ..., n. After deleting the interior interval yjc in
each dimension, the resulting set would be composed of 2" boxes. For large n,
this is too many boxes to handle separately. Hence we delete only a few (one,
two, or three) of the' largest of the intervals Yr. We then process each of the new
boxes separately.
258 E. Hansen

We would like to prevent the generation of long, narrow boxes. Thus a good
choice of which yjc to delete is the one(s) corresponding to the component for
which the smallest interval containing both Y~(1)and YS2) is largest. However, we
have chosen to delete the largest interval Yr.
Let us call the process we have described in this section the quadratic method.
We can combine the quadratic method with the Newton method. It is desirable
to do this as we now explain.
If the left endpoint of Hu(X ) is negative, then the quadratic method can give
rise to two new intervals y m and Yi(2) in place of X i. When trying to improve
Xi+ 1 (say), it is impractical to use y(1) and y(2) separately and we use X i,
instead. Thus the improvement of Xi is of no help when trying to improve Xi+ 1,
etc. Similarly, when applying the Newton step, if Ju(X) contains zero as an
interior point, we can obtain two subintervals in place of X~. Again, we cannot
conveniently use this fact in the remaining part of the Newton step.
We would like to do those steps first which are of help in subsequent steps.
Hence the following sequence is suggested. First try to improve X~ by the
quadratic method for each value of i = 1.... , n for which the left endpoint of
H , ( X ) is positive. Then apply the interval Newton method to the (old or new)
components for which O6BJu(X ) (i = 1, ..., n). Next use the quadratic method for
those components for which the left endpoint of Hii(X ) is non-positive. Finally,
complete the Newton step for those components with O~BJu(X ).
At each stage of either method, when trying to improve the i-th component
of the box, we use the currently best interval for the other components. This
may be the smallest interval containing two disjoint intervals in some cases. In
fact it would be possible for the quadratic method and the Newton method to
each delete disjoint sub-intervals for a given component. This would give rise to
three sub-intervals to be retained. However, it seems better to simplify this case
and only delete the larger of the two sub-intervals.
When both methods are completed, we may have several components
divided into two sub-intervals. If so, we find the one for which the largest
interior sub-interval has been deleted. We replace all the others by the smallest
sub-interval containing the two disjoint parts. We then divide the remaining
part of the current box into two sub-boxes by deleting the sub-interval for the
component in question. We could do this for more than one component, but
each deletion would double the number of boxes. It seems better to keep the
number of boxes small.

8. Choice of el

Suppose we want to bound the value f * of the global minimum to within a

tolerance ~1 but we do not care w h e r e f takes on this value. Then, as pointed out
in Sect. 7, we can delete points y where

f(y)>f-~l. (8.t)
Once we have found a point )~ where f(~2)=f is such that f - f * < = e l , our
algorithm will eventually delete all of X <~ if we use (8.1). However, ~ may be far
Global Optimization Using Interval Analysis 259

from the point x* where f is globally minimal. When all of X (~ is deleted, we

will know that
f - e 1 =<f* __<f
Choosing el > 0 will speed up our algorithm. However, if we wish to obtain
good bounds on x*, we must choose el =0. We then terminate our algorithm
when the remaining set of points is sufficiently small. See Sect. 13 for a
termination procedure.

9. Monotonicity

Another step in our algorithm makes use of the monotonicity off. Suppose, for
example, the i-th component gi(x) of the gradient is non-negative for all x e X .
Then the smallest value of f ( x ) for x e X must occur for xg equal to the left
endpoint of X i.
To make use of a fact such as this, we evaluate gg(X 1..... X,). The resulting
interval, which we denote by [a i, cog], contains g/(x) for all x e X . Denote X i
=Ix/L, xiR]. If a i > O , f ( x ) is smallest (in X) for x i = x ~. Hence we can delete all of
X except the points with xg=x~. If ag>O,f(x) cannot have a stationary point in
X. Hence we can delete all of X unless the boundary at x i = x ~ contains
boundary points of the initial box X (~ Similar results occur if coi __<0 or if cog< 0.
We evaluate gg(X~ . . . . . Xn) for i = 1.... , n and reduce the dimensionality of X
for any value of i for which crg__>0 or cog<0. Of course, we delete all of X, if
possible.
It is possible that we can reduce X in every dimension in this way. If so, only
a single point, say 2, remains. In this case, we evaluate f(2). I f f ( 2 ) > ~ we can
eliminate ~ and hence all of X is deleted by the process. Iff(k)_<_f, we reset f
equal to f(2). In this latter case, X is again deleted; but we store 2 for future
reference.

I0. Boundary Points

The process just described in Sect. 9 can sometimes eliminate points of the
boundary of X (~ which lie in X. Suppose that for some X, we find
gj(X 1.... , X , ) > 0 for some j = 1. . . . . n. Then we can delete all of X except for
any boundary points of X (~ occurring at x i = x~. Any other boundary points of
X ~~ which are in X are thus deleted.
The quadratic method of Sect. 7 deletes any point x w h e r e f ( x ) > f whether x
lies on the boundary of X ~~ or not. However, the Newton method of Sect. 6 and
the procedure in Sect. 5 (which considers convexity) cannot delete any boundary
points of X (~
Suppose we apply a step of the Newton method to a box X and obtain a new
box X' contained in x . A simple way to proceed is to retain the smallest box
containing both X' and all boundary points of X (~ which are in X. This will
260 E. Hansen

generally save points of X outside X' thus reducing the efficiency of the
procedure. In fact, it may be that the smallest box containing the boundary
points of X ~~ which are in X is X itself. If this were the case, we would bypass
the Newton step for the box X. This approach would rely upon the methods of
Sects. 7 and 9 to delete boundary points of X (~
This same idea can be used for the method of Sect. 5. I f f is not convex in X,
we can simply replace X by the smallest (perhaps degenerate) box, say X,
containing the boundary points of X (~ which lie in X. In this case, either )( = X
or else X is a degenerate box of dimension less than that of X.
Suppose we are given a box X. For this approach, if ) ( = X, we do not apply
either the Newton method or the convexity test. We could use the Newton
method in this case also or we might bypass its use whenever X contains
boundary points of X (~
A more straightforward procedure is to simply express the boundary of X (~
as 2n separate (degenerate) boxes of dimension n - 1 . The interior of X (~ can
then be treated as a box wherein the global minimum must be a stationary
point. However, the (n-1)-dimensional faces of X (~ have (n-2)-dimensional
boundaries which must, in turn, be separated from the interiors, and so on.
Finally the vertices of X (~ would have to be separated. These vertices alone are
2" points. Even for moderate values of n, this separation process produces too
many (degenerate) boxes. Thus it is better, in general, not to try to separate the
boundaries from X (~
These two approaches represent extreme cases. Intermediate methods might
be used wherein the boundaries of X (~ in a given box X are separated off under
special circumstances.
It should, perhaps, be pointed out that the Newton method can delete
boundary points of X (~ under certain circumstances. Suppose our algorithm has
produced a degenerate box X which is all or part of a face of X (~ In this
degenerate ( n - 1)-dimensional box, the Newton method can delete points which
are not in the (n-2)-dimensional boundary of the face of X ~~ Such deleted
points are, of course, on the boundary of X (~
In some examples, we shall know a priori that the global minimum is a
stationary point. In this case we are free to delete boundary points by any of our
procedures.

11. The List of Boxes

When we begin our algorithm, we shall have a single box X (~ We apply the
four procedures described in Sects. 5, 6, 7, and 9 to this box. It is possible that
none of these procedures can delete any of X (~ If so, we divide X ~~ in half in a
direction of its m a x i m u m width. We put one of these new boxes in a list L to be
processed and work on the other. These and subsequent boxes may also have to
be subdivided, thus adding to the list L of boxes yet to be processed. If boundary
points are handled appropriately, the four procedures described in Sects. 5, 6, 7
and 9 can each produce more than one new sub-box; and all but one are added
to the list. Thus the number of boxes in the list tends to grow initially.
Global Optimization Using Interval Analysis 261

Eventually, however, the boxes become small and often a box is entirely
eliminated. Thus the number of boxes in the list eventually decreases to one, or
just a few, or to none at all when e 1 is chosen to be nonzero.

12. Subdividing a Box

In the initial stages of our algorithm, we shall be applying it to large boxes. For
example, we begin by applying it to the entire initial box X (~ Thus it could be
that, for a given box X, none of the procedures described in Sects. 5, 6, 7, and 9
can delete any of X. When this occurs, we wish to subdivide X.
We could subdivide each component X~ of X into two parts. But this would
give rise to 2" sub-boxes. To prevent generation of too many sub-boxes, we shall
divide only one component in half. It is best to subdivide the largest component
X~ to prevent generation of a long, narrow box.
Suppose we divide X~=[x~, x~] in half giving two new boxes X' and X"
whose i-th components are X' i = [x L, Xi] and X' i' = [)Zi, xR], respectively, where 97i
=(xL+xR)/2. The boxes X' and X" have a boundary in common at x i = ~ i. I f f
had a global minimum on this common boundary, we would subsequently find
it twice. This is unlikely to be the case. To avoid having the same points in two
boxes, we could define one of them in terms of a half open interval. Thus we
could define X ri - [ xLi, xi).
It is simpler to always use closed intervals. The extra work of keeping track
of whether an interval contains a given endpoint is probably not worth the
effort. In practice, we have elected to avoid this problem. Thus we have always
used closed intervals only. In general, this does not cause the algorithm to find a
given global minimum more than once.

13. Termination

If we have chosen e l > 0 , we can continue our algorithm until X (~ is entirely

eliminated. As pointed out in Sect. 8, we then have f * bounded to within a error
el. In this case, we do not obtain a bound on x*.
If e l = 0 , we cannot eliminate all of X ~~ since we always retain a box or
boxes containing the point(s) x* where f ( x * ) = f * . As pointed out above, we
might also retain a box or boxes w h e r e i n f has a value very near f * but no value
equal to f * .
Suppose that at some stage in our algorithm, the list L contains s boxes.
Denote these boxes by X (1), , X (s). Let X~.~ denote the interval defining thej-th
component (dimension) of X ii'). Let w(X~i); denote the width of the interval X~i)
and let
w i = max [w(X~~
1 <=j<=n

That is, w~ is the m a x i m u m dimension of X ").

262 E. Hansen

We could continue processing boxes in the list L by our algorithm until

i wi<e
i=1
(13.1)

for some ,~2>0. This is provided 52 is chosen large enough that the prescribed
precision is attainable using (say) single precision arithmetic. However, it is
more convenient computationally to require only that

w i < e2 (13.2)

for each i = 1.... , s. If e 1 >0, we set e 2 = 0 for convenience.

Thus whenever a new box X ") is obtained by our algorithm we can check
whether (13.2) is satisfied. If so, we no longer apply our algorithm to X (i~ (except
as discussed below). If X ") contains a point x* wh6re f is a global minimum,
then the location of x* is bounded. In fact, if x ") is the center of X {~) and (13.2)
holds for X ~~ then

Ix}')-x~'E__<~2/2 ( / = 1 . . . . , n).

Let s denote the number of boxes remaining and denote the boxes by X (~) (i
= 1. . . . . s). As a final step, we want to assure that f * is bounded sufficiently
sharply. We do this as follows.
For each i = 1.... , s we evaluate f(X")); that is we evaluate f with interval
arguments X~ i) (j = 1. . . . . n). The result, say [Fi L, F~] contains the range of f for
all x e X ~), but will not be sharp, in general. However, if e 2 was chosen to be
small, the interval result should be 'close to sharp' since e 2 is an upper bound on
the largest dimension of any box X(i); and the smaller X (1) is, the sharper
[F~L, F~] is. (See 1-8].) Therefore, it is generally not necessary to use special
procedures to sharpen the computed interval.
Since [FiL, F/R] contains the range o f f ( x ) for all x e X Ci), we have

FiL < f (x) < FiR

for any x s X "). Denote

_f= min FiL
l<_i<s
Then
f < f (x)
for any x in any of the boxes X (1) (/= 1, ...,s). Therefore, since any global
minimum must occur at a point x* lying in one of the .boxes X (1), we have
f < f * . But also f * < f (as discussed above) and hence

f <f*<f (13.3)

We thus have bounds on f * . However, they may not be sharp enough since
our specified requirement is to bound f * to within, say, ~0; and it may be that f
- f > e 0. If this is the case, we shall improve our bounds. To do this, we find a
Global Optimization Using Interval Analysis 263

value j for which f =fi L. We apply our main algorithm to X (J). This will increase
fjL, in general. It might also decrease 3~ For exact interval arithmetic, this must
decrease f - - f j L since X ~J) will be reduced in size (even if it is merely subdivided).
Repeating this step for each j such that f=fjL, we must decrease f - f (at least,
if exact interval arithmetic is used) and hence eventually have

f - - f "(/30
so that f * is bounded to sufficient accuracy since (13.3) holds.
Because of rounding errors, we cannot reduce f - f arbitrarily, in practice.
Hence we assume eo is chosen commensurate with achievable accuracy using
(say) single precision arithmetic.
We also require that

for each box X ") (i = 1. . . . . s). For convenience, we can choose e 3 = eo to reduce
the number of quantities to be specified. Note that F~L<f since otherwise
f(x) >f for all x~X (i) in which case X (i) can be deleted. Hence
F/R ~-~f-~- e3 ~f-~- ~;0"~/33 < f * + ~o + g3
for every i = t .... , s. That is, every remaining box contains a point x at which
f(x) differs from f * by no more than eo+e3.

14. The Steps of the Algorithm

We now describe the steps involved in our algorithm. Initially, the list L of
boxes to be processed consists of a single box X (~ In general, divide the list L
into the list L 1 of intervals X (~ satisfying the condition wi < e2 (see (13.2)) and a
list L 2 which do not satisfy this condition.
We assume we have evaluated f at the center of X (~ and thus obtained an
initial value for )~ The subsequent steps are to be done in the following order
except as indicated by branching:
(1) Of the boxes in L2, choose one which has been in L 2 longest. Call it X. If
L 2 is empty, go to step (11) if e l > 0 . If L 2 is empty and e l = 0 choose a box
which has been in L1 longest and go to step 2. If both L 1 and L 2 are empty,
print the b o u n d s f - ~ l a n d f on f * and stop.
(2) Check for monotonicity. Evaluate g(X) as described in Sect. 9. For i
=1 .... , n, if gi(X)>O ( < 0 ) and the boundary of X at xi=x L ( = x R) does not
contain a boundary point of X (~ delete X and go to step (1). Otherwise, if
g~(X)>0 (_-<0), replace Xi= [X i, L x~] by [x~, x L] ,Lfl-XRi,XR3~J,"Rename the result X

again.
(3) Test for non-convexity as in Sect. 5. Let X' denote the smallest box in X
containing all the boundary points of X ~~ which lie in X. If X' -- X go to step 4.
Otherwise, evaluate h,(X~ . . . . . X.) for i = 1 , ..., n. If the resulting interval is
strictly negative for any value of i, replace X by X'. If X' is empty, go to step 1.
If X' is not empty, put it in the list L and go to step 1.
264 E. Hansen

(4) Begin use of the quadratic method of Sect. 7. For those values of i
=1 ..... n for which the left endpoint of Hii(X) is non-negative, solve the
quadratic for the interval Y~to replace X i. Rename the result X i.
(5) Begin use of the Newton method. For those values of i = i, ..., n for
which Oq~[BJ(X)]u, solve for the new interval to replace X~. Rename the result
X~. For a given value of i, omit this step if a reduction of X i will delete
boundary points of X (~ from the box X.
(6) Complete the quadratic method. For those values of i not used in step 4,
solve the quadratic for Y~. If Y~is a single interval, replace X~ by Yi, renaming it
X i. Otherwise save Y~for use in step 8.
(7) Complete the Nowton method. For those values of i not used in step 5,
solve for the new set (say) Yj. For a given value of i, omit this step if a reduction
of X i will delete boundary points of X C~ from the box X. If Y/ is a single
interval, replace Xi by Y/, renaming it X~.
(8) Combine the results from the quadratic and Newton methods for those
components X i for which both methods divided X~ into two sub-intervals. That
is, find the intersection Y/' of Y~ from step 6 and Y~' from step 7. If Yj' is
composed of three intervals, replace it by either Y~ or Y/, whichever has the
smallest intersection with X i. Of all the Y~",save the one (or two or three) which
deletes the largest subinterval of X i. That is, save that Y/' whose complement in
X i is largest. Let j be its index. For all relevant values of i ~j, replace Y~" by X~,
that is, ignore the fact that part of X i could be deleted.
(9) If Yj' exists; that is, if at least one interval Xj was divided into two sub-
intervals, say YS') and i1(2), subdivide the box X into two sub-boxes. These sub-
boxes will have the same components X~ as X except one will have j-th
component yj(1) and the other will have j-th component YS2~. If no such Yj'
exists, we may wish to subdivide the current box. Let X denote the box chosen
in step 1 and let X" denote the current box resulting from applying steps 2
through 8 to X. If the improvement of X" over X is so small that (say)

w(X") >0.75 w(X),

then divide X" in half in its greatest dimension.

(10) Evaluate f at the center of the box or boxes resulting after step9.
Update f as described in Sect. 4. Put the box(es) in the list L and go to step 1.
(11) Evaluate f ( X ~~ for each remaining box X ~) in L. Denote the result by
[F~L, FIR]. If FiR--FL>~o for any value of i, use X (') for X and go to step 4. If FiR
--FiL < eo for all i = 1. . . . . s, find
f = rain F~.
l<=i<_s

Then print the bounds f and f on f * and stop.

For e I >0, these steps bound f * to within an error e,. If e, =0, they found f *
to within eo; they bound x* to within ea; and they assure that for any point x in
any final box, f(x) exceedsf * by no more than eo + e3.
In this step, we sometimes branch to step 4. Note that we could go to step 2,
but it is unlikely that either step 2 or step 3 will be helpful. This is because we
expect each box remaining at this stage to contain a minimum.
Global Optimization Using Interval Analysis 265

15. A Numerical Example

We now illustrate the steps of our algorithm. We shall consider the so-called
three hump camel function.

f (x)=2x~_ l.O5x~ + ~x 16_ x l x2 + x 2 (15.1)

which has three minima and two saddle points. The gradient g(x) has com-
ponents
gl (x)= 4 x l - 4.2 x~ + x ~ - x 2,
g2(x)=2X2--Xl. (15.2)
The interval Jacobian J(X) (see Sect. 3) has elements

J I ~ ( X ) = 4 - 1 2 . 6 X 2 + 5 X 4,
J12(X)=J2t(X) = - 1,
Jzz(X)=2. (15.3)

As described in [41, a better formulation for J(X) could be derived which

would give smaller intervals, in general. However, we shall use the simpler form
given here.
Suppose that the box we choose in step 1 has first component X 1 =[1, 1.11.
In step 2 we find that, whatever X 2 is,

hll(X1, X2)= [ - 5.196, -2.55].

Since this is strictly negative, we know that f does not have a minimum in the
interval X1 for any value of x 2. Hence if X does not contain a boundary point of
X ~~ we can delete all of X.
Now suppose the box chosen in step l has components X 1 = [2, 3] and X 2
= [0, 1]. We find

h ~ (X1, X2)= [ - 29.4, 358.6], hz2(X ~, X2)= 2.

Since neither interval is negative, we cannot say that f is not convex in X.

Hence we go to step 3.
In step 3, we evaluate g(X) obtaining

gl(X) = [ - 74.4, 221.4], g2(X) = [ - 3, 01.

We see that g2(x) is non-positive for all x e X and hence f is smallest in X for
X2=I. Thus we can replace X by the degenerate box X' with components
X'1 = [2, 31 and X 2 = [1, 11.
If the box X had components X1 = [0, l] and X 2 = [2, 3], we would have
obtained
g l (X) - [ - 7.2, 33, g2 (X) = [3, 63.

In this case g2(x) is strictly positive and we can eliminate all of X unless the
boundary of X at x 2 = 2 contains a boundary point of X w). Suppose X~~ [0, 1]
266 E. Hansen

and X~~ 3]. Then X (~ has boundary points at x 2 = 2 for x~ = 0 and 1. We

could thus delete all of X except the points (0, 2) and (1,2). This is simple to do
in this two-dimensional problem. In higher dimensions, it might be simpler to
retain the entire boundary at x 2 = 2.
Now suppose that X is given by X~ = X 2 = [ 0 , 1]. Then g l ( X ) = [ - 5 . 2 , 5] and
g 2 ( X ) = [ - 1 , 2] so that we do not have monotonicity. Therefore, we do step 4
which involves the quadratic method of Section 7.
For this box, we obtain

Hll(X)=JlI(X)=[-8.6,9], H21(X)=2J21(X)=-2, H22(X)=J22(X)=2.

The center of the box is at x=(0.5, 0.5). We wish to evaluate f(x) and g(x). We
cannot obtain f(x) exactly using finite precision decimal arithmetic. Let us use
five significant decimal digits and evaluate f(x) using interval arithmetic to
bound rounding errors. Thus we replace the coefficient 1/6 by [0.16666, 0.16667]
and obtain
f(x) = [0.43697, 0.43699].
We also obtain
[[1.0062, 1.0063]]
g(x) = t 0.5 J"

Suppose we have previously obtained f = 0 . 2 and that we choose e~ =0. To

do step 4, we wish to solve (7.2) for points yeX where we know that f(y)>f
does not hold and hence f(y) < f might hold. If we first tried to solve for Y1, we
would rewrite (7.2) in the form (7.4). However, the left endpoint of Hx~(X) is less
than zero. Hence, solving (7.4) would give rise to two semi-infinite intervals.
Therefore, we defer this operation until step 6 and first solve for Y2 which will be
a single interval since the "left endpoint" of Hz2(X ) is positive.
We solve for Y2 using (7.5). As we have not yet solved for Y1, we use X I in its
place. Substituting into (7.5), we obtain

[ - 1.2412, 1.9652] + [0, 1] Y2 +~2 <0.

From Eq. (7.8), the solution set is

2 2 = [ - 1.7212, 1.1141].
Hence
Z2 = x 2 ']- Z2 = [ - 1.2212, 1.6141]
and
Y2-~-Z2usX2.=X2 .

Thus we have not deleted any of X 2.

Next we do step 5 which applies that part of the Newton method which
generates a single interval. The interval Jacobian is
Global Optimization Using Interval Analysis 267

The center of this interval matrix is

whose inverse is (approximately)

[-3.3333 - 1.6667 ]
B = I_- 1.6667 -0.33333J"

For simplicity of exposition, we shall compute B J ( X ) explicitly. We obtain

from (6.2) (with ~ replaced by X),

[[[-4.1877,-4.1872]q I-[-28.334,30.334] -0.0001-1

-1.844, - 1.8436]] + I_[- 14.668, 14.668] [1, 1.0001]J ( y - x ) = 0 . (15.4)

We try to improve X z first rather than X 1 because [BJ(X)]11 contains zero

while [BJ(X)]z z does not. Thus the first equation of (15.4) gives rise to two new
intervals while the second equation does not.
The second equation is

[ - 1.844, - 1.8436] + [ - 14.668, 14.668] (X 1 - x l ) + [1, 1.0001] (Y2 - x 2 ) = 0

where we have replaced Yl by X1. Solving for Y2, we obtain the interval

Z 2 = [-4.9904, 8.678].

The intersection Y z = Z 2 ~ X z equals X 2 so again no improvement has been

made.
Step 6 prescribes that we use the quadratic method to try to improve those
components of X not solved for in step 4. We solve (7.5) for points Yl where
f ( y ) < f We would use Y2 in place of X2; but they are equal. Substituting into
(7.5), we obtain

[0.08697, 0.83699] + [0.5062, 1.5063] Yl + [ - 4 . 3 , 4.5] ~ <0.

Using equation (7.11), we find that this quadratic has the solution set

Z1 = [ - oo, -0.10093] ~ [0.42555, oo].

Thus
Z 1 = 2 1 + x 1 = [ - oo, 0.39907] w [0.92555, oo]
and
Y~= Z~ w X 1 = [0, 0.39907] u [0.92555, 1]

note we have eliminated a subinterval of length 0.52648 from X 1.

In step 7, we use the Newton method to try to improve X~. We solve the first
equation of (15.4),

[ - 4.1877, - 4.1872] + [ - 28.334, 30,334] (y ~ - x ~)- 0.000t (X z - xz) = O,

268 E. Hansen

where we have now replaced I12 by X 2. Solving for Yl, we obtain the two semi-
infinite intervals

Z~3)= [ - o% 0.35223], Z~()= [0.63803, oo].

Their intersections with X 1 are (say) y~3) and y(4) where

u [0, 0.35223], Y1(4) = [0.63803, 1].

We wish to combine the results obtained using the quadratic method and the
Newton method. Thus we retain the intersection of
Y(I)w Y1(2) and Yt3)wY1~4)
which is
[0, 0.35223] t~ [0.92555, 1].

We have deleted a substantial portion of the original box X. The remaining

points compose the two boxes

0.35223]] [[0.92555,
[[0'[0,1 ] ] and [ [0,1] 1]].

In subsequent steps, our method would be applied separately to each of these

boxes.

16. Computational Results

We now describe some computational results obtained using the algorithm

described above. The computations were done on the Amdahl 470V/6-II com-
puter. In each case, we assumed it was known that the global minimum occurred
in the interior of the initial box. This speeds up the algorithm since boundary
points need not get special treatment.
This is consistent with the fact that we are really considering the uncon-
strained case. We intend to treat the constrained case in a later paper.
We give results for only one example which typifies the problems in two and
three dimensions which we have used. The example is the three hump camel
function given by (15.1).
This function has its global minimum at the origin. It has two local minima
at approximately [+1.75, +0.87] and two saddle points at approximately
[_+1.07, _+0.535]. Our initial box was defined by X l = X 2 = [ - 2 , 4 ] which
contains all these points. We chose e 1 = 0 and e0 =t32 =•3 = 10-4.
We find that after eight steps of our algorithm, we have six sub-boxes in our
list. In the next step a sub-box is entirely eliminated and after the fifteenth step,
only one sub-box remains but its width exceeds e2. After an additional step, we
obtain final box

X = [ [ _ 2 . 9 1 x 1 0 _ v , 3.56 x10_6]].
Global Optimization Using Interval Analysis 269

Here and in the following, we record results to only three decimals. This box
satisfies the error criterion requiring its width to be less than %. Evaluating f at
the center of this box, we obtain f = 1.12 x 10- lO
As prescribed in step 11, we evaluate f(X) and obtain [ - 1.24 x 10-11, 7.57
x 10-1~ Thus _f = - 1.24 x 10 - 1 1 and

f*~[_f, f ] = [ - 1.24 x 10 -11, 1.12 x 10-lo].

Since f - f < %, we have f * bounded to the prescribed tolerance. If we approxi-

mate f * by

(f+f)/2=4.98 x 10- 11,

then we know that the error is at most 6.22 x i0 11 in magnitude.
If we approximate x* by the center (3.27 x 10 -6, 1.63 x 10-6), then we know
that the error in x* is less than 3.85 x 10 . 6 and the error in x* is less than t.93
x 10 -6.
We have obtained x* to far more accuracy than required because of the
rapid rate of convergence of the interval Newton method used. The bound on
f * is much better than required simply because a given error bound on x*
automatically yields a much better bound on f * .
We also used this example with an initial box of width 2 x 106. This case
required 46 steps to run to completion. This illustrates that if we use a very large
box to assure containment of x*, the computing time need not increase
drastically.

17. Conclusion

We have presented an algorithm for solving the unconstrained minimization

problem assuming we have an initial box which is known to contain the
minimum.
It would certainly be possible to construct a highly oscillatory function for
which our algorithm would be prohibitively slow. However, it has converged
adequately rapidly for the test problems on which we have tried it. (See Sect. 16.)
We have assumed f(x)eC 2. The global minimization problem can also be
solved for f(x)EC 1. In this case, the Newton method cannot be used. The
quadratic method can be replaced by a corresponding linear method in which
we find points y at which f ( y ) < ~ This is done by noting that ifx~X and y~X,
then

f(y) = f ( x ) + ( y - x) T g(~)
for some ( e X . Thus we can solve for the approximate points y from

f (x) + (y- x) T g(X) <

For the problems of low dimension on which we have used this method, it was
less efficient then the quadratic method described in Sect. 7. We do not know the
relative efficiencies for large n.
It is possible to solve the global optimization case when f(x) is only
270 E. Hansen

c o n t i n u o u s b u t n o t differentiable. H o w e v e r , o u r a l g o r i t h m is v e r y slow. It e n t a i l s
a different a p p r o a c h t h a t w e h o p e to d e s c r i b e in a n o t h e r p a p e r .
T h e n o n l i n e a r c o n s t r a i n e d o p t i m i z a t i o n p r o b l e m c a n also b e s o l v e d by
i n t e r v a l m e t h o d s . A n e x t e n s i o n o f o u r a l g o r i t h m is r e q u i r e d . O u r e x p e r i e n c e in
this case is for h a n d c a l c u l a t i o n s only. A difficulty exists (currently) w h e n it is
difficult to find a p o i n t in the n e i g h b o r h o o d of x* w h i c h is w i t h o u t q u e s t i o n ,
feasible.
O n e o f the v i r t u e s o f i n t e r v a l a r i t h m e t i c is t h a t it is u s u a l l y p o s s i b l e to
f o r m u l a t e a n i t e r a t i v e a l g o r i t h m in s u c h a w a y t h a t it s t o p s a u t o m a t i c a l l y w h e n
t h e best p o s s i b l e result h a s b e e n o b t a i n e d for t h e finite p r e c i s i o n a r i t h m e t i c used.
W e p l a n to d o this for o u r a l g o r i t h m a n d t h u s p r e c l u d e t h e n e e d for specifying
t30' gl, ~2, a n d e 3.

Acknowledgments. The author is greatly indebted to Thomas Kratzke and Saumyendra Sengupta for
their assistance in programming and debugging the computer program with which the data in this
paper was obtained.
This research was supported by the U.S. Air Force Office of Scientific Research under grant
F49620-76-C-0003. The United State Government is authorized to reproduce and distribute reprints
for governmental purposes notwithstanding any copyright notation hereon.

References

1. Dixon, L.C.W., Szeg6, G.P.: Towards global optimization. Amsterdam: North Holland, 1975
2. Dixon. L.C.W., Szeg6, G.P.: Towards global optimization 2. Amsterdam: North Holland, 1977
3. Hansen, Eldon: On solving systems of equations using interval arithmetic. Math. Comp. 22, 374-
384 (1968)
4. Hansen, Eldon: Interval forms of Newton's method. Computing 20, 153-163 (1978)
5. Hansen, Eldon: Global optimization using interval analysis - the one-dimension case. Jour.
Optimiz. Theo. Applic. 29, 331-344 (1979)
6. Hansen, Eldon, and Roberta Smith: Interval arithmetic in matrix computations, II. SIAM J.
Num. Anal. 4, I-9 (1967)
7. Krawczyk, R.: Newton-Algorithmen zur Bestimmung von Nullstellen mit Fehlerschranken.
Computing 4, 187-201 (1969)
8. Moore, R.E.: Interval analysis. New York: Prentice-Hall (1966)
9. Moore, R.E.: On computing the range of a rational function ot n variables over a bounded
region. Computing 16, 1-15 (1976)
10. Moore, R.E.: A test for existence of solutions to nonlinear systems. SIAM J. Num. Anal. 14,
611-615 (1977)
11. Moore, R.E.: A computational test for convergence of iterative methods for nonlinear systems.
SIAM J. Num. Anal. 15, 1194-1196 (1978)
12. Moore, R.E.: Methods and applications of interval analysis. Philadelphia: SIAM (1979)
13. Nickel, Karl: On the Newton method in interval analysis. Mathematics Research Center Report
1136, University of Wisconsin (1971)
14. Skelboe, S.: Computation of rational interval functions, BIT 14, 87-95 (1974)

Received June 12, 1978