J. F.

TRAUB, Editor

The Simplex Method of Linear factored form as a product of transformations, and then
applying it to the right-hand sides of the systems.
Programming Using LU The basis matrix of any simplex step differs from t h a t
of the preceding step in only one column, so it is easier to
Decomposition update P-1 than to invert the new P. While this generally
produces satisfactory results, it is vulnerable to computa-
tional problems in two respects. First, if P-~ is continually
i~ICHARD H. BARTELS AND GENE H. GOLUB
updated, computational inaccuracies imposed by limited-
S t a n f o r d University,* Stanford, California
precision arithmetic (round-off errors) are allowed to
propagate from step to step. Second, the updating, in
whatever way it is carried out, is equivalent to premulti-
Standard computer implementations of Dantzlg's simplex plying p-1 by a matrix of the form:
method for linear programming are based upon forming the
inverse of the basic matrix and updating the inverse after every
0 - yo/yk
step of the method. These implementations have bad round-off
error properties. This paper gives the theoretical background 0
for an implementation which is based upon the LU decomposi-
tion, computed with row interchanges, of the basic matrix. The - yk-i/y~
implementation is slow, but has good round-off error behavior.
1/yk
The implementation appears as CACM Algorithm 350.
KEY WORDS AND PHRASES: simplex method, linear programming, LU de- - yk+i/Yk
composition, round-off errors, computational stability
CR CATEGORIES: 5.41 0

-- y,,ri/Y~
1. LU Decomposition
The linear programming problem:
I n the presence of round-off errors this computation can be
maximize drx quite inaccurate if yk is small relative to the other Yi -
(1) Both of these problems can be eliminated if, instead of
I
subject to Ix > 0
Gx = b
p - l , the LU decomposition of P is computed using row
interchanges to select pivots. The problem of solving a
where d r = [do, . . . , d,_~], b r = [b0, . . . , bin-a], and linear system based upon P is then reduced to t h a t of

e =
I00 00:]
: :
backsolving two triangular linear systems. The numerical
stability of this scheme is generally quite good (see [4]).
Advantage can be taken of the similarity between any two
Lg~-i,0 • " • gm-l,~-l_j
successive matrices so as to economize on the computation
are given, with b > 0, is commonly solved by a two-phase of the LU decompositions. For details see [1, Secs. 5-6].
process built upon the simplex method of G. B. Dantzig Additional accuracy is obtained in the program (see
[2]. Each step of this method requires the solution of three Algorithm 350) by iteratively refining the solution to
systems of linear equations involving a common matrix of problem (1), after it has been found, according to the
coefficients, the basis matrix P. I t is the usual practice to scheme given in [4, p. 121].
solve these systems by forming P-~, either directly or in The algorithm requires ha~6 + 0 ( n 2) multiplications per
exchange on the average. Thus the algorithm is most ef-
This paper gives the theoretical background of Algorithm 350, fectively used for very ill-conditioned problems or in con-
"Simplex-Method Procedure Employing LU Decomposition," by junction with other implementations of the simplex algo-
the same authors, which appears on pages 275-278.
* Computer Science Department. This project was supported rithm which produces an initial basis more cheaply. A fast
in part by NSF under project GP948 and ONI~ under project iNK and accurate version of the simplex algorithm is described
044 211. in [5] but this requires additional storage.

266 C o m m u n i c a t i o n s of t h e ACM V o l u m e 12 / Number 5 / May, 1969

2. The Two-Phase Solution Process columns of G) be used to construct the following matrix:
T h e possibility of degeneracy is ignored, as is c u s t o m a r y .
P = e0 [ --" e,~-~-i g~-~ [ -'" g,,-1
PHASE 1 (Find a basic feasible solution.)

maximize -- e Ta Suppose t h a t p - l b ~ O. (This situation arises if the vari-

ables are n u m b e r e d so t h a t x~_~, • • • , x~-i are the slack

I
Gx + a = b
subject to Ix >_ 0, a ~ 0,
(2) variables and the constraints are ordered so t h a t
g~_~ = e,~_~, . . . , g , - t = era-l, w h e r e b y P becomes the
identity matrix. ) T h e n p r o b l e m (2) can be replaced b y the
where a r = [a0, " . , a,,_l] is a vector of "artificial vari- problem:
ables" and e r = [1, - . . , 1] (m c o m p o n e n t s ) . T h e solution maximize --~T5
is obtained b y starting with a = b as a basic feasible solu-
tion and applying the simplex method. As each a~ becomes
nonbasic, it m a y be dropped f r o m the p r o b l e m altogether.
Since - - e T a < 0 for all a, p r o b l e m (2) has a solution. I f
subject to (4)
m a x ( - - e r a ) ~ O, t h e n p r o b l e m (1) has no basic feasible
solution. Otherwise, the second phase is entered.
PHASE 2 ( F i n d an optimal basic feasible solution.)
x (1) >_ x (2) _> 0, ~> 0
I f the solution to problem (2) contains only components
of x as basic variables, these become the initial basic where x ( 1 ) = Ix0, . . - , x . . . . 1], x (2)= [Xn--K, " " , Xn--t],
variables for solving p r o b l e m (1) b y the simplex method. G (I) and G (2) are appropriate submatrices of G, 5 =
I f artificial variables, say a~0, . . . , ai~ , remain basic b u t [a0, • • • , a . . . . 1], ~ = [1, . . . , 1] (m - K components), a n d i
at zero level in the solution of (2), an equivalent p r o b l e m is the (m -- ~)-order identity matrix. T h e simplex m e t h o d
to ( 1 ) is solved: is begun with a0, . - . , a . . . . 1, x . . . . - . . , x~_l as basic
variables. Since there are fewer artificial variables to be
maximize drx
driven to zero in p r o b l e m (4) t h a n in p r o b l e m (2), we ex-
pect problem (4) to require on the average less work to
G ejo • "• eil 0 X solve.
0 1 ... 1 1 A reduction of c o m p u t a t i o n b e y o n d t h a t described in [1]
aJ o can be arranged for the L U decompositions calculated in
phase 1. E a c h time an artificial variable becomes nonbasic,
a row and column interchange can be applied to the basis
subject to (3) matrix so t h a t it always has the partitioned f o r m
a5 l

x >_ 0, s > 0, aj 0 ~ 0, ... ,aj~ _> 0

where I is the identity matrix of appropriate order, and R
where ek represents the kth column of the identity matrix and S are composed of components of G. T h e L U decompo-
(column n u m b e r i n g begun at zero). T h e additional vari- sition of P is k n o w n when the decomposition S = ~11 has
able s, together with the additional constraint been c o m p u t e d ; viz.
l
s + a i , = O,
i=O P ~

holds the artificial variables at zero level t h r o u g h o u t

further computation. T h e simplex m e t h o d is applied to
T h e triangular systems of equations induced b y this for-
problem (3) with s and the basic variables f r o m problem
m a t are solved in a particularly efficient m a n n e r b y Al-
(2) as the initial set of basic variables. (For an example of
g o r i t h m 350.
this two-phase process in operation see [3, Sec. 7.7].)
4. Example Application
3. Some Computational Details
Let V be a r a n d o m variable restricted to a finite set of
Phase 1 is skipped if a basic feasible set of variables for values v0, • • • , v=_l. Let p~ be the probability t h a t V has
p r o b l e m (1) is k n o w n a priori. If a partial set of basic value v~. I f certain m o m e n t s
variables is k n o w n for p r o b l e m (1), the c o m p u t a t i o n of
phase 1 can be restructured to take this into account. T o be ~k -~ E vjkpj , k ~ {k0, "'" , kin--l}
jffi0
precise, let K (1 < ~ < m - 1) columns of G ( w i t h o u t loss
of generality these can be made g~_~, • • • , g~-l, the last are k n o w n together with the values v¢, estimates of the

Volume 12 / Number 5 / May, 1969 Communications of the ACM 267

p~ a n d of u n k n o w n m o m e n t s gl can b e m a d e v i a linear g: = 10.5, g3 = 40.5 a n d v~ = i for i = 0, . . . , 6. T h e re-
p r o g r a m m i n g ; viz. sults are listed below.
Actual
a~<p~<fll and 7z_<g~_<81, Computed upper probabil~
bound ity
where
•083333333325 .015625
a~ = - - m a x ( - - x ~ ) , fl~ = m a x ( x l ) • 19999999998 .09375
•50000000001 .234375
u--1 n--1
•83333333335 . 3125
7, = - m a x ( - ~ v/xi), 5, = m a x ( ~ vitxi), •49999999999 .234375
5=0 5=0
•20000000001 .09375
all s u b j e c t to t h e c o n s t r a i n t s •083333333334 .015625
n--1
L o w e r b o u n d s for t h e p~ c o m p u t e d f r o m t h e a b o v e d a t a
X0>0, "'", Xn-1 >__ 0, ~Xi---- 1,
1=0 in a s i m i l a r f a s h i o n were all zero. C o m p u t e d b o u n d s for
n--1
#4 t h r o u g h glo were as follows.
5=0 Computed Actual Computed
lower moment upper
bound bound
T h e basis m a t r i c e s e n c o u n t e r e d in solving t h e s e p r o b -
lems are s u b m a t r i c e s of a V a n d e r m o n d e m a t r i x , m a k i n g 165.5 168.0 175.5
700.5 738.0 850.5
t h e m s o m e w h a t p o o r l y c o n d i t i o n e d ; so i t is i m p o r t a n t t h a t
3035.5 3396.75 4495.5
t h e simplex m e t h o d be carried o u t a c c u r a t e l y . O n l y t h e 13390• 5 16251• 75 25150.5
o b j e c t i v e f u n c t i o n differs f r o m one e s t i m a t i o n p r o b l e m to 59965.5 80335.5 145435.5
t h e n e x t ; so t h e s o l u t i o n of a n y one of t h e p r o b l e m s w o u l d 272200.5 408280• 5 856210.5
p r o v i d e an i n i t i a l basis for t h e o t h e r p r o b l e m s . P h a s e 1 1251135• 5 2124764.25 5088055.5
w o u l d n e e d to b e c a r r i e d o u t o n l y once. RECEIVED SEPTEMBER, 1967; REWSED NOVEMBER, 1968
T h u s , for e x a m p l e , to c o m p u t e u p p e r b o u n d s for
REFERENCES
p0, "" • , p~-~ f r o m g i v e n v a l u e s v0, . . • , v~_~ a n d g i v e n
m o m e n t s go, • • • , g ~ - ~ , one w o u l d set u p a r r a y s G[i, j] 1. BARTELS, R. H., AND GOLUB, G . H . Stable numerical m e t h o d s
for o b t a i n i n g t h e C h e b y s h e v solution to a n o v e r d e t e r m i n e d
v / a n d b[i] --= gl a n d execute s y s t e m of equations• Comm. ACM 11, 6 (June 1968), 401-406.
kappa : = 0; fori := 0stepluntiln-- ldod[i] := 0; 2. DANTZm, G . B . Linear Programming and Extensions• Prince-
for i := 0 s t e p 1 u n t i l n -- 1 d o t o n U. Press, P r i n c e t o n , N.J., 1963•
b e g i n d[i] := 1; 3. HADLEY, G. Linear Programming. Addison-Wesley, Reading,
i f i>_ l t h e n d [ i - - 1] := 0; Mass•, 1962.
linprog(m, n, kappa, G, b, d, x, z, ind, infeas, unbdd, sing); 4. WlI~KINSON, J. H. Rounding Errors in Algebraic Processes.
upper bound[i] := z; P r e n t i c e - H a l l Series in A u t o m a t i c C o m p u t a t i o n , Englewood
end; Cliffs, N.J., 1963.
5. BARTELS, RICHARD H• A numerical i n v e s t i g a t i o n of the sim-
A s a n i l l u s t r a t i o n , t h e p r e c e d i n g was r u n o n S t a n f o r d plex method• Tech. Rep. No. CS 104, C o m p u t . Sci. Dep.,
U n i v e r s i t y ' s B5500 c o m p u t e r w i t h t h e d a t a go = 1, #~ = 3, S t a n f o r d U., Stanford, Calif., J u l y 1968.

Rough and Ready Error Estimates Two expressions are derived for use in estimating the error
in the numerical integration of analytic functions in terms of
in Gaussian Integration of the maximum absolute value of the function in an appropriate
region of regularity. These expressions are then specialized
Analytic Functions to the case of Gaussian integration rules, and the resulting
error estimates are compared with those obtained by the use of
tables of error coefficients.
PHILIP RABINOWITZ
The Weizmann Institute of Science,* Rehovoth, Israel
D a v i s a n d R a b i n o w i t z [1] p r e s e n t e d a m e t h o d for esti-
m a t i n g t h e error c o m m i t t e d in i n t e g r a t i n g a n a n a l y t i c
KEY WORDS AND PHRASES: numerical integration, analytic functions,error f u n c t i o n b y a n a r b i t r a r y i n t e g r a t i o n rule
estimates, Gaussian integration, tabulated error coefficients, computable
error coefficients, Cauchy integral formula, Chebyshev polynomials
CR CATEGORIES: 5.1 I, 5.16
I(f) = w,f(z,).

* D e p a r t m e n t of Applied M a t h e m a t i c s I f t h e error is d e n o t e d b y E(f) = f i l l ( x ) dx -- I ( J ) , we

268 Communications o f t h e ACM V o l u m e 12 / N u m b e r 5 / M a y , 1969