Unitary Triangularization of a N o n s y m m e t r i c Matrix*

ALSTON S. HOUSEHOLDER

Oak Ridge National Laboratory, Oak Ridge, Tennessee

A method for the inversion of a nonsymmetric matrix, due to J. W. Givens, has been in use at Oak Ridge National Laboratory and has proved to be highly stable numerically but to require a rather large number of arithmetic operations, including a total of n ( n -- 1)/2 square roots. Strictly, the method achieves the triangularization of the matrix, after which any standard method m a y be employed for inverting the triangle. The triangular form is brought about by means of a sequence of n ( n -- 1)/2 plane rotations, whose product is an orthogonal matrix. Each rotation requires the extraction of a square root. The advantage in using the method lies in the fact that an orthogonal matrix is perfectly conditioned. Hence the condition of the matrix cannot deteriorate through successive transformations. In fact, if ore defines the condition number of a matrix A to be [1] ~(A) = [] A ][ [I A-I[[, where the norm is the spectral norm, then for any orthogonal matrix W, 7(W) = 1 and the condition of any matrix is preserved under multiplication by an orthogonal matrix:
, y ( W A ) = "y(A ).

T o look at the m a t t e r another way, if

where R is an upper triangle, then

so that R is precisely the triangle one would obtain from the application of the Choleski square-root method to the positive definite matrix A r A . I t is, in fact, the matrix to which yon Neumann and Goldstine [2] are led in their study of Gaussian elimination as apphed to a positive definite matrix. T o obtain the precise triangle that would result from Gaussian elimination with A r A , one has only to remove as a factor the diagonal of R:
R = DU,

where U has a unit diagonal. The purpose of the present note is to point out that the same result can be obtained with fewer arithmetic operations, and, in particular, for inverting a square matrix of order n, at most 2(n - 1) square roots are required, instead of n ( n - 1)/2. For n > 4, this is a saving of (n - 4)(n - 1)/4 square roots.
* Received June, 1958 339

340

ALSTON

S.

HOUSEHOLDER

I t is as easy to discuss the general complex case, and the method is based upon the almost self-evident LEMMA. For a n y vector a ~ O, and a n y u n i t vector v, a u n i t vector u exists such
that (I 2 u u * ) a = II a [[ v,

where [[ a [[ represents the E u c l i d e a n n o r m : [[ a ]]2 = a*a. The c o m p u t a t i o n requires two square roots and a single reciprocation

The proof exhibits the computation. Let a = ]]a]] > O. This represents one of the necessary square roots. I t is required t h a t
a 2 ( u * a ) u = av.

(1)

Let
= 2u*a. (2)

Then
~u = a ~v, (3)

2a(c~

v'a),

(4)

which accounts for the other square root. Clearly

since a is the Euclidean length of a, and v*a the projection of a upon the unit vector v. Hence a and g are both real and can be taken non-negative. If a = v'a, then the lemma is verified with u = 0; otherwise take g > 0 defined b y (4), and one verifies easily t h a t u defined b y (3) is effective. T h e single reciprocation . --1 necessary is in g N o w let a be the first column of A and take v = e~, the first column of the identity. Application of the l e m m a provides a unitary matrix U1 = I - 2ulu1* such t h a t the first column of U1A is null except in the first element. The result is equivalent to the application of n -- 1 plane rotations, with one slight difference, that det ( I -- 2uu*) = --1, whence this transformation reverses the orientation of the configuration One continues after suppressing the first row and first column of the transformed matrix. After n -- 1 steps, at most, the matrix A is triangularized:
U U n - l U n - 2 . . " U1,
UA = R.

Evidently det A = ( - 1 ) ~-1 det R.

UNITARY

TRIANGULARIZATION

341

An interesting byproduct of the reduction is a simple proof of the theorem of Hadamard, that I d e t A I ~ H]]a~]l, 1 where a, is the ,th column of A. This results from the fact that the Euclidean norm is preserved under multiplication b y a unitary matrix, and that each diagonal element of R is the norm of the projection of that column upon a certain subspace. I t is evident that when the reduction is applied to a matrix A of n columns and N > n rows, there results again an upper triangle of non-null elements, and a trapezoid of zeros, the triangle representing the factorization of the normal matrix A*A required for a least squares solution. Returning to the case of a square matrix, if the vectors u, are stored as generated and applied to the successive columns of the matrix, there are required the n - 1 reciprocations, 2(n - 1) square roots, and (n -- 1)(2n 2 + 5n + 9)/3 ~ 2n3/3 multiplications for the formation of R. In this count it is assumed that one forms o~, t~-1, u, and then each u'at and uu*a~, i > 1.
/ \

Inversion of R requires ( n +3 2 ) multiplications; and if R -1 is multiplied by the U~ in reverse order, then (n -- 1)(4n 2 + 7n + 12)/6 multiplications are required. Altogether, for the formation of A -~, the number of multiplications is (3n 3 --F 4n 2 + 5n - 10)/2 -+- 3n3/2, as compared with approximately 5n3/2 multiplications required if one were to form (A*A)-'A*. K y Fun has pointed out that half the square roots and all the reciprocations are evaded in the triangularization proper if one forms t~I-

2(#~t)(ttu)*.

Instead of R, a scalar multiple of R is formed in this way. This is feasible for floating-point computations, but probably not for fixed point. If the matrix
A = (al, a2, -.. , a~)

is scaled at the outset so that [[ a, I1 =< 1 for every z, then all elements remain within range throughout the triangularization, since a unitary transformation leaves the Euclidean norm invariant. Hence no scaling problems arise in the actual triangularization. Moreover, when R - ' is formed, if this is similarly

342

ALSTON S. HOUSEHOLDER

scaled no further scaling is required in multiplying b y the matrices U~ to form A -I. Only in forming R -I itself may intermediate scaling be required. The unitary matrix employed here is obviously suggested b y a somewhat more general one previously used [1]. A similar form is used by Steenrod [3] for theoretical purposes. REFERENCES 1. A. S. HOUSEHOLDER(1958), A class of methods for inverting matrices. J. Soc. Ind. Appl. Math. 6, 189-195. 2. JOHN YON NEUMANNand H. H. GOLDSTINE(1947), Numerical inverting of matrices of high order, Bull. Am. Math. Soc. 55: 1021-99. 3. NORM&NE. STEENROD(1951), The topology of fibre bundles Princeton