Numer. Math.

64, 241-269 (1993)

Numerische
Mathematik
9 Springer-Verlag1993

A Jacobi eigenreduction algorithm

for definite matrix pairs
K. Veseli~
Fachbereich Mathematik und Informatik, Fernuniversit/itGesamthochschule, Fleyer Strasse 204,
W-5800 Hagen, Federal Republic of Germany
Received July 11, 1990/RevisedVersion received May 11, 1992

Summary. We propose a Jacobi eigenreduction algorithm for symmetric definite

matrix pairs A, J of small to medium-size real symmetric matrices with j 2 = I,
J diagonal (neither J nor A itself need be definite). Our Jacobi reduction works
only on one matrix and uses J-orthogonal elementary congruences which include
both trigonometric and hyperbolic rotations and preserve the symmetry through-
out the process. For the rotation parameters only the pivotal elements of the
current matrix are needed which facilitates parallelization. We prove the global
convergence of the method; the quadratic convergence was observed in all experi-
ments. We apply our method in two situations: (i) eigenreducing a single real
symmetric matrix and (ii) eigenreducing an overdamped quadratic matrix pencil.
In both cases our method is preceded by a symmetric indefinite decomposition and
performed in its "one-sided" variant on the thus obtained factors. Our method
outdoes the standard methods like standard Jacobi or qr/qI in accuracy in spite of
the use of hyperbolic transformations which are not orthogonal (a theoretical
justification of this behaviour is made elsewhere). The accuracy advantage of our
method can be particularly drastic if the eigenvalues are of different order. In
addition, in working with quadratic pencils our method is shown to either converge
or to detect non-overdampedness.

Mathematics Subject Classification (1991): 65F15

1. Introduction and preliminaries

Real symmetric matrices S and T are said to be simultaneously diagonalizable if

there exists a real non-singular matrix G such that the simultaneous congruence
transformation

(1.1) GTSG = S', GTTG = T'

yields diagonal matrices S', T'. We consider here the most important class of pairs
S, T which are simultaneously diagonalizable, namely the class of definite matrix
 242 K. Veseli6

pairs. A pair S, T of real symmetric matrices is called positive or negative definite if

there is a real/~ such that the matrix
S-/zT
is positive or negative definite (cf. Stewart [34]). The number/~ is called a definitiz-
ing shift. It is easily seen (see Th. A1) that the set ~r of all definitizing shifts is an
open interval separating the eigenvalues of the, say, positive definite pair S, T:
(1.2) 2 ; . . . . ,2~-_,, < J < 2~- . . . . . 2 + , J = (2,-_,,, 2~),
where (m, n - m) is the inertia of T.
In this paper we present a Jacobi-like method for performing the eigen-
reduction (1.1) for special pairs A, J, where J is diagonal and j 2 = I. We reduce the
pair A, J by iteratively transforming a single matrix A as
A ~ CXAC, withCXJC=J.
The property of C above is called J-orthogonality. Since all J-orthogonal
matrices form a multiplicative group we ultimately find a J-orthogonal Z such that
(1.3) ZTAZ = diag.
For an indefinite J the transformation matrices C consist of standard (trigonomet-
ric) as well as of hyperbolic plane rotations.
Definite pairs seldom arise in the normalized form A, J. If the matrix T is
non-singular, then we can make the symmetric indefinite decomposition (see
e.g. [5] ).

(1.4) T = W J W x, J =
0]
-I,_,, "

This leads to an equivalent pair

(1.5) A = W-1SW-T,.].

Note that in this way the initial symmetry of the problem is preserved during
the whole diagonalizing process. If a definitizing shift # is known in advance then
the step (1.5) can be made with the positive definite matrix T ' = T - # S thus
reducing to a common Cholesky decomposition with J = I. The eigenreduction
may then be continued by diagonalizing a single symmetric matrix A. However, the
transition from S, T to `4, J may be unstable even if T itself is positive definite (see
" [15], Ex. 8.7.2). There are pairs where this transition is safe ( [2, 39] ) and this issue
deserves further research. A possible symmetry preserving method for a general
symmetric definite pair is the Jacobi-like algorithm of Falk and Langemeyer [13]
for which the global convergence has been proved in [19] whereas [32] proves the
quadratic convergence for distinct eigenvalues. With multiple eigenvalues this
method usually lacks the quadratic convergence [20]. This calls for a further
investigation. In this paper we consider cases in which the reduction (1.5) is
relatively simple. We shall present two main applications of our Jacobi method:

d . Diagonalization of a single real symmetric matrix, say, T. Then S -- I and the

pair .4, J from (1.5) is trivially positive definite. Actually, in this case the Jacobi
process is better made on the matrix pair ,4 = WxW, J, where .4 is kept and
A Jacobi eigenreduction algorithm for definite matrix pairs 243

updated in the factorized form (this is the "one-sided" or "implicit" variant of the
Jacobi method) so that, upon convergence, the factor W contains the eigenvectors
of T. This seems rather a roundabout way to diagonalize a real symmetric matrix
and uses hyperbolic transformations besides! Nevertheless, this algorithm is appar-
ently the closest of all known algorithms to the ideal: if the matrix is "well-behaved"
i.e. if small relative errors in the matrix entries cause small relative errors in the
eioenvalues then compute the latter with nearly the same accuracy. For positive
definite T we have J = I and our method reduces to a known one [37] which is
shown to attain the above ideal [7]. For indefinite T we still obtain a positive
definite current matrix A whereas the initial indefiniteness is "packed" in J which
does not change during the Jacobi process (note that indefinite J means the use of
hyperbolic transformations). Pairs A, J with A positive definite allow a very sharp
"relative" perturbation theory [38]. This is a basis for an error analysis, performed
in the forthcoming dissertation of Slapni6ar [33] whose results are a non-trivial
generalization of those in [7]. As the Slapni~ar's analysis shows, the orthogonality
of the transformation plays only a secondary role in the error estimates. Our
method is generally more accurate than the standard ql or Jacobi method, applied
directly to the initial matrix T. Our method is also generally faster than the
standard Jacobi method. This is due to the factorized form of the current matrix as
well as to the fact that the decomposition (1.4) is a preconditioner reducing the
number of the Jacobi iterations. The rounding errors in performing (1.4) are, as
a rule, more serious than those in the Jacobi method that follows ([33]) and it is
certainly of great help if the factor W is accessible without computing T itself.

~ . Overdamped pencils. Consider the quadratic eigenvalue problem

(1.6) (M2 z + D2 + K ) x = 0

The matrices M, K, D are real symmetric and positive definite. The pencil (1.6) is
called overdamped [10], if
(1.7) (xTDx) 2 > 4 x T M x x T K x

holds for any non-vanishing x. By setting 2 = 1//~ and

we obtain a linear symmetric eigenvalue problem

(1.9) Sz = #Tz ,

which is equivalent to (1.6). Note that both matrices in (1.9) are indefinite. Using
the decomposition

(1.10) T = W J W T, J = I ' W= L1 '

where M = L2L2~, K = L I L T are the usual Cholesky decompositions, we obtain

a pair A, J with

(1.11) A= LllL 2 L ; 1 D L [ T ], A -a= L21L 1

244 K. Veseli6

Now, the overdampedness of the quadratic pencil is equivalent to the positive

definiteness of the pair A, J (for the proof see Th. A5). Here the definitizing shift is
not known in advance. Moreover, for given M, K, D we are interested either to
eigenreduce or to detect non-overdampedness as quickly as possible. Our algo-
rithm either converges or detects indefiniteness relatively quickly and may thus be
useful tool to check the definiteness of a pair. I
Again, the transition from S, T into A, J appears to be the most dangerous part
of the eigenreduction. The implicit variant works only with the Cholesky factors of
the matrices M, K, D (Sect. 3) thus increasing the accuracy, if M, K, D are given in
the factorized form. An error analysis, analogous to that of [33] is not available for
that case as yet.
The plan of the article is as follows. Section 2 contains the description of the
method and the convergence results. We prove the global convergence with the
row-cyclic and the threshold strategies. We also prove that on a pair whose
definiteness is unknown the algorithm either converges or discovers the indefinite-
ness in finitely many steps. The convergence proofs are facilitated by the fact that
our Jacobi method is trace reducing. If the matrix A itself is positive definite then
we present an alternative convergence proof which uses the fact that the so-called
H a d a m a r d measure, is increasing. The importance of this proof lies in the fact that
it compares the quantities [a,s [ and ~ as the threshold and stopping
criterion. This returns the computed eigenvalues with desired relative accuracy.
Section 3 describes the implicit method and its modification for the cases d
and ~ above. Section 4 presents experimental comparison of our Jacobi
methods with the standard Q L / Q R algorithms. The experimental results show
that on most single symmetric matrices our method shows better relative
accuracy than the standard Q L or Jacobi method. The advantage may be
quite drastic on matrices whose eigenvalues are of different order of magnitude.
The rare known examples where our method is weak are of a very particular
structure and it is easily seen that the fault lies already on the indefinite symmetric
decomposition T = WJ W T rather than on the Jacobi method that follows. These
difficulties could be related to the fact, that, in contrast to the positive definite case,
we are still not in a position to simply describe the set of all well behaved matrices
(see [38]).
The experiments with overdamped pencils show the same behaviour: although
it uses hyperbolics our method was never much worse and sometimes much better
than the (non-symmetric) QR method, applied on the single matrix JA or even the
Q Z method, applied directly to the pair S, T. On some examples we show that
accuracy of our method can be drastically increased, if the Cholesky factors of the
input matrices are accurately known. In the Appendix we prove some theorems on
definite pairs which are of interest for our method. The most important result is
that the eigenvector condition is independent of the algorithm and is asymp-
totically proportional to the inverse of the square root of the distance to the nearest
non-definite pair - - a result as much seldom as favorable.

1 Recently, Scott and Ward [30] proposed an algorithm for a partial eigenreduction of large
sparse matrices generated by overdamped pencils. Our algorithm can be considered as its
counterpart for small full matrices. It could therefore be used as a subspace eigenreduction routine
within the mentioned algorithm of Scott and Ward.
A Jacobi eigenreduction algorithm for definite matrix pairs 245

2. The Jacobi method

The Jacobi method for reducing a definite pair A, J consists of an iterative

application of the congruence transformation
(2.1) A'= CTAC ,

where C is a plane transformation i.e.

(2.2) C LCs, csd

(the non-displayed elements are those of the unit matrix) having the property
(2.3) CT J C = J .

It is immediately seen that (2.2) and (2.3) imply

([cos x -sinx 1
(2.4) F c" c,s~=,)l_sinx cosx_l' l<r,s<m or m + l < r , s < n ,
kC~ c~s_J )Fcoshy sinhy 1
[,Lsinhy coshyJ' otherwise.

The parameter x or y is chosen so that the r, s-element of the transformed

matrix is annihilated i.e.
2a,, rt n
(2.5) tan 2x = , - - =
art -- ass 4 < x < ~ ,

or

2ars
(2.6) tanh 2y = t, t=
art + a~s "

In (2.6) of course,
(2.7) larr + a~st > 21a,~],
has to be valid. We call the two types of transformations in (2.4) the trigonometric
and the hyperbolic transformation, respectively. In a real computation we use the
stable Rutishauser formulae (or their hyperbolic analogs) from 1-41]).
If (2.7) is violated at some step, then the algorithm breaks down.
Theorem 2.1. Let A' be given by (2.1)-(2.3). Then the pair A', J is positive (negative)
definite i f and only if A, J is such. In this case we have f o r r < m < s

art + ass > 6 (art + ass < -- 6) ,

where 6 is the length o f the interval J in (1.2). Moreover

(2.8)
I--IJ
2a,s
a,, + a~
<
=
1
(ar, + a ~ ) 2 "
246 K. Veseli6

Proof The first assertion follows from

A' - #J = CT(A - IJJ)C .

If A - / ~ J is, say, positive definite and r < m < s then the matrix

a,~ ass + #

is positive definite as well. Thus, the p a r a b o l a _ # 2 _ (as, - a,,,)# + ar, a s s - a,~ is

positive for any # ~ J i.e.

6 < x/(a,, + a , , ) z _ 4a,~,

which proves (2.8) and thus the whole Theorem. Q.E.D.

Thus, for definite pairs our method never b r e a k s d o w n .

As far as the transformation formulae (2.1)-(2.4) are concerned our method is
closely related to the algorithm of Eberlein [12], which can be shown to preserve
the J - s y m m e t r y [35]. Its convergence is satisfactory for the case of real and
non-defective eigenvalues. The only difference is that here the formula (2.6) for the
hyperbolic p a r a m e t e r y needs only pivotal elements a,,, a,,, ass of the current
matrix in contrast to the costly norm-reducing formula (15) in [12]. This formula
requires access to two rows and two columns of the matrix at every pivot which
makes it less convenient for implicit or parallel computations. The algorithms from
[35, 36] have the same weakness.
O n the other hand, our method is closely related to the method for a general
definite pair A, B ([13, 16, 18, 19, 42]). However, the much simpler structure of our
m e t h o d as well as some nice monotonicity and invariance properties allow us to
strengthen the convergence results considerably.
Finally, our transformations (2.4) with the parameters as in (2.5), (2.6), respec-
tively, appear to be a special case of those proposed in [11] Ch. VII, Sect. II, for
general non-symmetric matrices. There, however, the process has only a modest
success since it m a y diverge or even break down on quite well-conditioned
matrices. O u r class of matrices (A, 3 definite) behaves essentially better as is seen
from Th. 2.1 and the convergence theory which we now present.
Let A be symmetric and A' = C T A C with a J - o r t h o g o n a l C. We shall use the
following invariance properties
TrJA' = TrJA,
(2.9)
T r ( ( J A ' ) 2) = Tr((JA) z) ,

which are easily verified. Writing

(2.10)
F
A = LAT2 A2~ '
F
= LA~ Ai:J'

where the partition is that of d in (1.5), the invariance properties (2.9) can be
expressed as
(2.11) TrA'I 1 - TrA ~2 = TrA 1i - TrA 2z ,
(2.12) l[a]l[I 2 + Ila~2ll z - 211A'12112 = Ilaixll 2 + 11A22112 -2llA12[I 2
A Jacobi eigenreduction algorithm for definite matrix pairs 247

Here II It denotes the Euclidean norm. Applying (2.12) to the transformation

(2.13) [a'rr a ' r s ] = ~ c , r cs,-] ~a~, a,s] [-c,r c~s]

a'rs a',~J L C , s cssJ ka,s assd kCs, cssJ
(C as in (2.2)-(2.6)) we obtain
(2.14) t2 = a~,
a', ff + ass 2 + ass
2 - 2ars
2 ,

for a hyperbolic and

(2.15) a'r2 + a'~2 = a,2~ + ass2 + 2a2

for a trigonometric transformation. Subtracting (2.14) from (2.12) we obtain

(2.16) $2(A'11) + $2(A~2) -- 211A'12 It 2 = S2(All) + $ 2 ( A 2 2 )
- 21tA12112 + 2a2
for a hyperbolic transformation. Here

$2(A) = Z a2
i,j
iee j
for a general matrix A. Subtracting (2.15) from (2.12) and using the identity
lIAr2 [] = IIA 12 II ,

valid for any trigonometric transformation, we obtain

(2.17) S2(A ') = S2(A) - 2aZs.
After a hyperbolic transformation the only changed diagonal elements are
(2.18) a',, = ar, coshZy + 2a,~coshy sinhy + asssinhZy
(2.19) a'ss = arrsinh2y + 2a,scoshy sinhy + as~coshZy.
Although the Jacobi method never breaks down, large hyperbolic parameters
may be unsuitable because a highly conditioned C may only temporarily increase
the condition of the current eigenvector matrix which means instability. (Note that
the final condition of the eigenvector matrix depends only on the pair A, J
(Cor. A.2) and grows slowly as the matrix A approaches the boundary of the
definiteness region. (Th. A.4).) In order to curb such "local" condition growth we
modify the formula (2.6) as

(2.20)
~
/' = tanh 2y = It,
t ma~sign t, [t] > tmax
otherwise ,

with 0 __<tma x ~ 1 and t from (2.6). Another plausible precaution consists in

switching off hyperbolic transformations during, say, the first cycle.

Lemma 2.2. Let A, J be, say, positive definite. Perform a hyperbolic step with any
parameter 33. Then
(2.21) TrA - TrA' = (a,, + as~)[1 - cosh 233 + t sinh 233],
248 K. Veseli6

(t from (2.6)). The above function is strictly concave in ~ and takes its maximum at
29 = artanh t. With 33 = artanh ?, ?from (2.20)

(2.22) TrA - TrA' = (a,~ + a=)( 1

\
~2
(2.23) > (arr + ass)
(1 + x/1 - ?2)

holds. Moreover, let Yl, Y2 . . . . be the sequence of the hyperbolic parameters in

a Jacobi method. Then the trace of the current matrix A has a limit tro and

tanh 22y k TrA ~ - tro

<
k=l 1 + ,/1- tanh22yk 5

where A 1 is the initial matrix, 3 is the length of the interval J in (1.2). In particular,
the choice l?I = t tm,x ( can occur only finitely many times.

Proof The formulae (2.21) and (2.22) are directly verified. From (2.22) and the
invariance of the trace under trigonometric transformations it follows that TrA is
non-increasing. It is obviously bounded from below by #TrJ, p any definitizing
shift, and has therefore a limit tro. Now (2.23) follows from a r t + a~ > 3
(Th. 2.1). Q.E.D.

Note that for the validity of the identities (2.21), (2.22) no definiteness of the pair
A, J is necessary. It is sufficient that an r, s-hyperbolic transformation is possible i.e.
that (2.7) holds and that art + as~ is, say, positive.
The estimate (2.23) gives an upper bound to the condition of the hyperbolic
transformation independently of the choice of tm,~. For matrices near the bound-
ary of the definiteness region, however, this bound is of no use and a convenient
t~,,. should be introduced. It is plausible that tmax should be as close to 1 as
possible while still producing a tolerable condition of the hyperbolic matrix C in
(2.4). Our experiments suggested t max = 0.8 as a good value (see Sect. 4 below). In
this ease

(2.24) lcoshxl ~ x / ~ , lsinhxl ~ ~ , / ~ l x t ~ 0.5.

By (2.22) the choice (2.24) yields at least

(2.25) t2a~ 9 1 + `/1 - t 2 _~ t m2 a x = 40%

1+,/1- 2
traax t2 - I + , / 1 -- tm.x
2

of the optimal trace reduction obtained by (2.6).

Theorem 2.3. Let A, J be definite. Then the Jacobi algorithm with the row-cyclic
strategy is convergent i.e. Ak converges to a diagonal matrix as k -* oo.

Proof Let
A1 = A , A2 = C ~ A 1 C 1 . . . .
A Jacobi eigenreduction algorithm for definite matrix pairs 249

be the sequence generated by the Jacobi algorithm. For y sufficiently smalt we have

Ecoshy sinhy-]=[ cos~ sin~][1-tanh2y

sinh y cosh y [ -sin ~ cos ~ 2tanh y
0 I 1
1 + tanh2y x/1 _ tanh4y

where the equation

tan~=tany, -~ <~< ~,

gives a one-to-one correspondence between y and ~. Moreover, ~ tends to zero

together with y (Lemma 2.2). The Jacobi sequence can now be written as

(2.26) A1 =A, A2 =R~A1R1 +F1 . . . .

where R1, R2 . . . . are plane rotations (with the pivot pairs chosen in the row-
cyclic order) whose rotation angles lie in [-re/4, re/4]. In addition, the r(k), s(k)
(and s(k), r(k)) element of R~Ak Rk as well as the whole matrix Fk tends to zero
with k ~ o9 (note that Fk vanish for trigonometric transformations). Since the
whole sequence Ak is bounded we can use a result of Hari ([17], Lemma 2.2) which
immediately implies S 2(A k) ~ 0. The proof that the diagonal elements are conver-
gent, too, is very much the same as with the standard symmetric Jacobi method and
is omitted. Q.E.D.

Remark. Parallel strategies.

For implementation on multiprocessor arrays cyclic strategies different from
the row (or column) cyclic strategy are used ([3, 4, 21, 25, 29, 31]). The convergence
proofs for these strategies should carry over without difficulties (the proofs are
essentially a matter of combinatorics and are independent of the nature of the
elementary matrices C used in the transformation).

Another important class of strategies uses thresholds i.e. in sweeping the matrix
in a cyclic order a rotation is performed, if the pivot off-diagonal is larger than
a given threshold. The simplest way is just to take one, small enough, threshold and
use it as the stopping criterion as well. The following convergence theorem will
include matrix pairs A, J whose definiteness is not known in advance. We will show
that the method either converges or detects the indefiniteness in finitely many steps.
For this purpose we shall introduce three criteria simple to check at each step.
The criteria will be shown to hold for any definite matrix pair. We will then show
that the algorithm converges--at least with a threshold strategy - - if they are never
violated. The three criteria are:

Criterion I. The inequality (2.7) has to hold whenever a hyperbolic transformation

is due on the pivot pair r, s.

Criterion II. For all pivots r, s with r < m < s (hyperbolic transformations) the
quantity

(2.27) a = sign(a,, + ass),

is constant.
250 K. Veseli6

Criterion III. Before starting the algorithm c o m p u t e the quantities

- min au, a=l ,

O{1 ~ l<i~m

- max aii , (7 ~ - 1 ,
l<_i<m

- max aii, a = 1 ,
O~1 .~_ m < j < n

- max aii , t7 = -- 1
m<j<=n

The current matrix A k has to satisfy the following conditions at every stage of
the process

a=l: a=-l:
~(k)
a~) > c t z , l<i<_m; "u <a2, l<i<_m;
(k) (k)
a~j > c t l , m<j<n; a;j <~1, m<j<n.

It is clear that I and II must be fulfilled for any definite pair A, J (Th. 2.1). N o w
consider III e.g. for a positive definite pair (a = 1). Since both A 1 - g J and
Ak -- # J are positive definite for some ~t we obtain

al,*' > , , , al? > 1 _< i_< m,

(1) (k)
ajj > - - p , ajj > --I~, m<j<n.

Thus, the conditions of I I I follow.

ComputationaUy, the Criterias I and II do not require any significant effort.
The same is the case with III, if one remembers that only two diagonal elements are
changed at any step. So, after fully applying III on the initial matrix only two
comparisons, e.g. for a = 1 (tr from (2.27))

a,, > , 2 , ass > a l , (hyperbolic)

a , > "2, as~ > ct2, r, s < m (trigonometric)
a,, > ~tl, a= > a l , r,s > m (trigonometric)

have to be made on the current matrix A after the transformation on r, s has been
performed.
W e now state our most general convergence theorem.

Theorem 2.4. L e t the Jacobi method follow any cyclic strategy by rotating on all r, s
with

(2.28) [a,sl > tr,

for a given positive threshold a. I f the Criteria 1 - I I 1 are fulfilled then the number o f
steps is finite. I f a sequence a 1, a t . . . . ~ 0 o f such thresholds is given then the Jaeobi
algorithm is convergent.
A Jacobi eigenreduction algorithm for definite matrix pairs 251

P r o o f Let A ~ = A, A 2 . . . .
be the sequence of matrices generated by the Jacobi
algorithm. We use the natural notation
(l~ A(k) 7

[A "a12 | , (k))

Let tr from Criterion II be, say, 1. Again (2.22) (see the remarks following
L e m m a 2.2) implies that TrA]~, TrA ~k~ and T r A k are non-increasing sequences.
According to Criterion III we have

(2.29)
j=l

for 1 < i < m and similarly

n
(k) ~--(k) ~'~ (k)
(2.30) ~a ~ ajj = lrAE2 -- ~., all __<TrA - (n - m - 1)~i ,
i=m+l
i*j

for m < i < n. Thus, the sequences al~ ) are bounded in k. The same is therefore true
for TrA~ kF, Z r A ,E(k)
2 and T r A k as well. For k large enough (2.22) and (2.25) yield

(2.31) T r A (k) - TrA ~+1) > m m ~ - ,

9 f2~ ~ at ....
}

where (by (2.29), (2.30))

M = T r A i - ( n - 2 ) m a x ( [ ~ l [ , t~21)

> TrAk - (m- 1)e2 -- ( n - - m - 1)al

tk) (k)
ar(k)r(k ) d- as(k)s(k) > 0 .

Since the sequence Ak is bounded (2.3t) implies that hyperbolic rotations occur
only finitely m a n y times. The remaining trigonometric transformations are also
finitely m a n y since (2.17) implies S 2 ( A k + l ) < S 2 ( A k ) - - 20" 2. N o w it is clear that
with a sequence a l, a 2 . . . . ~ 0 of thresholds the off-diagonals will tend to zero.
The proof of the convergence of the diagonal elements is again omitted. Q.E.D.

Another, even more important, threshold strategy rotates, if

(2.32)

where a is a given threshold with 0 < a < 1 and stops, if (2.32) is false for all r 4= s
and a is small enough. This stopping criterion is useful, if the algorithm is expected
to compute the eigenvalues with uniformly small relative accuracy. Indeed, if
]a,, t < a I x f ~ , l [a,~ I for all r = s then by [2] the eigenvalues lie in the union of the
intervals

[ ++_a . - n a l a . . ] , ++_a . + nala.,[].

252 K. Veseli6

Although there is little doubt that our algorithm converges under this strategy as
well (our experiments confirm that) we have a p r o o f only for the important special
case where A itself is positive definite (case d ) .

Theorem 2.5. Let A be positive definite. Then all assertions of Th. 2.4 are true, if the
threshold criterion (2.28) is substituted by (2.32).

Proof. By
detA
X~(A) -
a l l 9 . . ann
denote the H a d a m a r d measure of A. The positive definiteness implies
0 < o~g(A) < 1, where 9f'(A) = 1 holds, if and only if A is diagonal. Since the
transformation (2.1) preserves both the positive definiteness and the determinant
we have
1 - a'
X~(A ') = X~(A) - - ,
1--~x
with
a,2 ars ~2
~X= - - (Xr --
arrass ' arr,as s, "
With tanh y = t from (2.6) we have a' = 0 and
3~(Ak)
3f(Ak+l) > -
1 - - tr
Thus, there are only finitely m a n y rotations for a given threshold. With the
modified parameter (2.20) a straightforward, if lengthy, calculation leads to
1 2
fftma
-- x
3V(Ak+~) > 3V(A,)
(1 - atmax)2 '
with the same conclusion as in Th. 2.4. 2 Q.E.D.

The quadratic convergence of the method has recently been proved in [9] in full
analogy to the standard Jacobi algorithm.
If the m e t h o d is convergent then the value tro from L e m m a 2.2 is given by
n--tit

tro= 2i ~ - ~ 27 9
i=1 i=1

Thus, the value

TrA - tro
(2.33) p(A) --

depends only on A and is shown to be an upper b o u n d of the spectral condition of

the eigenvector matrix of the pair A, J (see Th. A.3, A.4 below).

2 The use of the Hadamard measure in proving the convergence ofa Jacobi method goes back to
Gose [16]. Note that this type of proof makes absolutely no difference between trigonometric and
hyperbolic rotations.
A Jacobi eigenreduction algorithm for definite matrix pairs 253

3. The implicit method

Here we consider the so-called implicit or o n e - s i d e d variant of our algorithm. It is

particularly suitable in treating matrices of type ed and ~ from Sect. 1. 3 Since there
are significant differences between these two cases we shall present them separately.

d. D i a g o n a l i z a t i o n o f a single real s y m m e t r i c m a t r i x A. First perform the de-

composition ([5])

(3.1) A = G J G T, J =
['0 ~ -I

where G has full column rank, the diagonal blocks in d need not have the same
'

dimension and one of them m a y be void. 4 The matrix

= GTG

is positive definite and our Jacobi method is applied to the pair ~{, J by updating
the factor
(3.2) G1 = G, G k + 1 --- G k C k .

U p o n convergence the current matrix Gk becomes a G | of the form

G~ = GF,
where the matrix F is J-orthogonal and
FTi{F = FTGTGF = GTGoo = 7

is diagonal. The column normalization now gives a matrix

U =- G~]: -1/2 = G F 7 -1/2

with o r t h o n o r m a l columns i.e. U T U = I. In fact, U contains the eigenvectors for

the non-vanishing eigenvalues of A:
A U = G J G T G F 7 - 1/2 = G J F - T F T GT G F 7 - 1/2

= GJF-TT1/2 = GFF-1jF-T71/2 = G F J y 1/2 = U J 7 1 / 2 ,

where the diagonal matrix Jy 1/2 carries the non-vanishing eigenvalues of A. The
current matrix A k is not updated, except the diagonals whose updating is non-
expensive (see Rutishauser formulae in [41]). The pivot element d~ ), needed for the
rotation parameter, is computed as the scalar product of the columns of Gk. All
d(k)
,, are refreshed from G k after every sweep. In this way only one square array of
storage suffices to compute both the eigenvalues and the eigenvectors. The
stopping criterion

(3.3) ar~
-~(k) I <
= a ~ / l a ( ~ ) I u~
_~(~) t, for a l l r 4 : s

3 The idea of implicit Jacobi algorithms goes back to Hestenes 1-22] (see also [6, 18, 23, 37]).
, Here we obviously have to decide about the numerical rank of A. This is a delicate question
which deserves a careful treatment. In view of the fact that small eigenvalues may be equally well
defined as the large ones a simple possibility is to accept rank defect only, if during the
decomposition (3.1) exact zeros are encountered (I-33]).
254 K. Veseli6

yields not only a desired relative accuracy of the eigenvalues but also the desired
orthogonality of the eigenvector matrix. 5 In the case J = I (A positive definite) this
method reduces to a known one in 1-37, 7].

Diagonalizing the pair A, J from (1.11). Here A and J have the same inertia; more
precisely

(3.4) A = H J H T, H = LL?IL3 ,

where D = L a L ~. We introduce the auxiliary pair

(3.5) A= HTjH,J.

L e m m a 3.1. The pair A, J is positive definite, if and only if the pair fi, J is negative
definite.

P r o o f By
J H -T(A~, J ) J H r = (A, J)

the two pairs have always the same eigenvalues. By (1.6) all eigenvalues have
negative real parts. If A, J is negative definite then atl eigenvalues are real and
negative and there exists a Z with

(3.6) z T J Z = J, z T ~ 4 z = - ~J ,

where a is a positive definite diagonal matrix. Set

(3.7) F = JHZ~t-1/2P, P = 0 "

Then

(3.8) F T A F = -- P~tJP, FTjF = J.

The second equality in (3.8) follows easily from the second one in (3.6) (note that
PJP =J). F o r the first equality in (3.8) we have
-

F T,4F = Pet - 1/2 ZTHJHTjHJHTZ~ - 1/2p

= p ~ - 1 / 2 Z T H J H T Z Z - 1 J Z - T Z T H J H T Z o t - UEp

= Pot - 1/2 ( _ g j ) j ( _ g j ) ~ - 1/2 p = Po~JP,

which is diagonal. N o w
pFT(A, J ) A F P = - P(-ctJ, J) = ZT(--/L --J),

5 It is a little annoying that before stopping one void cycle must be run with some n2/2 scalar
products in order to check (3.3). However, this increases the whole operational count by not more
than 4-5%.
A Jacobi eigenreduction algorithm for definite matrix pairs 255

and A, J is positive definite. Conversely, if A, J is positive definite then (3.8) holds

and for Z = H - ~JFP~ i/2 we have

z T J Z - cx1/2P J F - 1H - T J H - i F -TJPct 1/2

= ct i/2 PJ(PctJP) - 1jpc~ 1/2 = j

and
Z TAZ = ~ 1/2 p F T j F p a i/2 = _ aJ .

Here, as above, we conclude that (.4, J) is negative definite. Q.E.D.

Now the algorithm runs on ,4, given by the factor H which upon con-
vergence and J-normalization yields the eigenvector matrix H ~ - 1 / 2 =
H Z a - 1/2p for the original pair. All remaining details are as in case M above. 6
The stability problems in computing H in (3.4) are obviously of a similar nature
as those with A in (1.11). However, H contains only the factors L l, L 2, L 3 and an
improvement in accuracy can be expected if these factors are available without
previously forming K, M, D [1] or are computed by using higher precision arith-
metic in the Cholesky decompositions. In this case our implicit method reminds of
a singular value decomposition and may share its advantages in accuracy, if the
matrices K, M, D have high condition numbers (see Example 4.6 below).
The operational count for obtaining H is 5(n/2) 3/6 = 5n3/48 which is about 14
times less than that of one Jacobi cycle. Since one Jacobi process takes generally
not less than 6-8 cycles the cost for computing H can be considered as negligible.
There is one interesting aspect of our implicit algorithm. The transition
from the matrix A = H J H T to d = H T j H is nothing else but one step of a
"J-symmetric" LR method for the pair A, J. This carries usually some non-
negligible diagonalization effect. Indeed, according to (1.11) we have

(3.9) TrA = TrL?IDL~ ~ = TrDK-i

A=F L;~ML;T-LTK;~L3 L;~ML;T]

(3.10)
L L3iML3 T L 3 1 M L 3 T]

(3.11) Tr(- A) = T r ( - H T j H ) = T r D K - i _ 2TrMD-1

which is always less than TrA (note that the definiteness sign changes with the
transition from A t o / ] ! ) .
Similar remarks hold for the transition from a single matrix A = GJG r to the
auxiliary pair GTG, J in case ~r This effect is stressed by the fact that the indefinite
symmetric decomposition A = GJG T is made with total pivoting. This has some
further favorable stability consequences ([33]).

6 The implicit method is much more difficult to design for matrix pairs A, J where A and J are
both indefinite and have different inertiae. In fact, in this case it is easily shown that the
definiteness of the original pair does not carry over to the auxiliary one.
256 K. Veseli6

4. Experimental results

We present here some experimental results obtained by three algorithms:

(1) jsm is obtained from the Rutishauser algorithm jacobi [41] by merely
substituting the trigonometric formulae by the hyperbolic ones on the pivot
pairs p, q with 1 < p < m, m < q < n. It needs some 4n 3 operations per cycle.
(2) jsmf uses in addition fast rotations (both trigonometric and hyperbolic,
cf. [28]). It needs some 2n 3 operations per cycle. Its storage need is about the
same as with jsm both of them working on two n x n arrays.
(3) jsmp is the implicit algorithm described in Sect. 3 and uses fast rotations as well.
It needs some 3n3/2 operations per cycle and uses one square array, accessed
only columnwise. 7
The output ofjsm is almost identical to that ofjsrnf, except that the former is
about twice as slow. Both methods are also used on a single symmetric matrix thus
reducing to the standard Jacobi method.
The stopping criteria have been taken in accordance with the respective
machine precision macheps. In particular, Jacobi processes were stopped after

Ia,s [ < l O m a c h e p s ~
was satisfied for all r < s. The above inequality was also used as a threshold within
the row-cyclic strategy.
The hyperbolic parameter y was bounded according to (2.20) with tmax 0.8. =

The first sweep consisted of only trigonometric transformations. The algorithms

were compared with (symmetric) ql and the non-symmetric qr algorithm from [41],
respectively. The latter is used on the single non-symmetric matrix JA (this is
certainly the proper known alternative for a pair A, J whose definitizing shift is not
known in advance; the transition to a single matrix JA just means changing some
signs in A). We let qr compute only the right eigenvectors. Their J-orthogonality
was as a rule only slightly weaker than that of Jacobi (which was always attained to
the full precision). U p o n multiple or clustered eigenvalues, however, qr fails to give
J-orthogonality and needs some more calculation to obtain the left eigenvectors.
We have tested matrices of order n < 50. Our experiments were mainly done on
an MS-DOS-operated PC with the Intel 80286 and 80287 processors using Turbo
Pascal with macheps~, 2 . 1 0 - 1 6 . (All time data displayed below will concern this
environment. Some experiments were repeated on an IBM 370 under Delft Fast
Algol 60, 1/2/77 with the same precision. The results were similar to those displayed
below.) All our time data have only a relative meaning: as the time unit we have
simply taken the time for the compared ql/qr algorithm, s
Occasionally, in comparing accuracies we also used the double-precision Q R
and Q Z method from P C M A T L A B as well as ql/qr and Jacobi in Turbo Pascal
single (macheps,~ 10 - s) and extended (macheps,~ 10 - 20) precision arithmetic.

7 In fact, we use two different versions of that algorithm: one for case ~1 and one for case ~ in
Sect. 4. There will be no confusion in using the same name for both.
s Our time experiments are generally made on matrices, generated in some random way. If the
eigenvalues are about to be separated by the Gershgorin circles Jacobi methods become faster.
This could be efficiently exploited ff approximate eigenvectors are known and are used for
a pre-transformation of the matrix.
A Jacobi eigenreduction algorithm for definite matrix pairs 257

We now give the results of our numerical tests.

The first three examples treat single real symmetric indefinite matrices (for
positive definite matrices our jsmp reduces to an algorithm whose high accuracy
was proved in [7]). We have tried some hundreds of variously generated random
matrices. On no such example our jsmp was essentially less accurate than its
competitors ql and standard Jacobi algorithm. On well-behaved matrices with
eigenvalues of different order our method was usually much better. Typical com-
puting times were:
n jsm jsmp ql
20 3.4 2.6 1
30 4 3 1
50 4.2 2.9 1
We compare single precision algorithms jsmp, jsmp and qr on three examples.

4.1.1

I -10-101~ 1 1 1
A= 1 1 1 .
1 1 -1.0.10 s

We display the computed eigenvalues of single precision algorithms, compared

with the "correct" values, which consist of the figures common to the extended
precision jsmp and ql, rounded to 8 places.
correct jsm jsmp qr
+ 1.0000000-10 ~176+ 1.0000000-10 ~176+ 1.0000000.10 ~176+ 1.0240000-1003
- 1.0000000-10 ~ -- 1.0000000-10 ~ - 1.0000000.10 ~ - 9.9999744.1007
- 1.0000000.101 o -- 1.0000000 9101 o - 1.0000000 9 101 o - 1.0000001 9 10 l~

On such matrices (in [2] they are called scaled diagonally dominant) both Jacobi
methods are good. It is interesting to note that on many matrices the accuracy ofql
or even standard Jacobi, may strongly depend on the permutaton of rows and
columns of the initial matrix. On the present example, however, no permutation
helped ql to compute one single digit of the smallest eigenvalue correctly whereas
jsmp and jsm were insensitive to such permutations.

4.1.2

I
11 11 a1 11 1
A= 1 a 1 1 "
I 1 1 b

We took the following choices: (i) a = - 5- 107, b = 10 6, (ii) a = - 10 9,

b = 106, (iii) a = - 10 lo, b = 10 8. In spite of its condition number this matrix is
well-behaved ([38]). Here, too, we have tried the initial matrix in all 24 permuta-
tions of rows and columns. All algorithms computed the large eigenvalues correctly
258 K. Veseli6

in all cases. Also, the small eigenvalue 1 + O(1/b) was always correct withjsmp. The
accuracy ofjsm was excellent for some permutations, independently of the choice
(i), (ii), (iii) and catastrophic for other permutations. The situation with ql was
amazing: in both (i) and (ii) there was exactly one permutation yielding correct
small eigenvalue, but these permutations were different. For (iii) no permutation
helped ql to get any correct digit?

4.2

A=
[110 ]
1 0
1 0 I0-7

The well-behavedness of this matrix is partly based on its zero pattern ([38])
and it has no positive definite analogue. Here jsmp got no correct digit of the tiny
eigenvalue. This is not surprising, if one remembers that the indefinite symmetric
decomposition [5] begins by a 1 x 1 Gaussian elimination step on the pivot 1, 1
which has disastruous consequences. If [5] is told to make a 2 x 2 step instead, then
everything is accurate. It seems to be a non-trivial open problem to modify the
pivoting strategy of the indefinite symmetric decomposition to cover all such cases
(cf. also [33]).

4.3 Oscillator ladder matrix

The matrix A - I has the form (1.11) with

M = I, C = diag(c x. . . . . c,,), m = n/2,

-1 - 12 "'-
K = I 2 1
9 ~ " " --1

-1 2

If all ci are equal then both the eigenvalues and the eigenvectors of A are exactly
known (here both A -1 and the corresponding H - x are generated by the explicit
formulae for the Cholesky factors). The computing times were

cd~=4 c~=6
n ~m ~mf jsmp qr jsm ~mf jsmp qr
20 1.8 1.2 1.2 1 1.6 1 1 1
30 2 1.2 1.1 1 1.9 1.1 0.9 1
50 2.3 1.3 1.2 1 2.2 1.2 1.1 1

9 Although our experienceis limited to low dimensions,ql seemsto produce often "unexpectedly"
accurate small eigenvaluesfor some initial permutations (we see no simple rule to single out that
permutations), if the condition number does not exceed 1/macheps.Somewhere above this limit
the accuracy is worsening rapidly.
A Jacobi eigenreduction algorithm for definite matrix pairs 259

For ci = 4 and n ~ ~ the definiteness interval shrinks to a point and the

condition of the eigenvalue problem grows to infinity. Indeed, taking here n = 50
and cl = 3.9 the pair has already 6 non-real eigenvalues. In this case jsmp dis-
covered the non-definiteness in only 1/10 of the usual time. On pairs which are
"more indefinite" this discovering goes much faster. Only if the pair is very near the
indefiniteness limit Jacobi methods will need longer times either to converge or to
discover the indefiniteness.
The observed accuracy of the eigenvalues was equally good for all tested
methods. In the case of large damping a natural dichotomy takes place: the "plus"
eigenvalues are absolutely much smaller than the "minus" ones. Then Jacobi
methods became better. For example, with n = 10, c~ = 1.00004.105 the two
absolutely smallest eigenvalue were, computed by a stable exact formula
-8.10108123391679.10 -7 -6.90250922120190.10 -6

whereas our algorithms yielded

jsm -8.10108123391675.10 -7 -6.90250922647791.10 -6
jsmf -8.10108123391675.10 -7 -6.90250922647791.10-6
jsmp -8.10108123391684-10 -7 -6.90250922120188.10 -6
qr -8.10103887527447.10 -7 -6.90251181367785.10 -6

Note that here we were not able to improve the accuracy of qr results by
lowering its tolerance epsilon.
The P C M A T L A B QZ algorithm was here equally good. It yielded
-8.10108123391679.10 -7 -6.90250922120189.10 -6

4.4 Local growth o f the eigenvector condition

We consider here pairs A, J with

A = ~ vlv T + ~lI = WW T "~ ~1I, 1 < r < n < 50,

i=l
where r / > O, and W = (vl . . . . . v,) and v~ are vectors with random components
between _+ 0.5. Such a pair has two interesting properties: (i) Even for r / = 0 it is
"almost always" diagonalizable. (ii) For ~/= 0, r < min(m, n - m) the pair is not
definite (and lies therefore on the boundary of the definiteness region). In fact, (i) is
seen from the formula
1
(A -- 2J) - 1 = i ( J __ J W ( ~ I -- w T j w ) -1 w T j ) ,

which shows that the eigenvalue 0 is defective only if W r J W i s singular (and this is
"almost never" the case). Since all non-vanishing eigenvalues are non-defective (i)
follows. Now, if the pair W W x, J were positive definite, then according to Th. A.1
the form xX,lx would be definite on the nullspace of W W r (which is "almost
always" n - r-dimensional). By the Sylvester inertia theorem this is impossible, if
n - r > m and n - r > n - m. Thus, (ii) holds.
260 K. Veseli6

We have tried a total of 3630 matrices for n = 10, 20, 30, 50 and various choices
of r, m, r/producing matrices from the boundary of the definiteness region or its
small neighbourhood (r/= 20i. macheps, i = 0, 1, 2, 3). The tested matrices were
distributed over the dimension n as follows

dimension n 10 20 30 50
number of tested matrices 1680 670 360 220
We have measured two condition numbers of the current eigenvector matrix
Z = (zl . . . . . z~):
IlZll~ and llZll~ = max[lz/l[ 2
Here obviously
1 < [tZllz < IlZ[l~, n < llZ[l~

Possible measures for the stability of the eigenvector computation are the
quantities
maximal l[Z II~ maximal llZ I[
finalllZ[l~ " ~= finaIllZIl~
The following table shows the maximum values of the measured (~, (l:
n 10 20 30 50
(E 2 4.4 2.8 1.4
~t 4.25 9 10.1 6.2
(the final conditions themselves never exceeded some hundreds i.e. the generated
matrices had well conditioned eigenvectors). On all other tested types of matrices
(e.g. those with r = n) these quotients were always about one. On matrices
A = GrG obtained by (3.1) with total pivoting the danger of high condition is
essentially smaller and well-controlled (s. [33]).
We also have tested a number of matrices with high eigenvector condition (they
were obtained by taking r = n thus producing a positive definite A and then fitting
a negative ~/so as to bring the pair A, J to the boundary of the definiteness region).
In all such cases both values (E, (t were 1. The results also fully confirmed the
perturbation estimate (A. 11) from Th. A.4. Moreover, we have also compared the
relative accuracy of the computed QR and Jacobi eigensolutions in single precision
(macheps,~lO-S). The comparison was made against extended precision
(macheps ~ 10- 20) results of both QR and Jacobi. Here Jacobi was hardly essen-
tially worse and often better than QR - - irrespectively of the condition of the
eigenvector matrix.

4.5 General overdamped pencils

The pair A, J is obtained from M, D, K which are generated as sums of n random

dyadics. In order to insure the overdampedness
D=D+M+~K, ~>1
A Jacobi eigenreduction algorithm for definite matrix pairs 261

is set. F o r ct = 2 we o b t a i n e d the following times.

n j s m f jsmp qr
20 1.1 1 1
30 1.2 1.1 1
50 1.4 1.2 1

We have also measured the times for reducing to A a n d H. F o r n = 50 they were

a b o u t one percent of the total a m o u n t .
Here, too, we made relative accuracy comparisons analogous to those in 5.4
with the same results. As soon as the eigenvalues differed significantly in order of
m a g n i t u d e Jacobi became better, both in the eigenvalues a n d in the eigenvectors.l~

4.6

This is an example of a particularly badly conditioned overdamped system:

l+e 0.5 0
Mo=

Ei ~176 ~l
0.5
0
2+e
0
0
e
'

K = diag(e, 2 + e, 4 + e, e), D = Do + 10vv a', Do = M o + 1 . 5 K .

The bad c o n d i t i o n is based o n the fact that M, D, K have a c o m m o n eigen-

vector with the eigenvalues e, 2.5e, e a n d e is chosen as small. O n this eigenvector
the quadratic eigenvalue problem (1.6) has exactly k n o w n eigenvalues - 2 , - t / 2 ,
independently of the value e. F o r n = 8 and e = 10-12 the o b t a i n e d eigenvalues
were

qr ~mf jsmp o
-1.99002011775738-10 - I - -1.99002011664866-10 - 1 --1.99002011664867.10 - I
-3.75778707876245.10-1 -- 3.75778706961464.10 - 1 --3.75778706961460.10 -1
-5.00059269396540.10 -1 - -5.00059269396839.10 - 1 --4.99999999999999.10 -1
- 1.00000000000000.10 -1 - -9.99999999999975.10 - 1 --9.99999999999975.10 -1
-1.62422129292789 -- 1.62422129303876 -- 1.62422129303876
-1.80099798742062 - -1.80099798833540 --1.80099798833542
--1.99994073060351 - 1.99994073060316 - - 2 . ~ 2
-2.(X)OOOXXXX)O090.1013 -- 2.00000000000090- I013 --2.000000(0)O30090.1013

lo It would be certainly interesting to make an equally exhaustive comparison with the QZ

algorithm, applied directly to the pair M, S. Hovewer, the only QZ code we had at hand was the
MATLAB one, which is available only in double precision. Our results below indicate that the QZ
algorithm although more accurate than QR cannot compete with Jacobi. Here some more
experimental investigation is indicated.
262 K. Veseli6

jsmp PCMATLAB QZ perturbation theory

- 1.99002011664867.10 -1 -1.99002011664867-10-1 - 1.99002011664720.10 -1
-3.75778706961462.10 -1 -3.75778706961462.10-1 -3.75778706961357.10 -1
-5.00281420444726.10-1 -5.00814898903433-10-i -0.5(exact)
-9.99999999999975.10-1 -9.99999999999975.10 - i -9.99999999999975.10 -1
-1.62422129303876 -1.62422129303876 -i.62422129303864
-1.80099798833542 -1.80099798833541 -1.80099798833528
-1.99905265259173 -1.99784809630606 - 2 (exact)
-2.00000000000089.1013 -2.00000000000090.10 la -2.00000000000015.1013

H e r e jsmpo differs f r o m jsmp in t a k i n g m o r e a c c u r a t e C h o l e s k y factors as

follows: M o , D o , K are f a c t o r i z e d a c c u r a t e l y a n d the sensitive C h o l e s k y f a c t o r s
L1 a n d L3 are c o m p u t e d by the k n o w n d y a d i c - u p d a t i n g a l g o r i t h m ( [15-1 ) w h i c h
a v o i d s the s q u a r i n g . T h e last c o l u m n a b o v e is o b t a i n e d u s i n g the e x a c t f o r m u l a e for
the l i m i t i n g case e = 0 c o r r e c t e d by the s e c o n d o r d e r p e r t u r b a t i o n t h e o r y in e. N o w
jsmpo is clearly s u p e r i o r in the a c c u r a c y - - e v e n to Q Z .

4.7 Another badly conditioned overdamped pencil

M , D, K are as follows

3 2 2
M=I, K= 2 3 2 ' D= 1 2 1,

2 2 3 1 1

w i t h e 1 = 1 . 1 1 . 1 0 - 12. T h e e i g e n v a l u e s are e x a c t l y k n o w n a n d up to an e 2 r e l a t i v e
e r r o r t h e y are e q u a l to - 5/e ~, - 1/e ~ (triple), - e 1 (triple), - 9 e l / 5 . H e r e we
h a v e c o m p a r e d several P C M A T L A B r o u t i n e s w i t h jsmf. W e h a v e tried the m a t r i x
A~ = A -1 f r o m (1.11) u s i n g P C M A T L A B a n d its f u n c t i o n chol. W e t h e n c o m -
p a r e d v a r i o u s P C M A T L A B Q R / Q Z possibilities w i t h o u r jsmf a l g o r i t h m . T h e
results b e l o w a r e labeled as follows

ansqr o b t a i n e d by P C M A T L A B eig(J * A ~),

ansqzl o b t a i n e d by P C M A T L A B eio(A 1, J ) ,
ansqz2 o b t a i n e d by P C M A T L A B ones(8, t)./eig(J, A x ),
ansqz3 o b t a i n e d by P C M A T L A B eio(T, S),
ansqz4 o b t a i n e d by P C M A T L A B ones(8, 1)./eig(S, T).

H e r e T, S are t h e m a t r i c e s f r o m (1.9). T h e o b t a i n e d e i g e n v a l u e s w e r e

ansqr ansqz l ansqz2

--4.50450450450450.10 + 12 -4.50450450450450 10 +12 --4.50450450450450-10 + 12
-- 9.00900900900901.10 § 11 - 9.00900900900901 10+11 --9.00900900900901.10 + 11
-- 9.00900900900901 910 + 11 -9.00900900900901 10 TM --9.00900900900901-10 + 11
-- 9.00900900900901.10 + ix - 9,00900900900900 10+11 -- 9.00900900900901.10 + 11
--9.13967715116028.10 - ~ - 1.77905414160207 10-12 -- 1.99799999999999-10 - 12
- - 2.20163083800163.10- o6 - 1.01441465334759 10-12 -- 1.45894819453302.10 - 1z
+ 2.12686427028049.10 -o5 - 7.52426853806064 10-13 - 1.11000000000000-10 - 12

+ 4.18744903335842.10 - o5 + 4.66343697748042-10 - xI + 8.59909442443810.10 - ~3

A Jacobi eigenreduction algorithm for definite matrix pairs 263

ansqz3 ansqz4 jsmf

-4.50450450450450-10 +,2 -- 1.18651723887948.10 + lz --4.50450450450450.10 + t 2
-9.00900900900901.10 +ii -- 2.40428417339426.10 § i 1 --9.00900900900899.10 § i 1
-6.18613755414509-10 +li --3.66748085228670-10 § ao --9.00900900900899.10 § t 1
-2.58525280928511.10 +li -- 1.99800000000000.10 - 12 --9.00900900900899-10 § 11
--1.99800000000000.10-12 -- 1.11000000000000 9 10 - 12 -- 1.99800000000000.10 - 12
-1.I1000000000000-10 -12 - 1.11000000000000 9 10 - 12 -- 1.11000000000001 9 10 - t2
--1.11000000000000-10 -x2 - 1.11000000000000 91 0 - 1 2 -- 1.1 t000000000000 9 10- i2
-1.t1000000000000.t0 -12 + 3.08570876918623.10 + i2 -- 1.11000000000000-10 - 12

Here only jsmf was able to reproduce both the small and the large eigenvalues
correctly. Note also that neither A nor A - 1 is diagonally d o m i n a n t in the sense of
[2] thus indicating further classes of matrices on which Jacobi methods are
accurate.

A. Some properties of definite pairs

In this section we present some theoretical results mainly concerning the eigen-
vector condition of definite pairs. The results are mostly new. Although useful in
c o n n e c t i o n with our algorithm, they may have i n d e p e n d e n t interest. A particularly
strong result is T h e o r e m A.4 which describes the behaviour of the condition of the
eigenvectors of a pair A, J near the b o u n d a r y of the definiteness region. We prove
that the c o n d i t i o n n u m b e r is O(1/x/d), d the distance to the nearest indefinite pair.
Thus, although possible, the b a d condition is highly i m p r o b a b l e and, which is
much more i m p o r t a n t , it can be sensibly improved by only a slight change of the
matrix.
The first two assertions of the following theorem are elementary a n d their proof
is included for the sake of completeness. The last assertion is i m p o r t a n t a n d seems
to be new.

Theorem AI. Let the pair A, B be, say, positive definite and let B be non-singular.
Then
(i) There exists a n o n - s i n g u l a r Z such that

(A.1) Z T A Z = ( o+ --A_O ), ZTBZ=j

where A + . diag(2~-,
. . . . 2,,+ ), A _ = diag(21- . . . .2~--m)
. . and

(A.2) j = ( lm O)
0 -- In- m

where (m, n - m) is the inertia of B.

(ii) F o r any definitizing shift p we have
iF ..... 2~+ > ~ > 2i- . . . . . ;t2,-,

(iii) Any two matrices Z and Zo performing (A.1) are connected by

Z = ZoU
264 K. Veseli6

where

and U 1,2 is an orthogonal martix of order m, n - m, respectively. The diagonal

blocks A • and A ~ , obtained by Z, Zo, respectively, are equal up to a permutation.

Proof. Let # be any definitizing shift. Then as it is well known there exists a non-
singular Z 1 such that
ZT(A -- l , B ) Z l , fl = Z T B Z l
are diagonal where the first m diagonal elements of fl are positive and the remaining
ones are negative. Taking Z = Z 1 ]ill - 1/2 we obtain Z T B Z = J and ZT(A -- # B ) Z
diagonal. Then

ZTAZ = 0 -A _

is diagonal as well, which proves (i). (ii) follows from (i) and the positive definiteness
of
ZT(A_#B)z=(A+ -# 0 ) .
0 -A-+I~
N o w we prove (iii). Let Z o and Z perform (A.1) i.e.

ZTAZo =
0)
-A o , Z T B Z o = J, A~:diagonal.

Then for D = Z - 1 Z o we have

DTjD = J,
and

,A3, _Ao):OT( o )o
By the J-orthogonality of D we have

,A4) o(O 0 A~ D-1 = A_

0)
N o w (A.3), (A.4) together with A ~ < # < A ~ imply that A • and A ~ are equal up
to a permutation and that D c o m m u t e s with J i.e. it must be of the form

o(T0
where by the J-orthogonality of D the matrices U 1, U 2 are orthogonal. Thus
(U~ 0)
Z = Zo 0 U2

and this is the assertion (iii). Q.E.D.

A Jacobi eigenreduction algorithm for definite matrix pairs 265

Corollary A2. All matrices Z performing the transformation (A.1) for a definite pair
A, B have the same (Euclidean, spectral) condition number.

Theorem A3. Let the condition of Theorem A.1 be fullfilled with B = J. Then the
Euclidean condition of Z is bounded by
(A.5) c o n d E Z = flZllEIIZ -x II~ -< n + 2p(A) ,
with
~, n-m
(A.6) tro = T r A + - T r A _ = 2+ - ~, 27
i=1 i=l

(p(A) from (2.33)).

Proof. By the J-orthogonality of Z the left hand side of (A.5) reads simply T r Z T Z .
A straightforward computation shows that the polar decomposition of any
J - o r t h o g o n a l matrix reads

(A.7) Z=H(X)( U~O U20 ) ,

x/I + X X T X )
(A.8) H(X) = XT x/I + x T x ,

where X is some m x (n - m) matrix and U1, U2 are some orthogonal matrices of

order m, n - m, respectively. N o t e that
H(X) -1 = H ( - X), H(X) J-orthogonal, symmetric, positive definite.

Let now (A.1) hold. Then

+ 0 )Z -1
(A.9)
-Z_

where

A+ = diag(2 + . . . . . 2+),
(A.10)
A _ = diag(2i- . . . . . 2~,_,).
Therefore

T r A = T r [ x / I + X X T A +x/I + X X T -- X A _ X T + X T A +X

- ~/1 + X T X A _~/1 + X TX]

= tro + 2 T r [ A + X X T - A - x T x ] >1 tro + 2 6 T r X T X .
By
TrZTZ = TrH(X)2 = n + 4TrXTX

the formula (A.5)follows. Q.E.D.

266 K. Veseli6

The formula (A.5) is, of course, only an upper estimate for the condition
number of Z and there are matrices for which this estimate is pessimistic. In such
cases it is likely that a non definite pair A 1, J could be found in a small neigbour-
hood of A, J.
In the following theorem we derive an asymptotic estimate of the condition of
the diagonalizing J-orthogonal matrix as A approaches the boundary of the
definiteness region. Without loss of generality we can assume that the spectral
point where the plus and the minus eigenvalues come together is zero.

Theorem A4. Let A and V be positive semidefinite such that the definiteness interval
of A, J is void (this implies that A is singular). Let, in addition, A + V be positive
definite or, equivalently,
Ax=O ~ xTVx>O.

Then A + ~ V is positive definite for any e > 0 and the spectral condition of any
J-orthogonal matrix Z diagonalizing the pair A + e V, J is estimated by

(A. 11) condZ = ]lZlIllZ -1 II ~ cons t ' e - 1 / 2 + O(1),

(see (A.15) below for a more explicit estimate).

Proof. For e small enough there exists a J-orthogonal F such that

A/ll 0 AP12
(A.12) FT(A + eV)F = 0 A'oo 0
A ?2 0 A ~22

where F = F ~ + gF 1 -t- 0(~ 2 ) is analytic at e = 0,

is positive definite even for e = O,

A'oo = A~ + e(F~176 + O(e 2)

and the pair A~ Jo has only one multiple eigenvalue zero. Here

J = diag(I, Jo, - I ) ,
is partitioned as in (A.12). The existence of such an F is guaranteed by the analytic
perturbation theory (see Kato [24], Ch. II w where F is called the transforma-
tion function). For e > 0 Abo is positive definite and the pair Abo, Jo is diagonal-
ized by a matrix Zo, obtained as follows. We take L with A~,o = L L T and find an
orthogonal U such that
UTL-1JoL-T U = a-lJo ,

where a is a diagonal positive definite matrix. Putting

Z 0 = L-TUo~I/2
A Jacobi eigenreduction algorithm for definite matrix pairs 267

we obtain
'7 T A ! "7
Z~JoZo = Jo, Z,o,aooZ, o = ~ .
We have IIL-Tll = I I Z ~ 1 I11]2 The smallest eigenvalue of Abo is analytic at e = 0
and reads

IIA~ 1 II-i = w + 0(~ 2) ,

where
yTVy
v= inf
Ay=0 YT(F ~ 1 7 6 -ly > 0 .

On the other hand, let/~ be any of the diagonal elements of ~. Then/~ > 0 and

(A'oo - ktJo)x = (A~ - #Jo + A)x = O,

where A = A ~o - A ~ and x is some non-vanishing vector. This implies

(A.13) IIA(A~ - M o ) -1 II _-__1

(note that A~ - #Jo is invertible for every #:~0). Since d o A ~ is nilpotent and
A ~ is positive semidefinite it is readily seen that already the square of J o A ~ or,
equivalently, the matrix A oooJoA ooo vanishes. Thus,

Jo JoA~
(A.14) (A~ - # S o ) - 1 = _ __ +
# #:

N o w (A.13) and (A.t4) imply

1 < IIAtIII(A - # J o ) - l l l < IlZll + < IlZll + ~y 9

F r o m this we obtain
/~ < IIAtl + Ilf~

and thus,
t)~1/211 ~ ~IIAII + IIF~ ~A-HIIAII.
Putting
Z = F diag(I, Zo, I)

we obtain
z T J Z = J, ZT A ' Z = d i a g .

Now

(A.15) IlZlt = llZ -1 II 5 [1FIIIlZo I1 ~ IIFII ll(Abo)-1 II 1/2~ItAII + lfF ~ tI~ItAIIllAII

ItF~ V IlAIlll vll ]1/4

=~L ~ + ~

By squaring this we obtain (A.11). Q.E.D.

268 K. Veseli6

Theorem AS. The system described by (1.6) is overdamped if and only if the pair A, J
from (1.11) is positive definite.

Proof. It is well k n o w n (see e.g. [10]) that (1.7) holds, if and only if there exists
a real # such that
--(M/,t 2 + D,u + K)

is positive definite. By the positive definiteness of K, D and M the shift # is always

negative. N o w

(A.16) T-/~S=-#(S- #-IT)

[, -- btI 0 -~2M
0
+/~D + K)
]['o ;']
Thus, the positive definiteness of the pair S, T and therefore the positive definite-
ness of A, J is equivalent to the overdampedness condition (1.8). Q.E.D.

Acknowledgements. The first version of this article is more than six years old and during this time
it has incited several follow-on papers I-8,9, 33, 38] which, in turn, influenced our present version.
In addition, Stapni~ar, in preparing his thesis 1-331made a first implementation of our eigenreduc-
tion in case ~r above. This implementation was used in our experiments. The author thanks the
authors of these papers for early communicating their results and experiences. He also thanks I.
Slapni~ar, Hagen and A. Suhadolc, Ljubljana, for many valuable discussions.

References

1. Argyris, J.H., Bronlund, O.E. (1975): The natural factor formulation of stiffness for the matrix
deplacement method. Comput. Meth. Appl. Mech. Engrg. 40, 97-119
2. Barlow, J., Demmel, J. (1990): Computing accurate eigensystems of scaled diagonally
dominant matrices. SIAM J. Numr. Anal. 27, 762-791
3. Brent, R.P., Luk, F.T. (1985): The solution of singular-value and symmetric eigenvalue
problems on multiprocessor arrays SIAM J. Sci. Statist. Comput. 6, 69-84
4. Brent, R.P., Luk, F.T., van Loan, C. (1985): Computation of the singular value decomposition
using mesh-connected processors. J. VLSI Computer Systems 1, 242-270
5. Bunch, J.R., Parlett, B.N. (1971): Direct methods for solving symmetric indefinite systems of
linear equations. SIAM J. Numer. Anal. 8, 639-655
6. Chartres, B.A. (1962): Adaptation of the Jacobi Method for a Computer with Magnetic Tape
Backing Store. Comput. J. 5, 51-60
7. Demmel, J., Veselir, K. (1992): Jacobi's method is more accorate than QR, Courant Institute
preprint 1989. SIAM J. Matr. Anal. Appl. (to appear)
8. Donner, K. (1990): Uber ein Jacobi-Verfahren ftir definite hermitesche Matrix-Paare. Fern-
universit/it Hagen, Diplomarbeit
9. Drma~, Z., Hari, V. (1993): On quadratic convergence bounds for the J-symmetric Jacobi
method. Numer. Math. (to appear)
10. Duffin, R.J. (1955): A minimax theory for overdamped networks. J. Rat. Mech. Anal. 4,
221-233.
11. Durand, E. (1972): Solutions numeriques des equations algrbriques. Masson, Paris
12. Eberlein, P.J., Boothroyd, J. (1968): Solution to the eigenprobtem by a norm reducing Jacobi
type method. Numer. Math. 11, 1-12 (also [41-] p. 327)
13. Falk, S., Langemeyer, P. (1960): Das Jacobische Rotationsverfahren fiir reellsymmetrische
Matrizenpaare I, II. Elektronische Datenverarbeitung, pp. 30-43
A Jacobi eigenreduction algorithm for definite matrix pairs 269

14. Falk, S. (1984): Ober halbimplizite und implizite Algorithmen zur endlichen und iterativen
Transformationen von Matizenpaaren. ZAMM 64, 437439
15. Golub, G.H., van Loan, Ch.F. (1989): Matrix computations. Hopkins University Press,
Baltimore
16. Gose, G. (1979): Das Jacobl-Verfahren ffir A x = 2 B x . ZAMM 59, 93-101
17. Hari, V. (1982): On the global convergence of the Eberlein method for real matrices. Numer.
Math. 39, 361-369
18. Hari, V. (1984): On cyclic Jacobi methods for the positive definite generalized eigenvalue
problem. Fernuniversitht Hagen, Dissertation
19. Hari, V.: On the global convergence of cyclic Jacobi methods for the generalized eigen-
problem. Department of Mathematics, University of Zagreb, preprint
20. Haft, V. (1991): On pairs of almost diagonal matrices. Linear Algebra Appl. 148, 193-223.
21. Hari, V., Veseli6, K. (1987): On Jacobi methods for singular value decompositions. SIAM J.
ScL Stat. Comp. 8, 741-754
22. Hestenes, M.R. (1958): Inversion of matrices by biorthogonalization and related results.
J. STAM 6, 51-90
23. Kaiser, H.F. (1972): The JK method: a procedure for finding the eigenvalues and eigenvectors
of a real symmetric matrix. Compout. J. 15, 271-273
24. Kato, T. (1966): Perturbation theory for linear operators, Springer, Berlin Heidelberg
New York
25. Luk, F.T., Park, H. (1987): A proof of convergence for two parallel Jacobi SVD algorithms.
Proc. SPIE 826. Advanced algorithms and signal processing II, San Diego, CA
26. Moler, C.B., Stewart, G.W. (1973): An algorithm for generalized matrix eigenvalue problems.
SIAM J. Numer. Anal. 10, 241-256
27. Nash, J.C. (1974): A one-sided transformation method for the singular value decomposition
and algebraic eigenvalue problem. Comput. J. 18, 74-76
28. Rath, W. (1982): Fast Givens rotations for orthogonal similarity transformations. Numer.
Math. 40, 47-56
29. Sameh, A.H. (1971): On Jacobi and Jacobi-like algorithms for a parallel computer. Math.
Comput. 25, 579-590
30. Scott, D.S., Ward, R.C. (1982): Solving symmetric-definite quadratic 2-matrix problems
without factorization. SIAM J. Sci. Star. Comput. 3, 58-110
31. Shroff, G., Schreiber, R. (1988): On the convergence of the cyclic Jacobi method for parallel
block orderings. Rensselaer Polyt. Inst. Dept. Comput. Sci. Report No. 88-11
32. Stapni6ar, I., Hari, V. (1991): On the quadratic convergence of the Falk-Langemeyer method.
SIAM J. Math. Anal. 12, 84-114
33. Slapni6ar, I. (1992): Accurate symmetric eigenreduction by a Jacobi method, Fernuniversit~it
Hagen, Ph.D. Thesis
34. Stewart, G.W. (1979): Perturbation bounds for the definite generalized eigenvalue problems.
Linear Alg. Appl. 23, 69-85
35. Veseli6, K. (1976): Some convergent Jacobi-like procedures for diagonalizing J-symmetric
matrices. Numer. Math. 27, 67-75
36. Veseli6, K. (1983): Global Jacobi method for a symmetric indefinite problem S x = 2 T x
Comput. Meth. Appl. Mech. Engrg. 38, 273-290
37. Veseli6, K., Haft, V. (1989): A note on a one sided Jacobi algorithm. Numer. Math. 56,
627-633
38. Veseli6, K., Slapni6ar, I. (1992): Floating-point perturbations of Hermitian matrices. Fern-
universitat Hagen, preprint
39. Wang, X., (t991): A few propositions on upper bounds for max 0 ~ m~ M x ( A , , , ) / x ( A o ) . Fern-
universitht Hagen, preprint
40. Wilkinson, J.H. (1968): Almost diagonal matrices with multiple or close eigenvalues Linear
Algebra Appl. 1, 1-12
41. Wilkinson, J.H., Reinsch, C. (1971): Handbook of Automatic Computation II Linear Algebra.
Springer, Berlin Heidelberg New York
42. Zimmermann, K. (1965): On the convergence ofa Jacobi process for ordinary and generalized
eigenvalue problems. ETH Zfirich Dissertation No. 4305