LINEAR

ALGEBRA

AND ITS APPLICATIONS

Global Convergence
Shifts*

of Tridiagonal

QR

409

Algorithm

with Origin

J. H. WILKINSON
National

Physical

Teddington,

Laboratory

England

1. INTRODUCTION

The QR algorithm with origin shifts [2-51 has proved a particularly

effective method for solving the symmetric eigenvalue problem. It can
be expressed in the form

R,QtT
+ k,I = At+l>

Q,(A,i 41) = R,,

where the Q, are orthogonal

t=

1,2,...,

and the R, are upper-triangular.

A t+1

(1)
Since

(2)

QtAtQt'p

symmetry is preserved throughout ; moreover if A, is tridiagonal this is

true of all A, and in practice the general symmetric eigenvalue problem
is usually solved by employing a preliminary reduction to tridiagonal
form. We shall therefore consider only the case when the A, are tridiagonal
and will write
(f)

ak,k+l

(*)

ak+l,k

(*I

ek+l

(3)

We assume that ek(*)# 0 (k = 2,. . . , n) because otherwise A, could be

split into smaller tridiagonal matrices. Hence all roots of A, (and therefore
of all A,) are simple, and roots of equal modulus can exist, if at all, only
in pairs, each pair consisting of two equal and opposite roots.
Two choices of origin shift are commonly used:
(a)

k, = tin(t);

* Dedicated

(4)

to Professor

A. M. Ostrowski

Linear
Copyright

Algebra

on his 75th birthday.

and Its Applications

1968 by American

Elsevier

1, 409-420

Publishing

Company,

(1908)
Inc.

(b)

k, = A,, where

A, is the eigenvalue
a!!!. ,

is closer

(3

to d I.

the algorithm
with

is always

blocks

immediately

convergent

of orders

available.

so far been concerned

that,

behavior

with shifts
in choice

A, tends

of the convergence
version,

of the shifted

since in the early stages

In this connection

for performing

This strateg!,

diagonal
are

of the process

has

i.e.,

k, = 0 for all t,
There remained

arc not necessaril!

A, might oscillate indefinitely.

is of interest.

simultaneously

k, and k, given by the eigenvalues

(b) above.

to block

the eigenvalues

version.

the shifts

the matrices

the following comment

a method

is used throughotlt, then

so that

with the unshifted

close to any of the eigenvalues,

has described

in that

1 and 2 only,

Analyses

and with the asymptotic

the danger

matrix

a,,()I

\Ve shall show thzt if either of these strategies

form

P,,f)

Ie,,()
which

of the 2

certainlv

Francis

[3, ii]

two steps of QR

of the 2 x 2 matrix
leads to oscillation

used

with the

matris
2

__~I
2

-1
I

-1

-I.

\Ye first prove a simple lemma

1.

~.EMI;MA

With either of the strutegies (a)

07

(b)

above all elements

idkt)l and lektt)i aye bounded b! maxIA, 1 and all elements derived during the
triangularization

of all .4, (and hence all elements of all R,) are hounded b~l

2 maxlil,l.
Proof.

Since

all A, arc orthogonally

similar

to -4, WC hnvc~

;A,& =J ~A,,I, = max~L,I.

Rut

every

element

of l/A,!/ is bounded

by

)(A,l&

(7
Hence

ldk(l, le, (1 < masli,j

k and t.
Kow k, is an eigenvalue

(8)

for all relevant

strategy

(a) or (b).

Hence

of a principal

submatrix

from the separation

of -4, whether
theorem

we USC

RR

ALGORITHM

WITH

SHIFTS

411

I& < maxIAl,

(9)

ORIGIK

giving

II4llz + llW2 G 2m44.

II4 - ~,~lls,<

PO)

It is well known that the Euclidean norm of any column (or row) of a
matrix is not greater than the 12 norm of the matrix itself, and in the
orthogonal triangularization
of A, - K,I to give R, the Euclidean norm
of each column remains unaltered throughout.
Hence all elements derived
during all triangularizations
are bounded by 2 maxIAil. We denote this
bound by M from now on.
2.

THE

BASIC

RELATIONS

The matrix A, - k,l is reduced to upper-triangular

form
multiplication
by 12 - 1 plane rotations
and we may write
R$,

n...

R$R$(A,

by pre-

- k,l) = R,,

(11)

where R$ is a rotation in the (i, i) plane. We denote the cosine and sine
associated with Rtt!
&,,1 1 by c!,.~ and s!!~, respectively. With either strategy
we write
$) = &(I - k,
and denote

the configuration

just before premultiplication

(12)
by Rf)_,,,

by

(13)

Clearly

pp)_
1

Linear

[(A$_1)2 + (ek()2]2,
Algebra

and Its ApPlications

(14)
1, 409-420

(1968)

J. H. WIT,KINSON

(15)

(16j

(17)

and

A,+,

( R$T,

is obtained

k,l

by

postmultiplying

Hence

(R$)?, . . . , (Rif_ , ,,) .

e.(
* i 1 = p;yt),

3.

GLOBAL

Mith

successively

b!

j = -YJ
_) . . . ) 92 - 1,

(19)

e v t 1)= x (Usvi
II
It ,I )

(20)

(jn(f t 1 z-_k, -}- ~~l,(~)p.

(21)

CONVERGENCE

strategy

this

we have

WITH

(a) we have

ct. II

STRATEGY

(a)

k, = d,,ct) and from

_

II

this

it follows

that

p,,,()c~f)
, (Q))2,

(22)

giving

The

elements

le+jt)( therefore

a limit L (say).
and column

n and combine

d,(t) being accepted

the convergence

monotonically

with a matrix

and thus

of order PZ-

of e,t) is ultimately

cubic

in this case.

e(l)
n 10.

und Its Applications

I,

tend

to

omit row

1, the current

The proofs given in [5, 71 show that

as an eigenvalue.

If L > 0 we show that

Linear Algebra

decrease

If L = 0 then e,() + 0 and we can ultimately

40!)-420

(1968)

QR ALGORITHM

Proof.

WITH

From

ORIGIN

SHIFTS

413

(22)

and the right-hand side tends to unity if Iez I--+L > 0. Hence I&, 1I~lf)/~+
1 and since both factors are bounded
above by unity
@i/ + 1 and
IQ/2 * 1. The former implies that for any positive E
IstL,I <E

for all sufficiently

large t.

(25)

Now from Eq. 19

Hence et)_, -+ 0 and we can ultimately omit the last two rows and columns
and work with a matrix of order 12 - 2.
Note that in this case the usual proof [5, 71 showing ultimate
convergence does not apply and it is easy to see that convergence
be quite slow. Consider, for example, the matrix

Ial > Ibl.

with

One step of QR using

strategy

(28)

(a) gives the matrix

(a2 + c2)@

0
be(a2 + Ed)-W

cubic
may

be(a2 + &2)-l/2
0
ba(a2 + E~)-~/~

ba(a2 + E~)-~/~
0

I*

W-9

It is clear that the convergence of eiii to zero is merely linear and that
if the eigenvalues
are f A,, & 1, then e!?i is ultimately
diminished
by
the factor i2,/3, per iteration.
If A, is close to As convergence is very slow.
Linear

Algebra

and Its

Applications

1, 409-

420 (1968)

Perhaps

a more

interesting

example

is the matris

(30)

1 (ignoring quantities of order 2). Since

which has the roots 1 & -.)-t. ,the shift in the first step is zero we cannot expect to make much progress
because

the shifted

modulus.

matrix

-After one iteration

has roots which are all of mucli the same

WC have

in fact

(31)

(X)
Although
regime

convergence

This example
IO-lo.

In general

iterations
then

was investigated
ezCt)changes

but ultimately

almost

immediate.

the accompanying
necessary
usually

is ultimately

cubic

it is a long time

this

values of c from 10 5 to
number

of

place and convergence

is

a rapid change

takes

\I:ith e == lo-lo

the history

tabulation.

to achieve

with various

very slowly for a considerable

It will be seen that

convergence

compared

required.

lWO( 1.OOOO0000 000)

lo-I(

l.OOOO 0000 001)

10---~(1.0000 0000 317)

10-y1.0000

0077 208)

IO-(1.0001

8767 107)

lW~(1.0488

2126 954)

IO-(1.755%

4888 838)

10-7 (1.1404 8440 541)

10-l

(2.9029 8947 528)

lW(4.7686

0295 751)

lo-z(1.6906

7771 311)

Algebra

md

Its

Applicntiovs

1, 4OB-4%)

of /eQCt)I
is gi\,cn in
47 iterations

were

with the 2 or 3 which arc

1W( 1.0000 0000 000)

Lznear

before

is established.

(1968)

QR

ALGORITHM

4. GLOBAL

WITH

ORIGIN

CONVERGENCE

WITH

415

SHIFTS

(b)

STRATEGY

Although the behavior of the iterates with strategy (a) is interesting

it is not of much practical importance since strategy (b) is almost invariably
used.

With strategy

(b) it is clear that

a,(, -

(eJ)) 2
8n 1

(33)

and
/a:L,l > (Q).

(34)

With this strategy it is not in general true that le,,@)Iis monotonic, but we
now prove that j&!_,e,()/ is monotonic decreasing. From Eqs. (14-21)
it is easy to show that
x,(f) z

$
(( n 11

s,(y~&l)

(35)

e V-I-1)= x,(os V)
n
n )

(36)

while
(0 V)
(I)
Y*-~ = e,-,k2.

(37)

Hence
(33)
from which it follows that
jen(t+l)j < leiL,I

(39)

because all other factors on the right of (38) are bounded in modulus by
unity. We have also
(40)
and from (38) and (40)

Linear

Algebra

and

Its Applications

1,

416

J, H. WILKINSON

Since

we have

and each of the factors on the right of IejttL,e,()Iis bounded by unity.

Hence

finally
lejt?i)e,(+r)( <

le$e,()/

and this means that leE),e,@)j + L (say).

we must ultimately
E. However,

I\ Ie()
n-1 1this

since [e,cr+r) <

The alternative

L = 0 then given any E > 0

reach a value of t for which either le$)r\ < E or lenCf)i<

reach a value of t for which

if

If

(44)

is that

means that in any case we must

IeltCt)j
< E.

L >O. We show that this is impossible.

For

L > 0,
(45)

and hence from

(43)

giving
jc,*(lI- 0,

Ic;Lr; -+ 0.

(47)

Now

~~
(e,(f))2---- =
[(e,(f))2+ (xKd21
and hence A$,

--* 0.

(48)

But
P&

= d!?,cj?_,

and since cam, ---t0 and xz,

Linear Algebra

(SnV))2 +

and Its Applications

e~l_,~j?_~,s~~,

+ 0 and dt:,
1, 409-420

is bounded
(1968)

(49)
this means that

QR ALGORITHM

WITH

ORIGIN

417

SHIFTS

e$,c~!_&_t),

+ 0.

(50)

However, @s/ and @.il -+ 1; hence e!Li -+ 0 and from (39) this implies
that je,()1 + 0. In all cases therefore efltr)+ 0, a most satisfactory conclusion.
.j. ASYMPTOTIC

BEHAVIOR

WITH

STRATEGY

(b)

The proofs of ultimate cubic convergence given in [5, 71 do not apply

when strategy (b) is used. We show that strategy (b) does ultimately
give quadratic convergence at least. We require a simple lemma.
LEMMA

2.

If

B is a real symmetric matrix and

QB=

R,

(51)

where Q is orthogonal and R is u#r-triangdar,

Proof.

then lr,,& 3 minlLJB)/.

We have

IIBQTeA = llRTenlla
= IrJ
But minIrZ,(B)
1=

miq+111B4I2 and

Proof of quadratic convergeme.

(52)

hence lrnnl> min &i(B).

Since the eigenvalues of A, are distinct

let
6 = minjl, -

$1.

(53)

Suppose we have reached the stage at which e,,() = E so that the shifted
A, is of the form

&

e/&

1)

(54)

where B is the leading principal tridiagonal submatrix of order n - 1.

Let h, . . . , pn_1 be the eigenvalues of B and Ai, . . . , A, be the eigenvalues
of the shifted A,. Then for some ordering of the A+
IA -/&I G e,
Linear

Algebra

lk+~G.,
and Its Applications

(55)
1, 409-420 (1968)

J.

418

H.

WILKINSON

and writing
fJ =,L& -

ii +

(56)

\\reha\,c

(57)
remembering

that

the separation

_At the penultimate

configuration

Completing

Hence

is the samr as that of the J.,.

of the A

stage in the reduction

in the last

two rows is

the transformation

we ha1.e

of the shifted

A, to

R, the

QX

ilLGORITHM

WITH

ORIGIN

419

SHIFTS

We see that convergence is at least quadratic though we will expect it

to be cubic in general, since there is no reason to expect that @,

will

continue to decrease with e, @)

. On the contrary, after adding back the
shift we expect @,
to be tending, in general, to some other eigenvalue
and hence a$i will be tending to the difference between two eigenvalues.
The convergence generally seems to improve rapidly even when an
initial step is merely quadratic. Consider for example the shifted matrix

(63)
One step of the algorithm gives

(1 +ees)li2

(1 + &2)12
&a/(1+ G)
&s/(1+ Es)
&Z/(1+ Es) e/(1 + e2) 1

so that convergence in this step is quadratic.

shifted matrix is

(64)

The next shift is

and the

(1 + Es)@
(1 +OEa)l!s - &+ &S/(1+ 9)
E2/(1+ E2)

.G/(l + ES)
-

E3/(i +

(65)

E2)

After the next step e, is of order e5.

ti. GENERAL

COMMENTS

In practice the convergence is generally cubic with either strategy,

but strategy (b) has proved to be quite significantly superior and it is
reassuring to know that it has global convergence properties.
The following considerations show why strategy (b) is usually superior.
If the QR is carried out without shift of origin then all subdiagonal
elements tend to zero. When shifts are incorporated the tendency for
e, @) to diminish is, of course, very marked, but usually e:), gets quite
small. Consider a matrix in which e,(j) = er and et), = e2, e1 being small
These terms have been left deliberately
and e2 being fairly small.
imprecise. With strategy (a) we take k, = d,(f) and usually this approximates a root of A, with an error which is of the order of magnitude of
Linear

Algebra

and Its Applications

1, 409-420 (1968)

J. H. WILKIXSON

420
~2.

On the other hand with strategy

(b) the shift usually

approximates

a root of A, with an error which is of the order of magnitude

From

the general

theory

we expect

is of the order of &/,I;_,

modulus

of the shifted

e, to be diminished

where A, and A;-,

matrix.

Hence

after

are the two roots of least

one step of strategy

by a factor of order sr2 and with strategy

is diminished
by a factor

of order e12~22. This effect

the QR algorithm

progresses

of the remaining

et(t) tend

usually

since toward
to be fairly

of e12e22.

by a factor which
(a) e,,

(b) it is diminished

becomes

more marked

the end of the process

as

several

small.

ACKNOWLEDGMENTS

The work described

above has been carried out at the Sational

Physical

Laboratory.
REFERESCES
1 F. L. Bauer, Numerische
symmetrischer
2 H. Bowdler,
algorithms

Matrizen,

Absch%tzung
Aplikace

R. S. Martin,

for symmetric

293-306.
3 J. G. F. Francis, TheQR
26.5-271, 332-345.
4 V. N. Kublanovskaya,
value problem,

und Berechnung

Matematiky

C. Reinsch,

tridiagonal

5 J. H. Wilkinson,

The Algebraic Eigenealw

1965.
6 J. H. Wilkinson,

The QH algorithm

The QH and QL

A:zwzeriscke Mathematik

for the solution of the complete

eigen-

555-570.
Problem, Oxford Univ. Press, London,

for real symmetric

J. 8(1965), 77-84.
5, 1968

Linear Algebra and Ifs Applications

ll(1968).

4(19til-1962),

matrices

with multiple

eigenvalues, Computer J. 8(1965), 85-87.

7 J. H. Wilkinson, Convergence of the LR, [JR, and related algorithms,

Received March

nicht-

178-189.

Parts 1 and 11, ComputevJ.

On some algorithms

Zh. vych. mat. 1(1961),

van Eigenwerten

and J. H. Wilkinson,

matrices,

Transformation,

10(19&j),

1, 409-480

(1968)

Comnputev