LECTURES ON NUMERICAL

METEOROLOGY
Roger K. Smith and Wolfgang Ulrich
December 5, 2008
Contents
1 Partial Dierential Equations 6
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 Some denitions . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.3 First order equations . . . . . . . . . . . . . . . . . . . . . . . 12
1.4 Linear second-order equations . . . . . . . . . . . . . . . . . . 15
1.5 Parabolic equations . . . . . . . . . . . . . . . . . . . . . . . . 19
1.6 Hyperbolic equations . . . . . . . . . . . . . . . . . . . . . . . 21
1.7 Elliptic equations . . . . . . . . . . . . . . . . . . . . . . . . . 22
2 Finite Dierence Methods 30
2.1 The basic ideas . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.2 The advection equation . . . . . . . . . . . . . . . . . . . . . . 33
2.3 Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.4 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.5 Time dierencing schemes . . . . . . . . . . . . . . . . . . . . 41
2.5.1 Two-level schemes . . . . . . . . . . . . . . . . . . . . 41
2.5.2 Three-level schemes . . . . . . . . . . . . . . . . . . . . 42
2.6 Properties of schemes - the oscillation equation . . . . . . . . 43
2.6.1 Two-level schemes . . . . . . . . . . . . . . . . . . . . 44
2.6.2 Euler scheme . . . . . . . . . . . . . . . . . . . . . . . 45
2.6.3 Backward scheme . . . . . . . . . . . . . . . . . . . . . 45
2.6.4 Trapezoidal scheme . . . . . . . . . . . . . . . . . . . . 45
2.6.5 Matsuno scheme . . . . . . . . . . . . . . . . . . . . . 46
2.6.6 Heun scheme . . . . . . . . . . . . . . . . . . . . . . . 46
2.6.7 Phase change of schemes . . . . . . . . . . . . . . . . . 47
2.6.8 Euler and backward schemes . . . . . . . . . . . . . . . 47
2.7 Three level schemes, computational modes . . . . . . . . . . . 48
2.7.1 Leapfrog scheme . . . . . . . . . . . . . . . . . . . . . 48
2.7.2 Stability of the leapfrog scheme . . . . . . . . . . . . . 50
2.7.3 The Adams-Bashforth scheme . . . . . . . . . . . . . . 51
2.8 Properties of schemes applied to the friction equation. . . . . . 52
1
2.8.1 Noniterative two-level schemes - see Eq. (2.44) . . . . . 52
2.9 A combination of schemes . . . . . . . . . . . . . . . . . . . . 53
3 The advection equation 56
3.1 Schemes with centred second-order space dierencing . . . . . 56
3.2 Energy method . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.3 Lax-Wendro scheme . . . . . . . . . . . . . . . . . . . . . . . 60
3.4 Computational dispersion . . . . . . . . . . . . . . . . . . . . 63
3.5 Group velocity . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.6 Schemes with uncentred dierencing . . . . . . . . . . . . . . 69
3.7 Schemes with centred fourth-order-space dierencing . . . . . 73
3.8 The two-dimensional advection equation . . . . . . . . . . . . 76
3.9 Aliasing error and nonlinear instability . . . . . . . . . . . . . 78
3.10 Ways to prevent nonlinear instability . . . . . . . . . . . . . . 81
3.11 Arakawas conservative scheme . . . . . . . . . . . . . . . . . . 81
3.12 Conservative schemes for the primitive equations . . . . . . . . 87
4 The gravity and inertia-gravity wave equations 90
4.1 One dimensional gravity waves . . . . . . . . . . . . . . . . . . 90
4.2 Two-dimensional gravity waves . . . . . . . . . . . . . . . . . 92
4.3 Inertia-gravity waves . . . . . . . . . . . . . . . . . . . . . . . 96
4.4 The normal form of the gravity wave equations . . . . . . . . 103
4.5 The leapfrog scheme and the Eliassen grid . . . . . . . . . . . 104
4.6 The Eliassen grid . . . . . . . . . . . . . . . . . . . . . . . . . 107
4.7 Economical explicit schemes . . . . . . . . . . . . . . . . . . . 107
4.7.1 The forward-backward scheme . . . . . . . . . . . . . . 108
4.8 Implicit and semi-implicit schemes . . . . . . . . . . . . . . . 110
4.9 The implicit scheme (trapezoidal rule) . . . . . . . . . . . . . 111
4.10 The semi-implicit method of Kwizak and Robert . . . . . . . . 112
4.11 The splitting or Marchuk method . . . . . . . . . . . . . . . . 114
4.12 Two-grid-interval noise suppression . . . . . . . . . . . . . . . 116
4.13 Time noise and time ltering . . . . . . . . . . . . . . . . . . . 118
4.13.1 Example of ltering . . . . . . . . . . . . . . . . . . . . 121
5 Methods for Solving Elliptic Equations 123
5.1 The Poisson equation . . . . . . . . . . . . . . . . . . . . . . . 123
5.2 General considerations . . . . . . . . . . . . . . . . . . . . . . 124
5.3 Richardsons Method . . . . . . . . . . . . . . . . . . . . . . . 125
5.4 Liebmanns method . . . . . . . . . . . . . . . . . . . . . . . . 127
5.5 Southwells Residual Relaxation Method . . . . . . . . . . . . 127
5.6 Successive Over-Relaxation (SOR) Method . . . . . . . . . . . 128
2
5.7 Tactics and Strategy for SOR . . . . . . . . . . . . . . . . . 131
5.8 Other Iterative Methods . . . . . . . . . . . . . . . . . . . . . 131
5.9 Fourier-Series Methods . . . . . . . . . . . . . . . . . . . . . . 132
5.10 Neumann Boundary conditions in SOR Method . . . . . . . . 135
5.11 Example of Relaxation Methods . . . . . . . . . . . . . . . . . 136
5.12 General elliptic equations . . . . . . . . . . . . . . . . . . . . . 138
6 Nonoscillatory Advection Schemes 141
6.1 Computational stability of nonoscillatory advection schemes . 142
6.2 General ux-corrected-transport (FCT) procedure . . . . . . . 144
6.3 Semi-Lagrangian approximations for atmospheric ows using
nonoscillatory advection schemes . . . . . . . . . . . . . . . . 148
7 Spectral Methods 156
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
7.2 An Example of the Spectral Method . . . . . . . . . . . . . . 161
8 Finite Element Methods 163
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
8.2 What is the nite element method? . . . . . . . . . . . . . . . 163
8.3 Simple operations with linear nite elements . . . . . . . . . . 166
8.3.1 Dierentiation . . . . . . . . . . . . . . . . . . . . . . . 166
8.3.2 Multiplication . . . . . . . . . . . . . . . . . . . . . . . 168
8.3.3 Second derivatives . . . . . . . . . . . . . . . . . . . . 169
8.4 Eciency, accuracy and conservation . . . . . . . . . . . . . . 171
8.4.1 Eciency . . . . . . . . . . . . . . . . . . . . . . . . . 171
8.4.2 Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . 172
8.4.3 Conservation . . . . . . . . . . . . . . . . . . . . . . . 172
3
Numerical Weather Prediction
The story goes that in 1904 Wilhelm Bjerknes was the rst to point out
that the future state of the atmosphere could be predicted by integrating the
partial dierential equations that govern the behaviour of the atmosphere,
using as initial elds the observed state of the atmosphere at a particular
time. However, the equations are too complicated for analytic solutions to
be found and one must resort to numerical methods. We refer now to such
integrations as numerical weather prediction, commonly abbreviated NWP.
The rst attempt at NWP was carried out by Lewis Frey Richardson
during the First World War. At this time the calculations had to be carried
out by hand and were very tedious and time-consuming. The result was a
spectacular failure, but the details were published in what has become one
of the most famous books in meteorology (Richardson, 1922).
Richardson estimated that it would require a team of 64,000 persons to
carry out a 24 hour forecast in 24 hours. This together with the unrealistic
nature of his calculation, which predicted a surface pressure change of 145
mb in 6 hours, cast doubt on the practicality of the method! Several later
developments changed this pessimistic view. Courant, Friedrichs and Lewy
(1928) found that space and time increments chosen to discretize the dier-
ential equations have to meet a certain stability requirement. Later, mainly
through the work of Rossby and his collaborators in the late 1930s, it was
found that the large-scale motions in the atmosphere could be represented
approximately by a rather simple equation expressing the conservation of
absolute vorticity following the motion of air columns. Finally, at the end
of World War II the rst electronic computer ENIAC (Electronic Numeri-
cal Integrator and Computer) was constructed. This computer was used by
Charney, Fjrtoft and von Neumann in the late 1940s for the rst successful
numerical forecast, based on integration of the absolute vorticity conservation
equation (Charney et al. 1950).
In the last four decades, progress in the accuracy and sophistication of
NWP models has been swift, partly through the development of improved
numerical algorithms for solving the equations and also because of the as-
4
tonishing technological developments in computer engineering.
In these lectures we attempt to survey some of the basic results from the
theory of partial dierential equations (PDEs) before describing a range of
numerical techniques for the solution of such equations. The emphasis will
be on techniques that have application to numerical weather prediction.
The basic theory of PDEs is covered in Chapter 1. The various techniques
for their solution are covered in subsequent chapters.
The main body of notes is concerned with grid point methods (Chapter
2-5). Later grid spectral methods and nite element methods are discussed.
Further reading
G. J. Haltiner, 1971: Numerical weather prediction. John Wiley, New
York, 317 pp.
G. Stephenson, 1970: An introduction to partial dierential equations
for science students. Longmon, 152 pp.
I. N. Sneddon, 1957: Elements of partial dierential equations. McGraw-
Hill, New York, 327 pp.
5
Chapter 1
Partial Dierential Equations
1.1 Introduction
The dierential equations describing atmospheric motion including those
used for numerical weather prediction (NWP) are examples of partial dif-
ferential equations. Partial dierential equations (PDEs) arise in problems
involving more than one independent variable and those with which we shall
be concerned here are ones that arise in the representation of various types of
physical processes such as diusion, wave propagation, or equilibrium states
of various physical systems.
An example is the equation for the temperature distribution, T(x, t), of
a heated rod (Fig. 1.1). In this case, the two independent variables are the
distance along the rod, x, and the time t.
Figure 1.1: Temperature distribution in a heated rod.
Physical laws show that for a rod with a uniform diusivity k, the tem-
perature satises the equation
T
t
= k

2
T
x
2
. (1.1)
Understanding the solution of this problem provides an understanding of
a range of problems of interest to the atmospheric scientist. An example
6
is the temperature distribution in the ground due to the diurnal variation
of heating at the earths surface. Knowledge about the temperature of the
earths surface as a function of time is needed for the complete solution of the
equations for the atmosphere, itself.
It is useful to have means of picturing the solution to a PDE. In the case
of Eq. (1.1), the solution can be expressed as a surface, z = T(x, t), in a
three-dimensional space (x, t, z), as shown in Fig 1.2. The domain of the
solution, D, is the region 0 t < and a x b. The temperature
distribution at some time t
0
> 0 is the curve z = T(x, t
o
), where the plane
t = t
0
intersects the solution curve. The curve z = T(x, 0) is the initial
temperature distribution.
Figure 1.2: Solution surface and solution domain, D, for Eq. (1.1).
Equation (1.1) relates locally (i.e. at any point x, t) the slope of the sur-
face in the t-direction to the rate-of-change of the slope in the x-direction. We
shall see that in order to obtain a unique solution, we need to say something
about the behaviour of the surface at the edges of the solution domain: at
t = 0 and at x = a, b. It would seem physically reasonable, for example, that
in order to predict the future evolution of the temperature, we need to know
the initial state, i.e., the initial temperature distribution in the rod, T(x, 0).
It would seem reasonable also that we need to know something about what
is happening at the ends of the rod at any particular time. We shall return
7
to consider such problems later.
In contrast to the general solution of ordinary dierential equations (ODEs),
which involve arbitrary constants of integration, the general solution of PDEs
involves arbitrary functions. Consider, for example, the equation

2
u
xy
= 0 (1.2)
Integrating partially with respect to x we obtain
u
y
= F(y),
where F(y) is an arbitrary function of y. We may integrate once again, this
time with respect to y to obtain
u(x, y) = f(y) + g(x), (1.3)
where f(y) =
_
F(y) dy and g(x) are arbitrary functions. To determine f(y)
and g(x) we need to have some additional information about the problem, for
example, the initial conditions (if time is one of the independent variables)
and/or boundary conditions. An example of the latter are the conditions that
might be required to solve the heated rod problem.
To be specic, suppose that we wish to nd u(x, y) satisfying (1.2) in the
region x 0, y 0 and that we are given that u = x when y = 0 and u = y
when x = 0. Then the surface u(x, y) must intersect the plane x = 0 in the
line u = y and the plane y = 0 in the line u = x (Fig. 1.3).
Let us see how the functions f(y) and g(x) in (1.3) are determined. We
are given that
u(x, 0) = f(0) +g(x) = x
and
u(0, y) = f(y) + g(0) = y
It follows that
u(x, y) = x + y f(0) g(0).
The only way this can satisfy the equation and the boundary conditions
is if f(0) and g(0) are both zero, whereupon
u(x, y) = x + y.
It is easily veried that this equation satises the equation and the boundary
condition. It may be shown also to be unique.
8
Figure 1.3: Solution surface u(x, t) for Eq. (1.2) satisfying the conditions
u = x at y = 0 and u = y at x = 0.
Clearly, boundary conditions are an essential part of the problem and in
order to obtain a meaningful solution, we need sucient boundary conditions
but not too many. We know, for example, that for a third-order ODE, we
require just three conditions to determine a unique solution; with only two
conditions the problem is under-prescribed and, with four conditions it is
over-prescribed.
The question is: what is a sucient set of boundary conditions for a
PDE? We shall see that this depends on the type of equation, which, in
turn, reects the nature of the physical problem. Often two dierent types
of boundary condition lead to dierent types of solution of the same PDE.
Therefore methods of solution are normally built around boundary conditions
and on the physically expected type of solution.
A relevant problem in the atmosphere is that of numerical weather pre-
diction (NWP), which is what this course is all about. With some approx-
imations (see e.g. Smith, Lectures on Dynamical Meteorology, Chapter 8),
one can formulate an equation for the evolution of atmospheric pressure as
a function of space and time, p(x, y, z, t). It is physically reasonable to ex-
pect that to be able to predict the pressure at some time in the future, we
9
would need to know the pressure distribution at some particular time, e.g.,
p(x, y, z, 0). Furthermore, it is clear that if we are making the prediction for
a limited region on the earth, for example in the European region, we shall
need to know about pressure systems that are entering or leaving this region.
That is, we shall need to specify the pressure, or some related quantity such
as the horizontal pressure gradient, at the boundary of the computational
domain for all time t. Again, as in the heated rod problem, we expect to
require an initial condition and some boundary conditions.
1.2 Some denitions
To continue our discussion we need to introduce some basic denitions:
Operators Consider the expression

2
u
t
2
c
2

2
u
x
2
.
This describes a set of manipulations which assigns to any suciently dier-
entiable function u(x, t) a new function of x and t. Such an assignment of
one function to another is called an operator. We may denote it by L. We
write the function that it assigns to u by L[u] or Lu.
If u(x, t) and v(x, t) are any twice dierentiable functions, so is an arbi-
trary linear combination of them, au+bv, where a and b are constants. Then
L[au + bv] is dened as
L[au + bv] = aL[u] + bL[v].
Denition An operator with these properties is called a linear oper-
ator. The order of an operator refers to the degree of the highest derivative
(as in theory of ODEs). For example
L[u] =
u
x
ku is a rst-order operator ,
L[T] =
T
t
k

2
T
x
2
and L[u] =

2
u
xy
are second-order operators.
An equation of the type
L[u] = f,
where L is a linear operator and f is a function of the independent variables
only is called a linear partial dierential equation. If f = 0 it is a non
homogeneous equation; if f = 0 it is homogeneous.
10
Examples:

2
u
x
2
+

2
u
y
2
= 0
u
x
= ku are homogeneous equations,
u
x
= kx,

2
u
x
2
+

2
u
y
2
= kxy, are nonhomogeneous equations.
In general, homogeneous means that every term includes the dependent vari-
able to the same power. For example, the nonlinear equation
u
u
x

_

2
u
y
2
_
2
= 0
is homogeneous. Then if u is a solution, so is u, for any constant .
Note 1.
The denition of a linear equation refers only to the operator and does not
consider the boundary conditions. Thus, while the solutions of a linear equa-
tion may be added (or linearly superposed) to generate other solutions, dif-
ferent solutions may satisfy dierent boundary conditions. However, super-
position may enable us to construct a solution satisfying given boundary
conditions from ones which do not.
Note 2.
Superposition does not apply to nonlinear or to nonhomogeneous equations.
For example, if u
1
nd u
2
are solutions of
L[u] = u
u
x


2
u
x
2
= ky,
then

x
(u
1
+ u
2
)

2
x
2
(u
1
+ u
2
)
u
2
x
+ u
2
u
1
x
+ 2ky = ky.
11
1.3 First order equations
One of the simplest rst-order PDEs is the one-dimensional advection equa-
tion,
T
t
+ U
T
x
= 0, (1.4)
where U is a constant. To understand the solution method it may be helpful
to see how the equation might arise.
Suppose we y horizontally in a research aircraft along a straight track
with speed c in an atmosphere at rest. Suppose that there is horizontal
temperature gradient in this atmosphere
1
in the direction of ight and that
the background temperature varies with time. What will be the rate-of-
change of temperature recorded in the aircraft? Suppose that in a short time
interval t, the change in temperature is T. Then by the chain rule for
dierentiation,
T =
T
t
t +
T
x
x,
so that the total rate-of-change of temperature is
T
t
=
T
t
+
x
t
T
x
.
Now x = ct and in the limit as t 0, the total rate-of-change of T is
DT
Dt
=
T
t
+ c
T
x
. (1.5)
The rst term on the right hand side of this expression is the local rate-of-
change of temperature associated with the time variation of the background
eld (it might be the normal diurnal variation of temperature, for example);
the second term is associated with the motion of the aircraft in the presence
of the horizontal temperature gradient. Suppose now that the air mass is
moving in the x-direction with uniform speed U and that we drift with the
air in a balloon, again measuring temperature. Then, the rate of temperature
change we measure in the balloon is given by (1.5) with c = U. Since we
are drifting with the air, we cannot know about the horizontal temperature
gradient within the air mass and must measure the local rate-of-change of
temperature at the point of the air parcel with which we are drifting, say
T
0
/t; i.e.
1
In actual fact, such an atmosphere cannot be at rest (why?), but this is immaterial
for the present illustration. The assumption can be rigorously justied if the ight speed
of the aircraft is much larger than the wind speed.
12
DT
Dt
=
T
t
+ U
T
x
=
T
o
t
, (1.6)
If the local rate-of-change of air temperature at this point is zero, then
T
t
+ U
T
x
= 0. (1.7)
This is therefore the PDE representing the behaviour of temperature as a
function of x and t in a uniformly translating air mass in which there is a
horizontal temperature gradient. Equation (1.7) is simply a statement of the
fact that the temperature of an air parcel being carried along with the air
mass remains constant.
Knowing how equations such as (1.7) arise provides a clue to their solu-
tion. Suppose know that a certain quantity q(x, t) satises the PDE
q
t
+
q
x
= 0, (1.8)
We recognize the operator L /t+/x, where is a constant, as repre-
senting a total rate-of-change following a point moving with speed dx/dt = ,
(cf. expression (1.5)). That is, if one moves with a point x(t) = x
0
+ t,
the operator L[q] will tell us the rate-of-change of q. Equation (1.8) tells us
that this rate-of-change is zero; in other words q is a constant along a line
x = x
0
+ t. Of course, it may vary from line to line and since each line is
characterized by the constant of integration x
0
, we can say that q = q(x
0
), or
alternatively, q = q(x t). If we know the distribution of q at time t = 0,
say q = Q(x), then it is clear that the solution of (1.8) at time t is
q = Q(x t). (1.9)
All this becomes clearer when we sketch the solution surface in the (x, t, z)-
space, analogous to Fig. 1.2. The lines x = x
0
+ t are a family of parallel
lines in the plane z = 0 and intersect the plane t = 0 at x = 0 . The equation
says that the height of the solution surface is always the same along such a
line. Then, it is not too dicult to see that the intersection of this surface
with the plane t = constant is a curve which is identical with the curve
z = Q(x) at t = 0, but displaced in the x-direction by a distance t. Thus
the solution represents a disturbance with arbitrary shape Q(x) translating
uniformly with speed in the positive x-direction if > 0, or in the negative
x-direction if < 0.
It is clear that information about the initial distribution of q propagates
or is carried along the lines x = x
0
+t in the plane z = 0. These lines are
called the characteristic curves, or simply the characteristics of the equation.
13
Figure 1.4: Solution surface for Eq. (1.8).
If were a function of t and/or x, the solution procedure would be the
same. Essentially we would solve rst for the characteristic curves by inte-
grating the equation
dx
dt
= (x, t). (1.10)
The solution of this ODE involves an integration constant that determines
where the characteristic intersects the x-axis. Then we need to construct the
solution surface which has the same value along each characteristic in the
x t plane as that in the initial plane, t = 0. Of course, for a general
(x, t), some characteristics may never cross the initial line, but in a physical
problem, this would not be expected to be the case (why?).
If the equation is nonhomogeneous, for example Eq. (1.6) with T
0
(t)
prescribed, the procedure is again to solve for the characteristics, but in this
case, T is not constant along the characteristics, but varies at the rate T
0
/t
(see Ex. 1.1 below).
Note that the characteristics of a linear equation depend on the coe-
cient(s) of the derivatives, but that is all. However, in a nonlinear equation
such as
u
t
+ u
u
x
= 0, (1.11)
they depend upon the solution also. For example, the characteristics of (1.11)
satisfy the equation
dx
dt
= u. (1.12)
Equation (1.11) is not much more dicult to treat than the linear equation
(1.8), but the solution is markedly dierent and may develop discontinuities.
14
Again, (1.11) says that u is constant along a characteristic given by (1.12)
and this being the case, (1.12) can be integrated to give
x = x
0
+ ut (1.13)
along a characteristic. Thus the characteristics are again straight lines in the
x t plane, but their slope depends on the value of the solution u, which
is equal to the initial value, say u
0
(x
0
). Thus depending on the slope of the
initial condition u(x, 0) = u
0
(x), the characteristics may diverge or converge.
If they converge, there will be a nite time at which two characteristics meet
at some point and the solution will become discontinuous there (see Ex. 1.2).
This happens in the case of the nonlinear shallow-water equations without
diusion.
1.4 Linear second-order equations
The general linear homogenous equation of second-order is
a

2
u
x
2
+ 2b

2
u
xy
+ c

2
u
y
2
+ 2f
u
x
+ 2g
u
y
+ hu = 0, (1.14)
where a, b, c, f, g, h are either constants or more generally functions of x
and y. The diusion equation (1.1) is an example for which y = t, a = k,
g
1
2
and all other coecients are zero. Another example is the so-called
wave equation:

2
u
t
2
= c
2
0

2
u
x
2
, (1.15)
where y = t, a = 1, b = 0, c
2
0
and f = g = h = 0. Note that the form of
(1.14) resembles the equation for a conic section,
ax
2
+ 2bxy + cy
2
+ 2fx + 2gy + h = 0, (1.16)
where a, b, c .... are constants. Equation (1.16) represents an ellipse,
parabola, or hyperbola according as ac b
2
> 0, = 0 or < 0. Thus we
classify the PDE (1.14) as
elliptic when ac b
2
> 0
parabolic when ac b
2
= 0
hyperbolic when ac b
2
< 0
It follows that the diusion equation is parabolic and the wave equation is
hyperbolic. We shall see that the type of boundary and/or initial conditions
15
required to determine a solution of (1.14) depend on the type of equation
according to the above classication. We begin by studying a slightly simpler
form of (1.14), namely
a

2
u
x
2
+ 2b

2
u
xy
+ c

2
u
y
2
= 0, (1.17)
where a, b, and c are constants. This is known as Eulers equation. To obtain
the general solution we make a linear transformation of the independent
variables to and given by
= px + qy and = rx + sy,
where the constants p, q , r and s will be chosen later. Now, using the chain
rule for partial dierentiation,
u
x
=
u

x
+
u

x
= p
u

+ r
u

,
and
u
y
=
u

y
+
u

y
= q
u

+ s
u

.
It follows that

2
u
x
2
=

x
_
u
x
_
=
_
p

+ r

__
p
u

+ r
u

_
= p
2

2
u

2
+ 2pr

2
u

+ r
2

2
u

2
,
and

2
u
y
2
=

y
_
u
y
_
=
_
q

+ s

__
q
u

+ s
u

_
= q
2

2
u

2
+ 2qs

2
u

+ s
2

2
u

2
,
Then, substitution into (1.17) gives
(ap
2
+ 2bpq + cq
2
)

2
u

2
+ 2[apr + csq + b(rq + sp)] +

2
u

+ (ar
2
+ 2brs + cs
2
) = 0. (1.18)
16
Now we choose p = 1, r = 1, and q and s to be the two roots
1
,
2
of the
quadratic equation
a + 2b + c
2
= 0. (1.19)
Then (1.18) becomes
[a + b(
1
+
2
) + c
1

2
]

2
u

= 0. (1.20)
Noting that
1
+
2
= 2b/c and
1

2
= a/c, Eq. (1.20) reduces to
2
c
(ac b
2
)

2
u

= 0. (1.21)
If c = 0 and if the equation is not parabolic, then

2
u

= 0, (1.22)
which has the same form as (1.2), with solution (cf. Eq. 1.3)
u(, ) = F() + G(), (1.23)
where F and G are arbitrary functions. In terms of x and y, it follows that
u(x, y) = F(x +
1
y) + G(x +
2
y). (1.24)
The functions F and G will need to be determined from suitably-prescribed
initial and/or boundary conditions.
In the case of the wave equation (1.15), a = 1, b = 0, c = c
2
0
and y = t.
Then ac b
2
< 0 and the equation is hyperbolic. Equation (1.19) reduces to
1
2
/c
2
0
= 0, whereupon
1
= c
0
,
2
= c
0
so that the general solution is
u(x, t) = F(x c
0
t) + G(x + c
0
t). (1.25)
In analogy with (1.9), the solution corresponds with the sum of two shapes
F(x) and G(x), the former translating in the positive x-direction with speed
c
0
, the latter propagating with the same speed in the negative x-direction. In
this case, there are two sets of characteristic curves: = x c
0
t = constant
and = x + c
0
t = constant. Part of the initial information (u(x, 0) =
F(x) +G(x)) propagates along one set of characteristics and part propagates
along the other set (see Ex. 1.6). Clearly, to have enough information to
determine F and G we need an extra initial condition. This is not surprising
since the equation involves the second derivative with respect to time rather
than just the rst (note that the ODE dx/dt = f(t) requires only one initial
17
derivative condition whereas the equation d
2
x/dt
2
= f(t) requires two). We
might guess that an appropriate additional condition would be to prescribe
u/t = v
0
(x) at t = 0, equivalent to specifying the initial slope of the
solution surface at t = 0 as well as its height. From (1.25), this implies that
c
0
dF
dx
+ c
0
dG
dx
= v
0
(x),
whereupon
F + G =
1
c
o
_
x
0
v
o
(x

)dx

(1.26)
Since
F + G = u
0
(x), (1.27)
the full solution of (1.15) follows readily in terms of u
0
(x) and v
0
(x) (see Ex.
1.6).
For hyperbolic equations in general, acb
2
< 0 and the roots of (1.19) are
real and distinct. For elliptic equations, acb
2
> 0 and the roots are complex
conjugate pairs. Thus elliptic equations have complex characteristics.
In the case of a parabolic equation, ac = b
2
and the roots of (1.19) are
equal; in fact
1
=
2
= b/c. In this case we choose p = 1 and q = b/c,
whereupon the rst and second terms of (1.18) vanish and then provided
that r and s are not both zero, it reduces to

2
u

2
= 0. (1.28)
This equation may be integrated directly to give
u(x, y) = F() + G(), (1.29)
where F and G are arbitrary functions of . Now since p = 1 and q = b/c,
say, then
= x + y and = rx + sy,
where r and s are arbitrary and not both zero. For simplicity we may choose
r = 0 and s = 1 to obtain
u(x, y) = F(x + y) + yG(x + y). (1.30)
Note that a parabolic equation, unlike a hyperbolic equation, has only one
set of characteristics given by = x + y = constant.
If one considers the solution surface z = u(x, y) in (x, y, z)-space, it is
clear that a knowledge of u(x, 0) would determine the function F. However,
it is not immediately clear what condition is suitable to determine G.
18
While the method of obtaining the general solution of Eulers equation
shows some important features about the structure of solution, the general
solutions are not usually easy to t directly to any prescribed initial or bound-
ary conditions.
Often the general solutions to PDEs are far too general to be practically
useful and specic solutions require initial and/or boundary conditions to be
prescribed. In trying to see what boundary conditions might be appropriate,
it is enlightening to introduce three physical problems which lead to equa-
tions of the three basic types we have discussed: hyperbolic, parabolic and
elliptic. We regard these problems as prototype problems for understanding
the structure of solutions to the equations.
1.5 Parabolic equations
A prototype for studying parabolic systems is the heated rod problem dis-
cussed in section 1.1. Suppose that the initial temperature distribution in
the rod is T
0
(x) and that for T > 0, the ends are held at uniform tempera-
ture T
a
and T
b
, respectively - in our physical thought experiment we assume
that the rod is insulated along its length so that no heat escapes and that
the ends are held in contact with heat reservoirs, which maintain the tem-
perature there at the required values. It is reasonable to expect that if the
rod has a uniform conductivity k, the temperature distribution along it will
eventually become linear in x, i.e.
T(x, ) = T
a
+ (T
b
T
a
)(x a)/(b a). (1.31)
It is easy to check that this is a steady solution of Eq. (1.1) satisfying the
required boundary conditions on T at x = a and b. Let us dene the deviation
temperature at time t, (x, t), to be the actual temperature T(x, t) minus the
steady state temperature given by (1.31). Then clearly (x, t) satises (1.1)
because the latter is a linear equation; moreover (a, t) = 0 and (b, t) = 0
for all time, because T(a, t) = T(a, ) and T(b, t) = T(a, ).
We try now to obtain a solution for (x, t) by the method of separation
of variables. We set (x, t) = X(x)(t) in the equation for and divide by
to obtain
1

d
dt
=
k
X
d
2
X
dx
2
, (1.32)
Since the left-hand side is a function of t only and the right-hand side is
function of x only, the only way that (1.32) can be satised as x and t vary
independently is if both sides are the same constant, say . This observation
19
enables us to reduce the PDE to two ODEs, namely
d
dt
= , (1.33)
and
d
2
X
dx
2


k
X = 0. (1.34)
In order to satisfy the boundary conditions on for all time we must have
X(a) = X(b) = 0. The solution of (1.33) has the form
=
o
e
t
(1.35)
and since (x, t) is the transient part of the solution for T(x, t) that must
decay to zero as t , it is clear that must be taken to be negative.
Then, writing m
2
= /k > 0 in (1.34), we see that solutions for X must
have the form
X(x) = Acos mx + Bsin mx (1.36)
The algebra in implementing the boundary conditions on X is simplied if
we relocate the origin of x so that a = 0. Then the condition X(0) = 0
requires that A = 0 and the condition X(b) = 0 requires that Bsin mb = 0.
If we choose B = 0, the solution for X is trivial and of little use to us. If
B = 0 then sin mb = 0, whereupon mb must be an integer multiple n, say, of
. It follows that possible solutions for have the form
B
n
exp
_
n
2

2
kt/b
2
_
sin (nx/b)
and by the principle of superposition, the general solution for (x, t) satisfying
(0, t) = 0 and (b, t) = 0 is the Fourier series
(x, t) =

n=1
B
n
exp(n
2

2
kt/b
2
) sin
_
nx
b
_
. (1.37)
Given the initial condition that T(x, 0) = T
0
(x) we nd that
T(x, 0) T
0
(x, ) =

n=1
B
n
sin
_
nx
b
_
,
which relates the Fourier coecients B
n
to the initial temperature distribu-
tion in the rod, i.e.,
B
n
=
2
b
_
b
0
[T
0
(x) T(x, )] sin
_
nx
b
_
dx.
The solution of this problem brings out clearly the constraints on parabolic
equations in general. It shows that one initial condition is required together
with two boundary conditions. Again, this ts with the idea that the time
derivative is only rst order whereas the space derivative is second order.
20
1.6 Hyperbolic equations
The prototype problem for a hyperbolic equation considers the vibrations
of a stretched string with the ends at x = 0 and x = b, say (Fig. 1.5).
The lateral displacement of the string, y(x, t), satises the wave equation
(1.15), where the constant c
2
0
is proportional to the tension in the string and
inversely proportional to its mass per unit length.
Figure 1.5: Schematic of a vibrating string.
The boundary conditions are clearly that y(0, t) and y(b, t) = 0 and since
the equation is derived by applying Newtons second law to each element of
the string, we can guess that it will be necessary to know both the positions
and velocity of each element, i.e. y(x, 0) and y/t at t = 0. Substituting
y(x, t) = X(x)T(t) in this case and dividing through by y(x, t) gives
1
X
d
2
X
dx
2
=
1
c
2
o
T
d
2
T
dt
2
= m
2
,
where m
2
= constant. We choose a negative separation constant in this case
in order to be able to satisfy the boundary conditions at x = 0 and x = b;
these imply that X(0) and X(b) = 0. Then X satises
d
2
X
dx
2
+ m
2
X = 0,
with appropriate solutions
X = B
n
sin
_
nx
b
_
(n = 1, 2, 3, ....),
as before (see section 1.5). The equation for T is similar in form and has
solutions
T = A
n
cos
_
nc
o
t
b
_
+ C
n
sin
_
nc
o
t
b
_
.
21
Then the general solution for y satisfying y(0, t) = 0 and y(b, t) = 0 for all
time is
y(x, t) =

n=1
_
A
n
cos
_
nc
o
t
b
_
+ C
n
sin
_
nc
o
t
b
__
sin
_
nx
b
_
, (1.38)
where the constants A
n
, C
n
have been redened to incorporate the constants
B
n
. Note that, in contrast to the solution of the diusion equation, there
are now two sets of Fourier coecients to be determined. The rst set is
obtained by putting t = 0 in (1.38) so that
y(x, t) =

n=1
A
n
sin
_
nx
b
_
. (1.39)
Then
A
n
=
2
b
_ b
0
y(x, 0) sin
_
nx
b
_
dx. (1.40)
The other set is obtained by dierentiating (1.38) with respect to time and
the setting t = 0 so that
y
t

t=0
= v
o
(x) =
c
0
b

n=0
nC
n
sin
_
nx
b
_
, (1.41)
or
C
n
=
_
2
nc
o
__
b
0
v
o
(x) sin
_
nx
b
_
dx. (1.42)
Again, the analysis shows rather well how the physical problem is precisely
constrained by a certain combination of initial and boundary conditions. In
particular it indicates how the formulation would fall apart if one or more of
these conditions were unavailable. For example the physical problem would
be clearly ill-conceived if we removed either boundary condition at the end
of the string!
1.7 Elliptic equations
Elliptic equations tend to arise in problems involving equilibrium or bal-
anced states. The prototype physical problem considers the equilibrium of
a stretched membrane subject to a distribution of forces normal to it (Fig.
1.6). It can be shown that the lateral displacement of the membrane z(x, y)
22
Figure 1.6: Equilibrium displacement of a stretched membrane.
subject to the force distribution F(x, y) per unit area normal to it satises
the elliptic PDE:

2
z
x
2
+

2
z
y
2
= F (x, y). (1.43)
Equation (1.43) is called Poissons equation. Typically the membrane
would have zero displacement along its boundary (e.g. a drum skin), i.e.
an appropriate boundary condition would be z = 0 along all of its closed
sides. Suppose that the membrane is stretched across a closed piece of wire
which does not lie exactly in a plane, but whose displacement is known and
suppose that there is no force distribution across the membrane. Then the
displacement satises the equation

2
z
x
2
+

2
z
y
2
= 0 (1.44)
subject to the displacement being prescribed along the entire closed boundary
formed by the wire. Equation (1.44) is called Laplaces equation.
In section 1.4 we showed that the transformation = x+
1
y, = x+
2
y
reduces Eulers equation (1.17) to the form
2
u/ = 0, where
1
and
2
are the roots of (1.19), assuming that acb
2
= 0. This transformation proves
useful for a hyperbolic equation where
l
and
2
are real, but is of limited
use for elliptic equation where
l
and
2
are complex conjugates and the
characteristics are complex. Suppose that in the latter case,

1
= p + iq and
2
= p iq, (1.45)
where
p = b/c, q = (ac b
2
)
1/2
/c. (1.46)
Instead of transforming to and , we transform to and where
=
1
2
( + ) = x + py, (1.47)
23
and
=
1
2i
( ) = qy. (1.48)
Then

x
=

x
+

and

y
=

y
+

y
p

+ q

,
whereupon Eulers equation reduces to
1
c
(ac b
2
)
_

2
u

2
+

2
u

2
_
= 0. (1.49)
Similarly, the general elliptic equation can be transformed to the canonical
form

2
u
x
2
+

2
u
y
2
+ e
u
x
+ f
u
y
+ gu = 0. (1.50)
Other problems in which elliptic equations arise are:
(i) Steady heat ow in a medium with uniform conductivity The
heat ux in such a medium is q = kT, where T is the temperature and
k is the conductivity. If there are no heat sources or sinks in the medium,
q = 0, whereupon T satises
2
T = 0.
(ii) Steady irrotational incompressible ow of a homogeneous uid
Irrotational means that v = 0, where v is the velocity vector. This
implies the existence of a velocity potential such that v = . Steady,
incompressible and homogeneous implies that v = 0, whereupon satises

2
= 0.
Since elliptic problems arise naturally for steady problems, problems
concerned essentially with states of equilibrium, initial values are not rel-
evant. Thus typical problems are boundary value problems in which the
boundary data are prescribed on given closed boundary curves (or sur-
faces/hypersurfaces for equations with more than two independent variables).
The two most commonly occurring problems are:
(a) The Dirichlet problem, e.g.

2
u
x
2
+

2
u
y
2
= 0
with u prescribed as a function of position along the whole boundary of the
domain
24
(b) The Neumann problem, e.g.

2
u
x
2
+

2
u
y
2
= 0
with u/n prescribed as a function of position along the whole boundary
of the domain, where /n is the normal derivative along the boundary.
Note that in this case, the prescribed function must satisfy a compatibility
condition (see below).
A third problem is the mixed problem where au + bu/n is prescribed
along the boundary, possibly with a or b zero along segments of the boundary.
Figure 1.7:
Greens theorem If P(x, y) and Q(x, y) are functions dened inside and
on the boundary C of a closed domain S, then
_ _
s
_
P
x
+
Q
y
_
dxdy =
_
C
(Pdy Qdx), (1.51)
where
_
C
is taken in the positive (counter-clockwise) direction.
The proof may be found in many books on advanced calculus, e.g. R.P.
Gillespie, Integration, Oliver and Boyd, p56, 20.
Let n be the outward unit normal to C and let ds be an element of the
curve C. Then
dx = ds cos = j n ds,
and (1.51) may be written as
_ _
s
_
P
x
+
Q
y
_
dxdy =
_
C
(Pi Qj) nds. (1.52)
25
This is the two-dimensional form of the divergence theorem. In the case of
Neumanns problem for Poissons equation,
_
C
u
n
ds =
_ _
s
_

2
u
x
2
+

2
u
y
2
_
dx dy =
_ _
s
F dxdy, (1.53)
the rst equality being obtained by setting P = u/x and Q = u/y in
(1.52). Noting that u/n = n u, it follows that u/n can be prescribed
only subject to the constraint on F implied by Eq. (1.53). Clearly, for
Laplaces equation we must have
_
C
u
n
ds = 0. (1.54)
An example of an incorrectly-posed boundary value problem for Laplaces
equations was given by Hadamard. It is shown diagrammatically in Fig. 1.8.
Figure 1.8: Specication of Hadamards problem for Laplaces equation.
These boundary conditions are the same as those that are appropriate for
the wave equation.
It turns out that one can nd a unique solution to this problem, but the
solution is not continuous with respect to the data. By this we mean that
an arbitrarily small change in f(x) can change the solution by an arbitrarily
large amount.
26
Example
Let f(x) = e[sin(4n+1)x], where n is a positive integer. It is easy to verify
that a solution which satises the boundary conditions is
u
n
(x, y) = e

n
cosh [(4h + 1)y] .
By taking n suciently large, one can make f and any number of its deriva-
tives arbitrarily small, but
u
n
(
1
2
, y) = e

n
cosh [(4n + 1)y]
can be made arbitrarily large for any xed y > 0 by choosing n suciently
large. Hence the solution is not continuous with respect to the data and the
problem is incorrectly posed. It is easy to see why the problem is ill-posed by
considering a physical analogy. Suppose that u represents the temperature
distribution in a block of material and write u = TT
0
. The physical analogy
is that of a block of material of unit width as shown in Fig. 1.8. If T(x) > 0,
the temperature gradient between y = 0 and the side-boundaries implies a
steady heat ux through y = 0 towards the side boundaries. However, the
condition T/y = 0 at y = 0 is inconsistent with this as it implies zero
heat ux. Note also that the absence of any condition as y allows
an arbitrary heat ux through that boundary and therefore a multiplicity of
solutions.
Figure 1.9: Physical analogue to Hadamards problem. Arrows indicate di-
rection of heat ow.
27
Exercises
1.1 Solve the PDEs
(a)
T
t
+ U
T
x
= q(1 e
t
),
(b)
T
t
+ Ue
t
T
x
= 0.
in the domain x , t 0, given that T = T
0
(x) at t = 0.
Assume that U, and q are constants.
1.2 Obtain the characteristics of the PDE
u
t
+ u
u
x
= 0
in the domain x , t 0, given that u = 1e
x
at t0. Show
that the characteristics converge in the region x > 0 and diverge in the
region x < 0.
1.3 A model for the temperature variation T(z, t) in the ground as a func-
tion of depth z and time t is to assume that T(z, t) satises the diusion
equation
T
t
= k

2
T
z
2
,
where k is a constant. The surface temperature, T(0, t), and the tem-
perature at great depth, T(, t), are prescribed. Taking T(0, t) =
T
s
+ cos(t) to represent the diurnal temperature variation at the
earths surface resulting from insolation and nocturnal cooling and
T(, t) = T

, and assuming that T

s
, and are constants, cal-
culate T(z, t). Is it realistic to have T
s
and T

dierent? Indeed, how

realistic is this model?
1.4 Determine the nature of each of the following equations (i.e. whether
elliptic, parabolic or hyperbolic) and obtain the general solution in each
case:
(a) 3

2
u
x
2
+ 4

2
u
xy


2
u
y
2
= 0, (b)

2
u
x
2
2

2
u
xy
+

2
u
y
2
= 0,
(c) 4

2
u
y
2
+ 2

2
u
x
2
= 0, (d)

2
u
y
2
+ 2

2
u
x
2
0,
28
(e)

2
u
y
2
+ 2

2
u
x
2
= 0.
1.5 A function u(r, t) satises the equation
1
r
2

r
_
r
2
u
r
_
=
1
c
2

2
u
t
2
,
where c is constant. By introducing the new dependent variable v(r, t)
and writing = r + ct, = r ct, reduce this equation to

2
v

= 0.
Hence show that the general solution u(r, t) has the form u(r, t) =
1
r
[f(r+ct)+g(rct)], where f and g are arbitrary (twice dierentiable)
functions.
1.6 Using Eqs. (1.26) and (1.27), show that the solution of the wave equa-
tion satisfying the initial conditions u = u
0
(x) and u/t = v
0
(x) at
t = 0 is
u(x, t) =
1
2
[u
0
(x ct) + u
0
(x + ct)] +
1
2
c
_ x+ct
xct
v
0
(s) ds.
This is known as DAlemberts solution.
29
Chapter 2
Finite Dierence Methods
2.1 The basic ideas
The most direct way to solve PDEs numerically is through the use of nite-
dierence approximations to the derivatives. The domain of integration
spanned by the independent variables is covered with a nite grid of points
at discrete space and/or time intervals. The dependent variables are stored
and evaluated only at these grid points and derivatives are approximated by
values at adjacent grid points. In this way the dierential equation is reduced
to a system of algebraic equations for the function values at each grid point.
These equations are solved numerically.
As an example consider the ordinary dierential equation (ODE),
du
dx
= f(u, x), (2.1)
on the interval a x b. We divide the interval up into a set of N +1 grid
points, x
i
= a + x, i = 1, . . . , N + 1, where x = (b a)/N is the grid
length. Let u
i
denote the value of u at the grid point x; i.e., u
i
= u(x
i
).
There are various ways to approximate the derivative at the grid point:
I) A forward dierence,
_
du
dx
_
i
=
u
i+1
u
i
x
,
II) A backward dierence,
_
du
dx
_
i
=
u
i
u
i1
x
,
30
III) A centred (or central) dierence,
_
du
dx
_
i
=
u
i+1
u
i1
2x
.
The names are clear when one considers the graphical representation of these
quantities (Fig. 2.1). One gains the impression from the gure that the
centred dierence may be a more accurate approximation to the tangent
curve at x
i
than either of the other dierences. We show now that this is the
case. Suppose u(x) is expanded as a Taylor series about the point x
i
. Then
u
i+1
= u
i
+
_
du
dx
_
i
x +
1
2
_
d
2
u
dx
2
_
i
x
2
+
1
6
_
d
3
u
dx
3
_
i
x
3
(2.2)
and
u
i1
= u
i

_
du
dx
_
i
x +
1
2
_
d
2
u
dx
2
_
i
x
2

1
6
_
d
3
u
dx
3
_
i
x
3
. (2.3)
Figure 2.1: Graphical representation of forward, backward and centred dif-
ferences.
It follows from (2.2 and 2.3) that
_
du
dx
_
i
=
u
i+1
u
i
x
x
_
1
2
_
d
2
u
dx
2
_
i
+ . . .
_
, (2.4)
_
du
dx
_
i
=
u
i
u
i1
x
+ x
_
1
2
_
d
2
u
dx
2
_
i
. . .
_
(2.5)
31
and by subtracting (2.3) from (2.2) that
_
du
dx
_
i
=
u
i+1
u
i1
2x
x
_
1
6
_
d
3
u
dx
3
_
i
+ .....
_
. (2.6)
Thus the error in replacing the derivative by a forward or backward dier-
ence is 0(x) while that for the central dierence is 0(x
2
). The error,
characterized by the term in x or x
2
in the foregoing equations, is called
the truncation error of the approximation. The leading power of x in the
truncation error is the order of accuracy of the approximation. Clearly, for
an approximation to be consistent it must be at least rst-order accurate,
and consistency in this sense is a requirement of any nite-dierence approx-
imation. It is clear that knowledge of a discrete set of values u
i
, even if the
approximations were perfect, gives less information than knowledge of the
function u(x). Suppose that the function u(x) dened on the interval (0, L)
is represented by a Fourier series
u(x) =
1
2
a
0
+

n=1
[a
n
cos(2nx/L) + b
n
sin(2nx/L)]. (2.7)
If only N +1 values of u(x) are known, we cannot expect to determine more
than N+1 of the Fourier coecients a
0
, a
1
, a
2
, ...., b
1
, b
2
, ... . It is intuitively
obvious from Fig. 2.2 that to resolve a sine wave of a given wavelength, a
minimum of three points are required. That is, 2 .
Figure 2.2: Representation of a sine wave of wavelength requires a minimum
of three grid points.
It is clear also from Fig. 2.2 that a wave of wavelength cannot be
distinguished from one of wavelength 2. In view of this, it is reasonable
to expect that with only N + 1 points, only waves longer than a certain
32
wavelength in (2.7) will be resolved, i.e. only the coecients a
0
and a
1
, a
2
,
...a
N/2
, b
1
, b
2
, ... b
N/2
can be determined (assuming that N is even). Note
that the shortest wave has wavelength L/(N/2) = 2L/N = 2x, conrming
the intuitive limit suggested by Fig. 2.2. This is an important result: a grid
with a spacing x can not resolve wavelengths shorter than 2x.
2.2 The advection equation
Many of the important ideas can be illustrated by reference to the advection
equation (1.4) which we write in the form
u
t
+ c
u
x
= 0. (2.8)
We divide the (x, t)-plane into a series of discrete points (ix, nt) and
denote the approximate solution for u at this point by u
n
i
(see Fig. 2.3). We
have seen that the true solution propagates along the characteristics in the
x t plane, which are given by dx/dt = c. This suggests replacing the time
derivative by a forward dierence and the space derivative by a backward
dierence, an idea that is explored further in Chapter 3: see Fig. 3.8. Then
a possible nite-dierence scheme for the equation is
u
n+1
i
u
n
i
t
+ c
u
n
i
u
n
i1
x
= 0. (2.9)
Figure 2.3: A nite-dierence grid for nding an approximate solution to
(2.3).
33
This scheme is called a forward and upstream scheme, upstream indicat-
ing the position of the point i1 relative to the advection velocity c, assumed
here to be positive.
Equation (2.9) is only one of many possible consistent nite-dierence
schemes for (2.8). There are many schemes that approach the PDE as x
and t approach zero.
The dierence between the numerical solution and the true solution is
u
n
i
u(ix, nt), (2.10)
which represents the error of the numerical solution. Since u(x, t) is normally
unobtainable, we cannot usually determine this error. However, we can esti-
mate the accuracy of a scheme in terms of the grid spacings x, t in the
following way. For Eq. (2.9), for example, we substitute the true solution
(assumed known) in the dierence equation (2.9) to obtain
u(ix, (n + 1)t) u(ix, nt)
t
+
c
u(ix, nt) u((i 1)x, nt)
x
= . (2.11)
The quantitiy e is called the truncation error of the nite-dierence scheme.
It is a measure of how closely the true solution satises the equation for
the scheme and therefore a measure of the accuracy of the scheme. Using
expressions equivalent to (2.4) and (2.5) it follows at once that
= t
_
1
2

2
u
t
2
+
1
6

3
u
t
3
t + ....
_
c x
_
1
2

2
u
x
2

1
6

3
u
x
3
x ....
_
. (2.12)
As before, these are the terms that are truncated o to make the PDE
reduce to the nite dierence scheme. The order of accuracy of a nite-
dierence scheme is the lowest power of x and t that appears in the
truncation error. The scheme (2.9) is rst order accurate, because
= O(t) + O(x) = O(t, x).
It is useful to make a distinction between orders of accuracy in space and in
time, especially when the lowest powers of x and t are not the same. A
necessary condition for the consistency of a scheme is that it be at least of
the rst order of accuracy.
34
2.3 Convergence
The truncation error of a consistent scheme can be made arbitrarily small
by making t and x suciently small. However, we cannot be sure that
this will reduce the error of the numerical solution given by (2.10). There
are two questions that emerge:
(a) How does the error u
n
i
u(ix, nt) behave when, for a xed total
time nt, the increments x, t approach zero?
(b) How does this error behave when, for xed values of x and t, the
number of time steps increases?
If in (a) the error approaches zero as the grid is progressively rened (i.e. as
t, x 0), the solution is called convergent. If a scheme gives a convergent
solution for any initial conditions, then the scheme itself is called convergent.
We may show that consistency of a scheme does not guarantee conver-
gence. For example, we have noted that (2.9) is a consistent scheme for
(2.8). However, additional conditions must be satised for it to be a conver-
gent scheme.
Consider the numerical solution when the grid lines and characteristics
are as shown in Fig. 2.4. The characteristic passing through the origin
passes also through the grid point A, denoted by a square. Thus, according
to our discussion in 1.3, the true value of u at A is equal to the value of
u at the origin. However, in the numerical solution, the value of u at A is
computed using the values at points denoted by circles. The domain spanned
by these points is called the domain of dependence of the numerical scheme.
For the grid shown, the grid point at the origin is outside this domain and
therefore cannot aect the numerical solution at A. Therefore the error of
the numerical solution at A can be arbitrarily large.
It is obvious that the situation is not improved if t and x are reduced
in proportion to one another since the domain of dependence would remain
the same. Renement of the grid in this way cannot reduce the error of the
numerical solution. Clearly, a necessary condition for the convergence of a
scheme for Eq. (2.8) is that the characteristic dening the true solution at a
grid point lies within the domain of dependence of the numerical solution at
that point. This will happen when the slope of the characteristics is greater
than slope of the line AB, i.e. when
ct x. (2.13)
This necessary condition for the convergence of (2.9) is the condition found
by Courant et al. (1928) referred to in the Prologue.
35
Figure 2.4: A possible grid for the solution of (2.9) showing the characteristics
through the point A and the domain of dependence of this point, (i.e. the
set of points, denoted by open circles, that aect the solution at A).
2.4 Stability
The answer to question (b) raised above depends on the stability of the
numerical solution.
A diculty in dening stability arises because the true solution does not
have to be bounded (consider for example an unstable Eady-wave governed
by the quasi-geostrophic potential vorticity equation). When we know that
the true solution is bounded, as is the case of (2.8), we can construct a
denition based on the error u
n
i
u(ix, nt).
Denition: The solution u
n
i
is stable if the error remains bounded as n
increases. Moreover, nite-dierence scheme is stable if it gives a stable
solution for any initial conditions.
Stability of a scheme is a property of great importance. There exist
consistent schemes of high order of accuracy that still give solutions diverging
unacceptably far from the true solution. Thus, conditions for stability should
be known. There are three methods for doing this which we illustrate using
the forward and upstream scheme (2.9).
Direct method: We know that the true solution is bounded so it suces
to test the boundedness of the numerical solution. We may rewrite (2.9) as
u
n+1
i
= (1 )u
n
i
+ u
n
i1
, (2.14)
where
36
m = ct/x.
If 0 1, which happens to be the necessary condition for convergence
(2.13), we have

u
n+1
i

(1 ) |u
n
i
| +

u
n
i1

. (2.15)
We apply this at the point i where at time level n+1,

u
n+1
i

is a maximum,
Max
(i)

u
n+1
i

. Since the right-hand-side is increased only if each of the terms

is replaced with Max
(i)
|u
n
i
|, we obtain from (2.15),
Max
(i)

u
n+1
i

Max
(i)
|u
n
i
| .
This proves the boundedness of the numerical solution. Thus 0 1 is a
sucient condition for the stability of (2.9).
While the direct method is simple, it is successful only for a rather limited
number of schemes.
Energy method: This method is of much wider applicability and can be
used as well for nonlinear equations.
If we know that the true solution is bounded, we test whether

i
(u
n+1
i
)
2
is bounded also. If it is, then every u
n
i
must be bounded and the stability
of the scheme has been proved. The method is called the energy method
because in physical applications u
2
is often proportional to some form of
energy.
Squaring (2.14) and summing over i gives

i
(u
n+1
i
)
2
=

i
_
(1 )
2
(u
n
i
)
2
+ 2(1 )u
n
i
u
n
i1
+
2
(u
n
i1
)
2

. (2.16)
We assume a cyclic boundary condition
u
1
= u
N
whereupon

i
(u
n
i1
)
2
=

i
(u
n
i
)
2
, (2.17)
37
Now we use Schwarzs inequality which states that

ab
_

a
2
_

b
2
.
Equation (2.16) then shows that

i
u
n
i
u
n
i1

i
(u
n
i
)
2

i
(u
n
i1
)
2
=

i
(u
n
i
)
2
. (2.18)
Using (2.17) and (2.18) it follows that if 1 0, (2.16) gives the inequality

i
(u
n+1
i
)
2

_
(1 )
2
+ 2(1 ) +
2

i
(u
n
i
)
2
,
or

i
(u
n+1
i
)
2

i
(u
n
i
)
2
.
Therefore 1 1 0, together with the cyclic boundary condition, is a
sucient condition for the stability of (2.14).
Von Neumanns method: This method, otherwise called the Fourier se-
ries method is the most frequently used one for establishing the stability of a
nite-dierence scheme. However, we cannot use it for nonlinear equations,
but must apply it to the linearized versions of these.
A solution to a linear equation can be expressed in the form of a Fourier
series, each harmonic of which is a solution also. The idea is to test for the
stability of a single harmonic solution. Then a necessary condition for the
stability of the scheme is that it is stable to all admissible harmonics.
As an illustration of the method we consider again the advection equation
(2.8), which has an analytic solution in the form of a single harmonic.
u(x, t) = Re[U(t)e
ikx
]. (2.19)
Here U(t) is the wave amplitude and k the wavenumber. Substitution into
(2.8) gives
dU
dt
+ ikcU = 0,
which has the solution
U(t) = U(0)e
ikct
,
38
U(0) being the initial amplitude. Hence
u(x, t) = Re
_
U (0) e
ik(xct)

, (2.20)
as expected from the general solution (cf. 1.9).
In the von Neumann method we look for an analogous solution of the
nite-dierence equation (2.14). We substitute
u
n
j
= Re
_
U
(n)
e
ikjx

, (2.21)
where U(n) is the amplitude at level n. This is a solution provided that
U
(n+1)
= (1 )U
(n)
+ U
(n)
e
ikx
. (2.22)
We can now study the behaviour of the amplitude U(n) as n increases. We
dene an amplication factor || by
U
(n+1)
= U
(n)
. (2.23)
Then

U
(n)

= ||
n

U
0
< B

,
where B is a nite number, i.e.
nln || < ln
_
B/|U
(0)
|
_
= B

, say.
Since n = t/t, this necessary condition for stability becomes
ln < (B

t). (2.24)
If we require boundedness of the solution for nite time t, condition (2.24)
is that
ln|| O(t).
Now dene || = 1 + . Then since for || << 1, ln(1 + ) , the
stability criterion is equivalent to 0(t), or
|| 1 +O(t). (2.25)
This is the von Neumann necessary condition for stability.
The von Neumann condition allows an exponential growth of the solution
when || > 1, but no faster. This is necessary when the true solution grows
exponentially. However, when we know that the true solution doesnt grow,
as in the problem at hand, it is customary to replace (2.25) by a sucient
condition
|| 1. (2.26)
39
In the present example, substitution of (2.23) into (2.22) gives
= 1 + e
ikx
, (2.27)
whereupon
||
2
= 1 2(1 )(1 coskx). (2.28)
It follows once again that 1 1 0 is a sucient condition for the
stability of (2.14).
Equation (2.28) gives further information about the numerical solution.
We consider ||
2
as a function of (= ct/x) for various xed values of
kx. These curves are all parabolas satisfying ||
2
= 1 at = 0 and = 1
and with a minimum value at =
1
2
that depends on kx (Fig. 2.5). The
absolute minimum ||
2
= 0 occurs when cos k = 1, i.e. kx = , cor-
responding with a wavelength L = 2/k = 2x. Recall that this is the
minimum resolvable wavelength. Figure 2.5 shows plots of ||
2
against for
L = 2x, 4x and 8x. Clearly, within the stable region 0 < < 1, the
scheme is damping, but the degree of damping decreases as the wavelength
increases. Since the true solution has a constant amplitude, the damping
indicates an error due to nite dierencing, At the shortest resolvable wave-
length, the damping may be very large unless t is extremely small.
Figure 2.5: Plots of ||
2
versus for the nite-dierence scheme (2.14) and
for various values of L.
40
2.5 Time dierencing schemes
We consider time dierencing schemes rst in the context of ODEs of the
type
dU
dt
= f(U, t), U = U(t), (2.29)
where t denotes the time. We divide the time axis into segments of equal
length t and denote the approximate value of U at time nt by U
(n)
. We
assume that U
(0)
, U
(1)
, ... , U
(n)
are known and seek to construct a scheme for
computation of an approximate value U
(n+1)
. There are many possibilities.
2.5.1 Two-level schemes
These relate U
(n+1)
to U
(n)
by approximating the exact formula
U
(n+1)
= U
(n)
+
_ (n+1)t
nt
f(U, t)dt. (2.30)
Euler (or forward) scheme: In this, f(U, t) is assumed constant, equal
to the value at time nt. Then
U
(n+1)
= U
(n)
+ t f
(n)
,
where
f
(n)
= f(U
(n)
, nt). (2.31)
The truncation error of this scheme is O(t), i.e., it is a rst-order accurate
scheme. Because f is computed at one end of the interval, the scheme is
said to be uncentred in time. Such schemes are invariably only rst-order
accurate.
Backward scheme: Now f(U, t) is assumed constant, equal to f
(n+1)
, i.e.
U
(n+1)
= U
(n)
+ t f
(n+1)
. (2.32)
Since f
(n+1)
depends on U
(n+1)
, the scheme is called implicit. For an ODE it
may be simple to solve such a dierence equation for U
(n+1)
, but for PDEs
this requires solving a set of simultaneous equations, one for each grid point
of the computational region.
If the value of f
(n)
does not depend on U
(n+1)
, such as in (2.31), the
scheme is called explicit.
The truncation error of (2.32) is also O(t).
41
Trapezoidal scheme: Here f is assumed to vary linearly between its val-
ues at nt and (n + 1)t so that its mean value is used for the integral,
i.e.
U
(n+1)
= U
(n)
+
1
2
t (f
(n+1)
+ f
(n)
). (2.33)
This is an implicit scheme also, but its truncation error is O(t
2
).
Next we consider a pair of iterative schemes:
Matsuno (or Euler-backward) schemes: First a step is made using the
Euler scheme. The value of U
n+1

so obtained is used to approximate f

(n+1)
,
say f
(n+1)

, which is then used to make a backward step. Therefore

U
(n+1)

= U
(n)
+ t f
(n)
,
U
(n+1)
= U
(n)
+ t f
(n+1)

,
_
, (2.34)
where
f
(n+1)

= f(U
(n+1)

, (n + 1)t).
This is an explicit scheme and is rst-order accurate.
Heun scheme: This is similar to the previous one, it is explicit and second-
order accurate, but the second step is made using the trapezoidal scheme,
i.e.
U
(n+1)

= U
(n)
+ t f
(n)
,
U
(n+1)
= U
(n)
+
1
2
t
_
f
(n)
+ f
(n+1)

_
.
_
(2.35)
2.5.2 Three-level schemes
Except at the rst time step, one can store the value U
(n1)
and construct
schemes using the time history of U. These are three-level schemes. They
approximate the formula
U
(n+1)
= U
(n1)
+
_ (n+1)t
(n1)t
f(U, t) dt, (2.36)
or they can use the additional value to improve the approximation to f in
(2.30). Two examples are:
42
Leapfrog scheme: In this, f is taken to be constant, equal to the value
at time nt, whereupon
U
(n+1)
= U
(n1)
+ 2t f
(n)
. (2.37)
This is probably the scheme most widely used in the atmospheric sciences.
It is second-order accurate with truncation error O(t
2
).
Adams-Bashforth scheme: The scheme that is usually called the Adams-
Bashforth scheme in the atmospheric sciences is, in fact, a simplied version
of the original Adams-Bashforth scheme, which is fourth-order accurate. The
simplied version is obtained when f in (2.30) is approximated by a value
obtained at the centre of the interval t by a linear extrapolation using
values f
(n1)
and f
(n)
. This gives
U
(n+1)
= U
(n)
+ t (
3
2
f
(n)

1
2
f
(n1)
). (2.38)
This is an explicit, second-order accurate scheme.
Milne-Simpson scheme: In this, Simpsons rule is used to calculate the
integral in (2.36), giving an implicit scheme.
2.6 Properties of schemes - the oscillation equa-
tion
The stability and other important properties of the foregoing time dierenc-
ing schemes depend on the form of the function f(U, t). In order to discuss
these properties we need to prescribe this function. For applications to at-
mospheric models it is of particular interest to consider the case f = iU,
i.e., the ODE:
dU
dt
= iU, U = U(t). (2.39)
We shall refer to this as the oscillation equation. We allow U to be
complex (e.g. u + iv) so that, in general, the equation represents a system
of two equations. The parameter is taken to be real and is called the
frequency. Note that the time variation of a single harmonic of the advection
equation satises this equation if = kc (cf. Eq. (2.19) and the equation
below it).
The general solution of (2.39) is U(t) = U(0)e
it
, or, for discrete values
t = nt,
43
U(nt) = U(0)e
int
. (2.40)
If we consider the solution in the complex plane, its argument rotates by
t each time step and there is no change in amplitude. We use Neumanns
method to analyse the properties of various schemes. Let
U
(n+1)
= U
(n)
, (2.41)
and
= ||e
i
. (2.42)
Then the numerical solution can be written
U
(n)
= ||
n
U
(0)
e
in
. (2.43)
Here represents the change in argument (or phase change) of the numerical
solution during each time step. Since we know that the amplitude of the true
solution does not change, we shall require 1 for the stability.
Denition 1: We say that a scheme is unstable, neutral or damping,
according as || > 1, = 1 or < 1, respectively.
Denition 2: We say that a scheme is accelerating, has no eect on
the phase speed, or decelerating, according as
/(t) > 1, = 1 or < 1.
For accuracy it is desirable to have both the amplication and the relative
speed, (/t)/, close to unity.
2.6.1 Two-level schemes
The three non-iterative two-level schemes in section 2.5.1 can be described
by a single nite-dierence equation
U
(n+1)
= U
(n)
+ t (f
(n)
+ f
(n+1)
), (2.44)
with a consistency requirement that + = 1. The Euler scheme has = 1,
= 0; the backward scheme = 0, = 1 and the trapezoidal scheme =
1
2
,
=
1
2
.
Applied to the oscillation equation, (2.44) gives
U
(n+1)
= U
(n)
+ it (U
(n)
+ U
(n+1)
). (2.45)
Then
U
(n+1)
= U
(n)
, (2.46)
44
where
=
1 +ip
1 ip
=
1 p
2
+ i( + )p
1 +
2
p
2
, (2.47)
and
p = wt. (2.48)
2.6.2 Euler scheme
= 1 +ip, || = (1 +p
2
)
1/2
. (2.49)
This scheme is always unstable. If t << 1/||, then p << 1 and || =
1 +
1
2
p
2
+ . . ., i.e. || = 1 + O(t
2
). Then || 1 is an order of magnitude
less than the maximum allowed by the von Neumann stability condition.
Nevertheless, an indiscriminate use of the Euler scheme for the solution of
the atmospheric equations leads to amplication at a quite unacceptable rate.
2.6.3 Backward scheme
= (1 +
1
4
ip)/(1 +p
2
), || = (1 +p
2
)

1
2
. (2.50)
This scheme is stable irrespective of the size of t; it is an unconditionally-
stable scheme. However, since || < 1, it is damping with the amount of
damping increasing as the frequency (and hence p) increases. The latter
property is often considered desirable of a scheme, because it leads to the
selective removal of undesirable high-frequency noise.
2.6.4 Trapezoidal scheme
=
_
1
1
4
p
2
+ ip
_
/
_
1 +
1
4
p
2
_
, || = 1. (2.51)
This scheme is always neutral.
Note that both implicit schemes are stable irrespective of the size of t.
The iterative two-level schemes can be described by a single equation also:
U
(n+1)

= U
(n)
+ t f
(n)
,
U
(n+1)
= U
(n)
+ t
_
f
(n)
+ f
(n+1)

_
+ = 1.
_

_
(2.52)
45
The Matsuno scheme has = 0, = 1; the Heun scheme =
1
2
, =
1
2
.
When applied to the oscillation equation, (2.52) gives
U
(n+1)

= U
(n)
+it U
(n)
U
(n+1)
= U
(n)
+it
_
U
(n)
+ U
(n+1)

_
_
(2.53)
whereupon
U
(n+1)
= U
(n)
,
with
= 1 p
2
+ ip. (2.54)
2.6.5 Matsuno scheme
= 1 p
2
+ ip, || = (1 p
2
+ p
4
)
1/2
. (2.55)
This scheme is stable if |p| 1 so that to achieve stability we must ensure
that
t 1/||. (2.56)
Accordingly, the Matsuno scheme is conditionally stable. The higher the
frequency, the more restrictive the stability condition.
Dierentiating (2.55) gives
d||
dp
= p(2p
2
1)/(1 p
2
+ p
4
)
1
/
2
.
Thus the amplication factor of this scheme has a minimum when p = 1/

2.
Therefore, if we can choose t so that 0 < p < 1/

2 for all the frequen-

cies present, then the scheme will reduce the relative amplitudes of higher
frequencies.
2.6.6 Heun scheme
= 1
1
2
p
2
+ ip, || = (1 +
1
4
p
4
)
1/2
.. (2.57)
This is always > 1 so that the Heun scheme is always unstable. However, for
small p,
|| = 1 +
1
8
p
4
+ . . . , (2.58)
i.e. || = 1 + O(t
4
). The instability is therefore quite weak. Experience
shows that it can be tolerated when we can choose t suciently small.
46
The results of the ve two-level schemes are summarized in Fig. 2.6.
Since the amplication factors are all even functions of p, it is only necessary
to show the curves for p 0.
Figure 2.6: The amplication factors as a function of p = t for the ve
two level schemes and for the true solution.
2.6.7 Phase change of schemes
It is of interest to consider the phase change per time step, , together with
the relative phase change per time step, /p, the former being dened by
= tan
1
[Im()/Re()]. (2.59)
2.6.8 Euler and backward schemes
From (2.49) and (2.50)
/p = (1/p) tan
1
p, (2.60)
and since |(1/p) tan
1
p | < 1, |/p| < 1 for a stable scheme. Thus both
schemes are decelerating. For p = 1, /p = /4.
47
Matsuno scheme Equation (2.55) gives
/p = (1/p) tan
1
[p/(1 p
2
)]. (2.61)
For stability |p| 1 and for relatively small frequencies (p << 1),
/p = 1 +
2
3
p
2
+ ......
Therefore, the Matsuno scheme is accelerating. When p = 1, /p = /2. The
analysis of phase errors of schemes applied to the oscillation equation is not
so important as an analysis of the amplication factor. Firstly, phase errors
do not aect the stability of a scheme. Moreover, when the schemes are used
to solve the PDEs of atmospheric motion, additional phase errors occur as a
result of space dierencing. These are usually the dominant source of error.
2.7 Three level schemes, computational modes
2.7.1 Leapfrog scheme
When applied to the oscillation equation, Eq. (2.37) gives
U
(n+1)
= U
(n1)
+ 2it U
(n)
. (2.62)
Three-level schemes such as this require two initial conditions to initialize
the calculation. Physically, a single initial condition should have been ade-
quate. It turns out that, besides this physical initial condition, three-level
schemes require a computational initial condition U
(1)
. This will have to be
determined by using a two-level scheme for the rst step. Now (2.23) gives
U
(n+1)
= U
(n1)
U
(n+1)
=
2
U
(n1)
. (2.63)
whereupon (2.62) reduces to

2
2ip 1 = 0, (2.64)
which is a quadratic equation for . The solutions are

1
=
_
1 p
2
+ ip

2
=
_
1 p
2
+ ip. (2.65)
It follows that there are two solutions of the form U
(n+1)
= U
(n)
. In general,
an m-level scheme gives m1 solutions of this form. Each possible solution
is called a mode. If such a solution represents an approximation to the true
solution, it is clear that must tend to unity as t 0, i.e. as p 0. In
(2.64),
1
1, but
2
1 as 0. Thus the solution with
1
is called
the physical mode; that with =
2
is not an approximation to the solution
and is called the computational mode.
48
A simple example. Consider the case = 0. Then (2.39) becomes
dU
dt
= 0 with solution U = constantU
(0)
.
The leapfrog scheme gives U
(n+1)
= U
(n1)
and from (2.64) = 1, or

1
= 1,
2
= 1.
Suppose the rst step happens to give U
(1)
= U
(0)
. Then the solution
with
1
gives the true solution, and consists of the physical mode only.
In contrast, if the rst step gives U
(1)
= U
(0)
, then, for all n, U
(n+1)
=
U
(n)
; i.e. the solution would consist entirely of the computational mode.
Apparently a good choice for the computational initial condition is crucial
for a satisfactory numerical solution to the problem.
In general, since (2.62) is linear, its solution will be a linear combination
of the two solutions
U
(n)
1
=
n
1
U
(0)
1
and U
(n)
2
=
n
2
U
(0)
2
, i.e.,
U
(n)
= a
n
1
U
(0)
1
+ b
n
2
U
(0)
2
, (2.66)
where a and b are constants. To satisfy the two initial conditions (U
(0)
, U
(1)
specied) requires that
U
(0)
= aU
(0)
1
+ bU
(0)
2
,
U
(0)
= a
1
U
(0)
1
+ b
2
U
(0)
2
,
which can be solved for aU
(0)
1
and bU
(0)
2
. Then (2.66) gives
U
(n)
= [
n
1
(U
(1)

2
U
(0)
)
n
2
(U
(1)

1
U
(0)
)]/(
1

2
). (2.67)
Clearly, the amplitudes of the physical and computational modes are
proportional to

U
(1)

2
U
(0)

and

U
(1)

1
U
(0)

,
respectively, and these depend on U
(1)
. If we were able to choose U
(1)
=

1
U
(0)
, the solution would consist of the physical mode only; if it happened
that U
(1)

2
U
(0)
, the solution would consist entirely of the computational
mode. Unfortunately, for more complicated functions f(U, t) than studied
here, it is frequently not possible to determine
1
and
2
and we have to live
with the presence of the computational mode.
Normally, U
(1)
is computed using one of the two-level schemes. The
simplest method is to use the Euler scheme, but use of a higher-order scheme
such as the Heun scheme gives a smaller amplitude of the computational
mode.
49
2.7.2 Stability of the leapfrog scheme
Case |p| < 1 In (2.65), 1 p
2
> 0 and ||
1
= ||
2
1.
In this range of p, both the physical mode and the computational mode
are stable and neutral. For the phase change, using (2.59) and (2.65),

1
= tan
1
_
p/
_
(1 p
2
)
_

2
= tan
1
_
p/
_
(1 p
2
)
_
(2.68)
and for 0 p < 1,
2
=
1
. As p 0,
1
p while
2
p
so that for small t, the physical mode approximates the true solution and
the behaviour of the computational mode is quite dierent. For p < 0,

2
=
1
. The phase change is plotted in Fig. 2.7 as a function of
x = p/(1 p
2
)
1/2
.
Figure 2.7: Phase change of the physical and of the computational mode for
the leapfrog scheme.
For accuracy of the physical mode,
1
should closely approximate the
phase change of the true solution, p. For |p| << 1, (2.68) gives

1
= p +
1
6
p
3
+ . . . ,
i.e. the leapfrog scheme is accelerating. However, the acceleration is four
times less than that of the Matsuno scheme.
50
One can illustrate the behaviour of the two modes in the complex plane.
Note that
U
(n)
1
= U
(0)
1
e
in
1
, U
(n)
2
= U
(0)
2
e
in(
1
. (2.69)
For simplicity assume that
1
= /8 and that Im (U
(0)
) = 0 at t = 0. As
seen in the gure, the physical mode rotates in the positive sense by an angle

1
in each time step t, while in the case p > 0, the computational mode
rotates by an angle
1
.
2.7.3 The Adams-Bashforth scheme
The stability analysis of this scheme when applied to the oscillation equation
is left as an exercise (Ex. 2.1.). It turns out that the physical mode of the
scheme is always unstable, but like the Heun scheme, the amplication is
only by a fourth order term and it can be tolerated when t is suciently
small. It has a useful property that the computational mode is damped.
Figure 2.8: The physical and computational modes for the leapfrog scheme
when =

8
and when the imaginary parts are zero at the initial moment.
The rst ve time steps are shown.
51
2.8 Properties of schemes applied to the fric-
tion equation.
We consider now the properties of schemes when applied to the equation
dU
dt
= U, U = U(t), > 0. (2.70)
Following Mesinger and Arakawa (1976), we refer to this as the friction equa-
tion. An equation of this type arises if we try to solve the diusion equation
[cf. Eq. (1.1)] .
u
t
=

2
u
x
2
> 0 > 0. (2.71)
Then, the substitution u(x, t) = Re
_
U(t)e
ikx

gives
dU
dt
= k
2
U, (2.72)
equivalent to Eq. (2.70) if =
2
. Alternatively, if U = u + iv represents
the velocity vector in the complex plane, Eq. (2.70) describes the motion of
a particle of unit mass subject to a frictional retardation proportional to the
velocity. The general solution of (2.70) is
U(t) = U(0)e
t
(2.73)
Thus both the real and imaginary parts of U (e.g. u and v) decrease expo-
nentially with time.
The properties of schemes applied to Eq. (2.70) will be analyzed using
the von Neumann method.
2.8.1 Noniterative two-level schemes - see Eq. (2.44)
Applied to the friction equation, (2.44) gives
U
(n+1)
= U
(n)
t (U
(n)
+ U
(n+1)
). (2.74)
Let
K = t. (2.75)
Then (2.74) may be written
U
(n+1)
=
1 K
1 +K
U
(n)
. (2.76)
52
Euler scheme
= 1, = 0; = 1 K.
Hence the stability requires that |1 k| 1, or 0 < K 2. We see at once
that the stability criteria of particular schemes do not have to be the same
when they are applied to dierent equations. In the case of (2.76), we would
wish to be more demanding in the choice of t by choosing K < 1; this
prevents the solution (2.76) from oscillating from time step to time step.
Backward scheme
= 0, = 1; = 1/(1 +K).
Now stability requires only that K > 0. Furthermore, the solution does not
oscillate in sign.
Trapezoidal scheme
=
1
2
, =
1
2
; = (1 K)/(1 +K).
Iterative two-level-schemes The scheme (2.52) when applied to (2.70)
gives
U
(n+1)
= (1 K + K
2
)U
(n)
. (2.77)
Therefore both the Matsuno and Heun schemes are stable for suciently
small values of K.
Leapfrog scheme One can show that when this scheme is applied to (2.71)
the computational mode is unstable (see Ex. 2.2). The scheme is therefore
unsuitable for solving the friction equation.
Adams-Bashforth scheme One can show that this scheme is stable for
suciently small values of K and that the computational mode is damped
(see Ex. 2.3).
2.9 A combination of schemes
The question arises: what do we if the equation contains both the oscillation
and the friction term, i.e.
53
dU
dt
= iU U. (2.78)
For example, we may wish to use the leapfrog scheme because of the oscilla-
tion term iU, but we have just seen that it cannot be used for the friction
term U. In this and in similar situations we can use dierent schemes for
the dierent terms; e.g. we might use the leapfrog scheme for the oscillation
term and the forward scheme for the friction term. This would give
U
(n+1)
= U
(n1)
+ 2t(i U
(n)
U
(n1)
). (2.79)
Other combinations would be possible also.
Exercises
(2.1) Show that when applied to the oscillation equation, the Adams-Bashforth
scheme - Eq. (2.38) - gives
U
(n+1)
= U
(n)
+ t (
3
2
U
(n)

1
2
U
(n1)
). (2.80)
Show further that the substitution (2.63) into this equation gives

_
1 +
3
2
ip
_
+
1
2
ip = 0. (2.81)
Deduce that for small p, the roots of this equation are approximately

1
=
_
1
1
2
p
2

1
8
p
4
. . .
_
+ i
_
p +
1
4
p
3
+ . . .
_

2
=
_
1
2
p
2
+
1
8
p
4
. . .
_
+ i
_
1
2
p
1
4
p
3
+ . . .
_
(2.82)
and that the amplication factors are: |
1
| = 1 +
1
4
p
2
+ . . . , |
2
| =
1
2
p + . . .. Note that the scheme is weakly unstable.
(2.2) Show that when applied to the friction equation (2.70), the leapfrog
scheme gives
U
(n+1)
= U
(n1)
2ktU
(n)
, (2.83)
and that the (complex) amplication factor has solutions

1
= K +

1 +K
2

2
= K

1 +K
2
,
(2.84)
where K = kt. Show that the solution associated with
1
is the
physical mode and that associated with
2
is the computational mode.
54
Deduce that the computational mode is always unstable. It follows
that the leapfrog scheme is unsuitable for the solution of the friction
equation.
(2.3) Show that when applied to the friction equation (2.70), the Adams-
Bashforth scheme has the (complex) amplication factor
=
1
2
_
1
3
2
K
_
1 K +
9
4
K
2
_
. (2.85)
Deduce that this scheme is stable for suciently small values of K and
that the computational mode is damped.
(2.4) The linearized shallow-water equations for unidirectional waves on a
homogeneous layer of nonrotating uid of uniform undisturbed depth
H are
u
t
= g
h
x
,
h
t
= H
u
x
, (2.86)
where h(x, y) is the disturbed depth and u(x, y) is the horizontal ve-
locity perturbation. Show that solutions have the form
(u (x, t) , h (x, t)) = Re
__
u,

h
_
e
ik(xct)
_
, (2.87)
where c =

gH. Taking central space dierences, show that the cor-

responding dierential-dierence equations have solutions of the form
(u
j
, h
j
) = Re
__
u,

h
_
e
ik(jxc

t)
_
, (2.88)
and that the corresponding dispersion relation is
c

=
_
gH
sin kx
kx
. (2.89)
Show that the dispersion equation in the analogous two-dimensional
formulation of the continuous system, corresponding to (2.86), is
2
=
gH (k
2
+ l
2
) , where, for example,
h (x, y, t) = Re
_

he
i(kx+lyt)
_
.
55
Chapter 3
The advection equation
We return now to consider nite-dierence schemes for the advection equa-
tion. This will set the stage for our later study of the equations governing
atmospheric motion. We begin with the simplest one-dimensional linear form
of the equation already discussed in section
3.1 Schemes with centred second-order space
dierencing
The one-dimensional linear advection equation is
u
t
+ c
u
x
= 0, c = constant, (3.1)
where u = u(x, t), and its general solutionis u = f(x ct), where f is an
arbitrary function. One of the nite-dierence schemes for this equation is
the forward and upstream scheme (2.9) or (2.14) which we showed in Chapter
2 to have excessive damping. If the space derivative in (3.1) is approximated
by a central dierence, we obtain
u
j
t
= c
u
j+1
u
j1
2x
. (3.2)
A number of schemes for (3.1) can be constructed by approximating the time
derivative in (3.2) by one of the methods in section 2.2. For example, the
leapfrog scheme gives
u
n+1
j
u
n1
j
2t
= c
u
n
j+1
u
n
j1
2x
. (3.3)
56
The properties of schemes constructed in this way can be inferred from
the known properties of time-dierencing schemes applied to the oscillation
equation. We substitute in (3.2) a tentative solution in the form
uj = Re[U(t)e
ikjx
] (3.4)
whereupon
dU
dt
= i
_

c
x
sin kx
_
U. (3.5)
This is a form of the oscillation equation studied in section 2.2 if
=
c
x
sin kx. (3.6)
Leapfrog scheme If we apply this to (3.5) we obtain
U
(n+1)
= U
(n1)
+ 2i
_
c
t
x
sin kx
_
U
(n)
. (3.7)
Let
p = c
t
x
sin kx. (3.8)
Then, according to our earlier analysis we require |p| 1 for stability, i.e.

ct
x
sin kx

1
for any possible k. But | sinkx| 1, so the criterion reduces to
t/x 1, (3.9)
which is, again, the CFL criterion.
Note that the maximum value of |p|, i.e. the minimum stability, is asso-
ciated with the wave with kx =

2
. This is the component with wavelength
4x, twice the shortest resolvable wavelength 2x. There are two solutions
for U(0), the physical and the computational mode,
U
(n)
1
=
n
1
U
(0)
2
, U
(n)
2
=
n
2
U
(0)
2
(3.10)
where
1
and
2
are given by (2.65). In the stable case, we have

1
= e
i
, qquad = tan
1
_
p/
_
(1 p
2
)
_
(3.11)
57

2
= e
i()
= e
i
, (3.12)
where the + or sign is taken according as p > 0 or p < 0. Combining
(3.10) with (3.4) we obtain the physical mode
u
n
j
= Re[U
(0)
1
e
ik(jx+

kt
nt)
], (3.13)
and the computational mode
u
n
j
= Re[(1)
n
U
(0)
2
e
ik(jx+

kt
nt)
]. (3.14)
These expressions, should be compared with the true solution (2.20), i.e.,
u(x, t) = Re[U(0)e
ik(xct)
]. (3.15)
It is clear that the phase speed of the physical mode, c
1
, is equal to /kt.
Equation (3.12) shows that as t 0, p and Eq. (3.8) shows that
as x 0, p ckt. It follows that, as x, t 0, c
1
c; i.e. the
phase speed of the physical mode approaches the phase speed of the solution,
while c
2
c. In addition, the computational mode changes sign at all grid
points from time step to time step, because of the factor (1)
n
in (3.14).
Matsuno scheme First the approximate values u
(n+1)
j
are calculated us-
ing the forward scheme, i.e.,
u
(n+1)

j
u
n
j
t
= c
u
n
j+1
u
n
j1
2x
. (3.16)
Then the are used in the backward scheme, that is
u
n+1
j
u
n
j
t
= c
u
(n+1)

j+1
u
(n+1)

j1
2x
. (3.17)
We eliminate the starred quantities in (3.17) using (3.16) to obtain
u
n+1
j
u
n
j
t
= c
u
n
j+1
u
n
j1
2x
+ c
2
t
u
n
j+2
2u
n
j
+ u
n
j2
(2x)
2
. (3.18)
Without the last term, this represents the nite-dierence equation obtained
when the forward scheme is used for the time derivative in Eq. (3.2). The
third term approaches zero as x, t 0 and (3.18) is therefore a consistent
scheme for the advection equation.
58
3.2 Energy method
We show now another example of the use of the energy method for testing
stability. This method ca be applied also to non-linear equations, and to
study the eect of boundary conditions on stability. The method can be
used to test the stability of a group of schemes for solving (3.1). Many
schemes of interest can be written as
u
n+1
j
u
n
j
=
1
2
(u

j+1
u

j1
) (3.19)
where
m = ct/x, (3.20)
and u

j
is a linear function of a number of values u
n
j
.
Non-iterative two-level schemes take
u

j
= u
n
j
+ u
n+1
j
. (3.21)
Iterative two-level schemes, take
u

j
= u
n
j

1
2
(u
n
j+1
u
n
j1
). (3.22)
Adam-Bashforth scheme take
u

j
=
2
3
u
n
j

1
2
u
n
j1
. (3.23)
Here we analyse the stability of the non-iterative two-level schemes.
First we multiply (3.19) by u

j
and sum over j to obtain

j
u

j
(u
n+1
j
u
n
j
) =
1
2

j
u

j
(u

j+1
u

j1
).
If cyclic boundary conditions are assumed, the right-and side vanishes where-
upon

j
u

j
(u
n+1
j
u
n
j
) = 0. (3.24)
Now

j
1
2
_
(u
n+1
j
)
2
(u
n
j
)
2

j
1
2
(u
n+1
j
+ u
n
j
)(u
n+1
j
u
n
j
).
59
Substituting (3.21) into (3.24) and writing = 1 gives

j
1
2
_
_
u
n+1
j
_
2

_
u
n
j
_
2
_
=
_

1
2
_

j
_
u
n+1
j
u
n
j
_
2
. (3.25)
Thus if >
1
2
the scheme is unstable; if =
1
2
it is neutral; if <
1
2
the
total energy,

1
2
_
u
n
j
_
2
, decreases monotonically with time.
3.3 Lax-Wendro scheme
Unlike the schemes described so far, the two-step Lax-Wendro scheme can-
not be constructed by an independent choice of nite-dierence approxi-
mations to the space and to the time derivative of the advection equation.
Consider the stencil shown in Fig. 3.1. First, provisional values of u are
calculated at the centre of the two rectangular meshes of the stencil, denoted
by stars. This is done using centred space and forward time dierencing.
Figure 3.1: The space-time stencil for the Lax-Wendro scheme.
The values of u
n
j+
1
2
and u
n

1
2
are obtained by taking arithmethic averages of
the values u
n
j
at the nearest grid points. Then
u
n+
1
2
j+
1
2

1
2
(u
n
j+1
+ u
n
j
)
1
2
t
= c
u
n
j+1
u
n
j
x
(3.26)
60
u
n+
1
2
j
1
2

1
2
(u
n
j
+ u
n
j1
)
1
2
t
= c
u
n
j
u
n
j1
x
. (3.27)
Using these provisional values a further step is made, this one is time-centred
in both space and time, i.e.,
u
n+1
j
u
n
j
t
= c
u
n+
1
2
j+
1
2
u
n+
1
2
j
1
2
x
. (3.28)
Elimination of the provisional values from (3.28) using Eqs. (3.26) - (3.27)
gives
u
n+1
j
u
n
j
t
= ct
u
n
j+1
u
n
j1
2x
+
1
2
c
2
t
u
n
j+1
2u
n
j
+ u
n
j1
x
2
. (3.29)
This dierence equation is rather similar to (3.18) obtained using simple
centred space dierencing and the Matsuno time dierence, the only dier-
ence being in the last term. Again, this approaches zero as x, t 0;
with t xed it now approaches
1
2
c
2
t(
2
u/x
2
), again a diusion term,
but with a coecient half the previous one. Furthermore, this term is now
calculated over an interval 2x, and its damping eect will be a maximum
at that wavelength. This kind of dependence on wavelength is often consid-
ered desirable of a nite dierence scheme. This is because there are serious
problems with nite-dierence around 2x. It is often possible to alleviate
these problems by using a dissipative scheme which preferentially damps the
two-grid interval waves.
While (3.18) was rst-order accurate in time, (3.29) has truncation error
O[(x)
2
+ (t)
2
] and is therefore second-order accurate in both space and
time.
Stability properties Substitute
u
n
j
= Re[U(n)e
ikjx
], (3.30)
into (3.29) to obtain
U
(n+1)
= [1
2
(1 cos kx) isin kx]U
(n)
). (3.31)
Therefore
61
= 1
2
(1 cos kx) imsin kx. (3.32)
It is easy to show that
|| =
_
1 4
2
_
1
2
_
sin
4 kx
2

1
2
. (3.33)
Since

sin
1
2
kx

1, the expression in brackets cannot fall below 14

2
(1

2
), which is unity if = 1 and has a minimum when
2
=
1
2
. Hence || 1.
Thus like the Lax-Wendro scheme, the scheme is stable if
|c|
t
x
1,
again, the CFL condition. The scheme is damping for || < 1. For a given
wavelength, the amplication factor has a minimum value

min
= (1 sin 4
1
2
kx)
1/2
, (3.34)
which equals zero for kx =
1
2
, or 2/k = 2x and approaches unity as
k 0 , i.e. as the wavelength tends to innity. The amplication factors
for waves of wavelength 2x and 4x are shown in Fig. 3.2. The amount
of damping is generally quite large for shorter wavelengths. Unfortunately,
it depends also on the time step and on the advection velocity (through
m). This is a disadvantage of the Lax-Wendro scheme because there is no
reason why the amount of damping should depend on these quantities and it
is either impractical or impossible to control the damping by changing them.
For example, for small values of m, expansion of (3.33) gives
|| = 1 2
2
sin 4
1
2
kx + . . .
showing that for a given amount of time (a xed value of nt) the total
damping will be approximately proportional to t. However, we would prefer
to choose t to give the best accuracy and stability properties, not to give
the optimum amount of damping. The Lax-Wendro scheme has been widely
used in atmospheric models on account of its reasonably good behaviour. It
is second-order accurate, explicit, not unconditionally unstable, and has no
computational mode. None of the schemes obtained by combining centred
space dierencing with one of the seven time dierencing schemes studied in
section 2.2 has all of these advantages. The dissipation of the scheme will
not be harmful if its total eect is negligible compared with the physical
dissipation, and it can be useful for controlling the shortest waves. If the
physical dissipation is very small or non-existent, it is better to use a neutral
scheme.
62
Figure 3.2: Amplication factor of the Lax-Wendro scheme as a function of
= ct/x, for waves of wavelength 2x and 4x.
3.4 Computational dispersion
The linear advection equation (3.1) has no dispersion in the sense that all
wave modes have the same phase speed c. We shall show that this is generally
not true of the nite-dierence approximation to it.
Consider the equation
u
j
t
+ c
u
j+1
u
j1
2x
= 0, (3.35)
that is obtained by approximating u/x by a centred space dierence. An
equation of this type is a dierential-dierence equation. Since the time
derivative has retained its dierential from, any error in (3.35) is due to the
space dierencing. Equation (3.35) has a solution in the form of a single
harmonic component
u
j
= Re
_
U (t) e
ikjx

, (3.36)
provided that
63
dU
dt
+ ik
_
c
sin kx
kx
_
U = 0, (3.37)
while (3.1) with the substitution gives
dU
dt
+ ikc U = 0. (3.38)
Thus the solutions of (3.35) propagate with the phase speed c

given by
c

= c
sin kx
kx
, (3.39)
which is a function of the wavenumber k. Thus, the nite dierencing in
space gives rise to a dispersion of the waves. As kx increases from zero,
the phase speed c

decreases monotonically from c and becomes zero for the

shortest resolvable wavelength 2x, when kx = .
In summary, all waves propagate at a speed that is less than the true phase
speed c, with this decelerating eect increasing as the wavelength decreases.
The two-grid interval wave is stationary.
It is clear why the two-grid interval wave is stationary when we look at
the plot of that wave, shown in Fig. 3.3. For such a wave, u
j+1
= u
j1
at all
grid points, whereupon (3.35) gives a zero value for u
j
/t.
The consequence of a reduction of the true advection speed is a gen-
eral retardation of the advection process. Moreover, the dependence of this
reduction on wavelength is particularly serious for the shortest waves.
If the pattern that is being advected represents a superposition of more
than one wave, the false dispersion will result in a deformation of that pat-
tern. In the atmosphere, small-scale patterns such as fronts, shear lines, etc.
represent a superposition of many waves with a signicant proportion of the
energy in the shortest waves resolvable. For this reason, in NWP models,
such patterns, if present in the initial elds, are rather rapidly deformed,
until they acquire a form which is less sharp than in the beginning. Since
such small-scale features are often associated with signicant weather, the
eect of computational dispersion require careful attention.
3.5 Group velocity
In linear dispersive wave theory, the group velocity is dened by
c
g
=
d
dk
=
d
dk
(kc), (3.40)
64
Figure 3.3: Plot of a two-grid interval wave, with a wavelength of 2x.
where = (k) is the dispersion relationship that relates frequency to
wavenumber k. In the case of the linear advection equation (3.1), c is a
constant, whereupon c
g
= c. However, for the dierential-dierence equation
(3.35),
c

g
=
d
dk
(kc

) = c cos kx. (3.41)

Thus as kx increases from zero, the group velocity c

g
decreases mono-
tonically from c
g
and becomes equal to c
g
for the shortest resolvable wave-
length 2x. The results are summarized in Fig. 3.4. For the exact advection
equation (3.1), both individual harmonic waves and wave packets (i.e. places
where superposition of waves results in a maximum amplitude of a group of
neighbouring wavenumbers) propagate at the same constant speed c = c
g
.
Introduction of the centred space nite dierence in (3.35) makes both the
phase speed and the group velocity decrease as the wavenumber increases.
The error is especially great for the shortest resolvable wavelengths; waves
with wavelengths less than 4x even have a negative group velocity. Thus
wave packets made up of these waves propagate in the direction opposite to
the advection velocity and opposite to the direction of individual waves.
As an example, consider the sinusoidal-like function Y (x) that is slowly-
varying in space on the scale of a grid length. Let us dene
u
j
= (1)
j
Y
j
, (3.42)
65
Figure 3.4: Phase speed and group velocity for the linear advection equa-
tion and the corresponding dierential-dierence equation with second-order
space dierencing.
shown in Fig. 3.5. Thus the function Y (x) is the envelope of the function
u
j
. Suppose that we calculate the advection of u
j
using (3.35), i.e.
Y
j
t
c
Y
j+1
Y
j1
2x
= 0,
which is obtained by substituting (3.42) into (3.35). It is clear that the
advection of Y
j
is governed by the same equation as for the advection of u
j
,
except that the advection velocity appears with an opposite sign! Therefore,
as the individual short waves of the function u
j
slowly propagate in the
positive x direction, the envelope Y (x), which has much longer wavelength,
propagates relatively fast in the opposite direction.
When Y (x) is a sine function, so that it consists of only a single harmonic,
it propagates with no change in shape. Then, because of (3.42), u
j
must
also be advected with no change in shape, from which we conclude that u
j
consists also of a single harmonic component. If, on the other hand, the
function Y (x) consisted of a number of harmonic components, the shapes of
both Y (x) and u
j
would change during the advection process as a result of
the computational dispersion of these components.
66
Figure 3.5: Functions u
j
and Y (x), illustrating the propagation of an enve-
lope of short as waves when their advection is calculated using second-order
space dierencing.
Analytic solution of (3.35). It is possible to solve Eq. (3.35) analytically.
With the re-scaled time variable, = ct/x, the equation can be written
2
d
d
u
j
() = u
j1
() u
j+1
(). (3.43)
This is the recurrence formula of the Bessel function of the rst kind of order
j, J
j
(), i.e., a solution is
u
j
() = J
j
(). (3.44)
Several of these functions, of the order, are shown in Fig. 3.6. Note that j,
(1)
j
J
j
= J
j
.
Figure 3.6: The Bessel functions J
0
(), J
1
() and J
2
(). Note that J
1
() =
J
1
(), J
2
() = J
2
().
67
Now in (3.43), j can take any integer value, since the location for which
we choose j = 0 is arbitrary. Thus, a solution that is more general than (3.44)
is
u
j
() = J
jp
(t),
where p is an arbitrary integer. Moreover, because Eq. (3.43) is linear, a still
more general solution is
u
j
() =

p=
a
p
J
jp
(), (3.45)
where a
p
are arbitrary constants. Now, J
0
(0) = 1 and J
n
(0) = 0 for n = 0.
Hence,
p = j, u
j
(0) = a
j
.
It follows that the constants in (3.45) can be chosen to satisfy an arbitrary
initial condition u
j
= u
j
(0), j. Therefore (3.45) is the general solution of
(3.43) .
Suppose that the initial condition consists of a spike function
U
j
(0) =
_
1 for j = 0
0 for j = 0
(3.46)
shown at the top of Fig. 3.7.
It follows from (3.43) that du
j
/d = 0 at = 0, except at points j 1
and j = 1, where it is equal to
1
2
and
1
2
, respectively.
If the spike function were an initial condition for the advection equation
(3.1), it would simply translate to the right with uniform speed while re-
maining unchanged in shape. However, in the dierence equation (3.35), the
disturbance spreads out in both directions as series of waves; i.e. it disperses.
Figure 3.7 illustrates an example of computational dispersion associated
with second-order space dierencing. The main disturbance is advected at a
speed only slightly less than the physical one. It mostly consists of the longer
wave components which have an advection speed not much dierent from the
physical advection velocity. One sees also the propagating of a group of short
waves in the direction opposite to that of the physical advection. Since the
appearance of these waves contradicts the physical properties of the advection
equation, such waves are called parasitic waves.
For the spike function, the solution of the dierential-dierence equation
is quite unsatisfactory as an approximation to the true solution. However,
this is an extreme example, although it does illustrate one of the limitations
involved with the numerical technique.
68
Figure 3.7: The analytic solution of (3.41) for the initial condition condition
shown in the uppermost diagram for = 5 and = 10.
3.6 Schemes with uncentred dierencing
The space dierencing in the advection equation (3.1) can be approximated
also using uncentred dierencing. Two possibilities are to use a forward or
backward scheme, according to whether c is negative or positive, i,e.,
u
j
t
+ c
u
j
u
j1
x
= 0 for c > 0, (3.47)
or
69
u
j
t
+ c
u
j+1
u
j
x
= 0 for c < 0, (3.48)
These equations are again dierential-dierence equations. In both cases,
the dierences are calculated on the side from which the advection veloc-
ity reaches the centre point. For this reason we call Eqs. (3.47)-(3.48) an
upstream dierence schemes.
A series of schemes for the advection equation can be constructed by ap-
proximating the time derivatives in Eqs. (3.47)-(3.48) by one of the many
possible consistent methods. The resulting schemes will be only rst order
accurate. However, they have one advantage over centred schemes in space
when applied to the advection equation: with upstream dierences, a distur-
bance cannot propagate in the direction opposite to the physical advection.
Thus no parasitic waves will contaminate the numerical solution. Suppose
that a forward dierence is used for the time derivative in (3.1) and that
c > 0; then (3.47) gives
u
n+1
j
u
n
j
t
+ c
u
n
j
u
n
j1
x
= 0 (3.49)
This scheme was used in Chapter 2 where it was found to be damping,
with the amount of damping depending on the wavelength. The maximum
damping occurred for the 2x wave. The analytic solution of the dierence
equation (3.49) was discussed by Wurtele (1961).
The advantage that is achieved, at least in principle, by using upstream
dierencing as compared with centred or downstream dierencing (e.g. 3.47
for c > 0), can be illustrated by considering the domain of inuence of a grid
point in dierent schemes.
We stay with the case c > 0. As shown in Chapter 1 (section 1.3), a
grid point value can be said to propagate along the characteristics x ct =
constant. Figure 3.8 shows a grid point marked by a circle with the associated
characteristic passing through it. With upstream dierencing as in (3.49), the
value at the grid point will inuence the values at points within the domain
shaded by vertical lines. The gure shows also the domains of inuence
with centred and downstream dierencing. One of the domains of inuence,
that given by upstream dierencing, is clearly the best approximation to the
characteristic line representing the domain of inuence in the solution.
The foregoing discussion suggests constructing a scheme for Eq. (3.1) by
tracing a characteristic back from a point (jx, (n + 1)t) to intersect the
previous time level t = nt and calculating the value at the point of inter-
section by interpolation (see Fig. 3.9). We then set u
n+1
j
= u

. If we choose
70
Figure 3.8: Domains of inuence of a grid point for the scheme (3.49) with
upstream dierencing as well as for schemes with centred dierencing, down-
stream dierencing and the true solution.
a linear interpolation procedure that employs values at two neighbouring
points at the time nt, we obtain
u
n+1
j
= u
n
j1
+
u
n
j
u
n
j1
x
(x ct).
This is identical to the scheme (3.49), with upstream dierencing. If a
quadratic interpolation procedure is chosen using three neighbouring points,
one obtains the Lax-Wendro scheme (Ex. 3.1).
Analytic solution of 3.47 Further insight into the properties of schemes
that can be obtained from (3.47) may be obtained from an analytic solution to
this dierential-dierence equation. For small values of t this solution will
approximate the solution obtained from the dierence schemes. As before,
we scale the time by writing = ct/Dx. Then Eq. (3.47) becomes
d
d
u
j
() + u
j
() u
j1
() = 0. (3.50)
It may easily be veried that a solution of this equation is the Poisson
frequency function.
u
j
() =
_
e

jp
(jp)!
for j p
0for j < p
(3.51)
Again, p is an arbitrary integer which relates to the fact that the location
of the point j = 0 is arbitrary. An example of the Poisson frequency function
71
Figure 3.9: Sketch for the construction of schemes by calculation of a previous
value on a characteristic passing through the point (jx, (n + 1)t).
is shown in Fig. 3.10 for the case when = 4. The inclusion of a vertical
scale is unnecessary because the area enclosed by the graph is always unity.
In particular, when = 0, the histogram consists of a single rectangle with
its ordinate equal to unity. Thus it corresponds with the initial pulse-like
disturbance (3.46) studied earlier. As increases, (3.51) transforms into
a skew bell-shaped histogram of the type shown in Fig. (3.10). Its mean
position on the x-axis is

jp=0
(j p)
e

jp
(j p)!
,
i.e., it moves at unit non-dimensional speed. Thus the mean position prop-
agates at the physical advection velocity. However, the maximum point of
the histogram logs behind as indicated by the skewed-shape of the histogram
shown in Fig. 3.10. Negative values of u
j
, which would be physically unre-
alistic, do not occur and there are no parasitic waves on the opposite side
of zero from the direction of the physical advection. Finally, since the total
amount of the advected quantity is exactly conserved, but the disturbance is
damped quite rapidly. As before, a more general solution than (3.51) is the
linear combination
j

p=
a
p
e

jp
(j p)!
, (3.52)
where a
p
are arbitrary constants. Putting = 0 in this formula gives
72
Figure 3.10: The Poisson frequency function (3.51) when = 4.
u
j
(0) = a
j
,
so that the constants ap can be chosen to satisfy arbitrary initial conditions.
The form of (3.52) indicates that the value u
j
(t) at a point j can be attributed
to the superposition of the eect of the initial values at that point and of the
initial values at all the points located upstream of it.
Figure 3.11 compares the solution (3.45) for centred dierencing with that
from (3.52) for upstream dierencing for the initial disturbance
u
j
(0) =
_
1 for j = 1, 0 , 1,
0 otherwise,
at three nondimensional times. If the grid distance is 300 km and c =
15 ms
1
, then 5 units of nondimensional time corresponds with about 1 day
in physical time. Therefore, the damping eect of upstream dierencing is
seen to be appreciable.
3.7 Schemes with centred fourth-order-space
dierencing
Many of the diculties described above, in particular the phase speed error
and the computational dispersion, result from the approximation used for
73
Figure 3.11: Analytic solutions of the exact advection equation (heavy solid
line), of the equation using centred-dierencing (dashed line), and of the
equation using upstream dierencing (thin solid line), for three dierent val-
ues of non-dimensional time.
space dierencing. Therefore we are led to consider schemes using high-order
space dierencing.
Using the Taylor series expansion discussed in section 2.1, one can show
that
u
x
=
4
3
u
j+1
u
j1
2x

1
3
u
j+2
u
j2
4x
+ O(x
4
),
which is a fourth-order accurate approximation to u/x (see Ex. 3.2). Then,
in analogy with (3.2), the dierential-dierence approximation to (3.1) is
u
j
t
+ c
_
4
3
u
j+1
u
j1
2x

1
3
u
j+2
u
j2
4x
_
= 0. (3.53)
Then, taking u
j
(t) = Re[U(t)e
ikjx
] as before, we obtain the phase speed
c = c
_
4
3
sin kx
kx

1
3
sin 2kx
2kx
_
, (3.54)
instead of c = c(sin kx/kx) for the case of second-order dierencing. For
small kx, the latter formula gives
74
c

= c
_
1
1
3!
(kx)
2
+ . . .
_
,
whereas Eq. (3.54) gives
c

c
_
1
4
5!
(kx)
4
+ . . .
_
.
Figure 3.12: Phase speed for the linear advection equation, c, and for the cor-
responding dierential-dierence equations with second-order (c

) and with
fourth-order (c

) centred-space dierencing.
Thus, even though the decelerating eect is still present, the phase speed
error is considerably reduced. The two phase speeds c

and c

are shown in
Fig. 3.12 for all values of k. There is clearly a very signicant increase in
accuracy of the phase speed for large-scale and medium-scale waves. How-
ever, as the wavelength approaches its minimum value of 2x, the increase
in phase speed obtained by fourth order dierencing diminishes, until, nally,
the wave with wavelength 2x is again stationary. Moreover, for short waves
the slope of the phase speed curve is greater than with second-order dier-
encing implying that the computational dispersion of these waves is greater.
Thus a short wave disturbance will be distorted more rapidly.
The use of additional grid points for higher order space dierencing does
introduce some side diculties. As in the case of higher order time dierenc-
ing, it leads to the appearance of computational modes, but in space. More-
over, the formulation of boundary conditions becomes more complicated.
For small-scale disturbances, comparable to two grid intervals in space, no
nite-dierence scheme is very satisfactory
75
3.8 The two-dimensional advection equation
The two-dimensional linear advection equation is
u
t
+ U
u
x
+ V
u
y
= 0, (3.55)
where u = u(x, y, t) is a function of two space variables x, y and of time t,
and U, V are components of the advection velocity, assumed constants. The
advection speed is
c =

U
2
+ V
2
. (3.56)
The corresponding dierential-dierence equation is

t
u
j,m
= U
u
j+1,m
u
j1,m
2x
V
u
j,m+1
u
j,m1
2y
, (3.57)
where the space derivatives are approximated by second-order central dier-
ences. Here, denotes u(jx, my) and the grid-points are located at points
x = jx, y = my. The substitution
u
j,m
= Re
_

U(t)e
i(kx+ly)
_
, (3.58)
into (3.57) gives the oscillation equation
d

U
dt
= i
_

U
x
sin kx
V
y
sin ly
_

U. (3.59)
If the leapfrog scheme is used for the time derivative, the stability criterion
is

_
U
x
sin kx +
V
y
sin ly
_
t

1 (3.60)
This equation has to be satised for all possible values of the wavenumbers k
and l. Let us consider the case x = y = d, say. As before, the minimum
resolvable wavelength in the x-direction occurs when kd = . Likewise, the
minimum resolvable wavelength in the y-direction occurs when ld = . Hence
the admissible region of wavenumber space for the square grid is the shaded
region shown in Fig. 3.12. Consider the function
(k, l, U, V ) =
U
x
sin kx +
V
y
sin ly 0
in this region. This function has an extremum where /k = 0 and /l =
0, i.e., at kd =

2
, ld

2
, the centre of the region. At this point a wave has
76
wavelength components 2/k = 4d, 2/l = 4d. For a given value of the
advection speed c(U
2
+ V
2
)
1
2
1
2
, the left-hand-side of (3.59) has a maximum
value at this point if /U = 0 and /V = 0, i.e., if 1U/|c|
2
U
2
)
1
2
1
2
=
0, which requires that U = V . In this case the advection velocity makes an
angle of /4 with the x-axis and U = V = c/

2. Thus the stability criterion

(3.60) becomes

2c
t
x
1. (3.61)
Figure 3.13: Admissible region of wavenumber for a square two-dimensional
grid, with grid length d.
It follows that we must choose a smaller time step (1/

2 times smaller)
than in the one-dimensional case to ensure stability. The minimum stability
is associated with wavelengths in both the x and y directions twice as long as
the shortest resolvable wavelength of 2d, just as in the one-dimensional case.
The two-dimensional wavenumber of this wave is (k
2
+
2
)
1/2
which is greater
by a factor of

2 than wavenumbers along the axes, and its wavelength is

shorter by the same factor. This applies to all waves with k = l.
77
3.9 Aliasing error and nonlinear instability
We consider now the one-dimensional nonlinear advection equation
u
t
+ u
u
x
= 0, (3.62)
discussed in section 1.3. As shown there, the general solution of this equation
is
u = F(x ut), (3.63)
where F is some arbitrary function. Here we consider the eect of the multi-
plication in (3.62). When performed in nite-dierences, it results in an error
related to the inability of the discrete grid to resolve wavelengths shorter than
2x, i.e. wavenumbers greater than k
max
= /x. Consider, for example,
the function
u = sin kx, (3.64)
where k < k
max
. Substitution into the nonlinear term of (3.62) gives
u(u/x) = k sin kx cos kx =
1
2
k sin 2kx.
Hence, if the wavenumber in (3.64) lies in the interval
1
2
k
max
< k k
max
,
the nonlinear term will give a wavenumber that is outside the range that can
be resolved by the grid. Such a wave cannot be properly reproduced in a
nite-dierence calculation.
Consider a wave for which k > k
max
, e.g., let l = x as indicated by
the continuous line in Fig. 3.16. Knowing only the values at grid points it
is not possible to distinguish this wave from the one shown by the dashed
line. Thus from the considerations explained in the section 2.1, where it is
assumed that only the longest waves are present, there will be an error. This
is called an aliasing error.
In a more general case, suppose that the function u consists of a number
of harmonic components
u =

n
u
n
.
Then the nonlinear term will contain products of harmonics of dierent wave-
lengths, such as
sin k
1
xcos k
2
x = [sin(k
1
+ k
2
)x + sin(k
1
k
2
)x].
78
Figure 3.14: A wave of wavelength
4
3
x, misrepresented by the nite dier-
ence grid as a wave of wavelength 4x.
Thus, even if a nite-dierence calculation is begun with waves which all
have k k
max
nonlinear interaction will lead ultimately to wave components
with k > k
max
and these waves will be mispresented.
Note that for all k,
sin kx = sin[2k
max
(2k
max
k)]x,
and since k
max
= /x, this becomes
sin kx = sin
2
x
x cos
_
2
x
k
_
x cos
2
x
x sin
_
2
x
k
_
x.
In particular, at grid points x = jx,
sin
2
x
jx = 0, cos
2
x
jx = 1,
whereupon
sinkjx = sin(2k
max
k)jx. (3.65)
It follows that, knowing only grid point values, we cannot distinguish the
wavenumbers k from the wavenumbers 2k
max
k. Further, if k > k
max
,the
wavenumber k will be misrepresented as the wavenumber
k

= 2k
max
k. (3.66)
Therefore, as shown in Fig. 3.14, the resulting wave has wavenumber k

which is less that k

max
by an amount equal to that by which k is greater
than k
max
. We can think of the wavenumber k

as being an image obtained

79
Figure 3.15: Misrepresentation of a wavenumber k > k
max
in accordance
with (3.66).
by the reection of k across the value k
max
into the admissible range of
wavenumber.
In the example considered above ( =
4
3
x), k =
3
2
(x)
1
and k

2
(x)
1
, the wavenumber seen by the nite dierence grid. The latter
wave has wavelength 4x.
The consequence of aliasing error in a numerical integration of the equa-
tions of atmospheric motion is a cascade of energy into scales that cannot
be resolved. After a time, this energy can be expected to grow beyond phys-
ically acceptable limits. Indeed, it is found that if no measures are taken
to prevent or control this build up, the calculation suers a catastrophic
breakdown. The phenomenon is called nonlinear instability and was rst
discovered by Norman Phillips in 1956 in an early attempt to model the
atmospheric general circulation.
Phillips started from an atmosphere at rest and sought to integrate the
vorticity equation for a model time of 30 days. The calculation came to a
catastrophic end before this time, a fact that Phillips attributed initially to
excessive truncation error. Therefore he repeated the experiment with re-
duced space and time steps. However, the breakdown still occurred at about
the same time. Later he repeated the experiment, but after every two hours
of simulated time he carried out a harmonic analysis of the vorticity elds,
eliminating all components with k >
1
2
k
max
1
. This solved the instability
problem.
1
Orszag (1971) showed that, in fact, it is sucient to eliminate only the top one-
third of wavenumbers, because if waves with k >
2
3
k
max
are ltered out all aliases satisfy
k >
2
3
k
max
and will be removed.
80
3.10 Ways to prevent nonlinear instability
If we wish to carry out an integration for an extended period of time it is
necessary to suppress nonlinear instability. Possible methods are:
I) To use a dierencing scheme that has a built-in damping of the shortest
waves, e.g. the Lax-Wendro scheme. Indeed, Kasahara (1969) showed
that it is sucient to apply the scheme intermittently at relatively long
time intervals.
II) To add a dissipative term to a scheme that is not dissipative, enabling
the amount of dissipation to be better controlled.
III) To use a Lagrangian formulation of the advection terms instead of an
Eulerian formulation. More about such schemes will be presented later.
IV) To use schemes that are free of the spurious inow of energy into the
shorter waves, instead of suppressing the amplitude of these waves.
Such schemes were developed by Arakawa (1966, 1972).
3.11 Arakawas conservative scheme
Consider a homogeneous, incompressible, inviscid, two-dimensional ow. The
vorticity ()-streamfunction () equations are

t
= u = (u), (3.67)
u = k , (3.68)
=
2
, (3.69)
in standard notation. Suppose that the ow is contained in a closed domain
D so that there is no ow across the domain boundary S. Then

t
_
D
dV =
_
D
(u)dV =
_
D
u ndS = 0,
where n is a unit vector normal to S and pointing from D. It follows that
the total (or mean) vorticity is conserved.
Multiplying (3.67) by gives
81

t
_
1
2

2
_
=
1
2

2
u
_
,
and application of the divergence theorem to this shows that the total or mean
square vorticity is conserved also. Vorticity squared is called enstrophy.
Finally, multiplying (3.67) by and using (3.69) gives

t
_
1
2

2
_
=
1
2

2
u
_
. (3.70)
The left-hand-side of this can be written

t
( )
_

t
_
=
_

t
_

_

t
_
and the right-hand-side can be rearranged as
(u) = (u) u .
Substitution of these into (3.70) gives

_

t
_
=
_

t
_
+ (u) u .
The last term vanishes because of (3.68), while the rst term is

t
_
1
2
()
2
_
,
just the rate-of-change of kinetic energy. Again with the use of Gausss
theorem, it follows that

t
_
D
1
2
u
2
dV = 0,
i.e. the mean kinetic energy is conserved in D.
In summary, for any closed region, the model expressed by (3.67) - (3.69)
conserves mean kinetic energy, mean vorticity, and mean enstrophy. Ob-
viously it would be desirable and perhaps even necessary to retain these
properties when the dierential equation is replaced by a dierence equa-
tion.
Suppose that D is a rectangular region of dimensions L
x
and L
y
over
which the streamfunction can be represented as a double Fourier series. The
implication is that the streamfunction is periodic in the x-direction with
82
period L
x
and is zero along the upper and lower boundary, but kinetic energy,
vorticity and enstrophy are still conserved.
=

m=0

n=1
_
a
m,n
(t) cos
2mx
L
x
+ b
m,n
(t) sin
2mx
L
x
_
sin
ny
L
y
. (3.71)
If we calculate the product u in (3.67) and use standard trigonometric
identities, it is found that if a term in u with wavenumbers m
1
, n
1
is multi-
plied by a term in with wavenumbers m
2
,n
2
, the resulting term involves
the four pairs of wavenumbers
m
1
+ m
2
, n
1
+ n
2
m
1
m
2
, n
1
+ n
2
m
1
+ m
2
, n
1
n
2
m
1
m
2
, n
1
n
2
(3.72)
It follows that there is a transfer of vorticity and energy between dierent
wavenumbers associated with this term. Again, nonlinear interaction re-
sults in the exchange of energy within the total spectrum and may develop
wavenumbers which previously did not exist.
Now the individual terms of (3.72) are orthogonal eigenfunctions of the
Helmholtz equation

m,n
+ m
2
m,n

m,n
= 0, (3.73)
where the eigenvalues are given by

2
m,n
= (2/L
x
)
2
+ (n/L
y
)
2
. (3.74)
Because the eigenfunctions
m,n
are orthogonal, the mean kinetic energy and
mean enstrophy are
K =
1
2

(
m,n
)
2
=

K
m,n
=
1
2

2
m,n

2
m,n
(3.75)

2
=

(
2

m,n
)
2
2

2
m,n
K
m,n
=

4
m,n

2
m,n
, (3.76)
where the bar indicates a domain average
2
. Since K and
2
are conserved,
so also is their ratio
2
/K = 2
2
, where is an average wavenumber. In
2
The penultimate expression in (3.76) follows from the vector identity =
()
2
.
83
other words, a systematic energy cascade towards higher wavenumbers is not
possible in this ow. Furthermore, to obtain the enstrophy the contributions
K
m,n
are multiplied by the wavenumber squared. Therefore the fraction of
energy that can ow into high wavenumbers is clearly limited, and the higher
the wavenumber, the more it is limited. These results were obtained by
Fjrtoft (1953).
Charney (1966) showed that the situation can be illustrated by a simple
mechanical analogy. Equations (3.75)-(3.76) give
K
2
=

2
m,n
K
m,n
.
As shown in Fig. (3.16), we can imagine a semi-innite weightless rod on
which a weight K is suspended at a distance =
2
to the left of the point
at which the rod is suspended, and weights K
1
= K
1,1
, K
2
= K
1,2
, K
3
K
1,3
,
are suspended at distances
1
=
1,1
,
2
=
2
1,2
,
3
=
2,1
, to the right of
that point. The rod, as dened, would be in mechanical equilibrium. Its
left-hand-side cannot change, while on the right-hand-side an interchange of
mass between the weights is permitted, but only so as not to disturb the
equilibrium, i.e., the total moment of the forces. Since the value of for a
particular K is xed, it follows that at least three components must always
take part in an energy transfer (see Ex. 3.3). In particular, very little energy
can be expected to accumulate in the highest wavenumbers through a cascade
of energy from the lowest wavenumbers.
Figure 3.16: A mechanical analogy of the interchange of energy between
harmonic components.
Consider now the numerical solution of Eqs. (3.67) - (3.69). These can
be written as a single equation

t
(
2
) = J(,
2
), (3.77)
using the Jacobian notation, J(f, g) = (f/x)(g/y) (g/x)(f/y). If
a nite-dierence scheme could be constructed so as to conserve the average
84
values of the kinetic energy and enstrophy, a systematic transfer of energy
towards the highest wavenumbers could not occur and the nonlinear insta-
bility found by Phillips would be avoided. The method for doing this is due
Arakawa op. cit.. The idea is to nd a form of the Jacobian term in (3.77)
that has the appropriate conservation properties.
Note that for any two scalar functions f and g,
J(f, g) = k (fg) = k (g f).
Using Stoke s circulation theorem it follows that the domain average of J is
zero, i.e.,
J(f, g) = 0, (3.78)
if either f or g is a constant along the boundary of the domain. Moreover,
under the same conditions,
fJ(f, g) = 0 , gJ(f, g) = 0. (3.79)
Now there are three alternative ways to write the Jacobian in (3.77), i.e.,
j(, ) =

x

y


y

x
(3.80)
=

x
_

y
_

y
_

x
_
(3.81)
=

y
_

x
_

x
_

y
_
. (3.82)
Based on these, we consider possible ways to construct second-order accurate
nite-dierence approximations to the Jacobian. With the simplest centred
space dierencing, we require values of and from a box of nine adjacent
grid points to evaluate (3.80)-(3.82) - see Fig. 3.17. Let d be the grid size
and
n
,
n
the values of and at the point n.
Then the corresponding approximations to (3.80-3.82) are
J
++
0
= [(
1

3
)(
2

4
) (
2

4
)(
1

3
)]/4d
2
(3.83)
J
+x
0
= [
1
(
5

8
)
3
(
6

7
)
2
(
5

6
) +
4
(
8

7
)]/4d
2
(3.84)
J
x+
0
= = [
2
(
5
y6) z4(
8

7
)
1
(
5

8
) +
3
(
6

7
)]/4d
2
. (3.85)
The superscripts + and x denote the position of the points from which values
of and , respectively, are used to form the approximations. A more general
approximation to the Jacobian is
85
Figure 3.17: Stencil used for the nite-dierence approximation to the Jaco-
bian in (3.80)-(3.82).
J(, ) = J
++
+ J
x+
+ J
+x
, (3.86)
with the consistency requirement that + + = 1. We shall seek to choose
, and to ensure conservation of mean kinetic energy and enstrophy.
Consider the gain in enstrophy at point zero in g. 3.17 due to the value at
point 1 and vice versa. The results, excluding the factor 4d
2
, are (3.87)

0
J
++
0

0

1
(
2

4
) + other terms

1
J
++
1

1

0
(
5

8
) + other terms (3.87)
Since these terms do not cancel and no other opportunity for the product

1
occurs over the grid, the sum J
++
cannot vanish in general, and
2
will
not be conserved (i.e.,
2
/t == 0). Similarly

0
J
+x
0

0

5
(
1

2
) + . . .

5
J
+x
5

5

0
(
y
2
1
) + . . .

0
J
x+
0

0

1
(
5

8
) + . . .

1
J
x+
1

1

0
(
2

4
) + . . . (3.88)
From (3.87) and (3.88) it is clear that both J
+x
and
1
2
(J
++
+J
x+
) conserve

2
. A similar examination of the nite-dierence approximation of J(, z)
shows that
86
J
x+
and J
+x 1
2
(J
++
+ J
+x
) conserve K =
1
2
u
2
.
Combining the two results it follows that the
1
3
(J
++
J
+x
+ J
x+
) conserves both
2
and K. (3.89)
Consequently it conserves also the mean wavenumber. These conservation
properties prevent nonlinear instability. Aliasing is still present in the form of
phase errors, but the latter results also from linear nite-dierence truncation
errors. The expression (3.89) is called the Arakawa-Jacobian.
Arakawa (1966) showed also how to construct an approximation of fourth-
order accuracy to the Jacobian with the same conservation properties.
3.12 Conservative schemes for the primitive
equations
Consider the nonlinear advection equation
u
t
+ u
u
x
= 0 (3.90)
and its corresponding kinetic energy equation

t
_
1
2
u
2
_
+

x
_
1
3
u
3
_
= 0. (3.91)
Integration of (3.91) with respect to x gives
_
L
0

t
_
1
2
u
2
_
dx =
1
3
_
u
3
L
u
3
0
_
. (3.92)
If u
0
= u
L
, including the case of zero ux at the boundaries when both are
zero, then

t
_
L
0
1
2
u
2
dx = 0, i.e. the kinetic energy is conserved.
If the interval [0, L] is divided into n equal segments with end points at
x
j
= jd where d = L/n (j = 0, 1, . . . n), then the right-hand side of (3.92)
can be written as the sum

1
3
[(u
3
1
u
3
0
) +(u
3
2
u
3
1
) +. . . +(u
3
j
u
3
j1
) +(u
3
j+1
u
3
j
) +. . . +(u
3
n
u
3
n1
)]
87
Note that when the variables are discretized, the integral becomes a sum
over the domain. Moreover, for conservation to occur, the successive terms
in the sum must be of the form (A
j+1
A
j
), so that all intermediate terms
cancel as in the above expression. Suppose one were to use centred space
dierencing to approximate (3.90); i.e.,
u
j
t
+ u
j
u
j+1
u
j1
2d
= 0. (3.93)
Multiplying this by u
j
and forming the kinetic energy integral as above gives

t
_
L
0
1
2
u
2
dx =
1
2
n

j =0
(u
2
j
u
j+1
u
2
j
u
j1
). (3.94)
It is clear that the terms in the summation are not of the form (A
j+1
A
j
),
whereupon the scheme does not conserve kinetic energy. This is despite the
fact that it is a consistent and linearly stable scheme for t suciently small.
Another possibility would be to use the ux form of the equation, i.e.,
u
j
t
=

x
(
1
2
u
2
j
) = (u
2
j+1
u
2
j1
)/4d.
This would give for the right-hand side of (3.94),

1
4
n

j =0
(u
2
j+1
u
j
u
j
u
2
j1
),
in which again, the terms are not of the form (A
j+1
A
j
). Hence this does
not lead to an energy conserving scheme either. A further possibility would
be to use the dierence form
u
j
t
=
1
6d
(u
j+1
+ u
j
+ u
j1
)(u
j+1
u
j1
).
Multiplying this by u
j
and summing leads to the right-hand side for (3.94):

1
6
n

j=0
_
(u
2
j
u
j+1
+ u
j
u
2
j+1
) (u
2
j1
u
j
+ u
j1
u
2
j
)

,
in which the terms are of the form (A
j
A
j1
). Accordingly the scheme is
energy conserving for interior points. Although this nite-dierence form is
too simple for more complex systems of equations, it does suggest the use of
some kind of averaging technique to achieve the conservation property.
88
Exercises
3.1. Show that by using a quadratic interpolation formula for tracing back
the characteristic to the time level t = nt in Fig. (3.9), one obtains
the Lax-Wendro scheme.
3.2. Derive the fourth-order accurate approximation
u
x
=
4
3
u
j+1
u
j1
2x

1
3
u
j+2
u
j2
4x
+ O(x
4
),
on page 68 using a Taylor series.
3.3. Verify the statement on page 77 concerning Fig. 3.15 that at least
three components must always take part in an energy transfer.
89
Chapter 4
The gravity and inertia-gravity
wave equations
Like advection, wave propagation is a particularly important process con-
tained in the equations for atmospheric motion. Accordingly, we shall be
concerned with investigating numerical methods for treating the terms in
the equations that represent wave propagation. We begin with the simplest
linearized system governing the horizontal propagation of gravity- or inertia-
gravity waves.
4.1 One dimensional gravity waves
For perturbations to a layer of homogenous uid of uniform undisturbed
depth H, there are two equations:
u
t
= g
h
x
, (4.1)
h
t
= H
u
x
, (4.2)
where u(x, t) and h(x, t) are the perturbation horizontal velocity and uid
depth, respectively. Equation (4.1) is the linearized form of the momentum
equation while (4.1b) is the depth-integrated form of the continuity equation
(see e.g. DM, Chapter 11). We look for solutions of (4.1)-(4.2) of the form
[u(x, t), h(x, t)] = Re[( u,

h)e
i(kxt)
] (4.3)
where u and

h are constants that satisfy the homogeneous algebraic equations
u = gk

h,

h = Hk u.
90
These equations can be combined to give the dispersion relation

2
= gHk
2
. (4.4)
The phase speed of waves is therefore
c =
h
t
= H
u
x
. (4.5)
It follows that gravity waves can propagate along the x-axis in either direction
with speed

gH. In particular, they are nondispersive.

When the space derivatives are approximated by central dierences, the
dierential-dierence equations corresponding with Eqs. (4.1) - (4.2) are
u
j
t
= g
h
j+1
h
j1
2x
, (4.6)
h
j
t
= H
u
j+1
u
j1
2x
. (4.7)
The solution (4.3) then takes the form
(u
j
, h
j
) = Re[( u,

h) e
i(kjxt)
], (4.8)
and substitution of these into Eqs. (4.6) - (4.7) gives
u = g
sin kx
x

h ,

hH
sin kx
x
u.
These combine to give the dispersion relation

2
= gH
_
sin kx
x
_
2
. (4.9)
Thus, with the nite-dierence approximation to the space derivatives, the
gravity waves propagate with the phase speed
c

=
_
gH
_
sin kx
kx
_
(4.10)
= c
sin kx
kx
. (4.11)
Again, space dierencing leads to computational dispersion. The formula
(4.11) is the same as that obtained when considering the advection equation
(see e.g. 3.39). It follows that both the phase speed and group velocity
depend on the wavenumber as shown in Fig. 3.4. The phase speed decreases
91
as the wavelength decreases, and the wave with wavelength 2x is stationary.
Nevertheless, there is an important dierence between this problem and the
advection problem in that there are now two dependent variables.
We have assumed that u and h are carried at every grid point as shown
in Fig. 4.1a. However, as far as the system (4.6) - (4.7) is concerned, u
j
depends only on h at adjacent points and h
j
depends only on u at adjacent
points. Thus the grid contains two elementary subgrids, with the solution
on one of these subgrids being completely decoupled from the other. It would
appear to be sucient to calculate only one of these solutions by using the
grid shown in Fig. 4.1b. Such a grid, with variables carried at alternate
points in space, is called a staggered grid. The computation time needed
to solve (4.6) - (4.7) on this grid is halved, whereas the truncation error is
the same. Moreover, the waves with kx >
1
2
have been eliminated and
these are just the waves with large phase speed errors and negative group
velocities. This amounts to a signicant improvement in the properties of
the scheme, because, when using the staggered grid, the phase and group
velocity diagram in Fig. 3.4 is reduced to its left half, covering waves with
wavelengths no shorter than 4x.
Figure 4.1: A grid with two dependent variables u and h. In (a) these are
stored at every grid point; in (b) at alternate grid points.
If we wish to include waves with wavelengths between 4Dx and 2Dx in
our calculation we can halve the grid length and will still require no more
computation time than if we had used a nonstaggered grid.
4.2 Two-dimensional gravity waves
In two-dimensions, the linearized shallow-water equations are
u
t
= g
h
x
,
v
t
= g
h
y
,
92
h
t
= H u. (4.12)
Substituting the wave solution
(u, v, h) = Re[( u, v,

h)
i(kx+lyx)
], (4.13)
we obtain the dispersion relation

2
= gH(k
2
+ l
2
). (4.14)
The phase speed, c = w/|k|, where k = (k, l) is the wavenumber vector. It
follows again that c =

gH.
The construction of a suitable grid, either non-staggered or staggered,
presents a number of possibilities. Three of these, denoted (A), (E) and (C)
are shown in Fig. 4.2.
Let d

denote the shortest distance between the grid points. With the
same value of d

, the lattice (E) will have half the number of variables per
unit area than the lattice (A), while lattice (C) will have only a quarter of
the number. The lattice (E) can be obtained by a superposition of two (C)
lattices and the lattice (A) by a superposition of two (E) lattices, or of four
(C) lattices.
Figure 4.2: Three types of lattice for the nite-dierence solution of 4.12.
The admissible range of wavenumbers in the wavenumber plane can be
determined by considering the shortest resolvable wavelengths. In lattice (E),
the lines joining the nearest points with the same variable makes an angle
of /4 with the grid lines while with the other two lattices these lines are
along the grid lines. Fig. 4.3 shows the admissible range of wavenumbers.
A halving of the number of variables is accompanied by a halving of the
admissible region of the wavenumber plane.
The same standard nite-dierence approximations can be used for the
space derivatives in (4.12) for all three lattices. We dene the dierence
operator
93

x
h = [h(x + d

, y) h(x d

, y)]/2d

,
and similarly for
y
h. Then (4.12) can be approximated by
h
t
= H(
x
u +
y
v). (4.15)
Substituting the appropriate wave solutions (analogous to 4.3) we obtain the
dispersion relation

2
= gH
sin
2
kd

+ sin
2
ld

d
2
. (4.16)
Figure 4.3: Admissible regions of wavenumbers for the three types of lattice
shown in Fig. 4.2.
If we dene X = d

k, Y = ld

, the ratio of the phase speed given by

(4.16), c

, to the true phase speed

gH, can be written as

c

gH
=

sin
2
X + sin
2
Y
X
2
+ Y
2
. (4.17)
This formula reduces to (4.10) when applied to the one-dimensional case.
The values of the relative phase speed (4.17) on the wavenumber range
admitted by lattice (E) are shown in Fig. 4.4. By symmetry about the line
l = k, only half of the region needs to be shown. Fig. 4.3 shows that lat-
tice (C) admits only the left half of the triangular region shown in Fig. 4.4.
Clearly lattice (C) gives a more accurate phase speed for gravity waves than
the other two lattices. Unfortunately, because it does not carry the two ve-
locity components at the same points, there is some diculty if the Coriolis
terms have to be included. Of the other lattices, the staggered lattice (E) is
94
much superior to the non-staggered lattice (A). A result with the same trun-
cation error can be achieved in about half of the computation time, (exactly
half if the equations are linear), and a signicant fraction of wavenumbers
that are associated with large phase speed errors and computational dis-
persion are eliminated. The additional time needed for a calculation on an
(A) lattice is spent on waves that can hardly be expected to improve the
integration.
Despite its superiority over lattice (A), the lattice (E) is not free of compu-
tational problems. As with the non-staggered one-dimensional grid discussed
in section 4.1, the solution of (4.15) on each of the two type (C) subgrids
forming the (E) lattice are independent and can diverge from each other.
This can lead to serious problems. For example, if the values of the depen-
dent variables on one of these (C) lattices are constants, they will constitute
a stationary solution on that lattice, no matter what the values of the vari-
ables on the other (C) lattices are. Two stationary solutions, with dierent
constant values on each of these complementary lattices, will give a station-
ary wave represented by the right-hand corner of the triangular region in Fig.
4.4, with a zero phase speed. This wave is usually referred to as the two-
grid-interval wave. In the same way, the (A) lattice admits four independent
stationary solutions, with dierent constant values on each of its four type
(C) subgrids.
Figure 4.4: Relative phase speed of gravity waves when the space derivatives
in (4.12) are approximated by space-centred nite-dierence analogues.
The two-grid-interval wave can easily be generated when boundary condi-
tions are articially prescribed, and, with more complete equations, in cases
when gravity waves are generated inside the computational region. These
95
can be caused by heating, for example through the release of latent heat,
and also by the inuence of mountains. When gravity waves are excited
involving variables of one of the (C) subgrids only, for example by forcing
at individual grid points or lines of points, the gravity wave will propagate
through the variables (C) of this subgrid only. The variables of the other (C)
subgrid will be inuenced only through the Coriolis and advection terms on
a much larger time scale. Thus physical eects which may excite relatively
long waves in the atmosphere may excite spurious waves with wavelengths
of approximately two grid intervals in a computation. When these reach an
excessive amplitude, some remedial measures have to be taken - see later.
4.3 Inertia-gravity waves
We consider now the eect of centred space dierencing on inertia-gravity
waves. The linearized shallow-water equations with rotation present are
u
t
= g
h
x
+ fv,
v
t
= g
h
y
fu,
h
t
= H u, . (4.18)
These dier from Eqs. (4.12) through the appearance of the two Coriolis
terms. Since these do not involve derivatives, they are dicult to calculate
on the (C) lattice, which was ideal for pure gravity waves. For this reason
we need to reconsider the problem of the distribution of the variables.
Equations (4.18) admit two distinct types of motion: low-frequency, quasi-
geostrophic and quasi-nondivergent ow; and high-frequency inertia-gravity
waves (see e.g. DM, Chapter 11). Inertia-gravity waves are continually ex-
cited in the atmosphere, but as they are dispersive, a local accumulation of
wave energy disperses with time. This process is known as geostrophic ad-
justment. The remaining motion is in approximate geostrophic balance and
changes only slowly with time. Here we are concerned with the correct repre-
sentation of this process, which is governed essentially by the inertia-gravity
wave equations.
We are interested both in waves caused by physical eects, and in those
caused by inadequacies of the initial data and of the numerical procedures.
Accordingly, we shall investigate the eect of the space distribution on the
dispersive properties of the inertia-gravity waves. We accomplish this using
the simplest centred approximations for the space derivatives, leaving the
time derivatives in their dierential form. The discussion follows that of
96
Winnigho and Arakawa, as presented by (Arakawa, 1972; Arakawa et al.,
1974).
We consider ve ways of distributing the dependent variables in space,
shown in Fig. 4.5. Let d denote the shortest distance between neighbouring
points carrying the same dependent variable. In the panel, d is the same
for each of the ve lattices. Therefore all the lattices have the same number
of dependent variables per unit area and the computation time required for
an integration on each of the lattices will about the same. However, the
properties of the solutions obtained will dier because of the eect of the
space arrangement of variables.
Using the subscripts shown in the gure, we dene the centred space-
dierencing operator by
(
x
a)
i,j
1
d

(a
i +
1
2
,j
, a
i
1
2
,j
),
where d

is the distance separating the points between which the nite-

dierence is taken. Thus, for lattices (A) to (D), d

= d, whereas for lattice

(E), d

2d. We dene also an average taken over the two points by

(a
x
)
i,j
=
1
2
(a
i+
1
2
,j
i
i
1
2
,j
).
The analogous quantities (
y
a)
i,j
and (a
y
)
i,j
are dened similarly, but with
respect to the y axis. Finally
(a
x
)
i,j
= (a
x
)
y
i,j
.
For each of the ve lattices we use the simplest centred approximations
for the space derivatives and Coriolis terms in (4.18) and obtain the ve
equation systems:
u
t
= g
x
h
x
+ fv,
v
t
= g
y
h
y
fu,
h
t
= H
_

x
u
x
+
y
v
y
_
, (4.19)
u
t
= g
x
h
y
+ fv,
v
t
= g
y
h
x
fu,
h
t
= H
_

x
u
y
+
y
v
x
_
, (4.20)
97
Figure 4.5: Five types of lattice considered for the nite-dierence solution
of Eqs. (4.1) - (4.2). Note that the grid-spacing in (E) is smaller than that
in (A) - (D) by a factor

2, because d is the same.

u
t
= g
x
h + fv
xy
,
v
t
= g
y
h fu
xy
,
h
t
= H (
x
u +
y
v) , (4.21)
u
t
= g
x
h
xy
+ fv
xy
,
v
t
= g
y
h
xy
fu
xy
,
h
t
= H
_

x
u
xy
+
y
v
xy
_
, (4.22)
u
t
= g
x
h + fv,
v
t
= g
y
h fu,
h
t
= H (
x
u +
y
v) . (4.23)
We analyze rst a one-dimensional case, in which the variables u, v and
h do not vary with y. Thus we have
u, v, h = u, v, h(x, t).
98
The system (4.18) then reduces to
u
t
= g
h
x
+ fv,
v
t
= fu,
h
t
= H
u
x
. (4.24)
Substituting the wave solutions (4.13) we obtain the frequency equation
which can be written as
_

f
_
2
= 1 +
gH
f
2
k
2
. (4.25)
Thus, as the radius of deformation
=
_
gH/f,
is never equal to zero, the frequency of the inertia-gravity waves is a mono-
tonically increasing function of k. Therefore, the group velocity /k is
never equal to zero. This is very important for the geostrophic adjustment
process, as it precludes a local accumulation of wave energy.
We now look at the eect of the nite dierencing in space in this case.
As the variables are assumed not to depend on y, the systems (4.19) - (4.23)
reduce to
u
t
= g
x
h
x
+ fv,
v
t
= fu,
h
t
= H
x
u
x
, (4.26)
u
t
= g
x
h + fv,
v
t
= fu,
h
t
= H
x
u, (4.27)
u
t
= g
x
h + fv
x
,
v
t
= fu
x
,

t
= H
x
u, (4.28)
99
u
t
= g
x
h
x
+ fv
x
,
v
t
= fu
x
,

t
= H
x
u
x
, (4.29)
u
t
= g
x
h + fv,
v
t
= fu,
h
t
= H
x
u. (4.30)
Substitution of wave solutions into these systems gives the frequency
equations
_

f
_
2
= 1 +
_

d
_
2
sin
2
kd, (4.31)
_

f
_
2
= 1 + 4
_

d
_
2
sin
2
kd
2
, (4.32)
_

f
_
2
= cos
2
kd
2
+ 4
_

d
_
2
sin
2
kd
2
, (4.33)
_

f
_
2
= cos
2
kd
2
+
_

d
_
2
sin
2
kd, (4.34)
_

f
_
2
= 1 + 2
_

d
_
2
sin
2
kd

2
. (4.35)
The non-dimensional frequency /f is now seen to depend on two parame-
ters, kd and /d.
We shall analyze the dispersion properties revealed by the expressions for
each of the ve lattices. The wave length of the shortest resolvable wave
along the x-axis is 2d for lattices (A) through (D), and for the lattice (E).
Thus, we have to consider the range 0 < kd

2 for lattices (A) through

(D), and the range 0 < kd

2 for the lattice (E).

Lattice (A): The frequency reaches a maximum at kd = /2. Thus,
the group velocity is zero for k equal to /(2d). If inertia-gravity waves of
approximately that wave number are excited near a point inside the com-
putational region, for example by nonlinear eects or by forcing through
heating or ground topography, the wave energy stays near the point. Beyond
this maximum value, for /2 < kd < , the frequency decreases as the wave
100
number increases. Thus, for these waves the group velocity has the wrong
sign. Finally, the two-grid-interval wave with kd = behaves like a pure
inertia oscillation, and its group velocity is again zero.
Lattice (B): The frequency increases monotonically throughout the range
0 < kd < . However, it reaches a maximum at the end of the range, so that
the group velocity is zero for the two-grid-interval wave with kd = .
Lattice (C): The frequency increases monotonically with kd if /d >
1
/
2
and decreases monotonically with kd if /d <
1
/
2
. It reaches an extreme
value again at kd = , associated with a zero group velocity. For /d =
1
/
2
the group velocity is equal to zero for all k.
Lattice (D): The frequency reaches a maximum at (/d)2coskd =
1
/
4
.
The two-grid-interval wave at kd = is stationary.
Lattice (E): The frequency reaches a maximum at kd = /

2. The short-
est resolvable wave with kd =

2 behaves like a pure inertia oscillation,

and its group velocity is again zero.
A summary of these results is shown in Fig. 3.2. It shows the functions
||/f, in the case /d = 2.
The gure vividly illustrates the inadequacy of the lattices (D) and (A).
The phase speed and dispersion properties of the remaining three lattices
are much better: however, zero group velocities occur with every lattice.
Thus, with any lattice there will be diculties in the geostrophic adjustment
process.
The dierence between the results for lattices (B) and (E) is interesting
because these two lattices can be obtained from one another by a rotation
through an angle of /4. If we consider the one-dimensional case in which
the dependent variables are constant along the lines y = x + c, we obtain
results for these two lattices that are exactly opposite to those in Fig. 4.6.
In general, we dene the coordinate system x

, y

by rotating the system

x, y in the positive direction through an angle of /4, and then, using the
relations change from variables u, v, h to new dependent variables u

, v

,
h. We nd that this transforms the system (4.20) into (4.23). Thus, the
dispersion properties of the lattices (B) and (E) can be considered equivalent.
A inertia-gravity wave in one of these lattices has phase speed and dispersion
properties identical to those of the same wave with its front rotated through
the angle of /4 in the other lattice.
u

2
2
(u + v), v

2
2
(u + v),
Obviously, we should consider also the two-dimensional case. The values
of ||/f that are obtained in the two-dimensional case for the true solution
and those using lattices (B) and (C) are shown in Fig. 4.7 with /d

= 2.
101
Figure 4.6: The functions ||/f given by (4.25) and (4.31)-(4.35), with /d =
2.
The diagram for lattice (E) can be obtained by a counter-clockwise ro-
tation of the (B) lattice diagram. The diagram for lattice (C) in the two-
dimensional case is seen to be a much better approximation to the exact
solution than the (B) or (E) lattice diagram. In the (B) lattice diagram the
dot-dashed line shows the maximum ||/f for a given ratio l/k; note that
there is no such line in the (C) lattice diagram and the exact solution. Such
a maximum occurs at only two corner points of the (C) lattice diagram.
Thus, with the (C) lattice, no waves have a group velocity with the wrong
sign. The situation, though, does depend on the parameter /d. Within a
stratied atmosphere the radius of deformation depends on the stability; if
the stability is so weak as to make l/d of the order 1 or less, the (C) lattice
loses the advantages shown in Fig. 4.7. However, for typical grid sizes used
in atmospheric models this is not the case and therefore Arakawa (Arakawa
and Lamb, 1976) concludes that the lattice (C) is the best lattice to simulate
the geostrophic adjustment process. Accordingly, it was used in the general
circulation model at the University of California at Los Angeles, and also in
the British operational model. The (B) or (E) lattices have a problem with
false low frequencies of the shortest waves. The two-grid-interval wave, that
was stationary as a pure gravity wave, now behaves like a pure inertia oscil-
lation. The diculty arises from decoupling of the gravity wave solutions on
the two complementary (C) type subgrids. Methods of dealing with this will
be discussed later.
102
Figure 4.7: The functions ||/f, for the true solution and for solutions of
systems (4.20) and (4.21), with /d = 2.
4.4 The normal form of the gravity wave equa-
tions
We consider the one-dimensional equations
u
t
+ c
u
x
+ g
h
x
= 0
h
t
+ c

x
+ H
u
x
= 0. (4.36)
We multiply the second of these equations by an arbitrary parameter , and
add the result to the rst equation. We obtain

t
(u + h) + (c + H)
u
x
+ (g + c)
h
x
= 0. (4.37)
We wish to choose so that
g + c
c + H
= (4.38)
to obtain an equation with only one dependent variable, u + h. The two
solutions are
=
_
g
H
. (4.39)
Substituting these into (4.37) we obtain
_

t
+
_
c +
_
gH
_

x
_ _
u +
_
g
H
h
_
= 0
103
_

t
+
_
c
_
gH
_

x
_
_
u
_
g

H
H
h
_
= 0. (4.40)
This is the normal form of (4.36). It shows that (4.36) is equivalent to a
system of two advection equations. The quantity u +
_
g/H

h is seen to
be advected at a velocity c +

gH in the direction of the x axis, while, at

the same time, the quantity is advected in the same direction at a velocity
c

gH. Thus, if the leapfrog scheme is used for the time dierencing and
we choose a grid that carries both u and h at every grid point, we obtain the
Courant-Friedrichs-Levy stability criterion
_
c +
_
gH
_
t
x
1. (4.41)
The advection velocity in the atmosphere is normally about an order of mag-
nitude less than the phase speed of external gravity waves, giving the ap-
proximate stability requirement
_
gH
t
x
< 1. (4.42)
When using the three-dimensional primitive equations, external gravity waves
are normally eliminated by permitting no vertical velocity at the upper
boundary. The highest phase speed admitted by the system is then that
of the Lamb waves, which for an isothermal atmosphere is

_
f
k
_
2
+
c
p
c
v
RT.
If we neglect the rst term and recall that the scale height of an isothermal
atmosphere is H

= RT/g, we see that the phase speed of the Lamb waves is

of the same order of magnitude as that of external gravity waves (300m/s).
The CFL stability condition thus means that a large amount of computer
time is required because of the presence of these fast moving waves. For this
reason implicit time dierencing schemes were developed, so that the choice
of the time step can be based solely on accuracy and not on stability.
4.5 The leapfrog scheme and the Eliassen
grid
We consider the leapfrog scheme with centred space dierencing applied to
the two-dimensional system of gravity wave equations
104
u
t
+ g
h
x
= 0,
v
t
+ g
h
y
= 0,
h
t
+ H v = 0. (4.43)
In nite-dierence notation
u
n+1
= u
n1
2gt
x
h
n
, v
n+1
= v
n1
2gt
y
h
n
h
n+1
= h
n1
2Ht(
x
u +
y
v)
n
. (4.44)
Substituting the wave solutions
u
n
= Re
_

n
ue
i(kx+ly)

,
v
n
= Re
_

n
v e
i(kx+ly)

,
h
n
= Re
_

he
i(kx+ly)
_
, (4.45)
we obtain the homogeneous system
(
2
1) u + i2

2gsin(X)

h = 0
(
2
1) v + i2

2gsin(Y )

h =
0i2

2H(sin (X) u + sin(Y ) v) + (

2
1)

h = 0 (4.46)
with X = kd

, Y = ld

, = t/(d

). Note that = t/d when lattice (E)

is chosen. The requirement that the determinant of (4.46) is equal to zero
gives six solutions for . Two of these are
= 1 (4.47)
and
= 1. (4.48)
The remaining four are given by

2
= 1 4A2
_
2A(2A1) (4.49)
where
A = gH
2
(sin
2
X + sin
2
Y ).
105
We can now analyze the solution (4.45) associated with the values found
for . Equation (4.47) gives a neutral and stationary solution. If either
sin X or sin Y is non-zero in this case, then according to (4.46) we have

h = 0, and the solution represents a physically acceptable translatory motion.

However, if sin X and sin Y are both equal to zero, the amplitudes of all three
dependent variables can take arbitrary values. In addition to the physically
acceptable solution where all the dependent variables are constant (k = l =
0), there is a solution with one or both of the wave numbers k and l equal
to /d

. This is the two-grid-interval wave. Again it appears as a false

computational solution; since it is stationary and it is not aected by the
introduction of time dierencing.
The second value = 1, represents a false computational mode in time,
with a period of 2Dt. This computational mode results from using a three
time-level scheme. To prove stability of the scheme the behaviour of the
remaining solutions given by (4.49) has to be investigated. They will all be
neutral for 2A < 1. To obtain the condition in the form Bt 1 we write

2A 1.
Since this has to be satised for all the admissible waves, we nd that the
CFL criterion in the two-dimensional case is now
2
_
gH 1 (4.50)
or
_
2gH
t
x
1. (4.51)
This is in agreement with the previous results (Exercise 4.1). The nondimen-
sional constant on the left is sometimes called the Courant number.
With solutions like (4.45), the frequency is given by
= || e
it
.
Thus, the expressions obtained for can be used to calculate the relative
phase speed using the relation (Exercise 4.x)
c

gH
=
1
t

gH

k
2
+ l
2
arc tan

im

re
. (4.52)
For a more explicit illustration of the eect of time dierencing, we can
perform a series expansion of (4.52). One obtains for

2A < 1/

2,
106
c

gH
=

sin
2
X + sin
2
Y
X
2
+ Y
2
(1 +
1
3
A+
3
10
A
2
+ . . . .
The factor multiplying the series in parenthesis describes the eect of space
dierencing as given by (4.17). The acceleration resulting from the leapfrog
time dierencing is benecial, as it reduces the phase error due to space
dierencing. The values of the relative phase speed show very little dierence
compared with Fig. 4.4. The relative phase speed is still poor.
4.6 The Eliassen grid
Eliassen (1956) pointed out the advantages of a space-time grid staggered
both in space and time as shown in Fig. 4.8. Such a grid is convenient
for the leapfrog scheme associated with centred space-dierencing which is
suitable for the linearized system:
u
t
+ g
h
x
fv = 0,
v
t
+ g
h
x
+ fu0,
h
t
+ H
_
u
x
+
v
y
_
= 0. (4.53)
The second grid can be obtained by shifting the grid a distance

2d

along the line y = x. This grid is called Eliassen grid.

4.7 Economical explicit schemes
The fact that we are now solving two equations, the equation of motion and
the continuity equation, suggests new ways of constructing time dierencing
schemes. The time step for gravity waves imposed by the CFL stability cri-
terion is generally much less than that required for an accurate integration of
the slower quasi-geostrophic motions. With these short timesteps, the errors
due to space dierencing are much greater than those due to time dier-
encing. Two explicit schemes that are more economical than the standard
leapfrog scheme will be given here. They both achieve economy by using a
dierent integration procedure for the height gradient terms of the equation
of motion and for the divergence term of the continuity equation. For brevity,
we call these terms the gravity wave terms in the governing equations.
107
Figure 4.8: Eliassen grid.
4.7.1 The forward-backward scheme
This scheme is obtained by rst integrating the gravity wave terms of either
the equation of motion or of the continuity equation forward, and then those
of the other equation backward in time. With centred space dierencing
(4.43) is approximated by
u
n+1
= u
n
gt
x
h
n
, vn + 1 = vn gt
y
h
n
,
h
n+1
= h
n
Ht(
x
u +
y
v)
n+1
, (4.54)
or by an analogous system in which the order of integration is reversed.
Substituting wave solution as before, we nd three solutions for . One of
these,
= 1, (4.55)
gives again a neutral and stationary solution. The remaining two are
= 1 A
_
A(A2) (4.56)
where the quantity A is dened as in the preceding section. Solutions (4.55)
and (4.56) are obtained for both versions of the scheme (Exercise .4.x.),
108
that is, no matter which of the two equations - the equation of motion or
the continuity equation - is rst integrated forward. Examination of the
amplication factors given by (4.56) shows that the scheme is stable and
neutral for A 2, that is, for

A 2.
To satisfy this for all admissible waves, we must have
2
_
gH 2. (4.57)
Thus the forward-backward scheme is stable and neutral with time steps
twice those allowed by the CFL criterion for the leapfrog scheme. The am-
plication factors of the forward-backward and of the leapfrog scheme are
equal within their regions of stability. We now compare their eect on the
phase speed by comparing the expression (4.56) for the forward-backward
scheme, with (4.49), for the leapfrog scheme. The right-hand side of (4.56),
with A replaced by 4A, is equal to the right-hand side of (4.49). Because
of the denition of A, this means that for the forward-backward scheme
is identical to
2
for the leapfrog scheme when the time steps used for the
forward-backward scheme are twice as long as those for the leapfrog scheme!
Thus, the forward-backward scheme gives the same result using only half the
computation time needed for the leapfrog scheme. In addition, as a two-
level scheme, it has no computational mode in time. To understand this
advantage over the leapfrog scheme we compare the nite-dierence ana-
logues that these two schemes give for the wave equation, since the system
of gravity wave equations is equivalent to a single wave equation. Consider
the one-dimensional version of this system:
u
t
+ g
h
x
= 0,
h
t
+ H
u
x
= 0 (4.58)
Eliminating one of the variables u, h we obtain a wave equation

2
h
t
2
gH

2
h
x
2
= 0. (4.59)
We can perform the same elimination for each of the nite-dierence schemes.
The forward-backward and space-centred approximation to (4.58) is
u
n+1
j
u
n
j
t
+ g
h
n
j+1
h
n
j1
2x
= 0,
h
n+1
j
h
n
j
t
+ H
u
n+1
j+1
u
n+1
j1
2x
= 0. (4.60)
We now subtract from the second of these equations an analogous equation
for the time level n1 instead of n, divide the resulting equation by t, and
109
nally, eliminate all u values from it using the rst of Eqs. (4.60), written
for space points j + 1 and j 1 instead of j. We obtain
h
n+1
j
2h
n
j
+ h
n1
j
(t)
2
gH
h
n
j+2
2h
n
j
+ h
n
j2
(2x)
2
= 0, (4.61)
This is a nite-dierence analogue of the wave equation (4.59). Note that
although each of the two equations (4.60) is only of rst order of accuracy
in time, the wave equation analogue equivalent to (4.60) is seen to be of the
second order of accuracy. If we use a leapfrog and space-centered approxima-
tion to (4.58), and follow an elimination procedure like that used in deriving
(4.61), we obtain
h
n+1
j
2h
n1
j
+ h
n3
j
(2t)
2
gH
h
n1
j+2
2h
n1
j
+ h
n1
j2
(2x)
2
= 0, (4.62)
This is also an analogue to the wave equation (4.59) of second-order accuracy.
However in (4.61) the second time derivative was approximated using values
at three consecutive time levels; in (4.62) it is approximated by values at
every second time level only, that is, at time intervals 2t. Thus, while
the time step required for linear stability with the leapfrog scheme was half
that with the forward-backward scheme (4.62) shows that we can omit the
variables at every second time step, and thus achieve the same computation
time as using the forward-backward scheme with double the time step. This
method was discussed in the previous section for the two-dimensional case,
it is the Eliassen grid. Thus, comparing (4.61) and (4.62) shows that the
economy accomplished by the forward-backward scheme is equivalent to that
accomplished with leapfrog time dierencing by the Eliassen grid. Both of
these methods avoid calculating the false time computational mode, and thus
save half of the computation time with no eect on the physical mode of the
solution. A disadvantage of the forward-backward scheme is that it is not
possible to use the leapfrog scheme for the advection terms. However the
second order accurate Adams-Bashforth scheme can be used for these terms.
4.8 Implicit and semi-implicit schemes
The time step permitted by the economical explicit schemes, twice that pre-
scribed by the CFL criterion, is still considerably shorter than that required
for accurate integration of the quasi-geostrophic motions. Thus we consider
implicit schemes which are stable for any choice of time step. We shall con-
sider here only the simplest of the implicit schemes, the trapezoidal rule
110
applied to the system (4.43) of pure gravity waves. For brevity it will simply
be called the implicit scheme.
Figure 4.9: Relative speed of gravity waves, with implicit time and centred
space dierencing, and a Courant number equal to 5.
4.9 The implicit scheme (trapezoidal rule)
We consider the nite-dierence scheme
u
n+1
= u
n
gt
1
2
(
x
h
n
+
x
h
n+1
),
v
n+1
= v
n
gt
1
2
(
y
h
n
+
y
h
n+1
),
h
n+1
= h
n
Ht[(
x
u +
y
v)
n
+ (
x
u +
y
v)
n+1
]. (4.63)
Substituting the wave solutions (4.45) we nd three solutions for . One
of these,
= 1, (4.64)
is again that associated with neutral a stationary solution. The remaining
two are
=
1
1 +
1
2
A
_
1
1
2
Ai

2A
_
(4.65)
Examination of (4.65) shows that it always gives amplication factors satis-
fying
111
|| = 1, (4.66)
and so the scheme is unconditionally stable and neutral. Using (4.65) and
(4.52) we nd for the relative phase speed of the nonstationary solutions,
c

gH
=
1

_
2gH(X
2
+ y
2
)
arctan
_

2A
1
1
2
A
_
. (4.67)
The numerical values given by (4.67) for the physical mode with 2

gH = 5
are shown in Fig. 4.9. The wave number region is the same as in the earlier
diagram, Fig. 4.4. Comparing the isolines of the present gure with those
of Fig. 4.4, where the eect of the space dierencing alone was considered,
shows that the eect of time dierencing on phase speed is now not negligible.
Implicit time dierencing is seen to result in a considerable retardation of
gravity waves of the same order of the magnitude as that due to centred
space dierencing. The solution of the implicit scheme for new variables
u
n+1
, v
n+1
is no longer trivial.
4.10 The semi-implicit method of Kwizak and
Robert
There is no advantage in using an implicit method for advection, Coriolis,
and other terms of governing equations in atmospheric models. They are as-
sociated with slower phase speeds, and should not require excessively small
time steps for linear stability when calculated explicitly. Since the trapezoidal
implicit scheme is a two-level scheme like the forward-backward scheme, it
is convenient to use Adams-Bashforth scheme for this purpose. Kwizak and
Robert (1971) chose, however, to use the leapfrog scheme. The usual pro-
cedure used for solving the semi-implicit dierence system for variables at
time level n + 1 will be illustrated for the shallow water equations. These
equations can be written in a compact form
u
t
= g
h
x
+ A
u
v
t
= g
h
y
+ A
v
,
h
t
= H v + A
h
, (4.68)
where A
u
, A
v
and A
h
denote the terms that were omitted in the system (4.43)
describing the propagation of pure gravity waves. These additional terms,
112
and implicit dierencing over a time interval 2t for the gravity wave terms
and centred space dierencing (4.68) is replaced by
u
n+1
= u
n1
gt(
x
h
n1
+
x
h
n+1
) + 2tA
n
u
,
v
n+1
= v
n1
gt(
y
h
n1
+
y
h
n+1
) + 2tA
n
v
,
h
n+1
= h
n1
Ht[(
x
u +
y
v)
n1
+ (
x
u +
y
v)
n+1
] (4.69)
+ 2tA
n
h
. (4.70)
We now apply the operator
x
to the rst and
y
to the second of these
equations, respectively and add the results. We introduce the notation

xx
h =
x
(
x
h)and
yy
h =
y
(
y
h).
We obtain
(
x
u +
y
v)
n+1
= (
x
u +
y
v)
n1
gt[(
xx
+

yy
)h
n1
+ (
xx
+
yy
)h
n+1
] + 2t (
x
A
u
+
y
A
v
)
n
Substituting the right-hand side into the third of Eqs. (4.69), and dening
the nite-dierence Laplacian by

h = (
xx
+
yy
)h
we nd that
h
n+1
= h
n1
2Ht(
x
u +
y
v)
n1
+ gH(t)
2
(
2

h
n1
+ h
n+1
)
+ 2
2

h
n+1
+ 2t[A
h
Ht(
x
A
u
+
y
A
v
)]
n
Using, in addition the denitions
F
n1
:= h
n1
2Ht(
x
u +
y
v)
n1
+ gH(t)
2

h
n1
,
G
n
:= 2t[Ah Ht(
x
A
u
+
y
A
v
)]
n
this can be written as
h
n+1
gH(Dt)
2
h
n+1
= F
n1
+ G
n
. (4.71)
113
The terms have been arranged to show that at time level n the right hand
side is known at all space grid points. Once the equation has been solved for
the values h
n+1
, u
n+1
and v
n+1
can be obtained directly from the rst and
the second of Eqs. (4.69). The algebraic system (4.71) is an approximation
to an elliptic equation

2
h + ah + b(x, y) = 0. (4.72)
4.11 The splitting or Marchuk method
Since there are dierent factors in a complex system of hydrodynamic equa-
tions for weather prediction, we will normally wish to use dierent schemes
for terms associated with them. Thus considering the linearized system with
advection and gravity wave terms,
u
t
+ c
u
x
+ g
h
x
= 0
h
t
+ c
h
x
+ H
u
x
= 0 (4.73)
we might wish to use one scheme for the advection terms, and another for the
gravity wave terms - in much the same way as was done for the semi-implicit
scheme. In such a situation, even though both of the schemes to be used
are stable considered one at a time, we cannot be certain that the scheme
obtained as a combination of the two will also be stable. These problems
can be avoided using the splitting method. The idea of this method is to
construct
schemes for a complex system of equations so that within each time step
this system is split into a number of simpler subsystems, which are then
solved consecutively one at a time. In case of (4.73), within a given time
step, we could rst solve the system of advection equations
u
t
+ c
u
x
= 0
h
t
+ c
h
x
= 0. (4.74)
Denote the provisional values u
n+1
, h
n+1
, obtained in this way by u

, h

.
Use these values at the beginning of the time step for solving the remaining
subsystem
114
u
t
+ g
h
x
= 0
h
t
+ H
u
x
= 0. (4.75)
The values u
n+1
, h
n+1
, obtained after solving this other subsystem, are now
taken as actual approximate values of these variables at the level n +1. The
procedure is repeated in each following time step. A solution obtained by
the splitting method will represent a consistent approximation to the true
solution. This can be proved easily for a particular choice of schemes for
solving the subsystems. The approximate values of the dependent variables
then have to approach the true values as the time step approaches zero.
Stability of the splitting method To study the stability of schemes constructed
by the splitting method, we consider the example above. Denote by la and lb
the values of l of the schemes chosen for the numerical solution of subsystems
(4.74) and (4.75), respectively. Then we have
u

= Re(
a

n
ue
ikx
), h

= Re(
a

he
ikx
) (4.76)
and
u
n+1
= Re(
b

n
ue
ikx
), h
n+1
= Re(
b
,
a

he
ikx
). (4.77)
Therefore, we nd,
=
b

a
, (4.78)
and
|| = |
b
| |
a
|. (4.79)
Thus, if both of the schemes chosen for the solution of subsystems (4.74)
and (4.76) - (4.79) are stable, the combined scheme constructed by the split-
ting method will also be stable. This conclusion can be generalized for an
arbitrary system of equations and number of subsystems.
When applying the splitting method, we do not necessarily have to use
equal time steps for each of the subsystems. This may be the main advantage
of the splitting method: we can chose a relatively long time step for the
subsystem governing a slow process, advection in the present example, and
then use a number of smaller steps to calculate the faster process. Signicant
economies can be accomplished in this way. A disadvantage of the splitting
method is that calculation of the eects of dierent physical factors one at
115
a time usually leads to an increase in the truncation error. The splitting
method was rst used in atmospheric models by Marchuk (1967); thus in
meteorology it is known as the Marchuk method.
4.12 Two-grid-interval noise suppression
A number of methods have been used to cope with two-grid-interval noise.
In many models dissipative schemes are used to give maximum damping for
the two-grid-interval wave, or lateral diusion is added with relatively large
diusion coecients. The appearance of two-grid interval noise is thereby
suppressed. However, instead of attacking the consequences of inadequacies
in a simulation of a physical process, it is generally better to look for a
method that would achieve a physical correct simulation of that process,
and thus eliminate the cause of the diculty. Mesinger (1973) showed how
two-grid-interval wave noise could be prevented in some cases even by using
centred dierencing. We consider the system of linearized equations (4.43).
Mesinger proposed a grid in which a height perturbation at a single grid
point is propagated by gravity waves to all the other height grid points.
Therefore there can be no grid-splitting and two-grid-interval noise in the
height eld. Since a velocity perturbation can propagate as a gravity wave
only by exciting height perturbations, the procedure will prevent false two-
grid-interval noise in all the variables. We shall illustrate this procedure
using the implicit scheme (4.63). The velocity components at regular veloc-
ity points are computed in the way as before, so the rst two equations of
that system remain unchanged. To calculate the velocity divergence in the
continuity equation we dene auxiliary velocity points midway between the
neighbouring height points, as shown by the circled numbers 5, 6, 7 and 8 in
Fig. 4.10.
Using the system x

, y

shown in this gure, components u

are needed
at points 5 and 7, and components v

at points 6 and 8. At the beginning of

the time step t these components are obtained by space-averaging, that is
u
n
=

2
2
_
u
y

+ v
y

_
n
, v

2
2
_
u
x

+ v
x

_
n
(4.80)
An overbar denotes a two-point average taken along the direction indicated
following the bar sign. Acceleration contributions are added to these initial
values to obtain values at the end of the time step
u
n+1
= u
n
gt
1
2
_

x
h
n
+
x
h
n+1
_
,
v

= v
n
gt
1
2
_

y
h
n
+
y
h
n+1
_
. (4.81)
116
Figure 4.10: Stencil used to denote the height and velocity grid point values
surrounding a height point.
The velocity divergence in the continuity equation can now be approximated
by
1
2
(
x
u +
y
v) +
1
2
(
x
u

+
y
v

) , (4.82)
giving weight to all eight directions of the lattice. In this way the implicit
approximation to the continuity equation may be obtained as
h
n+1
= h
n
Ht (
x
u +
y
v)
n
+
1
4
gH (t)
2
_

h
n
+
2

h
n+1
_
. (4.83)
Here the velocity components at level n + 1 have already been eliminated
using the rst two of Equations (4.63), and

=
1
4d
2
(h
1
+ h
2
+ h
3
+ h
4
+ 2(h
5
+ h
6
+ h
7
+ h
8
) 12h
0
). (4.84)
This is again a nite-dierence approximation to
2
h, but now it is calculated
using the height values of nine neighbouring points. Comparing this scheme
117
with the standard implicit scheme (4.63), the only modication is that this
nine-point Laplacian has replaced the ve-point Lapalcian in the continuity
equation. This allows propagation of gravity waves between all height points
of the grid, thus admitting no false space noise in the height eld. The mod-
ication has no eect on the unconditional stability of the implicit scheme;
however, instead of being neutral for all waves, the scheme now damps shorter
waves to some extent. The modied scheme has a smaller truncation error
than the unmodied scheme. It is important to be aware that this method
is not attempting to improve the calculation of short gravity waves of wave
lengths close to two grid intervals. At this scale the nite-dierence repre-
sentation is very poor, and signicant improvements in accuracy can hardly
be expected. The problem is that gravity waves, with longer wave lengths
can propagate independently on individual (C) type subgrids, and thus er-
roneously appear to have wave lengths close to two grid intervals. Thus we
are confronted with a kind of aliasing error. The proposed method enables
these waves to appear with wave length close to their physical value instead
in the noise region with wave lengths close to two grid intervals.
Mesinger denotes the application of the grid structure in Fig. 4.10 as
cceleration-modied versions of a particular scheme. The phase speed of
gravity waves as a function of wave numbers k and l is shown in Fig. 4.11 for
the implicit scheme, the Heun scheme and the leapfrog scheme. The shading
close to the right corner corresponds with a region of zero phase speed. Since
the solutions in these regions are damped, there is no danger that it can be
excited to a signicant amplitude.
4.13 Time noise and time ltering
In addition to the appearance of spurious short-wave noise in space, spurious
short-wave noise in time, that is high frequency noise can appear in numer-
ical models. As a result of initial conditions which are interpolated from
observations, the numerical forecasts will contain spurious gravity waves of
unrealistically large amplitudes. Experience shows that the noise generated
by assimilation of the observed data typically dies out to an acceptable level
in about 24 hours of simulated time due to geostrophic adjustment. However
it may be desirable to accelerate this adjustment by appropriate numerical
techniques. The Matsuno schemes can be used for this purpose.
Another method that can be used to increase the damping of high fre-
quency noise in atmospheric models is time ltering originally proposed by
Robert(1966). To apply this at least three consecutive values of the function
to be ltered are needed. It suces to consider one function only, which we
118
Figure 4.11: Relative phase speed of gravity waves as a function of wave
numbers k and l. a) for the implicit scheme. b) for the acceleration-
modied implicit scheme. c) the Heun-scheme. d) the acceleration-modied
Heun-scheme. e) the leapfrog-scheme. f) the acceleration-modied leapfrog-
scheme.
assume to be a solution of the oscillation equation. Thus we consider the
function
U(t) = U(0)e
it
, (4.85)
where the values U(t t), U(t) and U(t + t) are known. We shall rst
examine the eect of changing only the middle of these three values using
the relation
(t) = U(t) +
1
2
S[U(t t) 2U(t) + U(t + t)] (4.86)
known as the centred lter. The overbar now denotes the ltered value of
a function, and S is the lter parameter. The expression within the square
bracket in (4.86) is proportional to the simplest approximation to the second
derivative in time; thus, for suciently small positive values of S application
119
of the lter (4.86) will decrease the curvature in a graph of the three values
of U(t). For quantitative analysis of the eect of the lter we dene
U(t) = R U(t), (4.87)
where the complex factor R is called the response of the lter. When this is
substituted in into (4.85) and we use (4.86), we obtain
R = 1 S(1 cos t). (4.88)
It is convenient to dene
R = |R|e
i
.
We can then say that the phase change resulting from the centred lter
is zero, and that within the CFL stability criterion and for small positive
values of S the amplitude factor |R| exerts a damping eect increasing with
increasing frequencies.
When, however, a lter is continually applied during a numerical integra-
tion, the value U(t t) has already been changed prior to changing U(t).
It is then appropriate to consider the lter
(t) = U(t) + S[(t t) 2U(t) + U(t + t)]. (4.89)
Asselin (1972) calls this the basic time lter. A procedure like the one used
in deriving (4.87) now gives
R =
(2 S)
2
+ 2S
2
(1 cos t)
(2 S)
2
+ 4S(1 cos t)
e
it
. (4.90)
Thus, there is now a phase change that is dierent from zero; however it is
small for small values of t. The amplitude factor is not much dierent
from that of the centred lter for small values of S.
Using an analogous approach one can analyze the eect of smoothing and
ltering in space.
Extension of the centred lter to two space dimensions may be accom-
plished in two ways. First, one may smooth in each dimension, independently
of the other dimension. It can be shown that the nal result is independent
of the dimension in which one rst smooths, and also independent of the
order in which one applies the smoothing elements.
120
4.13.1 Example of ltering
We consider the eect of the centred lter applied to a two-dimensional array.
In Fig. 4.5a the example distribution is displayed. It consists of a high of the
product of single waves sin(x) sin(x) and a low, which has a Gaussian height
distribution. The maximum value is 10 and the minimum value is 10. The
Figure 4.12: Example of space ltering with the centred lter applied to a
two-dimensional array. a) undisturbed initial (ideal) distribution. Contour
interval in all panels 1 height unit, solid contours represent positive values,
dashed contours negative values. The extrema correspond with 10 and 10
height units. b) The initial distribution is disturbed by random numbers
with amplitude 2.5 height units. c) Height distribution after one iteration
with S = 0.5. The maximum error (maxerr) and mean error (meanerr) are
displayed on top of this panel and the following panel. d) As in c) but for 3
iterations.
number of intervals is 40 by 20. This distribution is disturbed with random
noise with a maximum amplitude of 2.5 at all inner grid points, shown in
Fig. 4.5b. The following panels display the eect of the centred lter applied
rst in x-direction to the interior of the whole array
121
h
x
(x, y) = h(x, y) +
1
2
S [h(x + x, y) 2h(x, y) + h(x x, y)] ,
and then in y-direction to the interior of the whole array
h
y
(x, y) = h(x, y) +
1
2
S [h(x, y + y) 2h(x, y) + h(x, y y)] .
Once the whole array is processed, the process is repeated, denoted as iter-
ations. Fig. 4.5 shows that the original pattern can be retrieved this way.
Best results are obtained with a few iterations. The lter parameter S and
the maximum and the mean error, with respect to the original eld Fig. 4.5
are displayed in a line above the panels.
Exercises
4.1 Write down the splitting method for the system (4.67) with the leapfrog
method for advection and the forward-backward-scheme for the gravity
wave terms. Derive a stability criterion.
4.2 Derive the elliptic nite-dierence equation (4.84).
4.3 Show, using the acceleration modied Heun-scheme with time step 2Dt
that the equation for the height-eld can be written
h
n+1
= h
n1
2Ht(
x
u +
y
v)
n1
+ 2gH(t)
2

h
n1
4.4 Show that the application of two smoothing elements, one in each di-
mension, the two elements may be combined into a single 9-point op-
erator.
122
Chapter 5
Methods for Solving Elliptic
Equations
In this chapter we consider methods for the solution of dierence equations
arising from discretization of elliptic partial dierential equations. As a spe-
cial case of elliptic equations we consider the Helmholtz-equation

2
= . (5.1)
This arises from the use of the semi-implicit method for shallow water equa-
tions as introduced in chapter 4 or as a Poisson-Equation ( = 0). Another
example is the inversion of pressure in nonhydrostatic Boussinesq equations.
The solutions of elliptic equations are known as boundary value problems.
The boundaries may be of periodic, of Dirichlet type ( prescribed), of Neu-
mann type (normal gradient

n
prescribed) or of Robin type (also known as
mixed type, a combination of Dirichlet and Neumann type). As a paradigm
of elliptic equations we chose the Poisson equation. For the iterative methods
we present here the extension to general elliptic equations is straightforward.
5.1 The Poisson equation
The discretized form of the Poisson equation, using second-order dierences
is

xx
+
yy
= (5.2)
resulting in a ve point formula

i+1,j
2
i,j
+
i1,j
x
2
+

i,j+1
2
i,j
+
i,j1
y
2
=
i,j
(5.3)
123
where
i,j
is known.
5.2 General considerations
In a rectangular domain where Max(i) = I and Max(j) = J, equation (5.3)
and the boundary equations give a system of N = (I 2) (J 2) simulta-
neous linear algebraic equations. Except for the source or non-homogeneous
term,
i,j
, it is the same block-tridiagonal system which results from a fully
implicit formulation of the two-dimensional diusion equation and the eco-
nomical tridiagonal algorithm is not applicable. The most elementary meth-
ods of solving such a system are Cramers rule and the various forms of
Gaussian elimination. For the problems of interest, N is a large number, and
these methods are inadequate. Cramers rule involves quite an unbelievable
number of operations - approximately (N + 1)! multiplications. Even if the
time were available ,the accuracy would be destroyed by round-o error. The
multiplications in Gaussian methods vary as N
3
, and the accuracy may be
expected to deteriorate for N much greater than 50 depending on the details
of the method and the computer word length.
In recent years, highly ecient direct methods have been developed. Dorr
(1970) reviews the block methods, cyclic reduction methods, tensor product
methods, the Fourier series methods, and a few others. All these methods
have one ore more of the following disadvantages: limited to rectangular do-
mains like L or T-shapes; limited to boundary conditions; large storage
requirements; not adaptable to non-Cartesian coordinates; eld size limited
(restricting the magnitude of I or J) due to accumulation of round-o error;
eld specication limited (e.g., I or J limited to the form 2
k
, k integer);
elaborate preliminary calculations specic to the mesh required; complex to
program; dicult to understand. However, the direct methods, particularly
the Fourier-series methods, are increasing in use especially for large problems
(I and J large). There exist commonly available ecient FORTRAN pro-
gram libraries for the direct (and also iterative) solution of elliptic equations
e.g. FISHPACK and ELLPACK. By comparison, the various iterative meth-
ods are very easy to understand and to program, and are quite exible. The
iterative methods have historically been used most often in computational
uid dynamics and will undoubtedly continue to be important. The latest
developments are the (iterative) multigrid methods.
124
5.3 Richardsons Method
The solution of elliptic equations by iteration is analogous to solving a time-
dependent problem to an asymptotic steady state. Suppose we consider a
time-dependent diusion equation for with a source term, , and with a
diusion coecient of unity

t
=
2
. (5.4)
We are not interested in the physical signicance of the transient solution,
but as a steady state of this solution is approached,

t
0, the solution
approaches the desired solution for the Poisson equation
2
= . In some
cases the analogy is exact. To illustrate the equivalence, we shall proceed
to derive Richardsons iterative method for the elliptic Poisson equation from
the Euler (forward) time-step with centred space dierences time-dependent
method for the parabolic diusion equation. Applying forward in time, cen-
tred in space (FTCS) dierencing to (5.4) we obtain

k+1
i,j

k
i,j
t
= (
xx
+
yy
)
k
i,j

i,j
. (5.5)
To avoid confusion, we momentarily restrict ourselves to x = y = .
Then Eq. (5.5) gives

k+1
i,j
=
k
i,j
+
t

2
_

k
i+1,j
+
k
i1,j
+
k
i,j+1
+
k
i,j1
4
k
i,j

2

i,j

. (5.6)
We rst demonstrate that
i,j
does not aect the stability properties of Eq.
(5.6). We consider Dirichlet boundary conditions and denote the exact so-
lution of the nite-dierence equation of the Poisson Eq. (5.2) by

. The
errors e
k
i,j
in the iterative values are then
e
k
i,j
=

i,j

k
i,j
. (5.7)
Substituting this into (5.2) gives

xx

+
yy

xx
e
k

yy
e
k
= . (5.8)
Since

exactly satises (5.2), then (5.8) reduces to the Laplace equation,

xx
e
k
+
yy
e
k
= 0. (5.9)
Since the boundary values are known,
k
=

or e = 0 on all boundaries.
Then the iteration (5.6) can be written for the error e as
125
e
k+1
i,j
= e
k
i,j
+
t

2
_
e
k
i+1,j
+ e
k
i1,j
+ e
k
i,j+1
+ e
k
i,j1
4e
k
i,j

. (5.10)
So the iteration (5.6) for is equivalent to the iteration (5.10) for e, which
clearly does not depend on . The stability restriction on Eq. (5.6) is just
t
2
/4 (Exercise 5.2). Since we wish to approach the asymptotic condi-
tion as rapidly as possible, we consider taking the largest possible t =
2
/4.
Substituting into (5.6) then gives

k+1
i,j
=
k
i,j
+
1
4
_

k
i+1,j
+
k
i1,j
+
k
i,j+1
+
k
i,j1
4
k
i,j

2

i,j

. (5.11)
The terms inside and outside the brackets cancel, giving

k+1
i,j
=
1
4
_

k
i+1,j
+
k
i1,j
+
k
i,j+1
+
k
i,j1

i,j

. (5.12)
This is Richardsons method for x = y = . The method is also known
as the Jacobi method or as iteration by total steps, or method of simultaneous
displacements, since each
k+1
i,j
is calculated independent of the sequence in
(i, j) and therefore, in a sense, simultaneously. This distinction is important
to the consideration of techniques for parallel-processing computers. For the
special case of Laplaces equation (
i,j
= 0), the method simply involves
setting the new iterative value equal to the arithmetic average of its four
neighbours. Equation (5.12) is the same result that is obtained from simply
solving the (steady) elliptic Eq. (5.3) for
i,j
and evaluating this term on the
left side of the equation at (k +1) and all terms on the right-hand side at k.
Dening the mesh aspect ratio = x/y this gives

k+1
i,j
=
1
2(1 +
2
)
_

k
i+1,j
+
k
i1,j
+
2

k
i,j+1
+
2

k
i,j1
x
2

i,j

. (5.13)
This is Richardsons method for x = y. The analysis of the convergence
rate can proceed as in the analysis of stability for the vorticity equation, now
writing
e
k+1
= e
k
(5.14)
for the error equation (5.10). The highest and lowest wavelength error com-
ponents damp most slowly ,i.e. have largest |()| .
126
5.4 Liebmanns method
Equation (5.13) is a two-level equation, requiring storage of
k+1
and
k
. If
we sweep in i , j and use new values wherever available in Eq. (5.13), we
obtain

k+1
i,j
=
1
2(1 +
2
)
_

k
i+1,j
+
k+1
i1,j
+
2

k
i,j+1
+
2

k+1
i,j1
x
2

i,j

, (5.15)
which is Liebmanns method, also known as Gauss-Seidel method. This
method may be programmed with only one storage level, interpreting (5.15)
as a FORTRAN replacement statement
. . .
do j = 2, jj 1
do i = 2, ii 1
(i, j) = fac

((i+1, j)+(i1, j)+bb

(i, j+1)+bb

(i, j1)dd

(i, j))
enddo
enddo
. . .
For the most resistant high and low wavenumber error components,
(Liebmann) = [ (Richardson)]
2
. (5.16)
Asymptotically, k Liebman iterations are worth 2k Richardson iterations,
and only require half the core storage.
5.5 Southwells Residual Relaxation Method
The original name for the method was just the relaxation method, but here
we use the term residual relaxation to distinguish it from Liebmans method
and other iterative procedures, which today are sometimes called relaxation
methods. The simplest form of Southwells residual relaxation method uses
the same equation as Richardsons method (5.13) to evaluate new
k+1
i,j
. The
dierence is that Eq. (5.13) is not applied indiscriminately to all mesh points
in a sweep of the matrix. Rather the residual r
i,j
is dened by
127
r
i,j
=

i+1,j
2
i,j
+
i1,j
x
2
+

i,j+1
2
i,j
+
i,j1
y
2

i,j
. (5.17)
When r
ij
= 0, the Poisson equation (5.3) is satised, but only at the point
(i, j). Thus, |r
i,j
| is indicative of how much the present estimate for all
i,j
is in error at (i, j). One then scans the eld for the largest |r
i,j
|, and sets
this r
ij
= 0 by calculating a new
i,j
from (5.13). This in turn changes r at
all neighbouring points and the scan is repeated. This method is not used
in todays computers because the time required to scan for Max(r
ij
) and
recalculate neighbouring r

s is not suciently shorter than the time needed

to apply Eq. (5.13). Southwells method was historically important because
a more rened version which evolved suggested the extrapolated Liebmann
method, more commonly known as the successive over-relaxation method.
5.6 Successive Over-Relaxation (SOR) Method
Frankel (1950) and independently Young (1954) developed a method of ap-
plying overrelaxation to Liebmanns method in a methodical manner suited
to electronic computers. Frankel called it the extrapolated Liebmann method,
and Young called it successive over-relaxation. Adding to Eq. (5.15) and re-
grouping gives

k+1
i,j
=
k
i,j
+
1
2(1+
2
)
_

k
i+1,j
+
k+1
i1,j
+
2

k
i,j+1
+
2

k+1
i,j1
x
2

i,j
2(1 +
2
)
k
i,j

.
(5.18)
Now as a solution is approached,
k+1
i,j

k
i,j
for all (i, j) the bracketed term
becomes zero identically by Eq. (5.3) and (5.18) becomes a statement of
convergence,
k+1
i,j
=
k
i,j
. If Liebmanns method is used and the bracketed
term is set identically to zero at the point (i, j),
k+1
i,j
=
k
i,j
. That is,
the residual r
ij
= 0. In the SOR method, the bracketed term in (5.18) is
multiplied by a relaxation factor , where = 0 thus, r
ij
= 0, but r
ij
0
as
k+1
i,j

k
i,j
, as before.

k+1
i,j
=
k
i,j
+

2(1+
2
)
_

k
i+1,j
+
k+1
i1,j
+
2

k
i,j+1
+
2

k+1
i,j1
x
2

i,j
2(1 +
2
)
k
i,j

.
(5.19)
For convergence, it is required that 1 < 2. Frankel and Young both
determined an optimum value,
0
, basing their optimality criterion on the
128
asymptotic reduction of the most resistant error. The optimum value de-
pends on the mesh, the shape of the domain, and the type of boundary
conditions. For the Dirichlet (constant boundary values) problem in a rect-
angular domain of size (I 1)x by (J 1)y with constant x and y,
it may be shown that

0
= 2
_
1

_
(5.20)
where
=
_
1
1 +
2
_
cos
_

I 1
_
+
2
cos
_

J 1
___
2
. (5.21)
With =
0
, the k required to reduce the error to some specied level varies
directly with the total number of equations N = (I 2) (J 2) for the
Liebmann method k N
2
. So the SOR method with the optimum
0
, some-
times referred to as the optimum over-relaxation method, is much better for
large problems. Analytic evaluation of
0
exists only for slightly more gen-
eral problems (e.g. Warlick and Young, 1970). Miyakoda (1962) has shown
that
0
increases for the case of the Neumann conditions at all boundaries.
For Dirichlet conditions along some boundaries and the Neumann conditions
along others, for varying
x
or
y
, and for L-shaped and most other non-
rectangular regions, no analytic evaluation exists. In such cases,
0
may be
determined experimentally, by solving successions of Laplace equations with
zero boundary conditions using dierent values of over 1 < < 2, and
monitoring convergence toward
k
i,j
= 0 for large k (the value of
0
does not
change with the source term, ). It is important to assure that all error com-
ponents are present in the initial condition. This is readily met by choosing

0
i,j
= 1 at all interior points. The process of experimentally nding
0
is
tedious because the convergence rate is usually very sensitive to near
0
.
An example is given in Fig. 5.1.
The curvature near
0
shows that it is usually best to slightly overestimate

0
than to underestimate it. The use of some guessed value, say = 1.1, is
seen to have very little good eect. The experimental determination of an
approximation to
0
is then always worthwhile, when the Poisson equation
must be solved many times, e.g. at every time step of the (barotropic)
vorticity equation. The SOR method described here is the original point
SOR of Frankel and Young. It uses advance (k + 1) values at neighbouring
points (I1, j) and (i, j1). It is possible to slightly improve the convergence
rate further by line SOR, which uses advance (k+1) values at 3 neighbouring
points. Ames (1969, page 147) states that line SOR will converge in fewer
129
Figure 5.1: Behaviour of SOR iteration for values of the relaxation parameter
. I = J21, x = Dy, which gives
0
= 1.7295.
iterations than point SOR by a factor of 1/

2 for x = y 0. However,
each iteration takes longer, because of the implicit tridiagonal solution itself.
Because of the simplicity and eectiveness, the point SOR method has been
the most popular of the iterative methods for solving the Poisson equation
in computational uid dynamics problems.
A simple modication of the SOR that is very easy to program is to use
the Liebmann method for only the rst iteration by setting = 1 thereafter,
=
0
is used.
130
5.7 Tactics and Strategy for SOR
The
0
given by Eqs. (5.20) - (5.21) is optimum in the sense of a strategic
or long-range manoeuvre. That is the total error

i,j
e
k
i,j
0 asymptotically
(as k ) the fastest for =
0
. But for a nite k the optimum may be
something less than
0
, depending on the initial error distribution (initial
guess). In fact, if we limit k = 1, then = 1 (Liebmanns method) gives the
greatest reduction in total error in a single sweep; thus Liebmanns method
is the optimum tactical or short-range manoeuvre. Numerical experiments
by Roache (1971) in a 21 21 grid indicated that the advantage of = 1
over =
0
, based on total error, could persist to k = 6 or 8. Further
=
0
, gave errors highly skewed along the diagonal, compared with the
results of = 1. If the direction were changed at each iteration (say, from
i i, j , to i , j , or others), convergence in the average is not aected,
but the skewness is decreased. The more ecient iterative methods do not
produce a truly symmetric calculation for symmetric boundary conditions,
a situation which may be viewed as another behavioural error. Note that
the crude Richardson method has the advantage of giving truly symmetric
calculations.
5.8 Other Iterative Methods
Alternating direction implicit (ADI) Methods
In analogy to Eq. (5.4) we write

k+1/2
=
k
+
t
2
_

xx

k+1/2
+
yy

(5.22)

k+1
=
k+1/2
+
t
2
_

xx

k+1/2
+
yy

k+1

. (5.23)
There are two fractional steps. The rst equation is implicit in x with a
tridiagonal form. The second is implicit in y with the same tridiagonal form.
Convergence is assured, provided the same Dt is used in both half-steps. It
might be thought that very large t would hasten the asymptotic ime
convergence, but actually an optimum t exists. The optimum t gives
convergence in slightly fewer iterations than the optimum SOR. However,
since each iteration of ADI takes longer, the optimum SOR method actu-
ally takes less computer time than this single parameter ADI method. The
real strength of the ADI method comes in choosing a sequence of iteration
131
parameters = 2x
2
/t. The corresponding Dt starts o large and drops
o. The optimum sequence is generally not attainable. The determination
of good parameters is beyond the scope of this text, the reader is referred to
Wachspress (1966).
In SOR methods, the number of iterations required for convergence in-
creases with N. For ADI methods applied to square regions, k
max
is almost
independent of N, so that for large enough N, ADI methods are preferable.
Other iterative methods are the conjugate gradient method and the multi-
grid methods. Implementations of these methods are available as FORTRAN
subprograms, e.g in the program-library ITPACK (package of iterative pro-
cedures). Because of its simplicity and acceptable convergence rate, the basic
SOR method (with = 1 for the rst iteration) will probably continue to be
the most popular iterative method for non-rectangular regions.
5.9 Fourier-Series Methods
The Fourier-series methods are based on the fact that an exact solution
of the nite dierence equation (5.3) can be expressed in terms of nite
eigenfunction expansions. For example, on a rectangular region of dimensions
X Y with M N interior points (N = I 2, M = J 2), with constant
x and y, and 0 an all boundaries, the exact solution of Eq. (5.3) can
be expressed as

i,j
=
_
2
N + 1
N

p=1
H
p,j
sin
px
i
X
, (5.24)
where x
i
= (i1)x. The H
pj
are solutions, for 1 p N, of the tridiagonal
dierence equations,
1
y
2
(H
p,j1
2H
p,j
+ H
p,j+1
) +
p
H
p,j
= V
p,j
(5.25)
with
H
p,1
= H
p,J
= 0 (5.26)
and
V
p,j
=
_
2
N + 1
N

q=1

q+1,j+1
sin
qpx
X
, (5.27)
132

p
=
2
x
2
_
cos
px
X
1
_
. (5.28)
The Fourier-series-method is reviewed by Dorr (1970). The method becomes
very ecient through the use of fast Fourier transforms which can be applied
for N = 2
r
3
s
5
t
with r, s, t integer. The method is simpler for periodic
boundary condition.
We do note, however, that although these methods in their basic forms
are quite restricted in the boundary formulation of the problem, they may
be applied in a modied procedure to more general problems. Consider rst
the case of a rectangular region with Dirichlet condition = f(x, y) on the
boundary, where f = 0 everywhere. An auxiliary function,
1
, is dened by
obtaining the exact solution of
2

1
= , with boundary conditions of
1
= 0
everywhere. Then a second auxiliary function,
11
, is dened by obtaining
the exact solution of the nite-dierence Laplace equation, , with the correct
boundary conditions
11
= f(x, y). The exact solution is obtained by the
separation of variables method of partial dierential equations applied to the
nite-dierence equation. Then the nal solution is obtained, because of
the linearity of the problem, by superposition. That is since
2

1
= and

11
= 0, then
2
(
1
+
11
= , and since
1
= 0 and
11
= f(x, y) on
boundaries, then
1
+
11
= f(x, y). So
1
+
11
satises
2
= and
= f(x, y) on boundaries.
If the zero-gradient Neumann boundary conditions are used, expansion in
a cosine series is appropriate. The problem of non-zero boundary specica-
tion of the normal gradient /n = g(x, y) can be solved as follows. An aux-
iliary function is written as
1
= 0 at all internal points,
1
= g(x, y)n
at the extremes i = I and j = J, and
1
= g(x, y) n at the extremes
i = 1 and j = 1. This
1
is a solution of the auxiliary discretized Poisson
equation
2

1
=
1
, with on boundaries, and
1
= 0 everywhere except at
points adjacent to boundaries, where
1

1
= 0. At a node two positions
in from a boundary,
2

1
= 0, since
1
= 0 at all the neighbouring points.
Dening
11
=
1
and
11
=
1
, the original problem is then con-
verted to nding the nite-dierence equation solution of
2

11
=
11
with
conditions
11
/n = 0 on all boundaries, which can be handled by a cosine
expansion. The desired solution is then
1
+
11
.
In a similar manner, non-rectangular regions may be solved by using a
rectangular mesh which overlaps the desired region. Consider the region
shown in Fig. 5.2a, formed by taking a small corner o of a rectangular
region. The boundary point (2,2) is not on the overlapping rectangle. Con-
sider = 0 on all boundaries. A rst auxiliary function
1
, is dened by
obtaining a solution to
2

1
=
1
in the overlapping mesh with
1
2,2
= 0.
133
Figure 5.2: Non-rectangular regions by a Fourier series method.
A second auxiliary function,
11
, is dened by obtaining a solution to the
equation
2

11
=
11
, where
11
is dened as
11
2,2
= 1, all other
11
i,j
= 0.
In electrical applications where is the potential and is the charge, this
technique is known as the unit charge method. Then a linear combination is
found which gives the desired
2,2
= 0 value without altering at internal
points. That is

2,2
= 0 = 1
1
2,2
+ a
11

11
2,2
(5.29)
or
a
11
=
1
2,2
/
11
2,2
. (5.30)
The denominator is never equal to zero. The nal solution is, by superposi-
tion,
=
1
+ a
11

11
. (5.31)
Although the at (2, 2) of the composite solution is a
11
+ (true) z
2,2
, the
solution is not aected, since (2, 2) is a boundary point of the composite
134
problem and therefore
2,2
does not enter into that problem.
When more than one boundary point is not on the overlapping rectangle,
each point requires an additional Poisson solution. In Fig. (5.2b), auxiliary
solutions
11
,
111
,
1111
, are associated with = 1 solutions at (2,4), (3,3),
(4,2), respectively. Corresponding to Eq. (5.29), the linear system dened
by

2,4
= 0 =
1
2,4
+ a
11
2,4
+ b
111
2,4
+ c
1111
2,4

3,3
= 0 =
1
3,3
+ a
11
3,3
+ b
111
3,3
+ c
1111
3,3

4,2
= 0 =
1
4,2
+ a
11
4,2
+ b
111
4,2
+ c
1111
4,2
(5.32)
must be solved for a, b, c, establishing an inuence coecient matrix to zero
the boundary values for later solution with new . For m boundary points,
m auxiliary Poisson equations must be solved, and a m-order linear system
like (5.32) must be solved by Gaussian elimination. However this need be
done only for the rst solution in a family of problems with dierent , all
in the same mesh.
5.10 Neumann Boundary conditions in SOR
Method
Neumann conditions require two special formulations: rst the incorporation
of the gradient condition in the SOR method. An obvious method of solution
is to sweep the mesh for the new (k + 1) iterative estimates at all interior
points, and then set new iterative (k + 1) estimates for the boundary values
from the known slope and the newly calculated adjacent interior points. As-
sume (i, jc) is on a boundary, then for a point using (5.19), we would have
the following. During interior sweep:

k+1
i,jc+1
=
k
i,jc+1
+

2(1+
2
)
_

k
i+1,jc+1
+
k+1
i1,jc+1
+
2

k
i,jc+2
+
2

k
i,jc
..
boundary
x
2

i,jc+1
2(1 +
2
)
k
i,jc+1

(5.33)
boundary-value determination:

k+1
i,jc
=
k+1
i,jc+1

n
y. (5.34)
This plausible method does not converge. The solution drifts, slowly but end-
lessly. The meteorologist Miyakoda (1962) recommends that the derivative
135
boundary condition is incorporated directly into the SOR dierence scheme
at interior points adjacent to the boundaries. Thus, an equation of the form
(5.19) is used only at interior points more than one node removed from the
boundary. At points adjacent to the boundary, Eq. (5.33) is replaced by

k+1
i,jc+1
=
k
i,jc+1
+

2(1+
2
)
_

k
i+1,jc+1
+
k+1
i1,jc+1
+
2

k
i,jc+2
+
2
_

k+1
i,jc+1

n
_
i,jc
y
_
x
2

i,jc+1
2(1 +
2
)
k
i,jc+1
_
(5.35)
which is solved algebraically for appearing on both sides. After convergence
is completed, the nal boundary values may be computed from Eq. (5.34).
Equation (5.35) diers from the cyclic use of (5.33) and (5.34) only in the
time level of the term

2
_

k+1
i,jc+1

n
_
i,jc
y
_
.
The second special formulation for Neumann conditions is that the boundary
gradients should be compatible with the source term, this applies not only to
SOR but also to direct methods. By Greens theorem, a continuum solution
to
2
= 0 over the area R exists only if
E =
_
R
dR
_
R

n
dl = 0.
Because of truncation error, the boundary values usually will fail to meet this
constraint, causing a slow divergence of SOR (and other iterative methods)
iteration. Miyakoda (1962) recommends that /n be determined to meet
this constraint. Usually, however, the discretized value of E is computed,
and then the modied equation
2
= E/R is solved.
Provided that /n is determined to second-order accuracy, the overall
method appears to be second-order accurate also. Experiments (Miyakoda,
1962) indicate that the optimum
0
is increased by the Neumann conditions.
With Neumann conditions if
i,j
is a solution, then
i,j
+ C, where C is a
constant, is also a solution. The particular solution is chosen by specifying
at one point. This one point may be omitted from the iteration scheme.
5.11 Example of Relaxation Methods
We demonstrate the convergence speed of relaxation methods using a simple
test case. We prescribe the solution

i,j
everywhere in a domain of 41 by 21
136
grid points. The initial distribution is displayed in Fig. 5.3. Then the right
hand side of the Poisson equation
i,j
is calculated from (5.3). We compare
the performance of Richardsons method (5.12), Liebmanns method (5.15)
or (5.18) and the SOR method (5.19) in obtaining the diskrete solution of
the Poisson-equation (5.3) for after k iterations with given
i,j
. We can
then compare the numerical solutions with , considering the maximum error

max
= Max
i,j
_

i,j

k
i,j

_
, the mean error

mean
=
1
I J
J

j=1
I

i=1
_

i,j

k
i,j
_
,
or the standard deviation
=
_
_
1
I J 1
J

j=1
I

i=1
_

i,j

k
i,j
_
2
_
_
1/2
.
Figure 5.3 shows the analytic solution and the convergence of Richardsons
method, Liebmanns method and the SOR method after k = 25 and k = 100
iterations. The accelerated SOR with = 1 for k = 1 (which corresponds
with Liebmanns method for the rst iteration only) can be shown theoret-
ically to converge faster than the original SOR method. In our example
Figure 5.3: Convergence of relaxation methods. a) the true solution. Middle
panels: 25 iterations. b) Richardsons method. c) Liebmanns method. d)
SOR method. Bottom panels: after 100 iterations. The maximum error is
displayed on top of the panels.
however, this acceleration has no (or negligible) benecial eect. This is ob-
vious in Fig. 5.4 where the maximum error is graphed versus the iteration
137
cycles. The accelerated SOR is indicated by a dashed line. The SOR con-
verges much faster than the other two methods up to 70 iterations roughly.
Then we have reached machine accuracy of our computer, smaller corrections
are indistinguishable. On a computer with higher precision (increased word
length) the error of the SOR would decrease further with the same slope.
When the analytic distribution is changed to a random number distribution
prescribed on every grid point, the convergence is similar except that the
accelerated SOR starts o with slightly less maximum error that persists up
to the 10th iteration.
Figure 5.4: Maximum error of relaxation methods versus iteration cycles.
The dashed line corresponds with the accelerated SOR method. It has no
(negligible) benecial eect. The error of SOR does not drop below 10-5
because of limited machine accuracy of our computer.
5.12 General elliptic equations
The most general two-dimensional elliptic equations that can be solved by an
ecient direct method (fast elliptic solver sometimes simply referred to as
138
fast Poisson solver) in the FISHPACK fortran subprogram (ecient FOR-
TRAN subprograms for the solution of elliptic partial dierential equations)
package (subroutine BLKTRI) solves the dierence equivalent corresponding
with the elliptic dierential equation
a(x)

x
2
+ b(x)

x
+ c(x) + d(y)

y
2
+ e(y)

y
+ f(y) = g(x, y)
More general forms are possible to be solved by recursive direct methods
where coecients may vary with x and y (however not in FISHPACK). Three-
dimensional elliptic partial dierential equations may be reduced in dimen-
sion to two-dimensional slices by application of the Fourier-series method.
Fast elliptic solvers may be useful for the solution of any elliptic partial dif-
ferential equation by use of an iteration. Let us assume that we want to
solve the general elliptic equation [] = RHS, where [] is a general
not necessary linear elliptic operator (more precisely, its dierence-equation
equivalent) and RHS is the right hand side. Then we chose the closest avail-
able operator (fast elliptic solver) L[] = RHS (e.g the one from BLKTRI)
and establish an iteration

k+1

= L
_

+
_

RHS
_
,
where is an over-relaxation factor (typical 1-1.3). Without proof it
is evident that if the iteration has converged
k+1
=
k
,
k+1
is a solution
of [
k+1
] = RHS as desired. This approach works for all kind of weakly
nonlinear Operators []. These kind of problems normally arises when coor-
dinate transformations are introduced (e.g. bottom topography). Typically
there are only a few iterations necessary to obtain convergence in case of
mildly nonlinear problems.
Exercises
5.1 Write (5.3) as a matrix equation A = for I = 4, J = 3. Explain the
band-structure of matrix A. Assume that
ij
= 0 at the boundaries of
the domain.
5.2 Show that the stability restriction on Eq. (5.10) is t
2
/4.
5.3 Write Eqs.(5.22)- (5.23) with subscripts and use
=
2x
2
t
.
Collect all new values on the left-hand-side, so that the tridiagonal
structure is obvious.
139
5.4 Show that (5.24)-(5.28) solve the dierence equation (5.3). Consider
the limiting case x 0.
5.5 Find suitable expressions that replace (5.24)-(5.28) in case of periodic
boundaries in the x-direction.
140
Chapter 6
Nonoscillatory Advection
Schemes
One complaint against traditional numerical methods for advection trans-
port is that they usually lead to negative values in the positive-denite scalar
elds (e.g., water substance, chemical tracers) or, in more general terms, to
spurious numerical over- and under- shoots in regions of steep gradients in
these variables. Traditional techniques that are free of these problems are
only rst order accurate (e.g. the upwind method). In the last two decades,
advanced nite-dierence methods for solving the transport problem have
been designed that are essentially free of spurious oscillations characteristic
of higher-order methods yet do not suer from the excessive implicit diu-
sion characteristic of low order methods. Although these essentially nonlinear
algorithms are far more complex than the traditional linear schemes, they
oer strong (linear and nonlinear) stability, maintain steep gradients of the
transported elds, and preserve the monotonicity and/or sign of the advected
variable. These properties make them potentially attractive tools for applica-
tions which combine transport with processes responsible for coupling the ad-
vected variables. In atmospheric applications, methods suppressing spurious
oscillations are usually considered in the context of advection transport equa-
tions and their most advertised properties are sign-preservation and overall
smooth appearance of of the solutions. Complexity of atmospheric equations
associated with multidimensionality, earth rotation, sphericity, orography,
phase-change processes, radiation, chemical reactions, and other natural ef-
fects makes stability, second order accuracy, generality, and computational
eciency the primary concerns of atmospheric models. Although there is a
wide availability of nonoscillatory techniques suitable for constant coecient
advection in one spatial dimension, there are few techniques adequate for
general ows.
141
6.1 Computational stability of nonoscillatory
advection schemes
The basic equation to be solved is the continuity equation describing the
transport of a nondiusive scalar quantity in M-dimensional space,

t
+
M

I=1

x
I
_
u
I
_
= 0, (6.1)
where =
_
t, x
1
, . . . , x
M
_
is the nondiusive scalar quantity u
I
= u
I
_
t, x
1
, . . . , x
M
_
is the I
th
velocity component, I = 1, . . . , M and (t, x) =
_
t, x
1
, . . . , x
M
_
are the time- and space-independent variables. Equation
(6.1) is usually referred to as ux form of the advection equation. In order to
compress the notation in the following numerical equations, we choose tradi-
tional n and i indices to denote, respectively, the temporal and spatial posi-
tion on a uniform computational grid (t
n
, x
i
) =
_
nt, i
1
X
1
, . . . i
M
X
M
_
and adopt e
I
for the unity vector in the I direction. A conservative advection
algorithm for integration of (6.1) may then be compactly written as:

n+1
i
=
n
i

M

I=1
_
F
I
i+1/2e
I
F
I
i1/2e
I
_
, (6.2)
where F
I
i+1/2e
I
X
I
is an approximation to the I component of the integrated
advective ux, evaluated at i +
1
2
e
I
position on the grid based on the infor-
mation provided in a local neighbourhood of the i +
1
2
e
I
grid point. Since
in (6.2) the time level of the uxes may be taken at any position, this equa-
tion represents the general form of an arbitrary nite-dierence ux-form
scheme. For the sake of illustration, the uxes from selected, elementary,
one-dimensional advection schemes are explicitly written as follows:
F
i+1/2
=
_

n
i+1/2

n
i
+
_

n
i+1/2

n
i+1
, (6.3)
F
i+1/2
=
1
2

n
i+1/2
_

n
i+1
+
n
i
_

1
2
_

n
i+1/2
_
2
_

n
i+1

n
i
_
, (6.4)
F
i+1/2
=
1
2

n+1/2
i+1/2
_

n+1/2
i+1
+
n+1/2
i
_
, (6.5)
where = (ut)/X is a local Courant number, dened on a staggered grid,
and [ ]
+
, [ ]

denote the non-negative- and non-positive-part operators,

respectively. Equations (6.3)-(6.5) provide uxes from, correspondingly, the
rst-order-accurate donor-cell (alias upwind, upstream) scheme (6.3), the
second-order-accurate (for uniform ow) Lax-Wendro scheme (6.3b), and
142
the second-order-accurate, centred in time and space (leapfrog) scheme (6.5).
A simple, compact form of the advective uxes in (6.3)-(6.5) is typical of
elementary advection schemes.
In order to assess the computational stability of nonoscillatory schemes,
consider rst a constant-sign eld (t, x) in (6.1) and an arbitrary, sign-
preserving advection algorithm in (6.2). For simplicity, assume that both
analytic and numerical uxes vanish at the boundaries of a computational
domain. Then the conservation form (6.2) implies

n
i
=

0
i
. (6.6)
Since the scheme preserves the sign of the transported quantity by assump-
tion, (6.6) is equivalent to
n

i
|
n
i
|

0
i

. (6.7)
Recalling that

| |
_
( )
2
_
1/2
, (6.7) implies that

i
(
n
i
)
2

0
i

_
2
B, n. (6.8)
In other words, total energy (entropy) of the sign-preserving solution is
uniformly bounded in time, which is to say that the sign-preserving solution
is computationally stable. Since for a suciently small time step advection
schemes can be designed which are sign-preserving for arbitrary ows (e.g.,
Smolarkiewicz, 1984; Smolarkiewicz and Clark, 1986), the inequality (6.8) is
a statement of both linear and nonlinear stability for such schemes.
The simplicity of the result in (6.8) is a direct consequence of the assump-
tion that is of a constant sign. For variable-sign elds, a similar result may
be obtained by noting that every may be uniquely decomposed into the
nonpositive and nonnegative part, = []
+
+[]

. Since the two parts have

disjoined supports, they are independent of each other, and they both satisfy
(6.1). Applying a sign-preserving advection scheme to both parts ensures
uniform boundedness of both []
+
and []

, and consequently of their sum.

Such arguments can be further elaborated (over a single time step) for the
transport equation with forcings and/or sources leading to the conclusion
that sign-preserving advection schemes
1
oer the means of controlling non-
linear stability in numerical models. This result is rarely appreciated in the
1
This also concerns monotonicity-preserving schemes, as every monotonicity-preserving
scheme is also sign-preserving (the oppsite is not true).
143
meteorological literature, where sign- and/or shape-preserving schemes are
usually considered in the context of their elementary (dening) properties,
and where Arakawa-type schemes (Arakawa, 1966) are thought to be the
obvious choice insofar as the nonlinearly stable, nite-dierence advection
transport algorithms are concerned.
6.2 General ux-corrected-transport (FCT)
procedure
Among a variety of existing monotonicity-preserving methods, the FCT schemes
originated by Boris and Book (1973), and later generalized by to fully multi-
dimensional algorithms by Zalesak (1979), become perhaps the only modern
nonlinear approach that has been adopted on a permanent basis in several
atmospheric and oceanic research models. It is important to realize that FCT
is a general concept that allows several degrees of freedom and may lead to
many dierent schemes. A number of known nonoscillatory techniques may
be viewed as either a particular realization of the FCT approach or as imple-
menting certain conceptual elements of FCT in their design. The basic idea
of the FCT approach is simple: The generic reason for the appearance of the
oscillations in the numerically generated higher-order-accurate solutions to
(6.1) is that the magnitude of certain uxes is overestimated with respect to
their analytic value. In contrast, the magnitude of the uxes given by rst-
order-accurate schemes is underestimated, which results in monotone but
heavily damped solutions (Zalesak, 1979). The FCT procedure overcomes
the problem of false oscillations by imposing appropriate limits on the trans-
port uxes from the higher-order-accurate algorithms. Consider an arbitrary,
higher-order-accurate advection algorithm for the integration of (6.1):

n+1
i
=
n
i

M

I=1
_
FH
I
i+1/2e
I
FH
I
i1/2e
I
_
. (6.9)
The high-order FH-ux may be arbitrarily cast into the sum of the ux from
a certain low-order nonoscillatory scheme and the residual, i.e.,
FH
I
i+1/2e
I
= FL
I
i+1/2e
I
+ A
I
i+1/2e
I
, (6.10)
where (6.10) denes the residual A-ux, which has a sense of correcting at
least the rst order truncation error terms in the transport uxes of the
low-order scheme, i.e.,
A
I
i+1/2e
I
t O(X, t) + HOT, (6.11)
144
where HOT has the usual meaning of higher order terms. Because of this
compensation of the leading truncation-error term in a low-order scheme, the
A-ux is traditionally referred to as the antidiusive ux. Using (6.10) in
(6.9) results in

n+1
i
=
n+1
i

M

I=1
_
A
I
i+1/2e
I
A
I
i1/2e
I
_
(6.12)
where denotes the solution given by the low-order scheme, which by
assumption satises

MAX
i

n+1
i

MIN
i
, (6.13)
where
MAX
i
and
MIN
i
are yet unspecied maximal and minimal values of
the scalar within the the i
th
grid box that achieve the monotonicity of the
scheme. Their explicit form will be discussed later in this section. Inasmuch
as
n+1
i
preserves the monotone character of the transported eld [by means
of (6.13)], the eventual oscillatory behaviour in
n+1
i
comes from overestimat-
ing the magnitude of certain A-uxes in (6.12). Thus to ensure ripple-free
solutions it is sucient to appropriately limit A-uxes such that
A
I
i+1/2e
I
= C
I
i+1/2e
I
A
I
i+1/2e
I
, (6.14)
where C-coecients, that in general are functions of the low- and high-order
solutions on the grid, are determined from the set of constraints
0 C
I
i+1/2e
I
1 (6.15)
and

MAX
i

n+1
i
=
n+1
i

M

I=1
_
A
I
i+1/2e
I
A
I
i1/2e
I
_

MIN
i
. (6.16)
When C
I
i+
1
2
e
I
is equal to zero or unity the resulting transport ux in (6.16)
becomes or respectively. The assumed convergence of the low-order schemes
involved in (6.10) together with (6.11), (6.14), and (6.15) ensure the conver-
gence of the -solutions in (6.16) as X, t 0.
The constraints in (6.15) and (6.16) allow one to derive formally the
explicit form of the C-coecients and, consequently, the explicit form of the
limited anti-diusive uxes in (6.14). The derivation provides maximized
A-uxes in (6.14) satisfying constraints (6.15) and (6.16):
145
A
I
i
+1/2e
I
= min
_
1,

i
,

i+1/2e
I
_
_
A
I
i+1/2e
I

+
+min
_
1,

i
,

i+1/2e
I
_
_
A
I
i+1/2e
I

,
(6.17)
where

i
=

MAX
i

n+1
i
A
IN
i
+
, (6.18)

i
=

n+1
i

MIN
i
A
OUT
i
+
, (6.19)
and A
IN
i
, A
OUT
i
are the absolute values of the total incoming and outgoing
A-uxes from the i
th
grid box,
A
IN
i
=
M

I=1
_
_
A
I
i1/2e
I

_
A
I
i+1/2e
I

_
,
and
A
OUT
i
=
M

I=1
_
_
A
I
i+1/2e
I

_
A
I
i1/2e
I

_
,
respectively. is a small value, e.g. 10
15
, which has been introduced
herein to allow for ecient coding of -ratios when A
IN
i
or A
OUT
i
vanish.
Equations (6.16), (6.17), (6.10), and Eqs. (6.18), (6.19) constitute a gen-
eral, arbitrary dimensional form of the FCT-algorithm discussed by Zale-
sak (1979). The arbitrary dimensionality of this procedure contrasts with
alternate-direction approach utilized by most other monotone schemes. In
order to determine

i
and

i
uniquely, one must specify the limiter
MAX
i
,

MIN
i
in (6.18), (6.19). The simple standard limiter (Zalesak, 1979) is

MAX
i
= max
I
_

n
ie
I
,
n
i
,
n
i+e
I
,
n+1
ie
I
,
n+1
i
,
n+1
i+e
I
_
(6.20)

MAX
i
= max
I
_

n
ie
I
,
n
i
,
n
i+e
I
,
n+1
ie
I
,
n+1
i
,
n+1
i+e
I
_
. (6.21)
The low-order, nonoscillatory -solutions appearing in Eqs. (6.20), (6.21)
constitute the original limiter of Boris and Book (1973). The limiter ef-
fectively prevents development of spurious oscillations in an arbitrary ow
eld. Zalesak (1979) extended the original limiter onto the local extrema
of the solution at the previous time step. The goal of this extension is to
improve the predictions in incompressible ows where the only extrema al-
lowed in an arbitrary grid point are those that were present in its immediate
146
environment (determined by the CFL stability criteria) at the previous time
step; in principle, i.e., with accuracy to round-o errors, the original Boris
and Book limiter is redundant in incompressible ows. Note that the lim-
iters (6.20), (6.21) ensure uniform boundedness of the solution, providing
uniformly bounded . Since uniform boundedness of the low-order solu-
tions is easily achievable, a nonlinear stability of the FCT approximations
is assured essentially by design. Note also that the FCT limiters in (6.20),
(6.21) impose tighter bounds on the nite-dierence solutions than the en-
ergy (entropy) constraint (6.8) of sign-preserving schemes considered in
the preceding section.
There are several degrees of freedom in the approach outlined. First, the
constraints (6.15) and (6.16) can be supplemented with some additional con-
ditions which may need to be satised by a nal approximation. Second, so
long as (6.13) holds, the limiters themselves may be, in principle, arbitrary.
This emphasizes that a nonoscillatory character of the solutions is under-
stood in a relative sense with respect to and the limiters, and that the
limiters essentially dene monotonicity of the resulting FCT scheme (for
instance, selecting weaker limiters
MAX
i
= and
MIN
i
= 0 leads to posi-
tive, but apparently oscillatory, advection schemes). This degree of freedom
is particularly useful where synchronized limiting of a system of equations
is concerned. Finally, the third degree of freedom is in the choice of the
high- and low-order schemes mixed by the FCT procedure. As there are no
essential restrictions imposed on the mixed schemes, a variety of FCT algo-
rithms may be designed. The formalism outlined does not even necessarily
require mixing of low- and high-order schemes but provides a general frame-
work for mixing any two algorithms attractive for a particular problem at
hand. Insofar as atmospheric applications are concerned, y evaluated with
the rst-order-accurate upwind scheme is usually considered as a low-order
solution , whereas the leapfrog advection schemes are usually selected (fol-
lowing Zalesak, 1979) for the high-order-accurate algorithms. However, as
the resulting FCT algorithm mixes the two schemes of dierent distributions
of the truncation errors, it is not necessarily optimal in terms of overall ac-
curacy. The leapfrog schemes yield excellent amplitude behaviour but large
phase errors, whereas the upwind scheme suers from large amplitude errors
but relatively small phase errors. Because the low- and high-order solutions
are shifted in phase, the FCT mixing of the two solutions (which should
eliminate dispersive ripples essentially without degrading the accuracy of the
high-order solutions) is inecient.
The latter point is illustrated in Fig. 6.1, which shows the result of
one-dimensional uniform advection of the irregular signal (heavy solid line).
The dashed line corresponds to the leapfrog solution, whereas the thin solid
147
line displays the solution obtained with the upwind/leapfrog FCT scheme.
Although the FCT procedure eciently removes the dispersive ripples, the
amplitudes of the initial perturbations are severly damped at the cost of
improving the phase error. The overall accuracy of the FCT solutions may
be improved by employing for the high-order scheme an algorithm of phase-
error characteristics similar to those of the low-order scheme. The nonlinear
Figure 6.1: Uniform advection of the irregular signal (heavy solid line) with
oscillatory scheme (dashed line) and with their non-oscillatory FCT versions
(thin solid lines).
MPDATA methods (Smolarkiewicz and Clark, 1986) are an excellent choice
for this pupose as they yield phase-errors similar to those in the upwind-
scheme. The MPDATA schemes, whose design is conceptually unrelated to
the FCT approach, are at least second-order-accurate for arbitrary ows and
already preserve the sign of the transported eld (therefore, for many prac-
tical applications they do not require additional FCT enhancement). Figure
6.1b shows the MPDATA solution (dashed line) and the fully nonoscillatory
solution obtained with the FCT version of this scheme (thin solid line), as
well as the analytic solution (heavy solid line). Comparison of the three
curves shows that the primary eect of the FCT modication is to remove
the overshoot and the undershoot present in the original MPDATA solution.
6.3 Semi-Lagrangian approximations for at-
mospheric ows using nonoscillatory ad-
vection schemes
Having established general FCT procedure and its degrees of freedom, we
narrow the discussion here to a special class of applications which empha-
sizes the utility and importance of all those one-dimensional nonoscillatory
148
schemes which are suitable for integrating merely a constant-coecient ad-
vection equation. Although such schemes are not particularly competitive in
applications addressing complex natural problems in Eulerian formulation,
they become accurate and powerful tools as nonoscillatory interpolation op-
erators inherent in semi-Lagrangian representation of equations governing
atmospheric phenomena.
The notion of a semi-Lagrangian approach is simple. It originates with a
Lagrangian form of the evolution equation for a uid variable
d
dt
= R (6.22)
and the particular interpretation of its integral
(x
i
, t) = (x
0
, t
0
) +
_
T
R dt (6.23)
where (x
i
, t) and (x
0
, t
0
) are the two points connected by a parcels trajectory
T, and R combines all sources and forces. Assuming that (x
i
, t) represents
points of a regular mesh, one computes backward-in-time the trajectories ar-
riving at (x
i
, t) and nds their departure points (x
0
, t
0
). Since the departure
points do not necessarily coincide with the points of the mesh, information
therein is not immediately available, and one has to invoke an interpola-
tion procedure to determine (x
0
, t
0
). In essence, similar action must be
taken with respect to the source term, with details depending upon a se-
lected method of the integrals approximation and the particular problem at
hand. The advertised advantages of semi-Lagrangian techniques are a cir-
cumvention of the CFL stability condition (a typical restriction of Eulerian
methods) and the convenience of a regular-mesh discretization (contrasting
with purely Langrangian techniques). A variety of semi-Lagrangian methods
has been designed exploiting dierent interpolation techniques and dierent
discretizations (see Staniforth and Cte, 1991 for a review).
Smolarkiewicz and Rasch (1990) suggested that semi-Lagrangian methods
represent a special case of a more general approach. Referring to elementary
properties of dierential forms and viewing a variable on the time-space
continuum, one can relate two values of the variable at any two points of
the continuum through Stokes theorem, gaining thereby a large degree of
freedom associated with the selection of points and connecting contours. Se-
lecting contours whose elements coincide with parcels trajectories leads to a
variety of semi-Lagrangian formulations. Moreover, this approach documents
that interpolation procedures - inherent to semi-Lagrangian approximations
- may be replaced with Eulerian algorithms suitable for integrating constant-
coecient advection equations while retaining their formal accuracy and such
149
attractive properties as monotonicity and/or sign preservation. In order to
show this latter point (which is of particular interest to applications), con-
sider the following interpolation problem: let us assume that at an arbitrary
instance t
0
(for conciseness, the dependence of on t
0
will be dropped in the
following discussion) a suciently continuous eld is known a priori in x
i
points of a uniform mesh in R
M
. For the sake of clarity, we restrict ourselves
to regular meshes with a constant grid interval X = (X
1
, . . . , X
M
)
such that x
i
= i X. The problem addressed is to provide an estimate
of , with certain desired accuracy, in a point of interest x
0
and provide
adequate estimates of the derivatives using information available on the grid,
or equivalently, to evaluate at x
0
based on the assumption that ts a
p-th order Lagrangian polynomial in between the grid points. We consider
here, however, an alternate approach (Smolarkiewicz and Grell, 1991).
As a consequence of the Stokes theorem,
(x
0
) (x
i
) =
_
c
dx (x) (6.24)
where C denotes an arbitrary contour connecting the point of interest x
0
with
an arbitrary x
i
of the computational grid. Exploiting the arbitrariness of the
contour selection in (6.24), we choose for C a line segment of the parametric
equation
x(x
i
, ) = (x
i
x
0
) +x
i
(6.25)
where the parameter [0, 1]. Implementing (6.25) in (6.24) gives
(x
i
, 1) = (x
i
, 0)
_
1
0
(x
i
x
0
) (x
i
, )d, (6.26)
where
(x
i
, 1) = (x
i
, 0)
_
1
0
(x
i
x
0
) (x
i
, )d. (6.27)
Since for xed x
0
and x
i
, the rst element of the scalar product appearing
under the integral in (6.26) is constant, (6.26) may be rewritten as
(x
i
, 1) = (x
i
, 0)
_
1
0
(U(x
i
, )d, (6.28)
where
U = x
i
x
0
. (6.29)
150
Equation (6.28) represents the integral of the equation

+ (U) = 0, (6.30)
over the interval [0, 1] at x
i
grid-point. In other words, (6.28) is a formal
solution of the advection equation (6.30) in which the free parameter plays
the role of a time independent variable, and the vector U dened in (6.29)
plays the role of a velocity eld. Therefore, the interpolation problem has
been expressed as the equivalent advection problem, and (6.28) may be ap-
proximated using, in principle, any known advection algorithm. Among the
variety of available advection algorithms, the foreward-in-time schemes are
the most attractive for applications, as they require information only from
x(x
i
, = 0) points in (6.25), where (x
i
, 0) = (x
i
) are known. Such an
approximation may be compactly written as

1
i
=
0
i
AS
i
(
0
, ), (6.31)
where
=
U
X
=
_
U
1
X
1
, ...,
U
M
X
M
_
is an eective Courant number vector (recall that = 1), and the nite
dierence ux-divergence operator AS identies an advection scheme dened
by its particular dependence on the values of and available on the mesh
in a neighbourhood of the x
i
grid point. The truncation error in the approx-
imation in (6.31) represents the error of estimating at the point of interest
x
0
. Choosing an arbitrary grid point x
i
that appears in the denition of the
eective advecting velocity (6.29), as the closest grid point to x
0
,
x
i
= [x
0
] = (NINT(x
1
0
/x
1
) x
1
, . . . , NINT(x
M
0
/x
M
) x
M
) (6.32)
(where (NINT denotes the nearest integer value), and using denitions (6.29)
and (6.27), the resulting interpolation algorithm may be compactly written
as
(x
0
) = ([x
0
]) AS
[x
0
]
_
,
[x
0
] x
0
x
_
. (6.33)
There is no gain in using (6.33) with most linear forward-in-time advec-
tion algorithms, however, where preservation of the monotonicity and sign
of the interpolated variable is concerned, (6.33) becomes a useful tool as
it allows to implement advanced monotone and sign-preserving advection
151
schemes. The simplicity and eciency of (6.33) is a consequence of the con-
stancy of the eective Courant number in (6.33), which allows for straightfor-
ward, alternate-direction (time-split) applications of one-dimensional advec-
tion schemes without degrading the formal accuracy of their constant coe-
cient limit. In contrast to fully multidimensional algorithms, the time-split
approximations employing one-dimensional advection schemes are simple and
versatile. Since

1
2

_
x
I
0

x
I
0
x
I

1
2
(6.34)
the computational stability is ensured for all one-dimensional forward-in-time
advection schemes, securing thereby the stability of a time-split algorithm for
arbitrary M.
A paradigm for the applicability: The integration of two-dimensional
Boussinesq-Equations
The utility of simple nonoscillatory schemes in (6.33) for semi-Lagrangian
integrations is illustrated in the example of a thermal convection. The govern-
ing system of equations consists of the momentum and temperature evolution
equations for an ideal, nonrotating, two-dimensional Boussinesq uid
dv
dt
=

+g

0
z (6.35)
d
dt
= 0. (6.36)
Here, v = (u, w) is the velocity vector in the (x, z) vertical plane,

(z) is the
pressure (normalized by a reference density,

= p/
0
), and g is the gravity.
denotes the potential temperature of a uid parcel, whereas

= (z)
represents its deviation from ambient, hydrostatic prole (z).
0
= (z) is
a reference potential temperature. The prognostic equations (6.35-6.36) are
accompanied by the incompressibility constraint
v = 0 (6.37)
characteristic of the Boussinesq approximation. Free-slip, rigid-lid upper and
lower boundaries, and open lateral boundaries are assumed. The adapted
semi-Lagrangian approximation to Eqs. (6.35)-(6.36) and (6.37) consists of
three distinct steps. First, departure points of the trajectories are evaluated
using the second-order-accurate implicit midpoint rule
x
0
= x
i
v(x
m,
t
m
)t, (6.38)
152
where the velocities at the midpoints of the trajectories (x
m
, t
m
) are predicted
with the rst-order-accurate integral of the momentum equations
v(x
m
, t
m
) + 0(t
2
) = v(x
0
, t
0
) v(x
0
, t
0
) +
1
2
t
dv
dt
(x
0
, t
0
). (6.39)
The implicit nature of the trajectory algorithm consisting of (6.38) and (6.39)
requires an iterative solution; the iteration converges provided
B =

v
x

t < 1 (6.40)
and one iteration (assuming the Euler backward approximation for the rst
guess) suces for the second-order-accurate approximation to the departure
points of the trajectories. In the second step, (6.35-6.36) are approximated
assuming the trapezoidal rule
_
T
Rdt
1
2
t (R (x
i
, t
0
+ t) + R(x
0
, t
0
)) (6.41)
for evaluating the trajectory integral (6.23); updating the temperature prior
to the momentum ensures availability at t0 + Dt in the vertical equation of
motion. While integrating momentum equations (6.35), pressure forces at
t
0
+t are yet unknown and must be determined from the incompressibility
constraint (6.37). In this third step, the approximate solution to (6.35) is
written as
v(x
i
, t
0
+ t) = v

(x
i
)
1
2
t

(x
i
, t
0
+ t) (6.42)
where v

(x
i
) combines all known terms appearing on the right hand side of
(6.41) (i.e., velocity, pressure gradient, and buoyancy terms evaluated at (x0,
t0), as well as the buoyancy term at (x
i
, t
0
+ t). Applying the divergence
(6.37) to (6.42) leads to the Poisson equation for the pressure

(x
i
, t + t) =
2
t
v

(x
i
) (6.43)
which is solved with standard methods, subject to (Neumann) boundary
conditions. Having established pressure at t
0
+t, (6.42) completes the pro-
cedure. The entire semi-Lagrangian algorithm for Eqs. (6.35)-(6.36) and
(6.37) is implicit with respect to all ordinary dierential equation integra-
tions, ensuring thereby computational stability regardless of the time step
employed; t is solely controlled by the convergence condition (6.40).
Figure 6.2a shows a solution to Eqs. (6.35)-(6.36) and (6.37) obtained
with the above-outlined solver which employs (6.33) in all interpolations in-
herent in semi-Lagrangian integrations. The AS operator is from the FCT
153
(ux-corrected-transport) advection scheme employed in the simple example
Fig. 6.1c. The experimental set-up assumes a symmetric thermal of the ini-
Figure 6.2: Isolines of the temperature perturbation of the inviscid buoy-
ant thermal at dimensionless time 7 simulated with the semi-Lagrangian
model. The contour interval is 0.2 of the initial amplitude: a) experi-
ment with nonoscillatory interpolation implementing FCT-advection scheme
in Fig. 6.1c; b) as in plate a), but with oscillatory interpolators.
tial radius r
0
= 250 m; the thermal is placed in the 8r
0
8r
0
domain resolved
with 200 200 uniform grid intervals X = Z = 0.04r
0
. The centre of
the thermal is located at the vertical symmetry axis of the domain r
0
+ Z
above the bottom boundary. The thermal has a uniform, initial temperature
excess of 0.5K relative to the neutrally stratied environment (z) = 300 K.
The eld is shown after 7 characteristic time scales T = r
0
/U, where U is the
characteristic velocity scale U = (2gBr
0
)
1
/
2
with B representing initial buoy-
ancy of the thermal. The time step of the computations is t 0.06T, which
results in B 0.5 in (6.40). The excellent performance of the algorithm is
evident in its ability to resolve ne interfacial structures and maintain sharp
across-the-interface gradients while introducing no spurious oscillations. The
highly inviscid yet smooth nature of computations is clear in Fig. 6.2a; the
initial thermals core may still be identied within narrow ( 2X wide) in-
terfacial convolutions engulng ambient uid. This performance is primarily
controlled by the monotonicity of the interpolation procedure in (6.33), which
by design preserves local convexity/concavity and sign, and ensures uniform
boundedness, of the interpolated variables. In the absence of forcing, these
properties hold along parcel trajectories consistently with analytic equations.
Non-monotonic schemes lead to poor results with excessive numerical noise
and a spurious secondary thermal in the wake of the main structure, evident
in Fig. 6.2b. Due to the Lagrangian format of the approximated equations,
the monotonicity constraint is ultimately imposed along a parcel trajectory.
154
Exercise
6.1. Reformulate the integration of the Boussinesq uid (6.35)-(6.36)-(6.43)
for an anelastic approximation (
0
(z) v) = 0. Energetic consider-
ations require that the same pressure variable = p/
0
as in (6.35)
must be used.
155
Chapter 7
Spectral Methods
7.1 Introduction
The numerical integration methods discussed thus far are based on the dis-
crete representation of the data on a grid or mesh of points covering the
space over which a prediction of the variables is desired. Then a local time
derivatives of the quantities to be predicted are determined by expressing
the horizontal and vertical advection terms, sources etc., in nite dierence
form. Finally, the time extrapolation is achieved by one of many possible al-
gorithms, for example leapfrog. The nite-dierence technique has a number
of associated problems such as truncation error, linear and nonlinear insta-
bility. Despite these diculties, the nite-dierence method has been the
most practical method of producing forecasts numerically from the dynami-
cal equations.
There is another approach called the spectral method which avoids some
of the diculties cited previously, in particular, nonlinear instability; however
the method is less versatile and the required computations are comparatively
time consuming. In a general sense, the mode of representation of data de-
pends on the nature of the data and the shape of the region over which the
representation is desired. An alternative to depiction on a mesh or grid of dis-
crete points is a representation in the form of a series of orthogonal functions.
This requires the determination of the coecients of these functions, and the
representation is said to be spectral representation or a series expansion in
wavenumber space. When such functions are used, the space derivatives can
be evaluated analytically, eliminating the need for approximating them with
nite-dierences.
As indicated earlier the choice of orthogonal functions depends in part
on the geometry of the region to be represented, and for meteorological data
156
a natural choice is a series of spherical harmonics. The rst published work
on the application of this technique to meteorological prediction is that of
Silberman (1954). He considered the barotropic vorticity equation

t
= v ( + f), (7.1)
in spherical coordinates, where
e

1
a

+ e

1
a sin

and v = v

+ v

t
=
1
a
_
v

+
v

sin

_
( + 2 cos ), (7.2)
=
1
a sin
_

(v

sin )
v

_
, (7.3)
where a is the earths radius, is the co-latitude, is the longitude, and v

and v

are the velocity components in the directions of increasing and .

In terms of a stream function , the velocity components from v = k
are
v

=
1
a

, v

=
1
a sin

and the vorticity equation becomes with =

2
s

2
s

t
=
1
a
2
sin
_

_
(
2
s
+ 2cos ), (7.4)
where

2
s
=
1
a
2
sin
_

_
sin

_
+
1
sin

2
_
.
The stream function is represented in terms of spherical harmonics which are
the solutions Y
m
n
of the equation
a
2

2
s
Y
m
n
+ n(n + 1)Y
m
n
= 0. (7.5)
These functions are expressible in the form
Y
m
n
= e
im
p
m
n
. (7.6)
If depend only on q if (7.6) is substituted into (7.5), the result is the ordinary
dierential equation (Exercise 7.1)
157
d
2
P
m
n
d
2
+ cot
dP
m
n
d
+
_
n(n + 1)
m
2
sin
2

_
P
m
n
= 0. (7.7)
This is a form of the Legendre equation, and its solutions P
m
n
are known as
Legendre functions of order m and degree n. The characteristic solutions of
(7.7) are expressible as a set orthonormal functions such that
_

0
p
m
n
p
m
s
sin d =
ns
, (7.8)
where
ns
is the Kronecker delta,

ns
= 1 if n = s
and

ns
= 0 if n = s.
The order m may take on negative values for which the legendre function is
P
m
n
= (1)
m
P
m
n
.
For integer values of n and m, the Legendre functions are simply polynomials,
the orders of which increase with n.
At any particular time t

the stream function may be expressed as the

nite sum

t=t
= a
2

n=|m|
m

m=m

K
m
n
Y
m
n
. (7.9)
Since the series is nite, disturbances of suciently small-scale are not repre-
sented, which constitutes a truncation error; however, this is not necessarily
undesirable. The harmonic coecients K
m
n
are complex, and the condition
for a real is that
K
m
n
= (1)
m
K
m
n
where the

denotes the complex conjugate. The stream function tendency
is given by
_

t
_
t=t

= a
2

n|m|
m

m=m
_
dK
m
n
dt
_
t=t

Y
m
n
. (7.10)
When the expressions for
partialpsi and /t are substituted into the vorticity equation (7.4) and then
158
this equation is multiplied by Y
m
n
sin and integrated from 0 to 2 with
respect to and from 0 to with respect to , the result is
_
dK
m
n
dt
_
t=t

=
2imK
n
(t)
n(n + 1)
(7.11)
+
i
2
n

s=|r|
m

r=m

k=|j|
m

j=m

K
j
k
(t

)K
r
s
(t

)H
jmr
kns
,
where H
jmr
kns
is zero unless j + r = m, in which case,
H
jmr
kns
=
s(s + 1) k(k + 1)
n(n + 1)
_

0
P
m
n
_
jP
j
k
dP
r
s
d
r
dP
j
k
d
P
r
s
_
d. (7.12)
The quantities H
jmr
kns
are called interaction coecients, which are zero unless
k + n + s = odd integer and |k s| < n < k + s.
Also m

2m

and n

2n

1. After the right side of the tendency equation

(7.11) has been calculated, the future values of the expansion coecients may
be determined by extrapolating forward in time as in the nite-dierence
technique; for example, the leapfrog method:
K
m
n
(t + t) = K
m
n
(t t) + 2t
dK
m
n
(t)
dt
. (7.13)
Robert (1966) pointed out that the components u and v of the wind eld
constitute pseudo-scalar elds on the globe, and as such are not well suited
to representation in terms of scalar spectral expansions; he suggested that
the variables
U = u cos f and V = v cos f, (7.14)
where denotes latitude, would be more appropriate for global spectral
representation.
The nondivergent ow eld may be represented as usual in terms of a
scalar stream function as
V = k .
Accordingly is seen to be expressed as
159
= k V =
2

and the quantities U and V as

U =
cos
a

and V =
1
a

(7.15)
The conservation of absolute vorticity may now be rewritten, with substitu-
tion of Eqs. (2) and (4), and an expansion into spherical polar coordinates
as

2
=
1
a cos
2

(U
2
) + cos

(V
2
)
_
2
V
a
, (7.16)
where , , a, and denote, respectively, latitude, longitude, and the earths
radius and rotation rate. The linear equations (7.15) provide specication of
the diagnostic quantities U and V in terms of the prognostic .
The more usual form of the barotropic vorticity equation is seen on sub-
stitution of Eqs. (7.15) into (7.16) to be

t
_

_
=
1
a
2
cos
_

2
a
2

. (7.17)
The calculation of the interaction coecients is a lengthy task. An advantage
of the method, however, is that nonlinear instability is avoided completely
because all nonlinear interactions are computed analytically and all con-
tributions to wave numbers outside the truncated series are automatically
eliminated.
Robert (1966) proposed a modication to Silbermanns method for nu-
merical integration of the primitive equations in which some simpler functions
are substituted for the spherical harmonics. These functions are in fact the
basic elements required to generate spherical harmonics, namely,
G
m
n
(, ) = e
im
cos
M
sin
n
.
Here and are the longitude and latitude respectively, M is the absolute
value of m and gives the number of waves along a latitude circle, and both
M and n are either positive integers or zero.
160
7.2 An Example of the Spectral Method
As an illustration of the spectral method, we present a simple example due
to Lorenz (1960). Consider the vorticity equation (7.1) applied to a plane
region over which the stream function is doubly periodic, that is,

_
x +
2
k
, y +
2
l
, t
_
= (x, y, t), (7.18)
where k and l are constants. Thus the area is nite but unbounded, and
in that respect it is analogous to the spherical earth. Note also that (7.1)
applies with a constant coriolis parameter so that rotation is not excluded.
Next assume that the stream function can be represented in terms of the
characteristic solutions of the equation

2
c = 0, (7.19)
which is the analogue to (7.5). The solutions are trigonometric functions;
thus is expressible as a double Fourier series, which for convenience may
be written as
=

m=

n=

1
m
2
k
2
+ n
2
l
2
[A
mn
cos(mkx + nly) + B
mn
sin(mkx + nly)] .
(7.20)
The coecients are as yet unknown functions of time, except for the ini-
tial values which are assumed known. It is apparent from (7.16) that the
characteristic values of (7.19) are c
mn
= (m
2
k
2
+ n
2
l
2
).
For actual prediction purposes a nite series of the form (7.20) is used and
the time derivatives of the coecients must be determined from the vorticity
equation. Consider a simple case where m and n take on only the values of
0, 1, 1.
Then after combining like terms, (7.20) is expressible in the form
=
A
10
k
2
cos kx
A
01
l
2
cos ly
A
11
k
2
+l
2
cos (kx + ly)

A
11
k
2
+l
2
cos (kx ly)
B
10
k
2
sin kx
B
01
l
2
sin ly

B
11
k
2
+l
2
sin (kx + ly)
B
11
k
2
+l
2
sin (kx ly).
It turns out that, if the Bs are zero initially, they will remain so; also if
A
11
A
11
initially, it will remain so. With these assumptions, Lorenz obtains
the maximum specication of the stream function for use with (7.1), namely,
=
A
l
2
cos ly
F
k
2
cos kx
2G
k
2
+ l
2
sin ly sin kx. (7.21)
161
Substituting this streamfunction into the vorticity equation (7.1), utiliz-
ing trigonometric identities, and nally equating coecients of like terms
leads to the following dierential equations for the coecients:
dA
dt
=
_
1
k
2

1
k
2
+ l
2
_
klFG K
1
FG,
dF
dt
=
_
1
l
2

1
k
2
+ l
2
_
klAG K
2
AG, (7.22)
dG
dt
=
1
2
_
1
l
2

1
k
2
_
klAF K
3
AF.
Note that the interaction coecients K
1
, K
2
, and K
3
are constants and
hence, remain the same throughout the period of integration. The set (7.22),
which is analogous to (12.53), can be solved analytically; however when the
spectral technique is applied, say to a hemisphere with real data, the resulting
system of equations is much more complex and must be solved by numerical
methods. If the leapfrog scheme is used here, the numerical integration of
(7.22) would be analogous to (7.13), that is,
A
n+1
= A
n1
+ 2tK
1
F
n
G
n
, (7.23)
etc., where the superscript denotes the time step. As usual, a forward step
must be used for the rst time step from the initial values of the coecients
A
0
, F
0
, and G
0
. To avoid linear instability, t must be a fairly small frac-
tion of the period of the most rapidly oscillating variable. Equation (7.23)
and similar ones for F and G permit the calculation of future values of the
coecient of the series (7.21); thus the prediction of a stream function is
achieved.
162
Chapter 8
Finite Element Methods
8.1 Introduction
In designing a numerical weather prediction model, one of the most funda-
mental aspects is the choice of discretization technique in each of the spatial
dimensions. In the vertical, by far the most popular choice is the nite-
dierence method; while in the horizontal, both nite-dierence and (espe-
cially for global models) spectral methods are widely employed. A third
possibility is the nite element method.
8.2 What is the nite element method?
The essence of the nite element method can be seen by considering various
ways of representing a function f(x) on an interval a x b. In the nite-
dierence method the function is dened only on a set of grid points; i.e.
f(x
j
) is dened for a set of x
j
e[a, b], but there is no explicit information
about how the function behaves between the grid points. In the spectral
method, on the other hand, the function is dened in terms of a nite set of
basis functions:
f(x) =
N

k=0
a
k
e
k
(x) (8.1)
where the basis functions e
k
(x) are global (e.g. Fourier series, or spherical
harmonics for two-dimensional functions on the surface of a sphere), and the
a
k
are the spectral coecients. Equation (8.1) denes f(x) everywhere on
the interval, and the representation is independent of any set of grid points.
163
In the nite-element method, the function is again in terms of a nite set
of basis functions:
f(x) =
N

k=0
a
k
e
k
(8.2)
but this time basis functions e
k
(x) are local, i.e. they are non-zero only on
a small-sub-interval. As in the spectral method, the a
k
are the coecients
of the basis functions, and f(x) is dened everywhere; but as in the nite-
dierence method, there is an underlying mesh of gridpoints (nodes) involved
in the representation.
To clarify this idea we consider the simple choice of linear nite elements.
The interval [a, b] is divided into subintervals by specifying a set of mesh
points, say x
0
, x
1
, . . . , x
N
. The basis function e
k
(x) is dened to be 1 at
x
k
, decreasing linearly to zero at x
k1
and x
k+1
, and to be zero outside the
interval [x
k1
, x
k+1
]. Thus the dening equations are:
e
k
(x) =
x x
k1
x
k
x
k1
, x [x
k1
, x
k
]
=
x
k+1
x
x
k+1
x
k
, x [ x
k
, x
k+1
]
= 0 otherwise
The situation is illustrated in Fig. 8.1; note that the mesh may be non-
uniform.
Figure 8.1: Linear nite element basis functions.
Suppose now that f(x) is given at the set of gridpoints x
k
, as in Fig. 8.2;
how do we determine the coecients a
k
of the basis functions, in order to use
164
the representation given by Eq. (8.2)? The answer is almost obvious: since
e
k
(x) is the only basis function which is non-zero at x
k
, we must have
a
k
= f(x
k
), 0 k N.
Between the grid points, say between x
k
and x
k+1
, just two of the basis
functions (e
k
and e
k+1
) are non-zero; since both are linear, f(x) as dened
by Eq. (8.2) is linear on the subinterval [x
k
, x
k+1
]. Thus the behaviour of
f(x) between grid points is determined simply by linear interpolation. At
this point, it may seem that we have gained very little over the simple grid
point representation of f(x). The benets of the representation in terms of
linear basis functions will become clear in the next section, when we consider
elementary operations with functions.
Finally in this section, we note that nite element basis functions can
be even simpler (piecewise constant on subintervals) or more complicated
(piecewise quadratic, cubic, and so on).
Figure 8.2: Representation of f(x).
165
8.3 Simple operations with linear nite ele-
ments
In this section we show how to perform three simple operations with linear
elements in one dimension: dierentiation, multiplication, and taking second
derivatives.
8.3.1 Dierentiation
Suppose we are given u
i
= u(x
i
) at a set of nodes x
i
(0 i N), and we
wish to dierentiate u to nd v = u
x
. We start by expanding u and v in
terms of the basis functions e
i
(x), as shown in section 2:
u(x) =
N

i=0
u
i
e
i
(x), v(x) =
N

i=0
v
i
e
i
(x),
where the coecients v
i
(0 i N) are the unknowns of our problem. The
series for u can be dierentiated term by term, so that v = u
x
becomes
N

i=0
v
i
e
i
(x) =
N

i=0
u
i
e

i
(x). (8.3)
In Eq. (8.3) we use the notation e

i
(x) to denote the x-derivative of the basis
function, to avoid multiple subscripts. From the denition of the (piecewise
linear) basis functions, e

i
(x) is piecewise constant.
The next step is to apply the Galerkin technique, namely to orthogonalize
the error in (8.3) to the basis: i.e., set
v u
x
, e
k
= 0 for all k(0 k N)
where the inner product, , , is dened by
f(x), g(x) =
x
N
_
x
0
f(x)g(x)dx.
It is easily seen that this is equivalent simply to multiplying both sides of
(8.3) by e
k
(x) and integrating from x
0
to x
N
:
x
N
_
x
0
N

i=0
v
i
e
i
(x)e
k
(x)dx =
x
N
_
x
0
N

i=0
u
i
e

i
(x)e
k
(x)dx. (8.4)
166
Since everything is well-behaved we can exchange the order of the integration
and the summation.
Moreover, since v
i
and u
i
are coecients of basis functions, they are
not functions of x, so we can take them outside the integrals. Hence (8.4)
becomes:
N

i=0
v
i
x
N
_
x
0
e
i
(x)e
k
(x)dx =
N

i=0
u
i
x
N
_
x
0
e

i
(x)e
k
(x)dx. (8.5)
The integrands remaining in (8.5) depend only on the mesh, not on the func-
tions u and v. It is easily seen that for example e
i
(x)e
k
(x) is zero everywhere
unless i = k or i = k 1; in fact it is a simple exercise to show that:
x
N
_
x
0
e
i
(x)e
k
(x)dx =
h
k1
6
for i = k 1
=
h
k1
+h
k
3
for i = k
=
h
k
6
for i = k + 1
= 0 otherwise
where h
k
= x
k+1
x
k
. Similarly,
x
N
_
x
0
e

i
(x)e
k
(x)dx =
1
2
for i = k 1
=
1
2
for i = k + 1
= 0 otherwise.
The outcome is most easily expressed in matrix/vector notation: let
u = (u
0
, u
1
, . . . , u
N
)
T
v = (v
0
, v
1
, . . . , v
N
)
T
,
then using the above results Eq. (8.5) becomes
Pv = P
x
u (8.6)
where P is tridiagonal; P
x
is also tridiagonal, but with zeros on the diagonal
itself. The matrix P is diagonally dominant, and the usual algorithm for
solving tridiagonal systems can safely be used.
It is very instructive to consider (8.6) in the case of a uniform mesh; at
interior points, we have just
1
6
(v
k1
+4v
k
+v
k+1
)
1
2h
(u
k+1
u
k1
), (8.7)
167
where h is the grid length.
If we perform a Taylor series analysis of (8.7), we nd that on a uniform
mesh we have 4th-order accuracy. General nite-element theory tells us that
we only have a right to expect 2nd-order accuracy in the derivative if the
basis functions are linear; here we have an example of superconvergence in
which the (second-order) errors happen to cancel by symmetry.
Notice also that the manipulations leading to (8.6) automatically yield
the appropriate equations for determining v = u
x
at the boundary points x
0
and x
N
in contrast to the nite-dierence case (especially for fourth-order
schemes) where in eect we have to invent additional values beyond the
boundary.
The right-hand side of (8.7) is of course just the usual second-order nite-
dierence approximation to v = u
x
, which is known to be smooth (Fourier
analysis shows that derivative is underestimated, the degree of underestima-
tion depending on the number of grid points per wavelength). The left-hand
side of (8.7) - i.e., the matrix P - is itself a three-point smoothing lter. To
solve (8.6) we multiply the right-hand side by the inverse of P, which is thus
a de-smoothing or sharpening operation. This simple argument provides
some insight into the superiority of linear nite elements over second-order
nite-dierences. In nite element parlance, P is often called the mass ma-
trix.
8.3.2 Multiplication
Suppose now we are given u
i
= u(x
i
) and v
i
= v(x
i
) at a set of nodes
x
i
(0 i N) and we wish to nd the product w = u
v
. Again we expand u
and v in terms of the basis functions e
i
(x), and similarly let
w(x) =
N

i=0
w
i
e
i
(x).
Applying the Galerkin techniques as before,
x
N
_
x
0
N

i=0
w
i
e
i
(x)e
k
(x)dx =
x
N
_
x
0
_
N

i=0
u
i
e
i
(x)
__
N

j=0
v
j
e
j
(x)
_
e
k
(x)dx. (8.8)
and we obtain a matrix problem
Pw = r
168
where P is the same tridiagonal matrix as in the previous sub-section. The
right-hand side can also be expanded in a similar way; it is easily seen for
example that the integral is zero unless both i and j are equal to k 1, k or
k+1. In practice it is more ecient (much more ecient in 2 or 3 dimensions)
to evaluate the integral by numerical quadrature. The right-hand side of (8.8)
is a piecewise cubic, and can be evaluated exactly by using an appropriate
integration formula. In fact we have a choice between Gaussian quadrature
and using Simpsons formula; Staniforth and Beaudoin (1986) show that the
second alternative is twice as ecient than a straightforward term-by-term
evaluation).
Notice that in deriving (8.8) we orthogonalized the error to the basis, so
that the result w(x) is an alias-free representation of the product uv, just as
in the case of the spectral method. The product is fourth-order accurate on
a uniform mesh (Cullen and Morton, 1980).
8.3.3 Second derivatives
Finally, suppose we are given u
i
= u(x
i
) at the nodes x
i
(0 i N), and we
wish to obtain the second derivative v = u
xx
. If we let
v(x) =
N

i=0
v
i
e
i
(x)
and proceed exactly as in Section 8.3.1, we obtain the folllowing analogue of
Eq. (8.4):
x
N
_
x
0
N

i=0
v
i
e
i
(x)e
k
(x)dx =
x
N
_
x
0
N

i=0
u
i
e

i
(x)e
k
(x)(dx). (8.9)
Clearly we are in trouble here, since e

i
(x) is zero everywhere. The trick is
to rewrite right-hand side of (8.9) as
x
N
_
x
0
u
xx
e
k
(x)dx
and then integrate by parts. We obtain
x
N
_
x
0
N

i=0
v
i
e
i
(x)e
k
(x)(dx) = [u
x
e
k
(x)
x
N
x
0

x
N
_
x
0
u
x
e

k
(x)dx. (8.10)
169
Now we can use the expansion of u in terms of the basis functions to replace
u
x
in the integration:
x
N
_
x
0
N

i=0
v
i
e
i
(x)e
k
(x)(dx) = [u
x
e
k
(x)
x
N
x
0

x
N
_
x
0
N

i=0
u
i
e

i
(x)e

k
(x)dx,
and thus
N

i=0
v
i
x
N
_
x
0
e
i
(x)e
k
(x)(dx) = [u
x
e
k
(x)
x
N
x
0

N

i=0
u
i
x
N
_
x
0
e

i
(x)e

k
(x)dx. (8.11)
The left-hand side of (8.11), in matrix/vector notation, is just the familiar Pv
again. The rst term on the right-hand side is zero except at the boundary
points. The second term on the right-hand side contains the easily-evaluated
integrals.
x
N
_
x
0
e

i
(x)e

k
(x)dx =
1
h
k1
for i = k 1
=
_
1
h
k1
+
1
h
k
_
for i = k
=
1
h
k
for i = k + 1
where again h
k
= x
k+1
x
k
.Thus (8.11) has the form
Pv = P
xx
u
where P and P
xx
are both tridiagonal.
On a uniform grid at interior points, (8.11) becomes
1
6
(v
k1
+ 4v
k
+ v
k+1
) =
1
h
2
(u
k1
2u
k
+ u
k+1
) (8.12)
and again the right-hand side is just the usual second-order nite dierence
approximation. Unlike the case of taking a rst derivative, however, inverting
the mass matrix does not provide fourth-order accuracy the accuracy remains
second-order.
There is a way round this problem, provided we are willing to bend the
rules of the Galerkin technique. If we replace the left-hand side of (8.12) by
1
12
(v
k1
+ 10v
k
+ v
k+1
),
in eect using dierent mass matrix, then we recover fourth-order accuracy
on a uniform grid, with no extra computational work.
170
8.4 Eciency, accuracy and conservation
8.4.1 Eciency
In the one-dimensional examples of the previous section, there was only one
way to divide the domain into subintervals. As soon as we move to higher-
dimensional problems, there is a choice. For example, in two dimensions
we might choose to subdivide the domain into triangles (especially if the
boundary of the domain is irregular) or into rectangles. In either case, linear
nite-element basis functions can be dened on the subdivisions. Staniforth
(1987) has powerfully argued the case for using a rectangular mesh if the
geometry of the problem allows it.
The fundamental reason is the cost of inverting the mass matrix P. On
a rectangular mesh, the linear nite-element basis functions are separable:
e
kl
(x, y) = e
k
(x)e
l
(y)
where e
kl
is the basis function centred at the mesh point (k, l). as a result,
the two-dimensional mass matrix can be inverted simply by solving a set of
tridiagonal systems in the x-direction, followed by a set of tridiagonal systems
in the y-direction (or vice-versa). Another way of looking at this is that the
two-dimensional mass matrix is just a tensor product (P
x
P
y
), where P
x
and P
y
are the mass matrices associated with the one-dimensional sets of
basis functions e
k
(x) and e
l
(y). This tensor product algorithm for inverting
the two-dimensional mass matrix was rst demonstrated by Staniforth and
Mitchell (1977).
On a triangular mesh the separability of the basis functions is lost, and
inverting the mass matrix is much more dicult. If the mesh is completely
regular then a reasonably ecient FFT-based direct method could be used,
but generally it is necessary to resort to iterative methods or approximate
inverses (Cullen, 1974b). In engineering applications of the nite element
method, the mass matrix is sometimes simply approximated by the identity
matrix to circumvent this problem (mass lumping), but as we have seen in
Section 8.3.1 this can seriously compromise the accuracy of the solution.
Staniforth (1987) also points out several other eciency tricks which
exist for linear nite elements on rectangles, but which do not carry over
to triangular meshes. In three dimensions, the arguments for trilinear basis
functions on bricks rather than on tetrahedra are even stronger.
171
8.4.2 Accuracy
So far we have considered only methods based on linear nite elements. Is it
worth using higher-order elements? In several respects, the answer seems to
be no. Quadratic elements can be less accurate than linear elements (no su-
perconvergence properties), so the extra expense is not likely to be justied.
Cubic elements do have superconvergence properties and can provide a high
order of accuracy, but they are much more expensive to compute with than
linear elements; also the additional degrees of freedom can result in compu-
tational modes (noise) being excited. Staniforth (1987) puts it succinctly:
the law of diminishing returns seems to apply.
8.4.3 Conservation
Finite-dierence schemes for nonlinear problems are often designed to con-
serve invariants of the original partial dierential equations. In general, such
conservation properties are not maintained in simple linear nite element
schemes. They can be recovered for example by spatial staggering of the ele-
ments, or by choosing dierent orders of element for dierent variables (Lee
et al., Clie, 1981; Girard, 1983; Steppeler, 1987b). The extent to which the
extra computation is worthwhile seems to be rather debatable, and in any
case is certain to be problem-dependent.
172
Bibliography
Ames, W. F. (1969). Numerical methods for partial dierential equations.
Barnes and Noble Inc., New York.
Arakawa, A. (1966). Computational design for long-term numerical inte-
grations of the equations of atmospheric motion. J. Comput. Phys.,
1:119143.
Arakawa, A. (1972). Design of the ucla general circulation model. Numer-
ical simulation of weather and climate, Dept of Meteorology, Univ. of
California, Los Angeles.
Arakawa, A. et al. (1974). The ucla atmospheric general circulation model.
Technical report, University of California, Los Angeles.
Arakawa, A. and Lamb (1976). Computational design of the ucla general cir-
culation model. In Methods in Computational Physics. Academic Press,
New York.
Asselin (1972). Frequency lter for time integrations. Mon. Wea. Rev.,
100:487490.
Bjerknes, V. (1904). Das problem von der wettervorhersage, betrachtet vom
standpunkt der mechanik und der physik. Meteor. Z., 21:17.
Boris, J. P. and Book, D. L. (1973). Flux-corrected transport I: SHASTA a
uid transport algorithm that works. J. Comput. Phys., 11:3869.
Charney, J., Fjortoft, R., and von Neumann, J. (1950). Numerical integration
of the barotropic vorticity equations. Tellus, 2:237254.
Charney, J. G. (1966). Some remaining problems in numerical weather pre-
diction. In Advances in numerical weather prediction, pages 6170, Hart-
ford, Conn. Travelers Research Center, Inc.
173
Courant, R., Friedrichs, K., and Lewy, H. (1928). uber die partiellen dieren-
zengleichungen der mathematischen physik. Mathematische Annalen,
100:3274.
Cullen, M. J. P. (1974). Integration of the primitive equations on a shere
using the nite element method. Quart. J. Roy. Meteor. Soc., 100:555
562.
Dorr, F. W. (1970). The direct solution of the discrete poisson equation on
a rectangle. SIAM Review, 12:248263.
ECMWF (1991). Numerical methods in atmospheric models. In Seminar
proceedings, volume 1. European Centre for Medium-Range Weather
Forecasts.
Eliassen, A. (1956). A procedure for numerical integration of the primitive
equations of the two-parameter model of the atmosphere. Sci. Rep 4,
Dept. of Meteorology, UCLA.
Fjortoft, R. (1953). On the changes in the spectral distribution of kinetic
energy for two-dimensional, non-divergent ows. Tellus, 5:225230.
Frankel (1950). Convergence rates of iterative treatments of partial dieren-
tial equations. Math tables and other aids to computation, 4:6575.
Gillespie, R. P. (1955). Integration. Oliver and Boyd, Edinburgh.
Kasahara, A. (1969). Simulation of the earths atmosphere. Ncar manuscript
69-27, National Center for Atmospheric Research, Boulder, Colo.
Kwizak, M. and Robert, A. (1971). A semi implicit scheme for grid point
atmospheric models of the primitive equations. Mon. Wea. Rev., 99:32
36.
Marchuk (1967). Numerical methods in weather prediction. Academic Press.
Mesinger (1973). A method for construction of second-order accuracy dif-
ference schemes permitting no false two-grid-interval wave in the height
eld. Tellus, 25:444458.
Mesinger, F. and Arakawa, A. (1976). Numerical methods used in atmo-
spheric models. In GARP Publications Series, number 14, page 64 pp.
WMO/ICSU Joint Organizing Committee.
174
Miyakoda (1962). Contribution to the numerical weather prediction com-
putation with nite dierence. Japanese J. of Geophysics, 3:75190.
Orszag (1971). On the elimination of aliasing in nite-dierence schemes by
ltering high wave number components. J. Atmos. Sci., 28:1074 pp.
Phillips, N. A. (1956). The general circulation of the atmosphere: a numerical
experiment. Quart. J. Roy. Meteor. Soc., 82:123 pp.
Richardson, L. F. (1922). Weather prediction by numerical process. Cam-
bridge Univ. Press, London. Reprinted by Dover.
Roache, P. J. (1972). Computational uid dynamics. Hermosa publishers,
Albuquerque, New Mexico.
Robert, A., J. (1966). The integration of a low order spectral form of the
primitive meteorological equations. J. Meteor. Soc. Japan, 44:237245.
Silberman, I. (1954). Planetary waves in the atmosphere. J. Meteor., 11:27
34.
Smolarkiewicz, P. K. (1984). A fully multidimensional positive denite advec-
tion transport algorithm with small implicit diusion. J. Comp. Phys.,
54:325362.
Smolarkiewicz, P. K. and Clark (1986). The multidimensional positive def-
inite advection transport algorithm: Further development and applica-
tions. T. J. Comp. Phys., 86:355375.
Smolarkiewicz, P. K. and Grell, G. A. (1992). A class of monotone interpo-
lation schemes. J. Comp. Phys., 101:431440.
Smolarkiewicz, P. K. and Rasch, P. J. (1990). Monotone advection on the
sphere: An eulerian versus semi-lagrangian approach. J. Atmos. Sci.,
48:793810.
Staniforth and Mitchell (1977). A semi-implicit nite-element barotropic
model. Mon. Wea. Rev., 105:154169.
Staniforth, A. (1987). Review: Formulating ecient nite-element codes for
ows in regular domains. Int. J. Num. Meth. Fluids, 7:116.
Staniforth, A. and Cote, J. (1991). Semi-lagrangian integration schemes for
atmospheric models a review. Mon. Wea. Rev., 119:22062223.
175
Steppeler, J. (1987a). Energy conserving galerkin nite element schemes
for the primitive equations of numerical weather prediction. J. Comp.
Phys., 69:258264.
Steppeler, J. (1987b). Quadratic galerkin nite element schemes for the
vertical discretization of numerical forecast models. Mon. Wea. Rev.,
115:15751588.
Wachspress, E. L. (1966). Iterative solution of elliptic systems. Prentice-Hall,
Englewood Clis, New Jersey.
Wurtele (1961). On the problem of truncation error. Tellus, 13:379391.
Young, D. (1954). Iterative methods for solving partial dierential equations
of elliptic type. Transactions of the American Mathematical Society,
76:92111.
Zalesak, S. T. (1979). Fully multidimensional ux-corrected transport algo-
rithms for uids. J. Comp. Phys., 31:335362.
176