Chapter 2

2D Continuous Space Signals and Systems

Contents
Introduction . . . . . . . . . . . . . . . . . . . . . . . . .
Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Classification of signals/images . . . . . . . . . . . . .
Periodic images . . . . . . . . . . . . . . . . . . . . .
Simple image transformations . . . . . . . . . . . . .
Simple image classes . . . . . . . . . . . . . . . . . .
Important 2D signals . . . . . . . . . . . . . . . . . .
2D Dirac impulse . . . . . . . . . . . . . . . . . . . .
Local image phase . . . . . . . . . . . . . . . . . . .
Systems in 2D (Imaging systems) . . . . . . . . . . . . . .
Classification of systems . . . . . . . . . . . . . . . .
Impulse response . . . . . . . . . . . . . . . . . . . .
Shift invariance . . . . . . . . . . . . . . . . . . . . .
Convolution . . . . . . . . . . . . . . . . . . . . . . .
Magnification and LSI systems . . . . . . . . . . . . .
Representation of signals by orthogonal bases . . . . . . .
Fourier series . . . . . . . . . . . . . . . . . . . . . .
Fourier transforms . . . . . . . . . . . . . . . . . . . . .
Two-dimensional Fourier transform . . . . . . . . . . .
Relation to 1D Fourier transform . . . . . . . . . . . .
2D Fourier transform properties . . . . . . . . . . . .
Properties that are analogous to 1DFT properties . . . .
Generalized Fourier transforms . . . . . . . . . . . . .
Properties that are unique to the 2D FT . . . . . . . . .
Hankel transform . . . . . . . . . . . . . . . . . . . .
Other 2D transforms . . . . . . . . . . . . . . . . . .
Bandwidth and time-bandwidth products . . . . . . . .
Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . .
Signal reconstruction/interpolation . . . . . . . . . . .
Numerical evaluation of Fourier transforms via the FFT
Summary . . . . . . . . . . . . . . . . . . . . . . . .

2.1

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

2.2
2.3
2.3
2.5
2.8
2.11
2.13
2.15
2.19
2.20
2.21
2.25
2.28
2.29
2.32
2.33
2.40
2.44
2.44
2.45
2.47
2.47
2.49
2.51
2.51
2.55
2.56
2.57
2.60
2.64
2.67

c J. Fessler, September 7, 2009, 22:51 (student version)

2.2

Introduction
Many image processing papers address the following scenario: one is given a digital image and one would like to process that
image to form another image, or to apply an algorithm for extracting some information from it. However, except for computergenerated images (like digital animation), generally digital images originate from some (continuous) real-world scene, recorded by
an imaging system that includes analog components like lenses. For a complete view of image processing, one must be familiar
with the basic principles of such imaging systems. Many such systems are (approximately) linear and shift-invariant, so many of
the classical signals and systems concepts apply, albeit generalized to 2D (or 3D) rather than in 1D.
So our first goal is a brief review of 2D continuous or analog signals and systems. (Essentially EECS 216|306|316 in 2D).
Overview
Definitions
Simple signal transformations (shift, scale). New: rotation, warping.
Signal classes (even, odd, periodic, ...). New: circular symmetry, separable.
Important signals (impulse functions, sinusoids)
System classes: (linear, stable, invertible, causal, static, shift-invariant). New: rotation invariant.
Impulse response (point-spread function)
Linear shift-invariant (LSI) systems
Superposition integral, convolution, and properties. New: magnification
LSI system properties in terms of PSF
Orthogonal signal representation
Fourier series / eigenfunctions / properties
2D Fourier transform
2D FT properties (convolution etc.). New: rotation, separability, circular symmetry
2D sampling / recovery via interpolation. New: non-Cartesian sampling.
What major 1D topics are absent?
??

??

This review will emphasize the similarities and differences between the 1D and 2D formulae. This treatment serves to reinforce
signals and systems concepts learned previously, while simultaneously introducing a few new concepts and properties that are
unique to 2D (and higher) problems.
References: Bracewells two books [1, 2].

c J. Fessler, September 7, 2009, 22:51 (student version)

2.3

Signals

Classification of signals/images
Mathematically, a signal is a function of one or more independent variables. An image is a function of two (or perhaps 3) variables.
A function is a mapping from one set of values, called the domain, to another set called the range. When introducing a new
function, such as one called f , one generally writes f : D 7 R and then describes the domain D, the range R, and the value
f (x) R that the function assigns to an arbitrary member x of the domain D.
To say that an image is a function of two or three variables is a rather broad and vague definition. To get more specific we make
some classifications!
Domain dimension
One way to classify signals is by the dimension of the domain of the function, i.e., how many arguments the function has.
A one-dimensional (1D) signal is a function of a single variable, e.g., time or elevation above sea level. In this case the range
is a subset of R = (, ), the set of all real numbers.
An M -dimensional signal is a function of M independent variables. In this case, the range is a subset of RM = (, )M ,
the set of all M -tuples of real numbers.
Example. A sequence of black and white (really: grayscale) TV pictures I(x, y, t) is a scalar-valued function of three arguments:
spatial coordinates x and y, and time index t, so it is a 3D signal.
We focus primarily on two-dimensional (2D) signals, i.e., images, generally considering the independent variables to be spatial
position (x, y).
Range dimension
Another way to classify signals is by the dimension of the range of the function, i.e., the dimension of the space of values the
function can take.
scalar or single-channel signals
multichannel signals
Example. A BW TV picture is scalar valued, whereas a color TV picture can be described as a three-channel signal, where the
components of the signal represent red, green, and blue (RGB). Here ~g(x, y) = [gR (x, y) gG (x, y) gB (x, y)]. We will consider
both scalar (grayscale) images and multichannel (e.g., color) images.
Discrete- vs. Continuous-Space Images
Continuous-space images or analog images
Defined for all locations/coordinates (x, y) R2 = (, )2 , or at least in some (usually rectangular) subset of R2 , i.e., the
domain is this subset.
2
2
Example. g(x, y) = e(x +y ) , < x, y < .
Discrete-space images
Defined only for integer locations/coordinates (n, m) Z2 , or at least in some (usually rectangular) subset of Z2 , i.e., the
domain is this subset. Note that Z , {. . . , 2, 1, 0, 1, 2, . . .} . Often, g[n, m] denotes the image value at some discrete
location (xn , ym ). The discrete locations are called pixels. The image value at this location is called a pixel value. For
example, it often happens that xn = nX , and ym = mY , where X and Y are horizontal and vertical pixel spacings. In
other words, the pixels form a rectangular grid. However, this is not the only possibility. (Note that it is incorrect to think of
g[n, m] as being zero for non-integer values of (n, m).)
Some of the common ways in which discrete-space images arise are:
Sampling a continuous image at discrete positions
Integrating a continuous image over each of a set of pixel regions
Processing another discrete-space image
Focus on continuous-space images
As this chapter focuses on continuous-space images, from now on, unless stated otherwise, all images are presumed to be
continuous-space images.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.4

Notation
We will (usually) use parentheses for the arguments of continuous-space images, e.g., g(x, y), and square brackets for the
arguments of discrete-space images, e.g., g[n, m]. When it is helpful to further distinguish the two, we will add the subscripts a
and d, as in ga (x, y) and gd [n, m].
In these notes, as throughout much of engineering, when we write g(x, y), there are two possible interpretations: g(x, y) might
mean the value of the image at location (x, y) or it might refer to the entire image. The correct interpretation will usually be
clear from context. Another approach is to simply write g or g(, ) when referring to the entire image.
Image support
The domain of an image is commonly called its support or support region. An image is said to have finite or bounded support
if there is a number S < such that if (x, y) is in the support/domain, then |x| S and |y| S.
Value characteristics
A continuous-valued image can take any value in some continuous interval.
A discrete-valued image only takes values from a countable set of possible values, e.g., a finite set as in {0, 1, 2, . . . , 255} for
an 8-bit image, or a countably infinite set, as in Z.
Typically arises from quantization of a continuous-valued image (e.g., A-to-D conversion), or from counting (e.g., nuclear
imaging).
A binary image only takes two values, typically 0 or 1.
Typically arises from thresholding a continuous-valued or discrete-valued image, or in the formation of halftone grayscale
images.
For generality, we will consider both real-valued and complex-valued images. A real-valued image has f (x, y) R. A
complex-valued
image has f (x, y) C, where C denotes the set of all complex numbers of the form u + v, with u, v R and

= 1.
Some image values have meaningful physical units, such as lumens. But often the image values are just relative intensities.
Deterministic vs Random images
Whether a given set of images is considered to be deterministic or random is a matter of the philosophy of whatever mathematical
or statistical models that we assume. We will consider both deterministic and stochastic models for images.
Energy and power images
The energy of an image g(x, y) with R2 as its support is defined as
Z Z
E,
| g(x, y) |2 dx dy .

The average power of an image g(x, y) with R2 as its support is defined as

Z T Z T
1
P , lim
| g(x, y) |2 dx dy
T (2T )2 T T
ZZ
1
| g(x, y) |2 dx dy .
= lim
R R2
2
2
{(x,y) : x +y <R}

Using circles of radius T and 2T that inscribe and surround the square [T, T ] [T, T ], one can easily show that the above
two definitions of power are equivalent.
If E is finite (E < ), then g(x, y) is called an energy image, or is said to be square integrable, and P = 0.
If E is infinite, then P can be either finite or infinite. If P is finite and nonzero, then g(x, y) is called a power image.
Some images are neither energy images nor power images, such as g(x, y) = x2 , for which E = and P = . Such signals
are generally of little practical engineering importance.
When an image has finite support, the integrals defining the energy and power are taken only over the support region and instead
of taking a limit in the definition of power, the integral is simply divided by the area of the support. In any event, power is
energy per unit area.
Using notation from functional analysis, let L2 (S) denote the set of all real-valued functions (e.g., images) with support S and
finite energy. For example, we often consider the L2 (R2 ), the collection of all finite energy images over the Cartesian plane.
For complex-valued images, | g(x, y) |2 = g(x, y) g (x, y), where g (x, y) denotes the complex conjugate of g(x, y), where
(u + v) = u v.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.5

Periodic images
A 1D signal g(t) is called periodic with period T > 0 iff g(t) = g(t + T ) for all t R.
It would be tempting (but incorrect) to say that a 2D signal (image) g(x, y) is periodic if g(x + TX , y + TY ) = g(x, y), x, y R
for some TX , TY > 0. However, aperiodic signals such as g(x, y) = e|xy| satisfy this condition (for any TX = TY > 0). So
we need a better condition.
An image g(x, y) is called periodic with period (TX , TY ) > 0 iff
g(x + TX , y) = g(x, y + TY ) = g(x, y),

x, y R.

(2.1)

Otherwise g(x, y) is called aperiodic.

Ordinarily, periodic images are presumed to have support equal to R2 , i.e., infinite support. One could also consider an image
with finite support to be periodic, if the above periodicity property holds with (x, y) limited to the support.
If g(x, y) is periodic with period (TX , TY ), then
g(x, y) = g(x + TX , y) = g((x + TX ) + TX , y) = g(x + 2TX , y) = = g(x + nTX , y + mTY ),

n, m Z.

The following fact is readily verified.

An image that is periodic with period (TX , TY ), is also periodic with period (nTX , mTY ), for any integers n, m 6= 0.
Example. The image
g(x, y) = sin(2[ux + vy])
is periodic with period (TX , TY ) = (k/u, l/v) for any choices of nonzero integers k and l.
If one examines a slice through the image at angle along the line described parametrically by (x(t), y(t)) = (t cos , t sin ) ,
with t ranging from to , then one obtains the 1D signal
g (t) = sin(2[u cos + v sin ]t),
which is periodic with period 1/(u cos + v sin ). Notice how slices in different directions yield 1D signals with different periods.
Example. The signal g(x, y) = e|sin(2x/5) + sin(2y/3)| is periodic with period (5, 3). The following figure shows this signal using
a (hopefully self explanatory) gray scale display.

A periodic 2D signal g(x,y)

12

0
0

10
x

15

c J. Fessler, September 7, 2009, 22:51 (student version)

Fact. Periodic signals are power signals with P =

interval of length T .

1
TX TY

2.6

hTX i hTY i

|g(x, y)| dx dy, where

hT i

denotes an integral over any

Fundamental period?
In 1D, at this point we usually define the fundamental period to be the smallest value of the period satisfying (2.1).
How do we define smallest value in 2D?
As we will see, there is not a unique way to define a smallest period.
Suppose an image g(x, y) is periodic with some period (TX , TY ) > 0, not necessarily the fundamental period. Define TX,0 to be
the smallest value of T for which (T, TY ) is a period of g(x, y). Define TY,0 to be the smallest value of T for which (TX,0 , T ) is
a period of g(x, y). Then it would be natural to define (TX,0 , TY,0 ) as the fundamental period of g(x, y). But such a definition
would only be sensible if it were independent of whether we minimized over x or y first.
Is the preceding definition invariant to ordering of minimization?
Another possible approach is to define the fundamental period to be the smallest period of any 1D signal produced by a slice
1
through the
image. For example, for the sinusoidal signal above, the slice with smallest period has angle = tan (v/u) and
period 1/ u2 + v 2 . However, it is not clear how this definition, which gives a scalar quantity, would correspond to our original
definition of period, which involves a tuple.
Yet another approach is to consider any undominated period (TX , TY ) to be fundamental, where undominated means there exists
no other period (TX , TY ) such that TX TX and TY TY . With this definition, there may be a set of fundamental periods, rather
than just one.
Lim [3, p. 51] discusses a more efficient definition of period in a problem at the end of the first chapter. We can say an image is
periodic iff
g(x, y) = g(x + T11 , y + T12 ) = g(x + T21 , y + T22 ),
x, y.
Although Lim does not state this, presumably one must require that T11 > 0, T22 > 0, T21 6= T11 , and T12 6= T22 . With this
definition, apparently one would like to minimize |T11 T22 T12 T21 |. Is there a unique minimizer?
The bottom line is that there seems not to be a canonical definition of fundamental period for 2D (or higher) signals. Fortunately,
defining the fundamental period is not terribly important, because the main place we use the concept of period is with the Fourier
series (FS), and for FS it is not essential to use the fundamental periodany period will suffice.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.7

Periodic functions and modulo

Suppose f (x) is a 1D signal with support [a, b] and we define a periodic signal using superimposed replicates as follows:
g(x) =

k=

f (x k).

If b a so there is no overlap of the replicates, then an alternative expression that is more convenient for numerical implementation is:
g(x) = f (mod(x a, ) + a),
where the modulo function is defined (for y 6= 0) by
mod(x, y) , x x/y y,
and the floor function t denotes the largest integer that is t. Note that mod(x, y) [0, y).
Caution: M ATLABs rem function differs from mod for negative arguments.
Here is example M ATLAB code, for a case where = 3.
% fig_mod1.m illustrate periodic functions via modulo
% shifted triangle with support [s-1,s+1]
f = @(x,s) (abs(x-s) <= 1) .* (1-abs(x-s));
t = linspace(-7,7,141);
s = 5; % arbitrary shift parameter for illustration
a = s-1; % left edge of support
tm = mod(t-a, 3) + a;
clf, subplot(211), plot(t, f(t,s)), ylabel f(t), title Original signal
axis tight, xtick(-7:7)
subplot(212), plot(t, f(tm,s)), title(Periodic extension with \Delta=3)
xlabel t, ylabel f(4 + mod(t-4, 3)), axis tight, xtick(-7:7)
savefig fig_mod1

Original signal

f(t)

0.5

0
7 6 5 4 3 2 1

f(4 + mod(t4, 3))

Periodic extension with =3

1

0.5

0
7 6 5 4 3 2 1

0
t

c J. Fessler, September 7, 2009, 22:51 (student version)

2.8

Simple image transformations

Now we describe some simple mathematical operations, i.e., transformations, that transform an image into another image. Some
of these operate directly on the value of an image, while others operate directly on the independent space variables x and y, and
only indirectly on the image values. In other words, some operate on the range and some on the domain of the signal.
Amplitude transformations
Affine amplitude transformation:

g(x, y) = a f (x, y) +b.

Nonlinear amplitude transformation:

a f (x, y) +b > gmax

255,
g(x, y) =
a f (x, y) +b, gmin a f (x, y) +b gmax

0,
a f (x, y) +b < gmin

This is often used to map an image to the range [0, 255] for display. Performed with M ATLABs imagesc function.
Spatial transformations
Space shifting or translation (e.g., camera panning):
g(x, y) = f (x x0 , y y0 )
Mirroring (space reversal):

g(x, y) = f (x, y)

(M ATLABs transpose, fliplr, and flipud commands)

Space scaling (zooming, magnification, minification, or shrinking):

g(x, y) = f (ax, ay)

Ordinarily, we take a > 0, because if a < 0, then mirroring occurs in addition to space scaling.
Zooming/magnification corresponds to a < 1, and shrinking/minification corresponds to a > 1.
Rotation (counterclockwise by radians):

g(x, y) = f x cos + y sin , x sin + y cos .

{z
} |
{z
}
|
x

(2.2)

This is performed (in discrete-space) by M ATLABs imrotate function.

This relationship is perhaps easiest understood in polar coordinates. We write an image g(x, y) in polar coordinates as
g (r, ) = g(r cos , r sin ),
in other words: (x, y) = (r cos , r sin ). Usually we drop the subscript and just write g(r, ).
In polar coordinates, image rotation is simply:
g (r, ) = f (r, ).

(2.3)

Exercise. Verify that (2.2) and (2.3) are equivalent.

General spatial transformation (e.g., morphing or warping an image)

g(x, y) = f (TX (x, y), TY (x, y)),

where here TX : R2 R and likewise for TY . Used, for example, in video coding to approximately match one frame to the next
account for scene motion between frames.
Performed with M ATLABs interp2 or griddata functions.
One nice method for spatial transformations is parameterized using thin-plate splines.
There are numerous additional transformations
that will constitute a considerable fraction of our efforts in this course, e.g.,
r
2

2 

gradient magnitude operation: g(x, y) =

+ y f (x, y) , histogram equalization, etc.
x f (x, y)

c J. Fessler, September 7, 2009, 22:51 (student version)

2.9

Spatial shift:

Original:
f(x,y)

g(x,y) = f(x+30, y40)

1

128

128

128

128

0
128

0
x

128

128

0
x

128

Mirror in y:

Affine amplitude transform:

g(x,y) = f(x,y)

2 + 3 f(x,y)

128

128

128

128

2
128

0
x

128

128

0
x

128

Example. The figures above and below illustrate several of the

amplitude and spatial transformations.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.10

Rotate:

Magnify:

g(x,y) = f(x cos(/8) + y sin(/8), x sin(/8) + y cos(/8))

g(x,y) = f(x/1.3, y/1.3)

128

128

128

128

6.19e05

0.000547
128

0
x

128

128

0
x

128

Warp:

Minify:

g(x,y) = f(2x, 2y)

g(x,y) = f(x, 128 (y/128) )

1

128

128

128

128

0.00203
128

0
x

128

0
128

0
x

128

c J. Fessler, September 7, 2009, 22:51 (student version)

2.11

Simple image classes

Symmetric images
There are several ways to define symmetry properties of 2D images. For example, in examining brain images, we expect left-right
symmetry. For establishing properties of the 2D FT, however, the most useful symmetry properties are the following.
g(x, y) has even symmetry iff g(x, y) = g(x, y), x, y
g(x, y) has odd symmetry iff g(x, y) = g(x, y), x, y
g(x, y) is Hermitian symmetric iff g(x, y) = g (x, y), x, y
We can decompose any image g(x, y) into its even and odd parts, just as in the 1D case.
g(x, y) = Ev {g(x, y)} + Od {g(x, y)}
Ev {g(x, y)} ,

1
1
[g(x, y) + g(x, y)] , Od {g(x, y)} , [g(x, y) g(x, y)] .
2
2

An alternate way to describe a symmetry property is to describe a simple image operation and then to consider the set of images
that are left unchanged for by the operation. For example, an image has even symmetry iff mirroring (i.e., space reversal) produces
the same image. That is, an image has even symmetry iff it is invariant to mirroring. As another example, an image has odd
symmetry iff the transformation g(x, y) = f (x, y) produces the same image.
Circular symmetry

(No 1D analog!)

Another symmetry condition is particularly important for imaging problems.

We say an image g(x, y) is circularly symmetric or radially symmetric iff it is invariant to any rotation, or equivalently, iff
p
p
g(x, y) = g(s, t) whenever x2 + y 2 = s2 + t2

or equivalently, iff

g(x, y) = gR

p

x2 + y 2 ,

x, y

for some 1D function gR (). Often we just write g(r) rather

pthan gR (r), and the reader must remember that if we consider r to be a
scalar, then g(r) is a 1D function, but if we consider r = x2 + y 2 , where x and y are both variables, then g(r) is a 2D function.
Example: see pillbox and 2D gaussian images below.

Question: How would we define the radially symmetric component of an image?

Question: Can we decompose any image into a radially symmetric component plus some other meaningful component?
Rotational symmetry
An image g(x, y) is said to have n-fold rotational symmetry iff it is invariant to a rotation by angle 2/n, equivalently, iff
g(x, y) = g(cos(2/n) x sin(2/n) y, sin(2/n) x + cos(2/n) y),

x, y.

Example: rect2 (x, y) has four-fold rotational symmetry.

Caution: All circularly symmetric images have n-fold rotational symmetry for every given n N , {1, 2, . . .}, but the reverse is
not true.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.12

Separable images

(No 1D analog!)

For convolution, Fourier transforms, and other analyses, we often simplify results by exploiting separability in one of the two
following forms.
An image g(x, y) is called separable in Cartesian coordinates (or just separable) iff
g(x, y) = gX (x)gY (y),
for some 1D functions gX () and gY ().
An image g(x, y) is called separable in polar coordinates (or polar separable) iff
p

g(x, y) = gR
x2 + y 2 g (tan1 (y/x)),

or equivalently: iff g(r, ) = gR (r) g (), for some 1D functions gR () and g (), where r =

Occasionally we might also have use for functions that are additively separable:

p
x2 + y 2 , = tan1 (y/x).

g(x, y) = gX (x) + gY (y).

Exercise. What images are both (Cartesian) separable and radially symmetric? (Be as general as possible.)
Exercise. What images are both additively separable and radially symmetric?
Exercise. Does circular symmetry imply polar separability, or vice versa?

??

c J. Fessler, September 7, 2009, 22:51 (student version)

2.13

Important 2D signals
The pillbox image or disk image is


p

1, |t| < 1/2
2
2
g(x, y) , rect
x + y = rect(r), where rect(t) ,
0, otherwise.

A closely-related image is the circ function, which is unity over the unit disk:

r
p
x2 + y 2 = circ(r) = rect
.
g(x, y) = circ
2

(2.4)

(2.5)

A box image or 2D rect is defined by

g(x, y) = rect2

x x0 y y0
,
wX
wY





x x0
y y0
, rect
rect
.
wX
wY

This function is unity within a box of size wX by wY centered at (x0 , y0 ), and is zero elsewhere.
The 2D gaussian image is

g(x, y) = er = ex ey .

(2.6)

The class of 2D sinusoidal images is given by

g(x, y) = cos(2[ux + vy] + ), where u, v R+ = (0, ) and R.
u and v are called the spatial frequencies of the sinusoid. What are their units?

??

The class of 2D complex exponential signals is given by

g(x, y) = e2(ux+vy) , where u, v R and C.
These are complex signals, so we visualize them by displaying their real and imaginary parts, which are sinusoidal signals.

1
1

0.5

0.5

0.5

1
1

0.5

0.5

Im[g(x,y)] for fX=2 fY=0

Im[g(x,y)] for fX=2 fY=1

0.5

0.5

0.5

0.5

0.5

0.5

0.5

0
x

0.5

0.5

0.5

0
x

0.5

What kind of symmetry does a 2D complex exponential signal have?

Which of the above are radially symmetric?
Which of the above are separable?

Re[g(x,y)] for fX=2 fY=1

0.5

0.5

Re[g(x,y)] for fX=2 fY=0

0.5

0.5

0.5

0
x

0.5

0.5

0
x

0.5

??

??

??

Is the 2D function rect(r/3 1) radially symmetric?

??

Displaying images in M ATLAB

To efficiently generate (samples of) 2D signals in M ATLAB and to display them as images, the ndgrid (or perhaps meshgrid)
commands in conjunction with imagesc are very useful. Study the following example.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.14

Profile
1
1
0.8

0.5

0.6

g (x,0)

0.8
0.6

0
0.4

0.4

0.5
0.2
1
1

0.5

0.5

0.2

0
1

0.5

0.5

0.5

0.5

0.5

0
x

0.5

1
1
0.8

0.5

0.6

0.6

g (x,0)

0.8

0.4

0.4

0.5
0.2
1
1

0.5

0.5

0.2

0
1

1
1
0.8

0.5

0.6

0.6
0

g (x,0)

0.8

0.4

0.4

0.5
0.2
1
1

0.5

0
x

0.5

0.2
0
1

% fig_signal2a.m illustrate some 2d signals

nx = 32; x = [-nx/2:nx/2-1]/nx * 2;
ny = nx; y = [-ny/2:ny/2-1]/ny * 2;
[xx yy] = ndgrid(x,y); % useful trick for
g1 = double(sqrt(xx.2 + yy.2) < 1/2);
g2 = (abs(xx) < 1/2) .* (abs(yy) < 1/2);
g3 = exp(-7 * (xx.2+yy.2));
clf, clim = [0 1.01]; % grayscale limits
colormap(1-gray(256))
subplot(321), imagesc(x,y,g1, clim), axis
subplot(323), imagesc(x,y,g2, clim), axis
subplot(325), imagesc(x,y,g3, clim), axis

x,y grid!

xy square, colorbar
xy square, colorbar
xy square, colorbar, xlabel x, ylabel y

subplot(322), plot(x, g1(:,ny/2+1)), axis([-1 1 0 1.1])

xlabel , ylabel g_1(x,0), title(Profile)
subplot(324), plot(x, g2(:,ny/2+1)), axis([-1 1 0 1.1])
xlabel , ylabel g_2(x,0)
subplot(326), plot(x, g3(:,ny/2+1)), axis([-1 1 0 1.1])
xlabel x, ylabel g_3(x,0)
orient tall, print(fig_signal2a, -deps)
clf, c = ones(size(g1)); % a single mesh color
subplot(321), mesh(xx, yy, g1, c), xlabel x, ylabel y
subplot(323), mesh(xx, yy, g2, c), xlabel x, ylabel y
subplot(325), mesh(xx, yy, g3, c), xlabel x, ylabel y
orient tall, print(fig_signal2b, -deps)
Figure 2.1: Illustration of simple 2D images, displayed via grayscale, profiles, i.e., g(x, 0), and mesh plots, and the M ATLAB code
that generated those plots.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.15

2D Dirac impulse
The 1D Dirac impulse (t) plays a central role in the analysis of 1D systems.
The 2D Dirac impulse 2 (x, y) has an equally important role for analysis of 2D systems.
Although the Dirac impulse is sometimes called the Dirac delta function in the engineering literature, it is not really a function.
In the branch of mathematics that deals rigorously with such entities, they are called distributions.
We define a Dirac impulse by specifying its properties, rather than by giving an explicit formula for 2 (x, y) itself. Basically, it
is defined by what it does as an integrand, although there are also a couple of other useful properties.
Major properties

Sifting property

R R

g(x, y) 2 (x x0 , y y0 ) dx dy = g(x0 , y0 ) if g(x, y) is continuous at (x0 , y0 ).

This is by far the most important property, and it is the key to simplifying the analysis of linear systems.
More generally, the above property holds if the integration limits are replaced by an open set that contains the point
(x0 , y0 ), e.g., the set (x0 , x0 + ) (y0 , y0 + ) for some > 0.
The sifting integral is zero if the integral is over any closed set that does not contain the point (x0 , y0 ).
The integral is undefined if the point (x0 , y0 ) is on the boundary of the region of integration.
Sampling property g(x, y) 2 (x x0 , y y0 ) = g(x0 , y0 ) 2 (x x0 , y y0 ) if g(x, y) is continuous at (x0 , y0 ).
Minor properties (theseRR
all follow from the sifting property!)
unit area property
2 (x, y) dx dy = 1
(The integral can be over any open set containing the origin.)
scaling property ??
symmetry property 2 (x, y) = 2 (x, y)
support property 2 (x x0 , y y0 ) = 0 if x 6= x0 or y 6= y0 .
2
relationship with unit step function: 2 (x, y) = x y u(x)u(y)
Separability: 2 (x, y) = (x) (y) (not essential)
No ordinary function can have all these properties. For example, any ordinary function that is zero everywhere except at (x, y) =
(0, 0) will integrate to zero, rather than to unity.
However, there are many functions approximate the properties of a Dirac impulse, for example, many tall and narrow pulse
functions that integrate to unity. The approximation improves as the pulse becomes taller and narrower. As an example, for
intuition one can express the Dirac impulse as the limit of a unit-volume 2D function, e.g., a 2D Gaussian:
2 (x, y) = lim 2 (x, y; ),

2 (x, y; ) = 2 e

(x2 +y 2 )

The preceding formula is not really rigorous. However, what we really mean by the above equality is that for points (x, y) at which
g(x, y) is continuous, in the limit, integrating g(x , y ) times 2 (x , y ; ) over an interval containing (x, y) has the same effect as
integrating g(x , y ) times 2 (x , y ):
Z Z
g(x , y ) 2 (x x, y y; ) dx dy = g(x, y) .
(2.7)
lim

In other words, 2 (x, y) is just an idealization of a tall narrow pulse like 2 (x, y; ). The type of limit shown above can be studied
rigorously, and holds for many pulse-like functions 2 (x, y; ), not just for Gaussians.
The Gaussian choice is useful when (many) derivatives of 2 (x, y) are needed. More frequently, no derivatives are needed, and the
rect choice 2 (x, y; ) = 2 rect2 (x, y) is more convenient.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.16

Sifting property
The sifting property written above can also be viewed as inducing a decomposition of g(x, y) into (weighted) elementary functions,
namely off-center Dirac impulses. Define
2 (x, y; x , y ) , 2 (x x , y y ),
which one can visualize as follows.

y
Impulse

Impulse

The usual 2D Dirac impulse located at the origin is simply

2 (x, y; 0, 0) = 2 (x, y),
whereas 2 (x, y; x , y ) is a Dirac impulse located at (x , y ).
We can then rewrite the sifting property as follows:
g(x, y) =

ZZ

g(x , y ) 2 (x, y; x , y ) dx dy . (Picture)

Comb functions
The 1D comb or shah or impulse train function is defined as follows:

comb(x) ,

n=

(x n) .

...
1

(2.8)
x

We often need non-integer spacing:

 



X
X
x
x
comb
=

n = TX
(x nTX ) .
TX
TX
n=
n=

TX TY

n= m=

2 (x nTX , y mTY ) .

These functions are useful for analyzing sampling.

Dirac impulses with nonlinear arguments


What does (sin(x)) or 2 sin(x), y 2 mean? Study the descriptions on the next pages.

...

The 2D comb function is defined as a separable product that makes a bed of nails function:

 
 

x
y
x y
, comb
comb
,
comb2
TX TY
TX
TY

(2.9)

...

(2.10)
x

c J. Fessler, September 7, 2009, 22:51 (student version)

2.17

1D Dirac impulses with nonlinear arguments

For the 1D Dirac impulse, one can argue by simple change-of-variables that
X

(f (x)) =

{xn : f (xn

where the xn s are the roots of the equation f (x) = 0 and f =

(x xn )
,
|f(xn )|
)=0}

(2.11)

d
dx f .

As can be seen from the derivation below, this formula derives from the fact that at each root xn , the derivative becomes a local
scale factor. This formula is valid if the set of roots is finite (or perhaps even countable), if f is continuously differentiable and
one-to-one in the neighborhood of each root, and if f is nonzero in the neighborhood of each root.
Derivation.
Let {Bn } be a set of disjoint open intervals such that xn Bn but xm
/ Bn for n 6= m. By the support property of the Dirac
impulse:
Z
XZ
(f (x)) q(x) dx =
(f (x)) q(x) dx .

Now making the change of variables x = f (x) we have

Z
Z
(f (x)) q(x) dx =
=

q(f 1 (0))
dx
=
|f(f 1 (x ))|
|f(f 1 (0))|
(x xn )
q(x) dx,
|f(xn )|

(x ) q(f 1 (x ))

f (Bn )

Bn

Bn

q(xn )
=
|f(xn )|

where we have applied the usual sifting property twice.

Example. The scaling property (ax) =

1
|a|

(x) for a 6= 0 is a simple special case of the general result above.

 2 
x 9
1
as 0.
rect

Example. To understand x2 9 , consider what happens to
Rx
To examine this formally, let F (x) = f (t) dt, so that
Z


 2
1
x 9
f (x) x 9 dx = lim f (x) rect
dx
0

Z 9+/2
Z
1
1 9/2
=
f (x) dx +
f (x) dx
9/2
9+/2
i 1h
i
p
p
p
1h p
F ( 9 + /2) F ( 9 /2) +
F ( 9 /2) F ( 9 + /2)
=

"
"
#
#

1/4
1/4
1/4
1/4

F (3) p
+ F (3) p
p
p

9 + /2
9 /2
9 /2
9 + /2 =0


Z
1
1
1
1
f (3) + f (3) = f (x)
(x 3) + (x + 3) dx
=
6
6
6
6
2

using the chain rule for derivatives. Thus we conclude, consistent with (2.11), that

1
1
x2 9 = (x 3) + (x + 3) .
6
6

 2 
p
p
Another explanation is that the support of rect x 9 is /2 < x2 9 < /2 or equivalently 9 /2 < |x| < 9 + /2.

< |x| < 3 + 12

or equivalently 21 < |x|3
< 1.
For small , that support is approximately 3 12





 2 
 2 
/6  2

1
1 6
1 6
x+3
x 9
x3
x+3
+
rect
or
equivalently

+
Thus rect x 9 rect x3
rect
rect
rect
/6
/6

6
/6
6
/6 .

c J. Fessler, September 7, 2009, 22:51 (student version)

2.18

2D Dirac impulses with nonlinear arguments

What about in 2D? What is 2 (f (x, y), g(x, y))?
Assume that the set of roots that simultaneously solve f (x, y) = 0 and g(x, y) = 0 is finite, and that f and g are differentiable near
their zeros. Let (xn , yn ) denote the coordinate pairs where both f (xn , yn ) = 0 and g(xn , yn ) = 0. Then
2 (f (x, y), g(x, y)) =

X 2 (x xn , y yn )
|dn |

where dn is the (assumed nonzero) Jacobian determinant of

f (x, y)
g(x, y)

at (xn , yn ), i.e., dn = det

"

x f (x, y)

x g(x, y)

y f (x, y)

y g(x, y)

Proof. Let {Bn } be a set of small disjoint disks such that (xn , yn ) Bn . Then for any continuous function q(x, y):
ZZ
X ZZ
2 (f (x, y), g(x, y)) q(x, y) dx dy
2 (f (x, y), g(x, y)) q(x, y) dx dy =
=
=

X ZZ

Bn

X 1
1
q(xn , yn )
du dv =
|J(u, v)|
dn
T (Bn )
n
n
#
Z Z "X
X Z Z 2 (x xn , y yn )
2 (x xn , y yn )
q(x, y) dx dy
q(x, y) dx dy =
dn
dn
n
n
2 (u, v) q(x, y)

where we made the change of variables u = f (x, y), v = g(x, y).

This is the basic idea. Some of the notation details need to be worked out...

c J. Fessler, September 7, 2009, 22:51 (student version)

2.19

What does 2 (f (x, y), g(x, y)) mean if the set of simultaneous roots of f (x, y) = 0 and g(x, y) = 0 is uncountable?
Impulse ribbons
In some applications we need 1D Dirac impulses in the context of a 2D problem, which requires different sifting properties than
the usual formula. One can always determine the appropriate sifting property by returning to a first-principles expression like (2.7).
Typically a 2D integral containing a 1D Dirac impulse sifts to a 1D integral.
For example, in tomography, two ways to express a line integral at angle and radial distance r0 are as follows:
ZZ
ZZ
(x cos + y sin r0 ) g(x, y) dx dy =
(s r0 ) g(s cos l sin , s sin + l cos ) ds dl
Z
=
g(r0 cos l sin , r0 sin + l cos ) dl,
by making the change of variables

s
l

cos
sin

sin
cos



x
y

Curvilinear Dirac impulses

Bracewell [2, p. 130] shows that a 1D curvilinear Dirac impulse of the form (f (x, y)) has strength

1
|grad f(x,y) | , f (x, y) = 0
S(x, y) =
0,
otherwise,
where the slope in the direction normal to the contour where f (x, y) = 0 is |grad f (x, y) | =
the following sifting property [2, p. 132]:
ZZ
Z
S(s) g(x, y) ds,
g(x, y) (f (x, y)) dx dy =

f )2 + ( y
f )2 . This leads to
( x

where C is the curve where f (x, y) = 0, and ds is the element of arc length along the curve.
Transformation of variables
The following result from multivariable calculus [4, 5] was used above.
Suppose T : Rn Rn satisfies the following:
T is continuously differentiable.
T is one-to-one on a set R Rn , where R has a boundary consisting of finitely many smooth sets and where R and its boundary
are contained in the interior of the domain of T .
det{T }, the Jacobian determinant of T , is nonzero on R.
Then if f is bounded and continuous on T (R) then
Z
Z
f (x1 , . . . , xn ) dx1 dxn =
f (T (x1 , . . . , xn )) |det{T (x1 , . . . , xn )}| dx1 dxn .
T (R)

Exercise. Find a suitable sifting property for (r r0 ) =

Exercise. Find a suitable sifting property for

1
r

(r).


p
x2 + y 2 r0 . Consider all possible r0 values.

Local image phase

The concept of local phase of a signal or image can be important in certain applications. It is defined in 1D via the Hilbert
transform. See [6] for a discussion of local phase for 1d and 2d signals in the context of image registration.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.20

Systems in 2D (Imaging systems)

An imaging system, or just system, operates on a 2D signal called the input image and produces a 2D signal called the output
image.
For continuous-space systems, the input signal f (x, y) is transformed by the system into the output signal g(x, y). (We focus
on 2D-to-2D systems here, but of course 3D-to-3D and 3D-to-2D systems are also of considerable interest. Extensions to 3D are
straightforward.)
This mapping can be depicted as follows.
Input Image
f(x,y)
f(x,y,z)

System Operator
S

Output Image
g(x,y)
g(x,y,z)

Design Parameters

g = S[f]

Roughly speaking, there are two broad types of imaging systems: image capture systems and image processing systems. A
camera and an X-ray scanner are examples of image capture systems. Zoom lenses, optical color filters, photographic film, and
analog photocopiers are examples of continuous-space image processing systems. A scanner / digitizer, a contrast enhancer, a
denoiser, an image compressor and an edge detector are examples of image processing systems where the output is a discrete-space
image.
In an image capture system, f (x, y) is typically the true image, or true spatial distribution of some physical quantity of interest,
often called the object, whereas g(x, y) is the observed image or recorded image. The goal is usually for g(x, y) to be as similar
as possible to f (x, y).
(In image restoration problems the goal is to recover f (x, y) from g(x, y).)
In an image processing system, f (x, y) is typically the image produced by some image capture system, which might be noisy or
blurry for example, and g(x, y) is some enhanced version of f (x, y). (Typically this is done digitally and thus with a discrete-space
system, although optical / analog image enhancement is also possible.)
The input-output relationship for an imaging system is described by a system operator or system function S and is written
g = S[f ], where S : L2 (R2 ) L2 (R2 ).
In other words, S maps one function to another function, i.e., and image to another image. The relationship is very often expressed
using the following somewhat dangerously imprecise notation:
g(x, y) = S[f (x, y)] .
The problem with that notation is that it suggests that the value of g(x, y) at any point (x, y) depends only on the input image at
that same spatial location. This is rarely the case. More precise, but somewhat cumbersome, notation is:
g(x, y) = (S[f ])(x, y).
For convenience, we will often simply write one of the following:
S

f g or f (x, y) g(x, y) or f (x, y) S g(x, y) .

Example. Overhead projector...
What is the system function S? For most image capture systems, it is a blurring function. Often we can model the image capture
system operator as the convolution of the object with the 2D (or 3D) point spread function.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.21

Classification of systems
We can divide system characteristics into the following two categories.
Amplitude properties
A-1 linearity
A-2 stability
A-3 invertibility
Spatial properties
S-1 causality
S-1 separability
S-2 memory
S-3 shift invariance
S-4 rotation invariance
We will save linearity (the most important?) for last.
A-2 Stability
A system is bounded-input bounded-output (BIBO) stable iff every bounded input produces a bounded output.
f , if Mf s.t. | f (x, y) | Mf < , x, y, then there must exist an Mg s.t. | g(x, y) | Mg < , x, y.
Usually Mg will depend on Mf .
Otherwise the system is called unstable, and it is possible that a bounded input signal will make the output blow up.
Example: moving average filter with X , Y > 0
g(x, y) =
=

Z y+Y /2 Z x+X /2
1
f (x , y ) dx dy
X Y yY /2 xX /2


Z Z
x x y y
1
f (x , y ) dx dy .
rect2
,
X
Y
X Y

Suppose | f (x, y) | Mf < , x, y, so f (x, y) is a bounded input. Then by the triangle inequality for integration1:


 1 Z y+Y /2 Z x+X /2




| g(x, y) | = 
f (x , y ) dx dy 
 X Y yY /2 xX /2

Z y+Y /2 Z x+X /2
1
|f (x , y )| dx dy

X Y yY /2 xX /2
Z y+Y /2 Z x+X /2
1

Mf dx0 dy0
X Y yY /2 xX /2
= Mf ,
so | g(x, y) | Mf . Thus the output signal is also bounded for a bounded input, so this system is BIBO stable.
We will derive a simple test for BIBO stability shortly that applies to LSI systems (but not more generally).
It is easy to postulate hypotheticalR systems
that are unstable, like g(x, y) = 1/ f (x, y),
x Ry
or the ideal integrator: g(x, y) = f (x , y ) dx dy .
But it is difficult to think of real continuous-space systems that are unstable.
The issue of stability is particularly relevant when designing discrete-space filters.

1 The

P
R R
P
usual triangle inequality is |a + b| |a| + |b|. Generalizing to summation and integration yields: n fn n |fn | and f |f | .

(2.12)

c J. Fessler, September 7, 2009, 22:51 (student version)

2.22

A-3 Invertibility
A system S is called invertible iff each (possible) output signal is the response to only one input signal.
Otherwise S is not invertible.
If a system S is invertible, then there exists a system S 1 such that

f (x, y) S g(x, y) S 1 f (x, y) .

Design of S 1 is important in many image processing applications.
Mathematically:
S 1 [S[f ]] = f.
Invertibility is a particularly important concept in imaging because often we would like to undo degrading effects like optical
blurring (e.g., in images from the infamous Hubble telescope).
Example: Is the moving average filter (2.12) an invertible system?

??

S-1 Causal systems

The concept of causality is very important in 1D systems (when the independent variable is time). A 1D system is causal if the
output at any time t0 depends only on the values of the input signal for times t t0 .
In imaging systems, the independent variables are spatial position, not time, so causality is usually unimportant. (Of course in real
time systems like video, where the independent variables are both space and time, one must consider causality with respect to the
time variable.)
There are some (mostly older) papers in the image processing literature that address raster-scan based image processing methods,
where the image is filtered, for example, by starting in the upper left hand corner and working lexicographically down towards
the lower right hand corner. Such processing could be described as causal (whereas the moving average filter above would be
noncausal by such a definition).
S-1 Separable systems
The above definition of causality is a restriction on the how the system function S behaves in terms of the domain of the input
signal. The concept of a separable system is also a restriction on the behavior in terms of the domain of the input signal. Roughly
speaking, a separable system S processes the rows and columns of an image independently, and can be written as
S = Sh Sv or S = Sv Sh ,
where denotes function composition, i.e., S[f ] = Sh [Sv [f ]]. Here, Sh denotes a horizontal acting operator where if g = Sh [f ],
then for any given y0 , the output values g(x, y0 ) depend only on the input values f (, y0 ), i.e., the input values in the same row.
Likewise, Sv denotes a vertical acting operator where if g = Sv [f ], then for any given x0 , the output values g(x0 , y) depend only
on the input values f (x0 , ), i.e., the input values in the same column.
Separable systems essentially require only 1D operations, so they are often used in image processing to minimize computation.
Example. Is the moving average filter (2.12) a separable system?

??

Challenge: prove or disprove: if S = Sh Sv then does there always exist Sh and Sv such that S = Sv Sh , even if S is nonlinear?
If not, what if S is linear?

c J. Fessler, September 7, 2009, 22:51 (student version)

2.23

S-2 Memory
A system whose output g(x, y) at any point (x, y) only depends on the input image f (x, y) at the same location (x, y) could be
called memoryless, if we stretch the English usage of memory somewhat. For example, a detector that converts amplitude to
intensity (with no other effects) via the relation g(x, y) = [f (x, y)]2 would be memoryless.
Continuous-space image capture systems are virtually never memoryless, due to the spreading of light when propagating through
space. But discrete-space image processing operations can be categorized by those that use pixel neighborhoods vs those that
work independently pixel-by-pixel. So the memoryless concept is more relevant to digital image processing than to image capture
systems.

S-3 Shift-invariance
Systems whose input-output behavior is independent of the specific spatial location are called shift-invariant or space-invariant.
Why do we focus on shift-invariant systems?
??

??

A system S is called shift invariant iff

S

f (x, y) g(x, y) implies that f (x x0 , y y0 ) g(x x0 , y y0 )

(2.13)

for every input image f (x, y) and shift x0 ,y0 .

Otherwise the system is called shift variant or space variant.
Example: Is the moving average system above shift invariant?
Example: Is an overhead projector shift invariant?

??

??

Example: Is the imaging zooming system defined by g(x, y) = f (x/2, y/2) shift invariant?
Example: Is the ideal mirror system defined by g(x, y) = f (x, y) shift invariant?

??

??

S-4 Rotation-invariance
Usually the object being imaged has an orientation that is unknown relative to the recording devices coordinate system. Often we
would like both the imaging system and the image processing operations applied to the recorded image to be independent of the
orientation of the object.
A system S is called rotationally invariant iff
S

f (x, y) g(x, y) implies that f (x, y) g (x, y) where f (x, y) , f (x cos + y sin , x sin + y cos )
and g (x, y) is defined similarly, for every input image f (x, y) and rotation .
Example: Is the moving average system above rotationally invariant?

??

How could we modify the moving-average system to make it rotationally invariant?

?? ??

(2.14)

c J. Fessler, September 7, 2009, 22:51 (student version)

2.24

A-3 Linear systems

Why linearity?
??

??
??
??

In the context of 2D continuous-space systems, we will say S is a linear system (function) iff it is a linear operator on the vector
space of functions L2 (R2 ) described in the Appendix, i.e., iff
S[f1 + f2 ] = S[f1 ] + S[f2 ] superposition property
for all images f1 , f2 and all real (or complex) constants , .
Example: Is the moving average system above linear?

??

Two special cases of the above condition for linearity are particularly important in the context of linear systems.
If the input to a linear system is scaled by a constant, then the output is scaled by that same constant.
S[f ] = S[f ]

scaling property.

If the sum of two inputs is presented to the system, then the output will be simply the sum of the outputs that would have resulted
from the inputs individually.
S[f1 + f2 ] = S[f1 ] + S[f2 ] additivity property.
We can extend the superposition property to finite sums: by applying proof-by-induction:

"K
X

k fk =

k=1

K
X

k=1

k S[fk ].

For a system to be called linear, we require the superposition property to hold even for complex input signals and complex scaling
constants. (This presumes that the system is defined to permit both real and complex inputs and to produce both real and complex
outputs.)
Example: Is g(x, y) = real(f (x, y)) a linear system?

??

Exercise. Find a system that satisfies the scaling property but not the additivity property.
Thus, both the additivity and scaling properties are required for a system to be linear.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.25

Impulse response (point spread function) (PSF) of a linear system

(The PSF is perhaps the most important tool in the analysis of imaging systems. It is analogous to the impulse response used in 1D
signals and systems.)
The superposition property of a linear system enables a very simple method to characterize the system function.
Decompose the input into some elementary functions.
Compute the response (output) of the system to each elementary function.
Determine the total response of the system to the desired (complex) input by simply summing the individual outputs.
A simple and powerful function for decomposition is the Dirac impulse 2 (x, y).
The key property of the Dirac impulse is the sifting property:
Z Z
ZZ

f (x, y) =
f (x , y ) 2 (x x , y y ) dx dy =
f (x , y ) 2 (x, y; x , y ) dx dy ,

where the latter form expresses the image as a decomposition of an image into (weighted) elementary functions, namely off-center
Dirac impulses, and where 2 (x, y; x , y ) = 2 (x x , y y ), was defined earlier.
PSF
Suppose the input to a system S is an impulse centered at (x , y ), i.e., the input is 2 (x, y; x , y ), where x and y are considered
fixed, and x, y are the independent variables of the input image. Let h(x, y; x , y ) denote the corresponding output signal:
S

2 (x, y; x , y ) h(x, y; x , y ) .
The function h(x, y; x , y ), the output of the system S when the input is an impulse function located at (x , y ), is called the point
spread function (PSF) of the system, or (less frequently in the context of imaging systems) the impulse response of the system.
Example: What is the PSF of an ideal mirror? The input-output relationship is g(x, y) = f (x, y), so

??

Example: What is the PSF of an ideal magnifying lens? For a magnification of 2, the ideal input-output relationship is
g(x, y) = f (x/2, y/2), for some 0 < < 1/4 related to the absorption of the lens and the energy spreading. Thus
S

2 (x x , y y )

??

More generally, the impulse response of the system f (x, y) g(x, y) = f (x/a, y/b) is

??

Example: What is the impulse response of the moving average filter (2.12)?
Substitute f (x, y) = 2 (x x , y y ) in (2.12), and we see


Z Z
1
x x y y

h(x, y; x , y ) =
2 (x x , y y ) dx dy
rect2
,
X
Y
X Y


x x y y
1
.
rect2
,
=
X Y
X
Y
The first two examples also illustrate that linearity is distinct from shift-invariance!

c J. Fessler, September 7, 2009, 22:51 (student version)

2.26

Superposition integral

for any linear system

Consider a linear system with system function S operating on input f (x, y). We want to find a simple expression for the output
image g(x, y) in terms of the input image f (x, y). We would like an expression that is more informative than g = S[f ]. The PSF,
the impulse decomposition, and the superposition property are the key ingredients.
To aid interpretation, first consider a finite weighted sum of input impulses:
f (x, y) =

X
n

n 2 (x xn , y yn ) g(x, y) =

n h(x, y; xn , yn ) .

Now generalize to a continumm using the strong superposition property (see Appendix):
ZZ
ZZ
S
=
f (x , y ) S[2 (, ; x , y )](x, y) dx dy
f (x, y) =
f (x , y ) 2 (x, y; x , y ) dx dy g(x, y)
ZZ
=
f (x , y ) S[2 (x, y; x , y )] dx dy
ZZ
=
f (x , y ) h(x, y; x , y ) dx dy ,
where we have substituted in the definition of h. The quoted equals warns you of the dangerous notation discussed on p. 2.20.
Thus we have derived the following input-output relationship for linear systems, known as the superposition integral:
g(x, y) =

f (x , y ) h(x, y; x , y ) dx dy .

(2.15)

This is the general superposition integral for the case where we have decomposed the input into impulse functions.
The superposition integral tells us that once we have found the PSF of the imaging system for all input coordinates, then the output
is fully determined for any input. Thus, a linear system function S is characterized fully by its PSF h.
Thus we will rarely (if ever) explicitly work with S, for linear systems.
Example. Consider an idealized inkjet printer with a dirty head such that each time a dot is written the ink smears into the next
dot over in the direction of print head movement. We might model this smear as


y
x 
x /2 
rect
rect
3/4
.

2
2
where is the dot width. Then the input-output relationship is:





ZZ
x x
x x /2
y y

rect
3/4
dx dy .
g(x, y) =
f (x , y ) rect

2
2
For an image capture system, the ideal PSF would be simply an impulse function:
h(x, y; x , y ) = 2 (x x , y y ),
so that the system output would exactly equal the input: g = S[f ] = f .
In real systems there is always some blurring (imperfect resolution), if not other distortions as well. This blurring means that an
input impulse function gets spread out at the output, hence the term.
Analysis of imaging systems is often firstly concerned with:
finding h for the system,
and then understanding how h is influenced by the various system parameters (such as detector spacing, etc.).
Then we can design system parameters to improve h(x, y).
Particularly useful as a rule-of-thumb for examining h is the width of the PSF, often specified by the full-width at half maximum (FWHM). (Picture)

c J. Fessler, September 7, 2009, 22:51 (student version)

2.27

Example: Consider the following 1D pinhole camera.

Input

f (x)

d1
Ideal Pinhole Aperture

d2
Output

g(x)

x d2 /d1 0 x

How do we find the PSF? It varies by imaging system. For this system we can use simple geometric reasoning. For incoherent
illumination, this system is linear (in intensity).
If the input were an impulse function at x = x , then for an ideal (infinitesimal) pinhole the output would be an impulse function
at x = x d2 /d1 = mx , where m , d2 /d1 is called the source magnification factor [7, p. 44]. (For now we ignore the fact
that light intensity decreases with propagation distance.)
Thus by simple ray-tracing we see that (for an ideal pinhole) the PSF would be given by:
h(x; x ) = (x mx ) .
Substituting into the 1D superposition integral (cf. (2.15)) and simplifying by the sifting property yields the input-output relation:
g(x) =

1
f (x/m).
|m|

Note that the system is highly shift-variant, due to the mirroring and (spatial) scaling / magnification. On the other hand, as will be
discussed later, with a simple coordinate change for the input image, the system can be considered to be shift-invariant!
2

Example. If the input is f (x) = e(x3) (a Gaussian bump centered at x = 3), then we can use the superposition integral to
compute the output:
Z
Z

2
1 ( x3m
)2 .
m
e
g(x) =
f (x )h(x; x ) dx =
e(x 3) (x mx ) dx =
|m|

Thus the output is a Gaussian bump centered at 3|m| and stretched out (magnified) by the factor |m|.
The superposition integral is easier than a direct attempt to find g(x).
If the pinhole has finite width w, then the PSF is not an impulse function, but rather a rectangular function of width w(d1 + d2 )/d1
centered at x d2 /d1 :


x mx

,
h(x; x ) = rect
wM

where M = 1 + d2 /d1 is the aperture magnification factor. If the aperture plane has finite thickness, then the width of the
rectangle will vary with x , which causes further shift-variance.
If we account for the 1/r2 falloff with distance of the intensity of incoherent radiation, then the PSF of an ideal pinhole will be:
h(x; x ) = (x mx )

1
,
r2 (x)

where the distance between the source point and the detector point is
p
p
r(x) = (d1 + d2 )2 + (x + xd1 /d2 )2 = (d1 + d2 ) 1 + (x/d2 )2 .

This intensity effect adds an additional form of shift-variance, although we can often ignore it for points near the axis (paraxial
approximation).
The effect of a finite detector aperture of width W could be described by truncating the PSF by multiplying by rect(x/W ). This
also makes the system shift-varying.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.28

The PSF depends on system design parameters: d2 , d1 , and w. Specifically, the ratio d2 /d1 determines a magnification factor.
When is magnification useful?
??

??

Shift invariance or Space invariance of linear systems

In any linear system where the point spread function is the same for all input points (except for a shift), we can greatly simplify
general superposition integral.
Fact. A linear system satisfies the shift invariant or space invariant condition (2.13) iff the point spread function satisfies
h(x, y; x , y ) = h(x x , y y ; 0, 0),

x, y, x , y .

(2.16)

Proof. We must show that (2.13) holds iff (2.16) holds.

(Sufficiency.) Suppose (2.16) holds. Then (2.15) becomes
Z Z
Z
S
f g g(x, y) =
h(x, y; x , y ) f (x , y ) dx dy =

h(x x , y y ) f (x , y ) dx dy .

Making the change of variables (x , y ) = (x + x1 , y + y1 ) we have

Z Z
g(x x1 , y y1 ) =
h(x x1 x , y y1 y ) f (x , y ) dx dy

Z Z
=
h(x x , y y ) f (x x1 , y y1 ) dx dy ,

which clearly shows that the general shift invariance condition (2.13) holds.
(Necessity.) If (2.16) does not hold for any input location (x , y ), then we can consider the impulse input image f (x, y) = 2 (x, y)
and another (translated) input image, 2 (x x , y y ), and the corresponding output images will not satisfy (2.13).
2
......................................................................................................................
For shift-invariant systems, the (0, 0) notation in (2.16) is superfluous. Therefore, to simplify notation we define
h(x, y) , h(x, y; 0, 0),
so that (with a little notation recycling) we can simply write
h(x, y; x , y ) = h(x x , y y ) .
Because h(x, y) is the response of the system to an impulse at the origin, if the system is linear and shift invariant, then we can
determine the PSF by measuring the response to a single point-source input.
Example. X-ray CT phantom with tungsten wire...
To test for shift invariance of a linear system, first determine the system output for an impulse at the origin. Then shift or translate
the input impulse by some arbitrary distance. Does the system output shift by the same distance, the same direction, and retain the
same shape? If the answer yes (for any shift), then the system is shift invariant.
Example. Are the mirror, magnification, or 1D pinhole systems shift invariant?
Example. Is the moving average filter (2.12) shift invariant?

??

??

Exercise. To show that a given linear system is shift-varying, does it suffice to check for counter-examples using impulse input
functions (e.g., as used in the mirror example above)? If so, prove it. If not, explain why.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.29

Convolution
For linear shift-invariant (LSI) systems, the input-output relationship simplifies from the general superposition integral (2.15) for
linear systems to the following 2D convolution integral:
g(x, y) =

f (x , y ) h(x x , y y ) dx dy .

(2.17)

We have the following notations for convolution:

g = f h, or g(x, y) = (f h)(x, y), or dangerously imprecisely: g(x, y) = f (x, y) h(x, y) .
The dual s indicate 2D convolution. Note2 : L2 (R2 ) L1 (R2 ) L2 (R2 ).
LSI systems are easier to describe than linear shift-variant systems, because we only need the two-argument PSF h(x, y), rather
than the general PSF h(x, y; x , y ) with its 4 arguments. Furthermore, for LSI systems we can simplify the convolution using
Fourier methods.
Unfortunately, in most image capture systems, the PSF is not exactly shift invariant! For example, in phased-array ultrasound the
PSF in the region of the transmit focus is vastly different from the PSF near the face of the array.
However, the PSF of most imaging systems varies slowly over space. (If this were not the case then the images might be very
difficulty to interpret visually.) Consequently, for the purposes of system design and analysis, the PSF can often be considered shift
invariant over local regions of the image.
Linearity alone does not imply shift-invariance, as in the 1D pinhole example above.
One rarely performs the 2D convolution integral (2.17) analytically.
2D convolution properties

(relevant to LSI systems)

The following properties of 2D convolution are easily verified directly from the 2D convolution integral by manipulations that are
essentially identical to those for 1D convolution.
Commutative property:
f h = h f
Associative property:
[f h1 ] h2 = f [h1 h2 ]
Distributive property:
f [h1 + h2 ] = [f h1 ] + [f h2 ]
The order of serial connection of LSI systems does not affect the overall impulse response: h1 h2 = h2 h1 .
f (x, y) 2 (x, y) = f (x, y)
Shift property: f (x, y) 2 (x x , y y ) = f (x x , y y )
Shift-invariance: If g(x, y) = f (x, y) h(x, y), then f (x x1 , y y1 ) h(x x2 , y y2 ) = g(x x1 x2 , y y1 y2 )
Separability property: [f1 (x)f2 (y)] [h1 (x)h2 (y)] = [f1 (x) h1 (x)] [f2 (y) h2 (y)]
Convolving two separable functions yields a separable function.
What is (x) (y)? ??
Circular symmetry property: if f (x, y) and h(x, y) are circularly symmetric, then so is f (x, y) h(x, y).
Scaling property.
Exercise. If g(x, y) = f (x, y) h(x, y), then what is f (2x, 2y) h(2x, 2y)?

2 For

a more precise conditions under which convolution is defined rigorously, see EECS 600.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.30

LSI system properties via PSF

Because an LSI system is characterized completely by its PSF, we should be able to express all the other five system properties in
terms of conditions on the PSF.
causal: h(x, y) = 0 whenever x < 0 or y < 0?
That would be the natural analog to the 1D case, but it is essentially irrelevant for imaging.
memoryless (static): ??
stable: ??
Exercise: state and prove the appropriate condition for stability.
rotational invariance.
Under what conditions on the PSF h(x, y) is an LSI system rotationally invariant?

??

invertible: For what type of h(x, y) is an LSI system invertible? ??

If h(x, y) f (x, y) = 0signal for some nonzero signal f (x, y), then the system with PSF h(x, y) is not invertible
Exercise. State and prove the condition on h(x, y) that ensures an LSI system is rotationally invariant.
Separability
Fact. An LSI imaging system is separable if and only if its PSF h(x, y) is separable, i.e., h(x, y) = h1 (x)h2 (y) for some 1D
functions h1 and h2 .
In which case 2D convolution simplifies into two sets of 1D convolution operations:
f (x, y) h(x, y) = f (x, y) (h1 (x)h2 (y)) = f (x, y) (h1 (x) (y) (x) h2 (y)) = (f (x, y) h1 (x) (y)) (x) h2 (y),
where
f (x, y) (h1 (x) (y)) =

ZZ

f (x x , y y ) h1 (x ) (y ) dx dy =

f (x x , y) h1 (x ) dx .


 
 

Example. The moving average filter (2.12) is separable because its PSF is h(x, y) = rect2 xX , yY = rect xX rect yY ,
which is separable. In fact, we can see that this system is separable by rewriting slightly the original input-output relation:
#
Z y+Y /2 "
Z x+X /2
1
1

f (x , y ) dx dy .
g(x, y) =
Y yY /2 X xX/2
The inner integral acts only along the x direction, i.e., it is a horizontal acting operation, whereas the outer integral acts only in
the y direction, i.e., it is a vertical acting operation.
The 2D convolution integral for any LSI system with a separable PSF can be so decomposed:

Z Z
h(x, y) = h1 (x)h2 (y) = g(x, y) = h(x, y) f (x, y) =
h1 (x) f (x, y) dx h2 (y) dy

Z Z
=
h2 (y) f (x, y) dy h1 (x) dx .
Differential equations?
At this point in a 1D signals and systems course, most texts address the specific case of systems described by linear constant
coefficient differential equations, such as RLC circuits. Indeed, the analysis of such systems occupies a substantial portion of the
effort in such courses. Although there are certainly some important differential equations relevant to imaging, such as the wave
equation, there is not a clear analog to RLC systems. This is a significant difference between 1D (time) signals and systems and
multidimensional signal processing.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.31

Effect of PSF on spatial resolution

Pictures illustrating PSF and spatial resolution:
Object

PSF

Output

0.5

0.5

0.5

0
0

50

0
0

50

0
0

0.5

0.5

0.5

0
0

50

0
0

50

0
0

0.5

0.5

0.5

0
0

50

0
0

50

0
0

0.5

0.5

0.5

0
0

50

0
0

50

0
0

50

50

50

50

Resolution criteria
Sometimes an image contains the superposition of two closely spaced impulse-like features, i.e., two closely spaced dots. If their
spacing is sufficiently close their appearance will be indistinguishable from, simply, one dot. If they are distinguishable, then one
says they are resolved. It sometimes important to develop a mathematical criteria for determining when this will happen.
For example, from [8, p. 326]:
the question of when two closely spaced point sources are barely resolved is a complex one and lends itself to a variety
of rather subjective answers. According to the so-called Rayleigh resolution criterion, two equally bright points
are barely resolved when the first zero of the Airy pattern [PSF] of the image of one point exactly coincide with the
central maximum of the [PSF] of the image of the second point. ... An alternative definition is the so-called Sparrow
criterion, which states that two point sources are just resolved if the second derivative of the image intensity pattern
vanishes at the point midway between the [sum of the two PSFs].
Exercise. Find the just-resolved
distances
frequency

 (according to Rayleighs resolution criterion) for a system having
 a square


, where umax =
response H(u, v) = rect2 2uumax , 2uvmax and for one having a disk frequency response H() = rect 2umax

5mm1. Which system would you rather use for imaging and why?
Hint: M ATLABs fzero function may be useful here.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.32

Magnification and LSI systems

As illustrated by the 1D pinhole example above, any system with magnification or mirroring is, strictly speaking, a shift-variant
system, because the PSF does not depend solely on the difference between input and output coordinates.
However, we can recast such systems into equivalent shift-invariant systems by suitable coordinate changes for the input signal.
This is reasonable in the context of imaging systems because the coordinates that we use are somewhat arbitrary. (In contrast, in
the context of time-domain systems, coordinate changes are nontrivial.)
We will show the required coordinate change for the example of a 2D pinhole camera. Among other things, this example will
illustrate the type of manipulations that are useful for imaging systems analysis.
Input
f(x,y)

Pinhole Aperture
a(x,y)

Output
g(x,y)

(x ,y )
h(x, y; x , y )

d1

d2

Consider the simple pinhole camera system shown above, assumed linear, with a pinhole opening described by a real aperture
function 0 a(x, y) 1, where a(x, y) denotes the fraction of light incident on the point (x, y) that passes through.
If the thickness of the aperture plane is infinitesimal (and ignoring 1/r2 intensity falloff), then by geometry, the PSF is
h(x, y; x , y ) = h(x mx , y my ), where h(x, y) = a(x/M, y/M ),
2
= 1 m is called the object magnification factor (but
where m = d2 /d1 is the source magnification factor, and M = d1d+d
1
might better be called the aperture magnification factor). The minus sign in the definition of m represents the mirroring effect.

By the superposition integral, the output intensity is written in terms of the PSF by
Z Z
Z Z
g(x, y) =
f (x , y ) h(x, y; x , y ) dx dy =
f (x , y ) h(x mx , y my ) dx dy .

We can now rewrite the system equation using the simple change of variables (xm , ym ) = (mx , my ), leading to:
Z Z
g(x, y) =
fm (xm , ym ) h(x xm , y ym ) dxm dy m = (fm h)(x, y).

This is now a simple convolution of the modified image fm (x, y) , m12 f (x/m, y/m), which is a magnified, scaled and mirrored
version of the input image, obtained via the coordinate change above, and the PSF h(x, y), which is a magnified version of the
aperture function.
To understand the 1/m2 in the definition of fm , consider that for an infinitesimal pinhole, the system magnifies the image by the
factor m in each dimension. Thus the spatial extent of the input image is extended by m2 in the output, But because the total
radiation striking the output plane does not change (i.e., passive camera system), the intensity of the output image is reduced by
the factor 1/m2 .
The magnified image fm (x, y) is what the system output would be for an ideal pinhole aperture, i.e., when a(x, y) = 2 (x, y).

c J. Fessler, September 7, 2009, 22:51 (student version)

2.33

Representation of signals by orthogonal bases

The next topic usually discussed in a 1D signals and systems text is Fourier series. Although Fourier methods are very important
in imaging, we are also often interested in other representations of 2D signals, such as wavelets. We first describe the general
representation of signals by orthogonal bases, and then consider Fourier series as an important special case.
In engineering it is often very useful to represent signals by a linear combination of simpler signals:
g(x, y) =

ck k (x, y) .

(2.18)

This process of signal analysis is particularly convenient if we choose a set of signals {k } that are orthogonal.
Review of inner products and orthogonality
The concept of orthogonality is an extension of the geometric concept of perpendicular lines. Suppose two lines in the plane
are perpendicular and intersect at the origin. Let ~u and ~v denote two vectors pointing along those lines. Because the lines are
perpendicular, we say ~u and ~v are orthogonal vectors, meaning their inner product or dot product is zero. In Euclidean 2-space,
the standard inner product of ~u and ~v is defined by
h~u, ~v i = ~u ~v = ~v T ~u = u1 v1 + u2 v2 .
More generally, for n-dimensional complex vectors x, y Cn , the standard inner product is
hx, yi = y x =

n
X

xk yk ,

k=1

where y denotes the Hermitian transpose of a vector (or matrix).

To generalize the concept of orthogonality to an other setting (i.e., another vector space) beyond ordinary Euclidean space, the
key is to define an appropriate inner product. Any inner product must satisfy the following conditions:

1. hx, yi = hy, xi (Hermitian symmetry), where denotes complex conjugate.

2. hx + y, zi = hx, zi + hy, zi (additivity)
3. hx, yi = hx, yi (scaling)
4. hx, xi 0 and hx, xi = 0 iff x = 0. (positive definite)
Using the above conditions, one can show that any inner product is a continuous function [9, p. 49], and is a bilinear form that
satisfies:
*
+
XX
X
X
n l hxn , ym i,
(2.19)
n xn ,
m y m =
n

for any complex constants {n } and {m } if

xn and

ym are convergent series.

Related to the concept of inner product is the concept of the norm of a vector, defined in terms of an inner product as follows:
p
kxk , hx, xi.
p
pPn
pPn

2
In Euclidean space this measures the length of a vector, i.e., k~uk = h~u, ~ui =
i=1 ui ui =
i=1 |ui | .
Also related is the concept of the distance between two vectors:
kx yk .

c J. Fessler, September 7, 2009, 22:51 (student version)

2.34

Inner products and orthogonality for 2D images

Here we are working with 2D continuous-space images, which are functions defined on R2 , rather than simple vectors in Rn or
Cn . For images with domain B R2 , the natural definitions of inner product, norm (squared) and distance are:
ZZ
hf, gi =
f (x, y) g (x, y) dx dy
B
ZZ
2
2
kf k = hf, f i =
|f (x, y)| dx dy
B

kf gk =

Z Z

|f (x, y) g(x, y)| dx dy

1/2

The norm of f is the square root of its energy.

Two images f and g are called orthogonal if hf, gi = 0.
A set of 2D signals {k } is called orthogonal iff the inner product of each pair of distinct signals in the set is zero:

ZZ
Ek , l = k
k (x, y) l (x, y) dx dy =
hk , l i =
= Ek [l k],
0,
l 6= k
B
where Ek = kk k2 is the energy of k (x, y).
The 1D Kronecker impulse or Kronecker delta function. (this one is a function!) is defined by [l] =

1, l = 0
0, l =
6 0.

If Ek = 1 for all k, then we call the signals orthonormal, because they are normalized to have unit energy (and thus unit norm).
For specific 2D examples, we will often want to write:
g(x, y) =

ck,l k,l (x, y)

(2.20)

k= l=

rather than (2.18), i.e., we will often index the basis signals with double subscripts.
In this case, we say {k,l } is an orthogonal set of signals if hk1 ,l1 , k2 ,l2 i = 0 whenever k1 6= k2 or l1 6= l2 .
In the following examples, we will use the following useful fact:

Z T
0,
k = 1, 2, . . .
sin(2kx/T + ) dx =
= T sin [k] .
T
sin
,
k=0
0
Example. (separable) harmonic sinusoids k,l (x, y) = sin(2kx/TX ) sin(2ly/TY ) for k, l = 1, 2, . . . are orthogonal on the set
B = [0, TX ] [0, TY ] because
hk,l , mn i =

TXZ TY
0

k,l (x, y) mn (x, y) dx dy = . . . =

TX TY
[k m] [l n] .
4

Example. harmonic complex exponentials k,l (x, y) = e2(kx/TX +ly/TY ) over B = [0, TX] [0, TY ] for k, l Z.
hk,l , mn i =
=

TXZ TY
0

k,l (x, y) mn (x, y) dx dy =

TX TY [k m] [l n] .

TXZ TY
0

e2(km)x/TX e2(ln)y/TY dx dy

c J. Fessler, September 7, 2009, 22:51 (student version)

2.35

Generalized Fourier series

We will be interested in sets of orthogonal signals {k } on the set B such that if g(x, y) has finite energy over that set, then we can
represent g(x, y) in terms of the k functions by the following convergent series3 :
g(x, y) =

ck k (x, y),

(x, y) B,

(2.21)

g(x, y) k (x, y) dx dy .

(2.22)

k=

where

ck =

hg, k i
2

kk k

1
=
Ek

ZZ

The ck values are called Fourier coefficients with respect to the orthogonal set {k }, which is called an orthogonal basis. The
series is called an orthogonal series representation for g or simply an orthogonal series or orthogonal representation. A set of
orthogonal functions {k } is called complete if representations of the form (2.21) are possible for every g L2 (B), i.e., for all
signals with finite energy over B.
The formula (2.22) indicates that we can compute each ck independently! This is a consequence of choosing an orthogonal set of
signals. If we had chosen a non-orthogonal set, then to find the ck values, in general we would have to solve a large system of
simultaneous equations. (An interesting exception is the case of over-complete wavelet expansions, called frames, which can be
non-orthogonal yet simple formula for the coefficients still exist.)
Where did the ck formula (2.22) originate? There are two perspectives one can take to answer this question: the self consistency
perspective, and the best approximation perspective. Both lead to the same formula (2.22).
Self consistency perspective
If (2.21) is true, then
*
+

X
X
X
hg, k i =
cl hl , k i =
cl Ek [k l] = ck Ek ,
cl l , k =
l=

l=

l=

so ck = hg, k i /Ek , which is exactly (2.22).

But this is somewhat circular reasoning because how do we know (2.21) is correct in the first place?
Best approximation perspective
In practice often we must use a finite series, rather than the infinite series, meaning we make the approximation:
g(x, y) gN (x, y) ,

N
X

ck k (x, y) .

k=N

In this case, (2.21) does not hold exactly, so the previous derivation is inapplicable.
What would be a reasonable rationale for choosing the ck values in the finite series context?
Choose the ck values to make the approximation error as small as possible!
The approximation error is the difference between the actual signal g(x, y) and its finite series approximation:
eN (x, y) = g(x, y)

N
X

ck k (x, y) .

k=N

One reasonable measure of the approximation error (reasonable because it is analytically tractable for one thing) is the energy
2
of the error signal. So we would like to choose the ck values to minimize keN (x, y)k . This is essentially one form of least
squares fitting. The solution to this error minimization problem gives the formula in (2.22) for ck , regardless of N !

3 The equality in (2.21) is not quite the usual equality, but rather that the two sides are equivalent in the L sense, i.e., that the norm of the difference of the two
2
sides is zero.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.36

Proof:
keN (x, y)k

2 *
+
N
N
N


X
X
X


ck k = g
ck k , g
cl l
g


k=N

N
X

kgk

l=N
N
X

kgk +

k=N

kgk +

N
X

k=N

k=N

cl hg, l i
|ck |2 Ek

N
X

k=N
N
X

l=N

apply bilinearity (2.19):

l=N

ck hk , gi +

cl hg, l i

N
X

N
X

k=N l=N

N
X

k=N

ck cl hk , l i

ck hk , gi

use orthogonality:

rearrangement trick:


2
N
X


1
1

E k  ck
hg, k i
|hg, k i|2 .
Ek
Ek
k=N

The first and third terms are independent of ck , so the error signal energy is minimized when the middle term is equal to zero,
which happens when and only when ck is given by (2.22). Geometrically, this choice is called a projection of g(x, y) onto the
signal k (x, y).
The fact that
RR (2.22) is optimal regardless of N is a result of measuring error by energy. If we instead had used, for example, the
L1 norm: B |eN (x, y)| dx dy (integrated absolute error) as the measure of error, then the optimal choice for the ck values is not
(2.22), and is not independent of N !
From the above formula for the approximation error energy, we can deduce that when we choose the ck values according to (2.22),
then the error energy is
N
X
X
2
2
Ek |ck |2 .
keN (x, y)k = kgk
Ek |ck |2 =
k=N

|n|>N

This shows that the energy of the approximation error goes to zero, i.e., the finite series converges to the true image g(x, y), iff
lim

N
X

k=N

Ek |ck |2 = kgk2 .

It follows that the above is a necessary and sufficient condition for a set of orthogonal functions to be complete. Unfortunately,
neither this condition, nor any other condition for completeness, is particularly easy to check. (See [9, p. 61] for an example of
proving completeness of polynomials over intervals in 1D.)
Finally, we note that when we have an orthogonal series representation as in (2.21), the sense in which equality holds is that the
difference between g(x, y) and the summation while not necessarily zero for all (x, y) has zero energy. This means they can
differ on a finite or countably infinite set of points, or even on a set of measure zero. Such differences are, generally speaking,
unobservable. For example, the output of any LSI system will not be influenced by such differences.
Parsevals theorem
The relationship that we know as Parsevals theorem for Fourier series holds for general orthogonal expansions:
*
+

X
X
X
X
X
X
X
2
kgk = hg, gi =
ck cl hk , l i =
ck cl Ek [k l] =
Ek |ck |2 ,
ck k ,
cl l =
k=

k= l=

l=

k= l=

k=

(2.23)

which follows directly from the orthogonality of the k functions. So kgk =

More generally, if f =

k= ck k

and g =

k=

ZZ

| g(x, y) |2 dx dy =

k=

Ek |ck |2 .

dk k where the k functions are orthogonal, then hf, gi =

k=

Ek ck dk .

c J. Fessler, September 7, 2009, 22:51 (student version)

2.37

Gram-Schmidt
We can orthogonalize any set of signals by applying the Gram-Schmidt orthogonalization procedure. The procedure for signals
is completely analogous to that for vectors, except that we use the inner products and norms that are appropriate for signals.
Separable Orthogonal Bases
There are many well known orthogonal sets of 1D signals, such as harmonic complex exponentials, the Harr basis, etc. We can
easily construct sets of 2D orthogonal signals from such 1D sets.
Suppose {k } is an orthogonal set of 1D signals over the interval [a, b]. A simple approach to constructing orthogonal sets of 2D
signals uses separability:
k,l (x, y) = k (x)l (y).
(2.24)
Over what set B is this set orthogonal?

??

Separable 2D signals are simple and convenient, but may not always be the best choice for a given application.
Example. Harmonic complex exponentials are separable, because k,l (x, y) = e2(kx/TX +ly/TY ) = e2kx/TX e2ly/TY .

Challenge: do complete, orthogonal 1D bases lead, via separability, to complete orthogonal 2D bases?
Uses of orthogonal representations
Finite support signals, where g(x, y) = 0 except for (x, y) B
Periodic signals. Apply representation over one period. If the representation is accurate over one period and the k functions
have the same period, then the representation is equally good over all periods.
Finite energy signals with infinite support. Less frequently we use orthogonal representations for finite energy signals with
infinite support.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.38

Example: The Harr basis.

A set of 1D signals that is orthogonal and complete on the interval (0,1) and that consists of pieces of square waves as illustrated
below:
Harr basis
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
0

0.2

0.4

0.6

0.8

Each signal takes values 1, -1, and 0.

Due to its completeness, any (real, finite-energy) signal can be approximated arbitrarily well (in terms of the L2 norm) using the
Harr basis with a sufficiently large number of terms N .
Here is an example where x(t) = sin(2t).
Harr approximation to sin(2 t)
1
0.8
0.6
0.4

x(t)

0.2
0
0.2
0.4
0.6
0.8
1
0

0.2

0.4

0.6

0.8

I made these plots in M ATLAB using a discretized approximation to the integral (2.22) for computing the ck values, i.e., for large
N:
PN 1

i=0 x(i/N )k (i/N )

ck P
.
N 1
2
i=0 |k (i/N )|

The natural corresponding 2D basis consists of the separable form (2.24).

There are many possible complete sets of orthogonal signals, all of which can approximate all finite-energy signals to arbitrary
accuracy. One large family is called wavelets, of which the Harr basis is a member.
Why do we focus on the Fourier series rather than the Harr basis?

?? ??

c J. Fessler, September 7, 2009, 22:51 (student version)

2.39

Eigenfunctions of LSI systems

Of all the many orthogonal bases, the set of harmonic complex exponentials is particularly important.
The reason is simple: complex exponentials are eigenfunctions of LSI systems:
e2(ux+vy) LSI h(x, y) g(x, y) = e2(ux+vy) h(x, y)
Z Z

=
e2[(xx )u+(yy )v] h(x , y ) dx dy = e2(ux+vy) H(u, v)

where H(u, v) is the 2D Fourier transform of h(x, y):

Z Z
H(u, v) ,

h(x, y) e2(ux+vy) dx dy .

(2.25)

So the response due to a complex exponential input signal is just that same complex exponential input signal scaled by H(u, v),
which is why the complex exponentials are called eigenfunctions of LSI systems.
So if we can decompose a signal into complex exponential components, it is easy to determine the output of an LSI system by
applying superposition.
Exercise. Some, but not most, LSI systems have very many eigenfunctions in addition to complex exponentials. Give two distinct
examples of such LSI systems. One example is the do nothing system where g(x, y) = f (x, y). For this system, every 2D signal
is an eigenfunction. (You provide two other systems.)
Are there other finite-power signals besides complex exponentials that are eigenfunction of all LSI systems?

??

c J. Fessler, September 7, 2009, 22:51 (student version)

2.40

Fourier series (the classical 2D case)

If g(x, y) is periodic with period (TX , TY ), then we can represent g(x, y) using a 2D Fourier series:
g(x, y) =

g(x, y) e2(xk/TX +yl/TY ) dx dy .

ck,l e2(xk/TX +yl/TY )

(2.26)

k= l=

ck,l =

1
TX TY

TY

TX

(2.27)

This series uses basis functions {k,l } with k,l = e2(xk/TX +yl/TY ) . These are orthogonal and complete on any support rectangle
whose width is a multiple of TX and height is a multiple of TY . Because the formula for the coefficients ck,l does not depend on
which rectangle is chosen, the Fourier series expansion applies to all such rectangles, i.e., it applies to all x, y. (Note that the
summation formula in (2.26) is periodic with period (TX ,TY ). Note also that the integration in (2.27) can be taken over any
rectangle of width TX and height TY . Finally, notice that the k,l s have energy TX TY over any TX TY rectangle. Therefore, they
each have unity power.)
One of the most important applications of the Fourier series is in the context of deriving sampling theorems.
Convergence
Because the harmonic complex exponentials are complete, equality in (2.26) holds in the sense that the difference between the
left and right-hand sides has zero energy. Equivalently, the error in the difference between g(x, y) and the summation of terms
from N to N converges to zero as N , i.e., kg gN k 0 as N for any square integrable g, where
gN (x, y) ,

N
X

N
X

ck,l e2(xk/TX +yl/TY ) .

k=N l=N

As in the 1D case, if g(x, y) satisfies additional conditions, then the right-hand side of (2.26) equals g(x, y) at all points x, y
where g(x, y) is continuous.
Certainly if g(x, y) is separable, then all convergence properties are inherited directly from the 1D case.
The Gibbs phenomena will also be present for signals with discontinuities.
If g(x, y) has continuous 1st and 2nd derivatives, then the 2D FS is pointwise and uniformly convergent.
This means that gN (x, y) g(x, y) for any x, y as N . (Not true for discontinuous images!)
Properties
If g(x, y) is real, then the coefficients are Hermitian symmetric: ck,l = ck,l , and we can write:
g(x, y) = c00 +

k=1

2|ck0 | cos(2(k/TX) x + ck0 ) +

k= l=1

2|ck,l | cos(2[xk/TX + yl/TY ] + ck,l ) .

The other usual 1D properties (time shift, conjugation, etc.) also all carry over directly.
If g(x, y) is separable, i.e., g(x, y) = gX (x)gY (y), where gX (x) has period TX and gY (y) has period TY , then ck,l is the product of
the 1D Fourier series coefficients of gX (x) and gY (y), i.e.,
"
#"
#
Z TX
Z TY
1
1
2xk/TX
2yl/TY
ck,l =
dx
dy .
gX (x) e
gY (y) e
TX 0
TY 0
Parsevals theorem follows directly from the orthogonality of the k,l s (see (2.23)):
1
TX TY

TY

TX

| g(x, y) |2 dx dy =

k= l=

|ck,l |2 .

This shows that power in the space domain equals the sum of the powers in each frequency component.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.41

...

Example: Consider the 2D bed of nails function:

y

g(x, y) =

n= m=

2 (x nTX , y mTY ) .
...

From (2.27), this g(x, y) has FS coefficients ck,l =

g(x, y) =

1
TX TY ,

so we can write:

k= l=

1
e2(xk/TX +yl/TY ) .
TX TY

(However, this g(x, y) is not square integrable, so the FS does not converge so the above equality is just an engineering convention.)
The comb or shah or impulse train function is defined as follows:

comb(x) =

n=

(x n) .

...
1

(2.28)
x

We often need non-integer spacing:

 



X
X
x
x
comb
=

n = TX
(x nTX ) .
TX
TX
n=
n=

(2.29)

The above FS example shows the following equality (it can be useful in deriving sampling theorems):
comb2

x y
,
TX TY

 
 

X
x
y
, comb
comb
=
TX
TY

Note that because comb2 (, ) is separable, so is its Fourier series.

k= l=

e2(xk/TX +yl/TY ) .

(2.30)

c J. Fessler, September 7, 2009, 22:51 (student version)

2.42

Example:
Consider the 2D signal g(x, y) shown below, defined to be unity where white, and zero where black.
Is this signal separable?

?? What is c00 ? ??

We calculate the coefficients using (2.27):

ck,l

( R1
x e2kx dx,
l=0

R01 1
=
e
dy dx =
2(kx+ly) x
dx,
l
6= 0
e
0
0
0 2l
y=0

1/2, k = l = 0

k=l=0
, l = 0, k 6= 0
k2
1/2,
i

,
l
=
0,
k
=
6
0
=
=
,
k = 0, l 6= 0
k2
l2


R 1 1  2kx

2(k+l)x

,
k = l 6= 0

e
dx,
l
=
6
0
l2

0 i2l

0,
otherwise.
Z

2(kx+ly)

So most of the coefficients are zero, except where l = 0 or or k = 0 or k = l.

Are the coefficients separable?

?? Thus

g(x, y) =

1 X 1
+
[ sin(2kx) + sin(2ky) + sin(2k(x y))].
2
k
k=1

FS synthesis g4(x,y)

g(x,y)
1

0.8

1.5

1.2

1.5

0.8

0.6
y

0.6
1

0.4

0.4
0.5

0.2

0.5

1
x

1.5

0.2

0.5

0
0

0.2
0

0.5

1
x

1.5

% fig_fs1.m Illustrate 2D FS
x = linspace(0,2,101); y = x; tx = 1; ty = 1;
[xx,yy] = ndgrid(x,y);
xm = mod(xx,tx); ym = mod(yy,ty);
% handy trick for periodic functions
gxy = ym < xm;
% think about this trick!
clf, subplot(221), imagesc(x,y,gxy), axis xy, axis square, colorbar
xlabel x, ylabel y, title g(x,y)
g = 1/2 * ones(size(xx));
% DC term
for k=1:4
g = g + 1/(k*pi) * (-sin(2*pi*k*xx) + sin(2*pi*k*yy) + sin(2*pi*k*(xx-yy)));
end
subplot(222), imagesc(x,y,g), axis xy, axis square, colormap gray, colorbar
xlabel x, ylabel y, title(sprintf(FS synthesis g_{%d}(x,y), k))
print(fig_fs1, -deps)

c J. Fessler, September 7, 2009, 22:51 (student version)

2.43

Fourier series for finite support functions

The preceding pages describe how to expand a periodic function using Fourier series. Using the above analysis, it is easy to see
that one can also use Fourier series to describe a finite support function. Indeed, mathematical treatments of Fourier series usually
focus on finite support cases.
If g(x, y) is a 2D function with support [a, a + TX] [b, b + TY], then we can represent g(x, y) using a 2D Fourier series as follows:
 P
P
2(xk/TX +yl/TY )
, a x a + TX & b y b + TY
k=
l= ck,l e
g(x, y) =
0,
otherwise
ck,l =

1
TX TY

a+TX
a

b+TY

g(x, y) e2(xk/TX +yl/TY ) dx dy .

Notice that the above synthesis expression is almost identical to (2.26) except for the braces to limit the support, and the analysis
expression is similar to (2.27) except that the region of integration covers the support.
If we were given only the Fourier series coefficients {ck,l } and asked to synthesize g(x, y), we would need to ask for additional
(support) information to determine whether to use (2.26) or the above version with braces. Of course the two formula are identical
over the support, so the distinction between the two is important only if one might evaluate the synthesis expression for arguments
outside the support.
This discussion is a preview of the discrete Fourier series (DFS) and the discrete Fourier transform (DFT). Those two transforms also involve synthesis formulae that are identical except for support considerations.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.44

Fourier transforms
There are many transforms that are useful for image analysis. Of these, why is the Fourier transform particularly important?
Reason 1: Convolution Property
A shift-invariant PSF reduces the entire system equation to a convolution integral. Such a system is easier to analyze in the
Fourier domain, because convolutions in the space domain become simply multiplications in the Fourier domain. That is, the
transform of the output function is the transform of the input function multiplied by the system transfer function H, the transform
of the PSF.
The preceding concept is the most important property of Fourier transforms for our purposes.
Reason 2: Eigenfunctions of LSI systems
As shown previously, complex exponential signals are the eigenfunctions of LSI systems, and the eigenvalues corresponding to
those eigenfunctions are exactly the values of the FT of the impulse response at the frequency of the signal:
e2(ux+vy) LTI h(x, y) H(u, v) e2(ux+vy) ,
F

2
H(u, v). (See derivation preceding (2.25).) The fact that the FT formula (2.25) fell out of the convolution
where h(x, y)
with the complex exponential signal is by itself a compelling motivation to study that integral further.

The above two reasons are why Fourier methods are so important for analyzing LSI systems.
Two-dimensional Fourier transform
The continuous-space 2D Fourier transform (2D FT) of a function g(x, y) is another function G(u, v) defined as
Z

G(u, v) = F2 [g(x, y)] =

g(x, y) e2(ux+vy) dx dy . (analysis equation)

The inverse Fourier transform of G(u, v) is defined by

g(x, y) =

F21 [G(u, v)]

G(u, v) e2(ux+vy) du dv . (synthesis equation)

Like the Fourier series, the above is a decomposition into complex exponentials, except that now there are a continuum of exponentials, combined with an integral, rather than a sum.
Other notation:
G = F2 [g],

G(u, v) = (F2 [g])(u, v),

g = F21 [G],

2
G(u, v),
g(x, y)

g(x, y) = (F2 [G])(x, y),

F

2
G(u, v) .
g(x, y)

We refer to u and v as spatial frequencies by analogy to 1D transforms where = 2f where f is frequency.

What are the units of u and v? Hint: the units of x and y are distances, for example centimeters.

??

Physically, each component

G(u, v) can be considered that component of the function g(x, y) resulting from a plane wave of

wavevector K = 2 u2 + v 2 and propagating along direction , where = tan1 (v/u).

What are the units of G(u, v)?

??

c J. Fessler, September 7, 2009, 22:51 (student version)

2.45

Relation to 1D Fourier transform

One can think of the 2D Fourier transform as resulting from first applying the 1D Fourier transform to g(x, y) with x as the
independent variable and y as a constant, to obtain a function of u and y, and then applying the 1D Fourier transform to this
function with y as the independent variable and u as a constant. That is,
G(u, v) =

F1D,y {F1D,x {g(x, y)}}


Z Z
g(x, y) e2ux dx e2vy dy .

where F1D,y and F1D,x denote the 1D Fourier transforms with respect to x and y, respectively.
Relation to Fourier series
One can derive the 2D FT from the Fourier series by letting the period(s) go to infinity, in analogy to the usual 1D derivation.
Here is a pseudo-proof that F21 [F2 [g]] = g:
F21 [F2 [g]]

=
=
=
=
=

lim

lim

T
Z T

T
Z T
T

G(u, v) e2(ux+vy) du dv
Z

g(x , y )
g(x , y )

"
h

g(x , y ) e2(x u+y v) dx dy e2(ux+vy) du dv

#
Z Z

lim

e2[(xx )u+(yy )v] du dv dx dy

i
lim T 2 sinc2 (T (x x ), T (y y )) dx dy

g(x , y ) 2 (x x , y y ) dx dy = g(x, y) .

(This is typical of how FT properties are shown, by exchanging order of integration, changes of variables, etc.) To make this
rigorous, one must identify regularity conditions on g(x, y) that allow the exchange of order of limits and integration.
Existence
An integral with infinite limits represents a shorthand for a limit of an integral with endpoints that go to infinity, as seen above.
Anytime one works with such integrals, for a mathematically rigorous treatment one should pause and consider when the limit
(e.g., limT above) actually exists.
There are two common sets of sufficient conditions that ensure that the 2D Fourier transform of an image g(x, y) exists and that
the inverse transform relation also holds.
1. g(x, y) has finite energy, i.e., it is square integrable.
In this case, the synthesis equation says that g(x, y) equals the synthesis integral in the sense that their difference has zero
energy.
2. Each of the following holds:
g(x, y) is absolutely integrable (over all of R).
g(x, y) has only a finite number of discontinuities and a finite number of maxima and minima in any finite region.
g(x, y) has no infinite discontinuities.

In this case, the Fourier transform exists and the synthesis integral holds exactly at all x, y except where there is a discontinuity in g(x, y).

We will also apply F2 to functions that satisfy neither of these conditions, such as impulse functions.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.46

Null functions

(a cautionary note)

Do the above conditions guarantee that taking a FT of a function g and then taking the inverse FT will give you back
exactly the same function g? No! Consider the function

1, x = 0, y = 0
g(x, y) =
0, otherwise.
Because this function (called a null function) has no area, G(u, v) = 0, so the inverse FT g =RR
F21 [G] is simply g = 0, which
2
2
does not equal g exactly! However, g and g are equivalent in the L2 (R ) sense that kg gk = (g g)2 = 0, which is more
than adequate for any practical problem.

If we restrict attention to continuous functions g (that are absolutely integrable), then it will be true that g = F21 [F2 [g]].
Most physical functions are continuous, or at least do not have type of isolated points that the function g above has. Moreover,
most physical systems are insensitive to the addition of a null function to its input. Therefore the above mathematical subtleties
need not deter our use of transform methods.
We can safely restrict attention to functions g for which g = F21 [F2 [g]] or at least to functions such that g = F21 [F2 [g]] + n,
where n is a null function.
R
Lerchs theorem: if f and g have the same Fourier transform, then f g is a null function, i.e., |f (x) g(x)| dx = 0.
In 1D, when there is a (finite) discontinuity in the signal, taking the FT and then the inverse FT recovers the midpoint of the
discontinuity. What is the corresponding result in 2D?
Conjecture:
1
0 2

(F21 [F2 [g]])(x, y) = lim

g(x + cos , y + sin ) d.

Exercise: prove or disprove.

Example. Here is the FT of an MR head image (thanks to Doug Noll), and the corresponding images synthesized from only the
low spatial frequencies and only the high spatial frequencies.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.47

2D Fourier transform properties

F

2
G(u, v) to indicate that G is the FT of g, i.e., G = F2 [g].
We use the notation g(x, y)

Properties that are analogous to 1DFT properties

Linearity The 2D Fourier transform operation is linear:
X
k

2
k gk (x, y)

Convolution

k Gk (u, v) .

2
G(u, v) H(u, v)
(g h)(x, y)

The convolution of two images has a spectrum that is the product of the spectra of the two images. This is a very important relation
for imaging.
Cross correlation [2, p. 47]

(g h)(x, y) =

g (x , y ) h(x + x, y + y) dx dy

Z
Z

g (x x, y y) h(x, y) dx dy

G(u, v) H (u, v) .

The complex conjugate product of the spectra of two signals is the FT of their complex cross-correlation function.
Cross-correlation is used for object detection via matched filtering. Discrete-space implementation is M ATLABs xcorr2 command.
An important special case is image autocorrelation:
Z Z
F2
| G(u, v) |2 .
(g g)(x, y) =
g (x , y ) g(x + x, y + y) dx dy

Although | G(u, v) |2 is often called the power spectral density, the term energy density spectrum is more appropriate.
Magnification (scaling)
F

2
For , 6= 0, g(x, y)

??

Stretching in one domain proportionally contracts in the other domain, together with a constant weighting factor.
Shift (in space)
F

2
G(u, v) e2(ua+vb)
g(x a, y b)

Translation of a function in one space introduces a linear phase shift in the transform domain.
Is F2 a shift invariant operator? ??
Derivative

d
F2
2u G(u, v)
g(x, y)
dx

DC value (historical term, not necessarily any current here.)

Z Z
G(0, 0) =

g(x, y) dx dy .

c J. Fessler, September 7, 2009, 22:51 (student version)

2.48

Duality
F

2
2
g (u, v) .
G(u, v), then G (x, y)
If g(x, y)

Self similarity
F

2
G(u, v), then g(x, y) + g(x, y) + G(x, y) + G(x, y) is its own Fourier transform!
If g(x, y)

Example: Gaussian, comb, derivative of Gaussian

Parsevals Theorem or Power Theorem
Z Z
Z
g(x, y) h (x, y) dx dy =

G(u, v) H (u, v) du dv

in particular: Rayleighs theorem (1889):

Z Z

| g(x, y) | dx dy =

| G(u, v) |2 du dv .

Exercise. Prove Parsevals theorem for the 2D FT.

Multiplication (dual of convolution property)
F

2
G(u, v) H(u, v) .
g(x, y) h(x, y)

Modulation (dual of shift property)

F

2
G(u u, v v) .
g(x, y) e2(ux+vy)

This property is at the heart of how holography works, and is important in deriving sampling theorems.
Hermitian symmetry properties
Conjugation
F

Proof.

RR

g (x, y) e2(ux+vy) dx dy =

Hermitian symmetry

RR

2
G(u, v) .
g (x, y)

g(x, y) e2[(u)x+(v)y] dx dy

= G (u, v) .

If g(x, y) = g (x, y), then G(u, v) is real.

(This follows readily from the conjugation property.
By duality, if g(x, y) is real (which is usually the case), then G(u, v) is Hermitian symmetric.
There are many other such relationships, such as: if g(x, y) = g(x, y) then G(u, v) is even.
Example. One application of the Hermitian property is in half k-space MRI, where one acquires only half of the Fourier samples
of an (assumed real) object, and fills in the other half using Hermitian symmetry [10, 11].

c J. Fessler, September 7, 2009, 22:51 (student version)

2.49

Generalized Fourier transforms

There are a number of important images for which, although not square or absolutely integrable, we would nevertheless like to use
Fourier theory. These include constants, exponentials, sinusoids, and periodic functions. In addition, e would like to apply Fourier
theory to the Dirac impulse and its cousin the comb function. In all such cases, the Dirac impulse and its defining properties play
a principal role. Here is a list of such functions and their Fourier transforms.
Constant function
F

2
2 (u, v) .
1

Impulse function
F

2
1,
2 (x, y)

2
e2(ua+vb) .
2 (x a, y b)

Complex exponential
F

2
2 (u u0 , v v0 ) .
e2(xu0 +yv0 )

Sinusoid
F

2
cos(2[xu0 + yv0 ])

1
[2 (u u0 , v v0 ) + 2 (u + u0 , v + v0 )] .
2

Periodic signals
One approach to finding the 2D FT of a periodic signal uses the 2D FS:
g(x, y) =

k= l=

2
G(u, v) =
ck,l e2(xk/TX +yl/TY )

k= l=

ck,l 2 (u k/TX , v l/TY ) .

(2.31)

Often periodic signals arise as the superposition of shifted replicates of a basic pattern:
g(x, y)

n= m=

"

f (x nX , y mY )

(2.32)
#



x
y
1 1
= f (x, y)
comb2
,
2 (x nX , y mY ) = f (x, y)
X Y
X Y
n= m=
F

G(u, v) = F (u, v) comb2 (X u, Y v)

X
X
1
=
F (k/X , l/Y ) 2 (u k/X , v l/Y ) .
X Y

(2.33)

k= l=

Such a replicated pattern signal is periodic with period (TX , TY ) = (X , Y ).

Comparing the right hand side of (2.31) and (2.33), we see that the FS coefficients of g(x, y) are samples of the FT of the pattern
f:


l
k
1
.
F
,
ck,l =
X Y
X Y
This expression is usually the easiest way to find 2D FS coefficients of a periodic signal of the form (2.32).
Comb

(an application of the previous result)

F

2
comb2 (u, v) .
comb2 (x, y)

What are the values of X ,Y and f here?

??

c J. Fessler, September 7, 2009, 22:51 (student version)

2.50

Example. (This example arose in the analysis of B-spline image interpolation.)

P
3
n
Find the Fourier series representation of the periodic signal g(x) =
n= (1) sinc (x n) .
P
P

3
3
First note that g(x) = l= sinc (x 2l) sinc (x 2l 1) = l= f (x 2l), so g(x) is 2-periodic,


F
where f (x) = sinc3 (x) sinc3 (x 1) F (u) = [rect(u) rect(u) rect(u)] 1 e2u .
1


k
2 , k = 1
=
Hence the FS coefficients are ck = 12 F k2 = [rect rect rect](k/2) 1(1)
2
0, otherwise.
Thus by FS synthesis: g(x) = 21 e2x/2 + 21 e2x/2 = cos(x) .
P
3
n
In summary, we have the following remarkable equality: cos(x) =
n= (1) sinc (x n) .
Example. The following figure illustrates some of the 2D FT properties.

modulate

shift

scale

rect2(x,y)

g(x,y)

|G(u,v)|

Real[G(u,v)]

Imag[G(u,v)]

c J. Fessler, September 7, 2009, 22:51 (student version)

2.51

Properties that are unique to the 2D FT

Rotation
F

2
G(u, v), then (see (2.14)):
If g(x, y)

2
G (u, v) = G(u cos + v sin , u sin + v cos ) .
g (x, y)

(2.34)

Exercise. Prove the rotation property of the 2D FT.

Rotational symmetry
If an image g(x, y) has n-fold rotational symmetry, then so does its 2D FT G(u, v).
This is a trivial consequence of the preceding property.
Separability (Cartesian)
Images that are (Cartesian) separable have (Cartesian) separable Fourier transforms:
F

2
G(u, v) = GX (u)GY (v),
g(x, y) = gX (x)gY (y)

where GX and GY are the 1D Fourier transforms of gX (x) and gY (y) respectively.
Separable functions are often used in filtering, where the filtering operation is a convolution such that
F

2
G(u, v) H(u, v) = G(u, v) HX (u)HY (v).
(g h)(x, y) = g(x, y) [hX (x)hY (y)]

We would also like a result for polar separable images. To form such a result, we must first digress into the Hankel transform.
Hankel transform
On [12, p. 360: 9.1.21] of Abramowitz and Stegun, the kth-order Bessel function Jk is expressed:
Z
Z
Z 2
k z cos
()k 2 z cos k
1
Jk (z) =
e
cos(k) d =
e
e d =
ez sin ek d .
0
2
2 0
0
(Note that Jk (z) = (1)k Jk (z). And Jk solves z 2 f + z f + (z 2 k 2 )f = 0.)
The kth-order Hankel transform of the 1D function g(r) is defined by:
Gk () = 2

g(r)Jk (2r)r dr .

(2.35)

The most important property of the kth-order Hankel transform is that it is related to the 2D FT as follows:
F

2
F (, ) = ek ()k Gk (),
f (r, ) = g(r) ek

(2.36)

where we write the spectrum

in polar coordinates as follows: F (, ) = F ( cos , sin ) (adopting the standard notation

recycling), where = u2 + v 2 and = tan1 (v/u).

Proof:
F (, ) =
=

f (x, y) e

Z 2
Z

g(r)

ek

g(r)

2(x cos +y sin )

dx dy =


Z
ek e2r cos() d r dr = ek

Z

f (r, ) e2r(cos cos +sin sin ) r d dr

g(r)

Z

ek e2r cos() d r dr = ek ()k 2

0
k


ek() e2r cos() d r dr
g(r)Jk (2r) r dr = ek ()k Gk ().

A function having the form f (r, ) = g(r) e is called circularly harmonic, and the result (2.36) shows that circularly harmonic
functions have circularly harmonic transforms.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.52

Separability (Polar)
If g(x, y) is separable in polar coordinates, i.e., if g(r, ) = gR (r) g (), where g () is 2-periodic, then it is natural to express
the 2D FT in polar coordinates (although in general the 2D FT is not quite separable in those coordinates).
Because g () is 2-periodic, we use the 1D Fourier series to express it in terms of its angular harmonics:

g () =

ck ek , where ck =

k=

1
2

g () ek d .

Thus a polar separable function g has the following 2D FT:

g(r, ) =

k=

X
 F2

G(, ) =
ck ek ()k Gk ().
ck gR (r) ek
k=

This expression is the sum of polar separable functions, but is not itself polar separable.
Circular Symmetry
By far the most frequent use of polar separability is in problems where g(x, y) has circular symmetry, so g () = 1 and
g(x, y) = g(r). In this case only the k = 0 term above is nonzero, so the 2D FT is also circularly symmetric:
Z
F2
g(r)J0 (2r) r dr,
G(, ) = G() = 2
g(x, y) = g(r)
0

where J0 is the 0th-order Bessel function, defined by [12, p. 360: 9.1.18]:

J0 (z) =

cos(z sin ) d =

Thus
G() = 2

cos(z cos ) d =

1
2

ez cos d =

g(r)J0 (2r) r dr .

Similarly: g(r) = 2

1
2

ez sin d .

G()J0 (2r) d .

(2.37)

This is called the Hankel Transform (of zero order). It is often called the Fourier-Bessel transform in lens/diffraction theory.
Example: The functions g (r) = 1/r for (0, 2) are important in tomography [13] [14, p. 414]. Their Hankel transforms are
G () = 2

where r = r and c , 2
Duality

R
0

1
J0 (2r) r dr = 2
r

J0 (2r)r1 dr =

c
1
r dr
J
(2r
)
= 2 ,
0

(r /)

2 (/2) sin(/2)

. Note that c1 = 1 because (1/2) =

(for circularly symmetric functions)

F

2
2
g (r).
G() G (r)
g(r)

......................................................................................................................
Exercise. Find a version of Parsevals theorem that is appropriate for circularly symmetric functions.
What is the scaling property for the Hankel transform?

??

There are fast Hankel transform methods [15] that facilitate the numerical use of (2.35).

c J. Fessler, September 7, 2009, 22:51 (student version)

2.53

Example:
F2

Pillbox: rect(r) jinc() ,

J1 ()
2 ,

/4,

6= 0
= 0,

2
e
Gaussian: er

Hankel Transform: G()

1D Function
1

g(r) = rect(r)

G() = jinc()

0
0
0

0.5

1
r

1.5

2D image: g(x,y)

2D Fourier Transform: G(u,v)

0.5

0.5

0
1

0
1

0
y

0
5

The Hankel transform and the 2D FT are intimately related.

Hankel transform in higher dimensions
p
2
2
2
In dimensions d > 2, if a function has spherical
p symmetry, i.e., f (x1 , x2 , . . . , xd ) = f0 ( x1 + x2 + + xd ), then so will its
Fourier transform F (u1 , u2 , . . . , ud ) = F0 ( u21 + u22 + + u2d ), and [16, p. 1836]:
Z
2
f0 (r)Jd/21 (2r)rd/2 dr .
F0 () = d/21

0
OTF and MTF
For LSI imaging systems, the optical transfer function (OTF) is defined as the normalized frequency response
OTF(u, v) ,

H(u, v)
,
H(0, 0)

and the modulation transfer function (MTF) is defined as its magnitude:

MTF(u, v) , |OTF(u, v)| =
Why? See [17, p. 98].

| H(u, v) |
.
| H(0, 0) |

c J. Fessler, September 7, 2009, 22:51 (student version)

2.54

2D FT Examples
Circularly symmetric functions
Spectrum

Function
Separable functions
Spectrum

Function

1
2 (r)
r

2 (x, y) =

2 (u, v)

comb2 (x, y)

comb2 (u, v)

(r r0 ), r0 > 0

2r0 J0 (2r0 )

1
r
1
, 0<<2
r

er

2
2
(/2) sin(/2)

rect(r)
rect2 (x, y)

sinc2 (u, v)

e|x| e|y|

2
2
2
1 + (2u) 1 + (2v)2

ear
er

sinc2 (u, v) , sinc(u) sinc(v),

e [1, p. 274]

sinc(t) ,

G(u, v) = sinc2 (u, v)

g(x, y) = rect2 (x, y)

sin t
t ,

1,

t 6= 0
t = 0.
G() = jinc()

g(r) = rect(r)

0.5

0.5

0.5

0.5

0
y 1 1

0
5

v 5 5

0
y 1 1

0.5

0.5

2
0.5

0
x

0.5

0.5

0.5
2

v 5 5

0.5
y

0.5

0
5

0.5

0
1

0.5

0
x

0.5

0
1

J1 ()
2 ,

6= 0
/4,
= 0,
J1 (2)
4 jinc(2) =

2a
[(2)2 + a2 ]3/2
jinc() =

circ(r) = rect(r/2)

0.6

0.4

0.2

c J. Fessler, September 7, 2009, 22:51 (student version)

2.55

Other 2D transforms
A number of other 2D continuous-space transforms exist, such as the following.
2D Laplace transform
One could define a (bilateral) 2D Laplace transform as follows:
Z Z
G(s1 , s2 ) =
g(x, y) e(s1 x+s2 y) dx dy,

where s1 and s2 are both complex numbers.

A primary use of 1D Laplace transforms is in the solution of 1D differential-equations (and analysis of LTI systems described by
differential-equations). Because 2D LSI systems are rarely described by differential equations, the 2D Laplace transform is much
less frequently used.
Hartley transform
The 2D Hartley transform and its inverse are defined by:
Z Z
H(u, v) =
g(x, y) cas(2(ux + vy)) dx dy

g(x, y) =

H(u, v) cas(2(ux + vy)) du dv,

where cas() , cos + sin . The Hartley transform of a real image is itself real, and fast algorithms exist, e.g., [15, 18].
Radon transform
The Radon transform of a 2D image f (x, y) is the collection of all line integral projections through that image, defined as
follows:
Z
p (r) =

f (r cos sin , r sin + cos ) d .

This transform is the foundation for tomographic imaging like X-ray computed tomography (CT).
Projection slice theorem
Because p (r) is a 1D function of r (for fixed ), it has a 1D Fourier transform P (u). Of course the 2D image f (x, y) has a 2D
Fourier transform F (u, v) = F (, ). The projection slice theorem states that the 1D FT of p (r) is a slice through the 2D
Fourier transform of f (x, y):


P (u) = F (, ) 
.
=u

The field of tomographic imaging essentially is built upon this property, because it shows that the set of line-integral projections
(at all angles ) contain sufficient information to describe the 2D FT of an image f (x, y), and hence f (x, y) itself.
Frequency distance principle
This is an interesting topic related to what happens if you take the 2D FT of a sinogram p (r). See [1921].

c J. Fessler, September 7, 2009, 22:51 (student version)

2.56

Bandwidth and time-bandwidth products

1D bandwidth relationships
There are many definitions of bandwidth (frequency domain) and pulse width (time domain).
For symmetric spectra, root mean-squared bandwidth or RMS bandwidth is a natural definition:
vR
u 2
u u |G(u)|2 du
.
urms , t R
2
|G(u)| du

(2.38)

Example. Gaussian signal. For t0 > 0:

g(t) = e(t/t0 ) G(u) = t0 e(ut0 ) .

(2.39)

The RMS bandwidth is:

(Use change of variables x = ut0 .)
vR
vR
v R
s
u 2
u 2 2 2(ut )2
u 1 2 2x2
0
u u | G(u) |2 du u
u
x
e
dx
u
t
e
du
1 1
1
2/(8)
0
R

.
urms = t R
= t R 2 2(ut )2
= t t0
=
=
2
2
2x
0
t0
t0 2
du
dx
t0 e
2/2
| G(u) | du
t0 e
For symmetric signals, root mean-squared pulse width or RMS pulse width is natural: trms

vR
u 2
u t |g(t)|2 dt
.
, t R
2
|g(t)| dt

Example. Gaussian signal in (2.39) above. The RMS pulse width is:
(Use change of variables x = t/t0 .)
v R
vR
vR
u 3 2 2x2
u 2 2(t/t )2
u 2
0
u t |g(t)|2 dt u
x e
dx
t0
t e
dt u
1
t
t
R
R
= t
= t0 .
=
trms =
R
2
2
2
2(t/t
)
2x
0
2

dt
dx
t0 e
e
|g(t)| dt

Time-bandwidth product

Using the Cauchy-Schwarz inequality, one can show [22, p. 178] (for signals satisfying certain regularity conditions):
urms trms

1
.
4

(2.40)

This uncertainty relation is similar to the Heisenberg uncertainty principle. For discrete-time and other extensions, see [23].
So a signal cannot be localized in both time and in frequency. Extreme cases:
impulse function - narrow in time, wide spectrum
sinusoid signal - periodic (infinite duration) in time, narrow spectrum
gaussian signal: time-bandwidth product is as narrow as possible
Example. For a Gaussian signal: 2urms trms = 1/2, which is optimal in a time-bandwidth sense.
F

Example. g(t) = tri(t) G(u) = sinc2 (u).

R1 2

t (1t)2 dt
1
t2rms = 0R 1 (1t)2 dt = 1/30
1/3 = 10 , so trms = 1/ 10
0
rR
2

u |sinc2(u)|2 du
0R
2urms = 2
= 3 1.73.

sinc4(u) d
0

Thus 2urms trms =

n-dimensional case

0.32.

p
3/10 0.548 1/2, but fairly close.

(with possibly
asymmetric signals and spectra)
vR
vR
u
u
2
2
2
2
u k~x
u
k~u ~k |G(~u)| d~u
~ k |g(~x)| d~x
t
and
t
,
inf
are natural
For n-dimensional signals, urms , infn t R
R
rms

2
2

~ Rn

~ R
u)| d~u
x)| d~x
|G(~
|g(~
n
definitions, where, for n = 2, ~x = (x, y) and ~u = (u, v). And one can show [14, p. 108] that urms trms
.
4

c J. Fessler, September 7, 2009, 22:51 (student version)

2.57

Sampling
Thus far we have described images, such as the output of an image capture system, as a continuous functions g(x, y). This is
convenient for analysis, but in any digital imaging system, we can only record a (finite) 2D array of numbers {gd [n, m]} that
should somehow represent g(x, y).
A word about notation in this section is in order. When considering primarily continuous-space images, we use g(x, y) rather than
the more cumbersome ga (x, y). When considering primarily discrete-space images (later on), we will use g[n, m] rather than the
more cumbersome gd [n, m]. But in this section we require both continuous- and discrete-space, so we include the subscripts for
clarity. The following table summarizes the various functions considered.
Signal domain
Continuous space
Impulse synthesized
Discrete space

signal
ga (x, y)
gs (x, y)
gd [n, m]

transform
Ga (u, v)
Gs (u, v)
Gd (X , Y )
G[k, l]

transform type
2D FT
2D FT
2D DSFT
2D DFT

As a realistic concrete example, in some digital mammography systems, the intensity of a transmitted X-ray beam is recorded in
analog format on some continuous detector (e.g., photographic film, or a rewritable equivalent). The intensity that is recorded
is our ga (x, y). Then a digitization device (such as a laser-scanning system or CCD camera) samples ga (x, y) over a regular grid,
and sends (quantized) values to the digital computer.
Body

Intensity

Discrete Detectors

ga (x, y)

Point Samples

gd [n, m]

X-ray Source

Example. An optical scanner is another device that converts a continuous, analog image (picture) into a sampled, digital form.
Ideal uniform rectilinear sampling
The most common4 form of 2D sampling uses a 2D uniformly sampled grid, which matches raster-scan displays and printers, and
is convenient for storage. (It is usually not the most efficient sampling scheme however, as shown in Lim Pr. 1.35.)
Let X and Y be the sampling intervals in the x and y directions respectively, and define:
gd [n, m] = ga (nX , mY ) .
This equality represents idealized point sampling.
The ratios 1/X and 1/Y can be called the sampling rates in the x and y directions, respectively.
What are the units of these sampling rates?

??

However, especially when considering non-Cartesian sampling, it is more relevant to quote the sampling density of a given
samping pattern, which will have units samples per unit area. For rectiliniear sampling, the sampling density is 1/(X Y ).
Non-ideal sampling
Exercise. Generalize the three main results of Shannons ideal impulse sampling theory to the more realistic case where we
account for finite detector size in the sampling relationship:
Z (m+1/2)Y Z (n+1/2)X
1
gd [n, m] =
ga (x, y) dx dy .
X Y (m1/2)Y (n1/2)X
(Note: even further generalizations are possible, e.g., [25].)
4 That is, the most common in digital systems. In human retinas, the sampling of rods and cones is systematically nonuniform (higher concentration in fovea),
and some semiconductor sensors have been developed that mimic this nonuniform sampling property [24].

c J. Fessler, September 7, 2009, 22:51 (student version)

2.58

What are the questions to answer when analyzing sampling?

Q1. ??

??
Q3. ??
Q4. ??
Q2.

For analysis, it is convenient to use a grid of Dirac impulses (bed of nails) to synthesize the following continuous-space function
gs (x, y) from the samples {gd [n, m]}. This manipulation is purely analyticalno such impulses exist in practice. The hypothetical
sample carrying continuous-space image is defined by
gs (x, y)

n= m=

= ga (x, y)

= ga (x, y)

gd [n, m] X Y 2 (x nX , y mY )

n= m=

X
X

n= m=

= ga (x, y) comb2

(2.41)

X Y 2 (x nX , y mY ) by sampling property of 2 (x, y)

2 (x/X n, y/Y m) by scaling property of 2 (x, y)

x
y
,
X Y



(2.42)

2
comb2 (u, v), it follows from the modulation and scaling properties of the 2D FT that the spectrum
Because comb2 (x, y)
Gs (u, v) of the hypothetical impulse sampled image gs (x, y) is related to the spectrum Ga (u, v) of the original image ga (x, y) as
follows:

2
Gs (u, v) =
gs (x, y)

Ga (u, v) [X Y comb2 (uX , vY )]

Ga (u k/X , v l/Y ),

2 (u k/X , v l/Y ) .

k= l=

because
X Y comb2 (uX , vY ) =

k= l=

(2.43)

Thus Gs (u, v) consists of the sum of shifted copies of the spectrum Ga (u, v) as illustrated below.

Before

After sampling/synthesizing

...
Ga (u, v)

v
Gs (u, v)

v max

...

u
1/Y
1/X

max

...
Now we can answer the questions posed above.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.59

Q1. Can we recover ga (x, y) from gd [n, m]?

In general one cannot recover ga (x, y) from the samples {gd [n, m]}, no matter how closely spaced. However, there are classes of
functions for which it is possible. One such class, perhaps the most important one, is the class of band-limited functions.
We say ga (x, y) (or Ga (u, v)) is band-limited to (umax , v max ) iff Ga (u, v) = 0 for |u| umax or |v| v max . (Picture)
Q2. How finely must we sample?
If Ga (u, v) is band-limited to (umax , v max ) and if 1/X > 2umax and 1/Y > 2v max , then there is no overlap of the replicates
of Ga (u, v) in Gs (u, v). Overlap is called aliasing (higher spatial frequencies will alias as lower spatial frequencies).
2 max{umax , v max } is called the Nyquist sampling rate (twice the highest spatial frequency).
If X < 1/(2umax ) and Y < 1/(2v max) then we say the image (or system) is adequately sampled, otherwise we say the image
is undersampled.


Example. Consider sampling the image g(x, y) = cos 2[ 21 X + X ]x + 2[ 21 Y + Y ]y with sampling intervals X and Y ,
where 0 < X <

1
2X

and 0 < Y <

1
2Y .

Then, g(x, y) is undersampled, producing samples

gd [n, m] =
=



1
1
cos 2[
+ X ]nX + 2[
+ Y ]nY
2X
2Y
cos(n + X nX + n + Y nY ) .

At 
these sampling intervals, another lower
 frequency image has the same samples. Specifically, consider the signal f (x, y) =
1
1
cos 2[ 2X X ]x + 2[ 2Y Y ]y , whose samples are
fd [n, m] =
=
=
=



1
1
X ]nX + 2[
Y ]nY
cos 2[
2X
2Y
cos(n X X n + n Y Y n)

cos(X X n n + Y Y n n) because cos() = cos()

cos([X X + 2]n n + [Y Y + 2] n)

cos(X X n + n + Y Y + n)

gd [n, m] .

That is, f (x, y) and g(x, y) have the same samples. They are said to be aliases of each other. Alternatively, we say that g(x, y) has
aliased to f (x, y).
Non-band-limited images
If ga (x, y) is not band-limited, then in general no algorithm can recover ga (x, y) from gd [n, m] perfectly.
Other information (such as nonnegativity) would be required. For example, an active area of research in signal processing is
to consider signal models based on assumptions of sparsity with respect to some representation. In these compressive sensing
formulations, one usually still uses linear measurements, but uses a nonlinear signal recovery method, e.g., [26].
However, for reasonably behaved images g(x, y), one can approximately recover g(x, y) from its samples by taking X and Y
sufficiently small. In fact, the energy between g(x, y) and the recovered approximation from the samples can be made arbitrarily
small by making X and Y small enough. This is the subject of approximation theory and it is relevant to signal and image
processing because rarely are signals exactly band-limited.
Avoiding aliasing
How can we avoid aliasing for non-band-limited images?

??

c J. Fessler, September 7, 2009, 22:51 (student version)

2.60

Signal reconstruction/interpolation
Q3. How can we recover a (bandlimited) image ga (x, y) from its samples {gd [n, m]}?
(Of course we cannot store all of ga (x, y) on a computer, but we might want values of ga (x, y) on a grid finer than the original
sampling. If the function is bandlimited and adequately sampled then this can be done!) Or we might like to design a suitable 2D
D/A converter (i.e., a video monitor) that takes a digital input signal gd [n, m] and recreates the analog ga (x, y) at its output.
The key to methods for recovering ga (x, y) from {gd [n, m]} is the following fact.
If Ga (u, v) is bandlimited to (1/(2X ), 1/(2Y )) (and hence adequately sampled), then
Ga (u, v) = Gs (u, v) rect2 (uX , vY ).

(2.44)

This relationship follows directly from the form of Gs (u, v) given in (2.43) above.
The ideal lowpass filter H(u, v) = rect2 (uX , vY ) selects out the central replicate. This replicate is precisely the spectrum
Ga (u, v) of the original signal. This is easily seen in the spectrum, but is perhaps puzzling in the space domain!
Frequency-domain reconstruction method:
Construct gs (x, y) from samples {gd [n, m]}.
Take 2D FT of gs (x, y) to get Gs (u, v)
Truncate Gs (u, v) to the bandlimited region (set everything outside to zero) to get Ga (u, v).
Take inverse Fourier transform to get ga (x, y).
This method is impractical because Ga (u, v) is continuous; on a computer one could store only a finite number of samples of
the spectra, so one would not recover ga (x, y) exactly at any coordinates. (However, a discrete approximation to this approach is
sometimes applied; see M ATLABs interpft routine.)
Space-domain reconstruction method:
Taking the inverse 2D FT of key property (2.44) above yields:



1
x
y
ga (x, y) = gs (x, y)
sinc2
,
X Y
X Y
"
#


X X
=
gd [n, m] X Y 2 (x nX , y mY )
n= m=

ga (x, y) =

gd [n, m] sinc2

n= m=

x nX y mY
,
X
Y



x
y
1
sinc2
,
X Y
X Y

(2.45)

Thus we can recover ga (x, y) by interpolating the samples gd [n, m] using sinc functions. This is called sinc interpolation.
The sinc function has infinite extent, so in practice approximate interpolators are used more frequently (such as B-splines). See
M ATLABs interp2 routine.
Non-rectangular spectral support (over sampling)
If Ga (u, v) is band-limited with a support region that is smaller than [1/(2X ), 1/(2X )] [1/(2Y ), 1/(2Y )], the usual
rectangle, then (2.44) is not the only approach that will work in the frequency domain. In fact, there will be many ideal interpolators.

Example. If X = Y = and Ga (u, v) is zero when u2 + v 2 > 1/(2), (Picture) , then we could choose to replace the rect
function in (2.44) with a circ function rect(). The space-domain reconstruction formula corresponding to (2.45) becomes
ga (x, y) =

gd [n, m] jinc(?)

n= m=

because

1
2

2
rect() . This is called jinc interpolation.
jinc(r/)

?? ,

(2.46)

c J. Fessler, September 7, 2009, 22:51 (student version)

2.61

The following figures compare sinc interpolation with some of the practical alternatives available with M ATLABs interp2
routine.

Nearest
10

10

6
x

8 10

5
10
flop 24012

5
10
flop 44971

Sinc

Cubic

True g (x,y)

10

10

10

Linear

10

Samples of ???

5
10
flop 157201

5
10
flop 2883225

Why are M ATLABs choices so poor in this example?

5
x

??

What causes the remaining artifacts in the sinc interpolated image?

??

One should not conclude from this example that sinc interpolation is always preferable!
Alternative reconstruction methods
Other reconstruction methods, generally referred to as interpolation methods, will be discussed in Chapter 8.

10

c J. Fessler, September 7, 2009, 22:51 (student version)

2.62

Video monitors
Standard computer monitors reconstruct an analog image from discrete-space data gd [n, m], but they do not use the above sinc
interpolation method. Typically these monitors use a 2D version of sample and hold, which is mathematically equivalent to
convolving gs (x, y) with X1Y rect2 xX , yY , where X Y is the pixel size. Thus the reconstructed image is
gr (x, y)

so
Gr (u, v) =

"



x
1
y
rect2
,
X Y
X Y



X
X
x nX y mY
,
,
=
gd [n, m] rect2
X
Y
n= m=
= gs (x, y)

k= l=

Ga (u k/X , v l/Y ) sinc2 (X u, Y v) . (Picture)

(2.47)

(2.48)

This means that there can be some aliasing (imperfect reconstruction) in such a video display.
What happens of ga (x, y) is not bandlimited?
If ga (x, y) is not bandlimited or is otherwise sampled below the Nyquist rate, then aliasing will occur. Aliasing means more than
one frequency component contributes to a single component of the Fourier transform.
Fortunately (from an aliasing perspective, but unfortunately from a resolution perspective) many imaging systems are intrinsically
nearly bandlimited, so aliasing can be minimized by appropriate sampling. However, the cost in terms of number of detector
elements may be high. For example, digital mammography systems are approaching 4K by 4K pixels, presenting a nontrivial data
storage problem alone, as well as a challenge for fast image processing.
Physical objects are space-limited, so they cannot be band-limited! How to sample?
Sampling for non-band-limited signals
Aliasing is unavoidable in this case, but a practical rule-of-thumb that is sometimes used is to use a sample spacing that is half the
FWHM of the PSF.
Example. The triangular PSF h(x) = tri(x/w) has FWHM = w and frequency response H(u) = w sinc2 (wu) . For this H(u),
1
= w/2. So we have
umax 1/w, the location of first zero. (Picture) Thus a reasonable sampling interval is X = 2umax
X = FWHM/2. This is not exact Nyquist sampling because H(u) is not band-limiting, but it is a reasonable starting point in
practice. Although the peak of the first sidelobe of sinc2 is about 0.047, often the amount of aliasing will be less because the object
spectrum usually peaks near zero and then decreases for higher |u|.
Exercise. For the video monitor reconstruction example given in the notes, how much more finely than the Nyquist rate must one
sample to guarantee that the amplitude of all aliased components will be attenuated down to an value that is less than 10% of their
original values?
Hint: this probably requires a small numerical calculation for which fzero or fsolve may be useful.
What can be said about uniform sampling in polar coordinates?

??

c J. Fessler, September 7, 2009, 22:51 (student version)

2.63

In the figure below (from Doug Noll) there is a linear variation of frequencies in both x and y. That is, the x component of the
frequencies varies from 0.25 to 0.75 cycles/unit and the y component of the frequencies varies from 0.25 to 0.75 cycles/unit. The
upper image is the original signal and the lower image is the signal sampled at x = y = 1 unit. The dashed lines mark the +/0.5 unit-1 line that corresponds the Nyquist limit. Only the spectral components in the central box can be represented. In the
attached spectral plots, the blue dots correspond to Dirac impulses of the true frequencies. The green dots are the spectral replicates
in the case of sampling. Only the upper left quadrant has no aliasing. The upper left and lower right appear to have the same
frequency, but the latter is aliased. The upper right and lower left also appear to have the same frequency, but from different
aliasing mechanismsaliasing of the x component and y-component, respectively.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.64

Numerical evaluation of Fourier transforms via the FFT

In imaging work, we often need to evaluate the FT G(u, v) of an image g(x, y) for which the analytical expression for G(u, v) is
unknown. In such cases we compute G(u, v) numerically using the fast Fourier transform (FFT).
We derive the method in 1D but the principles generalize to higher dimensions.
The expression for the 1D FT of a signal g(x) is:
G(u) =

g(x) e2ux dx .

Assume that we have an analytical expression for g(x), but are unable (or unwilling) to find G(u) analytically. To evaluate the
1D FT numerically for a signal g(x) that is roughly symmetric about the origin x = 0, we can approximate the integral by a
summation:
N/21
X

(2.49)
g(nX ) e2unX .
G(u) G(u)
, X
n=N/2

Note that X takes the role of dx. Note that the sum is finite to enable computation. We must use a finite value for N even if the
signal g(x) has infinite support. So it is feasible to use the summation (2.49) only if g(x) is at least approximately finite support,
meaning that g(x) 0 as |x| . We should first plot g(x) and choose some xmax beyond which g(x) is essentially zero and
we must choose N and X such that
N X 2xmax .
(2.50)
Of course we cannot evaluate the summation (2.49) for every conceivable value of u. Usually we are content with a finite set of
equally spaced u values, say {ku : k = K/2, . . . , K/2 1}, for some frequency domain sample spacing u that we must

choose. Defining G[k] , G(k

u ) we have
N/21

G[k] = X

g(nX ) e2knX u ,

n=N/2

k = K/2, . . . , K/2 1.

(2.51)

This is a sum of N terms to be evaluated for K arguments, so one could implement it directly at a cost of O(KN ) flops. This gets
expensive for large K and N . To reduce computation, we would like to evaluate (2.51) using an FFT. This will be possible only
if we choose X and u such that X u = 1/L for some integer L. In that case, we can evaluate (2.51) using a L-point DFT
(using zero-padding if L > N ).
Usually (but not always) we choose L = K = N , which requires that we choose X and u such that
X u =

1
.
N

In this case, (2.51) simplifies to

N/21

G[k] = X

g(nX ) e2kn/N ,

n=N/2

k = N/2, . . . , N/2 1,

(2.52)

which is almost the usual N -point DFT formula with an extra factor of X out front to account for the dx in the original integral.
The approximation (2.49) involves sampling g(x), which is reasonable if g(x) is at least approximately band-limited. Typically
g(x) is not exactly band-limited, but there will be some frequency umax beyond which G(u) is approximately zero, in which case
we say g(x) is essentially band-limited. To avoid aliasing, we will want to satisfy the Nyquist criterion, so we want
X <

1
.
2umax

We often do not know umax in advance, so we must make a guess, compute G[k], and then look for signs of aliasing (spectral
overlap). If such overlap is present, then we can decrease the sampling interval X . Then we must also increase N accordingly to
satisfy (2.50). To get a pretty plot of G(u), we may need to decrease u , and then again increase N accordingly.

c J. Fessler, September 7, 2009, 22:51 (student version)

2.65

The fftshift and ifftshift commands in M ATLAB

We say (2.52) is almost the usual DFT because the usual DFT has a summation from n = 0 to N 1, not from N/2 to N/2 1.
Likewise, the usual DFT involves arguments k from 0 to N 1, not from N/2 to N/2 1. If we define

g(nX ),
n = 0, . . . , N/2 1
g[n] ,
g((n N )X ), n = N/2, . . . , N 1,
then we can rewrite (2.52) as follows:
G[k] = X

N
1
X

g[n] e2kn/N ,

k = N/2, . . . , N/2 1.

n=0

(2.53)

In M ATLAB, we use the command ifftshift to form g[n] from g(nX ).

Furthermore, the periodicity property of the N -point DFT is that G[k] = G[k mod N ] . Thus, we first evaluate
=
G[k]

N
1
X

g[n] e2kn/N ,

n=0

and then we form G[k] by

G[k] =

k = 0, . . . , N 1,

k = 0, . . . , N/2 1
X G[k],

X G[k + N ], k = N/2, . . . , 1.

In M ATLAB, we use the command fftshift to get G[k] from G[k].

Note the fftshift and ifftshift commands are identical in the usual case when N is even, as has been assumed in the
above derivations. When N is odd they differ, and the difference is important.
The 2D case
The 2D FT can be computed rapidly using the 2D Fast Fourier Transform (FFT), e.g., the fft2 routine in Matlab. Now there
is a double integral so both X and Y are needed. Again, fftshift is often necessary. In 2D, fftshift swaps the four
quadrants appropriately.
Example. Consider the signal g(x, y) = 1/(1 + x2 + y 2 )3/2 . What does its 2D FT look like?
g(x,y)

G(u,v) by FFT
1
5

6
0.8

0.5

0.6

0.4

2
0.5

0.2

6
8

0
x

1
1

0.5

Profiles

0.5

G(u,v) exact

G(u,v) FFT
G(u,v) exact

0
u

6
5

0.5

5
v

G(u,0)

4
4
0

3
2

0.5

1
0

0.2

0.2
u

0.4

0.6

0.8

1
1

0.5

0
u

0.5

c J. Fessler, September 7, 2009, 22:51 (student version)

Here is the mfile that produced that figure. Notice the use of ndgrid.
% ft_example2.m Illustrate approximate 2D FT via FFT
% hankel pair: 1 / (a2 + r2)3/2 <-> 2 pi / a exp(-2 pi a * q)
nx = 28; ny = nx;
xmax = 23;
dx = 2*xmax / nx; dy = dx;
x = [-nx/2:nx/2-1] * dx;
y = [-ny/2:ny/2-1] * dy;
[xx yy] = ndgrid(x, y); % make 2D array from 1D sampling coordinates
gxy = 1 ./ (1 + xx.2 + yy.2).1.5; % g(x,y) sampled
Guv = dx * dy * fftshift(fft2(ifftshift(gxy))); % dx dy to be quantitative
%Guv = reale(Guv);
frac = max(abs(imag(Guv(:)))) / max(abs(Guv(:)))
if frac > 1e-13, warning non-negligible imaginary part, end
Guv = real(Guv); % discard (almost) negligible imaginary part
nu = nx;
nv = ny;
du = 1/(nx*dx); dv = 1/(ny*dy);
u = [-nu/2:nu/2-1] * du;
v = [-nv/2:nv/2-1] * dv;
[uu vv] = ndgrid(u,v); % grid of frequency domain sample locations
Guv_true = 2*pi * exp(-2*pi*sqrt(uu.2+vv.2));
clf
subplot(221)
imagesc(x, y, gxy), axis image, axis xy
xlabel x, ylabel y, title g(x,y), colormap(1-gray(256)), colorbar
ax = [-1 1 -1 1]*1;
subplot(222)
imagesc(u, v, Guv), axis image, axis xy, axis(ax)
xlabel u, ylabel v, title G(u,v) by FFT, colormap(1-gray(256)), colorbar
subplot(224)
imagesc(u, v, Guv_true), axis image, axis xy, axis(ax)
xlabel u, ylabel v, title G(u,v) exact, colormap(1-gray(256)), colorbar
subplot(223)
imagesc(u, v, abs(Guv-Guv_true)), axis image, axis xy, axis(ax)
xlabel u, ylabel v, title G(u,v) error, colormap(1-gray(256)), colorbar
iv = nv/2+1;
plot(u, Guv(:,iv), ., u, Guv_true(:,iv), -)
xlabel u, ylabel G(u,0), title Profiles
legend(G(u,v) FFT, G(u,v) exact)
axis([-0.3 0.8 0 7])
return
%savefig ft_example2

2.66

c J. Fessler, September 7, 2009, 22:51 (student version)

2.67

Summary
This chapter has introduced 2D signals, 2D systems, 2D Fourier series, 2D Fourier transforms, and 2D sampling. The majority of
the concepts in 2D generalize readily from those in 1D.

Bibliography
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25]
[26]
[27]

R. Bracewell. The Fourier transform and its applications. McGraw-Hill, New York, 1978.
R. N. Bracewell. Two-dimensional imaging. Prentice-Hall, New York, 1995.
J. S. Lim. Two-dimensional signal and image processing. Prentice-Hall, New York, 1990.
P. E. Varberg. Change of variables in multiple integrals. Amer. Math. Monthly, 78(1):425, January 1971.
R. E. Williamson, R. H. Crowell, and H. F. Trotter. Calculus of vector functions. Prentice-Hall, New Jersey, 1972.
M. Mellor and M. Brady. Phase mutual information as a similarity measure for registration. Med. Im. Anal., 9(4):33043,
August 2005.
A. Macovski. Medical imaging systems. Prentice-Hall, New Jersey, 1983.
J. W. Goodman. Statistical optics. Wiley, New York, 1985.
D. G. Luenberger. Optimization by vector space methods. Wiley, New York, 1969.
G. McGibney, M. R. Smith, S. T. Nichols, and A. Crawley. Quantitative evaluation of several partial Fourier reconstruction
algorithms used in MRI. Mag. Res. Med., 30(1):519, July 1993.
Z. P. Liang, F. E. Boada, R. T. Constable, E. M. Haacke, P. C. Lauterbur, and M. R. Smith. Constrained reconstruction
methods in MR imaging. Reviews of Magnetic Resonance in Medicine, 4:67185, 1992.
M. Abramowitz and I. A. Stegun. Handbook of mathematical functions. Dover, New York, 1964.
K. T. Smith and F. Keinert. Mathematical foundations of computed tomography. Appl. Optics, 24(23):39507, 1985.
L. Grafakos. Classical and modern Fourier analysis. Pearson, NJ, 2004.
J. A. Ferrari, D. Perciante, and A. Dubra. Fast Hankel transform of nth order. J. Opt. Soc. Am. A, 16(10):25812, October
1999.
R. M. Lewitt. Multidimensional digital image representations using generalized Kaiser-Bessel window functions. J. Opt. Soc.
Am. A, 7(10):183446, October 1990.
I. A. Cunningham. Applied linear-systems theory. In J. Beutel, H. L. Kundel, and R. L. Van Metter, editors, Handbook of
Medical Imaging, Volume 1. Physics and Psychophysics, pages 79160. SPIE, Bellingham, 2000.
C. H. Paik and M. D. Fox. Fast Hartley transforms for image processing. IEEE Trans. Med. Imag., 7(2):14953, June 1988.
R. M. Lewitt, P. R. Edholm, and W. Xia. Fourier method for correction of depth-dependent collimator blurring. In Proc. SPIE
1092, Med. Im. III: Im. Proc., pages 23243, 1989.
S. J. Glick, B. C. Penney, M. A. King, and C. L. Byrne. Noniterative compensation for the distance-dependent detector
response and photon attenuation in SPECT. IEEE Trans. Med. Imag., 13(2):36374, June 1994.
W. Xia, R. M. Lewitt, and P. R. Edholm. Fourier correction for spatially variant collimator blurring in SPECT. IEEE Trans.
Med. Imag., 14(1):10015, March 1995.
R. Bracewell. The Fourier transform and its applications. McGraw-Hill, New York, 3 edition, 2000.
D. L. Donoho and P. B. Stark. Uncertainty principles and signal recovery. SIAM J. Appl. Math., 49(3):90631, June 1989.
B. A. Wandell, A. E. Gamal, and B. Girod. Common principles of image acquisition systems and biological vision. Proc.
IEEE, 90(1):517, January 2002.
W. Sun and X. Zhou. Reconstruction of band-limited signals from local averages. IEEE Trans. Info. Theory, 48(11):295563,
November 2002.
R. G. Baraniuk. Compressive sensing. IEEE Sig. Proc. Mag., 24(4):11821, 2007.
H. Stark. Sampling theorems in polar coordinates. J. Opt. Soc. Am., 69(11):151925, November 1979.