Lecture 18

Linear Filters
Relevant section from book by Boggess and Narcowich: Section 2.3, p. 108
The idea of a lter is very important in signal and image processing. A lter F takes a signal f
and processes it to produce a modied signal

f. Examples of processing include noising/denoising,
blurring/deblurring and zooming. Mathematically, we shall require F to map a suitable space of
functions, say S to itself.
In what follows, we let f L
2
(R) denote a possibly complex-valued signal. For mathematical
ease, we shall also assume that f(t) is piecewise-continuous, which is reasonable for the applications
we consider. Therefore, our space S will be the set of piecewise-continuous, L
2
functions on R. A
linear lter is simply a linear mapping L : S S which obeys the usual criteria for linearity:
1. L(f +g) = L(f) +L(g),
2. L(cf) = cL(f) for c R (or C).
Time-invariant linear lters
Another useful property of linear lters is time-invariance. For an a R, let S
a
: L
2
(R L
2
(R)
denote the continuous shift operator:
S
a
: f f
a
, where f
a
(t) = f(t a). (1)
Note that S
a
is a linear operator/lter.
A linear lter L is said to be time-invariant if
LS
a
= S
a
L. (2)
In other words, if we apply S
a
to a function f, and then apply L to it, i.e.,
g(t) = L(S
a
f) = L(f
a
) = L(f(t a)), (3)
the result is the same as if we rst applied L to f and then S
a
, i.e.,
h(t) = S
a
(Lf) = (Lf)
a
= (Lf)(t a), (4)
175
so that g(t) = h(t).
We consider a couple of examples below.
1. Convolution: We consider the convolution of a signal f(t) with a function g(t),
(Lf)(t) = (f g)(t) =
_

f(t s)g(s) ds. (5)

We also assume that (t) is such that the above integral exists for all f S. One condition
that guarantees this is that g(t) have nite support, i.e., g(t) = 0 outside a nite interval [a, b].
Clearly, L is a linear operator.
Let h(t) = (Lf)(t) so that
(Lf)
a
(t) = h
a
(t) = h(t a) = (f g)(t a). (6)
By denition
h(t a) =
_

f(t a s)g(s) ds
=
_

f
a
(t s)g(s) ds
= (f
a
g)(t)
= L(f
a
)(t). (7)
Therefore (Lf)
a
= L(f
a
), so L is time-invariant.
2. An integral operator: Now consider the following integral operator
(Lf)(t) =
_
t
0
f(s) ds, (8)
which may be viewed as a kind of sum over nite history of f. Clearly, L is a linear operator.
We now investigate if it is time-invariant. First,
(Lf)
a
(t) = (Lf)(t a)
=
_
ta
0
f(s) ds. (9)
But
(Lf
a
)(t) =
_
t
0
f
a
(s) ds
176
=
_
t
0
f(s a) ds
=
_
ta
a
f(u) du (u = s a, du = ds, etc.)
= (Lf)
a
(t). (10)
So the lter L is not time-invariant.
Note that the problem is caused by the dierence in the lower integration limits of the two
integrals, i.e., 0 and a. This, in turn, arises from the fact that the integration starts at time 0.
This indicates that the following linear integral operator,
(Lf)(t) =
_
t

f(s) ds, (11)

in other words, a sum over the entire history of f(t), is time-invariant.
We shall now state the main result of this section for linear, time-invariant lters.
Theorem 1: Let L : L
2
(R L
2
(R) be a linear, time-invariant lter. Then there exists an integrable
function h such that
Lf = f h, for all f L
2
(R). (12)
Note: This result tells us that all of the work involved in the design of a linear time-invariant lter
may be focussed on the integration kernel h.
In order to prove the above property, we shall require the following result.
Theorem 2: Let L be a linear, time-invariant lter. Then for any R, there is a function H()
such that
L(e
it
) =

2H()e
it
. (13)
A few comments before we prove this result.
1. We consider as input to L functions of the form e
it
, i.e., oscillatory functions. (Note that they
are not L
2
functions.)
177
2. The output of such a function is an oscillatory function with the same frequency. Only its
amplitude, originally 1, may be modied.
3. The

2 factor is placed there for convenience, as will be seen below.

Proof: Let
h

(t) = L(e
it
) (14)
denote the output of the linear lter L, when e
it
is the input. Since L is time-invariant, it follows
that
L(e
i(ta)
) = h

(t a), for a R. (15)

(Here, e
i(ta)
is a shifted input function. Time-invariance implies that the output is shifted by the
same amount.) But L is linear, so we can express the LHS of the above equation as
L(e
i(ta)
= e
ia
L(e
it
)
= e
ia
h

(t). (16)
Combining the results of the last two equations, we have
h

(t a) = e
ia
h

(t). (17)
Since a is arbitrary, we set a = t:
h

(0) = e
it
h

(t), (18)
or
h

(t) = e
it
h

(0), (19)
We may now substitute this result into Eq. (14):
L(e
it
) = h

(0)e
it
. (20)
Note that h

(0) is dependent only upon and not t, i.e., it is a function of . We then dene a new
function H() as follows,
h

(0) =

2H() =H() =
1

2
h

(0). (21)
The proof is complete.
The above result leads to a proof of Theorem 1.
178
Lecture 19
Linear lters (contd)
We now use the result from the end of the last lecture to prove the following important result, stated
earlier:
Theorem 1: Let L : L
2
(R L
2
(R) be a linear, time-invariant lter. Then there exists an integrable
function h such that
Lf = f h, for all f L
2
(R). (22)
Proof of Theorem 1: We now claim that the function H() determines L, or vice versa. To show
this, consider a function f L
2
(R). Then, by the Fourier inversion theorem,
f(t) =
1

2
_

F()e
it
d, (23)
in the L
2
sense if f(t) is piecewise continuous, then equality holds at points t where f is continuous.
Here, F() is the FT of f(t). Now let g = Lf, which means that we apply L to both sides of the
above equation:
g(t) = (Lf)(t)
=
1

2
L
__

F()e
it
d
_
. (24)
What we would now like to do is to pass the linear operator L inside the integral to operate on the
function e
it
. If f(t) and F() are piecewise continuous, hence Riemann-integrable, then this transfer
may be justied using the Riemann sum denition of the integral, as shown in the book by Boggess
and Narcowich. In the more general case of L
2
, a little more analysis is required, but the desired result
holds. Therefore, we shall simply write,
g(t) =
1

2
_

F()L(e
it
) d
=
_

F()H()e
it
d (from previous Theorem)
=

2F
1
[F()H()]
= (f h)(t) (from Convolution Theorem). (25)
This completes the proof: The action of the linear operator L on a function f(t) is equivalent to
convolving f with h, the inverse FT of the response function H().
179
Special case: The role of the function h(t) is made clear if we consider the special case of input
function, f(t) = (t), the Dirac delta function. In this case, the input represents an impulse. The
output is given by
(Lf)(t) = (L)(t) = ( h)(t)
=
_

(t s)h(s) ds
= h(t). (26)
In other words, h(t) is the result of having as input the impulse function (t). As a result, h(t) is
referred to as the impulse response function.
Applications of the result Lf = f h
Given a linear lter L and a signal f, let g = Lf. Then
g(t) = (Lf)(t) = (f g)(t). (27)
Taking Fourier transforms of both sides, we have, from the Convolution Theorem,
G() =

2F()H(). (28)
Therefore, the action of the linear lter L can be performed in the frequency deomain, provided that
you know H(). This will explain why a lot of the signal/image processing literature is devoted to
lter banks, which take the FT, F(), of a signal and divide it into various components, for example,
low- and high-frequency components.
For example, let us return to the frequency-thresholding example that we studied earlier, where
you set to zero all frequency components F() for all || >
0
, where
0
is a threshold. The system
response function H() that produces this thresholding is given by
H() =
_
_
_
1

2
, ||
0
,
0, || >
0
.
(29)
The 1/

2 factor arises because of the

2 factor in Eq. (28) which comes from the Convolution

Theorem.
As mentioned in the previous lecture, H() is an example of a low-pass lter where low frequen-
cies are basically allowed to pass and high frequencies are modied, possibly blocked. In general, a
low-pass lter H() would have the following characteristics,
180
1. H()
1

2
as 0,
2. H() 0 as || .
In contrast, a high-pass lter blocks out low frequencies and leaves high frequencies untouched. As
such, it will have the following characteristics,
1. H() 0 as 0,
2. H()
1

2
as || .
High-pass lters are useful in applications where one wishes to amplify the changes in a signal, for
example, the sharpening of edges in images.
181
Appendix to Lecture 19
(This section was not covered in class and is included here only for the sake of
interest.)
Linear lters (contd)
Causal lters
We now return to the low-pass lter discussed at the end of the previous lecture,
H() =
_
_
_
1

2
, ||
0
,
0, || >
0
.
(30)
Let us examine the action of this lter in the time domain. recalling that if g = Lf, then
g(t) = (f h)(t). (31)
We must compute h(t) = F
1
[F()]:
h(t) =
1

2
_

H()e
it
d
=
1
2
_

0

0
e
it
d
=
1
2
1
it
_
e
it

0
=
1
2
1
it
_
e
i
0
t
e
i
0
t

=
1

sin(
0
t)
t
. (32)
Now suppose that we lter the simple input function dened as follows: For some t
0
> 0,
f
t
0
(t) =
_
_
_
1, 0 t t
0
,
0, t < 0 or t > t
0
.
(33)
In other words, f(t) starts at t = 0 and ends at t = t
0
. The eect of the linear lter L is given by
g(t) = (Lf
t
0
)(t) = (f
t
0
h)(t)
=
_

f
t
0
(s)h(t s) ds
=
1

_
t
0
0
sin[
0
(t s)]
t s
ds. (34)
182
Now use the change of variable u =
0
(t s), so that du =
0
ds, ds =
1

0
du and t s =
u

0
.
Then
g(t) = (Lf
t
0
)(t) =
1

_

0
t

0
(tt
0
)
sin u
u
du
=
1

[Si(
0
t) Si(
0
(t t
0
))], (35)
where
Si(x) =
_
x
0
sin u
u
du, Si(0) = 0. (36)
The function Si(x) is known as the Sine integral function. A plot of g(t) for the special case
0
= 1
and t
0
= 1 is presented below.
-0.1
-0.05
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
-10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10
t
A plot of g(t) = (Lf
t0
)(t) for the case
0
= t
0
= 1.
There is a very noteworthy feature of this result: Recall that the input function f(t) is zero for t < 0.
However, the output function g(t) = (Lf)(t) is non-zero for t < 0. In other words, the output exists
before the input function arrives!
This result may be viewed as a violation of causality: the existence of a response before the actual
(nonzero) cause of the response appears in time. As such, the linear lter L is said to be acausal.
In some applications, causality may be a highly desired, even mandatory, feature any processing
that is applied to a time series should aect the future signal. After all, what good is it to modify a
signal that has already appeared?
On the other hand, in applications where time-ordering is not required, acausality may not be
objectionable. This is the case in the processing of single images (as opposed to video sequences),
where a signal/image is viewed as a function of a spatial variable as opposed to a temporal variable.
183
That being said, the remainder of this section deals with causal linear lters, with an eye to
applications that demand causality.
Denition: A (linear) lter L is said to be causal if the output sequence (Lf)(t) begins after the
input signal f(t) arrives. Another way of stating this is as follows: The signal (Lf)(t) does not depend
on values f(u) for u > t.
A necessary and sucient condition for a time-invariant linear lter to be causal is provided by
the following result.
Theorem: Let L be a linear, time-invariant lter with piecewise-continuous response function h, i.e.,
Lf = f h. Then L is causal if and only if h(t) = 0 for t < 0.
Proof: (i) We rst consider the = portion of the statement and show that if h(t) = 0 for t < 0,
then L is causal, i.e., (Lf)(t) = 0 for t < 0 for any signal f(t). Consider a signal f(t) such that
f(t) = 0 for t < 0. Then
(Lf)(t) = (f h)(t) =
_

0
f(s)h(t s) ds. (37)
(The integration is from 0 instead of since f(t) = 0 for t < 0.) By assumption, h(t s) = 0 for
t s < 0 or, equivalently, t < s. Now if t < 0 then t s < 0 since s 0 in the above integration.
This implies that h(t s) = 0 which, in turn, implies that (Lf)(t) = 0 for t < 0 for any signal f(t).
Therefore L is causal.
(ii) Now consider the = portion of the statement: We assume that L is causal and wish to
show that h(t) = 0 for t < 0. In what follows, we provide only a sketch of the proof, leaving out the
technical details that would be required in a formal proof. Since L is causal, then for any signal f(t)
such that f(t) = 0, t < 0,
(Lf)(t) =
_

0
f(s)h(t s) ds = 0 for t < 0. (38)
Now suppose that h(t
0
) = 0 for some t
0
< 0 without loss of generality we assume that h(t
0
) > 0.
Since h is piecewise continuous, there exists an interval containing t
0
on which h(t) is positive. Well
assume, again without loss of generality, that this interval has the form (t
0
, t
0
+) for some > 0.
Now consider a continuous, nonnegative signal f(t) which has the following properties: There exists
184
a t
1
, where 0 < t
1
< |t
0
|, such that f(t) 0 on (t
1
, t
1
+) and f(t) = 0 elsewhere. Note that this
implies that f(t) = 0 for t < 0.
Let us now examine the integral in Eq. (38). By construction, f(s) is nonzero only in the -
interval centered at s = t
1
. Furthermore, when s = t
1
, t s = t t
1
= t
0
when t = t
0
+t
1
< 0. This
implies that h(t s) = h(t
0
) > 0. Therefore, there is a strictly positive contribution to the integral,
implying that (Lf)(t) > 0 for t = t
0
+t
1
< 0. This contradicts Eq. (38). Therefore, h(t) must be zero
for t < 0.
Example: An example of a causal lter important in the history of signal processing and ltering is
the so-called Butterworth lter (named after S. Butterworth, an audio engineer of the early 1900s),
h(t) =
_
_
_
Ae
t
, t 0,
0, t < 0,
(39)
where A, > 0.
The system response function, H(), which is the Fourier transform of h(t), is given by
H() =
1

2
_

0
e
t
e
it
dt
=
1

2
A
+i
. (40)
Note that H() 0 as || so that H() is a low-pass lter. Recall from the last lecture (p.
155) that in this particular formalism of linear lters, there was the normalization requirement that
H(0) =
1

2
. This implies that A = , in which case we express H() as follows,
H() =
1

+i
=
1

2
_
1
1 +i
_

_
_
. (41)
The above lter is a special case of a family of Butterworth lters, H
n
(), with the following
characteristics,
H
n
() =
1
B
n
()
, (42)
where B
n
() is a polynomial of degree n in such that
|H
n
()|
2
=
1
1 +
_

0
_
2n
. (43)
For convenience, we have omitted the multiplicative 1/(2) factor. The value
0
is a kind of charac-
teristic cuto frequency:
|H
n
(
0
)|
2
=
1
2
. (44)
185
In all cases, |H
n
()| 0 as || , with
|H
n
()|
n
= O(||
n
) as || . (45)
The decay rates of these low-pass lters increase with n. One of the desirable features of these lters
is that for || >
0
, the graph of |H
n
()|
2
quickly becomes quite at. Plots of |H
n
()|
2
for n = 1, 2
and 5 are shown in the gure below.
-0.5
0
0.5
1
1.5
-5 -4 -3 -2 -1 0 1 2 3 4 5
omega
n=5
n=2
n=1
Plots of |H
n
()|
2
vs. , cf. Eq. (43), for n = 1, 2 and 5. Here
0
= 1. The factor 1/(2) has been omitted.
As n , the graph of |H
n
()|
2
vs. n approaches the following box-car function,
B(t) =
_
_
_
1, ||
0
,
0, || >
0
.
(46)
Interestingly enough, recall that the boxcar function B(t) corresponds to an acausal lter its inverse
FT is a sinc function which, when convolved with a signal f(t), will produce an acausal output.
186
The Sampling Theorem
Electronic storage and transmission of signals and images has been of obvious importance in our civi-
lization. From the telephone, to radio, and then to television, engineers and scientists have consistently
been faced with the basic question of how to store and transmit information as eciently as possible.
In the not-too-distant-past pre-digital age, the transmission and storage of audio and video (except
for still images) was analog, i.e. continuous in time, in the form of reel-to-reel tapes and videotape. The
advent of computers ushered in the digital age, where continuous signals were replaced by sequences of
bits, i.e., 0s and 1s. This led to digital storage devices that mimicked the storage of information in
computer memory: oppy disks of various sizes, followed by audio digital tape, compact discs (CDs)
and, most recently, DVDs.
As mentioned earlier, there has always been, and perhaps always will be, the fundamental question
of how to store and transmit information as eciently as possible. Back in the pre-digital age of analog
communication, Claude Shannon of Bell Labs (later to be AT &T Labs) provided a basic reference
point for communication theory in a celebrated paper. (C. Shannon, A mathematical theory of
communication, Bell System Technical Journal, vol. 27, pp. 379, 623 (1948).) Shannons classic
paper gave birth to rapid advances in information and communication theory.
That being said, Shannon was actually not the rst to come up with this fundamental result. There is a
very interesting history behind the Sampling Theorem and so-called cardinal series, to be introduced below.
A brief discussion is given in the introductory chapter of the book, Introduction to Shannon Sampling and
Interpolation Theory, by R.J. Marks II (Springer-Verlag, NY, 1991). Marks writes that one historian (H.S.
Black) credits the mathematician Cauchy for understanding the idea of sampling a function with nite support,
citing a work of 1841. Another researcher (Higgins, 1985) disputes this claim and credits the mathematician
E. Borel in an 1897 paper. The British mathematician E.T. Whittaker published a highly cited paper on the
sampling theorm in 1915. (E.T. Whittaker, On the functions which are represented by the expansions of the
interpolation theory, Proc. Roy. Soc. Edinburgh, vol. 35, pp. 181-194 (1915).) Whittakers formula was
later called the cardinal series by his son, J.M. Whittaker. V.A. Kotelnikov reported the sampling theorem
in a Soviet journal in 1933. As stated earlier, Shannon showed the importance of the sampling theorem to
communication theory in his 1948 paper, in which he cited Whittakers 1915 paper. A number of other events
in the development of the cardinal series are listed by Marks.
187
In any case, Shannons paper was fundamental in showing the application of the Sampling Theorem
to communications, thereby attracting the attention of the communications research community.
The basic question asked by Shannon and others was as follows. Suppose that we have a contin-
uous, or analog, signal f(t) for example, an audio signal that is sampled to produce discrete data
points, as we discussed in earlier lectures, i.e.,
f[n] = f(nT), n Z. (47)
Here, T > 0 is the sampling period. Can we reconstruct f(t) perfectly for all t R from these
samples?
Before we examine the Sampling Theorem of Shannon et al., let us step back and think a little
about this problem. Suppose that you were given the data points f[n]. What could one do in an eort
to construct f(t), or at least approximations to it?
The simplest response would be to attempt various interpolations of the points f[n]. And the
simplest interpolation would be:
Piecewise constant interpolation: We dene the following approximation g
0
(t) to f(t): For n Z,
g
0
(t) = f(nT), nT t < (n + 1)T, (48)
sketched schematically below.
t
0
T T 2T 3T 4T
y
y = f(t)
Piecewise constant approximation g
0
(t) to continuous signal f(t).
There is one obvious drawback to this approach: g(t) is discontinuous at the sample points, which
would probably be disastrous for audio signals. (In two-dimensions, it is not such a bad approximation
188
for images. In fact, digital images are piecewise constant approximations to a real continuous photo
or scene.)
There is another way of looking at this approximation which will be quite useful in our later
discussions. Let us dene the fundamental basis function (t) for t R.
(t) =
_
_
_
1, 0 t < T,
0, otherwise.
(49)
Then our piecewise constant function g
0
(t) may be written as
g
0
(t) =

n=
f(nT)(t nT). (50)
Each translate (t nT) has value 1 over the interval [nT, (n +1)T) and is zero outside this interval.
This is what permits us to write Eq. (50). The set of all translates (t nT), n Z, serves as a basis
for all functions on R that are piecewise constant on the intervals [nT, (n + 1)T). In fact, these basis
functions are orthogonal to each other. This idea will be important in our study of wavelets.
Piecewise linear interpolation: Now dene the approximation g
1
(t) to f(t) as follows: For n Z,
g
1
(t) =
t nT
T
f(nT) +
(n + 1)T t
T
f((n + 1)T), nT t < (n + 1)T. (51)
By construction, g
1
(nT) = f(nT) for all n, and the graph of g
1
(t) from f(nT) to f((n + 1)T) is a
straight line, as sketched below.
t
0
T T 2T 3T 4T
y
y = f(t)
Piecewise linear approximation/interpolation g
1
(t) to continuous signal f(t).
We may also view the function g
1
(t) as a linear combination of basis functions which are translates
of a fundamental basis function h(t). To see this, consider the sketch below, where we have drawn
189
triangular hat functions that have bases on the intervals [(n 1)T, (n + 1)T] and apices at points
nT with heights f(nT).
t
0
T T 2T 3T 4T
y
y = g1(t)
Piecewise linear approximation/interpolation g
1
(t) to continuous signal f(t), viewed as a linear combination of
triangular hat functions.
Each triangular function is a translation and vertically scaled version of the following function, which
is sketched in the gure below.
h(t) =
_

_
t
T
+ 1, T t < 0,
1
t
T
, 0 t < T
0, otherwise.
(52)
T T
t
0
1
y = h(t)
y
Triangular hat function h(t) whose translates comprise a basis for piecewise linear functions.
The fact that h(0) = 1 dictates that the triangular function below the sample point at t = nT
must be multiplied by the sample value f(nT). And the fact that h(T) = h(T) = 0 produces the
linear interpolation between adjacent sample values. As a result, the function g
1
(t) may be written as
g
1
(t) =

n=
f(nT)h(t nT). (53)
190
Notice the similarity in form between Eqs. (50) and (53). The translates h(t nT), n Z, form a
(nonorthogonal) basis for piecewise linear functions over the intervals [nT, (n + 1)T) on R.
Higher-order interpolations: It is possible to construct kth degree polynomials that interpolate
between the sample points f(nT) and f((n + 1)T) using k + 1 consecutive sample points that con-
tain these points. These polynomial interpolation functions are called splines and will comprise the
interpolation function g
k
(t).
We now return to the Sampling Theorem. Shannons idea was to restrict attention to bandlim-
ited functions: functions f(t) with Fourier transforms F() that were identically zero outside a nite
interval, assumed to be the symmetric interval [, ] for some > 0, i.e.,
F() = 0 for || > . (54)
is known as the band limit of f (or F).
Does this sound like an articial constraint? Perhaps, but, in fact, it is practical, for the following
reasons.
1. Sounds made by the human voice are contained well within the frequency range of an 88-key
piano keyboard:
low A at 27.5 Hz , (1 Hz = 1 cycle/second)
high C at 4186 Hz.
Therefore, speech signals are essentially bandlimited, with = 4200 2 = 8400.
Note: is the angular velocity, in units of radians/unit time. There are 2 radians/cycle.
Equivalently, we have 1/(2) cycle/radian.
2. The human ear can hear sounds in roughly the range 20-20,000 Hz. As such, audible sounds are
bandlimited, with 20, 000 2 = 40, 000.
191
Lecture 20
The Sampling Theorem (contd)
Once again, we dene bandlimited functions as follows:
A function f(t) is said to be bandlimited, or -bandlimited, if there exists an > 0 such that
F() = 0 for || > . (55)
In practice, one generally tries to nd the smallest such frequency for which (55) holds.
Associated with the angular frequency band limit (radians/second) is the frequency
=

2
Hz (cycles/second), (56)
known as the Nyquist frequency. The Nyquist rate is given by
2 =

Hz. (57)
Its importance will become clear after we study the Sampling Theorem.
The Whittaker-Shannon Sampling Theorem: Let f(t) be an -bandlimited function, with
Fourier transform F() that satises Eq. (55) for some > 0. Furthermore, assume that F() is
piecewise continuous on [, ]. Then f = F
1
F is completely determined at any t R by its values
at the points t
k
=
k

, k = 0, 1, 2, , as follows,
f(t) =

k=
f
_
k

_
sin(t k)
t k
(58)
=

k=
f
_
k

_
sinc
_
t

k
_
. (59)
Furthermore, the series on the right converges uniformly on closed subsets of R.
Note: Please note that we are now using the signal/image processing denition of the sinc function,
i.e.,
sinc(x) =
_
_
_
sinx
x
, x = 0,
1, x = 0.
(60)
192
This form of the sinc function includes the factor of in the denition.
Just a couple of comments before we prove this theorem.
1. First of all, the series on the right, in whatever form, is known as the cardinal series. (As
mentioned in the introduction to this lecture, the origin of this term is due to J.M. Whittaker.)
2. From Eqs. (56) and (57), we may express the cardinal series in the following alternate forms:
f(t) =

k=
f
_
k
2
_
sinc (2t k) (61)
=

k=
f (kT) sinc
_
t
T
k
_
, (62)
where
T =
1
2
=

(63)
is the sampling period. Note that
T =
1
2
_
2

_
. (64)
In other words, the period T is one-half the period associated with the bandwidth . Another
way to put this is as follows:
The sampling frequency is twice the bandwidth frequency .
The above is in terms of angular frequency. In terms of cycles per unit time, this explains why
the Nyquist rate of sampling is twice the Nyquist frequency (associated with the bandwidth).
We now prove the W-S Theorem.
Proof: We exploit the fact that the Fourier transform is supported on the nite interval [, ]
and expand F() in terms of a Fourier series. Well use the complex exponential functions u
k
() =
e
ik/
that form an orthogonal set on [, ]. (Youve seen these before, but for the independent
variable t [L, L] instead of [, ].) The (unnormalized) Fourier series for F() has the form
F() =

k=
c
k
e
ik/
, (65)
where
c
k
=
1
2
_

F()e
ik/
d. (66)
193
Because F() is assumed to be piecewise continuous on [, ], this series will converge uniformly.
Since F() = 0 for || > , we may change the limits of integration to extend over the entire real
line:
c
k
=
1
2
_

F()e
ik/
d
=

2
2
1

2
_

F()e
i(
k

)
d, (67)
where we have rewritten the integrand slightly so that it has the form of an inverse Fourier transform
(which also explains the introduction of the factor 1/

2). Indeed, we may write that

c
k
=

2
2
f
_

_
. (68)
Now substitute this result into Eq. (65) to obtain
F() =

2
2

k=
f
_

_
e
ik/
. (69)
We now replace k with k (or let l = k, then replace l with k) to give
F() =

2
2

k=
f
_
k

_
e
ik/
. (70)
Well now use this result for F() in the expression of f(t) as an inverse Fourier transform:
f(t) =
1

2
_

F()e
it
d
=
1

2
_

F()e
it
d (since F() = 0 for || > )
=
1
2
_

k=
f
_
k

_
e
i[
k

t]
d
=
1
2

k=
f
_
k

_ _

e
i[
k

t]
d. (71)
The uniform convergence of the Fourier series to F() permits the interchange of summation and
integration that produced the nal line.
We now evaluate the integrals:
_

e
i[
k

t]
d =
1
i
1
_
k

_
e
i[kt]
e
i[kt]
_
=
2
k

t
sin(k t). (72)
194
Substitution into Eq. (71) yields the desired result,
f(t) =

k=
f
_
k

_
sin(t k)
(t k)
=

k=
f
_
k

_
sinc
_
t

k
_
. (73)
Once again, this is the cardinal series for an -bandlimited function f(t).
Recall that if f(t) is -bandlimited, i.e., its Fourier transform F() is identically zero outside the
interval [, ] for some > 0, then it may be reconstructed exactly for any t R from samples
taken at times t
k
=
k

, k Z. The reconstruction is performed via the so-called cardinal series:

f(t) =

k=
f
_
k

_
sin(t k)
(t k)
=

k=
f
_
k

_
sinc
_
t

k
_
. (74)
We now make some comments on this result:
1. For a xed t R, the cardinal series can converge slowly, since the sinc function decays on the
order of O(1/k) as k . (Actually, the function f(t) also decays to zero as t since it
is assumed to be in L
2
(R), which can improve the convergence somewhat.) There are ways to
increase the convergence rate, including oversampling, which may be addressed later.
2. Since the sampling period is given by T = /, functions f(t) with higher band limit values
must be sampled more frequently. This makes sense: functions with higher values naturally
have higher frequency components. It is necessary to sample the function more often in order
to capture the greater variability produced by these higher frequencies.
3. From the previous comment, it is desirable to nd the smallest for which the Fourier transform
F() vanishes outside the interval [, ]. In this way, the function f(t) does not have to
sampled at too high a rate. That being said, a cardinal series employing a higher value of
than necessary will still converge to f(t).
On the other hand, if we use an value that is too small, i.e., the cardinal series will not converge
to f(t). This is because the Fourier transform F() is not being properly approximated by the
195
Fourier series. For example, if = 1 and you use a value of 1/2 in the cardinal series, you
are only approximating F() over [1/2, 1/2] and ignoring the portions of F() that lie in the
intervals [1/2, 1] and [1, 1/2]. This is the problem of undersampling, to which we shall
return below.
4. The above cardinal series may be rewritten as follows,
f(t) =

k=
f
_
k

_
sinc
_

_
t
k

__
=

k=
f (kT) sinc
_
1
T
(t kT)
_
. (75)
In this form, we see that it is a sum of shifted sinc functions that are multiplied by the sample
values f(kT), where T = /. Moreover, the sinc functions are shifted by multiples of T as
well. In other words, the cardinal series has the same general form as the series for piecewise
constant and piecewise linear interpolations of f(t), shown in the previous lecture.
This indicates that the translated shifted sinc functions may be viewed as basis functions. More-
over, each sinc function is nonzero only at the point where it is multiplied by a sample point
f(kT) at the other sample points it is zero. As such, the cardinal series provides a quite
sophisticed interpolation of the sample points f(kT). This property is sketched in the gure
below.
-1
0
1
2
3
4
0 1 2 3 4 5
y=f(t)
y=f(4)*sinc(t-4)
Plot of a generic function f(t) along with the components f(kT)sinc
_
1
T
(t kT)
_
of its cardinal series. Here,
T = 1. The component k = 4 is labelled.
196
Example: Consider the Fourier transform given by
F() =
_
_
_

2(1
2
), 1 1,
0, otherwise.
(76)
In this case, the band limit of F() is = 1. We solve for the function f = F
1
F:
f(t) =
1

2
_

F() e
it
d
=
_
1
1
(1
2
)e
it
d
.
.
.
=
4
t
3
[sin t t cos t]. (77)
(Details left as an exercise.) Note that despite the t
3
term in the denominator, f(t) is continuous at
t = 0: f(t)
4
3
as t 0. (If f(t) were discontinuous at t = 0, then the Fourier transform F()
would not be bandlimited.)
In this case, since = 1, the sampling period is given by T =

= . As such, the cardinal series

for f(t) has the form
f(t) =

k=
f(k)
sin(t k)
t k
. (78)
The sample values f(kT) are given by
f(0) =
4
3
and f(k) =
4 cos(k)
(k)
2
=
(1)
k+1
(k)
2
, k = 0. (79)
In the gure below are plotted several partial sums for this cardinal series having the form
S
N
(t) =
N

k=N
f(k)
sin(t k)
t k
, (80)
on the interval [20, 20]. To the accuracy of the plot, perfect reconstruction of f(t) is achieved over
this interval for N = 10 (21 terms) and even N = 5 (11 terms). For N = 3, errors are visible
only for t 15. The case N = 0, i.e., using only the f(0) term, corresponds to a one-sinc-function
approximation to f(t). In all gures, the true f(t) is shown as a dotted plot.
Now suppose that we did not know the exact bandlimiting value for a function f(t). We would
then have to consider cardinal series that are constructed with estimates

of , having the form

g(t) =

k=
f
_
k

_
sin(

t k)

t k
,

< . (81)
197
-0.5
0
0.5
1
1.5
-20 -15 -10 -5 0 5 10 15 20
-0.5
0
0.5
1
1.5
-20 -15 -10 -5 0 5 10 15 20
N = 10 N = 5
-0.5
0
0.5
1
1.5
-20 -15 -10 -5 0 5 10 15 20
-0.5
0
0.5
1
1.5
-20 -15 -10 -5 0 5 10 15 20
N = 3 N = 0
Approximations to f(t) =
4
t
3
[sin t t cos t] using partial sums S
N
(t) of cardinal series given in main text.
Where seen, the dotted plots correspond to the true values of f(t).
198
As mentioned earlier, if

, then the above cardinal series will converge to f(t). since it is still
taking into consideration the interval [, ] on which the Fourier transform F() is supported. On
the other hand, if

< , then the above cardinal series will not converge to f(t) since the portions
of the spectrum lying outside the interval [

] are not being approximated properly, as sketched

in the gure below.

F()

0
These parts of the spectrum F() are ignored by undersampling
The plots shown in the next gure demonstrate this problem. We have used a value of

= 0.8 in
the cardinal series for the function f(t) of the previous example, where the true band limit is = 1.
The partial sums for N = 10 and 20 are virtually identical, which indicates that convergence of the
cardinal series to a limit has been obtained. This limit, however, is not f(t).
-0.5
0
0.5
1
1.5
-20 -15 -10 -5 0 5 10 15 20
-0.5
0
0.5
1
1.5
-20 -15 -10 -5 0 5 10 15 20
N = 10 N = 20
Approximations to f(t) =
4
t
3
[sin t t cos t] using partial sums of cardinal series, when the bandwidth is
underestimated. In the calculations above, we have used

= 0.8, which is below the true band limit = 1.

Signicant errors are incurred with N = 10. No improvement for N = 20. Plot of true f(t) is dotted.
199