EECS 206

Part 4: Sampling
(Chapter 4 of DSP First)

Lecture Notes
We have already discussed the basic idea of sampling a continuous-time signal to obtain a
discrete-time signal. Here's a reminder:

Sampling:

Sampling a continuous-time signal x(t) produces a discrete-time signal

x[n] = x(nTs)

where Ts = "sampling interval", fs = 1/Ts = "sampling rate" or "sampling

frequency"

(It might seem more realistic to describe sampling as x[n] = x(nTs+τ), where τ is
some time "offset". However, "time zero" is just some arbitrary reference time, so we
can assume it is chosen so that τ=0.)

Interpolation/reconstruction
This part of the course is mostly about about the reverse problem, namely, converting
a discrete-time signal to a continuous-time signal, which is called "interpolation" or
"reconstruction" or "discrete-time to continuous-time conversion".

We begin with a discussion of engineering tasks that require sampling, and see that some,
but not all, of them involve converting discrete-time signals to continuous-time. We will
then focus mainly on one of these tasks.

August 22, 2002 1 DLN -- Pt 4: Sampling

EECS 206

Sampling is the first component of an analog-to-digital converter

Analog-to-digital conversion:

Three components: sampler, quantizer and binary encoder

quantized
continuous- discrete- discrete-
time signal time signal time signal bits
x(t) x[n] ^
x[n] binary b[m]
sampler quantizer
encoder

analog-to-digital converter

Analog to digital conversion is abbreviated A/D conversion or ADC. It is also called

"digitization".

Quantization:

The "quantizer" takes the input sample x[n] and "rounds" it to the nearest of a finite
set of "quantization levels".
The quantization levels are ordinarily of the form

a, a+∆, a+2∆, ..., a+(M-1)∆

where M is the number of levels, a is the first level, and ∆ is the "level spacing".

Example: M = 8, a = 0, ∆ = 0.1
∆

x
-.35 -.05 .05 .35

Binary encoding:

The "binary encoder" assigns a distinct binary sequence, called a codeword, to each
quantization level.
m
If M = 2 , then the codewords have m bits. It is often said that this is an "m-bit
quantizer" or that this quantizer "encodes with m-bits per sample".

Example continued: m = 3

000 001 010 011 100 101 110 111

x
-.35 -.05 .05 .35

Example: A continuous-time signal x(t), its samples x[n], the quantized samples ^x[n],
and the bits produced by the encoder b[m].

(plots to be shown here)

August 22, 2002 2 DLN -- Pt 4: Sampling

EECS 206

Tasks requiring analog-to-digital conversion:

A. Digitization for digital storage/transmission

e.g. for speech, audio, images, video, ...

reconstructed
continuous- continuous-
time signal bits bits time signal
x(t) analog-to- binary digital-to- ~
b[m] b[m] x(t)
digital storage/ analog
converter transmission converter

digital-to-analog conversion

discrete- continuous-
bits time signal time signal
~
b[m] binary x[n] x(t)
interpolator
decoder

digital -to-analog converter

The goals are to get as good a reconstructed continuous-time signal as possible, using
as few bits/second as possible.
Example continued: The interpolated output of the digital-to-analog converter ^x(t).
(plot to be shown here)

B. Signal recovery (noise reduction)

e.g. for radio signal recovery, noisy audio signals, ...

reconstructed
continuous- continuous-
time signal bits bits time signal
x(t)=s(t)+n(t) analog-to- b[m] digital ^
b[m] digital-to- ^s(t)
digital processor analog
converter converter

The goal is to have the reconstructed signal ^s(t) be as similar as possible to s(t).

C. Signal detection

e.g. for radar, sonar, dollar change machines,

continuous-
time signal bits
decision
x(t)=s(t)+n(t) analog-to- b[m] digital about s(t)
digital processor
converter

The goal is to make decisions about s(t), such as a "signal present" or "signal not
present" decision, that are as reliable as possible. For this task, no digital-to-analog
conversion is needed, i.e. no reconstruction of x(t) or s(t) is needed.

D. Other
Many other systems use sampling and/or ADC, for example, MRI and other digital
imaging systems and sampled control systems.

August 22, 2002 3 DLN -- Pt 4: Sampling

EECS 206

Discrete-Time to Continuous-Time Conversion, aka Interpolation or

Reconstruction

The discussion that follows will primarily address Task A, digitization for digital
storage and/or transmission. The main issue is how to convert a discrete-time signal
into a continuous-time signal.
However, the discussion also has relevance to the other tasks as well as will be
discussed later. Note that several of our lab assignments involve Task C, signal
detection. With task B in mind, later in the course we will spend considerable time
(Chapters 5-8) developing "digital filters".

Two simple interpolators:

a. zero-order hold (Section 4.4.3)

(show example here)

b. linear interpolation (Section 4.4.4)

(show example here)

Interpolation with pulses (Section 4.4.2):

∞
~
x(t) = ∑ x[n] p(t-nTs),
n=-∞
where p(t) is some basic interpolation pulse.

Examples:
i. zero-order hold is the special case
 1, -T s/2≤t≤T s/2
p(t) =  (show plot here)
0, otherise

ii. linear interpolation is the special case

 1-|t|/T s, -T s≤t≤T s
p(t) =  (show plot here)
0, otherise

iii. parabolic interpolation (Section 4.4.5)

p(t) = ... see Figure 4.17, p. 103 (show plot here)

iv. Other p's are possible and useful.

Other interpolators:
Given a set of samples there are infinitely many ways to interpolate. That is, there
are infinitely many ways to draw a continuous-time signal that passes through the
samples. All of these may be considered to be interpolations, even if they are not
pulse-type interpolations. For example, a set of samples and three rather arbitrary
interpolations are shown below

August 22, 2002 4 DLN -- Pt 4: Sampling

EECS 206

Quality of the interpolated signal

Though we won't emphasize this much, one can use MSE to measure the quality of
the interpolated signal, i.e.
t
2
1 ~ 2
MSE = t ∫ (x(t)- x(t)) dt .
2-t1 t1

Questions:
• Among all possible interpolators, what is best?
• Among all possible pulse-type interpolators, what is best?
• For a given type of interpolation how does quality depend on the sampling
frequency fs?
• Is there a tradeoff between quality and complexity/cost of interpolators?

Basic guiding principles:

Given a discrete-time signal s[n], a good interpolation method should produce a

continuous-time signal s(t) such that
(a) s(t) has s[n] as its samples, i.e. s(nTs) = s[n], for each n
(b) s(t) is as smooth as possible.

The motivation for (a) is self-evident. The motivation for (b) is a kind of Occam's
razor principle, which postulates that the simplest explanation for some
phenomenon is the best explanation. Here we assert that the smoothest and least
fluctuating interpolation is the best interpolation, because it is in some sense the
simplest. For example, in the previous figure, one can easily identify the smoothest
and least fluctuating interpolation of the three shown. More generally, we look for
interpolations whose spectrum is concentrated at the lowest possible frequencies,
because interpolations with larger high frequency components will fluctuate more
and be less smooth. With smooth interpolations in mind, parabolic interpolation is
better than linear, which in turn is better than zero-order hold.

The effect of increasing the sampling rate f s :

• With zero-order hold, linear interpolation, parabolic interpolation and most
pulse-type interpolations, it should be intuitive that ~x(t) becomes a better
approximation to x(t) as fs increases. For example, it is intuitively clear that

MSE→ 0 as fs→∞

On the other hand, we'd prefer to be able to use as small a sampling rate as
possible, because a smaller sampling rate generates fewer samples for us to have
to store and/or to process. As a result, at some point MSE is sufficiently small
and further increases in the sampling rate are not worthwhile, i.e. when
increasing the sampling rate, there is a point of diminishing returns.
• Surprisingly, however, there is one particular choice of p(t) that creates perfect
interpolations. And, surprisingly, fs need not grow without limit. Instead it is
only required that fs be larger than twice the frequency of all spectral compo-
nents of the signal. This remarkable result stems from the "sampling theorem".

August 22, 2002 5 DLN -- Pt 4: Sampling

EECS 206

The Sampling Theorem (Sections 4.1.2 and 4.5)

If the sampling frequency fs is greater than the twice the frequency of all spectral
components of the signal x(t), then
∞
x(t) = ∑ x(nTs) p*(t-nT s)
n=-∞
where
π
sin(Tst)
p*(t) = π
Tst
Notes:

1. This theorem shows that under appropriate conditions, the signal x(t) equals,
without approximation, the interpolation ~x(t) produced from its samples using the
pulse p*(t).

2. Example: Suppose x(t) = 2 cos(2π(3)t +.1) + 2 cos(2π(5)t +.2) . Plot the

spectrum. The theorem shows that interpolation from the samples of x(t) equals
x(t) if we choose sampling rate fs > 10.

3. Interpolator using p*(t) is called "ideal interpolation". This particular pulse is

often called a "sinc function" or "sinc pulse". Notice that it has infinite support and
that due to the sine function in its numerator its value is zero at times ±Ts, ±2Ts,
±3Ts, .... When t =0, p*(0) = sin(0)/0 appears to be undefined. However, since
sin(x)/x → 1 as x → 0, we define p*(0) = 1. A central portion of p*(t) is
shown below.
p(t) for Ts = .1
1

0.8

0.6

0.4

0.2

-0.2

-0.4
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
t in seconds

4. Example: Illustration of the interpolation of a set of samples using the sinc pulse:

(show picture here)

5. This is a remarkable and surprising theorem. A complete derivation is beyond the

scope of EECS 206, but is included in EECS 306. It requires the frequency domain
analysis of aperiodic continuous-time signals via the "continuous-time Fourier
transform". Later we'll have just a brief discussion about its derivation.
The theorem is often called the Shannon Sampling Theorem, after UM alumnus
Claude Shannon who published it in his pioneering 1948 paper on the theory of
communications, which among other things made the sampling theorem widely
known to engineers. The earliest versions of the theorem go back to 1847.

6. Let fmax denote the highest frequency of any spectral component of the signal
x(t). If fmax < ∞, then x(t) is said to be "bandlimited" because its bandwidth is
finite. Moreover, we say "x(t) is bandlimited to frequency fmax".

August 22, 2002 6 DLN -- Pt 4: Sampling

EECS 206

7. The Sampling Theorem applies to bandlimited signals, e.g. a finite sum of sinusoids.
It shows that such signals can be perfectly recovered from their samples.
Moreover, it indicates that the sampling frequency fs need not grow without
bound to obtain very good interpolations. We need only have

fs > 2 fmax

or equivalently
fs
fmax <
2
2fmax is often called the "Nyquist frequency".

8. If a signal x(t) has fmax = ∞, then it is not bandlimited and the sampling theorem
does not apply. For example, a periodic square wave is not bandlimited. We'll
discuss sampling nonbandlimited signals later.

9. The fact that the pulse p*(t) has infinite support can make it difficult to build a
system that implements ideal interpolation. For example, whereas zero-order hold
and linear interpolation use just one and two samples, respectively, when producing
~
the value of x(t) at any particular time t, the ideal interpolator uses an infinite
number of samples.
In practice, few systems attempt to use ideal interpolation. Instead most use zero-
order hold, linear interpolation, or some other simple scheme. Because of this,
they generally need to use a sampling rate that is the larger than the Nyquist rate
2fmax. The ratio fs/(2fmax) is sometimes called "the oversampling ratio".
If ideal interpolation is not commonly used, what then is the value of the sampling
theorem? Its main value is in the understanding that it provides. For example, it
tells us that good interpolation is possible only when fs > 2fmax.

10. It can be shown (e.g. in EECS 306) that the signal produced by the ideal
interpolator, namely,
∞
~
x(t) = ∑ x(nTs) p* (t-T s) ,
n=-∞

is itself bandlimited to frequency fs/2 and that it is the only signal that is
bandlimited to frequency fs/2 that passes through the samples. That is, any other
interpolation of the samples has components at frequencies greater than fs/2. Thus,
~
s(t) is clearly the "smoothest" possible interpolation of the samples.
This is a very important property.

11. What's so special about 2fmax? What goes wrong when fs ≤ 2fmax?

Example: Consider sampling the signal

x0(t) = cos( 2πf0t + φ)

with sampling rate fs such that f0 = 1.1 fs. Notice that

fs < f0 = fmax < 2fmax.

Draw the signal, its samples, and the linear interpolation of the samples.

August 22, 2002 7 DLN -- Pt 4: Sampling

EECS 206

Notice that the interpolation looks like a sinusoid with a much lower frequency.
Let us see what is happening. The sampled signal is

x0[n] = cos( 2πf0nTs + φ )

= cos( 2π(1.1)fsnTs + φ )

= cos( 2π(1.1) n + φ ) since fs = 1/Ts

since frequency 2π(.1) and
= cos( 2π (.1) n + φ ) frequency 2π(1.1) are equivalent

Now, observe that the samples of x0(t) are exactly the same as the samples
from the sinusoid x1(t) = cos( 2πf1t+φ) with the much lower frequency f1 =
0.1 fs:

x1[n] = cos( 2πf1nTs + φ )

= cos( 2π(.1)fsnT s + φ )

= cos( 2π(.1) n + φ ) since fs = 1/Ts

Thus we see that sampling produces x0(t) and x1(t) at the given sampling
frequency produces x0[n] and x1[n] that are identical because they are
sinusoids with equivalent frequencies.
Recalling the basic principles of interpolation, we recognize that any reasonable
interpolator will attempt to produce the sinusoid x1(t) because it fluctuates less.
(Indeed, the samplng theorem indicates that the ideal interpolator would produce
x1(t) exactly, because fs is more than twice as large as the frequency of all of
its components.) Thus, we have the unpleasant situation that one signal x0(t) is
the input to the sampler, but a rather different signal x1(t) comes out of the
interpolator.

Example: Consider sampling the signal

x0(t) = cos( 2πf0t + φ)

with sampling rate fs such that f0 = 0.6 fs. Notice that

fs < 2f0 = 2fmax .

Draw the signal, its samples, and the linear interpolation of the samples.

Notice that the interpolation looks like a sinusoid with a lower frequency. Let us
see what is happening. The sampled signal is

x0[n] = cos( 2πf0nTs + φ )

= cos( 2π(.6)fsnT s + φ )

= cos( 2π(.6) n + φ ) since fs = 1/Ts

Now, observe that the samples of x0(t) are exactly the same as the samples
from the sinusoid x1(t) = cos( 2πf1t-φ) with the lower frequency f1 = 0.4 fs:

x1[n] = cos( 2πf1nTs - φ )

August 22, 2002 8 DLN -- Pt 4: Sampling

EECS 206

= cos( 2π(.4)fsnT s - φ )

= cos( 2π(.4) n - φ ) since fs = 1/Ts

= cos( -2π(.4) n + φ ) since cos(-θ) = cos(θ)

since frequency -2π(.4) and
= cos( 2π(.6) n + φ ) frequency 2π(.6) are equivalent

We see that sampling x0(t) and x1(t) produces x0[n] and x1[n] that are
identical because they are sinusoids with equivalent frequencies. And as in the
previous example, any reasonable interpolator will produce, at least
approximately, the sinusoid x1(t) because it has the lower frequency. (And the
ideal sampling would produce x1(t) exactly because fs is more than twice as
large as its frequency.)

12. Two continuous-time signals that have the same samples, such as in the previous
two examples, are said to be "aliases" of each other. "Aliasing" is said to occur
when, as in the previous example, a continuous-time signal x0(t) is sampled, but a
very different continuous-time signal x1(t) is produced, at least approximately, by
the interpolator. By "very" different we mean that the difference is not simply due
to a crude interpolation, like zero-order hold, but is due to the fact that the
interpolator has produced a signal that is, at least approximately, bandlimited to a
lower frequency than the original signal. It is also common to say that x0(t) has
"aliased" to x1(t).

13. Examples of sinusoids and complex exponentials that are aliases of each other:

(a) x0(t) = cos (2π f0t + φ ) and x1(t) = cos (2π (f0+mfs)t +φ)

where m is any positive or negative integer.

j(2πf0t+φ) j(2π(f0+mfs)t+φ)
(b) x0(t) = e and x1(t) = e

where m is any positive or negative integer.

(c) x0(t) = cos (2π f0t + φ ) and x1(t) = cos (2π (mfs - f0)t - φ)

where m is any positive or negative integer. In this case, it is sometimes said

that the frequencies of x0(t) and x1(t) are related by "folding".

14. From the above property we may deduce that any sinusoid x0(t) with frequency
greater than fs/2 has an alias x1(t) with frequency less than fs/2.
The following illustrates the higher frequencies that alias to a given frequency. It
also indicates whether the aliasing results in the same phase or the negative of the
phase.

-φ +φ -φ +φ -φ
f
1 3 5
fs fs fs 2 fs fs 3 fs
2 2 2

August 22, 2002 9 DLN -- Pt 4: Sampling

EECS 206

15. Let us return to the question of what goes wrong when fs < 2fmax or,
equivalently, when fmax > fs/2.
We begin by considering sinusoids. From the previous property, any sinusoid
x0(t) with frequency f0 = fmax > fs/2 has an alias with frequency f1 = fmax < fs/2,
which any reasonable interpolator will produce, at least approximately. This shows
clearly what goes wrong.
Next, consider an arbitrary periodic signal x(t). By the Fourier series theorem,
this signal is a sum of sinusoidal components. We also note that sampling is a linear
operation. Thus, the samples of the periodic signal are simply the sum of the sam-
ples from its sinusoidal components. If the signal has fmax > fs/2, then at least one
sinusoidal component will suffer aliasing, and consequently the original signal x(t)
suffers aliasing. In particular, the interpolator will produce, at least approximately,
the sum of the aliased sinusoids, rather than the sum of the original sinusoids.
In summary, if a signal has fmax > fs/2, then the interpolator produces an alias
with fmax < fs/2.

16. Derivation of the sampling theorem: Using Fourier series arguments like those in
the previous note, one can show that if a periodic signal x(t) has fmax < fs/2 (i.e.
all spectral components have frequencies less than fs/2), then no other periodic
signal with fmax < fs/2 has the same samples. This indiates that if fmax < fs/2,
then it should be possible to reconstruct the signal from its samples. However, to
show that this can be done with the sinc pulse based interpolator requires
continuous-time Fourier transform methods, which are beyond the scope of this
class. They are included in EECS 306.

17. What happens if we sample exactly at the Nyquist rate, i.e. with fs = 2fmax?
Aliasing might or might not occur. For example consider taking two samples per
period from a sinusoid, which means fs is exactly twice the frequency of the
sinusoid. These samples can be taken at the the zero crossings, in which case the
sinusoid aliases to the all zero signal. Or they can be taken at the peaks, in which
case aliasing does not occur. Or they can be taken at other times, in which case the
sinusoid has an alias at the same frequency but a different phase.
(draw pictures of each of these cases)

18. What happens if a signal is bandlimited and we sample at too low a frequency?
In the case of a periodic signal and ideal interpolation, sampling and interpolation
results reconstructs perfectly all spectral components at frequencies less than f s/2,
but all higher frequency spectral components alias to a frequency less than fs/2.

(illustrate this)

19. How to sample a signal that is not bandlimited?

There's no perfect way. One must pick a sampling rate fs. All spectral compo-
nents at frequencies above fs/2 will alias to frequencies below fs/2. If possible,
one chooses fs so large that the frequency components above fs/2 are very small.

(illustrate this)

If possible, one precedes the sampler with a "continuous-time filter" that eliminates

August 22, 2002 10 DLN -- Pt 4: Sampling

EECS 206

all frequency components above fs/2. This reduces the interpolation MSE by
approximately a factor of two. Continuous-time filters are discussed in EECS 306.

20. Sampling and interpolation for the signal recovery task.

In the signal recovery task, the signal x(t) = s(t) + n(t) is sampled with the goal of
eventually producing an approximation ^s(t) to s(t), the desired part of the signal.
Much of the reminder of the course will be devoted to designing filters as the
digital processor.

Though we are not trying to reconstruct x(t), it makes sense to sample it at a rate
greater than 2fmax for x(t), because then the samples contain all the information
in x(t). (From the samples, one could reconstruct x(t).) It is not essential that one
sample at a frequency significantly greater than 2fmax, but in some cases, this may
simplify the processing that must be performed.

The digital-to-analog converter, which is the last step of the signal recovery system,
is not actually reconstructing a signal from its samples. Rather, it is constructing a
signal ^s(t) from samples ^s[n] created by a digital processor. The interpolation
used by the digital-to-analog converter could be zero-order hold, linear
interpolation, ideal interpolation, or some other form of interpolation.

If ideal interpolation is chosen, then ^s(t) is determined by the formula

∞
^s(t) = ∑ ^s[n] p*(t-nTs)
n=-∞

There is also, sometimes, a shortcut to finding ^s(t). In particular, if ^s[n] happens

to be a sinusoid, e.g.
^s[n] = A cos(ωn+φ),
^ 0≤ω
^ ≤π

then we know that the ideal interpolator will produce the unique continuous-time
signal that is bandlimited to frequency fs/2 and has ^s[n] as its samples. What is
this signal? It easy to see by inspection that the following signal has these two
properties:
A cos(ωf
^ st+φ) .

Thus, it must be the output ^s(t) of the ideal interpolator.

This method can also be applied when the discrete-time signal is the sum of
sinusoids, by applying it separately to each sinusoidal component. It can also be
applied to arbitrary periodic signals, because by the DFT Theorem, any periodic
signal is the sum of sinusoids.

21. Sampling for signal detection.

In the signal detection problem, we sample the signal x(t) = s(t) + n(t) with the
goal of making a decision about s(t) based on the samples. The system does not
output a continuous-time signal. As in the signal recovery problem, it makes sense
to sample x(t) at a rate greater than 2fmax, because in this case, the samples
contain all the information that was originally in x(t). Sometimes sampling at a
significantly higher rte simplifies the processing that must be done.

August 22, 2002 11 DLN -- Pt 4: Sampling