DCSP-2: Fourier Transform

Jianfeng Feng
Department of Computer Science Warwick Univ., UK

Jianfeng.feng@warwick.ac.uk

http://www.dcs.warwick.ac.uk/~feng/dcsp.html
Data transmission
Channel characteristics,
Signalling methods (ADC)

Interference and noise

Fourier transform

Data compression and encryption

Bandwidth
The range of frequencies occupied by the
signal is called its bandwidth.
Power

0 B Frequency
Nyquist-Shannon Theorem
The ADC process is governed by an important la
Nyquist-Shannon Theorem
(will be discussed in Chapter 3)

An analogue signal of bandwidth B can be

completely recreated from its sampled form
provided its sampled at a rate equal to at least
twice it bandwidth.
That is
S>2B
Example

• I will guess that B = 1 Hz

• Sample at 2B = 2 Hz: x[n] = [ 0 0 0 0 ]

• Intuitively, I would say it will not work

Example

• I will guess that B = 1 Hz

• Sample at 2B < 4 Hz: x[n] = [ 1 0 -1 0 1 0 -1 0 ]

• According to N-S Thm, we can fully recover the original signal

Example

• I will guess that B = 1 Hz

• Sample at 2B < 4 Hz: x[n] = [ 1 0 -1 0 1 0 -1 0 ]

• According to N-S Thm, we can fully recover the original signal

• Well, the blue line has the identical frequency, and x[n]. What is wrong?
Noise in a channel
Noise in a channel

Attenuation
Noise in a channel
Noise in a channel
Noise in a channel
SNR
Noise therefore places a limit on the channel at
which we can transfer information

Obviously, what really matters is the signal to

noise ratio (SNR).

This is defined by the ratio signal power S to noise

power N, and is often expressed in deciBels
(dB):

SNR=10 log10 (S/N) dB

Noise sources
Input noise is common in low frequency circuits and
arises from electric fields generated by electrical
switching.

It appears as bursts at the receiver, and when present

can have a catastrophic effect due to its large power.

Other peoples signals can generate noise: cross-talk is

the term give to the pick-up of radiated signals from
adjacent cabling.
Noise sources
When radio links are used, interference from other
transmitters can be problematic.

Thermal noise is always present. It is due to the random

motion of electric charges present in all media. It can be
generated externally, or internally at the receiver.

How to tell signal from noise?

Communication Techniques I

Time frequency
Fourier Transform

bandwidth
noise power
Communication Techniques I

Time frequency
Fourier Transform

bandwidth
noise power
Communication Techniques II
Time, frequency and bandwidth

We can describe a signal in two ways.

• One way is to describe its evolution in time domain,

as we usually do.
• The other way is to describe its frequency content,
in frequency domain: what we will learn

The
Your heartbeat
• Ingredients:
a frequency ω (units: radians)
an initial phase φ (units: radians)
an amplitude A (units depending on underlying measurement)

• a trigonometric function
e.g. x[n]= A cos(ωn+φ)
cosine wave, x(t), has a single frequency,
w =2 p/T
where T is the period i.e. x(t+T)=x(t).
What do we expect?
Power

1 Hz
Time Fre
What do we expect?
Power

1 Hz
Time Fre
What do we expect?
Power

1 Hz
Time Fre
What do we expect?
Power

1 Hz
Time Fre
What do we expect?
Power

1 Hz
Time Fre
Fourier Transform I
This representation is quite general.

In fact we have the following theorem due to Fourier.

Any signal x(t) of period T can be

represented as the sum of a set of
cosinusoidal and sinusoidal waves of
different frequencies and phases
Fourier Transform II
In mathematics, the continuous Fourier transform is
one of the specific forms of Fourier analysis.

As such, it transforms one function into another, which

is called the frequency domain representation of the
original function (which is often a function in the time-
domain).

In this specific case, both domains are continuous and

unbounded.

The term Fourier transform can refer to either the

frequency domain representation of a function or to
the process/formula that "transforms" one function into
the other.
Fourier Transform III
Fourier Transform IV
• Continuous time (analogous signals): FT
(Fourier transform)
it is in theory (in Warwick, we need it)

• Discrete time: DTFT (infinity digital signals)

it is in theory (discrete version)

• DFT: Discrete Fourier transform (finite digital signals

what we can use, one line in Matlab (fft))
History of FT I
• Gauss computes trigonometric series efficiently in
1805

• Fourier invents Fourier series in 1807

• People start computing Fourier series, and develop tricks Good comes up
with an algorithm in 1958

• Cooley and Tukey (re)-discover the fast Fourier transform algorithm in

1965 for N a power of a prime

• Winograd combined all methods to give the

most efficient FFTs
History of FT II

Gauss
History of FT III

Fourier
History of FT IV

Jianfeng Feng
History of FT V

Prof Feng
Complex Numbers
Euler Formular
Exp(j a) = cos a + j sin a
The complex eponential

• the trigonometric function of choice in DSP

is the complex exponential:

• x[n] = Aexp(j(ωn+φ))

= A[cos(ωn + φ) + j sin(ωn + φ)]

The complex eponential
Most beautiful Math Formula

exp ( j π ) + 1 = 0

•Where e is Euler's number

•J is the imaginary unit

 Fourier's Song
• Integrate your function times a complex
exponential
• It's really not so hard you can do it with your
pencil • Or make the pulse wide, and the sinc
• And when you're done with this calculation grows dense,
You've got a brand new function - the Fourier The uncertainty principle is just
Transformation common sense.
• What a prism does to sunlight, what the ear • From time into frequency - from
does to sound frequency to time
• Fourier does to signals, it's the coolest trick • Let's do some examples... consider a
around sine
• Now filtering is easy, you don't need to • It's mapped to a delta, in frequency -
convolve not time
All you do is multiply in order to solve. • Now take that same delta as a function
• From time into frequency - from frequency to of time
time Every operation in the time domain • Mapped into frequency - of course - it's
• Has a Fourier analog - that's what I claim a sine!
• Think of a delay, a simple shift in time • Sine x on x is handy, let's call it a sinc.
• It becomes a phase rotation - now that's truly • Its Fourier Transform is simpler than
sublime! you think.
And to differentiate, here's a simple trick • You get a pulse that's shaped just like a
Just multiply by J omega, ain't that slick? top hat...
Integration is the inverse, what you gonna do?
Divide instead of multiply - you can do it too. • Squeeze the pulse thin, and the sinc
grows fat.
 Example
Frequency-space Image space

k1 IFT y

k2 x
FT
Fun: Decoding dream (Horikawa et al.
Science, 2013)
DCSP-3: Fourier Transform II

Jianfeng Feng
Department of Computer Science Warwick Univ., UK

Jianfeng.feng@warwick.ac.uk

http://www.dcs.warwick.ac.uk/~feng/dcsp.html
This week’s summary

• Introduce FT in layman’s language

(yesterday)

• How to calculate it (today)

This week’s summary
• Introduce FT in layman’s language
(yesterday)

• How to calculate it (today)

You might find the world is different !
http://venturebeat.com/2014/11/02/jonathan-rothbergs-butterfly-network-has-raised-
100m-for-medical-imaging-tech/
This week’s summary
• Introduce FT in layman’s language
(yesterday)

• Talk about continuous FT since it is clean and simple

• Come back to it on how to numerically calculate it later on

• Intuition

• Fourier theorem

• Examples

• Bandwidth
Fourier Theorem
This representation is quite general.
In fact we have the following theorem due to Fourier.

Any signal x(t) of period T can be represented

as the sum of a set of cosinusoidal and
sinusoidal waves of different frequencies
Intuition of FT

Two dimensional space (all points)

(a, b) = a (1,0) + b (0,1)

Signal space (all functions of t)

x (t) = a sin(ω t) + b sin (2 ω t)

Intuition of FT

Two a point in it

(a, b) = a (1,0) + b (0,1)

Intuition of FT

Two a point in it

(a, b) = a (1,0) + b (0,1)

Signal = coef basis coef basis

Intuition of FT

Two a point in it

(a, b) = a (1,0) + b (0,1)

x(t) = ?

Signal = coef basis coef basis

Intuition of FT

Two a point in it

(a, b) = a (1,0) + b (0,1)

x(t) = a F1(t) +b F2(t)

Signal = coef basis coef basis

Intuition of FT
• It is a branch of mathematics called
functional analysis: treat each function as a point in a functional space

• It is the starting pointe of modern mathematics

• It is also the actual power of mathematics

• Branch space, Hilbert space etc.

Intuition of FT

How to calculate these coefficients?

o
(a, b) = a (1,0) + b (0,1)

Functional
x (t) = a cos(ω t) + b cos (2 ω t)

w = 2 pi /T
Intuition of FT
1. Coefficient is obtained via Inner
product
<(x, y), (m,n)> = xm+yn
2. All bases are orthogonal
<(0, 1), (1,0)> = 0, (0,1) (1, 0)

(a, b) = a (1,0) + b (0,1)

Therefore

<(a, b), (1,0)> = a <(1,0), (1,0)> + b <(0,1), (1,0)> = a

<(a, b), (0,1)> = a <(1,0), (0,1)> + b <(0,1), (0,1)> = b

Intuition of FT
Two dimensi coefficient is obtained via Inner product
(a, b) = a (1,0) + b (0,1)
<(a, b), (1,0)> = a <(1,0), (1,0)> + b <(0,1), (1,0)> = a

Functional coefficient is obtained via inner product

x (t) = a cos(ω t) + b cos (2 ω t)
<x (t), cos(ω t) > = <a cos(ω t), cos(ω t) > + <b cos (2 ω t), cos(ω t) > = a
(?)
Intuition of FT

Two dimension orthogonal basis

(a, b) = a (1,0) + b (0,1)

Functional space basis

x (t) = a cos(ω t) + b cos (2 ω t)

Intuition of FT
It turns out that we can define the inner product
in the signal space

T /2
2
 x(t),y(t ) 
T  /2
T x(t )y(t )dt
T
2
 cos(wt ),cos(2wt ) 
T T cos(wt ) cos(2wt )dt

Intuition of FT Cos ( ω t)
Cos ( 2* ω t)

Cos ( 2 ω t) * Cos ( 2* ω t)

T
2
 cos(wt ),cos(2wt ) 
T T cos(wt ) cos(2wt )dt

Intuition of FT Cos ( 2 ω t)
Cos ( 2* ω t)

Cos ( 2 ω t) * Cos ( 2* ω t)

• { cos (n ω t ), n=1,2…} are orthogonal

• they form orthogonal bases

Intuition of FT
It turns out that we can define the inner product
in the signal space

2 / T ,n  m
 cos(mwt ),cos(nwt ) 
0,n  m
In particular, we have

x (t) = a cos(ω t) + b cos (2 ω t)

T /2

ò
2
a = <X(t), cos(ω t)> = x(t) cos(w t)dt
T -T /2
b=?
Intuition of FT

X
X1 X6

(1 0 0 0 0 0) X2 X3 x4 (0 0 0 0 0 1)
X5

(0 0 1 0 0 0) (0 1 0 1 0 0)

(0 1 0 0 0 0 ) (0 1 0 0 1 0)
• In an n-dim Euclidean space, we decompose any vector X in terms of orthogonal bases

• Coefficient is obtained by inner product between a basis and X

• We sometime call xi weight

• X = x1 (1 0 0 0 0 0) + x2(0 1 0 0 0 0 ) + … + x6 (0 0 0 0 0 1)
Intuition of FT
X(t)
A6 ……….
A1
cos (ω t )
A2 A3 A4
A5

x(t )  A1 cos(wt )  more

Intuition of FT
X(t)
A6 ……….
A1
cos (ω t )
A2 A3 A4
A5

cos (2 ω t )

x(t )  A1 cos(wt )  A2 cos(2wt )  more

Intuition of FT
X(t)
A6 ……….
A1
cos (ω t )
A2 A3 A4
A5
cos (3 ω t )

cos (2 ω t )

x(t )  A1 cos(wt )  A2 cos(2wt )  A3 cos(3wt )  more

Intuition of FT
X(t)
A6 ……….
A1
cos (ω t )
A2 A3 A4
A5
cos (3 ω t ) cos (4 ω t )

cos (2 ω t )

x(t )  A1 cos(wt )  A2 cos(2wt )  A3 cos(3wt )  A4 cos(4wt )  more

Intuition of FT
X(t)
A6 ……….
A1
cos (ω t )
A2 A3 A4
0
cos (3 ω t ) cos (4 ω t )

cos (2 ω t )

x(t )  A1 cos(wt )  A2 cos(2wt )  A3 cos(3wt )  A4 cos(4wt )  more

Intuition of FT
X(t)
A6
A1
cos (ω t )
A2 A3 A4
0
cos (3 ω t ) cos (4 ω t )

cos (2 ω t )

x(t )  A1 cos(wt )  A2 cos(2wt )  A3 cos(3wt )  A4 cos(4wt )  A6 cos(6wt )

Intuition of FT
X(t)
A6 ……….
A1
cos (ω t )
A2 A3 A4
0
cos (3 ω t ) cos (4 ω t )

cos (2 ω t )

• In an n-dim Euclidean space, we decompose any vector X in terms of orthogonal bases

• Coefficient is obtained by inner product between a basis and X

• We sometime call xi weight

FT: Fourier Thm in terms of Sin and Cos
• simply a generalization of common knowledge of the Euclidean space

• { 1, cos (nωt), sin(nwt), n=1,2…} are orthogonal and complete

• they form orthogonal bases

FT: Fourier Thm in terms of Sin and Cos
• simply a generalization of common knowledge of the Euclidean space

• { 1, cos (nωt), sin(nwt), n=1,2…} are orthogonal and complete

• they form orthogonal bases

FT: Fourier Thm in terms of Sin and Cos
• simply a generalization of common knowledge of the Euclidean space

• { 1, cos (nωt), sin(nwt), n=1,2…} are orthogonal and complete

• they form orthogonal bases

¥ ¥
x(t) = A0 + å An cos(nw t) +å Bn sin(nw t)
n=1 n=1

ì T /2

ò
1
ï A0 =< 1, x(t) >= x(t ) dt
ï T -T /2
ï T /2

ò
2
ï An =< cos(nw t), x(t) >= x(t) cos(nw t) dt
ï T
í -T /2

ï T /2

ò
2
ï Bn =< sin(nw t), x(t) >= x(t)sin(nw t) dt
ï T -T /2

ï 2p
ï w=
î T
where A0 is the d.c. term, and T is the period of the waveform.
FT: Fourier Thm in terms of Sin and Cos
• simply a generalization of common knowledge of the Euclidean space

• { 1, cos (nωt), sin(nwt), n=1,2…} are orthogonal and complete

• they form orthogonal bases

¥ ¥
x(t) = A0 + å An cos(nw t) +å Bn sin(nw t)
n=1 n=1

ì T /2

ò
1
ï A0 =< 1, x(t) >= x(t ) dt
ï T -T /2
ï T /2

ò
2
ï An =< cos(nw t), x(t) >= x(t) cos(nw t) dt
ï T
í -T /2

ï T /2

ò
2
ï Bn =< sin(nw t), x(t) >= x(t)sin(nw t) dt
ï T -T /2

ï 2p
ï w=
î T
where A0 is the d.c. term, and T is the period of the waveform.
FT: Fourier Thm in terms of Sin and Cos
• simply a generalization of common knowledge of the Euclidean space

• { 1, cos (nωt), sin(nwt), n=1,2…} are orthogonal and complete

• they form orthogonal bases

¥ ¥
x(t) = A0 + å An cos(nw t) +å Bn sin(nw t)
n=1 n=1

ì T /2

ò
1
ï A0 =< 1, x(t) >= x(t ) dt
ï T -T /2
ï T /2

ò
2
ï An =< cos(nw t), x(t) >= x(t) cos(nw t) dt
ï T
í -T /2

ï T /2

ò
2
ï Bn =< sin(nw t), x(t) >= x(t)sin(nw t) dt
ï T -T /2

ï 2p
ï w=
î T
where A0 is the d.c. term, and T is the period of the waveform.
FT: Fourier Thm in terms of Sin and Cos
• simply a generalization of common knowledge of the Euclidean space

• { 1, cos (nωt), sin(nwt), n=1,2…} are orthogonal and complete

• they form orthogonal bases

¥ ¥
x(t) = A0 + å An cos(nw t) +å Bn sin(nw t)
n=1 n=1

ì T /2

ò
1
ï A0 =< 1, x(t) >= x(t ) dt
ï T -T /2
ï T /2

ò
2
ï An =< cos(nw t), x(t) >= x(t) cos(nw t) dt
ï T
í -T /2

ï T /2

ò
2
ï Bn =< sin(nw t), x(t) >= x(t)sin(nw t) dt
ï T -T /2

ï 2p
ï w=
î T
where A0 is the d.c. term, and T is the period of the waveform.
Example I

x(t)=1, 0< t < p, 2p < t < 3p, 0 otherwise

Hence x(t) is a signal with a period of 2p

p 2p 3p 4p
Example I
ì p

ò dt = 2
1 1
ï A0 =
ï 2p 0
ï
ï p

ò cos(nt)dt = np sin(np ) = 0, n = 1, 2,...

2 1
í An =
ï 2p 0
ï p

ò sin(nt)dt = np (1- cos(np )), n = 1, 2,...

2 1
ï Bn =
ï
î 2p 0

Finally, we have

1 2 1 1
x(t) = + [sin(t) + sin(3t) + sin(5t) +...]
2 p 3 5
Example I |X(F)|

2/p

Frequency domain
1 2 1 1
x(t ) = + [sin(t ) + sin(3t ) + sin(5t ) + ...]
2 p 3 5

The description of a signal in terms of its constituent frequencies is called its

frequency (power) spectrum.

Example I
|X(F)|

2/p

Time domain
Frequency domain
Example I
• A periodic signal is uniquely determined by its coefficients {An, Bn}.

• If we truncated the series into finite term, the signal can be

approximated by a finite sines as shown below (compression, MP3,
MP4, JPG, … )

one term
Two terms
Three terms
Five terms
Example II (understanding music)
Example II
a. Pure tone: Script1_1.m

This confirms our earlier believe that it is a signal

with a finite bandwidth (N-S sampling Thm)
Example II
b. Different waveforms Script1_2.m

This confirms our earlier believe that it is a signal

without bandlimit (N-S sampling Thm)
 Bandwidth can be properly defined

• Bandwidth is the difference between the upper and lower frequencies in

a set of frequencies. It is typically measured in hertz
 Bandwidth can be properly defined
Power

0 B Frequency

• Bandwidth is the difference between the upper and lower frequencies in

a continuous set of frequencies. It is typically measured in hertz
Example II
C. Approximation (compression)
Script1_3.m

Without doing much, we can compress the original data now,

as in Example I.