RANDOM PROCESS

Dr.P.Indumathi
Associate Professor
Department of Electronics Engineering
MIT Campus, Anna University
indu@mitindia.edu
 Outline of the lecture
• Why random process?
• What is random process?
• Stationary process
• Ergodic process
• Gaussian process
• Central limit theorem
• Transmission of a random process through a
LTI filter
28-02-2018 Lecture at Velammal Engineering College 2
• 1. Problems in Correlation & Covariance
functions, Power Spectral Density
• 2. Properties of Ergodic Processes, Gaussian
Process,
• 3. Transmission of a Random Process Through
a LTI filter
• 4. Analysis of noise using random process

28-02-2018 Lecture at Velammal Engineering College 3

 Initial word ….
• Trying to capture the feel of random process
in this lecture!
• But the complete understanding will prevail
only after you learn all the ‘mathematical
relations’ associated with the related
concepts!!
• If time permits little mathematics also…

28-02-2018 Lecture at Velammal Engineering College 4

 Deterministic signals
• Deterministic signals – values at any instant t
are known from their analytical or graphical
description
• Complete CERTAINTY about their values at any
instant t

28-02-2018 Lecture at Velammal Engineering College 5

 Example deterministic signals

28-02-2018 Lecture at Velammal Engineering College 6

 How much information a deterministic
signal convey?

28-02-2018 Lecture at Velammal Engineering College 7

• Such signals which can be specified with
certainty cannot convey information!

28-02-2018 Lecture at Velammal Engineering College 8

• Information is related to uncertainty.
• The higher the uncertainty about a signal to
be received, higher its information content.
• Hence signals that convey information must
be unpredictable.

28-02-2018 Lecture at Velammal Engineering College 9

 Also…
• Noise signals – unwanted electrical
“disturbance” that interfere with proper
reception and reproduction of transmitted
signals are also unpredictable.

28-02-2018 Lecture at Velammal Engineering College 10

 To sum up…
• These unpredictable message signals and
noise waveforms are examples of random
process.

28-02-2018 Lecture at Velammal Engineering College 11

• The outcome of a random experiment may be
a real number (as in the case of rolling a die)
or it may be nonnumerical and described by
phrases (Heads or Tails in tossing a coin).

28-02-2018 Lecture at Velammal Engineering College 12

• From a mathematical point of view, it is
desirable to have numerical values for all
outcomes.Hence we assign a real number to
each sample point according to some rule.

28-02-2018 Lecture at Velammal Engineering College 13

• If there are m sample points,η1,η2,……ηm, then
using some convenient rule, we assign a real
number x(ηi) to sample point ηi (i = 1,2,...m)
• In case of tossing a coin, for example, “T”= -1,
and “H”=1
• Thus x(.) is a function that maps η1,η2,……ηm
into real numbers x1,x2,……..,xm.

28-02-2018 Lecture at Velammal Engineering College 14

• IN OTHER WORDS, we now have a random
variable X that takes on values x1,x2,……..,xm.

• That is X is Random Variable and x1,x2,……..,xm

are the values of the random variable

28-02-2018 Lecture at Velammal Engineering College 15

• The notion of Random Process is an extension
of Random Variable.

28-02-2018 Lecture at Velammal Engineering College 16

Through an example we shall get to random
process….
• Consider, the temperature X of a certain city at
noon.
• X is indeed a random variable and it takes on
different values everyday.
• To get the complete statistics of X, we need to
record values of X at noon over many days ( a
large number of trials)

28-02-2018 Lecture at Velammal Engineering College 17

• From this data, we can determine pX (x), the
PDF(Probability Density Function) of the random
variable X (the temperature at noon).
• But the temperature is also a function of time. At 1 pm,
for example the temperature may have an entirely
different distribution from that of temperature at noon.
• Thus random variable X is a function of time and can be
expressed as X(t) ! OK…

28-02-2018 Lecture at Velammal Engineering College 18

• A random variable that is a function of time is
called a Random Process or Stochastic
Process.
• A RANDOM PROCESS IS A COLLECTION OF AN
INFINITE NUMBER OF RANDOM VARIABLES.

28-02-2018 Lecture at Velammal Engineering College 19

 To qualify as a random process….
• X could be a function of any other variable
such as distance.
• A random process may also be a function of
more than one variable.
• Like say Z(x,t) could be a random process.

28-02-2018 Lecture at Velammal Engineering College 20

 Ensemble
• The collection of all possible sample functions
is known as the ensemble of the random
process X(t)
• The number of waveforms (sample functions )
may be finite or infinite.
• In X(t) which is temperature of a city, the
ensemble has infinite waveforms.

28-02-2018 Lecture at Velammal Engineering College 21

 Finite number of sample functions – an
example
• If output of a binary signal generator is
considered over 0 to 10T, how many
waveforms are there in the ensemble?
• Finite or infinite?
• Why?

28-02-2018 Lecture at Velammal Engineering College 22

• Atmost 210 waveforms are in the ensemble!

28-02-2018 Lecture at Velammal Engineering College 23

28-02-2018 Lecture at Velammal Engineering College 24
 Ensemble statistics
• What is it?
• When we deal with random process(es) we do
not know which sample function will occur in
a trial.
• Hence for any specification, characterisation
or optimisation, we need to average over the
entire ensemble.

28-02-2018 Lecture at Velammal Engineering College 25

 Autocorrelation function of a random
process
• Autocorrelation leads to spectral information
of random process…. (How? )

28-02-2018 Lecture at Velammal Engineering College 26

• The frequency content of a process depends on the
rapidity of the amplitude change with time. And this
can be measured by ‘correlating’ amplitudes at t1 and
t1 +𝛕
• Example: Consider a random process X(t) as a slowly
varying process compared to another random
process Y(t).The amplitudes at t1 and t1 +𝛕 are
similar in X(t) , i.e., they have stronger correlation in
X(t).
• How will the ‘correlation’ be for Y(t)?
28-02-2018 Lecture at Velammal Engineering College 27
• For Y(t) the amplitudes at t1 and t1 +𝛕 have
little resemblance, that is they have weaker
correlation.
• Correlation – measure of ‘similarity’
• Autocorrelation –It is computed by multiplying
amplitudes at t1 and t2 and then averaging the
product over the ensemble.

28-02-2018 Lecture at Velammal Engineering College 28

• The two auto-correlation functions RX(𝛕) and
RY(𝛕)

• Their possible variations for varying 𝛕

28-02-2018 Lecture at Velammal Engineering College 29

28-02-2018 Lecture at Velammal Engineering College 30
• Stationary Process – The statistical parameters
do not vary with time
• Wide Sense Stationary Process
• Ergodic Process – Time average is ensemble
average…
• (What is Time average ? What is ensemble
average?....) (contd..)

28-02-2018 Lecture at Velammal Engineering College 31

• Ensemble average is average “across the process”
• Time average is average “along the process”
• (A naïve Example): Tossing unbiased coin at 50
places at noon. The average outcome is ensemble
average. (Time is fixed here)
• Tossing a unbiased coin 50 times in the
Auditorium of Velammal Engg College and finding
the average outcome. This is time average.

28-02-2018 Lecture at Velammal Engineering College 32

 Central Limit Theorem
• “The sum of many i.i.d random random
variables is a Gaussian random variable”
• Many is how many?
• Of course Infinity
• Implications of the theorem

28-02-2018 Lecture at Velammal Engineering College 33

• Gaussian process, Gaussian noise, Gaussian
random variable – why are they important in
Communications?

28-02-2018 Lecture at Velammal Engineering College 34

 Gaussian or normal random
variable (an example variable x)

28-02-2018 Lecture at Velammal Engineering College 35

 Mean and standard deviation
• In the Gaussian distribution in the slide before
mean = some a; Mean can assume any finite
value.
• The parameter σ is called the standard
deviation and can assume any finite and
positive value.
• A Gaussian random variable with mean 0 and
variance 1 is called standard normal

28-02-2018 Lecture at Velammal Engineering College 36

 Why is Gaussian random variable very
important in Communications?
• Thermal noise which is the major source of
noise in Communication systems, has a
Gaussian distribution.

28-02-2018 Lecture at Velammal Engineering College 37

 Gaussian process….
• Thermal noise in electronic devices which is
produced by the random movement of
electrons due to thermal agitation can be
closely modelled by a Gaussian distribution….

28-02-2018 Lecture at Velammal Engineering College 38

Why is thermal noise a Gaussian process?...

• Consider a resistor. The free electrons in a

resistor move as a result of thermal agitation.
• The movement of electrons is quite random
and can be in any direction…

28-02-2018 Lecture at Velammal Engineering College 39

 (Contd…)
• However the velocity of the electrons is a
function of the ambient temperature.
• The higher the temperature, the higher the
velocity of the electrons…
• The movement of these electrons generate a
current with a random value..

28-02-2018 Lecture at Velammal Engineering College 40

 (Contd…)
• We can consider each electron in motion as a
tiny current source, whose current is a
random variable that can be positive or
negative depending on the direction of the
movement of the electron…

28-02-2018 Lecture at Velammal Engineering College 41

 (Contd…)
• We can assume that at least a majority of
these sources behave independently and
therefore the total current is the sum of a
large number of independent and identically
distributed random variables…

28-02-2018 Lecture at Velammal Engineering College 42

 (Contd..)
• The total current generated by all electrons ,
(which is nothing but generated thermal
noise) is the sum of the currents of all these
current sources…

28-02-2018 Lecture at Velammal Engineering College 43

 So Central Limit theorem sums it
up…!!!
• Now by applying the Central Limit theorem,
we conclude that this total current has a
Gaussian distribution…
• That is why, thermal noise can be very well
modelled by a Gaussian random process..

28-02-2018 Lecture at Velammal Engineering College 44

 Why is it AWGN?
• AWGN – Additive White Gaussian Noise
• White – prevalent at all frequencies (just as
white colour is made of all colours)
• Additive - as this noise is added to the signal in
wired communication systems more
commonly…

28-02-2018 Lecture at Velammal Engineering College 45

 Gaussian Process
• Why is noise as a Gaussian process – poses
“imminent” threats to the transmitted signal
in a Communication System?
• Asymptotic nature

28-02-2018 Lecture at Velammal Engineering College 46

 Transmission of a Random Process through
a LTI filter..

• What is the expected output process? Why?

28-02-2018 Lecture at Velammal Engineering College 47

• So…………

28-02-2018 Lecture at Velammal Engineering College 48

• The ball is in your court.

28-02-2018 Lecture at Velammal Engineering College 49

• ALL THE BEST FOR YOUR EXAMS

28-02-2018 Lecture at Velammal Engineering College 50