Chapter 4: Random Processes

Random Processes
Outline
 Introduction
 Definition of a Random Process
 Characterization of Random Processes
 Mean, Correlation, and Covariance Functions
 Classification of Random Processes
 Power Spectral Densities of Random Processes
 Response of Linear Systems to Random Inputs

2
 Introduction

 The theory of random processes was first developed in connection

with the study of fluctuations and noise in physical systems.
 A random process is the mathematical model of an empirical
process whose development is governed by probability laws.
 Random processes provides useful models for the studies of such
diverse fields as statistical physics, communication and control,
time series analysis, population growth, and management sciences.

3
 Definition of a Random Process

 A random process is a family of random variables {X(t), tϵT} defined on a

given probability space, indexed by the parameter t, where t varies over an
index set T.
 In a random process {X(t), tϵT}, the index set T is called the parameter set of
the random process.
 The values assumed by X(t) are called states, and the set of all possible values
forms the state space E of the random process.
 If the index set T of a random process is discrete, then the process is called a
discrete-time random process.

4
 Definition of a Random Process Cont’d…..

 A discrete-time random process is also called a random sequence and

is denoted by {Xn , n = 1, 2, 3, . . .).

 If T is continuous, then we have a continuous-time random process.

 In fact, a random process {X(t), tϵT} is a function of two arguments

{X(t, ω), tϵT, ωϵΩ}.

 For a fixed time t=tk, X(tk, ω) = Xk(ω) is a random variable denoted by

X(tk), as ω varies over the sample space Ω.

5
 Definition of a Random Process Cont’d…..

 On the other hand, for a fixed sample point ωi ϵΩ, X(t, ωi) = Xi(t)
is a single function of time t, called a sample function or a
realization of the process.
 The totality of all sample functions is called an ensemble.

 Of course if both ω and t are fixed, X(tk , ωi) is simply a real

number.
 In the following discussion, we use the notation X(t) to represent
X(t, ω).

6
 Characterization of Random Processes

 If X(t) is a random process, then for fixed t=t1, X1=X(t1) represents a

random variable.
 Its distribution function is given by:

FX ( x1 , t1 )  P{ X (t )  x1 }
 Notice that FX(x, t) depends on t, since for a different t, we obtain a
different random variable.
 The first-order probability density function of the process X(t) is
defined as:

dFX ( x1 , t1 )
f X ( x1 , t1 ) 
dx1

7
 Characterization of Random Processes Cont’d…..
 For t = t1 and t = t2, X(t) represents two different random variables X1 = X(t1) and X2 =
X(t2) respectively.
 Their joint distribution is given by:

 The second-order 2 , t1 , t 2 ) 
FX ( x1 , xprobability (t1 )  xof1 ,theX random
P{ Xfunction
density (t2 )  process
x2 } X(t) is:

 Similarly represents the nth order density function of the

process X(t).  2 FX ( x1 , x2 , t1 , t2 )
f X ( x1 , x2 , t1 , t2 ) 
x1 x2
f X ( x1 , x2 ,  xn , t1 , t 2  , t n )

8
 Mean, Correlation, and Covariance Functions

 As in the case of random variables, random processes are often

described by using statistical averages.
 The mean of X(t) is defined by:

 X (t )  E X (t )
where X(t) is treated as a random variable for a fixed value of t.

 In general, μX(t) is a function of time, and it is often called the

ensemble average of X(t).

9
 Mean, Correlation, and Covariance Functions …...

 A measure of dependence among the random variables of X(t) is

provided by its autocorrelation function, defined by:

R XX (t1 , t 2 )  E X (t1 ) X ( 2 )

 Note that:

R XX (t1 , t 2 )  R XX (t 2 , t1 ) and R XX (t , t )  E X 2 (t ) 
 The autocovariance function of X(t) is defined by:

C XX (t1 , t 2 )  CovX (t1 ) , X (t 2 )  EX (t1 )   X (t1 )X (t 2 )   X (t 2 )

 R XX (t1 , t 2 )   X (t1 )  X (t 2 )

10
 Mean, Correlation, and Covariance Functions …...

 It is clear that if the mean of X(t) is zero, then:

C XX (t1 , t 2 )  R XX (t1 , t 2 )
 Note that the variance of X(t) is given by:


 X 2 (t )  Var X (t )  E X (t )   X (t )2 
 If X(t) is a complex random process, then its autocorrelation function
RXX(t1, t2) and autocovariance function CXX(t1, t2) are defined, respectively,
by:

 
R XX (t1 , t 2 )  E X (t1 ) X * (t 2 ) and


C XX (t1 , t 2 )  E X (t1 )   X (t1 )X (t 2 )   X (t 2 )
*

11
 Classification of Random Processes

i. Stationary Processes
 A random process {X(t), tϵT} is said to be stationary or strict-sense
stationary (SSS) if, for all n and for every set of time instants (ti ϵT, i = 1,2,
. . . , n),

FX ( x1 ,........, x n , t1 , ....., t n )  FX ( x1 , ........, x n , t1   , ......, t n   )

 Hence, the distribution of a stationary process will be unaffected by a shift
in the time origin, and X(t) and X(t+τ) will have the same distributions for
any τ.
 Nonstationary processes are characterized by distributions depending on
the points t1, t2, . . . , tn.

12
 Classification of Random Processes Cont’d……

ii. Wide-Sense Stationary Processes

 A random process X(t) is wide-sense stationary (WSS) if:

1. E X (t )   X (constant )

2. R XX (t1 , t 2 )  E X (t1 ) X (t 2 )  R XX  t 2  t1 

 Note that a strict-sense stationary process is also a WSS process,

but, in general, the converse is not true.

13
 Classification of Random Processes Cont’d……

 Other types of random processes include:

 Independent Processes
 Markov Processes
 Normal Processes
 Ergodic Processes
 Poisson Processes

14
 Power Spectral Densities of Random Processes

 The autocorrelation function of a continuous-time random

process X(t) is defined as:

R XX ( )  E X (t ) X (t   )

 Properties of RXX(τ):

1. R XX ( )  R XX ( )

2. R XX ( )  R XX (0)


3. R XX (0)  E X 2 (t )  0 
15
 Power Spectral Densities of Random Processes……

 In case of a discrete-time random process X(n), the

autocorrelation function of X(n) is defined by:

R XX (k )  E X (n) X (n  k )

 Properties of RX(k):

1. R XX ( k )  R XX (k )

2. R XX (k )  R XX (0)


3. R XX (0)  E X 2 (n)  0 
16
 Power Spectral Densities of Random Processes……

 Two processes X(t) and Y(t) are called (mutually) orthogonal if:

R XY ( )  0 , for all 
 Similarly, the cross-correlation function of two discrete-time jointly
WSS random processes X(n) and Y(n) is defined by:

R XY (k )  E X (n)Y (n  k )
 The various properties of RXY(k) similar to those of RXY(τ) can be
obtained by replacing τ by k in the above equations.

17
 Power Spectral Densities of Random Processes……

 The power spectral density (or power spectrum) SXX(ω) of a continuous-

time random process X(t) is defined as the Fourier transform of RXX(τ),
i.e. ,


S XX     R XX ( )e  j d

 Thus, taking the inverse Fourier transform of SX(ω), we obtain:

1 
R XX     S XX ( )e j d
2 
 The above equations are known as the Wiener-Khinchin relations.

18
 Power Spectral Densities of Random Processes……

 Properties of SXX(ω):

1. S XX ( ) is real and S XX ( )  0

2. S XX ( )  S XX ( )

 
3. E X (t )  R XX (0) 
2 1
2 



S XX ( )d

 Similarly, the power spectral density SXX(Ω) of a discrete-time

random process X(n) is defined as the Fourier transform of RXX(k):

S XX     XX
R ( k ) e  jk

k  
19
 Power Spectral Densities of Random Processes……

 Thus, taking the inverse Fourier transform of SXX(Ω), we obtain:

1 
R XX (k ) 
2 

S XX ()e jk d

 Properties of SXX(Ω):
1. S XX (  2 )  S XX ()

2. S XX () is real and S XX ()  0

3. S XX ()  S XX ()

 
3. E X (n)  R XX (0) 
2 1
2

S

XX (  ) d

20
 Power Spectral Densities of Random Processes……

 The cross power spectral density (or cross power spectrum) SXY(ω)
of two continuous-time random processes X(t) and Y(t) is defined as
the Fourier transform of RXY(τ):

S XY     R XY ( )e  j d


 Thus, taking the inverse Fourier transform of SXY(ω), we get:

1 
R XY     S XY ( )e j d
2 

21
 Power Spectral Densities of Random Processes……

 Properties of SXY(ω):
 Unlike SXX(ω), which is a real-valued function of ω, SXY(ω), in general, is a complex-
valued function.

1. S XY ( )  S YX ( )

(spectral ( )SXY(Ω) of two discrete-time random

*
2. cross
 Similarly, the S XYpower )  S XYdensity
processes X(n) and Y(n) is defined:


S XY ()   XY
R (
k  
k ) e  jk

22
 Power Spectral Densities of Random Processes……

 Taking the inverse Fourier transform of SXY(Ω), we get:

1 
R XY (k ) 
2 

S XY ()e jk d
 Properties of S (ω):
XY

 Unlike SXX(Ω), which is a real-valued function of Ω, SXY(Ω), in general, is a complex-

valued function.

1. S XY (  2 )  S XY ()

2. S XY ()  S YX ()
*
3. S XY ()  S XY ()
23
Example on Random Processes
Example:
Consider a random process X(t) defined by
X (t )  A cos( 0 t   )

where  0 and A are constants and  is a uniform

random variable over the interval (0, 2 )

a. Find the mean  X (t ).

b. Find the autocorrelation function R XX (t1 , t 2 ).

c. Find the autocovariance function C XX (t1 , t 2 ).

d . Determine whether X (t ) is WSS random process or not.

e. Find the power spectral density of X (t ).

24
Example on Random Processes Cont’d……

Solution:
a.  X (t )  E X (t )  E A cos( 0 t   )  AE cos( 0 t   )

But , cos( 0 t   )  cos( 0 t)cos - sin( 0 t)sin

  X (t )  E X (t )  AE cos( 0 t)cos - sin( 0 t)sin 

 A cos( 0 t) E cos    A sin( 0 t) E sin  

1 2
E cos     cos d  0
2 0

1 2
Similarly , E sin     sin d  0
2 0

  X (t )  E X (t )  0

25
Example on Random Processes Cont’d……

Solution:
b. R XX (t1 , t 2 )  E X (t1 ) X (t 2 )

 E A cos( 0 t1   ) A cos( 0 t 2   )

 A 2 E cos( 0 t1   ) cos( 0 t 2   )

A2
 E cos  0 (t 2  t1 )  cos( 0 (t1  t 2 )  2 )
2
But , E cos  0 (t 2  t1 )  cos  0 (t 2  t1 ) and

E cos( 0 (t1  t 2 )  2 )  0

A2
 R XX (t1 , t 2 )  cos  0 (t 2  t1 )
2
26
Example on Random Processes Cont’d……

Solution:
c. C XX (t1 , t 2 )  R XX (t1 , t 2 )   X (t1 )  X (t 2 )

A2
 cos  0 (t 2  t1 )  0
2
A2
 C XX (t1 , t 2 )  cos  0 (t 2  t1 )
2
d . Since the mean is constant and the autocorrelation function

depends on time difference only, X (t ) is a WSS random process.

27
Example on Random Processes Cont’d……

Solution:
e. Since X (t ) is a WSS random process, the autocorrelation

function can be simply written as :

A2
R XX ( )  cos( 0 )
2
The power spectral density of X (t ) is given by :

S XX ( )   R XX ( )e  j d


But from Fourier transform pair table, we have :

FT cos( 0 t )   (   0 )   (   0 )

A 2 A 2
 S XX ( )   (   0 )   (   0 )
2 2
28
 Response of Linear Systems to Random Inputs
 If a WSS random process X(t) with autocorrelation function RXX(τ) is applied to a linear
system with impulse response h(t), then the cross correlation function RXY(τ) and the output
autocorrelation function RYY(τ) are given as follows.

X(t h(t) Y(t)

)
R XY ( )  R XX ( ) * h * ( )

And ,

RYY ( )  R XY ( ) * h( )

 R XX ( ) * h * ( ) * h( )
29
 Response of Linear Systems to Random Inputs…..
 Using properties of Fourier transform, we get:

f (t ) 
FT
F ( ) and g (t ) 
FT
G ( )

 property,
 Then using the above 
f (t ) * gthe(t )crossFT
F ( power
and output )G (spectral
) densities can be
evaluated as:

 
S XY ( )  FT R XX ( ) * h * ( )  S XX ( ) H * ( )

And ,

S YY ( )  FT RYY ( )  FT R XY ( ) * h( )
2
 S XY ( ) H ( )  S XX ( ) H ( )

30
Example on Response of Linear Systems
Example:
Consider a WSS random process X(t) with autocorrelation
function given by:
a 
R XX ( )  e , where a is a real positive constant

Let the random process X(t) be applied to the input of an LTI

system with impulse response given by:
h(t )  e  bt u (t ) , where b is a real positive constant

Find the autocorrelation function of the output Y(t) of the

system.

31
Example on Response of Linear Systems Cont’d……

Solution:
The frequency response H ( ) of the system is :
1
H ( )  FT h(t ) 
j  b
The power spectral density of X (t ) is :
2a
S XX ( )  FT R XX ( )  2
  a2
Then, the power spectral density of Y (t ) is given by :

2  1  2a 
S YY ( )  S XX ( ) H ( )   2 2  2 
   b    a 
2

32
Example on Response of Linear Systems Cont’d……

Solution:
a  2b  b  2a 
 S YY ( )  2  2 2 
 2  2 2 
a  b b    b  a  b b    b 
2 2

Taking the inverse Fourier transform of both sides of the above

equation, we obtain :

RYY ( )  2
1
a  b 2 b

ae
b 
 be
a 


33
Exercise on Random Processes
1. Consider a random process X(t) defined by
X (t )  A cos(0t   )
where 0 and  are constants and A is a uniform
random variable over the interval (0, 2)
a. Find the mean  X (t ).
b. Find the autocorrelation function RXX (t1 , t 2 ).
c. Find the autocovariance function C XX (t1 , t 2 ).
d . Determine whether X (t ) is WSS random process or not.
e. Find the power spectral density of X (t ).

34
Exercise on Random Processes
2. Consider a random process X(t) defined by:
X (t )  A sin( 0 t   )
where A and  are independent random variables which are
  
uniformly distributed over the intervals [0, 1] and  , 
 2 2
respectively and  0 is a constant.
a. Find the mean  X (t ).
b. Find the autocorrelation function R XX (t1 , t 2 ).
c. Find the autocovariance function C XX (t1 , t 2 ).
d . Determine whether X (t ) is WSS random process or not.
e. Find the power spectral density of X (t ).

35
Exercise on Random Processes Cont’d……

3. Two random processes X(t) and Y(t) are given by:

X (t )  A cos(0t   ) and Y (t )  A sin(0t   )

where A and 0 are constants and  is a uniform random
variable over the interval (0, 2 ).
a. Find the cross correlation function of X (t ) and Y (t ).
b. Verify tha t RXY (- )  RXY ( )

36
Exercise on Random Processes Cont’d……

4. Consider a discrete-time WSS random process X(n) with

autocorrelation function given by:

RXX (k )  2e |k |

Find the power spectral density of X(n).

37