Moments and Moment generating function

Definition: rth raw moment:

Suppose X is a random variable, discrete or continuous. Then the rth raw moment about the
origin is given by

r'  E ( X r ), r  0,1, 2,...

 x r p x if X is discrete with probability mass function p x
x
 
r
  x f ( x)dx if X is continuous with probability density function f ( x)


Definition: rth central moment:

Suppose X is a random variable, discrete or continuous with mean . . Then the rth central
moment (about the mean) is given by

 r  E (( X   ) r ), r  0,1,2,...
 ( x   ) r p x if X is discrete with probabilit y mass function p x

x
 
  ( x   ) r f ( x)dx if X is continuous with probabilit y density function f ( x)



*From the above definitions, you can already infer that

 mean = E ( X )   '
1
 Variance = E ( X   )2  
2
This means that mean or expectation is a moment of the first degree, i.e. the first raw moment,
and variance is the second central moment. What about moments of higher degrees? What
purpose do they serve?
Since the mean alone is not an accurate representative of the data, we use variance or standard
deviation to check how scattered the data is about the mean. The moments of the third degree
tell us about the skewness of the data, I.e, whether more values are towards the right or the
left of the mean. The moments of the fourth degree lead us to kurtosis, which is the flatness of
the curve once you trace the graph.

Relation between raw and central moments:

We have

 
2
 Variance = 𝐸(𝑋 2 ) − [𝐸(𝑋)]2 = E ( X   )2   , i.e.    '   '
2 2 2 1
 1  E( X   )  0
Now, let us attempt to generalize this connection between raw and central moments to the r th
moment.

r  E (( X   )r ), r  0,1,2,...
 E ( X r  rC X r 1  rC X r  2 2  rC X r  3 3  ...  (1)r  r )
1 2 3
r
 E ( X )  rC E ( X r  1 )  rC2 E ( X r  2  2 )  rC3E ( X r  3 3)  ...  (1)r E (  r )
1
 E ( X r )  rC E ( X r 1)  rC E ( X r  2 ) 2  rC E ( X r  3)  3  ...  (1)r E (  r )
1 2 3
(  is a constant and E (k )  k , for all constants k )
 r  r'  rC  '   rC  '   rC  '   ...  (1)r  r
1 r 1 2 r 2 3 r 3

Since it is easier to find the raw moments than the central moments, this relation can be very
helpful, especially after the next part, which is a still easier method to find the raw moments.
Finding raw moments can still be tedious using a table, as you have to calculate a lot of values.
It would be much easier if we could just develop a formula (or function!) that gives us all the
moments at one go. That is what we shall look at now.

Moment Generating Function:

Suppose X is a random variable, discrete or continuous. The Moment generating function (mgf
or MGF) is denoted by M (t ) and is defined as:
X

M (t )  E (etX )
X
  etx p , if X is a discrete r.v. with probability mass function P( X  x)  p
x x x
  
  etx f ( x)dx, if X is a continuous r.v. with probability density function f ( x)
 

Remark: If the mgf exists for a random variable X, we will be able to obtain all the moments of
X. It is very plainly put, one function that generates all the moments of X.

Result: Suppose X is a random variable (discrete or continuous) with moment generating

function M (t ) then the rth raw moment is given by
X
 tr
coefficient of in the exp ansion of M X (t )

r   r
' r!
, r  1, 2,...
 d  M (t ) 
 dt r 
 X t 0
Proof: We have,
M (t )  E (etX )
X
t 2 X 2 t3 X 3 tr X r
 E (1  tX    ...   ...)
2! 3! r!
t2 t3 tr
 1  tE ( X )  E ( X 2 )  E ( X 3)  ...  E ( X r )  ...
2! 3! r!
t 2 t 3 t r
i.e. M (t )  1  t  '   '   '  ...  r'  ... ____(i)
X 1 2! 2 3! 3 r!
t r
 r'  coefficient of in the exp ansion of M (t ) _____( A)
r! X
Now, differentiating (i) w.r.t t we get

( M X (t ))   '   '  t 2 '  ...  t r 1r'  ...

d 2t 3 r
____(ii)
dt 1 2 2 6 3 r!

Putting t  0 in (ii) we get,

𝑑
[𝑀 (𝑡)]𝑡=0 = 𝜇1′
𝑑𝑡 𝑋
Now, differentiating (ii) w.r.t t we get

d d d2
( M (t ))  M (t )
dt dt X dt 2 X
3(2) ' r (r 1) r 2 '
 t 2 '  t   ...  t r  ... ____ (iii)
2 6 3 r!
Putting t  0 in (iii) we get,
d2
( M X (t ))   '
dt 2 2
2
d
i.e.  '  2 M X (t ) t 0 ______( II )
2 dt
Generalizing (I) and (II) we get,

dr
r'  [ r M X (t )] _____ ( B)
dt t 0

(A) and (B) give the required result.

Examples:

1. A random variable takes values 1 and -1 with probabilities 1/2 each. Find its moment
generating function
Solution: We have the probability distribution of X to be given by

X -1 1

P(X=x) 1/2 1/2

We have the moment generating function

M (t )  E (etX )
X
  etx px
x
 et (1) ( )  et (1) ( )
1 1
2 2

e et t

2
 M (t )  cosh t
X
𝑡𝑟
So what this means is that if you were to write the expansion of cosht, the coefficients of 𝑟!
would give you all the moments of the above probability distribution.

2. A r.v. X takes values 0 and 1 with probabilities q and p respectively with q+p=1. Find the mgf
of X and show that all the moments about the origin equal p. (Bernoulli distribution)

Solution: We have the probability distribution of X to be given by:

X 0 1

P(X=x) q p

Therefore the mgf of X is given by:

M (t )  E (etX )
X
  etx p x
x
1
  etx p x
x0
 et*0q  et*1 p
i.e. M (t )  q  pet
X
We have the rth raw moment to be given by
 tr
coefficient of in the exp ansion of M X (t )

r   r
' r!
, r  1, 2,...
 d  M (t ) 
 dt r 
 X t 0
Now
M (t )  q  pet
X
 t t2 t3 tr 
 M (t )  q  p 1     ...   ... ......(1)
X  1! 2! 3! r! 
tr
 r'  coefficient of in M (t )
r! X
 r'  p ( from (1))
Hence all the moments about the origin equal p.

1
3. A r.v. X has the probability distribution P( X  x)  3C , x  0,1,2,3 Find the mgf of X
8 x
and hence find the mean and variance.
Solution: We have the probability distribution of X to be given by:

X 0 1 2 3

P(X=x) (1/8)3C0 =1/8 (1/8)3C1 =3/8 (1/8)3C2 =3/8 (1/8)3C3=1/8

Therefore the mgf of X is given by:

M (t )  E (etX )
X
  etx p x
x
3
  etx p x
x0

1 3 3 1
M (t )  et*0   et*1   et*2   et*3 
X 8 8 8 8
1

i.e. M (t )  1  3et  3e2t  e3t
X 8 
We have the rth raw moment to be given by
 tr
coefficient of in the exp ansion of M X (t )
' 
r   r r!
, r  1, 2,...
 d
 M (t ) 
 dt r  X t 0

Now

1

M (t )  1  3et  3e2t  e3t ......(1)
X 8 
dr
 r'  r  M (t ) 
dt  X t 0
d 1

 Mean   '   1  3et  3e2t  e3t 
1 dt  8

t 0

1

i.e  '  0  3et  6e2t  3e3t
1 8 t 0 
1 12
  '   3  6  3 
1 8 8
3
i.e  ' 
1 2

Again,
d2 
E( X 2 )   '  M (t ) 
2 dt 2  X t 0

i.e  ' 
d 2 1
2 dt 2  8  
1  3et  3e2t  e3t 

t 0

d 1

  3et  6e2t  3e3t 
dt  8

t 0

1

i.e  '  3et  12e2t  9e3t
2 8 t 0 
1 24
    3  12  9  
'
1 8 8
i.e  '  3
2

 
2 9 3
Variance     '   '  3   Variance 
2 2 1 4 4
4. A r.v. X has the following density function: f ( x)  e x , x  0,   0 Find the mgf of X
and hence find the mean and variance. (Exponential distribution)

Solution: We have the mgf of X to be given by:


M (t )  E (etX )   etx f ( x)dx
X 
 tx  x
  e e dx
0

   e x(t  )dx
0

e  x (t  ) 

   M (t ) 
(t   )  X  t
0
We have the rth raw moment to be given by
 tr
coefficient of in the exp ansion of M X (t )

r   r
' r!
, r  1, 2,...
d  M X (t ) 
 dt r 
 t 0

Now

M (t )  ......(1)
X  t
dr
 r'  r  M (t ) 
dt  X t 0
d   
 Mean   '  
1 dt    t t 0
  (1)  1
i.e  '     Mean   ' 
1  (  t )2  1 
 t 0
Again,
d2 
E( X 2 )   '  M (t ) 
2 dt 2  X t 0

d   
i.e  '  
2 dt  (  t )2 
 t 0
  (2(  t )(1)   2 
i.e  '     '  
2  (  t )4  2   2 
 t 0

 
2 2
2 1 1
 Variance     '   '     Variance 
2 2 1 2    2
 3
5. Suppose a r.v. X has the mgf: M X (t )  . Obtain the mean and standard deviation of X.
3t
Solution: We have the rth raw moment to be given by
 tr
coefficient of in the exp ansion of M X (t )
r'  
 r
r!
, r  1, 2,...
d  M X (t ) 
 dt r 
 t 0

3
Now M (t )  ........(1)
X 3t

dr
 r'  r  M (t ) 
dt  X t 0
d  3 
 Mean   '  
1 dt  3  t t 0
 3(1) 
i.e  '   
1  (3  t )2 
 t 0
1
 Mean   ' 
1 3

Again,
d2 
E( X 2 )   '  M (t ) 
2 dt 2  X t 0

d  3 
i.e  '  
2 dt  (3  t )2 
 t 0
 3(2(3  t )(1) 
i.e  '   
2  (3  t )4 
 t 0
2
 '   
2 9
Therefore,

 
2 2
2 1
Variance    '  '   
2 2 1 9  3

1
 Variance 
9

1 1
 S tan dard deviation  
9 3