ECO 227Y1 Foundations of Econometrics

Kuan Xu

University of Toronto
kuan.xu@utoronto.ca

November 23, 2023

Kuan Xu (UofT) ECO 227 November 23, 2023 1 / 64

Ch 4 Continuous Variables and Their Probability
Distributions

1 Introduction
2 The Probability Distribution for a Continuous Random Variable
3 The Expected Value for Continuous Random Variables
4 The Uniform Probability Distribution
5 The Normal Probability Distribution
6 The Gamma Probability Distribution
7 The Beta Probability Distribution
8 Some General Comments
9 Other Expected Values
10 Tchebysheff’s Theorem

Kuan Xu (UofT) ECO 227 November 23, 2023 2 / 64

Introduction

Real life examples—the uncountably infinite number of points in an

interval;
Informally, a random variable that can take on any value in an
interval is called a continuous random variable;
Key difference: For a discrete random variable, it is possible to assign
a nonnegative probability to each of its possible values. But for a
continuous random variable, it is impossible to assign non-zero
probabilities to all the points on a line interval while satisfying the
requirement that the probabilities of its distinct possible values sum
to 1.
We need a different method to describe the probability distribution
for a continuous random variable.

Kuan Xu (UofT) ECO 227 November 23, 2023 3 / 64

The Probability Distribution for a Continuous Random
Variable (1)

We need to introduce the distribution function (or cumulative distribution

function) associated with a random variable.

Distribution Function
Let Y denote any random variable. The distribution function of Y ,
denoted by F (y ), is such that F (y ) = P(Y ≤ y ) for −∞ < y < ∞.

The nature of the distribution function—step or continuous

function—determines whether the random variable is discrete or
continuous.

Kuan Xu (UofT) ECO 227 November 23, 2023 4 / 64

The Probability Distribution for a Continuous Random
Variable (2)
Example: Suppose that Y has a binomial distribution with n = 2 and
p = 1/2. Please find F (y ).
Solution: The probability function for Y is given by
   y  2−y
2 1 1
p(y ) = ,
y 2 2
for y = 0, 1, 2.
We have p(0) = 1/4, p(1) = 1/2 and p(2) = 1/4. Therefore,



0, for y < 0

1/4, for 0 ≤ y < 1
F (y ) = P(Y ≤ y ) =
3/4, for 1 ≤ y < 2



1, for y ≥ 2

Note that this probability distribution function is a step function.

Kuan Xu (UofT) ECO 227 November 23, 2023 5 / 64
The Probability Distribution for a Continuous Random
Variable (3)

Fig. 4.1, p. 159

Figure: Binomial Distribution Function

Kuan Xu (UofT) ECO 227 November 23, 2023 6 / 64

The Probability Distribution for a Continuous Random
Variable (4)

Note that the probability distribution is for any random variable (discrete
or continuous random variable). It must satisfies a set of properties.

Theorem 4.1—Properties of Distribution Function

If F (y ) is a distribution function, then
1 F (−∞) = limy →−∞ F (y ) = 0.
2 F (∞) = limy →∞ F (y ) = 1.
3 F (y ) is a nondecreasing function of y . If y1 and y2 are any values
such that y1 < y2 , then F (y1 ) ≤ F (y2 ).

Note that the above properties must be satisfied by any random variable.
Now we are ready to define a continuous random variable.

Kuan Xu (UofT) ECO 227 November 23, 2023 7 / 64

The Probability Distribution for a Continuous Random
Variable (5)

Continuous Random Variable

A random variable Y with distribution function F (y ) is said to be
continuous if F (y ) is continuous, for −∞ < y < ∞.

Kuan Xu (UofT) ECO 227 November 23, 2023 8 / 64

The Probability Distribution for a Continuous Random
Variable (6)

Fig. 4.2, p. 159

Figure: Distribution Function for a Continuous Random Variable

Kuan Xu (UofT) ECO 227 November 23, 2023 9 / 64

The Probability Distribution for a Continuous Random
Variable (7)

Probability Density Function

Let F (y ) be the distribution for a continuous random variable Y . Then
f (y ), given by
dF (y )
f (y ) = = F ′ (y )
dy
wherever the derivative exists, is called the probability density function for
the random variable Y .
Ry
Remarks: It turns out F (y ) = −∞ f (t)dt.
Remarks: The probability density function is a theoretical model for the
frequency distribution (histogram) of a population of measurements.

Kuan Xu (UofT) ECO 227 November 23, 2023 10 / 64

The Probability Distribution for a Continuous Random
Variable (7)

Theorem 4.2—Properties of Density Function

If f (y ) is a density function for a continuous random variable, then
1 f (x) ≥ 0 for all y , where −∞ < y < ∞.
R∞
−∞ f (y )dy = 1.
2

Kuan Xu (UofT) ECO 227 November 23, 2023 11 / 64

The Probability Distribution for a Continuous Random
Variable (8)

Example:
Let

0, for y < 0,

F (y ) = y , for 0 ≤ y ≤ 1,

1, for y > 1.


d Find f (y ) and graph it.

Solution:
 d(0)
 dy = 0, for y < 0,
dF (y )  d(y )
f (y ) = = dy = 1, for 0 ≤ y ≤ 1,
dy  d(1)

dy = 0, for y > 1.

Kuan Xu (UofT) ECO 227 November 23, 2023 12 / 64

The Probability Distribution for a Continuous Random
Variable (9)

We show F (y ) and f (y ) below.

Fig. 4.4, p. 162 Fig. 4.5, p. 163

(a) F (y ) (b) f (y )

Kuan Xu (UofT) ECO 227 November 23, 2023 13 / 64

The Probability Distribution for a Continuous Random
Variable (10)

If Y is a continuous random variable and a and b are constants such that

a < b, then P(Y = a) = 0 and P(Y = b) = 0 and Theorem 4.3 implies
that

P(a < Y < b) = P(a ≤ Y < b)

= P(a < Y ≤ b)
= P(a ≤ Y ≤ b)

Kuan Xu (UofT) ECO 227 November 23, 2023 14 / 64

The Probability Distribution for a Continuous Random
Variable (11)

Quantile
Let Y denote any random variable. If 0 < p < 1, the pth quantile of Y ,
denoted by ϕp , is the smallest value such that P(Y ≤ ϕp ) = F (ϕp ) ≥ p. If
Y is continuous, ϕp is the smallest value such that
F (ϕp ) = P(Y ≤ ϕp ) = p.

Remarks: The median of Y is a special case for quantile at p = 1/2, ϕ.5 .

Theorem 4.3—P(a ≤ Y ≤ b)
If the random variable Y has density function f (y ) and a < b, then the
probability that Y falls in the interval [a, b] is
Z b
P(a ≥ Y ≥ b) = f (y )dy = F (b) − F (a).
a

Kuan Xu (UofT) ECO 227 November 23, 2023 15 / 64

The Probability Distribution for a Continuous Random
Variable (12)

Fig. 4.8, p. 164

Figure: P(a ≤ Y ≤ b)

Kuan Xu (UofT) ECO 227 November 23, 2023 16 / 64

The Probability Distribution for a Continuous Random
Variable (12)

Example: Given f (y ) = ( 38 )y 2 for 0 ≤ y ≤ 2. Please find P(1 ≤ Y ≤ 2).

Solution:
2
Z 2   3   
3 2 3 y 3 7 7
P(1 ≤ Y ≤ 2) = y dy = = = .
1 8 8 3 8 3 8
1

Kuan Xu (UofT) ECO 227 November 23, 2023 17 / 64

The Expected Value (1)

The Expected Value

The expected value of a continuous random variable Y is
Z ∞
E (Y ) = yf (y )dy ,
−∞

pr ovided that the integral exists.

Remarks: Technically, E (Y ) is said to exist if

Z ∞
|y |f (y )dy < ∞.
−∞

It is the case for all expectations that we discuss.

Kuan Xu (UofT) ECO 227 November 23, 2023 18 / 64

The Expected Value (2)

Theorem 4.4—E (g (Y ))
Let g (Y ) be a function of Y ; t hen the expected value of g (Y )′ is given by
Z ∞
E [g (Y )] = g (y )f (y )dy ,
−∞

provided that the integral exists.

Kuan Xu (UofT) ECO 227 November 23, 2023 19 / 64

The Expected Value (3)

Theorem 4.5—Properties
Let c be a constant and let g (Y ), g1 (Y ), g2 (Y ), . . . , gk (Y ) be functions of
a continuous random variable Y . Then the following results hold:
1 E (c) = c.
2 E [cg (Y )] = cE [g (Y )].
3 E [g1 (Y ) + g2 (Y ) + · · · + gk (Y )] =
E [g1 (Y )] + E [g2 (Y )] + · · · + E (gk (Y )).

Kuan Xu (UofT) ECO 227 November 23, 2023 20 / 64

The Expected Value (4)

Example: Y has probability density function

(
(3/8)y 2 , for 0 ≤ y ≤ 2
f (y ) =
0, elsewhere

Please find E (Y ) and V (Y ). Solution:

2
Z 2
2 4
E (Y ) = y (3/8)y dy = (3/8)(1/4)y = 1.5.
0
0

2
Z 2 Z 2
2 2 2 4 5
E (Y ) = y (3/8)y dy = (3/8)y dy = (3/8)(1/5)y = 2.4.
0 0
0
2 2 2
V (Y ) = E (Y ) − [E (Y )] = 2.4 − 1.5 = 2.4 − 2.25 = .15.

Kuan Xu (UofT) ECO 227 November 23, 2023 21 / 64

The Uniform Probability Distribution (1)
Intuition: A bus (customer) arrives at the bus station (store checkout
counter) with the same relative frequency between any two smaller time
intervals (say 2 minutes) over the entire 10 minute time intervals.

Fig. 4.9, p. 174

Figure: Density of Y

Kuan Xu (UofT) ECO 227 November 23, 2023 22 / 64

The Uniform Probability Distribution (2)

Uniform Probability Distribution

If θ1 < θ2 , a random variable Y is siad to have a continuous uniform
probability distribution on the interval (θ1 , θ2 ) if and only if the density
function of Y is (
1
, θ1 ≤ y ≤ θ2
f (y ) = θ2 −θ1
0, elsewhere

Remarks: Sometimes it is written as Y ∼ Uniform(θ1 , θ2 ).

Kuan Xu (UofT) ECO 227 November 23, 2023 23 / 64

The Uniform Probability Distribution (3)

Parameters
The constants that determine the specific form of a density function are
called parameters of the density function.

Remarks: θ1 and θ2 are the parameters of the uniform probability density

function.
Example: Arrivals of customers at a checkout counter follow a Poisson
distribution. It is known that, during a given 30-minute period, one
customer arrived at the counter. Find the probability that the customer
arrived during the last 5 minutes of the 30-minute period.
Solution: The actual time of arrival follows a uniform distribution over the
interval (0, 30). If Y denotes the arrival time, then
30
30 − 25
Z
1
P(25 ≤ Y ≤ 30) = dy = = 1/6.
25 30 30

Kuan Xu (UofT) ECO 227 November 23, 2023 24 / 64

The Uniform Probability Distribution (4)

Theorem 4.6—Mean and Variance of Uniform Probability Distribution

If θ1 < θ2 and Y is a random variable uniformly distributed on the interval
(θ1 , θ2 ), then
θ1 + θ 2
µ = E (Y ) =
2
and
(θ2 − θ1 )2
σ 2 = V (Y ) = .
12
Note
Z ∞
1
 
E (Y ) = y dy
−∞ θ2 − θ1
θ2
y2
" #
1
 
=
θ2 − θ1 2 θ1

θ22 − θ12 2 2
= ∵ θ2 − θ1 = (θ2 + θ1 )(θ2 − θ1 )
2(θ2 − θ1 )
θ2 + θ1
= .
2

Kuan Xu (UofT) ECO 227 November 23, 2023 25 / 64

The Uniform Probability Distribution (5)
Z ∞
1
 
2 2
E (Y ) = y dy
−∞ θ2 − θ1
 " 3 #θ2
1 y

=
θ2 − θ1 3 θ1

θ23 − θ13 3 3 2 2
= ∵ θ2 − θ1 = (θ2 − θ1 )(θ2 + θ2 θ1 + θ1 )
3(θ2 − θ1 )
θ22 + θ2 θ1 + θ12
= .
3

2 2
V (Y ) = E (Y ) − [E (Y )]
θ22 + θ2 θ1 + θ12 θ2 + θ1 2
= −[ ]
3 2
4(θ22 + θ2 θ1 + θ12 ) 3(θ2 + θ1 )2
= −[ ]
12 12
4(θ22 + θ2 θ1 + θ12 ) 3(θ22 + 2θ2 θ1 + θ12 )
= −[ ]
12 12
(θ22 − 2θ2 θ1 + θ12 )
=
12
(θ2 − θ1 )2
= .
12

Kuan Xu (UofT) ECO 227 November 23, 2023 26 / 64

The Normal Probability Distribution (1)

Mostly widely used continuous probability distribution;

It approximates the distributions of some random variables.

Normal Probability Distribution

A random variable Y is said to have a normal probability distribution if
and only if, for σ > 0 and −∞ < µ < ∞, the density function of Y is

1 (y −µ)2
f (y ) = √ e− 2σ 2 ,
2πσ 2
where −∞ < y < ∞.

Remarks: Sometimes it is written as Y ∼ N(µ, σ 2 ) with µ and σ 2 being

the parameters.

Kuan Xu (UofT) ECO 227 November 23, 2023 27 / 64

The Normal Probability Distribution (2)

Theorem 4.7—Mean and Variance of Normal Probability Distribution

If Y is a normally distributed random variable with parameters µ and σ 2 ,
then
E (Y ) = µ
and
V (Y ) = σ 2 .

Remarks: The proof of the above theorem is postponed to the discussion

of the moment-generating function.

Kuan Xu (UofT) ECO 227 November 23, 2023 28 / 64

The Normal Probability Distribution (3)

To find P(a ≤ Y ≤ b), we need to evaluate

Z b (y −µ)2
1
√ e− 2σ 2 dy .
a 2πσ 2
But there is no closed-form expression. We rely on the numerical
integration techniques. In R, we use pnorm(y0 , µ, σ) to generate
P(Y ≤ y0 ) and qnorm(p, µ, σ) to generate P(Y ≤ ϕp ) = p.

Kuan Xu (UofT) ECO 227 November 23, 2023 29 / 64

The Normal Probability Distribution (4)
Example: Let Z ∼ N(0, 1). Please find
1 P(Z > 2),
2 P(−2 ≤ Z ≤ 2),
3 P(0 ≤ Z ≤ 1.73).
Solution:
1 P(Z > 2):
> 1-pnorm(2,0,1)
[1] 0.02275
2 P(−2 ≤ Z ≤ 2):
> 1-2*pnorm(-2,0,1)
[1] 0.9545
3 P(0 ≤ Z ≤ 1.73) :
> pnorm(1.73,0,1) - .5
[1] 0.45818
Kuan Xu (UofT) ECO 227 November 23, 2023 30 / 64
The Normal Probability Distribution (5)

Example: Let Y ∼ N(75, (10)2 ). Find P(80 ≤ Y ≤ 90).

Solution: Let Z = Y σ−µ ∼ N(0, 1).

80 − 75
z1 = = .5
10
90 − 75
z2 = = 1.5
10
P(80 ≤ Y ≤ 90) = P(.5 ≤ Z ≤ 1.5) = .2471

> pnorm(1.5,0,1) - pnorm(.5,0,1)

[1] 0.24173

Kuan Xu (UofT) ECO 227 November 23, 2023 31 / 64

The Gamma Probability Distribution (1)

Gamma distributions have been used to model continuous variables that

are always positive and have skewed distributions. It is often used to
describe the time between independent events that have consistent
average time intervals between them, such as rainfalls, insurance claims,
wait time, etc

Fig. 4.15, p. 185

Figure: A Skewed Probability Density Function

We will use its special case more often in econometrics — chi-square

distributions.

Kuan Xu (UofT) ECO 227 November 23, 2023 32 / 64

The Gamma Probability Distribution (2)

Gamma Probability Distribution

A random variable Y is said to have a gamma distribution with parameters
α > 0 and β > 0 if and only if the density function of Y is
( α−1 −y /β
y e
f (y ) = β α Γ(α) , 0 ≤ y ≤ ∞,
0, elsewhere,

where Z ∞
Γ(α) = y α−1 e −y dy .
0

Remarks: Sometimes it is written as Y ∼ Gamma(α, β).

Kuan Xu (UofT) ECO 227 November 23, 2023 33 / 64

The Gamma Probability Distribution (3)
For the gamma distribution, α is a shape parameter while β is a scale
parameter.

Fig. 4.16, p. 186

Figure: Gamma Density Functions, β = 1

Kuan Xu (UofT) ECO 227 November 23, 2023 34 / 64

The Gamma Probability Distribution (4)

If α is an integer, the distribution function of a gamma-distributed random

variable can be expressed as a sum of certain Poisson probabilities. If α is
not
R d an integer, it is not possible to have a closed-form solution to
c f (x)dx, where 0 < c < d < ∞.
Note that the case for α = 1 is an exception.

Kuan Xu (UofT) ECO 227 November 23, 2023 35 / 64

The Gamma Probability Distribution (5)

Theorem 4.8 — Mean and Variance of Gamma Probability

Distribution
If Y has a gamma distribution with parameters α and β, then

µ = E (Y ) = αβ

and
σ 2 = V (Y ) = αβ 2 .

Recall Γ(α) = (α − 1)Γ(α − 1) and Γ(1) = 1. If α is an integer,

Γ(n) = (n − 1)!.

Kuan Xu (UofT) ECO 227 November 23, 2023 36 / 64

The Gamma Probability Distribution (6)

To show E (Y ) = αβ, write

y α−1 e −y /β
Z ∞ Z ∞ !
E (Y ) = yf (y )dy = y dy .
−∞ 0 β α Γ(α)

By definition,
y α−1 e −y /β
Z ∞ !
dy = 1.
0 β α Γ(α)

Hence, Z ∞ 
α−1 −y /β α
y e dy = β Γ(α).
0

Kuan Xu (UofT) ECO 227 November 23, 2023 37 / 64

The Gamma Probability Distribution (7)

Using the above in E (Y ), we get

Z ∞
1 α −y /β
 βαΓ(α)
E (Y ) = y e dy = = αβ.
β α Γ(α) 0 Γ(α)
| {z }
β α+1 Γ(α+1)

Find E (Y 2 ).
y α−1 e −y /β
Z ∞ ! Z ∞
2 2 1 α+1 −y /β
E (Y ) = y dy = y e dy
0 β α Γ(α) β α Γ(α) 0

2
1 α+2 β (α + 1)αΓ(α) 2
= [β Γ(α + 2)] = = α(α + 1)β
β α Γ(α) Γ(α)

Find V (Y ) = E (Y 2 ) − [E (Y )]2 .

2 2 2 2 2 2 2 2
V (Y ) = α(α + 1)β − (αβ) = α β + αβ − α β = αβ .

Kuan Xu (UofT) ECO 227 November 23, 2023 38 / 64

The Gamma Probability Distribution (8)

Two special cases of the gamma probability distribution are of particular

importance in econometrics—the chi-square distribution and the
exponential distribution.
Chi-square Distribution
Let v be a positive integer. A random variable Y is said to have a
chi-square distribution with v degrees of freedom if and only if Y is a
gamma-distributed random variable with parameters α = v /2 and β = 2.

Remarks, we often denote this as Y ∼ χ2 (v ). The probability density of Y

is  v −1
 y 2 v e −y /2 , 0 ≤ y ≤ ∞,
f (y ) = 2 2 Γ( v2 )
0, elsewhere.

Kuan Xu (UofT) ECO 227 November 23, 2023 39 / 64

The Gamma Probability Distribution (9)

Theorem 4.9—Mean and Variance of Chi-square Distribution

If Y is a chi-square random variable with v degrees of freedom, then

µ = E (Y ) = v

and
σ 2 = V (Y ) = 2v .

If we apply Theorem 4.8 (E (Y ) = αβ and V (Y ) = αβ 2 ) with α = v /2

and β = 2.
How to tell a gamma distribution is a chi-square distribution?
Gamma α β µ = αβ σ 2 = αβ 2 χ2 µ = v = 2α σ 2 = 2v = 4α
3/2 2 µ=3 σ 2 = (3/2)22 = 6 3 6
4/2 2 µ=4 σ 2 = (4/2)(22 ) = 8 4 8

Table: χ2 (v ) and Gamma(α, β)

Kuan Xu (UofT) ECO 227 November 23, 2023 40 / 64

The Gamma Probability Distribution (10)
When the gamma density function has α = 1, it is called the exponential
density function.
Exponential Distribution
A random variable Y is said to have an exponential distribution with
parameter β > 0 if and only if the density function of Y is
(
1 −y /β
βe , 0 ≤ y < ∞,
f (y ) =
0, elsewhere.

Remarks: Let α = 1 in the gamma density function:

( α−1 −y /β
y e 1 −y /β
β α Γ(α) = β e , 0 ≤ y ≤ ∞,
f (y ) =
0, elsewhere.
This density function is often useful for modeling the length of life of
electronic components. Note that α is a shape parameter whereas β is a
scale parameter.
Kuan Xu (UofT) ECO 227 November 23, 2023 41 / 64
The Gamma Probability Distribution (11)

Theorem 4.10—Mean and Variance of Exponential Distribution

If Y is an exponential random variable with parameter β, then

µ = E (Y ) = β

and
σ 2 = V (Y ) = β 2 .

Remarks: This is the result of applying Theorem 4.8 directly with α = 1.

That is,
E (Y ) = αβ = β
and
V (Y ) = αβ 2 = β 2 .

Kuan Xu (UofT) ECO 227 November 23, 2023 42 / 64

The Gamma Probability Distribution (12)

Example: Let Y be the length of life of electronic components and it follows an exponential probability density function. In
addition, if a > 0 and b > 0, show P(Y > a + b|Y > a) = P(Y > b).
Solution: We know
P(Y > a + b)
P(Y > a + b|Y > a) = ,
P(Y > a)

because the intersection of events (Y > a + b) and (Y > a) is the event (Y > a + b). Note

Z ∞ ∞
1 −y /β −y /β −(a+b)/β
P(Y > a + b) = e dy = −e =e .
a+b β a+b

Similarly, Z ∞
1 −y /β −a/β
P(Y > a) = e dy = e .
a β

Combining these results yields

e −(a+b)/β −b/β
P(Y > a + b|Y > a) = =e = P(Y > b).
e −a/β

This shows the memory-less property of this distribution.

Kuan Xu (UofT) ECO 227 November 23, 2023 43 / 64

The Beta Probability Distribution (1)

The beta density function is a two-parameter density function defined over

the closed interval 0 ≤ y ≤ 1. It is often used as a model for proportion.
Beta Probability Distribution
A random variable Y is said to have a beta probability distribution with
parameters α > 0 and β > 0 if and only if the density function of Y is
( α−1
y (1−y )β−1
B(α,β) , 0 ≤ y ≤ 1,
f (y )
0, wlsewhere,

where Z 1
Γ(α)Γ(β)
B(α, β) = y α−1 (1 − y )β−1 dy = .
0 Γ(α + β)

Kuan Xu (UofT) ECO 227 November 23, 2023 44 / 64

The Beta Probability Distribution (2)
The graphs of beta density functions vary widely for different values of the
parameters. See

Fig. 4.17, p. 195

Figure: Beta Density Functions

Kuan Xu (UofT) ECO 227 November 23, 2023 45 / 64

The Beta Probability Distribution (3)

Note that we are not restricted to 0 ≤ y ≤ for the beta density function as
we can always transform the random variable over the interval c ≤ y ≤ d
y −c
by defining y ∗ = d−c such that 0 ≤ y ∗ ≤ 1.
The cumulative distribution function for the beta random variable is
commonly called the incomplete beta function and is denoted by
Z y α−1
t (1 − t)β−1
F (y ) = dt = Iy (α, β)
0 B(α, β)

In R, we use pbeta(y0 , α, 1/β) to generate P(Y ≤ y0 ) and

qbeta(p, α, 1/β) to generate the pth quantile, ϕp , such that
P(Y ≤ ϕp ) = p.

Kuan Xu (UofT) ECO 227 November 23, 2023 46 / 64

The Beta Probability Distribution (4)

Theorem 4.11—Mean and Variance of Beta Probability Distribution

If Y is a beta-distributed random variable with parameters α > 0 and
β > 0, then
α
µ = E (Y ) =
α+β
and
αβ
σ 2 = V (Y ) = .
(α + β)2 (α + β + 1)

Kuan Xu (UofT) ECO 227 November 23, 2023 47 / 64

The Beta Probability Distribution (5)

To find µ = E (Y ).

µ = E (Y )
Z ∞
= yf (y )dy
−∞

y α−1 (1 − y )β−1
Z 1 !
= y dy
0 B(α, β)
Z 1
1 α β−1
= y (1 − y ) dy
B(α, β) 0
B(α + 1, β)
= ∵α>0⇒α+1>0
B(α, β)
Γ(α + 1)Γ(β) Γ(α + β)
= ·
Γ(α + β + 1) Γ(α)Γ(β)
αΓ(α)Γ(β) Γ(α + β)
= · ∵ Γ(n) = (n − 1)Γ(n − 1)
(α + β)Γ(α + β) Γ(α)Γ(β)
α
= .
α+β

Kuan Xu (UofT) ECO 227 November 23, 2023 48 / 64

The Beta Probability Distribution (6)

To find σ 2 = V (Y ), find E (Y 2 ) first.

Z 1
2 Γ(α + β) α+1 β−1
E (Y ) = y (1 − y ) dy
Γ(α)Γ(β) 0
Γ(α + β) Γ(α + 2)Γ(β)
= ·
Γ(α)Γ(β) Γ(α + β + 2)
Γ(α + β) (α + 1)Γ(α + 1)Γ(β)
= ·
Γ(α)Γ(β) (α + β + 1)Γ(α + β + 1)
Γ(α + β) (α + 1)αΓ(α)Γ(β)
= ·
Γ(α)Γ(β) (α + β + 1)(α + β)Γ(α + β)
(α + 1)α
= .
(α + β + 1)(α + β)

Kuan Xu (UofT) ECO 227 November 23, 2023 49 / 64

The Beta Probability Distribution (7)

2 2 2
σ = V (Y ) = E (Y ) − [E (Y )]
2
(α + 1)α α

= −
(α + β + 1)(α + β) α+β
α2 + α α2
= −
(α + β + 1)(α + β) (α + β)2
(α2 + α)(α + β) α2 (α + β + 1)
= −
(α + β + 1)(α + β)2 (α + β + 1)(α + β)2
α3 + α2 β + α2 + αβ α3 + α2 β + α2
= −
(α + β + 1)(α + β)2 (α + β + 1)(α + β)2
αβ
=
(α + β + 1)(α + β)2

Kuan Xu (UofT) ECO 227 November 23, 2023 50 / 64

Some General Comments

Density functions are theoretical models for real data;

Which model to use?
1 Theoretical consideration;
2 Match data.
Does it matter if we use the wrong model?
1 Errors in inference;
2 Some methods are insensitive to asumptions about the underlying
population distributions.

Kuan Xu (UofT) ECO 227 November 23, 2023 51 / 64

Other Expected Values (1)

kth Moment about the Origin

If Y is a continuous random variable, then the kth moment about the
origin is given by
µ′k = E (Y k ), k = 1, 2, . . . .
The kth central moment is given by

µk = E [(Y − µ)k ], k = 1, 2, . . . .

Kuan Xu (UofT) ECO 227 November 23, 2023 52 / 64

Other Expected Values (2)
Example: Find µ′k for the uniform random variable with θ1 = 0 and θ2 = θ.
Solution: By definition,
θ
∞ θ
y k+1
Z Z  
1
µ′k = E (Y ) = k k
y f (y )dy = yk
dy =
−∞ 0 θ θ(k + 1)
0

θk
= .
k +1
Thus,
θ
µ′1 = µ = ,
2
θ2
mu2′ = ,
3
and
θ3
mu3′ = .
4
Kuan Xu (UofT) ECO 227 November 23, 2023 53 / 64
Other Expected Values (3)

Moment-generating Function
If Y is a continuous random variable, then the moment-generating
function of Y is given by

m(t) = E (e tY ).

The moment-generating function is said to exist if there exists a constant

b > 0 such that m(t) is finite for |t| ≤ b.

Kuan Xu (UofT) ECO 227 November 23, 2023 54 / 64

Other Expected Values (4)

Note
Z ∞
tY ty
E (e ) = e f (y )dy
−∞

t2y 2 t3y 3 x2 x3
Z ∞ !
x
= 1 + ty + + + ··· f (y )dy ∵ e = 1 + x + + + ···
−∞ 2! 3! 2! 3!

t2 t3
Z ∞ Z ∞ Z ∞ Z ∞
2 3
= f (y )dy + t yf (y )dy + y f (y )dy + y f (y )dy + · · ·
−∞ −∞ 2! −∞ 3! −∞

′ t2 ′ t3 ′
= 1 + tµ1 + µ2 + µ3 + · · ·
2! 3!

Also note
d k m(t) ′
= µk .
dt k t=0

Kuan Xu (UofT) ECO 227 November 23, 2023 55 / 64

Other Expected Values (5)
Example: Find the moment-generating function for a gamma distributed random variable.
Solution:

Z ∞ " α−1 −y /β #
tY ty y e
m(t) = E (e )=
e dy
0 β α Γ(α)
Z ∞
1 1
  
α−1 x
= y exp −y −t dy exp(x) = e here
β α Γ(α) 0 β
Z ∞ " #
1 α−1 −y
= y exp dy
β α Γ(α) 0 β/(1 − βt)

Note that the integral of the variable factor of any density function must equal the reciprocal of the constant factor. That is, if
f (y ) = cg (y ), where c is a constant, then

Z ∞ Z ∞
f (y )dy = cg (y )dy = 1
−∞ −∞

so Z ∞
1
g (y )dy = .
−∞ c

Also recall the gamma distribution function is

y α−1 e −y /β
f (y ) = .
β α Γ(α)

Kuan Xu (UofT) ECO 227 November 23, 2023 56 / 64

Other Expected Values (6)

1 ).
Applying the above result to the integral in m(t) noting [β/(1 − βt)] > 0 (when β > 0, (1 − βt) > 0 which implies t < β

α−1
g (y ) = y exp{−y /[β/(1 − βt)]}

is the variable factor of a gamma density function with parameters α > 0 and β/(1 − βt) > 0. Thus, we obtain

α
1 β 1
 
m(t) = Γ(α) = for t < 1/β.
β α Γ(α) 1 − βt (1 − βt)α

Kuan Xu (UofT) ECO 227 November 23, 2023 57 / 64

Other Expected Values (6)

Example: Expand the moment-generating function of the above example into a power series in t and thereby obtain µ′k .
Solution: Recall m(t) = (1 − βt)−α . Using the expansion for a binomial term of the form (x + y )n ,1 we have

−α −α−1
m(t) = (1 − βt) = 1 + (−α)(1) (−βt)

(−α)(−α − 1)(1)−α−2 (−βt)2 (−α)(−α − 1)(−α − 2)(1)−α−3 (−βt)3

+ + + .···
2! 3!

t 2 [α(α + 1)β 2 ] t 3 [α(α + 1)(α + 2)β 3 ]

= 1 + t(αβ) + + + ···
2! 3!

Because µ′k is the coefficient of t k /k!, we find

′
µ1 = µ1 = αβ, (See Theorem 4.8)

′ 2
µ2 = α(α + 1)β ,

′ 3
µ3 = α(α + 1)(α + 2)β ,

and so on. In general, µ′k

= α(α + 1)(α + 2) · · · (α + k − 1)β . Recall k

σ 2 = V (Y ) = E (Y 2 ) − [E (Y )]2 = α(α + 1)β 2 − [αβ]2 = (α2 + α)β 2 − α2 β 2 = αβ 2 . (See Theorem 4.8)

1          
Given Cjn = nj = j!(n−j)! , (x + y )n = y = n0 x n y 0 + n1 x n−1 y 1 + n2 x n−2 y 2 + · · · + nn x 0 y n .
n! Pn n−j j
j=0 x
Kuan Xu (UofT) ECO 227 November 23, 2023 58 / 64
Other Expected Values (8)
Example: First, let Z ∼ N(0, 1). Find the moment-generating-function of Z as follows:
Z ∞
Zt zt 1 − 1 z2
mZ (t) = E (e ) = e √ e 2 dz
−∞ 2π
Z ∞ Z ∞
1 zt− 1 z 2 1 − 1 z 2 +zt− 1 t 2 + 1 t 2
= √ e 2 dz = √ e 2 2 2 dz
−∞ 2π −∞ 2π
Z ∞ Z ∞
1 1
− (z−t) + t2 1 2 1 t 2 1 − (z−t)2
1
= √ e 2 2 dz = e 2 √ e 2 dz
−∞ 2π −∞ 2π
| {z }
=1

Second, let Y = a + bZ , where a and b are constants and Z ∼ N(0, 1). Find the moment-generating function for Y .
tY t(a+bZ ) ta tbZ
mY (t) = E (e ) = E (e ) = E (e e )
ta tbZ ta
= e E (e ) = e mZ (tb).
Let a = µ and b = σ. Then
µt µt 1 (tσ)2 µt+ 1 t 2 σ 2
mY = e mZ (tσ) = e e2 =e 2 .
2
This result can be verified. We can use mY (t) to show Y ∼ N(µ, σ ).

µt+ 1 t 2 σ 2
 
d e 2
dmY (t) 1 t 2 σ2
 
2 µt+ 2
= = (µ + tσ )e
dt t=0
dt t=0 t=0
= µ = E (Y ).

µt+ 1 t 2 σ 2
 
d (µ + tσ 2 )e 2
d 2 mY (t)
=
dt dt

Kuan Xu (UofT) ECO 227 November 23, 2023 59 / 64

Other Expected Values (9)

Recall duv
dx
= uv ′ + vu ′ .

µt+ 1 t 2 σ 2
 
d (µ + tσ 2 )e 2
1 t 2 σ2 2 µt+ 1 t 2 σ2
 
2 2 µt+ 2
= (µ + tσ ) e +σ e 2
dt t=0
2 2 2
= µ + σ = E (Y ).

2 2 2 2 2 2
V (Y ) = E (Y ) − [E (Y )] = µ + σ − µ = σ .

This explains Theorem 4.7.

Theorem 4.12—Moment-generating Function for g (Y )

Let Y be a random variable with density function f (y ) and g (Y ) be a function of Y . Then the moment-generating function for
g (Y ) is Z ∞
tg (Y ) tg (y )
E [e ]= e f (y )dy .
−∞

Kuan Xu (UofT) ECO 227 November 23, 2023 60 / 64

Other Expected Values (10)
Example: Let g = Y − µ, where Y is a normally distributed random variable with mean µ and variance σ 2 . Then find the
moment-generating function for g (Y ).
Solution:
2 2
Z ∞ " #
tg (Y ) t(Y −µ) t(y −µ) exp[−(y − µ) /2σ ]
m(t) = E [e ] = E [e ]= e √ dy
−∞ σ 2π

Let u = y − µ. Then, du = dy and

Z ∞
1 tu −u 2 /(2σ 2 )
m(t) = √ e e du
σ 2π −∞
Z ∞
1 1
   
2 2
= √ exp − (u − 2σ tu) du
σ 2π −∞ 2σ 2

2 2
Multiply and divide the right-hand-side by e t σ /2 to get

exp[−(1/2σ 2 )(u 2 − 2σ 2 tu + σ 4 t 2 )]
Z ∞
t 2 σ 2 /2
m(t) = e √ du
−∞ σ 2π
Z ∞ 2 2 2
t 2 σ 2 /2 exp[−(u − σ t) /2σ ]
= e √ du.
−∞ σ 2π

R∞ exp[−(u−σ 2 t)2 /2σ 2 ]

−∞
√ du continas a normal density function with mean σ 2 t and variance σ 2 . The integral is equal to 1.
σ 2π
Hence,
t 2 σ 2 /2
m(t) = e .

Kuan Xu (UofT) ECO 227 November 23, 2023 61 / 64

Tchebysheff’s Theorem (1)

We restate Tchebysheff’s theorem for a continuous random variable.

Theorem 4.13—Tchebysheff’s Theorem
Let Y be a random variable with finite mean µ and variance σ 2 . Then, for
any k > 0,
1
P(|Y − µ| < kσ) ≥ 1 − 2
k
or
1
P(|Y − µ| ≥ kσ) ≤ 2 .
k

Kuan Xu (UofT) ECO 227 November 23, 2023 62 / 64

Tchebysheff’s Theorem (2)

Note
Z ∞
2 2
V (Y ) = σ = (y − µ) f (y )dy
−∞
Z µ−kσ Z µ+kσ Z ∞
2 2 2
= (y − µ) f (y )dy + (y − µ) f (y )dy + (y − µ) f (y )dy
−∞ µ−kσ µ+kσ

R µ+kσ
Note µ−kσ (y − µ)2 f (y )dy ≥ 0. Also note (y − µ)2 ≥ k 2 σ 2 for all values of y between −∞ and µ − kσ or between
µ + kσ and ∞. Therefore,
Z µ−kσ Z ∞
2 2 2 2 2
V (Y ) = σ ≥ k σ f (y )dy + k σ f (y )dy
−∞ µ+kσ

2 2 2 2 2
σ ≥ k σ [P(Y ≤ µ − kσ) + P(Y ≥ µ + kσ)] = k σ P(|Y − µ| ≥ kσ).

2 2
Dividing by k σ , we obtain
1
P(|Y − µ| ≥ kσ) ≤ ,
k2
or, equivalently,
1
P(|Y − µ| < kσ) ≥ 1 − ,
k2

Kuan Xu (UofT) ECO 227 November 23, 2023 63 / 64

 The End

Kuan Xu (UofT) ECO 227 November 23, 2023 64 / 64