0% found this document useful (0 votes)
57 views6 pages

3 Mathematical Expectation: 3.1 Univariate Random Variable

This document defines mathematical expectation and provides formulas for calculating the mean, variance, and moment-generating function of random variables. It states that the mathematical expectation of a function g(X) of a random variable X, denoted E(g(X)), is the average value of g(X). The mean (μ) is the expectation of X (E(X)), the variance (V(X)) is the expectation of (X - μ)2, and the moment-generating function φ(θ) is the expectation of eθX. Formulas are provided showing that the mean of aX + b is aE(X) + b, the variance of aX + b is a2

Uploaded by

Lalit Narayan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
57 views6 pages

3 Mathematical Expectation: 3.1 Univariate Random Variable

This document defines mathematical expectation and provides formulas for calculating the mean, variance, and moment-generating function of random variables. It states that the mathematical expectation of a function g(X) of a random variable X, denoted E(g(X)), is the average value of g(X). The mean (μ) is the expectation of X (E(X)), the variance (V(X)) is the expectation of (X - μ)2, and the moment-generating function φ(θ) is the expectation of eθX. Formulas are provided showing that the mean of aX + b is aE(X) + b, the variance of aX + b is a2

Uploaded by

Lalit Narayan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

3 Mathematical Expectation

⎧ 




⎪ g(x i )p i = g(xi ) f (xi ), (Discrete RV),
3.1 Univariate Random Variable ⎪

⎨ i i
E(g(X)) = ⎪
⎪  ∞





⎩ g(x) f (x) dx, (Continuous RV).
Definition of Mathematical Expectation (਺ֶత‫ظ‬଴஋):
−∞
Let g(X) be a function of random variable X. The mathemat-
The following three functional forms of g(X) are important.
ical expectation of g(X), denoted by E(g(X)), is defined as
follows:

86 87

1. g(X) = X. 2. g(X) = (X − μ)2 .

The expectation of X, E(X), is known as mean (ฏ‫)ۉ‬ The expectation of (X − μ)2 is known as variance (෼
of random variable X. ࢄ) of random variable X, which is denoted by V(X).
⎧




⎪ xi f (xi ), (Discrete RV),


⎨ i V(X) = E((X − μ)2 )
E(X) = ⎪ ⎧

⎪  ∞ ⎪



⎪ ⎪
⎪ (xi − μ)2 f (xi ), (Discrete RV),


⎩ x f (x) dx, (Continuous RV), ⎪

⎨ i
−∞ =⎪⎪  ∞
= μ, (or μ x ). ⎪




⎩ (x − μ)2 f (x) dx, (Continuous RV),
−∞

When a distribution of X is symmetric, mean indicates = σ2 , (or σ2x ).


the center of the distribution.

88 89

If X is broadly distributed, σ2 = V(X) becomes large. 3. g(X) = eθX .


Conversely, if the distribution is concentrated on the The expectation of eθX is called the moment-generating

center, σ2 becomes small. Note that σ = V(X) is function (ੵ཰฼ؔ਺), which is denoted by φ(θ).
called the standard deviation (ඪ४ภࠩ).
φ(θ) = E(eθX )
⎧




⎪ eθxi f (xi ), (Discrete RV),


⎨ i
=⎪⎪  ∞





⎩ eθx f (x) dx, (Continuous RV).
−∞

90 91

Some Formulas of Mean and Variance: Note that we have i xi f (xi ) = E(X) from the defini-

tion of mean and i f (xi ) = 1 because f (xi ) is a prob-
1. Theorem: E(aX + b) = aE(X) + b, where a and b are
ability function.
constant.
If X is a continuous random variable,
Proof:
 ∞
When X is a discrete random variable, E(aX + b) = (ax + b) f (x) dx
−∞
  ∞  ∞
E(aX + b) = (axi + b) f (xi ) =a x f (x) dx + b f (x) dx
i −∞ −∞
 
=a xi f (xi ) + b f (xi ) = aE(X) + b.
i i ∞
= aE(X) + b. Similarly, note that we have −∞
x f (x) dx = E(X) from

92 93


the definition of mean and −∞
f (x) dx = 1 because 3. Theorem: V(aX + b) = a2 V(X), where a and b are
f (x) is a probability density function. constant.

2. Theorem: V(X) = E(X 2 ) − μ2 , where μ = E(X). Proof:

Proof: From the definition of the mathematical expectation,


V(aX + b) is represented as:
V(X) is rewritten as follows:


V(X) = E((X − μ)2 ) = E(X 2 − 2μX − μ2 ) V(aX + b) = E ((aX + b) − E(aX + b))2


= E(X 2 ) − 2μE(X) + μ2 = E(X 2 ) − μ2 . = E (aX − aμ)2 = E(a2 (X − μ)2 )

= a2 E((X − μ)2 ) = a2 V(X)


The first equality is due to the definition of variance.

94 95

The first and the fifth equalities are from the definition Proof:
of variance. We use E(aX + b) = aμ + b in the second E(X) and V(X) are obtained as:
equality.
X − μ E(X) − μ
E(Z) = E = = 0,
σ σ
4. Theorem: The random variable X is assumed to be

1 μ 1
distributed with mean E(X) = μ and variance V(X) = V(Z) = V X− = 2 V(X) = 1.
X−μ σ σ σ
σ2 . Define Z = . Then, we have E(Z) = 0 and
σ The transformation from X to Z is known as normal-
V(Z) = 1.
ization (ਖ਼‫ن‬Խ) or standardization (ඪ४Խ).

96 97
Example 1.5: In Example 1.2 of flipping a coin three times First, μ = E(X) is computed as:
(Section 1.1), we see in Section 2.1 that the probability func- 
n 
n 
n
n!
μ = E(X) = x f (x) = x f (x) = x p x (1 − p)n−x
tion is written as the following binomial distribution: x=0 x=1 x=1
x!(n − x)!
n! 
n
n!
P(X = x) = f (x) = p x (1 − p)n−x , = p x (1 − p)n−x
x!(n − x)! x=1
(x − 1)!(n − x)!
n
(n − 1)!
for x = 0, 1, 2, · · · , n, = np p x−1 (1 − p)n−x
x=1
(x − 1)!(n − x)!

where n = 3 and p = 1/2. 


n
n !   
= np p x (1 − p)n −x = np,
x !(n − x )!

When X has the binomial distribution above, we obtain E(X), x =0

V(X) and φ(θ) as follows. where n = n − 1 and x = x − 1 are set.

98 99

Second, in order to obtain σ2 = V(X), we rewrite V(X) as: 


n
(n − 2)!
= n(n − 1)p2 p x−2 (1 − p)n−x
x=2
(x − 2)!(n − x)!
σ2 = V(X) = E(X 2 ) − μ2 = E(X(X − 1)) + μ − μ2 . 
n
n !   
= n(n − 1)p2 p x (1 − p)n −x
x =0
x !(n − x )!
E(X(X − 1)) is given by:
= n(n − 1)p , 2


n 
n
E(X(X − 1)) = x(x − 1) f (x) = x(x − 1) f (x)
x=0 x=2
where n = n − 2 and x = x − 2 are re-defined.
n
n!
= x(x − 1) p x (1 − p)n−x
x=2
x!(n − x)!

n
n!
= p x (1 − p)n−x
x=2
(x − 2)!(n − x)!

100 101

Therefore, σ2 = V(X) is obtained as: Finally, the moment-generating function φ(θ) is represented
as:
σ2 = V(X) = E(X(X − 1)) + μ − μ2

n
n!
= n(n − 1)p + np − n p = −np + np = np(1 − p).
2 2 2 2 φ(θ) = E(eθX ) = eθx p x (1 − p)n−p
x=0
x!(n − x)!

n
n!
= (peθ ) x (1 − p)n−p = (peθ + 1 − p)n .
x=0
x!(n − x)!

In the last equality, we utilize the following formula:



n
n!
(a + b)n = a x bn−x ,
x=0
x!(n − x)!

which is called the binomial theorem.

102 103
Example 1.6: As an example of continuous random vari-
ables, in Section 2.1 the uniform distribution is introduced,  ∞
σ = V(X) = E(X ) − μ =
2 2
x2 f (x) dx − μ2
2
which is given by: −∞
⎧  1 1 1
1 2
⎪ 1


⎨ 1, for 0 < x < 1, = x2 dx − μ2 = x3 − = ,
f (x) = ⎪ 3 0 2 12


⎩ 0,
0
otherwise.
When X has the uniform distribution above, E(X), V(X) and  
∞ 1
θX θx
φ(θ) are computed as follows: φ(θ) = E(e ) = e f (x) dx = eθx dx
−∞ 0
1 1
  = [ eθx ]10 = (eθ − 1).
∞ 1 1 1 1 θ θ
μ = E(X) = x f (x) dx = x dx = x2 = ,
−∞ 0 2 0 2

104 105

Example 1.7: As another example of continuous random E(X) is obtained as:


variables, we take the standard normal distribution:  ∞  ∞
1
xe− 2 x dx
1 2
E(X) = x f (x) dx = √
1 −∞ 2π −∞
f (x) = √ e− 2 x ,
1 2
for −∞ < x < ∞ 1 − 12 x2 ∞
2π = √ −e = 0,
−∞

When X has a standard normal distribution, i.e., when X ∼
because lim −e− 2 x = 0.
1 2

N(0, 1), E(X), V(X) and φ(θ) are as follows. x→±∞

106 107

V(X) is computed as follows: The first equality holds because of E(X) = 0.


 ∞ In the fifth equality, use the following integration formula,
V(X) = E(X 2 ) = x2 f (x) dx
 ∞ −∞ called the integration by parts:
1  b
x2 √ e− 2 x dx b  b
1 2
=
−∞ 2π h(x)g (x) dx = h(x)g(x) − h (x)g(x) dx,
 ∞ a
a
a
d(−e− 2 x )
1 2
1
= √ x dx
where we take h(x) = x and g(x) = −e− 2 x in this case.
1 2
2π −∞ dx
 ∞
1 − 12 x2
∞ 1
e− 2 x dx
1 2
In the sixth equality, lim −xe− 2 x = 0 is utilized.
1 2
= √ x(−e ) + √
−∞ x→±∞
 2π

2π −∞
The last equality is because the integration of the standard
1 − 12 x2
= √ e dx = 1.
−∞ 2π normal probability density function is equal to one.

108 109
φ(θ) is derived as follows: Example 1.8: When the moment-generating function of X
 ∞  ∞ is given by φ x (θ) = e 2 θ (i.e., X has a standard normal distri-
1 2

θx 1
eθx √ e− 2 x dx
1 2
φ(θ) = e f (x) dx =
bution), we want to obtain the moment-generating function
−∞ ∞
−∞
 ∞2π
1 − 12 x2 +θx 1
√ e− 2 ((x−θ) −θ ) dx of Y = μ + σX.
1 2 2
= √ e dx =
−∞
 ∞2π −∞ 2π
1 Let φ x (θ) and φy (θ) be the moment-generating functions of X
θ
1 2
− 1 2 θ
1 2
= e2 √ e 2 (x−θ)
dx = e 2 .
−∞ 2π and Y, respectively. Then, the moment-generating function

The last equality holds because the integration indicates the of Y is obtained as follows:

normal density with mean θ and variance one.


φy (θ) = E(eθY ) = E(eθ(μ+σX) ) = eθμ E(eθσX ) = eθμ φ x (θσ)
1
= eθμ e 2 σ θ = exp(μθ + σ2 θ2 ).
1 2 2

2
110 111

Example 1.8(b): When X ∼ N(μ, σ2 ), what is the moment- 3.2 Bivariate Random Variable
generating function of X?
Definition: Let g(X, Y) be a function of random variables
 ∞
φ(θ) = eθx f (x) dx X and Y. The mathematical expectation of g(X, Y), denoted
−∞∞
1 by E(g(X, Y)), is defined as:
1
= eθx √ exp − 2 (x − μ)2 dx ⎧
2πσ 2 2σ ⎪


−∞∞



g(xi , y j ) f (xi , y j ), (Discrete),
1 1 ⎪
⎨ i j
= √ exp θx − (x − μ)2
dx E(g(X, Y)) = ⎪⎪  ∞ ∞
−∞ 2πσ2 2σ2 ⎪


 ∞
1 ⎪

⎩ g(x, y) f (x, y) dx dy, (Continuous).
1 1
= exp(μθ + σ2 θ2 ) √ exp 2
(x − μ − σ2 θ)2 dx −∞ −∞
2 −∞ 2πσ2 2σ
The following four functional forms are important, i.e., mean,
1 2 2
= exp(μθ + σ θ ). variance, covariance and the moment-generating function.
2

112 113

1. g(X, Y) = X: 2. g(X, Y) = (X − μ x )2 :

The expectation of random variable X, i.e., E(X), is The expectation of (X − μ x )2 is known as variance of X.
given by:
V(X) = E((X − μ x )2 )
⎧ ⎧

⎪ ⎪




⎪ xi f (xi , y j ), (Discrete), ⎪
⎪ (xi − μ x )2 f (xi , y j ), (Discrete)

⎪ ⎪

⎨ i j
⎨ i j =⎪
E(X) = ⎪
⎪  ∞ ∞ ⎪
⎪  ∞ ∞


⎪ ⎪


⎪ x f (x, y) dx dy, ⎪

⎩ (x − μ x )2 f (x, y) dx dy, (Continuous)
⎩ (Continuous),
−∞ −∞ −∞ −∞
= μx . = σ2x .

The case of g(X, Y) = Y is exactly the same formula- The variance of Y is also obtained in the same way, i.e.,
tion as above, i.e., E(Y) = μy . V(Y) = σ2y .

114 115
3. g(X, Y) = (X − μ x )(Y − μ y ):

The expectation of (X − μ x )(Y − μy ) is known as co-


Cov(X, Y) = E((X − μ x )(Y − μy ))
variance of X and Y, which is denoted by Cov(X, Y) ⎧


⎪ (xi − μ x )(y j − μy ) f (xi , y j ),



and written as: ⎪

⎪ i j


⎨ (Discrete),
=⎪⎪  ∞ ∞





⎪ (x − μ x )(y − μy ) f (x, y) dx dy,



⎩ −∞ −∞ (Continuous).

Thus, covariance is defined in the case of bivariate ran-


dom variables.

116 117

4. g(X, Y) = eθ1 X+θ2 Y :

The mathematical expectation of eθ1 X+θ2 Y is called the


moment-generating function, which is denoted by:

φ(θ1 , θ2 ) = E(eθ1 X+θ2 Y )


⎧




⎪ eθ1 xi +θ2 y j f (xi , y j ), (Discrete)


⎨ i j
=⎪⎪  ∞ ∞





⎩ eθ1 x+θ2 y f (x, y) dx dy, (Continuous)
−∞ −∞

In Section 5, the moment-generating function in the


multivariate cases is discussed in more detail.

118

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy