Chapter 4: Random Variable, Expectation, and

Variance

Li-Pang Chen, PhD

Department of Statistics, National Chengchi University

©Fall 2024

1 / 71
Outlines

1 Random Variables

2 Cumulative Distribution Function

3 Expectation

4 Variance

5 Jointly Distributed Random Variables and Their Mean and Variance

2 / 71
4.1 Random Variables

Definition (Random variables)

A variable measured or observed as the result of an experiment. By
chance, the variable can have different values.

Simply speaking, it indicates that the outcomes occur randomly in an

experiment of chance.

3 / 71
4.1 Random Variables

Examples:
(1) The number of students absent in class, which can be 0, 1, 2, · · · .
⇒ The number of absence is a random variable.
(2) Quiz scores in statistical course.
(3) Heights of male/female students.
From examples above, we find that random variables can be discrete
(outcome ∈ N) or continuous (outcome ∈ R).

4 / 71
4.1 Random Variables

Here we give formal definitions of discrete/continuous random variables.

Definition
(a) Discrete random variable: a random variable that can assume only
certain separated values.
(b) Continuous random variable: a random variable that assumes an
infinite number of values within a given range.

5 / 71
4.1 Random Variables

Mathematically, we usually denote capital letters, such as X , as

random variables; little letters, such as x, represent real values of
possible outcomes.
When a random variable X is defined, we may wonder what the
probability of specific outcome is.

6 / 71
4.1 Random Variables

Notation:
If X is a discrete random variable, then the probability mass function
(pmf) is defined as

p(x) ≜ P(X = x)

for all possible outcomes x. That is,

x x1 x2 x3 ··· xn
p(x) p(x1 ) p(x2 ) p(x3 ) ··· p(xn )
Properties of pmf: Suppose x1 , x2 , · · · , xn are outcomes. Then a pmf
satisfies
(a) P(X = xi ) > 0 for i = 1, 2, · · · .
(b) P(X = a) = 0 if a is NOT outcome of X .
Pn
(c) P(X = xi ) = 1.
i=1

7 / 71
4.1 Random Variables

Example: Let X be a random variable of tossing a fair die.

Outcome: 1, 2, · · · , 6.
A fair die: P(X = i) = 16 for i = 1, · · · , 6.
Moreover, P(X = a) = 0 if a ̸∈ {1, · · · , 6}.

8 / 71
4.1 Random Variables
Example: Let X be a random variable that is defined as the sum of two
fair dice. Find the pmf of X .

9 / 71
4.1 Random Variables
Example: Find the value of k which makes P(X = x) below a pmf.
x 0 1 2 3
P(X = x) k 2k 0.3 4k

10 / 71
4.1 Random Variables

On the other hand, if X is a continuous random variable, it is

impossible to specify a specific value because the outcome x ∈ R.
We say f (x) is the probability density function (pdf) if f (x) is a
nonnegative function for a continuous random variable X defined in
x ∈ Ω ⊆ (−∞, ∞). Here Ω is called support, which is regarded as the
domain of f (x).
Moreover, the probability of a set B ⊆ Ω for X is defined as
Z
P(X ∈ B) = f (x)dx. (1)
B

In particular, if B = [a, b] is an interval with a < b, then (1) becomes

Z b
P(a < X < b) = f (x)dx.
a

11 / 71
4.1 Random Variables

Aside 1: Review of integral of commonly used functions.

Let C be an arbitrary constant.
1
(a) x r dx = r +1 x r +1 + C for all r ≥ 0.
R
R 1
(b) x dx = ln |x| + C .
(c) x1r dx = −r1+1 x −r +1 + C for all r > 1.
R

(d) e αx dx = α1 e αx + C for all α ̸= 0.

R

(e) xe αx dx = α1 xe αx − α12 e αx + C for all α ̸= 0.

R

12 / 71
4.1 Random Variables

Aside 2: Some properties of integral.

R R
(a) kf (x)dx = k f (x)dx for k ∈ R.
R R R
(b) f (x)dx ± g (x)dx = f (x) ± g (x)dx.
Rb Rc Rb
(c) a f (x)dx = a f (x)dx + c f (x)dx for some constant c ∈ [a, b].
R
(d) Suppose F (x) is a function such that F (x) = f (x)dx. Then
Rb
a f (x)dx = F (b) − F (a).
(e) If f (x) and g (x) are continuous functions, and f (x) < g (x) for
Rb Rb
x ∈ [a, b], then a f (x)dx < a g (x)dx.
Rb Ra
(f) a f (x)dx = − b f (x)dx.

13 / 71
4.1 Random Variables

Properties of pdf: Suppose B = [a, b] with a < b, and let Ω denote the
support of X . Then
(a) f (x) ≥ 0 for all x ∈ Ω.
(b) P(X ∈ B) = P(a < X < b) > 0.
Ra
(c) P(a < X < a) = a f (x)dx = 0.
(d) P(X ∈ Ω) = 1 and P(X ∈ Ωc ) = 0.

14 / 71
4.1 Random Variables

Example: Consider a continuous random variable X whose pdf is given by

K (4x − 2x 2 ) 0 < x < 2


f (x) =
0 otherwise

(a) What is the value K ?

(b) Calculate P(X > 1).

15 / 71
4.1 Random Variables

Example: Let X be a continuous random variable with pdf

kx 2

 0<x ≤1
f (x) = k(2 − x) 1 < x ≤ 2
0 otherwise


(a) Find the value k.

(b) Calculate P(0.5 < X < 1.5).

16 / 71
4.2 Cumulative Distribution Function

Recall: In Section 2.2, we discussed the number of subjects less than

a specific point.
Now we consider the probability that outcomes are less than a specific
value.
For example,
(1) value of tossing a die is less than 5.
(b) students’ statistics midterm grades are less than 90.

17 / 71
4.2 Cumulative Distribution Function

Let X be a random variable. Then the interested probability can be

defined as

F (x) ≜ P(X ≤ x).

We call F (x) as the cumulative distribution function (CDF).

18 / 71
4.2 Cumulative Distribution Function

Formula: Let X be a random variable, then its CDF is defined as

Type P Discrete RContinuous

x
F (x) P(X = xi ) −∞ f (u)du.
i:xi ≤x<xi+1

19 / 71
4.2 Cumulative Distribution Function

Properties of CDF:
(a) F (−∞) = 0 and F (∞) = 1.
(b) F (x) is an increasing function.
(c) F (b) − F (a) = P(a < X ≤ b).

20 / 71
4.2 Cumulative Distribution Function
Example: Suppose that X is a discrete random variable with pmf
P(X = 1) = 12 , P(X = 2) = 31 , P(X = 3) = 16 .
Find F (x) and draw a plot of CDF.

21 / 71
4.2 Cumulative Distribution Function
Example: The range of random variable X is Ω = {1, 2, 3, 4, 5}. For
x ∈ Ω, the CDF for X is given by
x 1 2 3 4 5
F (x) 0.1k 0.2 0.5k k 4k 2
(a) Find k and P(2 < X ≤ 4).
(b) Find the pmf.

22 / 71
4.2 Cumulative Distribution Function
Example: Continue an early example
K (4x − 2x 2 ) 0 < x < 2

f (x) =
0 otherwise
Find F (x) and calculate P(X > 1).

23 / 71
4.3 Expectation

Definition (Expectation)
Let X be a random variable. The expectation or expected value of X is
denoted as µ ≜ E (X ).
In particular,
(a) if X is a discrete random variable with possible outcomes x1 , · · · , xn ,
then
n
X
µ= xi P(X = xi );
i=1

(b) if X is a continuous random variable under the support (−∞, ∞),

then
Z ∞
µ= xf (x)dx.
−∞

24 / 71
4.3 Expectation

Interpretation: µ is a weighted average of the possible values that X

can take on, each value being weighted by the probability that X
assume it.
Note: µ is not a value that X could possibly assume, so it should not
be interpreted as the value that we expect X to have but rather as the
average value of X in a large number of repetitions of the experiment.

25 / 71
4.3 Expectation
Example: Calculate µ:
(a) X isthe random variable of tossing a fair die.
1 if A occurs
(b) I = is called an indicator function, where
0 if A does not occur
A is an event.
(c) X is a continuous random variable with pdf
 1
f (x) = 1.5 if 0 < x < 1.5
0 otherwise

26 / 71
4.3 Expectation

Properties of the expectation:

(A) The expected value of some function g (x).

Definition
(a) if X is a discrete random variable with pmf p(X = x), then
X
E {g (X )} = g (x)P(X = x);
x

(b) if X is a continuous random variable with pdf f (x), then

Z ∞
E {g (X )} = g (x)f (x)dx.
−∞

27 / 71
4.3 Expectation

The expected value of some functions g (x).

Interpretation: It can be regarded as a weighted average of the possible
values g (x) with the weight given to g (x) being equal to the
probability.
Special case: if g (x) = x, then we have µ.

28 / 71
4.3 Expectation

(B) Expectation of linear transformation.

Suppose that X is a random variable. Let a, b denote nonzero real
numbers.
Q: What is E (aX ± b)?

29 / 71
4.3 Expectation

Example: Suppose that X has the following pmf

P(X = 0) = 0.2, P(X = 1) = 0.5, P(X = 2) = 0.3.

Calculate E (X 2 ).

30 / 71
4.3 Expectation
Example: The time, in hours, it takes to locate and repair an electrical
breakdown in a certain factory is a random variable, call X , whose pdf is
given by

1 if 0 < x < 1
f (x) =
0 otherwise
If the cost involved in a breakdown of duration x is x 3 , what is the
expected cost of such a breakdown?

31 / 71
4.4 Variance

In Section 4.3, E (X ) only shows the weighted average of the possible

values of X , but it does not tell us anything about the variation or
spread of these values.
Recall: variance is a measure to describe the variation. In addition, in
Section 2.3.2, we defined variance as
N
1 X
σ2 = (Xi − µ)2 .
N
i=1

⇒ Measuring the possible variation of X shows how far apart X

would be from its mean.
Inspired by this idea, the variance of a random variable X can be
defined.

32 / 71
4.4 Variance

Definition
If X is a random variable with mean µ = E (X ), then the variance of X ,
denoted by var(X ), is defined as

var(X ) = E (X − µ)2 .


p
Moreover, var(X ) is called standard deviation of X .

A convenient expression of var(X ) is given by

var(X ) = E (X 2 ) − {E (X )}2 .

33 / 71
4.4 Variance

Remark: Here E (X − µ)2 and E (X 2 ) can be computed by property



(A) with a function g (X ) specified as g (X ) = E (X − µ)2 or

g (X ) = E (X 2 ).

34 / 71
4.4 Variance

Example: Let X denote the random variable of tossing a fair die.

Calculate var(X ).

35 / 71
4.4 Variance

1 if A occurs
Example: Recall that I = . Calculate var(X ).
0 otherwise

36 / 71
4.4 Variance
Example: X is a continuous random variable with pdf
 1
f (x) = 1.5 if 0 < x < 1.5
0 otherwise
Find var(X ).

37 / 71
4.4 Variance
Property of var(X ):
Suppose that X is a random variable and a, b are nonzero constants.
Q: What is var(aX ± b)?

38 / 71
4.5 Jointly Distributed Random Variables

In the preceding sections, we discussed the univariate random variable

X and its computation of mean and variance.
In this section, we particularly discuss two random variables, say X
and Y , as well as their properties.

39 / 71
4.5 Jointly Distributed Random Variables

Definition
When X and Y are both discrete random variable with possible values
x1 , x2 , · · · , and y1 , y2 , · · · ,, respectively, then the joint probability mass
function of X and Y at X = xi and Y = yj is defined as

P(xi , yj ) = P(X = xi , Y = yj ) (2)

∀i = 1, · · · , j = 1, · · · .

40 / 71
4.5 Jointly Distributed Random Variables

Definition (Marginal distribution)

If X and Y are discrete random variables and joint pmf is (2), then the
∞
P
marginal pmf of X = xi is pX (xi ) = p(xi , yj ); the marginal pmf of
j=1
∞
P
Y = yj is pY (yj ) = p(xi , yj ).
i=1

41 / 71
4.5 Jointly Distributed Random Variables
Example: Suppose that 3 batteries are randomly chosen from a group of
3 new, 4 used but still working, and 5 defective batteries. If we let X and
Y denote, respectively, the number of new and used but still working
batteries that are chosen, then find the joint probability mass function of
X and Y .

42 / 71
4.5 Jointly Distributed Random Variables
Example: Suppose that 15 percent of the families in a certain community
have no children, 20 percent have one, 35 percent have two, and 30
percent have three children; suppose further that each child is equally
likely (and independently) to be a boy or girl. Let B be the number of
boys and let G denote the number of girls. If a family is chosen at random
from this community, then find the joint pmf of B and G .

43 / 71
4.5 Jointly Distributed Random Variables

Recall: if X is a continuous random variable, then we usually consider

an interval/range.
In Section 4.1, we consider the probability of X for an interval [a, b],
i.e., P(a < X < b).
Naturally, it can be extended to bivariate case.
Suppose X and Y are continuous random variables, then in this case,
we have two-dimensional plane.

44 / 71
4.5 Jointly Distributed Random Variables

Definition
Let X and Y denote continuous random variables, and f : R2 → R+ is a
bivariate continuous function. Suppose that C is a set in two-dimensional
plane, then the probability of (X , Y ) in a set C is defined as
ZZ


P (X , Y ) ∈ C = f (x, y )dydx. (3)
(x,y )∈C

In particular, if C = {(x, y ) : x ∈ [a, b], y ∈ [c, d]}, then (3) becomes

P (X , Y ) ∈ C = P X ∈ [a, b], Y ∈ [c, d]
Z bZ d
= f (x, y )dydx.
a c

45 / 71
4.5 Jointly Distributed Random Variables

Similar to discrete random variables, we can define marginal

probability by “integrating out” other random variables (i.e., summing
up other random variables in discrete version).
That is, suppose that the support of f (x, y ) is R2 , then the marginal
density function of X and Y are, respectively, given by
Z ∞ Z ∞
fX (x) = f (x, y )dy and fY (y ) = f (x, y )dx. (4)
−∞ −∞

Note: In (4), if we integrate a function with respect to x (or y ), then

we treat y (or x) as a constant.

46 / 71
4.5 Jointly Distributed Random Variables

Moreover, the marginal probability of X ∈ [a, b] and Y ∈ [c, d] are,

respectively, given by
Z b Z b Z ∞ 
P(X ∈ [a, b]) = fX (x)dx = f (x, y )dy dx
a a −∞

and
Z d Z d Z ∞ 
P(Y ∈ [c, d]) = fY (y )dy = f (x, y )dx dy
c c −∞

47 / 71
4.5 Jointly Distributed Random Variables
Example: The joint density function of X and Y is given by
 −x−2y
2e 0 < x < ∞, 0 < y < ∞
f (x, y ) =
0 otherwise
Compute (a) P(X > 1, Y < 1) and (b) P(X < a) for some constant
a > 0.

48 / 71
4.5 Jointly Distributed Random Variables

Independence:

Recall: in Section 3.4, we said two events E and F are independent if

P(E ∩ F ) = P(E )P(F ).
Similar idea can be applied to two random variables. If X and Y are
independent, then joint pmf/pdf of X and Y can be decomposed as
the product of pmf/pdf of X and Y .

49 / 71
4.5 Jointly Distributed Random Variables

Here we state the formal definition.

Definition
The random variables X and Y are said to be independent if for any two
sets of real numbers A and B,

P(X ∈ A, Y ∈ B) = P(X ∈ A)P(Y ∈ B).

50 / 71
4.5 Jointly Distributed Random Variables

When X and Y are continuous, then we take A and B as intervals,

and f (x, y ) = fX (x)fY (y ).
When X and Y are discrete, then taking A = {a} and B = {b}, we
have p(x, y ) = pX (x)pY (y ) for all x and y .
In general, suppose that there are n independent random variables
X1 , · · · , Xn , then for all sets of real numbers A1 , · · · , An , we have
n
Y
P(X1 ∈ A1 , · · · , Xn ∈ An ) = P(Xi ∈ Ai ).
i=1

51 / 71
4.5 Jointly Distributed Random Variables

Example: Continue two examples in pages 42 and 43 and check the

independence of two random variables.

52 / 71
4.5 Jointly Distributed Random Variables
Example: Continue an early example that X and Y are two random
variables whose joint density function is given by
 −x−2y
2e 0 < x < ∞, 0 < y < ∞
f (x, y ) =
0 otherwise
Check the independence of X and Y .

53 / 71
4.5 Jointly Distributed Random Variables

Conditional probability:

Recall: Again, in Section 3.4, we defined the conditional probability:

∩F )
for two events E and F , we have P(E |F ) = P(EP(F ) , provided that
P(F ) > 0.
Now, suppose that X and Y are two random variables, how can we
define “conditional probability”?

54 / 71
4.5 Jointly Distributed Random Variables

If X and Y are discrete random variables, then the conditional pmf of

X given Y = y is defined as

pX |Y (x|y ) ≜ P(X = x|Y = y )

P(X = x, Y = y )
=
P(Y = y )
p(x, y )
= .
pY (y )

55 / 71
4.5 Jointly Distributed Random Variables

If X and Y are continuous random variables, then the conditional pdf

of X given Y = y is defined as

f (x, y )
fX |Y (x|y ) ≜ .
fY (y )

56 / 71
4.5 Jointly Distributed Random Variables

Example: Continue an example in page 43 and find the conditional pmf of

B given G = 1.

57 / 71
4.5 Jointly Distributed Random Variables
Example: Suppose that p(x, y ), the joint pmf of X and Y , is given by
p(0, 0) = 0.4, p(0, 1) = 0.2, p(1, 0) = 0.1, p(1, 1) = 0.3.
Calculate the conditional pmf of X given Y = 1

58 / 71
4.5 Jointly Distributed Random Variables
Example: The joint pdf of X and Y is given by
 12
f (x, y ) = 5 x(2 − x − y ) 0 < x < 1, 0 < y < 1
0 otherwise
Compute the conditional pdf of X given Y = y for y ∈ (0, 1).

59 / 71
4.5 Jointly Distributed Random Variables

Expectation:

Suppose that X and Y are two random variables with joint pmf
p(x, y ) or pdf f (x, y ).
Let g (x, y ) denote an arbitrary function of X and Y . Then the
expected value of g (x, y ) is defined as
PP
E {g (X , Y )} = g (x, y )p(x, y ) (discrete version).
Rx∞ y R ∞
E {g (X , Y )} = −∞ −∞
g (x, y )f (x, y )dydx (continuous version).

60 / 71
4.5 Jointly Distributed Random Variables
In particular, we consider g (x, y ) = x + y . Then we have
E {g (X , Y )} = E (X + Y ) = E (X ) + E (Y ).
⇒ expectation has the property of additivity.
Justification by continuous random variables:

61 / 71
4.5 Jointly Distributed Random Variables

How about E (X − Y )?
In general, the additivity still holds for n (n > 2) random variables.
That is, for X1 , · · · , Xn , we have
n
X
E (X1 + X2 + · · · + Xn ) = E (Xi ).
i=1

62 / 71
4.5 Jointly Distributed Random Variables
Example: A construction firm has recently sent in bids for 3 jobs worth
(in profits) 10, 20, and 40 (thousands) dollars. If its probabilities of
winning the job are respectively 0.2, 0.8, and 0.3, what is the firm’s
expected total profit?

63 / 71
4.5 Jointly Distributed Random Variables
Example: A secretary has typed N letters along with their respective
envelopes. The envelopes get mixed up when they fall on the floor. If the
letters are placed in the mixed-up envelopes in a completely random
manner (that is, each letter is equally likely to end up in any of the
envelopes), what is the expected number of letters that are placed in the
correct envelopes?

64 / 71
4.5 Jointly Distributed Random Variables

Variance and Covariance:

In Section 4.4, we defined variance of a random variable.

When two random variables are available, similar to Section 2.5 in
Chapter 2, we can discuss covariance of two random variables.

65 / 71
4.5 Jointly Distributed Random Variables

Definition (covariance)
Let X and Y be two random variables. Then the covariance of X and Y is
defined as

cov(X , Y ) = E {(X − µX )(Y − µY )} ,

where µX = E (X ) and µY = E (Y ).

Remark: Similar to the discussion of variance, cov(X , Y ) has a simpler

formulation:

cov(X , Y ) = E (XY ) − E (X )E (Y ).

66 / 71
4.5 Jointly Distributed Random Variables

Properties:
(a) cov(X , Y ) = cov(Y , X ).
(b) cov(X , X ) = var(X ).
(c) cov(aX , Y ) = acov(X , Y ) for a ̸= 0.
(d) for three random variables X , Y , Z ,
cov(X + Z , Y ) = cov(X , Y ) + cov(Z , Y ).
Pn m
P  P n Pm
(e) In general, cov Xi , Yj = cov(Xi , Yj ).
i=1 j=1 i=1 j=1

67 / 71
4.5 Jointly Distributed Random Variables

Similarto the expectation, we can consider the variance of g (X , Y ),

say var g (X , Y ) .
In particular, we mainly study g (X , Y ) = X + Y , and thus,
var(X + Y ).
Q: The expectation has the additivity property, does the variance
have this property as well, say var(X + Y ) = var(X ) + var(Y )?
A: NO! In general, we have

var(X + Y ) = var(X ) + var(Y )+2cov(X , Y ).

Exercise: How about var(X − Y )?

68 / 71
4.5 Jointly Distributed Random Variables

Theorem
If X and Y are independent random variables, then cov(X , Y ) = 0, and
thus, var(X + Y ) = var(X ) + var(Y ).

69 / 71
4.5 Jointly Distributed Random Variables

Example: Compute the variance of the sum obtained when 10

independent rolls of a fair die are made.

70 / 71
4.5 Jointly Distributed Random Variables

Example: Compute the variance of the number of heads resulting from 10

independent tosses of a fair coin.

71 / 71