Discrete Random Variables
Discrete Random Variables
Example 5.1. If one rolls a die, let X denote the outcome, i.e. taking values 1, 2, 3, 4, 5, 6.
Example 5.2. If one rolls a die, let Y be 1 if an odd number is showing, and 0 if an even
number is showing.
Example 5.3. If one tosses 10 coins, let X be the number of heads showing.
P(X = x) = P X −1 (x) .
X
pX (xi ) = P(S) = 1.
i∈N
Let X be the number showing if we roll a die. The expected number to show up on a roll of
a die should be 1 · P(X = 1) + 2 · P(X = 2) + · · · + 6 · P(X = 6) = 3.5. More generally, we
dene
59
60 5. DISCRETE RANDOM VARIABLES
We need absolute convergence of the sum so that the expectation does not depend on
the order in which we take the sum to dene it. We know from calculus that we need
to be careful about the sums of conditionally convergent series, though in most of the
examples we deal with this will not be a problem. Note that pX (x) is nonnegative for
all x, but x itself can be negative or positive, so in general the terms in the sum might
have dierent signs.
Example 5.5. If we toss a coin and X is 1 if we have heads and 0 if we have tails, what
is the expectation of X?
Solution :
1
2, x = 1
pX (x) = 12 , x = 0
0, all other values of x.
Hence EX = (1)( 12 ) + (0)( 21 ) = 1
2
.
This turns out to sum to 1. To see this, recall the formula for a geometric series
1
1 + x + x2 + x3 + · · · = .
1−x
If we dierentiate this, we get
1
1 + 2x + 3x2 + · · · = .
(1 − x)2
5.1. DEFINITION, PROPERTIES, EXPECTATION, MOMENTS 61
We have
EX = 1( 14 ) + 2( 81 ) + 3( 16
1
) + ···
h i
1 1 1
= 4 1 + 2( 2 ) + 3( 4 ) + · · ·
1 1
= = 1.
4
(1 − 12 )2
Example 5.8. Consider a discrete random variable taking only positive integers as values
1
with P (X = n) = n(n+1)
. What is the expectation EX ?
Solution : First observe that this is indeed a probability since we can use telescoping partial
sums to show that
∞
X 1
= 1.
n=1
n(n + 1)
Then
∞ ∞ ∞
X 1
X X 1
EX = n · P (X = n) = n· = = +∞,
n=1 n=1
n(n + 1) n=1 n + 1
so the expectation of X is innite.
If we list all possible values of a discrete random variable X as {xi }i∈N , then we can write
X ∞
X
EX = xpX (x) = xi pX (xi ) .
{x:pX (x)>0} i=1
Proposition 5.1
If X is a random variable on a nite sample space S, then
X
EX = X(ω) P ({ω}) .
ω∈S
62 5. DISCRETE RANDOM VARIABLES
Proof. For each i ∈ N we denote by Si the event {ω ∈ S : X (ω) = xi }. Then {Si }i∈N
is a partition of the space S into disjoint sets. Note that since S is nite, then each set Si is
nite too, moreover, we only have a nite number of sets Si which are non-empty.
∞ ∞ ∞
!
X X X X
EX = xi p (xi ) = xi P (X = xi ) = xi P ({ω})
i=1 i=1 i=1 ω∈Si
∞
! ∞
X X X X
= xi P ({ω}) = X(ω)P ({ω})
i=1 ω∈Si i=1 ω∈Si
X
= X(ω)P ({ω}) ,
ω∈S
∞ ∞ ∞
!
X X X
EZ = zk P(Z = zk ) = zk P(Z = zk , X = xi )
k=1 k=1 i=1
∞ ∞
!
X X
= zk P(X = xi , Y = zk − xi )
k=1 i=1
∞
XXX ∞ ∞
= zk P(X = xi , Y = zk − xi , Y = yj ).
k=1 i=1 j=1
∞ X
X ∞
EZ = (xi + yj )P(X = xi , Y = yj )
i=1 j=1
∞
XX ∞ ∞ X
X ∞
xi P(X = xi , Y = yj ) + yj P(X = xi , Y = yj )
i=1 j=1 i=1 j=1
∞ ∞
! ∞ ∞
!
X X X X
= xi P(X = xi , Y = yj ) + yj P(X = xi , Y = yj )
i=1 j=1 j=1 i=1
X∞ ∞
X
= xi P(X = xi ) + yj P(Y = yj ) = EX + EY,
i=1 j=1
Note that if we have a countable sample space all these sums converge absolutely and so we
can justify writing this similarly to Proposition 5.1 as
X
E [X + Y ] = (X(ω) + Y (ω)) P (ω)
ω∈S
X
= (X(ω)P (ω) + Y (ω)P (ω))
ω∈S
X X
= X(ω)P (ω) + Y (ω)P (ω)
ω∈S ω∈S
= EX + EY.
64 5. DISCRETE RANDOM VARIABLES
Using induction on the number of random variables linearity holds for a collection of random
variables X1 , X2 , . . . , Xn .
Corollary
If X1 , X2 , . . . , Xn are random variables, then
Example 5.9. Suppose we roll a die and let X be the value that is showing. We want to
2
nd the expectation EX .
The only possibility where things could go wrong is if more than one value of X leads to
2 1 1
the same value of X . For example, suppose P(X = −2) = , P(X = −1) = , P(X = 1) =
8 4
3
8
, P(X = 2) = 14 . Then if Y = X 2 , P(Y = 1) = 58 and P(Y = 4) = 38 . Then
EX 2 = (1) 58 + (4) 83 = (−1)2 41 + (1)2 38 + (−2)2 18 + (2)2 14 .
2
P 2
But even in this case EX = x x P(X = x).
Theorem 5.2
For a discrete random variable X taking values {xi }∞
i=1 and a real-valued function g
dened on this set, we have
∞
X ∞
X
Eg(X) = g (xi ) P (X = xi ) = g (xi ) p (xi ) .
i=1 i=1
∞
X ∞
X
Eg(X) = cp (xi ) = c p (xi ) = c · 1 = c.
i=1 i=1
Denition (Moments)
EX n is called the nth moment of a random variable X. If M := EX is well dened,
then
Var (X) = E(X − M )2
is called the variance of X . The square root of Var (X) is called the standard deviation
of X p
SD (X) := Var(X).
X
EX n = xn pX (x).
x:pX (x)>0
The variance measures how much spread there is about the expected value.
Example 5.11. We toss a fair coin and let X = 1 if we get heads, X = −1 if we get tails.
Then EX = 0, so X − EX = X , and then Var X = EX 2 = (1)2 12 + (−1)2 21 = 1.
Example 5.12. We roll a die and let X be the value that shows. We have previously
7
calculated EX = . So X − EX equals
2
5 3 1 1 3 5
− ,− ,− , , , ,
2 2 2 2 2 2
1
each with probability . So
6
Var X = (− 25 )2 16 + (− 32 )2 61 + (− 12 )2 16 + ( 21 )2 16 + ( 23 )2 16 + ( 25 )2 16 = 35
12
.
Using the fact that the expectation of a constant is the constant we get an alternate expres-
sion for the variance.
Var X = EX 2 − (EX)2 .
66 5. DISCRETE RANDOM VARIABLES
(
1 if outcome is heads (H)
X=
0 if outcome is tails (T).
Example 5.14. Let X be the amount of liability (damages) a driver causes in a year. In
this case, X can be any dollar amount. Thus X can attain any value in [0, ∞).
Example 5.15. Toss a coin 3 times. Let X be the number of heads that appear, so that
X can take the values 0, 1, 2, 3. What are the associated probabilities to each value?
Solution :
1 1
P (X = 0) =P ((T, T, T )) = = ,
23 8
3
P (X = 1) =P ((T, T, H) , (T, H, T ) , (H, T, T )) = ,
8
3
P (X = 2) =P ((T, H, H) , (H, H, T ) , (H, T, H)) = ,
8
1
P (X = 3) =P ((H, H, H)) = .
8
Example 5.16. Toss a coin n times. Let X be the number of heads that occur. This
random variable can take the values 0, 1, 2, . . . , n. From the binomial formula we see that
1 n
P(X = k) = n .
2 k
© Copyright 2017 Phanuel Mariano, Patricia Alonso Ruiz, Copyright 2020 Masha Gordina.
68 5. DISCRETE RANDOM VARIABLES
Example 5.17. Suppose we toss a fair coin, and we let X be 1 if we have H and X be 0
if we have T. The probability mass function of this random variable is
1
2 x=0
1
pX (x) = x = 1,
2
0 otherwise .
Often the probability mass function (PMF) will already be given and we can then use it to
compute probabilities.
Example 5.18. The PMF of a random variable X taking values in N ∪ {0} is given by
λi
pX (i) = e−λ , i = 0, 1, 2, . . . ,
i!
where λ is a positive real number.
λ0
P (X = 0) = pX (0) = e−λ = e−λ .
0!
(b) Find P (X > 2).
Solution : note that
P (X > 2) = 1 − P (X 6 2)
= 1 − P (X = 0) − P (X = 1) − P (X = 2)
= 1 − pX (0) − pX (1) − pX (2)
−λ −λ λ2 e−λ
=1−e − λe − .
2
5.2.2. Expectation. We dened the expectation in Denition 5.2 in the case when X
is a discrete random variable X taking values {xi }i∈N . Then for a random variable X with
PMF pX (x) the expectation is given by
X ∞
X
E [X] = xpX (x) = xi pX (xi ) .
x:p( x)>0 i=1
Example 5.19. Suppose again that we have a coin, and let X(H) = 0 and X (T ) = 1.
What is EX if the coin is not necessarily fair?
Example 5.20. Let X be the outcome when we roll a fair die. What is EX ?
1 1 1 1 21 7
EX = 1 · + 2 · + · · · + 6 · = (1 + 2 + 3 + 4 + 5 + 6) = = = 3.5.
6 6 6 6 6 2
5.2. FURTHER EXAMPLES AND APPLICATIONS 69
Note that in the last example X can never be 3.5. This means that the expectation may not
be a value attained by X. It serves the purpose of giving an average value for X.
Example 5.21. Let X be the number of insurance claims a person makes in a year. Assume
2 2
that X can take the values 0, 1, 2, 3 . . . with P (X = 0) = , P (X = 1) = , . . . , P (X = n) =
3 9
2
. Find the expected number of claims this person makes in a year.
3n+1
Solution : Note that X has innite but countable number of values, hence it is a discrete
2
random variable. We have that pX (i) = 3i+1 . We compute using the denition of expectation,
Example 5.22. Let S = {1, 2, 3, 4, 5, 6} and assume that X(1) = X(2) = 1, X(3) =
X(4) = 3, and X(5) = X(6) = 5.
(1) Using the initial denition, the random variable X takes the values 1, 3, 5 and pX (1) =
pX (3) = pX (5) = 31 . Then
1 1 1 9
EX = 1 · + 3 + 5 = = 3.
3 3 3 3
(2) Using the equivalent denition, we list all of S = {1, 2, 3, 4, 5, 6} and then
1 1 1 1 1 1
EX = X(1)P ({1}) + · · · + X(6) · P ({6}) = 1 + 1 + 3 + 3 + 5 + 1 = 3.
6 6 6 6 6 6
5.2.3. The cumulative distribution function (CDF). We implicitly used this char-
acterization of a random variable, and now we dene it.
FX (x) := P (X 6 x) ,
for any x ∈ R.
70 5. DISCRETE RANDOM VARIABLES
1
pX (0) = P (X = 0) =
8
3
pX (1) = P (X = 1) =
8
3
pX (2) = P (X = 2) =
8
1
pX (3) = P (X = 3) = .
8
Find the CDF for X and plot the graph of the CDF.
0.8
0.6
0.4
0.2
0
−1 0 1 2 3 4
P (X = −1) = 0.2,
P (X = 0) = 0.5,
P (X = 1) = 0.3.
Let Y = X 2, nd E[Y ].
Solution : Note that Y takes the values 02 , (−1)2 and 12 , which reduce to 0 or 1. Also notice
that pY (1) = 0.2 + 0.3 = 0.5 and pY (0) = 0.5. Thus, E[Y ] = 0 · 0.5 + 1 · 0.5 = 0.5.
72 5. DISCRETE RANDOM VARIABLES
Note that EX 2 = 0.5 . While (EX)2 = 0.01 since EX = 0.3 − 0.2 = 0.1. Thus in general
EX 2 6= (EX)2 .
In general, there is a formula for g(X) where g is function that uses the fact that g(X) will
be g(x) for some x such that X = x. We recall Theorem 5.2. If X is a discrete distribution
that takes the values xi , i ≥ 1 with probability pX (xi ), respectively, then for any real valued
function g we have that
X∞
E [g (X)] = g (xi ) pX (xi ).
i=1
Note that
∞
X
2
EX = x2i pX (xi )
i=1
will be useful.
Example 5.26. Let us revisit the previous example. Let X denote a random variable such
that
P (X = −1) = 0.2
P (X = 0) = 0.5
P (X = 1) = 0.3.
Let Y = X 2. Find EY .
Solution : We have that
∞
X
EX = 2
x2i pX (xi ) = (−1)2 (0.2) + 02 (0.5) + 12 (0.3) = 0.5
i=1
5.2.5. Variance. The variance of a random variable is a measure of how spread out the
values of X are. The expectation of a random variable is quantity that help us dierentiate
between random variables, but it does not tell us how spread out its values are. For example,
consider
X = 0 with probability 1
(
−1 p = 21
Y =
1 p = 12
(
−100 p = 12
Z = .
100 p = 12
What are the expected values? The are 0, 0 and 0. But there is much greater spread in Z
than Y and Y than X. Thus expectation is not enough to detect spread, or variation.
Example 5.27. Calculate Var(X) if X represents the outcome when a fair die is rolled.
5.2. FURTHER EXAMPLES AND APPLICATIONS 73
Proposition 5.4
For any constants a, b ∈ R we have that Var (aX + b) = a2 Var (X).
5.3. Exercises
Exercise 5.1. Three balls are randomly chosen with replacement from an urn containing
5 blue, 4 red, and 2 yellow balls. Let X denote the number of red balls chosen.
Exercise 5.2. Two cards are chosen from a standard deck of 52 cards. Suppose that
you win $2 for each heart selected, and lose $1 for each spade selected. Other suits (clubs
or diamonds) bring neither win nor loss. Let X denote your winnings. Determine the
probability mass function of X.
Exercise 5.3. A nancial regulator from the FED will evaluate two banks this week. For
each evaluation, the regulator will choose with equal probability between two dierent stress
tests. Failing under test one costs a bank 10K fee, whereas failing test 2 costs 5K. The
probability that the rst bank fails any test is 0.4. Independently, the second bank will fail
any test with 0.5 probability. Let X denote the total amount of fees the regulator can obtain
after having evaluated both banks. Determine the cumulative distribution function of X.
Exercise 5.4. Five buses carry students from Hartford to campus. Each bus carries,
respectively, 50, 55, 60, 65, and 70 students. One of these students and one bus driver are
picked at random.
(a) What is the expected number of students sitting in the same bus that carries the ran-
domly selected student?
(b) Let Y be the number of students in the same bus as the randomly selected driver. Is
E[Y ] larger than the expectation obtained in the previous question?
Exercise 5.5. Two balls are chosen randomly from an urn containing 8 white balls, 4
black, and 2 orange balls. Suppose that we win $2 for each black ball selected and we lose
$1 for each white ball selected. Let X denote our winnings.
Exercise 5.6. A card is drawn at random from a standard deck of playing cards. If it is
a heart, you win $1. If it is a diamond, you have to pay $2. If it is any other card, you win
$3. What is the expected value of your winnings?
Exercise 5.7. The game of roulette consists of a small ball and a wheel with 38 numbered
pockets around the edge that includes the numbers 1 − 36, 0 and 00. As the wheel is spun,
the ball bounces around randomly until it settles down in one of the pockets.
(a) Suppose you bet $1 on a single number and random variable X represents the (monetary)
outcome (the money you win or lose). If the bet wins, the payo is $35 and you get
5.3. EXERCISES 75
your money back. If you lose the bet then you lose your $1. What is the expected prot
on a 1 dollar bet?
(b) Suppose you bet $1 on the numbers 1 − 18 and random variable X represents the
(monetary) outcome (the money you win or lose). If the bet wins, the payo is $1
and you get your money back. If you lose the bet then you lose your $1. What is the
expected prot on a 1 dollar bet ?
Exercise 5.8. An insurance company nds that Mark has a 8% chance of getting into a
car accident in the next year. If Mark has any kind of accident then the company guarantees
to pay him $10, 000. The company has decided to charge Mark a $200 premium for this one
year insurance policy.
(a) Let X be the amount prot or loss from this insurance policy in the next year for the
insurance company. Find EX , the expected return for the Insurance company? Should
the insurance company charge more or less on its premium?
(b) What amount should the insurance company charge Mark in order to guarantee an
expected return of $100?
Exercise 5.9. A random variable X has the following probability mass function: pX (0) =
1 1 1 1
, pX (1) = , pX (2) = , pX (3) = . Find its expected value, variance, and standard
3 6 4 4
deviation, and plot its CDF.
Exercise 5.10. Suppose X is a random variable such that E [X] = 50 and Var(X) = 12.
Calculate the following quantities.
(a) E [X 2 ],
(b) E [3X
+ 2],2
(c) E (X + 2) ,
(d) Var [−X],
(e) SD (2X).
Exercise 5.11. Does there exist a random variable X such that E [X] = 4 and E [X 2 ] = 10?
Why or why not? (Hint: look at its variance)
Exercise 5.12. A box contains 25 peppers of which 5 are red and 20 green. Four peppers
are randomly picked from the box. What is the expected number of red peppers in this
sample of four?
76 5. DISCRETE RANDOM VARIABLES
k 3−k
3 4 7
P(X = k) = · .
k 11 11
Solution to Exercise 5.2: The random variable X can take the values −2, −1, 0, 1, 2, 4.
Moreover,
13
2
P(X = −2) = P(2♠) = 52 ,
2
13 · 26
P(X = −1) = P(1♠ and 1(♦ or ♣)) = 52
,
2
26
2
P(X = 0) = P(2(♦ or ♣)) = 52 ,
2
13 · 13
P(X = 1) = P(1♥ and 1♠) = 52
,
2
P(X = 2) = P(1♥ and 1(♦ or ♣)) = P(X = −1),
P(X = 4) = P(2♥) = P(X = −2).
Thus the probability mass function is given by pX (x) = P(X = x) for x = −2, −1, 0, 1, 2, 4
and pX (x) = 0 otherwise.
Solution to Exercise 5.3: The random variable X can take the values 0, 5, 10, 15 and 20
depending on which test was applied to each bank, and if the bank fails the evaluation or
not. Denote by Bi the event that the ith bank fails and by Ti the event that test i applied.
Then
0.8
0.6
0.4
0.2
0
−5 0 5 10 15 20 25
0 x < 0,
0.3 0 6 x < 5,
0.55 5 6 x < 10,
FX (x) =
0.85 10 6 x < 15,
0.95 15 6 x < 20,
1 x > 20.
Solution to Exercise 5.4: Let X denote the number of students in the bus that carries
the randomly selected student.
50 55
(a) In total there are 300 students, hence P(X = 50) = 300 , P(X = 55) = , P(X = 60) =
300
60 65 70
, P(X = 65) = and P(X = 70) = 300 . The expected value of X is thus
300 300
50 55 60 65 70
E[X] = 50 + 55 + 60 + 65 + 70 ≈ 60.8333.
300 300 300 300 300
1
(b) In this case, the probability of choosing a bus driver is , so that
5
1
E[Y ] = (50 + 55 + 60 + 65 + 70) = 60
5
which is slightly less than the previous one.
4
2 6
P (X = 4) = P ({BB}) = 14 = ,
2
91
2
2 1
P (X = 0) = P ({OO}) = 14 =
2
91
4 2
8
P (X = 2) = P ({BO}) = 1 141 = ,
2
91
8 2
16
P (X = −1) = P ({W O}) = 1 141 = ,
2
91
4 8
32
P (X = 1) = P ({BW }) = 1 141 = ,
2
91
8
2 28
P (X = −2) = P ({W W }) = 14 =
2
91
On average the company will lose $600 dollars. Thus the company should charge more.
1 1 1 1 34
EX = 0 · +1· +2· +3· = .
3 6 4 4 24
5.4. SELECTED SOLUTIONS 79
0.8
0.6
0.4
0.2
0
−1 0 1 2 3 4
Var(X) = E X 2 − (EX)2
2
1 1 1 1 34
= 0 · − 12 + 22 · + 32 · −
2
3 6 4 4 24
2
82 34 812
= − 2 = 2.
24 24 24
Taking the square root gives us
√
2 203
SD(X) = .
24
Solution to Exercise 5.11: Using the hint let's compute the variance of this random
2 2 2
variable which would be Var(X) = E [X ] − (EX) = 10 − 4 = −6. But we know a random
variable cannot have a negative variance. Thus no such a random variable exists.