MATH F113: Probability & Statistics

Anupama Sharma

Department of Mathematics,
BITS PILANI K K Birla Goa Campus, Goa

Semester I, 2023-24

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
 Module 2: Discrete Distributions

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
 Definition: Let X be a random variable with pmf f and expected
value µ. The variance of X , denoted by Var [X ] or 2 , is given by

Var [X ] = E [(X µ)2 ].

Definition: Let X be a random variable with variance 2. The

standard deviation of X , denoted by , is given by
p p
= 2 = Var [X ].

Note that the variance measure variability by considering (X µ),

the di↵erence between r.v. and its mean. The di↵erence is squared
so that negative values will not cancel the positive ones.

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
 Theorem: Computational formula for 2

2
= E [X 2 ] (E [X ])2 .

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
 Theorem: Computational formula for 2

2
= E [X 2 ] (E [X ])2 .

Proof: By definition,
2
= Var [X ] = E [(X µ)2 ]

2
) = E [(X µ)2 ]
= E [(X 2 2 · µ · X + µ2 )]
= E [X 2 ] 2 · µ · E [X ] + µ2
= E [X 2 ] 2 · µ · µ + µ2 [Since E [X ] = µ]
2 2
= E [X ] µ
2
= E [X ] (E [X ])2

This completes the proof.

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
 Rules of Variance:
Let X and Y be random variables and c be any real number. Then,
(i) Var c = 0.
(ii) Var cX = c 2 Var X .
(iii) If X and Y are independent, then

Var (X + Y ) = Var X + Var Y .

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
 Rules of Variance:
Let X and Y be random variables and c be any real number. Then,
(i) Var c = 0.
(ii) Var cX = c 2 Var X .
(iii) If X and Y are independent, then

Var (X + Y ) = Var X + Var Y .

Proposition: If H(x) = aX + b is linear function, then

V [aX + b] = a2 V [X ]

where, a and b are constants.

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
 Definition: Let X be a random variable. The k-th ordinary
moment for X is defined as E [X k ] where k is a natural number.

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
 Definition: Let X be a random variable. The k-th ordinary
moment for X is defined as E [X k ] where k is a natural number.

Moment generating function – is a generating function that

encodes the moments of a distribution.

Definition: Let X be a random variable with p.m.f. f . The

moment generating function for X (mgf), denoted by mX (t), is
given by X
mX (t) = E [e tX ] = e tx f (x)
all x
provided this expectation is finite for all real numbers t in some
open interval containing zero.

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
 What it means if mgf exists for a r.v. X?

– Mean of X can be found by evaluating first derivative of mgf

at t = 0, i.e.
µ = E [X ] = mX 0 (0)

– Variance of X can be found by evaluating the first and

second derivatives of mgf at t = 0, i.e.

= E [X 2 ] (E [X ])2 = mX 00 (0) (mX 0 (0))2

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
 Theorem: Let mX (t) be the moment generating function for a
random variable X . Then
d k mX (t)
= E [X k ].
dt k t=0

Proof: Let us consider that z = tX . The Maclaurian series

expansion for e tX gives,

(tX )2 (tX )3
e tX = 1 + tX + + + ···
2! 3!
By taking the expected value of both sides in the above equation,
we get
(tX )2 (tX )3
E [e tX ] = E [1 + tX + + + ···]
2! 3!
t 2 E [X 2 ] t 3 E [X 3 ]
mX (t) = 1 + tE [X ] + + + ···
2! 3!

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
 Di↵erentiate the above equation w.r.t. t, we get

dmX (t) t 2 E [X 3 ]
= E [X ] + tE [X 2 ] + + ···
dt 2!
Note that, when this derivative is evaluated at t = 0, every term
except the first term becomes 0. Hence,

dmX (t)
= E [X ].
dt t=0

Taking the second derivative and evaluating at t = 0, gives,

d 2 mX (t)
= E [X 2 ].
dt 2 t=0

If we continue this, we get

dmX (t)
= E [X k ].
dt k t=0

This completes the proof.

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
Some points to remember about mgf

1. Using moment generating function is a very powerful tool but

unfortunately mgf does not always exist so we can not use it every
time even if we are dealing with independent variables.

2. Uniqueness: The moment generating function of a

distribution, if it exists, uniquely determine the distribution. This
implies that corresponding to a given probability distribution, there
is one and only one mgf (provided it exists) and corresponding to a
given mgf, there is only one probability distribution.

3. The variable t is not related to X, it is a dummy variable.

4. mX (0) = 1 for any valid mgf. So whenever you compute an

mgf, plug in 0 and check if you get 1.

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
 Theorem: Let X be a random variable with moment generating
function mX (t). For any constants a and b, the moment
generating function of Y = aX + b is

mY (t) = e bt mX (at).

Theorem: Let X1 and X2 be independent random variables with

moment generating functions mX1 (t) and mX2 (t) respectively. Let
Y = X1 + X2 . The moment generating of Y is

mY (t) = mX1 (t)mX2 (t).

Theorem: Let X and Y be random variables with moment

generating functions mX (t) and mY (t) respectively. If
mX (t) = mY (t) for all t in some open interval about 0, then X
and Y have same distribution.

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
Uniform random variable
Uniform distribution
– the most fundamental of all is the discrete uniform
distribution.
– symmetric distribution where a finite number of values are
“equally likely” to be observed, i.e. every one of the n events
has equal probability 1/n.
– for example, possible outcomes of rolling a fair die; drawing
a spade, a heart, a club, or a diamond from a deck of card

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
Uniform random variable
Uniform distribution
– the most fundamental of all is the discrete uniform
distribution.
– symmetric distribution where a finite number of values are
“equally likely” to be observed, i.e. every one of the n events
has equal probability 1/n.
– for example, possible outcomes of rolling a fair die; drawing
a spade, a heart, a club, or a diamond from a deck of card

Uniform random variable: A random variable X has a discrete

uniform distribution if each of n values in its range has equal
probability. Then

f (x) = P(X = x) = 1/n, x = 1, 2, ..., n,

where, f (x) represents the pmf of X. We write X ⇠ Unif (p).

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
Uniform random variable
n
X
Mean, E [X ] = xf (x) = (n + 1)/2.
x=1

n
X (n + 1)(2n + 1)
We have, E [X 2 ] = x 2 f (x) =
6
x=1

Variance, V [X ] = 2 = E [X 2 ] (E [X ])2 = (n2 1)/12.

Moment generation function,

( t
e e (n+1)t
n(1 e t ) , t 6= 0
mX (t) =
1, t = 0.

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
Bernoulli random variable

A Bernoulli trial is a random experiment in which there are only

two possible outcomes – success and failure.

Examples
1. Tossing a coin and considering heads as success and tails as
failure.
2. Checking items from a production line: success = not defective,
failure = defective.
3. Phoning a call centre: success = operator free; failure = no
operator free.
4. Simulating the spread of an epidemic: success =disease
transmission; failure = no disease transmission.

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
Bernoulli random variable
A r.v. X is said to be a Bernoulli random variable if its probability
mass function is given by

f (1) = P(X = 1) = p
f (0) = P(X = 0) = 1 p.

where p, 0  p  1, is the probability that the trail is a success.

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
Bernoulli random variable
A r.v. X is said to be a Bernoulli random variable if its probability
mass function is given by

f (1) = P(X = 1) = p
f (0) = P(X = 0) = 1 p.

where p, 0  p  1, is the probability that the trail is a success.

Mean, E [X ] = p,
Variance, V [X ] = p(1 p),
Moment generation function, mx (t) = (1 p) + e t p.
We write
X ⇠ Bern(p).
The number p is called the parameter of the distribution.
MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
Binomial random variable

Consider n independent trials, each of which result in a success

with probability p or in failure with probability (1 p) are
performed.

The trials are identical and independent, and probability p of

success remains same from trial to trial.

The random variable X represents the number of successes that

occur in these n trails, then X is said to be a Binomial random
variable and the distribution of X is called the Binomial
distribution with parameters n and p. We write X ⇠ Bin(n, p).

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
Binomial random variable

Consider n independent trials, each of which result in a success

with probability p or in failure with probability (1 p) are
performed.

The trials are identical and independent, and probability p of

success remains same from trial to trial.

The random variable X represents the number of successes that

occur in these n trails, then X is said to be a Binomial random
variable and the distribution of X is called the Binomial
distribution with parameters n and p. We write X ⇠ Bin(n, p).

Note that, Bernoulli distribution is a special case of Binomial

distribution with parameter (1,p)

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
Examples

Consider the experiment where one ball is drawn without

replacement from a box containing 20 red and 30 blue balls, and
the number of red balls drawn is recorded. Is this a binomial
experiment?

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
Examples

Consider the experiment where one ball is drawn without

replacement from a box containing 20 red and 30 blue balls, and
the number of red balls drawn is recorded. Is this a binomial
experiment?
No! The key here is the lack of independence - since the balls are
drawn without replacement, the ball drawn on the first will a↵ect
the probability of later balls.

A fair die is rolled ten times, and the number of 6’s is recorded.
Is this a binomial experiment?

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
Examples

Consider the experiment where one ball is drawn without

replacement from a box containing 20 red and 30 blue balls, and
the number of red balls drawn is recorded. Is this a binomial
experiment?
No! The key here is the lack of independence - since the balls are
drawn without replacement, the ball drawn on the first will a↵ect
the probability of later balls.

A fair die is rolled ten times, and the number of 6’s is recorded.
Is this a binomial experiment?
Yes! There are fixed number of trials (ten rolls), each roll is
independent of the others, there are only two outcomes (either
it’s a 6 or it isn’t), and the probability of rolling a 6 is constant.

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
Binomial random variable

Theorem: Let a random variable X has a Binomial distribution

with parameters n and p. Then its probability mass function is
given by
⇢ n x
p (1 p)n x x = 0, 1, . . . , n
b(x; n, p) = x
0 otherwise

where 0 < p < 1 and n is a positive integer.

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
Binomial random variable
Example: Five fair coins are tossed. Assuming the outcomes
independent, find the pmf of number of heads obtained.

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
Binomial random variable

For X ⇠ Bin(n, p), the cdf will be denoted by

x
X
B(x; n, p) = P(X  x) = b(y ; n, p) x = 0, 1, . . . , n
y =0

Going back to the previous example,

Find the probability of at least two heads.

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa
Binomial random variable
Example: Bits are sent over a communications channel in packets
of 12. If the probability of a bit being corrupted over this channel
is 0.1 and such errors are independent, what is the probability that
no more than 2 bits in a packet are corrupted? If 6 packets are
sent over the channel, what is the probability that at least one
packet will contain 3 or more corrupted bits?

MATH F113 (Probability & Statistics) Anupama Sharma, BITS Pilani, Goa