Introduction to Probability & Statistics

Lecture - 9

Prof. Pritam Ranjan

 Joint Distribution - Chapter 10

Pritam Ranjan / OM&QT 1

Concepts to cover
• Joint distribution (n = 2)

• Expectation: conditional vs. unconditional (n = 2)

• Generalization for n ≥ 2

• Generalization for continuous R.V.’s

Pritam Ranjan / OM&QT 2

 Joint distribution (n = 2)

Pritam Ranjan / OM&QT 3

Recall – Survey data
• Questionnaire example: The question, ”Do you smoke?” was
asked to 100 people. Results are shown in the table.

Yes No
Male 19 41
Female 12 28
Total 100

Question: What is the probability that a male smokes?

Pritam Ranjan / OM&QT 4

Joint probability table
• Questionnaire example: The question, ”Do you smoke?” was
asked to 100 people. Results are shown in the table.

Yes No
Male 0.19 0.41
Female 0.12 0.28
Total 1

Question: Are “smoking habit” and “gender” independent?

Pritam Ranjan / OM&QT 5

Independent R.V.s

Suppose X , Y are two random variable

• Events: [X = u] and [Y = v ] are independent iff

PX ,Y (u, v ) = PX (u)PY (v )

• RVs: X and Y are independent iff

PX ,Y (u, v ) = PX (u)PY (v )

for all possible sets of values of {u, v }

Pritam Ranjan / OM&QT 6

Assuming independence
The joint distribution of X and Y is characterized by

• define
PX ,Y (u, v ) = P(X = u)P(Y = v )
for all possible sets of values of {u, v }

• define marginals using a table

x 0 1 2
P(X = x) 0.19 0.41 0.4

• define marginals using a function

 k
1
P(X = k ) = , for k = 1, 2, 3, ..., ∞, and 0 otherwise.
2
Pritam Ranjan / OM&QT 7
Not independent ?
The joint distribution of X and Y is characterized by
• define
PX ,Y (u, v ) = P(X = u, Y = v )
for all possible sets of values of {u, v }

• using a table
Yes No
Male 0.19 0.41
Female 0.12 0.28
Total 1
• using a function

Pritam Ranjan / OM&QT 8

Not independent ?
The joint distribution of X and Y is characterized by
• define
PX ,Y (u, v ) = P(X = u, Y = v )
for all possible sets of values of {u, v }

• using a table
• using a function
1
P(X = k, Y = r ) = , for k = 1, 2, ..., 5; r = 1, 2, ..., 5; k 6= r ,
20
0 otherwise.

Pritam Ranjan / OM&QT 9

Example
• The following table shows the joint distribution of two random
variables, X - salary of a new employee and Y - the cost of the
benefits (% salary).
Y \X 80 K 100 K 140 K
0.10% 0.3 0.3 0.15
0.15% 0.1 0.1 0.05

Question: Are “salary” and “benefits” independent?

Pritam Ranjan / OM&QT 10

Example - insurance policies
A large insurance agency services a number of customers who have purchased both a
homeowner’s policy and an automobile policy from the agency. For each type of policy,
a deductible amount must be specified. For an automobile policy, the choices are $100
and $250, whereas for a homeowner’s policy, the choices are 0, $100, and $200.
Suppose an individual with both types of policy is selected at random from the agency’s
files. Let X be the deductible amount on the auto policy and Y be the deductible
amount on the homeowner’s policy, and the joint pmf be given by

Y 0 100 200
X 100 0.2 0.10 0.20
250 0.05 0.15 0.30

Find P(Y ≥ 100)

Pritam Ranjan / OM&QT 11

 Expectation (n = 2)

Pritam Ranjan / OM&QT 12

Example
• The following table shows the joint distribution of two random
variables, X - salary of a new employee and Y - the cost of the
benefits (% salary). Thus, the total cost of hiring is T = X (1 + Y )
Y \X 80 K 100 K 140 K
0.10% 0.3 0.3 0.15
0.15% 0.1 0.1 0.05

Question: What is the expected cost of hiring an employee?

Pritam Ranjan / OM&QT 13

Expectation
• For any h(X , Y ), the expectation is given by
XX
E[h(X , Y )] = h(u, v )pX ,Y (u, v )
u v

• Simple example: h(X , Y ) = X

• Simple example: h(X , Y ) = Y
• Simple example: h(X , Y ) = XY

Pritam Ranjan / OM&QT 14

Expectation
• Rules of Expectation:

- E[h(X , Y ) + g(X , Y )] = E[h(X , Y )] + E[g(X , Y )] (Always)

- If X and Y are independent, E(XY ) = E(X )E(Y )

- If X and Y are independent, E[h(X )g(Y )] = E[h(X )]E[g(Y )]

Pritam Ranjan / OM&QT 15

Example

Find the correlation between X and Y .

Pritam Ranjan / OM&QT 16

Covariance
• Define covariance
Cov (X , Y ) = E(XY ) − E(X )E(Y )

• Define correlation
Cov (X , Y )
Corr (X , Y ) = p p
Var (X ) Var (Y )

Pritam Ranjan / OM&QT 17

Example - insurance policies
A large insurance agency services a number of customers who have purchased both a
homeowner’s policy and an automobile policy from the agency. For each type of policy,
a deductible amount must be specified. For an automobile policy, the choices are $100
and $250, whereas for a homeowner’s policy, the choices are 0, $100, and $200.
Suppose an individual with both types of policy is selected at random from the agency’s
files. Let X be the deductible amount on the auto policy and Y be the deductible
amount on the homeowner’s policy, and the joint pmf be given by

Y 0 100 200
X 100 0.2 0.10 0.20
250 0.05 0.15 0.30

Find the expected deductible amount on the homeowner’s policy given that the
deductible on the auto policy is $250.

Pritam Ranjan / OM&QT 18

Conditional expectation
• Simplest case
X
E(X |Y = y) = u P(X = u|Y = y)
u

Pritam Ranjan / OM&QT 19

Example - warranty policy
Let X denote the number of Canon digital cameras sold during a particular week by a
certain store. The pmf of X is

x 0 1 2 3 4
pX (x) .1 .2 .3 .25 .15

Sixty percent of all customers who purchase these cameras also buy an extended
warranty. Let Y denote the number of purchasers during this week who buy an
extended warranty.
• What is P(X = 4, Y = 2)?
• Calculate P(X = Y ).
• Determine the joint pmf of X and Y and then the marginal pmf of Y .

Pritam Ranjan / OM&QT 20

Conditional vs. unconditional expectation
• Simplest case
E(X ) = Ey [Ex (X |Y = y )]

Pritam Ranjan / OM&QT 21

Example
• An assembly line involves sampling 20 finished items per day
and checks for no. of defectives. If p is the probability of
producing a defective item then p is assumed to be a random
variable (due to chance causes/ non-systematic error) with
P[p = 0.01] = 0.2, P[p = 0.5] = 0.6, P[p = 0.9] = 0.18,
P[p = 0.95] = 0.02. Find the expected number of defectives.

Pritam Ranjan / OM&QT 22

 Generalization

Pritam Ranjan / OM&QT 23

General case: 2 → n
Suppose X1 , X2 , ..., Xn denote a sequence of random variable, and we
are interested in finding their joint distribution, then
• we need to define (via a table or a function)

P(X1 = t1 , X2 = t2 , ..., Xn = tn )

for all possible sets of values of {t1 , t2 , ..., tn }

• If Xi ’s are independent (e.g., in a random sample):

P(X1 = t1 , X2 = t2 , ..., Xn = tn ) = P(X1 = t1 ) · · · P(Xn = tn )

for all possible sets of values of {t1 , t2 , ..., tn }

Pritam Ranjan / OM&QT 24

Continuous case
• X and Y are said to be independent iff

fX ,Y (u, v ) = fX (u)fY (v ) ∀ u, v

• In general define

fX ,Y (u, v ) → fX1 ,X2 ,...,Xn (t1 , t2 , ..., tn )

• If Xi ’s are all independent (e.g., in a random sample)

fX1 ,X2 ,...,Xn (t1 , t2 , ..., tn ) = fX1 (t1 ) · · · fXn (tn )

Pritam Ranjan / OM&QT 25

Continuous case - FYI
• conditional distribution
fX ,Y (u, v )
fX |Y (u|v ) =
fY (v )
• conditional expectation
Z
Ex (X |Y = y) = u fX |Y (u|y )du
u

• unconditional expectation E(X ) = Ey [Ex (X |Y = y )]

Z Z Z 
fX ,Y (u, v )
E(X ) = Ex (X |Y = v )fY (v )dv = u du fY (v )dv
v v u fY (v )

Pritam Ranjan / OM&QT 26

 Applications

Pritam Ranjan / OM&QT 27

Application - Inference
• Likelihood: L(θ)

Let {X1 , X2 , ..., Xn } be a random sample from f (x, θ), then

L(θ) = fX1 ,X2 ,...,Xn (x1 , x2 , ..., xn ; θ)

Pritam Ranjan / OM&QT 28

Application - Modern Portfolio Theory
• MPT: a mathematical framework for assembling a portfolio of
Pk
assets (i.e., find wi ’s for Rp = i=1 wi Ri ) such that the expected
return is maximized for a given level of risk.

i.e. maximize E(Rp ) s.t. Var (Rp ) = c

Pritam Ranjan / OM&QT 29

Application - Sampling distribution
Distribution of a statistic - a function of data !!

Objective: Estimate θ using a random sample {X1 , ..., Xn } from fX (θ)

Pritam Ranjan / OM&QT 30

Homework

• Book problems

• Finish unsolved examples from notes, textbook, problem set.

Pritam Ranjan / OM&QT 31