ECON-UA 20 Analytical Statistics

Lecture 3: Independence, Conditional

Probability

Konrad Menzel
September 10, 2024
New York University

1
First Problem Set Due Tuesday next Week

ˆ As a reminder, the first problem set is due on Tuesday, 9/17 at 8pm. The
deadline is sharp, no late submissions will be accepted.
ˆ You should be able to do all problems with material covered in lectures
and recitations this week. You are expected to be very precise in proofs
and derivations, documenting all formal steps.
ˆ You are encouraged to work together and discuss homework problems in
groups, but each student must submit their own, separate written
answers.
ˆ You may consult any notes from class and textbooks.
ˆ Assignments (with or without answers) must never be posted on
“tutoring” websites like CourseHero or Chegg.
ˆ Electronic submissions can be uploaded via Brightspace. You can give me
paper submissions at the end of the lecture on the day the assignment is
due, or drop them off in the TAs’ mailboxes on the 6th floor in 19W4 St
before the 8pm deadline.

2
Independent Events
Independent Events

ˆ We want to define a notion that the occurrence of A does not ”affect”

the likelihood of the occurrence of B.
ˆ For example, if we toss a coin two times, the outcome of the second toss
should not be influenced in any way by the outcome of the first.

Definition The events A and B are said to be independent if

P(A ∩ B) = P(A)P(B)

Independence is therefore a property of the probability distribution, not

necessarily of the nature of the events. In some examples we may have an
intuition what it means for events to be independent, but in most cases we’ll
have to check the formal condition.

3
Independent Events - Examples

ˆ For example suppose we roll a fair die once, and consider the events
A = {2, 4, 6} and B = {1, 2, 3, 4}.

4
Independent Events - Examples

ˆ For example suppose we roll a fair die once, and consider the events
A = {2, 4, 6} and B = {1, 2, 3, 4}.
ˆ Counting outcomes, P(A) = n(A)
n(S)
= 3
6
= 21 , and similarly, P(B) = 4
6
= 23 .
ˆ The probability of the intersection of events is
2 1
P(A ∩ B) = P({2, 4}) = = = P(A)P(B)
6 3
so the events are independent even though they resulted from the same
roll.

4
Independent Events - Examples

ˆ For example suppose we roll a fair die once, and consider the events
A = {2, 4, 6} and B = {1, 2, 3, 4}.
ˆ Counting outcomes, P(A) = n(A)
n(S)
= 3
6
= 21 , and similarly, P(B) = 4
6
= 23 .
ˆ The probability of the intersection of events is
2 1
P(A ∩ B) = P({2, 4}) = = = P(A)P(B)
6 3
so the events are independent even though they resulted from the same
roll.
ˆ However this is no longer true if the die had instead been manipulated so
that P(6) = 38 , whereas for all other numbers n = 1, . . . , 5, P(n) = 18 :
1 1 3 5 4 1
P(A) = + + = , P(B) = =
8 8 8 8 8 2
and
2 1 5
P(A ∩ B) = = < P(A)P(B) =
8 4 16

4
Independent Events - Complements

Proposition If A and B are independent, then A and B C are also

independent.

5
Independent Events - Complements

Proposition If A and B are independent, then A and B C are also

independent.

Proof: Since we can partition A into the disjoint events A ∩ B and A ∩ B C ,

we can write
(P3) Indep.
P(A ∩ B C ) = P(A) − P(A ∩ B) = P(A) − P(A)P(B)
= P(A)(1 − P(B)) = P(A)P(B C )

proving independence □

5
Independence of a Collection of Events

We can extend the definition of independence to more than two events:

Definition A collection of events A1 , A2 , . . . are independent if for any

subset Ai1 , Ai2 , . . . of these events (all indices being different)

P(Ai1 ∩ Ai2 ∩ . . . ) = P(Ai1 ) · P(Ai2 ) · . . .

ˆ For example for three events A, B, C ,

P(A∩B) = P(A)P(B), P(A∩C ) = P(A)P(C ), P(B∩C ) = P(B)P(C )

and
P(A ∩ B ∩ C ) = P(A)P(B)P(C )

6
Example

ˆ For example let the sample space be S = {s1 , s2 , s3 , s4 }, and P(si ) = 1

4
for
all outcomes.
ˆ Then each of the events
A = {s1 , s2 }, B = {s1 , s3 }, C = {s1 , s4 }
1
occurs with probability 2
.

7
Example

ˆ For example let the sample space be S = {s1 , s2 , s3 , s4 }, and P(si ) = 1

4
for
all outcomes.
ˆ Then each of the events
A = {s1 , s2 }, B = {s1 , s3 }, C = {s1 , s4 }
1
occurs with probability 2
.
ˆ The probability for the event A ∩ B is
1
P(A ∩ B) = P({s1 }) = = P(A)P(B)
4
and likewise for any other pair of events, so the events are pairwise
independent.

7
Example

ˆ For example let the sample space be S = {s1 , s2 , s3 , s4 }, and P(si ) = 1

4
for
all outcomes.
ˆ Then each of the events
A = {s1 , s2 }, B = {s1 , s3 }, C = {s1 , s4 }
1
occurs with probability 2
.
ˆ The probability for the event A ∩ B is
1
P(A ∩ B) = P({s1 }) = = P(A)P(B)
4
and likewise for any other pair of events, so the events are pairwise
independent.
ˆ However, taken together, the full collection is not independent since
1 1
P(A ∩ B ∩ C ) = P({s1 }) = ̸= = P(A)P(B)P(C )
4 8

ˆ Intuitively, once we know that both A and B occurred, we know for sure
that C occurred.
7
Conditional Probability
Conditional Probability

Suppose the occurrence of A affects the occurrence (or non-occurrence) of B

and vice versa. How do we describe the probability of B given knowledge about
A?

ˆ We already argued heuristically that if the events are independent, then A

does not reveal any information about B. But what if it does?

8
Conditional Probability

Suppose the occurrence of A affects the occurrence (or non-occurrence) of B

and vice versa. How do we describe the probability of B given knowledge about
A?

ˆ We already argued heuristically that if the events are independent, then A

does not reveal any information about B. But what if it does?
ˆ If we roll a fair die, and I tell you that in fact the outcome was an even
number, i.e that the event B = {2, 4, 6} occurred. What’s the probability
of the event A of having rolled a number greater or equal to 4?

8
Conditional Probability

Suppose the occurrence of A affects the occurrence (or non-occurrence) of B

and vice versa. How do we describe the probability of B given knowledge about
A?

ˆ We already argued heuristically that if the events are independent, then A

does not reveal any information about B. But what if it does?
ˆ If we roll a fair die, and I tell you that in fact the outcome was an even
number, i.e that the event B = {2, 4, 6} occurred. What’s the probability
of the event A of having rolled a number greater or equal to 4?
ˆ Since there are only three equally likely possibilities in B, and the
intersection with A, A ∩ B = {4, 6} contains two equally likely outcomes,
we’d intuitively expect the answer to be 23 .
ˆ I.e. we basically simplified the sample space, to S̃ = B = {2, 4, 6}, and
calculated the simple probability for the redefined problem. We next
formalize this idea as conditional probability.

8
Conditional Probability

Definition Suppose A and B are events defined on S such that P(B) > 0.
The conditional probability of A given that B occurred is given by
P(A ∩ B)
P(A|B) =
P(B)

B ≡ S′
A

A ∩ B ≡ A′

9
Conditional Probability and Independence

ˆ In the definition of conditional probability, the numerator redefines which

outcomes in A are possible once B is known to have occurred, the
denominator does the same for the whole sample space S, where
P(B ∩ S) = P(B).
ˆ If A and B are independent, then from the definition
P(A ∩ B) P(A)P(B)
P(A|B) = = = P(A)
P(B) P(B)
so B occurring tells us nothing about A, and the conditional probability is
the same as the unconditional probability.

10
Example: Betting Markets

ˆ On platforms like PredictIt, you can trade assets which will pay 1$ if a
given event Ai (e.g. Yankees win the World Series) occurs. Such an
asset/contract is called an contingent claim for the event Ai .
ˆ Agents can ask to buy or sell these assets at any prices at any time before
the event (or its complement) is realized. If two agents agree to a
transaction at a particular price πt (Ai ) at time t, the market price is
updated accordingly.

11
Example: Betting Markets

ˆ On platforms like PredictIt, you can trade assets which will pay 1$ if a
given event Ai (e.g. Yankees win the World Series) occurs. Such an
asset/contract is called an contingent claim for the event Ai .
ˆ Agents can ask to buy or sell these assets at any prices at any time before
the event (or its complement) is realized. If two agents agree to a
transaction at a particular price πt (Ai ) at time t, the market price is
updated accordingly.
ˆ On the political market for the 2008 US Presidential election on Intrade,
you could trade assets for the events
ˆ Ai that candidate i wins the presidential elections (without
conditioning on nomination)
ˆ Bi that candidate i wins the nomination of her/his party
ˆ Ck that the nominee of party k wins the election

11
Example: Betting Markets

Why can we interpret prices πt (Ai ) for these assets as probabilities?

ˆ If optimizing agents are allowed to trade any combination of positive or negative numbers of assets, and the
set of tradable assets correspond to a sigma algebra of events, then any violation of Kolmogorov’s axioms
would present an opportunity to make bets that return a positive profit for sure (“arbitrage opportunity”).
ˆ For axiom (P1), if any agent offered any contingent claim at a price πi < 0, then buying a unit of that
asset guarantees a positive profit for sure and regardless of the likelihood of event Ai , since the payout at
the end of the contract can’t be negative.
ˆ For axiom (P3), if the sum of prices for events corresponding to a partition A1 , . . . , Ak of the event A add
up to less than πt (A), then buying one unit each of A1 , . . . , Ak and selling one of A earns a profit
πt (A) − (πt (A1 ) + · · · + πt (Ak )) > 0 and leaves the payoff at the end of the contract unchanged,
regardless of whether A occurs or not. If they add up to more, then we could execute the opposite trade
ˆ For axiom (P2), if asset prices for a partition of the sure event (e.g. “Democratic” or “Republican” or
”other” candidate wins) sum up to less than 1$, we should buy now and receive a sure payoff of 1$ in the
future.
ˆ An “arbitrage-free” market is a market in which any of those transactions have already taken place and
forced asset prices to adjust to eliminate any further arbitrage opportunities.

12
PredictIt price data for the winner of the US Presidential election 2024. 13
PredictIt price data for the Democratic Nominating contest. 14
Example: Betting Markets

ˆ On June 27, 2024 there was a public debate between Donald Trump and
Joe Biden, then the presumed nominees of their parties. Biden’s
performance ultimately lead to doubts about his candidacy, and was
eventually replaced by Kamala Harris.
ˆ We can now look at the data to infer whether the prediction markets
“thought” that she was going to be a stronger candidate, i.e. was the
probability P(Ai |Bi ) of Harris winning the presidency conditional on being
nominated greater than Biden’s?
ˆ We assume that a candidate who is not nominated by the party has no
chances of winning the presidency, so that
P(Ai ∩ Bi ) P(Ai )
P(Ai |Bi ) = =
P(Bi ) P(Bi )

15
Implied probabilities for Biden and Harris winning the general election
conditional on being nominated by the DNC.

16
Law of Total Probability

ˆ In order to distinguish it from the conditional probabilities P(A|Bi ), P(A)

is also called the marginal probability of A.
ˆ The relationship between marginal and conditional probabilities is given
by the Law of Total Probability:

Theorem (Law of Total Probability) Suppose that B1 , . . . , Bn is a

partition of the sample space S such that P(Bi ) > 0 for every
i = 1, . . . , n. Then
Xn
P(A) = P(A|Bi )P(Bi )
i=1

for any event B.

17
Law of Total Probability - Proof

ˆ From the definition of a conditional probability,

P(A|Bi )P(Bi ) = P(A ∩ Bi ) for any event Bi .
ˆ Since B1 , . . . , Bn is a partition of the sample space, (A ∩ B1 ), . . . , (A ∩ Bn )
are disjoint and exhaustive for A - i.e. constitute a partition for A.
ˆ Therefore, by the axiom (P3) on probabilities of unions of disjoint sets,
Pn
i=1 P(A ∩ Bi ) = P(A) □

B1
B2
B3
A

B4
B5 B6 S

18
Application: Composition Effects

In medical data we can often find that patients treated by older, and more
experienced, heart surgeons have in fact higher post-operative death rates than
those operated by younger surgeons.

ˆ Say we observe a death rate of 6.0% for experienced surgeons, and only
5.5% for inexperienced surgeons. Does this mean that the surgeons’ skill
decreases with age?
ˆ Probably not - let’s suppose there are four different types of procedures a
surgeon may have to perform - single, double, triple, and quadruple
bypass. The complexity of the procedure and the risk to the patient
increase in the number of bypasses, and patients who are generally
”sicker” may tend to require a more complicated procedure.

19
Application: Composition Effects

ˆ To illustrate this point, we can construct a hypothetical example where

for each procedure, patients of the experienced surgeon face significantly
lower death rates, but the overall patient mortality is lower for
inexperienced surgeons:

Inexperienced Experienced
Procedure Death Rate Percentage of Cases Death Rate Percentage of Cases
Single Bypass 4.0% 50.0% 2.0% 25.0%
Double Bypass 6.0% 40.0% 4.0% 25.0%
Triple Bypass 10.0% 9.0% 6.0% 25.0%
Quadruple Bypass 20.0% 1.0% 12.0% 25.0%
Total 100.0% 100.0%

20
Application: Composition Effects

ˆ To illustrate this point, we can construct a hypothetical example where

for each procedure, patients of the experienced surgeon face significantly
lower death rates, but the overall patient mortality is lower for
inexperienced surgeons:

Inexperienced Experienced
Procedure Death Rate Percentage of Cases Death Rate Percentage of Cases
Single Bypass 4.0% 50.0% 2.0% 25.0%
Double Bypass 6.0% 40.0% 4.0% 25.0%
Triple Bypass 10.0% 9.0% 6.0% 25.0%
Quadruple Bypass 20.0% 1.0% 12.0% 25.0%
Total 5.5% 100.0% 6% 100.0%

20
Application: Composition Effects

ˆ In the notation of the Law of Total Probability, the overall death rate
P(A) for experienced surgeons can be computed from the death rate
conditional on procedure Bi , P(A|Bi ), and the base rates/proportions
P(Bi ) of cases corresponding to each procedure.
ˆ If experienced surgeons are assigned a disproportionately large share of
risky cases (their average (better: marginal) death rate is higher than
that of inexperienced surgeons, even though they perform better
conditional on each treatment category.
ˆ This phenomenon is often referred to as a composition effect. The
variable ”type of procedure” would be treated as what statisticians call a
confounding factor.

21
Summary

ˆ Pairs of events A and B may be related and convey information about

each other.
ˆ We formalized that idea by defining the conditional probability P(A|B).
ˆ If A and B are independent, P(A|B) = P(A), or equivalently,
P(A ∩ B) = P(A)P(B).
ˆ We can relate conditional probabilities P(A|B) to the marginal probability
P(A) using the law of total probability.

22
Conditional Independence
Conditional Independence

Another closely related concept is that of conditional independence:

Definition Two events A and B are said to be independent conditional on

event C if
P(A ∩ B|C ) = P(A|C )P(B|C )

It is important to note that

ˆ unconditional independence does not imply conditional independence

ˆ conditional independence does not imply unconditional independence

i.e. whether A and B are independent depends crucially on what we are

conditioning on.

23
Conditional Independence - Example

Consider the experiment of a roll of a fair die.

(1) Making two independent events dependent: Consider the events

A = {1, 2, 3, 4} and B = {2, 4, 6}. We already saw in a previous example
that these events are independent:
4 3 1
P(A)P(B) = P({1, 2, 3, 4})P({2, 4, 6}) = · = = P({2, 4}) = P(A∩B)
6 6 3

24
Conditional Independence - Example

Consider the experiment of a roll of a fair die.

(1) Making two independent events dependent: Consider the events

A = {1, 2, 3, 4} and B = {2, 4, 6}. We already saw in a previous example
that these events are independent:
4 3 1
P(A)P(B) = P({1, 2, 3, 4})P({2, 4, 6}) = · = = P({2, 4}) = P(A∩B)
6 6 3

ˆ Now let event C = {3, 6}. Then

P(A ∩ C ) P({3}) 1 P({6}) P(B ∩ C )
P(A|C ) = = = = = = P(B|C )
P(C ) P({3, 6}) 2 P(C ) P(C )

24
Conditional Independence - Example

Consider the experiment of a roll of a fair die.

(1) Making two independent events dependent: Consider the events

A = {1, 2, 3, 4} and B = {2, 4, 6}. We already saw in a previous example
that these events are independent:
4 3 1
P(A)P(B) = P({1, 2, 3, 4})P({2, 4, 6}) = · = = P({2, 4}) = P(A∩B)
6 6 3

ˆ Now let event C = {3, 6}. Then

P(A ∩ C ) P({3}) 1 P({6}) P(B ∩ C )
P(A|C ) = = = = = = P(B|C )
P(C ) P({3, 6}) 2 P(C ) P(C )

ˆ However,
P(∅)
P(A ∩ B|C ) = = 0 ̸= P(A|C )P(B|C )
P(C )

24
Conditional Independence - Example

(2) Making two dependent events independent: Let D = {2, 3, 4} and

E = {2, 4, 6}.
ˆ We can check that D and E are dependent: Clearly P(D) = P(E ) = 21 ,
however,
1 1
P(D ∩ E ) = P({2, 4}) = ̸= = P(D)P(E )
3 4

25
Conditional Independence - Example

(2) Making two dependent events independent: Let D = {2, 3, 4} and

E = {2, 4, 6}.
ˆ We can check that D and E are dependent: Clearly P(D) = P(E ) = 21 ,
however,
1 1
P(D ∩ E ) = P({2, 4}) = ̸= = P(D)P(E )
3 4

ˆ But if we condition on F = {3, 4, 5, 6},

P({4}) 1
P(D ∩ E |F ) = =
P({3, 4, 5, 6}) 4
whereas
P({3, 4}) P({4, 6}) 1 1 1
P(D|F )P(E |F ) = · = · =
P({3, 4, 5, 6}) P({3, 4, 5, 6}) 2 2 4

25