Probability Theory

1
Outline:
 1. Conditional Probability

 2. Bayes’ Theorem

2
3.6 Conditional probability
 Probability of an event is meaningful
iff it refers to a given sample.

 P(A|S): the probability of A given

some space S. If respect to more
samples.

3
Ex.
 500 machines. Improper assemble I=
30;Defective D= 15; Both
I and D = 10.
15
 P(D)=? 500 I DDand
andI I
20 1010
 P(D|I)=? 30
10

 P(D and I)=? 10

500

4
Conditional prob.
 Theorem. If A and B are any events is
S and P(B) is not empty, the
conditional probability of A given B is:

P( A  B)
P( A | B) 
P( B)

5
Ex.
 A coin is flipped twice. If we assume
that all four points in the sample
space are equally likely, what is the
probability that both flips result in
heads, given that the first flip does?
Solution: Let E = {(H,H)} be the event that both flips land
heads, and F={(H,H), (H,T)} denote the event that the first
flip lands heads, then the desired probability is given by

P( E  F ) 1/ 4
P( E | F )    1/ 2
P( F ) 2/ 4

6
Ex.
 Suppose a family has two children. We
assume that the probability of having a
baby boy is ½.

 Now suppose we wish to find the probability

that the family has one boy and one girl,
but we also have the information that at
least one of the children is a boy.

 What is the probability?

7
Ex.
 Celine is undecided as to whether to take a
French course or a chemistry course. She
estimates that her probability of receiving
an A grade would be ½ in a French course,
and 2/3 in a chemistry course.

 If Celine decides to base her decision on

the flip of a fair coin, what is the probability
that she gets an A in chemistry?

8
Solution
 If we let C be the event that Celine
takes chemistry and B denote the
event that she receive an A in
whatever course she takes, then the
desired probability is P(CB).

P(CB)  P(C ) P( B | C )
1 2 1
  
2 3 3

9
 Multiplication rule

P( E1E2 En )  P( E1 ) P( E2 | E1 ) P( E3 | E1E2 ) P( En | E1 En 1 )

10
Multiplication rule – Ex.
 An ordinary deck of 52 playing cards
is randomly divided into 4 piles of 13
cards each. Compute the probability
that each pile has exactly 1 Ace?

11
Solution: another approach
 Define events
 E1 = {the ace of spades is in any one of the piles}
 E2 = {the ace of spades and the ace of hearts are in
different piles}
 E3 = {the aces of spades, hearts, and diamonds are all in
different piles}
 E4 = {all aces are in different piles}

 P(E1) = 1, P(E2|E1)=39/51, P(E3|E1E2)=26/50,

 P(E4|E1E2E3) = 13/49.
 P(E1E2E3E4) = P(E1)P(E2|E1)P(E3|E1E2)P(E4|E1E2E3)
 = 0.105

12
Independent
 A is independent of B if and only if

P(A|B)=P(A)
or

P(A and B) = P(A) * P(B)

13
EX.
 Suppose that we toss 2 dice.
 Let E denote the event that the sum of
the dice is 6 and
 F denote the event that the first die
equals 4.

 Q: are the events E and F

independent?

14
Solution
 P(EF) = P({4, 2}) = 1/36.
 P(E) = 5/36 P(F) = 1/6

 P(E)P(F) = 5/216
 Conclusion: E and F are not
independent!!
 Question: how about the event E
changed to: the sum is 7.
15
Properties
 If E and F are independent, then so
are E and F

 If E is independent of F and is also

independent of G. Is E then
necessarily independent of FG?

 Ans: No!!
16
Generalization
 Theorem 3.8 If A and B are any
events in S, then
P( A  B)  P( A | B) P( B)  P( B | A) P( A)
if P(A)>0,P(B)>0.

If independent
P( A  B)  P( A) P( B)

17
Discussion
 P(A|B) and P(A ∩ B)

 A, B are independent, mutually

exclusive, what is the difference?

18
Bayes’ Theorem
B2
B1
B3
Event A

B6 B5 B4

B1 , B2 , , Bn  A partition of sample S

Bi  B j  , for i  j S n
i 1 Bi

P( A)   in1 P( A  Bi )   in1 P( A | Bi ) P( Bi )
19
Bayes’ Theorem
 An experiment depends on the outcomes of
various intermediate stages.
 EX. An assembly plant receives its voltage
regulators.
60% from B1, 30% from B2, 10% from B3
Perform according to specifications:
B1: 95%, B2: 80%, B3: 65%.

Event: a voltage regulator received by the plant

performs according to specifications.
20
Tree diagram

P(A|B1)=0.95
B1
A
0.6

0.3 B2 P(A|B2)=0. 8
A

0.1
P(A|B3)=0.65
B3 A

21
Solution

P( A)  P ( A  B1)  P ( A  B 2)  P( A  B3)

P ( A)  P ( B1) P ( A | B1)  P ( B 2) P ( A | B 2)  P( B3) P( A | B3)

 0.6 * 0.95  0.3 * 0.8  0.1* 0.65
 0.875

22
Rule of total probability
 Theorem 3.10.

If B1 , B2 , Bn are mutually exclusive

events of which one must occur, then
n
P( A)   P( Bi ) P( A | Bi )
i 1

23
Special case

P( A)  P( A | B) P( B)  P( A | B )(1  P( B))

24
EX.
 A laboratory blood test is 95 percent
effective in detecting a certain disease
when it is, in fact, present. However, the
test also yield a “false positive” result for 1
percent of the health person tested.

 Q: If .5 percent of the population actually

has the disease, what is probability a
person has the disease given that the test
result is positive?
25
 Only 0.323!

Solution
 Let D be the event that the tested
person has the disease.
 Let E be the event that the test result
is positive.
 The desired probability is P(D|E)
P( D  E )
P( D | E ) 
P( E )
P( D) P( E | D)

P( E | D) P( D)  P( E | D)(1  P( D))
0.005  0.95

0.95 * 0.005  0.01* (1  0.005) 26
 0.323
Bayes’ Theorem
 Theorem. If B1 , B2 ,, Bn are mutually
exclusive events of which one must
occur, then
P( Br ) P( A | Br )
P( Br | A)  n

 P( B ) P( A | B )
i 1
i i
The prob. Of
reaching A via
P( Br  A) P( B ) P( A | Br ) the r-th branch
“Effect” A was P( Br | A)   n r
 P( Bi ) P( A | Bi )
P( A)
“caused” by of the tree
i 1
the event Br

27
EX.

P( B1 ) P( A | B1 )
P( B1 | A) 
P( B1) P( A | B1)  P( B 2) P( A | B 2)  P( B3) P( A | B3)
0.6 * 0.95

0.6 * 0.95  0.3* 0.8  0.1* 0.65
 0.65

28
Expectation
 Expectation: If the probability of
obtaining the amounts
a1 , a2 ,, or ak are p1 , p2 ,, and pk

then the mathematical expectation is

E  a1 p1  a2 p2    an pn

29
Motivations
 The expected value of x is a weighted
average of possible values that X can
take on, each value being weighted
by the probability that X assumes it.

 Frequency interpretation

30
Summary
 Conditional probability
P( A  B)
P( A | B)  , if P( B)  0
P( B)

 Independent of events A, B
P( A | B)  P( A),
P( A  B )  P ( A) P ( B )

 Bayes’ rule, Bi disjoint

n
P( A)   P( Bi ) P( A | Bi )
i 1
31