LECTURE NOTES

on

PROBABILITY and STATISTICS

Eusebius Doedel
 SAMPLE SPACES

DEFINITION :
The sample space is the set of all possible outcomes of an experiment.

EXAMPLE : When we flip a coin then sample space is

S = {H , T },
where
H denotes that the coin lands ”Heads up”
and
T denotes that the coin lands ”Tails up”.

For a ”fair coin ” we expect H and T to have the same ”chance ” of

occurring, i.e., if we flip the coin many times then about 50 % of the
outcomes will be H.
We say that the probability of H to occur is 0.5 (or 50 %) .
The probability of T to occur is then also 0.5.

1
EXAMPLE :

When we roll a fair die then the sample space is

S = {1, 2, 3, 4, 5, 6}.

1
The probability the die lands with k up is 6
, (k = 1, 2, · · · , 6).

When we roll it 1200 times we expect a 5 up about 200 times.

The probability the die lands with an even number up is

1 1 1 1
+ + = .
6 6 6 2

2
EXAMPLE :
When we toss a coin 3 times and record the results in the sequence
that they occur, then the sample space is

S = { HHH , HHT , HT H , HT T , T HH , T HT , T T H , T T T } .

Elements of S are ”vectors ”, ”sequences ”, or ”ordered outcomes ”.

We may expect each of the 8 outcomes to be equally likely.

1
Thus the probability of the sequence HT T is 8
.

The probability of a sequence to contain precisely two Heads is

1 1 1 3
+ + = .
8 8 8 8

3
EXAMPLE : When we toss a coin 3 times and record the results
without paying attention to the order in which they occur, e.g., if we
only record the number of Heads, then the sample space is
n o
S = {H, H, H} , {H, H, T } , {H, T, T } , {T, T, T } .
The outcomes in S are now sets ; i.e., order is not important.
Recall that the ordered outcomes are
{ HHH , HHT , HT H , HT T , T HH , T HT , T T H , T T T } .

Note that
{H, H, H} corresponds to one of the ordered outcomes,
{H, H, T } ,, three ,,
{H, T, T } ,, three ,,
{T, T, T } ,, one ,,

Thus {H, H, H} and {T, T, T } each occur with probability 18 ,

while {H, H, T } and {H, T, T } each occur with probability 38 .

4
Events

In Probability Theory subsets of the sample space are called events.

EXAMPLE : The set of basic outcomes of rolling a die once is

S = {1, 2, 3, 4, 5, 6},

so the subset E = { 2 , 4 , 6 } is an example of an event.

If a die is rolled once and it lands with a 2 or a 4 or a 6 up then we

say that the event E has occurred.

We have already seen that the probability that E occurs is

1 1 1 1
P (E) = + + = .
6 6 6 2

5
The Algebra of Events

Since events are sets, namely, subsets of the sample space S, we can
do the usual set operations :

If E and F are events then we can form

Ec the complement of E
E∪F the union of E and F
EF the intersection of E and F

We write E ⊂ F if E is a subset of F .

REMARK : In Probability Theory we use

Ec instead of Ē ,

EF instead of E ∩ F ,

E⊂F instead of E ⊆ F .

6
If the sample space S is finite then we typically allow any subset of
S to be an event.

EXAMPLE : If we randomly draw one character from a box con-

taining the characters a, b, and c, then the sample space is
S = {a , b , c} ,
and there are 8 possible events, namely, those in the set of events
n o
E = { } , {a} , {b} , {c} , {a, b} , {a, c} , {b, c} , {a, b, c} .

If the outcomes a, b, and c, are equally likely to occur, then

1 1 1
P ({ }) = 0 , P ({a}) = , P ({b}) = , P ({c}) = ,
3 3 3
2 2 2
P ({a, b}) = , P ({a, c}) = , P ({b, c}) = , P ({a, b, c}) = 1 .
3 3 3
For example, P ({a, b}) is the probability the character is an a or a b.

7
We always assume that the set E of allowable events includes the
complements, unions, and intersections of its events.

EXAMPLE : If the sample space is

S = {a , b , c , d} ,

and we start with the events

n o
E0 = {a} , {c, d} ,

then this set of events needs to be extended to (at least)

n o
E = { } , {a} , {c, d} , {b, c, d} , {a, b} , {a, c, d} , {b} , {a, b, c, d} .

EXERCISE : Verify E includes complements, unions, intersections.

8
Axioms of Probability

A probability function P assigns a real number (the probability of E)

to every event E in a sample space S.

P (·) must satisfy the following basic properties :

• 0 ≤ P (E) ≤ 1 ,

• P (S) = 1 ,

• For any disjoint events Ei , i = 1, 2, · · · , n, we have

P (E1 ∪ E2 ∪ · · · ∪ En ) = P (E1 ) + P (E2 ) + · · · P (En ) .

9
Further Properties

PROPERTY 1 :

P (E ∪ E c ) = P (E) + P (E c ) = 1 . ( Why ? )

Thus
P (E c ) = 1 − P (E) .

EXAMPLE :
What is the probability of at least one ”H” in four tosses of a coin?

SOLUTION : The sample space S will have 16 outcomes. (Which?)

1 15
P (at least one H) = 1 − P (no H) = 1 − = .
16 16

10
PROPERTY 2 :
P (E ∪ F ) = P (E) + P (F ) − P (EF ) .

PROOF (using the third axiom) :

P (E ∪ F ) = P (EF ) + P (EF c ) + P (E c F )

= [P (EF ) + P (EF c )] + [P (EF ) + P (E c F )] − P (EF )

= P(E) + P(F) - P(EF) . ( Why ? )

NOTE :
• Draw a Venn diagram with E and F to see this !

• The formula is similar to the one for the number of elements :

n(E ∪ F ) = n(E) + n(F ) − n(EF ) .

11
So far our sample spaces S have been finite.
S can also be countably infinite, e.g., the set Z of all integers.
S can also be uncountable, e.g., the set R of all real numbers.

EXAMPLE : Record the low temperature in Montreal on January

8 in each of a large number of years.
We can take S to be the set of all real numbers, i.e., S = R.
(Are there are other choices of S ?)
What probability would you expect for the following events to have?

(a) P ({π}) (b) P ({x : − π < x < π})

(How does this differ from finite sample spaces?)

We will encounter such infinite sample spaces many times · · ·

12
Counting Outcomes

We have seen examples where the outcomes in a finite sample space

S are equally likely , i.e., they have the same probability .

Such sample spaces occur quite often.

Computing probabilities then requires counting all outcomes and

counting certain types of outcomes .

The counting has to be done carefully!

We will discuss a number of representative examples in detail.

Concepts that arise include permutations and combinations.

13
Permutations

• Here we count of the number of ”words ” that can be formed

from a collection of items (e.g., letters).

• (Also called sequences , vectors , ordered sets .)

• The order of the items in the word is important;

e.g., the word acb is different from the word bac .

• The word length is the number of characters in the word.

NOTE :
For sets the order is not important. For example, the set {a,c,b} is
the same as the set {b,a,c} .

14
EXAMPLE : Suppose that four-letter words of lower case alpha-
betic characters are generated randomly with equally likely outcomes.
(Assume that letters may appear repeatedly.)

(a) How many four-letter words are there in the sample space S ?
SOLUTION : 264 = 456, 976 .

(b) How many four-letter words are there are there in S that start
with the letter ”s ” ?
SOLUTION : 263 .

(c) What is the probability of generating a four-letter word that

starts with an ”s ” ?
SOLUTION :
263 1 ∼
4
= = 0.038 .
26 26
Could this have been computed more easily?

15
EXAMPLE : How many re-orderings (permutations) are there of
the string abc ? (Here letters may appear only once.)
SOLUTION : Six, namely, abc , acb , bac , bca , cab , cba .

If these permutations are generated randomly with equal probability

then what is the probability the word starts with the letter ”a ” ?
SOLUTION :
2 1
= .
6 3

EXAMPLE : In general, if the word length is n and all characters

are distinct then there are n! permutations of the word. ( Why ? )

If these permutations are generated randomly with equal probability

then what is the probability the word starts with a particular letter ?

SOLUTION :
(n − 1)! 1
= . ( Why ? )
n! n

16
EXAMPLE : How many

words of length k

can be formed from

a set of n (distinct) characters ,

(where k ≤ n ) ,

when letters can be used at most once ?

SOLUTION :
n (n − 1) (n − 2) · · · (n − (k − 1))
= n (n − 1) (n − 2) · · · (n − k + 1)
n!
= ( Why ? )
(n − k)!

17
EXAMPLE : Three-letter words are generated randomly from the
five characters a , b , c , d , e , where letters can be used at most
once.

(a) How many three-letter words are there in the sample space S ?
SOLUTION : 5 · 4 · 3 = 60 .

(b) How many words containing a , b are there in S ?

SOLUTION : First place the characters
a,b
i.e., select the two indices of the locations to place them.
This can be done in
3 × 2 = 6 ways . ( Why ? )
There remains one position to be filled with a c , d or an e .
Therefore the number of words is 3 × 6 = 18 .

18
(c) Suppose the 60 solutions in the sample space are equally likely .

What is the probability of generating a three-letter word that

contains the letters a and b ?

SOLUTION :
18
= 0.3 .
60

19
EXERCISE :

Suppose the sample space S consists of all five-letter words

having distinct alphabetic characters .

• How many words are there in S ?

• How many ”special” words are in S for which only the second
and the fourth character are vowels, i.e., one of {a, e, i, o, u, y} ?

• Assuming the outcomes in S to be equally likely, what is the

probability of drawing such a special word?

20
Combinations

Let S be a set containing n (distinct) elements.

Then
a combination of k elements from S ,
is
any selection of k elements from S ,

where order is not important .

(Thus the selection is a set .)

NOTE : By definition a set always has distinct elements .

21
EXAMPLE :

There are three combinations of 2 elements chosen from the set

S = {a , b , c} ,

namely, the subsets

{a, b} , {a, c} , {b, c} ,

whereas there are six words of 2 elements from S ,

namely,

ab , ba , ac , ca , bc , cb .

22
In general, given
a set S of n elements ,

the number of possible subsets of k elements from S equals

 
n n!
≡ .
k k! (n − k)!

 
n
REMARK : The notation is referred to as
k
”n choose k ”.

 
n n! n!
NOTE : = = = 1,
n n! (n − n)! n! 0!

since 0! ≡ 1 (by “convenient definition” !) .

23
PROOF :

First recall that there are

n!
n (n − 1) (n − 2) · · · (n − k + 1) =
(n − k)!

possible sequences of k distinct elements from S .

However, every sequence of length k has k! permutations of itself,

and each of these defines the same subset of S.

Thus the total number of subsets is

 
n! n
≡ .
k! (n − k)! k

24
EXAMPLE :
In the previous example, with 2 elements chosen from the set

{a , b , c} ,

we have n = 3 and k = 2 , so that there are

3!
= 6 words ,
(3 − 2)!
namely
ab , ba , ac , ca , bc , cb ,
while there are
 
3 3! 6
≡ = = 3 subsets ,
2 2! (3 − 2)! 2
namely
{a, b} , {a, c} , {b, c} .

25
EXAMPLE : If we choose 3 elements from {a , b , c , d} , then
n = 4 and k = 3 ,
so there are
4!
= 24 words, namely :
(4 − 3)!

abc , abd , acd , bcd ,

acb , adb , adc , bdc ,
bac , bad , cad , cbd ,
bca , bda , cda , cdb ,
cab , dab , dac , dbc ,
cba , dba , dca , dcb ,
while there are
 
4 4! 24
≡ = = 4 subsets ,
3 3! (4 − 3)! 6
namely,
{a, b, c} , {a, b, d} , {a, c, d} , {b, c, d} .

26
EXAMPLE :

(a) How many ways are there to choose a committee of 4 persons

from a group of 10 persons, if order is not important?
SOLUTION :
 
10 10!
= = 210 .
4 4! (10 − 4)!

(b) If each of these 210 outcomes is equally likely then what is the
probability that a particular person is on the committee?
SOLUTION :
   
9 10 84 4
/ = = . ( Why ? )
3 4 210 10

Is this result surprising?

27
(c) What is the probability that a particular person is not on the
committee?
SOLUTION :
   
9 10 126 6
/ = = . ( Why ? )
4 4 210 10

Is this result surprising?

(d) How many ways are there to choose a committee of 4 persons

from a group of 10 persons, if one is to be the chairperson?
SOLUTION :
     
10 9 9 9!
= 10 = 10 = 840 .
1 3 3 3! (9 − 3)!

QUESTION : Why is this four times the number in (a) ?

28
EXAMPLE : Two balls are selected at random from a bag with
four white balls and three black balls, where order is not important.

What would be an appropriate sample space S ?

SOLUTION : Denote the set of balls by

B = {w1 , w2 , w3 , w4 , b1 , b2 , b3 } ,
where same color balls are made “distinct” by numbering them.

Then a good choice of the sample space is

S = the set of all subsets of two balls from B ,
because the wording ”selected at random ” suggests that each such
subset has the same chance to be selected.

The number of outcomes in S (which are sets of two balls) is then

 
7
= 21 .
2

29
EXAMPLE : ( continued · · · )
(Two balls are selected at random from a bag with four white balls
and three black balls.)
• What is the probability that both balls are white?
SOLUTION :    
4 7 6 2
/ = = .
2 2 21 7

• What is the probability that both balls are black?

SOLUTION :    
3 7 3 1
/ = = .
2 2 21 7

• What is the probability that one is white and one is black?

SOLUTION :      
4 3 7 4·3 4
/ = = .
1 1 2 21 7
(Could this have been computed differently?)

30
EXAMPLE : ( continued · · · )
In detail, the sample space S is
n
{w1 , w2 }, {w1 , w3 }, {w1 , w4 }, | {w1 , b1 }, {w1 , b2 }, {w1 , b3 },
{w2 , w3 }, {w2 , w4 }, | {w2 , b1 }, {w2 , b2 }, {w2 , b3 },
{w3 , w4 }, | {w3 , b1 }, {w3 , b2 }, {w3 , b3 },
| {w4 , b1 }, {w4 , b2 }, {w4 , b3 },
———– ———– ———–
{b1 , b2 }, {b1 , b3 }, o
• S has 21 outcomes, each of which is a set . {b2 , b3 }
1
• We assumed each outcome of S has probability 21
.
• The event ”both balls are white” contains 6 outcomes.
• The event ”both balls are black” contains 3 outcomes.
• The event ”one is white and one is black” contains 12 outcomes.
• What would be different had we worked with sequences ?

31
EXERCISE :

Three balls are selected at random from a bag containing

2 red , 3 green , 4 blue balls .

What would be an appropriate sample space S ?

What is the the number of outcomes in S ?

What is the probability that all three balls are red ?

What is the probability that all three balls are green ?

What is the probability that all three balls are blue ?

What is the probability of one red, one green, and one blue ball ?

32
EXAMPLE : A bag contains 4 black balls and 4 white balls.
Suppose one draws two balls at the time, until the bag is empty.
What is the probability that each drawn pair is of the same color?

SOLUTION : An example of an outcome in the sample space S is

n o
{w1 , w3 } , {w2 , b3 } , {w4 , b1 } , {b2 , b4 } .

The number of such doubly unordered outcomes in S is

    
1 8 6 4 2 1 8! 6! 4! 2! 1 8!
= = = 105 (Why?)
4! 2 2 2 2 4! 2! 6! 2! 4! 2! 2! 2! 0! 4! (2!)4

The number of such outcomes with pairwise the same color is

     
1 4 2 1 4 2
· = 3 · 3 = 9. ( Why ? )
2! 2 2 2! 2 2

Thus the probability each pair is of the same color is 9/105 = 3/35 .

33
EXAMPLE : ( continued · · · )
The 9 outcomes of pairwise the same color constitute the event
(
n o
{w1 , w2 } , {w3 , w4 } , {b1 , b2 } , {b3 , b4 } ,
n o
{w1 , w3 } , {w2 , w4 } , {b1 , b2 } , {b3 , b4 } ,
n o
{w1 , w4 } , {w2 , w3 } , {b1 , b2 } , {b3 , b4 } ,
n o
{w1 , w2 } , {w3 , w4 } , {b1 , b3 } , {b2 , b4 } ,
n o
{w1 , w3 } , {w2 , w4 } , {b1 , b3 } , {b2 , b4 } ,
n o
{w1 , w4 } , {w2 , w3 } , {b1 , b3 } , {b2 , b4 } ,
n o
{w1 , w2 } , {w3 , w4 } , {b1 , b4 } , {b2 , b3 } ,
n o
{w1 , w3 } , {w2 , w4 } , {b1 , b4 } , {b2 , b3 } ,
)
n o
{w1 , w4 } , {w2 , w3 } , {b1 , b4 } , {b2 , b3 } .

34
EXERCISE :

• How many ways are there to choose a committee of 4 persons

from a group of 6 persons, if order is not important?
• Write down the list of all these possible committees of 4 persons.
• If each of these outcomes is equally likely then what is the
probability that two particular persons are on the committee?

EXERCISE :

Two balls are selected at random from a bag with three white balls
and two black balls.

• Show all elements of a suitable sample space.

• What is the probability that both balls are white?

35
EXERCISE :

We are interested in birthdays in a class of 60 students.

• What is a good sample space S for this purpose?

• How many outcomes are there in S ?

• What is the probability of no common birthdays in this class?

• What is the probability of common birthdays in this class?

36
 111|11111111|1111|11 .

The total number of positions is

17 + 3 = 20 .

The total number of nonnegative solutions is therefore

 
20 20! 20 × 19 × 18
= = = 1140 .
3 (20 − 3)! 3! 3×2

39
 CONDITIONAL PROBABILITY

Giving more information can change the probability of an event.

EXAMPLE :
If a coin is tossed two times then what is the probability of two
Heads?
ANSWER : 1
.
4

EXAMPLE :
If a coin is tossed two times then what is the probability of two Heads,
given that the first toss gave Heads ?
ANSWER : 1
.
2

45
NOTE :

Several examples will be about playing cards .

A standard deck of playing cards consists of 52 cards :

• Four suits :
Hearts , Diamonds (red ) , and Spades , Clubs (black) .

• Each suit has 13 cards, whose denomination is

2 , 3 , · · · , 10 , Jack , Queen , King , Ace .

• The Jack , Queen , and King are called face cards .

46
EXERCISE :

Suppose we draw a card from a shuffled set of 52 playing cards.

• What is the probability of drawing a Queen ?

• What is the probability of drawing a Queen, given that the card

drawn is of suit Hearts ?

• What is the probability of drawing a Queen, given that the card

drawn is a Face card ?

What do the answers tell us?

(We’ll soon learn the events ”Queen” and ”Hearts” are independent .)

47
The two preceding questions are examples of conditional probability .

Conditional probability is an important and useful concept.

If E and F are events, i.e., subsets of a sample space S , then

P (E|F ) is the conditional probability of E , given F ,

defined as
P (EF )
P (E|F ) ≡ .
P (F )
or, equivalently

P (EF ) = P (E|F ) P (F ) ,

(assuming that P (F ) is not zero).

48
 P (EF )
P (E|F ) ≡
P (F )

E
S S E
F
F

Suppose that the 6 outcomes in S are equally likely.

What is P (E|F ) in each of these two cases ?

49
 P (EF )
P (E|F ) ≡
P (F )

S E S F

Suppose that the 6 outcomes in S are equally likely.

What is P (E|F ) in each of these two cases ?

50
EXAMPLE : Suppose a coin is tossed two times.

The sample space is

S = {HH , HT , T H , T T } .

Let E be the event ”two Heads ” , i.e.,

E = {HH} .

Let F be the event ”the first toss gives Heads ” , i.e.,

F = {HH , HT } .
Then
EF = {HH} = E ( since E ⊂ F ) .

We have
1
P (EF ) P (E) 4 1
P (E|F ) = = = 2 = .
P (F ) P (F ) 4
2

51
EXAMPLE :
Suppose we draw a card from a shuffled set of 52 playing cards.
• What is the probability of drawing a Queen, given that the card
drawn is of suit Hearts ?
ANSWER :
1
P (QH) 52 1
P (Q|H) = = 13 = .
P (H) 52
13

• What is the probability of drawing a Queen, given that the card

drawn is a Face card ?
ANSWER :
4
P (QF ) P (Q) 52 1
P (Q|F ) = = = 12 = .
P (F ) P (F ) 52
3

(Here Q ⊂ F , so that QF = Q .)

52
The probability of an event E is sometimes computed more easily

if we condition E on another event F ,

namely, from

P (E) = P ( E(F ∪ F c ) ) ( Why ? )

= P ( EF ∪ EF c ) = P (EF ) + P (EF c ) ( Why ? )

and

P (EF ) = P (E|F ) P (F ) , P (EF c ) = P (E|F c ) P (F c ) ,

we obtain this basic formula

P (E) = P (E|F ) · P (F ) + P (E|F c ) · P (F c ) .

53
EXAMPLE :

An insurance company has these data :

The probability of an insurance claim in a period of one year is

4 percent for persons under age 30

2 percent for persons over age 30

and it is known that

30 percent of the targeted population is under age 30.

What is the probability of an insurance claim in a period of one year

for a randomly chosen person from the targeted population?

54
SOLUTION :

Let the sample space S be all persons under consideration.

Let C be the event (subset of S) of persons filing a claim.

Let U be the event (subset of S) of persons under age 30.

Then U c is the event (subset of S) of persons over age 30.

Thus
P (C) = P (C|U ) P (U ) + P (C|U c ) P (U c )
4 3 2 7
= +
100 10 100 10
26
= = 2.6% .
1000

55
EXAMPLE :

Two balls are drawn from a bag with 2 white and 3 black balls.

There are 20 outcomes (sequences) in S . ( Why ? )

What is the probability that the second ball is white ?

SOLUTION :

Let F be the event that the first ball is white.

Let S be the event that the second second ball is white.

Then
c c 1 2 2 3 2
P (S) = P (S|F ) P (F ) + P (S|F ) P (F ) = · + · = .
4 5 4 5 5

QUESTION : Is it surprising that P (S) = P (F ) ?

56
EXAMPLE : ( continued · · · )
Is it surprising that P (S) = P (F ) ?

ANSWER : Not really, if one considers the sample space S :

n
w 1 w2 , w1 b 1 , w1 b 2 , w1 b 3 ,

w2 w1 , w2 b 1 , w2 b 2 , w2 b 3 ,

b 1 w1 , b 1 w2 , b1 b2 , b1 b3 ,

b 2 w1 , b 2 w2 , b2 b1 , b2 b3 ,
o
b 3 w1 , b 3 w2 , b3 b1 , b3 b2 ,

where outcomes (sequences) are assumed equally likely.

57
EXAMPLE :

Suppose we draw two cards from a shuffled set of 52 playing cards.

What is the probability that the second card is a Queen ?

ANSWER :

P (2nd card Q) =

P (2nd card Q|1st card Q) · P (1st card Q)

+ P (2nd card Q|1st card not Q) · P (1st card not Q)

3 4 4 48 204 4 1
= · + · = = = .
51 52 51 52 51 · 52 52 13

QUESTION : Is it surprising that P (2nd card Q) = P (1st card Q) ?

58
A useful formula that ”inverts conditioning ” is derived as follows :

Since we have both

P (EF ) = P (E|F ) P (F ) ,

and
P (EF ) = P (F |E) P (E) .

If P (E) 6= 0 then it follows that

P (EF ) P (E|F ) · P (F )
P (F |E) = = ,
P (E) P (E)
and, using the earlier useful formula, we get
P (E|F ) · P (F )
P (F |E) = ,
P (E|F ) · P (F ) + P (E|F c ) · P (F c )

which is known as Bayes’ formula .

59
EXAMPLE : Suppose 1 in 1000 persons has a certain disease.
A test detects the disease in 99 % of diseased persons.
The test also ”detects” the disease in 5 % of healthly persons.
With what probability does a positive test diagnose the disease?

SOLUTION : Let
D ∼ ”diseased” , H ∼ ”healthy” , + ∼ ”positive”.
We are given that
P (D) = 0.001 , P (+|D) = 0.99 , P (+|H) = 0.05 .

By Bayes’ formula
P (+|D) · P (D)
P (D|+) =
P (+|D) · P (D) + P (+|H) · P (H)

0.99 · 0.001 ∼
= = 0.0194 (!)
0.99 · 0.001 + 0.05 · 0.999

60
EXERCISE :
Suppose 1 in 100 products has a certain defect.

A test detects the defect in 95 % of defective products.

The test also ”detects” the defect in 10 % of non-defective products.

• With what probability does a positive test diagnose a defect?

EXERCISE :
Suppose 1 in 2000 persons has a certain disease.

A test detects the disease in 90 % of diseased persons.

The test also ”detects” the disease in 5 % of healthly persons.

• With what probability does a positive test diagnose the disease?

61
More generally, if the sample space S is the union of disjoint events
S = F1 ∪ F2 ∪ · · · ∪ Fn ,
then for any event E
P (E|Fi ) · P (Fi )
P (Fi |E) = .
P (E|F1 ) · P (F1 ) + P (E|F2 ) · P (F2 ) + · · · + P (E|Fn ) · P (Fn )

EXERCISE :
Machines M1 , M2 , M3 produce these proportions of a article

Production : M1 : 10 % , M2 : 30 % , M3 : 60 % .

The probability the machines produce defective articles is

Defects : M1 : 4 % , M2 : 3 % , M3 : 2 % .

What is the probability a random article was made by machine M1 ,

given that it is defective?

62
Independent Events

Two events E and F are independent if

P (EF ) = P (E) P (F ) .

In this case
P (EF ) P (E) P (F )
P (E|F ) = = = P (E) ,
P (F ) P (F )

(assuming P (F ) is not zero).

Thus

knowing F occurred doesn’t change the probability of E .

63
EXAMPLE : Draw one card from a deck of 52 playing cards.
Counting outcomes we find
12 3
P (Face Card) = 52
= 13
,

13 1
P (Hearts) = 52
= 4
,

3
P (Face Card and Hearts) = 52
,

3
P (Face Card|Hearts) = 13
.
We see that
3
P (Face Card and Hearts) = P (Face Card) · P (Hearts) (= ).
52
Thus the events ”Face Card ” and ”Hearts ” are independent.

Therefore we also have

3
P (Face Card|Hearts) = P (Face Card) (= ).
13

64
EXERCISE :

Which of the following pairs of events are independent?

(1) drawing ”Hearts” and drawing ”Black” ,

(2) drawing ”Black” and drawing ”Ace” ,

(3) the event {2, 3, · · · , 9} and drawing ”Red” .

65
EXERCISE : Two numbers are drawn at random from the set
{1, 2, 3, 4}.

If order is not important then what is the sample space S ?

Define the following functions on S :

X( {i, j} ) = i + j , Y ( {i, j} ) = |i − j| .

Which of the following pairs of events are independent?

(1) X = 5 and Y = 2 ,

(2) X = 5 and Y = 1 .

REMARK :
X and Y are examples of random variables . (More soon!)

66
EXAMPLE : If E and F are independent then so are E and F c .

PROOF : E = E(F ∪ F c ) = EF ∪ EF c , where

EF and EF c are disjoint .
Thus
P (E) = P (EF ) + P (EF c ) ,
from which

P (EF c ) = P (E) − P (EF )

= P (E) − P (E) · P (F ) (since E and F independent)

= P (E) · ( 1 − P (F ) )

= P (E) · P (F c ) .

EXERCISE :
Prove that if E and F are independent then so are E c and F c .

67
NOTE : Independence and disjointness are different things !

E
S S E
F
F

Independent, but not disjoint. Disjoint, but not independent.

(The six outcomes in S are assumed to have equal probability.)

If E and F are independent then P (EF ) = P (E) P (F ) .

If E and F are disjoint then P (EF ) = P ( ∅ ) = 0 .

If E and F are independent and disjoint then one has zero probability !

68
Three events E , F , and G are independent if

P (EF G) = P (E) P (F ) P (G) .

and
P (EF ) = P (E) P (F ) .
P (EG) = P (E) P (G) .
P (F G) = P (F ) P (G) .

EXERCISE : Are the three events of drawing

(1) a red card ,

(2) a face card ,

(3) a Heart or Spade ,

independent ?

69
EXERCISE :

A machine M consists of three independent parts, M1 , M2 , and M3 .

Suppose that
9
M1 functions properly with probability 10
,

9
M2 functions properly with probability 10
,

8
M3 functions properly with probability 10
,
and that

the machine M functions if and only if its three parts function.

• What is the probability for the machine M to function ?

• What is the probability for the machine M to malfunction ?

70