SIT Internal

BCD1002 QA

Lecture 2:
Probability Theory
(Understanding Uncertainties in Life)
w
 SIT Internal

Basic Probability Concepts

Learning Objectives:

• Be familiar with terminologies used in basic probability

theory (e.g., experiment, simple event, event, sample space,
etc.)
• Construct a new event using union or intersection of two or
more events
• Be familiar with the concepts of mutual exclusivity and
independence
• Be familiar with the concept of conditional probability

Prepared by: Lee Ee Lian

 SIT Internal

What is Probability?

In general, probability is the chance that

something will happen.
 SIT Internal

Probabilities of Events

Mathematically, probability is defined as a number between 0

to 1 (inclusive) that is assigned to a random outcome of
uncertainties.

4
Experiment
Definition
An experiment is any activity or process whose
outcome is subject to uncertainty. Examples:
▪ selecting a card or cards from a deck
▪ tossing a coin once or several times
▪ surveying
▪ sampling guests/passengers of particular hotel/flight

Note: although the word experiment generally suggests a planned or carefully

controlled activity of laboratory testing
Outcome
Definition
A particular result of an experiment.
Examples:
▪ Selecting a “Queen” card from a deck
▪ Observing 3 heads when tossing a coin 3 times
▪ Surveying a person age > 50 years old
▪ Observing a dissatisfied customer from a particular flight
7

Simple Event
Definition
A simple event is the outcome that is observed on a
single repetition of the experiment.

▪ One and only one simple event can occur when the
experiment is performed.

▪ A simple event is denoted by E with a subscript

8

Event & Sample Space

Definition
An event is a collection of simple events

▪ Simple event is a direct outcome of the experiment

▪ Event is usually not a direct outcome of the experiment, but it
is related to the simple event(s) (or direct outcome(s))

Definition
The sample space of an experiment, denoted by S, is the set of all
possible simple events (all possible outcomes) of that experiment.
9

Example – Toss a Die

▪ Simple events:
(possible outcomes)
1 E1
Sample space:
2
E2 S ={E1, E2, E3, E4, E5, E6}
3 E3
S
4
•E1 •E3
E4
•E5
5
E5
•E2 •E4 •E6
6 E6
0

Example – Toss a Die

▪ Remember an event is a collection of simple
events
▪ What is the event (A) that we get an odd
number?
A ={E1, E3, E5}
S
•E1 •E3
•E5 A
Venn Diagram
•E2 •E4 •E6 for die tossing
1

Example – Toss a Die

▪ How about the event (B) that we get a
number greater than 2?

B ={E3, E4, E5, E6}

S
•E1 •E3
A •E5
•E2 •E4 •E6
Venn Diagram for
B die tossing
 SIT Internal

Probabilities of Events

Classical Probability
If each of the n outcomes is equally likely, then the probability of
a random outcome E is 1
P( E ) =
n
where n : total number of possible outcomes.
What is probability of throwing an E?

E 1 2 3 4 5 6
P(E) 1/6 1/6 1/6 1/6 1/6 1/6

12
 SIT Internal

Probabilities of Events

Eg. What is the probability of throwing an odd number?

Denote A = event of throwing 1, 3 or 5.
P(A) =

Eg. What is the probability of throwing a number larger than 4?

Denote B = event of throwing 5 or 6.
P(B) =

Eg. What is the probability of throwing either an odd or even

number?
Denote C = event of throwing 1,2,3,4,5 or 6.
P(C) =
13
 SIT Internal

RECAP: Probabilities of Events

Sample space S is defined as the set of all possible outcomes of a

random phenomenon, typically drawn using a Venn diagram with a
rectangle.
S
S = 1, 2, 3, 4, 5, 6
The sample space when tossing a die is S = 1, 2, 3, 4, 5, 6
1 2 3
Simple events: one unique event. Eg. A={4}
4 5 6

Events: an event made up of 2 or more simple events. Eg. B={4,5,6}

1 2 3

4 5 6
14
Characteristics of Probability for any event (A)
▪ 𝑃(𝐴) must be between 0 and 1

0 ≤𝑃 𝐴 ≤ 1
▪ 𝑃(𝐴) = 0 if the event A never occurs (A is an empty set)

▪ 𝑃(𝐴) = 1 if the event A always occurs (sure event)

▪ The closer 𝑃(𝐴) is to 1, the more likely that event A will occur

▪ Sum of the probabilities of all simple events in S equals 1 (i.e.,

𝑃 𝑆 =𝟏)
 Event Relations and Probability Rules
▪ Sometimes the event of interest can be formed as a
combination of several other events. Let 𝐴 and 𝐵 be two
events defined on sample space 𝑆. There are three
important relationships between the events:

1. The union of events 𝐴 or 𝐵, denoted by AB

2. The intersection of events 𝐴 and 𝐵, denoted by AB
3. The complement of an event 𝐴, denoted by Ac, which
is the event that A does not occur
Union of Events
▪ The union of two events, A and B, denoted by AB, is
the event that either A or B or both occur when the
experiment is performed.
S

Venn
A B diagram

Rules of Addition
▪ For any two events, A and B, the probability of their
union, P(AB), is
P( A  B) = P( A) + P( B) − P( A  B)
Mutually Exclusive Events
▪ If two events A and B are mutually exclusive (i.e.,
no simple events in common), then A  B = (null set)
and P(A  B) = 0.
Then P(A  B) = P(A) + P(B)

A B Venn
diagram
 SIT Internal

Mutually Exclusive Events - EXAMPLE

Two events are mutually exclusive or disjoint if they cannot both

occur. P( A  B) = 0

B
A

E. g. What is the probability of throwing 1 and 3?

A = event of throwing 1 and 3.
P(A) = P(E1 ∩ E3)= 0
19
 SIT Internal

Probabilities of Events (Union)

Eg. What is the probability of throwing an odd

number? 1 2 3
A = event of throwing 1 or 3 or 5.
4 5 6
P(A) = P(E1 U E3 U E5)

Eg. What is the probability of throwing a number

1 2 3
larger than 4?
B = event of throwing 5 or 6. 4 5 6
P(B) = P(E5 U E6)

Eg. What is the probability of throwing either an

odd or even number? 1 2 3
C = event of throwing 1 or 2 or 3 or 4 or 5 or 6. 4 5 6
20
P(C) = P(E1 U E2 U E3 U E4 U E5 U E6)
 SIT Internal

Probabilities of Events (Union)

Eg. What is the probability of throwing an odd

number? 1 2 3
A = event of throwing 1 or 3 or 5.
4 5 6
P(A) = P(E1 U E3 U E5)= (1/6+1/6+1/6)= 1/2

Eg. What is the probability of throwing a number

1 2 3
larger than 4?
B = event of throwing 5 or 6. 4 5 6
P(B) = P(E5 U E6) = (1/6+1/6)=1/3

Eg. What is the probability of throwing either an

odd or even number? 1 2 3
C = event of throwing 1 or 2 or 3 or 4 or 5 or 6. 4 5 6
21
P(C) = P(E1 U E2 U E3 U E4 U E5 U E6) = 1
 SIT Internal

Probabilities of Events (Intersection)

Eg. What is the probability of throwing 1 and 3?

A = event of throwing 1 and 3. 1 2 3
P(A) = P(E1 ∩ E3)
4 5 6

Eg. What is the probability of throwing an odd

number which larger than 4?
1 2 3
B = event of throwing (1 or 3 or 5) and (5 or 6).
P(B) = P( (E1 U E3 U E5) ∩( E5 U E6)) = P(E5) 4 5 6

Eg. What is the probability of throwing 1 and 2

and 3 and 4 and 5 and 6?
C = event of throwing 1 and 2 and 3 and 4 and 5 1 2 3
and 6. P(C) = P(E1 ∩ E2 ∩ E3 ∩ E4 ∩ E5 ∩ E6) 4 5 6
22
 SIT Internal

Probabilities of Events (Intersection)

Eg. What is the probability of throwing 1 and 3?
A = event of throwing 1 and 3.
1 2 3
P(A) = P(E1 ∩ E3)= 0
P(E1 ∩ E3)=P(E1) +P(E3) –P(E1 U E3) =1/6 +1/6-2/6=0 4 5 6
( if using addition rule)

Eg. What is the probability of throwing an odd

1 2 3
number which larger than 4?
B = event of throwing (1 or 3 or 5) and (5 or 6). 4 5 6
P(B) = P( (E1 U E3 U E5) ∩( E5 U E6)) = P(E5) = 1/6

Eg. What is the probability of throwing 1 and 2

and 3 and 4 and 5 and 6? 1 2 3
C = event of throwing 1 and 2 and 3 and 4 and 5
4 5 6
and 6. P(C) = P(E1 ∩ E2 ∩ E3 ∩ E4 ∩ E5 ∩ E6)= 0 23
 SIT Internal

Probabilities of Events (Complement)

Definition: The probability of a complementary event Ac is

P( Ac ) = 1 − P( A) S
Ac
The probability of a complementary event is the
A
probability that the event does not happen.
Example:
• Tossing a die, sample space
S = 1, 2, 3, 4, 5, 6
• Event A getting an odd number A =  1, 3, 5

Probability of NOT getting an odd number :

1 1
P( Ac ) = 1 − P( A) = 1 − =
2 2
24
Example
Select a student at random
from the classroom of 120.

A: male Brown Not Brown

P(A) = 60/120 Male 20 40
B: female Female 30 30

A and B are P(B) = 1- P(A)

complementary, so = 1- 60/120 = 60/120
that
Using Rule of Addition - Exercise
A sample of 200 tourists in Singapore shows 120 went to Sentosa,
100 went to Botanic Gardens, and 60 visited both.

What is the probability of tourists who went to Sentosa or Botanic

Gardens?
 SIT Internal

Conditional Probability

Conditional probability is the updated probability of an event A

given that another event B has occurred.
“given”

P( A  B)
P( A | B) =
P( B)
S
“Given that B has already
A occurred” = “Given that B is
B now the new (updated) Sample
space”

27
 SIT Internal

Conditional Probability
Example: given an even number, what is the probability of getting a 6?
• Tossing a die, sample space
S = 1, 2, 3, 4, 5, 6 S
B
A =  6
A6
• Event A is getting a ‘6’ 2, 4
• Event B is getting an even number
B = 2, 4, 6
1
Intuitively, from the diagram, P( A | B) =
3
Or we can use conditional probability:
P( A  B)
P( A | B) = = (1/6)/(1/2) = 1/3
P( B)
28
 SIT Internal

Conditional Probability
Example: Note that the P(A|B)
• Tossing a die, sample space and P(B|A) represent
S = 1, 2, 3, 4, 5, 6 different probabilities
• Event A is getting a number larger than 4: A={5,6} and are NOT the same!
• Event B is getting an even number: B={2,4,6}
P( A  B) S
P( A | B) =
P( B)
A B
P( B  A) 1
P( B | A) = = 6
=1
P( A) 1 5
6 66 2
 P( A | B) 4
P( A  B) = P( A | B) P( B)
P( A  B) = P( B | A) P( A)
29
 SIT Internal

Conditional Probability
Example: Note that the P(A|B)
• Tossing a die, sample space and P(B|A) represent
S = 1, 2, 3, 4, 5, 6 different probabilities
• Event A is getting a number larger than 4: A={5,6} and are NOT the same!
• Event B is getting an even number: B={2,4,6}
P( A  B) S
P( A | B) = = (1/6)/(1/2) = 1/3
P( B)
A B
P( B  A) 16
P( B | A) = = (1/6)/(1/3)
=1 = 1/2
P( A) 1 5
6 66 2
 P( A | B) 4

P( A  B) = P( A | B) P( B) = (1/3)*(1/2) = 1/6

P( A  B) = P( B | A) P( A) = (1/2)*(1/3) = 1/6 30
 SIT Internal

Conditional Probability

Since
P( A  B) P( A  B)
P( A | B) = and P( B | A) =
P( B) P( A)

We have:

Rules of Multiplication

P( A  B) = P( A | B) P( B)

P( A  B) = P( B | A) P( A)

These are known as the general multiplication rules of probability. 31

Rule of Multiplication - Exercise
A golfer has 12 golf shirts in his closet. Suppose 9 of these shirts are
white and the others are blue. He gets dressed in the dark, so he just
grabs a shirt and puts in on. He plays golf two days in a row and does
not return the shirts to the closet. What is the probability both shirts
are white?
9 8
P(W1 and W2) = P(W1)P(W2|W1) = ( )( ) = .55
12 11
 SIT Internal

Dependent Events
= The occurrence of one event has an effect on the probability of the occurrence
of another event.

Example: P(A|B) depends on P(B)

• Tossing a die, sample space S
S = 1, 2, 3, 4, 5, 6
A B
• Event A is getting a number larger than 4: A={5,6}
5
• Event B is getting an even number: B={2,4,6} 6 2
P( A  B) 4
P( A | B) = = (1/6)/(1/2) = 1/3 = P(A)!!!
P( B)
S
• If event B is getting an even number larger
than 2: B={4,6} A
B
P( A  B) 5
P( A | B) = = (1/6)/(1/3) = 1/2 6
P( B) 4

33
 Independent Events
Definition
▪ Two events, A and B, are said to be independent if and only if the
probability of event A is not influenced or changed by the occurrence
of event B, or vice versa.
▪ Example: toss a single die two times. Define two events:
A: observe a 2 on the first toss
B: observe a 2 on the second toss
INDEPENDENCE The occurrence of one event has no effect on the probability of
the occurrence of another event.

Regardless of whether event A has or has not occurred, probability of

observing a 2 on the second toss is still 1/6.
𝑃 𝐵 given that 𝐴 occurred = 1Τ6
𝑃 𝐵 given that 𝐴 did not occur = 1Τ6
Thus, A and B are independent events.
 SIT Internal

Independent Events

Definition: For independent events A and B

P( A | B) = P( A) and P( B | A) = P( B)

If the conditional probability of A given that B has occurred remains

unchanged as the original probability of A, then A and B are said to
be independent events.

Thus, P 𝐴 ∩ 𝐵 = 𝑃(𝐴) ∙ 𝑃(𝐵).

35
 SIT Internal

Independent Events

Definition: For independent events A and B

P( A | B) = P( A) and P( B | A) = P( B)

Example: consider tossing two fair dice

1
• Event A is getting a ‘6’ on the first die P ( A) =
6
1
• Event B is getting also a ‘6’ on the second die P( B) =
6
Intuitively, probability of getting a ‘6’ on the second die should not be
influenced by the outcome of the first dice, i.e. P( B | A) = P(B) = 1
6
P( A  B) = P( B | A) P( A) = P( B) P( A)
1
P( A  B) =
1 1
x =
6 6 36 36
 SIT Internal

Independent Events

For independent events,

P( A | B) = P( A) and P( B | A) = P( B)
We have:
P( A  B) = P( A | B) P( B) = P( A) P( B)

P( A  B) = P( B | A) P( A) = P( B) P( A)

These are known as the multiplication rule of independent events.

In fact, the multiplication rules can be extended to more than two
independent events:

P( A1  A2  ...  An ) = P( A1 ) P( A2 )...P( An )
37
 SIT Internal

Independent Events

Example: Consider tossing two fair coins. What is the probability that
both will be heads? Define A: first coin is Head. B: second coin is Head.

Tree diagram:

P(both heads)= P(A ∩B)

= 0.25
Since A and B is independent, P(A ∩ B) = P(A)P(B) = 0.5*0.5 = 0.25 38
 SIT Internal

Independent Events

Example: Consider tossing two fair coins. What is the probability of getting
at least one head? Define A: first coin is Head. B: second coin is Head.

Tree diagram:

P(at least one head)= P ( A  B ) = P ( A) + P ( B ) − P ( A  B )

39
 SIT Internal

Independent Events

Example: Consider tossing two fair coins. What is the probability of getting
at least one head? Define A: first coin is Head. B: second coin is Head.

Tree diagram:

P(at least one head)= P ( A  B ) = P ( A) + P ( B ) − P ( A  B )

P( A  B) = 0.5 + 0.5 − 0.25 = 0.75 40
 SIT Internal

Mutually Exclusive events vs Independent Events

• Mutually exclusive events will not occur at the same time.

• Independent events will not affect the probabilities of one another.

41
Exercise
If 60 % of Singaporeans consumes health supplements regularly, find
the probability that for a sample of 5 Singaporeans, all 5 consume
health supplements regularly. Find the probability that none consumes
of the 5 health supplements regularly.
 SIT Internal

Summary

Probabilities of events: P( A)

Conditional probabilities: Probability of event A will happen

given event B happens P( A | B)

Mutually exclusive events: if A happens, B will never happen

P( A  B) = 0

Independent events: the probability of event A happens are

not affected by event B P( A | B) = P( A)

Multiplicative rules of probabilities: applies for all cases

P( A  B) = P( A | B) P( B)
43
SUMMARY of Addition and Multiplication Rules
 SIT Internal

Tutorial Q1

a) Please give an example of mutually exclusive events. Define them in terms of

probability.
b) Please give an example of independent events. Define them in terms of probability.
c) If two events are mutually exclusive, are they independent?

45
 SIT Internal

Tutorial Q2

In a University comprising 1000 students, 500 use MRT to come to class and 300 use bus
to come to class. Given that 100 come to class by both MRT and bus, how many come to
class by means that do not involve MRT or bus?

46
 SIT Internal

Tutorial Q3
3 yellow and 4 green marbles are in a bag. A child is asked to randomly pick 2 marbles
without replacement.
a) What is the probability that the child picks a yellow marble on the second pick
(event B), given that she has already picked a green marble (event A)?
b) What is the probability that the child picks a green marble on the second pick
(event B’), given that she has already picked a green marble (event A)?
c) What is the probability that the child picks a green marble first and a yellow
marble on the second pick?
d) What is the probability that the child picks two green marbles?

47
 SIT Internal

Tutorial Q4
City residents were surveyed recently to determine readership of newspapers. 50% of the
residents read the morning paper, 60% read the evening paper and 20% read both
newspapers.
(a) Find the probability that a resident selected at random reads the morning or evening
paper.
(b) What is the probability of reading the evening paper given that the morning paper has
been read?

48
 SIT Internal

Tutorial Q5
Jane is in a class of 20 students (including her) that meets daily. Each day for a week
(Monday to Friday), a student in Jane’s class is randomly selected to answer a homework
problem. Once a student has been selected, he or she is not selected again for that week.
a) Given that Jane was not one of the four students selected earlier in the week, what is
the probability that she will be picked on Friday?
b) What is the probability that Jane is not picked up for the week?

49
 SIT Internal

Tutorial Q6
In a survey of 1929 students, the following data were obtained on “students’ first reason for
application to the university in which they matriculated.”
Reason for Application
Enrollment University University Cost Other Total
status Quality
or Convenience
Full-time 421 393 76 890
Part-time 400 593 46 1039
Total 821 986 122 1929

(a) What is the probability that university quality (event B) is the first reason for a student to
choose a university?
(b) For a full-time student (event A), what is the probability that university quality is the first
reason for choosing a university?
(c) Let A denote the event that a student is full-time, and let B denote the event that the
student lists university quality as the first reason for applying. Are events A and B
independent? Justify your answer.

50
 SIT Internal

Tutorial Q7
An investor holds two stocks, each of which can rise (R), remain unchanged (U), or decline
(D) on any day. Suppose that on a particular day, the first stock has the probabilities P(R) =
0.4, P(U) = 0.2, P(D) = 0.4, and the second stock has the probabilities P(R) = 0.8, P(U) = 0.1,
P(D) = 0.1. Assume that these stocks move independently.
a) Find the probability that both stocks decline.
b) Find the probability that exactly one stock rises.

51