Chapter 1, 2 of 3rd ed.

Chapter 1
Experiments, Models, and
Probabilities
Chien-Kang Huang ( 黃乾綱 )
臺大工科海洋系

1
 Before the Class
• Read the “A Message to Students from the Authors” in
Preface.

2
 Outline
1.1 Set Theory
1.2 Applying Set Theory to Probability (1.1, 1.4)
1.3 Probability Axioms (1.2)
1.4 Some Consequences of the Axioms (1.2)
1.5 Conditional Probability (1.3)
1.6 Independence (1.5)
1.7 Sequential Experiments and Tree Diagrams (2.1)
1.8 Counting Method (2.2)
1.9 Independent Trials (2.3)
1.10 Reliability Problems

3
 Getting Started with Probability
• The probability of an event is a number between 0 and 1.
– The proportion of times we expect a certain thing to happen.

• Two interpretation of probability:

– A physical property
– Related to the prior knowledge

• Difference between probability theory and physics view point.

– Probability theory – describe phenomena that cannot be predicted with
certainty.
– Physics – do the same thing in the same way, the result will always be
the same.
• Probability: While each outcome may be unpredictable, there
are consistent patterns to be observed when we repeat the
procedure a large number of times.

4
 Three purposes served by this book
• It introduces students to the logic of probability theory
• It helps students develop intuition into how the theory applies
to practical situations.
• It teaches students how to apply probability theory to solving
engineering problems.

5
 Logic of the subject
• Definitions:
– Establish the logic of probability theory

• Axioms:
– facts that we accept without proof.

• Theorems:
– Consequences that follow logically from definitions and axioms
– Each theorem has a proof.

• There are only 3 axioms of probability theory. These three

axioms are the foundation on which the entire subject rests.
– Dozens of Definitions  3 Axioms  Dozens of Theorems

6
1.1 Set Theory

7
 Terminologies and Symbols
• Terms:
– set, element, subset, universal set, null set (),
union, intersection, complement, differences,
mutually exclusive, collectively exhaustive.

• Symbols:
Symbols Usage
 xA
 cA
{} {all the students in this room}
| C = {x2 | x = 1, 2, 3, 4, 5}
… D ={x2 | x = 1, 2, 3, 4, 5, …}
 CD
= A = B iff B  A and A  B
 For any set A,   A 8
 Venn Diagram

AB

• if and only if, iff:

– Symbols ,  , 

– http://en.wikipedia.org/wiki/If_and_only_if

9
 Venn Diagram

AB AB
A

Ac

union intersection complement

x A B x A B
  x  Ac  x  A
x  A or x  B x  A and x  B

Three operations

10
 Venn Diagram

A–B A1 A2
A
A5
B
A4 A3

difference (n sets) mutually exclusive, collectively exhaustive

(2 sets) disjoint
x A B
 

Ai  A j   , i  j. A1  A2    An  S
x  A and x  B

A  B  A  BC
BC  S  B 11
 Shorthands
• Shorthand for unions and intersections of n sets:

A  A  A
i 1
i 1 2   An
n

A  A  A
i 1
i 1 2   An

12
Theorem 1.1 De Morgan’s Law

13
Quiz 1.1

14
Quiz 1.1 Solution

15
1.1

1.2 Applying Set Theory to

Probability

16
Page 4

Terminologies
• Probability is a number that describe a set. The higher the
number, the more probability there is.
• The basic model is a repeatable experiment. An experiment
consists of a procedure and observations. There is
uncertainty in what will be observed.
• For the most part, we will analyze models of actual physical
experiments. We create models because real experiments
generally are too complicated to analyze.
• We often will use the word experiment to refer to the model of
an experiment.

17
Example 1.1 ~ 1.3

18
Definition 1.1, 1.2, 1.3

Definition 1.1 ~ 1.4

Set Algebra Probability

Set Event
Universal set Sample space
Element Outcome 19
Example 1.4

20
Example 1.5

21
Example 1.6

22
Example 1.7

23
 Example 1.8

The set of events {G2, R2} is both mutually exclusive and collectively exhaustive.
However, {G2, R2} is not a sample space for the experiment because the
elements of the set do not completely describe the set of possible outcomes of
the experiment. The set {G2, R2} does not have the finest-grain property.
24
Example 1.9

25
Example 1.10

26
Theorem 1.7, Figure 1.1, Page 15

Theorem 1.2 Mutual Exclusive of Event Space

27
Example 1.14

Example 1.11

28
Quiz 1.2

29
Quiz 1.2 Solution

30
1.3 Probability Axioms

31
Definition 1.4

Definition 1.5

S: universal set, sample space The probability of an event is the proportion

s: elements of S, outcomes of the time that event is observed in a large
A: sets of elements, events number of runs of the experiment. The is the
relative frequency notion of probability.
32
Theorem 1.1, 1.2

Theorems 1.3 ~ 1.4

33
Theorem 1.4

Theorem 1.5

34
 Comments on Notation
Notation Meaning

P[] The probability of an event

P[A] A function that transforms event A to a number

between 0 and 1.

{si} A set with the single element si

P[{si}] The probability of {si}. P[{si}] = P[si]

P[si]

P[A  B] The probability of the intersection of two events.

P[A, B] P[A  B] = P[A, B] = P[AB]
P[AB]

35
Example 1.12

36
Theorem 1.5

Theorem 1.6 Equally Likely Outcomes

37
Example 1.13

38
Quiz 1.3

39
Quiz 1.3 Solution

40
1.4 Some Consequences of the
Axioms

41
Theorem 1.3

Theorem 1.7

42
Example 1.8

Theorem 1.8

43
Example 1.15

Example 1.14

44
Briefing in next page.
 Example 1.14
• Model of telephone usage
– Time: long (l) > 3 mins, brief (b)
V D F
– Type: voice(v), data(d), fax(f) L 0.3 0.12 0.15
– Sample space S = {lv, bv, ld, bd, lf, bf} B 0.2 0.08 0.15
– Event symbol: L, B, V, D, F

• The probability of a long call

– P[L] = P[LV] + P[LD] + P[LF] = 0.57

45
Quiz 1.4

46
Quiz 1.4 Solution

47
Quiz 1.4 Solution (continued)

48
1.5 Conditional Probability

49
 Prior Probability
• It is sometimes useful to interpret P[A] as our knowledge of
the occurrence of event A before an experiment takes place.
• Thus P[A] reflects our knowledge of the occurrence of A prior
to performing an experiment. Sometimes, we refer to P[A] as
the a priori probability, or the prior probability, of A.

50
 Conditional Probability
• In many practical situations, it is not possible to find out the
precise outcome of an experiment. Rather than the outcome
si, itself, we obtain information that the outcome is in the set B.
• Conditional probability describes out knowledge of A when we
know that B has occurred but we still don’t know the precise
outcome.
• Notation: P[A|B] : “the probability of A given B”

51
Example 1.9

Example 1.15

52
Briefing in next page.
 Example 1.15
• Testing two ICs come from the same silicon wafer.
• Observing: accepted (a), rejected (r)
• Sample space: S = {rr, ra, ar, aa}
• Event B = {first chip tested is rejected} = {rr, ra}
• Event A = {second chip tested is rejected} = {rr, ar}

• High quality  P[A] is very low.

• P[A|B] is higher than the a priori probability P[A] because of
the likelihood that dust contaminated the entire wafer.

53
Definition 1.5

Definition 1.6

54
Theorem 1.6

Theorem 1.9

55
Example 1.10

Example 1.16

• A = “second chip rejected”, P[A] =?

• B = “first chip rejected”, P[B] = ?
• P[AB] = ?
• P[A|B] = ?

Go to Example 1.15 56
Example 1.16 Solution

57
 Example 1.17

• A = “bottom card is the ace of clubs”

• B = “bottom card is a black card”
• P[A|B] =

58
Example 1.17 Solution

59
Example 1.11

Example 1.18

• A = {X1  2}. P[A] = ?

• B = {X2 > X1}. P[B] = ?
• P[A|B] = ?

60
Example 1.18 Solution

61
Theorem 1.9

Theorem 1.10 Law of Total Probability

62
Example 1.16

Example 1.19

63
Briefing in next page.
 Example 1.19
• Three machines B1, B2, and B3 for making 1k resistors.
– B1: 80% of resistors are within 50 of the nominal value.
3000 resistors per hour.
– B2: 90% of resistors are within 50 of the nominal value.
4000 resistors per hour.
– B3: 60% of resistors are within 50 of the nominal value.
3000 resistors per hour.

• What is the probability that the company ships a resistor that

is within 50 of the nominal value?
• A = “resistor is within 50 of the nominal value”
• P[A] = ?

64
Example 1.19 Solution

65
Theorem 1.10

Theorem 1.11 Bayes’ theorem

Theorem 1.10 + 1.11

PA Bi PBi 
PBi A  .
 PA Bi PBi 
m
i 1

66
Example 1.17

Examples 1.20

67
Example 1.20 Solution

68
Quiz 1.3

Quiz 1.5

69
Quiz 1.5 Solution

70
1.6 Independence

71
Definition 1.6

Definition 1.7 Two Independent Events

• Equivalent definition of independent events

P[A|B] = P[A], P[B|A] = P[B]

• Independent and disjoint are not synonyms.

– Disjoint events hare no outcomes in common, P[AB] = 0
– Disjoint  Axiom 3, add  union
– Independent  Definition 1.7, multiply  intersection

72
 Example 1.21

• S = {rrr, rrg, rgr, rgg, grr, grg, ggr, ggg}

• R2 = {rrr, rrg, grr, grg}, G2 = {rgr, rgg, ggr, ggg}, R2  G2 = .
P[R2G2] = P[R2]P[G2] ?

• R2 = {rrr, rrg, grr, grg}, R1 = {rgg, rgr, rrg, rrr}, R2  R1 = {rrg, rrr}
P[R2R1] = P[R2]P[R1] ?

73
Example 1.21 Solution

74
Example 1.19

Example 1.22

75
Briefing in next page.
 Example 1.22
• ICs undergo two test
• Mechanical test (detect the spacing between pins)
• Electrical test (check the input/output relationship)
• We assume that electrical failures and mechanical failures
occur independently.
• Probability of mechanical failures = 0.05
• Probability of electrical failures = 0.2
• What is the probability model?

76
Example 2.2 Solution

77
Definition 1.7

Definition 1.8

78
Example 1.20

Example 1.23

79
Definition 1.8

Definition 1.9

80
Quiz 1.5

Quiz 1.6

81
Quiz 1.6 Solution

82
Quiz 1.6 Solution (continued)

83
2.1

1.7 Sequential Experiments

and Tree Diagrams

84
 Subexperiments and Tree Diagram
• Many experiments consist of a sequence of subexperiments.

Theorem 1.10 + Theorem 1.9 (Axiom 3, Axiom 2)

Summation is 1

Conditional probability

85
Example 2.1

Example 1.24

86
 Example 1.24
• For the resistors of Example 1.19
• A: a randomly chosen resistor is “within 50 of the nominal
value.”
• Two-step procedure for testing a resistor.
– First, identify which machine (B1, B2, or B3) produced the resistor.
– Second, we find out if the resistor is acceptable.

• Sketch a sequential tree.

• What’s the probability of choosing a resistor from machine B2
that is not acceptable.

87
Example 2.2

Example 1.25

88
Example 1.25 Solution

89
Example 1.25 Solution (continued)

90
Example 1.26

91
Example 1.26 Solution

92
Example 1.27

93
Example 1.27 Solution

94
Quiz 2.1

Quiz 1.7

95
Quiz 1.7 Solution

96
2.2

1.8 Counting Method

97
Theorem 2.1
Definition 1.10 Fundamental Principle of
Counting

• Generally, if an experiment E has k subexperiments E1, …, Ek

where Ei has ni outcomes, then E has


k
n
i 1 i outcomes.

98
Example 2.5

Example 1.28

99
Example 1.29

100
Example 1.30

101
Theorem 2.2

Permutation, Theorem 1.12

• An ordered sequence of k distinguishable objects is called a
k-permutation.
n!
• Notation: n k  nn  1n  2  n  k  1 
n  k !

102
 Sampling without Replacement
• Replace choosing objects from a collection is also called
sampling, and the chosen objects are know as a sample.
• Picking a subset of the collection of objects, each subset is
called k-combination.

n
• Notation:   read as “n choose k”
k 

• k-combination of n objects + k-permutation of k objects

n
n k   k!
k 
103
Theorem 2.3

Theorem 1.13

• Binomial coefficient: the coefficient of xkyn–k in (x + y)n

• Observation    n k 
n n!  n 
k    
  k! k!n  k !  n  k 

• Constraint: 0  k  n. (Check the extended Definition 1.11)

104
Definition 2.1

Definition 1.11 nBinomial

chooseCoefficient
k

105
Example 2.8

Example 1.31

106
Example 2.9

Example 1.32

107
Theorem 2.4, 2.5
Sampling with Replacement
Theorem 1.14, 1.15

• Example 1.34 ~ 1.36

• Example 1.37 ~ 1.38

108
Example 2.11, 2.12

Example 1.34 ~ 1.36

109
Example 1.37

110
Example 2.14

Example 1.38

111
Theorem 2.6, 2.7

Theorem 1.16 ~ 1.17

112
Theorem 1.17 Proof

113
 Theorem 1.17
• For n repetitions of a subexperiment with sample space S =
{s0, …, sm-1}, the number of length n = n0 + … + nm-1
observation sequences with si appearing ni times is

 n  n!
  
 n0 ,, nm 1  n0! n1! nm 1!
Proof
 n  n  n0  n  n0  n1   n  n0    nm 2 
M        
 n0  n1  n2   nm1 

n! n  n0 !

n  n0    nm 2 !
n0! n  n0 ! n1! n  n0  n1 ! nm1! n  n0    nm1 !
= 0! = 1
114
Definition 2.2

Definition 1.12 Multinomial Coefficient

115
Quiz 2.2

Quiz 1.8

116
Quiz 1.8 Solution

117
2.3

1.9 Independent Trials

118
Theorem 2.8, Example 2.16

Theorem 1.18

119
Example 1.39 Solution

120
Example 2.17

Example 1.40

121
Example 1.41

122
Example 1.41 Solution

123
Theorem 2.9

Theorem 1.19

124
Example 2.18

Example 1.42

125
Example 1.43

126
Quiz 2.3

Quiz 1.9

127
Quiz 1.9 Solution

128
1.10 Reliability Problems

129
 Reliability Problem
• The operation consists of n components and each component
succeeds with probability p, independent of any other
component. Let Wi denote the event that component i
succeeds.
• Components in series
– The operation succeeds if all of its components succeed.
PW   PW1W2 Wn   p  p    p  p n
• Components in parallel
– The operation succeeds if any of its components succeed.

   
P W c  P W1cW2c Wnc  1  p 
n

 
PW   1  P W c  1  1  p 
n

130
 Figure 1.3

PW   PW1W2 Wn     

P W c  P W1cW2c Wnc  1  p 
n

 p  p   p  pn  
PW   1  P W c  1  1  p 
n

131
132
Example 1.44

133
Example 1.44 Solution

134
Quiz 1.10

135
Quiz 1.10 Solution

136