' $

Probability & Statistics

• The theory of probability is a branch of pure
mathematics. From a certain set of axioms and
definitions mathematicians build up the theory by
deduction.
• In contrast, for a physicist, statistics is a branch of
applied mathematics which is essentially inductive.
Nevertheless statistics is intimately connected with the
theory of probability.
Example: Suppose that it is known that when a coin a has
an a priori probability p = 21 of landing on one specific side
(head) and a probability (1 − p) of landing on the other
sides (tail).
• What is the probability of observing r heads out of n
tosses???
• This is a question in probability theory and the answer
is provided by the binomial distribution law [Chap. 5]:

n!
P (r; n) = pr (1 − p)n−r . (1)
r!(n − r)!

& %
c
Computational Physics - Phys 75.487/502 °Alain Bellerive - Carleton University 1
 ' $

Probability & Statistics (con’t)

• A complete different situation exists if one has NO a
priori knowledge of the probability p and decides to
perform an experiment to determined this parameter.
• A simple experiment would consist in tossing a coin
repeatedly and counting how many times the outcome
heads would occur. It would then be a question of
statistics to ask what the parameters p is like, given
that in n tosses, r heads were observed.
• A reasonable answer to this question is to say that the
most likely value of the parameter is given by:
r
p = p̃ = . (2)
n
The experiment has then given a point estimate p = p̃ for
the unknown parameter. However from the nature of the
experiment, we are not completely sure that the value p̃
obtained is identical to the true value of the parameter p.

& %
c
Computational Physics - Phys 75.487/502 °Alain Bellerive - Carleton University 2
 ' $

Experiment
• Intuitively one expects that if the coin experiment was
repeated with new sequences tosses, then one should
obtain a different estimate for p.
• Hence, instead of stating the result of the experiment in
terms of a single number p̃ one could give an interval
estimate [p1 , p2 ] for the parameter p.
• The faith that p lies within [p1 , p2 ] could then be
expressed by a confident level.

THIS IS STATISTICAL INFERENCE!

• Generality: larger(smaller) the interval, larger(smaller)
the certitude that [p1 , p2 ] really includes the true value
of p.
• Since the actual calculation of a confidence interval
require assumption about the probability P (r; n): to
make statistical inference it is necessary to know the
functional form of the binomial distribution.
• Thus, we shall state (not prove) the rules that govern
the theory of probability.

& %
c
Computational Physics - Phys 75.487/502 °Alain Bellerive - Carleton University 3
 ' $

Definition of Probabilities
• Physicist do not agree about the best way to define
probability!
• Frequentist versus Bayesian.
• In this course we will use the frequency approach.

ADOPT THE DEFINITION OF

PROBABILITY IN TERM OF THE LIMIT
OF RELATIVE FREQUENCY
OCCURRENCE!
• Thus, in a sequence of n trials of an experiment the
outcome of a specified EVENT (or class) occurs r
times:
r
P (E) = in the limit n → ∞ . (3)
n
• From this definition 0 ≤ P (E) ≤ 1, where P (E) = 0 if
the EVENT never occurs and P (E) = 1 if it always
occurs.
• Frequentist assumption: converge to the ’true’ value as
n → ∞.

& %
c
Computational Physics - Phys 75.487/502 °Alain Bellerive - Carleton University 4
 ' $

Random Variables and Sample Space

• Random variable: prior to an experiment the
outcome cannot be predicted with complete certainty.
All the outcome are said to be exclusive.
• Associate each outcome xi of the experiment with the
probability Pi such that:
X
n
P (X = xi ) = Pi with Pi = 1 (exhaustive). (4)
i=1

• Continuous: Random variable can have a continuum

of values within any finite interval Ω.
• The probability P (x ≤ X ≤ x + dx) is associated to the
event of getting a measurement within the interval
[x, x + dx].
• We define the probability density function f (x) for
the continuous random variable X by the equation:

f (x) dx = P (x ≤ X ≤ x + dx) . (5)

The exhaustive requirement is then
Z
f (x) dx = 1
Ω

& %
c
Computational Physics - Phys 75.487/502 °Alain Bellerive - Carleton University 5
 ' $

Definitions
Set The concept of a set is used to denote a collection of
objects with some common properties.
Element An object which belongs to a set A is said to be
an element of A.
Subset If every element of the set B is also an element of
the set A, we say the B is a subset of A.
Complement In a sample space Ω, the complement Ā is
the set of all elements in Ω that do not belong to A.
Union A ∪ B is the set of element that belong to A or B,
or both.
Intersection A ∩ B is the set of element that belong to A
and B. Use P (AB) ≡ P (A ∩ B).

1111111
0000000
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
A 0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
B
0000000
1111111

Venn Diagram

& %
c
Computational Physics - Phys 75.487/502 °Alain Bellerive - Carleton University 6
 ' $

Rules
Axioms
• P (E)Ω = 1
• P (A) ≥ 0
• P (A + Ā) = 1
N (A) N (B) N (AB)
• P (A) = N (E) P (B) = N (E) P (AB) = N (E)

Addition P (A ∪ B) = P (A) + P (B) − P (A ∩ B)

Mutually exclusive
P (A + B) ≡ P (A ∪ B) = P (A) + P (B)
Conditional Probability Consider A and B two subsets
of the sample space Ω. Suppose that we are interested
only in the subset A.
Q: How can we express the probability of the subset B
relative to the new sample space A?
A: This new probability is called the conditional
probability of B relative to A: P (B|A).
P (B|A) It should read: “probability of B given A”.

& %
c
Computational Physics - Phys 75.487/502 °Alain Bellerive - Carleton University 7
 ' $

Conditional Probability

N (AB) N (AB) N (AB)/N (E)

P (B|A) = = = . (6)
N (EA) N (A) N (A)/N (E)

P (AB)
P (B|A) =
P (A)

• Assume
X
n
E= Ai
i=1

• Then
X
n
P (B) = P (EB) = P (AB) = P (Ai )P (B|Ai )
i=1
E

A3 A4
A2 A5

A1 An

& %
c
Computational Physics - Phys 75.487/502 °Alain Bellerive - Carleton University 8
 ' $

Bayes Theorem
• Consider 1) A1 · · · An a partition of E and 2) B an
event of A.
• Then for 1 ≤ k ≤ n fixed

P (Ak B)
P (Ak |B) =
P (B)
Then
P (B|Ak )P (Ak )
P (Ak |B) =
P (B)
Finally

P (B|Ak )P (Ak )
P (Ak |B) = Pn
i=1 P (B|Ai )P (Ai )

Bayes theorem is concerned with conditional probabilities

where P (B) and P (A) are the unconditional (or a priori)
probabilities of B and A, respectively. This is a fundamental
theorem of probability theory, but its use in statistics is a
subject of some controversy (Bayesian Statistics).

& %
c
Computational Physics - Phys 75.487/502 °Alain Bellerive - Carleton University 9
 ' $

Bayesian Approach
An important school of statistical theory, in which statistics is
derived from a probability interpretation that includes the
degree of belief in an hypothesis. It thus refers not only to
repeatable measurements (as does the frequentist
interpretation). The interpretation of data can be described by
Bayes Theorem for an hypothesis H and experimental data d.
P (d|H)P (H)
P (H|d) = (7)
P (d)
Bayesian meaning of the different terms:

• P (H|d) is the degree of belief in the hypothesis H, after the

experiment which produced data d,

• P (H) is the prior probability of H being true,

• P (d|H) is the ordinary likelihood function used also by

non-Bayesian,

• P (d) is the prior probability of obtaining data d,

• It can be rewritten using the other terms as:

P
P (d) = i P (d|Hi )P (Hi ), where summation runs over all
hypotheses.

What is called a Bayesian viewpoint is the application of the

laws of probability to non-repeatable events: H is a hypothesis
or proposition, either true or untrue, and P (H) is interpreted as
the degree of belief in the proposition.
Remark: We will mainly use the frequentist approach in this course.
& %
c
Computational Physics - Phys 75.487/502 °Alain Bellerive - Carleton University 10
 ' $

Independence: Multiplication Rule

• The realization of A influenced the probability of B,
then we say that A and B are dependent:
P (AB)
P (B|A) = 6= P (B) (8)
P (A)
• But if A and B are independent if and only if:

P (B|A) = P (B) (9)

INDEPENDENCE IMPLIES:
P (AB) = P (A) P (B)

& %
c
Computational Physics - Phys 75.487/502 °Alain Bellerive - Carleton University 11
 ' $

Example: Relay Networks I

1 2 3

Relay Network
Question: The probability for the closing of each relay in
the circuit is some given number α. Assuming all relays act
independantly, find the probability for the flow of a current
between the terminal.
Hint: The current will flow (event E) if it can go in the
branch: i.e. E = E1 .and.E2 .and.E3 .
Rules: P (E) = P (E1 ∩ E2 ∩ E3 ).

Answer: P (E) = P (E1 ) P (E2 ) P (E3 ) = α3 .

& %
c
Computational Physics - Phys 75.487/502 °Alain Bellerive - Carleton University 12
 ' $

Example: Relay Networks II

1

2 3
Relay Network
Question: The probability for the closing of each relay in
the circuit is some given number α. Assuming all relays act
independantly, find the probability for the flow of a current
between the terminal.
Hint: The current will flow if it can go in one of the branch.
Rules: The current will flow if the condition E = E1 .or.(E2 .and.E3 )
is satistied; i.e. P (E) = P (E1 ∪ [E2 ∩ E3 ]).

Answer: Since P (A ∪ B) = P (A) + P (B) − P (A ∩ B).

P (E) = P (E1 ) + P (E2 ∩ E3 ) − P (E1 ∩ [E2 ∩ E3 ])

P (E) = P (E1 )+P (E2 ) P (E3 )−P (E1 ) P (E2 ) P (E3 ) = α+α2 −α3 .

& %
c
Computational Physics - Phys 75.487/502 °Alain Bellerive - Carleton University 13
 ' $

Example: Coincidence
Suppose the Cerenkov light, from particles traversing a
Cerenkov counter along their path, is detected by a
spherical arrangement of phototubes as illustrated in
Figure 1. In order to discriminate against accidental
triggering of the system, it is desirable to observe
coincidences between the signals from several phototubes.
Assume all phototubes to act independently. If the event E
of having a signal from one phototube corresponds to a
probability P (E) = ǫ = 0.93.
(a) What is the probability for the detection of the
Cerenkov light by all nine independent phototubes (c.f.
Figure 1a)?
(b) What is the probability for the detection of the
Cerenkov light when the tubes are grouped together three
by three (c.f. Figure 1b)? Each group is activated if at least
one of the tubes in the group has a signal.
(c) Which experimental set-up gives the largest detector
efficiency?
Ref: Probability and Statistics in Partcile Physics (Frodesen et al.)

& %
c
Computational Physics - Phys 75.487/502 °Alain Bellerive - Carleton University 14
 ' $

Example: Coincidence (con’t)

OR
OR

OR

(a) 9−fold coincidence (b) 3−fold coincidence

Figure 1: Arrangements of nine phototubes in a Cerenkov

counter.

& %
c
Computational Physics - Phys 75.487/502 °Alain Bellerive - Carleton University 15