Theory of Probability

Prof Tasneem Chherawala

NIBM

Uncertain Events
See a Zebra the classroom today?
Win in a lottery?
Toss a coin and get heads / roll a die (or two) and get 5/
pick a card from a deck and get an Ace?
Make a 10% profit from investment in shares over 1
month?
Face a default on money lent?
Earn positive returns on a fixed deposit?
Count the total number of hours today and get 24?

Probability
Probability quantifies how uncertain we are about a
future outcome / event
A probability refers to the percentage chance that
something will happen, from 0 (it is impossible) to 1 (it is
certain to occur), and the scale going from less likely to
more likely
An interpretation based on data - Probability can be
interpreted as the relative frequency of the outcomes
(values) of uncertain events (variable) after a great many
(infinitely many) repetitions / parallel independent trials of
an experiment

Why Measure Uncertainty

Is something at stake?

To make tradeoffs among uncertain events

Measure combined effect of several uncertain events
To communicate about uncertainty

To draw inferences about a population from a sample

Probability Theory in Finance

Financial decisions a game of chance!
What chance?
Probability of making or losing money in an investment!

Why chance?
Uncertainty and variability of future events (price movements
and value of investments)

Probability concepts help define financial risk by

quantifying the prospects for unintended and negative
outcomes (losses)
Probability also quantifies expected values of future
events, which gives us a fair estimate of current value of
investment

Probability Theory in Finance

Probability theory applications:
Make logical and consistent investment decisions
Manage expectations in an environment of risk

Hypothesis: No system for making financial choices

from those offered to us can both (1) be certain to avoid
losses and (2) have a reasonable chance of making us
rich
Expected values in a probability model are the prices of
alternative financial decisions

Some Definitions
Random Experiment: a process that leads to one of
several possible outcomes
Outcome: result of the experiment
Examples: Head in a coin toss, two heads when tossing two
coins, win in a lottery, 10% returns from the stock in one day,
Borrower repays Re 1 from Rs. 100 borrowed at 12% rate of
interest

Sample Space: a set of all possible outcomes of the

experiment relates to population data
Examples:

Toss one coin: S = {H, T}

Toss two coins / toss a coin twice: S = {HH, HT, TH, TT}
Lottery: S = {Win, Lose}
One day stock returns: S = {-100% to +Infinity?}
Repayment on Rs. 100 loan by borrower at 12% rate of int.: S =
{0,1,2,.., 100, 101,,112}

Sample Space
Finite sample space: finite number of outcomes in the space
S.

Countable infinite sample space: ex. natural numbers.

Discrete sample space: if it has finite or countable infinite
number of outcomes.
Continuous sample space: If the outcomes constitute a
continuum. Ex. All the points in a line.

Events
Event: subset of a sample space a combination of
possible outcomes of an uncertain process
Examples
Head in one toss of a coin: E1 = {H}
One head and one tail in two coin tosses: E2 = {HT, TH}
Mean of the dice values greater than or equal to 5 in two dice rolls:
E3 = {4.6, 5.5, 5.6, 6.4, 6.5, 6.6}
One day stock returns greater than equal to 10%: E4 = {10% to
+Infinity}
Loss to bank if Borrower doesnt repay principal: D =
{1%,2%,..,99%,100%}

Events
Mutually Exclusive Events: events which have no
outcome in common and thus cannot happen together
If a list of events is mutually exclusive, it means that only
one of them can possibly take place
Examples
Toss one coin: E1 = {H}, E2 = {T}
Toss two coins: E1: first toss is Head = {HT, HH}, E2: first toss is
Tail = {TH, TT}
Invest in a stock for one day: E1: One day stock returns greater
than equal to 10% = {10% to +Infinity}, E2: One day stock returns
between 2% to 5% = {2% to 5%}
How would E1 compare with E3: One day stock returns less than or
equal to 10% = {-100% to 10%)

Lend money: E1: Borrower repays entire principal =

{100,101,102,112}, E2: Borrower repays 50% of principal = {50}
How would E1 compare with E3: Borrower repays principal with 5%
interest = {105} ?

Properties of Probabilities defined

on a Sample Space
Sample space must be exhaustive: List all possible
outcomes
Outcomes in the sample space must be mutually
exclusive
The exhaustive and mutually exclusive characteristic of a
sample space together imply that
The probability (likelihood of occurrence) of any one
outcome or event must lie between 0 and 1
0 < P(E) <1

The sum of the probabilities of all the outcomes in the

sample space must be 1
P(S) = 1

Properties of Probabilities defined

on a Sample Space
By extension of the above rationale
The probability of any event would lie between 0
(impossible) and 1 (certain), or 0 < P(E) < 1.
There is no such thing as a negative probability (less
than impossible?) or a probability greater than 1
(more certain than certain?).
The sum of all probabilities of all outcomes would
equal 1, provided the outcomes are both mutually
exclusive and exhaustive.
If outcomes are not mutually exclusive, the
probabilities would add up to a number greater than
1, and if they were not exhaustive, the sum of
probabilities would be less than 1.

Classical / A Priori Approach to

Probabilities
When the range of possible uncertain outcomes in a
sample space is known and equally likely
Probability for each outcome or event can be determined
by logic
Examples
Tossing a fair coin outcomes can only be heads or tails and
both are equally likely
Rolling a fair die outcomes can only be 1,2,6 and all six are
equally likely
Drawing a card from a pack outcomes can only be 52, and all
are equally likely

The probability of each outcome in the above examples

has been determined by construction of the
coin/die/pack of cards

Defining A Priori Probabilities

In a sample of N mutually exclusive, exhaustive and
equally likely outcomes we assign a chance (or weight)
of 1/N to each outcome
We define the probability of an event for such a sample
as follows:
The probability of an event E occurring is defined as:
P(E) = n(E)/n(S)
n(E) is the number of outcomes favourable to E and
n(S) is the total number of equally likely outcomes in
the sample space S of the experiment
By extension, the probability of event E not occurring
P(not E) = 1-P(E) = 1- n(E)/n(S)

Example
What is the probability that a card drawn at random from
a deck of cards will be an ace?
Total no. of outcomes in the sample space?
Are they equally likely?
Event of interest (E)?
No. of Outcomes favourable to the Event?
P(E)?
What is the probability that a card drawn at random from
the deck will not be an ace?

A Priori Probabilities
The same principle can be applied to the problem of
determining the probability of obtaining different totals
from a pair of dice.
Possible Outcomes?
Are they equally likely?
Event 1 = A = sum of the two dice will equal 5
Event 2 = B = the absolute difference will equal 1
P (Event 1) =
P (Event 2) =

A Priori Probabilities
In certain cases where outcomes are not equally likely,
one can still deduce rationally the a priori probability of
an event
For example
If we forecast that a company is 70% likely to win a
bid on a contract (irrespective of how this probability
is derived), and we know this firm has just one
business competitor, then we can also make an a
priori forecast that there is a 30% probability that the
bid will go to the competitor

Empirical Probabilities
In finance, we cannot depend upon the exactness of a
process to determine a priori probabilities
The financial analyst would have to depend upon
historical occurrence of the event(s) or repeat an
experiment multiple times to determine the probability of
the event empirically
For example, the range of outcomes of returns on a
financial asset are virtually infinite and that too, not all
outcomes are a priori, equally likely
Thus, the financial analyst would have to observe many
movements in asset prices to determine the probability
of future price changes of a given magnitude
Of course, we know that past performance does not
guarantee future results, so a purely empirical approach
has its drawbacks

Defining Empirical Probabilities

The probability of an event (or outcome) is the proportion
of times the event occurs in a long run of repeated
experiment.
The empirical probability of a given outcome / event Z is
defined as
P(Z) = no. of Z occurrences/no. of trials of the experiment

This is the same as analysis of relative frequency of

observations in a sufficiently large sample
Consider a financial analyst who is interested in knowing
what will be the probable one day returns on a particular
stock.
He tracks the past 100 days of price movements and returns of
a particular stock.
Each of the 100 days would constitute trial and each days

Subjective Probabilities
Probability under this approach is simply defined as the
strength of belief that an event will occur
It is based upon experience and judgment
Such probabilities are applied to many business
problems where a priori probabilities are not possible,
nor are there sufficient empirical observations upon
which to base probability estimates
For example, subjective probability is incorporated in the
forecasting of company profits by investment analysts
Of course, subjective probabilities are unique to the
person making them and to the specific assumptions
made

Rules of Probability
There are a number of formal probability rules applied to
probability estimates
Which of these rules is applicable will depend upon
whether
We are concerned with a single event, in which case
the outcomes relate only to that event
We are concerned with combinations of several
events, for example the changes in Sensex and
Exchange Rates together
The combined events are independent or mutually
exclusive

Rules of Probability
The rules are
Complement rule: When we are concerned with
whether an event A will not occur
Multiplication rule: when we are concerned with event
A and B occurring together. This requires us to know
whether A and B are independent of each other
Addition rule: when we are concerned with event A or
B happening. This requires us to know whether A and
B are mutually exclusive

Intersection of Events and Joint

Probability
Joint probability: The probability of both event A and
event B occurring: P(A and B) / P(A B) / P(AB)
Intersection: Event defined as both A and B occur
Example Using a priori probabilities:
P(A and B) = No. of Outcomes favourable to the Joint Event /
Total no. of outcomes in the sample space

Event A is you draw a spade from a deck of cards

Event B is you draw a king
Intersection is event you draw a king of spades
Joint probability = P(A and B) = 1/52

Joint Probability Table

Job_status

Decision
accept

reject

governmen

0.116

0.072

military

0.003

0.003

misc

0.000

0.003

private_s

0.266

0.352

retired

0.014

0.005

self_empl

0.035

0.051

student

0.005

0.008

unemploye

0.013

0.056

Interpretation of 0.266?

Marginal Probability
Marginal probability: Probability of a single event
When outcomes are exhaustive and mutually exclusive,
can be calculated by adding all the joint probabilities
containing the single event

Marginal Probability
Job_status

Decision
accept

reject

Total

governmen

0.116

0.072

0.187

military

0.003

0.003

0.005

misc

0.000

0.003

0.003

private_s

0.266

0.352

0.617

retired

0.014

0.005

0.019

self_empl

0.035

0.051

0.086

student

0.005

0.008

0.013

unemploye

0.013

0.056

0.069

Total

0.451

0.549

1.000

Interpretation of 0.187, 0.451?

Conditional Probability
A conditional probability is the probability of an event
given that another event has occurred
Conditional probability assumes that one event has
taken place or will take place, and then asks for the
probability of the other (A, given B)
P(B | A): probability of B given A
P(A | B): probability of A given B
For example,
Probability of drawing a king given that a spade is drawn: P(king
| spade) = 1/13
Probability of drawing a spade given that a king is drawn:
P(spade | king) = 1/4
What is the probability that the total of two dice will be greater
than 8 given that the first die is a 6?

Conditional Probability in Terms of

Joint Probability
Job_status

Decision
accept

reject

Total

governmen

0.116

0.072

0.187

military

0.003

0.003

0.005

misc

0.000

0.003

0.003

private_s

0.266

0.352

0.617

retired

0.014

0.005

0.019

self_empl

0.035

0.051

0.086

student

0.005

0.008

0.013

unemploye

0.013

0.056

0.069

Total

0.451

0.549

1.000

Given that an application

is accepted, what is the
probability that the
customer is a Pvt. Sector
employee?
What kind of probability
asked for?
Imagine 1000 Customer
Applications. You would
expect 451 to be accepted
and 266 of those to be
Pvt. Sector employees
P(Pvt. S Empl | Accept) =
116/451 = 0.257

Conditional Probability in Terms of

Joint Probability
Joint Probability can be rescaled to find Conditional
Probability
P(A|B) = P(A and B) / P(B)
Can think about this as rescaling the Accept Column so
it sums to one
Interpretation

P(Pvt. S Empl | Accept) = 266/451 = 0.59

P(Non Pvt. S Empl | Accept) = 185/451 = 0.41
Means Pvt. S Employees are more likely to be accepted than
Non Pvt. Sector Employees?
P(Accept | Pvt. S Empl) = 266/617 = 0.43
P(Accept | Non Pvt. S Empl) = 185/383 = 0.483
Means more of the accepted applicants are Non Pvt. Sector
Employees than Pvt. Sector Employees?
If the banks objective was to specifically target a group for
marketing of the product, which probabilities would it rely upon to
give it an accurate picture?

Joint Probability in Terms of

Conditional Probability
Another way to calculate Joint Probability of Events A
and B is
P(A and B) = P(A) x P(B|A) = P(B) x P(A|B)
where P(B|A) is the conditional probability of B given A
and P(A|B) is the conditional probability of A given B
Example:
If we believe that a stock is 70% likely to return 15% in the next
year, as long as GDP growth is at least 8%, then we have made
our prediction conditional on a second event (GDP growth). In
other words, event A is the stock will rise 15% in the next year;
event B is GDP growth is at least 8%; and our conditional
probability is P(A | B) = 0.7
Now if we know that there is a 20% unconditional probability that
GDP will grow at 8% or above P(B) = 0.2
The probability that GDP will grow at least at 8% and stock will
return 15% is P(A and B) = 0.7 *0.2 = 0.14

Joint Probability Vs Conditional

Probability
Joint probability is not the same as conditional
probability, though the two concepts are often confused.
Joint probability sets no conditions on the occurrence of
events but simply provides the chance that both events
will happen together
In a problem, to help distinguish between the two, look
for qualifiers that one event is conditional on the other
(conditional) or whether they will happen concurrently
(joint).

Joint Probability and Independence

of Events
Independence of Event A and B: When Probability of
Event A occurring does not depend upon occurrence of
Event B and vice versa
Two events are independent if and only if
P(A | B) = P(A) [implies P(B | A) = P(B)]
If A and B are independent, then the probability that
events A and B both occur is:
P(A and B) = P(A) x P(B | A) = P(A) x P(B)
Examples:
What is the probability that a fair coin will come up with heads
twice in a row?
Now consider a similar problem: Someone draws a card at
random out of a deck, replaces it, and then draws another card
at random. What is the probability that the first card is the ace of
clubs and the second card is a club (any club)?

Joint Probability and Independence

of Events
Is the customer application being accepted independent
of being a Pvt. sector employee?
P(Accept | Pvt. Sector employee) = 0.43
P(Accept) = 0.451
The rule generalizes for more than two events provided
they are all independent of one another, so the joint
probability of three events P(ABC) = P(A) * (P(B) * P(C),
again assuming independence

Joint Probability and Mutually

Exclusive Events
The joint probability of two mutually exclusive events
occurring is 0
This is because by definition of mutually exclusive,
events A and B cannot occur together
For example:

Roll a die once: Event A = 6, Event B <5

Are A and B mutually exclusive?
P(A and B) = ?
Roll 2 dice: Event A = {1,4}, Event B = {4,1}
Are A and B mutually exclusive?
P(A and B) = ?
Roll 2 dice: Event A = {1,4}, Event B = at least one die shows 1
Are A and B mutually exclusive?

Union of Events and Probability

Union: Union of events A and B is event that either A or
B or both occur: (A or B) or (AUB)
P(A or B) is interpreted as the probability that at one of
the two events A and B will occur
If events A and B are mutually exclusive, P(A and B) =
0
P(A or B) = P(A) + P(B)
Example:
What is the probability of rolling a die and getting either a 1 or a
6?
Since it is impossible to get both a 1 and a 6, these two events
are mutually exclusive
P(1 or 6) = P(1) + P(6) = 1/6 + 1/6 = 1/3

Union of Events and Probability

If events A and B are not mutually exclusive, P(A and
B) 0
P(A or B) = P(A) + P(B) P(A and B)
The logic behind this formula is that when P(A) and P(B)
are added, the occasions on which A and B both occur
are counted twice. To adjust for this, P(A and B) is
subtracted.
Example:
What is the probability that a card selected from a deck will be
either an ace or a spade?
P(Ace) = 4/52
P(Spade) = 13/52
P(Ace and Space) = ?
P(Ace or Space) = ?

Union of Events and Probability

Consider the probability of rolling a die twice and getting
a 6 on at least one of the rolls. The events are defined in
the following way:
Event A: 6 on the first roll: P(A) = 1/6
Event B: 6 on the second roll: P(B) = 1/6
P(A and B) = 1/6 x 1/6 (why?)
P(A or B) = 1/6 + 1/6 - 1/6 x 1/6 = 11/36
The same answer can be computed using the following
admittedly convoluted approach:
Getting a 6 on either roll is the same thing as not getting
a number from 1 to 5 on both rolls.
This is equal to: 1 - P(1 to 5 on both rolls) = 1 5/6 x 5/6
= 1 25/36 = 11/36

Union of Events and Probability

Despite the convoluted nature of this method, it has the
advantage of being easy to generalize to three or more
events.
For example, the probability of rolling a die three times
and getting a six on at least one of the three rolls is?
1 - 5/6 x 5/6 x 5/6 = 0.421
In general, the probability that at least one of k
independent events will occur is:

1 - (1 - )k
where each of the events has probability of occurring

Union of Events and Probability

Example:
Assume that the bank had lent to 5 different borrowers,
each of whom was assigned a probability of default = 0.1
The bank wished to hedge the risk of its portfolio such
that it bought insurance such that the loss against at
least one borrower defaulting was made good
How would the insurance company estimate the
probability of at least one borrower defaulting, assuming
that the default event of each borrower was
independent?
P(At least one default) = 1 (1-0.1)5 = 0.41

Union of Events and Probability

Job_statu
s

Decision
accept

reject

Total

governme
n

0.116

0.072

0.187

military

0.003

0.003

0.005

misc

0.000

0.003

0.003

private_s

0.266

0.352

0.617

retired

0.014

0.005

0.019

self_empl

0.035

0.051

0.086

student

0.005

0.008

0.013

unemploy
e

0.013

0.056

0.069

Total

0.451

0.549

1.000

What is the probability that the

customer application is accepted
or the customer is a government
employee?
What kind of probability asked
for?
P(Accept) = 0.451
P(Govt. Employee) = 0.187
P(Accept and Govt. Employee =
0.116
P(Accept or Govt. Employee) =
0.451 + 0.187 0.116 = 0.522

Union of Events and Probabilities

Example:
A Fund manager has invested in the stocks of two
companies 1 and 2 and is interested in knowing what is
the probability that the equity price of either company will
rise
Are the two events mutually exclusive?
P(Co. 1 Equity ) = 0.55
P(Co. 2 Equity ) = 0.35
P(Co. 1 Equity and Co. 2 Equity ) = 0.3
P(Co. 1 Equity or Co. 2 Equity ) = 0.55+0.35-0.30 =
0.60

Union of Events and Probability

What if we want to know the probability of at least one
of 3 events A, B and C happening?

P( A B C) P( A) P(B) P(C) P( AB) P( AC) P(BC) P( ABC)

Probability Rules Summary

Multiplication Rule: Joint probability of any two events A
and B is:
P(A and B) = P(A | B) * P(B)
Follows from definition of conditional probability

Multiplication Rule Independent Events: If A and B are

independent, joint probability is:
P(A and B) = P(A) * P(B)

Addition Rule mutually exclusive events:

P(A or B) = P(A) + P(B)
Mutually exclusive if both cannot occur

Addition Rule: Probability that event A or event B or both

occur is
P(A or B) = P(A) + P(B) P(A and B)

Are mutually exclusive events independent?

Does pr(A|B) = pr(A)?
NO!
If B has happened, then A can not happen
pr(A|B) = 0
So
Mutually exclusive events are DEPENDENT.

Probability Tree and Total

Probability Rule
Decision

Job_status
accept

reject

Private
Sector

0.29

0.35

Non Pvt
Sector

0.16

0.20

Probability Tree
0.29

Pvt. S & Accept

Pvt. Sector | Accept: 0.59

Accept: 0.45

Not Pvt. Sector | Accept: 0.41

0.16
Pvt. Sector | Reject: 0.64

0.35

Not Pvt. S & Accept

Pvt. S & Reject

Reject: 0.55
Not Pvt. Sector | Reject: 0.36
0.20

Not Pvt. S & Reject

Total Probability Rule

The Total Probability Rule: The total probability rule
explains an unconditional probability of an event A, in
terms of that event's conditional probabilities in a series
of mutually exclusive, exhaustive scenarios of event B
P(A) = P(A | B) x P(B) + P(A | not B) x P(not B)
With the total probability rule, event A has a conditional
probability based on each scenario of event B (i.e. the
likelihood of event A, given that scenario), with each
conditional probability weighted by the probability of that
scenario for event B occurring
Example:
What is the probability of a customer employed in the Private
Sector:
P(Pvt. S. Empl) = P(Pvt. S Empl | Accept) x P(Accept) + P(Pvt. S
Empl | Reject) x P(Reject) =
0.59*0.45+0.64*0.55 = 0.61

Total Probability Rule

A model that predicts whether a borrower has defaulted
or not is 95 percent effective in predicting that a borrower
has defaulted when it actually has. However, the model
also yields a false positive result for 1 percent of the
non-defaulted borrowers. That is, there is 1 percent
chance that a borrower who has not defaulted will be
identified as a defaulted borrower by the model.
Q: If 0.5 percent of the banks portfolio has actually
defaulted, what is the probability that a borrower has
defaulted given that the model predicts default?

Probability Tree
0.00475
Def Pred | Default: 0.95

Default: 0.005

No Def Pred | Default: 0.05

0.00025
Def Pred | No Default: 0.01

0.00995

No Default: 0.995
No Def Pred | No Default: 0.99
0.98505

Total Probability Rule And Bayes

Theorem

Let D be the event that a borrower has defaulted

Let E be the event that the model predicts default
We need to estimate P(D|E)
We know: P(E|D)=0.95, P(E|notD)=0.01, P(D)=0.005,
P(notD)=1-P(D)=0.995

P( D E ) 0.00475

0.3231
P( E )
0.0147
P( D E ) P( ED) P( D) 0.95 0.005 0.00475
P( E ) P( ED) P( D) P( EnotD) P(notD) 0.95 0.005 0.01 0.995 0.0147

P( DE )