PROBABILITY AND STOCHASTIC PROCESSES

Course Contents
1. Introduction to probability and random variables
2. Discrete probability distributions
3. Continuous probability distributions
4. Parameter estimation and Hypothesis testing
5. Stochastic processes
6. Markov chains
Course Objectives
Introduction to the logic of probability
Develop intuition of how probability to practical situation
Apply probability theory to engineering problems
 Probability
How likely something is to happen.

Many events can't be predicted with total certainty. The best we can say is
how likely they are to happen, using the idea of probability.

Tossing a Coin

When a coin is tossed, there are two possible outcomes:

heads (H) or
tails (T)

We say that the probability of the coin landing H is .

And the probability of the coin landing T is .

Throwing Dice

When a single die is thrown, there are six possible

outcomes: 1, 2, 3, 4, 5, 6.

The probability of any one of them is 1/6.

Probability

In general:

Probability of an event happening = Number of ways it can happen/Total

number of outcomes
Example: the chances of rolling a "4" with a die

Number of ways it can happen: 1 (there is only 1 face with a "4" on it)

Total number of outcomes: 6 (there are 6 faces altogether)

So the probability = 1/6

Example: there are 5 marbles in a bag: 4 are blue, and 1 is red.

What is the probability that a blue marble gets picked?

Number of ways it can happen: 4 (there are 4 blues)

Total number of outcomes: 5 (there are 5 marbles in total)

So the probability = 4/5 = 0.8

Probability Line

We can show probability on a Probability Line :

Probability is always between 0 and 1

Probability is Just a Guide

Probability does not tell us exactly what will happen, it is just a guide

Example: toss a coin 100 times, how many Heads will come up?

Probability says that heads have a chance, so we can expect 50 Heads.

But when we actually try it we might get 48 heads, or 55 heads ... or anything
really, but in most cases it will be a number near 50.

Words

Some words have special meaning in Probability:

Experiment or Trial: an action where the result is uncertain.

Tossing a coin, throwing dice, seeing what pizza people choose are all examples
of experiments.

Sample Space: all the possible outcomes of an experiment

Example: choosing a card from a deck

There are 52 cards in a deck (not including Jokers)

So the Sample Space is all 52 possible cards: {Ace of Hearts, 2 of Hearts,

etc... }

The Sample Space is made up of Sample Points:

Sample Point: just one of the possible outcomes

Example: Deck of Cards

the 5 of Clubs is a sample point

the King of Hearts is a sample point
"King" is not a sample point. As there are 4 Kings that is 4 different sample
points.

Event: a single result of an experiment

Example Events:

Getting a Tail when tossing a coin is an event

Rolling a "5" is an event.

An event can include one or more possible outcomes:

Choosing a "King" from a deck of cards (any of the 4 Kings) is an

event
Rolling an "even number" (2, 4 or 6) is also an event

The Sample Space is all possible

outcomes.

A Sample Point is just one possible

outcome.

And an Event can be one or more of the

possible outcomes.

Hey, let's use those words, so you get used to them:

Example: Alex wants to see how many times a "double" comes up
when throwing 2 dice.

Each time Alex throws the 2 dice is an Experiment.

It is an Experiment because the result is uncertain.

The Event Alex is looking for is a "double", where both dice have the same
number. It is made up of these 6 Sample Points:

{1,1} {2,2} {3,3} {4,4} {5,5} and {6,6}

The Sample Space is all possible outcomes (36 Sample Points):

{1,1} {1,2} {1,3} {1,4} ... {6,3} {6,4} {6,5} {6,6}

These are Alex's Results:

Experiment Is it a Double?
{3,4} No
{5,1} No
{2,2} Yes
{6,3} No
... ...
After 100 Experiments, Alex has 19 "double" Events ... is that close to what
you would expect?

Probability: Complement
Complement of an Event: All outcomes that are NOT the event.

When the event is Heads, the complement is Tails

When the event is {Monday, Wednesday} the complement

is {Tuesday, Thursday, Friday, Saturday, Sunday}

When the event is {Hearts} the complement is {Spades,

Clubs, Diamonds, Jokers}

So the Complement of an event is all the other outcomes (not the ones we
want).

And together the Event and its Complement make all possible outcomes.

Probability
Probability of an event happening = Number of ways it can happen/Total
number of outcomes

Example: the chances of rolling a "4" with a die

Number of ways it can happen: 1 (there is only 1 face with a "4" on it)
Total number of outcomes: 6 (there are 6 faces altogether)

So the probability = 16

The probability of an event is shown using "P":

P(A) means "Probability of Event A"

The complement is shown by a little mark after the letter such as A' (or
sometimes Ac or A):

P(A') means "Probability of the complement of Event A"

The two probabilities always add to 1

P(A) + P(A') = 1

Example: Rolling a "5" or "6"

Event A is {5, 6}

Number of ways it can happen: 2

Total number of outcomes: 6

P(A) = 2/6 = 1/3

The Complement of Event A is {1, 2, 3, 4}

Number of ways it can happen: 4

Total number of outcomes: 6

P(A) = 4/6 = 2/3

Let us add them:

P(A) + P(A') = 1/3 + 2/3 = 3/3 = 1

Yep, that makes 1

It makes sense, right? Event A plus all outcomes that are not Event A make
up all possible outcomes.

Why is the Complement Useful?

It is sometimes easier to work out the complement first.

Example. Throw two dice. What is the probability the two scores
are different?

Different scores are like getting a 2 and 3, or a 6 and 1. It is quite a long list:

A = { (1,2), (1,3), (1,4), (1,5), (1,6),

(2,1), (2,3), (2,4), ... etc ! }
But the complement (which is when the two scores are the same) is only 6
outcomes:

A' = { (1,1), (2,2), (3,3), (4,4), (5,5), (6,6) }

And the probability is easy to work out:

P(A') = 6/36 = 1/6

Knowing that P(A) and P(A') together make 1, we can calculate:

P(A) = 1 P(A')
= 1 1/6
= 5/6

So in this case (and many others) it's easier to work out P(A') first, then find
P(A)

Random Variables
A Random Variable is a set of possible values from a random experiment.

Example: Tossing a coin: we could get Heads or Tails.

Let's give them the values Heads=0 and Tails=1 and we have a Random
Variable "X":
In short:

X = {0, 1}

Note: We can choose Heads=100 and Tails=150 or other values if we want! It is

our choice.

So:

We have an experiment (such as tossing a coin)

We give values to each event
The set of values is a Random Variable

Not Like an Algebra Variable

In Algebra a variable, like x, is an unknown value:

Example: x + 2 = 6

In this case we can find that x=4

But a Random Variable is different ...

A Random Variable has a whole set of values ...

... and it could take on any of those values, randomly.

 Example: X = {0, 1, 2, 3}

X could be 0, 1, 2, or 3 randomly.

And they might each have a different probability.

Capital Letters
We use a capital letter, like X or Y, to avoid confusion with the Algebra type of
variable.

Sample Space
A Random Variable's set of values is the Sample Space.

Example: Throw a die once

Random Variable X = "The score shown on the top face".

X could be 1, 2, 3, 4, 5 or 6

So the Sample Space is {1, 2, 3, 4, 5, 6}

Probability
We can show the probability of any one value using this style:

P(X = value) = probability of that value

Example (continued): Throw a die once

X = {1, 2, 3, 4, 5, 6}

In this case they are all equally likely, so the probability of any one is 1/6

P(X = 1) = 1/6
P(X = 2) = 1/6
P(X = 3) = 1/6
P(X = 4) = 1/6
P(X = 5) = 1/6
P(X = 6) = 1/6

Note that the sum of the probabilities = 1, as it should be.

Example: Toss three coins.

X = "The number of Heads" is the Random Variable.

In this case, there could be 0 Heads (if all the coins land Tails up), 1 Head, 2
Heads or 3 Heads.

So the Sample Space = {0, 1, 2, 3}

But this time the outcomes are NOT all equally likely.

The three coins can land in eight possible ways:

X = "number
of Heads"
 HHH 3

HHT 2

HTH 2

HTT 1

THH 2

THT 1

TTH 1

TTT 0

Looking at the table we see just 1 case of Three Heads, but 3 cases of Two
Heads, 3 cases of One Head, and 1 case of Zero Heads. So:

P(X = 3) = 1/8
P(X = 2) = 3/8
P(X = 1) = 3/8
P(X = 0) = 1/8
 Example: Two dice are tossed.

The Random Variable is X = "The sum of the scores on the two dice".

Let's make a table of all possible values:

1st Die
1 2 3 4 5 6
1 2 3 4 5 6 7
2nd
Die 2 3 4 5 6 7 8
3 4 5 6 7 8 9
4 5 6 7 8 9 10
5 6 7 8 9 10 11
6 7 8 9 10 11 12

There are 6 6 = 36 of them, and the Sample Space = {2, 3, 4, 5, 6, 7, 8, 9,

10, 11, 12}

Let's count how often each value occurs, and work out the probabilities:

2 occurs just once, so P(X = 2) = 1/36

3 occurs twice, so P(X = 3) = 2/36 = 1/18
4 occurs three times, so P(X = 4) = 3/36 = 1/12
5 occurs four times, so P(X = 5) = 4/36 = 1/9
6 occurs five times, so P(X = 6) = 5/36
7 occurs six times, so P(X = 7) = 6/36 = 1/6
8 occurs five times, so P(X = 8) = 5/36
9 occurs four times, so P(X = 9) = 4/36 = 1/9
10 occurs three times, so P(X = 10) = 3/36 = 1/12
11 occurs twice, so P(X = 11) = 2/36 = 1/18
 12 occurs just once, so P(X = 12) = 1/36

A Range of Values
We could also calculate the probability that a Random Variable takes on a range
of values.

Example (continued) What is the probability that the sum of

the scores is 5, 6, 7 or 8?

In other words: What is P(5 X 8)?

P(5 X 8) = P(X = 5) + P(X = 6) + P(X = 7) + P(X = 8) = (4+5+6+5)/36 =

20/36 = 5/9

Solving
We can also solve a Random Variable equation.

Example (continued) If P(X = x) = 1/12, what is the value of

x?

P(X = 4) = 1/12, and P(X = 10) = 1/12

So there are two solutions: x = 4 or x = 10

Notice the different uses of X and x:

X is the Random Variable "The sum of the scores on the two dice".
x is a value that X can take.

Continuous
 Random Variables can be either Discrete or Continuous :

Discrete Data can only take certain values (such as 1,2,3,4,5)

Continuous Data can take any value within a range (such as a person's
height)

All our examples have been Discrete.

Summary
A Random Variable is a set of possible values from a random experiment.

The set of possible values is called the Sample Space.

A Random Variable is given a capital letter, such as X or Z.

Random Variables can be discrete or continuous.

Random Variables:
Mean, Variance and
Standard Deviation
A Random Variable is a set of possible values from a random experiment.

Example: Tossing a coin: we could get Heads or Tails.

Let's give them the values Heads=0 and Tails=1 and we have a Random
Variable "X":
So:

We have an experiment (like tossing a coin)

We give values to each event
The set of values is a Random Variable

Learn more at Random Variables .

Mean, Variance and Standard Deviation

Example: Tossing a single unfair die

For fun, imagine a weighted die (cheating!) so we have these probabilities:

1 2 3 4 5 6
0.1 0.1 0.1 0.1 0.1 0.5

Mean or Expected Value:

When we know the probability p of every value x we can calculate the Expected
Value (Mean) of X:
 = xp

Note: is Sigma Notation , and means to sum up.

To calculate the Expected Value:

multiply each value by its probability

sum them up

Example continued:
x 1 2 3 4 5 6
p 0.1 0.1 0.1 0.1 0.1 0.5
xp 0.1 0.2 0.3 0.4 0.5 3

= xp = 0.1+0.2+0.3+0.4+0.5+3 = 4.5

The expected value is 4.5

Note: this is a weighted mean : values with higher probability have higher
contribution to the mean.

Variance: Var(X)

The Variance is:

Var(X) = x2p 2

To calculate the Variance:

 square each value and multiply by its probability
sum them up and we get x2p
then subtract the square of the Expected Value 2

Example continued:
x 1 2 3 4 5 6
p 0.1 0.1 0.1 0.1 0.1 0.5
x2p 0.1 0.4 0.9 1.6 2.5 18

x2p = 0.1+0.4+0.9+1.6+2.5+18 = 23.5

Var(X) = x2p 2 = 23.5 - 4.52 = 3.25

The variance is 3.25

Standard Deviation:

The Standard Deviation is the square root of the Variance:

= Var(X)

Example continued:
x 1 2 3 4 5 6
p 0.1 0.1 0.1 0.1 0.1 0.5
 x 2p 0.1 0.4 0.9 1.6 2.5 18

= Var(X) = 3.25 = 1.803...

The Standard Deviation is 1.803...

Let's have another example!

(Note that we run the table downwards instead of along this time.)

You plan to open a new McDougals Fried Chicken, and found

these stats for similar restaurants:
Percent Year's Earnings
20% $50,000 Loss
30% $0
40% $50,000 Profit
10% $150,000 Profit

Using that as probabilities for your new restaurant's profit, what is the
Expected Value and Standard Deviation?

The Random Variable is X = 'possible profit'.

Sum up xp and x2p:

Probability Earnings ($'000s)

p x xp x2p
0.2 -50 -10 500
0.3 0 0 0
0.4 50 20 1000
0.1 150 15 2250
 p = 1 xp = 25 x2p = 3750

= xp = 25

Var(X) = x2p 2 = 3750 252 = 3750 625 = 3125

= 3125 = 56 (to nearest whole number)

But remember these are in thousands of dollars, so:

= $25,000
= $56,000

So you might expect to make $25,000, but with a very wide deviation possible.

Let's try that again, but with a much higher probability for $50,000:

Example (continued):

Now with different probabilities (the $50,000 value has a high probability
of 0.7 now):

Probability Earnings ($'000s)

p x xp x2p
0.1 -50 -5 250
0.1 0 0 0
0.7 50 35 1750
0.1 150 15 2250
p = 1 Sums: xp = 45 x2p = 4250

= xp = 45

Var(X) = x2p 2 = 4250 452 = 4250 2025 = 2225

 = 2225 = 47 (to nearest whole number)

In thousands of dollars:

= $45,000
= $47,000

The mean is now much closer to the most probable value.

And the standard deviation is a little smaller (showing that the values are more
central.)

Continuous
Random Variables can be either Discrete or Continuous :

Discrete Data can only take certain values (such as 1,2,3,4,5)

Continuous Data can take any value within a range (such as a person's
height)

Here we looked only at discrete data, as finding the Mean, Variance and
Standard Deviation of continuous data needs Integration .

Summary
A Random Variable is a variable whose possible values are numerical
outcomes of a random experiment.

The Mean (Expected Value) is: = xp

 The Variance is: Var(X) = x2p 2

The Standard Deviation is: = Var(X)