PROBABILITY THEORY AND

STOCHASTIC PROCESSES

1
 UNIT – 1

PROBABILITY AND RANDOM VARIABLE

PROBABILITY

Introduction

It is remarkable that a science which began with the consideration of games of chance
should have become the most important object of human knowledge.

A brief history

Probability has an amazing history. A practical gambling problem faced by the French nobleman
Chevalier de Méré sparked the idea of probability in the mind of Blaise Pascal (1623-1662), the
famous French mathematician. Pascal's correspondence with Pierre de Fermat (1601-1665), another
French Mathematician in the form of seven letters in 1654 is regarded as the genesis of probability.
Early mathematicians like Jacob Bernoulli (1654-1705), Abraham de Moivre (1667-1754), Thomas
Bayes (1702-1761) and Pierre Simon De Laplace (1749-1827) contributed to the development of
probability. Laplace's Theory Analytique des Probabilities gave comprehensive tools to calculate
probabilities based on the principles of permutations and combinations. Laplace also said,
"Probability theory is nothing but common sense reduced to calculation."

Later mathematicians like Chebyshev (1821-1894), Markov (1856-1922), von Mises (1883-
1953), Norbert Wiener (1894-1964) and Kolmogorov (1903-1987) contributed to new
developments. Over the last four centuries and a half, probability has grown to be one of the most
essential mathematical tools applied in diverse fields like economics, commerce, physical
sciences, biological sciences and engineering. It is particularly important for solving practical
electrical-engineering problems in communication, signal processing and computers.
Notwithstanding the above developments, a precise definition of probability eluded the
mathematicians for centuries. Kolmogorov in 1933 gave the axiomatic definition of probability
and resolved the problem.

Randomness arises because of

 random nature of the generation mechanism

 Limited understanding of the signal dynamics inherent imprecision in
measurement, observation, etc.
For example, thermal noise appearing in an electronic device is generated due to random motion of
electrons. We have deterministic model for weather prediction; it takes into account of the factors
affecting weather. We can locally predict the temperature or the rainfall of a place on the basis of
previous data. Probabilistic models are established from observation of a random phenomenon.
While probability is concerned with analysis of a random phenomenon, statistics help in building
such models from data.

2
Deterministic versus probabilistic models

A deterministic model can be used for a physical quantity and the process generating it provided
sufficient information is available about the initial state and the dynamics of the process generating
the physical quantity. For example,

 We can determine the position of a particle moving under a constant force if we know
the initial position of the particle and the magnitude and the direction of the force.

 We can determine the current in a circuit consisting of resistance, inductance

and capacitance for a known voltage source applying Kirchoff's laws.

Many of the physical quantities are random in the sense that these quantities cannot be predicted
with certainty and can be described in terms of probabilistic models only. For example,

 The outcome of the tossing of a coin cannot be predicted with certainty. Thus
the outcome of tossing a coin is random.

 The number of ones and zeros in a packet of binary data arriving through
a communication channel cannot be precisely predicted is random.

 The ubiquitous noise corrupting the signal during acquisition, storage and
transmission can be modelled only through statistical analysis.

How to Interpret Probability

Mathematically, the probability that an event will occur is expressed as a number between 0 and 1.
Notationally, the probability of event A is represented by P (A).

 If P (A) equals zero, event A will almost definitely not occur.

 If P (A) is close to zero, there is only a small chance that event A will occur.
 If P (A) equals 0.5, there is a 50-50 chance that event A will occur.
 If P(A) is close to one, there is a strong chance that event A will occur.
 If P(A) equals one, event A will almost definitely occur.

In a statistical experiment, the sum of probabilities for all possible outcomes is equal to one. This
means, for example, that if an experiment can have three possible outcomes (A, B, and C), then
P(A) + P(B) + P(C) = 1.

3
Applications

 Probability theory is applied in everyday life in risk assessment and in trade on financial
markets.
 Governments apply probabilistic methods in environmental regulation, where it is called
pathway analysis
 Another significant application of probability theory in everyday life is reliability. Many
consumer products, such as automobiles and consumer electronics, use reliability theory in
product design to reduce the probability of failure. Failure probability may influence a
manufacturer's decisions on a product's warranty.

THE BASIC CONCEPTS OF SET THEORY

Some of the basic concepts of set theory are:

Set: A set is a well defined collection of objects. These objects are called elements or members of
the set. Usually uppercase letters are used to denote sets.

The set theory was developed by George Cantor in 1845-1918. Today, it is used in almost every
branch of mathematics and serves as a fundamental part of present-day mathematics.

Examples of sets:

1. Describe the set of vowels.

If A is the set of vowels, then A could be described as A = {a, e, i, o, u}.

2.Describe the set of positive integers.

Since it would be impossible to list all of the positive integers, we need to use a rule to describe
this set. We might say A consists of all integers greater than zero.

3. Set A = {1, 2, 3} and Set B = {3, 2, 1}. Is Set A equal to Set B?

Yes. Two sets are equal if they have the same elements. The order in which the elements are
listed does not matter.

4. What is the set of men with four arms?

Since all men have two arms at most, the set of men with four arms contains no elements. It is the
null set (or empty set).

5. Set A = {1, 2, 3} and Set B = {1, 2, 4, 5, 6}. Is Set A a subset of Set B?

Set A would be a subset of Set B if every element from Set A were also in Set B. However, this is
not the case. The number 3 is in Set A, but not in Set B. Therefore, Set A is not a subset of Set B

4
 Some important sets used in mathematics are

N: the set of all natural numbers = {1, 2, 3, 4,......}

Z: the set of all integers = {....., -3, -2, -1, 0, 1, 2, 3,......}

Q: the set of all rational numbers

R: the set of all real numbers

Z+: the set of all positive integers

W: the set of all whole numbers

The different types of sets are explained below with examples.

1. Empty Set or Null Set:

A set which does not contain any element is called an empty set, or the null set or the void set and
it is denoted by ∅ and is read as phi. In roster form, ∅ is denoted by {}. An empty set is a finite set,
since the number of elements in an empty set is finite, i.e., 0.

For example: (a) the set of whole numbers less than 0.

 Clearly there is no whole number less than 0.

Therefore, it is an empty set.

(c) N = {x : x ∈ N, 3 < x < 4}

 Let A = {x : 2 < x < 3, x is a natural number}

Here A is an empty set because there is no natural number between 2 and 3.

Note:

∅ ≠ {0} ∴ has no element.

{0} is a set which has one element 0.

The cardinal number of an empty set, i.e., n(∅) = 0

2. Singleton Set:

A set which contains only one element is called a singleton set.

For example:

 A = {x : x is neither prime nor composite}

5
 It is a singleton set containing one element, i.e., 1.

 B = {x : x is a whole number, x < 1}

This set contains only one element 0 and is a singleton set.

 Let A = {x : x ∈ N and x² = 4}

Here A is a singleton set because there is only one element 2 whose square is 4.

Let B = {x : x is a even prime number}

Here B is a singleton set because there is only one prime number which is even, i.e., 2.

3. Finite Set:

A set which contains a definite number of elements is called a finite set. Empty set is also called a
finite set.

For example:
 The set of all colors in the rainbow.
 N = {x : x ∈ N, x < 7}
 P = {2, 3, 5, 7, 11, 13, 17,. . . . . 97}

4. Infinite Set:

The set whose elements cannot be listed, i.e., set containing never-ending elements is called an
infinite set.

For example:

 Set of all points in a plane

 A = {x : x ∈ N, x > 1}

 Set of all prime numbers

 B = {x : x ∈ W, x = 2n}

Note:

All infinite sets cannot be expressed in roster form.

For example:

The set of real numbers since the elements of this set do not follow any particular pattern.

6
5. Cardinal Number of a Set:

The number of distinct elements in a given set A is called the cardinal number of A. It is denoted
by n(A).

For example:

 A {x : x ∈ N, x < 5}

A = {1, 2, 3, 4}

Therefore, n(A) = 4

 B = set of letters in the word

ALGEBRA

B = {A, L, G, E, B, R}

Therefore, n(B) = 6

6. Equivalent Sets:

Two sets A and B are said to be equivalent if their cardinal number is same, i.e., n(A) = n(B). The
symbol for denoting an equivalent set is “↔”.

For example:

A = {1, 2, 3} Here n(A) = 3

B = {p, q, r} Here n(B) = 3

Therefore, A ↔ B

7. Equal sets:

Two sets A and B are said to be equal if they contain the same elements. Every element of A is an
element of B and every element of B is an element of A.

For example:

A = {p, q, r, s}

B = {p, s, r, q}

Therefore, A =

8. Disjoint Sets:

Two sets A and B are said to be disjoint, if they do not have any element in common.

7
 For example;

A = {x : x is a prime number}

B = {x : x is a composite number}.

Clearly, A and B do not have any element in common and are disjoint sets.

9. Overlapping sets:

Two sets A and B are said to be overlapping if they contain at least one element in common.

For example;
• A = {a, b, c, d}
B = {a, e, i, o, u}
• X = {x : x ∈ N, x < 4}

Y = {x : x ∈ I, -1 < x <

4}

Here, the two sets contain three elements in common, i.e., (1, 2, 3)

10. Definition of Subset:

If A and B are two sets, and every element of set A is also an element of set B, then A is called a
subset of B and we write it as A ⊆ B or B ⊇ A

The symbol ⊂ stands for ‗is a subset of‘ or ‗is contained in‘

• Every set is a subset of itself, i.e., A ⊂ A, B ⊂ B.

• Empty set is a subset of every set.

• Symbol ‗⊆‘ is used to denote ‗is a subset of‘ or ‗is contained in‘.

• A ⊆ B means A is a subset of B or A is contained in B.

• B ⊆ A means B contains A.

Examples;

1. Let A = {2, 4, 6}

B = {6, 4, 8, 2}

Here A is a subset of B

8
 Since, all the elements of set A are contained in set

B. But B is not the subset of A

Since, all the elements of set B are not contained in set A.

Notes:

i. If ACB and BCA, then A = B, i.e., they are

equal sets. Every set is a subset of itself.

Null set or ∅ is a subset of every set.

ii. The set N of natural numbers is a subset of the set Z of integers and we write N ⊂ Z.

iii. Let A = {2, 4, 6}

B = {x : x is an even natural number less than 8} Here A ⊂ B and B ⊂ A.

Hence, we can say A = B

iv. Let A = {1, 2, 3, 4}

B = {4, 5, 6, 7}

Here A ⊄ B and also B ⊄ C [⊄ denotes ‗not a subset of‘]

11. Super Set:

Whenever a set A is a subset of set B, we say the B is a superset of A and we write, B ⊇ A.

Symbol ⊇ is used to denote ‗is a super set of‘

For example;

A = {a, e, i, o, u}

B = {a, b, c,.............., z}

Here A ⊆ B i.e., A is a subset of B but B ⊇ A i.e., B is a super set of A

a. Proper Subset:

If A and B are two sets, then A is called the proper subset of B if A ⊆ B but B ⊇ A i.e., A ≠ B. The
symbol ‗⊂‘ is used to denote proper subset. Symbolically, we write A ⊂ B.

For example;

1. A = {1, 2, 3, 4}

Here n(A) = 4
9
 B = {1, 2, 3, 4, 5}

Here n(B) = 5

We observe that, all the elements of A are present in B but the element ‗5‘ of B is not present in A.

So, we say that A is a proper subset of B.

Symbolically, we write it as A ⊂ B

Notes:

No set is a proper subset of itself.

Null set or ∅ is a proper subset of every set.

2. A = {p, q, r}

B = {p, q, r, s, t}

Here A is a proper subset of B as all the elements of set A are in set B and also A ≠ B.

Notes:

No set is a proper subset of itself.

Empty set is a proper subset of every set.

12 Power Set:

The collection of all subsets of set A is called the power set of A. It is denoted by P(A). In P(A),
every element is a set.

For example;

If A = {p, q} then all the subsets of A will

be P(A) = {∅, {p}, {q}, {p, q}}

Number of elements of P(A) = n[P(A)] = 4 = 22

In general, n[P(A)] = 2m where m is the number of elements in set A.

13 Universal Set

A set which contains all the elements of other given sets is called a universal set. The symbol for
denoting a universal set is 𝖴 or ξ.

For example;

1. If A = {1, 2, 3} B = {2, 3, 4} C = {3, 5, 7}

then U = {1, 2, 3, 4, 5, 7}
10
[Here A ⊆ U, B ⊆ U, C ⊆ U and U ⊇ A, U ⊇ B, U ⊇ C]

2. If P is a set of all whole numbers and Q is a set of all negative numbers then the universal set is
a set of all integers.

3. If A = {a, b, c} B = {d, e} C = {f, g, h, i}

then U = {a, b, c, d, e, f, g, h, i} can be taken as universal set.

Operations on sets:
1. Definition of Union of Sets:

Union of two given sets is the smallest set which contains all the elements of both the sets.

To find the union of two given sets A and B is a set which consists of all the elements of A and all
the elements of B such that no element is repeated.

The symbol for denoting union of sets is

“𝖴”. Some properties of the operation of

union:

(i) A𝖴B = B𝖴A (Commutative law)

(ii) A𝖴(B𝖴C) = (A𝖴B)𝖴C (Associative law)

(iii) A 𝖴 Ф = A (Law of identity element, is the identity of 𝖴)

(iv) A𝖴A = A (Idempotent law)

(v) U𝖴A = U (Law of 𝖴) 𝖴 is the universal set.

Notes:

A 𝖴 Ф = Ф𝖴 A = A i.e. union of any set with the empty set is always the set itself.

Examples:

1. If A = {1, 3, 7, 5} and B = {3, 7, 8, 9}. Find union of two set A and B.

Solution:

A 𝖴 B = {1, 3, 5, 7, 8, 9}
No element is repeated in the union of two sets. The common elements 3, 7 are taken only once.

2. Let X = {a, e, i, o, u} and Y = {ф}. Find union of two given sets X and Y.

Solution:
11
X 𝖴 Y = {a, e, i, o, u}

Therefore, union of any set with an empty set is the set itself.

2. Definition of Intersection of Sets:

Intersection of two given sets is the largest set which contains all the elements that are common to
both the sets.

To find the intersection of two given sets A and B is a set which consists of all the elements which
are common to both A and B.

The symbol for denoting intersection of sets is “∩”.

Some properties of the operation of intersection

(i) A∩B = B∩A (Commutative law)

(ii) (A∩B)∩C = A∩ (B∩C) (Associative law)

(iii) Ф ∩ A = Ф (Law of Ф)

(iv) U∩A = A (Law of 𝖴)

(v) A∩A = A (Idempotent law)

(vi) A∩(B𝖴C) = (A∩B) 𝖴 (A∩C) (Distributive law) Here ∩ distributes over 𝖴

Also, A𝖴(B∩C) = (AUB) ∩ (AUC) (Distributive law) Here 𝖴 distributes over ∩

Notes:

A ∩ Ф = Ф ∩ A = Ф i.e. intersection of any set with the empty set is always the empty set.

Solved examples :

1. If A = {2, 4, 6, 8, 10} and B = {1, 3, 8, 4, 6}. Find intersection of two set A and B.

Solution:

A ∩ B = {4, 6, 8}

Therefore, 4, 6 and 8 are the common elements in both the sets.

2. If X = {a, b, c} and Y = {ф}. Find intersection of two given sets X and Y.

Solution:

X∩Y={}

3. Difference of two sets

12
If A and B are two sets, then their difference is given by A - B or B - A.

• If A = {2, 3, 4} and B = {4, 5, 6}

A - B means elements of A which are not the elements of

B. i.e., in the above example A - B = {2, 3}

In general, B - A = {x : x ∈ B, and x ∉ A}
• If A and B are disjoint sets, then A – B = A and B – A =

B Solved examples to find the difference of two sets:

1. A = {1, 2, 3} and B = {4, 5, 6}.

Find the difference between the two sets:

(i) A and B

(ii) B and A

Solution:

The two sets are disjoint as they do not have any elements in

common. (i) A - B = {1, 2, 3} = A

(ii) B - A = {4, 5, 6} = B

2. Let A = {a, b, c, d, e, f} and B = {b, d, f, g}.

Find the difference between the two sets:

(i) A and B

(ii) B and A

Solution:

(i) A - B = {a, c, e}

Therefore, the elements a, c, e belong to A but not to B

(ii) B - A = {g)

Therefore, the element g belongs to B but not A.

4. Complement of a Set

In complement of a set if U be the universal set and if A is a subset of U then the complement of
A is the set of all elements of U which are not the elements of A.

13
Symbolically, we denote the complement of A with respect to U as A‘.

14
Some properties of complement sets

(i) A 𝖴 A' = A' 𝖴 A = 𝖴 (Complement law)

(ii) (A ∩ A') = ϕ (Complement law) - The set and its complement are disjoint sets.

(iii) (A 𝖴 B)’ = A' ∩ B' (De Morgan‘s law)

(iv) (A ∩ B)' = A' 𝖴 B' (De Morgan‘s law)

(v) (A')' = A (Law of complementation)

(vi) Ф' = 𝖴 (Law of empty set - The complement of an empty set is a universal set.

(vii) 𝖴' = Ф and universal set) - The complement of a universal set is an empty set.

For Example; If S = {1, 2, 3, 4, 5, 6, 7}

A = {1, 3, 7} find A'.

Solution:

We observe that 2, 4, 5, 6 are the only elements of S which do not belong to A.

Therefore, A' = {2, 4, 5, 6}

Algebraic laws on sets:

1. Commutative Laws:

For any two finite sets A and B;

(i) A U B = B U A

(ii) A ∩ B = B ∩ A

2. Associative Laws:

For any three finite sets A, B and C;

(i) (A U B) U C = A U (B U C)

(ii) (A ∩ B) ∩ C = A ∩ (B ∩ C)

Thus, union and intersection are associative.

15
3. Idempotent Laws:

For any finite set A;

(i) A U A = A

(ii) A ∩ A = A

4. Distributive Laws:

For any three finite sets A, B and C;

(i) A U (B ∩ C) = (A U B) ∩ (A U C)

(ii) A ∩ (B U C) = (A ∩ B) U (A ∩ C)

Thus, union and intersection are distributive over intersection and union respectively.

5. De Morgan’s Laws:

For any two finite sets A and B;

(i) A – (B U C) = (A – B) ∩ (A – C)

(ii) A - (B ∩ C) = (A – B) U (A – C)

De Morgan‘s Laws can also we written as:

(i) (A U B)‘ = A' ∩ B'

(ii) (A ∩ B)‘ = A' U B'

More laws of algebra of sets:

6. For any two finite sets A and B;

(i) A – B = A ∩ B'

(ii) B – A = B ∩ A'

(iii) A – B = A ⇔ A ∩ B = ∅

(iv) (A – B) U B = A U B

(v) (A – B) ∩ B = ∅

(vi) (A – B) U (B – A) = (A U B) – (A ∩ B)

16
Definition of De Morgan’s law:

The complement of the union of two sets is equal to the intersection of their complements and the
complement of the intersection of two sets is equal to the union of their complements. These are
called De Morgan’s laws.

For any two finite sets A and B;

(i) (A U B)' = A' ∩ B' (which is a De Morgan's law of union).

(ii) (A ∩ B)' = A' U B' (which is a De Morgan's law of intersection).

Venn Diagrams:

Pictorial representations of sets represented by closed figures are called set diagrams or Venn
diagrams.

Venn diagrams are used to illustrate various operations like union, intersection and

difference. We can express the relationship among sets through this in a more significant

way.

In this,

•A rectangle is used to represent a universal set.

• Circles or ovals are used to represent other subsets of the universal set.

Venn diagrams in different situations

In these diagrams, the universal set is represented by a rectangular region and its subsets by circles
inside the rectangle. We represented disjoint set by disjoint circles and intersecting sets by
intersecting circles.

S.No Set &Its relation Venn Diagram

1 Intersection of A and B

17
2 Union of A and B

3 Difference : A-B

4 Difference : B-A

5 Complement of set A

6 A 𝖴 B when A ⊂ B

18
7 A 𝖴 B when neither A ⊂ B
nor B ⊂ A

8 A 𝖴 B when A and B are

disjoint sets

9 (A 𝖴 B)’ (A union B dash)

10 (A ∩ B)’ (A intersection
B dash)

11 B’ (B dash)

12 (A - B)’ (Dash of sets A

minus B)

19
13 (A ⊂ B)’ (Dash of A subset
B)

Problems of set theory:

1. Let A and B be two finite sets such that n(A) = 20, n(B) = 28 and n(A 𝖴 B) = 36, find n(A ∩ B).

Solution:

Using the formula n(A 𝖴 B) = n(A) + n(B) - n(A ∩ B).

then n(A ∩ B) = n(A) + n(B) - n(A 𝖴 B)

= 20 + 28 - 36

= 48 - 36

= 12

2. If n(A - B) = 18, n(A 𝖴 B) = 70 and n(A ∩ B) = 25, then find n(B).

Solution:

Using the formula n(A𝖴B) = n(A - B) + n(A ∩ B) + n(B - A)

70 = 18 + 25 + n(B - A)

70 = 43 + n(B - A)

n(B - A) = 70 - 43

n(B - A) = 27

Now n(B) = n(A ∩ B) + n(B - A)

= 25 + 27

= 52

20
3. In a group of 60 people, 27 like cold drinks and 42 like hot drinks and each person likes at
least one of the two drinks. How many like both coffee and tea?

Solution:

Let A = Set of people who like cold

drinks. B = Set of people who like hot

drinks.

Given

(A 𝖴 B) = 60 n(A) = 27 n(B) = 42

then; n(A ∩ B) = n(A) + n(B) - n(A 𝖴 B)

= 27 + 42 - 60

= 69 - 60 = 9

=9

Therefore, 9 people like both tea and coffee.

4. There are 35 students in art class and 57 students in dance class. Find the number of
students who are either in art class or in dance class.

• When two classes meet at different hours and 12 students are enrolled in both activities.

• When two classes meet at the same hour.

Solution:

n(A) = 35, n(B) = 57, n(A ∩ B) = 12

(Let A be the set of students in art class.

B be the set of students in dance class.)

(i) When 2 classes meet at different hours n(A 𝖴 B) = n(A) + n(B) - n(A ∩ B)

= 35 + 57 - 12

= 92 - 12

= 80

21
(ii) When two classes meet at the same hour, A∩B = ∅ n (A 𝖴 B) = n(A) + n(B) - n(A ∩ B)

= n(A) + n(B)

= 35 + 57

= 92

5. In a group of 100 persons, 72 people can speak English and 43 can speak French. How many
can speak English only? How many can speak French only and how many can speak both English
and French?

Solution:

Let A be the set of people who speak

English. B be the set of people who speak

French.

A - B be the set of people who speak English and not French.

B - A be the set of people who speak French and not English.

A ∩ B be the set of people who speak both French and

English. Given,

n(A) = 72 n(B) = 43 n(A 𝖴 B) = 100

Now, n(A ∩ B) = n(A) + n(B) - n(A 𝖴 B)

= 72 + 43 - 100

= 115 - 100

= 15

Therefore, Number of persons who speak both French and English =

15 n(A) = n(A - B) + n(A ∩ B)

⇒ n(A - B) = n(A) - n(A ∩ B)

= 72 - 15

22
 = 57

and n(B - A) = n(B) - n(A ∩ B)

= 43 - 15

= 28

Therefore, Number of people speaking English only =

57 Number of people speaking French only = 28

Probability Concepts

Before we give a definition of probability, let us examine the following concepts:

1. Experiment:

In probability theory, an experiment or trial (see below) is any procedure that can be
infinitely repeated and has a well-defined set of possible outcomes. An experiment is said to
be random if it has more than one possible outcome, and deterministic if it has only one
possible outcome. A random experiment that has exactly two (mutually exclusive) possible
outcomes is known as a Bernoulli trial.

Random Experiment:

An experiment is a random experiment if its outcome cannot be predicted precisely. One

out of a number of outcomes is possible in a random experiment. A single performance of
the random experiment is called a trial.

Mathematical description of an experiment:

A random experiment is described or modeled by a mathematical construct known as a probability
space. A probability space is constructed and defined with a specific kind of experiment or trial in
mind.
A mathematical description of an experiment consists of three parts:

1. A sample space, Ω (or S), which is the set of all possible outcomes.
2. A set of events , where each event is a set containing zero or more outcomes.
3. The assignment of probabilities to the events—that is, a function P mapping from events to
probabilities.

23
1. Sample Space: The sample space “Ѕ” is the collection of all possible outcomes of
random experiment. The elements of “Ѕ” are called sample points.

a. A sample space may be finite, countably infinite or uncountable.

b. A finite or countably infinite sample space is called a discrete sample space.

c. An uncountable sample space is called a continuous sample space

Ex:1. For the coin-toss experiment would be the results ―Head‖and ―Tail‖, which we may
represent by S={H T}.

Ex. 2. If we toss a die, one sample space or the set of all possible outcomes is

S = { 1, 2, 3, 4, 5, 6}

The other sample space can

be S = {odd, even}

Types of Sample Space:

i. Finite/Discrete Sample Space:

Consider the experiment of tossing a coin

twice. The sample space can be

S = {HH, HT, T H , TT} the above sample space has a finite number of sample points. It
is called a finite sample space.

ii. Countably Infinite Sample Space:

Consider that a light bulb is manufactured. It is then tested for its life length by
inserting it into a socket and the time elapsed (in hours) until it burns out is recorded. Let
the measuring instrument is capable of recording time to two decimal places, for example
8.32 hours.

Now, the sample space becomes count ably infinite i.e.

S = {0.0, 0.01, 0.02 ……………}

The above sample space is called a countable infinite sample space.

iii. Un Countable/ Infinite Sample Space:

If the sample space consists of unaccountably infinite number of elements then it is called

Un Countable/ Infinite Sample Space.

24
2. Event: An event is simply a set of possible outcomes. To be more specific, an event A is a
subset of the sample space S.

a. For a discrete sample space, all subsets are events.

Ex: For instance, in the coin-toss experiment the events A={Heads} and B={Tails} would be
mutually exclusive.

An event consisting of a single point of the sample space 'S' is called a simple event or elementary
event.

Some examples of event sets:

Example 1: tossing a fair coin

The possible outcomes are H (head) and T (tail). The associated sample space is .It
is a finite sample space. The events associated with the sample space are : and

Example 2: Throwing a fair die:

The possible 6 outcomes are:

. . .

The associated finite sample space is . Some events are

And so on.

25
Example 3: Tossing a fair coin until a head is obtained

We may have to toss the coin any number of times before a head is obtained. Thus the possible
outcomes are:
H, TH, TTH, TTTH,
How many outcomes are there? The outcomes are countable but infinite in number.
The countably infinite sample space is .

Example 4 : Picking a real number at random between -1 and +1

The associated Sample space is

Clearly is a continuous sample space.

Example 5: Drawing cards

Drawing 4 cards from a deck: Events include all spades, sum of the 4 cards is (assuming face cards
have a value of zero), a sequence of integers, a hand with a 2, 3, 4 and 5. There are many more
events.

Types of Events:

1. Exhaustive Events:

A set of events is said to be exhaustive, if it includes all the possible events.

Ex. In tossing a coin, the outcome can be either Head or Tail and there is no other possible
outcome. So, the set of events { H , T } is exhaustive.

2. Mutually Exclusive Events:

Two events, A and B are said to be mutually exclusive if they cannot occur together.

i.e. if the occurrence of one of the events precludes the occurrence of all others, then such a set of
events is said to be mutually exclusive otherwise “Non-. Mutually Exclusive Events”.

Ex. In tossing a coin, both head and tail cannot happen at the same time.

1. Equally Likely Events:

If one of the events cannot be expected to happen in preference to another, then such events
are said to be Equally Likely Events.( Or) Each outcome of the random experiment has an
equal chance of occurring.
Ex. In tossing a coin, the coming of the head or the tail is equally likely.

26
 2. Independent Events:

Two events are said to be independent, if happening or failure of one does not affect the happening
or failure of the other. Otherwise, the events are said to be dependent.

Probability Definitions and Axioms:

Classical Definition:

Let the sample space (denoted by ‘S’ ) be the set of all possible distinct outcomes to an experiment. The
probability of some event is
Number of ways the event can occur
Number of outcomes ∈S

provided all points in ‘S’ are equally likely.

For example, when a die is rolled the probability of getting a 2 is 1/6, because one of the six faces is a 2.

(OR)
Consider a random experiment with a finite number of outcomes ‘N’, and if all the outcomes of the experiment
are equally likely , the probability of an event ‘A’ is defined by

where

N= Total Number of outcomes

Example 1 A fair die is rolled once. What is the probability of getting a ‘6’ ?

Here and

Example 2 A fair coin is tossed twice. What is the probability of getting two ‘heads'?

Here and .
Total number of outcomes is 4 and all four outcomes are equally likely. Only outcome favorable to

‘A’ is {HH}

27
 Limitation:

 The classical definition is limited to a random experiment which has only a finite number of
outcomes. In many experiments like that in the above examples, the sample space is finite and
each outcome may be assumed ‘equally likely.' In such cases, the counting method can be
used to compute probabilities of events.

 The notion of equally likely is important here. Equally likely means equally probable. Thus
this definition presupposes that all events occur with equal probability . Thus the definition
includes a concept to be defined

 Relative-frequency based definition of probability

If an experiment is repeated ‘n’ times under similar conditions and the event ‘A’ occurs ‘ n A ‘ times,
then the probability of the event A is defined as

Example: Suppose a die is rolled 500 times. The following table shows the frequency each face.

We see that the relative frequencies are close to 1/6. How do we ascertain that these relative frequencies will
approach to 1/6 as we repeat the experiments infinite no of times?

The Relative frequency has the following properties:

• 0 ≤ P(A) ≤ 1
• P(A) =1 if and only if A occurs every time among the n repetitions.

Limitation:

Since we can never repeat an experiment or process indefinitely, we can never know the probability
of any event from the relative frequency definition. In many cases we can't even obtain a long
series of repetitions due to time, cost, or other limitations. For example, the probability of rain today
can't really be obtained by the relative frequency definition since today can‘t be repeated again.

Probability axioms:
Given an event in a sample space” “which is either finite with elements or countably
infinite with elements, then we can write

and a quantity P(E i) , called the probability of event , is defined such that

28
Axiom1: The probability of any event A is positive or zero. Namely P(A)≥0. The probability
measures, in a certain way, the difficulty of event A happening: the smaller the probability, the
more difficult it is to happen. i.e

Axiom2: The probability of the sure event is 1. Namely P(Ω)=1. And so, the probability is always
greater than 0 and smaller than 1: probability zero means that there is no possibility for it to happen
(it is an impossible event), and probability 1 means that it will always happen (it is a sure event).i.e

Axiom3: The probability of the union of any set of two by two incompatible events is the sum of
the probabilities of the events. That is, if we have, for example, events A,B,C, and these are two by
two incompatible, then P(A𝖴B𝖴C)=P(A)+P(B)+P(C). i.e Additivity:

, where and are mutually exclusive.

for , 2, ..., where , , ... are mutually exclusive

(i.e., ).

Main properties of probability: If A is any event of sample space S then

1. P(A)+P( )=1. Or P( ) = 1-P(A)

2. Since A 𝖴 S , P(A 𝖴 ) =1
3. The probability of the impossible event is 0, i.e P(Ø)=0
4. If A⊂B, then P(A)≤P(B).
5. If A and B are two incompatible events, and therefore, P(A−B)=P(A)
−P(A∩B).and P(B−A)=P(B)−P(A∩B).
6. Addition Law of probability:
P(AUB)=P(A)+P(B)-P(A∩B)

29
Rules of Probability:

Rule of Subtraction:

Rule of Multiplication:

Rule of addition:

30
Conditional probability

The answer is the conditional probability of B given A denoted by . We shall

develop the concept of the conditional probability and explain under what condition this
conditional probability is same as .

Let us consider the case of equiprobable events discussed earlier. Let sample points
be favourable for the joint event A∩B

31
This concept suggests us to define conditional probability. The probability of an event B under the
condition that another event A has occurred is called the conditional probability of B given A and
defined by

We can similarly define the conditional probability of A given B , denoted by .

From the definition of conditional probability, we have the joint probability of

two events A and B as follows

Problems:

Example 1 Consider the example tossing the fair die. Suppose

Example 2 A family has two children. It is known that at least one of the children is a girl. What is
the
probability that both the children are

girls? A = event of at least

one girl B = event of two

girls

32
Properties of Conditional probability:

1. If , then

We have,

2. Since

3. We have,

4. Chain Rule of Probability/Multiplication theorem:

We have,

We can generalize the above to get the chain rule of probability for n events as

33
Joint probability

Joint probability is defined as the probability of both A and B taking place, and is
denoted by P (AB) or P(A∩B)

Joint probability is not the same as conditional probability, though the two concepts are
often confused. Conditional probability assumes that one event has taken place or will take place,
and then asks for the probability of the other (A, given B). Joint probability does not have such
conditions; it simply asks for the chances of both happening (A and B). In a problem, to help
distinguish between the two, look for qualifiers that one event is conditional on the other
(conditional) or whether they will happen concurrently (joint).

Probability definitions can find their way into CFA exam questions. Naturally, there may
also be questions that test the ability to calculate joint probabilities. Such computations require
use of the multiplication rule, which states that the joint probability of A and B is the product of
the conditional probability of A given B, times the probability of B. In probability notation:

P(AB) = P(A | B) * P(B)

Given a conditional probability P(A | B) = 40%, and a probability of B = 60%, the joint
probability P(AB) = 0.6*0.4 or 24%, found by applying the multiplication rule.

P(AUB)=P(A)+P(B)-P(A‫ח‬B)

For independent events: P(AB) = P(A) * P(B)

Moreover, the rule generalizes for more than two events provided they are all independent of one
another, so the joint probability of three events P(ABC) = P(A) * (P(B) *P(C), again assuming
independence.

34
Total Probability theorem:

Let be n events such that

Then for any event B,

Proof: We have

Remark

(1) A decomposition of a set S into 2 or more disjoint nonempty subsets is called a partition of
S.The subsets form a partition of S if

(2) The theorem of total probability can be used to determine the probability of a complex
event in terms of related simpler events. This result will be used in Bays' theorem to be
discussed to the end of the lecture.

35
Bayes' Theorem:

This result is known as the Baye's theorem. The probability is called the a priori probability and
is called the a posteriori probability. Thus the Bays' theorem enables us
to determine the a posteriori probability from the observation that B has occurred. This
result is of practical importance and is the heart of Baysean classification, Baysean estimation etc.

Example1:
In a binary communication system a zero and a one is transmitted with probability 0.6 and 0.4
respectively. Due to error in the communication system a zero becomes a one with a probability
0.1 and a one becomes a zero with a probability 0.08. Determine the probability (i) of receiving a
one and (ii) that a one was transmitted when the received message is one

Solution:
Let S is the sample space corresponding to binary communication. Suppose be event of
Transmitting 0 and be the event of transmitting 1 and and be corresponding events of
receiving 0 and 1 respectively.

36
 Given and

Example 7: In an electronics laboratory, there are identically looking capacitors of three makes
in the ratio 2:3:4. It is known that 1% of , 1.5% of are defective.
What percentages of capacitors in the laboratory are defective? If a capacitor picked at defective
is found to be defective, what is the probability it is of make ?

Let D be the event that the item is defective. Here we have to find .

37
 Independent events

Two events are called independent if the probability of occurrence of one event does not affect the
probability of occurrence of the other. Thus the events A and B are independent if and

where and are assumed to be non-zero.

Equivalently if A and B are independent, we have

or --------------------

Two events A and B are called statistically dependent if they are not independent. Similarly, we
can define the independence of n events. The events are called independent if and
only if

Example: Consider the example of tossing a fair coin twice. The resulting sample space
is given by and all the outcomes are equiprobable.

Let be the event of getting ‗tail' in the first toss and be the event of
getting ‗head' in the second toss. Then

and

Again, so that

Hence the events A and B are independent.

38
Problems:

Example1.A dice of six faces is tailored so that the probability of getting every face is
proportional to the number depicted on it.

a) What is the probability of extracting a 6?

In this case, we say that the probability of each face turning up is not the same, therefore we cannot
simply apply the rule of Laplace. If we follow the statement, it says that the probability of each face
turning up is proportional to the number of the face itself, and this means that, if we say that the
probability of face 1 being turned up is k which we do not know, then:
P({1})=k, P({2})=2k, P({3})=3k,
P({4})=4k, P({5})=5k,P({6})=6k.
Now, since {1},{2},{3},{4},{5},{6} form an events complete system , necessarily
P({1})+P({2})+P({3})+P({4})+P({5})+P({6})=1

Therefore

k+2k+3k+4k+5k+6k=1

which is an equation that we can already solve:

21k=1

thus
k=1/21

And so, the probability of extracting 6 is P({6})=6k=6⋅(1/21)=6/21.

b) What is the probability of extracting an odd number?

The cases favourable to event A= "to extract an odd number" are: {1},{3},{5}. Therefore, since
they are incompatible events,
P(A)=P({1})+P({3})+P({5})=k+3k+5k=9k=9⋅(1/21)=9/21

39
Example2: Roll a red die and a green die. Find the probability the total is
5. Solution: Let represent getting on the red die and on the green die.

Then, with these as simple events, the sample space is

The sample points giving a total of 5 are (1,4) (2,3) (3,2), and

(4,1). (total is 5) =

Example3: Suppose the 2 dice were now identical red dice. Find the probability the total is 5.

Solution : Since we can no longer distinguish between and , the only

distinguishable points in S are

Using this sample space, we get a total of from points and only. If we assign
equal probability to each point (simple event) then we get (total
is 5) = .

Example4: Draw 1 card from a standard well-shuffled deck (13 cards of each of 4 suits -
spades, hearts, diamonds, and clubs). Find the probability the card is a club.
Solution 1: Let = { spade, heart, diamond, club}. (The points of are generally listed between

brackets {}.) Then has 4 points, with 1 of them being "club", so (club) = .
Solution 2: Let = {each of the 52 cards}. Then 13 of the 52 cards are clubs, so

40
Example 5: Suppose we draw a card from a deck of playing cards. What is the probability
that we draw a spade?

Solution: The sample space of this experiment consists of 52 cards, and the probability of each
sample point is 1/52. Since there are 13 spades in the deck, the probability of drawing a spade is

P(Spade) = (13)(1/52) = 1/4

Example 6: Suppose a coin is flipped 3 times. What is the probability of getting two tails and
one head?

Solution: For this experiment, the sample space consists of 8 sample points.

S = {TTT, TTH, THT, THH, HTT, HTH, HHT,

HHH}

Each sample point is equally likely to occur, so the probability of getting any particular sample
point is 1/8. The event "getting two tails and one head" consists of the following subset of the
sample space.

A = {TTH, THT, HTT}

The probability of Event A is the sum of the probabilities of the sample points in A. Therefore,

P(A) = 1/8 + 1/8 + 1/8 = 3/8

Example7: An urn contains 6 red marbles and 4 black marbles. Two marbles are
drawn without replacement from the urn. What is the probability that both of the marbles are
black?

Solution: Let A = the event that the first marble is black; and let B = the event that the second
marble is black. We know the following:

 In the beginning, there are 10 marbles in the urn, 4 of which are black. Therefore, P(A)
= 4/10.
 After the first selection, there are 9 marbles in the urn, 3 of which are black.
Therefore, P(B|A) = 3/9.

Therefore, based on the rule of multiplication:

P(A ∩ B) = P(A) P(B|A)

P(A ∩ B) = (4/10) * (3/9) = 12/90 = 2/15

41
RANDOM VARIABLE
INTRODUCTION
In many situations, we are interested in numbers associated with the outcomes of a random
experiment. In application of probabilities, we are often concerned with numerical values which are
random in nature. For example, we may consider the number of customers arriving at a service
station at a particular interval of time or the transmission time of a message in a communication
system. These random quantities may be considered as real-valued function on the sample space.
Such a real-valued function is called real random variable and plays an important role in describing
random data. We shall introduce the concept of random variables in the following sections.

Random Variable Definition

A random variable is a function that maps outcomes of a random experiment to real

numbers. (or)
A random variable associates the points in the sample space with real numbers

A (real-valued) random variable, often denoted by X(or some other capital letter), is a function
mapping a probability space (S; P) into the real line R. This is shown in Figure 1.Associated with
each point s in the domain S the function X assigns one and only one value X(s) in the range R.
(The set of possible values of X(s) is usually a proper subset of the real line; i.e., not all real
numbers need occur. If S is a finite set with m elements, then X(s) can assume at most an m
different value as s varies in S.)

42
Example1
A fair coin is tossed 6 times. The number of heads that come up is an example of a random
variable.
HHTTHT – 3, THHTTT -- 2.
These random variables can only take values between 0 and 6. The
Set of possible values of random variables is known as its Range.
Example2
A box of 6 eggs is rejected once it contains one or more broken eggs. If we examine 10 boxes of
eggs and define the randomvariablesX1, X2 as
1 X1- the number of broken eggs in the 10
boxes 2 X2- the number of boxes rejected
Then the range of X1 is {0, 1,2,3,4-------------- 60} and X2 is {0,1,2-------10}
Figure 2: A (real-valued) function of a random variable is itself a random variable, i.e., a
function mapping a probability space into the real line.

Example 3 Consider the example of tossing a fair coin twice. The sample space is S={
HH,HT,TH,TT} and all four outcomes are equally likely. Then we can define a random variable
as follows

Here .

Example 4 Consider the sample space associated with the single toss of a fair die.
The sample space is given by .

If we define the random variable that associates a real number equal to the number
on the face of the die, then .

Types of random variables:

There are two types of random variables, discrete and continuous.

1. Discrete random variable:

43
 A discrete random variable is one which may take on only a countable number of distinct
values such as 0, 1,2,3,4,.........Discrete random variables are usually (but not necessarily) counts. If
a random variable can take only a finite number of distinct values, then it must be discrete
(Or)
A random variable is called a discrete random variable if is piece-wise constant.
Thus is flat except at the points of jump discontinuity. If the sample space is discrete the
random variable defined on it is always discrete.

Plot of Distribution function of discrete random variable

• A discrete random variable has a finite number of possible values or an infinite sequence
of countable real numbers.
–X: number of hits when trying 20 free throws.
–X: number of customers who arrive at the bank from 8:30 – 9:30AM Mon-‐Fri.
–E.g. Binomial, Poisson...

2. Continuous random variable:

A continuous random variable is one which takes an infinite number of possible values. Continuous rando
variables are usually measurements. E
A continuous random variable takes all values in an interval of real numbers.
(or)
X is called a continuous random variable if i n absolutely continuous function
of x . Thus is continuous everywhere on and exists everywhere except at finite
or countably infinite points
3. Mixed random variable:
X is called a mixed random variable if has jump discontinuity at countable number
of points and increases continuously at least in one interval of X. For a such type RV X.

44
Plot of Distribution function of continuous and mixed random variables

45
 DISTRIBUTION AND DENSITY FUNCTIONS AND OPERATIONS ON ONE RANDOM
VARIABLE

Probability Distribution

The probability distribution of a discrete random variable is a list of probabilities associated with
each of its possible values. It is also sometimes called the probability function or the probability
mass function.

More formally, the probability distribution of a discrete random variable X is a function which
gives the probability p(xi) that the random variable equals xi, for each value xi:
p(xi) = P(X=xi)

It satisfies the following conditions:

a.
b.

Cumulative Distribution Function

All random variables (discrete and continuous) have a cumulative distribution function. It is a
function giving the probability that the random variable X is less than or equal to x, for every value
x.

Formally, the cumulative distribution function F(x) is defined to be:

for

For a discrete random variable, the cumulative distribution function is found by summing up the
probabilities as in the example below.

For a continuous random variable, the cumulative distribution function is the integral of its
probability density function.

Example
Discrete case : Suppose a random variable X has the following probability distribution p(xi):
xi 0 1 2 3 4 5
p(xi) 1/32 5/32 10/32 10/32 5/32 1/32
This is actually a binomial distribution: Bi(5, 0.5) or B(5, 0.5). The cumulative distribution
function F(x) is then:
xi 0 1 2 3 4 5

46
 F(xi) 1/32 6/32 16/32 26/32 31/32 32/32

F(x) does not change at intermediate values. For

example: F(1.3) = F(1) = 6/32 and F(2.86) = F(2)
= 16/32
Probability Distribution Function

The probability is called the probability distribution

function ( also called the cumulative distribution function , abbreviated as CDF ) of and
denoted by . Thus

Properties of the Distribution Function

1.

2. is a non-decreasing function of . Thus, if

Is right continuous.

3.

47
 .
4.
.
5.

We have,

6.

Example: Consider the random variable in the above example. We have

48
49
Thus we have seen that given , we can determine the probability of any event
involving values of the random variable .Thus is a complete description of the
random variable .

Example 5 Consider the random variable defined by

Find a) .

b) .

c) .
d) .

Solutio

50
Probability Density Function

The probability density function of a continuous random variable is a function which can be
integrated to obtain the probability that the random variable takes a value in a given interval.

More formally, the probability density function, f(x), of a continuous random variable X is the
derivative of the cumulative distribution function F(x):

Since it follows that:

If f(x) is a probability density function then it must obey two conditions:

a. that the total probability for all possible values of the continuous random variable X is 1:

b. that the probability density function can never be negative: f(x) > 0 for all x.

51
Example 1

Consider the random variable with the distribution function

The plot of the is shown in Figure 7 on next page.

The probability mass function of the random variable is given by

Value of the
pX(x)
random variable X
0

52
Properties of the Probability Density Function
1. .------- This follows from the fact that is a non-decreasing function

2.

3.

4.

Other Distribution and density functions of Random variable:

1. Bernoulli random variable:

Suppose X is a random variable that takes two values 0 and 1, with probability mass
functions

And

Such a random variable X is called a Bernoulli random variable, because it describes the
outcomes of a Bernoulli trial.

The typical CDF of the Bernoulli RV is as shown in Figure 2

53
Mean and variance of the Bernoulli random:

Remark

 The Bernoulli RV is the simplest discrete RV. It can be used as the building block
for many discrete RVs.
 For the Bernoulli RV,

Thus all the moments of the Bernoulli RV have the same value of

2. Binomial random variable

Suppose X is a discrete random variable taking values from the set . is called
a binomial random variable with parameters n and if

where

The trials must meet the following requirements:

a. the total number of trials is fixed in advance;

b. there are just two outcomes of each trial; success and failure;
c. the outcomes of all the trials are statistically independent;
d. all the trials have the same probability of success.

As we have seen, the probability of k successes in n independent repetitions of the Bernoulli

trial is given by the binomial law. If X is a discrete random variable representing the number of
successes in this case, then X is a binomial random variable. For example, the number of heads in ‗n
' independent tossing of a fair coin is a binomial random variable.

 The notation is used to represent a binomial RV with the parameters and

54
 .


 The sum of n independent identically distributed Bernoulli random variables is
a binomial random variable.
 The binomial distribution is useful when there are two types of objects - good, bad;
correct, erroneous; healthy, diseased etc

Example1:In a binary communication system, the probability of bit error is 0.01. If a block of 8 bits
are transmitted, find the probability that

(a) Exactly 2 bit errors will occur

(b) At least 2 bit errors will occur

(c) More than 2 bit errors will occur

(d) All the bits will be erroneous

Suppose is the random variable representing the number of bit errors in a block of 8 bits.
Then

Therefore,

55
The probability mass function for a binomial random variable with n = 6 and p =0.8

56
Mean and Variance of the Binomial Random Variable

57
58
 3. Poisson Random Variable

A discrete random variable X is called a Poisson random variable with the parameter if
and
PX(k)= (e-λλk)/k!

The plot of the pmf of the Poisson RV is shown

59
Mean and Variance of the Poisson RV

Example:The number of calls received in a telephone exchange follows a Poisson distribution

with an average of 10 calls per minute. What is the probability that in one-minute duration?

i. no call is received
ii. exactly 5 calls are received
iii. More than 3 calls are received.

60
Solution: Let X be the random variable representing the number of calls received. Given

Where Therefore,

i. probability that no call is received 0.000095

ii. probability that exactly 5 calls are received 0.0378

iii. probability that more the 3 calls are received

0.9897

Poisson Approximation of the Binomial Random Variable

The Poisson distribution is also used to approximate the binomial distribution when n
is very large and p is small.

61
Thus the Poisson approximation can be used to compute binomial probabilities for large n. It also
makes the analysis of such probabilities easier. Typical examples are:

 number of bit errors in a received binary data file

 Number of typographical errors in a printed page

Example 4 Suppose there is an error probability of 0.01 per word in typing. What is the probability
that there will be more than 1 error in a page of 120 words?

Solution: Suppose X is the RV representing the number of errors per page of 120 words.

Where Therefore,

In the following we shall discuss some important continuous random variables.

4. Uniform Random Variable

A continuous random variable X is called uniformly distributed over the interval [a, b],

62
 , if its probability density function is given by

Figure 1

We use the notation to denote a random variable X uniformly distributed over

the interval [a,b]

Distribution function

Figure 2 illustrates the CDF of a uniform random variable.

63
 Figure 2: CDF of a uniform random variable

Mean and Variance of a Uniform Random Variable:

64
 5. Normal or Gaussian Random Variable

The normal distribution is the most important distribution used to model natural and man made
phenomena. Particularly, when the random variable is the result of the addition of large number of
independent random variables, it can be modeled as a normal random variable.

continuous random variable X is called a normal or a Gaussian random variable with

parameters and if its probability density function is given by,

Where and are real numbers.

We write that X is distributed.

If and , and the random variable X is called the standard normal variable.

Figure 3 illustrates two normal variables with the same mean but different variances.

Figure 3

65
  Is a bell-shaped function, symmetrical about .
 Determines the spread of the random variable . If is small X is more
X
concentrated around the mean .

 Distribution function of a Gaussian variable is

Substituting , weget

where is the distribution function of the standard normal variable.

Thus can be computed from tabulated values of . The table was very useful in
the pre-computer days.

In communication engineering, it is customary to work with the Q function defined by,

Note that and

These results follow from the symmetry of the Gaussian pdf. The function is tabulated and the
tabulated results are used to compute probability involving the Gaussian random variable.

66
Using the Error Function to compute Probabilities for Gaussian Random Variables

The function is closely related to the error function and the complementary
error function .

Note that,

And the complementary error function is given

Mean and Variance of a Gaussian Random Variable

If X is distributed, then

Proof:

67
 6. Exponential Random Variable

A continuous random variable is called exponentially distributed with the parameter

if the probability density function is of

the PDF of Exponential Random Variable is

68
Example 1

Suppose the waiting time of packets in in a computer network is an exponential RV with

Rayleigh Random Variable

A Rayleigh random variable X is characterized by the PDF

where is the parameter of the random variable.

probability density functions for the Rayleigh RVs are illustrated in Figure

Figure 6

69
Mean and Variance of the Rayleigh Distribution

similarly

70
Relation between the Rayleigh Distribution and the Gaussian distribution

A Rayleigh RV is related to Gaussian RVs as follow: If and

are independent, then the envelope has the

Rayleigh distribution with the parameter .

We shall prove this result in a later lecture. This important result also suggests the
cases where the Rayleigh RV can be used.

Application of the Rayleigh RV


Modeling the root mean square error-

Modeling the envelope of a signal with two orthogonal components as in the case of a signal
of the following form:

Conditional Distribution and Density functions

We discussed conditional probability in an earlier lecture. For two events A and B

with , the conditional probability was defined as

Clearly, the conditional probability can be defined on events involving a Random Variable X

Conditional distribution function:

Consider the event and any event B involving the random variable X . The
conditional distribution function of X given B is defined as

We can verify that satisfies all the properties of the distribution function.
Particularly.

 And .
 .
 Is a non-decreasing function of .


71
Conditional Probability Density Function

In a similar manner, we can define the conditional density function of the

random variable X given the event B as

All the properties of the pdf applies to the conditional pdf and we can easily show that

Example 1 Suppose X is a random variable with the distribution function . Define

Case 1:

Then

72
And

Case 2:

and are plotted in the following figures.

Figure 1

73
74
Example2: Suppose is a random variable with the distribution function and
.

Then

For , .Therefore,

For , .Therefore,

Thus,

the corresponding pdf is given by

75
OPERATION ON RANDOM VARIABLE-EXPECTATIONS

Expected Value of a Random Variable

 The expectation operation extracts a few parameters of a random variable and

provides a summary description of the random variable in terms of these parameters.
 It is far easier to estimate these parameters from data than to estimate the distribution
or density function of the random variable.
 Moments are some important parameters obtained through the expectation operation.

Expected value or mean of a random variable

The expected value of a random variable is defined by

Provided exists.

76
 Is also called the mean or statistical average of the random variable and is denoted by

Note that, for a discrete RV with the probability mass function (pmf)
the pdf is given by

Thus for a discrete random variable with

Figure1 Mean of a random variable

Example 1

Suppose is a random variable defined by the pdf

77
 Then

Example 2

Consider the random variable with the pmf as tabulated below

Value of the random

0 1 2 3
variable x
pX(x)

Then

Example 3 Let X be a continuous random variable with

Then

78
 =
Hence EX does not exist. This density function is known as the Cauchy density function.

Expected value of a function of a random variable

Suppose is a real-valued function of a random variable as discussed in the

last class.
Then,

We shall illustrate the above result in the special case when is one-to-one
and monotonically increasing function of x In this case,

Figure 2

79
The following important properties of the expectation operation can be immediately
derived:

(a) If is a constant,

Clearly

(b) If are two functions of the random variable and are

constants,

The above property means that is a linear operator.

MOMENTS ABOUT THE ORIGIN:

Mean-square value

80
MOMENTS ABOUT THE MEAN

Variance

Second central moment is called as variance

For a random variable with the pdf and mean the variance of is denoted
by and

defined as

Thus for a discrete random variable with

The standard deviation of is defined as

Example 4

Find the variance of the random variable in the above example

Example 5

Find the variance of the random variable discussed in above example. As already computed

81
 For example, consider two random variables with pmf as shown below. Note that

each of has zero mean.The variances are given by and implying that
has more spread about the mean.

Properties of variance

(1)

(2) If then

(3) If is a constant,

nth moment of a random variable

We can define the nth moment and the nth central- moment of a random variable X by the
following relations

82
 Note that

 The mean is the first moment and the mean-square value is the
second moment
 The first central moment is 0 and the variance is the
second central moment

SKEWNESS

 The third central moment measures lack of symmetry of the pdf of a random variable

is called the coefficient of skewness and if the pdf is

symmetric this coefficient will be zero.
 The fourth central moment measures flatness or peakedness of the pdf of a random

variable. Is called kurtosis. If the peak of the pdf is sharper, then

the random variable has a higher kurtosis.

Characteristic function

Consider a random variable with probability density function The

characteristic function of denoted by is defined
as

Note the following:

 is a complex quantity, representing the Fourier transform of and

traditionally using instead of This implies that the properties of the
Fourier transform applies to the characteristic function.

We can get
from by the inverse transform

83
Example 1

Consider the random variable X with pdf given by

= 0 otherwise. The characteristics function is given by

Solution:

Example 2

The characteristic function of the random variable with

Characteristic function of a discrete random variable

Suppose X is a random variable taking values from the discrete set

with corresponding probability mass
function for the value

Then,

84
If Rx is the set of integers, we can write

In this case can be interpreted as the discrete-time Fourier transform with

substituting in the original discrete-time Fourier transform. The inverse relation is

Moments and the characteristic function

Given the characteristics function the nth moment is given by

To prove this consider the power series expansion of

Taking expectation of both sides and assuming to exist, we get

85
Taking the first derivative of with respect to at we get

Similarly, taking the derivative of with respect to at we get

Thus ,

TRANSFORMATION OF A RANDOM VARIABLE

Description:

Suppose we are given a random variable X with density fX(x). We apply a function g
to produce a random variable Y = g(X). We can think of X as the input to a black
box,and Y the output.

86