Lecture 1

This introductory article is laid out in 2 sections. Section 1 discusses the Last updated:
concepts of random experiment, while Section 2 depicts some basic terms 16FEB2021
associated with basics of probability.

Keywords: Probability; Random experiment 20 minutes read

Authors: Imran Khan, Shamimul Islam Difficulty: Easy

Section 1: Probability is for Random experiment

DEFINING PROBABILITY

In our daily life, we face many uncertainties. Probability is the likelihood of an event to occur. We
express such situations in two ways-

i. using qualitative terms like “Most probably”, “Certainly”, “Very Likely”, “Good or less chance
of..” etc. or,
ii. mathematically – in a formal way.

Some example statements are as follows.

Probability - using qualitative terms Probability – using mathematical expression

1. Since it is sunny and hot outside, it is very likely 1. There is an 80% chance of rain today.
we will not play cricket today.

2. A smoker has a much greater chance of getting 2. When you toss a coin, you have 50% chances
cancer than a non-smoker does. of getting head.

3. Based on experience, we predict that there is a 3. Based on his achievements this year, I am
good chance of fall of share market prices this 100% sure that he will get a raise.
season.

When used formally (mathematically), probability is a numerical estimate of the proportion of

uncertainty / likelihood with certain degree of confidence.

Probability is the basis of inferential statistics, because in inferential statistics, we make decisions under
uncertain conditions. We then use probability theory to evaluate the uncertainty in those decisions.

RANDOM EXPERIMENT

We apply probability theory on many phenomena around us. All these phenomena have a common
feature, which is, their outcomes will be different under the same condition.
Think about the following cases.

1. The number of heads when we toss a coin 100 times.

2. The number of child births in a day.

3. The number of defective t-shirts produced by same workers / machines / materials in a garment
factory.

4. The number of road accidents in the same highway in a given day.

In all 4 cases above, the outcome may be different, like, the number of road accidents today may not be
the same as the number of accidents yesterday. But importantly, there is no functional or deterministic
regularity in this outcome. That is, we cannot say that if we had 10 accidents yesterday, there will be 20
accidents today. We cannot determine the outcomes like this.

However, though there is no deterministic probability, the outcomes may differ in such a way that there
is a probabilistic regularity. In other words, although there is no way we can determine the outcome of
next experiment based on the results of previous experiments, we can at least understand the
probabilistic way to assume the outcomes.

An example is here. Think about a fair coin. The probability of getting a Head is 0.5 (50%). Assume, we
toss it 100 times and get 42 Heads (proportion of Head is 0.42). Keeping the conditions same, assume
that we toss it 1000 times and get 480 Heads (proportion of Head is 0.48). Thus, if this experiment is
repeated many more times, we can expect the proportion to get nearer to 0.5, which is the original
proportion.

This allows us to define random experiment.

A phenomenon that has probabilistic regularity (and not deterministic regularity) is known as random
experiment. In other words, a phenomenon that has uncertain individual outcomes, but the long-term
pattern of outcomes is predictable is known as a random experiment.

So, the coin tossing was an example of a phenomenon having property of probabilistic regularity. On the
other hand, Newton’s law F=ma is a deterministic one. If we know any two (like “F” and “m”), we can
find the 3rd variable “a”. There is no probability involved here.

The basis of probability theory is random outcomes of random experiments. Randomness is the
property of these outcomes, and so, they are chance measurements. This is the very first and most
important point to be understood before we start studying probability theory and distributions.

Section 2: Basic Terms (Formal definitions)

1. Random Experiment: is any action or process that generates observations (outcomes). A random
experiment can be defined by the following properties –

i. All possible outcomes of the experiment are known in advance.

 ii. Outcome of a single run of experiment is not known in advance.
iii. The experiment can be repeated under similar conditions.

An experiment having the above three properties is called a random experiment.

Example 1. 2 experiments are performed. The first one throwing a fair dice to read the outcome. The
second one is throwing a cricket ball to record its speed. Which one is an example of random
experiment? Why?

Answer. Throwing a dice is a random experiment because,

i. we know all possible outcomes (1 through 6);

ii. We do not know for sure the outcome of one instance of throwing it;

iii. We can repeat it.

Throwing a cricket ball and recording its speed is not a random experiment, because,

i. we don’t know all possible outcomes, and

ii. If we know F and m (Force and mass f the ball), we can find speed / acceleration “a” using F=ma
formula. This shows that this experiment outcomes are deterministic by formula. Since outcomes are
not probabilistic, the experiment is not a random one.

2. Outcome (of Random Experiment): The observed result of a random experiment is known as an
outcome. If a coin is tossed, the outcomes are Head H or Tail T.

In some texts, a term “Sample Point” is used for outcomes. That is, any possible outcome of a random
experiment is known as a sample point.

3. Probability of an outcome: The proportion of trials in which a particular outcome occurs in a very
long run of trials is known as the probability of that outcome. If a coin is tossed, probability of getting a
Head outcome is 0.5

4. Sample Space: The set of all possible outcomes of an experiment is known as a sample space. It is
denoted by S. Outcomes or sample points we discussed above are elements of the set named sample
space.

The following is an example of experiments, their outcomes, and sample spaces.

Experiment Outcomes / Sample points Sample Space

Tossing a Head H, Tail T S = {H, T}

coin once

Tossing a HH, HT, TH, TT S = {(H, H), (H,T), (T,H), (T,T)}

coin twice or we can write it as {HH, HT,
TH, TT}
 Sitting in an Pass, Fail S = {Pass, Fail}
exam

Throwing a 1, 2, 3, 4, 5, 6 S = {1,2,3,4,5,6}
dice

Inspecting a Good, Defective S = {Good, Defective}

product from
a machine

Throwing a (1,1), (1,2), … , (5,6), (6,6) – a total of 36 outcomes. For S = {(1,1),(1,2),(1,3),(1,4),

pair of dice every outcome of dice 1, there are outcomes of second (1,5), (1,6), (2,1),(2,2),(2,3),
dice. (2,4), (2,5), (2,6), … … … … …
(6,1),(6,2),(6,3),(6,4), (6,5),
(6,6)}

Image source:
http://web.math.princeton.edu/math_alive/3/Notes1.pdf

Drawing a sample space

By drawing either a Venn diagram or a tree diagram, we can illustrate the sample space for an
experiment.

1. A Venn diagram is a picture (a closed geometric shape such as a rectangle, a square, or a circle) that
depicts all the possible outcomes for an experiment.

2. In a tree diagram, each outcome is represented by a branch of the tree.

Venn and tree diagrams help us understand probability concepts by presenting them visually. Below
example shows how to draw these diagrams for statistical experiments.

Experiment Venn Diagram Tree Diagram

A coin is tossed. H =
Head, T = Tail
 2 coins tossed

5. Event: An event E is a set of possible outcomes of an experiment. In other words, an event E is a

subset of a Sample space S Example,

Let E be an event of odd outcomes when we throw a dice, i.e. E = {1,3,5}

If 2 dices are tossed (|S| = 62 = 36), an event E1 may be all outcomes that sums up to 7. So, E1 = {(1,6),
(2,5), (3,4), (4,3), (5,2) (6,1)}

Simple Event: An event with one single outcome is known as a simple event. For example,
getting a 6 when a dice is thrown, E= {6}

Compound Event: Sets or groups of simple events are known as compound event. For example,
getting an odd number when we throw a dice, E = {1,3,5}

Example 2. Suppose we randomly select two workers from a company and observe whether the worker
selected each time is a man or a woman. Write all the outcomes for this experiment. Draw the Venn and
tree diagrams for this experiment. List all the outcomes included in each of the following events and
state whether they are simple or compound events.

(a) Both workers are male.

(b) At most one worker is female

(c) Exactly one worker is male.

(d) The first worker must be a male.

Solution. Let M denote an event that a worker is male, and W denote an event that a worker is female.

This experiment has the following four outcomes -

MM = both workers are male

MW = the first person is male and the second is female

WM = the first worker is female and the second is male

WW = both workers are female

The Venn and tree diagrams in Figure 1 show these four outcomes.

(a) The event “both workers are male” will occur if MM is obtained. Thus,

Event outcomes of “both workers are male” = {MM}

Because this event has only one outcome, it is a simple event.

(b) The event “at most one worker is female” will occur if either none or one of the workers selected is
female. Consequently,

Event outcomes of “at most one worker is female” = {MM, MW, WM}

Because this event includes more than one outcome, it is a compound event.

(c) The event “exactly one worker is male” will occur if one of the two workers selected is male and the
other is female.

So, event outcomes for “exactly one worker is male” = {MW, WM}

Because this event includes more than one outcome, it is a compound event.

(d) The event “The first worker must be a male” will occur if first one of two workers selected is male.

So, the outcomes of the event “the first worker must be a male” = {MM, MW}

Because this event includes more than one outcome, it is a compound event.
Exercise 1. A box contains some IC chips, some of which are defective. Two ICs are picked up at random.
How many total outcomes are possible? Draw a tree diagram of this random experiment. List the
outcomes of the following events, and state if they are simple or compound events.

a. Both ICS are defective

b. At least one IC is good
c. At most one IC is good
d. Exactly one IC is good
e. The first IC is good, the second one is defective
f. Not more than one IC is defective

Exercise 2. Draw a tree diagram for 3 tosses of a coin. List all outcomes in a sample space S.

[End of part 1] ??

6. Favorable Cases: The outcomes (or sample points) in favor of an event are called favorable cases. For
example, {1}, {3} and {5} are favorable cases when we expect an odd outcome from throwing a dice.

7. Mutually Exclusive Event: Events that cannot occur together and have no common elements or
sample points are known as mutually exclusive events, also known as disjoint events.

Example, if a dice is thrown the odd outcomes and the event outcomes are disjoint events, as {1,3,5}
and {2,4,6} have no elements in common.

If a dice is thrown, then the set of prime numbers and even numbers are not mutually exclusive. Set of
prime numbers = {1,2,3,5} and even numbers = {2,4,6}. Since element {2} is common in both sets, they
are not mutually exclusive or not disjoint events.

8. Complement of an Event: If E is an event, then its complement comprises of all elements in sample
space S that are not in E.

Example. Let a dice is thrown as we define event E to be the odd outcomes. So, E = {1,3,5} and S =
{1,2,3,4,5,6}. Complement of E = {2,4,6}

9. Equally Likely Events: Equally likely events are those events which have an equal probability of
occurring. For example:

When we toss an unbiased coin, the probability of getting a heads is 1/2 and the probability of getting a
tails is 1/2. So, they are equally likely events.

If we have a bag of 3 blue and 2 red balls, the probability of drawing a blue ball is 3/5, while the
probability of drawing a red ball is 2/5. Thus, they are NOT equally likely events.

10. Independent Event: Independent events are events whose outcomes are not affected by any other
events.
Example : In a coin tossing experiment, the event showing a head doesn’t affect or is not influenced by
the event showing a tail. Another experiment may be that we throw a dice and toss a coin together. But
the outcome of the dice and the coin are not influenced by each other, and thus, independent.

Question. Are all Independent events mutually exclusive?

Answer: No. Mutually exclusive events are not necessarily independent events. Recall that mutually
exclusive events do not share any sample points with each other, so they cannot occur at the same time.
Thus, if event A and event B are mutually exclusive, they are actually inextricably DEPENDENT on each
other because appearance of event A nullifies occurrence of Event B and vice-versa.

For example, Flower A and Flower B can grow in the same area, but perhaps Flower A's growth impedes
Flower B's growth, so the probability of Flower B growth changes given Flower A's existence; they share
points and are dependent one each other, showing that you cannot assume mutual exclusivity from
dependence. – need better example

11. Exhaustive Events: Sets of all the possible outcomes of an experiment are known as exhaustive
events.

Example 3: A box contains three items that are labeled X, Y, and Z. Two items are selected at random
(without replacement) from this box. List all the possible outcomes for this experiment. Write the
sample space S.

solution: The experiment of selecting two items from the box without replacement has the following six
possible outcomes: XY, XZ, YX, YZ, ZA, ZY

The sample space is written as S = { XY, XZ, YX, YZ, ZA, ZY}

Example 4: A box contains a certain number of computer parts, a few of which are defective. Two parts
are selected at random from this box and inspected to determine if they are good or defective. How
many total outcomes are possible? Draw a tree diagram for this experiment. List all the outcomes
included in each of the following events. Indicate which are simple and which are compound events.

a. At least one part is good.

b. Exactly one part is defective.

c. The first part is good and the second is defective.

d. At most one part is good.

Let G = the selected part is good and D = the selected part is defective. The four outcomes for this
experiment are: GG, GD, DG, DD

a) {GG, GD, DG}; a compound event

b) {GD, DG}; a compound event

c) {GD}; a simple event

d) {GD, DG, DD}; a compound event

12. Event Space: All possible events for a given experiment. A coin tossing experiment has an event
space {H,T}.

Conclusion
Probability distributions are for probabilistic outcomes that comes from random experiment. But if the
outcomes come in a deterministic way (say, by a formula), then we cannot apply probability theory and
distributions of it. Thus, if we read ‘current I’ by giving 100 different ‘voltages V’ to an object of
‘resistance R’, the 100 outcomes of ‘I’ are not probabilistic. Rather, they are deterministic outcomes –
determined by Ohm’s law V = RI. Consequently, the experiment is not a random one and the outcomes
won’t have a probability distribution.

After reading this article, we expect the students to

i. Understand and define random experiment.

ii. Define basic terms associated with random experiment