Marginal distribution

In probability theory and statistics, the marginal distribution of a subset of a collection of random
variables is the probability distribution of the variables contained in the subset. It gives the probabilities of
various values of the variables in the subset without reference to the values of the other variables. This
contrasts with a conditional distribution, which gives the probabilities contingent upon the values of the
other variables.

Marginal variables are those variables in the subset of variables being retained. These concepts are
"marginal" because they can be found by summing values in a table along rows or columns, and writing
the sum in the margins of the table.[1] The distribution of the marginal variables (the marginal distribution)
is obtained by marginalizing (that is, focusing on the sums in the margin) over the distribution of the
variables being discarded, and the discarded variables are said to have been marginalized out.

The context here is that the theoretical studies being undertaken, or the data analysis being done, involves a
wider set of random variables but that attention is being limited to a reduced number of those variables. In
many applications, an analysis may start with a given collection of random variables, then first extend the
set by defining new ones (such as the sum of the original random variables) and finally reduce the number
by placing interest in the marginal distribution of a subset (such as the sum). Several different analyses may
be done, each treating a different subset of variables as the marginal distribution.

Definition

Marginal probability mass function

Given a known joint distribution of two discrete random variables, say, X and Y , the marginal distribution
of either variable – X for example – is the probability distribution of X when the values of Y are not taken
into consideration. This can be calculated by summing the joint probability distribution over all values of Y .
Naturally, the converse is also true: the marginal distribution can be obtained for Y by summing over the
separate values of X.

, and
 X
x1 x2 x3 x4 pY(y) ↓
Y
4 2 1 1 8
y1 32 32 32 32 32

3 6 3 3 15
y2
32 32 32 32 32

9 9
y3 32 0 0 0 32

16 8 4 4 32
pX(x) → 32 32 32 32 32

Joint and marginal distributions of a pair

of discrete random variables, X and Y,
dependent, thus having nonzero mutual
information I(X; Y). The values of the
joint distribution are in the 3×4 rectangle;
the values of the marginal distributions
are along the right and bottom margins.

A marginal probability can always be written as an expected value:

Intuitively, the marginal probability of X is computed by examining the conditional probability of X given a
particular value of Y, and then averaging this conditional probability over the distribution of all values of Y.

This follows from the definition of expected value (after applying the law of the unconscious statistician)

Therefore, marginalization provides the rule for the transformation of the probability distribution of a
random variable Y and another random variable X=g(Y):

Marginal probability density function

Given two continuous random variables X and Y whose joint distribution is known, then the marginal
probability density function can be obtained by integrating the joint probability distribution, f, over Y, and
vice versa. That is
where , and .

Marginal cumulative distribution function

Finding the marginal cumulative distribution function from the joint cumulative distribution function is easy.
Recall that:

For discrete random variables,

For continuous random variables,

If X and Y jointly take values on [a, b] × [c, d] then

and

If d is ∞, then this becomes a limit . Likewise for .

Marginal distribution vs. conditional distribution

Definition

The marginal probability is the probability of a single event occurring, independent of other events. A
conditional probability, on the other hand, is the probability that an event occurs given that another
specific event has already occurred. This means that the calculation for one variable is dependent on
another variable.[2]

The conditional distribution of a variable given another variable is the joint distribution of both variables
divided by the marginal distribution of the other variable.[3] That is,

For discrete random variables,

For continuous random variables,

Example

Suppose there is data from a classroom of 200 students on the amount of time studied (X) and the
percentage of correct answers (Y).[4] Assuming that X and Y are discrete random variables, the joint
distribution of X and Y can be described by listing all the possible values of p(xi,yj), as shown in Table.3.
 X
Time studied (minutes)
Y
x1 (0-20) x2 (21-40) x3 (41-60) x4(>60) pY(y) ↓
2 8 10
y1 (0-20) 200 0 0 200 200

10 2 8 20
y2 (21-40) 0
200 200 200 200

2 4 32 32 70
y3 (41-59) 200 200 200 200 200

20 30 10 60
y4 (60-79) 0 200 200 200 200

4 16 20 40
% correct

y5 (80-100) 0 200 200 200 200

14 30 86 70
pX(x) → 200 200 200 200 1

Two-way table of dataset of the relationship in a classroom of 200 students between the
amount of time studied and the percentage correct

The marginal distribution can be used to determine how many students scored 20 or below:

, meaning 10 students or 5%.

The conditional distribution can be used to determine the probability that a student that studied 60
minutes or more obtains a scored of 20 or below:
, meaning there is about a

11% probability of scoring 20 after having studied for at least 60 minutes.

Real-world example
Suppose that the probability that a pedestrian will be hit by a car, while crossing the road at a pedestrian
crossing, without paying attention to the traffic light, is to be computed. Let H be a discrete random variable
taking one value from {Hit, Not Hit}. Let L (for traffic light) be a discrete random variable taking one value
from {Red, Yellow, Green}.

Realistically, H will be dependent on L. That is, P(H = Hit) will take different values depending on whether
L is red, yellow or green (and likewise for P(H = Not Hit)). A person is, for example, far more likely to be
hit by a car when trying to cross while the lights for perpendicular traffic are green than if they are red. In
other words, for any given possible pair of values for H and L, one must consider the joint probability
distribution of H and L to find the probability of that pair of events occurring together if the pedestrian
ignores the state of the light.

However, in trying to calculate the marginal probability P(H  = Hit), what is being sought is the
probability that H  = Hit in the situation in which the particular value of L is unknown and in which the
pedestrian ignores the state of the light. In general, a pedestrian can be hit if the lights are red OR if the
lights are yellow OR if the lights are green. So, the answer for the marginal probability can be found by
summing P(H  |  L) for all possible values of L, with each value of L weighted by its probability of
occurring.

Here is a table showing the conditional probabilities of being hit, depending on the state of the lights. (Note
that the columns in this table must add up to 1 because the probability of being hit or not hit is 1 regardless
of the state of the light.)
 Conditional distribution:
L
Red Yellow Green
H
Not Hit 0.99 0.9 0.2

Hit 0.01 0.1 0.8

To find the joint probability distribution, more data is required. For example, suppose P(L  =  red)  = 0.2,
P(L = yellow) = 0.1, and P(L = green) = 0.7. Multiplying each column in the conditional distribution by the
probability of that column occurring results in the joint probability distribution of H and L, given in the
central 2×3 block of entries. (Note that the cells in this 2×3 block add up to 1).

Joint distribution:

L Marginal
Red Yellow Green probability
H
P(H)
Not Hit 0.198 0.09 0.14 0.428

Hit 0.002 0.01 0.56 0.572

Total 0.2 0.1 0.7 1

The marginal probability P(H = Hit) is the sum 0.572 along the H = Hit row of this joint distribution table,
as this is the probability of being hit when the lights are red OR yellow OR green. Similarly, the marginal
probability that P(H = Not Hit) is the sum along the H = Not Hit row.

Multivariate distributions
For multivariate distributions, formulae similar to those
above apply with the symbols X and/or Y being
interpreted as vectors. In particular, each summation or
integration would be over all variables except those
contained in X.[5]

That means, If X1 ,X2 ,…,Xn are discrete random

variables, then the marginal probability mass function
should be

Many samples from a bivariate normal distribution.

if X1 ,X2 ,…,Xn are continuous random variables,
The marginal distributions are shown in red and
then the marginal probability density function should
blue. The marginal distribution of X is also
be
approximated by creating a histogram of the X
coordinates without consideration of the Y
coordinates.

See also
Compound probability distribution
 Joint probability distribution
Marginal likelihood
Wasserstein metric
Conditional distribution

References
1. Trumpler, Robert J. & Harold F. Weaver (1962). Statistical Astronomy. Dover Publications.
pp. 32–33.
2. "Marginal & Conditional Probability Distributions: Definition & Examples" (https://study.com/
academy/lesson/marginal-conditional-probability-distributions-definition-examples.html).
Study.com. Retrieved 2019-11-16.
3. "Exam P [FSU Math]" (https://www.math.fsu.edu/~paris/Pexam/). www.math.fsu.edu.
Retrieved 2019-11-16.
4. Marginal and conditional distributions (https://www.khanacademy.org/math/ap-statistics/anal
yzing-categorical-ap/distributions-two-way-tables/v/marginal-distribution-and-conditional-dis
tribution), retrieved 2019-11-16
5. A modern introduction to probability and statistics : understanding why and how. Dekking,
Michel, 1946-. London: Springer. 2005. ISBN 9781852338961. OCLC 262680588 (https://w
ww.worldcat.org/oclc/262680588).

Bibliography
Everitt, B. S.; Skrondal, A. (2010). Cambridge Dictionary of Statistics. Cambridge University
Press.
Dekking, F. M.; Kraaikamp, C.; Lopuhaä, H. P.; Meester, L. E. (2005). A modern introduction
to probability and statistics. London : Springer. ISBN 9781852338961.

Retrieved from "https://en.wikipedia.org/w/index.php?title=Marginal_distribution&oldid=1143685949"