Some Notes on σ-algebra and Random Variables

Ali Fele Paranj

January 26, 2025

1 Introduction
The probability theory is so natural that one can start thinking about some basic problems
as soon as they start to know how to count! Because finding the answer to most of the
problems boils down to counting the number of all possible scenarios, and then reporting
the proportion of the outcomes of interest to the total number. The first shock in probabil-
ity theory is experienced when you sit in a undergraduate level probability course. Then
you will learn about the notion of a random variable that is a function! And even more
shockingly, there is nothing random about a random variable. Surviving so many abuse
of notations (like probability density function, probability function, distribution, law, etc)
then you might get a feeling that you understand what probability is saying and you can
imagine a peaceful life after your course. However, once one encounters the treatment of
probability theory in a graduate level course, at first, you might even think that you are
sitting in a wrong class (and continue thinking the same way for 4 more weak if not the
whole term!). That is because you will hear about different notions like σ-algebra, mea-
surable random variables, a filtration, adaptive random variable, and etc. The situation
becomes even more scary when you hear for the first time that conditional expectation is
a random variable! In this note, I will provide a more algebraic treatment of the notion
of σ-algebra which will help to understand these notions better. Also, by the end of these
notes you will have a more tangible sense of the damn sentence “we can think of σ-algebra
as information available to us!”. Seriously.
Things to be added or changed later: Notice that for any Ω the 2Ω already has the same
algebraic structure as discussed below, i.e. 2Ω = ⊕x∈Ω F2 . Then any σ-algebra will be a
subspace of this space 2Ω .

2 σ-Algebra
The notion of σ-algebra is easiest to understand when considered in a finite setting. We
are not going to review the definition of σ-algebra here. However, we want to highlight
that we can look at a σ-algebra of a sample space Ω in a way that resembles a vector
space. Then we can talk about subspaces of that vector spaces (which will be the notion
of sub σ-fields). We start with an explicit example.

1
2.1 Finite Case
Let Ω = {1, 2, 3} be our sample space. Then for σ-algebra we have many options. One
one extreme we can have F0 = {∅, Ω}, and on the other extreme we can have

F3 = {{1}, {2}, {3}, {1, 2}, {2, 3}, {1, 3}, {1, 2, 3}, Ω}

We can consider any of these collections as a collection of functions rather than a collection
of sets. For instance, we can define

F3 = {f : {1, 2, 3} → F2 },

where F2 = {0, 1} is a field with characteristic 2. For instance the set {1} will be the same
as the function f where f (1) = 1, f (2) = 0, and f (3) = 0. And when the domain of those
functions is finite, then the followings are equivalent representations of the same thing.
 
1 Ç å
  1 2 3
f (1) = 1, f (2) = 0, f (3) = 0, (1, 0, 0),  0 , , 100,
1 0 0
0

and the list above is not an exhaustive one. Thus we can also represent the elements of
F3 as                

 0 

 1 0 0 1 1 0 1 
               
F3 = 0 , 1 , 0 , 1 , 0 , 1 , 1 , 0 .

 
 0 0 1 0 1 1 1 0 


It is easy to see that F3 forms a vector field on F2 . In other way of seeing this is to observe
that
⊕3
F3 = F2 ,
i=1

i.e. F3 is a direct sum (equivalently direct product when we can dealing with finitely many
of spaces) of the underlying field F2 . But what about the case that we choose

F1 = {{1}, {2, 3}, {1, 2, 3}, ∅}.

Then how we can show this collection as a vector space in above? It turns out that we
can write it as
F1 = {f : {{1}, {2, 3}} → F2 }.
Observe that {{1}, {2, 3}} is in fact a partition of Ω. So in general we can summarize as
below.

Summary 1. Let Ω be a finite sample space, and P = {P1 , P2 , · · · , Pn } be any partition.

Then the σ-algebra generated by P, denoted as F = σ(P), is given as
∪
F ={ P | A ⊆ P}.
P ∈A

2
Or equivalently ⊕
F = {f : P → F2 } = F2 .
P ∈P

Observe that F has a vector space structure. The elements of the partition P is called the
atoms of F. Furthermore, the collection B = {fi } ⊂ F , where fi (Pj ) = δi,j forms a basis
for the F.

We will follow the second interpretation above, as it provides a more clear algebraic
structure. To understand the notion of measurable functions (i.e. random variable) we
need to introduce the dual basis to B. Consider the collection of functionals {1i }ni=1
defined as
1i : F → F2 , where 1i (fj ) = δi,j .
This collection forms a dual basis for F, i.e. it is a basis for F ′ .

Summary 2. In the same setting as the summary above, the collection {1i }ni=1 defined
as
1i : F → F2 , where 1i (fj ) = δi,j
forms a dual basis for F. With some abuse of notation, these functions
∪ are the similar to
the indicator function {1P }P ∈P when we view the σ-algebra as F = { P ∈A P | A ⊆ P}.

With this point of view, the notion of measurable functions becomes very easy to
grasp. A measurable function is simply an element of F ′ . I.e. X : Ω → R is measurable
if and only if we can write it as a linear combination of dual basis. With slight abuse of
notation (see the summary below for clarification) we can write:

∑
n
X= α i 1i ,
i=1

or with the notation of the indicator functions

∑
X= α i 1P .
P ∈P

Summary 3. X is F-measurable if and only if X ∈ F ′ in the sense that we can write X

as
∑n
X= α i 1i ◦ q
i=1

where q : Ω → P is the quotient map.

The following example demonstrates our setup above. However, this example might
sound quite silly, as we are over-killing our machinery for the purpose of demonstrating
the tools we have in our disposal.

3
Example 1. Consider an experiment of tossing a dice. So the sample space will be

Ω = {1, 2, 3, 4, 5, 6}.

Assume that we don’t have enough technology to determine the exact outcome of an
experiment. Instead, our measurement device can only report the outcome of the exper-
iments in mod 2, i.e. if the outcome is even or is odd. So the set of atomic events (i.e.
the σ-algebra) will be1
A = {{1, 3, 5}, {2, 4, 6}}.
And the σ-algebra of interest will be the sigma algebra generated by these atoms, i.e.

F = σ(A) = {{1, 3, 5}, {2, 4, 6}, Ω, ∅}.

Or in a more algebraic point of view we have

⊕
F= F2 = F2 ⊕ F2 = {f : A → F2 }.
p∈A

Fixing a basis for this space (as discussed in Summary 1) we can write the elements of
this vector space explicitly as
ñ ô ñ ô ñ ô ñ ô
¶ 0 1 1 0 ©
F= , , , .
1 0 1 0

Now suppose that we did some upgrade in our measurement system (or wrote down some
theory that enables more accurate measurements) our device can now measure the exact
value of the outcome of the experiment. So the atoms for this measurement device, i.e.
random variable will be

A′ = {{1}, {2}, {3}, {4}, {5}, {6}}.

So the generated σ-algebra will be

F ′ = σ(A′ ) = 2Ω ,

i.e. the set of all subsets of of Ω. Or in the algebraic point of view we have

F ′ = F 2 ⊕ F2 ⊕ F 2 .

Remark 1. As we saw in the example above, it is usually the random variable that induces
the σ-algebra rather than having a σ-algebra that determines which random variable is
measurable and which is not. It is because of this reason that we are generally interested
in the σ-algebra generated by a collection of random variables.
1
Observe that A is a partition of the set Ω.

4
2.2 Infinite Case
When considering finite sample spaces, most of the notions above seems to be too much
complexity for a system that can by analyzed easily. But the usefulness of the notions
above becomes more evident when we consider sample spaces with infinite cardinality,
specially the sample spaces associated to the cases where each outcome is the result of
a countably many repeated experiments. For instance, the sample space for an infinite
coin toss, which is the space of sequences of 0s and 1s. This system will be our running
example for the rest of this section.
Example 2 (Running Example). Imagine you tossing a coin infinitely many times. The
sample space for this will be

Ω = {(1, 0, . . . ), (0, 0, · · · ), (1, 1, . . . ), (1, 0, . . . )},

i.e. the set of all sequences composed of 1s and 0s. It is very hard to imagine this sample
space in any practical and useful way. But we can identify the elements of this sample
space with real numbers in (0, 1]. For each ω ∈ Ω we identify it with a ∈ (0, 1] where
the binary digits of a in the non-terminating expansion matches with ω. For instance let
ω1 = (0, 1, 0, 1, 0, 1, . . . ). The corresponding real number a1 will be

a1 = 0.01010101 . . . .

And for ω2 = (1, 0, 0, 0, . . . ) the corresponding real number a2 will be

a2 = 0.01111111 . . . .

Observe that we chose the non-terminating representation of 0.10000 . . . (for more details
see my lecture notes on the probability theory, or see Billingsley section 1.1). Now imagine
that one outcome ω ∈ Ω has occurred but we do not have enough machinery and technology
to exactly say which one has occurred. Instead, our machinery can only determine what
was the outcome of the coin flip on the fits toss. I.e. we have the following partition for
the sample space
A = {A0 , A1 },
where A0 denotes the set of all outcomes whose first entry is 0 and A1 denotes the set of
all outcomes whose first entry if 1. So the σ-algebra will be

F1 = σ(A) = {A0 , A1 , Ω, ∅}.

Or for the algebraic point of view

F1 = σ(A) = F1 ⊕ F2 .

Using Summary 2 the dual basis will be

B′ = {1A0 , 1A1 }

5
 Since it is hard to imagine the structure of Ω as above, we use our identification above
to construct the similar notions for Ω̃ = (0, 1]. With our identification of elements of Ω
with real numbers it is easy to see that
1 1
à = {(0, ], ( , 1]}.
2 2
So
1 1
F̃1 = {(0, ], ( , 1], Ω, ∅}.
2 2
And the dual basis functions will be

B̃′ = {1(0, 1 ] , 1( 1 ,1] }.

2 2

In the setting of example above, we are interested in some special random variables
that will be crucial for the rest of our analysis. So for the space (Ω, F1 ) as above we define

X1 : Ω → R, (x, . . . ) → x,

where ω1 is its first entry. Or in the corresponding space (Ω̃, F̃1 ) we can write this random
variable as
d1 : Ω̃ → R, 0.x · · · → x.
We call d1 as dyadic function.
Example 3. Assume that we have received some new measurement devices that can mea-
sure the second, and the third entry of each outcome respectively. Each of these devices
will result in the following measurable spaces

(Ω, F2 ), (Ω, F3 ),

where
F2 = σ(A2 ) = σ({A00 , A01 , A10 , A11 }),
where A10 is the set of all outcomes that start with 10, and a similar definition for other
sets. Similarly

F3 = σ(A3 ) = σ({A000 , A001 , A010 , A011 , A100 , A101 , A110 , A111 }).

This becomes more interesting when we consider the Ω̃, the real-interval counterpart of
these sets. We are not going to list all of them here but for instance we have
1 2 3 6 7 3 4
A00 = (0, ], A10 = ( , ], A110 = ( , ], A011 = ( , ], ....
4 2 4 8 8 8 8
It is also very informative to look at the dyadic functions d2 , and d3 associated with these
σ-algebras.

6
 First observe that
• d1 is F1 measurable.
• d2 is F2 measurable.
• d3 is F3 measurable.
• d4 is F4 measurable.
It is immediate to see the above as each of the dyadic functions above is simple a linear
combination of the dual basis elements of their corresponding σ-algebra. It is also easy to
see that these random variables are independent. For instance to check the independence
between d1 and d2 we have
P((1/2, 1] ∩ ((1/4, 1/2] ∪ (3/4, 1])) 1/4
P(d−1 −1
1 (1)|d2 (1)) = = = 1/2 = P(d−1
1 (1)).
P((1/4, 1/2] ∪ (3/4, 1])) 1/2
P((0, 1/2] ∩ ((0, 1/4] ∪ (2/4, 3/4])) 1/4
P(d−1 −1
1 (0)|d2 (0)) = = = 1/2 = P(d−1
1 (0)).
P((0, 1/4] ∪ (2/4, 3/4])) 1/2

3 Conditional Expectation
The algebraic point of view above helps with understanding the notion of conditional
expectation in a much more depth! The following summary covers most of the ideas we
have developed above.
Summary 4. For any sample space Ω we can associate a vector space given as
∏
2Ω = F2 ,
x∈Ω
∏ ⊕
where we can replace with if Ω is finite. The dual basis of this space will be {fx }x∈Ω
where fx (A) = 1A (x). Given any partition of the space, e.g. A, which are called the
atoms, we will have an associated subspace (up to an isomorphism) of Ω given as
⊕
F= F2 .
P ∈A

7
The dual basis vectors for this space is {fP }P ∈A where fP = 1P , i.e. assumes the value 1
on the partition P and 0 otherwise 2 .

Let X be a F-measurable random variable, and let G ⊂ F be a sub σ-algebra. Then

the conditional expectation of X with respect to G is really a projection of the functional
X (that can be expressed in therms of the dual basis of F) to a functional on G. For the
sale of concreteness consider the following example.
Example 4. Let Ω be a sample space and let An = {P1 , · · · , Pn }, Am = {P1′ , · · · , Pm′ } be

two partitions. Note that we can have the situation where An is a partition, i.e. An is
the finer partition that partitions each element of Am which is a courses partition. For
instance you can assume the finer partition to be {(0, 1/4], (1/4, 2/4], (2/4, 3/4], (3/4, 1/2]},
and the coarser partition to be {(0, 1/2], (1/2, 1]}. Let Fn = σ(An ), and Fm = σ(Am ) be
the corresponding σ-algebras. Let X be a An -measurable random variable. I.e. we can
write
∑n
X= α i 1P i .
i=1

Then we define the conditional expectation of X as

∑
m
E[X|Fn ] = E[X|Pi′ ]1Pi′ .
i=1

We can write this projection linear operator as a matrix by applying it on the dual basis
vectors of Fn . The matrix of this projection can be written as
 
E[1P1 |P1′ ] E[1P2 |P1′ ] · · · E[1Pn |P1′ ]
 E[1 |P ′ ] E[1 |P ′ ] · · · E[1 |P ′ ] 
 P1 2 P2 2 Pn 2 
M =  .. .. .. .

 . . . ··· 
′ ] E[1 |P ′ ] · · · E[1 |P ′ ]
E[1P1 |Pm P2 m Pn m

Observe that E[1P1 |P1′ ] = P(P1 |P1′ ). So we can write

 
P[P1 |P1′ ] P[P2 |P1′ ] · · · P[Pn |P1′ ]
 P[P |P ′ ] P[P |P ′ ] · · · P[P |P ′ ] 
 1 2 2 2 n 2 
M =  .. .. .. .

 . . . ··· 
P[P1 |Pm ′ ] P[P |P ′ ] · · · P[P |P ′ ]
2 m n m

Now consider the example below for a more explicit calculation.

Example 5. Let Ω = {a, b, c, d, e, f }. Consider the following partitions

A1 = {{a}, {b, c}, {d, e}, {f }}, A2 = {{a, b}, {c, d}, {e, f }},
2
Note that in principle, the domain of fP should be F . So we really mean fP ◦ q when we wrote fP
above, where q is the quotient map. See Summary 3 for more details

8
and let F1 = σ(A1 ) and F2 = σ(A2 ) be the corresponding σ-algebras. It is easy to check
that the dual basis for these spaces is

B1 = {1{a} , 1{b,c} , 1{d,e} , 1{f } }, B2 = {1{a,b} , 1{c,d} , 1{e,f } }

Let X be a F1 measurable random variable given as

Ç å
a b c d e f
X≡ .
1 3 3 5 5 7

We can write X as
 
1
 3
 
X = 11{a} + 31{b,c} + 51{d,e} + 71{f } :=   .
 5
7

We want to compute the conditional expectation E[X|F2 ]. First, we need to compute the
transformation matrix as in the example above. For instance

P({a} ∩ {a, b}) 1/6 1

(M )1,1 = P({a}|{a, b}) = = = .
P({a, b}) 2/6 2

And the full matrix will be given as

 
1 1
0 0
 21 2
1 1 
M= 2 2 2 0 .
1 1
0 0 2 2

So we can write  
  1  
1 1
0 0   2
2 2   3  
E[X|F2 ] =  0 1 1
0    = 4 .
1  5
2 2
1
0 0 2 2 6
7
So we can write
E[X|F2 ] = 21{a,b} + 41c,d + 61e,f .
Now let A3 be yet another partition given by

A3 = {{a, b, c, d}, {e, f }},

and let F3 = σ(A3 ) be the generated σ-algebra by this partition. We can also easily
calculate E[X|F3 ] by first computing the transformation matrix
ñ ô
′ 1/4 1/2 1/4 0
M = .
0 0 1/2 1/2

9
Thus we will have
 
ñ ô  1 ñ ô
1/4 1/2 1/4 0 3 3
E[X|F3 ] =  = .
0 0 1/2 1/2 5 6
7

Thus
E[X|F3 ] = 31{a,b,c,d} + 61{e,f } .

Viewing the conditioning operation as the matrices above gives us a very powerful
level of abstraction to think about these notions. Then we can utilize many different tools
from linear operator theory and spectral theory for different purposes in the setting of
probability theory. Also

Considering the example above, it is possible to construct the matrix M as

soon as we know the σ-algebras of interest. I.e. the transformation matrices
are purely determined by the σ-algebras and do not depend on any random
variable.

Example 6. (Two interesting extreme cases) Let X be a random variable (i.e. a measurable
function with respect to σ(X)), and let G be a σ-algebra. Then if

E[X|G] = X,

then we say that X is G-measurable. And if

E[X|G] = E[X],

then we say that X is independent of [G]. We will return to the coin tossing example to
demonstrate the shape of the transformation matrices in these two extreme cases. Let

A1 = {A0 , A1 }, A2 = {A00 , A01 , A10 , A11 }

as in Example 2. Equivalently, we can write these partitions as the subsets of (0, 1], i.e.

Ã1 = {(0, 1/2], (1/2, 1]},

and
Ã2 = {(0, 1/4], (1/4, 2/4], (2/4, 3/4], (3/4, 1]}.
Then we can write M : σ(A1 ) → σ(A2 ) as
 
1 0
1 0
 
M = .
0 1
0 1

10
And M ′ : σ(A2 ) → σ(A1 ) as
ñ ô
1/2 1/2 0 0
M′ = .
0 0 1/2 1/2

Consider the dyadic functions d1 that is F1 measurable, and d2 that is F2 measurable.

We can write
d1 = 1(0,1/2] , d2 = 1(1/4,2/4] + 1(3/4,1] .
Then it is easy to check that d2 is independent of F1 because
 
0
1 ñ1/2ô
 
E[d2 |F1 ] = M ′   = = E[d2 ].
 0 1/2
1

Furthermore, it is easy to see that d1 is F2 -measurable because

 
ñ ô 0
0 0
E[d1 |F2 ] = M =  ,
1 1
1

which is the same as d1 as both the RHS of the equation above and d1 assumes the same
values on all of the point in Ω.

11