Borel-Cantelli lemmas and the law of large

numbers
Gunnar Englund & Timo Koski
Matematisk statistik
KTH
2008

Introduction

Borel-Cantelli lemmas are interesting and useful results especially for proving
the law of large numbers in the strong form.
We consider a sequence events A1 , A2 , A3 , . . . and are intrested in the question
of whether infinitely many events occur or if possibly only a finite number of
them occur.
We set

[
\
Fn =
Ak and Gn =
Ak .
(1)
k=n

k=n

If Gn in (1) occurs, this means that all Ak for k n occur. If there is some
such n, this means in other words that from this n on all Ak occur for k n.
With

[
[
Ak
H=
Gn =
n=1

n=1 k=n

we have that if H occurs, then there is an n such that all Ak with k n occur.
Sometimes we denote H with lim inf Ak .
The fact that Fn occurs implies that there is some Ak for k n which occurs.
If Fn in (1) occurs for all n this implies that infinitely many of the Ak :s occur.
We form therefore

\
\
Ak .
Fn =
E=
n=1 k=n

n=1

If E occurs, then infinitely many of Ak :s occur. Sometimes we write this as

E = {An i.o.} where i.o. is to be read as infinitely often, i.e., infinitely many
times. E is sometimes denoted with lim sup Ak .
We need a couple of auxiliary results (the lemmas below) of probability calculus
(found, e.g., on page 3 of [1]) that are basic in the sense that they are derived
directly from the Kolmogorov axioms.

First, we consider a sequence of events Bk F that is increasing to B F .

This means that
B1 B2 B3 . . . Bn Bn+1 . . . B
and thus

k=1 Bk = B.
We can write this as
Bn B
Lemma 1 If Bn B, then P (B) = limn P (Bn ).

Proof: We can write

k=1 Bk = k=2 (Bk \ Bk1 ) B1 , since Bk are increasing.

P (B) = P (
k=1 Bk ) = P (k=2 (Bk \ Bk1 ) B1 )

But the sets in the decomposition are seen to be disjoint, and hence the axiom
of countable additivity yields
P

(
k=2 (Bk

\ Bk1 ) B1 ) =

P (Bk \ Bk1 ) + P (B1 )

k=2

= lim

n
X

P (Bk \ Bk1 ) + P (B1 )

k=2

Now we observe that since Bk1 Bk , we have

P (Bk \ Bk1 ) = P (Bk ) P (Bk1 ) .
Therefore we get a telescoping series
n
X

P (Bk \ Bk1 ) + P (B1 ) = P (Bn ) P (Bn1 ) + P (Bn1 ) P (Bn2 ) + . . . +

k=2

+P (B2 ) P (B1 ) + P (B1 ) = P (Bn ) .

In other words we have shown that
P (B) = lim P (Bn ) .
n

We take next a sequence of events Bk F that is decreasing. This means that

B1 B2 B3 . . . Bk Bk+1 . . .
k=1 Bk
and thus we write
Bn
k=1 Bk .

Lemma 2 If Bn
k=1 Bk , then P (k=1 Bk ) = limn P (Bn ).

Proof: We use the axiom of probability of complementary events

c

P (
k=1 Bk ) = 1 P ((k=1 Bk ) ) .

(2)

When we apply one of De Morgan, s rules we get

c

c
(
k=1 Bk ) = k=1 Bk .
c
Now we observe that if Bk Bk+1 , then Bkc Bk+1
, i.e., the complement
events of a decreasing sequence of events are an increasing sequence of events.
Thus the first lemma implies
c
c
P (
k=1 Bk ) = lim P (Bn ) .
n

Hence we have that

c
P ((
k=1 Bk ) ) = P (k=1 Bk )

= lim P (Bnc ) .
n

This we shall insert in (2) and get

c
P (
k=1 Bk ) = 1 lim P (Bn )
n

= 1 lim (1 P (Bn ))
n

= 1 1 + lim P (Bn ) = lim P (Bn ) .

n

This completes the proof.

The Borel-Cantelli results

One notes that Fn is a decreasing set of events. This is simply so because

!

[
[
[
[
Ak = An Fn+1
Ak = An
Fn =
k=n+1

k=n

and thus
Fn Fn+1 .
Thus the second lemma above gives
P (E) = P (

Fn ) = lim P (Fn ) = lim P (

n

n=1

We have, however, by Boole, s inequality that

P(

Ak )

Ak ).

k=n

P (Ak )

k=n

k=n

P
and this sum 0 as n , if the sum
1 P (Ak ) converges. This implies
that we have shown following proposition, also known as the Borel-Cantelli
lemma.
3

Proposition
1 Borel-Cantelli lemma
P
If n=1 P (An ) < then it holds that P (E) = P (An i.o) = 0, i.e., that with
probability 1 only finitely many An occur.
One can observe that no form of independence is required, but the proposition
holds in general, i.e., for any sequence of events.
There is a converse to the Borel-Cantelli lemma obtained if we assume that
the events A1 , A2 , . . . are independent .
Proposition 2 Converse Borel-Cantelli lemma
If A1 , A2 , . . . are independent and

P (An ) = ,

n=1

then it holds that P (E) = P (An i.o) = 1, i.e., it holds with probability 1 that
infinitely many An occur.
Proof: We have by independence
P(

Ak ) =

P (Ak ) =

(1 P (Ak )).

k=n

k=n

k=n

Since 1 x ex we get 1 P (Ak ) eP (Ak ) and

P(

Ak )

exp(

k=n

If now

n=1 P (An )

P (Ak )).

k=n

= , then the sum in the exponent diverges and we obtain

P(

Ak ) = 0.

k=n

Thus it holds also that

P(

Ak ) = 0,

n=1 k=n
,

which implies by De Morgan s rules that

P(

n=1 k=n

Ak ) = 1 P (

Ak ) = 1 0 = 1

n=1 k=n

i.e., that infinitely many Ak :n occur with probability 1.

Some examples of applications

Exempel 1 Let X1 , X2 , X3 . . . be independent equidistributed with continuous distribution. We let

(
1 if Xn > Xj for j = 1, 2, . . . n 1
Un =
0 annars.
This says simply that Un = 1 if Xn is a record, i.e., the largest value observed
so far. We set An = {Un = 1}.
We see that P (Un = 1) = 1/n, since the probability that the largest of n values
should occur in the round n is 1/n for reasons of symmetry. Furthermore,
A1 , A2 , . . . are independent. We have
P (Am Am+1 . . . Am+k ) = P (Am |Am+1 . . . Am+k )P (Am+1 . . . Am+k )
and Am and Am+1 . . . Am+k are, of course, independent since Am is only
concerned with the relative magnitudes of the m first of X-variables.
We get then, as

X
X
1
P (An ) =
= ,
n
n=1
n=1
that P (An i.o.) = 1 i.e., infinitely many An occur. We get infinitely many
records a result which perhaps (?) is self-evident. Furthermore, we get
E(

Un Un+1 ) =

n=1

E(Un Un+1 ) =

n=1

P (An )P (An+1 ) =

n=1

X
n=1

1
<
n(n + 1)

and hence 1 Un Un+1 is finite with probability 1. The conclusion is that there
only occurs a finitely number of double records, i.e., a record two times in a
row this result is by no means trivial.
Exempel 2 Let X1 , X2 , . . . be independent and equidistributed. Then it holds
that
Z
Z n+1
X
E(|X1 |) =
P (|X1 | > x)dx =
P (|X1| > x)dx
0

n=0 n

P (|X1 | > n) 1 +

n=0

but also
E(|X1 |) =

Z
X
n=0

n+1

P (|X1| > x)dx

X
n=0

P (|Xn | > n),

n=1

P (|X1 | > n + 1) =

P (|Xn | > n).

n=1

If now E(|X1 |) < we see that n=1 P (|Xn | > n) < which according
to Borel-Cantelli lemma implies
P (|Xn | > n i.o.) = 0. On the other hand, if
P
P
E(|X1 |) = and thereby
n=0 (|Xn | > n) = , the converse Borel-Cantelli
lemma entails that P (|Xn | > n i.o.) = 1. If E(|Xk |) are finite (respectively
infinite ) |Xn | will be larger than n infinitely many times with probability 0
(respectively 1).
5

Proof of the strong form of the law of large

numbers

We let X1 , X2 , . . . be independent equidistributed with E(Xi ) = m and Var(Xi ) =

2 < and define Sn = X1 + X2 + + Xn . We are interested in showing
the strong form of the law of large numbers (SLLN), i.e., that it holds with
probability one that Sn /n m as n . This means that we want to prove
that
Sn
= m) = 1,
P ( lim
n n
i.e., that there exists a set 0 with P (0 ) = 1 where for every 0 it holds
that
Sn
lim | m| = 0.
n n
We need in other words to prove that for every 0 and for every > 0
there is N(, ) so that if n N(, ) holds that |Sn /n m| .
Sn
It suffices to prove that | m| > can occur only a finite number of times,
n
i.e.,
Sn
lim P (| m| > some n N) = 0.
N
n
Note the distinction with regard to the law of large numbers in the weak form,
which says that that for all > 0
P (|

Sn
m| > ) 0 as n .
n

In words: for the law of large numbers in the strong form |Sn /n m| must be
small for all sufficiently large n for all 0 where P (0 ) = 1.
In tossing a coin we can code heads and tails with 0 and 1, respectively, and
we can identify an with a number in the intervall [0, 1] drawn at random,
where binary expansion gives the sequence of zeros and ones. The law of large
numbers says in this case that we will obtain with probability 1 a number
such that the proportion of 1:s in sequence converges towards 1/2. There can
be exceptional - - for example the sequence 000 . . . is possible, but such
exceptional sequences have the probability 0.
After these deliberations of pedagogic nature let us get on with the proof.
Proof of SLLN: Without restriction of generality we can assume that E(Xi ) =
m = 0, since we in any case can consider Xi m. We have V (Sn ) = n 2 . By
Chebyshev, s inequality it holds that
V (Sn )
n 2
2
P (|Sn | > n)
=
= 2.
(n)2
(n)2
n
P
is divergent so we cannot use BorelUnfortunately the harmonic series
1 1/nP
2
Cantelli lemma directly. But it holds that
1 1/n < and this means that
6

we can use the lemma for n2 , n = 1, 2, . . . . We have

P (|Sn2 | > n2 )

2
.
n2 2

Sn 2
| > i.o.) = 0
n2
which proves that (with probability 1) Sn2 /n2 0. We have in other words managed to prove that for the subsequence n2 , n = 1, 2, . . . there is convergence
with probability 1. It remains to find out what will happen between these n2 .
We define therefore
Dn = 2 max 2 |Sk Sn2 |,

In other words it holds by Borel-Cantelli lemma that P (|

n k<(n+1)

i.e., the largest of the deviation from Sn2 that can occur between n2 and (n+1)2 .
We get
(n+1)2 1
X
2
2
(Sk Sn2 )2 ,
Dn = 2 max 2 (Sk Sn2 )
n k<(n+1)

k=n2

where we used the rather crude inequality max(|x|, |y|) (|x| + |y|). This
entails
(n+1)2 1
X
2
E((Sk Sn2 )2 ).
E(Dn )
k=n2

But E((Sk Sn2 )2 ) = (k n2 ) 2 2n 2 as n2 k < (n + 1)2 and there are

2n terms in the sum and this entails
E(Dn2 ) (2n)(2n) 2 = 4n2 2 .
With Chebyshev, s inequality this gives
P (Dn > n2 )

4n2 2
4 2
=
.
(n2 )2
n2 2

In other words, Dn /n2 0 holds with probability 1. Finally this yields for k
between n2 and (n + 1)2 that
|

|Sn2 | + Dn
|Sn2 | + Dn
Sk
|

0.
k
k
n2

This means that we have succeeded in proving that Sn /n 0 with probability

1. We have done this under additional condition that Var(Xi ) = 2 , but with
a painstaking effort we can in fact prove that this additional condition is not
necessary.

Referenser
[1] Z. Brezniak and T. Zastawniak (2007): Basic Stochastic Processes. Springer Verlag, London.