Scribd
Upload
14 views3 pages

Probability II Upload Week 9

class notes

Uploaded by

Amitesh Sisodia
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
14 views3 pages

Probability II Upload Week 9

class notes

Uploaded by

Amitesh Sisodia
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

1.

C ONVERGENCE OF RANDOM VARIABLES

Let {Xn }n≥1 be a sequence of random variables, and X be another random variable defined on
the same probability space. We will discuss only real-valued random variables.

Convergence in quadratic mean (q.m.): Suppose that {Xn }n≥1 and X are such that E[Xn2 ] < ∞
for all n ≥ 1 and E[X 2 ] < ∞. Then, the sequence {Xn }n≥1 converges to X in q.m. if
h i
lim E (Xn − X)2 = 0.
n→∞

q.m. L2
This is denoted by Xn → X (or, Xn → X).
q.m. q.m.
By definition, Xn → X if and only if Xn − X → 0 as n → ∞.

Convergence in probability (P): The sequence {Xn }n≥1 converges to X in probability, denoted
P
by Xn → X if
lim P {ω : |Xn (ω) − X(ω)| > ε} = 0
n→∞

for every ε > 0.


P P
By definition, Xn → X if and only if Xn − X → 0 as n → ∞.

Convergence in q.m. implies convergence in P: By Chebyshev’s inequality, for each ε > 0


h i
E |Xn − X|2
0 ≤ P {|Xn − X| > ε} ≤ .
ε2
Therefore, E | Xn − X |2 → 0 implies that P {|Xn − X| > ε} → 0 as n → ∞.
 

Exercise 1. Let {Xn }n≥1 be a sequence of random variables with E (Xn ) = µn and Var (Xn ) = σn2
P
for n = 1, 2, . . . Suppose that limn→∞ µn = µ and limn→∞ σn2 = 0. Then, Xn → µ as n → ∞.

Convergence in P does not necessarily imply convergence in q.m.:


√
Example 2. Consider the U (0, 1) probability, and the sequence Xn = n1(0,1/n) for n ≥ 1 and
X = 0. Clearly, Xn converges in probability to X. (Why? - Note that |Xn − X| = |Xn |).
But, not in q.m., because E (Xn − X)2 = E Xn2 = 1 for every n ≥ 1.
   

1
Transformations: Let h : R → R be a continuous function.
P P
(1) If Xn −
→ X, then h(Xn ) −
→ h(X) as n → ∞.
q.m.
(2) If Xn −−→ X, then h(Xn ) does not necessarily converge to h(X) in q.m.
(Think of a counter-example!)

2
2. W EAK LAW OF LARGE NUMBERS

Let X1 , X2 , . . . be i.i.d random variables (independent random variables each having the same
marginal distribution). Assume that the second moment of X1 is finite. Then, µ = E[X1 ] and
σ 2 = Var(X1 ) are well-defined. (Why?)
Sn X1 +···+Xn
Let Sn = X1 + · · · + Xn (partial sums) and X̄n = n = n (sample mean). Then, by the
properties of expectation and variance, we have

σ2
E[Sn ] = nµ, Var(Sn ) = nσ12 , E[X̄n ] = µ, and Var(X̄n ) = .
n
√
In particular, s.d.(X̄n ) = σ/ n decreases with n. If we apply Chebyshev’s inequality to X̄n , we
get for any δ > 0 that
σ2
0 ≤ P{|X̄n − µ| ≥ δ} ≤ .
δ2n
This goes to zero as n → ∞ (with δ > 0 being fixed). This means that for large n, the sample mean
is unlikely to be far from µ (sometimes called the “population mean”). This is consistent with our
intuitive idea that if we toss a coin (with probability of head p) many times, we can get a better
guess of what the value of p is.

Weak law of large numbers (Jacob Bernoulli): With the above notations, for any δ > 0, we have

σ2
0 ≤ P{|X̄n − µ| ≥ δ} ≤ → 0 as n → ∞.
δ2n
This is very general, in the sense that we only assume the existence of variance. If Xk are assumed
to have more moments, one can get better bounds. For example, when Xk are i.i.d. Ber(p), we
have the following theorem.

Hoeffding’s inequality: Let X1 , . . . , Xn be i.i.d. Ber(p). Then, for any δ > 0


2
0 ≤ P{|X̄n − p| ≥ δ} ≤ 2e−2nδ .

A more general version


Let X1 , . . . , Xn be independent, bounded random variables with Xi ∈ [a, b] for i = 1, . . . , n and
−∞ < a ≤ b < ∞. Then, for all δ ≥ 0
2 2
0 ≤ P X̄n − E[X] ≥ δ ≤ 2e−2nδ /(b−a) .


Check the file ‘WLLN.R’.

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy