Scribd
Upload
91 views10 pages

Sums of Independent Random Variables: Scott She Eld

This document summarizes key concepts about summing independent random variables from the lecture: 1) When summing two independent random variables X and Y, the probability density function of their sum X + Y is the integral of the product of the individual densities over all pairs (x, y) such that x + y is less than or equal to the value being evaluated. 2) Summing independent identically distributed (i.i.d.) random variables results in certain standard distributions, such as the gamma distribution for exponentials or the negative binomial for geometric. 3) In general, if independent random variables are added, their means and variances add, so the sum will be normally distributed if the individual variables are

Uploaded by

DevendraReddyPoreddy
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
91 views10 pages

Sums of Independent Random Variables: Scott She Eld

This document summarizes key concepts about summing independent random variables from the lecture: 1) When summing two independent random variables X and Y, the probability density function of their sum X + Y is the integral of the product of the individual densities over all pairs (x, y) such that x + y is less than or equal to the value being evaluated. 2) Summing independent identically distributed (i.i.d.) random variables results in certain standard distributions, such as the gamma distribution for exponentials or the negative binomial for geometric. 3) In general, if independent random variables are added, their means and variances add, so the sum will be normally distributed if the individual variables are

Uploaded by

DevendraReddyPoreddy
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 10

18.

440: Lecture 23
Sums of independent random variables

Scott Sheffield

MIT

1
18.440 Lecture 23

Summing two random variables

. Say we have independent random variables X and Y and we


know their density functions fX and fY .
. Now let’s try to find FX +Y (a) = P{X + Y ≤ a}.
. This is the integral over {(x, y ) : x + y ≤ a} of
f (x, y ) = fX (x)fY (y ). Thus,
.
∞ a−y
P{X + Y ≤ a} = fX (x)fY (y )dxdy
−∞ −∞
∞
= FX (a − y )fY (y )dy .
−∞
. Differentiating both sides gives
d ∞ ∞
fX +Y (a) = da −∞ FX (a−y )fY (y )dy = −∞ fX (a−y )fY (y )dy .
. Latter formula makes some intuitive sense. We’re integrating
over the set of x, y pairs that add up to a.
2
18.440 Lecture 23
Independent identically distributed (i.i.d.)

. The abbreviation i.i.d. means independent identically


distributed.
. It is actually one of the most important abbreviations in
probability theory.
. Worth memorizing.

3
18.440 Lecture 23
Summing i.i.d. uniform random variables

. Suppose that X and Y are i.i.d. and uniform on [0, 1]. So


fX = fY = 1 on [0, 1].
. What is the probability density function of X + Y ?
R ∞ R1
. fX +Y (a) = −∞ fX (a − y )fY (y )dy = 0 fX (a − y ) which is
the length of [0, 1] ∩ [a − 1, a].
. That’s a when a ∈ [0, 1] and 2 − a when a ∈ [1, 2] and 0
otherwise.

4
18.440 Lecture 23
Review: summing i.i.d. geometric random variables

. A geometric random variable X with parameter p has


P{X = k} = (1 − p)k−1 p for k ≥ 1.
. Sum Z of n independent copies of X ?
. We can interpret Z as time slot where nth head occurs in
i.i.d. sequence of p-coin tosses.
. So Z is negative
ik−1binomial
i n−1 (n, p). So
P{Z = k} = n−1 p (1 − p) k−n p.

5
18.440 Lecture 23
Summing i.i.d. exponential random variables

. Suppose X1 , . . . Xn are i.i.d. exponential random variables with


parameter λ. So fXi (x) = λe −λx on [0, ∞) for all 1 ≤ i ≤ n.
n
. What is the law of Z = i=1 Xi ?
. We claimed in an earlier lecture that this was a gamma
distribution with parameters (λ, n).
λe −λy (λy )n−1
. So fZ (y ) = Γ(n) .
. We argued this point by taking limits of negative binomial
distributions. Can we check it directly?
. By induction, would suffice to show that a gamma (λ, 1) plus
an independent gamma (λ, n) is a gamma (λ, n + 1).

6
18.440 Lecture 23
Summing independent gamma random variables
. Say X is gamma (λ, s), Y is gamma (λ, t), and X and Y are
independent.
. Intuitively, X is amount of time till we see s events, and Y is
amount of subsequent time till we see t more events.
λe −λx (λx)s−1 −λy (λy )t−1
. So fX (x) = Γ(s) and fY (y ) = λe Γ(t) .
R∞
. Now fX +Y (a) = −∞ fX (a − y )fY (y )dy .
. Up to an a-independent multiplicative constant, this is
Z a Z a
−λ(a−y ) s−1 −λy t−1 −λa
e (a−y ) e y dy = e (a−y )s−1 y t−1 dy .
0 0

. Letting x = y /a, this becomes


R1
e −λa as+t−1 0 (1 − x)s−1 x t−1 dx.
. This is (up to multiplicative constant) e −λa as+t−1 . Constant
must be such that integral from −∞ to ∞ is 1. Conclude
that X + Y is gamma (λ, s + t).
7
18.440 Lecture 23
Summing two normal variables
. X is normal with mean zero, variance σ12 , Y is normal with
mean zero, variance σ22 .
−x 2 −y 2
. fX (x) √ 1 e 2σ 2 2
√ 1 e 2σ2 .
= 2πσ1
1 and fY (y ) = 2πσ2
R∞
. We just need to compute − y )fY (y )dy .
fX +Y (a) = −∞ fX (a
. We could compute this directly.
. Or we could argue with a multi-dimensional bell curve picture
that if X and Y have variance 1 then fσ1 X +σ2 Y is the density
of a normal random variable (and note that variances and
expectations are additive).
. Or use fact that if Ai ∈ {−1, 1} are i.i.d. coin tosses then
Pσ2 N
√1 2
N i=1 Ai is approximately normal with variance σ when
N is large.

. Generally: if independent random Pvariables P Xj are normal

(µj , σj2 ) then n n n 2


P
j=1 X j is normal ( j=1 µ j , j=1 σj ).
8
18.440 Lecture 23
Other sums

. Sum of an independent binomial (m, p) and binomial (n, p)?


. Yes, binomial (m + n, p). Can be seen from coin toss
interpretation.
. Sum of independent Poisson λ1 and Poisson λ2 ?
. Yes, Poisson λ1 + λ2 . Can be seen from Poisson point process
interpretation.

9
18.440 Lecture 23
MIT OpenCourseWare
http://ocw.mit.edu

18.440 Probability and Random Variables


Spring 2014

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy