Scribd
Upload
15 views2 pages

Stochastic Problem2

This document contains instructions for homework 1 of Statistics 251b/551b. It asks students to solve several problems regarding Markov chains. Specifically, it asks students to: 1) Show that every state i in a finite, irreducible Markov chain is positive recurrent by proving several steps including showing the expected return time EiTi is finite. 2) Consider a Markov chain on the states {0,1,2,...} with transition probabilities dependent on parameters α and β. Show that if α > 1/2, the chain is transient. 3) Use a fact about the binomial distribution to prove part 2, showing that the probability the chain is in state 0 after n steps is less than an exponentially decreasing

Uploaded by

aapirasteh1402
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
15 views2 pages

Stochastic Problem2

This document contains instructions for homework 1 of Statistics 251b/551b. It asks students to solve several problems regarding Markov chains. Specifically, it asks students to: 1) Show that every state i in a finite, irreducible Markov chain is positive recurrent by proving several steps including showing the expected return time EiTi is finite. 2) Consider a Markov chain on the states {0,1,2,...} with transition probabilities dependent on parameters α and β. Show that if α > 1/2, the chain is transient. 3) Use a fact about the binomial distribution to prove part 2, showing that the probability the chain is in state 0 after n steps is less than an exponentially decreasing

Uploaded by

aapirasteh1402
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Statistics 251b/551b, spring 2009

Homework # 1
Due: Monday 26 January
If you are not able to solve a part of a problem, you can still get credit for later
parts: Just assume the truth of what you unable to prove in the earlier part.
[1] Consider an irreducible Markov chain with a finite state space S. Show that every
state i is positive recurrent by following these steps. Write S\i for {j ∈ S : j 6= i}.
(i) [5 points] Explain why, for each j in S\i, there exists a positive integer N (j) such
Pi {XN (j) = i} > 0. Define N := 1 + maxj N (j). Deduce that the quantity δ
defined by
δ := minj∈S\i Pj {Ti ≤ N − 1}
is strictly positive and that Pj {Ti ≥ N } ≤ 1 − δ for all j in S\i.
(ii) [5 points] For all j and k in S\i, explain why
Pj {Ti ≥ 2N | XN = k, Ti ≥ N } = Pk {Ti ≥ N }
Hint: {Ti ≥ m} = ∩m−1
n=1 {Xn 6= i}.
(iii) [5 points] Deduce that Pj {Ti ≥ 2N } ≤ (1 − δ)2 for all j in S\i.
(iv) [bonus points] Show that Pj {Ti ≥ mN } ≤ (1−δ)m for all m ∈ N and all j ∈ S\i.
(v) [5 points] Deduce that Pi {Ti ≥ mN + 1} ≤ (1 − δ)m for each m in N.
(vi) [5 points] Explain why
X
Ti ≤ N + N 1{Ti ≥ mN + 1}.
m∈N

Hint: Consider what happens to the right-hand side when N < Ti ≤ 2N .


(vii) [5 points] Conclude that Ei Ti < ∞. That is, i is a positive recurrent state.
[2] [10 points] Consider a Markov chain with state space S = {0, 1, 2, . . . } and tran-
sition probabilities
α = P (i, i + 1) for all i ≥ 0
β = P (0, 0) = P (i, i − 1) for all i ≥ 1
where α+β = 1 and α > 1/2. Suppose we generate the chain by repeated tossing
of a coin that lands heads with probability α, moving one step to the right for a
head and one step to the left (or just stay where we are if Xn = 0) for a tail. Let
Sn denote the number of tails in the first n tosses.
(i) [5 points] Suppose the chain starts in state 0. Explain why Xn ≥ (n − Sn ) − Sn
for all n.
(ii) [5 points] Show that P0 {Xn = 0} ≤ P{Sn ≥ nβ(1+2δ)} where δ = (α −β)/4β.
(iii) [5 points] Use the Fact about the Binomial distribution (see next page) to deduce
that the chain is transient when α > 1/2.
FACT ( ABOUT THE B INOMIAL DISTRIBUTION ) Suppose Y ∼ Bin(n, θ) for some
0 < θ < 1. Then for each δ > 0,

P{Y ≥ nθ(1 + 2δ)} ≤ ρ−n where ρ = (1 + δ)δθ .

P ROOF For each s > 0

1{Y ≥ nθ(1 + 2δ)} ≤ (1 + s)Y −nθ(1+2δ)

because the right-hand side is always nonnegative and it is at least 1 when Y ≥


nθ(1 + 2δ). Take expectations, using the fact that E(1 + s)Y = (1 + θs)n , to
deduce that
 n  n
1 + 2sθ 1
P{Y ≥ nθ(1 + 2δ)} ≤
(1 + s)θ+θδ (1 + s)θδ

Choose s equal to δ, which makes the second term equal to ρ−n . The logarithm of
the first term then equals n times

log(1 + θδ) − θ(1 + δ) log(1 + δ).

Invoke the inequality


x
≤ log(1 + x) ≤ x for x > 0
1+x
to bound the last difference from above by θδ − θ(1 + δ)δ/(1 + δ) = 0.


You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy