Scribd
Upload
56 views2 pages

MTH6141 Random Processes, Spring 2012 Solutions To Exercise Sheet 5

The document provides solutions to exercises on random processes. It calculates probabilities, expected values, and recurrence properties for Markov chains. The key points are: 1) For a 3-state Markov chain, it calculates probabilities and expected times to return to each state. 2) It proves a theorem about null recurrence of states in a Markov chain. 3) For a Markov chain with transition probabilities depending on step length, it shows the chain is null recurrent. In 3 sentences or less, this summarizes the essential information and high level ideas conveyed in the document.

Uploaded by

aset999
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
56 views2 pages

MTH6141 Random Processes, Spring 2012 Solutions To Exercise Sheet 5

The document provides solutions to exercises on random processes. It calculates probabilities, expected values, and recurrence properties for Markov chains. The key points are: 1) For a 3-state Markov chain, it calculates probabilities and expected times to return to each state. 2) It proves a theorem about null recurrence of states in a Markov chain. 3) For a Markov chain with transition probabilities depending on step length, it shows the chain is null recurrent. In 3 sentences or less, this summarizes the essential information and high level ideas conveyed in the document.

Uploaded by

aset999
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

MTH6141 Random Processes, Spring 2012 Solutions to Exercise Sheet 5

1. If we start in state 1 we can return in 2 steps (1 2 1) or 3 steps , (1 2 3 1). These are the only possibilities, so So P(R1 = 2) = 2 3 2 1 7 1 P(R1 = 3) = 3 and E(R1 ) = 3 2 + 3 3 = 3 . If we start in state 2 then possible paths are similar (2 1 2 and 2 2 3 1 2) and the calculation is identical, yielding P(R2 = 2) = 3 , 2 1 7 1 P(R2 = 3) = 3 and E(R2 ) = 3 2 + 3 3 = 3 . If we start in state 3 then we may return at any odd number t = 2k + 1 of steps (k 1), via a path of the form 3 1 2 1 2 3. Thus 1 2 k1 P(R3 = 2k + 1) = 3 (3) for k 1 (and all other values for R3 have zero 1 probability). I.e., 2 (R3 1) is distributed geometrically with parameter 1/3. 1 Hence E( 1 (R3 1)) = 1 E(R3 ) 2 = 3, and E(R3 ) = 7. (Or sum the series 2 2 by hand.) By Theorem 1.14, the equilibrium distribution is given by wk = 1/ E(Rk ), so 3 3 1 , 7 , 7 ). This agrees with the answer obtained by solving wP = w (c.f. is ( 7 Exercise Sheet 3). 2. Suppose that a b. Then there exist r and s such that pab > 0 and pba > 0. Hence (r) (t) (s) r+t+s) p( pab pbb pba , for all t N. aa Now set u = r + s and = pab pba , to obtain the inequality we were required to prove. Now suppose that a is null recurrent. Recurrence is a class property (Theorem 1.12) and so state b is also recurrent. Now, by Theorem 1.13, since a is (t) (t) (t+u) null recurrent, paa 0 as t . Now pbb 1 paa , and the right-hand side converges to 0 as t . Hence the left-hand side also converges to 0 and, by Theorem 1.13, state b is null recurrent. 3. By considering the transition diagram we see that f00 = p0,1 p1,2 p2,3 . . . pt2,t1 pt1,0 = Now f00 =
t=1 (t) (r) (s) (r ) (s)

1 2

2 3

3 4

...

t1 t

1 t+1

1 . t(t + 1)

f00 =
t=1

(t)

1 = t(t + 1)

t=1

1 1 t t+1

=1

and so the chain is recurrent.

Also, E(R0 ) =

tf00 =
t=1 t=1

(t)

1 = t+1

and so the chain is null recurrent. The transition diagram for this process has the same shape as that for the success runs chain. Only the numerical values of the positive probabililties have changed. You could think of this process as being like the success runs chain except that the longer the current run of successes is the more likely it is to be extended. For instance a football team may be more condent during a long run of wins and hence more likely to win the next game. 4. As in the lecture, we argue that, in order to be at state 0 at time 2n, we must n have made exactly n increasing transitions and n decreasing. There are 2n possible such sequences. Each sequence occurs with probability pn (1 p)n . Thus the probability of making a transition from state 0 to state 0 in 2n steps n n is 2n p (1 p)n . It follows that p00 =
(2n)

2n n p (1 p)n 22n [p(1 p)]n = [4p(1 p)]n , n

n where we have used the fact that the binomial coecient 2n is certainly less 2n 2n 2n than the sum k=0 k = 2 of all coecients. The function r = r(p) = 1 and is everywhere else 4p(1 p) achieves a maximum of 1 at the point p = 2 1 strictly less than 1 (and non-negative). So when p = 2 , r < 1 and (t) p00 t=1 (2n) p00 n=1

n=1 (t)

rn

1 < , 1r

where we have used the fact that p00 = 0 when t is odd. Hence, by Lemma 1.12, state 0 is transient. By symmetry (or by Theorem 1.13) all states are transient. 5. Hint: Consider the symmetric random walk (Yt ) and the related stochastic process (Xt ) given by Xt = |Yt |. Is Xt a Markov chain? If so, what are its transition probabilities?

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy