S&DS 241 Lecture 10

Random walk: Gambler’s ruin

B-H: 2.7 (Example 2.7.3), math appendix A.4,A.8

1/28
Random walk
A particle starts at 0, and at each step it either moves 1 unit to the right
with probability p or to the left with probability q = 1 − p, independently.
q p

−4 −3 −2 −1 0 1 2 3 4

Let Sn be the particle’s position after n steps.

10 20 30 40 50

-2

-4

-6

-8

2/28
PMF of Sn
Let (
+1 with prob p
Zi = ith step =
−1 with prob q
Then
Sn = Z1 + · · · + Zn
| {z }
iid

3/28
PMF of Sn
Let (
+1 with prob p
Zi = ith step =
−1 with prob q
Then
Sn = Z1 + · · · + Zn
| {z }
iid

To relate to binomial,
• Let X= number of steps to the right ∼ Bin(n, p)

3/28
PMF of Sn
Let (
+1 with prob p
Zi = ith step =
−1 with prob q
Then
Sn = Z1 + · · · + Zn
| {z }
iid

To relate to binomial,
• Let X= number of steps to the right ∼ Bin(n, p)
• Then Sn = 2X − n ∈ {−n, −n + 2, . . . , n − 2, n} and
 
n n+j n−j
P (Sn = j) = P (X = (n + j)/2) = n+j p 2 q 2
2

3/28
PMF of Sn
Let (
+1 with prob p
Zi = ith step =
−1 with prob q
Then
Sn = Z1 + · · · + Zn
| {z }
iid

To relate to binomial,
• Let X= number of steps to the right ∼ Bin(n, p)
• Then Sn = 2X − n ∈ {−n, −n + 2, . . . , n − 2, n} and
 
n n+j n−j
P (Sn = j) = P (X = (n + j)/2) = n+j p 2 q 2
2

• Special case: symmetric random walk (p = 1/2)

 
n −n
P (Sn = j) = P (X = (n + j)/2) = n+j 2
2
3/28
Gambler’s ruin
Two gamblers, with a bankroll of $3 and $7, respectively, bet on
independent tosses of a fair coin. The first gambler wins $1 if a toss is
head; the second gambler wins $1 if a toss is tail. The game stops if
either runs out of money. What is the probability that the first gambler
wins all the money?

4/28
Gambler’s ruin: general version
Two gamblers, with a bankroll of $k and $n − k, respectively, bet on
independent tosses of a coin. The first gambler wins $1 if a toss is head,
with probability p; the second gambler wins $1 if a toss is tail with
probability q = 1 − p. The game stops if either runs out of money. What
is the probability that the first gambler wins all the money?

5/28
Gambler’s ruin: general version
Two gamblers, with a bankroll of $k and $n − k, respectively, bet on
independent tosses of a coin. The first gambler wins $1 if a toss is head,
with probability p; the second gambler wins $1 if a toss is tail with
probability q = 1 − p. The game stops if either runs out of money. What
is the probability that the first gambler wins all the money?
• In the language of random walk: start the walk at k, what is the
probability that the particle hits 0 before hitting n?
q p
wealth of 1st gambler
0 k n

5/28
Gambler’s ruin: general version
Two gamblers, with a bankroll of $k and $n − k, respectively, bet on
independent tosses of a coin. The first gambler wins $1 if a toss is head,
with probability p; the second gambler wins $1 if a toss is tail with
probability q = 1 − p. The game stops if either runs out of money. What
is the probability that the first gambler wins all the money?
• In the language of random walk: start the walk at k, what is the
probability that the particle hits 0 before hitting n?
q p
wealth of 1st gambler
0 k n

Goal
• How does winning probability depends on the initial wealth?
• How does winning probability depends on the chance of each toss?

5/28
Special case: Tennis (k = n − k = 2)
(Lecture 4) Alice plays tennis against Bob. The game is at deuce.
Suppose
• Alice wins each point with probability p and loses with probability
q =1−p
• Each point is played independently
• The game is won by the player who leads by 2 points

6/28
Special case: Tennis (k = n − k = 2)
(Lecture 4) Alice plays tennis against Bob. The game is at deuce.
Suppose
• Alice wins each point with probability p and loses with probability
q =1−p
• Each point is played independently
• The game is won by the player who leads by 2 points
Solution:
p2
P (Alice eventually wins) =
p2 + q 2

6/28
Special case: Cliff (k = 1, n = ∞)
(PSet 3) A drunkard is standing one step away
from the cliff on his left. He moves randomly, one
step at a time and independently, either to the
0 1 2 3 4
right (away from the cliff) with probability p or left
(toward the cliff) with probability 1 − p.

7/28
Special case: Cliff (k = 1, n = ∞)
(PSet 3) A drunkard is standing one step away
from the cliff on his left. He moves randomly, one
step at a time and independently, either to the
0 1 2 3 4
right (away from the cliff) with probability p or left
(toward the cliff) with probability 1 − p.

Solution:
(
1
1 0≤p≤ 2
P (eventually falls down the cliff) = 1−p 1
p 2 ≤p≤1
1.0

0.8

0.6

0.4

0.2

0.2 0.4 0.6 0.8 1.0

7/28
Fair coin: p = 1/2
Winning probabilities
Let Pk be the probability that the first gambler eventually wins if he
starts with a bankroll of $k and his opponent starts with $n − k, i.e.,

Pk = P (1st gambler eventually wins all the money starting with $k)
Qk = P (1st gambler eventually loses all the money starting with $k)

9/28
Winning probabilities
Let Pk be the probability that the first gambler eventually wins if he
starts with a bankroll of $k and his opponent starts with $n − k, i.e.,

Pk = P (1st gambler eventually wins all the money starting with $k)
Qk = P (1st gambler eventually loses all the money starting with $k)

9/28
Winning probabilities
Let Pk be the probability that the first gambler eventually wins if he
starts with a bankroll of $k and his opponent starts with $n − k, i.e.,

Pk = P (1st gambler eventually wins all the money starting with $k)
Qk = P (1st gambler eventually loses all the money starting with $k)

• Question: Is it true that Pk + Qk = 1?

9/28
Winning probabilities
Let Pk be the probability that the first gambler eventually wins if he
starts with a bankroll of $k and his opponent starts with $n − k, i.e.,

Pk = P (1st gambler eventually wins all the money starting with $k)
Qk = P (1st gambler eventually loses all the money starting with $k)

• Question: Is it true that Pk + Qk = 1?

• It is unclear from first principles that Pk + Qk = 1, because the third
possibility is that the game never ends (keep going back and forth)!

9/28
Winning probabilities
Let Pk be the probability that the first gambler eventually wins if he
starts with a bankroll of $k and his opponent starts with $n − k, i.e.,

Pk = P (1st gambler eventually wins all the money starting with $k)
Qk = P (1st gambler eventually loses all the money starting with $k)

• Question: Is it true that Pk + Qk = 1?

• It is unclear from first principles that Pk + Qk = 1, because the third
possibility is that the game never ends (keep going back and forth)!
• It turns out that the game ends with probability one.

9/28
Winning probabilities
Let Pk be the probability that the first gambler eventually wins if he
starts with a bankroll of $k and his opponent starts with $n − k, i.e.,

Pk = P (1st gambler eventually wins all the money starting with $k)
Qk = P (1st gambler eventually loses all the money starting with $k)

• Question: Is it true that Pk + Qk = 1?

• It is unclear from first principles that Pk + Qk = 1, because the third
possibility is that the game never ends (keep going back and forth)!
• It turns out that the game ends with probability one.
• Let’s first compute Pk . By definition: Pn = 1 and P0 = 0.
• Next: find a recursion for Pk

9/28
Law of total probability

Gambling process
• If the next toss is head, 1st gambler then has $(k + 1)
• If the next toss is tail, 1st gambler then has $(k − 1)

10/28
Law of total probability

Gambling process
• If the next toss is head, 1st gambler then has $(k + 1)
• If the next toss is tail, 1st gambler then has $(k − 1)
Therefore
1
Pk = P (1st gambler wins starting with $k + 1|next toss head)
2| {z }
Pk+1
1
+ P (1st gambler wins starting with $k − 1|next toss tail)
2| {z }
Pk−1

10/28
Difference equation (B-H Math Appendix A.5)

(
Pk = 12 Pk−1 + 12 Pk+1 , k = 1, . . . , n − 1 (recursion)
P0 = 0, Pn = 1 (boundary conditions)

11/28
Solving difference equation: focus on the increment

For 1 ≤ k ≤ n − 1, then
1 1
Pk = Pk−1 + Pk+1
2 2

12/28
Solving difference equation: focus on the increment

For 1 ≤ k ≤ n − 1, then
1 1
Pk = Pk−1 + Pk+1
2 2
1 1 1 1
⇐⇒ Pk + Pk = Pk−1 + Pk+1
2 2 2 2

12/28
Solving difference equation: focus on the increment

For 1 ≤ k ≤ n − 1, then
1 1
Pk = Pk−1 + Pk+1
2 2
1 1 1 1
⇐⇒ Pk + Pk = Pk−1 + Pk+1
2 2 2 2
⇐⇒ Pk − Pk−1 = Pk+1 − Pk
| {z } | {z }
∆k−1 ∆k

12/28
Solving difference equation: focus on the increment

For 1 ≤ k ≤ n − 1, then
1 1
Pk = Pk−1 + Pk+1
2 2
1 1 1 1
⇐⇒ Pk + Pk = Pk−1 + Pk+1
2 2 2 2
⇐⇒ Pk − Pk−1 = Pk+1 − Pk
| {z } | {z }
∆k−1 ∆k

⇐⇒ ∆0 = ∆1 = · · · = ∆n−1

12/28
Solving difference equation: focus on the increment

Recall boundary conditions: P0 = 0, Pn = 1. Then

1 = Pn = Pn − Pn−1 + Pn−1 − Pn−2 + · · · + P1 − P0 = n∆0

| {z } | {z } | {z }
∆n−1 ∆n−2 ∆0

13/28
Solving difference equation: focus on the increment

Recall boundary conditions: P0 = 0, Pn = 1. Then

1 = Pn = Pn − Pn−1 + Pn−1 − Pn−2 + · · · + P1 − P0 = n∆0

| {z } | {z } | {z }
∆n−1 ∆n−2 ∆0

and

Pk = Pk − Pk−1 + Pk−1 − Pk−2 + · · · + P1 − P0 = k∆0

| {z } | {z } | {z }
∆k−1 ∆k−2 ∆0

13/28
Solving difference equation: focus on the increment

Recall boundary conditions: P0 = 0, Pn = 1. Then

1 = Pn = Pn − Pn−1 + Pn−1 − Pn−2 + · · · + P1 − P0 = n∆0

| {z } | {z } | {z }
∆n−1 ∆n−2 ∆0

and

Pk = Pk − Pk−1 + Pk−1 − Pk−2 + · · · + P1 − P0 = k∆0

| {z } | {z } | {z }
∆k−1 ∆k−2 ∆0

Thus
k
Pk = , k = 1, . . . , n
n

13/28
Winning and losing probability

By symmetry:

Qk = P (1st gambler loses all the money starting with $k)

= P (2nd gambler wins all the money starting with $n − k)
n−k
= Pn−k =
n

14/28
Winning and losing probability

By symmetry:

Qk = P (1st gambler loses all the money starting with $k)

= P (2nd gambler wins all the money starting with $n − k)
n−k
= Pn−k =
n
So
k n−k
Pk = , Qk = , k = 1, . . . , n
n n
and
P (game never ends) = 1 − Pk − Qk = 0

14/28
Back to example
Two gamblers, with a bankroll of $3 and $7, bet on independent tosses
of a fair coin. The first gambler wins $1 if a toss is head; the second wins
$1 if a toss is a tail. The game stops if either runs out of money.

3
P (1st gambler wins all the money) =
10
7
P (2nd gambler wins all the money) =
10

15/28
Playing against casino

A gambler with a bankroll of $k bet on independent tosses of a fair coin

against the casino with $∞ bankroll. The gambler wins $1 if a toss is
head; the casino wins $1 if a toss is a tail. The game stops if either runs
out of money.
P (gambler wins) = 0
P (casino wins) = 1

16/28
Playing against casino

A gambler with a bankroll of $k bet on independent tosses of a fair coin

against the casino with $∞ bankroll. The gambler wins $1 if a toss is
head; the casino wins $1 if a toss is a tail. The game stops if either runs
out of money.
P (gambler wins) = 0
P (casino wins) = 1

• As long as the initial wealth k is finite, the gambler is doomed if the

game is fair
• Also explains the cliff problem for p = 1/2: P (eventual fall) = 1
regarding of the starting position.

16/28
Lesson

“Millionaires should always gamble, poor men never.”

— J. M. Keynes

17/28
Lesson

“Millionaires should always gamble, poor men never.”

— J. M. Keynes

“Gambling is risk-taking. It might be said the owner of a casino

gambles, takes risks, but he has the odds in his favour, so that’s
intelligent gambling. If I wanted to gamble, I’d buy the casino.”

— J. P. Getty

17/28
Biased coin: p ̸= 1/2
Law of total probability

Gambling process
• If the next toss is head, 1st gambler then has $(k + 1)
• If the next toss is tail, 1st gambler then has $(k − 1)

19/28
Law of total probability

Gambling process
• If the next toss is head, 1st gambler then has $(k + 1)
• If the next toss is tail, 1st gambler then has $(k − 1)
Therefore

Pk = p P (1st gambler wins starting with $k + 1|next toss head)

| {z }
Pk+1

+ q P (1st gambler wins starting with $k − 1|next toss tail)

| {z }
Pk−1

19/28
Difference equation

(
Pk = qPk−1 + pPk+1 , k = 1, . . . , n − 1 (recursion)
P0 = 0, Pn = 1 (boundary conditions)

Next we solve this difference equation (B-H Math Appendix A.4)

20/28
Solving difference equation: focus on the increment

For 1 ≤ k ≤ n − 1, then (recall q = 1 − p)

Pk = qPk−1 + pPk+1

21/28
Solving difference equation: focus on the increment

For 1 ≤ k ≤ n − 1, then (recall q = 1 − p)

Pk = qPk−1 + pPk+1
⇐⇒ qPk + pPk = qPk−1 + pPk+1

21/28
Solving difference equation: focus on the increment

For 1 ≤ k ≤ n − 1, then (recall q = 1 − p)

Pk = qPk−1 + pPk+1
⇐⇒ qPk + pPk = qPk−1 + pPk+1
⇐⇒ q (Pk − Pk−1 ) = p (Pk+1 − Pk )
| {z } | {z }
∆k−1 ∆k

21/28
Solving difference equation: focus on the increment

For 1 ≤ k ≤ n − 1, then (recall q = 1 − p)

Pk = qPk−1 + pPk+1
⇐⇒ qPk + pPk = qPk−1 + pPk+1
⇐⇒ q (Pk − Pk−1 ) = p (Pk+1 − Pk )
| {z } | {z }
∆k−1 ∆k
q
⇐⇒ ∆k = ∆k−1
p

21/28
Solving difference equation: focus on the increment

For 1 ≤ k ≤ n − 1, then (recall q = 1 − p)

Pk = qPk−1 + pPk+1
⇐⇒ qPk + pPk = qPk−1 + pPk+1
⇐⇒ q (Pk − Pk−1 ) = p (Pk+1 − Pk )
| {z } | {z }
∆k−1 ∆k
q
⇐⇒ ∆k = ∆k−1
p

In other words,  k
q
∆k = ∆0
p

21/28
Solving difference equation: focus on the increment
Recall boundary conditions: P0 = 0, Pn = 1. Then1

1 = Pn = Pn − Pn−1 + Pn−1 − Pn−2 + · · · + P1 − P0

| {z } | {z } | {z }
∆n−1 ∆n−2 ∆0
 n
q
q
 n−1 !
q 1 − p
= ∆0 1 + + ··· + = ∆0
p p 1 − pq

1 1−an
Recall finite geometric sum: 1 + a + · · · + an−1 = 1−a
(B-H A.8).
22/28
Solving difference equation: focus on the increment
Recall boundary conditions: P0 = 0, Pn = 1. Then1

1 = Pn = Pn − Pn−1 + Pn−1 − Pn−2 + · · · + P1 − P0

| {z } | {z } | {z }
∆n−1 ∆n−2 ∆0
 n
q
q
 n−1 !
q 1 − p
= ∆0 1 + + ··· + = ∆0
p p 1 − pq

Similarly:  k
q
1− p
Pk = ∆ 0
1 − pq

1 1−an
Recall finite geometric sum: 1 + a + · · · + an−1 = 1−a
(B-H A.8).
22/28
Solving difference equation: focus on the increment
Recall boundary conditions: P0 = 0, Pn = 1. Then1

1 = Pn = Pn − Pn−1 + Pn−1 − Pn−2 + · · · + P1 − P0

| {z } | {z } | {z }
∆n−1 ∆n−2 ∆0
 n
q
q
 n−1 !
q 1 − p
= ∆0 1 + + ··· + = ∆0
p p 1 − pq

Similarly:  k
q
1− p
Pk = ∆ 0
1 − pq

Thus  k
q
1− p
Pk =  n , k = 1, . . . , n
q
1− p

1 1−an
Recall finite geometric sum: 1 + a + · · · + an−1 = 1−a
(B-H A.8).
22/28
Winning and losing probability

Given
 k
q
1− p
Pk =  n
q
1− p

swapping k ↔ n − k and p ↔ q gives:

 n−k
p
1− q
Qk =  n
p
1− q

23/28
Winning and losing probability

Given
 k
q
1− p
Pk =  n
q
1− p

swapping k ↔ n − k and p ↔ q gives:

 n−k
p
1− q
Qk =  n
p
1− q

and
P (game never ends) = 1 − Pk − Qk = 0

23/28
Summary
• For p = 12 :
k
Pk = , k = 1, . . . , n
n
• For p ̸= 1
2
 k
q
1− p
Pk =  n , k = 1, . . . , n
q
1− p

How to reconcile these two results?

24/28
Summary
• For p = 12 :
k
Pk = , k = 1, . . . , n
n
• For p ̸= 1
2
 k
q
1− p
Pk =  n , k = 1, . . . , n
q
1− p

How to reconcile these two results? Take the limit of p → 12 .

24/28
Summary
• For p = 12 :
k
Pk = , k = 1, . . . , n
n
• For p ̸= 1
2
 k
q
1− p
Pk =  n , k = 1, . . . , n
q
1− p

How to reconcile these two results? Take the limit of p → 12 .

• Note that
   k−1 
q
1− p 1 + p + · · · + pq
q

Pk =    n−1 
q
1− p 1 + p + · · · + pq
q

24/28
Summary
• For p = 12 :
k
Pk = , k = 1, . . . , n
n
• For p ̸= 1
2
 k
q
1− p
Pk =  n , k = 1, . . . , n
q
1− p

How to reconcile these two results? Take the limit of p → 12 .

• Note that
   k−1 
q
1 + p + · · · + pq
q
H
1 −Hp
H
H p=q k
Pk =    n−1  = n
1H−Hpq 1 + pq + · · · + pq
H
H

24/28
Special case: k = 3, n − k = 7
Two gamblers, with a bankroll of $3 and $7, bet on independent tosses
of a fair coin. The first gambler wins $1 if a toss is head, with probability
p; the second wins $1 if a toss is a tail, with probability q = 1 − p. The
game stops if either runs out of money. What is the probability that the
first gambler wins all the money?

25/28
Special case: k = 3, n − k = 7
Two gamblers, with a bankroll of $3 and $7, bet on independent tosses
of a fair coin. The first gambler wins $1 if a toss is head, with probability
p; the second wins $1 if a toss is a tail, with probability q = 1 − p. The
game stops if either runs out of money. What is the probability that the
first gambler wins all the money?
• When p = 0.56

0.44 3


1− 0.56
P3 = ≈ 0.566 > 0.3
0.44 10


1− 0.56
P_3

• P3 vs p: 1.0

0.8

0.6

0.4

0.2

p
0.2 0.4 0.6 0.8 1.0

25/28
Special case: Tennis (k = n − k = 2)
(Lecture 4) Alice plays tennis against Bob. The game is at deuce.
Suppose
• Alice wins each point with probability p and loses with probability
q =1−p
• Each point is played independently
• The game is won by the player who leads by 2 points

 2
q
1− p 1 p2
P2 = P (Alice eventually wins) =  4 =  2 =
q q p2 + q 2
1− p 1+ p

26/28
Special case: Cliff (n = ∞)
(PSet 3) A drunkard is standing one step away
from the cliff on his left. He moves randomly, one
step at a time and independently, either to the
0 1 2 3 4
right (away from the cliff) with probability p or left
(toward the cliff) with probability q = 1 − p.

27/28
Special case: Cliff (n = ∞)
(PSet 3) A drunkard is standing one step away
from the cliff on his left. He moves randomly, one
step at a time and independently, either to the
0 1 2 3 4
right (away from the cliff) with probability p or left
(toward the cliff) with probability q = 1 − p.

1 − pq
(
1
0 p≤ 2
P1 = P (never fall starting at 1) = lim  n = q
n→∞ 1
1 − pq 1− p p> 2

27/28
Special case: Cliff (n = ∞)
(PSet 3) A drunkard is standing one step away
from the cliff on his left. He moves randomly, one
step at a time and independently, either to the
0 1 2 3 4
right (away from the cliff) with probability p or left
(toward the cliff) with probability q = 1 − p.

1 − pq
(
1
0 p≤ 2
P1 = P (never fall starting at 1) = lim  n = q
n→∞ 1
1 − pq 1− p p> 2

Similarly,
 k 
q 1
1− p
0 p≤ 2
Pk = P (never fall starting at k) = lim  n =  k
q
n→∞ q 1 − 1
1− p p p> 2

27/28
Special case: Cliff (n = ∞)
(PSet 3) A drunkard is standing one step away
from the cliff on his left. He moves randomly, one
step at a time and independently, either to the
0 1 2 3 4
right (away from the cliff) with probability p or left
(toward the cliff) with probability q = 1 − p.

1 − pq
(
1
0 p≤ 2
P1 = P (never fall starting at 1) = lim  n = q
n→∞ 1
1 − pq 1− p p> 2

Similarly,
 k 
q 1
1− p
0 p≤ 2
Pk = P (never fall starting at k) = lim  n =  k
q
n→∞ q 1 − 1
1− p p p> 2

(This is simply because 1 − Pk = (1 − P1 )k )

27/28
Special case: Cliff (n = ∞)

1
1 p≤ 2
P (eventually fall starting at k) =  q k
1

p p> 2

P (fall)
1.0

0.8
k=1

0.6

0.4
k=2
0.2
k=3

p
0.2 0.4 0.6 0.8 1.0

28/28