Stochastic Models

MA 310

Dr. Sudipta Das

Outline I

1 Renewal Process
Poisson process

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 2 / 51

Introduction I

Definition 1:-
Counting Processes: A stochastic process {N(t), t ≥ 0}is said to
be a counting process if N(t) represents the total number of
“events" that occur by time t.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 3 / 51

Introduction II

From its definition we see that for a counting process N(t) must
satisfy:
1 N(t) ≥ 0.
2 N(t) is integer valued.
3 If s < t, then N(s) ≤ N(t).
4 For s < t,
N(t) − N(s)
equals the number of events that occur in the interval (s, t].

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 4 / 51

Introduction III

Some examples of counting processes are the following:

1 If we let N(t) equal the number of persons who enter a particular
store at or prior to time t, then {N(t), t ≥ 0} is a counting process in
which an event corresponds to a person entering the store.
Note that if we had let N(t) equal the number of persons in the store
at time t, then {N(t), t ≥ 0} would not be a counting process.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 5 / 51

Introduction IV

2 If we say that an event occurs whenever a child is born, then

{N(t), t ≥ 0} is a counting process when N(t) equals the total
number of people who were born by time t.
N(t) must include persons who have died by time t
3 If N(t) equals the number of goals that a given soccer player scores
by time t, then {N(t), t ≥ 0} is a counting process. An event of this
process will occur whenever the soccer player scores a goal.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 6 / 51

Introduction V

A counting process is, sometimes, called a point process.

Recall that
Counting/point process N(t), t ≥ 0, is a random process taking
values in 0, 1, 2, . . . , such that N(0) = 0, and, for each ω, N(t, ω) is
a nondecreasing, right continuous step function.
Hence, it can be seen characterised by a random distribution of
points on R+ (i.e., the jump instants), and a sequence of integer
random variables corresponding to the jump at each point.
Therefore, it is called point process, also.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 7 / 51

Definition I

Definition 2:-
Poisson process:- A counting/point process {N(t), t ≥ 0}, is
called a Poisson process if
1 All jumps of N(t) are of unit size, with probability 1,
2 For all t, s ≥ 0, (N(t + s) − N(t)) is independent of {N(u) : u ≤ t},
3 For all t, s ≥ 0, distribution of N(t + s) − N(t) does not depend on t.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 8 / 51

Definition II

Remarks:-
For a point/counting process N(t), given t ≥ 0 and τ ≥ 0,

N(t + τ ) − N(t)

is called the increment of N(t) over the interval (t, t + τ ],

i.e., N(t + τ ) − N(t) is the cumulative value of the jumps in the
interval (t, t + τ ].

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 9 / 51

Definition III
The definition asserts that any increment of a Poisson process is
independent of the past of the process. However, we can conclude
more.
Consider t1 < t2 < . . . < tn then, by property (2) in the definition,

N(t1 ) q (N(t2 ) − N(t1 ))

Further, again using the same property,

N(t3 ) − N(t2 ) q (N(t1 ), N(t2 ))

or, equivalently,

(N(t3 ) − N(t2 )) q (N(t1 ), N(t2 ) − N(t1 )).

Thus, it follows that,

(N(t3 ) − N(t2 )), (N(t2 ) − N(t1 )), N(t1)

are mutually independent.

Thus we conclude that the increments of a Poisson process (over
disjoint intervals) are independent.
This is known as the independent increment property.
Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 10 / 51
Definition IV

By property (3) of the definition one can state that the distribution
of an increment depends only on the width of the interval over
which it is taken, not the location of the interval in time.
This is called the stationary increment property.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 11 / 51

Definition V

Thus we can say that a Poisson process is a counting/point

process with stationary and independent increments. In addition,
at each point of a Poisson process there is a unit jump.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 12 / 51

Properties I

Theorem 1:-
If the function f (x), x ≥ 0, is
1 right continuous at x = 0, and
2 non-increasing in x, and
3 for each x ≥ 0, y ≥ 0,

f (x + y ) = f (x)f (y ).

Then,
either f (x) = 0, x ≥ 0, or
there exists λ, 0 ≤ λ < ∞, such that

f (x) = e−λx ,

for x ≥ 0.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 13 / 51

Properties II

Lemma 1:-
For a Poisson process there exists a 0 ≤ λ < ∞, such that, for all
t ≥ 0,
P[N(t) = 0] = e−λt ,
and λ is called the rate of the process.
Remark:-
Define T1 := inf{t > 0 : N(t) 6= 0}, the first “jump" time of N(t).
This lemma asserts that the time until the first jump in a Poisson
process is exponentially distributed with mean 1/λ, i.e.,

P[T1 > t] = P ([N(t) = 0]) = e−λt .

Note: pdf of T1 , i..e., fT1 (t) = − dtd P[T1 > t] = λe−λt .

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 14 / 51

Properties III
Proof of Lemma 1:-
Using the stationary and independent increment property, one can
write

P(N(t + s) = 0) = P(N(t) = 0, N(t + s) − N(t) = 0)

= P(N(t) = 0)P(N(t + s) − N(t) = 0)
= P(N(t) = 0)P(N(s) = 0)

Let us write, for t ≥ 0,

f (t) = P(N(t) = 0).

Thus, we have established that f (·) satisfies the functional

equation: for all s, t ≥ 0,

f (t + s) = f (t)f (s).

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 15 / 51

Properties IV
Since N(0) = 0, we have

f (0) = 1.

Now, T1 , the first jump time of N(t). Hence,

f (t) = P(N(t) = 0) = P(T1 > t).

Thus, f (t) is the complementary c.d.f. of a nonnegative random

variable; hence,
f (t), t ≥ 0, is right continuous and
non-increasing.
It then follows from the above Theorem 1, that the only nonzero
solution of the functional equation is

f (t) = e−λt ,

for some λ, 0 ≤ λ < ∞.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 16 / 51
Properties V

Lemma 2:-
1
lim P(N(t) ≥ 2) = 0.
t→0 t

Remark:-
This result states that

P(N(t) ≥ 2) = o(t),

i.e., that the probability of there being 2 or more points of the

process in an interval of length t decreases to 0 faster than t, as t
decreases to 0.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 17 / 51

Properties VI

Lemma 3:-
1
lim P(N(t) = 1) = λ,
t→0 t
where λ is as obtained in Lemma 1.
Remark:-
In other words, P(N(t) = 1) can be approximated as λt + o(t) as
t → 0.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 18 / 51

Properties VII

Proof of Lemma 3:-

P(N(t) = 1) = 1 − P(N(t) = 0) − P(N(t) ≥ 2)

= 1 − e−λt − P(N(t) ≥ 2)
1 − e−λt
 
1 P(N(t) ≥ 2)
⇒ lim P(N(t) = 1) = lim −
t→0 t t→0 t t
= λ.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 19 / 51

Properties VIII

Theorem 2:-
If N(t), t ≥ 0, is a Poisson process then, for all t ≥ 0, and
k ∈ {0, 1, 2, . . .},

(λt)k −λt
P(N(t) = k ) = e ,
k!
where λ is the rate of the process.
Remark:-
Thus an increment of a Poisson process over an interval of length
t is Poisson distributed with parameter λt.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 20 / 51

Properties IX

Proof of Theorem 2:-

For 0 < α < 1, define
 
G(t) = E αN(t) ,

i.e., G(t) is the moment generating function of the random variable

N(t).
For notational simplicity, we have not retained α as an argument of
the moment generating function.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 21 / 51

Properties X

Now, using the stationary and independent increment property, we

obtain a functional equation for G(t) as follows
 
G(t + s) = E αN(t+s)
 
= E αN(t) · αN(t+s)−N(t)
   
= E αN(t) · E αN(t+s)−N(t)
   
= E αN(t) · E αN(s)
= G(t)G(s)

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 22 / 51

Properties XI

Since N(0) = 0, G(0) = 1;

and since N(t) increases with t, and 0 < α < 1, we also conclude
that G(t) is non-increasing with t.
Also, by previous lemmas, we can write
 
lim G(t) = lim E αN(t)
t→0 t→0
∞
!
X
0 k
= lim α P(N(t) = 0) + αP(N(t) = 1) + α P(N(t) = k )
t→0
k =2
∞
!
X
−λt k
= lim 1·e + α · (λt + o(t)) + α P(N(t) = k )
t→0
k =2
= 1 + 0 + lim o(t)
t→0
= 1.

establishing that G(t) is continuous from the right at t = 0.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 23 / 51

Properties XII

Now using Theorem 1 we conclude that the unique solution to this

functional equation is
G(t) = eg(α)t ,
for some constant g(α).
To obtain g(α), we observe that

G(t) − G(0)
g(α) = lim
t→0 t
∞
!
1 P(N(t)=1) 1 X k
= lim [P(N(t)=0)−1] + α + α P(N(t)=k )
t→0 t t t
k =2
∞
!
1 −λt λt + o(t) 1 X k
= lim [e − 1] + α + α P(N(t)=k )
t→0 t t t
k =2
= −λ + αλ.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 24 / 51

Properties XIII

Thus, we find that

G(t) = e−λt+λtα
∞
X e−λt (λt)k
= αk
k!
k =0
∞
X
= P(N(t) = k )αk .
k =0

It follows that, for k ∈ {0, 1, 2, . . .},

e−λt (λt)k
P(N(t) = k ) = .
k!

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 25 / 51

Properties XIV

Exercise 1:-
Prove that the finite dimensional distribution of a Poisson process:

P(N(t1 ) = i1 , N(t2 ) = i2 , . . . , N(tn ) = in )

e−λt1 (λt1 )i1 e−λ(t2 −t1 ) (λ(t2 − t1 ))(i2 −i1 )
= × × ···
i1 ! (i2 − i1 )!
e−λ(tn −tn−1 ) (λ(tn − tn−1 ))(in −in−1 )
× .
(in − in−1 )!

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 26 / 51

Properties XV

Remarks:-
We see that the finite dimensional distributions of a Poisson
process are completely characterised in terms of a single
parameter λ.
We shall see that λ is the rate of the Poisson point process.
It can easily be verified that E(N(t)) = λt, and Var (N(t)) = λt.
Hence, the variance to mean ratio of a Poisson process is 1.
Thus, the Poisson process is often used as a “benchmark" for the
variability in arrival processes.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 27 / 51

Properties XVI

Remarks:-
Suppose we are told that exactly one event of a Poisson process
has taken place by time t, and we are asked to determine the
distribution of the time at which the event occurred.
We can show that, the time of the event is uniformly distributed over
[0, t].

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 28 / 51

Properties XVII
Lemma 4:-
For two time points, s < t,
s
P[T1 < s|N(t) = 1] = ,
t
where T1 denote the time the first event occurs.
Proof:-
P[T1 < s, N(t) = 1]
P[T1 < s|N(t) = 1] =
P[N(t) = 1]
P[1 event in [0, s), 0 events in [s, t]]
=
P[N(t) = 1]
P[1 event in [0, s)]P[0 events in [s, t]]
=
P[N(t) = 1]
λse−λs e−λ(t−s)
=
λte−λt
s
=
t

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 29 / 51

Properties XVIII

Exercise 2:-
Prove that, if N(t) is a Poisson process with rate λ and let
0 = t0 < t1 < t2 < . . . < tn = t.

Then for all k , k1 , k2 , . . . , kn nonnegative integers such that

P n
i=1 ki = k we have

P(N(t1 ) = k1 , N(t2 ) − N(t1 ) = k2 , . . . , N(tn ) − N(tn−1 ) = kn |N(t) = k )

n  k
k! Y ti − ti−1 i
= Qn
i=1 ki !
t
i=1

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 30 / 51

Properties XIX

Remarks:-
Note that the time points ti , 1 ≤ i ≤ n − 1, partition the interval [0, t]
into consecutive subintervals.
In the exercise, the question being asked is that, given that exactly k
Poisson points are known to have occurred in the interval [0, t], what
is the probability that ki of them fell in the i th interval, 1 ≤ i ≤ n.
This result states that conditioned on there being k Poisson arrivals
in an interval [0, t], these k points are distributed over the interval
as if each one of them was independently and uniformly distributed
over the interval.
With this explanation, the form of the right hand side of the
conclusion in the exercise becomes self evident;
it is the multinomial probability distribution with k “trials" and n
alternatives in each trial, the i th alternative being that a point falls in
the i th sub-interval, and the probability of this alternative is
(ti − ti−1 )/t.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 31 / 51

Stopping time for Poisson process I

A random time T is called a stopping time for a Poisson process

N(t), t ≥ 0, if {T ≤ u} q {N(s) − N(u), s ≥ u}.
Remarks:-
Thus a random time T is a stopping time for a Poisson process
N(t) if the question
“Is T ≤ u?" can be answered independently of the increments of N(t)
after the time u.
Unlike the earlier definition of the stopping time, here we do not
require that the event {T ≤ u} be determined by N(t), t ≤ u.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 32 / 51

Stopping time for Poisson process II

Theorem 4:-
Let T be a stopping time for the Poisson process N(t), with
P(T < ∞) = 1, then

N(T + s) − N(T ), s ≥ 0,

is a Poisson process independent of T and of {N(t), t ≤ T }.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 33 / 51

Stopping time for Poisson process III

Remarks:
This theorem asserts that if T is a proper stopping time for the
Poisson process N(t) then
the increments of N(t) starting from T form a Poisson process, and
this process is independent of T as well as of the past of N(t) prior to
T.
This theorem follows from the strong Markov property of continuous
time Markov chains, we see later.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 34 / 51

Stopping time for Poisson process IV

Exercise 3:-
Given a Poisson process N(t), t ≥ 0, show that the following
assertions hold
The jump instants Tk , k ≥ 1, are stopping times.
For each t ≥ 0, T = t is a stopping time.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 35 / 51

Stopping time for Poisson process V

A Corollary of Theorem 4:-

Given a Poisson process N(t) with rate λ, and T a stopping time
with P(T < ∞) = 1,

P(N(T + s) − N(T ) = 0|N(s), s ≤ T ) = P(N(s) = 0) = e−λs .

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 36 / 51

Stopping time for Poisson process VI

Remarks:-
We saw in Exercise 3 that all jump times of a Poisson process are
stopping times.
Also, it follows from this corollary that the successive inter-jump
times are i.i.d. exponential with parameter λ.
Hence, we learn that the Poisson process is a renewal process with
i.i.d. exponential life-times, i.e.,
X1 , X2 , . . . , Xn , . . . ∼ Exp(λ), i.e., pdf :

fXn (x) = λe−λx

This is, in fact, an alternate characterisation of a Poisson process.

It follows Tn = ni=1 Xi ∼ Gamma(n, λ) i.e., pdf :
P

λn x n−1 −λx
fTn (x) = e
Γ(n)

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 37 / 51

Stopping time for Poisson process VII

Exercise 4:-
Suppose that people immigrate into a territory at a Poisson
process with rate λ = 1 per day.
1 What is the expected time until the tenth immigrant arrives?
2 What is the probability that the elapsed time between the tenth and
the eleventh arrival exceeds two days?

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 38 / 51

Stopping time for Poisson process VIII
Solution:-
1

Z ∞
E[T10 ] = xfS10 (x)dx
0
∞
(λx)10−1
Z
= xλe−λx dx
0 (10 − 1)!
= 10/λ
" 10
#
X
= 10 days. = E(Xi )
i=1

Z ∞
P[X11 > 2] = λe−λx dx
2
= e−2λ
= e−2 ≈ 0.133.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 39 / 51

Other Characterisations of Poisson process I

In Definition 2 a Poisson process was defined as a point process

with stationary and independent increments, and with unit jumps,
with probability 1.
In applications it is often useful to have alternate equivalent
characterisations of the Poisson process.
The following are two such characterisations.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 40 / 51

Other Characterisations of Poisson process II

Theorem 5:-
A point process N(t), t ≥ 0, is a Poisson process if and only if
1 for all t0 < t1 < t2 < . . . < tn , the increments

N(ti ) − N(ti−1 ), 1 ≤ i ≤ n,

are independent random variables, and

2 there exists 0 ≤ λ < ∞ such that

(λs)k e−λs
P(N(t + s) − N(t) = k ) = .
k!

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 41 / 51

Other Characterisations of Poisson process III

Proof:-
The “only if" assertion follows since it has already been shown that
the original definition of the Poisson process (i.e., Definition 2)
implies these properties (see Theorem 2).
As for the “if part" of the proof, note that the stationary and
independent increment property follows immediately.
Also the Poisson distribution of the increments implies that the time
between successive jumps is 0 with zero probability, thus
completing the proof.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 42 / 51

Other Characterisations of Poisson process IV

Theorem 6:-
A point process N(t), t ≥ 0, is a Poisson process with parameter λ
if and only if the successive jump times Tk , k ≥ 0, are renewal
instants with i.i.d. exponentially distributed inter-renewal times
with mean λ1 .
Remark: It follows from ERT (Theorem 3.1) that, with probability 1,

lim N(t)/t = λ.
t→∞

Hence λ is the rate of the Poisson point process.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 43 / 51

Other Characterisations of Poisson process V

Proof:
The “only if" part has already been established by the Corollary of
Theorem 4.
For the “if part,": skip the proof

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 44 / 51

Other Characterisations of Poisson process VI

Exercise 5:-
Events occur according to a Poisson process with rate λ = 2 per
hour.
What is the probability that no event occurs between 8 P.M. and 9
P.M.?
Starting at noon, what is the expected time at which the fourth
event occurs?
What is the probability that two or more events occur between 6
P.M. and 8 P.M.?
Exercise 6:-
Customers arrive at a bank at a Poisson rate λ. Suppose two
customers arrived during the first hour. What is the probability that
both arrived during the first 20 minutes?
at least one arrived during the first 20 minutes?

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 45 / 51

Other Characterisations of Poisson process VII

Exercise 7:-
An insurance company pays out claims on its life insurance
policies in accordance with a Poisson process having rate λ = 5
per week. If the amount of money paid on each policy is
exponentially distributed with mean Rs. 20 lakh, what is the mean
and variance of the amount of money paid by the insurance
company in a four-week span?

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 46 / 51

Other Characterisations of Poisson process VIII

Exercise 8:- (Splitting)

Consider a Poisson process N(t) with rate λ, and denote its jump
instants by {T1 , T2 , . . .}. Also, consider an independent Bernoulli
process Zk , k ≥ 1, ; i.e., such that Zk are i.i.d. with

1 w.p. p
Zk =
0 w.p. 1 − p

Now define two new point processes N (1) (t) and N (2) (t) as
follows:
each point Tk , 1 ≤ k ≤ N(t), is a point of N (1) (t) if Zk = 1, else Tk
is a point of N (2) (t).
Show that N (1) (t) and N (2) (t) are two independent Poisson
processes with rates pλ and (1 − p)λ, respectively.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 47 / 51

Other Characterisations of Poisson process IX

Exercise 9:- (Superposition)

Let N (1) (t) and N (2) (t) be two independent Poisson processes
with rates λ1 and λ2 . Define the point process N(t) by

N(t) := N(1)(t) + N(2)(t),

i.e., each point of N (1) (t) and of N (2) (t) is assigned to N(t).
Show that N(t) is a Poisson process with rate λ = λ1 + λ2 .

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 48 / 51

Other Characterisations of Poisson process X

Exercise 10:-
Customers arrive in a Poisson process, N(t), t ≥ 0, with rate λ to
two queues Q1 and Q2. The first customer is assigned to Q1, the
next is assigned to Q2, and so on the customers are assigned
alternately to the two queues.
Show that the arrival process of customers to Q1 and Q2 are
renewal processes, and find their life-time distributions.
Show that these renewal processes are not independent.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 49 / 51

Other Characterisations of Poisson process XI

Exercise 11:-
{N1 (t)} and {N2 (t)} are two independent Poisson processes;
{N1 (t) + N2 (t)} is their superposition. Consider an observer at
(1)
Tn , the nth epoch of {N1 (t)}.
Obtain the distribution of the time until the next epoch of {N2 (t)}.
Obtain the probability that the next epoch of {N1 (t) + N2 (t)} is an
epoch in {N1 (t)}.
Obtain the distribution of the time until the next epoch of
{N1 (t) + N2 (t)}.
Obtain the mean number of epochs of {N1 (t)} that the observer
sees before the next epoch of {N2 (t)}.

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 50 / 51

Other Characterisations of Poisson process XII

Exercise 12:-
There are two types of claims that are made to an insurance
company. Let Ni (t) denote the number of type i claims made by
time t, and suppose that {N1 (t), t ≥ 0} and {N2 (t), t ≥ 0} are
independent Poisson processes with rates λ1 = 10 and λ2 = 1.
The amounts of successive type 1 claims are independent
exponential random variables with mean Rs. 1 lakh whereas the
amounts from type 2 claims are independent exponential random
variables with mean Rs. 5 lakh.
A claim for Rs. 4 lakh has just been received; what is the
probability it is a type 1 claim?

Sudipta Das (RKMVERI) Stochastic Models MCS-PSP-2021 51 / 51