Queuing Theory

Kazi Saeed Alam

Lecturer, Dept. of CSE, KUET
Courtesy: Prof. Dr. Sk. Md. Masudul Ahsan

1
 Queuing System
🞂 Waiting for service is part of our daily life. We queue up for
🞂 Bus, check out counter etc.

🞂 Not only for human, jobs wait to be processed on a machine

🞂 Study of queue quantifies the phenomenon of waiting lines using

🞂 Average queue length, average waiting time, average facility utilization

🞂 Thereby reduce the adverse impact of waiting to tolerable levels

2
 Queuing System
🞂 A queuing system is a facility consisting of one or several servers designed to perform
certain tasks or process certain jobs and a queue of jobs waiting to be processed.

🞂 Examples of queuing systems are:

🞂 a shared computer executing tasks sent by its users;

🞂 an internet service provider whose customers connect to the internet, browse, and disconnect;

🞂 a printer processing jobs sent to it from different computers;

🞂 a medical office serving patients; and so on.

3
 Queuing System
🞂 Main components

4
 Queuing System - Input process
🞂 Usually called the arrival process.
🞂 Arrivals are called customers.
🞂 We assume that no more than one arrival can occur at a given instant.
🞂 If more than one arrival can occur at a given instant, we say that bulk arrivals are
allowed.
🞂 Models in which arrivals are drawn from a small population are called finite source
models.
🞂 If a customer arrives but fails to enter the system, we say that the customer has balked.

5
 Queuing System - Output process
🞂 To describe the output process of a queuing system, we usually specify a probability
distribution – the service time distribution – which governs a customer’s service time.
🞂 Two arrangements of servers:
🞂 Servers are in parallel if all server provide the same type of service and a customer need only
pass through one server to complete service.
🞂 Servers are in series if a customer must pass through several servers before completing service.

6
 Queuing System - Queue discipline
🞂 Describes the method used to determine the order in which customers are served.
🞂 The most common - FCFS discipline (first come, first served), served in order of their
arrival.
🞂 LCFS discipline (last come, first served), the most recent arrivals are the first to enter
service.
🞂 Customer is randomly chosen from waiting line - SIRO discipline (service in random
order).
🞂 Priority queuing disciplines - classifies each arrival into one of several categories.
🞂 Each category is then given a priority level, and within each priority level served on an FCFS
basis.

7
 Modeling Arrival Processes
🞂 We define ti to be the time at which the ith customer arrives.
🞂 we define Ti = ti+1 - ti to be the ith interarrival time.
🞂 In modeling the arrival process we assume that the Ti’s are independent, continuous random
variables described by A having a density function a(t).
🞂 The assumption that each interarrival time is governed by the same random variable implies
that the distribution of arrivals is independent of the time of day or the day of the week.
🞂 This is the assumption of stationary interarrival times.
🞂 Stationary-(whose distribution characteristics do not change over time)

8
Modeling Arrival Processes
🞂 The independence assumption means, for example, that the value of T 2 has no effect on the
value of T3, T4, or any later Ti
🞂 Stationary interarrival ties is often unrealistic, but we may often approximate reality by
breaking the time of day into segments.
🞂 A negative interarrival time is impossible. This allows us to write

🞂 We define λ to be the arrival rate or units of arrivals per hour

🞂 So, 1/λ to be the mean or average interarrival time or units of hours per arrival.

9
 Modeling Arrival Processes
🞂 Lemma 1: If A has an exponential distribution, then for all nonnegative values of t and
h,

(5)

🞂 For example, if h=4, then (5) yields, for t=5, t=3, t=2, and t=0

🞂 For reasons that become apparent, a density that satisfies the equation is said to have the
no-memory property.
🞂 The no-memory property of the exponential distribution is important because it implies
that if we want to know the probability distribution of the time until the next arrival,
then it does not matter how long it has been since the last arrival.
10
 Modeling the Service Process
🞂 We assume that the service times of different customers are independent random variables and that each
customer’s service time is governed by a random variable S having a density function s(t).
🞂 We let 1/µ be then mean service time for a customer.
🞂 so µ is service rate.
🞂 As with interarrival times, we hope that service times can be accurately modeled as exponential random
variables

🞂 Unfortunately, actual service times may not be consistent with the no-memory property.
🞂 For this reason, we often assume that s(t) is an Erlang distribution with shape parameters k and rate
parameter μ.

The Erlang distribution was developed by A. K. Erlang to examine the number

of telephone calls which might be made at the same time to the operators of
the switching stations.
11
 The Kendall-Lee Notation for Queuing Systems
🞂 Standard notation used to describe many queuing systems.
🞂 The notation is used to describe a queuing system in which all arrivals wait in a single
line until one of s identical parallel servers id free. Then the first customer in line enters
service, and so on.
🞂 To describe such a queuing system, Kendall devised the following notation.
🞂 Each queuing system is described by six characters:
1/2/3/4/5/6

13
 The Kendall-Lee Notation for Queuing Systems
🞂 The first characteristic specifies the nature of the arrival process. The following
standard abbreviations are used:
M = Interarrival times are independent, identically distributed (iid) having an exponential
distribution.
D = Interarrival times are iid and deterministic
Ek = Interarrival times are iid Erlangs with shape parameter k.
GI = Interarrival times are iid and governed by some general distribution

14
 The Kendall-Lee Notation for Queuing Systems
🞂 The second characteristic specifies the nature of the service times:
M = Service times are iid and exponentially distributed
D = Service times are iid and deterministic
Ek = Service times are iid Erlangs with shape parameter k.
G = Service times are iid and governed by some general distribution

15
 The Kendall-Lee Notation for Queuing Systems
🞂 The third characteristic is the number of parallel servers.
🞂 The fourth characteristic describes the queue discipline:
🞂 FCFS = First come, first served
🞂 LCFS = Last come, first served
🞂 SIRO = Service in random order
🞂 The fifth characteristic specifies the maximum allowable number of customers in the
system.
🞂 The sixth characteristic gives the size of the population from which customers are
drawn.

16
 The Kendall-Lee Notation for Queuing Systems
🞂 In many important models 4/5/6 is GD/∞/∞. If this is the case, then 4/5/6 is often
omitted.
🞂 M/E2/4/FCFS/10/∞
🞂 might represent a health clinic with 4 doctors,
🞂 exponential interarrival times,
🞂 two-phase Erlang service times,
🞂 an FCFS queue discipline, and
🞂 a total capacity of 10 patients.

17
 Birth-Death Processes

18
 Birth-Death Processes
🞂 We will use it to answer questions about several different types of queuing systems.
🞂 Number of people present in any queuing system at time t to be the state of the queuing
systems at time t.
🞂 We call πj the steady state, or equilibrium probability, of state j.
🞂 The behavior of Pij(t) before the steady state is reached is called the transient behavior of the
queuing system.
🞂 A birth-death process is a continuous-time stochastic process for which the system’s state at
any time is a nonnegative integer.

Pij(t) = transition probability that the system is in state j at time t starting from state i, at t=0

Pij(t) =the probability that j people will be present in the queuing system at time
t, given that at time 0, i people are present
19
Birth-Death Processes
You may think it like DTMC, but better not to These are transition
probability
1-λ1-μ1 1-λ2-μ2 1-λj - μj
1-λ0
λ0 λ1 λ2 λj-1 λj
0 1 2 …. j ….
μ1 μ2 μ3 μj μj+1
These are transition
Sometimes, simplified to the following one rate in BD process

λ0 λ1 λ2 λj-2 λj-1 λj λj+1

0 1 2 …. 0
j-1 j j+1 ….
μ1 μ2 μ3 μj-1 μj μj+1

20
 Laws of Motion for Birth-Death
🞂 Law 1
🞂 With probability λjΔt+o(Δt), a birth occurs between time t and time t+Δt.
🞂 A birth increases the system state by 1, to j+1.
🞂 The variable λj is called the instantaneous birth rate in state j.
🞂 In most queuing systems, a birth is simply an arrival.
🞂 Law 2
🞂 With probability µjΔt+o(Δt), a death occurs between time t and time t + Δt. A death decreases the
system state by 1, to j-1.
🞂 The variable µj is the death rate in state j.
🞂 In most queuing systems, a death is a service completion.
🞂 Note that µ0 = 0 must hold, or a negative state could occur.
🞂 Law 3
🞂 Births and deaths are independent of each other.
21
 Birth-Death Processes
🞂 M/M/1/FCFS/∞/∞ queuing system in which interarrival times are exponential with parameter λ and
service times are exponentially distributed with parameter μ.
🞂 If the state (number of people present) at time t is j, then the no-memory property of the exponential
distribution implies that the probability of a birth during the time interval [t,t+ Δt] will not depend on
how long the system has been in state j.
🞂 Thus, may be determined as if an arrival had just occurred at time t.
🞂 Then the probability of a birth occurring during [t,t+ Δt] is

22
 M/M/1/FCFS/∞/∞ queuing system

23
 M/M/3/FCFS/∞/∞ queuing system
🞂 Interarrival times are exponential with λ = 4 and service times are exponential with μ=5.

• If either interarrival times or service times are nonexponential, then the birth–death
process model is not appropriate.
• Suppose, for example, that service times are not exponential, and we are
considering an M/G/1/FCFS/∞/∞ queuing system.
• Since the service times for an M/G/1/FCFS/∞/∞ system may be nonexponential,
the probability that a death (service completion) occurs between t and t+ Δt will
depend on the time since the last service completion.
• This violates law 2, so we cannot model an M/G/1/FCFS/∞/∞ system as a birth–
death process.
24
Birth-Death Processes
λ0 λ1 λ2 λj-2 λj-1 λj λj+1

0 1 2 …. 0
j-1 j j+1 ….
μ1 μ2 μ3 μj-1 μj μj+1

For a continuous Markov chain which is currently in state i, the probability

that the chain will leave state i and go to state j(j ≠ i) in the next infinitesimal
amount of time ∆t, no matter how long the chain has been in state i, is

Or simply

qij is the instantaneous transition rate of leaving state i for state j

25
Birth-Death Processes – steady state probabilities
λ0 λ1 λ2 λj-2 λj-1 λj λj+1

0 1 2 …. 0
j-1 j j+1 ….
μ1 μ2 μ3 μj-1 μj μj+1

we define πj(t) = P[X(t) = j] the chain in a particular state at time t

probability change in an infinitesimal amount of time Δt

probability that the system will be

in state j - 1 at time t and transit to j at time t +Δt =
in state j + 1 at time t and transit to j at time t +Δt =

in state j at time t and doesn’t leave j at time t +Δt = 26

Birth-Death Processes – steady state probabilities
λ0 λ1 λ2 λj-2 λj-1 λj λj+1

0 1 2 …. 0
j-1 j j+1 ….
μ1 μ2 μ3 μj-1 μj μj+1

dividing by ∆t and taking limits,

For j=0,

27
Steady-State Probabilities for Birth-Death Processes

For t→∞,
called steady state probability

Notice the simplified

notion
For j=0,

🞂 The above equations are often called the flow balance equations, or conservation of flow
equations, for a birth-death process.

= Expected no. of entrances into state j

= Expected no. of departures from state j
28
 Solution of Balance Eq.
🞂 We obtain the flow balance equations for a birth-death process:

λ0 λ1 λ2 λj-2 λj-1 λj λj+1

0 1 2 …. 0
j-1 j j+1 ….
μ1 μ2 μ3 μj-1 μj μj+1

29
Solution of Balance Eq.

The requirement is that

30
 Solution of Balance Eq.
🞂 If is finite, we can solve for π0:

🞂 It can be shown that if is infinite, then no steady-state distribution exists.

🞂 The most common reason for a steady-state failing to exist is that the arrival rate is at
least as large as the maximum rate at which customers can be served.

31
Solution of Balance Eq.

Another way of writing flow balance eqn

32
 Flow balance Eqn of B-D process
🞂 Global eqn

🞂 Local eqn

33
Birth Death Process - Summary
λ0 λ1 λ2 λj-2 λj-1 λj λj+1

0 1 2 …. 0
j-1 j j+1 ….
μ1 μ2 μ3 μj-1 μj μj+1

34
 M/M/1 Queuing System

35
M/M/1/GD/∞/∞ Queuing System
λ λ λ λ λ λ λ

0 1 2 …. 0
j-1 j j+1 ….
μ μ μ μ μ μ

Here, λj = λ j= 0, 1, 2, … Be remember
μ0 = 0 • State represents no.
μj = μ j= 1, 2, … of customer in the q

Let, the traffic intensity be ρ = λ /μ and 0 ≤ ρ < 1

36
M/M/1/GD/∞/∞ Queuing System
λ λ λ λ λ λ λ

0 1 2 …. 0
j-1 j j+1 ….
μ μ μ μ μ μ

If ρ ≥ 1, the infinite sum “blows up”, thus, no steady-state distribution exists.

So, rest of this section, we assume ρ < 1, 37

 M/M/1/GD/∞/∞ Queuing System
🞂 The steady state parameters we are interested in
🞂 L = average number of customers in the queuing system
🞂 Lq = expected no. of customers waiting in the queue
🞂 LS = expected no. of customers in service
= average number of busy server = s′
🞂 L = Lq + LS

38
 M/M/1/GD/∞/∞ Queuing System
🞂 The steady state parameters we are interested in
🞂 W = average time a customer spends in the system
= average response time
🞂 Wq = average time a customer spends in line
🞂 Ws = average time a customer spends in service
🞂 W = Wq +Ws
Little’s Queuing Formula L = λeff W

🞂 For any queuing system in which a steady-state distribution exists, the following relations hold:
L = λW
Lq = λWq
Ls = λWs

🞂 Utilization = s′/s
39
 M/M/1/GD/∞/∞ Queuing System
🞂 Derivation of L
🞂 We know that E(X) = ∑[x f(x)] and P(X=x) =f(x)
🞂 X = no. of customer in the system during steady state
🞂 X can have values j = 0, 1, 2, … with probability πj

40
 M/M/1/GD/∞/∞ Queuing System
🞂 Derivation of L
🞂 Another way

41
 M/M/1/GD/∞/∞ Queuing System
🞂 Derivation of Lq
🞂 Let, the system in state j=0,1,2,… with probability πj
🞂 i.e. there are j customer in the system now
🞂 Since only 1 server, there are j-1 customer is waiting in queue

42
 M/M/1/GD/∞/∞ Queuing System
🞂 Derivation of LS

= s′

If we already know L and Lq

= s′

If we already know L and LS

43
 M/M/1/GD/∞/∞ Queuing System

44
 M/M/1/GD/∞/∞ Queuing System

45
 M/M/1/GD/∞/∞ Queuing System
🞂 Example 4 (Winston)
🞂 Suppose that all car owners fill up when their tanks are exactly half full.
🞂 At the present time, an average of 7.5 customers per hour arrive at a single-pump gas
station.
🞂 It takes an average of 4 minutes to service a car.
🞂 Assume that interarrival and service times are both exponential.
1. What is the probability that the teller is idle?
2. For the present situation, compute L , Lq and W, Wq.

46
 M/M/1/GD/∞/∞ Queuing System
🞂

47
 Solutions
🞂 We have an M/M/1/GD/∞/∞ system with
λ = 7.5 cars per hour µ = 60/4=15 cars per hour.
ρ = 7.5/15 = 0.5
1. Idle probability π0 = 1 − ρ = 0.5

2. L = ρ/(1 − ρ)= 0.5/(1 − 0.5) = 1,

W = L/λ = 1/7.5 = 0.13 hour = 8 mins.

Lq = ρ2/(1- ρ)= 0.25/(1 − 0.5) = 0.5

Wq = Lq/λ = 0.5/7.5 =1/15 hr = 4 mins.

🞂 Hence, in this situation, everything is under control, and long lines appear to be unlikely.
48
 Solutions
3. We now have an M/M/1/GD/∞/∞ system with
λ = 2*7.5 =15 cars per hour
µ = 60*3/10=18 cars per hour.
ρ = 15/18 =5/6

🞂 Thus, panic buying has cause long lines.

49