Информационные процессы, Том 11, № 1, 2011, стр. 114–139.

⃝
c 2011 Efrosinin,Sztrik.
MATHEMATICAL MODELS

Stochastic analysis of a controlled queue

with heterogeneous servers and constant retrial rate1
D. Efrosinin∗ , J. Sztrik∗∗
∗
Peoples’ Friendship University of Russia, Moscow, Russia
Johannes Kepler University, Linz, Austria
∗∗
Debrecen University, Debrecen, Hungary
Received February, 10, 2011

Abstract—In this paper we analyze a controlled retrial queue with several exponential hetero-
geneous servers in which the time between two successive repeated attempts is independent
of the number of customers applying for the service. The customers upon arrival are queued
in the orbit or enters service area according to the control policy. This system is analyzed as
controlled quasi-birth-and-death (QBD) process. It is showed that the optimal control policy
is of threshold and monotone type. We give the explicit formula for the approximation to the
optimal threshold levels and propose value iteration algorithm for the exact calculation of the
levels. The steady-state analysis is performed using matrix-geometric approach. The main per-
formance characteristics are calculated for the system under optimal threshold policy (OTP)
and compared with the same characteristics for the model under scheduling threshold policy
(STP) and other heuristic policies, e.g. the usage of the Fastest Free Server (FFS) or Random
Server Selection (RSS).

1. INTRODUCTION
Different types of multi-server retrial queueing systems under a variety of modifications for sin-
gle and multiple server cases have found applications in local area networks and communication
protocols. For some concrete examples we refer to [3, 5, 21]. The multi-server retrial queues have
been extensively studied for homogeneous servers (equal service intensities) [2, 4, 7] and for hetero-
geneous servers (different service intensities) [13, 20]. The queues with heterogeneous servers are
extensively used for modeling real systems, e.g. the models with servers that have different types of
processors as a consequence of system upgrade, nodes in telecommunication networks with links of
different capacities, nodes in wireless systems serving different mobile users, are assumed to involve
heterogeneous servers.
The system with heterogeneous servers are mostly investigated with respect to the heuristic
service policies, e.g. when the servers must be activated according to the Fastest Free Server (FFS)
or Random Service Selection (RSS) policies. The motivation for considering the controlled queues
comes from the fact that many of the studied systems with heterogeneous servers do not incorporate
facilities (controllers) that allow to pursue different control policies, possibly based upon state
dependent decisions. But if the servers are heterogeneous then the controller may considerably
improve the system’s performance in comparison with mentioned above heuristic policies, e.g. by
reducing the sojourn time (or the number of customers in the system): it is better to wait until
the faster server will be idle then to occupy the slower one. It was shown in [18, 19] that classical
queueing systems under optimal control policy are superior in performance to the homogeneous
ones with the same total service time.
1
Research is supported by the Austro-Hungarian Cooperation (OMAA) Grant No 77öu10, 2010.
 STOCHASTIC ANALYSIS OF A CONTROLLED HETEROGENEOUS QUEUE 115

The queueing systems that assumed to be dynamically controlled may be treated in the frame-
work of Markov decision models [14, 15, 16]. Controlled classical queues with heterogeneous servers
were studied in [17, 18]. It was shown that the dynamic programming value function has specific
monotonicity properties that imply a threshold structure of the optimal control policy which min-
imizes the mean number of customers in the system. It means that for each server there exists
some threshold level that specifies the number of customers in the queue when this server must
be activated. Analogous results as a conjecture were formulated in [6] for the queues with classical
retrial policy and phase-type interarrival and service time distributions. But the given proof for
the monotonicity of increments in exponential case is not complete since it is necessary to prove
some additional convexity properties for the value function similar to the inequalities shown for the
classical queue in [9].
The analysis of controlled queue with constant retrial rate includes the following contributions:
(a) We prove the convex property of the value function that confirms the threshold structure of
the optimal control policy.
(b) Analysis of the boundary to the areas with different optimal threshold levels we obtain the
approximation formula for the explicit calculation of threshold levels.
(c) We model the system by means of a quasi-birth-and-death (QBD) process with policy depen-
dent infinitesimal matrix and demonstrate the application of the general theory of matrix-geometric
solutions [12] for deriving the steady-state distribution and stability conditions for different control
policies.
(d) We calculate the main performance characteristics for the system with different control
policies and study the influence of these policies on the quality of service. It is shown that the
heterogeneous system may be superior in performance to the homogeneous one with the same total
service time.
The paper is organized as follows. In Section 2, we describe the mathematical model and specify
the optimization problem. In Section 3, we investigate the structural properties of the optimal
control policy which appears to be of threshold and monotone type. In Section 4, we apply the results
from the theory of QBD processes to the steady-state analysis of the system under optimal control
policy. The steady-state distributions for the system under other control policies are investigated
in Section 5. In Section 6, we derive the performance measures and in Section 7, we present some
illustrative numerical results for the system under different control policies.
In further sections we will use the notations e, ej and I for the suitably dimensioned column-
vector consisting of 1’s, column vector with 1 in the j-th (beginning from 0-th) position and 0
elsewhere, and an identity matrix, respectively.

2. DESCRIPTION OF THE MATHEMATICAL MODEL

Consider the queueing model M/M/c in which primary customers arrive according to a Poisson
stream with rate λ, c heterogeneous exponential servers with rates µ1 > µ2 > · · · > µc , constant
retrial rate γ > 0 and the number of places in the retrial orbit 2 ≤ K ≤ ∞. According to the control
policy an arriving customer can join the orbit or have direct access to the accessible idle servers.
The arrival process, service times and retrial times are assumed to be mutually independent.
Let Q(t) is the number of customers in the retrial orbit at time t, Dj (t), 1 ≤ j ≤ c describe the
states of the servers at this time,
{
0, if the j-th server is idle at time t and
Dj (t) =
1, if the j-th server is busy.

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

116 EFROSININ,SZTRIK

The observed process

{X(t)}t≥0 = {Q(t), D1 (t), . . . , Dc (t)}t≥0 (1)

is a continuous-time Markov process with state space defined as

E = {x = (q, d1 , . . . , dc ); 0 ≤ q ≤ K, dj = {0, 1}, 1 ≤ j ≤ c} ≡ N × {0, 1}c ,

where q and dj , 1 ≤ j ≤ c denote, respectively, the number of customers on the orbit and states of
the servers and q(x), dj (x) specify the components of the state x. Let J0 (x) and J1 (x) be the sets
of labels assigned to idle and busy servers, respectively, i.e.

J0 (x) = {j : dj (x) = 0}, J1 (x) = {j : dj (x) = 1}.

The control decision in this model are the choice of assigning or not assigning a server whenever
primary arrival or retrial customer tries to enter a service area. Define by U = {0, 1, . . . , c} a set of
admissible controls and by U (x) ⊆ U
{
J0 (x) ∪ {0}, for x if q(x) < K,
U (x) =
J0 (x), for x if q(x) = K < ∞.

a set of admissible controls in state x. Thus each time t− just before a transition upon arrival
(primary or retrial) when the system state is X(t− ) = x one of available in this state control
actions u ∈ U (x) must be chosen, where u = 0 has the meaning ”send the job to the orbit”and
u = k ≥ 1 has the meaning ”switch on server k”.
A deterministic control policy is then meant by any function f : E → U subject to the constraint
f (x) ∈ U (x) for any x ∈ E. When f is adopted as a policy, u = f (x) is applied whenever the system
is in state x.
The components axy (u) of the infinitesimal generator of the Markov process {X(t)}t≥0 depend
on the control action u and satisfies


 λ, for y = x + eu , u ∈ U (x)


µ , for y = x − e , j ∈ J (x),
j j 1
axy (u) =

 γ, for y = x − e0 + eu , u ∈ U (x − e0 )


0, otherwise
∑
and ax (u) = −axx (u) = y̸=x axy (u).
For the model under consideration we look for some deterministic control policy f ∗ which min-
imizes the long-run average sojourn time. This is equivalent to minimizing the long-run average
number of customers in the system
[∫ T ]
1 f
g(f ) = lim Ex (Q(t) + D1 (t) + D2 (t))dt , (2)
T →∞ T 0

where Efx denotes the expectation with respect to the probability measure induced by the policy f
on the process {X(t)} with initial state x. The policy f ∗ is said to be optimal if

g = g(f ∗ ) ≤ g(f ), x ∈ E

for any other admissible stationary deterministic policy f .

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

 STOCHASTIC ANALYSIS OF A CONTROLLED HETEROGENEOUS QUEUE 117

3. ON OPTIMAL CONTROL POLICY

3.1. Optimality of always using faster server
Our objective is to prove that the faster server upon arrival of a primary or retrial customer
must be switched on whenever it is idle. To prove it we need the following property of the dynamic
programming operator B
[ ( ∑ )]
1
Bv(x) = min t
x e+ axy (u)v(y) − g , x ∈ E (3)
u ax (u)
y̸=x

mapping any positive function v : E → R+ into a positive function.

Lemma 1. Assume the function v : E → R+ has the following properties

1. v(x) ≤ v(x + ei ), i ∈ J0 (x),

2. v(x) ≤ v(x + e0 ),
3. v(x + ej ) ≤ v(x + ei ), i, j ∈ J0 (x), µj ≥ µi ,
4. v(x + e1 ) ≤ v(x + e0 ).

Then the operator B defined by (3) preserves these properties for the function v.

Proof. To prove the properties of lemma 1 we rewrite the equation (3) in the form
(
1
Bv(x) = ∑ xt e + λ min v(x + eu ) + 1{q(x)>0} γ min v(x − e0 + eu ) (4)
λ + γ + cj=1 µj u∈U (x) u∈U (x−e0 )
∑ ∑ )
+ 1{q(x)=0} γv(x) + µj v(x − ej ) + µj v(x) − g .
j∈J1 (x) j∈J0 (x)
∑c
without loss of generality assuming that the rate λ + γ + j=1 µj = 1 For the inequality 1 taking
the last equation for the states x + ei and x we get

Bv(x + ei ) − Bv(x)
= 1 + λ[ min {v(x + ei + eu )} − min {v(x + eu )}]
u∈U (x+ei ) u∈U (x)

+ 1{q(x)>0} γ[ min {v(x + ei − e0 + eu )} − min {v(x − e0 + eu )}]

u∈U (x+ei −e0 ) u∈U (x−e0 )

+ 1{q(x)=0} γ[v(x + ei ) − v(x)]

∑ ∑
+ µl [v(x + ei − el ) − v(x − el )] + µl [v(x + ei ) − v(x)] ≥ 0.
l∈J1 (x+ei ) l∈J0 (x+ei )

It can be easily seen that all terms in the right hand side of the equation are nonnegative due to
the property of the function v(x): v(x) ≤ v(x + ei ). Indeed we have

min {v(x + ei + eu )} ≥ min {v(x + eu )} ≥ min {v(x + eu )}

u∈U (x+ei ) u∈U (x+ei ) u∈U (x)

min {v(x + ei − e0 + eu )} ≥ min {v(x + eu − e0 )} ≥ min {v(x + eu − e0 )}

u∈U (x+ei −e0 ) u∈U (x+ei −e0 ) u∈U (x−e0 )

where the inequalities follow from the monotonicity assumption of the function v and from the
relations U (x + ei ) ⊂ U (x) and U (x + ei − e0 ) ⊂ U (x − e0 ) together with the fact that the minimum
does not increase upon expanding the minimization set.

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

118 EFROSININ,SZTRIK

The inequality 2 can be shown similarly. Indeed,

Bv(x + e0 ) − Bv(x)
= 1 + λ[ min {v(x + e0 + eu )} − min {v(x + eu )}]
u∈U (x+e0 ) u∈U (x)

+ 1{q(x)>0} γ[ min {v(x + eu )} − min {v(x − e0 + eu )}]

u∈U (x) u∈U (x−e0 )

+ 1{q(x)=0} γ[ min {v(x + eu )} − v(x)]

u∈U (x)
∑ ∑
+ µl [v(x + e0 − el ) − v(x − el )] + µl [v(x + e0 ) − v(x)] ≥ 0,
l∈J1 (x) l∈J0 (x)

due to the monotonicity properties of the function v.

For the inequality 3 we take the equation (4) for the states x + ei and x + ej

Bv(x + ei ) − Bv(x + ej )
= λ[ min {v(x + ei + eu )} − min {v(x + ej + eu )}]
u∈U (x+ei ) u∈U (x+ej )

+ 1{q(x)>0} γ[ min {v(x + ei − e0 + eu )} − min {v(x + ej − e0 + eu )}]

u∈U (x+ei −e0 ) u∈U (x+ej −e0 )

+ 1{q(x)=0} γ[v(x + ei ) − v(x + ej )] + µj [v(x + ei ) − v(x)] − µi [v(x + ej ) − v(x)]

∑ ∑
+ µl [v(x + ei − el ) − v(x + ej − el )] + µl [v(x + ei ) − v(x + ej )] ≥ 0.
l∈J1 (x+ei +ej ) l∈J0 (x+ei +ej )

The last expression is nonnegative if µj ≥ µi since

min {v(x + ei + eu )} = min {min{v(x + ei + ej ), v(x + ei + eu )}} ≥

u∈U (x+ei ) u∈U (x+ei +ej )

min {min{v(x + ei + ej ), v(x + ej + eu )}} = min {v(x + ej + eu )}

u∈U (x+ei +ej ) u∈U (x+ej )

by virtue of inequality 3 for the function v. The same is true for the term with coefficient γ if we
take the inequality 3 for the x − e0 .
At last for the inequality 4 we have

Bv(x + e0 ) − Bv(x + e1 )
= λ[ min {v(x + e0 + eu )} − min {v(x + e1 + eu )}]
u∈U (x+e0 ) u∈U (x+e1 )

+ 1{q(x)>0} γ[ min {v(x + eu )} − min {v(x + e1 − e0 + eu )}]

u∈U (x) u∈U (x+e1 −e0 )

+ 1{q(x)=0} γ[ min v(x + eu ) − v(x + e1 )] + µ1 [v(x + e0 ) − v(x)]

u∈U (x)
∑ ∑
+ µl [v(x + e0 − el ) − v(x + e1 − el )] + µl [v(x + e0 ) − v(x + e1 )] ≥ 0.
l∈J1 (x+e1 ) l∈J0 (x+e1 )

by virtue of the inequalities 3 and 4.

Theorem 1. The optimal policy has the following property: Whenever upon arrival (primary or
retrial) the fast server is available, this server should serve a customer.

Proof. As in Lin and Kumar [10] the proof combines the value iteration algorithm with lemma
1. The value function v(x) satisfies

Bv(x) = v(x), x ∈ E.

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

 STOCHASTIC ANALYSIS OF A CONTROLLED HETEROGENEOUS QUEUE 119

Let
[ ( ∑ )]
1
vn+1 (x) = min t
x e+ axy (u)vn (y) − gn = Bvn (x), x ∈ E.
u ax (u) y

In spite of the operator B not being a contractive operator for the long-run average problem, if
we put the starting iteration v0 (x) = 0, x ∈ E, then the function vn is a monotonously increasing
function so that
v(x) = lim B n v0 (x), x ∈ E.
n→∞

Note that the initial choice of v0 satisfies the assumption of lemma 1. Hence all vn as well as the
limit have this property, therefore it holds also for the value function v ∗ (x).
The structure of the equation (4) allows to express the optimal policy f (x) in each state x ∈ E
through the value function v ∗ (x) in the form

f (x) = arg min{v(x + eu )}, x ∈ E. (5)

u∈U (x)

Therefore the the assumption of lemma 1 for the value function implies that if it is necessary to
switch on some server it must be the fastest available one.

3.2. Optimality of threshold policy for c = 2

We have shown that the first server must be activated whenever the customer arrives and it
is free. Note that the values v(x + e2 ) and v(x + e0 ) for d1 (x) = 1 can not be compared in the
same way as before, this assertion does not allow us to determine whether a second server must be
activated. But we can show that the increments v(x + e0 ) − v(x + e2 ) are monotone with respect
to the queue length.

Lemma 2. Assume the positive function v : E → R+ has the following property

3. v(x + e0 ) − v(x + e2 ) ≤ v(x + 2e0 ) − v(x + e0 + e2 ),

4. v(x + e0 ) − v(x) ≤ v(x + e2 + e0 ) − v(x + e2 ),
5. 2v(x + e0 ) ≤ v(x) + v(x + 2e0 ).

Then the operator B defined by (3) preserves these properties for the function v.

Proof. The inequality 3 of the function v is known as superconvexity, the inequality 2 - as

supermodularity and the inequality 3 - as convexity. Note, that the summing 3 and 4 implies 5.
We have to show that the operator B preserves the mentioned properties. To prove the inequality
3 we substitute the equation (4) in corresponding point, then we have

Bv(x + e0 ) − Bv(x + e2 ) − Bv(x + 2e0 ) + Bv(x + e0 + e2 )

= λ[ min {v(x + e0 + eu )} − min {v(x + e2 + eu )}
u∈U (x+e0 ) u∈U (x+e2 )

− min {v(x + 2e0 + eu )} + min {v(x + e0 + e2 + eu )}]

u∈U (x+2e0 ) u∈U (x+e0 +e2 )

+ 1{q(x)>0} γ[ min {v(x + eu )} − min {v(x − e0 + e2 + eu )}

u∈U (x) u∈U (x+e2 −e0 )

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

120 EFROSININ,SZTRIK

− min {v(x + e0 + eu )} + min {v(x + e2 + eu )}]

u∈U (x+e0 ) u∈U (x+e2 )

+ 1{q(x)=0} γ[ min {v(x + eu )} − v(x + e2 )

u∈U (x)

− min {v(x + e0 + eu )} + min {v(x + e2 + eu )}]

u∈U (x+e0 ) u∈U (x+e2 )

+ µ1 [v(x + e0 − e1 ) − v(x + e2 − e1 ) − v(x + 2e0 − e1 ) + v(x + e0 + e2 − e1 )]

+ µ2 [v(x + e0 ) − v(x) − v(x + 2e0 ) + v(x + e0 )] ≤ 0

The last two terms with the coefficients µ1 and µ2 are nonpositive with respect to the inequalities
3 and 5 of lemma 2. To prove the assertion that the term with coefficient λ is nonpositive we
distinguish two possible cases
1. f (x + e2 ) = f (x + 2e0 ) = 0. In this case we have

min {v(x + e0 + eu )} − v(x + e2 + e0 ) − v(x + 3e0 ) + min {v(x + e0 + e2 + eu )}

u∈U (x+e0 ) u∈U (x+e0 +e2 )

≤ v(x + 2e0 ) − v(x + e2 + e0 ) − v(x + 3e0 ) + v(x + 2e0 + e2 ) ≤ 0

according to the assumption of lemma 2 in the state x + e0 .

2. f (x + e2 ) = 0 and f (x + 2e0 ) = 2. In this case we get

min {v(x + e0 + eu )} − v(x + e2 + e0 ) − v(x + 2e0 + e2 ) + min {v(x + e0 + e2 + eu )}

u∈U (x+e0 ) u∈U (x+e0 +e2 )

≤ v(x + e0 + e2 ) − v(x + e2 + e0 ) − v(x + 2e0 + e2 ) + v(x + e0 + e2 + e0 ) = 0

For the term with coefficient 1{q(x)>0} γ the nonpositivity follows from the previous cases by
replacing x with x − e0 . Inequality of the term with coefficient 1{q(x)=0} γ can be shown similarly
by considering two cases f (x + e0 ) = 0 and f (x + e0 ) = 2.
For the inequality 4 we have

Bv(x + e0 ) − Bv(x) − Bv(x + e2 + e0 ) + Bv(x + e2 )

= λ[ min {v(x + e0 + eu )} − min {v(x + eu )}
u∈U (x+e0 ) u∈U (x)

− min {v(x + e2 + e0 + eu )} + min {v(x + e2 + eu )}]

u∈U (x+e2 +e0 ) u∈U (x+e2 )

+ 1{q(x)>0} γ[ min {v(x + eu )} − min {v(x − e0 + eu )}

u∈U (x) u∈U (x−e0 )

− min {v(x + e2 + eu )} + min {v(x + e2 − e0 + eu )}]

u∈U (x+e2 ) u∈U (x+e2 −e0 )

+ 1{q(x)=0} γ[ min {v(x + eu )} − v(x)

u∈U (x)

− min {v(x + e2 + eu )} + v(x + e2 )]

u∈U (x+e2 )

+ µ1 [v(x + e0 − e1 ) − v(x − e1 ) − v(x + e2 + e0 − e1 ) + v(x + e2 − e1 )]

+ µ2 [v(x + e0 ) − v(x) − v(x + e0 ) + v(x)] ≤ 0

The last two terms with coefficients µ1 and µ2 are nonpositive with respect to the inequality 3.
For the term with λ we have two subcases, namely
1. f (x) = f (x + e2 + e0 ) = 0.

min {v(x + e0 + eu )} − v(x + e0 ) − v(x + 2e0 + e2 ) + min {v(x + e2 + eu )}

u∈U (x+e0 ) u∈U (x+e2 )

≤ v(x + 2e0 ) − v(x + e0 ) − v(x + 2e0 + e2 ) + v(x + e2 + eu ) ≤ 0

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

 STOCHASTIC ANALYSIS OF A CONTROLLED HETEROGENEOUS QUEUE 121

according to the assumption of lemma 2 in the state x + e0 .

2. f (x) = 2 and f (x + e2 + e0 ) = 0.

min {v(x + e0 + eu )} − v(x + e2 ) − v(x + 2e0 + e2 ) + min {v(x + e2 + eu )}

u∈U (x+e0 ) u∈U (x+e2 )

≤ 2v(x + e0 + e2 ) − v(x + e2 ) − v(x + 2e0 + e2 ) ≤ 0

using the inequality 5.

The term with the coefficient 1{q(x)>0} γ is nonpositive by applying the inequality 3 to the point
x − e0 . At last the term with the coefficient 1{q(x)=0} γ is nonnegative due to the relation

min {v(x + eu )} − v(x) − min {v(x + e2 + eu )} + v(x + e2 )

u∈U (x) u∈U (x+e2 )

≤ v(x + e0 ) − v(x) − v(x + e2 + e0 ) + v(x + e2 ) ≤ 0.

Theorem 2. The optimal policy for the retrial queueing system M/M/2 with heterogeneous
servers and constant retrial rate is of threshold and monotone type, i.e. the fastest idle server must
be switched on whenever a primary or retrial customer arrives and another one must be switched
on if and only if the fastest server is busy and the orbit length reaches the threshold level q ≥ q2∗ .

Proof. The assumption of lemma 2 holds for the value function v according to the monotone
convergence property of the operator B, i.e.

v(x + e0 ) − v(x + e2 ) ≤ v(x + 2e0 ) − v(x + e0 + e2 ), x = (q, 1, 0), q ≥ 0.

This inequality shows that the advantage to send the customer to the orbit and leave the second
server idle decreases when the orbit size increases. According to the definition of the optimal control
f (x), x ∈ E in (5) the following relations hold

f (x + e0 ) = 0 ⇒ f (x) = 0
f (x) = 2 ⇒ f (x + e0 ) = 2.

Hence there exists a level of the orbit size q2∗ when upon arrival the second server must be activated.
And if in some state x the optimal control is to allocate a customer to the orbit then this control
is optimal for all the states y.

3.3. Calculation of threshold levels

The analytical representation of the threshold level is quite complicated, but by means of the
value iteration algorithm it can be done numerically. If future arrivals are not taken into account
(scheduling problem), i.e. when the objective is to minimize the sojourn time for an individual
customer, the explicit solution for the threshold level exists

Theorem 3 (Scheduling threshold policy). If λ = 0 then there exists a threshold level

⌊ ( )⌋
γ µ1
q2∗ = −1 , (6)
µ1 + γ µ2

such that if q ≥ q2∗ in state x = (q, 1, 0) then upon retrial arrival it is optimal to dispatch a customer
to the slower server, if q > q2∗ then the slower server must be idle.

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

122 EFROSININ,SZTRIK

Proof. Consider the system with customers without future arrival, i.e. λ = 0. For this system it
is obviously that the mean number of customers g = 0. Therefore the relative function v satisfies
the recurrent equations for q ≥ 1 which after some transformations have the form
µ1 + µ2 1
v(0, 1, 1) = , v(0, 0, 1) = , (7)
µ1 µ2 µ2
( )
q q (q − 1)q 1 1
v(q − 1, 1, 0) = + v(q − 1, 0, 0) = + + ,
µ1 µ1 2 γ µ1
1
v(q − 1, 1, 1) = (q + 1 + µ1 v(q − 1, 0, 1) + µ2 v(q − 1, 1, 0))
µ1 + µ2
1
= + v(q − 1, 1, 0).
µ2
Since q = q2∗ is an optimal orbit size to switch on the second server, it is necessary to require the
following condition: The value function in the state x = (q − 1, 1, 1), in which the controller begins
to use the second server with respect to the optimal policy must be less then the value function in
the state, in which the controller does not use the second server, i.e.
v(q − 1, 1, 1) < v(q, 1, 0).
Substituting the corresponding expressions (7) into the last inequality we get
( ) ( )
1 q (q − 1)q 1 1 q + 1 q(q + 1) 1 1
+ + + < + + .
µ2 µ1 2 γ µ1 µ1 2 γ µ1
Thus the second server must be switched on if
⌊ ( )⌋
γ µ1
q> −1 .
µ1 + γ µ2
Further we obtain an approximation to the optimal threshold level q2∗ in general case with arrivals.
This approximation can be obtained by analysis of the asymptote to the boundary between the
areas where the optimal threshold value is q2∗ and q2∗ + 1. Note that the term µµ12 − 1 represents
the threshold level for the scheduling problem in case of classical queue, see e.g. [1] and can be
interpreted as the average number of customers that can be served by the first server while the
second one is busy. The relation (6) has additional term µ1γ+γ that denotes the probability of the
retrial arrival in the state with busy faster server. Now we can provide the following assertion.
Assertion 1. The approximation to the threshold level of the retrial system with arrivals is of the
form
⌊ ⌋
∗ ∗ γ ∗
q2 ≈ q̃2 = q̃ , (8)
µ1 + γ 2 c
⌊ √ ⌋
∗ µ1 − λ + (µ1 − λ)2 + 4µ2 λ
q̃2 c = −1 , (9)
2µ2
where q̃2∗ c is an approximation to the threshold level of the ordinary queue.

Proof. Consider an ordinary queueing system without retrials. To calculate the value q2∗ c we
display the optimal thresholds as a function of µ1 /λ and µ2 /λ and get semi-infinite areas with
boundaries that can be approximated by the lines
λ/µ1 q∗
λ/µ2 = ∗ − ∗ 2 2.
q2 + 1 (q2 + 1)
The latter expression implies the relation (9).

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

 STOCHASTIC ANALYSIS OF A CONTROLLED HETEROGENEOUS QUEUE 123
(a) (b)
Μ1 Μ1
Λ 10 Λ 10
20 20
30 30
40 40

40 40

30 30
Γ Γ
Λ Λ
20 20

10 10

5 5
10 10
Μ2 15 Μ2 15
Λ Λ

Figure 1. Areas with optimal threshold level q2∗ = 1, 2, 3, 4 and corresponding boundary surfaces

This relation coincides with the result obtained in [11] for the analogous model with probabilistic
decision rule. The figure 1 shows that the obtained asymptotic surfaces quite good approximate
the actual boundaries. Numerical results show that the difference between performance values g
for the approximation and optimal threshold level never exceeds 1.5 percent.

Assertion 2. In heavy traffic case, λ ≈ µ1 + µ2 , we can assume the suboptimality of the Fastest
Free Server (FFS) discipline.

Proof. It is also confirmed by the approximation formula (8). Indeed, taking into account that
µ1 ≥ µ2 we get,
⌊ √ ⌋ ⌊√ ( ) ( ) ⌋
µ1 − (µ1 + µ2 ) + (µ1 − (µ1 + µ2 ))2 + 4µ2 (µ1 + µ2 ) µ2 2 µ2 µ2
−1 ≤ + 1+ − 1 < 1.
2µ1 2µ1 µ1 µ1

Assertion 3. In light traffic case, λ < µ2 , we can assume the suboptimality of the scheduling
threshold policy (STP).

Proof. Under the given condition we get the following upper and low bounds for the approxi-
mation (9)
v
⌊ √ ⌋ ⌊ u( (
u 1 µ ) √ )2 ⌋
µ1 − λ + (µ1 − λ) + 4µ2 λ
2 µ1 λ t 1 λ λ
−1 ≤ − + − + −1
2µ2 2µ2 2µ2 2 µ2 µ2 µ2
⌊ √ ⌋ ⌊ ⌋
µ1 λ λ µ1
= − + −1 ≤ ,
µ2 µ2 µ2 µ2
⌊ √ ⌋ ⌊ √ ( ) ⌋
µ1 − λ + (µ1 − λ)2 + 4µ2 λ µ1 λ 1 µ1 λ 2
−1 ≥ − + − −1
2µ2 2µ2 2µ2 4 µ2 µ2
⌊ ⌋ ⌊ ⌋
µ1 λ µ1
= − −1 ≥ −2 .
µ2 µ2 µ2

Numerical investigation of the systems with more than two servers leads to the following conjecture
about the approximation to the threshold levels.

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

124 EFROSININ,SZTRIK

Assertion 4 (Approximation to the threshold levels). Using the asymptote to the boundary be-
tween the area where the optimal threshold for the j-th server is qj∗ and the area with optimal
threshold qj∗ + 1, the following approximations to the optimal threshold levels can be calculated by
v
⌊ ( [∑ u( j−1 )2 ] )⌋
γ 1
j−1 u ∑
qj∗ ≈ q̃j∗ = ∑j−1 µk − λ + t µk − λ + 4(j − 1)µj λ − (j − 1) . (10)
γ+ k=1 µj
2µj
k=1 k=1

In case of more than two servers the exact values of optimal thresholds can be calculated numerically.
The system with infinite retrial group will be approximated by the system with finite but enough
large orbit capacity. We have used a C++ program implementing a policy iteration algorithm:

Algorithm 1.
Step 1. Selection of initial control policy, e.g. FFS
{ }
1
f0 (x) = arg min , x ∈ E.
u∈J0 (x) µu

Step 2. For the given control policy fi (x) solve the system of equations iteratively

v0i (x) = 0, x ∈ E, gni = λvni (ef i (0) )

( ∑
1
i
vn+1 (x) = ∑ xt e + λvni (x + efi (0) ) + µj vni (x − ej )
λ + j∈J1 (x) µj + 1{q(x)>0} γ
j∈J1 (x)
)
+ 1{q(x)>0} γvni (x − e0 + efi (x−e0 ) ) − gni , x ∈ E.

Step 3. For the given solution of previous system of equations v i (x) find the improved policy by

fi+1 (x) = arg min{v i (x + eu )}, x ∈ E

u∈U (x)

Step 4. Repeat Step 1 and Step 2 until the policies for two neighbor iterations will be equal for all
states, i.e.
fi (x) = fi+1 (x) = f ∗ (x) = arg min{v ∗ (x + eu )}, x ∈ E.
u∈U (x)

Step 5. Calculate the optimal threshold level q2∗ by

q2∗ = q + 1 if f ∗ (q − 1, 1, 0) = 0 and f ∗ (q, 1, 0) = 2.

4. STEADY-STATE DISTRIBUTION OF THE SYSTEM UNDER OPTIMAL POLICY

In this section we derive the equilibrium state distribution under the optimal threshold policy
(OTP). The derivation works via a standard matrix-geometric approach [12], taking into account
the special structure of the boundary states where not all servers are active. To distinguish the
system under OTP from other control policies which will be discussed later we will use the upper
index ′ (1)′ for the concerned values. Let q2∗ be the threshold level for activation of the second
server. As it was mentioned above they can be found numerically (e.g. using the policy iteration
algorithm [8]).
Consider a Markov process {X(t)}t≥0 defined by (1) with a state space E. This process is a
QBD process with block - three-diagonal infinitesimal matrix. Note that the blocks have different
sizes depending on the queue length.

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

 STOCHASTIC ANALYSIS OF A CONTROLLED HETEROGENEOUS QUEUE 125

4.1. Finite retrial group

First consider the system with finite retrial group, i.e. K < ∞. In this case the states are
partitioned as follows:

– block 0 includes the single state: (0, 0, 0);

– block 1 includes the states: (0, 0, 1), (0, 1, 0), (1, 0, 0);
– block 2 includes the states: (0, 1, 1), (1, 0, 1), (1, 1, 0), (2, 0, 0);
– blocks i, 3 ≤ i ≤ K include the states:
(i − 2, 1, 1), (i − 1, 0, 1), (i − 1, 1, 0), (i, 0, 0);
– block K + 1 includes the states:
(K − 1, 1, 1), (K, 0, 1), (K, 1, 0);
– block K + 2 includes the single state: (K, 1, 1);

Denote by Λ(1) the infinitesimal matrix of dimension 4(K + 1) for the system under OTP,
Λ(1) = (11)
 (1) 
−λ A1 0 0 0 0 ... 0
 (1) (1) (1) (1) 
 D0 −(C1 − B1 ) A2 0 0 0 ... 0 
 
 0 D1
(1) (1)
−(C2 − B2 )
(1)
A3
(1)
0 0 ... 0 
 
 0 (1) (1)
−(C2 − B2 )
(1) (1)
0 
 0 D2 A3 0 ... 
 
 .. .. .. .. .. 
 . . . . . ... 0 
 ,
 0 (1) (1)
−(C3 − B3 )
(1) (1)
0 
 ... 0 D2 A4 ... 
 
 .. .. .. .. .. .. .. .. 
 . . . . . . . . 
 
 (1) (1) (1) (1) 
 0 ... 0 0 D3 −(C3 − B3 ) A5 0 
 (1) 
 0 ... 0 0 0 D3
(1)
−(C4 − B4 ) A6 
(1) (1)
(1)
0 ... 0 0 0 0 D4 −M

where
(1) (1) (1) (1) (1) (1) (1) (1) (1)
λ + A1 e = D0 e − (C1 − B1 )e + A2 e = D1 e − (C2 − B2 )e + A3 e =
(1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1)
D2 e − (C2 − B2 )e + A3 e = (D2 − (C3 − B3 ) + A4 )e = D3 e − (C3 − B3 )e + A5 e =
(1) (1) (1) (1) (1)
D3 e − (C4 − B4 )e + A6 e = D4 e − M = 0,
(1) (1)
M = µ1 + µ2 . Matrices Ai and Bi represent primary and retrial arrivals, respectively, depending
on whether the queue length are above or below threshold level:
     
  λ 0 00 λ000 λ00
λ 0 0 0 λ
( ) 0 0 0    
 , A(1) =  λ 0 0 0  , A(1) =  λ 0 0  ,
= 0 λ 0 , A 2 =  0 0 λ 0  , A3 = 
(1) (1) (1)
A1 0 0 λ 0 4 λ 0 0 0 5 λ 0 0
00λ0
0 0 λ0 00λ0 00λ
   
    000 0 0000  
λ 000 γ 0 0 000
0  
 , B (1) =  γ 0 0 0  , B (1) =  γ 0 0  .
=  λ  , B1 =  0 0 0  , B 2 = 
(1) (1) (1)
A6 0 0 0 0 3 γ 0 0 0 4
λ 0γ 0 γ 00
00γ 0 00γ 0

(1)
Matrices Ci represent cases when the system stays at certain states:
 
  λ+M 0 0 0
λ + µ2 0 0  0 λ + µ2 + γ 0 0 
= 0 λ + µ 1 0  , C2 =  ,
(1) (1)
C1  0 0 λ + µ1 0 
0 0 λ+γ
0 0 0 λ+γ
 
λ+M 0 0 0  
 0 λ+M 0 0
 λ + µ2 + γ 0 0  , C4 =  0 .
(1) (1)
C3 = λ + µ2 + γ 0
0 0 λ + µ1 + γ 0 
0 0 λ + µ1 + γ
0 0 0 λ+γ

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

126 EFROSININ,SZTRIK
(1)
Matrices Di represent departures with elements depending on active servers:
   
  µ1 µ2 0 0 µ1 µ2 0  
µ2  0 0 µ1 µ2 0
0 µ2  
 , D(1) =  0 0 0 µ2  ( )
=  µ1  , D1 =  , D  0 0 0 µ2  , D(1) = 0 µ1 µ2 .
(1) (1) (1)
D0  0 3 =
0 µ1  2 0 0 0 µ1  4
0 0 0 0 µ1
0 0 0 0 0 0 0

(1) (1)
Denote by π (1) = (π0 , π1 , . . . ) the row-vector of the steady-state probabilities,
(1)
π (1) = {πx(1) = π(q,d1 ,d2 ) : x ∈ E} = lim P{X(t) = x},
t→∞
(1)
by {πk: k ≥ 0} — the subvectors that specify the states with k jobs in the system.
Obviously the row-vector π (1) of the steady-state probabilities of the system under optimal policy
satisfies the equations
π (1) Λ = 0, π (1) e = 1. (12)
The probabilities for the system states can be represented in the form of recursive relation with
(1)
some matrices Mk ,
(1) (1) (1)
πk = πk+1 Mk , k = 0, 1, . . . , K + 1,
namely
(1) (1) (1) 1 (1) (1)
π0 = π1 D0 = π1 M0 ;
λ
(1) 1
A )−1 = π2 M1 ;
(1) (1) (1) (1) (1) (1) (1) (1)
π1 = π2 D1 (C1 − B1 − D0
λ 1
= π3 D2 (C2 − B2 − M1 A2 )−1 = π3 M2 ;
(1) (1) (1) (1) (1) (1) (1) (1) (1)
π2
π3 = π4 D2 (C2 − B2 − M2 A3 )−1 = π4 M3 ;
(1) (1) (1) (1) (1) (1) (1) (1) (1)

= πi+1 D2 (C2 − B2 − Mi−1 A3 )−1 = πi+1 Mi , 4 ≤ i ≤ q2∗ ;

(1) (1) (1) (1) (1) (1) (1) (1) (1)
πi
= πi+1 D2 (C3 − B3 − Mi−1 A4 )−1 = πi+1 Mi , q2∗ + 1 ≤ i ≤ K − 1;
(1) (1) (1) (1) (1) (1) (1) (1) (1)
πi
πK = πK+1 D3 (C3 − B3 − MK−1 A4 )−1 = πK+1 MK ;
(1) (1) (1) (1) (1) (1) (1) (1) (1)

πK+1 = πK+2 D4 (C4 − B4 − MK A5 )−1 = πK+2 MK+1 ,

(1) (1) (1) (1) (1) (1) (1) (1) (1)
(13)
where all inverses are well defined because the matrices involved are clearly non-singular using their
probabilistic interpretation as transition matrices of a transient chain.
(1)
To calculate the values πk it is necessary:

Algorithm 2.
(1)
Step 1. Evaluate the matrices Mk , k = 0, . . . , K + 1 according to relation (13).
(1) (1)
Step 2 Evaluate the value πK+2 = π(K,1,1) from the normalization condition
 
∑ (1)
K+2 ∑
K+1 ∏
K+1 ∑
K+1 ∏
K+1
Mj e = πK+2 1 + Mj e .
(1) (1) (1) (1) (1)
1= πk e = πK+2 + πK+2 (14)
k=0 i=0 j=K+1−i i=0 j=K+1−i

(1)
Step 3. Substitute πK+2 into

(1) (1)
∏
K+1
(1)
πi = πK+2 Mj , i = 0, . . . , K + 1.
j=K+1−i

Note that this method works efficiently as long as K < ∞ is not too large. But for large K the
matrix geometric solution corresponding to K = ∞ is a good approximation.

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

 STOCHASTIC ANALYSIS OF A CONTROLLED HETEROGENEOUS QUEUE 127

4.2. Infinite retrial group

In case of infinite retrial group K = ∞ the infinitesimal matrix Λ(1) has infinite size and is
obtained from the matrix (11) by removing the last three rows.
We consider first the matrix-geometric part of the equations, above the threshold level q2∗ :

(1) (1) (1) (1) (1) (1) (1)

πq∗ +j A4 + πq∗ +j+2 D2 = πq∗ +j+1 (C3 − B3 ), j ≥ 0.
2 2 2

Conjecturing the matrix-geometric form

(1) (1)
πq∗ +j = πq∗ (R(1) )j
2 2

where R(1) is the minimal non-negative solution to the matrix equation

(1) (1) (1) (1)
(R(1) )2 D2 − R(1) (C3 − B3 ) + A4 = 0, (15)

which is typically solved numerically using the following iteration procedure:

R(1) (0) = 0, (16)

B3 )−1 B3 )−1 ,
(1) (1) (1) (1) (1) (1)
R(1) (n + 1) = A4 (C3 − + (R(1) )2 (n)D2 (C3 −

where the iteration halts when entries in R(1) (n + 1) and R(1) (n) differ in absolute value by less
that a given small constant.
The general theory [12] states that the necessary and sufficient condition for stability is

(1) (1)
p(1) D2 e > p(1) A4 e,

(1) (1) (1) (1) (1) (1)

where p(1) = (p0 , p1 , p2 , p3 ) is a stationary probability vector given by p(1) (A4 − (C3 −
(1) (1)
B3 ) + D2 ) = 0, p(1) e = 1.

(1) (1)
Theorem 4. For the system under optimal policy, the stationary vector p(1) of A4 − (C3 −
(1) (1)
B3 ) + D2 is given by

(1) (λ + γ)2 (λ + µ2 + γ)
p0 = ,
(λ + µ1 + γ)((λ + γ)(λ + 2µ1 + γ) + µ2 M )
(1) µ1 (1)
p1 = p ,
λ + µ2 + γ 0
(1) µ2 (λ + M + γ) (1)
p2 = p0 ,
(λ + γ)(λ + µ2 + γ)
(1) µ1 µ2 (2(λ + γ) + M ) (1)
p3 = p .
(λ + γ)2 (λ + µ2 + γ) 0

The system is stable if and only if the load factor ρ(1) satisfies

λ(λ + γ)2 (λ + µ2 + γ)
ρ(1) = < 1. (17)
M γ(λ + γ)2 + γµ1 µ2 (3(λ + γ) + µ1 ) + µ22 γ(λ + µ1 + γ)

Proof. By elementary calculations.

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

128 EFROSININ,SZTRIK

If
∑the system is stable then all eigenvalues of R must be less than 1 and the geometric sequence
∞ (1) −1 converges.
j=0 (R ) = (I − R )
(1) j

Equations for the boundary states below the threshold level are still to be solved, namely:

(1) (1) (1) 1 (1) (1)

π0 = π1 D0 = π1 M 0 ;
λ
(1) 1
A )−1 = π2 M1 ;
(1) (1) (1) (1) (1) (1) (1) (1)
π1 = π2 D1 (C1 − B1 − D0
λ 1
= π3 D2 (C2 − B2 − M1 A2 )−1 = π3 M2 ;
(1) (1) (1) (1) (1) (1) (1) (1) (1)
π2
π3 = π4 D2 (C2 − B2 − M2 A3 )−1 = π4 M3 ;
(1) (1) (1) (1) (1) (1) (1) (1) (1)

= πi+1 D2 (C2 − B2 − Mi−1 A3 )−1 = πi+1 Mi , 4 ≤ i ≤ q2∗ − 1;

(1) (1) (1) (1) (1) (1) (1) (1) (1)
πi
πq∗ = πq∗ (Mq∗ −1 A3 + R(1) D2 )(C2 − B2 )−1 .
(1) (1) (1) (1) (1) (1) (1)
(18)
2 2 2

The following algorithm is used to in the calculations:

Algorithm 3.
Step 1. Solve (15) for matrix R(1) , using iterations (16).
Step 2. Evaluate the Matrices Mj for j = 0, . . . , q2∗ − 1.
(1)

(1)
Step 3. Evaluate the value πq∗ by solving the normalisation condition
2

∞ q2∗ −1 q2∗ −1 ∞
∑ (1) (1)
∑ ∏ (1) (1)
∑
1= πk e = πq ∗ Mj e + πq∗ (R(1) )j e (19)
2 2
k=0 i=0 j=q2∗ −1−i j=0
[q∑
∗
2 −1 q2∗ −1
∏ ]
(1) (1) (1) −1
= πq∗ Mj e + (I − R ) e .
2
i=0 j=q2∗ −1−i

with the equation

(1) (1) (1) (1) (1) (1)
πq∗ (Mq∗ −1 A3 − (C2 − B2 ) + R(1) D2 ) = 0.
2 2

(1)
Step 4. Substitute πq∗ in
2

q2∗ −1
(1) (1)
∏ (1)
πi = πq ∗ Mj (20)
2
j=q2∗ −1−i

for the values i = 0, . . . , q2∗ − 1 and calculate

(1) (1)
πq∗ +j = πq∗ (R(1) )j (21)
2 2

for j > 0.

5. STEADY-STATE DISTRIBUTIONS OF THE SYSTEM UNDER HEURISTIC POLICIES

To measure the advantages of optimal threshold policy (OTP) we consider four alternative
models, namely

– Scheduling threshold policy (STP)

– Fastest free server selection (FFS)

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

 STOCHASTIC ANALYSIS OF A CONTROLLED HETEROGENEOUS QUEUE 129

– Random server selection (RSS)

– System with homogeneous servers (HS) and the same total service time.

The models STP, FFS, RSS and HS will be denoted by index m = {2, 3, 4, 5}, respectively. The
formulas for the calculation of the steady-state distribution of the system under the STP are exactly
the same as for the optimal policy with only one exception that the threshold level may differ from
the optimal one. For the homogeneous system to calculate the performance characteristics we use
the steady-state distribution obtained in [19].
In the next subsections calculation procedures are treated for systems under the FFS and RSS
policies.

5.1. Finite retrial group

In STP case the fastest server must be busy whenever the arrival occurs whereas the slower
server can be switched on only if upon arrival of a primary or retrial customer the orbit has length
defined by (6). The policy FSS means that the fastest free server must be occupied upon an arrival
of a primary or retrial customer. Under the policy RSS arrivals choose any free server with equal
probability.
Since the system under FFS and RSS control policies is described by the same Markov process
{X(t)}t≥0 with the state space E as under the optimal control policy, we have a similar states
partitioning.
Analogously as in previous sections we can write down the three-diagonal infinitesimal block
matrices Λ(m) , m = {2, 3, 4} of dimension 4(K + 1). It is obvious that for STP matrix has the same
form as for the optimal policy. For policies FFS and RSS policies in case K < ∞ the infinitesimal
matrices are of the form,

 (m) 
−λ A1 0 0 0 ... 0
 (m) (m) (m) (m) 
 D0 −(C1 − B1 ) A2 0 0 ... 0 
 
 0 D1
(m) (m)
−(C3 − B3 )
(m)
A4
(m)
0 ... 0 
 
 0 (m) (m) (m)
−(C3 − B3 )
(m)
0 
 0 D2 A4 ... 
Λ(m) 
= . 
.. .. .. .. 
 . . . . . . 0 
 
 (m) (m) (m) (m) 
 0 ... 0 D3 −(C3 − B3 ) A5 0 
 (m) 
 0 ... 0 0 D3
(m)
−(C4 − B4 ) A6 
(m) (m)
(m)
0 ... 0 0 0 D4 M

 
λ000
=  λ 0 0 0  , Ai = Ai , i = 4, 5, 6,
(3) (1) (3) (3) (1) (3) (1) (3) (1) (3) (4) (1)
where A1 = A1 , A2 B1 = B 1 , B3 = B 3 , B4 = B4 = B4 ,
00λ0
(3) (4) (1) (3) (4) (1) (3) (4) (1)
C1 = C1 = C 1 , Ci
= Ci = Ci , i = 3, 4, Di = Di = Di , 0 ≤ i ≤ 4,
   
  λ 0 0 0 λ 0 0
λ 0 0 0 λ 0 0  
(λ λ )
 0 λ 0 0
= 2 2 0 , A2 =  λ 0 0 0  , A4 =  , A5 =  
(4) (4) (4) (4)
A1 λ 0 0 ,
λ 0 0 0
0 λ λ
0
2 2 0 λ λ
0 0 λ λ
 2 2 2 2
  0 0 0 0
0 0 0 γ 0 0 0
= A6 , B1 =  0 0 0  , B3 =  
(4) (1) (4) (4)
A6 γ 0 0 0
γ γ
2 2
0 γ γ
0 2 2 0

The steady-state probability vector π (m) , m = {2, 3, 4} satisfies the system

π (m) Λ(m) = 0, π (m) e = 1.

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

130 EFROSININ,SZTRIK

To solve the system for FFS and RSS policies we represent the equations in the form

(m) (m) (m) 1 (m) (m)

π0 = π1 D0 = π1 M0 ;
λ
(m) 1
A )−1 = π2 M1 ;
(m) (m) (m) (m) (m) (m) (m) (m)
π1 = π2 D1 (C1 − B1 − D0
λ 1
= π3 D2 (C3 − B3 − M1 A2 )−1 = π3 M2 ;
(m) (m) (m) (m) (m) (m) (m) (m) (m)
π2
− Mi−1 A4 )−1 = πi+1 Mi
(m) (m) (m) (m) (m) (m) (m) (m) (m)
πi = πi+1 D2 (C3 − B3 , 3 ≤ i ≤ K − 1;
MK−1 A4 )−1 = πK+1 MK ;
(m) (m) (m) (m) (m) (m) (m) (m) (m)
πK = πK+1 D3 (C3 − B3 −
MK A5 )−1 = πK+2 MK+1 ,
(m) (m) (m) (m) (m) (m) (m) (m) (m)
πK+1 = πK+2 D4 (C4 − B4 − m = {3, 4}. (22)

(m)
To calculate the values πk , m = {3, 4} it is necessary:

Algorithm 4.
(m)
Step 1. Evaluate matrices Mk , k = 0, . . . , K + 1 by relation (22).
(m)
Step 2 Evaluate the value πK+2 from the normalization condition
 
∑
K+2 ∑
K+1 ∏
K+1 ∑
K+1 ∏
K+1
e = πK+2 1 + e .
(m) (m) (m) (m) (m) (m)
1= πk e = πK+2 + πK+2 Mj Mj (23)
k=0 i=0 j=K+1−i i=0 j=K+1−i

(m)
Step 3. Substitute πK+2 into
(m) (m)
∏
K+1
(m)
πi = πK+2 Mj .
j=K+1−i

for i = 0, . . . , K + 1.

5.2. Infinite retrial group

In case of infinite retrial group, K = ∞, the infinitesimal matrices Λ(m) , m = {2, 3, 4} are
obtained from the above matrices by removing the last three rows. In this case we conjecture the
matrix-geometric form
(m) (m)
π2+j = π2 (R(m) )j , j ≥ 0

where the matrix R(m) is to be found by solving the following quadratic equation
(m) (m) (m) (m)
(R(m) )2 D2 − R(m) (C3 − B3 ) + A4 = 0. (24)

(m) (m)
As before the necessary and sufficient condition for stability is p(m) D2 e > p(m) A4 e, where p(m)
(m) (m) (m) (m)
is a stationary probability vector given by p(m) (A4 − (C3 − B3 ) + D2 ) = 0, p(m) e = 1.

Theorem 5. For the system under STP and FFS control policies, the stationary probability vector
(m) (m) (m) (m)
p(m) of A4 − (C3 − B3 ) + D2 , m = {2, 3} is given by formulas 4. The system is stable if
and only if the load factor ρ(m) , defined by (17) satisfies

ρ(m) < 1, m = {2, 3}.

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

 STOCHASTIC ANALYSIS OF A CONTROLLED HETEROGENEOUS QUEUE 131

Theorem 6. For the system under RSS control policy, the stationary probability vector p(4) of
(4) (4) (4) (4)
A4 − (C3 − B3 ) + D2 is given by

(4) (λ + γ)2
p0 = ,
(λ + γ)(λ + M + γ) + 2µ1 µ2
(4) µ1 (4)
p1 = p ,
λ+γ 0
(4) µ2 (4)
p2 = p ,
λ+γ 0
(4) 2µ1 µ2 (4)
p3 = p .
(λ + γ)2 0

The system is stable if and only if the load factor ρ(4) satisfies

λ(λ + γ)2
ρ(4) = < 1. (25)
M γ(λ + γ) + 2γµ1 µ2

Proof. By elementary calculations.

(m) (m)
Probabilities π0 and π1 satisfy relations

(m) (m) (m) 1 (m) (m)

π0 = π1 D 0 = π1 M0 , (26)
λ
(m) 1
A1 )−1 = π2 M1
(m) (m) (m) (m) (m) (m) (m) (m)
π1 = π2 D1 (C1 − B1 − D0 .
λ
In case m = {3, 4} the following algorithm is used for the calculations:

Algorithm 5.
Step 1. Solve equations (24) for matrix R(m) , using iterations starting from R(m) (0) = 0.
(m) (m)
Step 2. Evaluate matrices M0 and M1 .
(m)
Step 3. Evaluate value π2 by solving the normalizing condition
∞
∑ ∞
∑
(m) (m) (m) (m) (m) (m) (m)
1= πk e = π2 M1 M0 e + π2 M 1 e + π2 (R(m) )j e (27)
k=0 j=0
[ ]
(m) (m) (m) (m) (m) −1
= π2 M1 M0 e + M1 e + (I − R ) e .

with the equation

(m) (m) (m) (m) (m) (m)
π2 (M1 A2 − (C3 − B3 ) + R(m) D2 ) = 0.

(m)
Step 4. Substitute π2 into

(m) (m) (m) (m) (m) (m) (m)

π0 = π2 M 1 M0 , π1 = π2 M1 (28)

and calculate
(m) (m)
π2+j = π2 (R(m) )j , j > 0. (29)

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

132 EFROSININ,SZTRIK

6. SYSTEM PERFORMANCE CHARACTERISTICS

By means of steady-state probabilities the main performance measures in case K = ∞ can be
derived as follows.

Corollary.
Utilization of the system
(m) (m)
U = 1 − π0 (30)

Utilization of the first and second server

∗ 
q2 −1
∑
U 1 = π1 e1 +  πi + πq∗ (I − R(m) )−1  (e0 + e2 ), m = {1, 2}
(m) (m) (m) (m)
(31)
2
i=2
 
q2∗ −1
∑
= π1 e0 +  + πq∗ (I − R(m) )−1  (e0 + e1 ), m = {1, 2}
(m) (m) (m) (m)
U2 πi
2
i=2

= π1 e1 + π2 (I − R(m) )−1 (e0 + e2 ), m = {3, 4}

(m) (m) (m)
U1
= π1 e0 + π2 (I − R(m) )−1 (e0 + e1 ), m = {3, 4}
(m) (m) (m)
U2

Mean number of busy servers

∗ 
q2 −1
∑
= π1 (e0 + e1 ) +  πi + πq∗ (I − R(m) )−1  (2e0 + e1 + e2 ) = U 1 + U 2 , m = {1, 2}
(m) (m) (m) (m) (m) (m)
C
2
i=2
(32)
= π1 (e0 + e1 ) + π2 (I − R(m) )−1 (2e0 + e1 + e2 ) = U 1
(m) (m) (m) (m) (m)
C + U 2 , m = {3, 4}

Mean number of customers waiting in the queue

∗ 
q2 −1
∑
= π1 e2 +  iπi + πq∗ (R(m) (I − R(m) )−1 + q2∗ I)(I − R(m) )−1  e
(m) (m) (m) (m)
Q (33)
2
i=2
 
q2∗ −1
∑
− + πq∗ (I − R(m) )−1  (2e0 + e1 + e2 ), m = {1, 2}
(m) (m)
πi
2
i=2

= π1 e2 + π2 [(R(m) (I − R(m) )−1 + 2I)(I − R(m) )−1 e − (I − R(m) )−1 (2e0 + e1 + e2 )], m = {3, 4}
(m) (m) (m)
Q

Mean number of customers in the system

q2∗ −1
∑
e + πq∗ (R(m) (I − R(m) )−1 + q2∗ I)(I − R(m) )−1 e = C
(m) (m) (m) (m) (m)
N = iπi +Q (34)
2
i=1

Mean waiting and sojourn times

(m) (m)
(m) Q (m) N
W = , T = (35)
λ λ

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

 STOCHASTIC ANALYSIS OF A CONTROLLED HETEROGENEOUS QUEUE 133

Overall rate of retrials

(m) (m) (m) (m)
γ̃1 = γ(1 − π0 − π1 (e0 + e1 ) − π2 e0 ) (36)

Rate of successful retrials

   
q2∗ −1 q2∗ −1
∑ ∑
−γ e2 +  + πq∗ (I − R)−1  e0  , m = {1, 2}
(m) (m) (m) (m) (m)
γ̃2 = γ̃1 πi πi (37)
2
i=2 i=3
(m) (m) (m) −1
γ̃2 = γ̃1 − γπ2 (I − R) e0 , m = {3, 4}

(m)
γ̃2
Fraction of successful retrials (m)
γ̃1
Blocking probability
 
q2∗ −1 q2∗ −1
∑ ∑
e2 +  + πq∗ (I − R(m) )−1  e0 , m = {1, 2}
(m) (m) (m) (m) (m)
Pblocking = π1 e1 + πi πi (38)
2
i=2 i=2
(m) (m) (m) −1
Pblocking = π2 (I −R ) e0 , m = {3, 4}

Probability of immediate access

q2∗ −1
∑
(e1 + e3 ) + πq∗ (I − R)−1 (e1 + e2 + e3 ), m = {1, 2}
(m) (m) (m) (m) (m)
Paccess = π0 + π1 (e0 + e2 ) + πi
2
i=2
(39)
+ π1 e + π2 (I − R(m) )−1 (e1 + e2 + e3 ), m = {3, 4}
(m) (m) (m) (m)
Paccess = π0

Mean busy period

( )
(m) 1 1
Tbusy = (m)
−1 . (40)
λ π0

Mean time in a regenerative cycle during which the orbiting customer is blocked
( ) (m)
(m) (m) 1 (m) Pblocking
T blocking = Pblocking + Tbusy = (m)
(41)
λ λπ0

7. NUMERICAL RESULTS AND COMPARISON ANALYSIS

Consider the system M/M/2 with primary arrival rate λ, retrial rate γ and service rates µ1 and
µ2 . By means of Mathematica package the following procedures were developed :

– calculation of steady-state probabilities under optimal and empirical service disciplines, by using
formulas (18–21) and (26)–(29),

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

134 EFROSININ,SZTRIK

– calculation of the performance characteristics under different service disciplines, by using for-
mulas (30–41).

Next we present some numerical results to show the effect of parameters on the performance
characteristics. In the tables below performance measures for the system under different selection
policies m = {1, 2, 3, 4} and for the homogeneous system m = {5} are given for varying values of
primary arrival, retrial and service rates, respectively

(1) (2) (3) (4) (5) (1) (2) (3) (4) (5)
λ, µ1 , µ2 , γ U U U U U N N N N N
0.1,2.2,0.3,8.5 0.0459 0.0459 0.0552 0.1648 0.0769 0.0482 0.0482 0.0575 0.1717 0.0802
0.1,2.2,0.3,1.5 0.0485 0.0484 0.0554 0.1650 0.0771 0.0510 0.0510 0.0577 0.1722 0.0803
0.1,1.3,1.2,8.5 0.0744 0.0744 0.0744 0.0771 0.0769 0.0775 0.0775 0.0775 0.0803 0.0802
0.1,1.3,1.2,1.5 0.0745 0.0745 0.0745 0.0772 0.0771 0.0777 0.0777 0.0777 0.0805 0.0803
1.3,2.2,0.3,8.5 0.6775 0.6706 0.6706 0.8526 0.7034 1.6820 1.8538 1.8520 2.0310 1.5904
1.3,2.2,0.3,1.5 0.8692 0.8799 0.8814 0.9364 0.8043 4.4750 4.6931 4.4960 5.5240 2.8376
1.3,1.3,1.2,8.5 0.6992 0.6992 0.6992 0.7037 0.7034 1.5799 1.5799 1.5800 1.5900 1.5904
1.3,1.3,1.2,1.5 0.7997 0.7997 0.7997 0.8040 0.8043 2.7940 2.7940 2.7940 2.8250 2.8376

Table 1. Utilization and mean number of customers in the system

(1) (2) (3) (4) (5) (1) (2) (3) (4) (5)
λ, µ1 , µ2 , γ W W W W W T T T T T
0.1,2.2,0.3,8.5 0.0275 0.0275 0.0012 0.0036 0.0017 0.4821 0.4821 0.5754 1.7173 0.8017
0.1,2.2,0.3,1.5 0.0546 0.0546 0.0025 0.0074 0.0034 0.5096 0.5096 0.5765 1.7218 0.8034
0.1,1.3,1.2,8.5 0.0016 0.0016 0.0016 0.0017 0.0017 0.7753 0.7753 0.7753 0.8028 0.8017
0.1,1.3,1.2,1.5 0.0033 0.0033 0.0033 0.0034 0.0034 0.7770 0.7770 0.7770 0.8046 0.8034
1.3,2.2,0.3,8.5 0.6437 0.9152 0.5253 0.5779 0.4234 1.2939 1.4260 1.4246 1.5623 1.2234
1.3,2.2,0.3,1.5 2.5848 2.8059 2.5317 3.2481 1.3828 3.4421 3.6101 3.4586 4.2493 2.1827
1.3,1.3,1.2,8.5 0.4202 0.4202 0.4202 0.4231 0.4234 1.2154 1.2154 1.2154 1.2233 1.2234
1.3,1.3,1.2,1.5 1.3545 1.3545 1.3545 1.3728 1.3828 2.1492 2.1492 2.1492 2.1727 2.1828

Table 2. Mean waiting time and mean sojourn time of a customer

The following figures represent the system utilization (Figure 2(a,b)), mean number of cus-
tomers in the system (Figure 3(a,b)), mean waiting time (Figure 4(a,b)) and mean sojourn time
of customers (Figure 5(a,b)) for service intensities µ1 = 2.2, µ2 = 0.3, retrial intensity γ = 2.5 for
figures labeled by ”a”and γ = 18.5 for figures labeled by ”b”with varying primary arrival intensity
0.05 ≤ λ ≤ 1.7. Analyzing the figures we have noticed the following.
1. When the primary arrival rate λ makes relative small contribution to the load factor of the sys-
tem then the OPT and STP coincide that leads to the equal values of the performance measures.
Otherwise, the policies as well as the performance measures are different.
2. When the primary arrival rate is quite large and the load factor tends to 1 (”heavy traffic”) then
difference between the policies can be neglected.
3. The curve of the mean waiting time for the FFS lies below other graphs, i.e. this policy minimizes
the mean number of jobs in the orbit. This does not contradict with the optimality of the OTP
since this policy optimizes the mean sojourn time, that is confirmed by the corresponding figure.
4. All curves are monotone except for the mean waiting time in case of the OTP and mean sojourn
time in case of the RSS policy. In first case as λ increases the threshold levels of the OTP decreases
that leads to the reduction of the waiting time in the queue. In second case the convexity of the
curve is connected with the mean service time that contributes to the mean sojourn time. If the
rate is very small a new arrival most likely sees two servers idle and under the RSS policy with
equal probability occupies them. Upon increasing of the arrival rate the slower server will be made
more busy and fastest server is used more intensively, that resulting in less service time.

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

 STOCHASTIC ANALYSIS OF A CONTROLLED HETEROGENEOUS QUEUE 135
(a) (b)

1 1

0.8 0.8

0.6 0.6
— HmL

— HmL
U

U
0.4 0.4

OTP OTP
0.2 STP 0.2 STP
FFS FFS
RSS RSS
HS HS
0.25 0.5 0.75 1 1.25 1.5 0.25 0.5 0.75 1 1.25 1.5
Λ Λ

Figure 2. System utilization versus primary arrival and retrial rate (a) γ = 2.5 (b) γ = 18.5

(a) (b)

4 4

3.5 3.5

3 3

2.5 2.5
— HmL

— HmL

2 2
N

1.5 1.5

1 OTP 1 OTP
STP STP
FFS FFS
0.5 RSS 0.5 RSS
HS HS
0.2 0.4 0.6 0.8 1 1.2 1.4 0.25 0.5 0.75 1 1.25 1.5
Λ Λ

Figure 3. Mean number of customers versus primary arrival and retrial rate (a) γ = 2.5 (b) γ = 18.5

5. In comparison to the classical queues [19] where the system with heterogeneous servers under
OTP is always superior in performance to the homogeneous one with the same total service time,
in case of retrials the system with homogeneous servers has advantage over heterogeneous one only
if µ2 < λ for large µ1 /λ and small γ/λ or vice versa.
The next figures represent the system utilization (Figure 6(a)), mean number of customers in
the system (Figure 6(b)), mean waiting time (Figure 7(a)) and mean sojourn time of a customer
(Figure 7(b)) for service intensities µ1 = 2.2, µ2 = 0.3, primary arrival rate λ = 0.5 with varying
retrial rate 0.05 ≤ γ ≤ 10.
With respect to the given performance measures we observe the following.
1. For small retrial rate the threshold control policies prescribe to use the slower server when q2∗ = 1,
that coincides with the heuristic policies (FFS, RSS). As the retrial rate increases the advantage of
the threshold policies with respect to the mean number of customers in the system or mean sojourn
time becomes more evident.
2. For large retrial rate the illustrated curves converges to their asymptotic values that resemble
very close to the corresponding performance measures of the classical M/M/2 queue.
3. All curves monotone decrease except for the mean waiting time in case of threshold policies

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

136 EFROSININ,SZTRIK
(a) (b)

1.75 1
1.5
0.8
1.25

— HmL
— HmL

1 0.6

W
W

0.75
0.4
0.5 OTP OTP
STP STP
FFS 0.2 FFS
0.25
RSS RSS
HS HS
0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 0.25 0.5 0.75 1 1.25 1.5
Λ Λ

Figure 4. Mean waiting time versus primary arrival and retrial rate (a) γ = 2.5 (b) γ = 18.5

(a) (b)

3 3

2.5 2.5

2 2
— HmL

— HmL

1.5 1.5
T
T

1 1
OTP OTP
STP STP
0.5 FFS 0.5 FFS
RSS RSS
HS HS
0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 0.25 0.5 0.75 1 1.25 1.5
Λ Λ

Figure 5. Mean sojourn time versus primary arrival and retrial rate (a) γ = 2.5 (b) γ = 18.5

(OPT, STP). It can be explained by the fact that as γ increases the threshold levels also increase
that in turn leads to the significant increasing of the mean waiting time.
(m)
Figure 8 displays the influence of γ on T blocking , m = {1, 2, 3, 4}, graphs are labeled by (a,b,c,d),
respectively.
1. Pictures show that the curves for all the models start by decreasing to a minimum and
then converges to their asymptotic values. These results coincide with the corresponding outcomes
obtained in [2] for the uncontrollable homogeneous retrial queues with direct access to the server
facility.
2. The step structure of the curves in case of threshold policies (OTP, STP) we interpret also by
changing of the threshold levels which implies the significant changing of the mean blocking time.
3. When the retrial rate is small the mean blocking time for threshold policies (OTP, STP) is quite
small in comparison with the heuristic policies (FFS, RSS). But as γ increases, the level q2∗ increases
as well, that leads to the larger mean blocking time.

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

 STOCHASTIC ANALYSIS OF A CONTROLLED HETEROGENEOUS QUEUE 137
(a) (b)

1 1
OTP OTP
STP STP
FSS FFS
0.8 0.8
RSS RSS
— (m)

— (m)
0.6 0.6
U

N
0.4 0.4

2 4 6 8 10 2 4 6 8 10
Γ Γ

Figure 6. (a) System utilization and (b) mean number of customers in the system versus retrial rate

(a) (b)

0.8 3
OTP OTP
STP STP
FFS 2.5 FSS
0.6
RSS RSS

2
— (m)

— (m)

0.4
W

1.5

0.2
1

2 4 6 8 10 2 4 6 8 10
Γ Γ

Figure 7. (a) Mean waiting time and (b) mean sojourn time of a customer versus retrial rate

8. CONCLUSION

The present paper considered a controllable queueing system with heterogeneous servers, con-
stant retrial rate. It was proved that similarly to the classical queueing systems the optimal control
policy of the retrial system is of threshold and monotone type, i.e. the fastest server must be used
whenever the customer comes to service area, while the slower server has to be switched on only
if the number of customers in orbit reaches some prespecified level. We have derived the formula
for the calculation of the optimal threshold levels for scheduling problem and approximation for
the general case and presented an efficient algorithm for the numerical calculation of these levels in
general case. Examining some numerical results we conjecture the suboptimality of the STP in so
called ”light traffic case”when the arrival rate is smaller that the slowest server and FSS in ”heavy
traffic case”when the load factor tends to 1.
We showed that the controllable model with heterogeneous servers and given control policy may
be treated as QBD process with block states and transition submatrices. Thus the general theory
of matrix-geometric solutions was applicable for developing numerical methods.

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

138 EFROSININ,SZTRIK
(a) (b)

0.6 0.6

0.5 0.5
blocking

blocking
0.4 0.4
— HmL

— HmL
T

T
0.3 0.3
OTP STP

5 10 15 20 25 30 5 10 15 20 25 30
Γ Γ

(c) (d)

0.26 0.48

0.25 FFS RSS

0.478

0.24
blocking

blocking 0.476
0.23
— HmL

— HmL

0.474
T

0.22

0.472
0.21

0.47
5 10 15 20 25 30 5 10 15 20 25 30
Γ Γ

Figure 8. Mean blocking time in a regenerative cycle versus retrial rate

By using the structured properties of the optimal control policy we derived an efficient algorithm
for the calculation of steady state probabilities and mean performance characteristics for the system
under optimal and other heuristic control policies. Some performance measures were calculated and
compared under different control policies. Numerical analysis showed that for the relative small load
factor the threshold policies (OTP, STP) have a great advantage in mean number of customers in
the system or mean sojourn time over heuristic policies (FFS, RSS) while the difference between
threshold policies was negligible. As the load factor tends to 1 the advantages of the threshold
policies decreased and the performance measures converged to similar values. Other performance
characteristics under optimal policy may be worse as under heuristic policies because the optimal
policy corresponds to the certain optimization criterion that minimizes the specified performance
measure. Necessary and sufficient conditions for the stability of the systems were obtained. In case
of constant retrial rate these conditions depended on all system parameters.
The presented methods can also be extended to some other models, e.g. with more than two
servers or where interarrival and service times are dependent on a modulating Markov process, or
with phase type times. These cases are not discussed because they would require very complicated
notation to write down the algorithms, but the extensions are natural and possible in principle.

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011

 STOCHASTIC ANALYSIS OF A CONTROLLED HETEROGENEOUS QUEUE 139

REFERENCES
1. A.K. Agrawala, E.G. Coffman et al. A stochastic optimization algorithm minimizing expected flow times
on uniform processors. IEEE Trans. Comp. 1984; 33: 351–356.
2. J.R. Artalejo, A. Gomez-Corral, M.F. Neuts. Analysis of multiserver queues with constant retrial rate.
European Journal of Operations Research 2001; 135: 569–581.
3. L. Breuer, V. Klimenok, A. Birukov, A. Dudin and Udo R. Krieger. Modeling the access to a wireless
network at hot spots. European Transactions on Telecommunication 2005; 16: 309–316.
4. Srinivas R. Chakravarthy, A. Krishnamoorty, V.C. Joshua. Analysis of a multi-server retrial queue with
search of customers from the orbit. Performance Evaluation 2006; 63: 776–798.
5. B.D. Choi, Y.W. Shin, W.C. Ahn. Retrial queues with collision arising from unslotted CSMA/CD
protocol. Queueing systems 1992; 11: 335–356.
6. D. Efrosinin, L. Breuer. Threshold policies for controlled retrial queues with heterogeneous servers.
Annals of Operation Research 2006; 141: 139–162.
7. G.I. Falin, J.G.C. Tempelton. Retrial queues. Chapman and Hall, London, 1997.
8. R.A. Howard. Dynamic programming and Markov processes. Wiley Series, 1960.
9. G. Koole. A simple proof of the optimality of a threshold policy in a two-server queueing system. Systems
and Control Letters, 1995; 26: 301–303.
10. W. Lin, P.R. Kumar. Optimal control of a queueing system with two heterogeneous servers. IEEE
Transactions on Automatic Control 1984; 29: 696–703.
11. R. Larsen, A.K. Agrawala. Control of a heterogeneous servers two-server exponential queueing system.
IEEE Transactions on software engineering 1983; 4:522–526.
12. M.F. Neuts Matrix-geometric solutions in stochastic models. Baltimore, Lnd.: The John Hopkins Univ.
Press, 1981.
13. B. Pourbabai. Markovian queueing systems with retrials and heterogeneous servers. Computers and
Mathematics with Applications 1987; 13: 917–923.
14. M. L. Puterman. Markov Decision Process. Wiley series in Probability and Mathematical Statistics, 1994.
15. M. Ross Non-discounted denumerable markovian decision models. The Annals of Mathematical Statistics,
V. 39(2). pp.412–423
16. V. V. Rykov. Controllable queueing systems (In Russian). Itogi nauki i techniki. Teoria verojatnostey i
Matematicheskoy Statistiki. Teoreticheskaia Kibernetika 1975, 12: 45-152.
17. V. Rykov. Monotone Control of Queueing Systems with Heterogeneous Servers. Queueing systems 2001;
37: 391–403.
18. V. Rykov, D. Efrosinin. Optimal control of queueing systems with heterogeneous servers. Queueing
Systems 2004; 46: 389-407.
19. V. Rykov, D. Efrosinin. On performance characteristics for queueing systems with heterogeneous servers.
Autom. and Remote Control, 2008; 1: 64–82.
20. J. Sztrik, J. Roszik. Performance analysis of finite-source retrial queueing systems with heterogeneous
non-reliable servers and different service policies. Research report, Univercity Debrecen 2001.
21. P. Tran-Gia, M. Mandjes. Modeling of customer retrial phenomenon in cellular mobile networks. IEEE
Journal of Selected Areas in Communications 1997; 15: 1406–1414.

ИНФОРМАЦИОННЫЕ ПРОЦЕССЫ ТОМ 11 №1 2011