Ant Colony Optimization

Presented by :VISHAL RAJ

 Presentation Outline
• Section I (Introduction)
• Historical Background
• Ant System
• Modified algorithms

• Section II (Applications)

• Section III (Conclusions)

• Conclusions, limitations
 Section 1

• Introduction (Swarm intelligence)

• Natural behavior of ants
• First Algorithm: Ant System
• Improvements to Ant System
 Swarm intelligence

• Collective system capable of accomplishing difficult

tasks in dynamic and varied environments without any
external guidance or control and with no central
coordination

• Achieving a collective performance which could not

normally be achieved by an individual acting alone

• Constituting a natural model particularly suited to

distributed problem solving
 HISTORY
• Ant algorithms were first proposed by Dorigo
and colleagues 1991 as a multi-agent approach
to difficult combinatorial optimization problems.
• Ant algorithms were inspired by the observation
of real ant colonies.
• An important and interesting behavior of
• ant colonies is their foraging behavior, and, in
particular, how ants can find the shortest
• paths between food sources and their nest.
Natural behavior of ant
Most of the ideas of ACO stem from real ants. In
particular, the use of:
(a) a colony of cooperating individuals,

(b) an (artificial) pheromone trail for local stigmergetic communication,

(c) a sequence of local moves to find shortest paths

(d) a stochastic decision policy using local information and no lookahead.

To study in controlled conditions the ants’ foraging behavior, the binary bridge
Experiment has been set up by Deneubourg et al.

• In the model, the probability of choosing a branch at a certain time depends on the total
amount of pheromone on the branch, which in turn is proportional to the number of ants
that used the branch until that time.

• Let Um and Lm are no. of ants ,where Um and Lm =m(total no of ants).

• This functional form of the probability of choosing one branch over the other was obtained
from experiments on trail following the parameters h and k allow us to fit the model to
experimental data.
• The probability of choosing lower branch= (1-Pu(m))
 The Ant Colony Optimization Approach
In the ant colony optimization (ACO) metaheuristic a colony of artificial ants cooperates
in finding good solutions to difficult discrete optimization problems. Cooperation is a
key design component of ACO algorithms: The choice is to allocate the computational
resources to a set of relatively simple agents (artificial ants) that communicate
indirectly by stigmergy. Good solutions are an emergent property of the agents’
cooperative interaction.

• Artificial ants live in a discrete world and their moves consist of transitions from
discrete states to discrete states.

• Artificial ants have an internal state. This private state contains the memory of the
ants’ past actions.

• Artificial ants deposit an amount of pheromone that is a function of the quality of

the solution found.

• Artificial ants’ timing in pheromone laying is problem dependent and often does
not reflect real ants’ behavior. For example, in many cases artificial ants update
pheromone trails only after having generated a solution.
• Artificial ants’ timing in pheromone laying is problem dependent and often does
not reflect real ants’ behavior. For example, in many cases artificial ants update
pheromone trails only after having generated a solution.

• To improve overall system efficiency, ACO algorithms can be enriched with extra
capabilities such as lookahead, local optimization, backtracking, and so on that
cannot be found in real ants. In many implementations ants have been hybridized
with local optimization procedures , while, so far, only Michel and Middendorf have
used a simple one-step lookahead function and there are no examples of backtracking
procedures added to the basic ant capabilities, except for simple recovery procedures
used by Di Caro and Dorigo
 How to implement in a program

•Ants: Simple computer agents

•Move ant: Pick next component in the const. solution

•Pheromone: quantity of pheromone  k

i, j

•Memory: MK or TabuK

•Next move: Use probability to move ant

 A simple TSP example []

[]

A
B
2

[]

C
3

[]

4
D
E []

5
 Iteration 1

[A] [B]

2
1
A
B
[C]

3
C

[D] [E]

4
D 5
E
How to build next sub-solution?

[A]

1
A
[A]
B

1
[A]
C

1
[A,D]
[A]

1 1
D
E
 Iteration 2
[E,A] [C,B]

5
3
A
B
[B,C]

2
C

[A,D]
[D,E]

1
D 4
E
 Iteration 3

[D,E,A] [E,A,B]

4 5
A
B
[A,D,C]

1
C

[B,C,D]
[C,B,E]

2
D 3

E
 Iteration 4
[B,C,D,A] [D,E,A,B]

2 4
A
B
[E,A,B,C]

5
C

[C,B,E,D]
[A,DCE]

D 3
1
E
 Iteration 5
[C,B,E,D,A] [A,D,C,E,B]

3 1
A
B
[D,E,A,B,C]

4
C

[E,A,B,C,D]

[B,C,D,A,E]

D 5

E 2
 Path and Pheromone Evaluation
[A,D,C,E,B]
Q
 if ( i , j )  tour
 ik, j   Lk
0

1 otherwise
[B,C,D,A,E]

2
[C,B,E,D,A]

[D,E,A,B,C]

[E,A,B,C,D]

5
End of First Run

Save Best Tour (Sequence and length)

All ants die

New ants are born

Ant System (Ant Cycle) Dorigo [1] 1991
t = 0; τij(t)=c for ∆τij=0
Initialize
Place the m ants on the n nodes

Update tabuk(s)
Tabu list management

 [  ij ( t )]  [  ij ]  Choose the city j to move


  
if j  allowed k to. Use probability
pijk ( t )    [  ik ( t )] [  ik ]
kallowedk
 Move k-th ant to town j.

0 otherwise Insert town j in tabuk(s)

Compute the length Lk of every ant

Update the shortest tour found

= (i,j)
For every edge
Compute  ij ( t  n )   ij ( t )   ij
For k:=1 to m do
Q
 if ( i , j )  tour described by tabu k
 k
i,j   Lk
0
 otherwise
 ij :  ij   ijk
Yes

Set t = t + n; ∆τij=0 NC<NCmax

&& not
stagn.
No

End
 Stopping Criteria

• Stagnation
• Max Iterations
 Improvements to AS

• Daemon actions are used to apply centralized

actions
– Local optimization procedure
– Bias the search process from global information
 Improvements to AS

• MMAS (St¨utzle and Hoos)

 min   ij   max
• pheromone trails are only updated offline by the daemon
pheromone

• trail values are restricted to an interval [min; max]

• trails are initialized to their maximum value

 Improvements to AS

• Elitist strategy (Dorigo et al).

e / Lgb
( t ) if arc(i, j)  T gb
 ijgb ( t )  
0 otherwise

• ASrank (Bullnheimer, Hartl, and Strauss)

w1
 ij ( t  1 )  ( 1   ) ij ( t )   ( w  r ) ijr ( t )  w ijgb ( t )
r 1
 Improvements to AS

• ACS (Dorigo and Gambardella)

– Strong elitist strategy
– Pseudo-random proportional rule
pheromone update rule:

With Probability (1- q0):

 [  ij ( t )]  [  ij ] 
  [  ( t )]  [  ]  if j  allowed
 k
pij ( t )  
k
ik ik
kallowedk


0 otherwise
 Improvements to AS

• ACS (Pheromone update)

 ij ( t  1 )  ( 1   ) ij ( t )   ijbest ( t )

– Update pheromone trail while building the solution

– Ants eat pheromone on the trail
– Local search added before pheromone update
 Applications

• Traveling Salesman Problem

• Quadratic Assignment Problem
• Network Model Problem
• Vehicle routing
• Graph colouring problem
• Finding largest common subsequece problem
• Job scheduling problem
Travelling Salesman Problem (TSP)

TSP PROBLEM : Given N cities, and a distance function d between

cities, find a tour that:

1. Goes through every city once and only once(Hamiltonian circuit)

2. Minimizes the total distance.

• Problem is NP-hard

• Classical combinatorial
optimization problem to
test.
ACO for the Traveling Salesman Problem

The TSP is a very important problem in the context of

Ant Colony Optimization because it is the problem to
which the original AS was first applied, and it has later
often been used as a benchmark to test a new idea
and algorithmic variants.

The TSP was chosen for many reasons:

• It is a problem to which the ant colony metaphor

• It is one of the most studied NP-hard problems in the combinatorial optimization
• it is very easily to explain. So that the algorithm behavior is not obscured by
too many technicalities.
Ant Systems Algorithm for TSP
Initialize

Place each ant in a randomly chosen city

For Each Ant

Choose NextCity(For Each Ant)

yes more cities

to visit

No

Return to the initial cities

Update pheromone level using the tour cost for each ant

No
Stopping
criteria

yes

Print Best tour

Rules for Transition Probability

1. Whether or not a city has been visited

Use of a memory(tabu list): J ik : set of all cities that are to be visited

2. N ij = 1 d ijvisibility:Heuristic desirability of choosing city j when in city i.

3.Pheromone trail: Tij (t ) This is a global type of information

Transition probability for ant k to go from city i to city j while building its route.

a = 0: closest cities are selected

Trail pheromone in AS

After the completion of a tour, each ant lays some pheromone

 K ij (t ) for each edge that it has used. depends on how well the ant
has performed.

Trail pheromone decay =

ACS : Ant Colony System for TSP
Performance

• Algorithm found best solutions on small problems

(75 city)
• On larger problems converged to good solutions –
but not the best
• On “static” problems like TSP hard to beat specialist
algorithms
• Ants are “dynamic” optimizers – should we even
expect good performance on static problems
• Coupling ant with local optimizers gave world
class results….
 Quadratic Assignment Problem

• The quadratic assignment problem is the problem of assigning n facilities to n locations so that
the cost of the assignment, which is a function of the way facilities have been assigned to
locations, is minimized

Maniezzo, Colorni, and Dorigo applied exactly the same algorithm as AS using the QAP-
specific min-max heuristic to compute the (n) values . The resulting algorithm, AS-QAP, was
tested on a set of standard problems and turned out to be of the same quality as metaheuristic
approaches such as simulated annealing and evolutionary computation.

Similar results were obtained by St¨utzle and Hoos with their MMAS-QAP algorithm
 Job-Shop Scheduling Problem

Given a set M of machines and a set J of jobs consisting of an ordered sequence of

operations to be executed on these machines, the job-shop scheduling problem is that
of assigning operations to machines so that the maximum of the completion times of all
operations is minimized and no two jobs are processed at the same time on the same
machine.

Colorni, Dorigo, and Maniezzo (1994) applied AS to the job-shop scheduling problem
(JSP),

The basic algorithm they applied was exactly the same as AS, where the ´ heuristic
value was computed using the longest remaining processing time heuristic. Due to
the different nature of the constraints with respect to the TSP they also defined a new
way of building the ant’s tabu list. AS-JSP was applied to problems of dimensions up
to 15 machines and 15 jobs always finding solutions within 10% of the optimal value
Vehicle Routing Problem
 Work to date

Problem name Authors Algorithm name Year

Traveling salesman Dorigo, Maniezzo & Colorni AS 1991
Gamberdella & Dorigo Ant-Q 1995

Dorigo & Gamberdella ACS &ACS 3 opt 1996

Stutzle & Hoos MMAS 1997

Bullnheimer, Hartl & Strauss ASrank 1997

Cordon, et al. BWAS 2000
Quadratic assignment Maniezzo, Colorni & Dorigo AS-QAP 1994

Gamberdella, Taillard & Dorigo HAS-QAP 1997

Stutzle & Hoos MMAS-QAP 1998

Maniezzo ANTS-QAP 1999

Maniezzo & Colorni AS-QAP 1994
Scheduling problems Colorni, Dorigo & Maniezzo AS-JSP 1997

Stutzle AS-SMTTP 1999

Barker et al ACS-SMTTP 1999

den Besten, Stutzle & Dorigo ACS-SMTWTP 2000

Merkle, Middenderf & Schmeck ACO-RCPS 1997
Vehicle routing Bullnheimer, Hartl & Strauss AS-VRP 1999

Gamberdella, Taillard & Agazzi HAS-VRP 1999

 Work to date
Problem name Authors Algorithm name Year

Connection-oriented Schoonderwood et al. ABC 1996

network routing White, Pagurek & Oppacher ASGA 1998

Di Caro & Dorigo AntNet-FS 1998

Bonabeau et al. ABC-smart ants 1998

Connection-less Di Caro & Dorigo AntNet & AntNet-FA 1997
network routing Subramanian, Druschel & Chen Regular ants 1997

Heusse et al. CAF 1998

van der Put & Rethkrantz ABC-backward 1998
Sequential ordering Gamberdella& Dorigo HAS-SOP 1997
Graph coloring Costa & Hertz ANTCOL 1997

Shortest common supersequence Michel & Middendorf AS_SCS 1998

Frequency assignment Maniezzo & Carbonaro ANTS-FAP 1998

Generalized assignment Ramalhinho Lourenco & Serra MMAS-GAP 1998
Multiple knapsack Leguizamon & Michalewicz AS-MKP 1999
Optical networks routing Navarro Varela & Sinclair ACO-VWP 1999
Redundancy allocation Liang & Smith ACO-RAP 1999
Constraint satisfaction Solnon Ant-P-solver 2000
 Some inherent advantages

• Positive Feedback accounts for rapid discovery

of good solutions
• Distributed computation avoids premature
convergence
• The greedy heuristic helps find acceptable
solution in the early solution in the early stages
of the search process.
• The collective interaction of a population of
agents.
 Disadvantages in Ant Systems

• Slower convergence than other Heuristics

• Performed poorly for TSP problems larger
than 75 cities.
• No centralized processor to guide the AS
towards good solutions
 Conclusions
• ACO is a recently proposed metaheuristic approach
for solving hard combinatorial optimization problems.
• Artificial ants implement a randomized construction
heuristic which makes probabilistic decisions.
• The a cumulated search experience is taken into
account by the adaptation of the pheromone trail.
• ACO Shows great performance with the “ill-
structured” problems like network routing.
• In ACO Local search is extremely important to obtain
good results.
Thank you