Tabu Search

UW Spring 2005 INDE 516

Project 2
Lei Li, HongRui Liu, Roberto Lu

Introduction
Tabu socially or culturally proscribed: forbidden to be
used, mentioned, or approached because of social or
cultural rather than legal prohibitions.
(http://encarta.msn.com/dictionary_1861698691/taboo.html)
Glover, F. 1986. Future Paths for Integer Programming
and Links to Artificial Intelligence. Computers and
Operations Research. Vol. 13, pp. 533-549.
Hansen, P. 1986. The Steepest Ascent Mildest Descent
Heuristic for Combinatorial Programming. Congress on
Numerical Methods in Combinatorial Optimization, Capri,
Italy.
Li, Liu, Lu

UW Spring 2005 INDE 516

Overview of Tabu Search

Tabu search is based on introducing
flexible memory structures in conjunction
with strategic restrictions and aspiration
levels as a means for exploiting search
spaces [1].
Meta-heuristic that guides a local heuristic
search procedure to explore the solution
space beyond local optimum by use of a
Tabu list.
Li, Liu, Lu

UW Spring 2005 INDE 516

Overview of Tabu Search

Originated from surrogate constraint methods and
cutting plane approaches [2]
Used to solve combinatorial (finite solution set)
optimization problems
A dynamic neighborhood search method

Use of a flexible memory to restrict the next

solution choice to some subset of neighborhood
of current solution
Li, Liu, Lu

UW Spring 2005 INDE 516

Tabu Search Strategy

3 main strategies [7]:
Forbidding strategy: control what enters the
tabu list
Freeing strategy: control what exits the tabu
list and when
Short-term strategy: manage interplay
between the forbidding strategy and freeing
strategy to select trial solutions

Li, Liu, Lu

UW Spring 2005 INDE 516

Parameters of Tabu Search [5]

Li, Liu, Lu

Local search procedure

Neighborhood structure
Aspiration conditions
Form of tabu moves
Addition of a tabu move
Maximum size of tabu list
Stopping rule
UW Spring 2005 INDE 516

Basic Ingredients of Tabu Search

A chief way to exploit memory in tabu search is to classify a subset of the

moves in a neighborhood as forbidden (or tabu) [1].

A neighborhood is constructed to identify adjacent solutions that can be

reached from current solution [8].

The classification depends on the history of the search, and particularly on

the recency or frequency that certain move or solution components, called
attributes, have participated in generating past solutions [1].

A tabu list records forbidden moves, which are referred to as tabu moves
[5].

Tabu restrictions are subject to an important exception. When a tabu move

has a sufficiently attractive evaluation where it would result in a solution
better than any visited so far, then its tabu classification may be overridden.
A condition that allows such an override to occur is called an aspiration
criterion[1].

Li, Liu, Lu

UW Spring 2005 INDE 516

Basic Tabu Search Algorithm [4]

Step 1: Choose an initial solution i in S. Set i* = i and k=0.
Step 2: Set k=k+1 and generate a subset V* of solution in N(i,k) such
that either one of the Tabu conditions is violated or at least one of the
aspiration conditions holds.

Step 3: Choose a best j in V* and set i=j.

Step 4: If f(i) < f(i*) then set i* = i.
Step 5: Update Tabu and aspiration conditions.
Step 6: If a stopping condition is met then stop. Else go to Step 2.

Li, Liu, Lu

UW Spring 2005 INDE 516

Tabu Search Stopping Conditions

Some immediate stopping conditions could be the following [4]:
1. N(i, K+1) = 0. (no feasible solution in the neighborhood of solution i)
2. K is larger than the maximum number of iterations allowed.
3. The number of iterations since the last improvement of i* is larger
than a specified number.
4. Evidence can be given than an optimum solution has been obtained.
Hillier and Lieberman [5] outlined the tabu search stopping criterion by,
for example, using a fixed number of iterations, a fixed amount of
CPU time, or a fixed number of consecutive iterations without an
improvement in the best objective function value. Also stop at any
iteration where there are no feasible moves into the local
neighborhood of the current trial solution.

Li, Liu, Lu

UW Spring 2005 INDE 516

Flowchart of a Standard Tabu

Search Algorithm [7]
Initial solution
(i in S)

Update Tabu &

Aspiration
Conditions

No

Create a candidate
list of solutions

Evaluate solutions

Stopping conditions
satisfied ?

Choose the best

admissible solution

Yes
Final solution
Li, Liu, Lu

UW Spring 2005 INDE 516

10

Minimum spanning Tree

Given a connected, undirected graph, a
spanning tree of that graph is a subgraph which
is a tree and connects all the vertices together.
A single graph can have many different
spanning trees. We can also assign a weight to
each edge, which is a number representing how
unfavorable it is, and use this to assign a weight
to a spanning tree by computing the sum of the
weights of the edges in that spanning tree. A
minimum spanning tree or minimum weight
spanning tree is then a spanning tree with
weight less than or equal to the weight of every
other spanning tree..
Li, Liu, Lu

UW Spring 2005 INDE 516

11

Li, Liu, Lu

UW Spring 2005 INDE 516

12

 One example would be a cable TV company laying cable

to a new neighborhood. If it is constrained to bury the
cable only along certain paths, then there would be a
graph representing which points are connected by those
paths.
Some of those paths might be more expensive, because
they are longer, or require the cable to be buried deeper;
these paths would be represented by edges with larger
weights.
A spanning tree for that graph would be a subset of
those paths that has no cycles but still connects to every
house. There might be several spanning trees possible.
A minimum spanning tree would be one with the lowest
total cost.

Li, Liu, Lu

UW Spring 2005 INDE 516

13

Example [5]
Minimum spanning tree problem with constraints.
Objective: Connects all nodes with minimum costs
Costs

20

30
10

20

30
10

25
15

25

40

15

40

An optimal solution without

considering constraints

Constraints 1: Link AD can be included only if link DE also is included. (penalty:100)

Constraints 2: At most one of the three links AD, CD, and AB can be included.
(Penalty of 100 if selected two of the three, 200 if all three are selected.)
Li, Liu, Lu

UW Spring 2005 INDE 516

14

Example
Iteration 1
Cost=50+200 (constraint penalties)
B
20

30
10

25
Delete 15

40 Add

Add

Delete

Cost

BE
BE
BE

CE
AC
AB

75+200=275
70+200=270
60+100=160

CD
CD

AD
AC

60+100=160
65+300=365

DE
DE
DE

CE
AC
AD

85+100=185
80+100=180
75+0=75

New cost = 75 (iteration 2)

( local optimum)

Constraints 1: Link AD can be included only if link DE also is included. (penalty:100)

Constraints 2: At most one of the three links AD, CD, and AB can be included.
(Penalty of 100 if selected two of the three, 200 if all three are selected.)
Li, Liu, Lu

UW Spring 2005 INDE 516

15

Example
Tabu list: DE
Iteration 2 Cost=75
Delete

20

30
10

25
15

40

Add

Add

Delete

Cost

AD
AD
AD

DE*
CE
AC

Tabu move
85+100=185
80+100=180

BE
BE
BE

CE
AC
AB

100+0=100
95+0=95
85+0=85

CD
CD

DE*
CE

60+100=160
95+100=195

Tabu
* A tabu move will be considered only if it would
result in a better solution than the best trial
solution found previously (Aspiration Condition)
Iteration 3 new cost = 85 Escape local optimum

Constraints 1: Link AD can be included only if link DE also is included. (penalty:100)

Constraints 2: At most one of the three links AD, CD, and AB can be included.
(Penalty of 100 if selected two of the three, 200 if all three are selected.)
Li, Liu, Lu

UW Spring 2005 INDE 516

16

Example
Tabu list: DE & BE
Iteration 3 Cost=85
B

Tabu

20

30
10

25
15

Add

40

Tabu

Delete

Add

Delete

Cost

AB
AB
AB

BE*
CE
AC

Tabu move
100+0=100
95+0=95

AD
AD
AD

DE*
CE
AC

60+100=160
95+0=95
90+0=90

CD
CD

DE*
CE

70+0=70
105+0=105

* A tabu move will be considered only if it would

result in a better solution than the best trial
solution found previously (Aspiration Condition)
Iteration 4 new cost = 70 Override tabu status

Constraints 1: Link AD can be included only if link DE also is included. (penalty:100)

Constraints 2: At most one of the three links AD, CD, and AB can be included.
(Penalty of 100 if selected two of the three, 200 if all three are selected.)
Li, Liu, Lu

UW Spring 2005 INDE 516

17

Example
Optimal Solution
Cost = 70

B
20

30
10

25

15

Li, Liu, Lu

40

Additional iterations only find

inferior solutions

UW Spring 2005 INDE 516

18

Pros and Cons

Pros:
Allows non-improving solution to be accepted in order to escape
from a local optimum
The use of Tabu list
Can be applied to both discrete and continuous solution spaces
For larger and more difficult problems (scheduling, quadratic
assignment and vehicle routing), tabu search obtains solutions
that rival and often surpass the best solutions previously found
by other approaches [1].

Cons:
Too many parameters to be determined
Number of iterations could be very large
Global optimum may not be found, depends on parameter
settings
Li, Liu, Lu

UW Spring 2005 INDE 516

19

Advanced Topics
Intensification: penalize solutions far
from the current solution
Diversification: penalize solutions close
to the current solution

Li, Liu, Lu

UW Spring 2005 INDE 516

20

Some Convergence Results

Memory Tabu Search converges to the
global optimum with probability one if
randomly generated vectors (x) follows
Gaussian or uniform distribution [6].
Convergent Tabu Search converges to
the global optimum with probability one [3].

Li, Liu, Lu

UW Spring 2005 INDE 516

21

References
[1] Glover, F., Kelly, J. P., and Laguna, M. 1995. Genetic Algorithms and Tabu Search:
Hybrids for Optimization. Computers and Operations Research. Vol. 22, No. 1, pp.
111 134.
[2] Glover, F. and Laguna, M. 1997. Tabu Search. Norwell, MA: Kluwer Academic
Publishers.
[3] Hanafi, S. 2001. On the Convergence of Tabu Search. Journal of Heuristics. Vol. 7,
pp. 47 58.
[4] Hertz, A., Taillard, E. and Werra, D. A Tutorial on Tabu Search. Accessed on April
14, 2005: http://www.cs.colostate.edu/~whitley/CS640/hertz92tutorial.pdf
[5] Hillier, F.S. and Lieberman, G.J. 2005. Introduction to Operations Research. New
York, NY: McGraw-Hill. 8th Ed.
[6] Ji, M. and Tang, H. 2004. Global Optimizations and Tabu Search Based on Mamory.
Applied Mathematics and Computation. Vol. 159, pp. 449 457.
[7] Pham, D.T. and Karaboga, D. 2000. Intelligent Optimisation Techniques Genetic
Algorithms, Tabu Search, Simulated Annealing and Neural Networks. London:
Springer-Verlag.
[8] Reeves, C.R. 1993. Modern Heuristic Techniques for Combinatorial Problems. John
Wiley & Sons, Inc.

Li, Liu, Lu

UW Spring 2005 INDE 516

22