HEURISTIC SEARCH

4.0 Introduction 4.3 Using Heuristics I n Games

4.1 An Algorithm for Heuristic Search 4.4 Complexity Issues
4.2 Admissibility, Monotonicity, and 4.5 Epilogue and References
Informedness
4.6 Exercises

George F Luger

ARTIFICIAL INTELLIGENCE 5th edition

Structures and Strategies for Complex Problem Solving

Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 1

 More formally, why heuristic
functions work?
 In any search problem where there are at most b choices
at each node and a depth of d at the goal node, a naive
search algorithm would have to, in the worst case, search
around O(bd) nodes before finding a solution
(Exponential Time Complexity).

 Heuristics improve the efficiency of search algorithms

by reducing the effective branching factor from b to
(ideally) a low constant b* such that
 1 =< b* << b

2
 Heuristic Functions
 A heuristic function is a function f(n) that gives an estimation on
the “cost” of getting from node n to the goal state – so that the
node with the least cost among all possible choices can be
selected for expansion first.

 Three approaches to defining f:


f measures the value of the current state (its “goodness”)


f measures the estimated cost of getting to the goal from the current
state:

f(n) = h(n) where h(n) = an estimate of the cost to get
from n to a goal


f measures the estimated cost of getting to the goal state from the
current state and the cost of the existing path to it. Often, in this
case, we decompose f:

f(n) = g(n) + h(n) where g(n) = the cost to get to n (from
initial state) 3
 Approach 1: f Measures the Value
of the Current State

 Usually the case when solving optimization problems

 Finding a state such that the value of the metric f is
optimized

 Often, in these cases, f could be a weighted sum of a

set of component values:

 N-Queens

Example: the number of queens under attack …

 Data mining

Example: the “predictive-ness” (a.k.a. accuracy) of a rule
discovered 4
 Approach 2: f Measures the Cost
to the Goal

A state X would be better than a state Y if the

estimated cost of getting from X to the goal
is lower than that of Y – because X would be
closer to the goal than Y

• 8–Puzzle
h1: 8 The number of misplaced tiles
(squares with number).
h2: The sum of the distances of the tiles
from their goal positions.
h2 = 3+1+2+2+2+3+3+2= 18 5
 Approach 3: f measures the total cost
of the solution path (Admissible
Heuristic Functions)
 A heuristic function f(n) = g(n) + h(n) is admissible if h(n) never
overestimates the cost to reach the goal.
 Admissible heuristics are “optimistic”: “the cost is not that much …”
 However, g(n) is the exact cost to reach node n from the initial state.
 Therefore, f(n) never over-estimate the true cost to reach the goal
state through node n.
 Theorem: A search is optimal if h(n) is admissible.
 I.e. The search using h(n) returns an optimal solution.
 Given h2(n) > h1(n) for all n, it’s always more efficient to use h2(n).
 h2 is more realistic than h1 (more informed), though both are optimistic.

6
 Traditional informed
search strategies
 Greedy Best first search
 “Always chooses the successor node with the best f value”
where f(n) = h(n)
 We choose the one that is nearest to the final state among
all possible choices

 A* search
 Best first search using an “admissible” heuristic function f
that takes into account the current cost g
 Always returns the optimal solution path
7
 Admissible 30

Heuristics
● E.g., for the 8-puzzle

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 Admissible 31

● E.g., for the 8-puzzle

Heuristics
– h (n)
1 = number of misplaced tiles
– h (2 n) = total Manhattan distance
(i.e., no. of squares from desired location of each tile)

h 1(S) =?
● h2(S) =? Artificial Intelligence: Informed 15 February
Search 2024

Philipp Koehn
 Admissible 32

● E.g., for the 8-puzzle

Heuristics
– h (n)
1 = number of misplaced tiles
– h (2 n) = total Manhattan distance
(i.e., no. of squares from desired location of each tile)

h 1(S) =? 6
● h2(S) =? 4+0+3+3+1+0+2+1
Artificial= 14
Intelligence: Informed 15 February
Search 2024

Philipp Koehn
 Dominan 33

ce
● If h 2( n) ≥ h1 (n) for all n (both admissible)
→ h2 dominates h1 and is better for search

● Typical search costs (d = depth of solution for 8-puzzle)

d = 14 IDS =
3,473,941 nodes
A∗(h1) = 539 nodes A∗(h2) = 113 nodes
d = 24 IDS ≈
54,000,000,000 nodes
A∗(h1) = 39,135 nodes A∗(h2) = 1,641
nodes

●Given any admissible heuristics ha, hb,

h(n) = max(ha(n), hb(n))

Artificial Intelligence: Informed 15 February
is also admissible and dominates
Search ha , hb 2024

Philipp Koehn
The successive stages of open and closed that generate this graph are:

Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 2

Fig 4.16 State space generated in heuristic search of the 8-puzzle graph.

Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 3

Fig 4.17 Open and closed as they appear after the 3rd iteration of heuristic
search

Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 4

Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 5
Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 6
Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 7
Fig 4.18 Comparison of state space searched using heuristic search with space searched by
breadth-first search. The proportion of the graph searched heuristically is shaded.
The optimal search selection is in bold. Heuristic used is f(n) = g(n) + h(n) where
h(n) is tiles out of place.

Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 8

Fig 4.22 Heuristic measuring conflict applied to states of tic-tac-toe.

Luger: Artificial Intelligence, 5th edition. © Pearson Education Limited, 2005 12

 Local Search for
• Optimization
Note that for many types of problems, the path to a goal is
irrelevant (we simply want the solution – consider the 8-
queens).
• If the path to the goal does not matter, we might consider a
class of algorithms that don’t concern themselves with
paths at all.

• Local search algorithms operate using a single current

node (rather than multiple paths) and generally move only
to neighbors of that node.

• Typically, the paths followed by the search are not retained

(the benefit being a potentially substantial memory
savings).
 Local Search for
• Local search Optimization
– Keep track of single current state
– Move only to neighboring states
– Ignore paths

• Advantages:
– Use very little memory
– Can often find reasonable solutions in large or
infinite (continuous) state spaces.

• “Pure optimization” problems

– All states have an objective function
– Goal is to find state with max (or min) objective
value
– Does not quite fit into path-cost/goal-state
 Local Search for
Optimization
• In addition to finding goals, local search algorithms are
useful for solving pure optimization problems, in which
we aim to find the best state according to an objective
function.

• Nature, for example, provides an objective function –

reproductive fitness.

• To understand local search, we consider state-space

landscapes, as shown next.
 Optimizatio
n
• So what is optimization?

• Find the minimum or maximum of an objective

function (usually: given a set of constraints):
Local Search for
Optimization

State-Space Landscape
•If elevation corresponds with cost, then the
aim is to find the lowest valley – a global
minimum; if elevation corresponds to an
objective function, then the aim is to find
the highest peak – a global maximum.
 Hill-Climbing (Greedy Local
Search)
function HILL-CLIMBING( problem) return a state that is a local
maximum
input: problem, a problem
local variables: current, a node.
neighbor, a node.

current  MAKE-NODE(INITIAL-STATE[problem])
loop do
neighbor  a highest valued successor of current
if VALUE [neighbor] ≤ VALUE[current] then return STATE[current]
current  neighbor

Minimum version will reverse inequalities and

look for lowest valued successor
 Hill-
Climbing
• “A loop that continuously moves towards increasing value”
– terminates when a peak is reached
– Aka greedy local search
• Value can be either
– Objective function value
– Heuristic function value (minimized)

• Hill climbing does not look ahead of the immediate

neighbors
• Can randomly choose among the set of best successors
– if multiple have the best value

• “Climbing Mount Everest in a thick fog with

amnesia”
 Hill-Climbing (8-
queens)

• Need to convert to an optimization

problem!
 Hill-Climbing (8-
• State queens)
– All 8 queens on the board in some configuration

• Successor function
– move a single queen to another square in the same
column.

• Example of a heuristic function h(n):

– the number of pairs of queens that are
attacking each other
– (so we want to minimize this)
 Hill-Climbing (8-
queens)

• h = number of pairs of queens that are attacking each

other
• h = 17 for the above state
 Hill-Climbing (8-
queens)

• Randomly generated 8-queens starting states…

• 14% the time it solves the problem
• 86% of the time it get stuck at a local minimum

• However…
– Takes only 4 steps on average when it
succeeds
– And 3 on average when it gets stuck
 Hill Climbing
Drawbacks
• Local
maxima

• Plateau
s

• Diagonal
ridges 1
7
 Escaping Shoulders: Sideways
• If no downhill (uphill)Move
moves, allow sideways
moves in hope that algorithm can escape
– Need to place a limit on the possible number of sideways
moves to avoid infinite loops
• For 8-queens
– Now allow sideways moves with a limit of 100
– Raises percentage of problem instances solved from
14 to 94%

– However….
• 21 steps for every successful solution
• 64 for each failure
1
8
 Tabu
• Search
Prevent returning quickly to the same state
• Keep fixed length queue (“tabu list”)
• Add most recent state to queue; drop oldest
• Never make the step that is currently tabu’ed

• Properties:
– As the size of the tabu list grows, hill-climbing will
asymptotically become “non-redundant” (won’t look at the
same state twice)
– In practice, a reasonable sized tabu list (say 100 or so)
improves the performance of hill climbing in many
problems
 Escaping Shoulders/local Optima
Enforced Hill Climbing
• Perform breadth first search from a local
optima
– to find the next state with better h function

• Typically,
– prolonged periods of exhaustive search
– bridged by relatively quick periods of hill-
climbing

2
• Middle ground b/w local and systematic 0
 Hill-climbing: stochastic
• variations
Stochastic hill-climbing
– Random selection among the uphill moves.
– The selection probability can vary with the steepness of the uphill
move.

• To avoid getting stuck in local minima

– Random-walk hill-climbing
– Random-restart hill-climbing
– Hill-climbing with both

2
1
 Hill Climbing: stochastic
variations
When the state-space landscape has local
minima, any search that moves only in the
greedy direction cannot be complete

Random walk, on the other hand, is

asymptotically complete

Idea: Put random walk into greedy hill-

climbing
 Hill-climbing with random
• restarts
If at first you don’t succeed, try, try again!
• Different variations
– For each restart: run until termination vs. run for a fixed
time
– Run a fixed number of restarts or run indefinitely

• Analysis
– Say each search has probability p of success
• E.g., for 8-queens, p = 0.14 with no sideways moves

– Expected number of restarts?

– Expected number of steps taken?

• If you want to pick one local search algorithm, learn this

 Hill-climbing with random
• At each step do one ofwalk
the two
– Greedy: With prob p move to the neighbor with largest
value
– Random: With prob 1-p move to a random neighbor

Hill-climbing with both

• At each step do one of the three
– Greedy: move to the neighbor with largest value
– Random Walk: move to a random neighbor
– Random Restart: Resample a new current state
 Simulated
Annealing
• Simulated Annealing = physics inspired twist on random walk
• Basic ideas:
– like hill-climbing identify the quality of the local improvements
– instead of picking the best move, pick one randomly
– say the change in objective function is d
– if d is positive, then move to that state
– otherwise:
• move to this state with probability proportional to d
• thus: worse moves (very large negative d) are executed less
often
– however, there is always a chance of escaping from local maxima
– over time, make it less likely to accept locally bad moves
– (Can also make the size of the move random as well, i.e., allow
 Physical Interpretation of Simulated
• A Physical Analogy:Annealing
• imagine letting a ball roll downhill on the function surface
– this is like hill-climbing (for minimization)
• now imagine shaking the surface, while the ball rolls,
gradually reducing the amount of shaking
– this is like simulated annealing

• Annealing = physical process of cooling a liquid or

metal until particles achieve a certain frozen crystal
state
• simulated annealing:
– free variables are like particles
– seek “low energy” (high quality) configuration
– slowly reducing temp. T with particles moving around randomly
Simulated
Annealing
 Best-First 5

Search

● Idea: use an evaluation function for each node

– estimate of “desirability”

⇒ Expand most desirable unexpanded node

● Implementation:
fringe is a queue sorted in decreasing order of desirability

● Special cases
– greedy search
– A∗ search

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 Roman 6

ia

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 Roman 7

ia

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 Greedy 8

Search

● State evaluation function h(n) (heuristic)

= estimate of cost from n to the closest goal

● E.g., hSLD(n) = straight-line distance from n to Bucharest

● Greedy search expands the node that appears to be closest to goal

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 Romania with Step Costs 9

in km

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 Greedy Search 10

Example

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 Greedy Search 11

Example

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 Greedy Search 12

Example

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 Greedy Search 13

Example

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 Properties of Greedy 14

Search
● Complete? No, can get stuck in loops, e.g., with Oradea as goal,
Iasi → Neamt → Iasi → Neamt →

Complete in finite space with repeated-state checking

●Time? O(bm), but a good heuristic can give dramatic improvement

●Space? O(bm)—keeps all nodes in memory

●Optimal? No Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 15

a* search

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 A∗ 16

Search
● Idea: avoid expanding paths that are already expensive

● State evaluation function f (n) = g(n) + h(n)

– g(n) = cost so far to reach n
– h(n) = estimated cost to goal from n
– f (n) = estimated total cost of path through n to goal

● A∗ search uses an admissible heuristic

– i.e., h(n) ≤ h∗(n) where h∗(n) is the true cost from n
– also require h(n) ≥ 0, so h(G) = 0 for any goal G

● E.g., hSLD(n) never overestimates the actual road distance

● Theorem: A∗ search is optimal

Artificial Intelligence: Informed 15 February
Search 2024

Philipp Koehn
 A∗ Search 17

Example

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 A∗ Search 18

Example

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 A∗ Search 19

Example

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 A∗ Search 20

Example

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 A∗ Search 21

Example

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 A∗ Search 22

Example

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 A∗ Search 23

Example

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 A∗ Search 24

Example

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 A∗ Search 25

Example

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 A∗ Search 26

Example

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 A∗ Search 27

Example

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn
 Optimality of 28

A∗
● Suppose some suboptimal goal G2 has been generated and is in the queue

● Let n be an unexpanded node on a shortest path to an optimal goal G

since h(G )2 = 0
f (G2) =
>g(G) since G2 is suboptimal
g(G2) ≥ f (n) since h is admissible

● Since f (G2) > f (n), A∗Artificial

will never terminate
Intelligence: at G2
Informed 15 February
Search 2024

Philipp Koehn
 Properties of 29

A∗

● Complete? Yes, unless there are infinitely many nodes with f ≤ f (G)

● Time? Exponential in [relative error in h × length of solution]

● Space? Keeps all nodes in memory

● Optimal? Yes—cannot expand fi+1 until fi is finished A∗ expands all

nodes with f (n) < C∗

A∗ expands some nodes with f (n) = C∗
A∗ expands no nodes with f (n) > C∗

Artificial Intelligence: Informed 15 February

Search 2024

Philipp Koehn