Backtracking algorithm

Lecturer: Musika Fredrick

Backtracking algorithm
 Backtracking search is an exhaustive search algorithm
(depth-first search) that systematically assigns all possible
 combinations of values to the variables and checks if these
assignments constitute a solution
o It is a modified depth first search of a tree,
which determines problem solutions by
systematically searching the solution space for
the given problem instance.
o It involves forming a solution and checking at
every step if this has any chance of success.
 If the solution at any point seems not
promising, that solution is ignored
o That is if the algorithms determines that a node
can lead to nothing or dead end,
 we go back (backtrack) to the nodes
parent and proceed with the search on the
next child.
o Backtracking search is complete in the sense that it is
guaranteed to find a solution if one exists.
o On the downside, the worst-case running time of
backtracking search is exponential.

Terminology:

o Problem state is each node in the depth first search

tree.
o Solution states are the problem states ‘S’ for which
the path from the root node to ‘S’ defines a tuple in
the solution space.
o Answer states are those solution states for which
the path from root node to s defines a tuple that is a
member of the set of solutions.
o State space is the set of paths from root node to
other nodes
o Live node is a node that has been generated but
whose children have not yet been generated.
 o E-node is a live node whose children are currently
being explored. In other words, an E-node is a node
currently being expanded.
o Dead node is a generated node that is not to be
expanded or explored any further. All children of a
dead node have already been expanded.
o Branch and Bound refers to all state space search
methods in which all children of an E-node are
generated before any other live node can become the
E-node.
Application
 Backtracking algorithm is applied to some specific types of
problems,
o Decision problem used to find a feasible solution of the
problem.
o Optimisation problem used to find the best solution that
can be applied.
o Enumeration problem used to find the set of all feasible
solutions of the problem.

ALGORITHM
 Step 1 − if current_position is goal, return success
 Step 2 − else,
 Step 3− if current_position is an end point, return failed.
 Step 4− else, if current_position is not end point, explore and repeat above steps.

Example 1
N-Queen problem

o In N-Queen problem, we are given an NxN chessboard and

we have to place n queens on the board in such a way that
no two queens attack each other.
o A queen will attack another queen if it is placed in
horizontal, vertical or diagonal points in its way.
ALGORITHM
 Step 1 − Start from 1st position in the array.
 Step 2 − Place queens in the board and check.
o Do,After placing the queen, mark the position as a part of the
solution and then recursively check if this will lead to a
solution.
o Now, if placing the queen doesn’t lead to a solution, trackback
and go to step (1) and place queens to other rows.
o If placing queen returns a lead to solution return TRUE.
 Step 3 − If all queens are placed return TRUE.
 Step 4 − If all rows are tried and no solution is found, return FALSE

Solution

{0 , 1 , 0 , 0}
{0 , 0 , 0 , 1}
{1 , 0 , 0 , 0}
{0 , 0 , 1 , 0}

Example 2

Traveling Sale Person (TSP) using Backtracking:

o Consider the graph shown below with 4 vertices.

o We will assume that the starting node is 1 and the ending

node is obviously 1.
 Then 1, {2, … ,4}, 1 forms a tour with some cost
which should be minimum.
 The vertices shown as {2, 3, …. ,4} forms a
permutation of vertices which constitutes a tour
 We can also start from any vertex, but the tour should
end with the same vertex.
o Using the depth first order we take the tour
1. {1, 2, 3, 4, 1} with cost 30 + 5 + 20 + 4 = 59.
This the best tour so far

2. We back track and then take the tour {1, 2, 4, 3, 1}

with cost 30 + 10 + 20 + 6 = 66 and is discarded,
as the best tour so far is 59.
3. This is repeated for all the other combination and
the best tour is saved.
The optimal tour cost is found to be 25
Example 3
4 – Queens Problem
in 4- queens problem, we have 4 queens to be placed on a 4*4 chessboard,
satisfying the constraint that no two queens should be in the same row, same
column, or in same diagonal.

o The above figure shows graphically the steps that the

backtracking algorithm goes through as it tries to find a
solution.
o The dots indicate placements of a queen, which were tried
and rejected because another queen was attacking.
o In Figure (b) the second queen is placed on columns 1 and
2 and finally settles on column 3.
o In figure (c) the algorithm tries all four columns and is
unable to place the next queen on a square.
o Backtracking now takes place.
o In figure (d) the second queen is moved to the next
possible column, column 4 and the third queen is placed on
column 2.
 o The boards in Figure (e), (f), (g), and (h) show the
remaining steps that the algorithm goes through until a
solution is found.

Example 4

Graph Coloring (m-colorabiltiy decision problem)

o Let G be a graph and m be a given positive integer. We

want to discover whether the nodes of G can be colored in
such a way that no two adjacent nodes have the same
color, yet only m colors are used
o It has been found that 4 colors can be used to color any
graph and hence a solution to the above problem

Solution
 Thank you

End

Branch and Bound algorithm

Lecturer: Musika Fredrick

Branch and Bound algorithm
 Branch-and-bound is a general technique for improving the
searching process by systematically enumerating all
candidate solutions and disposing of obviously impossible
solutions.
 Branch-and-bound usually applies to those problems that
have finite solutions, in which the solutions can be
represented as a sequence of options.
 Just like backtracking, we will use bounding
functions to avoid generating subtrees that do not
contain an answer node.
 The first part of branch-and-bound, branching, requires
several choices to be made so that the choices branch out into
the solution space.
  In these methods, the solution space is organized as
a treelike structure.
 Figure below shows an instance of TSP and a solution
tree, which is constructed by making choices on the
next cities to visit.

 Branching out to all possible choices guarantees that no

potential solutions will be left uncovered
 The branch-and-bound algorithm handles this problem
by bounding and pruning.
o Bounding refers to setting a bound on the solution quality
(e.g., the route length for TSP),
o and pruning means trimming off branches in the solution
tree whose solution quality is estimated to be poor
 Bounding and pruning are the essential concepts of the
branch-and-bound technique, because they are used to
effectively reduce the search space-hence reduced
complexity
 The number under a leaf node of the solution tree represents
the length of the corresponding route.
 For incomplete branches, an expression in the form of a + b is
shown
o a -is the length of the traversed edges, and
o b- is a lower bound for the length of the remaining route
that has not been explore(minimum spanning tree)

The sum of these two numbers provides the basis for

bounding

.
o If the sum a + b is over or equal to the current upper
bound, the solutions on that branch guarantees to be
worse than the current best solution, and the branch can
be pruned.
 For example, for the unfinished route A→B→E=7
and a minimum spanning tree is built for nodes A, C, D, and E,
and its value is =12 and a+b=19
o Every time a partial route is extended by a vertex, a lower
bound for the length of the rest of the route is computed
o In the case of A→B→C→D→E→A the, the upper bound
is 21
 o If we consider the next route as of A→B→C→E→D→A
the new upper bound is 15 and the route of 21 is pruned
and 15 is taken and the currents best value
 In this graph, most of the nodes will be pruned since
a+b is greater than 15.
 In summary, in using an exhaustive search, a total of 89
nodes will be searched
o but with branch-and-bound only 20 node will be searched
 Branch-and-bound accelerates the search process by reducing
the solution space en masse.
o Although branch-and-bound algorithms generally do not
possess proven time complexity, their efficiency has made
them the first choice for many problems, especially for
NP-complete problems.
 Branch-and-bound mainly addresses optimization problems,
because bounding is often based on numerical comparisons
Thank you

End