ULC403

Design and Analysis of Algorithms

Backtracking

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 1 / 48
Agenda

■ Backtracking: The general method

■ 8-queens problem using Backtracking
■ Sum of Subsets Problem using Backtracking
■ Graph coloring using Backtracking
■ 0/1 Knapsack problem using Backtracking

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 2 / 48
8-Queens Problem
■ Classical combinatorial problem: 8 queens problem is to place eight
queens on an 8 × 8 chessboard so that no two “attack”, that is,
• no two of them are on the same row, column, or diagonal.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 3 / 48
Representing the solution

■ Number the rows and columns of the chessboard 1 through 8.

■ The queens can also be numbered 1 through 8.

• Since each queen must be on a different row, queen i is to be
placed on row i.

■ All solutions to the 8-queens problem can be represented as 8-tuples

(x1 ,· · · , x8 ), where xi is the column of the ith row where the ith queen
is placed.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 4 / 48
Representing the solution

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 5 / 48
Explicit constraints

Definition. Explicit constraints (EC) are rules that restrict each xi to

take on values only from a given set.

■ It depends on the particular instance I of problem being solved.

■ All tuples that satisfy the EC define a possible solution space for I.
■ Explicit constraint for 8-queens problem are
• Si = {1,2,3,4,5,6,7,8}, 1 ≤ i ≤ 8
• The solution space consists of 88 8-tuples.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 6 / 48
Implicit constraints
Definition. Implicit constraints (IC) are rules that determine which of the
tuples in the solution space of I satisfy the criterion function.
■ ICs describe the way in which the xi ’s must relate to each other.
■ The implicit constraints for 8-queens problem are that no two xi ’s
can be the same (i.e., all queens must be on different columns) and no
two queens can be on the same diagonal.
• reduces the size of the solution space from 88 to 8! tuples

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 7 / 48
Backtracking

■ Backtracking
■ represents one of the most general techniques.

■ is used to solve problems in which a sequence of objects is chosen

from a specified set
3 so that the sequence (optimal solution) satisfies some criterion.

■ The desired solution is expressed as an n-tuple (x1 , . . . , xn ) where the

xi ’s are chosen from some finite set Si .

■ The solution is based on finding one or more vectors that maximize,

minimize, or satisfy a criterion function P (x1 , . . . , xn ).

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 8 / 48
Backtracking Algorithms

■ The brute-force approach : Form all these n-tuples, evaluate each one
with criteria function P , and save those which yield the optimum.

■ Backtracking algorithms solves problem by systematically searching the

solution space (SS) for the given problem instance.

■ This search is facilitated by using a tree organization for the solution

space.

■ n-queens problem (generalized version)

• n queens problem is to place eight queens on an n × n chessboard
so that no two “attack”, that is, no two of them are on the same
row, column, or diagonal.
• The solution space consists of n! permutations of the n tuple
(1, 2, 3, . . . , n).
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 9 / 48
Permutation Tree for 4-queens problem
■ Permutation tree
• Edges are labeled by possible values of xi .
• Edges from level i to level i+1 are labeled with the values of xi .
• Edges from level-1 to level-2 specify the values for x1 .

Figure: Building permutation tree. Nodes are numbered as in DFS (see next
slide) . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 10 / 48
Expanding solution space for x1 =1, incomplete

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Figure:
(DAA,Not all problem
Dr. Ashish Gupta) states shown (see: Unit-II
ULC403 next page for complete state space 11 / 48
Solution space organization

■ The solution space is defined by all paths from the root node to a leaf
node.
■ Total leaf nodes: 4!=24. .
.
.
.
.
. . . . .
. . . .
. . . .
. . . .
. . . .
. . . . .
.
.
.
.
.
.
.
.
.

(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 12 / 48

Backtracking Algorithms

■ The brute-force approach : form all these n-tuples, evaluate each one
with criteria function P , and save those which yield the optimum.

■ Backtracking algorithms determine problem solutions by systematically

searching the solution space (SS) for the given problem instance.
• Basic idea: Build up the solution vector one component at a time
and
• to use modified criterion functions (bounding functions) to test
whether the vector being formed has any chance of success.

■ Advantage: if it is realized that the partial vector can in no way

lead to an optimal solution, then remaining possible test vectors
(from that point) can be ignored entirely.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 13 / 48
Terminology: Tree organization of solution space

■ Problem state : each node in the tree defines a problem state.

■ All paths from the root to other nodes defines the state space of the
problem.
■ Solution states are those problem states ‘s’ for which the path from
the root node to ‘s’ defines a tuple in the solution space.
■ Answer states are those solution states ‘s’ for which the path from
root node to s defines a tuple that is a member of the set of solutions.
■ The tree organization of the SS is referred to as the state space tree.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 14 / 48
Figure: All nodes in red are solution states. One answer state is shown in green.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 15 / 48
Finding Answer nodes for any problem

■ Conceive a state space tree for the given problem.

• systematically generate the problem states

• determining which of these are solution states, and

• determining which solution states are answer states.

■ How to generate the problem states ?

• Depth first node generation (stack)

• Breath first node generation (queue)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 16 / 48
More Terminology

■ Live node is a node that has been generated but all of whose children
have not yet been generated.

■ E-node is a live node whose children are currently being explored /

generated. In other words, an E-node is a node currently being ex-
panded.

■ Dead node is a generated node that is not to be expanded further or

all children of a dead node have already been expanded.

■ Generation of the problem states

• we have a list of live nodes at any point in time based on how
nodes are generated.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 17 / 48
Depth first generation of problem states

■ Depth first generation of problem states

• As soon as a new child C of the current E-node R is generated,
this child becomes the new E-node.

• Then R will become the E-node again when the subtree C has
been fully explored.

■ The second method

• The E-node remains the E-node until it is dead.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 18 / 48
The difference

■ In both methods,bounding functions are

• used to kill live nodes without generating all their children.

• carefully applied/done to ensure the generation of answer nodes

as specified in the problem.

■ Backtracking: Depth first generation with bounding functions.

■ Branch and bound: State generation method in which the E-node

remains the E-node until it is dead leads to Branch-and-bound methods.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 19 / 48
Backtracking: 4-queens

■ Bounding function: if (x1 , x2 , . . . , xi ) is the path to the current E-

node, then all the children nodes with parent-child labelings xi+1 are
such that (x1 , x2 , . . . , xi+1 ) represents a chessboard configuration in
which no two queens are attacking.
■ We start with the root node as the only live node.
• This becomes the E-node and the path is ().
• We generate one child. Let us assume that the children are gen-
erated in ascending order.
• Thus node number 2 of figure is generated and the path is now
(1). Node 2 becomes the E-node.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 20 / 48
 Figure: Backtracking on 4-queens problem.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 21 / 48
4-queens problem

Figure: Portion of the permutation tree that is generated during backtracking

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 22 / 48
Backtracking

■ Assumption: all answer nodes are to be found.

■ Let (x1 , x2 , . . . , xi ) be a path from the root to a node in a state space
tree.
■ Let T (x1 , x2 , . . . , xi ) be the set of all possible values of xi+1 such that
(x1 , x2 , . . . , xi+1 ) is also a path to a problem state.
■ Assume the existence of bounding function Bi+1 such that if
Bi+1 (x1 , x2 , . . . , xi+1 ) is false for a path (x1 , x2 , . . . , xi+1 ), then the
path cannot be extended to reach an answer node.
■ Candidates for position i+1 of the n-tuple solution vector are those
values which
• are generated by T and
• satisfy Bi+1 .

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 23 / 48
Recursive backtracking algorithm

Figure: Initially invoked with Backtrack(1);

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 24 / 48
General iterative backtracking method

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 25 / 48
The 8-QUEENS PROBLEM
■ Solution vector: (x1 , x2 , . . . , x8 )
• All x′i s will be distinct since no queens can be placed in the same
column.
• How do we test whether 2 queens are on the same diagonal ?

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 26 / 48
 Figure: Two queens lies on the same diagonal iff |j − l| = |i − k|

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 27 / 48
All solutions to 8-Queens problem

Figure: How to code T and B ?

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 28 / 48
Can a new queen be placed?

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 29 / 48
Backtracking
SUM OF SUBSETS PROBLEM

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 30 / 48
Sum of subsets problem

■ Given n positive numbers wi , 1 ≤ i ≤ n, and a value m, this problem

requires finding all subsets of wi whose sums are m.
+ Example : n = 4, (w1 , w2 , w3 , w4 ) = (11, 13, 24, 7) and m = 31.
+ Possible solutions: (11, 13, 7) and (24, 7).

■ Representing solution : Let us represent the solution vector by giving

the indices of the wi .
• Solution/Answer vectors: (1,2,4) and (3,4). All solutions are k-
tuples (x1 , x2 ,· · · , xk ), 1 ≤ k ≤ n.
• Variable tuple size representation
• Different solutions may have different-sized tuples.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 31 / 48
Sum of subsets problem

■ Given n positive numbers wi , 1 ≤ i ≤ n, and a value m, this problem

requires finding all subsets of wi whose sums are m.
■ Constraints for sum of subsets (Variable tuple size representation)
• Explicit constraints: xi ∈ {j|j is an integer and 1 ≤ j ≤ n}.
• Implicit constraints:
• No two xi can be the same.
• The sum of the corresponding wi ’s be m.
• Avoid generating multiple instances of the same subset (for
example, (1, 2, 4) and (1, 4, 2): xi ≤ xi+1 , 1 ≤ i < k repre-
sent the same subset).

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 32 / 48
Sum of Subsets: SST for Variable tuple size

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 33 / 48
Fixed-tuple sized formulation
■ Each solution can be represented by an n-tuple (x1 , x2 , . . . , xn ) such
that xi ∈ {0,1} 1≤ i ≤ n.
• xi =0 if wi is not chosen and xi =1 if wi is chosen.
• Solutions: (1,1,0,1) and (0,0,1,1)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 34 / 48
Fixed-tuple sized formulation

Figure: D-search order

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 35 / 48
Sum of subsets using Backtracking
■ Representation: Fixed tuple size strategy.
■ Children of any node can be generated easily.
■ Bounding function: Bk (x1 , x2 , . . . , xk )= true iff

• if above is false, (x1 , x2 , . . . , xk ) cannot lead to an answer node.

■ What if (assume) the wi ’s are initially arranged in non-decreasing
order ?
■ Results in strengthen bounding function, (x1 , x2 , . . . , xk ) cannot lead
to an answer node if

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 36 / 48
Sum of subsets: Bounding function

■ The bounding function (for k th position / level) we use are therefore

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 37 / 48
Bounding function in action

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 38 / 48
Recursive backtracking: SoS

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 39 / 48
SoS: Recursive calls in backtracking

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 40 / 48
 . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
(DAA, Dr. Ashish Gupta) ULC403 : Unit-II 41 / 48