Design and Analysis of Algorithms Module 5 BCS402

MODULE 5
BACKTRACKING
Syllabus:

Backtracking: N-Queen’s Problem, Sum of Subset Problem. Branch-and-Bound. : Travelling Sales

Person problem,0/1 Knapsack problem NP and NP-Complete Problems : Basic concepts,
nondeterministic algorithms, P, NP, NP Complete, and NP-Hard classes

Backtracking: General method

Backtracking is a systematic method of searching for the solution to a combinatorial problem by means
of an algorithm. Problems that have exponential time complexity can be best solved using backtracking
method.

Principal idea of backtracking: is to construct solutions by selecting one component at a time and
evaluate such partially constructed candidates as follows:
• If a partially constructed solution can be developed further without violating the problem’s
constraints, it is done by taking the first remaining legitimate option for the next component
• If there is no legitimate option for the next component, then remaining alternate components need
not be considered. In this case the algorithm backtracks to replace the last component of the
partially constructed solution with its next option

The problems that can be solved using backtracking are:

N-Queens problem
Subset-Sum problem
Hamiltonian Circuit problem

The output of a backtracking algorithm can be expressed as n-tuple:

State: (x1, x2, x3, …xn)
The n-tuple (x1, x2, x3, …xn) is called as state/solution of the problem.
State Space: The set of all possible states is called state space (solution space). It is denoted by S.
State Space Tree: The tree corresponding to the state space is called state space tree which is normally
constructed using depth first search method.
• Its root represents an initial state before the search for a solution begins.
• The nodes of the first level in the tree represent the choices made for the first component of
solution; the nodes of the second level represent the choices for the second component and so on.
• A node in a state-space tree is said to be promising if it corresponds to a partially constructed
solution that may still lead to a complete solution otherwise it is called non-promising node.
• Leaves represent either non promising dead ends or complete solutions found by the algorithm.

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 1

Design and Analysis of Algorithms Module 5 BCS402

The following pseudo code gives the general form of a backtracking algorithm

Algorithm Backtrack(X[1…..i])
// Gives a template of a generic backtracking algorithm
// input: X[1…i] specifies first i promising components of a solution
// output: All the tuples representing the problems solutions

if X[1…i] is a solution write X[1…i]

else
for each element x ϵ Si+1 consistent with X[1….i] and the constraints do
X[i+1] x
Backtrack(X[1….i+1])

N-Queens problem
Definition: The problem is to place n queens on an n × n chessboard so that no two queens attack each
other by being in the same row or in the same column or on the same diagonal.
For n = 1, the problem has a trivial solution.
For n=2 and n=3, no solution.
For n=4, there are 2 solutions as follows:

Soln 1: (x1, x2, x3, x4)=(2, 4, 1, 3) Soln 2: (x1, x2, x3, x4)=(3, 1, 4, 2)

Solution: The four queen’s problem can be solved by starting with the empty board and then place queen
1 in the first possible position of its row, which is in column 1 of row 1. (1, 1)
To place queen 2 in 2nd row, the first acceptable position is column 3. Queen 2 cannot be placed in
column 1 (because of straight column attack from queen 1) and in column 2 (because of diagonal attack).
(2, 3)
This proves to be a dead end because there is no acceptable position for queen 3. So, the algorithm
backtracks and puts queen 2 in the next possible position at (2, 4). Then queen 3 is placed at (3, 2),
which proves to be another dead end. The algorithm then backtracks all the way to queen 1 and moves it
to (1, 2). Queen 2 then goes to (2, 4), queen 3 to (3, 1), and queen 4 to (4, 3), which is a solution to the
problem. The state-space tree of this search is shown below:

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 2

Design and Analysis of Algorithms Module 5 BCS402

Sum of subsets problem

Definition: Given a set of n positive integers S = {s1, . . . , sn}, we have to find a subset of S whose sum
is equal to a given positive integer ‘d’. Some instances of this problem may have no solutions.

For example: Solve the given subset problem. A = {1, 2, 5, 6, 8} and d = 9. There are two solutions: {1,
2, 6} and {1, 8}.

Solution: The set’s elements have to be sorted in increasing order.

S1 ≤ S2 ≤ S3……. ≤ Sn
The state-space tree can be constructed as a binary tree.

Problem 1: S= {3, 5, 6, 7} and d=15

The following gives the state space tree

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 3

Design and Analysis of Algorithms Module 5 BCS402

The root of the tree represents the starting point, with no decisions about the given elements made as yet.
Its left and right children represent, respectively, inclusion and exclusion of s1 in a set being sought.
Similarly, going to the left from a node of the first level corresponds to inclusion of s2 while going to the
right corresponds to its exclusion, and so on. Thus, a path from the root to a node on the ith level of the
tree indicates which of the first I numbers have been included in the subsets represented by that node.
We record the value of s’, the sum of these numbers, in the node. If s’ is equal to d, we have a solution to
the problem. If s’ is not equal to d, we can terminate the node as non-promising if either of the following
two inequalities holds:

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 4

Design and Analysis of Algorithms Module 5 BCS402

Branch and Bound

Branch and Bound is a method of systematically searching for a solution in state-space tree. It is used to
find the optimal solution. It consists of generating the partial solution by adding an item and discarding
the ones that cannot possibly lead to a feasible solution or optimal solution based on
• Lower bounds for minimization problems
• Upper bound for maximization problems.
The various problems that can be solved using branch and bound are:
• Knapsack problem
• Assignment problem
• Travelling Sales Person problem

Difference between Backtracking and Branch and Bound

Parameter Backtracking Branch and Bound

Problems Backtracking is used for solving Decision Branch-and-Bound is used to solve
solved Problem optimization problems
Approach Backtracking is used to find all possible When it realizes that it already has a better
solutions available to a problem. When it optimal solution that the pre-solution leads
realizes that it has made a bad choice, it to, it abandons that pre-solution. It
undoes the last choice by backing it up. It completely searches the state space tree to
searches the state space tree until it has get optimal solution.
found a solution for the problem.

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 5

Design and Analysis of Algorithms Module 5 BCS402

Traversal Backtracking traverses the state space tree Branch-and-Bound traverse the tree either
by DFS manner. in DFS or BFS.
Function Backtracking involves feasibility function. Branch-and-Bound involves a bounding
function.
Efficiency Backtracking is more efficient. Branch-and-Bound is less efficient.
Next move Next move from current state can lead to Next move is always towards better
bad choice. solution.
Solution On successful search of solution in state Entire state space tree is search in order to
space tree, search stops. find optimal solution.
Applications Useful in solving N-Queen Problem, Sum Useful in solving Knapsack Problem,
of subset, Hamilton cycle problem, graph Travelling Salesman Problem.
coloring problem

Travelling Salesman Problem

TSP is an example for minimization problem. So we need to calculate lower bound by the formula
lb(lower bound)= s/2 where ‘s’ is sum of two smallest distances from each city.

Problem 1: solve the following TSP by using Branch and Bound

Solution: The following points are considered to solve the problem:

• Tour starts at ‘a’
• Node ‘b’ is visited before node ‘c’.

Lower bound(lb) =
(a,b) lb=14
(a, d) lb= [ (1+5) + (3+6) + (1+2) + (5+3) + (2 +3) ] / 2 = 16
(a, e) lb = [ (1+8) + (3+6) + (1 + 2) + (3 + 4) + (2 + 8)] / 2 = 19
a, b, c lb = [(1+3) + (3+6) + (1 + 6) + (3 + 4) + (2 + 3)] / 2 = 16
a, b, d lb = [(1+3) + (3+7) + (1 + 2) + (3 + 7) + (2 + 3)] / 2 = 16
a, b, e lb = [(1+3) + (3+9) + (1 + 2) + (3 + 4) + (2 + 9)] / 2 = 19

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 6

Design and Analysis of Algorithms Module 5 BCS402

State space tree:

So tour = a → b→ d → e → c → a with the cost 3 + 7 + 3 + 2 + 1 = 16.

Problem 2: solve the following TSP by using Branch and Bound

Lower bound (lb) = [(7 + 10) + (2 + 5) + (5 + 11) + (2 + 7)] / 2 = 25

(a, b) = [(7 + 10) + (10 + 2) + (5 + 11) + (2 + 7)] / 2 = 27
(a, d) = [(7 + 10) + (5 + 2) + (5 + 11) + (2 + 7)] / 2 = 25

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 7

Design and Analysis of Algorithms Module 5 BCS402

State space tree

Path= a → d → b → c → a with the cost 7 + 2 + 5 + 17 = 31

0/1 Knapsack Problem

Branch and bound solution: W.K.T knapsack is a maximization problem. So our objective is to

maximize . This maximization can be achieved by minimizing – in branch and

bound approach.
The modified knapsack problem statement is as follows:

Note: Definitions:
Live node: A node which has been generated and whose children have not yet been generated.
E-node: The live node whose children are currently being generated is called the E-node.
Dead node: A node which is not expanded further.
c^(x): approximate cost used for computing the least cost c(x).
u(x): denotes the upper bond.
LC Branch and Bound solution
LC stands for Least Cost

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 8

Design and Analysis of Algorithms Module 5 BCS402

The steps to be followed are:

1. Draw the state space tree
2. Compute c^(.) and u(.) for each node
3. If c^(x) > upper kill node x.
4. Otherwise the minimum – u(.) becomes E-node. Generate children for E-node.
5. Repeat step 3 and 4 until all the nodes gets covered.

Problem 1: Consider the knapsack instance n=4, (p1, p2, p3, p4) = (10, 10, 12, 18), (w1, w2, w3, w4) =
(2, 4, 6, 9), and m=15.
Construct state space tree:
The c^(.) and u(.) of the first node is calculated as follows:
To get u(1), the function scans through the objects from left to right. It adds all objects into the knapsack
until the first object that doesn’t fit is encountered. So u(1) = Items 1, 2 and 3 are added to bag.

– = – (10 + 10 + 12)
u(1) = -32
Item 4 cannot be selected, since it exceeds the knapsack capacity.

= -32 – 3/9 * 18 = -38

With x1=1 (with item 1): u(2) and c^(2) = same as u(1) and c^(1)
u(2) = -32
c^(2) = -38
Without item 1: u(2) = -(10 +12) = -22
c^(2) = -22 – (15- (4 + 6))/9 * 18
= -22 – 5 * 2 = -22- 10
= -32

With item 2: Since item 2 is included in all previous steps, it is same as u(2) and c^(2).
u(2) = -32
c^(2) = -38
Without item 2: u(2) = -(10 + 12) = -22
c^(2) = -22 – (15- (2 + 6))/9 * 18
= -22 – 7 * 2 = -22- 14
= -36

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 9

Design and Analysis of Algorithms Module 5 BCS402

With item 3: Since item 2 is included in all previous steps, it is same as u(2) and c^(2).
u(2) = -32
c^(2) = -38
Without item 3: u(2) = -(10 + 10) = -20
c^(2) = -20 – (15- (2 + 4))/9 * 18
= -20 – 9 * 2 = -20- 18
= -38

With item 1, 2, 3 and 4. It is not feasible to add item 4 since it exceeds the maximum capacity of the
knapsack.
With item 1, 2, without item 3 and item 4 : u(2) = -(10 + 10 + 18) = -38
c^(2) = -20 – (15- (2 + 4))/9 * 18
= -20 – 9 * 2 = -20- 18
= -38

With item 1 and 2 and Without item 3 and 4: u(2) = -(10 + 10) = -20
c^(2) = -20 – (15- (2 + 4))/9 * 0
= -20 – 0 = -20
= -20

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 10

Design and Analysis of Algorithms Module 5 BCS402

Soln: node 8 gives the answer (node with minimum –u(.) value). We get maximum profit with item 1, 2,
and 4.
So the answer is (1, 1, 0, 1) with the maximum profit 38.

FIFO Branch and Bound solution

Same as above we need to calculate both c^(.) and u(.). But the node first generated will be expanded
first.

– = – (10 + 10 + 12)
u(1) = -32

Item 4 cannot be selected, since it exceeds the knapsack capacity.

= -32 – 3/9 * 18 = -38

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 11

Design and Analysis of Algorithms Module 5 BCS402

With x1=1 (with item 1): u(2) and c^(2) = same as u(1) and c^(1)
u(2) = -32
c^(2) = -38
Without item 1: u(2) = -(10 +12) = -22
c^(2) = -22 – (15- (4 + 6))/9 * 18
= -22 – 5 * 2 = -22- 10
= -32

With item 2: Since item 2 is included in all previous steps, it is same as u(2) and c^(2).
u(2) = -32
c^(2) = -38
Without item 2: u(2) = -(10 + 12) = -22
c^(2) = -22 – (15- (2 + 6))/9 * 18
= -22 – 7 * 2 = -22- 14
= -36

Without item 1 and with item 2:

u(2) = -(10 +12) = -22
c^(2) = -22 – (15- (4 + 6))/9 * 18
= -22 – 5 * 2 = -22- 10
= -32
Without item 1 and without item 2:
u(2) = -(12 +18) = -30
c^(2) = -30 – (15- (6 + 9))/9 * 18
= -30 – 0 = -30
= -30. KILL this node since c^(2) > u(2)

With item 1, 2, and With item 3: Since item 2 is included in all previous steps, it is same as u(2) and
c^(2).
u(2) = -32
c^(2) = -38

With item 1, 2 and Without item 3: u(2) = -(10 + 10) = -20

c^(2) = -20 – (15- (2 + 4))/9 * 18
= -20 – 9 * 2 = -20- 18
= -38

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 12

Design and Analysis of Algorithms Module 5 BCS402

With item 1, 2, 3 and 4: It is not feasible to add item 4 since it exceeds the maximum capacity of the
knapsack.
With item 1, 2, 3 and without 4: same as parent node
u(2) = -32
c^(2) = -38

With item 1, 2, without item 3 and item 4 : u(2) = -(10 + 10 + 18) = -38
c^(2) = -20 – (15- (2 + 4))/9 * 18
= -20 – 9 * 2 = -20- 18
= -38

With item 1 and 2 and Without item 3 and 4: u(2) = -(10 + 10) = -20
c^(2) = -20 – (15- (2 + 4))/9 * 0
= -20 – 0 = -20
= -20

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 13

Design and Analysis of Algorithms Module 5 BCS402

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 14

Design and Analysis of Algorithms Module 5 BCS402

Complexity Classes
In computer science, problems are divided into classes known as Complexity Classes. In complexity
theory, a Complexity Class is a set of problems with related complexity. Complexity classes are useful in
organizing similar types of problems. The problems set can be divided into following types
• Problems that cannot be solved by computers.
• Problems that can be efficiently solved (solved in Polynomial time) by computers.
• Problems for which no efficient solution (only non-polynomial (exponential time) algorithms)
exist.

The common resources required by computers to obtain solution are time and space i.e, how much time
the algorithm takes to solve a problem and the corresponding memory usage.
• The time complexity of an algorithm is used to describe the number of steps required to solve a
problem, but it can also be used to describe how long it takes to verify the answer.
• The space complexity of an algorithm describes how much memory is required for the algorithm
to operate.
• An algorithm having time complexity of the form O(nk) for input n and constant k is called
polynomial time solution. These solutions scale well. On the other hand, time complexity of the
form O(kn) is exponential time.

Types of Complexity Classes:

1. P –class Problems: P class stands for Polynomial Time. It is the collection of decision problems
(problems with a "yes" or "no" answer) that can be solved by a deterministic machine (computers) in
polynomial time. Problems whose solution times are bounded by polynomials of small degree.
Features:
• The solution to P problems is easy to find.
• P is often a class of computational problems that are solvable and tractable. Tractable means that
the problems can be solved in theory as well as in practice. But the problems that can be solved in
theory but not in practice are known as intractable.

Example: Ordered search O(n log n), sorting O(n log n), polynomial evaluation O(n), string editing
O(mn) etc.

2. NP-class Problems: NP class stands for Non-deterministic Polynomial Time. It is the

collection of decision problems that can be solved by a non-deterministic machine (note that
computers are deterministic) in polynomial time. Problems whose best- known algorithm are non-
polynomial.
Features:
• The solutions of the NP class might be hard to find since they are being solved by a non-
deterministic machine but the solutions are easy to verify. So they are Intractable Problems
• Problems of NP can be verified by a deterministic machine in polynomial time.

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 15

Design and Analysis of Algorithms Module 5 BCS402

• These problems requires vast amount of time to execute even for moderate values of n. Such
problems cannot be solved for large values of ‘n’.

Example: TSP O(n22n) , Knapsack O(2n/2)

Problems which don’t have polynomial time algorithm are computationally related and can be
classified as follows:

1. NP-Hard Problems: An NP-hard problem is at least as hard as the hardest problem in NP and it is
a class of problems such that every problem in NP reduces to NP-hard. If an NP-hard problem
can be solved in polynomial time, then all NP-complete problems can be solved in polynomial
time.
Features:
• All NP-hard problems are not in NP.
• It takes a long time to check them. This means if a solution for an NP-hard problem is given then
it takes a long time to check whether it is right or not.
• A problem A is in NP-hard if, for every problem L in NP, there exists a polynomial-time
reduction from L to A.
Some of the examples of problems in Np-hard are:
• Halting problem.
• Qualified Boolean formulas.
• No Hamiltonian cycle.

2. NP-complete Problems: A problem is NP-complete if it is both NP and NP-hard. NP-complete

problems are the hard problems in NP. A problem that is NP-complete has the property that it can
be solved in polynomial time if and only if all other NP-complete problems can also be solved in
polynomial time.
Features:
• NP-complete problems are special as any problem in NP class can be transformed or reduced into
NP-complete problems in polynomial time.
• If one could solve an NP-complete problem in polynomial time, then one could also solve any
NP problem in polynomial time.
Some example problems include:
• Hamiltonian Cycle.
• Satisfiability.
• Vertex cover.

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 16

Design and Analysis of Algorithms Module 5 BCS402

The relation between these problems are explained in the following diagram

Figure: Commonly believed relationship between P and NP

Figure: Commonly believed relationship among P, NP, NP-complete, and NP-hard problems

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 17

Design and Analysis of Algorithms Module 5 BCS402

Non-deterministic algorithms
Non deterministic algorithm contains operations whose outcomes are limited to specified sets of
possibilities. The following three functions are used:
1. Choice (S) arbitrarily chooses one of the elements of set S.
2. Failure () signals an unsuccessful completion.
3. Success () signals a successful completion.

Problem 1: Searching x on A[1 : n], n ≥ 1 on success returns j if A[j] = x or returns 0 otherwise

j= Choice (1 , n) ;
if (A[j] == x) then {write (j) ; Success(); }
write(0), Failure( );

Problem 2: Sorting array A[1 : n] of positive integers in ascending order

Algorithm NSort (A, n) // sort n positive integers
{
for i = 1 to n do B[i] = 0; // Initialize B[]
for i=1 to n do
{
j=Choice (1, n);
if ( B[j] ≠ 0 ) then Failure ( );
B[ j ] = A [ i ];
}

for i=1 to n-1 do // verify order

if (B[i] > B [i+1] ) then Failure( );
write (B[1: n]);
Success ( );
}

Manasa Sandeep, Dept. of CSE, DSATM 2024-25 Page 18