CS1410 – Design and Analysis of Algorithms

CS1410 DESIGN AND ANALYSIS OF ALGORITHMS

UNIT V BACKTRACKING,BRANCH AND BOUND AND APPROXIMATION ALGORITHMS

P, NP and NP-Complete Problems--Backtracking –n-Queens problem –Hamiltonian Circuit

Problem –Subset Sum Problem-Branch and Bound –Assignment problem –Knapsack Problem –
Traveling Salesman Problem-Approximation Algorithms for the Traveling Salesman problem
and the Knapsack problem

5.1 LIMITATIONS OF ALGORITHM POWER

Some problems cannot be solved by any algorithm. Other problems can be solved
algorithmically but not in polynomial time. And even when a problem can be solved in
polynomial time by some algorithms, there are usually lower bounds on their efficiency
 We start with methods for obtaining lower bounds, which are estimates on a minimum
amount of work needed to solve a problem. In general, obtaining a nontrivial lower
bound even for a simple-sounding problem is a very difficult task.
 Decision trees allows us, among other applications, to establish lower bounds on the
efficiency of comparison-based algorithms for sorting and for searching in sorted
arrays. As a result, we will be able to answer such questions as whether it is possible to
invent a faster sorting algorithm than mergesort and whether binary search is the
fastest algorithm for searching in a sorted array.
 Intractability: which problems can and cannot be solved in polynomial time. This well-
developed area of theoretical computer science is called computational complexity
theory. We present the basic elements of this theory and discuss informally such
fundamental notions as P, NP, and NP-complete problems, including the most important
unresolved question of theoretical computer science about the relationship between P
and NP problems.

5.2 LOWER BOUND ARGUMENTS

Lower bound: an estimate on a minimum amount of work needed to solve a given problem
Examples:
 Number of comparisons needed to find the largest element in a set of n numbers
 Number of comparisons needed to sort an array of size n
 Number of comparisons necessary for searching in a sorted array
 Number of multiplications needed to multiply two n-by-n matrices
 Lower bound can be
o an exact count
o an efficiency class ()
 Tight lower bound: there exists an algorithm with the same efficiency as the lower
bound

Problem Lower bound Tightness

sorting (comparison- (nlog n) yes
based)
searching in a sorted (log n) yes
array
element uniqueness (nlog n) yes
n-digit integer (n) unknown
multiplication
multiplication of n-by-n (n2) unknown
matrices

Methods for Establishing Lower Bounds

 Trivial lower bounds
 Information-theoretic arguments (decision trees)
 Adversary arguments
 Problem reduction

St. Joseph’s College of Engineering Page 1 of 20

CS1410 – Design and Analysis of Algorithms
5.2.1 Trivial Lower Bounds
Trivial lower bounds: based on counting the number of items that must be processed in input
and generated as output
Examples
 Finding max element -- n steps or n/2 comparisons
 Polynomial evaluation
 Sorting
 Element uniqueness
 Hamiltonian circuit existence
Conclusions
 May and may not be useful
 Be careful in deciding how many elements must be processed

5.2.2 Decision trees

Decision tree — A convenient model of algorithms involving Comparisons in which:
 Internal nodes represent comparisons
 Leaves represent outcomes (or input cases)
Decision tree for 3-element insertion sort

Decision Trees and Sorting Algorithms

 Any comparison-based sorting algorithm can be represented by a decision tree (for each
fixed n)
 Number of leaves (outcomes) n!
 Height of binary tree with n! leaves log2n!
 Minimum number of comparisons in the worst case log2n! for any comparison-based
sorting algorithm, since the longest path represents the worst case and its length is the
height
 log2n!n log2n (by Sterling approximation)
 This lower bound is tight (mergesort or heapsort)

Ex. Prove that 5 (or 7) comparisons are necessary and sufficient for sorting 4 keys (or 5 keys,
respectively).

5.2.3 Adversary Arguments

Adversary argument: It’s a game between the adversary and the (unknown) algorithm. The
adversary has the input and the algorithm asks questions to the adversary about the input.
The adversary tries to make the algorithm work the hardest by adjusting the input
(consistently). It wins the “game” after the lower bound time (lower bound proven) if it is able
to come up with two different inputs.
Example 1: “Guessing” a number between 1 and n using yes/no questions (Is it larger
than x?)

Adversary: Puts the number in a larger of the two subsets generated by last question

St. Joseph’s College of Engineering Page 2 of 20

CS1410 – Design and Analysis of Algorithms

Example 2: Merging two sorted lists of size n

a1<a2< … <an and b1<b2< … <bn

Adversary: Keep the ordering b1<a1<b2<a2< … <bn<an in mind and answer comparisons
consistently

Claim: Any algorithm requires at least 2n-1 comparisons to output the above ordering
(because it has to compare each pair of adjacent elements in the ordering)
Ex: Design an adversary to prove that finding the smallest element in a set of n
elements requires at least n-1 comparisons.

5.2.4 Lower Bounds by Problem Reduction

Fact: If problem Q can be “reduced” to problem P, then Q is at least as easy as P, or
equivalent, P is at least as hard as Q.

Reduction from Q to P: Design an algorithm for Q using an algorithm for P as a subroutine.

Idea: If problem P is at least as hard as problem Q, then a lower bound for Q is also a lower
bound for P.
Hence, find problem Q with a known lower boundthat can be reduced to problem P in
question.
Example: P is finding MST for n points in Cartesian plane, and Q is element uniqueness
problem (known to be in (nlogn))

Reduction from Q to P: Given a set X = {x1, …, xn} of numbers (i.e. an instance of the
uniqueness problem), we form an instance of MST in the Cartesian plane: Y = {(0,x1), …,
(0,xn)}. Then, from an MST for Y we can easily (i.e. in linear time) determine if the elements
in X are unique.

Classifying Problem Complexity

Is the problem tractable,i.e., is there a polynomial-time (O(p(n)) algorithm that solves it?
Possible answers:
 yes (give example polynomial time algorithms)
 no
o because it’s been proved that no algorithm exists at all (e.g., Turing’s halting
problem)
o because it’s been be proved that any algorithm for it would require exponential
time
 unknown. How to classify their (relative) complexity using reduction?

Problem Types: Optimization and Decision

 Optimization problem: find a solution that maximizes or minimizes some objective
function
 Decision problem: answer yes/no to a question
Many problems have decision and optimization versions.
E.g.: traveling salesman problem
 optimization: find Hamiltonian cycle of minimum length
 decision: find Hamiltonian cycle of length  L
Decision problems are more convenient for formal investigation of their complexity.
Class P
P: the class of decision problems that are solvable in O(p(n)) time, where p(n) is a polynomial
of problem’s input size n
Examples:
 Searching
 Element uniqueness
 Graph connectivity
 Graph acyclicity
 Primality testing (finally proved in 2002)

St. Joseph’s College of Engineering Page 3 of 20

CS1410 – Design and Analysis of Algorithms
Class NP
NP (nondeterministic polynomial): class of decision problems whose proposed solutions can
be verified in polynomial time = solvable by a nondeterministicpolynomial algorithm
A nondeterministic polynomial algorithm is an abstract two-stage procedure that:
 Generates a solution of the problem (on some input) by guessing
 Checks whether this solution is correct in polynomial time
By definition, it solves the problem if it’s capable of generating and verifying a solution on one
of its tries
Why this definition?
 led to development of the rich theory called “computational complexity”

Example: CNF satisfiability

Problem: Is a boolean expression in its conjunctive normal form (CNF) satisfiable, i.e., are
there values of its variables that make it true?
This problem is in NP. Nondeterministic algorithm:
 Guess a truth assignment
 Substitute the values into the CNF formula to see if it evaluates to true
Example: (A | ¬B | ¬C) & (A | B) & (¬B | ¬D | E) & (¬D | ¬E)
Truth assignments:
ABCDE
0 0 0 0 0
. . .
1 1 1 1 1
Checking phase: O(n)
What other problems are in NP?
 Hamiltonian circuit existence
 Partition problem: Is it possible to partition a set of n integers into two disjoint subsets
with the same sum?
 Decision versions of TSP, knapsack problem, graph coloring, and many other
combinatorial optimization problems.
 All the problems in P can also be solved in this manner (but no guessing is necessary),
so we have:
 P NP
 Big (million dollar) question: P = NP ?
NP-Complete Problems
A decision problem D is NP-complete if it is as hard as any problem in NP, i.e.,
 D is in NP
 every problem in NP is polynomial-time reducible to D

Cook’s theorem (1971): CNF-sat is NP-complete

Other NP-complete problems obtained through polynomial- time reductions from a known NP-
complete problem

St. Joseph’s College of Engineering Page 4 of 20

CS1410 – Design and Analysis of Algorithms

Examples: TSP, knapsack, partition, graph-coloring and hundreds of other problems of

combinatorial nature

P = NP ?Dilemma Revisited
 P = NP would imply that every problem in NP, including all NP-complete
problems,could be solved in polynomial time
 If a polynomial-time algorithm for just one NP-complete problem is discovered, then
every problem in NP can be solved in polynomial time, i.e. P = NP

 Most but not all researchers believe that P NP , i.e.P is a proper subset of NP. If P 
NP, then the NP-completeproblems are not in P, although many of them are very
useful in practice.

5.3 COPING WITH THE LIMITATIONS

Backtracking & Branch-and-Bound are the two algorithm design techniques for solving
problems in which the number of choices grows at least exponentially with their instance size.
Both techniques construct a solution one component at a time trying to terminate the process
as soon as one can ascertain that no solution can be obtained as a result of the choices
already made. This approach makes it possible to solve many large instances of NP-hard
problems in an acceptable amount of time.
Both Backtracking and branch-and-bound uses a state-space tree-a rooted tree whose
nodes represent partially constructed solutions to the problem. Both techniques terminate a
node as soon as it can be guaranteed that no solution to the problem can be obtained by
considering choices that correspond to the node’s descendants.

5.4 BACKTRACKING

Backtracking constructs its state-space tree in the depth-first search fashion in the
majority of its applications. If the sequence of choices represented by a current node of the
state-space tree can be developed further without violating the problem’s constraints, it is
done by considering the first remaining legitimate option for the next component. Otherwise,
the method backtracks by undoing the last component of the partially built solution and
replaces it by the next alternative.
A node in a state-space tree is said to be promising if it corresponds to a partially

St. Joseph’s College of Engineering Page 5 of 20

CS1410 – Design and Analysis of Algorithms
constructed solution that may still lead to a complete solution; otherwise it is called non
promising. Leaves represent either nonpromising dead ends or complete solutions found by
the algorithms.

5.5 N – QUEENS PROBLEM

 N – Queens problem is solved by using backtracking.
 The problem is to place n queens on n * n chess board.
 The following constraints are used to place queens on the chess board.
o No two queens should be placed on the same diagonal
o No two queens should be placed on the same column
o No two queens should be placed on the same row
2 Queens Problem The 2 - Queens problem is not solvable. Because 2-Queens can be
placed on 2 * 2 chessboard as

Q Q Q
Q

Illegal Illegal

Q Q
Q Q

Illegal Illegal
There is no solution when the value of n = 1 , 2 and 3
4 - Queens Problem
 The solution for the 4 – Queens’s problem is easily obtained using the backtracking
algorithm.
 The aim of the problem is to place 4 – Queens on the chessboard in such a way that
none of the queens hit each other.
Solving Procedure
Step 1:
 Let us assume the queens are placed in the row wise. The 1 st Q is placed I 1st row at
(1,1)
Q1

Step 2:
 The second Queen has to be placed in 2 nd row. It is not possible to place the Q 2 at the
following places.
(2,1) – placing the queens in the same column
(2,2) – placing the queens in the same diagonal.
So the queen can be placed in (2,3)
Q1
Q2

St. Joseph’s College of Engineering Page 6 of 20

CS1410 – Design and Analysis of Algorithms

Step 3:
 The third Queen has to be placed in 3 rd row. It is not possible to place the Q 3 at the
following places.
(3,1) – placing the queens in the same column
(3,2) – placing the queens in the same diagonal.
(3,3) – placing the queens in the same column
(3,4) – placing the queens in the same diagonal.

Q1
Q2

X X X X

Backtracking to the previous solution

 So this tells that Q3 can not be placed at 3rd row and it will not given a solution. So
backtracking to the previous step ie) step 2. So instead of placing the Q 2 at (2,3), Q2
can be placed in (2,4).
Q1
Q2

Step 4:
 The third Queen has to be placed in 3 rd row. It is not possible to place the Q 3 at the
following places.
(3,1) – placing the queens in the same column
(3,3) – placing the queens in the same diagonal.
(3,4) – placing the queens in the same column
So the queen has to be placed in (3,2)
Q1
Q2

Q3

Step 5:
 The fourth Queen has to be placed in 4th row. It is not possible to place the Q 4 at the
following places.
(4,1) – placing the queens in the same column
(4,2) – placing the queens in the same column
(4,3) – placing the queens in the same diagonal.
(4,4) – placing the queens in the same column

Q1
Q2

Q3

X X X X
Backtracking to the previous solution

St. Joseph’s College of Engineering Page 7 of 20

CS1410 – Design and Analysis of Algorithms
 So this tells that Q4 can not be placed at 4th row and it will not given a solution. So
backtracking to the previous step ie) step 4. So instead of placing the Q 3 at (3,2), Q3
can be placed in (3,3) (or) (3,4), that is also not possible because it comes in a same
diagonal and same column. So again backtracking to the previous step of step 4 ie)
step3 and do the same process. Finally Q 3 is placed in (3,1)
Q1
Q2

Q3

Step 6:

Q1
Q2

Q3

Q4

Basic Terminologies
Solution Space
 Tuples that satisfy the constraints
 The solution space can be organized into a tree
State Space
 State Space is the set of all paths from root node to other nodes.
State Space Tree
 State Space Tree is the tree organization of the solution space.
 In backtracking technique while solving a given problem a tree is constructed based on
the choices made.
 Such a tree with all possible solutions is called State Space Tree.
Promising and Non-Promising Node
 A node in State Space Tree is said to be promising
 The node which are not promising for solution in a state space tree is called non –
promising node.
Q1= 1 Q1= 3
1
Q1= 2

2 6 Q2 = 4Q2 = 1 1
Q2 = 3

3 4 7 1
Not a Q3 = 2 Q3 = 1Q4 = 2
Solution

5Not a 8 Q4 = 3Q4 = 2 1
Solution

9 1
Solution Solution

St. Joseph’s College of Engineering Page 8 of 20

CS1410 – Design and Analysis of Algorithms
8 – Queens Problem
 The solution for the 8 Queens problem is easily obtained using backtracking algorithm.
 The aim of the problem is to place 8 queens on 8 * 8 chessboard in such a way that
none of queens hit each other.
 The following constraints are used to place queens on the chess board.
 No two queens should be placed on the same diagonal
 No two queens should be placed on the same column
 No two queens should be placed on the same row
 Let us assume two queens are placed at positions ( i , j ) and ( k , l ) then, the two
queens are said to be in same diagonal, if the following conditions are satisfied.
i+j=k+1
i–j=k–1
 The above two equations can be rearranged as follows
i -k = 1-j
i–k=j–1
 Combining these two equations
Abs ( i – k ) = Abs ( j - 1 )
 If this condition is true, then the queens are in the same diagonal.
 In any stage, we are unable to place a queen then we have to backtrack and change
the position of the previous queen.
 This is repeated till we place all the 8 queens on the board.
Algorithm
Algorithm Nqueens(k,n)
Begin
for i=1 to n do
if place(k,i) then
x[k] = I
if ( k = n) then
write (x[1:n])
else
Nqueens(k+1,n)
end for
End
Algorithm place(k,i)
Begin
for j=1 to k-1 do
if x[j] = i
return false
else
return true
end for
End
Time complexity of 8 Queen problem is O(k-1)

Q1

Q1

St. Joseph’s College of Engineering Page 9 of 20

CS1410 – Design and Analysis of Algorithms
Q2
Q3
Q4
Q5
Q6
Q7
Q8

1 Q1= 1
Q1= 2 Q1= 3

2
Q2 = 3 Q 29= 4 1 Q2 = 6

3 1
1 Q3 = 2
Q3 = 6 Q3 = 1
Q4 = 7 1 Q4 = 3
4 1
Q4 = 3

Q5 = 1 5 Q5 = 7 Q5 = 5
1 1

Q6 = 2
Not a Q6 = 2 Solution
6 1 1

Q7 = 4 Q7 = 8

7 1
Not a
Q8 = 5 Solution

8 2
Solution
Analysis
Time complexity of 8 Queen’s problem is O ( k – 1 )
2
5.6 SUM OF SUBSETS
 The sum of subsets is solved using the backtracking method.
Problem Definition
 Given n distinct positive numbers (weights) W i, 1 ≤ i ≤ n, and m, find all subsets of the
Wi whose sum is m.

St. Joseph’s College of Engineering Page 10 of 20

CS1410 – Design and Analysis of Algorithms
 The element Xi of the solution vector is either one (or) zero depending on whether the
weight Wi is included or not.
Xi = 1 , Wi is included
Xi = 0 , Wi is not included
Solution to Sum of Subsets Problem
 Sort the weights in ascending order.
 The root of the space tree represents the starting point, with no decision about the
given elements.
 The left and right child represents inclusion and exclusion of X i in a set.
 The node to be expanded, check it with the following condition.
k
Σ W i Xi + Wi +1 ≤ m
i=1
 The bounded node can be identified with the following condition.
 The choice for the bounding function is Bk (X 1,……… Xk ) =true iff
k n
Σ W i Xi + Σ Wi ≥m
i=1 i=k+1
 Backtrack the bounded node and find alternative solution.
 Thus a path from the root to a node on the ith level of the tree indicates which of the
first i numbers have been included in the subsets represented by that node.
 We can terminate the node as non promising if either of the 2
ingratiation holds
S’ + Wi +1 > m (where S’ in too large)
S’ + Wi +1 < m (where S’ in too small)
k
S’ = Σ W i Xi
i=1
Constraints
Implicit Constraints
 No two elements in the subset can be same.
Explicit Constraints
 The tuple values can be any of the value between ‘1’ and ‘n’ and needed to be
ascending order.
Procedure
 Let ‘S’ be a set of elements and ‘m’ be the expected sum of subsets.
Step 1 Start with an empty set.
Step 2 Add to the subset, the next element from the list
Step 3 If the subset is having sum ‘m’ then stop with that subset as the solution.
Step 4 If the subset if not feasible (or) if we have reached the end of the set then
backtrack through the subset until we find the most suitable value.
Step 5 If the subset is feasible, then repeat step 2.
Step 6 If we have visited, all the elements without finding a suitable subset and no
backtracking is possible then stop without solution.
Example
Consider a set S = {3, 5, 6, 7} m=15. Solve it for obtaining sum of subsets?

Subset Sum Action

3 3 < 15 Add next element

3, 5 8 < 15 Add next element
3, 5, 6 14 < 15 Add next element
3, 5, 6, 7 21 > 15 Backtrack ( sum exceeds )
Condition satisfies. Subset =
3,5,7 15 = 15
sum. Solution obtained

St. Joseph’s College of Engineering Page 11 of 20

CS1410 – Design and Analysis of Algorithms
Solution set X = { 1, 1, 0, 1 }
Algorithm
Algorithm sumofsubset(s,m,n)
Begin
solution := 0
for i <- 1 to n
solution <- solution + Wi
if ( solution > m )
solution <- solution – Wi
Xi <- 0
end if
Xi <- 1
end for
write “ The tuples X[1………n]
End

5.7 HAMILTONIAN CIRCUIT PROBLEM

Consider the problem of finding a Hamiltonian circuit in the graph in Figure a. Without
loss of generality, we can assume that if a Hamiltonian circuit exists, it starts at vertex a.
accordingly, we make vertex a the root of the state-space tree (Figure b). The first component
of our future solution, if it exists, is a first intermediate vertex of a Hamiltonian circuit to be
constructed. Using the alphabet order to break the three-way tie among the vertices adjacent
to a, we select vertex b. From b, the algorithm proceeds to c, then to d, then to e, and finally
to f, which proves to be a dead end. So the algorithm backtracks from f to e, then to d, and
then to c, which provides the first alternative for the algorithm to pursue. Going from c to e
eventually proves useless, and the algorithm has to backtrack from e to c and then to b. From
there, it goes to the vertices f , e, c, and d, from which it can legitimately return to a, yielding
the Hamiltonian circuit a, b, f , e, c, d, a. If we wanted to find another Hamiltonian circuit, we
could continue this process by backtracking from the leaf of the solution found.

5.8 BRANCH-AND-BOUND

It is an algorithm design technique that enhances the idea of generating a state-space

tree with the idea of estimating the best value obtainable from a current node of the decision
tree: if such an estimate is not superior to the best solution seen up to that point in the
processing, the node is eliminated from further consideration.
A feasible solution, is a point in the problem’s search space that satisfies all the
problem’s constraints, while an optimal solution is a feasible solution with the best value of
the objective function compared to backtracking branch-and-bound requires 2 additional
items:

St. Joseph’s College of Engineering Page 12 of 20

CS1410 – Design and Analysis of Algorithms
 A way to provide, for every node of a state-space tree a bound on the best value of the
objective function on any solution that can be obtained by adding further components
to the partial solution represented by the node.
 The best value of the best solution seen so far.
If this information is available, we can compare a node’s bound with the value of the best
solution seen so far: if the bound value is not better than the best solution seen so far- i.e.,
not smaller for a minimization problem and not larger for a maximization problem- the node is
nonpromising and can be terminated because no solution obtained from it can yield a better
solution than the one already available.
In general, we terminate a search path at the current node in a state-space tree of a
branch & bound algorithm for any one of the following three reasons:
 The value of the node’s bound is not better than the value of the best solution seen so
far.
 The node represents no feasible solutions because the constraints of the problem are
already violated.
 The subset of feasible solutions represented by the node consists of a simple point.
Compare the value of the objective function for this feasible solution with that of the
best solution seen so far and update the latter with the former if the new solution is
better.

5.9 ASSIGNMENT PROBLEM

It is a problem where each job will be assigned to each person. And no 2 jobs can be
assigned to same person and no 2 person should be assigned with the same job.
Select one element in each row of the cost matrix C so that:
• no two selected elements are in the same column
• the sum is minimized

Example
Job 1 Job 2 Job 3 Job 4
Person a 9 2 7 8
Person b 6 4 3 7
Person c 5 8 1 8
Person d 7 6 9 4

Complete state-space tree for the instance of the assignment problem solved with the best-
first branch-and-bound algorithm (Figure).

St. Joseph’s College of Engineering Page 13 of 20

CS1410 – Design and Analysis of Algorithms

5.10 KNAPSACK PROBLEM

First order the items of a given instance in descending order by their value-to-weight
ratios. Then the first item gives the best payoff per weight unit and the last one gives the
worst payoff per weight unit, with ties resolved arbitrarily:
v1/w1 ≥ v2/w2 ≥ . . . ≥ vn/wn.
Each node on the ith level of this tree, 0 ≤ i ≤ n, represents all the subsets of n items that
include a particular selection made from the first i ordered items. This particular selection is
uniquely determined by the path from the root to the node:
 Branch going to the left indicates the inclusion of the next item, and
 Branch going to the right indicates its exclusion.
We record the total weight w and the total value v of this selection in the node, along with
some upper bound ub on the value of any subset that can be obtained by adding zero or more
items to this selection.
ub = v + (W − w)(vi+1/wi+1).

Example: Consider the following knapsack problem

Item weight value value/weight
1 4 $40 10
2 7 $42 6
3 5 $25 5
4 3 $12 4

5.11 TRAVELING SALESPERSONS PROBLEM

Lower bound can be obtained by finding the smallest element in the intercity distance
matrix D and multiplying it by the number of cities n. we can compute a lower bound on the
length l of any tour as follows. For each city i, 1≤ i ≤ n, find the sum si of the distances from
city i to the two nearest cities; compute the sum s of these n numbers, divide the result by 2,
and, if all the distances are integers, round up the result to the nearest integer:
lb = ceiling function(s/2).

St. Joseph’s College of Engineering Page 14 of 20

CS1410 – Design and Analysis of Algorithms
Example:

5.13 APPROXIMATION ALGORITHMS FOR TRAVELING SALESMAN PROBLEM

Nearest-neighbor algorithm

The following well-known greedy algorithm is based on the nearest-neighbor heuristic:

always go next to the nearest unvisited city.
 Step 1 Choose an arbitrary city as the start.
 Step 2 Repeat the following operation until all the cities have been visited: go to the
unvisited city nearest the one visited last (ties can be broken arbitrarily).
 Step 3 Return to the starting city.

Example 1 For the instance represented by the graph in Figure, with a as the starting vertex,
the nearest-neighbor algorithm yields the tour (Hamiltonian circuit) sa: a − b − c − d − a of
length 10.

St. Joseph’s College of Engineering Page 15 of 20

CS1410 – Design and Analysis of Algorithms

Multifragment-heuristic algorithm

 Step 1 Sort the edges in increasing order of their weights. (Ties can be broken
arbitrarily.) Initialize the set of tour edges to be constructed to the empty set.
 Step 2 Repeat this step n times, where n is the number of cities in the instance being
solved: add the next edge on the sorted edge list to the set of tour edges, provided this
addition does not create a vertex of degree 3 or a cycle of length less than n;
otherwise, skip the edge.
 Step 3 Return the set of tour edges.

Example 2 As an example, applying the algorithm to the graph in Figure yields {(a, b), (c, d),
(b, c), (a, d)}. This set of edges forms the same tour as the one produced by the nearest-
neighbor algorithm.

In general, the multifragment-heuristic algorithm tends to produce significantly better

tours than the nearest-neighbor algorithm, but the performance ratio of the multifragment-
heuristic algorithm is also unbounded.

Twice-around-the-tree algorithm

 Stage 1: Construct a minimum spanning tree of the graph (e.g., by Prim’s or Kruskal’s
algorithm)
 Stage 2: Starting at an arbitrary vertex, create a path that goes twice
around the tree and returns to the same vertex
 Stage 3: Create a tour from the circuit constructed in Stage 2 by making shortcuts to
avoid visiting intermediate vertices more than once

Example 3:

St. Joseph’s College of Engineering Page 16 of 20

CS1410 – Design and Analysis of Algorithms

Total length tour=39

Christofides Algorithm
It also uses a minimum spanning tree but does this in a more sophisticated way than
the twice-around-the-tree algorithm. The Christofides algorithm obtains a multigraph by
adding to the graph the edges of a minimum-weight matching of all the odd-degree vertices
in its minimum spanning tree. (The number of such vertices is always even and hence this can
always be done.) Then the algorithm finds an Eulerian circuit in the multigraph and transforms
it into a Hamiltonian circuit by shortcuts, exactly the same way it is done in the last step of
the twice-around-the tree algorithm.

Example 4 Let us trace the Christofides algorithm in Figure a. The graph’s minimum
spanning tree is shown in Figure b. It has four odd-degree vertices: a, b, c, and e. The
minimum-weight matching of these four vertices consists of edges (a, b) and (c, e). (For this
tiny instance, it can be found easily by comparing the total weights of just three alternatives:
(a, b) and (c, e), (a, c) and (b, e), (a, e) and (b, c).) The traversal of the multigraph, starting at
vertex a, produces the Eulerian circuit a − b − c − e − d − b − a, which, after one shortcut,
yields the tour a − b − c − e − d − a of length 37.

5.14 APPROXIMATION ALGORITHMS FOR THE KNAPSACK PROBLEM

The knapsack problem, another well-known NP-hard problem: given n items of known
weights w1. . . wn and values v1, . . . , vn and a knapsack of weight capacity W, find the most
valuable subset of the items that fits into the knapsack.

Greedy algorithm for the discrete knapsack problem

St. Joseph’s College of Engineering Page 17 of 20

CS1410 – Design and Analysis of Algorithms
 Step 1 Compute the value-to-weight ratios ri= vi/wi, i = 1. . . n, for the items given.
 Step 2 Sort the items in non-increasing order of the ratios computed in Step 1. (Ties
can be broken arbitrarily.)
 Step 3 Repeat the following operation until no item is left in the sorted list: if the
current item on the list fits into the knapsack, place it in the knapsack and proceed to
the next item; otherwise, just proceed to the next item.

Example: consider the instance of the knapsack problem with the knapsack capacity 10 and
the item information as follows:
Item weight value
1 7 $42
2 3 $12
3 4 $40
4 5 $25
Computing the value-to-weight ratios and sorting the items in nonincreasing order of these
efficiency ratios yields
Item weight value value/weight
1 4 $40 10
2 7 $42 6
3 5 $25 5
4 3 $12 4

The greedy algorithm will select the first item of weight 4, skip the next item of weight
7, select the next item of weight 5, and skip the last item of weight 3. The solution obtained
happens to be optimal for this instance

Greedy algorithm for the continuous knapsack problem

 Step 1 Compute the value-to-weight ratios vi/wi, i = 1. . . n, for the items given.
 Step 2 Sort the items in non-increasing order of the ratios computed in Step 1. (Ties
can be broken arbitrarily.)
 Step 3 Repeat the following operation until the knapsack is filled to its full capacity or
no item is left in the sorted list: if the current item on the list fits into the knapsack in
its entirety, take it and proceed to the next item; otherwise, take its largest fraction to
fill the knapsack to its full capacity and stop.

Example: consider the instance of the knapsack problem with the knapsack capacity 10 and
the item information as follows:
Item weight value
1 7 $42
2 3 $12
3 4 $40
4 5 $25
Computing the value-to-weight ratios and sorting the items in nonincreasing order of these
efficiency ratios yields
Item weight value value/weight
1 4 $40 10
2 7 $42 6
3 5 $25 5
4 3 $12 4

The algorithm will take the first item of weight 4 and then 6/7 of the next item on the
sorted list to fill the knapsack to its full capacity. This algorithm always yields an optimal
solution to the continuous knapsack problem. The items are ordered according to their
efficiency in using the knapsack’s capacity. If the first item on the sorted list has weight w1
and value v1, no solution can use w1 units of capacity with a higher payoff than v1. If we
cannot fill the knapsack with the first item or its fraction, we should continue by taking as
much as we can of the second most efficient item, and so on.

UNIT V UNIVERSITY QUESTION:

St. Joseph’s College of Engineering Page 18 of 20

CS1410 – Design and Analysis of Algorithms
PART-A
1. Define P and NP problems. (Dec 2018)
2. Give an example for sum of subset problem. (Dec 2018)
3. How is lower bound by problem reduction? (May 2018)
4. What are tractable and non-tractable problems? (May 2018)
5. Explain promising and non promising node. (Dec 2017)
6. Differentiate feasible solution and optimal solution. (Dec 2017)
7. Define P and NP problems. (May 2017)
8. What is articulation point in a graph? (May 2017)
9. State: Planar coloring graph problem. (May 2017)
10. Write the formula for decision tree for searching a sorted array (Dec 2016)
11. State the reason for terminating search path at the current node in a branch and bound
algorithm. (Dec 2016)
12. Give the purpose of lower bound (May 2016)
13. What is Euclidean minimum spanning tree problem? (May 2016)
14. What is state space graph? (May 2016)
15. Draw the decision tree for comparison of three values. (Dec 2015)
16. How NP-hard problems are different from NP-complete (May 2015)
17. Define Hamiltonian circuit problem. (May 2015)
PART-B
1. Consider the travelling salesman problem instance defined by the following cost matrix.
Draw the state space tree and show the reduced matrix corresponding to each node. (Dec
2018)

[ ]
∝ 20 30 10 11
15 ∝ 16 4 2
3 5 ∝ 2 4
19 6 18 ∝ 3
16 4 7 16 ∝
2. Discuss the approximation algorithm for NP-hard problems. (Dec 2018)
3. Explain the 4-Queens problem using backtracking. Write the algorithms. Give the estimated
cost for all possible solutions of 4-queens problem. Specify the implicit and explicit
constraints. (Dec 2018) (Dec 2016)
4. What is class NP? Discuss about any five problems for which no polynomial time algorithm
has been found. (May 2018)
5. Elaborate on the nearest-neighbor algorithm and multiragment-heiristic algorithm for TSP
problem. (May 2018)
6. Apply branch and bound algorithm to solve the travelling salesman problem for the
following: (May 2017)

7. Give the methods for establishing lower bounds (Dec 2017)

8. Find the optimal solution using branch and bound for the following assignment problem.
(Dec 2017)
Job 1 Job Job Job
2 3 4
A 9 2 7 8
B 6 4 3 7
C 5 8 1 8
D 7 6 9 4

9. Find a Hamiltonian circuit or disprove its existence in the graph given below. (Dec 2017)

St. Joseph’s College of Engineering Page 19 of 20

CS1410 – Design and Analysis of Algorithms

10. Describe the backtracking solution to solve 8-queens problem. (May 2017)
11. Describe the approximation algorithm for NP-hard problems. (May 2017)
12. Solve the following instance of knapsack problem by branch and bound algorithm. (Dec
2016)
Item weight value
1 5 $40
2 7 $35
3 2 $18
4 4 $4
5 5 $10
6 1 $2
Capacity W = 15.
13. State the subset sum problem and complete the state space tree of the backtracking
algorithm applied to the instance A = {3, 5, 6, 7} and d =15 of the subset sum problem. (May
2016)
14. State the relationships among the complexity class algorithms with the help of neat
diagram. (Dec 2015)
15. What is approximation algorithm? Give example. (Dec 2015)
16. Suggest an approximation algorithm for travelling salesman problem. Assume that the
cost function satisfies the triangle inequality. (May 2015)
17. Explain how job assignment problem could be solved, given n tasks and n agents where
each agent has a cost to complete each task, using branch and bound technique. (May 2015)

St. Joseph’s College of Engineering Page 20 of 20