DIGITAL NOTES

ON
DESIGN & ANALYSIS OF
ALGORITHMS
Unit-V

Prepared by,

Dr. S. Ravichandran

Associate Professor/CSE,

GITAM University,

Hyderabad.
 CSEN3001 - DESIGN AND ANALYSIS OF ALGORITHMS

UNIT 5 Branch and Bound & Algebraic 9 hours,P - 6

Problems hours
Branch and Bound: The method, traveling sales person problem, 0/1
knapsack problem,
efficiency considerations.
Algebraic Problems: The general method, Evaluation and Interpolation.

General Method:
Branch and bound, or BnB, is an algorithm design paradigm that
solves combinatorial and discrete optimization problems. Many
optimization issues, such as crew scheduling, network flow problems,
and production planning, cannot be solved in polynomial time. Hence,
BnB is a paradigm that is widely used to solve such problems.

Advantages and Disadvantages

Let us understand the advantages and disadvantages of BnB through the points
below:

Advantages
 Time Complexity: The BnB algorithm does not explore all nodes in the tree.
Thereby, the time complexity is significantly lesser than most other algorithms.
 Optimal Solution: If the branching is done reasonably, the algorithm can find the
optimal solution in a reasonable period.
 Clear Pattern: The BnB algorithm does not repeat the notes to explore the tree for
the candidate solutions; instead, it follows a minimal path to derive the optimal
solution.

Disadvantages
 Time-consuming: Based on the size of the problem, the number of nodes that are
computed might be too large in a worst-case scenario, making it a time-consuming
process.
 Parallelization: The branching out of possible solutions provides scope for
speculative parallelism. However, when alternative actions are considered for the
said action, the branch and bound calculator finds difficulty

Traveling Salesperson problem using

branch and bound
Given the vertices, the problem here is that we have to travel each vertex exactly once and reach back to
the starting point. Consider the below graph:
As we can observe in the above graph that there are 5 vertices given in the graph. We have to find the
shortest path that goes through all the vertices once and returns back to the starting vertex. We mainly
consider the starting vertex as 1, then traverse through the vertices 2, 3, 4, and 5, and finally return to
vertex 1.

The adjacent matrix of the problem is given below:

Now we look at how this problem can be solved using the branch n bound

Let's first understand the approach then we solve the above problem.

The graph is given below, which has four vertices:

Suppose we start travelling from vertex 1 and return back to vertex 1. There are various ways to travel
through all the vertices and returns to vertex 1. We require some tools that can be used to minimize the
overall cost. To solve this problem, we make a state space tree. From the starting vertex 1, we can go to
either vertices 2, 3, or 4, as shown in the below diagram.
From vertex 2, we can go either to vertex 3 or 4. If we consider vertex 3, we move to the remaining
vertex, i.e., 4. If we consider the vertex 4 shown in the below diagram:

From vertex 3, we can go to the remaining vertices, i.e., 2 or 4. If we consider the vertex 2, then we move
to remaining vertex 4, and if we consider the vertex 4 then we move to the remaining vertex, i.e., 3
shown in the below diagram:
From vertex 4, we can go to the remaining vertices, i.e., 2 or 3. If we consider vertex 2, then we move to
the remaining vertex, i.e., 3, and if we consider the vertex 3, then we move to the remaining vertex, i.e., 2
shown in the below diagram:

The above is the complete state space tree. The state space tree shows all the possibilities. Backtracking
and branch n bound both use the state space tree, but their approach to solve the problem is different.
Branch n bound is a better approach than backtracking as it is more efficient. In order to solve the
problem using branch n bound, we use a level order. First, we will observe in which order, the nodes are
generated. While creating the node, we will calculate the cost of the node simultaneously. If we find the
cost of any node greater than the upper bound, we will remove that node. So, in this case, we will
generate only useful nodes but not all the nodes.
Let's consider the above problem.

As we can observe in the above adjacent matrix that 10 is the minimum value in the first row, 2 is the
minimum value in the second row, 2 is the minimum value in the third row, 3 is the minimum value in
the third row, 3 is the minimum value in the fourth row, and 4 is the minimum value in the fifth row.

Now, we will reduce the matrix. We will subtract the minimum value with all the elements of a row. First,
we evaluate the first row. Let's assume two variables, i.e., i and j, where 'i' represents the rows, and 'j'
represents the columns.

When i = 0, j =0

M[0][0] = ∞-10= ∞
When i = 0, j = 1

M[0][1] = 20 - 10 = 10

When i = 0, j = 2

M[0][2] = 30 - 10 = 20

When i = 0, j = 3

M[0][3] = 10 - 10 = 0

When i = 0, j = 4

M[0][4] = 11 - 10 = 1

The matrix is shown below after the evaluation of the first row:

Consider the second row.

When i = 1, j =0

M[1][0] = 15-2= 13

When i = 1, j = 1

M[1][1] = ∞ - 2= ∞

When i = 1, j = 2

M[1][2] = 16 - 2 = 14

When i = 1, j = 3
M[1][3] = 4 - 2 = 2

When i = 1, j = 4

M[1][4] = 2 - 2 = 0

The matrix is shown below after the evaluation of the second row:

Consider the third row:

When i = 2, j =0

M[2][0] = 3-2= 1

When i = 2, j = 1

M[2][1] = 5 - 2= 3

When i = 2, j = 2

M[2][2] = ∞ - 2 = ∞

When i = 2, j = 3

M[2][3] = 2 - 2 = 0

When i = 2, j = 4

M[2][4] = 4 - 2 = 2

The matrix is shown below after the evaluation of the third row:

Consider the fourth row:

When i = 3, j =0

M[3][0] = 19-3= 16

When i = 3, j = 1

M[3][1] = 6 - 3= 3

When i = 3, j = 2

M[3][2] = 18 - 3 = 15

When i = 3, j = 3

M[3][3] = ∞ - 3 = ∞

When i = 3, j = 4

M[3][4] = 3 - 3 = 0

The matrix is shown below after the evaluation of the fourth row:

Consider the fifth row:

When i = 4, j =0

M[4][0] = 16-4= 12

When i = 4, j = 1

M[4][1] = 4 - 4= 0

When i = 4, j = 2
M[4][2] = 7 - 4 = 3

When i = 4, j = 3

M[4][3] = 16 - 4 = 12

When i = 4, j = 4

M[4][4] = ∞ - 4 = ∞

The matrix is shown below after the evaluation of the fifth row:

The above matrix is the reduced matrix with respect to the rows.

Now we reduce the matrix with respect to the columns. Before reducing the matrix, we first find the
minimum value of all the columns. The minimum value of first column is 1, the minimum value of the
second column is 0, the minimum value of the third column is 3, the minimum value of the fourth
column is 0, and the minimum value of the fifth column is 0, as shown in the below matrix:

Now we reduce the matrix.

Consider the first column.

When i = 0, j =0

M[0][0] = ∞-1= ∞

When i = 1, j = 0

M[1][0] = 13 - 1= 12

When i = 2, j = 0
M[2][0] = 1 - 1 = 0

When i = 3, j = 0

M[3][0] = 16 - 1 = 15

When i = 4, j = 0

M[4][0] = 12 - 1 = 11

The matrix is shown below after the evaluation of the first column:

Since the minimum value of the first and the third columns is non-zero, we will evaluate only first and
third columns. We have evaluated the first column. Now we will evaluate the third column.

Consider the third column.

When i = 0, j =2

M[0][2] = 20-3= 17

When i = 1, j = 2

M[1][2] = 13 - 1= 12

When i = 2, j = 2

M[2][2] = 1 - 1 = 0

When i = 3, j = 2

M[3][2] = 16 - 1 = 15
When i = 4, j = 2

M[4][2] = 12 - 1 = 11

The matrix is shown below after the evaluation of the third column:

he above is the reduced matrix. The minimum value of rows is 21, and the columns is 4. Therefore, the
total minimum value is (21 + 4) equals to 25.

Let's understand that how to solve this problem using branch and bound with the help of a state-
space tree.

To make a state-space tree, first, we consider node 1. From node 1, we can go either to nodes 2, 3, 4, or
5 as shown in the below image. The cost of node 1 would be the cost which we achieved in the above-
reduced matrix, i.e.., 25. Here, we will also maintain the upper bound. Initially, the upper bound would-
be infinity.

The path of the tour would be 1->4->2->5->3.

efficiency considerations
The efficiency of a branch-and-bound algorithm A (c) is measured in th T(c): the
number of nodes decomposed in A6 prior to halt in A9

What is Branch and bound?

Branch and bound is one of the techniques used for problem solving. It is similar to the backtracking
since it also uses the state space tree. It is used for solving the optimization problems and minimization
problems. If we have given a maximization problem, then we can convert it using the Branch and bound
technique by simply converting the problem into a maximization problem.

Branch and bound algorithms are used to find the optimal solution for combinatory,
discrete, and general mathematical optimization problems. In general, given an NP-
Hard problem, a branch and bound algorithm explores the entire search space of possible
solutions and provides an optimal solution.
A branch and bound algorithm consist of stepwise enumeration of possible candidate
solutions by exploring the entire search space. With all the possible solutions, we first build
a rooted decision tree. The root node represents the entire search space:

Branch and bound work efficiently on the combinatory optimization problems. Given
an objective function for an optimization problem, combinatory optimization is a process to
find the maxima or minima for the objective function. The domain of the objective function
should be discrete and large
 0-1 Knapsack problem using Branch and Bound
he objective of the 0/1 Knapsack problem is to find a subset of objects such that the total
value is maximized, and the sum of weights of the objects does not exceed a given
threshold W. An important condition here is that one can either take the entire object or
leave it. It is not possible to take a fraction of the object.

Fox example,

Now, consider the 0/1 knapsack problem with n objects and total weight W. We define the
upper bound(U) to be the summation of v ixi (where vi denotes the value of that objects, and
xi is a binary value, which indicates whether the object is to be included or not), such that
the total weights of the included objects is less than W. The initial value of U is calculated at
the initial position, where objects are added in order until the initial position is filled.
Consider the problem with n =4, V = {10, 10, 12, 18}, w = {2, 4, 6, 9} and W = 15. Here, we
calculate the initital upper bound to be U = 10 + 10 + 12 = 32. Note that the 4th object
cannot be included here, since that would exceed W. For the cost, we add 3/9 th of the final
value, and hence the cost function is 38. Remember to negate the values after calculation
before comparison.

Time Complexity: O(2n)

Example: Consider the instance M=15, n=4, (p1, p2, p3, p4) = 10, 10, 12, 18 and (w1, w2, w3, w4)=(2, 4, 6, 9).

Solution: knapsack problem can be solved by using branch and bound technique. In this problem we will
calculate lower bound and upper bound for each node.
Arrange the item profits and weights with respect of profit by weight ratio. After that, place the first item
in the knapsack. Remaining weight of knapsack is 15-2=13. Place next item w2 in knapsack and the remaining
weight of knapsack is 13-4=9. Place next item w 3, in knapsack then the remaining weight of knapsack is 9-6=3.
No fraction are allowed in calculation of upper bound so w4, cannot be placed in knapsack.

Profit= p1+p2+ p3,=10+10+12 So, Upper

bound=32

To calculate Lower bound we can place w4 in knapsack since fractions are allowed in calculation of lower
bound.

Lower bound=10+10+12+ (3/9*18)=32+6=38

Knapsack is maximization problem but branch bound technique is applicable for only minimization
problems. In order to convert maximization problem into minimization problem we have to take negative sign
for upper bound and lower bound.

Therefore, upper bound (U) =-32 Lower bound

(L)=-38

We choose the path, which has minimized difference of upper bound and lower bound. If the difference is
equal then we choose the path by comparing upper bounds and we discard node with maximum upper bound.
Now we will calculate upper bound and lower bound for nodes 2, 3

For node 2, x1=1, means we should place first item in the knapsack.
U=10+10+12=32, make it as -32
L=10+10+12+ (3/9*18) = 32+6=38, we make it as -38
For node 3, x1=0, means we should not place first item in the knapsack.
U=10+12=22, make it as -22
L=10+12+ (5/9*18) = 10+12+10=32, we make it as -32

Next we will calculate difference of upper bound and lower bound for nodes 2, 3 For node 2, U-
L=-32+38=6
For node 3, U-L=-22+32=10
Choose node 2, since it has minimum difference value of 6.

Now we will calculate lower bound and upper bound of node 4 and 5. Calculate difference of lower and upper
bound of nodes 4 and 5.
For node 4, U-L=-32+38=6 For node 5, U-
L=-22+36=14
Choose node 4, since it has minimum difference value of 6
Now we will calculate lower bound and upper bound of node 6 and 7. Calculate difference of lower and upper
bound of nodes 6 and 7.
For node 6, U-L=-32+38=6 For node 7,
U-L=-38+38=0
Choose node 7, since it has minimum difference value of 0.

Now we will calculate lower bound and upper bound of node 8 and 9. Calculate difference of lower and upper
bound of nodes 8 and 9.
For node 8, U-L=-38+38=0 For node 9,
U-L=-20+20=0
Here, the difference is same, so compare upper bounds of nodes 8 and 9. Discard the node, which has
maximum upper bound. Choose node 8, discard node 9 since, it has maximum upper bound.
Consider the path from 12478
X1=1 X2=1 X3=0
X4=1
The solution for 0/1 knapsack problem is ( x1, x2, x3, x4)=(1, 1, 0, 1)
Maximum profit is:
∑pixi=10*1+10*1+12*0+18*1 10+10+18=38