BRANCH AND BOUND

Dr. K Shahu Chatrapati

 Introduction
• Branch and bound is used to find optimal solution
to many optimization problems.
• It is a state space search method that
systematically enumerates all candidate solutions.
• Children of a node are generated before
expanding any of its children.
• Discards large subsets of fruitless candidates by
using upper and lower estimated bounds of
quantity being optimized.
 Definitions
• Live node is a node that has been generated
but whose children have not yet been
generated.
• E-node is a live node whose children are
currently being explored.
• Dead node is a generated node that is not to
be expanded or explored any further. All
children of a dead node have already been
expanded.
 General Method
• Both BFS and DFS generalize to branch-and-
bound strategies.
• BFS is a FIFO search in terms of live nodes. List
of nodes is a queue.
• DFS is a LIFO search in terms of live nodes. List
of nodes is a stack.
• Just like backtracking, we will use bounding
functions to avoid generating sub trees that do
not contain an answer node.
Backtracking Solution to 4 Queens Problem

Figure 1
Portion of the Tree Generated during
Backtracking

Figure 2
Portion of 4-Queens State Space Tree
Generated by FIFO Branch-and-Bound

Figure 3
 Problem with LIFO and FIFO Branch-and-
Bound
• In both LIFO and FIFO branch-and-bound the selection rule
for the next E-node is rather rigid and in a sense blind.
• The selection rule for the next E-node does not give any
preference to a node that has a very good chance of
getting the search to an answer node quickly.
• When node 30 is generated, it should have become
obvious to the search algorithm that this node will lead to
an answer node in one move.
• However, the rigid FIFO rule first requires the expansion of
all live nodes generated before node 30 was expanded.
 Least Cost (LC) Search
 Rank the live nodes by using a cost function 𝑐(.)
 The next E-node is selected on the basis of this ranking function
 Assign ranks based on the expected additional computational effort

 For any node 𝑥, the cost could be given by

(cost) to reach a solution from the current live node.

1. Number of nodes in sub tree 𝑥 that need to be generated before an

2. Number of levels to the nearest answer node in the sub tree 𝑥.

answer can be reached.

 Problem with the above techniques to compute the cost at node 𝑥 is “by the

there is no need to explore 𝑥 again.

time the cost of a node is determined, that sub tree has been searched and

𝑔ො(.) of their cost.

 Therefore, search algorithms usually rank nodes on the basis of an estimate

 Let 𝑔ො(𝑥) be an estimate of the addition effort needed to reach an answer node
from 𝑥. Node 𝑥 is assigned a rank using a function 𝑐Ƹ (.) defined as:
ሺ 𝑥ሻ = 𝑓൫ℎሺ 𝑥ሻ ൯ + 𝑔ො(𝑥)
𝑐Ƹ
where ℎሺ 𝑥ሻ is the cost of reaching 𝑥 from the root and 𝑓(.) is non decreasing
function.
 LC Branch-and-Bound Search

 A search strategy that uses a cost function 𝑐Ƹ ሺ 𝑥ሻ =

𝑓൫ℎሺ 𝑥ሻ ൯ + 𝑔 ሺ . ሻ is
ො(𝑥) to select the next E-node with least 𝑐Ƹ
called LC-search.

 In BFS we use 𝑔 ොሺ 𝑥ሻ ≡ 0 and 𝑓൫ℎሺ 𝑥ሻ ൯ as the level of

 Both BFS and DFS are special cases of LC-search.

node 𝑥.
 In DFS, we use 𝑓൫ℎሺ 𝑥ሻ ൯ ≡ 0 and 𝑔 ොሺ 𝑥ሻ ≥ 𝑔
ොሺ 𝑦ሻ whenever 𝑦
is a child of 𝑥.
 An LC-search coupled with bounding functions is called

 If 𝑥 is an answer node, cሺ 𝑥ሻ is the cost of reaching 𝑥

LC branch-and-bound search.

 If 𝑥 is not an answer node, cሺ 𝑥ሻ = ∞, provided the

from the root of the state space tree.

sub tree 𝑥 contains no answer node.

 If sub tree 𝑥 contains an answer node, cሺ 𝑥ሻ is the cost
of the minimum cost answer node in sub tree 𝑥.
Example 15-Puzzle Arrangements

Figure 4
Part of the State Space Tree for 15-Puzzle
Problem

Figure 5
First Ten Steps of Depth First Search

Figure 6
 An Intelligent Solution to 15-Puzzle Problem
 Associate a cost cሺ 𝑥ሻ with each node 𝑥 in the state space tree.

The cost cሺ 𝑥ሻ can be the length of a path from root to a nearest goal node (if
any) in the sub tree with root 𝑥.


 In Figure 6, 𝑐ሺ 1ሻ = 𝑐ሺ 4ሻ = 𝑐ሺ 10ሻ = 𝑐ሺ 23ሻ = 3

 Begin with root as E-node and generate a child node with 𝑐(.) value same as
the root node.

compute the function 𝑐(.) specified above.

 Unfortunately, this is an impractical strategy as it is not possible to easily

We can compute estimate 𝑐Ƹ ሺ 𝑥ሻ of 𝑐(𝑥) as 𝑐Ƹ

ሺ 𝑥ሻ = 𝑓൫ℎሺ 𝑥ሻ ൯ + 𝑔
ො(𝑥) where 𝑓ሺ 𝑥ሻ
is the length of the path from the root to node 𝑥 and 𝑔 ො(𝑥)is an estimate of the


length of a shortest path from 𝑥 to a goal node in the sub tree with root 𝑥.

 One possible choice for 𝑔

ො(𝑥) is the number of nonblank tiles not in their goal
position.
Figure 6
Control Abstraction for LC-Search
 Bounding

• A branch-and-bound method searches a state space tree using

any search mechanism in which all children of the E-node are

• Each answer node 𝑥 has a cost 𝑐ሺ 𝑥ሻ and we have to find a

generated before another node becomes the E-node.

minimum-cost answer node. Common strategies include LC,

• Use a cost function 𝑥 such that 𝑐Ƹ ሺ 𝑥ሻ ≤ 𝑐ሺ 𝑥ሻ provides lower

FIFO, and LIFO.

bound on the solution obtainable from any node 𝑥.

• If 𝑈 is the upper bound on the cost of a minimum-cost
ሺ 𝑥ሻ > 𝑈may be killed.
solution, then all live nodes 𝑥 with 𝑐Ƹ
• All answer nodes reachable from 𝑥 have cost 𝑐ሺ 𝑥ሻ ≥ 𝑐Ƹ ሺ 𝑥ሻ >
𝑈.
• Starting value for 𝑈can be obtained by some heuristic or set
to ∞.
• Each time a new answer node is found, the value of 𝑈 can be
updated.
 Bounding
(Optimization/Minimization Problem)
• A maximization problem can be easily converted to a minimization problem
by changing the sign of the objective function.
• Formulate the search for an optimal solution as a search for a least-cost

• Define the cost function 𝑐(.) such that 𝑐(𝑥) is minimum for all nodes
answer node in a state space search tree.

• The easiest way to do this is to use the objective function itself for 𝑐(.).
representing an optimal solution.

• For nodes representing feasible solutions, 𝑐(𝑥) is the value of the objective

• For nodes representing infeasible solutions, 𝑐ሺ 𝑥ሻ = ∞.

function for that feasible solution.

• For nodes representing partial solutions 𝑐(𝑥) is the cost of the minimum-cost
node in the sub tree with root 𝑥.
• Since 𝑐(𝑥) is in general as hard to compute as the original optimization

𝑐Ƹ
(𝑥) such that 𝑐Ƹ (𝑥) ≤ 𝑐(𝑥) for all 𝑥.
problem is to solve, the branch-and-bound algorithm will use an estimate

• Any node representing a feasible solution will be an answer node but only
minimum-cost answer nodes correspond to an optimal solution
 Job Sequencing with Deadlines

• To use the branch and bound technique to solve any problem, it

is first necessary to conceive a state space tree for the problem.
ሺ 𝑥ሻ , 𝑢ሺ 𝑥ሻ and 𝑐ሺ 𝑥ሻ the nodes in the tree.
• Next, define 𝑐Ƹ
• The solution space for the problem of job sequencing with
deadlines consists of all possible subsets of the jobs. The solution
space can be organized into a tree by means of either of the two
formulations: fixed tuple and variable tuple formulation.
• Let 𝑆𝑥 be the subset of jobs selected at node 𝑥. If 𝑚 =
ሺ 𝑥ሻ = σ𝑖<𝑚 𝑝𝑖 .
max{𝑖|𝑖 ∈𝑆𝑥}, then 𝑐Ƹ
𝑖∉𝑆𝑥

• A simple upper bound 𝑢ሺ 𝑥ሻ on the cost of a minimum-cost

answer node in the sub tree 𝑥 is 𝑢ሺ 𝑥ሻ = σ𝑖∉𝑆𝑥 𝑝𝑖 .
Example: Job Sequencing with Deadlines (variable tuple size
formulation
J1 J2 J3 J4
Profit 5 10 6 3
Deadline 1 3 2 1
Processing Time 1 2 1 1

Figure 7
 Example: Job Sequencing with Deadlines (fixed tuple size
formulation
J1 J2 J3 J4
Profit 5 10 6 3
Deadline 1 3 2 1
Processing Time 1 2 1 1

Figure 8
FIFO Branch-and-Bound for Job Sequencing (Variable Tuple Size)
• 𝑈= ∞ or 𝑈= σ1≤𝑖≤𝑛 𝑝𝑖
• 𝑢ሺ 2ሻ = 19,𝑢ሺ 3ሻ = 14,𝑢ሺ 4ሻ =
18,𝑢ሺ 5ሻ = 21
• Variable 𝑈 is updated to 14 when node

• Since 𝑐Ƹሺ 4ሻ > 𝑈&𝑐Ƹ ሺ 5ሻ > 𝑈, nodes 4

3 is generated

& 5 get killed

• E-node=2, 𝑢ሺ 6ሻ = 9 and so 𝑈 is
• Only nodes 2 and 3 remain alive

ሺ 7ሻ = 10 > 𝑈 and so node 7 gets

• 𝑐Ƹ
updated to 9

killed

• E-node=3, 𝑢ሺ 9ሻ = 8 and so 𝑈 is
• Node 8 is infeasible and it gets killed

ሺ 10ሻ = 11 > 𝑈 and so node 10 gets

• 𝑐Ƹ
updated to 8

killed
• Next E-node is 6, both its children are
Figure 9 infeasible
• 9’s only child is infeasible, leaving 9
as the minimum-cost answer node
 LC Branch-and-Bound

 Begin with 𝑈= ∞ and node 1 as E-node

Update 𝑈= 14 when node 3 is generated; kill nodes 4 and 5

 Generate nodes 2,3,4 and 5 in that order

Node 2 is the next E-node as 𝑐Ƹሺ2ሻ = 0 &𝑐Ƹሺ3ሻ = 5



Generate nodes 6,7 and 8; update 𝑈= 9 when node 6 is



generated; kill 7 because 𝑐Ƹሺ7ሻ = 10 > 𝑈; node 8 is infeasible



Node 6 is the E-node because 𝑐Ƹሺ6ሻ = 0 < 𝑐Ƹሺ3ሻ = 5; both

 The two live nodes are 3 and 6


Node 3 is the next E-node; update 𝑈= 8 upon generating

children of 6 are infeasible

node 0; node 10 is killed because 𝑐Ƹሺ10ሻ = 11 > 𝑈



 Node 9 is the next E-node; its only child is infeasible

Figure 10  No more live nodes; algorithm terminates with node 9 as the
minimum-cost answer node
 0/1 Knapsack Problem

Hence, we replace the objective function σ 𝑝𝑖 𝑥𝑖 by

• Knapsack problem is a maximization problem.

the function − σ 𝑝𝑖 𝑥𝑖 .
• Clearly, σ 𝑝𝑖 𝑥𝑖 is maximized iff − σ 𝑝𝑖 𝑥𝑖 is
minimized.
• The modified knapsack problem is stated as
follows:
Minimize − σ 𝑝𝑖 𝑥𝑖
Subject to σ 𝑤𝑥𝑖 ≤ 𝑚,
𝑥𝑖 = 0 or 1,1 ≤ 𝑖 ≤ 𝑛
 0/1 Knapsack Problem
 We define two functions 𝑐Ƹ ሺ 𝑥ሻ and 𝑢ሺ 𝑥ሻ such that 𝑐Ƹ
(𝑥) ≤ 𝑐ሺ 𝑥ሻ ≤ 𝑢(𝑥).
 Let 𝑥 be a node at level 𝑗, 1 ≤ 𝑗≤ 𝑛 + 1. At node 𝑥 assignments have
already been made to 𝑥𝑖 , 1 ≤ 𝑖 < 𝑗; hence 𝑢ሺ 𝑥ሻ = − σ1≤𝑖<𝑗 𝑝𝑖 𝑥𝑖 .
𝑢ሺ 𝑥ሻ can
computing 𝑢ሺ 1ሻ and 𝑐Ƹ ሺ 1ሻ . The computation of 𝑢ሺ 1ሻ and 𝑐Ƹ ሺ 1ሻ is done as
 An improved value for be obtained by first

follows:
o The objects are scanned from left to right and the objects are added into

negation of the total profit of all the objects in the knapsack is the 𝑢(1).
the knapsack until the first object that doesn’t fit is encountered. The

o In the calculation of ෝ 𝑐 ሺ 1ሻ , the fraction (that fills the knapsack) of the first

the total profit of all objects in the knapsack i.e., 𝑢ሺ 1ሻ and the negation
object that doesn’t fit is also considered. It is the sum of the negation of

of the profit corresponding to the fraction (that fills the knapsack) of the

 The left child of node 1 corresponds to the inclusion of object 1 (𝑖.𝑒., 𝑥1 = 1).
first object that doesn’t fit.

The right child of node 1 corresponds to discarding of object 1 (𝑖.𝑒., 𝑥1 = 0).

 Let 𝑦 be the child of node 𝑥 obtained by removing an object 𝑖 in the
knapsack corresponding to node 𝑥, then
𝑢ሺ 𝑦ሻ = 𝑢ሺ 𝑥ሻ − 𝑝𝑖 .
ሺ 𝑦ሻ is computed from 𝑐Ƹ
𝑐Ƹ ሺ 𝑥ሻ by subtracting the cost of including
o

object 𝑖 and adding the cost of filling the remaining capacity of

o

knapsack.
 0/1 Knapsack Problem LC Search

Figure 11
0/1 Knapsack Problem FIFO Branch and
Bound

Figure 12
State Space Tree for Traveling Salesperson
Problem for n=4

Figure 13
 Least Cost Branch and Bound (LCBB) Search for
Travelling Salesperson Problem

To use least cost branch and bound

to search the travelling salesperson

𝑐(𝑥)and two other

state tree, we must define a cost

functions 𝑐Ƹ(𝑥)and 𝑢(𝑥)such that

function

𝑐Ƹ
(𝑥) ≤ 𝑐(𝑥) ≤ 𝑢(𝑥)
 LCBB Search for Travelling Salesperson Problem

• A row(column) is said
to be reduced iff it
contains at least one
zero and all remaining
entries are non-
negative.
• A matrix is reduced iff
every row and column
is reduced.
 LCBB Search for Travelling Salesperson Problem

• We can associate a reduced cost matrix with every

• Let 𝐴be the reduced cost matrix for node 𝑅. Let 𝑆 be a

node in the traveling salesperson state space tree.

child of 𝑅such that the tree edge (𝑅,𝑆) corresponds to

including edge (𝑖,𝑗) in the tour.
• If 𝑆is not a leaf, then the reduced cost matrix for 𝑆may

1. Change all entries in row 𝑖 and column 𝑗 of 𝐴 to ∞. This

be obtained as follows:

prevents edges leaving vertex 𝑖 and entering vertex 𝑗.

2. Set 𝐴ሺ 𝑗,1ሻ = ∞. This prevents the use of edge (𝑗,1).
3. Reduce all rows and columns in the resulting matrix except

4. If 𝑟 is the total amount subtracted in step 3, then

for rows and columns containing ∞

𝑐Ƹሺ 𝑆ሻ = 𝑐Ƹ
ሺ 𝑅ሻ + 𝐴ሺ 𝑖,𝑗ሻ + 𝑟
LCBB Search for Travelling Salesperson Problem

Figure 13. State Space Tree Generated by

LCBB
LCBB Search for Travelling Salesperson Problem
LCBB Search for Travelling Salesperson Problem
LCBB Search for Travelling Salesperson Problem
LCBB Search for Travelling Salesperson Problem
LCBB Search for Travelling Salesperson Problem
Thank You