Chapter 1

Asymptotic Analysis
LEARNING OBJECTIVES

 Algorithm  In order traversal

 Recursive algorithms  Data structure
 Towers of Hanoi  Worst-case and average-case analysis
 Time complexity  Asymptotic notations
 Space complexity  Notations and functions
 SET representation  Floor and ceil
 TREE representation  Recurrence
 Preorder traversal  Recursion-tree method
 Post-order traversal  Master method

ALGORITHM 2. Posterior analysis: This analysis is done after the execution.

It is dependent on system architecture, CPU, OS etc. it
An algorithm is a finite set of instructions that, if followed, accom-
provides non-uniform exact values.
plishes a particular task.
All algorithms must satisfy the following.
Recursive Algorithms
• Input: Zero or more quantities are externally supplied. A recursive function is a function that is defined in terms of itself.
• Output: Atleast one quantity is produced. An algorithm is said to be recursive if the same algorithm is
• Definiteness: Each instruction should be clear and unambiguous. invoked in the body.
• Finiteness: The algorithm should terminate after finite number
of steps. Towers of Hanoi
• Effectiveness: Every instruction must be very basic.
There was a diamond tower (labeled A) with 64-golden disks. The
Once an algorithm is devised, it is necessary to show that it disks were of decreasing size and were stacked on the tower in
computes the correct answer for all possible inputs. This process decreasing order of size bottom to top. Besides this tower there
is called algorithm validation. Analysis of algorithms refers to the were 2 other diamond towers (labeled B and C) we have to move
task of determining how much computing time and storage an the disks from tower A to tower B using tower C for intermediate
algorithm requires. storage. As the disks are very heavy, they can be moved only one at
a time. No disk can be on top of a smaller disk .

Analyzing Algorithms
The process of comparing 2 algorithms rate of growth with respect
to time, space, number of registers, network, bandwidth etc is
called analysis of algorithms.
This can be done in two ways
Tower A
1. Priori Analysis: This analysis is done before the execution;
the main principle behind this is frequency count of Tower B
fundamental instruction.
This analysis is independent of CPU, OS and system Tower C
architecture and it provides uniform estimated values. Figure 1 Towers of Hanoi
3.82 | Unit 3 • Algorithms

Assume that the number of disks is ‘n’. To get the largest To move ‘3’ disks from tower A to tower ‘C’ requires 7 disk
disk to the bottom of tower B, we move the remaining (n movements
– 1) disks to tower C and then move the largest to tower B. \ For ‘n’ disks, the number of disk movements required
Now move the disks from tower C to tower B. is 2n – 1 = 23 – 1 = 7
Example:
Time complexity
T(n) = 1 + 2T(n – 1)
3 T(n) = 1 + 2(1 + 2 (T(n – 2)))
2 T(n) = 1 + 2 + 22 T(n – 2)
1 T(n) = 1 + 2 + 22 (1 + 2T(n – 3))
A B C T(n) = 1 + 2 + 22 + 23 + T(n – 3)
T(n) = 1 + 2 + 22 + … + 2i–1 + 2i T(n – i)
n −1
T ( n ) = ∑ 2i
2 i =0
1 3
A B C The time complexity is exponential, it grows as power of 2.
\ T(n) @ O(2n)

Space complexity
The space complexity of an algorithm is the amount of
1 2 3
memory it needs to run to completion. The measure of the
A B C
quantity of input data is called the size of the problem. For
example, the size of a matrix multiplication problem might
be the largest dimension of the matrices to be multiplied.
The size of a graph problem might be the number of edges.
3 The limiting behavior of the complexity as size increases is
1 2 called the asymptotic time complexity.
A B C
• It is the asymptotic complexity of an algorithm which
ultimately determines the size of problems that can be
solved by the algorithm.
• If an algorithm processes inputs of size ‘n’ in time cn2 for
3 some constant c, then we say that the time complexity of
2 1 that algorithm is O(n2), more precisely a function g(n) is
A B C said to be O( f (n)) if there exists a constant c such that
g(n) ≤ c( f (n)) for all but some finite set of non-negative
values for n.
• As computers become faster and we can handle larger
problems, it is the complexity of an algorithm that deter-
3 2 1 mines the increase in problem size that can be achieved
A B C with an increase in computer speed.
• Suppose we have 5 algorithms Algorithm 1 – Algorithm 5
with the following time complexities.

Algorithm Time Complexity

2
3 1 Algorithm – 1 n
A B C Algorithm – 2 n log n
Algorithm – 3 n2
Algorithm – 4 n3
Algorithm – 5 2n
3
2 The time complexity is, the number of time units required
1 to process an input of size ‘n’. Assume that input size ‘n’ is
A B C 1000 and one unit of time equals to 1 millisecond.
 Chapter 1 • Asymptotic Analysis | 3.83

The following figure gives the sizes of problems that can • Main drawback of using adjacency matrix is that it
be solved in one second, one minute, and one hour by each requires |V|2 storage even if the graph has only O(|V|)
of these five algorithms. edges.
• Another representation for a graph is by means of lists.
Maximum
The adjacency list for a vertex v is a list of all vertices
Problem Size
Time W adjacent to V. A graph can be represented by |V| adja-
Algorithm Complexity 1 sec 1 min 1 hour cency lists, one for each vertex.
Algorithm – 1 n 1000 6 × 104 3.6 × 106
Algorithm – 2 n log n 140 4893 2.0 × 105 Example:
Algorithm – 3 n2 31 244 1897
1 2
3
Algorithm – 4 n 10 39 153
Algorithm – 5 2n 9 15 21

4 3
From the above table, we can say that different algorithms
will give different results depending on the input size. Figure 2 Directed graph
Algorithm – 5 would be best for problems of size 2 ≤ n ≤ 9,
Algorithm – 3 would be best for 10 ≤ n ≤ 58, Algorithm – 2
1 2 3 4
would be best for 59 ≤ n ≤ 1025, and Algorithm – 1 is best
for problems of size greater than 1024. 1 0 1 0 1
2 0 0 1 0

3 0 0 0 0

SET REPRESENTATION  
4 0 1 1 0
A common use of a list is to represent a set, with this rep-
resentation the amount of memory required to represent a Figure 3 Adjacency matrix
set is proportional to the number of elements in the set. The
amount of time required to perform a set operation depends
on the nature of the operation. Vertex – 1 2 4 0

• Suppose A and B are 2 sets. An operation such as A ∩ B Vertex – 2

3 0
Vertex – 3
requires time atleast proportional to the sum of the sizes
of the 2 sets, since the list representing A and the list rep- Vertex – 4 2 3 0
resenting B must be scanned atleast once.
• The operation A ∪ B requires time atleast proportional to Figure 4 Adjacency lists
the sum of the set sizes, we need to check for the same
element appearing in both sets and delete one instance of There are edges from vertex – 1 to vertex – 2 and 4, so the
each such element. adjacency list for 1 has items 2 and 4 linked together in the
• If A and B are disjoint, we can find A ∪ B in time inde- format given above.
pendent of the size of A and B by simply concatenating
the two lists representing A and B. • The adjacency list representation of a graph requires stor-
age proportional to |V| + |E|, this representation is used
when |E|< < |V|2.
GRAPH REPRESENTATION
A graph G = (V, E) consists of a finite, non-empty set of
vertices V and a set of edges E. If the edges are ordered pairs
(V, W) of vertices, then the graph is said to be directed; V is TREE REPRESENTATION
called the tail and W the head of the edge (V, W). There are A directed graph with no cycles is called a directed acyclic
several common representations for a graph G = (V, E). One graph. A directed graph consisting of a collection of trees is
such representation is adjacency matrix, a V X V matrix called a forest. Suppose the vertex ‘v’ is root of a sub tree,
M of 0’s and 1’s, where the ijth element, m[i, j] = 1, if and then the depth of a vertex ‘v’ in a tree is the length of the
only if there is an edge from vertex i to vertex j. path from the root to ‘v’.
• The adjacency matrix representation is convenient for • The height of a vertex ‘v’ in a tree is the length of a long-
graph algorithms which frequently require knowledge of est path from ‘v’ to a leaf.
whether certain edges are present. • The height of a tree is the height of the root
• The time needed to determine whether an edge is present • The level of a vertex ‘v’ in a tree is the height of the tree
is fixed and independent of |V| and |E|. minus the depth of ‘v’.
3.84 | Unit 3 • Algorithms

1 15
6 18
2 13
7 8 16
3 4
9 11 14
17
5 10 12
(c)
Left child Right child Figure 6 (a) Pre-order, (b) Post-order (c) In-order
1 2 6
2 3 4 Post-order traversal
3 0 0 A post-order traversal of T is defined recursively as follows:
4 0 5
5 0 0 1. Visit in post-order the sub trees with roots v1, v2, v3,
6 7 8 … vk in that order.
7 0 0 2. Visit the root r.
8 0 9
9 0 10 In-order Traversal
10 0 0
An in-order traversal is defined recursively as follows:
Figure 5 A binary tree and its representation
1. Visit in in-order the left sub tree of the root ‘r’.
• Vertex 3 is of depth ‘2’, height ‘0’ and the level is 2 2. Visit ‘r’.
(Height of tree - depth of ‘3’ = 4 – 2 = 2). 3. Visit in inorder the right sub tree of r.
• A binary tree is represented by 2 arrays: left child and
right child. Example: Consider the given tree
• A binary tree is said to be complete if for some integer
C
k, every vertex of depth less than k has both a left child
and a right child and every vertex of depth k is a leaf. A D
B
complete binary tree of height k has exactly (2k+1 – 1)
E
vertices.
A
• A complete binary tree of height k is often represented by
a single array. Position 1 in the array contains the root.
What are the pre-order, post-order and in-order traversals of
The left child of the vertex in position ‘i’ is located at
the above tree?
position ‘2i’ and the right child at position ‘2i + 1’.
Solution: Pre-order – CBADE
Tree Traversals Post-order – ABEDC
Many algorithms which make use of trees often traverse the In-order – ABCDE
tree in some order. Three commonly used traversals are pre-
order, postorder and inorder.
DATA STRUCTURE
Pre-order Traversal A data structure is a way to store and organize data in-order
A pre-order traversal of T is defined recursively as follows: to facilitate access and modifications. No single data struc-
ture works well for all purposes, it is important to know the
1. Visit the root.
strengths and limitations of several data structures.
2. Visit in pre-order the sub trees with roots v1, v2 … vk in
that order.
Efficiency
11 Algorithms devised to solve the same problem often differ
18
dramatically in their efficiency. Let us compare efficiencies
16 17 of Insertion sort and merge sort; insertion sort, takes time
12
14
17 16
equal to C1n2 to sort ‘n’ elements, where C1 is a constant
13 15 13 that does not depend on ‘n’. It takes time proportional to
12
n2, merge sort takes time equal to C2nlog n, C2 is another
18
15 constant that also does not depend on ‘n’. Insertion sort has
14 11
a smaller constant factor than merge sort (C1 < C2) constant
(a) (b) factors are far less significant in the running time.
 Chapter 1 • Asymptotic Analysis | 3.85

Merge sort has a factor of ‘log n’ in its running time, ASYMPTOTIC NOTATIONS
insertion sort has a factor of ‘n’, which is much larger.
Asymptotic notations are mostly used in computer science
Insertion sort is faster than merge sort for small input sizes,
to describe the asymptotic running time of an algorithm.
once the input size ‘n’ becomes large enough, merge sort
As an example, an algorithm that takes an array of size n
will perform better. No matter how much smaller C1 is than
as input and runs for time proportional to n2 is said to take
C2. There will always be a crossover point beyond which
O(n2) time.
merge sort is faster.
5 Asymptotic Notations:
Example: Consider 2 computers, computer A (faster
computer), B (slower computer). Computer A runs insertion • O (Big-oh)
sort and computer B runs merge sort. Each computer is • q (Theta)
given 2 million numbers to sort. Suppose that computer A • W (Omega)
executes one billion instruction per second and computer B • o (Small-oh)
executes only 10 million instructions per second, computer • w
A is 100 times faster than computer B (C1 = 4, C2 = 50).
How much time is taken by both the computers? How to Use Asymptotic Notation
Solution: Insertion sort takes C1 * n2 time for Algorithm Analysis?
Merge sort takes C2 * n * log n time Asymptotic notation is used to determine rough estimates
C1 = 4, C2 = 50 of relative running time of algorithms. A worst-case anal-
Computer A takes ysis of any algorithm will always yeild such an estimate,
because it gives an upper bound on the running time T(n) of
4 × ( 2 × 106 ) 2 instructions the algorithm, that is T(n) g(n).
≅ 4000 seconds
10 9 instructions/second Example:

Computer B takes a←0 1 unit 1 time

for i ← 1 to n do{ 1 unit n times

50 × 2 × 106 × log( 2 × 106 ) insturctions
=
10 7 instructions/second for j ← 1 to i do{ 1 unit n(n + 1)/2 times
= 209 seconds a←a+1 1 unit n(n +1)/2 times
By using an algorithm whose running time grows more
slowly, even with an average compiler, computer B runs Where the times for the inner loop have been computed as fol-
20 times faster than computer A. The advantage of merge lows: For each i from 1 to n, the loop is executed i times, so the
n
sort is even more pronounced when we sort ten million total number of times is 1 + 2 + 3 + ⋯ + n = ∑ i = n(n + 1)/ 2
numbers. As the problem size increases, so does the relative i =1
advantage of merge sort. Hence in this case
T(n) = 1 + n + 2n (n +1)/2 = n2 + 2n + 1
If we write g(n) = n2 + 2n + 1, then T(n) ∈ q(g(n)),
Worst-case and average-case analysis That is T(n) ∈ q(n2 + 2n + 1), we actually write T(n) ∈ q(n2),
In the analysis of insertion sort, the best case occurs when as recommended by the following rule:
the array is already sorted and the worst case, in which the • Although the definitions of asymptotic notation allow one
input array is reversely sorted. We concentrate on finding to write, for example, T(n) ∈ O(3n2 + 2).
the worst-case running time, that is the longest running time
We simplify the function in between the parentheses as
for any input of size ‘n’.
much as possible (in terms of rate of growth), and write
• The worst-case running time of an algorithm is an upper instead T(n) ∈ O(n2)
bound on the running time for any input. It gives us a For example: T(n) ∈ q(4n3 – n2 + 3)
guarantee that the algorithm will never take any longer. T(n) ∈ q(n3)
• The ‘average-case’ is as bad as the worst-case. Suppose  n 
that we randomly choose ‘n’ numbers and apply inser- For instance O  ∑ i  , write O(n2) after computing the sum.
tion sort. To insert an element A[j], we need to determine  i =1 
where to insert in sub-array A [1 … J – 1]. On average • In the spirit of the simplicity rule above, when we are to
half the elements in A[1 … J – 1] are less than A[j] and compare, for instance two candidate algorithms A and B
half the elements are greater. So tj = j/2. The average-case having running times (TA(n) = n2 – 3n + 4 and TB(n) = 5n3
running time turns out to be a quadratic function of the + 3, rather than writing TA(n) ∈ O(TB(n)), we write TA(n)
input size. ∈ q(n2), and TB(n) ∈ q(n3), and then we conclude that A
3.86 | Unit 3 • Algorithms

is better than B, using the fact that n2(quadratic) is better • The lower order terms of an asymptotically positive func-
than n3(cubic) time, since n2 ∈ O(n3). tion can be ignored in determining asymptotically tight
bounds because they are insignificant for large n.
• A small fraction of the highest order term is enough to
Order of Growth dominate the lower order term. Thus setting C1 to a value
In the rate of growth or order of growth, we consider only that is slightly smaller than the coefficient of the highest
the leading term of a formula. Suppose the worst case run- order term and setting C2 to a value that is slightly larger
ning time of an algorithm is an2 + bn + c for some constants permits the inequalities in the definition of q-notation to
a, b and c. The leading term is an2. We ignore the leading be satisfied. If we take a quadratic function f (n) = an2 +
term’s constant coefficient, since constant factors are less bn + c, where a, b and c are constants and a > 0. Throwing
significant than the rate of growth in determining compu- away the lower order terms and ignoring the constant
tational efficiency for large inputs. Thus we can write, the yields f (n) = q (n2).
worst-case running time is q(n2). • We can express any constant function as q(n0), or q(1) we
We usually consider one algorithm to be more efficient shall often use the notation q(1) to mean either a constant
than another if its worst-case running time has a lower order or a constant function with respect to some variable.
of growth. Due to constant factors and lower order terms,
this evaluation may be in error for small inputs. But for O-Notation
large inputs, q(n2) algorithm will run more quickly in the We use O-notation to give an upper bound on a function,
worst-case than q(n3) algorithm. within a constant factor.

q-Notation Cg(n)
A function f (n) belongs to the set q(g(n)) if there exists a f(n)
positive constant C1 and C2 such that it can be “sand witched”
between C1g(n) and C2g(n) for sufficiently large n. We write
f (n) ∈ q (g(n)) to indicate that f (n) is a member of q (g(n))
or we can write f (n) = q (g(n)) to express the same notation. n0 n

The above figure shows the intuition behind O-notation. For

C 2g(n) all values ‘n’ to the right of n0, the value of the function f (n)
f (n) is on or below g(n). We write f (n) = O(g(n)) to indicate that
C 1g(n) a function f (n) is a member of the set O(g(n)).
f (n) = q(g(n)) implies f (n) = O(g(n)). Since q notation is
stronger notation than O-notation set theoretically, we have
n0 n q(g(n)) ⊆ O(g(n)). Thus any quadratic function an2 + bn
The above figure gives an intuitive picture of functions f (n) + c, where a > 0, is in q(n2) also shows that any quadratic
and g(n), where we have that f (n) = q (g(n)), for all the val- function is in O(n2) when we write f (n) = O(g(n)), we are
ues of ‘n’ to the right of no, the value of f (n) lies at or above claiming that some constant multiple of g(n) is an asymp-
C1g(n) and at or below C2g(n). g(n) is asymptotically tight totic upper bound on f (n), with no claim about how tight an
bound for f (n). The definition of q(g(n)) requires that every upper bound it is.
member f (n) ∈ q(g(n)) be asymptotically non-negative, that The O-notation is used for asymptotically upper bound-
is f (n) must be non-negative whenever ‘n’ is sufficiently large. ing a function. We would use O (big-oh) notation to represent
The q-notation is used for asymptotically bounding a a set of functions that upper bounds a particular function.
function from both above and below. We would use q(theta) Definition We say that a function f (n) is big oh of g(n) writ-
notation to represent a set of functions that bounds a par- ten as f (n) = O(g(n)) if there exists positive constants C and
ticular function from above and below. n0 such that
0 ≤ f (n) ≤ Cg(n), ∀ n ≥ no
Definition: We say that a function f (n) is theta of g(n) writ-
ten as f (n) = q(g(n)) if such exists positive constants C1, C2
and n0 such that 0 ≤ C1g(n) ≤ f (n) ≤ C2 g(n), ∀ n ≥ n0. Solved Examples
Example: Let f (n) = 5.5n – 7n, verify whether f (n) is
2 Example 1: let f (n) = n2
q(n2). Lets have constants c1 = 9 and n0 = 2, such that 0 ≤ Then f (n) = O(n2)
f (n) ≤ C1 n2, ∀n ≥ n0. From example, 4 we have constants f (n) = O(n2log n)
C2 = 3, and n0 = 2.8, such that 0 ≤ C2 n2 ≤ f (n), ∀n ≥ n0. f (n) = O(n2.5)
To show f (n) is q(n2), we have got hold of two constants f (n) = O(n3)
C1 and C2. We fix the n0 for q as maximum {2, 2.8} = 2.8. f (n) = O(n4) … so on.
 Chapter 1 • Asymptotic Analysis | 3.87

Example 2: Let f (n) = 5.5n2 – 7n, verity whether f (n) is is on or above Cg(n). For any 2 functions f (n) and g(n) we
O(n2) have f (n) = q(g(n)) if f (n) = O(g(n)) and f (n) = W(g(n)).
From the above statement we can say that, an2 + bn + c =
Solution: Let C be a constant such that
q(n2) for any constants a, b and c, where a > 0, immediately
7 implies that
5.5n 2 − 7 n ≤ Cn 2 , or n ≥
c − 5.5 \ an2 + bn + c = W(n2)

Fix C = 9, to get n ≥ 2 \ an2 + bn + c = O(n2)

So our n0 = 2 and C = 9
This shows that there exists, positive constants C = 9 and n0 Example 4: Let f (n) = 5.5n2 - 7n.
= 2 such that Verity whether f (n) is W(n2)
0 ≤ f (n) ≤ Cn2, ∀ n ≥ n0 Solution: Let C be a constant such that 5.5n2 – 7n ≥ Cn2 or
Example 3: 7
n≥ . Fix C = 3, to get n ≥ 2.8. So, our n0 = 2.8 and
h(n) = 3n + 10n + 1000 log n ∈ O(n )
3 3 5.5 − C
C=3
h(n) = 3n3 + 10n + 1000 log n ∈ O(n4) This shows that there exists positive constants C = 3 and
n0 = 2.8, such that 0 ≤ Cn2 ≤ f (n), ∀n ≥ n0.
• Using O-notation, we can describe the running time of
an algorithm by inspecting the algorithm’s overall struc-
ture. For example, the doubly nested loop structure of the Cg(n)
insertion sort algorithm yields an O(n2) upper bound on f(n)
the worst-case running time. The cost of each iteration of
the inner loop is bounded from above by O(1) (constant), 0 ≤ f (n) ≤ Cg(n), ∀ n ≥ n0
the inner loop is executed almost once for each of the n2 n0
pairs.
• O(n2) bound on worst-case running time of insertion sort (a) f (n) = O(g(n))
also applies to its running time on every input.
• The q(n2) bound on the worst-case running time of inser- f (n)
tion sort, however, does not imply a q(n2) bound on the
running time of insertion sort on every input, when the
input is already sorted, insertion sort runs in q(n) time. Cg(n)

0 ≤ Cg(n) ≤ f (n), ∀ n ≥ n0
W (omega)-notation
n0

(b) f (n) = W(g(n))

f (n)
Cg(n)
C 1g(n)
f (n)

n0 n
C 2g(n)

The W-notation is used for asymptotically lower bound- 0 ≤ C 2g(n) ≤ f(n) ≤ C 1g(n), ∀ n ≥ n0
ing a function. We would use W(big-omega) notation to n0
represent a set of functions that lower bounds a particular
function. (c) f (n) = q(g(n))
Figure 7 A diagrammatic representation of the asymptotic notations
Definition We say that a function f (n) is big-omega of g(n) O, W and q
written as f (n) = W(g(n)) if there exists positive constants C • W-notation describes a lower bound; it is used to bound
and n0 such that the best-case running time of an algorithm. The best-case
0 ≤ Cg(n) ≤ f (n) , ∀ n ≥ n0 running time of insertion sort is W(n). The running time
of insertion sort falls between W(n) and O(n2), since it
The intuition behind W-notation is shown in the above falls anywhere between a linear function of ‘n’ and a
figure. For all values ‘n’ to the right of n0, the value of f (n) quadratic function of ‘n’.
3.88 | Unit 3 • Algorithms

• When we say that the running time of an algorithm is x − 1 <  x  ≤ x ≤  x  < x + 1 for any integer n,
W(g(n)), we mean that no matter what particular input of
size ‘n’ is chosen for each value of n, the running time on n n
that input is at least a constant times g(n), for sufficiently  2  +  2  = n,
   
large ‘n’.
For any real number n ≥ 0 and integer a, b > 0
O-notation
n
The asymptotic upper bound provided by O-notation may a  n 
or may not be asymptotically tight. The bound 2n3 = O(n3)   =  
is asymptotically tight, but the bound 2n = O(n2) is not.  b   ab 
 
We use O-notation to denote an upper bound that is not  
asymptotically tight.
n
w -notation a  n 
  =  
By analogy, w-notation is to W-notation as o-notation is to  b   ab 
O-notation. We use w-notation to denote a lower bound that  
 
is not asymptotically tight.
It is defined as
f (n) ∈ w(g(n)) if and only if g(n) ∈ o(f (n)) Polynomials
Given a non-negative integer k, a polynomial in n of degree
k
‘k’ is a function p(n) of the form p (n) = ∑ ai ni
Comparison of functions i =0
Transitivity Where the constants a0, a1, … ak are the coefficients of
1. f (n) = q(g(n)) and g(n) = q(h(n)) the polynomial and ak ≠ 0.
⇒ f (n) = q(h(n)) For an asymptotically positive polynomial p(n) of degree
2. f (n) = O(g(n)) and g(n) = O(h(n)) k, we have p(n) = q(nk)
⇒ f (n) = O(h(n))
3. f (n) = W(g(n)) and g(n) = W(h(n)) Exponentials
⇒ f (n) = W(h(n))
4. f (n) = o(g(n)) and g(n) = o(h(n)) For all real a > 0, m and n, we have the following identities:
⇒ f (n) = o(h(n)) a0 = 1
5. f (n) = w(g(n)) and g(n) = w(h(n)) a1 = a
⇒ f (n) = w(h(n)) 1
a −1 =
Reflexivity a
1. f (n) = q(f (n)) (am)n = amn
2. f (n) = O(f (n)) (am)n = (an)m
3. f (n) = W(f (n)) aman = am+n
x 2 x3 ∞
xi
Symmetry ex = 1 + x + + +⋯ = ∑
f (n) = q(g(n)) if and only if g(n) = q(f (n)) 2 ! 3! i =0 i !

Transpose symmetry • For all real x, we have inequality ex ≥ 1 + x

1. f (n) = O(g(n)) if and only if g(n) = W (f (n)) • If x = 0, we have 1 + x ≤ ex ≤ 1 + x + x2
2. f (n) = o(g(n)) if and only if g(n) = w(f (n))
Logarithms
NOTATIONS AND FUNCTIONS lg n = log2n (binary logarithm)
ln n = logen (natural logarithm)
Floor and Ceil lgk n = (log n)k (exponentiation)
For any real number ‘x’, we denote the greatest integer less lg lg n = lg (lg n) (composition)
than or equal to x by  x  called as floor of x and the least For all real a > 0, b > 0, c > 0 and n,
integer greater than or equal to x by  x  called as ceiling of x. logc (ab) = logca + logcb
 Chapter 1 • Asymptotic Analysis | 3.89

Substitution Method
log b a n = n log b a
In this method one has to guess the form of the solution.
log c a It can be applied only in cases when it is easy to guess the
log b a =
log c b form of the answer. Consider the recurrence relation

logb (1/a) = –logba T(n) = 2T(n/2) + n

1 We guess that the solution is T(n) = O(n log n) we have

log b a = to prove that
log a b
T(n) ≤ c n log n (\ c > 0)
c a
a logb = c logb Assume that this bound holds for  n/2 
T(n/2) ≤ c(n/2). log (n/2) + n
Factorials T(n) ≤ 2(c(n/2 log (n/2)) + n
n! is defined for integers n ≥ 0 as ≤ cn log n – cn log2 + n
1 if n = 0 ≤ cn log n – cn + n
n! =  ≤ cn log (\ c ≥ 1)
(n − 1)!* n n > 0
A weak upper bound on the factorial function is n! ≤ nn
since each of the n terms in the factorial product is almost n. Recursion-tree Method
n! = o(nn) In a recursion-tree, each node represents the cost of single
n! = w(2n) sub problem somewhere in the set of recursive function
lg (n!) = q(n log n) invocations. We sum the costs within each level of the tree
to obtain a set of per-level costs, and then we sum all the
Iterated Logarithm per-level costs to determine the total cost of all levels of
the recursion. Recursion trees are useful when the recur-
The notation lg*n is used to denote the iterated logarithm.
rence describes the running time of a divide-and-conquer
Let ‘lg(i) n’ be as defined above, with f (n) = lg n. The log-
algorithm.
arithm of a non-positive number is undefined, ‘lg(i) n’ is
defined only if lg(i–1) n > 0; Example:
The iterated logarithm function is defined as lg*n = min Consider the given recurrence relation
{i ≥ 0 : lg(i) n ≤ 1}. This function is a very slowly growing T(n) = 3T (n/4) + q(n2)
function.
We create a recursion tree for the recurrence
lg*2 = 1
lg*4 = 2 T(n) = 3T(n/4) + Cn2
lg*16 = 3
lg*65536 = 4 Cn 2
lg*(265536) = 5

RECURRENCES T (n/4) T (n/4) T (n/4)

When an algorithm contains a recursive call to itself,
its running time can often be described by a recurrence. The Cn2 term at the root represents the cost at the top level
A recurrence is an equation that describes a function in of recursion, and the three sub trees of the root represent the
terms of its value on smaller inputs. For example, the costs incurred by the sub problems of size n/4.
worst-case running time T(n) of the merge-sort can be
described as
Cn 2
T(n) = q (1) if n = 1
2T (n/2) + q (n) if n > 1
The time complexity of merge-sort algorithm in the worst- C(n/4)2
C(n/4)2
case is T(n) = q(n log n) C(n/4)2
There are 3 methods to solve recurrence relations: T (n/16)
T (n/16) T (n/16) T (n/16)
1. Substitution method T (n/16) T (n/16) T (n/16)
T (n/16) T (n/16)
2. Recursion-tree method
3. Master method Figure 8 Recursion tree for T(n) = 3T (n/4) + cn2
3.90 | Unit 3 • Algorithms

Cn 2

C (n /4)2 C (n /4)2 C (n /4)2

C (n/16)2 C (n /16)2 C (n /16)2 C (n /16)2 C (n/16)2 C (n /16)2 C (n /16)2 C (n /16)2 C (n /16)2

T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1)

Figure 9 Expanded Recursion tree with height log4n (\ levels log4n + 1)

The sub-problem size for a node at depth ‘i’ is n/4i, at this Master Method
depth, the size of the sub-problem would be n = 1, when n/4i Let a ≥ 1 and b > 1 be cons-tants, let f (n) be a function
= 1 or i = log4n, the tree has log4n+1 levels. and let T(n) be defined on the non-negative integers by the
• We have to determine the cost at each level of the tree. recurrence
Each level has 3 times more nodes than the level above,
so the number of nodes at depth ‘i’ is 3i. T(n) = aT(n/b) + f (n)
• Sub problem sizes reduce by a factor of ‘4’ for each level T(n) can be bounded asymptotically as follows
we go down from the root, each node at depth i, for i = 0,
1, 2 … log4n–1, has a cost of c(n/4i)2. 1. If f (n) = O(nlogba–∈) for some constant ∈ > 0, then T(n)
= q(nlogba)
Total cost over all nodes at depth i, for i = 0, 1, … log4n–1 2. If f (n) = q(nlogba) then T(n) = q(nlogba. log n)
2 i
n  3 3. If f (n) = W(nlogba+∈) for some constant ∈ > 0, and
= 3i * c  i  =   cn 2
 4   16  if af (n/b) ≤ cf (n) for some constant c < 1 and all
The last level, at depth log4n has 3i nodes = 3log4n = nlog43 each sufficiently large n, then T(n) = q (f (n)).
contributing cost T(1), for a total cost of nlog43 T(1), which is
q (nlog43) cost of the entire tree is equal to sum of costs over Note: In the first case, not only must f (n) be smaller than n
all levels. logba, it must be polynomially smaller. That is, f (n) must be
2 asymptotically smaller than nlogba by a factor of n∈, for some
3 2  3 constant ∈ > 0.
T (n) = cn 2 + cn +   cn 2 + ⋯ +
16  16  In the third case, not only must f (n) be larger than nlogba,
log 4n −1 it must be polynomially larger and in addition satisfy the
 3 regularity condition af (n/b) ≤ Cf (n).
  cn 2 + ⋯ + θ (n log 43 )
 16 
log 4n−1 i Example: Consider the given recurrence relation T(n)
 3 2
= ∑
i =0
  cn + θ (n 4 )
 16 
log 3
= 9T(n/3) + n.
To apply master theorem, the recurrence relation must be in
∞
 3
i the following form:
< ∑   cn 2 + θ (n log 43 )
i = 0  16  T(n) = aT(n/b) + f (n)

=
1
cn 2 + θ (n log 43 ) a = 9, b = 3, f (n) = n
 3
1−   nlogba = nlog39 = n2
 16 
Since f (n) = O(nlog39–∈), where ∈ = 1
16
= 2 cn + θ (n log 43 ) = O(n 2 ) We can apply case 1 of the master theorem and the solution
13 is T(n) = q(n2).
 Chapter 1 • Asymptotic Analysis | 3.91

EXERCISES
Practice Problems 1 10. Solve the recurrence relation using master method:
Directions for questions 1 to 15: Select the correct alterna- T(n) = 4T (n/2) + n2
tive from the given choices. (A) q(n log n) (B) q(n2 log n)
(C) q(n )
2
(D) q(n3)
1. What is the time complexity of the recurrence relation
n 11. Arrange the following functions according to their
T (n) = 2T   + n 2 ? order of growth (from low to high):
2
(A) q(n2) (B) q(n) (A) 3 n , 0.001n 4 + 3n3 + 1, 3n , 22 n
(C) q(n3) (D) q(n log n) (B) 3n , 22 n , 3 n , 0.001n 4 + 3n3 + 1
2. What is the time complexity of the recurrence relation
n (C) 22 n , 3 n , 3n , 0.001n 4 + 3n3 + 1
by using masters theorem T (n) = 2T   + n ?
2 (D) 3
n , 22 n , 3n , 0.001n 4 + 3n3 + 1
(A) q(n2) (B) q(n)
12. The following algorithm checks whether all the ele-
(C) q(n3) (D) q(n log n)
ments in a given array are distinct:
3. What is the time complexity of the recurrence relation
Input: array A[0 … n – 1]
n
by using master theorem, T (n) = 2T   + n 0.51 Output: true (or) false
4
(A) q(n2) (B) q(n) For i ← 0 to n – 2 do
(C) q(n3) (D) (n0.51) For j ← i + 1 to n – 1 do
4. What is the time complexity of the recurrence relation if A[i] = A[ j] return false
n
using master theorem, T (n) = 7T   + n 2 ? return true
3 The time complexity in worst case is
(A) q(n2) (B) q(n) (A) q(n2) (B) q(n)
(C) q(n3) (D) (log n) (C) q(log n) (D) q(n log n)
5. Time complexity of f (x) = 4x2 - 5x + 3 is 13. The order of growth for the following recurrence rela-
(A) O(x) (B) O(x2) tion is T(n) = 4T(n/2) + n3, T(1) = 1
3/2
(B) O(x ) (D) O(x0.5) (A) q(n) (B) q(n3)
6. Time complexity of f (x) = (x2 + 5 log2 x)/(2x + 1) is (C) q(n )2
(D) q(log n)
(A) O(x) (B) O(x2) n
(C) O(x3/2) (D) O(x0.5) 14. Time complexity of T (n) = 2T   + 3 is
4
( )
7. For the recurrence relation, T (n) = 2T  n  + lg n,
which is tightest upper bound? (A) q ( n log n) (B) q ( n log n )
(A) T(n) = O(n2) (B) T(n) = O(n3)
(C) q ( n ) (D) q(n2)
(C) T(n) = O(log n) (D) T(n) = O(lg n lg lg n)
15. Consider the following three claims
8. Consider T(n) = 9T(n/3) + n, which of the following is
(I) (n + k)m = q(nm), where k and m are constants
TRUE?
(II) 2n + 1 = O(2n)
(A) T(n) = q(n2) (B) T(n) = q(n3)
(III) 22n + 1 = O(2n)
(C) T(n) = W(n ) 3
(D) T(n) = O(n)
Which one of the following is correct?
9. If f (n) is 100 * n seconds and g(n) is 0.5 * n seconds then
(A) I and III (B) I and II
(A) f (n) = g(n) (B) f (n) = W(g(n))
(C) II and III (D) I, II and III
(C) f (n) = w(g(n)) (D) None of these

Practice Problems 2 2. n = Ω(log n) means

Directions for questions 1 to 15: Select the correct alterna- (A) To the least n is log n
tive from the given choices. (B) n is log n always
1. Arrange the order of growth in ascending order: (C) n is at most log n
(A) O(1) > O(log n ) > O(n) > O(n2) (D) None of these
(B) O(n) > O(1) > O(log n) > O(n2) 3. Which of the following is correct?
(C) O(log n) > O(n) > O(1) > O(n2) (i) q (g(n)) = O(g(n)) ∩ W(g(n))
(D) O(n2) > O(n) > O(log n) > O(1) (ii) q (g(n)) = O(g(n)) ∪ W(g(n))
3.92 | Unit 3 • Algorithms

(A) (i) is true (ii) is false (B) Both are true (C) Investigation of worst case is more complex than
(C) Both are false (D) (ii) is true (i) is false average case.
4. 2n = x (n ), x is which notation?
2 3 (D) None of these
(A) Big-oh (B) Small-oh 11. Time complexity of T(n) = T(n/3) + T(2n/3) + O(n) is
(C) W – notation (D) q – notation (A) O(1)
5. Master method applies to recurrence of the form T(n) (B) O(n log n)
= a T(n/b) + f (n) where (C) O(log n)
(A) a ≥ 1, b > 1 (B) a = 1, b > 1 (D) O(n2)
(C) a > 1, b = 1 (D) a ≥ 1, b ≥ 1 12. Solve the recurrence relation to find T(n): T(n) = 4(n/2)
6. What is the time complexity of the recurrence relation +n
using master method? (A) q(n2) (B) q(log2n)
(C) q(n2 log2n) (D) q(n3)
n
T (n) = 4T   + n 13. What is the worst case analysis for the given code?
2 int search (int a[ ], int x, int n)
(A) q(n )2
(B) q(n) {
(C) q(log n) (D) q(n log n) int i;
7. Use the informal definitions of O, q W to determine these for (i = 0 ; i < n; i ++)
assertions which of the following assertions are true. if (a [i] = = x)
(A) n(n + 1)/2 ∈ O(n3) (B) n(n + 1)/2 ∈ O(n2) return i;
(C) n(n + 1)/2 ∈ W(n) (D) All the above return –1;
}
8. Match the following: (A) O(n) (B) O(n log n)
(i) Big-oh (A) ≥ (C) O(log n) (D) O(n2)
(ii) Small-o (B) ≤ 14. Find the time complexity of the given code.
(iii) W (C) = void f (int n)
{
(iv) q (D) <
if (n > 0)
(v) w (E) >
{
(A) (i) – D, (ii) – A, (iii) – C, (iv) -B , (v) – E f (n/2);
(B) (i) – B, (ii) – D, (iii) – A, (iv) – C, (v) – E f (n/2);
(C) (i) – C, (ii) – A, (iii) – B, (iv) – E, (v) – D }
(D) (i) – A, (ii) – B, (iii) – C, (iv) – D, (v) – E }
(A) q(n2)
9. Which one of the following statements is true? (B) q(n)
(A) Both time and space efficiencies are measured as (C) q(n log n)
functions of the algorithm input size. (D) q(2n)
(B) Only time efficiencies are measured as a function
of the algorithm input size. 15. The running time of the following algorithm procedure
(C) Only space efficiencies are measured as a function A(n)
of the algorithm input size. if n ≤ 2
(D) Neither space nor time efficiencies are measured return (1)
as a function of the algorithm input size. else
return ( A( n ))
10. Which of the following is true?
is described by
(A) Investigation of the average case efficiency is con-
siderably more difficult than investigation of the (A) O( n log n)
worst case and best case efficiencies.
(B) O(log n)
(B) Investigation of best case is more complex than
average case. (C) O(log log n)
(D) O(n)
 Chapter 1 • Asymptotic Analysis | 3.93

PREVIOUS YEARS’ QUESTIONS

1. The median of n elements can be found in O(n) time. these n numbers needs to be determined. Which of the
Which one of the following is correct about the com- following is TRUE about the number of comparisons
plexity of quick sort, in which median is selected as needed? [2007]
pivot? [2006] (A) At least 2n – c comparisons, for some constant c,
(A) q(n) (B) q(n log n) are needed.
(C) q(n2) (D) q(n3) (B) At most 1.5n − 2 comparisons are needed.
2. Given two arrays of numbers a1 … an and b1 … bn where (C) At least n log2 n comparisons are needed.
each number is 0 or 1, the fastest algorithm to find the (D) None of the above.
largest span (i, j) such that ai + ai+ 1 + … + aj = bi + bi + 7. Consider the following C code segment:
1 + … + bj, or report that there is not such span, [2006] int IsPrime(n)
(A) Takes O(3n) and W(2n) time if hashing is permit- {
ted int i,n;
(B) Takes O(n3) and W(n2.5) time in the key compari-
for(i=2; i<= sqrt(n); i ++)
son model
if (n%i = =0)
(C) Takes Q(n) time and space
{printf(“Not Prime\n”); return 0;}
(D) Takes O( n ) time only if the sum of the 2n ele-
return 1;
ments is an even number
}
3. Consider the following segment of C-code:
Let T(n) denote the number of times the for loop is exe-
int j, n;
cuted by the program on input n. Which of the following
j = 1; is TRUE? [2007]
while (j <=n)
j = j*2; (A) T (n) = Ο( n ) and T (n) = Ω( n )
The number of comparisons made in the execution of (B) T (n) = Ο( n ) and T (n) = Ω(1)
the loop for any n > 0 is: [2007]
(C) T (n) = Ο(n) and T (n) = Ω( n )
(A) log 2 n  + 1 (B) n
(D) None of the above
(C) log 2 + n  (D) log 2 n  + 1 8. The most efficient algorithm for finding the number
4. In the following C function, let n ≥ m. of connected components in an undirected graph on
int gcd(n,m) n vertices and m edges has time complexity [2008]
{ (A) Q(n) (B) Q(m)
if (n%m = =0) return m; (C) Q(m + n) (D) Q(mn)
n = n%m;
9. Consider the following functions:
return gcd(m,n);
}
f (n) = 2n
g(n) = n!
How many recursive calls are made by this function?
[2007] h(n) = nlogn
(A) Θ(log 2 n) (B) Ω( n ) Which of the following statements about the asymp-
totic behavior of f (n), g(n), and h(n) is true? [2008]
(C) Θ(log 2 log 2 n) (D) Θ( n ) (A) f (n) = O(g(n)); g(n) = O(h(n))
5. What is the time complexity of the following recursive (B) f (n) = W(g(n)); g(n) = O(h(n))
function: (C) g(n) = O(f (n)); h(n) = O(f (n))
int DoSomething (int n) { (D) h(n) = O(f (n)); g(n) = W(f (n))
if (n <= 2) 10. The minimum number of comparisons required to
return 1; determine if an integer appears more than n/2 times
else in a sorted array of n integers is [2008]
return(DoSomething (floor(sqrt(n)))+ n);} [2007] (A) Q(n) (B) Q(log n)
2 (C) Q(log * n) (D) Q(1)
(A) Θ(n ) (B) Θ(n log 2 n)
11. We have a binary heap on n elements and wish to
(C) Θ(log 2 n) (D) Θ(log 2 log 2 n) insert n more elements (not necessarily one after
6. An array of n numbers is given, where n is an even another) into this heap. The total time required for this
number. The maximum as well as the minimum of is [2008]
3.94 | Unit 3 • Algorithms

(A) Q(log n) (B) Q(n) executed on input of size n. Which of the following is
(C) Q(n log n) (D) Q(n2) ALWAYS TRUE? [2012]
12. The running time of an algorithm is represented by (A) A(n) = W(W(n)) (B) A(n) = Q(W(n))
the following recurrence relation: [2009] (C) A(n) = O(W(n)) (D) A(n) = o(W(n))
17. The recurrence relation capturing the optimal execu-
n n ≤ 3 tion time of the Towers of Hanoi problem with n discs

T ( n) =   n  is [2012]
T  3  + cn otherwise (A) T(n) = 2T(n – 2) + 2
  
(B) T(n) = 2T(n – 1) + n
Which one of the following represents the time com-
(C) T(n) = 2T(n/2) + 1
plexity of the algorithm?
(D) T(n) = 2T(n – 1) + 1
(A) q(n) (B) q(n log n) 18. A list of n strings, each of length n, is sorted into
(C) q(n2) (D) q(n2 log n) lexicographic order using the merge sort algorithm.
13. Two alternative packages A and B are available for The worst-case running time of this computation is
processing a database having 10k records. Package A [2012]
requires 0.0001 n2 time units and package B requires (A) O(n log n) (B) O(n2log n)
10n log10 n time units to process n records. What is (C) O(n2 + log n) (D) O(n2)
the smallest value of k for which package B will be 19. Consider the following function:
preferred over A? [2010]
int unknown (int n) {
(A) 12 (B) 10
(C) 6 (D) 5 int i, j, k = 0;
14. An algorithm to find the length of the longest mono- for (i = n/2; i < = n; i++)
tonically increasing sequence of numbers in an array for (j = 2; j < = n; j = j*2)
A[0 : n - 1] is given below. k = k + n/2;
Let L denote the length of the longest monotonically return (k);
increasing sequence starting at index in the array.
}
Initialize Ln-1 = 1,
The return value of the function is [2013]
For all i such that 0 ≤ i ≤ n - 2 (A) Q(n2) (B) Q(n2log n)
Li {= 1 + Li +1 , if A[i ] < A [i + 1], (C) Q(n3) (D) Q(n3log n)
1 otherwise 20. The number of elements that can be sorted in Q(log n)
time using heap sort is [2013]
Finally the length of the longest monotonically
(A) Q(1)
increasing sequence is Max (L0, L1,…Ln–1)
Which of the following statements is TRUE? [2011] (B) Θ( log n )
(A) The algorithm uses dynamic programming para-
 log n 
digm. (C) Θ  
(B) The algorithm has a linear complexity and uses  log log n 
branch and bound paradigm. (D) Q(log n)
(C) The algorithm has a non-linear polynomial com-
21. Which one of the following correctly determines the
plexity and uses branch and bound paradigm.
solution of the recurrence relation with T(1) = 1
(D) The algorithm uses divide and conquer paradigm.
n
15. Which of the given options provides the increasing T (n) = 2T   + log n ? [2014]
order of asymptotic complexity of functions f1, f2, f3 2
and f4? [2011] (A) q(n) (B) q(n log n)
(C) q(n2) (D) q(log n)
f1(n) = 2n
22. An algorithm performs (log N)1/2 find operations, N
f2(n) = n3/2 insert operations, (log N)1/2 delete operations, and (log
f3(n) = n log2n N)1/2 decrease-key operations on a set of data items
f 4 (n) = n log 2 n with keys drawn from a linearly ordered set. For a
(A) f3, f2, f4, f1 (B) f3, f2, f1, f4 delete operation, a pointer is provided to the record
(C) f2, f3, f1, f4 (D) f2, f3, f4, f1 that must be deleted For the decrease – key opera-
tion, a pointer is provided to the record that has its
16. Let W(n) and A(n) denote respectively, the worst-
key decreased Which one of the following data struc-
case and average-case running time of an algorithm
tures is the most suited for the algorithm to use, if the
 Chapter 1 • Asymptotic Analysis | 3.95

goal is to achieve the best total asymptotic complexity (A) 4 (B) 5

considering all the operations? [2015] (C) 2 (D) 3
(A) Unsorted array 28. Let f(n) = n and g(n) = n (1 + sin n)
, where n is a positive
(B) Min-heap integer. Which of the following statements is/are cor-
(C) Sorted array rect? [2015]
(D) Sorted doubly linked list
I. f(n) = O(g(n))
23. Consider the following C function.
II. f(n) = Ω(g(n))
int fun1(int n) { (A) Only I (B) Only II
int i, j, k, p, q=0; (C) Both I and II (D) Neither I nor II
for (i=1; i<n; ++i) { 29. A queue is implemented using an array such that
p=0; ENQUEUE and DEQUEUE operations are per-
for (j=n; j>1; j=j/2) formed efficiently. Which one of the following state-
++p; ments is CORRECT (n refers to the number of items
for (k=1; k<p; k=k*2) in the queue)? [2016]
++q; (A) Both operations can be performed in O(1) time.
} (B) At most one operation can be performed in O(1)
return q; time but the worst case time for the other opera-
} tion will be W (n).
(C) The worst case time complexity for both opera-
tions will be W (n).
Which one of the following most closely approxi-
(D) Worst case time complexity for both operations
mates the return value of the function fun1? [2015]
will be W (logn).
(A) n3 (B) n(log n)2
(C) n log n (D) n log(log n) 30. Consider a carry look ahead adder for adding two n -
bit integers, built using gates of fan - in at most two.
24. An unordered list contains n distinct elements. The
The time to perform addition using this adder is
number of comparisons to find an element in this list
[2016]
that is neither maximum nor minimum is [2015]
(A) q(n log n) (B) q(n) (A) Q(1) (B) Q(log(n))
(C) q(log n) (D) q(1) (C) Q n ( ) (D) Q(n)
25. Consider a complete binary tree where the left and the 31. N items are stored in a sorted doubly linked list. For a
right subtrees of the root are max-heaps. The lower delete operation, a pointer is provided to the record to
bound for the number of operations to convert the tree be deleted. For a decrease - key operation, a pointer is
to a heap is [2015] provided to the record on which the operation is to be
(A) Ω(log n) (B) Ω(n) performed.
(C) Ω(n log n) (D) Ω(n2) An algorithm performs the following operations on
n the list in this order: Q (N) delete, O (logN) insert,
26. Consider the equality ∑ i 3 = and the following O (log N) find, and Q (N) decrease - key. What is the
choices for X i =0
time complexity of all these operations put together?
1. q(n ) 4
[2016]
2. q(n5) (A) O (log2N) (B) O(N)
3. O(n5) (C) O(N2) (D) Q (N2logN)
4. Ω(n3)
32. In an adjacency list representation of an undirected
The equality above remains correct if X is replaced by simple graph G = (V,E), each edge (u,v) has two adja-
[2015] cency list entries:[v] in the adjacency list of u, and
(A) Only 1 [u] in the adjacency list of v. These are called twins of
(B) Only 2 each other. A twin pointer is a pointer from an adja-
(C) 1 or 3 or 4 but not 2 cency list entry to its twin. If |E| = m and |V| = n, and
(D) 2 or 3 or 4 but not 1 the memory size is not a constraint, what is the time
27. Consider the following array of elements complexity of the most efficient algorithm to set the
<89, 19, 50, 17, 12, 15, 2, 5, 7, 11, 6, 9, 100> twin pointer in each entry in each adjacency list?
[2016]
The minimum number of interchanges needed to con-
(A) Q(n2) (B) Q(n + m)
vert it into a max-heap is [2015]
(C) Q(m2) (D) Q(n4)
3.96 | Unit 3 • Algorithms

33. Consider the following functions from positive inte- }

gers to real numbers: Time complexity of fun in terms of Q notation is
100 [2017]
10, n , n, log2 n, .
n
(
(A) Θ n n ) (B) Q(n2)
The CORRECT arrangement of the above functions
(C) Q(n log n) (D) Q(n2 log n)
in increasing order of asymptotic complexity is:
[2017] 36. A queue is implemented using a non-circular sin-
gly linked list. The queue has a head pointer and a
100 tail pointer, as shown in the figure. Let n denote the
(A) log 2 n, ,10, n, n
n number of nodes in the queue. Let enqueue be imple-
100 mented by inserting a new node at the head, and
(B) ,10, log 2 n, n , n dequeue be implemented by deletion of a node from
n
the tail.
100
(C) 10, , n , log 2 n, n
n
100
(D) , log 2 n, 10, n , n head
n tail

34. Consider the recurrence function Which one of the following is the time complexity
of the most time-efficient implementation of enqueue
2T
T (n) = 
 ( n ) +1 n>2 and dequeue, respectively, for this data structure?
[2018]
2,
 0<n≤2 (A) q(1), q(1) (B) q(1), q(n)
(C) q(n), q(1) (D) q(n), q(n)
Then T(n) in terms of Q notation is [2017]
(A) Q (log log n) (B) Q (log n) 37. Consider the following C code. Assume that unsigned
long int type length is 64 bits.
(C) Θ ( n) (D) Q (n) unsigned long int fun (unsigned long int n) {
unsigned long int i, j = 0, sum = 0;
35. Consider the following C function.
for (i = n; i > 1. i = i/2) j++;
int fun (int n) {
for (; j > 1; j = j/2) sum++;
int i, j;
for(i = 1; i <= n; i++) { return (sum);
for (j = l; j < n; j += i) { }
printf{“ %d %d”,i, j); The value returned when we call fun with the input 240
} is: [2018]
} (A) 4 (B) 5
(C) 6 (D) 40
 Chapter 1 • Asymptotic Analysis | 3.97

ANSWER KEYS
EXERCISES
Practice Problems 1
1. A 2. D 3. D 4. A 5. B 6. A 7. D 8. A 9. A 10. B
11. A 12. A 13. B 14. A 15. B

Practice Problems 2
1. A 2. A 3. A 4. B 5. A 6. A 7. D 8. B 9. A 10. A
11. B 12. A 13. A 14. B 15. C

Previous Years’ Questions

1. B 2. C 3. A 4. C 5. 6. B 7. B 8. C 9. D 10. A
11. B 12. A 13. C 14. A 15. A 16. C 17. D 18. B 19. B 20. C
21. A 22. A 23. D 24. D 25. A 26. C 27. D 28. D 29. A 30. B
31. C 32. B 33. B 34. B 35. C 36. B 37. B
 Chapter 2
Sorting Algorithms

LEARNING OBJECTIVES

 Sorting algorithms  Binary search trees

 Merge sort  Heap sort
 Bubble sort  Sorting–performing delete max operations
 Insertion sort  Max-heap property
 Selection sort  Min-heap property
 Selection sort algorithm  Priority queues

SORTING ALGORITHMS digit is in the set {1, 2,… k}, then radix sort can sort the num-
bers in q (d (n + k)) time. Where ‘d’ is constant. Radix sort runs
Purpose of sorting in linear time.
Sorting is a technique which reduces problem complexity and • Bucket sort, requires knowledge of the probabilistic distribution
search complexity. of numbers in the input array.
• Insertion sort takes q (n2) time in the worst case. It is a fast
inplace sorting algorithm for small input sizes.
• Merge sort has a better asymptotic running time q (n log n), but
it does not operate in inplace. MERGE SORT
• Heap sort, sorts ‘n’ numbers inplace in q (n log n) time, it uses a Suppose that our division of the problem yields ‘a’ sub problems,
data structure called heap, with which we can also implement a
1
priority queue. each of which is   th size of the original problem. For merge
• Quick sort also sorts ‘n’ numbers in place, but its worst – case b
running time is q (n2). Its average case is q (n log n). The con- sort, both a and b are 2, but sometimes a ≠ b. If we take D(n) time
stant factor in quick sort’s running time is small, This algorithm to divide the problem into sub problems and C(n) time to combine
performs better for large input arrays. the solutions of the sub problems into the solution to the original
• Insertion sort, merge sort, heap sort, and quick sort are all com- problem. The recurrence relation for merge sort is
parison based sorts; they determine the sorted order of an inpu-
θ (1) if n ≤ c,
tarray by comparing elements. T ( n) = 
• We can beat the lower bound of W (n log n) if we can gather aT ( n /b) + D( n) + C ( n) otherwise
information about the sorted order of the input by means other
than comparing elements. Running time is broken down as follows:
• The counting sort algorithm, assumes that the input numbers are Divide: This step computes the middle of the sub array, which
in the set {1, 2, …. k}. By using array indexing as a tool for takes constant time q (1).
determining relative order, counting sort can sort n numbers in
q (k + n) time. Thus counting sort runs in time that is linear in Conquer: We solve 2 sub problems of size (n/2) each recursively
size of the input array. which takes 2T(n/2) time.
• Radix sort can be used to extend the range of counting sort. If Combine: Merge sort procedure on an n-element sub array takes
there are ‘n’ integers to sort, each integer has ‘d’ digits, and each time q (n).
 Chapter 2 • Sorting Algorithms | 3.99

• Worst case running time T(n) of merge sort The array is already sorted, but our algorithm does not
know if it is completed. The algorithm needs one whole
0(1) if n ≤1 pass without any swap to know it is sorted.
T ( n) = 
 aT ( n / 2) + θ ( n) if n > 1 Third pass:
cn
(1 2 4 5 8) → (1 2 4 5 8)
cn
(1 2 4 5 8) → (1 2 4 5 8)
cn cn
cn (1 2 4 5 8) → (1 2 4 5 8)
2
2 (1 2 4 5 8) → (1 2 4 5 8)
Finally the array is sorted, and the algorithm can terminate.
log n cn cn cn cn cn
4 4 4 4 Algorithm
void bubblesort (int a [ ], int n)
{
int i, j, temp;
c c c c c c c c cn
for (i=0; i < n-1; i++)
{
Total: cn log n + cn for (j=0; j < n – 1 – i; j++)
Figure 1 Recurrence tree
if (a [j] > a [j + 1])
{
The top level has total cost ‘cn’, the next level has total cost temp = a [j + 1];
c(n/2) + c(n/2) = cn and the next level has total cost c(n/4) a [j + 1] = a [j];
+ c(n/4) + c(n/4) + c(n/4) = cn and so on. The ith level has a [j] = temp;
total cost 2i c (n/2i) = cn. At the bottom level, there are ‘n’ }
nodes, each contributing a cost of c, for a total cost of ‘cn’. }
The total number of levels of the ‘recursion tree’ is log n + 1. }
There are log n + 1 levels, each costing cn, for a total cost
of cn (log n + 1) = cn log n + cn ignoring the low–order term INSERTION SORT
and the constant c, gives the desired result of q (n log n). Insertion sort is a comparison sort in which the sorted array is
built one entry at a time. It is much less efficient on large lists
than more advanced algorithms such a quick sort, heap sort,
BUBBLE SORT (or) merge sort. Insertion sort provides several advantages.
Bubble sort is a simple sorting algorithm that works by
• Efficient for small data sets.
repeatedly stepping through the list to be sorted, compar-
• Adaptive, i.e., efficient for data set that are already sub-
ing each pair of adjacent items, and swapping them if they
stantially sorted. The complexity is O(n + d), where d is
are in the wrong order. The pass through the list is repeated
the number of inversions.
until no swaps are needed, which indicates that the list is
• More efficient in practice than most other simple quad-
sorted. The algorithm gets its name from the way smaller
ratic, i.e., O(n2) algorithms such as selection sort (or)
elements ‘bubble’ to the top of the list.
bubble sort, the best case is O(n).
Example: Take the array of numbers ‘5 1 4 2 8’and sort the • Stable, i.e., does not change the relative order of elements
array from lowest number to greatest number using bubble with equal keys.
sort algorithm. In each step, elements underlined are being • In-place i.e., only requires a constant amount O(1) of
compared. additional memory space.
First pass: • Online, i.e., can sort a list as it receives it.
(5 1 4 2 8) → (1 5 4 2 8), here algorithm Algorithm
compares the first 2 elements and swaps them
Insertion sort (A)
(1 5 4 2 8) → (1 4 5 2 8), swap (5 > 4)
For (j ← 2) to length [A]
(1 4 5 2 8) → (1 4 2 5 8), swap (5 > 2)
Do key ← A [j]
(1 4 2 5 8) → (1 4 2 5 8), since these ele-
i ←j – 1;
ments are already in order, algorithm does not swap them.
While i > 0 and A [i] > key
Second pass: {
(1 4 2 5 8) → (1 4 2 5 8) Do A [i + 1] ← A [i]
(1 4 2 5 8) → (1 2 4 5 8), swap since (4 > 2) i ← i - 1
(1 2 4 5 8) → (1 2 4 5 8) }
(1 2 4 5 8) → (1 2 4 5 8) A [i + 1] ← key
3.100 | Unit 3 • Algorithms

Every repetition of insertion sort removes an element Analysis

from the input data, inserting it into the correct position Selection sort is not difficult to analyze compared to other
in the already sorted list, until no input element remains. sorting algorithms, since none of the loops depend on the
Sorting is typically done in–place. The resulting array after data in the array selecting the lowest element requires scan-
K iterations has the property where the first k + 1 entries are ning all n elements (this takes n – 1 comparisons) and then
sorted. In each iteration the first remaining entry of the input swapping it into the first position. Finding the next lowest
is removed, inserted into the result at the correct position, element requires scanning the remaining n – 1 elements and
with each element greater than X copied to the right as it is so on, for (n – 1) + (n – 2) + … + 2 + 1 = n(n – 1)/2 ∈ q(n2)
compared against. X. comparisons.
Each of these scans requires one swap for n – 1 elements
Performance (the final element is already in place).
• The best case input is an array that is already sorted. In this
case insertion sort has a linear running time (i.e., q (n)). Selection sort Algorithm
• The worst case input is an array sorted in reverse order. First, the minimum value in the list is found. Then, the first
In this case every iteration of the inner loop will scan element (with an index of 0) is swapped with this value.
and shift the entire sorted subsection of the array before Lastly, the steps mentioned are repeated for rest of the array
inserting the next element. For this case insertion sort has (starting at the 2nd position).
a quadratic running time (O(n2)).
Example 1: Here’s a step by step example to illustrate the
• The average case is also quadratic, which makes insertion
selection sort algorithm using numbers.
sort impractical for sorting large arrays, however, inser-
tion sort is one of the fastest algorithms for sorting very Original array: 6 3 5 4 9 2 7
small arrays even faster than quick sort. 1st pass → 2 3 5 4 9 6 7 (2 and 6 were swapped)
2nd pass → 2 3 5 4 9 6 7 (no swap)
Example: Following figure shows the operation of
3rd pass → 2 3 4 5 9 6 7 (4 and 5 were swapped)
insertion sort on the array A = (5, 2, 4, 6, 1, 3). Each part
4th pass → 2 3 4 5 6 9 7 (6 and 9 were swapped)
shows what happens for a particular iteration with the value
5th pass → 2 3 4 5 6 7 9 (7 and 9 were swapped)
of j indicated. j indexes the ‘Current card’ being inserted.
6th pass → 2 3 4 5 6 7 9 (no swap)
j j Note: There are 7 keys in the list and thus 6 passes were
5 2 4 6 1 3 2 5 4 6 1 3 required. However, only 4 swaps took place.
Example 2: Original array: LU, KU, HU, LO, SU, PU
1st pass → HU, KU, LU, LO, SU, PU
j j 2nd pass → HU, KU, LU, LO, SU, PU
2 4 5 6 1 3 2 4 5 6 1 3 3rd pass → HU, KU, LO, LU, SU, PU
4th pass → HU, KU, LO, LU, SU, PU
5th pass → HU, KU, LO, LU, PU, SU
j Note: There were 6 elements in the list and thus 5 passes
1 2 4 5 6 3 1 2 3 4 5 6 were required. However, only 3 swaps took place.

Read the figure row by row. Elements to the left of A[ j] that BINARY SEARCH TREES
are greater than A[ j] move one position to the right and A[ j] Search trees are data structures that support many
moves into the evacuated position. dynamic, set operations, including SEARCH, MINIMUM,
MAXIMUM, PREDECESSOR, SUCCESSOR, INSERT
SELECTION SORT and DELETE. A search tree can be used as a dictionary and
as a priority Queue. Operations on a binary search tree take
Selection sort is a sorting algorithm, specifically an
time proportional to the height of the tree. For a complete
in-place comparison sort. It has O(n2) complexity, making it
binary tree with ‘n’ nodes, basic operations run in q(log n)
inefficient on large lists.
worst-case time. If the tree is a linear chain of ‘n’ nodes, the
The algorithm works as follows:
basic operations take q(n) worst-case time.
1. Find the minimum value in the list. A binary search tree is organized, in a binary tree such a
2. Swap it with the value in the first position. tree can be represented by a linked data structure in which
3. Repeat the steps above for the remainder of the list each node is an object. In addition to key field, each node
(starting at the second position and advancing each contains fields left, right and P that point to the nodes cor-
time). responding to its left child, its right child, and its parent,
 Chapter 2 • Sorting Algorithms | 3.101

respectively. If the child (or) parent is missing, the appropri- the partially sorted array. After removing the largest item, it
ate field contains the value NIL. The root node is the only reconstructs heap, removes the largest remaining item, and
node in the tree whose parent field is NIL. places, it in the next open position from the end of the par-
tially sorted array. This is repeated until there are no items
Binary search tree property left in the heap and the sorted array is full. Elementary
The keys in a binary search tree are always stored in such a implementations require two arrays one to hold the heap
way as to satisfy the binary search tree property. and the other to hold the sorted elements.
Let ‘a’ be a node in a binary search tree. If ‘b’ is a node • Heap sort inserts the input list elements into a binary
in the left sub tree of ‘a’, key [b] ≤ key [a] heap data structure. The largest value (in a max-heap) or
If ‘b’ is a node in the right sub tree of ‘a’ then key [a] ≤ key the smallest value (in a min-heap) is extracted until none
[b]. remain, the value having been extracted in sorted order.
8 Example: Given an array of 6 elements: 15, 19, 10, 7, 17,
10 16, sort them in ascending order using heap sort.
7
14 Steps:
6 9

Figure 2 Binary search tree. 1. Consider the values of the elements as priorities and
build the heap tree.
The binary search tree property allows us to print out all 2. Start delete Max operations, storing each deleted
keys in a binary search tree in sorted order by a simple element at the end of the heap array.
recursive algorithm called an inorder tree.
If we want the elements to be sorted in ascending order, we
Algorithm need to build the heap tree in descending order-the greatest
element will have the highest priority.
INORDER-TREE-WALK (root [T ])
INORDER-TREE-WALK (a) 1. Note that we use only array, treating its parts
differently,
1. If a ≠ NIL 2. When building the heap-tree, part of the array will be
2. Then INORDER-TREE-WALK (left [a]) considered as the heap, and the rest part-the original
3. Print key [a] array.
4. INORDER-TREE-WALK (right [a]) 3. When sorting, part of the array will be the heap and
It takes q(n) time to walk an n-node binary search tree, since the rest part-the sorted array.
after the initial call, the procedure is called recursively twice Here is the array: 15, 19, 10, 7, 17, 6.
for each node in the tree.
Let T(n) denote the time taken by IN-ORDER-TREE-
WALK, when it is called on the root of an n-node subtree. Building the Heap Tree
INORDER-TREE-WALK takes a small, constant The array represented as a tree, which is complete but not
amount of time on an empty sub-tree (for the test x ≠ NIL). ordered.
So T(1) = C for some positive constant C.
For n > 0, suppose that INORDER-TREE-WALK is 15 19 10 7 17 16
15
called on a node ‘a’ whose left subtree has k nodes and
whose right subtree has n – k – 1 nodes. 19 10
The time to perform in order traversal is
T(n) = T(k) + T(n – k – 1) + d.
7 17
For some positive constant ‘d’ that reflects the time to 16

execute in-order (a), exclusive of the time spent in recursive

Start with the right most node at height 1 – the node at posi-
calls T(n) = (c + d) n + c.
tion 3 = size/2. It has one greater child and has to be perco-
For n = 0, we have (c + d) 0 + c = T(0),
lated down.
For n > 0,
T(n) = T(k) + T(n – k – 1) + d
15
= ((c + d)(k + c) + ((c + d) (n – k – 1) + c) + d 15 19 10 7 17 16
= (c + d) n + c – (c + d) + c + d = (c + d)n + c
19 10
HEAP SORT
Heap sort begins by building a heap out of the data set, and 7 17 16
then removing the largest item and placing it at the end of
3.102 | Unit 3 • Algorithms

After processing array [3] the situation is: 17 16 7 15 19

15
15 19 16 7 17 10 10

19 16 Percolate down the hole

10
7 17 17 16 7 15 19 17
10

Next comes array [2]. Its children are smaller, so no perco- 10 16

lation is needed.
The last node to be processed is array[1]. Its left 7 15
child is the greater of the children. The item at array
Percolate once more (10 is less than 15, so it cannot be
[1] has to be percolated down to the left, swapped with
inserted in the previous hole)
array [2].
17
15 17 15 16 7 19
15 19 16 7 17 10
15 16
19 16 10

7
7 17 10
Now 10 can be inserted in the hole
As a result:
17
17 15 16 7 10 19
19
19 15 16 7 17 10 15 16
15 16
7 10
7 17 10
Repeat the step B till the array is sorted.
The children of array [2] are greater and item 15 has to be
moved down further, swapped with array [5]. Heap sort analysis
Heap sort uses a data structure called (binary) heap binary,
19 heap is viewed as a complete binary tree. An Array A that
19 17 16 7 15 10
represents a heap is an object with 2 attributes: length [A],
17 16 which is the number of elements in the array and heap size
[A], the number of elements in the heap stored within array A.
7 15
No element past A [heap size [A]], where heap size [A] ≤
10
length [A], is an element of the heap.
There are 2 kinds of binary heaps:
Now the tree is ordered, and the binary heap is built.
1. Max-heaps
Sorting-performing Delete 2. Min-heaps
Max Operations In both kinds the values in the nodes satisfy a heap-property.
Delete the top element Max-heap property A[PARENT (i)] ≥A[i]
Store 19 in a temporary place, a hole is created at the top. The value of a node is almost the value of its parent. Thus the
largest element in a max-heap is stored at the root, and the
sub tree rooted at a node contains values no larger than that
17 16 7 15 10
contained at the node itself.
17 16
19 Min-heap property For every mode ‘i’ other than the root
[PARENT (i)] ≤ A[i]. The smallest element in a min-heap
7 15 10 is at the root.
Max-heaps are used in heap sort algorithm.
Swap 19 with the last element of the heap. As 10 will be Min-heaps are commonly used in priority queues.
adjusted in the heap, its cell will no longer be a part of the
heap. Instead it becomes a cell from the sorted array
 Chapter 2 • Sorting Algorithms | 3.103

Basic operations on heaps run in time almost propor- Priority Queues

tional to the height of the tree and thus take O(log n) The most popular application of a heap is its use as an effi-
time cient priority queue.
• MAX-HEAPIFY procedure, runs in O(log n) time. A priority queue is a data structure for maintaining a set
• BUILD-MAX-HEAP procedure, runs in linear time. S of elements, each with an associated value called a key. A
• HEAP SORT procedure, runs in O(n log n) time, sorts an max-priority queue supports the following operations:
array in place. INSERT: INSERT (s, x) inserts the element x into the set S.
• MAX-HEAP-INSERT This operation can be written as S ← S U {x}.
HEAP- EXTRACT-MAX MAXIMUM: MAXIMUM (S) returns the element of S
HEAP-INCREASE-KEY with the largest key
HEAP-MAXIMUM
EXTRACT-MAX: EXTRACT-MAX(S) removes and returns
All these procedures, run in O(log n) time, allow the heap
the element of S with the largest key.
data structure to be used as a priority queue.
• Each call to MAX-HEAPIFY costs O(log n) time, and there INCREASE-KEY: INCREASE-KEY(s, x, k) increases
are O(n) such calls. Thus, the running time is O(n log n) the value of element x’s key to the new value k, which is
• The HEAPSORT procedure takes time O(n log n), since assumed to be atleast as large as x’s current key value.
the call to BUILD-MAX-HEAP takes time O(n) and each One application of max–priority queue is to schedule
of the (n - 1) calls to MAX-HEAPIFY takes time O(log n). jobs on a shared computer.

EXERCISES

Practice Problems 1 i←j–1

Directions for questions 1 to 15: Select the correct alterna- While i > 0 and A [ i ] > key
tive from the given choices. do A [i + i] ← A [ i ]
1. Solve the recurrence relation T(n) = 2T(n/2) + k.n i←i–1
where k is constant then T(n) is A [i + 1] ← key
(A) O(log n) (B) O(n log n) (A) Selection sort (B) Insertion sort
(C) O(n) (D) O(n2) (C) Quick sort (D) Merge sort
2. What is the time complexity of the given code? 5. What is the order of elements after 2 iterations of the
Void f(int n) above-mentioned sort on given elements?
{
if (n > 0) 8 2 4 9 3 6
f (n/2);
} (A) 2 4 9 8 3 6
(A) q(log n) (B) q(n log n)
(C) q(n2) (D) q(n)
(B) 2 4 8 9 3 6
3. The running time of an algorithm is represented by the
following recurrence relation;
(C) 2 4 6 3 8 9
n n≤3

T ( n) =   n  (D) 2 4 6 3 8 9
T  3  + cn otherwise
  
Common data for questions 6 and 7:
What is the time complexity of the algorithm?
6. The following pseudo code does which sort?
(A) q(n) (B) q(n log n)
(C) q(n2) (D) q(n2 log n) 1. If n = 1 done
2. Recursively sort
Common data for questions 4 and 5:
A [1…[n/2] ]and
4. The following pseudo code does which sorting? A [[n/2] + 1 … n]
xsort [A, n] 3. Combine 2 ordered lists
for j ← 2 to n (A) Insertion sort (B) Selection sort
do key ← A [ i ] (C) Merge sort (D) Quick sort
3.104 | Unit 3 • Algorithms

7. What is the complexity of the above pseudo code? sorted part of the array. If binary search is used instead
(A) q(log n) (B) q(n2) of linear search to identify the position, the worst case
(C) q(n log n) (D) q(2n) running time would be.
8. Apply Quick sort on a given sequence 6 10 13 5 8 3 2 (A) q (n log n)
11. What is the sequence after first phase, pivot is first (B) q (n2)
element? (C) q (n(log n)2)
(A) 5 3 2 6 10 8 13 11 (D) q (n)
(B) 5 2 3 6 8 13 10 11 13. Consider the process of inserting an element into a
(C) 6 5 13 10 8 3 2 11 max heap where the max heap is represented by an
(D) 6 5 3 2 8 13 10 11 array, suppose we perform a binary search on the path
9. Selection sort is applied on a given sequence: from the new leaf to the root to find the position for
the newly inserted element, the number of comparisons
89, 45, 68, 90, 29, 34, 17. What is the sequence after 2
performed is:
iterations?
(A) q (log n) (B) q (log log n)
(A) 17, 29, 68, 90, 45, 34, 89
(C) q (n) (D) q (n log n)
(B) 17, 45, 68, 90, 29, 34, 89
(C) 17, 68, 45, 90, 34, 29, 89 14. Consider the following algorithm for searching a given
(D) 17, 29, 68, 90, 34, 45, 89 number ‘X’ in an unsorted array A[1 … n] having ‘n’
 n  distinct values:
10. Suppose there are log n sorted lists of   elements
 log n  (1) Choose an ‘i’ uniformly at random from 1 … n
each. The time complexity of producing sorted lists of (2) If A [i ] = x
all these elements is: (hint: use a heap data structure)
Then stop
(A) q(n log log n) (B) q(n log n)
(C) W(n log n) (D) W(n3/2) else
11. If Divide and conquer methodology is applied on power- goto(1);
ing a Number Xn. Which one the following is correct? Assuming that X is present in A, what is the expected
(A) Xn = Xn/2 ⋅ Xn/2 number of comparisons made by the algorithm before
n −1 n −1 it terminates.
(B) X n = X 2
⋅X 2
. X (A) n (B) n – 1
n +1 n (C) 2n (D) n/2
(C) X n = X ⋅ X 2 2
15. The recurrence equation for the number of additions
(D) Both (A) and (B)
A(n) made by the divide and conquer algorithm on
12. The usual q(n2) implementation of insertion sort to input size n = 2K is
sort an array uses linear search to identify the posi- (A) A(n) = 2A(n/2)+ 1 (B) A(n) = 2A(n/2) + n2
tion, where an element is to be inserted into the already (C) A(n) = 2A(n/4) + n2 (D) A(n) = 2A(n/8) + n2

Practice Problems 2 (a) A result of cutting a problem size by a constant

Directions for questions 1 to 15: Select the correct alterna- factor on each iteration of the algorithm.
tive from the given choices. (b) Algorithm that scans a list of size ‘n’.
(c) Many divide and conquer algorithms fall in this
1.
category.
Linear Search Binary search
Input Array W (n ) W (n ) (d) Typically characterizes efficiency of algorithm
128 128 8
with two embedded loops.
elements (A) i – b, ii – c, iii – a, iv – d
1024 1024 x (B) i – a, ii – b, iii – c, iv – d
elements (C) i – c, ii – d, iii – a, iv – b
Find x value? (D) i – d, ii – a, iii – b, iv – c
(A) 10 (B) 11 3. Insertion sort analysis in worst case
(C) 12 (D) 13 (A) q (n)
2. Choose the correct one (B) q (n2)
(i) log n (ii) n (C) q (n log n)
(iii) n log n (iv) n2 (D) q (2n)
 Chapter 2 • Sorting Algorithms | 3.105

4. From the recurrence relation. Of merge sort 10. Which one of the following in-place sorting algorithm
T(n) = 2T (n/2) + q(n). needs the minimum number of swaps?
Which option is correct? (A) Quick sort (B) Insertion sort
I. n/2 II. 2T III. q (n) (C) Selection sort (D) Heap sort
(a) Extra work (divide and conquer) 11. As the size of the array grows what is the time com-
(b) Sub-problem size plexity of finding an element using binary search (array
of elements are ordered)?
(c) Number of sub-problems (A) q(n log n) (B) q(log n)
(A) III – b, II – a, I – c (B) I – b, II – c, III – a (C) q(n2) (D) q(n)
(C) I – a, II – c, III – b (D) I – c, II – a, III – b
12. The time complexity of heap sort algorithm is
5. What is the number of swaps required to sort ‘n’ ele- (A) n log n (B) log n
ments using selection sort, in the worst case? (C) n2 (D) None of these.
(A) q(n) (B) q(n2)
13. As part of maintenance work, you are entrusted with
(C) q(n log n) (D) q(n2 log n)
the work of rearranging the library books in a shelf in a
6. In a binary max heap containing ‘n’ numbers, the proper order, at the end of each day. The ideal choices
smallest element can be found in time will be_____.
(A) O(n) (B) O(log n) (A) Heap sort (B) Quick sort
(C) O(log log n) (D) O(1) (C) Selection sort (D) Insertion sort
7. What is the worst case complexity of sorting ‘n’ num- 14. The value for which you are searching is called
bers using quick sort? (A) Binary value
(A) q(n) (B) q(n log n) (B) Search argument
(C) q(n2) (D) q(n !) (C) Key
8. The best case analysis of quick sort is, if partition splits (D) Serial value
the array of size n into 15. To sort many large objects and structures it would be
(A) n/2 : n/m (B) n/2 : n/2 most efficient to _____.
(C) n/3 : n/2 (D) n/4 : n/2 (A) Place them in an array and sort the array
9. What is the time complexity of powering a number, by (B) Place the pointers on them in an array and sort the
using divide and conquer methodology? array
(A) q (n2) (B) q (n) (C) Place them in a linked list and sort the linked list
(C) q(log n) (D) q(n log n) (D) None of the above

PREVIOUS YEARS’ QUESTIONS

1. What is the number of swaps required to sort n ele- 4. The minimum number of comparisons required to
ments using selection sort, in the worst case? [2009] find the minimum and the maximum of 100 numbers
(A) q(n) is ––––––. [2014]
(B) q(n log n) 5. Suppose P, Q, R, S, T are sorted sequences having
(C) q(n2) lengths 20, 24, 30, 35, 50 respectively. They are to be
(D) q(n2 log n) merged into a single sequence by merging together
two sequences at a time. The number of comparisons
2. Which one of the following is the tightest upper that will be needed in the worst case by the optimal
bound that represents the number of swaps required algorithm for doing this is –––––. [2014]
to sort n numbers using selection sort? [2013]
(A) O(log n) (B) O(n) 6. You have an array of n elements. Suppose you imple-
(C) O(n log n) (D) O(n2) ment quick sort by always choosing the central
element of the array as the pivot. Then the tight-
3. Let P be a quick sort program to sort numbers in est upper bound for the worst case performance is
ascending order using the first element as the pivot. [2014]
Let t1 and t2 be the number of comparisons made by P (A) O(n2) (B) O(n log n)
for the inputs [1 2 3 4 5] and [4 1 5 3 2] respectively. (C) q(n log n) (D) O(n3)
Which one of the following holds? [2014]
7. What are the worst-case complexities of insertion and
(A) t1 = 5 (B) t1 < t2
deletion of a key in a binary search tree? [2015]
(C) t1 > t2 (D) t1 = t2
3.106 | Unit 3 • Algorithms

(A) q(log n) for both insertion and deletion (A) O(1) (B) O(d) but not O(1)
(B) q(n) for both insertion and deletion (C) O(2d) but not O(d) (D) O(d2d) but not O(2d)
(C) q(n) for insertion and q(log n) for deletion 10. Assume that the algorithms considered here sort the
(D) q(log n) for insertion and q(n) for deletion input sequences in ascending order. If the input is
8. The worst case running times of Insertion sort, Merge already in ascending order, which of the following are
sort and Quick sort, respectively, are: [2016] TRUE? [2016]
(A) Θ(n log n), Θ(n log n),and Θ(n2) I. Quicksort runs in Θ (n2) time
(B) Θ(n2), Θ(n2),and Θ(n log n) II. Bubblesort runs in Θ (n2) time
(C) Θ(n2), Θ(n log n),and Θ(n log n) III. Mergesort runs in Θ (n) time
(D) Θ(n2), Θ(n log n),and Θ(n2) IV. Insertion sort runs in Θ (n) time
9. An operator delete(i) for a binary heap data structure (A) I and II only (B) I and III only
is to be designed to delete the item in the i-th node. (C) II and IV only (D) I and IV only
Assume that the heap is implemented in an array and i
11. A complete binary min - heap is made by including
refers to the i-th index of the array. If the heap tree has
each integer in [1,1023] exactly once. The depth of
depth d (number of edges on the path from the root
a node in the heap is the length of the path from the
to the farthest leaf), then what is the time complexity
root of the heap to that node. Thus, the root is depth 0.
to re-fix the heap efficiently after the removal of the
The maximum depth at which integer 9 can appear is
element? [2016]
_____ . [2016]

ANSWER KEYS

EXERCISES
Practice Problems 1
1. B 2. A 3. A 4. B 5. B 6. C 7. C 8. B 9. A 10. B
11. D 12. A 13. A 14. B 15. A

Practice Problems 2
1. B 2. B 3. B 4. B 5. A 6. A 7. C 8. B 9. C 10. C
11. B 12. A 13. D 14. C 15. B

Previous Years’ Questions

1. A 2. B 3. C 4. 148 5. 358 6. A 7. B 8. D 9. B 10. D
11. 8
 Chapter 3
Divide-and-conquer

LEARNING OBJECTIVES

 Divide-and-conquer  Performance of quick sort

 Divide-and-conquer examples  Recurrence relation
 Divide-and-conquer technique  Searching
 Merge sort  Linear search
 Quick sort  Binary search

DIVIDE-AND-CONQUER A problem of size n

Divide-and-conquer is a top down technique for designing algo-
rithms that consists of dividing the problem into smaller sub prob-
lems hoping that the solutions of the sub problems are easier to
find and then composing the partial solutions into the solution of
Sub-problem 1 Sub-problem 2
the original problem. of size n/2 of size n/2
Divide-and-conquer paradigm consists of following major phases:
• Breaking the problem into several sub-problems that are similar
to the original problem but smaller in size.
• Solve the sub-problem recursively (successively and independently)
• Finally, combine these solutions to sub-problems to create a A solution to sub- A solution to sub-
solution to the original problem. problem 1 problem 2

Divide-and-Conquer Examples
• Sorting: Merge sort and quick sort
• Binary tree traversals
• Binary Search A solution to the
• Multiplication of large integers original problem
• Matrix multiplication: Strassen’s algorithm
• Closest-pair and Convex-hull algorithm
Figure 1 Divide-and-conquer technique.

3. Merge the two sorted sub lists back into one sorted list
MERGE SORT 4. The key of merge sort is merging two sorted lists into one,
Merge sort is a sorting algorithm for rearranging lists (or any other such that if we have 2 lists
data structure that can only be accessed sequentially, e.g., file X(x1 ≤ x2 ≤ x3 … ≤ xm) and
streams) into a specified order. Y (y1 ≤ y2 ≤ y3 … ≤ yn) the resulting list is z (z1 ≤ z2 ≤ … ≤ zm+n)
Merge sort works as follows:
1. Divide the unsorted list into two sub lists of about half the size. Example 1: L1 = {3, 8, 9}, L2 = {1, 5, 7}
2. Sort each of the two sub lists. Merge (L1, L2) = {1, 3, 5, 7, 8, 9}
3.108 | Unit 3 • Algorithms

Example 2: Make bold

99 6 86 15 58 35 86 4 0
<Pivot 1 >Pivot 1 >Pivot
99 6 86 15 58 35 86 4 0 Low Pivot 1 Pivot High

99 6 86 15 58 35 86 4 0 Divide: Partition the array A [p - r] into 2 sub arrays A [p

- q – 1] and A [q + 1 - r] such that each element of A [p - q
99 6 86 15 58 35 86 4 0 – 1] is less than or equal to A[q], which is, in turn, less than
or equal to each element of A [q + 1 - r]
4 0
Conquer: Sort the 2 sub arrays A [p - q – 1] and A [q +
Merge: 1 - r] by recursive calls to quick sort.

0 4 6 15 35 58 86 86 99 Combine: Since the sub arrays are sorted inplace, no work

is needed to combine them.
Sort left partition in the same way. For this strategy to
be effective, the partition phase must ensure that the pivot,
6 15 86 99 0 4 35 58 86
is greater than all the items in one part (the lower part) and
less than all those in the other (upper) part. To do this, we
choose a pivot element and arrange that all the items in the
lower part are less than the pivot and all those in the upper
6 99 15 86 35 58 0 4 86 part are greater than it. In the general case, the choice of
pivot element is first element.
(Here number of elements/2 is pivot)
99 6 86 15 58 35 86 0 4
Quick sort (A, 1, 12)
38 81 22 48 13 69 93 14 45 58 79 72
14 58 22 48 13 38 45 69 93 81 79 72
4 0

Quick sort (A, 1, 7) Quick sort (A, 9, 12)

38 58 22 48 13 14 45 93 81 79 72
38 45 22 14 13 48 58 72 79 81 93
Implementing Merge Sort
Merging is done with a temporary array of the same size as
the input array. quick sort (A, 1, 5) quick sort (A, 9, 10)
Pro: Faster than in-place since the temp array holds the result- 38 45 22 14 13 72 79
13 14 22 45 38 72 79
ing array until both left and right sides are merged into the temp
array then the temp array is appended over the input array.
Con: The memory required is doubled. The double mem- quick sort (A, 1, 2) quick sort (A, 4, 5)
ory merge sort runs O(N log N) for all cases, because of its 13 14 45 38
Divide-and-conquer approach. 13 14 38 45

T(N) = 2T(N/2) + N
Figure 2 Tree of recursive calls to quick sort.
= O(N log N)
• Quick sort is a sorting algorithm with worst case run-
QUICK SORT ning time O(n2) on an input array of n numbers. Inspite
Quick sort is an example of Divide-and-conquer strategy. In of this slow worst case running time, quick sort is often
Quick sort we divide the array of items to be sorted into two the best practical choice for sorting because it is effi-
partitions and then call the quick sort procedure recursively cient on the average: its expected running time is O(n
to sort the two partitions, i.e., we divide the problem into log n) and the constants hidden in the O-notation are
two smaller ones and conquer by solving the smaller ones. quite small
The conquer part of the quick sort routine looks like this • Quick sort algorithm is fastest when the median of the array
is chosen as the pivot element. This is because the resulting
<Pivot >Pivot partitions are of very similar size. Each partition splits itself
Low Pivot High in two and thus the base case is reached very quickly.
 Chapter 3 • Divide-and-conquer | 3.109

Example: Underlined element is pivot. The recursion tree for this recurrence has cost ‘cn’ at every
level, until a boundary condition is reached at depth log10n =
3 1 4 5 9 2 6 8 7
q (log n). The recursion terminates at depth log10/8n = q(log n).
3 1 4 5 9 2 6 8 7 The total cost of quick sort is O(n log n)

3 1 4 2 5 9 6 8 7
SEARCHING
Two searching techniques are:
3 1 4 2 9 6 8 7
• Linear search
3 1 4 2 9 6 8 7 • Binary search
1 2 4 3 6 7 8 9
Linear Search
4 3 Linear search (or) sequential search is a method for find-
6 8 9
1 ing a particular value in list that consists of checking every
3 4 one of its elements, one at a time and in sequence, until the
1 2 3 4 6 7 8 9
desired one is found. Linear search is a special case of brute
force search. Its worst case cost is proportional to the num-
ber of elements in the list.
1 2 3 4 5 6 7 8 9

Figure 3 The ideal quick sort on a random array Implementation

boolean linear search (int [ ] arr, int target)
Performance of Quick Sort {
• Running time of quick sort depends on whether the par- int i = 0;
titioning is balanced or unbalanced, it depends on which while (i < arr. length) {
elements are used for partitioning. If the partitioning is if (arr [i] = = target){
balanced, the algorithm runs asymptotically as fast as
return true;
merge sort. If the partitioning is unbalanced, it runs as
}
slowly as insertion sort.
+ + i;
• The worst case of quick sort occurs when the partitioning
}
routine produces one sub-problem with n – 1 elements and
return false;
one with ‘1’ element. If this unbalanced partitioning arises
}
in each recursive call, the partitioning costs q (n) time.
Example:
Recurrence Relation
Consider the array
T(n) = T(n – 1) + T(1) + q (n)
10 7 1 3 –4 2 20
(\ T(0) = q (1))
= T(n – 1) + q (n) Search for 3
If we sum the costs incurred at each level of the recursion 10 7 1 3 –4 2 20
we get an arithmetic series, which evaluates to q (n2). 3?

• Best case partitioning–PARTITION produces 2 sub prob- Move to next element

10 7 1 3 –4 2 20
lems, each of size no more than n/2, since one is of size
3?
 n/2  and one of size  n/ 2  – 1
The recurrence for the running time is then Move to next element
10 7 1 3 –4 2 20
T(n) ≤ 2T(n/2) + q(n)
3?
The above Recurrence relation has the solution T(n) = O(n
Move to next element
log n) by case 2 of the master theorem.
10 7 1 3 –4 2 20
• The average–case time of quick sort is much closer to the
3?
best than to the worst case
Element found; stop the search.
For example, that the partitioning algorithm always pro-
duces a 8-to-2 proportional split, which at first seems unbal- Binary Search
anced. The Recurrence relation will be A binary search algorithm is a technique for finding a particu-
T(n) ≤ T(8n/10) + T(2n/10) + cn lar value in a linear array, by ruling out half of the data at each
3.110 | Unit 3 • Algorithms

step; a binary search finds the median, makes comparison, to Example: Value being searched 123
determine whether the desired value comes before or after it,
and then searches the remaining half in the same manner. A 2 6 7 34 76 123 234 567 677 986
binary search is an example of Divide-and-conquer algorithm.
First (1) mid(5) Last(10)
Implementation 2 6 7 34 76 123 234 567 677 986
function binary search (a, value, left, right)
{ First (6) mid(8) Last(10)
if right < left 2 6 7 34 76 123 234 567 677 986
return not found
mid: = floor ((right –left)/2) + left
First (6) Last(7)
if a [mid] = value Mid (6)
return mid
2 6 7 34 76 123 234 567 677 986
if value < a[mid]
return binary search (a, value, left, mid –1) else return binary search
(a, value, mid + 1, right) First, mid, last (6)
}

EXERCISES
Practice Problems 1 4. What is the depth first search order of the given graph?
Directions for questions 1 to 15: Select the correct alterna-
1 4
tive from the given choices.
1. How many comparisons are required to search an item
89 in a given list, using Binary search? 2

4 8 19 25 34 39 45 48 66 75 89 95
3 5
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]

(A) 3 (B) 4 (A) 14325

(C) 5 (D) 6 (B) 12435
(C) 14253
2. Construct a Binary search tree with the given list of
(D) 12354
elements:
300, 210, 400, 150, 220, 370, 450, 100, 175, 215, 250 5. When pre-order traversal is applied on a given tree,
what is the order of elements?
Which of the following is a parent node of element
250? 1
(A) 220
(B) 150
(C) 370 2 3
(D) 215
3. What is the breadth first search order of the given tree? 4 5

a
(A) 1–2–4–5–3
(B) 1–4–2–5–3
b c (C) 1–2–4–3–5
(D) 1–2–3–4–5
d e f g
6. What is the order of post-order traversal and in-order
traversals of graph given in the above question?
h (A) 4 – 2 – 5 – 1 – 3 and 4 – 5 – 2 – 3 – 1
(B) 4 – 5 – 2 – 3 – 1 and 4 – 2 – 5 – 1 – 3
(A) acbhdefg (B) abcdefgh (C) 4 – 5 – 2 – 1 – 3 and 4 – 2 – 5 – 1 – 3
(C) adbcefgh (D) aebcdfgh (D) 4 – 5 – 2 – 3 – 1 and 4 – 2 – 5 – 3 – 1
 Chapter 3 • Divide-and-conquer | 3.111

7. Find the number of bridges in the given graph (C) 0 1 2 3 4 5 6 7 8 9

h
(D) 9 8 6 4 2 3 0 1 5 7
f n
b t
12. Consider the following graph:

a c e k l u s v a

d i g m q o
e
b f
j r p w

(A) 12 (B) 13 h
(C) 11 (D) 10
8. Match the following:
g
I. In-order 1. ABCDEFGHI
II. Pre-order 2. DBHEIAFCG
Among the following sequences
III. Post-order 3. ABDEHICFG
IV. Level-order 4. DHIEBFGCA I. a b e g h f II. a b f e h g
For the tree III. a b f h g e IV. a f g h b e
Which are depth first traversals of the above graph?
A (A) I, II and IV only (B) I and IV only
(C) I, III only (D) I, III and IV only
B C 13. The breadth first search algorithm has been imple-
mented using the queue data structure. One possible
order of visiting the nodes is
D E F G
A B C

H I

F E D
(A) I – 2, II – 3, III – 4, IV – 1
(B) I – 3, II – 1, III – 4, IV – 2
(C) I – 1, II – 2, III – 3, IV – 4 (A) A B C D E F (B) B E A D C F
(D) I – 4, II – 3, III – 2, IV – 1 (C) E A B D F C (D) Both (A) and (B)
9. A complete n-array tree in which each node has ‘n’
14. An undirected graph G has ‘n’ nodes. Its adjacency
children (or) no children.
matrix is given by an n × n square matrix.
Let ‘I’ be the number of internal nodes and ‘L’ be the
(i) Diagonal elements are 0’s
number of leaves in a complete n-ary tree.
(ii) Non-diagonal elements are 1’s
If L = 51 and I = 10 what is the value of ‘n’?
Which of the following is true?
(A) 4 (B) 5
(C) 6 (D) Both (A) and (B) (A) Graph G has no minimum spanning tree
(B) Graph G has a unique minimum spanning tree of
10. A complete n-ary tree is one in which every node has 0
cost (n –1)
(or) n children. If ‘X ’ is the number of internal nodes of a
(C) Graph G has multiple distinct minimum spanning
complete n-ary tree, the number of leaves in it is given by
trees, each of cost (n – 1)
(A) X(n – 1) + 1 (B) Xn – 1
(D) Graph G has multiple spanning trees of different cost.
(C) Xn + 1 (D) X(n + 1) + 1
11. The numbers 7, 5, 1, 8, 3, 6, 0, 9, 4, 2 are inserted in the 15. Which of the following is the breadth first search tree
given order into an initially empty binary search tree. for the given graph?
The binary search tree uses the usual ordering on natu- a b c d
ral numbers. What is the in-order traversal sequence of
the resultant tree?
(A) 7 5 1 0 3 2 4 6 8 9
(B) 0 2 4 3 1 6 5 9 8 7 h g f e
3.112 | Unit 3 • Algorithms

(A) a (B) a (C) a (D) a

e b e
b e b b
c f c f c f c f
d g d g d g d g

h h

Practice Problems 2 (A) C (B) A

Directions for questions 1 to 15: Select the correct alterna-
tive from the given choices.
B A B C
1. Which of the following algorithm design technique is
used in finding all pairs of shortest distances in a graph? (C) A (D) B
(A) Divide-and-conquer
(B) Greedy method
(C) Back tracking C B C A
(D) Dynamic programming 6. Consider an undirected unweighted graph G. Let a
breadth first traversal of G be done starting from a node r.
2. Let LASTPOST, LASTIN and LASTPRE denote the
Let d(r, u) and d(r, v) be the lengths of the shortest
last vertex visited in a post-order, in-order and pre-
paths from r to u and v respectively in ‘G’. If u is visited
order traversals respectively of a complete binary tree.
before v during the breadth first travel, which of the fol-
Which of the following is always true?
lowing is correct?
(A) LASTIN = LASTPOST
(A) d(r, u) < d(r, v) (B) d(r, u) > d (r, v)
(B) LASTIN = LASTPRE
(C) LASTPRE = LASTPOST (C) d(r, u) ≤ d (r, v) (D) None of these
(D) LASTIN = LASTPOST = LASTPRE 7. In a complete 5-ary tree, every internal node has exactly
3. Match the following: 5 children. The number of leaves in such a tree with ‘3’
internal nodes are:
X : Depth first search (A) 15 (B) 20
Y : Breadth first search (C) 13 (D) Can’t predicted
Z : Sorting 8. Which of the following algorithm is single pass that is they
a : Heap do not traverse back up the tree for search, create, insert
b : Queue etc.
(A) Depth first search (B) Pre-order traversal
c : Stack (C) B-tree traversal (D) Post-order traversal
(A) X – a, Y – b, Z – c
(B) X – c, Y – a, Z – b 9. Which of the following is the adjacency matrix of the
(C) X – c, Y – b, Z – a given graph?
(D) X – a, Y – c, Z – b a b
4. Let G be an undirected graph, consider a depth first tra-
versal of G, and let T be the resulting DFS Tree. Let ‘U’
be a vertex in ‘G’ and let ‘V’ be the first new (unvisited) c d
vertex visited after visiting ‘U’ in the traversal. Which
of the following is true? (A) 0 1 1 1 (B) 1 1 1 1
1 0 0 
(A) {U, V} must be an edge in G and ‘U’ is a descend-
 0 0 0   0 0
ant of V in T.
1 0 0 1 0 0 0 1
(B) {U, V} must be an edge in ‘G’ and V is a descend-    
ant of ‘U ’ in T. 1 0 1 0 0 0 1 0
(C) If {U, V} is not an edge in ‘G’ then ‘U’ is a leaf in T.
(C) 1 1 1 1 (D) 1 1 1 1
(D) if {U, V} is not an edge in G then U and V must 0
have the same parent in T.
0
 0 0 0   0 0 0 
0 0 0 1 1 0 0 1
5. Identify the binary tree with 3 nodes labeled A, B and C    
on which preorder traversal gives the sequence C, B, A. 1 0 1 0 0 0 1 0
 Chapter 3 • Divide-and-conquer | 3.113

10. Which one of the following is the post-order traversal of Which of the following is the resultant binary search tree
the given tree? after deletion of 33?
a
(A) 22

c
b 11 43

f
d e 10 16 32 44

(A) d e a f c b a (B) d e b f c a
(C) e b d f c a (D) a b c d e f 55

(B) 22
Common data for questions 11 and 12:
11. The pre-order traversal of a tree is a b d h i e c f g. Which
of the following is the correct tree?
11 44
(A) a

d 10 16 43 55
b
e c
h i
45
f g
(C) 22
(B) a
c
b 11 32
f g
d e
10 16 43 44
h i
(C) a
55
b
c (D) 22
f g
d e

h i 11 55
(D) a
c 10 16 43 44
b
f g
e d
43
h i
14. Match the following:
12. Which of the following is in-order traversal of the above
tree? I. Articulation Point 1. An edge whose removal
(A) a b h d e i f g c (B) a b d h e i f g c disconnects graph
(C) h d i b e a f c g (D) i d h b e a f c g II. Bridge 2. A vertex whose removal
disconnects graph
13. Consider the below binary search tree
III. Bi connected 3. Maximal set of edges such
22 component that any two edges in the set
lie on a common simple cycle

11 33 (A) I – 1, II – 2, III – 3 (B) I – 2, II – 1, III – 3

(C) I – 2, II – 3, III – 1 (D) I – 1, II – 2, III – 3
10 16 32 44 15. If x is the root of an n-node subtree, then the inorder-
tree-walk takes
(A) q (n) (B) q (n2)
43 55 (C) q (n )3
(D) q (n log n)
3.114 | Unit 3 • Algorithms

PREVIOUS YEARS’ QUESTIONS

1. Which one of the following is the tightest upper (C) (a+left_end+1, n-left_end-1, k-left_end-1) and
bound that represents the time complexity of insert- (a, left_end, k)
ing an object into a binary search tree of n nodes? (D) (a, n-left_end-1, k-left_end-1) and (a, left_end, k)
[2013] 5. Assume that a mergesort algorithm in the worst case
(A) O(1) (B) O(log n) takes 30 seconds for an input of size 64. Which of the
(C) O(n) (D) O(n log n) following most closely approximates the maximum
2. Consider a rooted n node binary tree represented input size of a problem that can be solved in 6 min-
using pointers. The best upper bound on the time utes? [2015]
required to determine the number of sub trees having (A) 256 (B) 512
exactly 4 nodes is O (na logb n). The value of a + 10b (C) 1024 (D) 2048
is _______ [2014]
6. The given diagram shows the flowchart for a recur-
3. Which one of the following is the recurrence equation sive function A(n). Assume that all statements, except
for the worst case time complexity of the Quicksort for the recursive calls, have O (1) time complexity. If
algorithm for sorting n(≥2) numbers? In the recur- the worst case time complexity of this function is O
rence equations given in the options below, c is a con- (na), then the least possible value (accurate up to two
stant. [2015] decimal positions) of α is ____. [2016]
(A) T(n) = 2T(n/2) + cn
Flowchart for Recursive Function A(n)
(B) T(n) = T(n – 1) + T(1) + cn
(C) T(n) = 2T(n – 1) + cn
(D) T(n) = T(n/2) + cn
4. Suppose you are provided with the following function
declaration in the C programming language.
int partition (int a[ ], int n);
The function treats the first element of a [ ] as a pivot,
and rearranges the array so that all elements less than
or equal to the pivot is in the left part of the array, and
all elements greater than the pivot is in the right part
in addition, it moves the pivot so that the pivot is the
last element of the left part. The return value is the
number of elements in the left part. 7. Let A be an array of 31 numbers consisting of a
sequence of 0’s followed by a sequence of 1’s. The
The following partially given function in the C pro-
problem is to find the smallest index i such that A[i]
gramming language is used to find the k th smallest
is 1 by probing the minimum number of locations in
element in an array a [ ] of size n using the partition
A. The worst case number of probes performed by an
function. We assume k ≤ n.
optimal algorithm is . [2017]
int kth_smallest (int a [ ], int n, int k) [2015]
8. Match the algorithms with their time complexities:
{
int left_end = partition(a, n); Algorithm Time complexity
if (left_end+1 == k) {
(P) Towers of Hanoi with n disks (i) Θ (n2)
return a [left_end];
) (Q) Binary search given n sorted (ii) Θ (n log n)
if (left_end+1 > k) { numbers
return kth_smallest (__________); (R) Heap sort given n numbers (iii) Θ (2n)
} else {
at the worst case
return kth_smallest (__________);
} (S) Addition of two n × n (iv) Θ (log n)
} matrices
The missing argument lists are respectively [2017]
(A) (a, left_end, k) and (a+left_end+1, n-left_end-1, (A) P → (iii), Q → (iv), R → (i), S → (ii)
k-left_end-1) (B) P → (iv), Q → (iii), R → (i), S → (ii)
(B) (a, left_end, k) and (a, n-left_end-1, k-left_end-1) (C) P → (iii), Q → (iv), R → (ii), S → (i)
(D) P → (iv), Q → (iii), R → (ii), S → (i)
 Chapter 3 • Divide-and-conquer | 3.115

ANSWER KEYS
EXERCISES
Practice Problems 1
1. A 2. A 3. B 4. C 5. A 6. B 7. B 8. A 9. C 10. A
11. C 12. D 13. A 14. C 15. A
Practice Problems 2
1. B 2. B 3. C 4. B 5. A 6. D 7. C 8. C 9. A 10. B
11. B 12. C 13. A 14. B 15. A
Previous Years’ Questions
1. C 2. 1 3. B 4. A 5. B 6. 2.2 to 2.4 7. 5 8. C
 Chapter 4
Greedy Approach

LEARNING OBJECTIVES

 Greedy approach  Graph traversal

 Knapsack problem  Breadth first traversal
 Fractional knapsack problem  Depth first search
 Spanning trees  Huffman codes
 Prim's algorithm  Task-scheduling problem
 Kruskal's algorithm  Sorting and order statistics
 Tree and graph traversals  Simultaneous minimum and maximum
 Back tracking  Graph algorithms

GREEDY APPROACH Solved Examples

In a greedy method, we attempt to construct an optimal solution Example 1: (Making change)
in stages.
Problem: Accept n dollars, to return a collection of coins
• At each stage we make a decision that appears to be the best with a total value of n dollars.
(under some criterion) at the time. Configuration: A collection of coins with a total value of n.
Objective function: Minimize number of coins returned.
• A decision made at one stage is not changed in a later stage, so
Greedy solution: Always return the largest coin you can.
each decision should assure feasibility.
• Some problems that use greedy approach are: • Coins are valued $.30, $.020, $0.05, $0.01 use a greedy choice
property and make $.40 by using 3 coins.
1. Knapsack problem
Solution: $0.30 + $0.05 + $0.05 = $0.40
2. Minimum spanning tree
3. Prims algorithm Fractional Knapsack Problem
4. Kruskals algorithm Given: A set S of n items, with each item i having
• bi - a positive benefit
• wi - a positive weight
KNAPSACK PROBLEM Goal: Choose items with maximum total benefit but with weight
The knapsack problem is a problem in combinatorial optimization: atmost W.
given a set of items, each with a weight and a value, determine If we are allowed to take fractional amounts, then this is the frac-
the number of each item to include in a collection so that the total tional knapsack problem.
weight is less than or equal to a given limit and the total value is
• In this case, let xi denote the amount we take of item i.
as large as possible. We have n kinds of items, 1 through n. Each
kind of item i has a value Vi and a weight Wi, usually assume that • Objective: Maximize ∑ bi ( xi /wi )
i∈S
all values and weights are non-negative. The maximum weight that
we can carry in the bag is W. • Constraint: ∑x
i∈S
i
≤W
 Chapter 4 • Greedy Approach | 3.117

Example 2: PRIM’S ALGORITHM

Items Prim’s algorithm is a greedy algorithm that finds a minimum
1 2 3 4 5 spanning tree for a connected weighted undirected graph.
This means it finds a subset of the edges that forms a tree
Weight 4 ml 8 ml 2 ml 6 ml 1 ml that includes every vertex, where the total weight of all the
Benefit $12 $32 $40 $30 $50
edges in the tree is minimized. The algorithm continuously
increases the size, of a tree, one edge at a time starting with
Value ($ per ml) 3 4 20 5 50
a tree consisting of a single vertex, until it spans all vertices.
“Knapsack” • Using a simple binary heap data structure and an adja-
cency list representation, prim’s algorithm can be shown
to run in time O(E log V) where E is the number of edges
and V is the number of vertices.
10 ml
Example:
1
Solution: 1 ml of 5, 2 ml of 3, 6 ml of 4, 1 ml of 2
6 5
• Greedy choice: Keep taking item with highest value (ben-
efit to weight ratio). 1
• Correctness: suppose there is a better solution, there is 2 5 4
an item i with higher value than a chosen item j. (i.e., vj 3 3 5
< vi). If we replace some j with i, we get a better solution. 2
6
4
Thus there is no better solution than the greedy one. 5 6
N = 3, m = 20 6
(P1, P2, P3) = (25, 24, 15)
(W1, W2, W3) = (18, 15, 10) Start:
Example 3: 1
1
4
1 4 2 1
2
X1 X2 X3 S WiXi S Pi Xi

1. 1/2 1/3 1/4 9 + 5 + 2.5 = 16.5 12.5 + 8 + 3.75 = 24.25 3

3
2. 1 2/15 0 18 + 2 + 0 = 20 25 + 3.2 + 0 = 28.2 4
5 6
5 6
3. 0 2/3 1 0 + 10 + 10 = 20 0 + 16 + 15 = 31
Iteration 1: U = {1, 3} Iteration 2: U = {1, 3, 6}
4. 0 1 1/2 0 + 15 + 5 = 20 0 + 24 + 7.5 = 31.5
1 1
4 4
(1), (2), (3), (4) are feasible ones but (4) is the optimum 2 1 2 1
solution. 2 2
5
3 3
4 4
5 6 5 6
SPANNING TREES
A spanning tree of a graph is just a sub graph that contains
Iteration 3: U = {1, 3, 6, 4} Iteration 4: U = {1, 3, 6, 4, 2}
all the vertices and is a tree. A graph may have many span-
ning trees. 1
• A sub graph that spans (reaches out to) all vertices of a 1 4
2
graph is called a spanning sub graph. 5
2
• A sub graph that is a tree and that spans all vertices of the
3
original graph is called a spanning tree. 3
4
• Among all the spanning trees of a weighted and con-
nected graph, the one (possibly more) with the least 5 6
total weight is called a Minimum Spanning Tree
(MST). Iteration 5: U = {1, 3, 6, 4, 2, 5}

Figure 1 An example graph for illustrating prim’s algorithm.

3.118 | Unit 3 • Algorithms

KRUSKAL’S ALGORITHM Backtracking is also known as depth first search (or)

branch and bound. Backtracking is an important tool for
Like prim’s algorithm, Kruskal’s algorithm also constructs
solving constraint satisfaction problems, such as cross-
the minimum spanning tree of a graph by adding edges to the
words, verbal arithmetic, sudoku and many other puzzles.
spanning tree one-by-one. At all points, during its execution
It is often the more convenient technique for parsing, for
the set of edges selected by prim’s algorithm forms exactly one
the knapsack problem and other combinational optimiza-
tree. On the other hand the set of edges selected by Kruskal’s
tion problems.
algorithm forms a forest of trees. Kruskals algorithm is con-
ceptually simple. The edges are selected and added to the • The advantage of backtracking algorithm is that they are
spanning tree in increasing order of their weights. An edge is complete, that is they are guaranteed to find every solu-
added to the tree only if it does not create a cycle. tion to every possible puzzle.
Example:
Graph Traversal
1
6 5
To traverse a graph is to process every node in the graph
exactly once, because there are many paths leading from
5 1
2 4
one node to another, the hardest part about traversing a
5
3 graph is making sure that you do not process some node
3 2
6 4 twice. There are general solutions to this difficulty.
5 6
6 1. When you first encounter a node, mark it as
REACHED. When you visit a node, check if it is
Start: marked REACHED, if it is, just ignore it. This is the
1 1 method our algorithms will use.
4 4
2 2 1 2. When you process a node, delete it from the graph.
1 Deleting the node causes the deletion of all the arcs
3 3 that lead to the node, so it will be impossible to reach
it more than once.
5 6 5 6

Initial configuration Setp 1: choose (1, 3)

General traversal strategy
1. Mark all nodes in the graph as NOT REACHED,
1 1
4 4 2. Pick a starting node, mark it as REACHED, and place
2
1
2
1
it on the READY list.
3. Pick a node on the READY list. Process it remove it
2 2
3 3 3 from READY. Find all its neighbors, those that are
NOT REACHED should marked as REACHED and
5 6 5 6 added to READY.
4. Repeat 3 until READY is empty.
Setep 2: choose (4, 6) Setep 3: choose (2, 5)
Example:
1 1
4 4 A B
2 1 2 1
5
2 2
3 3 3 3
4 4
C D
5 6 5 6

Step I: A = B = C = D = NOT REACHED

Setep 4: choose (3, 6) Setep 6: choose (4, 3)
Step II: READY = {A} . A = REACHED
Step III: Process A. READY = {B, C}.
TREE AND GRAPH TRAVERSALS B = C = REACHED
Back Tracking Step IV: Process C. READY = {B, D}.
Backtracking is a general algorithm technique that consid- D = REACHED
ers searching every possible combination in order to solve Step V: Process B. READY = {D}
an optimization problem. Step VI: Process D. READY = { }
 Chapter 4 • Greedy Approach | 3.119

The two most common traversal patterns are Unexplored edge

Discovery edge
• Breadth first traversal
Cross edge
• Depth first traversal
A A
Breadth First Traversal
B C D B C D
In breadth first traversal, READY is a QUEUE, not an arbi-
trary list. Nodes are processed in the order they are reached
(FIFO), this has the effect of processing nodes according to E F E F
their distance from the initial node. First, the initial node is
processed. Then all its neighbors are processed. Then all of
A A
the neighbors etc.
• Since a graph has no root, we must specify the vertex at B C D B C D
which to start the traversal.
• Breadth first tree traversal first visits all the nodes at depth
E F E F
zero (i.e., the root) then all the nodes at depth 1, and so on.

Procedure A A

First, the starting vertex is enqueued. Then, the following B C D B C D

steps are repeated until the queue is empty.
1. Remove the vertex at the head of the queue and call it E F E F
vertex.
2. Visit vertex A A
3. Follow each edge emanating from vertex to find the
adjacent vertex and call it ‘to’. If ‘to’ has not already B C D B C D
been put into the queue, enqueued it.
E F E F
Notice, that a vertex can be put into the queue at most once.
Therefore, the algorithm must some how keep track of the
vertices that have been enqueued. A

Procedure for BFS for undirected graph G(V, E) B C D

To perform BFS over a graph, the data structures required are
queue (Q) and the visited set (Visited), ‘V ’ is the starting vertex.
E F
Procedure for BFS(V)
Figure 2 Breadth–first search
Steps
1. Visit the vertex ‘V’ Depth First Search
2. Enqueue the vertex V A depth first traversal of a tree always starts at the root of
3. while (Q is not Empty) the tree. Since a graph has no root, when we do a depth first
(i) V = dequeue (); traversal, we must specify the vertex at which to begin. A
(ii) for all vertices J adjacent to V depth first traversal of a tree visits a node and then recur-
(a) if not visited (J) sively visits the sub trees of that node similarly, depth first
• Enqueue (J) traversal of a graph visits a vertex and then recursively vis-
• Visit the vertex ‘J’ its all the vertices adjacent to that node. A graph may con-
• end if. tain cycles, but the traversal must visit every vertex at most
• end for once.
• end while The solution to the problem is to keep track of the nodes
• Stop that have been visited.
Example:
Unexplored vertex
Procedure for DFS for undirected graph G(V, E)
A
To perform DFS over a graph, the data structures
A required are stack (S) and the list (visited), ‘V ’ is the
Visited vertex
start vertex.
3.120 | Unit 3 • Algorithms

Procedure for DFS(V) It is at this point that two traversal strategies differ.
Steps Breadth first adds B’s neighbors to the back of READY,
1. push the start vertex ‘V ’ into the stack S depth first adds them to the front.
2. while (S is not empty)
Breadth first
(i) pop a vertex V
(ii) if ‘V ’ is not visited • READY = [E, C, G]
(a) visit the vertex • Process E. READY = [C, G, F]
(b) Store ‘V ’ in visited • Process C. READY = [G, F, D]
(c) push all the adjacent vertices of ‘V ’ in to visited • Process G .READY = [F, D, H]
(iii) End if • Process F. READY = [D, H]
3. End while • Process D. READY = [H]
4. Stop. • Process H. READY = [ ]
Example: Depth First
A A
• READY = [C, G, E]
• Process C. READY = [D, G, E]
• Process D. READY = [G, E]
B D E B D E
• Process G. READY = [H, F, E]
• Process H. READY = [F, E]
• Process F. READY = [E]
C C • Process E. READY = [ ]

A A CONNECTED COMPONENTS
A graph is said to be connected if every pair of vertices
in the graph are connected. A connected component is a
B D E B D E maximal connected sub graph of ‘G’. Each vertex belongs
to exactly one connected component as does each edge.
C C • A graph that is not connected is naturally and obvi-
ously decomposed into several connected components
(Figure 4). Depth first search does this handily. Each restart
A A of the algorithm marks a new connected component.
• The directed graph in (Figure 5) is “Connected” Part of
it can be “Pulled apart” (so to speak, without “breaking”
B D E B D E any edges).
• Meaningful way to define connectivity in directed graph is:
C C ‘Two nodes U and V of a directed graph G = (V, E) con-
nected if there is a path from U to V ’, and one from V to U. This
relation between nodes is reflective, symmetric and transitive.
A As such, it partitions V into disjoint sets, called the strongly
connected components of the graph. In the directed graph of
figure 2 there are four strongly connected components.
B D E
1 2

C
3 5
4
Figure 3 Depth first search
6 8
• Let us compare two traversal orders on the following graph: 7
A B C D 12 13
11
9 10
E F G H
14
Initial steps:
READY = [A]. process A. READY = [B, E ]. process B. Figure 4 Undirected graph.
 Chapter 4 • Greedy Approach | 3.121

1 2 3 Fixed Length Coding

• Arrange all the characters in sequence (no particular
order is followed)
• a = 47, b = 12, c = 11, d = 14, e = 10, f = 6
Step I:
6
a : 47 b : 12 c : 11 d : 14 e : 10 f:6
4
5
Step II:
7 59 25 16
8

9 10 a : 47 b : 12 c : 11 d : 14 e : 10 f:6
11

Step III:
100
12
0 1

Figure 5 A directed graph and its strongly connected components

84 16
If we shrink each of these strongly connected compo-
nents down to a single node and draw an edge between two 0
1 0
of them if there is an edge from some node in the first to
some node in the second, the resulting directed graph has to 59 25 16
be a directed acyclic graph (DAG) – it has no cycles (figure 0 1 0 1 0 1
6). The reason is simple.
A cycle containing several strongly connected compo- a : 47 b : 12 c : 11 d : 14 e : 10 f:6
nents would merge them all to a single strongly connected
component. We interpret the binary code word for a character as
the path from the root to that character where ‘0’ means
1 2-4-5 3-6
‘go to the left child’, and 1 means ‘go to the right child’.
The above tree is not binary search tree, since the leaves
need not appear in sorted order.

Constructing Huffman Code

7-8-9-10-11-12
This algorithm builds the tree T corresponding to the opti-
mal code in a bottom-up manner. It begins with set of |C|
Every directed graph is a DAG of its strongly connected leaves and performs a sequence of |C|-1 ‘merging’ opera-
components. tions to create the final tree.
• A min-priority queue Q, keyed on f, is used to identify the
HUFFMAN CODES 2 least – frequent objects to merge together. The result of
For compressing data, a very effective and widely used the merger of 2 objects is a new object whose frequency
technique is Huffman coding. We consider the data to be a is the sum of the frequencies of the 2 objects that were
sequence of characters. Huffmans’s greedy algorithm uses merged.
a table of the frequencies of occurrence of the characters to • In the given example, there are 6 alphabets the initial
build up an optimal way of representing each character as a queue size is n = 6, and 5 merge steps are required to
binary string. build the tree. The final tree represents the optimal prefix
Example: Suppose we have a 1,00,000 – character data file, code. The code word for a letter is the sequence of edge
that we wish to store compactly. The characters in the file labels on the path from the root to the letter.
occur with the frequencies given below: a = 47, b = 12, c = 11, d = 14, e = 10, f = 6
Character a b c d e f
Step I: Arrange the characters in non-decreasing order
Frequency 47 12 11 14 10 6 according to their frequencies

f:6 e : 10 c : 11 b : 12 d : 14 a : 47
Solution: Two methods are used for compression of data are:
3.122 | Unit 3 • Algorithms

Let x and y be 2 characters in C having the lowest frequen- • A unit – time task is a job, such as a program to be run
cies. Then there exists an optimal prefix code for C in which on a computer, that requires exactly one unit of time to
the code words for x and y have the same length and differ complete.
only in the last bit • Given a finite set S of unit – time tasks, a schedule for S
16
is a permutation of S specifying the order in which these
tasks are to be performed.
0 1
• The first task in the schedule begins at time ‘0’ and fin-
ishes at time 1, the second task begins at time 1 and fin-
f:6 e : 10
ishes at time 2, and so on
• The problem of scheduling unit – time tasks with dead-
53
lines and penalties for a single processor has the follow-
0 1 ing inputs:
1. A set S = {a1, a2, … an} of n unit – time tasks:
23 30 2. A set of n integer deadlines d1, d2, … dn such that each
di satisfies 1 ≤ di ≤ n and task ai is supposed to finish
0 1 0 1
by time di.
c : 11 b : 12 d : 14 16 3. A set of n non-negative weights or penalties w1,
0 1
w2, … wn, such that we incur a penalty of wi if task ai
is not finished by time di and we incur no penalty if a
f:6 e : 10
task finishes by its deadline.
Example: Consider the following 7 tasks, T1, T2, T3, T4, T5
100 T6, T7. Every task is associated with profit and deadline.
0 1 Tasks T1 T2 T3 T4 T5 T6 T7
Deadline 4 2 4 3 1 4 5
a : 47 53 Profit 75 65 55 45 40 35 30

0 1
45 65 55 75 30
T4 T2 T3 T1 T7
0 1 2 3 4 5 6 7
23 30
0 T1 has highest profit, so it will be executed first and the
1 0 1
deadline of T1, is ‘4’ so T1 has to be executed within 4
c : 11 b : 12 d : 14
slots of time, same procedure is applied to other tasks
16
also.
0 1 The tasks which are not executed by CPU are T5 and T6.

f:6 e : 10 Profit: sum up the profits made by executing the tasks.

Profit = 45 + 65 + 55 + 75 + 30 = 270
Analysis: The analysis of the running time of Huffman’s
algorithm assumes that Q is implemented as a binary min- Analysis: We can use a greedy algorithm to find a maxi-
heap for a set C of ‘n’ characters, the initialization of Q can mum weight independent set of tasks. We can then create
be performed in O(n) time using the BUILD – MIN HEAP an optimal schedule having the tasks in A as its early tasks.
procedure. This method is an efficient algorithm for scheduling unit –
Each heap operation requires O(log n) time, and this time tasks with deadlines and penalties for a single processor.
will be performed exactly (n - 1) times, it contributes to The running time is O(n2) using GREEDY METHOD, since
O(n logn ) running time. Thus the total running time of each of the O(n) independent checks made by that algorithm
HUFFMAN on a set of ‘n’ characters is O(n log n). takes time O(n).

TASK-SCHEDULING PROBLEM SORTING AND ORDER STATISTICS

This is the problem of optimally scheduling unit – time
Minimum and Maximum
tasks on a single processor, where each task has a deadline,
along with a penalty that must be paid if the deadline is This algorithm determines, how many comparisons are nec-
missed. The problem looks complicated, but it can be solved essary to find minimum or maximum of a set of ‘n’ elements.
in simple manner using a greedy algorithm. Usually we can obtain maximum or minimum, by performing
 Chapter 4 • Greedy Approach | 3.123

(n - 1) comparisons; examine each element of the set in turn

and keep track of the smallest element seen so far.
δ (U ,V ) = {
{min{W (P ):u → v}if there is a path from
∞ ‘U ’ to ‘V ’ otherwise
Consider the following procedure.
Edge weights can be interpreted as metrics other than dis-
Assume that the set of elements reside in an array A
tances. They are often used to represent time, cost, penal-
where length [A] = n
ties, loss, or any other quantity.
MINIMUM (A)
• The breadth first search algorithm is a shortest-path algo-
Min ← A[1]
rithm that works on un weighted graphs, that is, graphs in
For i ←2 to length [A]
which each edge can be considered to have unit weight.
Do if min > A [i]
Then min ← A [i]
Return min. Negative-weight edges
Some of the instances of the single-source-shortest-paths
Simultaneous minimum and maximum problem, there may be edges whose weights are negative.
In some applications, we must find both the minimum and • If the graph G = (V, E) contains no negative weight cycles
the maximum of a set of ‘n’ elements. reachable from source S, then for all v ∈ V, the shortest –
We can find the minimum and maximum independently path weight d(S, V ) remains well defined, even if it has a
using (n - 1) comparisons for each, for a total of (2n – 2) negative value.
comparisons. • If there is a negative-weight cycle reachable from S,
shortest-path weights are not well defined.
• In fact, atmost 3  n/ 2  comparisons are sufficient to find
• No path from ‘S’ to a vertex on the cycle can be a shortest
both the minimum and the maximum.
path - a lesser weight path can always be found that fol-
• The strategy is to maintain the minimum and maximum
lows the proposed ‘shortest’ path and then traverses the
elements seen so far.
negative-weight cycle.
• Rather than processing each element of the input by compar-
• If there is a negative-weight cycle on some path form ‘S’
ing it against the current minimum and maximum at a cost
to ‘V’, we define d(S,V) = -∞.
of 2 comparisons per element, we process elements in pairs.
• Compare pairs of elements from the input with each Example: Consider the following graph, calculate the
other, and then we compare smaller to the current mini- shortest distance to all vertices from sources ‘S’.
mum and the larger to the current maximum, at a cost of a b
−5
3 comparisons for every 2 elements.
• Setting up initial values for the current minimum and
4 5
maximum depends on whether ‘n’ is odd or even. If ‘n’ is
7
odd, we set both the minimum and maximum to the value c
d
of the first element, and then we process the rest of the 6 9
S 0 g
elements in pairs.
−2
• If ‘n’ is even, we perform 1 comparison on the first 2 ele-
ments to determine the initial values of the minimum and 3 4
e 8
maximum and then process the rest of the elements in pairs. f

Analysis: If ‘n’ is odd the total number of comparisons

would be 3  n/ 2  . −5
If ‘n’ is even, we need 1 initial comparison followed by h i
3
3( n − 2) 3n
comparisons, for a total of − 2.
2 2
−7 4
\ The total number of comparisons is atmost 3  n/ 2 

j
GRAPH ALGORITHMS
Single Source Shortest Path Solution:
In a shortest-path problem, we are given a weighted • Shortest path from S to a is d(S, a) = W(S, a) = 4 (because
directed graph G = (V, E) with weight function W : E →R there is only one path from ‘S’ to ‘a’)
mapping edges to real-valued weights. The weight of path • Similarly, there is only one path from ‘s’ to ‘b’
P = < VO, V1 … VK > is the sum of the weights of its constitu- d(S, a) = W(S, a) + W(a, b) = 4 + (-5) = -1
ent edges. Shortest-path weight from U to V is defined by • Shortest-path from ‘s’ to ‘c’
3.124 | Unit 3 • Algorithms

There are infinitely many paths from ‘S’ to ‘c’ The algorithm maintains the invariant that Q = V - S at the
1. <S, c> start of each iteration of the while loop. Initially the min -
2. <S, c, d, c> priority queue Q contains all the vertices in V. ( S = ∅).
\
3. <S, c, d, c, d, c > and so on Each time through the while loop, a vertex ‘u’ is extracted
d <S, c> = 6 from Q = V - S and added to set S.
d (S, d, d, c) = 6 + 7- 2 = 11
• Each vertex is extracted from Q and added to S exactly
d (S, c, d, c, d, c) = 6 + 7 – 2 + 7 - 2 = 16
once, so the contents of while loop will be executed
d (S, c, d, c, d, c, d, c)
exactly |v| times.
= 6 + 7 - 2 + 7 - 2 + 7 - 2 = 21
• Dijkstra’s algorithm always chooses the ‘closest’ or
The cycle <c, d, c> has weight = 7 + (-2) = 5 > 0
‘lightest’ vertex in (V - S) to add to set S, we say that it
The shortest path from ‘S’ to ‘c’ is <s, c> with weight
uses a greedy strategy.
d (S, c) = 6 similarly, the shortest-path from ‘S’ to ‘d’ is
<s, c, d>, with weight d (S, d) = w (S, c) + W(c, d) = 13
Example: Consider the following graph, what is the
May there are infinitely paths from ‘S’ to ‘e’ shortest path?
1. <s, e>
a b
2. <s, e f, e> 2
3. <s, e, f, e, f , e> and so on 9 10
Since the cycle <e, f, e> has weight 4 + (-5) = -1 < 0. S 3 4 5 6
However, there is no shortest path from ‘S’ to ‘e’ by traversing
8
the negative-weight cycle <e, f, e> arbitrarily many times, we 6
2 d
can find paths from ‘s’ to ‘e’ with arbitrarily large negative c
weights,
So d(S, e) = -∞ Solution:
Similarly, d(S, f ) = - ∞
S V-S
• The shortest path from ‘S’ to ‘g’:
‘g’ can be reachable from ‘f ; we can also find paths with arbi- S abcd
trarily large negative weights from ‘s’ to ‘g’ and d(s, g) = -∞ Sc abd
• Vertices ‘h’, ‘i’ and ‘j’ also form a negative - weight
Scd ab
cycle. They are not reachable from ‘S’ so, d(S, h ) = d(S,
i) = d(S, j) = ∞ Scda b
Scdab ∅
Dijkstra’s Algorithm
Dijkstra’s algotithm solves the single-source shortest-path Distance from S to all vertices of (V - S)
problem on a weighted, directed graph G = (V, E), for the d[a] = 9
case in which all edge weights are non-negative. d[b] = ∞
d [c] = 6
• The running time of Dijkstra’s algorithm is lower than
d[d] = ∞
that of the Bellman–Ford algorithm.
9, ∞, 6, ∞ values are given to MIN-PRIORITY Queue ‘Q’,
• Dijkstra’s algorithm maintains a set ‘s’ of vertices whose
‘6’ is returned.
final shortest-path weights from the source ‘S’ have
already been determined.
• The algorithm repeatedly selects the vertex u ∈ (V - S)
with the minimum shortest-path estimate, adds ‘u’ to ‘S’
S 0
DIJKSTRA (G, W, S)
INITIALIZE - SINGLE - SOURCE (G, S) 6
6
S←∅ c
S ← V[G]
While Q ≠ 0 Distance from [Sc] to all vertices of (V - S)
do u ← EXTRACT - MIN(Q) d[b] = (S - c - b) = 6 + 10 = 16
S ← S U{u}
d[a] = min{(S - a) = 9, (s - c - a) = 10} = 9
For each vertex v ∈ Adj[u]
do RELAX (u, v, w) d[d] = min{∞, (S - c - d) = 6 + 2 = 8 } = 8
 Chapter 4 • Greedy Approach | 3.125

a b necessary to ensure that shortest-paths consist of a finite

number of edges.
• When there are no cycles of negative length, there is a
S 0 shortest-path between any two vertices of an n-vertex
graph that has atmost (n - 1) edges on it.
6
• A path that has more than (n - 1) edges must repeat atleast
2
c d one vertex and hence must contain a cycle
• Let distx[u] be the length of a shortest-path from the
Distance from [s c d] to [ab] source vertex ‘v’ to vertex ‘u’ under the constraint that
d[a] = min{9, (S - c - d - S - a) = 25} = 9 the shortest-path contains atmost ‘x’ edges. Then dist′[u]
d[b] = min{16, (S - c - d - b) =14} = 14 = cost [v, u] 1 ≤ u ≤ n when there are no cycles of negative
a b
length we can limit our search for shortest-paths to paths
with at most (n - 1) edges. Hence, distn-1[u] is the length
9 of an unrestricted shortest-path from ‘v’ to ‘u’.
S 0 The Recurrence Relation for dist is:
6 Distk[u] = min{distk-1[u], min{distk-1[i] + cost[i, u]}}
2 This recurrence can be used to compute distk from dist k-1
,
c d
for k = 2, 3, … , n - 1.
d[a] = min{14, (s - a - b) = 9 + 2 = 11} = 11 Example: Consider the given directed graph
−1
a b 2 5
2
9 5 1
−1 1
S 0 4
1 3 7
6
2 −1
4 2
c
4 6
−1
Analysis: It maintains the min-priority queue ‘Q’ by calling
three priority-queue operations: INSERT, EXTRACT-MIN, Find the shortest path from vertex ‘1’ to all other vertices
and DECREASE-KEY. We maintain the min-priority queue using Bellman–Ford algorithm?
by taking the vertices being numbered 1 to |v|. We store d[v]
Solution: Source vertex is ‘1’ the distance from ‘1’ to ‘1’ in
in the vth entry of an array. Each INSERT and DECREASE-
all ‘6’ iterations will be zero. Since the graph has ‘7’ verti-
KEY operation takes O(1) time, and each EXTRACT- MIN
ces, the shortest-path can have atmost ‘6’ edges. The follow-
operation takes O(v) time (\ we have to search through the
ing figure illustrates the implementation of Bellman–Ford
entire array) for a total time of O(v2 + E) = O(v2).
algorithm:
• If we implement the min - priority queue with a binary
1 2 3 4 5 6 7
min-heap. Each EXTRACT-MIN operation takes time
O(log V ), there are |V | such operations. 1 0 5 4 4 ∞ ∞ ∞
• The time to build the binary min-heap is O(v). Each 2 0 3 3 4 4 3 ∞
DECREASE-KEY operation takes time O(log V), and 3 0 2 3 4 2 3 5
there are still atmost |E | such operations. The total run- 4 0 2 3 4 1 3 3
ning time is O((V + E ) log V ), which is O(E log V ) if all
5 0 2 3 4 1 3 2
vertices are reachable form the source.
• We can achieve a running time of O(V log V + E) by imple- 6 0 2 3 4 1 3 2
menting the min-priority queue with a Fibonacci heap. 7 0 2 3 4 1 3 2

Bellman–Ford Algorithm Analysis

• Each iteration takes O(n2) time if adjacency matrices are
Bellman–Ford algorithm solves the single-source shortest-path
used and O(e) time if adjacency lists are used. Here ‘e’ is
problems in the case in which edge weights may be negative.
the number of edges in the graph.
• When negative edge lengths are permitted, we require • The time complexity is O(n3) when adjacency matrices
that the graph have no cycles of negative length. This is are used and O(N * E) when adjacency lists are used.
3.126 | Unit 3 • Algorithms

EXERCISES
Practice Problems 1 C 1 2 3 4
1 0 15 20 25
Directions for questions 1 to 14: Select the correct alterna-
2 10 0 14 15
tive from the given choices. 3 11 18 0 17
1. A thief enters a store and sees the following: 4 13 13 14 0
C
A B 1 2

$100 $80 $120

2 pds 2 pds 3 pds

4 3

His knapsack can hold 4 pounds, what should he steal (A) 1 - 2 - 4 - 3 - 1 (B) 2 - 3 - 4 - 1 - 2
to maximize profit? (Use 0-1 Knapsack). (C) 1 - 4 - 2 - 3 - 1 (D) 2 - 4 - 3 - 1 - 2
(A) A and B (B) A and C
(C) B and C (D) A, B and C 7. Calculate the maximum profit using greedy strategy,
knapsack capacity is 50. The data is given below:
2. By using fractional Knapsack, calculate the maximum
profit, for the data given in the above question? n=3
(A) 180 (B) 170 (w1, w2, w3) = (10, 20, 30)
(C) 160 (D) 150 (p1, p2, p3) = (60, 100, 120) (dollars)? (0/1 knapsack)
3. Consider the below figure: (A) 180 (B) 220
8 7 (C) 240 (D) 260
b c d
4 7
2 Common data for questions 8 and 9: Given that
4
a 11 i e a b c d e f
14
7 6
10 Frequency 45 13 12 16 9 5
8
g Fixed length code word 000 001 010 011 100 101
h f
1 2
8. Using Huffman code, find the path length of internal
What is the weight of the minimum spanning tree using nodes.
Kruskals algorithm? (A) 8 (B) 100
(A) 34 (B) 35 (C) 100 × 8 (D) 100/8
(C) 36 (D) 38 9. Using above answer, external path length will be
4. Construct a minimum spanning tree for the figure given (A) 18 (B) 108
in the above question, using prim’s algorithm. What are (C) 8 (D) None of these
the first three nodes, added to the solution set respec-
tively (consider ‘a’ as starting node). Common data for questions 10 and 11:
(A) b, c, i (B) h, b, c 10. Item 3
(C) c, i, b (D) h, c, b
Item 2
5. Consider the below graph, calculate the shortest dis- Item 1
50 kg
30 kg

tance from ‘S’ to ‘T ’?

20 kg
10 kg

4
A D
18
1 11 9
5 60/kg 100/kg 20/kg knapsack
2 13
S B E T
Using 0-1 knapsack select a subset of the three items
5 shown, whose weight must not exceed 50 kg. What is
16
2
the value?
C F
2 (A) 2220 (B) 2100
(C) 2600 (D) 2180
(A) 23 (B) 9
(C) 20 (D) 22 11. Which of the following gives maximum profit, using
6. Solve the travelling salesman problem, with the given fractional knapsack?
distances in the form of matrix of graph, which of the (A) x1 = 1, x2 = 1, x3 = 0 (B) x1 = 1, x2 = 1, x3 = 2/3
following gives optimal solution? (C) x1 = 1, x2 = 0, x3 = 1 (D) x1 = 1, x2 = 1, x3 = 1/3
 Chapter 4 • Greedy Approach | 3.127

12. Using dynamic programming find the longest common C2 = 10 paisa

subsequence (LCS) in the given 2 sub sequences: C3 = 5 paisa
x [1, … , m] C4 = 1 paisa
y [1, … , n] To represents 48 paisa, what is the minimum number of
x:ABCBDAB coins used, using greedy approach?
Y:BDCABA (A) 6 (B) 7
Find longest sequence sets common to both. (C) 8 (D) 9
(A) (BDAB, BCAB, BCBA) 14. Worst-case analysis of hashing occurs when
(B) (BADB, BCAB, BCBA) (A) All the keys are distributed
(C) (BDAB, BACB, BCBA) (B) Every key hash to the same slot
(D) (BDAB, BCAB, BBCA) (C) Key values with even number, hashes to slots with
even number
13. Let C1, C2, C3, C4 represent coins.
(D) Key values with odd number hashes to slots with
C1 = 25 paisa odd number.

Practice Problems 2 (A) b c

Directions for questions 1 to 15: Select the correct alterna-

tive from the given choices. a f d
1. Consider the given graph:
3 e
(B) b c
b d f
2 5 d
a
2
a 4 3 4 f e
6
7
c
5
e (C) b c

a
f d
8
e
Which one of the following cannot be the sequence of
edges added, in that order, to a minimum spanning tree (D) b c
using Kruskal’s algorithm?
a f
(A) (a - b), (d - f ), (b - f ), (d - c), (d - e) d
(B) (a - b), (d - f ), (d - c), (b - f ), (d - e)
e
(C) (d - f ), (a - b), (d - c), (b - f ), (d - e)
(D) (d - f ), (a - b), (b - f ), (d - e), (d - c) 4. Consider the following graph:
2. The worst case height analysis of B-tree is 4
b c
(A) O(n)
(B) O(n2) 3 2 5 6
(C) O(log n)
(D) O(n log n) a d e
7 4
3. Consider the given graph:
1 Find the shortest path using Dijkstra’s algorithm.
3
b c
6 (A) a - b - d - e (B) a - b - c - d
4 4
(C) a - c - d - e (D) a - b - c - e
a 5 5 d
f 5. Which statement is true about Kruskal’s algorithm?
(A) It is a greedy algorithm for the minimum spanning
2
6 8 tree problem.
(B) It constructs spanning tree by selecting edges in
e
increasing order of their weights.
Which of the following is the minimum spanning tree. (C) It does not accept creation of cycles in spanning tree.
(If we apply Kruskal algorithm). (D) All the above
3.128 | Unit 3 • Algorithms

6. Dijkstra’s algorithm bears similarity to which of the (A) I, II, III, IV (B) IV, III, I, II
following for computing minimum spanning trees? (C) IV, II, I, III (D) III, IV, II I
(A) Breadth first search (B) Prim’s algorithm 11. Let V stands for vertex, E stands for edges.
(C) Both (A) and (B) (D) None of these
For both directed and undirected graphs, the adjacency
7. Which of the following algorithm always yields a cor- list representation has the desirable property that the
rect solution for a graph with non-negative weights to amount of memory required is
compute shortest paths? (A) q(V ) (B) q(E)
(A) Prim’s algorithm (B) Kruskal’s algorithm (C) q(V + E ) (D) q(V - E )
(C) Dijkstra’s algorithm (D) Huffman tree
12. Which of the following is false?
8. Let the load factor of the hash table is number of keys (A) Adjacency-matrix representation of a graph per-
is n, cells of the hash table is m then mits faster edge look up.
(B) The adjacency matrix of a graph requires q(v2)
(A) ∝ = n/m (B) ∝ = m/n memory, independent of the number of edges in
m +1 n +1
(C) ∝ (D) ∝ the graph.
n m (C) Adjacency-matrix representation can be used for
9. To implement Dijkstra’s shortest path algorithm on weighted graphs.
unweighted graphs so that it runs in linear time, the (D) All the above
data structure to be used is: 13. Dynamic programming is a technique for solving prob-
(A) Queue lems with
(B) Stack (A) Overlapped sub problems
(C) Heap (B) Huge size sub problems
(D) B-tree (C) Small size sub problems
10. The development of a dynamic-programming algo- (D) None of these
rithm can be broken into a sequence of four steps, 14. The way a card game player arranges his cards, as he
which are given below randomly. picks them up one by one is an example of _____.
I. Construct an optimal solution from computed in- (A) Bubble sort (B) Selection sort
formation. (C) Insertion sort (D) None of the above
II. Compute the value of an optimal solution in a bot- 15. You want to check whether a given set of items is
tom-up fashion. sorted. Which method will be the most efficient if it is
III. Characterize the structure of an optimal solution. already in sorted order?
IV. Recursively defines the value of an optimal solution. (A) Heap sort (B) Bubble sort
The correct sequence of the above steps is (C) Merge sort (D) Insertion sort

PREVIOUS YEARS’ QUESTIONS

Data for question 1: We are given 9 tasks T1, T2 … T9. 3. To implement Dijkstra’s shortest path algorithm on
The execution of each task requires one unit of time. We unweighted graphs so that it runs in linear time, the
can execute one task at a time. Each task Ti has a profit Pi data structure to be used is: [2006]
and a deadline Di. Profit Pi is earned if the task is com- (A) Queue
pleted before the end of the Dith unit of time. (B) Stack
(C) Heap
Task T1 T2 T3 T4 T5 T6 T7 T8 T9
(D) B-Tree
Profit 15 20 30 18 18 10 23 16 25
Deadline 7 2 5 3 4 5 2 7 3 4. Consider the following graph: [2006]
2
1. What is the maximum profit earned? [2005]
(A) 147 (B) 165
b d
(C) 167 (D) 175 1 4
2 1
2. Consider a weighted complete graph G on the ver- a 3 3 f
5
tex set {v1, v2,…, vn} such that the weight of the edge 6 4
c e
( vi , v j ) is 2 i − j . The weight of the minimum span-
7
ning tree is: [2006]
(A) n - 1 (B) 2n - 2 Which one of the following cannot be the sequence
 n of edges added, in that order, to a minimum spanning
(C)   (D) n2 tree using Kruskal’s algorithm?
 2
 Chapter 4 • Greedy Approach | 3.129

(A) (a - b), (d - f), (b - f), (d - c), (d - e) int GetValue (struct CellNode *ptr) {
(B) (a - b), (d - f), (d - c), (b - f), (d - e) int value = 0;
(C) (d - f), (a - b), (d - c), (b - f), (d - e) if (ptr != NULL) {
(D) (d - f), (a - b), (b - f), (d - e), (d - c) if ((ptr->leftChild == NULL) &&
Common data for questions 5 and 6: A 3-ary max-heap (ptr->rightChild == NULL))
is like a binary max-heap, but instead of 2 children, value = 1;
nodes have 3 children. A 3-ary heap can be repre- else
sented by an array as follows: The root is stored in the value = value + GetValue(ptr->leftChild)
first location, a[0], nodes in the next level, from left + GetValue(ptr->rightChild);
to right, is stored from a[1] to a[3]. The nodes from }
the second level of the tree from left to right are stored return(value);
from a[4] location onward. An item x can be inserted
into a 3-ary heap containing n items by placing x in The value returned by GetValue when a pointer to the
the location a[n] and pushing it up the tree to satisfy root of a binary tree is passed as its argument is:
the heap property. (A) The number of nodes in the tree
(B) The number of internal nodes in the tree
5. Which one of the following is a valid sequence of
(C) The number of leaf nodes in the tree
elements in an array representing 3-ary max-heap?
(D) The height of the tree
[2006]
(A) 1, 3, 5, 6, 8, 9 (B) 9, 6, 3, 1, 8, 5 10. Let w be the minimum weight among all edge weights
(C) 9, 3, 6, 8, 5, 1 (D) 9, 5, 6, 8, 3, 1 in an undirected connected graph. Let e be a specific
6. Suppose the elements 7, 2, 10 and 4 are inserted, in edge of weight w. Which of the following is FALSE?
that order, into the valid 3-ary max-heap found in the [2007]
above question, Q-76. Which one of the following is (A) There is a minimum spanning tree containing e.
the sequence of items in the array representing the (B) If e is not in a minimum spanning tree T, then in
resultant heap? [2006] the cycle formed by adding e to T, all edges have
(A) 10, 7, 9, 8, 3, 1, 5, 2, 6, 4 the same weight.
(B) 10, 9, 8, 7, 6, 5, 4, 3, 2, 1 (C) Every minimum spanning tree has an edge of
(C) 10, 9, 4, 5, 7, 6, 8, 2, 1, 3 weight w.
(D) 10, 8, 6, 9, 7, 2, 3, 4, 1, 5 (D) e is present in every minimum spanning tree.
7. In an unweighted, undirected connected graph, the 11. The Breadth first search algorithm has been imple-
shortest path from a node S to every other node is mented using the queue data structure. One possible
computed most efficiently, in terms of time complex- order of visiting the nodes of the following graph is
ity, by [2007] [2008]
(A) Dijkstra’s algorithm starting from S. M N O
(B) Warshall’s algorithm
(C) Performing a DFS starting from S.
(D) Performing a BFS starting from S.
8. A complete n-ary tree is a tree in which each node has R
Q P
n children or no children. Let I be the number of inter-
nal nodes and L be the number of leaves in a complete (A) MNOPQR (B) NQMPOR
n-ary tree. If L = 41, and I = 10, what is the value of (C) QMNPRO (D) QMNPOR
n? [2007] 12. G is a graph on n vertices and 2n - 2 edges. The edges
(A) 3 (B) 4 of G can be partitioned into two edge-disjoint span-
(C) 5 (D) 6 ning trees. Which of the following is NOT true for G?
9. Consider the following C program segment where [2008]
CellNode represents a node in a binary tree: [2007] (A) For every subset of k vertices, the induced sub-
struct CellNode { graph has atmost 2k - 2 edges
struct CellNode *leftChild; (B) The minimum cut in G has atleast two edges
int element; (C) There are two edge-disjoint paths between every
struct CellNode *rightChild; pair of vertices
};
(D) There are two vertex-disjoint paths between eve-
ry pair of vertices
3.130 | Unit 3 • Algorithms

13. (A) {25, 12, 16, 13, 10, 8, 14}

−3 (B) {25, 14, 13, 16, 10, 8, 12}
b e
(C) {25, 14, 16, 13, 10, 8, 12}
2 2
1 1 1 (D) {25, 14, 12, 13, 10, 8, 16}
−5
a c h f 18. What is the content of the array after two delete oper-
2 3 2 3
ations on the correct answer to the previous question?
2
d g [2009]
(A) {14, 13, 12, 10, 8} (B) {14, 12, 13, 8, 10}
Dijkstra’s single source shortest path algorithm when
(C) {14, 13, 8, 12, 10} (D) {14, 13, 12, 8, 10}
run from vertex a in the above graph, computes the
correct shortest path distance to [2008] Common data for questions 19 and 20: Consider a com-
(A) Only vertex a plete undirected graph with vertex set {0, 1, 2, 3, 4}. Entry
(B) Only vertices a, e, f, g, h Wij in the matrix W below is the weight of the edge {i, j}.
(C) Only vertices a, b, c, d  0 1 8 1 4
(D) all the vertices  
 1 0 12 4 9 
14. You are given the post-order traversal, P, of a binary W =  8 12 0 7 3 
search tree on the n elements 1, 2,…, n. You have to  
determine the unique binary search tree that has P as 1 4 7 0 2
4 9 3 2 0
its post-order traversal. What is the time complexity  
of the most efficient algorithm for doing this? [2008]
(A) Q(log n) 19. What is the minimum possible weight of a spanning
(B) Q(n) tree T in this graph such that vertex 0 is a leaf node in
(C) Q(n log n) the tree T ? [2010]
(D) None of the above, as the tree cannot be uniquely (A) 7 (B) 8
determined (C) 9 (D) 10
15. Which of the following statement(s) is/are correct 20. What is the minimum possible weight of a path P
regarding Bellman–Ford shortest path algorithm? from vertex 1 to vertex 2 in this graph such that P
[2009] contains at most 3 edges? [2010]
P. Always finds a negative weighted cycle, if one (A) 7 (B) 8
exists. (C) 9 (D) 10
Q. Finds whether any negative weighted cycle is
reachable from the source. Common data for questions 21 and 22: A hash table of
(A) P only (B) Q only length 10 uses open addressing with hash function h(k) =
(C) Both P and Q (D) Neither P nor Q k mod 10, and linear probing. After inserting 6 values into
an empty hash table, the table is as shown below:
16. Consider the following graph: [2009]
0
b e
5 2 5 1
6 6 2 42
a 4 d 3 g 3 23
6
3 4 4 34
5
c f 5 52
6 6 46
Which one of the following is NOT the sequence 7 33
of edges added to the minimum spanning tree using 8
Kruskal’s algorithm? [2009] 9
(A) (b, e) (e, f) (a, c) (b, c) (f, g) (c, d)
21. Which one of the following choices gives a possi-
(B) (b, e) (e, f) (a, c) (f, g) (b, c) (c, d)
ble order in which the key values could have been
(C) (b, e) (a, c) (e, f) (b, c) (f, g) (c, d)
inserted in the table?
(D) (b, e) (e, f) (b, c) (a, c) (f, g) (c, d)
[2010]
Common data for questions 17 and 18: Consider a
(A) 46, 42, 34, 52, 23, 33
binary max-heap implemented using an array.
(B) 34, 42, 23, 52, 33, 46
17. Which one of the following array represents a binary (C) 46, 34, 42, 23, 52, 33
max-heap? [2009] (D) 42, 46, 33, 23, 34, 52
 Chapter 4 • Greedy Approach | 3.131

22. How many different insertion sequences of the key 24. What will be the cost of the minimum spanning tree
values using the same hash function and linear prob- (MST) of such a graph with n nodes? [2011]
ing will result in the hash table shown above? [2010] 1
(A) 10 (B) 20 (A) (11n 2 − 5n) (B) n2 - n + 1
12
(C) 30 (D) 40
(C) 6n - 11 (D) 2n + 1
23. A max-heap is a heap where the value of each parent
is greater than or equal to the value of its children. 25. The length of the path from V5 to V6 in the MST of
Which of the following is a max-heap? [2011] previous question with n = 10 is [2011]
(A) 11 (B) 25
(A) (C) 31 (D) 41
10

26. Consider the directed graph shown in the figure

8 6 below. There are multiple shortest paths between ver-
tices S and T. Which one will be reported by Dijkstra’s
shortest path algorithm? Assume that, in any iteration,
4 5 2 the shortest path to a vertex v is updated only when a
strictly shorter path to v is discovered. [2012]
1 2
(B) C 1 E 2
10 G
1
A 1
3 4
8 6 4
7 3
S
D 3
4 5 1 2 3 4
5
3 T
(C) 10
5
B
F

5 6 (A) SDT (B) SBDT

(C) SACDT (D) SACET
4 8 2 1 27. Let G be a weighted graph with edge weights greater
than one and G1 be the graph constructed by squaring
(D) the weights of edges in G. Let T and T1 be the mini-
5
mum spanning trees of G and G1, respectively, with
total weights t and t1. Which of the following state-
2 8 ments is TRUE? [2012]
(A) T 1 = T with total weight t1 = t2
(B) T 1 = T with total weight t1 < t2
1 4 6 1 (C) T 1 ≠ T but total weight t1 = t2
(D) None of the above
Common data for questions 24 and 25: An undirected
graph G (V, E) contains n(n > 2) nodes named V1, V2, …, 28. What is the time complexity of Bellman–Ford single-
Vn. Two nodes Vi, Vj are connected if and only if 0 < |i - j| source shortest path algorithm on a complete graph of
≤ 2. Each edge (Vi, Vj) is assigned a weight i + j. A sample n vertices? [2013]
graph with n = 4 is shown below. (A) Q(n2) (B) Q(n2 log n)
(C) Q(n3) (D) Q(n3 log n)
V3 7
V4 29. Consider the following operation along with Enqueue
4 and Dequeue operations on queues, where k is a
5
6 global parameter.
MultiDequeue(Q) {
V1 V2
3 m = k
3.132 | Unit 3 • Algorithms

while (Q is not empty) and (m > 0) { (A) 10, 8, 7, 3, 2, 1, 5 (B) 10, 8, 7, 2, 3, 1, 5

Dequeue (Q) (C) 10, 8, 7, 1, 2, 3, 5 (D) 10, 8, 7, 5, 3, 2, 1
m = m - 1 35. Consider the tree arcs of a BFS traversal from a
} source node W in an unweighted, connected, undi-
} rected graph. The tree T formed by the tree acrs is a
What is the worst case time complexity of a sequence data structure for computing [2014]
of n queue operations on an initially empty queue? (A) The shortest path between every pair of vertices
[2013] (B) The shortest path from W to every vertex in the
(A) Q(n) (B) Q(n + k) graph
(C) Q(nk) (D) Q(n2) (C) The shortest paths from W to only those nodes
that are leaves of T.
30. The preorder traversal sequence of a binary search (D) The longest path in the graph
tree is 30, 20, 10, 15, 25, 23, 39, 35, 42. Which one of
36. The number of distinct minimum spanning trees for
the following is the post order traversal sequence of
the weighted graph below is _______.
the same tree? [2013]
(A) 10, 20, 15, 23, 25, 35, 42, 39, 30 [2014]
(B) 15, 10, 25, 23, 20, 42, 35, 39, 30 2
(C) 15, 20, 10, 23, 25, 42, 35, 39, 30 2 1 2 1
(D) 15, 10, 23, 25, 20, 35, 42, 39, 30
31. Let G be a graph with n vertices and m edges. What is 2 2
the tightest upper bound on the running time of depth 2 1
1 1
first search on G, when G is represented as an adja-
cency matrix? [2014] 2 2
(A) q(n) (B) q(n + m)
(C) q(n2) (D) q(m2) 37. Suppose depth first search is executed on the graph
below starting at some unknown vertex. Assume that
32. Consider the directed graph given below. [2014]
a recursive call to visit a vertex is made only after
P Q first checking that the vertex has not been visited
earlier. Then the maximum possible recursion depth
(Including the initial call) is _______. [2014]
R S

Which one of the following is TRUE?

(A) The graph does not have any topological order-
ing.
(B) Both PQRS and SRQP are topological orderings.
(C) Both PSRQ and SPRQ are topological orderings. 38. Suppose we have a balanced binary search tree T
(D) PSRQ is the only topological ordering. holding n numbers. We are given two numbers L and
H and wish to sum up all the numbers in T that lie
33. There are 5 bags labeled 1 to 5. All the coins in a given
between L and H. suppose there are m such numbers
bag have the same weight. Some bags have coins of
in T. If the tightest upper bound on the time to com-
weight 10 gm. Others have coins of weight 11 gm. I
pute the sum is O(na logb n + mc logd n), the value of a
pick 1, 2, 4, 8, 16 coins respectively from bags 1 to
+ 10b + 100c + 1000d is -----.
5. Their total weight comes out to 323 gm. Then the
product of the labels of the bags having 11 gm coins [2014]
is ___ [2014] 39. The graph shown below has 8 edges with distinct inte-
ger edge weights. The minimum spanning tree (MST)
34. A priority queue is implemented as a max-heap.
is of weight 36 and contains the edges: {(A, C), (B, C),
Initially it has 5 elements. The level-order traversal
(B, E), (E, F), (D, F)}. The edge weights of only those
of the heap is : 10, 8, 5, 3, 2. Two new elements 1 and
edges which are in the MST are given in the figure
7 are inserted into the heap in that order. The level-
shown below. The minimum possible sum of weights
order traversal of the heap after the insertion of the
of all 8 edges of this graph is _______ [2015]
elements is [2014]
 Chapter 4 • Greedy Approach | 3.133

15 numbered 0 to 9 for i ranging from 0 to 2020?

B E
4 [2015]
(A) h(i) = i 2 mod 10
A F
2 (B) h(i) = i 3 mod 10
9 6 (C) h(i) = (11 * i2 ) mod 10
C D (D) h(i) = (12 * i) mod 10
44. Let G be a weighted connected undirected graph with
40. Consider two decision problems Q1, Q2 such that distinct positive edge weights. If every edge weight
Q1 reduces in polynomial time to 3-SAT and 3-SAT is increased by the same value, then which of the fol-
reduces in polynomial time to Q2. Then which one of lowing statements is/are TRUE? [2016]
the following is consistent with the above statement? P : Minimum spanning tree of G does not change.
[2015]
(A) Q1 is in NP, Q2 is NP hard. Q : Shortest path between any pair of vertices does
(B) Q2 is in NP, Q1 is NP hard. not change.
(C) Both Q1 and Q2 are in NP. (A) P only (B) Q only
(D) Both Q1 and Q2 are NP hard. (C) Neither P nor Q (D) Both P and Q
41. Given below are some algorithms, and some algo- 45. Let G be a complete undirected graph on 4 vertices,
rithm design paradigms. having 6 edges with weights being 1,2,3,4,5, and
6. The maximum possible weight that a minimum
1. Dijkstra’s Shortest Path i. Divide and Conquer weight spanning tree of G can have is _____ . [2016]
2. Floyd-Warshall algo- ii. Dynamic 46. G = (V,E) is an undirected simple graph in which each
rithm to compute all Programming edge has a distinct weight, and e is a particular edge
pairs shortest path of G. Which of the following statements about the
3. Binary search on a iii. Greedy design minimum spanning trees (MSTs) of G is/ are TRUE?
sorted array [2016]
4. Backtracking search on iv. Depth-first search I. If e is the lightest edge of some cycle in G, then
a graph every MST of G includes e
v. Breadth-first search II. If e is the heaviest edge of some cycle in G, then
Match the above algorithms on the left to the corre- every MST of G excludes e
sponding design paradigm they follow. (A) I only (B) II only
(A) 1–i, 2–iii, 3–i, 4–v (B) 1–iii, 2–iii, 3–i, 4–v (C) both I and II (D) neither I nor II
(C) 1–iii, 2–ii, 3–i, 4–iv (D) 1–iii, 2–ii, 3–i, 4–v 47. Breadth First Search (BFS) is started on a binary tree
42. A Young tableau is a 2D array of integers increasing beginning from the root vertex. There is a vertex t at
from left to right and from top to bottom. Any unfilled a distance four from the root. If t is the n-th vertex in
entries are marked with ∞, and hence there cannot be this BFS traversal, then the maximum possible value
any entry to the right of, or below a ∞. The following of n is _____. [2016]
Young tableau consists of unique entries. 48. Let G = (V,E) be any connected undirected edge-
weighted graph. The weights of the edges in E are pos-
1 2 5 14 itive and distinct. Consider the following statements:
(I) Minimum spanning Tree of G is always unique.
3 4 6 23
(II) Shortest path between any two vertices of G is
10 12 18 25 always unique.
31 ∞ ∞ ∞ Which of the above statements is/are necessarily true?
[2017]
When an element is removed from a Young tableau, (A) (I) only
other elements should be moved into its place so that (B) (II) only
the resulting table is still a Young tableau (unfilled (C) both (I) and (II)
entries maybe filled in with a ∞). The minimum num- (D) neither (I) nor (II)
ber of entries (other than 1) to be shifted, to remove 1
49. The Breadth First Search (BFS) algorithm has been
from the given Young tableau is _______ [2015]
implemented using the queue data structure. Which
43. Which one of the following hash functions on integers one of the following is a possible order of visiting the
will distribute keys most uniformly over 10 buckets nodes in the graph below? [2017]
3.134 | Unit 3 • Algorithms

51. Let G be a simple undirected graph. Let TD be a depth

M N O
first search tree of G. Let TB be a breadth first search
tree of G.
Consider the following statements.
(I) No edge of G is a cross edge with respect to TD.
(A cross edge in G is between two nodes neither
R Q P of which is an ancestor of the other in TD.)
(II) For every edge (u, v) of G, if u is at depth i and v
is at depth j in TB, then |i – j| = 1.
(A) MNOPQR
(B) NQMPOR Which of the statements above must necessarily be
(C) QMNROP true? [2018]
(D) POQNMR (A) I only (B) II only
(C) Both I and II (D) Neither I nor II
50. A message is made up entirely of characters from the
set X = {P, Q, R, S,T}. The table of probabilities for 52. Consider the following undirected graph G:
each of the characters is shown below:
4 x
Character Probability

P 0.22 1 3

Q 0.34 4 5

R 0.17
4
S 0.19

T 0.08 Choose a value for x that will maximize the number

Total 1.00 of minimum weight spanning trees (MWSTs) of G.
The number of MWSTs of G for this value of x is
If a message of 100 characters over X is encoded ______. [2018]
using Huffman coding, then the expected length of
the encoded message in bits is _________. [2017]

ANSWER KEYS

EXERCISES
Practice Problems 1
1. A 2. A 3. B 4. A 5. B 6. A 7. B 8. A 9. A 10. C
11. B 12. A 13. A 14. B

Practice Problems 2
1. D 2. C 3. A 4. A 5. D 6. C 7. C 8. A 9. A 10. D
11. C 12. C 13. A 14. C 15. D

Previous Years’ Questions

1. A 2. B 3. C 4. D 5. D 6. A 7. D 8. C 9. C 10. B
11. C 12. D 13. D 14. B 15. B 16. D 17. C 18. D 19. D 20. B
21. C 22. C 23. B 24. B 25. C 26. D 27. 28. C 29. A 30. D
31. C 32. C 33. 12 to 12 34. A 35. B 36. 6 to 6 37. 19 38. 110 39. 69
40. A 41. C 42. 5 43. B 44. A 45. 7 46. B 47. 31 48. A 49. D
50. 225 51. A 52. 4
 Chapter 5
Dynamic Programming
LEARNING OBJECTIVES

 Dynamic programming  Longest common subsequence

 Multi-stage graph  Optimal substructure of LCS
 All pairs shortest path problem  NP-hard and NP-complete
 Hashing methods  P-problem
 Mid-square method  NP-problem
 Folding method  P, NP, and Co-NP
 Resolving collisions  Cooks theorem
 Matrix chain multiplication  Non-deterministic search

DYNAMIC PROGRAMMING MULTI-STAGE GRAPH

Dynamic programming is a method for solving complex problems A multi-stage graph is a graph
by breaking them down into simpler sub problems. It is applicable
• G = (V, E) with V partitioned into K > = 2 disjoint subsets such
to problems exhibiting the properties of overlapping sub problems
that if (a, b) is in E, then a is in Vi, and b is in Vi + 1 for some sub
which are only slightly smaller, when applicable; the method takes
sets in the partition;
far less time than naive method.
• |V1| = |VK| = 1 the vertex S in V1 is called the source; the vertex t
• The key idea behind dynamic programming is to solve a given is called the sink.
problem, we need to solve different parts of the problem (sub prob- • G is usually assumed to be a weighted graph.
lems) then combine the solutions of the sub problems to reach an • The cost of a path from node V to node W is sum of the costs of
overall solution. Often, many of these sub problems are the same. edges in the path.
• The dynamic programming approach seeks to solve each sub • The ‘multi-stage graph problem’ is to find the minimum cost
problem only once, thus reducing the number of computations. path from S to t.
This is especially useful when the number of repeating sub prob-
lems is exponentially large. Example:
• There are two key attributes that a problem must have in order Stage I Stage II Stage III Stage IV Stage V Stage VI
for dynamic programming to be applicable ‘optimal sub struc-
ture’ and ‘overlapping sub-problems’. However, when the over- 8
2 5
lapping problems are much smaller than the original problem,
12
the strategy is called ‘divide-and-conquer’ rather than ‘dynamic
programming’. This is why merge sort-quick sort are not classi- 9
fied as dynamic programming problems. 1
3 13 15
Dynamic programming is applied for: 6
• Multi stage graph 10
• All pairs shortest path
14
4 7

Principle of Optimality 11
It states that whatever the initial state is, remaining decisions must
be optimal with regard to the state following from the first decision. Costs of edges
To solve a problem using dynamic programming strategy, it 1 – 2 → 10
must observe the principle of optimality. 1 – 3 → 20
3.136 | Unit 3 • Algorithms

1–4 → 30 Cost (III, 7) = min {cost(II, 2) + cost (2, 7),

2–5 → 10 Cost (II, 3) + cost (3, 7),
2–6 → 20 Cost (II, 4) + cost (4, 7)}
2–7 → 30 = min {10 + 30, 20 + 50, 30 + 30}
3–5 → 40 = 40 → Via the path 1 – 2 – 7
3–7 → 50
Step IV:
4–6 → 40 Cost (IV, 8) = min {cost (III, 5) + cost (5, 8),
4–7 → 30
Cost (III, 6) + cost (6, 8),
5–8 → 10
Cost (III, 7) + cost (7, 8)}
5–9 → 20
= min {20 + 10, 30 + ∞, 40 + ∞}
5 – 10 → 30
5 – 11 → 40 = 30 → Via path 1 – 2 – 5 – 8
6–9 → 20 Cost (IV, 9) = min {cost (III, 5) + cost (5, 9),
6 – 10 → 30 Cost (III, 6) + cost (6, 9),
7 – 10 → 30 Cost (III, 7) + cost (7, 9)}
7 – 11 → 20 = min {20 + 20, 30 + 20, 40 + ∞}
8 – 12 → 10 = 40 → Via the path 1 – 2 – 5 – 9
8 – 13 → 20
Cost (IV, 10) = min {cost (III, 5) + cost (5, 10),
8 – 14 → 30
9 – 13 → 20 Cost (III, 6) + cost (6, 10),
9 – 14 → 10 Cost (III, 7) + cost (7, 10}
10 – 13 → 10 = min {20 + 30, 30 + 30, 40 + 30}
10 – 14 → 20 = 50 → Via the path 1 – 2 – 5 – 10
11 – 13 → 10 Cost (IV, 11) = min {cost (III, 5) + cost (5, 11)
11 – 14 → 30
Cost (III, 6) + cost (6, 11),
12 – 15 → 20
Cost (III, 7) + cost (7, 11)}
13 – 15 → 10
= min {20 + 40, 30 + ∞, 40 + 20}
14 – 15 → 30
= 60 → Via the path 1 – 2 – 5 – 11
Solution Using Backward Cost or Via the path 1 – 2 – 7 – 11
Format: COST (Stage, node) = minimum cost of travelling
Step V:
to the node in stage from the source node (node 1)
Cost (V, 12) = min {cost (IV, 8) + cost (8, 12)
Step I: Cost (IV, 9) + cost (9, 12),
Cost (I, 1) = 0 Cost (IV, 10) + cost (10, 12),
Step II: Cost (IV, 11) + cost (11, 12)}
= min {30 + 10, 40 + ∞, 50 + ∞, 60 + ∞}
Cost (II, 2) = cost (I, 1) + cost (1, 2) = 0 + 10 = 10
Cost (II, 3) = cost (I, 1) + cost (1, 3) = 0 + 20 = 20 = 40 → Via the path 1 - 2 - 5 – 8 – 12
Cost (II, 4) = cost (I, 1) + cost (1, 4) = 0 + 30 = 30 Cost (V, 13) = min {cost (IV, 8) + cost (8, 13)
Step III: Cost (IV, 9) + cost (9, 13),
Cost (IV, 10) + cost (10, 13),
Cost (III, 5) = min {cost (II, 2) + cost (2, 5),
Cost (IV, 11) + cost (11, 13)}
cost (II, 3) + cost (3, 5),
= min {30 + 20, 40 + 20, 50 + 10, 60 + 10}
cost (II, 4) + cost (4, 5)
= 50 → Via the path 1 – 2 – 5 – 8 – 13
= min {10 + 10, 20 + 40, 30 + ∞}
= 20 → Via path 1 – 2 – 5 Cost (V, 14) = min {cost (IV, 8) + cost (8, 14)
Cost (IV, 9) + cost (9, 14),
Cost (III, 6) = min {cost (II, 2) + cost (2, 6),
cost (II, 3) + cost (3, 6), Cost (IV, 10) + cost (10, 14),
cost (II, 4) + cost (4, 6)} Cost (IV, 11) + cost (11, 14)}
= min {10 + 20, 20 + ∞, 30 + 40} = min {30 + 30, 40 + 10, 50 + 20, 60 + 30}
= 30 → via the path 1 – 2 – 6 50 → Via the path 1 – 2 – 5 – 9 – 14
 Chapter 5 • Dynamic Programming | 3.137

Step VI: Example:

1
Cost (VI, 15) = min {cost (V, 12) + cost (12, 15), V1 V2
Cost (V, 13) + cost (13, 15), 3
9
5
Cost (V, 14) + cost (14, 15)} V5
2 3
1
= min {40 + 20, 50 + 10, 50 + 30}
3
= 60 → Via the path 1 – 2 – 5 - 8 – 13 – 15 2
V3
V4
(or) 1 – 2 – 5 – 8 – 12 – 15 4
The weight matrix will be
ALL PAIRS SHORTEST PATH PROBLEM 1 2 3 4 5
(FLOYD–WARSHALL ALGORITHM) 1 0 1 ∞ 1 5
A weighted graph is a collection of points (vertices) con- 2 9 0 3 2 ∞
nected by lines (edges), where each edge has a weight (some 3 ∞ ∞ 0 4 ∞
real number) associated with it. 4 ∞ ∞ 2 0 3
Example: A graph in the real world is a road map. Each 5 3 ∞ ∞ ∞ 0
location is a vertex and each road connecting locations is an
edge. We can think of the distance travelled on a road from Let D(K) [i, j] = weight of a shortest path from vi to vj using
one location to another as the weight of that edge. only vertices from {v1, v2, … vk} as intermediate vertices in
the path.
• The Floyd–Warshall algorithm determines the shortest • D(0) = W
path between all pairs of vertices in a graph. • D(n) = D which is the goal matrix.
• The vertices in a graph be numbered from 1 to n. Consider
the subset {1, 2, … K} of these n vertices. How to compute D(K) from D(K–1)?
• Finding the shortest path from vertex i to vertex j that uses Case I: A shortest path from vi to vj restricted to using only
vertex in the set {1, 2, … K} only. There are two situations. vertices from {v1, v2, … vK} as intermediate verti-
ces does not use VK.
1. K is an intermediate vertex on the shortest path. Then D(K) [i, j] = D(K-1) [i, j]
2. K is not an intermediate vertex on the shortest path.
Case II: A shortest path from vi to vj restricted to using only
In the first situation, we can break down our shortest path vertices from {v1, v2 … vK} as intermediate vertices
into two paths: i to K and then K to j. Note that all the ver- does use VK. Then D(K) [i, j] = D(K-1) [i, K] + D(K-1)
tices from i to K are from the set {1, 2, … K – 1} and that [K, j]
all the intermediate vertices from K to j are from the set {1, Since D(K) [i, j] = D(K-1) [i, j]
2, … K –1}. Also in the second situation, we simply have
or D(K) [i, j] = D(K-1) [i, K] + D(K –1) [K, j]
that all intermediate vertices are from the set {1, 2, ... K –
We conclude: D(K) [i, j] = min{D(K-1) [i, j], D(K –1) [i,
1}. Now define the function D for a weighted graph with the
K] + D(K–1) [K, j]}
vertices {1, 2, … n} as follows.
D (i, j, K) = the shortest distance from vertex i to vertex j Example: 1
using the intermediate vertices. In the set {1, 2, … K}
Using the above idea, we can recursively define the func- 1 5
tion D.
4
2 3
D(i, j, K) = W(i, j) if K = 0
min (D(i, j, K – 1), D(i, K, K –1) + D(K, j, K – 1)) if K > 0 −3
2
• The first line says that if we do not allow intermediate verti-
ces, then the shortest path between two vertices is the weight 1 2 3
of the edge that connects them. If no such weightexists, we 1 0 4 5
usually define this shortest path to be of length infinity.
W = D° = 2 2 0 ∞
• The second line pertains to allowing intermediate vertices. It
3 ∞ –3 0
says that the minimum path from i to j through vertices {1,
2, … K} is either the minimum path from i to j through ver- 1 2 3
tices {1, 2, … K – 1} OR the sum of the minimum path from 1 0 0 0
vertex i to K through {1, 2, … K – 1} plus the minimum path P= 2 0 0 0
from vertex K to j through {1, 2, … K – 1}. Since this is the 3 0 0 0
case, we compute both and choose the smaller of these.
3.138 | Unit 3 • Algorithms

K = 1, vertex 1 can be intermediate node Example 2:

D1 [2, 3] = min (D°[2, 3], D°[2, 1] + D°[1, 3])
= min (∞, 7) 1
3 5
=7 1
D1 [3, 2] = min (D°[3, 2], D°[3, 1] + D°[1, 2]) 5
7
1 2
= min (-3, ∞) 3
6
1 4
= –3 50
4 3
2
1 2 3
D1 = 1 0 4 5
The final distance matrix and P
2 2 0 7
3 ∞ –3 0
1 2 3 4 5 6
1 0 2(6) 2(6) 4(6) 3 1
1 2 3
2 2(6) 0 2(6) 4(6) 5(6) 1
P= 1 0 0 0
D6 = 3 2(6) 2(6) 0 2 5(4) 1
2 0 0 1
3 0 0 0 4 4(6) 4(6) 2 0 3 3(3)
5 3 5(4) 5(4) 3 0 4(1)
K = 2, vertices 1, 2 can be intermediate nodes, 6 1 1 1 3(3) 4(1) 0
D2 [1, 3] = min (D [1, 3], D [1, 2] + D [2, 3])
= min (5, 4 + 7) = 5 The values in parenthesis are the non-zero P values.
D [3, 1] = min (D [3, 1], D [3, 2] + D [2, 1])
2

= min (∞, –3 + 2) Table 1 Divide and conquer vs dynamic programming.

= –1 1. This design strategy 1. This design strategy
divides the problem into chooses an optimal solu-
1 2 3 sub problems, conquer the tion for the problem, by
D2 = 1 0 4 5 each sub problem recur- recursively defining the
2 2 0 7 sively, finally combine all value of optimal solution,
the sub problem solutions, these values are computed
3 –1 –3 0
for the original problem. in bottom up fashion or top
down fashion.
1 2 3 2. each sub problem is solved 2. Each sub problem is
P= 1 0 0 0 recursively, and consumes solved only once and is
more time at each sub stored in table
2 0 0 1
problem
3 2 0 0
3. Sub problems are inde- 3. The sub problems are
pendent of each other e.g., dependent e.g., Traveling
K = 3 vertices 1, 2, 3 can be intermediate Binary search sales person problem
D3[1, 2] = min (D2[1, 2], D2[1, 3] + D2[3, 2])
= min (4, 5 + (–3))
=2 Dynamic Programming
D3[2, 1] = min (D2[2, 1], D2[ 2, 3] + D2[3, 1]) vs Greedy Method
= min (2, 7 + (–1)) The main difference between greedy method (GM) and
=2 dynamic programming (DP) methodology is, DP consid-
ers all possible solutions for the given problem and picks
1 2 3 the optimal one. Where as greedy, considers only one set of
1 0 2 5 solutions to the problem.
D3 = 2 2 0 7 The other difference between GM and DP is that, GM
3 –1 -3 0 considers the choice, which is best at that step, which is done
at each level of the sub problem. That is, it won’t reconsider
1 2 3 its choice. The choices reflect only present, won’t consider
1 0 3 0 the future choices, where as DP tries out all the best alterna-
P= 2 0 0 1 tives and finds the optimal solution. It implements principle of
3 2 0 0 optimality. At each stage of the problem, it decides based on
the previous decision made in the previous stage.
 Chapter 5 • Dynamic Programming | 3.139

HASHING METHODS h(i) = i

h(i + 1) = i + 1 (mod M)
Uniform Hash Function
h(i + 2) = i + 2 (mod M)
If the keys, K, are integers randomly distributed in [0, r] .
then hash function H(K) is given as
.
 mk  .
H (K ) =  
 r  While this ensures that consecutive keys do not collide, it
does not mean that consecutive array locations will be occu-
H(K) is a uniform hash function pied. We will see that in certain implementations this can
Uniform hashing function should ensure lead to degradation in performance.
1
ΣP ( K ) = ΣP ( K ) = ⋯ Σ P ( K ) = Multiplication method
K |h ( K ) = 0 K | h ( K ) =1 K |h ( K ) = 0 m
A variation on the middle-square method that alleviates its
P(K) = probability that a key K, occurs that is the number of deficiencies is called, multiplication hashing method. Instead
keys that map to each slot is equal. of multiplying the key x by itself, we multiply the key by a
carefully chosen constant ‘a’ and then extract the middle k
bits from the result. In this case, the hashing function is
Division method
Hashing an integer x is to divide x by M and then to use the M 
h( x) =  (ax mod W )) 
remainder modulo M. This is called the division method of W 
hashing. In this case the hash function is
h(x) = x mod M if we want to avoid the problems that the middle-square
method encounters with keys having a large number of
Generally this approach is quite good for just about any leading (or) trailing zero’s then we should choose an ‘a’ that
value of M. However, in certain situations some extra care is has neither leading nor trailing zero’s.
needed in the selection of a suitable value for M. For exam- Furthermore, if we, choose an ‘a’ that is relatively prime
ple, it is often convenient to make M an even number. But to W, then there exists another number ‘a’ such that aa′ = 1
this means that h(x) is even if x is even, and h(x) is odd of x (mod W). Such a number has the nice property that if we
is odd. If all possible keys are equiprobable, then this is not take a key x, and multiply it by ‘a’ to get ax, we can recover
a problem. However, if say even keys are more likely than the original key by multiplying the product again by a′,
odd keys, the function h(x) = x mod M will not spread the since a × a′ = aa′x = 1x.
hashed values of those keys evenly. The multiplication method for creating a hash function
• Let M be a power of two, i.e., M = 2k for some integer k > 1. operates in two steps:
In this case, the hash function h(x) = x mod 2k simply Step 1: Multiply the key K by a constant A in the
extracts the bottom k-bits of the binary representation of range 0 < A < 1 and extract the fractional part of KA.
x. While this hash function is quite easy to compute, it is Step 2: Multiply this value by M and take the floor of the
not a desirable function because it does not depend on all result.
the bits in the binary representation of x.
• For these reasons M is often chosen to be a prime number. In short the hash function is
Suppose there is bias in the way the keys are created that h(k ) =  M ⋅ ( KA mod 1) 
makes it more likely for a key to be a multiple of some
small constant, say two or three. Then making M a prime Where (KA mod 1) denotes the fractional part of KA, that
increases the likelihood that those keys are spread out is KA  KA
evenly. Also if M is a prime number, the division of x by
that prime number depends on all the bits of x, not just the Example:
bottom k-bits, for some small constant k. Let m = 10000, K = 123456 and A = 5 − 1
2
Example: Hash table size = 10
Key value = 112 = 0.618033
Hash function = h(k) = k mod M Then h(k ) = 10000 ⋅ (123456 ⋅ 0.61803 mod 1) 
= 112 mod 10 = 2
Disadvantage: A potential disadvantage of the division = 10000 ⋅ (76300.00412 mod 1) 
method is due to the property that consecutive keys map to
consecutive hash values. = 10000 ⋅ 0.00412  = 41
3.140 | Unit 3 • Algorithms

Practical issues 123456789 are inserted into the table at address 8.

• Easy to implement The folding method is distribution independent.
–On most machines multiplication is faster than division.
Resolving collisions In collision resolution strategy
–We can substitute one multiplication by shift operation.
algorithms and data structures are used to handle two hash
–We don’t need to do floating-point operations.
keys that hash to the same hash keys. There are a number of
• If successive keys have a large interval, A = 0.6125423371
collision resolution techniques, but the most popular are open
can be recommended.
addressing and chaining.
Mid-square method • Chaining: An array of linked list, Separate chaining
A good hash function to use with integer key values is the • Open Addressing: Array based implementation:
mid-square method. The mid-square method squares the key – Linear probing (Linear Search)
value, and then takes out the middle ‘r’ bits of the result, giv- – Quadratic probing (non-linear search)
ing a value in the range 0 to 2r – 1. This works well because – Double hashing (use two hash functions)
most (or) all bits of the key value contribute to the result. Separate chaining Every linked list has each element
Example: that collides to the similar slot. Insertion need to locate the
Consider records whose keys are 4-digit numbers in base accurate slot and appending to any end of the list in that slot
10. The goal is to hash these key values to a table of size wherever, deletion needs searching the list and removal.
100(i.e., a range of 0 to 99).
This range is equivalent to two digits in base 10.
That is r = 2. If the input is the number 4567, squaring Hash key = key % table size [0] 72
4 = 36% 8 [1]
yields an 8-digit number, 20857489. The middle two digits of 2 = 18% 8 [2] 10 18
this result are 57. All digits of the original key value (equiva- 0 = 72% 8
[3] 43
lently, all bits when the number is viewed in binary) contribute 3 = 43% 8
6 = 6% 8 [4] 36
to the middle two digits of the squared value. Thus, the result is
2 = 10% 8 [5] 5
not dominated by the distribution of the bottom or the top digit 5 = 5% 8 [6] 6
of the original key value. Of course, if the key values all tend 7 = 15% 8 [7] 15
to be small numbers, then their squares will only affect the low
order digits of the hash value. Figure 1 Separate chaining

Example: To map the key 3121 into a hash table of size Open addressing Open addressing hash tables are used to
1000, we square it (3121)2 = 9740641 and extract 406 as stock up the records straight inside the array. This approach
the hash value. is also known as closed hashing. This procedure is based on
probing. Well known probe sequence include:
Folding method • Linear probing: In which the interval between probes is
fixed often at 1.
The folding method breaks up a key into precise segments • Quadratic probing: In which the interval between probes
that are added to form a hash value, and still another tech- increases proportional to the hash value (the interval thus
nique is to apply a multiplicative hash function to each seg- increasing linearly and the indices are described by a
ment individually before folding. quadratic function).
Algorithm H(x) = (a + b + c) mod m. Where a, b, and c • Double hashing: In which the interval between probes is
represent the preconditioned key broken down into three computed by another hash function.
parts, m is the table size, and mod stands for modulo. In other (i) Linear probing: Linear probing method is used for
words: The sum of three parts of the pre conditioned key is resolving hash collisions of values of hash functions
divided by the table size. The remainder is the hash key. by sequentially searching the hash table for a free loca-
Example: tion. The item will be stored in the next available slot
Fold the key 123456789 into a hash table of ten spaces (0 in the table in linear probing. Also an assumption is
through 9) made that the table is not already full.
We are given x = 123456789 and the table size (i.e., m = 10) This is implemented via a linear search for an empty
Since we can break x into three parts any way, we will slot, from the point of collision.
break it up evenly. If the physical end of table is reached during the
Thus a = 123, b = 456 and c = 789 linear search, the search will again get start around to
H(x) = (a + b + c) mod M the beginning of the table and continue from there. The
H(123456789) = (123 + 456 + 789) mod 10 table is considered as full, if an empty slot is not found
= 1368 mod 10 = 8 before reaching the point of collision.
 Chapter 5 • Dynamic Programming | 3.141

[0] 72 [0] 72 [0]

Table size = 10 elements [1]
[1] [1] 5 Hash1(key) = key %10
[2]
Add the keys 10, 5 and Hash 2(key) = 7 − (key %7)
[2] 18 15 to the previous [2] 18 Insert keys: 89, 18, 49, 58 and 69 [3]

[3] 43
example. Hash key (89) = 89% 10 = 9 [4]
Hash key = key % table [3] 43
Hash key (18) = 18% 10 = 8 [5]
[4] 36 size [4] 36 Hash key (49) = 49% 10 = 9 (collision)
[6] 49
2 = 10% 8 = (7 − (49% 7)
[5] 5 = 5% 8 [5] 10 = (7 − (0)) [7]
7 = 15% 8 = 7 positions from [9] [8] 18
[6] 6 [6] 6
[9] 89
[7] 7 [7] 7
Figure 3 Double hashing

Figure 2 Linear probing

Limitation: A problem with linear probe method is

Insert keys = 58, 69
primary clustering. In primary clustering blocks of data Hash key (58) = 58% 10 = 8 a collision!
may possibly be able to form collision. Several attempts = (7 − (58% 7) = (7 − 2 ) = 5 positions from [8]
may be required by any key that hashes into the cluster Hash key (69) = 69% 10 = 9 a collision!
to resolve the collision. = (7 − (69 % 7)) = (7 − 6) = 1 position from [9]
(ii) Quadratic probing: To resolve the primary clustering [0] 69
problem, quadratic probing can be used. With quadratic [1]
probing, rather than always moving one spot, move i2 [2]
spots from the point of collision where i is the number [3] 58
of attempts needed to resolve the collision. [4]
[5]
[0] 49 [6] 49
[1] [7]
89% 10 = 9 [8] 18
[2] 58
18% 10 = 8 [9] 89
[3] 69 49% 10 = 9 → 1 attempt needed → 12 = 1 spot
[4] 58% 10 = 8 → 2 attempts needed → 22 = 4 spot
[5] 69% 10 = 9 → 2 attempts needed → 22 = 4 spot Figure 4 Double hashing
[6]
[7]
[8] 18
MATRIX-CHAIN MULTIPLICATION
[9] 89
We are given a sequence of n matrices m1, m2 … mn to be mul-
tiplied. If the chain matrices is < m1, m2, m3, m4>, the product
m1, m2, m3, m4 can be fully parenthesized in 5 distinct ways:
Limitation: Maximum half of the table can be used
as substitute locations to resolve collisions. Once the 1. (m1 (m2 (m3 m4)))
table gets more than half full, its really hard to locate 2. (m1 ((m2 m3) m4))
an unfilled spot. This new difficulty is recognized as 3. ((m1 m2) (m3 m4))
secondary clustering because elements that hash to the 4. ((m1 (m2 m3)) m4)
same hash key will always probe the identical substi- 5. (((m1 m2) m3) m4)
tute cells.
The way we parenthesize a chain of matrices can have a
(iii) Double hashing: Double hashing uses the idea of dramatic impact on the cost of evaluating the product. We
applying a second hash function to the key when a col- can multiply 2 matrices A and B only if they are compatible
lision occurs, the result of the second hash function i.e., the number of columns of A must equal the number
will be the number of positions from the point of col- of rows of B. If A is a (p × q) matrix and B is a (q × r)
lision to insert. There are some requirements for the matrix, the resulting matrix C is a (p × r) matrix. The time
second function: to compute C is the number of scalar multiplications, which
1. It must never evaluate to zero is (pqr).
2. Must make sure that all cells can be probed. Example: Consider the problem of a chain <m1, m2, m3> of
A popular second hash function is: three matrices. Suppose that the dimensions of the matrices
Hash(key) = R-(Key mod R) where R is a prime num- are (6 × 8), (8 × 14), (14 × 20) respectively. Which parenthe-
ber smaller than the size of the table. sization will give least number of multiplications?
3.142 | Unit 3 • Algorithms

Soluation: n
  + n = θ (n ) in all. The property of overlapping
2

(i) ((m1 m2) m3) 2

sub problems is the second hallmark of the applicability of
[m1]6 × 8 × [m2]8 × 14 = [m1 m2]6 × 14
dynamic programming.
Number of multiplications performed The first hall mark being optimal substructure.
= 6 × 8 × 14 = 672 Algorithm
[m1 m2]6 × 14 × [m3]14 × 20 = ((m1 m2) m3)6 × 20 1. n ← length [p] – 1
Number of multiplications performed 2. for i ← 1 to n
3. do m[i, i] ← 0
= 6 × 14 × 20 = 1680
4. for i ← 2 to n
Total number of multiplications 5. do for i ← 1 to n – i + 1
= 672 + 1680 = 2352 6. do j ← i + i – 1
(ii) (m1 (m2 m3)) 7. m[i, j] ← ∞
8. for k ← i to j – 1
[m2]8 × 14 × [m3]14 × 20 = [m2 m3]8 × 20
9. do q ← m [i, k] + m [k + 1, j] + Pi – 1 Pk Pj
Number of multiplications performed 10. if q < m [i, j]
= 8 × 14 × 20 = 2240 11. then m [i, j] ← q
[m1]6 × 8 × [m2 m3]8 × 20 = (m1 (m2 m3))6 × 20 12. S[i, j] ← k
13. return m and S
Number of multiplications performed
= 6 × 8 × 20 = 960 It first computes m[i, j] ← 0 for i = 1, 2 … n (the minimum
Total number of multiplications = 960 + 2240 = 3200 costs for chains of length 1). To compute m[i, i + 1] for i = 1,
∴ ((m1 m2) m3) gives least number of multiplications. 2, … n – 1(the minimum costs for chains of length l = 2 and
so on). At each step, the m[i, j] cost computed depends only
We need to define the cost of an optimal solution recur-
on table entries m[i, k] and m[k + 1, j] already computed. An
sively in terms of the optimal solutions to sub problems. For
entry m[i, j] is computed using the products Pi–1 Pk Pj for k
Matrix-chain multiplication problem, we pick as our sub
= i, i + 1, … j – 1. A simple inspection of the nested loop
problem the problems of determining the minimum cost of
structure of the above algorithm yields a running time of
a parenthesization of Ai Ai+1 … Aj for 1 ≤ i ≤ j ≤ n let m[i, j]
O(n3) for the algorithm.
be the minimum number of scalar multiplications needed to
compute the matrix Ai … j; for the full problem, the cost of
a cheapest way to compute A1 … N would be m[1, n]. We can
define m[i, j] recursively as follows:
LONGEST COMMON SUBSEQUENCE
A sub sequence of a given sequence is just the given
m [i, j] = m [i, k] + m [k + 1, j] + Pi-1 Pk Pj sequence with 0 or more elements left out. Formally, given a
sequence x = <x1, x2 … xm>, another sequence z = <z1, z2 …
If i = j, the problem is trivial. The chain consists of just one zk > is a subsequence of x if there exists a strictly increasing
matrix Ai … i = Ai, so that no scalar multiplications are nec- sequence <i1, i2 … ik> of indices of x such that for all j = 1,
essary to compute the product. 2 … k, we have xij = zj
Minimum cost of parenthesizing the product Ai Ai+1 …
Aj becomes Example: z = < B, C, D, B > is a subsequence of x = <A,
B, C, B, D, A, B> with corresponding index sequence <2,
0 if i = j 3, 5, 7>

m[i, j ] = min{m[i, k ] + m[k + 1, j ] Example: Given 2 sequences x and y, we say that a sequence
+ p p p } if i < j , i ≤ k < j
 i −1 k i z is a common sub sequence of x and y if z is a sub sequence
of both x and y.
The m[i, j] values give the costs of optimal solutions to sub If x = <A, B, C, B, D, A, B>
problems. y = <B, D, C, A, B, A>
At this point, to write a recursive algorithm based on The sequence <B, C, A> is a common subsequence of both
recurrence to compute the minimum cost m[1, n] for mul- x and y.
tiplying A1 A2 … An. However, this algorithm takes expo- The sequence <B, C, A> is not a longest common sub-
nential time, which is not better than the brute force method sequence (LCS) of x and y since it has length ‘3’ and the
of checking each way of parenthesizing the product. The sequence <B, C, B, A>, which is also common to both x
important observation we can make at this point is that we and y, has length 4. The sequence <B, C, B, A> is an LCS
have relatively few sub problems, one problem for each of x and y, as is the sequence <B, D, A, B>, since there is no
choice of i and j satisfying 1 ≤ i ≤ j ≤ n (or) common subsequence of length 5 or greater.
 Chapter 5 • Dynamic Programming | 3.143

• In the longest-common-sub sequence problem, we are NP, and a solution to the problem is somehow known, then
given 2 sequences x = <x1, x2, x3 … xm> and y = <y1, y2 … demonstrating the correctness of the solution can always be
yn> and wish to find a maximum length common subse- reduced to a single P (polynomial time) verification. If P
quence of x and y. and NP are not equivalent then the solution of NP-problems
• LCS problem can be solved efficiently using dynamic requires (in the worst case) an exhaustive search.
programming. A problem is said to be NP-hard, if an algorithm for
• A brute force approach to solve the LCS problem is to enu- solving it can be translated into one for solving any other
merate all subsequences of x and check each subsequence NP-problem. It is much easier to show that a problem is NP
to see if it is also a subsequence of y, keeping track of the than to show that it is NP-hard. A problem which is both
longest subsequence found. Each subsequence of x corre- NP and NP-hard is called an NP-complete problem.
sponds to a subset of the indices {1, 2 … m} of x. There are
2m subsequences of x, so this approach requires exponen- P versus NP-problems
tial time, making it impractical for long sequences. The P versus NP problem is the determination of whether
• The classes of sub problems correspond to pairs of ‘pre all NP-problems are actually P-problems, if P and NP are
fixes’ of 2 input sequences: not equivalent then the solution of NP-problem requires an
Given a sequence x = <x1, x2 … xm>, we define the ith exhaustive search, while if they are, then asymptotically
prefix of x, for i = 0, 1, … m, as faster algorithms may exist.
xi = <x1 x2 … xi>
NP-complete problem
Example: If x = <A, B, C, B, D, A, D>, then x4 = <A, B, C,
B> and x0 is the empty sequence. LCS problem has an opti- A problem which is both NP (verifiable in non-deterministic
mal sub-structure property. polynomial time) and NP-hard (any NP-problem can be
translated into this problem). Examples of NP-hard prob-
Optimal Substructure of LCS lems include the Hamiltonian cycle and travelling sales man
Let x = <x1, x2 … xm> and y = <y1, y2 … yn> be sequences problems.
and let z = <z1, z2 … zk> be any LCS of x and y then Example:
1. If xm = yn, then zk = xm = yn and zk–1 is an LCS of xm –1 and Circuit satisfiability is a good example of problem that we
yn – 1. don’t know how to solve in polynomial time. In this prob-
2. If xm ≠ yn, then zk ≠ xm implies that z is an LCS of xm–1 lem, the input is a Boolean circuit. A collection of and, or
and y. and not gates connected by wires. The input to the circuit is
3. If xm ≠ yn, then zk ≠ yn implies that z is an LCS of x and a set of m Boolean (true/false) values x1 … xm. The output
yn – 1. is a single Boolean value. The circuit satisfiability problem
asks, given a circuit, whether there is an input that makes
the circuit output TRUE, or conversely, whether the circuit
NP-HARD AND NP-COMPLETE always outputs FLASE. Nobody knows how to solve this
A mathematical problem for which, even in theory, no short- problem faster than just trying all 2m possible inputs to the
cut or smart algorithm is possible that would lead to a sim- circuit but this requires exponential time.
ple or rapid solution. Instead the only way to find an optimal
solution is a computationally intensive, exhaustive analysis in P, NP, and Co-NP
which all possible outcomes are tested. Examples of NP-hard • P is a set of yes/no problems that can be solved in poly-
problems include the travelling salesman problem. nomial time. Intuitively P is the set of problems that can
be solved quickly.
P-problem • NP is the set of yes/no problems with the following
property: If the answer is yes, then there is a proof of this
A problem is assigned to the P (polynomial time) class if
fact that can be checked in polynomial time. Intuitively
there exists at least one algorithm to solve that problem,
NP is the set of problems where we can verify a YES
such that number of steps of the algorithm is bounded by a
answer quickly if we have the solution in front of us.
polynomial in n, where n is the length of the input.
Example: The circuit satisfiability problem is in NP.
NP-problem If the answer is yes, then any set of m input values that pro-
duces TRUE output is a proof of this fact, we can check the
A problem is assigned to the NP (non-deterministic poly-
proof by evaluating the circuit in polynomial time.
nomial time) class if it is solvable in polynomial time by a
non-deterministic turing machine. • Co-NP is the exact opposite of NP. If the answer to a
A P-problem (whose solution time is bounded by a pol- problem in co-NP is no, then there is a proof of this fact
ynomial) is always also NP. If a problem is known to be that can be checked in polynomial time.
3.144 | Unit 3 • Algorithms

• π is NP-hard ⇒ if π can be solved in polynomial time, Cooks Theorem

then P = NP. Cook’s theorem states that CNFSAT is NP-Complete
This is like saying that if we could solve one particular It means, if the problem is in NP, then the deterministic
NP-hard problem quickly, then we could solve any problem Turing machine can reduce the problem in polynomial time.
whose solution is easy to understand, using the solution to The inference that can be taken from these theorems is,
that one special problem as a subroutine. NP-hard problems if deterministic polynomial time algorithm exists for solv-
are atleast as hard as any problem in NP. ing satisfiability, then to all problems present in NP can be
solved in polynomial time.
• Saying that a problem is NP-hard is like saying ‘If I own
a dog, then it can speak fluent English’. You probably
don’t know whether or not I own a dog, but you’re prob- Non-deterministic Search
ably pretty sure that I don’t own a talking dog. Nobody Non-deterministic algorithms are faster, compared to
has a mathematical proof that dogs can’t speak English. deterministic ones. The computations are fast as it always
The fact that no one has ever heard a dog speak English chooses right step
is evidence as per the hundreds of examinations of dogs The following functions are used to specify these algorithms
that lacked the proper mouth shape and brain power, but 1. Choice (A), which chooses a random element from set A
mere evidence is not a proof nevertheless, no sane person 2. Failure (A), specifies failure
would believe me if I said I owned a dog that spoke fluent 3. Success ( ), Specifies success
English. So the statement ‘If I own a dog then it can speak
fluent English’ has a natural corollary: No one in their The non-deterministic search is done as follows.
right mind should believe that I own a dog ! Likewise if Let us consider an array S[1 … n], n ≥ 1 we need to get
a problem is NP-hard no one in their right mind should the indice of ‘i’ such that S[i] = t (or) i = 0. The algorithm
believe it can be solved in polynomial time. is given below.
Steps:
NP −hard
1. i = Choice (1, n);
2. if S[i] = t, then
(i) Print (i);
(ii) Success ( );
Co−NP NP 3. Print (0)
failure
4. Stop.
P
If the search is successful it returns the indice of array ‘S’,
NP−complete
otherwise it returns ‘0’, the time complexity is Ω(n).

EXERCISES
Practice Problems 1 4. Insert element 14 into the given hash table with double
Directions for questions 1 to 15: Select the correct alternative hashing? h1 (k) = k mod 13, h2 (k) = 1 + (k mod 11). The
from the given choices. element will occupy, which slot?
1. Hash the keys 12, 44, 13, 88, 23, 94, 11, 39, 20 using
0
the hash function with chaining (2 k + 5) mod 11, which
1 79
of the following slots are empty?
2
(A) 0, 1, 2, 3, 4 (B) 0, 2, 3, 4, 8, 10
3
(C) 0, 1, 2, 4, 8, 10 (D) 0, 1, 2, 4, 8
4 69
2. Using linear probing on the list given in the above ques-
5 98
tion with the same hash function, which slots are not
6
occupied? 7 72
(A) 3, 4 (B) 4, 5 8
(C) 3, 6 (D) 4, 6 9
3. In hashing, key value 123456 is hashed to which 10
address using multiplication method (m = 104)? 11 50
(A) 40 (B) 41 12
(C) 42 (D) 44
 Chapter 5 • Dynamic Programming | 3.145

(A) 7th (B) 8th 11. Consider the following input (322, 334, 471, 679, 989,
(C) 2nd (D) 9th 171, 173, 199) and the hash function is x mod 10 which
5. Consider the below given keys: statement is true?
257145368, 25842354, 12487654, 248645452. Find I. 679, 989, 199 hash to the same value
the hash values of keys using shift folding method? II. 471, 171, hash to the same value
(A) 770, 221, 153, 345 (B) 221, 770, 153, 345 III. Each element hashes to a different value
(C) 760, 770, 153, 345 (D) 815, 770, 153, 345
IV. All the elements hash to the same value
6. Consider the following two problems on unidirected (A) I Only (B) II Only
graphs. (C) I and II (D) III
β : Given G(V, E), does G have an independent set of 12. For the input 30, 20, 56, 75, 31, 19 and hash function
size |V|-4? h (k) = k mod 11, what is the largest number of key
α : Given G(V, E), does G have an independent set of comparisons in a successful search in the open hash
size 5? table.
Which of the following is true? (A) 4 (B) 3
(A) β is in P and α is in NP-Complete (C) 5 (D) 2
(B) β is in NP-Complete and α is in P 13. The keys 12, 18, 13, 2, 3, 23, 5 and 15 are inserted into
(C) Both α and β are NP-Complete an empty hash table of length 10 using open address-
(D) Both α and β are in P ing with hash function, h (k) = k mod 10 and linear
7. Let S be an NP-complete problem and Q and R be probing.
two other problems not known to be in NP. Q is pol- Which is the resultant hash table?
ynomial-time reducible to S and S is polynomial-time (A) (B) 0
0
reducible to R. Which one of the following statements 3 1
1
is true?
2 2 12 2
(A) R is NP-Complete (B) R is NP-Hard
(C) Q is NP-Complete (D) Q is NP-Hard 23 3 13 3

13 4 4
8. Let FHAM3 be the problem of finding a Hamiltonian cycle
in a graph G = (V, E) with |V| divisible by 3 and DHAM3 15 5 15 5
be the problem of determining if a Hamiltonian cycle 6 6
exists in such graphs. Which of the following is true? 7 7
(A) Both FHAM3 and DHAM3 are NP-hard 8 8
(B) FHAM3 is NP-hard but DHAM3 is not 9
9
(C) DHAM3 is NP-hard but FHAM3 is not
(D) Neither FHAM3 nor DHAM3 is NP-hard
(C) 0 (D) 0
9. Consider a hash table of size 7, with starting index ‘0’
and a hash function (3x + 4) mod 7. Initially hash table 1 1
is empty. The sequence 1, 3, 8, 10 is inserted into the 12 2 2 2
table using closed hashing then what is the position of 13 3 3 3
element 10? 2 4 12 4
(A) 1st (B) 2nd 3 5 13 5
(C) 6th (D) 0th
23 6 23 6
10. Place the given keys in the hash table of size 13, index 5 7
5 7
from ‘0’ by using open hashing, hash function is h(k)
18 8 18 8
mod 13.
15 9 15 9
Keys: A, FOOL, HIS, AND
(hint : Add the positions of a word’s letters in the alpha- 14. Which one of the following is correct?
bet, take A → 1, B → 2, C → 3. D → 4 … Z → 26). (A) Finding shortest path in a graph is solvable in poly-
Which of the following shows the correct hash nomial time.
addresses of keys? (B) Finding longest path from a graph is solvable in
(A) A – 1, FOOL – 10, HIS – 9, AND – 6 poly-nomial time.
(B) A – 1, FOOL – 9, HIS – 10, AND – 6 (C) Finding longest path from a graph is solvable in
(C) A – 0, FOOL – 6, HIS – 10, AND – 9 polynomial time, if edge weights are very small values.
(D) A – 0, FOOL – 9, HIS – 9, AND – 6 (D) Both (A) and (B) are correct
3.146 | Unit 3 • Algorithms

15. In the following pair of problems (B) II is solvable in polynomial time, I is NP complete
problem.
2 CNF Satisfiability 3 CNF Satisfiability
Vs . (C) Both are solvable in polynomial time
I II (D) None can be solved in polynomial time.
(A) I is solvable in polynomial time, II is NP complete
problem.

Practice Problems 2 9. Which of the following is TRUE?

(A) All NP-complete problems are NP-hard.
Directions for questions 1 to 15: Select the correct alterna-
(B) If an NP-hard problem can be solved in polyno-
tive from the given choices.
mial time, then all NP-complete problems can be
1. For NP-complete problems solved in polynomial time.
(A) Several polynomial time algorithms are available (C) NP-hard problems are not known to be NP-complete.
(B) No polynomial time algorithm is discovered yet (D) All the above
(C) Polynomial time algorithms exist but not discovered
10. If a polynomial time algorithm makes polynomial num-
(D) Polynomial time algorithms will not exist, hence
ber of calls to polynomial time subroutines, then the
cannot be discovered
resulting algorithm runs in
2. In the division method for creating hash functions, we (A) Polynomial time (B) No-polynomial time
map a key k into one of m slots by taking the remainder (C) Exponential time (D) None of these
of k divided by m. That is, the hash function is
11. If a polynomial time algorithm makes atmost constant
(A) h(k) = m mod k (B) h(k) = m mod m/k
number of calls to polynomial time subroutines, then
(C) h(k) = k mod m (D) h(k) = mk mod k
the resulting algorithm runs in
3. In the division method for creating hash function, which (A) Polynomial time (B) No-polynomial time
of the following hash table size is most appropriate? (C) Exponential time (D) None of these
(A) 2 (B) 7
12. When a record to be inserted maps to an already occu-
(C) 4 (D) 8
pied slot is called
4. Which of the following techniques are commonly used to (A) Hazard
compute the probe sequence required for open addressing? (B) Collision
(A) Linear probing (B) Quadratic probing (C) Hashing
(C) Double hashing (D) All the above (D) Chaining
5. Which of the following problems is not NP-hard? 13. Worst-case analysis of hashing occurs when
(A) Hamiltonian circuit problem (A) All the keys are distributed
(B) The 0/1 knapsack problem (B) Every key hash to the same slot
(C) The graph coloring problem (C) Key values with even number, hashes to slots with
(D) None of these even number
6. For problems x and y, y is NP-complete and x reduces (D) Key values with odd number hashes to slots with
to y in polynomial time. Which of the following is true? odd number
(A) If x can be solved in polynomial time, then so can y 14. Main difference between open hashing and closed
(B) x is NP-hard hashing is
(C) x is NP-complete (A) Closed hashing uses linked lists and open hashing
(D) x is in NP, but not necessarily NP-complete does not.
7. If P1 is NP-complete and there is a polynomial time (B) Open hashing uses linked list and closed hashing
reduction of P1 to P2, then P2 is does not
(A) NP-complete (C) Open hashing uses tree data structure and closed
(B) Not necessarily NP-complete uses linked list
(C) Cannot be NP-complete (D) None of the above
(D) None of these 15. The worst case scenario in hashing occurs when
8. A problem is in NP, and as hard as any problem in NP. (A) All keys are hashed to the same cell of the hash table
The given problem is (B) The size of hash table is bigger than the number of
(A) NP hard keys
(B) NP complete (C) The size of hash table is smaller than the number
(C) NP of keys
(D) NP-hard ∩ NP-complete (D) None of the above
 Chapter 5 • Dynamic Programming | 3.147

PREVIOUS YEARS’ QUESTIONS

1. Consider a hash table of size seven, with starting index and linear probing. What is the resultant hash table?
zero, and a hash function (3x + 4) mod7. Assuming [2009]
the hash table is initially empty, which of the follow- (A) 0 (B) 0
ing is the contents of the table when the sequence 1, 3, 1 1
8, 10 is inserted into the table using closed hashing? 2 12 2 12
Note that − denotes an empty location in the table. 3 23 3 13
[2007] 4 4
5 15 5 5
(A) 8, −, −, −, −, −, 10 (B) 1, 8, 10, −, −, −, 3
6 6
(C) 1, −, −, −, −, −, 3 (D) 1, 10, 8, −, −, −, 3 7 7
8 18 8 18
Common data for questions 2 and 3: Suppose the let- 9 9
1 1 1 1 1 1
ters a, b, c, d, e, f have probabilities , , , , , , (C) (D)
2 4 8 16 32 32 0 0
respectively. 1 1
2 12 2 12,2
2. Which of the following is the Huffman code for the 3 13 3 13,3,23
letter a, b, c, d, e, f ? [2007] 4 2 4
(A) 0, 10, 110, 1110, 11110, 11111 5 3 5 5,15
(B) 11, 10, 011, 010, 001, 000 6 23 6
(C) 11, 10, 01, 001, 0001, 0000 7 5 7
(D) 110, 100, 010, 000, 001, 111 8 18 8 18
9 15 9
3. What is the average length of the correct answer to
above question?
Common data for questions 7 and 8: A sub-sequence
[2007]
of a given sequence is just the given sequence with some
(A) 3 (B) 2.1875 elements (possibly none or all) left out. We are given two
(C) 2.25 (D) 1.9375 sequences X[m] and Y[n] of lengths m and n, respectively,
4. The subset-sum problem is defined as follows: Given with indexes of X and Y starting from 0.
a set S of n positive integers and a positive integer W,
determine whether there is a subset of S whose ele- 7. We wish to find the length of the longest common sub-
ments sum to W. sequence (LCS) of X[m] and Y[n] as l(m, n), where an
An algorithm Q solves this problem in O(nW) incomplete recursive definition for the function l(i, j)
time. Which of the following statements is false? to compute the length of the LCS of X[m] and Y[n] is
[2008] given below:
I(i, j) = 0, if either i = 0 or j = 0
(A) Q solves the subset-sum problem in polynomial
= expr1, if i, j > 0 and X[i - 1] = Y[ j - 1]
time when the input is encoded in unary
= expr2, if i, j > 0 and X[i - 1] ≠ Y [ j - 1]
(B) Q solves the subset-sum problem in polynomial
time when the input is encoded in binary Which one of the following options is correct? [2009]
(C) The subset sum problem belongs to the class NP (A) expr1 ≡ I(i - 1, j) + 1
(D) The subset sum problem is NP-hard (B) expr1 ≡ I(i, j - 1)
(C) expr2 ≡ max(I(i - 1, j), I(i, j - 1))
5. Let πA be a problem that belongs to the class NP. Then
which one of the following is TRUE? (D) expr2 ≡ max(I(i -1, j - 1), I(i, j))
[2009] 8. The values of l(i, j) could be obtained by dynamic
(A) There is no polynomial time algorithm for πA. programming based on the correct recursive defini-
(B) If πA can be solved deterministically in polyno- tion of l(i, j) of the form given above, using an array
mial time, then P = NP. L[M, N], where M = m + 1 and N = n + 1, such that
(C) If πA is NP-hard, then it is NP-complete. L[i, j] = l(i, j).
(D) πA may be undecidable. Which one of the following statements would be
TRUE regarding the dynamic programming solution
6. The keys 12, 18, 13, 2, 3, 23, 5 and 15 are inserted
for the recursive definition of l(i, j)? [2009]
into an initially empty hash table of length 10 using
(A) All elements of L should be initialized to 0 for
open addressing with hash function h(k) = k mod 10
the values of l(i, j) to be properly computed.
3.148 | Unit 3 • Algorithms

(B) The values of l(i, j) may be computed in a row (NPC)? [2014]

major order or column major order of L(M, N). (A) (B)
(C) The values of l(i, j) cannot be computed in either P NP P NP
row major order or column major order of L(M, N). NPC
(D) L[p, q] needs to be computed before L[r, s] if NPC
either p < r or q < s. P = NP P = NP = NPC
(C) (D)
9. The weight of a sequence a0, a1, …, an-1 of real num- NPC
bers is defined as a0 + a1/2 + … + an-1/2n-1. A subse-
quence of a sequence is obtained by deleting some
elements from the sequence, keeping the order of the 14. Consider a hash, table with 9 slots. The hash func-
remaining elements the same. Let X denote the maxi- tion is h(K) = K mod 9. The collisions are resolved
mum possible weight of a subsequence of a0, a1, …., by chaining. The following 9 keys are inserted in the
an-1. Then X is equal to [2010] order: 5, 28, 19, 15, 20, 33, 12, 17, 10. The maximum,
(A) max (Y, a0 + Y) (B) max (Y, a0 + Y/2) minimum, and average chain lengths in the hash table,
(C) max (Y, a0 + 2Y) (D) a0 + Y/2 respectively, are [2014]
(A) 3, 0 and 1 (B) 3, 3 and 3
10. Four matrices M1, M2, M3 and M4 of dimensions p × (C) 4, 0 and 1 (D) 3, 0 and 2
q, q × r, r × s and s × t respectively, can be multiplied
15. Consider two strings A = ‘qpqrr’ and B = ‘pqprqrp’.
in several ways with different number of total scalar
Let x be the length of the longest common subsequence
multiplications. For example when multiplied as ((M1
(not necessarily contiguous between A and B and let y
× M2) × (M3 × M4)), the total number of scalar multipli-
be the number of such longest common subsequences
cations is pqr + rst + prt. When multiplied (((M1 × M2)
between A and B. then x + 10y = –––––––– [2014]
× M3) × M4) the total number of scalar multiplications
is pqr + prs + pst. 16. Suppose you want to move from 0 to 100 on the
If p = 10, q = 100, r = 20, s = 5 and t = 80, then the number line. In each step, you either move right by
minimum number of scalar multiplications needed is a unit distance or you take a shortcut. A shortcut is
[2011] simply a pre-specified pair of integers i, j with i <
j. Given a shortcut i, j if you are at position i on the
(A) 248000
number line, you may directly move to j. Suppose
(B) 44000
T(k) denotes the smallest number of steps needed to
(C) 19000
move from k to 100. Suppose further that there is at
(D) 25000
most 1 shortcut involving any number, and in particu-
11. Assuming P ≠ NP, which of the following is TRUE? lar from 9 there is a shortcut to 15. Let y and z be
[2012] such that T(9) = 1 + min(T(y), T(z)). Then the value of
(A) NP-complete = NP the product yz is _______ [2014]
(B) NP-complete ∩ P = ∅
(C) NP-hard = NP 17. Consider the decision problem 2CNFSAT defined as
(D) P = NP-complete follows: [2014]
12. Which of the following statements are TRUE? {φ | φ is a satisfiable propositional formula in CNF
(i) The problem of determining whether there exists with at most two literals per clause}
a cycle in an undirected graph is in P. For example, φ = (x1 ∨ x2) ∧ (x1 ∨ x3) ∧ (x2 ∨ x4) is a
(ii) The problem of determining whether there exists Boolean formula and it is in 2CNFSAT.
a cycle in an undirected graph is in NP. The decision problem 2CNFSAT is
(iii) If a problem A is NP-Complete, there exists a non-
deterministic polynomial time algorithm to solve A. (A) NP-complete
[2013] (B) Solvable in polynomial time by reduction to di-
(A) 1, 2 and 3 (B) 1 and 2 only rected graph reach ability.
(C) 2 and 3 only (D) 1 and 3 only (C) Solvable in constant time since any input in-
13. Suppose a polynomial time algorithm is discov- stance is satisfiable.
ered that correctly computes the largest clique in (D) NP-hard, but not NP-complete
a given graph. In this scenario, which one of the
following represents the correct Venn diagram of 18. Consider a hash table with 100 slots. Collisions are
the complexity classes P, NP and NP-complete resolved using chaining. Assuming simple uniform
 Chapter 5 • Dynamic Programming | 3.149

hashing, what is the probability that the first 3 slots Match the algorithms to the design paradigms they
are unfilled after the first 3 insertions? [2014] are based on. [2017]
(A) (97 × 97 × 97)/1003 (A) (P) ↔ (ii), (Q) ↔ (iii),(R) ↔ (i)
(B) (99 × 98 × 97)/1003 (B) (P) ↔ (iii), (Q) ↔ (i), (R) ↔ (ii)
(C) (97 × 96 × 95)/1003 (C) (P) ↔ (ii), (Q) ↔ (i), (R) ↔ (iii)
(D) (97 × 96 × 95)/(3! × 1003) (D) (P) ↔ (i), (Q) ↔ (ii), (R) ↔ (iii)
19. Match the following [2014] 25. Assume that multiplying a matrix G1 of dimension p ×
(P) prim’s algorithm for minimum (i) Backtracking q with another matrix G2 of dimension q × r requires
spanning tree pqr scalar multiplications. Computing the product of
(Q) Floyd-Warshall algorithm for (ii) Greedy method n matrices G1G2G3, ..., Gn can be done by parenthesiz-
all pairs shortest paths ing in different ways. Define Gi Gi+1 as an explicitly
(R) Mergesort (iii) Dynamic programming computed pair for a given paranthesization if they are
(S) Hamiltonian circuit (iv) Divide and conquer
directly multiplied. For example, in the matrix multi-
plication chain G1G2G3G4G5G6 using parenthesization
(A) P–iii, Q–ii, R–iv, S–i (G1(G2G3))(G4(G5G6)), G2G3 and G5G6 are the only
(B) P–i, Q–ii, R–iv, S–iii explicitly computed pairs.
(C) P–ii, Q–iii, R–iv, S–i
Consider a matrix multiplication chain
(D) P–ii, Q–i, R–iii, S–iv
F1F2F3F4F5, where matrices F1, F2, F3,
20. Given a hash table T with 25 slots that stores 2000 F4, and F5 are of dimensions 2 × 25,
elements, the load factor ∝ for T is _______ [2015] 25 × 3, 3 × 16, 16 × 1 and 1 × 1000, respectively. In
21. Language L1 is polynomial time reducible to language the parenthesization of F1F2F3F4F5 that minimizes the
L2. Language L3 is polynomial time reducible to L2, total number of scalar multiplications, the explicitly
which in turn is polynomial time reducible to lan- computed pairs is/are: [2018]
guage L4. Which of the following is/are true? [2015] (A) F1F2 and F3F4 only
(1) if L4 ∈ P, then L2 ∈ P (B) F2F3 only
(2) if L1 ∈ P or L3 ∈ P, then L2 ∈ P (C) F3F4 only
(3) L1 ∈ P, if and only if L3 ∈ P (D) F1F2 and F4F5 only
(4) if L4 ∈ P, then L1 ∈ P and L3 ∈ P 26. Consider the weights and values of items listed below.
22. The Floyd - Warshall algorithm for all -pair shortest Note that there is only one unit of each item.
paths computation is based on [2016]
Item no. Weight Value
(A) Greedy paradigm (in Kgs) (in Rupees)
(B) Divide-and-Conquer paradigm 1 10 60
(C) Dynamic Programming paradigm
2 7 28
(D) Neither Greedy nor Divide-and-Conquer nor
3 4 20
Dynamic Programming paradigm.
4 2 24
23. Let A1,A2,A3, and A4 be four matrices of dimensions 10 × 5,
5 × 20, 20 × 10, and 10 ×5, respectively. The minimum The task is to pick a subset of these items such that
number of scalar multiplications required to find the their total weight is no more than 11 kgs and their
product A1 A2 A3 A4 using the basic matrix multiplica- total value is maximized. Moreover, no item may be
tion method is _____ . [2016] split. The total value of items picked by an optimal
24. Consider the following table: algorithm is denoted by Vopt. A greedy algorithm sorts
Algorithms Design Paradigms the items by their value-to-weight ratios in descend-
ing order and packs them greedily, starting from the
(P) Kruskal (i) Divide and Conquer
first item in the ordered list. The total value of items
(Q) Quicksort (ii) Greedy
picked by the greedy algorithm is denoted by Vgreedy.
(R) Floyd-Warshall (iii) Dynamic Programming
The value of Vopt – Vgreedy is ______. [2018]
3.150 | Unit 3 • Algorithms

ANSWER KEYS

EXERCISES
Practice Problems 1
1. B 2. A 3. B 4. D 5. A 6. C 7. C 8. A 9. B 10. B
11. C 12. B 13. C 14. A 15. A

Practice Problems 2
1. B 2. C 3. B 4. D 5. B 6. C 7. A 8. B 9. D 10. C
11. A 12. B 13. B 14. B 15. A

Previous Years’ Questions

1. B 2. A 3. D 4. B 5. C 6. C 7. C 8. B 9. C 10. B
11. B 12. A 13. D 14. A 15. 34 16. 150 17. B 18. A 19. C 20. 80
21. C 22. C 23. 1500 24. C 25. C 26. 16