UNIT 3

Computing a Binomial Coefficient

 A binomial coefficient C(n, k) can be defined as the coefficient

of x^k in the expansion of (1 + x)^n.

 A binomial coefficient C(n, k) also gives the number of ways,

disregarding order, that k objects can be chosen from among n
objects more formally, the number of k-element subsets (or k-
combinations) of a n-element set.

 The binomial coefficient can be found with Pascal's triangle or the

binomial coefficient formula. The formula involves the use of
factorials: (n!)/(k!( n-k)!), where k = number of items selected and
n = total items chosen from.

Input: n = 4 and k = 2
Output: 6
Explanation: 4 C 2 is 4!/(2!*2!) = 6
Input: n = 5 and k = 2
Output: 10
Explanation: 5 C 2 is 5!/(3!*2!) = 10
We have discussed O(n*k) time and O(k) extra space algorithm
in this post. The value of C(n, k) can be calculated in O(k) time and
O(1) extra space.
Approach:
Share
1. change r to n-r if r is greater than n-r. and create a variable to store
the answer.
2. Run a loop from 0 to r-1
3. In every iteration update ans as (ans*(n-i))/(i+1) where i is the loop
counter.
4. So the answer will be equal to ((n/1)*((n-1)/2)*…*((n-r+1)/r)
which is equal to nCr.
C(n, k)
= n! / (n-k)! * k!
= [n * (n-1) *....* 1] / [ ( (n-k) * (n-k-1) * .... * 1) *
( k * (k-1) * .... * 1 ) ]
After simplifying, we get
C(n, k)
= [n * (n-1) * .... * (n-k+1)] / [k * (k-1) * .... * 1]

Also, C(n, k) = C(n, n-k)

// r can be changed to n-r if r > n-r

// Program to calculate C(n, k)

#include <bits/stdc++.h>
using namespace std;
// Returns value of Binomial Coefficient C(n, k)
int binomialCoeff(int n, int k)
{
int res = 1;

// Since C(n, k) = C(n, n-k)

if (k > n - k)
k = n - k;
 // Calculate value of
// [n * (n-1) *---* (n-k+1)] / [k * (k-1) *----* 1]
for (int i = 0; i < k; ++i) {
res *= (n - i);
res /= (i + 1);
}

return res;
}
// Driver Code
int main()
{ int n = 8, k = 2;
cout << "Value of C(" << n << ", " << k << ") is "
<< binomialCoeff(n, k);
return 0; }
Output
Value of C(8, 2) is 28
Complexity Analysis:
Time Complexity: O(r) A loop has to be run from 0 to r. So, the time
complexity is O(r).
Auxiliary Space: O(1) As no extra space is required.
Dynamic Programming Binomial Coefficients:

Dynamic Programming was invented by Richard Bellman, 1950. It is a

very general technique for solving optimization problems. Using
Dynamic Programming requires that the problem can be divided into
overlapping similar sub-problems. A recursive relation between the
larger and smaller sub problems is used to fill out a table. The algorithm
remembers solutions of the sub-problems and so does not have to
recalculate the solutions.
Also optimal problem must have the principle of optimality (phase
coined by Bellman) meaning that the optimal problem can be posed as
the optimal solution among sub problems. This is not true for all optimal
problems.
Dynamic Programming requires:
1. Problem divided into overlapping sub-problems
2. Sub-problem can be represented by a table
2. Principle of optimality, recursive relation between smaller and larger
problems
Compared to a brute force recursive algorithm that could run
exponential, the dynamic programming algorithm runs typically in
quadratic time. (Recall the algorithms for the Fibonacci numbers.) The
recursive algorithm ran in exponential time while the iterative algorithm
ran in linear time. The space cost does increase, which is typically the
size of the table. Frequently, the whole table does not have to store.

Computing a Binomial Coefficient

Computing binomial coefficients is non optimization problem but can be
solved using dynamic programming.

Binomial coefficients are represented by C(n, k) or (nk) and can be used

to represent the coefficients of a binomail:
(a + b)n = C(n, 0)an + ... + C(n, k)an-kbk + ... + C(n, n)bn

The recursive relation is defined by the prior power

C(n, k) = C(n-1, k-1) + C(n-1, k) for n > k > 0
IC C(n, 0) = C(n, n) = 1

Dynamic algorithm constructs a nxk table, with the first column and
diagonal filled out using the IC.
Construct the table:
k
0 1 2 ... k-1 k

0 1
1 1 1
2 1 2 1
n .
.
.
 k 1 1
.
.
.
n-1 1 C(n-
1, k-1)
n 1 C(n, k)

The table is then filled out iteratively, row by row using the recursive
relation.
Algorithm Binomial(n, k)
for i ← 0 to n do // fill out the table row wise
for i = 0 to min(i, k) do
if j==0 or j==i then C[i, j] ← 1 // IC
else C[i, j] ← C[i-1, j-1] + C[i-1, j] // recursive relation
return C[n, k]
the cost of the algorithm is filing out the table. Addition is the basic
operation. Because k ≤ n, the sum needs to be split into two parts
because only the half the table needs to be filled out for i < k and
remaining part of the table is filled out across the entire row.
 A(n, k) = sum for upper triangle + sum for the lower rectangle
= ∑i=1k ∑j=1i-1 1 + ∑i=1n ∑j=1k 1
= ∑i=1k (i-1) + ∑i=1n k
= (k-1)k/2 + k(n-k) ε Θ(nk)

Floyd Warshall Algorithm


The Floyd-Warshall algorithm, named after its creators Robert Floyd
and Stephen Warshall, is a fundamental algorithm in computer science
and graph theory. It is used to find the shortest paths between all pairs of
nodes in a weighted graph. This algorithm is highly efficient and can
handle graphs with both positive and negative edge weights, making it
a versatile tool for solving a wide range of network and connectivity
problems.
Floyd Warshall Algorithm:
The Floyd Warshall Algorithm is an all pair shortest path algorithm
unlike Dijkstra and Bellman Ford which are single source shortest path
algorithms. This algorithm works for both the directed and undirected
weighted graphs. But, it does not work for the graphs with negative
cycles (where the sum of the edges in a cycle is negative). It
follows Dynamic Programming approach to check every possible path
going via every possible node in order to calculate shortest distance
between every pair of nodes.

Idea Behind Floyd Warshall Algortihm:

Suppose we have a graph G[][] with V vertices from 1 to N. Now we
have to evaluate a shortestPathMatrix[][] where
shortestPathMatrix[i][j] represents the shortest path between
vertices i and j.
Obviously the shortest path between i to j will have some k number of
intermediate nodes. The idea behind floyd warshall algorithm is to treat
each and every vertex from 1 to N as an intermediate node one by one.
The following figure shows the above optimal substructure property in
floyd warshall algorithm:

Floyd Warshall Algorithm Algorithm:

 Initialize the solution matrix same as the input graph matrix as a first
step.
 Then update the solution matrix by considering all vertices as an
intermediate vertex.
 The idea is to pick all vertices one by one and updates all shortest
paths which include the picked vertex as an intermediate vertex in the
shortest path.
 When we pick vertex number k as an intermediate vertex, we already
have considered vertices {0, 1, 2, .. k-1} as intermediate vertices.
 For every pair (i, j) of the source and destination vertices
respectively, there are two possible cases.
 k is not an intermediate vertex in shortest path from i to j.
We keep the value of dist[i][j] as it is.
 k is an intermediate vertex in shortest path from i to j. We
update the value of dist[i][j] as dist[i][k] +
dist[k][j], if dist[i][j] > dist[i][k] + dist[k][j]
Pseudo-Code of Floyd Warshall Algorithm :
For k = 0 to n – 1
For i = 0 to n – 1
For j = 0 to n – 1
Distance[i, j] = min(Distance[i, j], Distance[i, k] + Distance[k, j])
where i = source Node, j = Destination Node, k = Intermediate Node
Illustration of Floyd Warshall Algorithm :
Suppose we have a graph as shown in the image:

Step 1: Initialize the Distance[][] matrix using the input graph such that
Distance[i][j]= weight of edge from i to j, also Distance[i][j] = Infinity if
there is no edge from i to j.
Step 2: Treat node A as an intermediate node and calculate the
Distance[][] for every {i,j} node pair using the formula:
= Distance[i][j] = minimum (Distance[i][j], (Distance from i to A) +
(Distance from A to j ))
= Distance[i][j] = minimum (Distance[i][j], Distance[i][A] +
Distance[A][j])
Step 3: Treat node B as an intermediate node and calculate the
Distance[][] for every {i,j} node pair using the formula:
= Distance[i][j] = minimum (Distance[i][j], (Distance from i to B) +
(Distance from B to j ))
= Distance[i][j] = minimum (Distance[i][j], Distance[i][B] +
Distance[B][j])
Step 4: Treat node C as an intermediate node and calculate the
Distance[][] for every {i,j} node pair using the formula:
= Distance[i][j] = minimum (Distance[i][j], (Distance from i to C) +
(Distance from C to j ))
= Distance[i][j] = minimum (Distance[i][j], Distance[i][C] +
Distance[C][j])
Step 5: Treat node D as an intermediate node and calculate the
Distance[][] for every {i,j} node pair using the formula:
= Distance[i][j] = minimum (Distance[i][j], (Distance from i to D) +
(Distance from D to j ))
= Distance[i][j] = minimum (Distance[i][j], Distance[i][D] +
Distance[D][j])
Step 6: Treat node E as an intermediate node and calculate the
Distance[][] for every {i,j} node pair using the formula:
= Distance[i][j] = minimum (Distance[i][j], (Distance from i to E) +
(Distance from E to j ))
= Distance[i][j] = minimum (Distance[i][j], Distance[i][E] +
Distance[E][j])
Step 7: Since all the nodes have been treated as an intermediate node,
we can now return the updated Distance[][] matrix as our answer matrix.
// C++ Program for Floyd Warshall Algorithm
#include <bits/stdc++.h>
using namespace std;

// Number of vertices in the graph

#define V 4

/* Define Infinite as a large enough

value.This value will be used for
vertices not connected to each other */
#define INF 99999

// A function to print the solution matrix

void printSolution(int dist[][V]);

// Solves the all-pairs shortest path

// problem using Floyd Warshall algorithm
void floydWarshall(int dist[][V])
{

int i, j, k;
/* Add all vertices one by one to
the set of intermediate vertices.
---> Before start of an iteration,
we have shortest distances between all
pairs of vertices such that the
shortest distances consider only the
vertices in set {0, 1, 2, .. k-1} as
intermediate vertices.
----> After the end of an iteration,
vertex no. k is added to the set of
intermediate vertices and the set becomes {0, 1, 2, ..
k} */
for (k = 0; k < V; k++) {
// Pick all vertices as source one by one
for (i = 0; i < V; i++) {
// Pick all vertices as destination for the
// above picked source
for (j = 0; j < V; j++) {
// If vertex k is on the shortest path from
// i to j, then update the value of
 // dist[i][j]
if (dist[i][j] > (dist[i][k] + dist[k][j])
&& (dist[k][j] != INF
&& dist[i][k] != INF))
dist[i][j] = dist[i][k] + dist[k][j];
}
}
}

// Print the shortest distance matrix

printSolution(dist);
}

/* A utility function to print solution */

void printSolution(int dist[][V])
{
cout << "The following matrix shows the shortest "
"distances"
" between every pair of vertices \n";
for (int i = 0; i < V; i++) {
for (int j = 0; j < V; j++) {
 if (dist[i][j] == INF)
cout << "INF"
<< " ";
else
cout << dist[i][j] << " ";
}
cout << endl;
}
}

// Driver's code
int main()
{
/* Let us create the following weighted graph
10
(0)------->(3)
| /|\
5| |
| |1
\|/ |
(1)------->(2)
 3 */
int graph[V][V] = { { 0, 5, INF, 10 },
{ INF, 0, 3, INF },
{ INF, INF, 0, 1 },
{ INF, INF, INF, 0 } };

// Function call
floydWarshall(graph);
return 0;
}
Output
The following matrix shows the shortest distances between every pair
of vertices
0 5 8 9
INF 0 3 4
INF INF 0 1
INF INF INF 0
Complexity Analysis of Floyd Warshall Algorithm:
 Time Complexity: O(V3), where V is the number of vertices in the
graph and we run three nested loops each of size V
 Auxiliary Space: O(V2), to create a 2-D matrix in order to store the
shortest distance for each pair of nodes.
 Optimal Binary Search Tree

An Optimal Binary Search Tree (OBST), also known as a Weighted
Binary Search Tree, is a binary search tree that minimizes the expected
search cost. In a binary search tree, the search cost is the number of
comparisons required to search for a given key.
In an OBST, each node is assigned a weight that represents the
probability of the key being searched for. The sum of all the weights in
the tree is 1.0. The expected search cost of a node is the sum of the
product of its depth and weight, and the expected search cost of its
children.
To construct an OBST, we start with a sorted list of keys and their
probabilities. We then build a table that contains the expected search
cost for all possible sub-trees of the original list. We can use dynamic
programming to fill in this table efficiently. Finally, we use this table to
construct the OBST.
The time complexity of constructing an OBST is O(n^3), where n is the
number of keys. However, with some optimizations, we can reduce the
time complexity to O(n^2). Once the OBST is constructed, the time
complexity of searching for a key is O(log n), the same as for a regular
binary search tree.
The OBST is a useful data structure in applications where the keys
have different probabilities of being searched for. It can be used to
improve the efficiency of searching and retrieval operations in
databases, compilers, and other computer programs.
Given a sorted array key [0.. n-1] of search keys and an array freq[0.. n-
1] of frequency counts, where freq[i] is the number of searches
for keys[i]. Construct a binary search tree of all keys such that the total
cost of all the searches is as small as possible.
Let us first define the cost of a BST. The cost of a BST node is the level
of that node multiplied by its frequency. The level of the root is 1.
Examples:
Input: keys[] = {10, 12}, freq[] = {34, 50}
There can be following two possible BSTs
10 12
\ /
12 10
I II
Frequency of searches of 10 and 12 are 34 and 50 respectively.
The cost of tree I is 34*1 + 50*2 = 134
The cost of tree II is 50*1 + 34*2 = 118

Input: keys[] = {10, 12, 20}, freq[] = {34, 8, 50}

There can be following possible BSTs
10 12 20 10 20
\ / \ / \ /
12 10 20 12 20 10
\ / / \
20 10 12 12
I II III IV V
Among all possible BSTs, cost of the fifth BST is minimum.
Cost of the fifth BST is 1*50 + 2*34 + 3*8 = 142
1) Optimal Substructure:
The optimal cost for freq[i..j] can be recursively calculated using the
following formula.

We need to calculate optCost(0, n-1) to find the result.

The idea of above formula is simple, we one by one try all nodes as root
(r varies from i to j in second term). When we make rth node as root, we
recursively calculate optimal cost from i to r-1 and r+1 to j.
We add sum of frequencies from i to j (see first term in the above
formula)
The reason for adding the sum of frequencies from i to j:
This can be divided into 2 parts one is the freq[r]+sum of frequencies of
all elements from i to j except r. The term freq[r] is added because it is
going to be root and that means level of 1, so freq[r]*1=freq[r]. Now the
actual part comes, we are adding the frequencies of remaining elements
because as we take r as root then all the elements other than that are
going 1 level down than that is calculated in the subproblem. Let me put
it in a more clear way, for calculating optcost(i,j) we assume that the r is
taken as root and calculate min of opt(i,r-1)+opt(r+1,j) for all i<=r<=j.
Here for every subproblem we are choosing one node as a root. But in
reality the level of subproblem root and all its descendant nodes will be
1 greater than the level of the parent problem root. Therefore the
frequency of all the nodes except r should be added which accounts to
the descend in their level compared to level assumed in subproblem.
2) Overlapping Subproblems
Following is recursive implementation that simply follows the recursive
structure mentioned above.

// A naive recursive implementation of optimal binary

// search tree problem
public class GFG
{
// A recursive function to calculate cost of
// optimal binary search tree
static int optCost(int freq[], int i, int j)
{
// Base cases
if (j < i) // no elements in this subarray
return 0;
if (j == i) // one element in this subarray
return freq[i];

// Get sum of freq[i], freq[i+1], ... freq[j]

int fsum = sum(freq, i, j);

// Initialize minimum value

int min = Integer.MAX_VALUE;

// One by one consider all elements as root and

// recursively find cost of the BST, compare the
// cost with min and update min if needed
for (int r = i; r <= j; ++r)
{
int cost = optCost(freq, i, r-1) +
optCost(freq, r+1, j);
if (cost < min)
min = cost;
}

// Return minimum value

return min + fsum;
}

// The main function that calculates minimum cost of

// a Binary Search Tree. It mainly uses optCost() to
// find the optimal cost.
static int optimalSearchTree(int keys[], int freq[], int n)
{
// Here array keys[] is assumed to be sorted in
// increasing order. If keys[] is not sorted, then
// add code to sort keys, and rearrange freq[]
// accordingly.
return optCost(freq, 0, n-1);
}

// A utility function to get sum of array elements

// freq[i] to freq[j]
static int sum(int freq[], int i, int j)
{
int s = 0;
for (int k = i; k <=j; k++)
 s += freq[k];
return s;
}

// Driver code
public static void main(String[] args) {
int keys[] = {10, 12, 20};
int freq[] = {34, 8, 50};
int n = keys.length;
System.out.println("Cost of Optimal BST is " +
optimalSearchTree(keys, freq, n));
}
}
Output
Cost of Optimal BST is 142

Time complexity of the above naive recursive approach is exponential.

It should be noted that the above function computes the same
subproblems again and again. We can see many subproblems being
repeated in the following recursion tree for freq[1..4].
Since same subproblems are called again, this problem has
Overlapping Subproblems property. So optimal BST problem has both
properties (see this and this) of a dynamic programming problem. Like
other typical Dynamic Programming(DP) problems, recomputations of
same subproblems can be avoided by constructing a temporary array
cost[][] in bottom up manner.
Dynamic Programming Solution
Following is C/C++ implementation for optimal BST problem using
Dynamic Programming. We use an auxiliary array cost[n][n] to store
the solutions of subproblems. cost[0][n-1] will hold the final result.
The challenge in implementation is, all diagonal values must be filled
first, then the values which lie on the line just above the diagonal. In
other words, we must first fill all cost[i][i] values, then all cost[i][i+1]
values, then all cost[i][i+2] values. So how to fill the 2D array in such
manner> The idea used in the implementation is same as Matrix Chain
Multiplication problem,
// Dynamic Programming Java code for Optimal Binary Search
// Tree Problem
public class Optimal_BST2 {
/* A Dynamic Programming based function that calculates
minimum cost of a Binary Search Tree. */
static int optimalSearchTree(int keys[], int freq[], int n) {

/* Create an auxiliary 2D matrix to store results of

subproblems */
int cost[][] = new int[n + 1][n + 1];

/* cost[i][j] = Optimal cost of binary search tree that

can be formed from keys[i] to keys[j]. cost[0][n-1]
will store the resultant cost */

// For a single key, cost is equal to frequency of the key

for (int i = 0; i < n; i++)
cost[i][i] = freq[i];

// Now we need to consider chains of length 2, 3, ... .

// L is chain length.
for (int L = 2; L <= n; L++) {
 // i is row number in cost[][]
for (int i = 0; i <= n - L + 1; i++) {

// Get column number j from row number i and

// chain length L
int j = i + L - 1;
cost[i][j] = Integer.MAX_VALUE;
int off_set_sum = sum(freq, i, j);

// Try making all keys in interval keys[i..j] as

root
for (int r = i; r <= j; r++) {

// c = cost when keys[r] becomes root of

this subtree
int c = ((r > i) ? cost[i][r - 1] : 0)
+ ((r < j) ? cost[r + 1][j] : 0) +
off_set_sum;
if (c < cost[i][j])
cost[i][j] = c;
}
}
 }
return cost[0][n - 1];
}

// A utility function to get sum of array elements

// freq[i] to freq[j]
static int sum(int freq[], int i, int j) {
int s = 0;
for (int k = i; k <= j; k++) {
if (k >= freq.length)
continue;
s += freq[k];
}
return s;
}

public static void main(String[] args) {

int keys[] = { 10, 12, 20 };

int freq[] = { 34, 8, 50 };
int n = keys.length;
 System.out.println("Cost of Optimal BST is "
+ optimalSearchTree(keys, freq, n));
}}
Output
Cost of Optimal BST is 142

The Knapsack Problem and Memory Functions

To design a dynamic programming algorithm, we need to derive a
recurrence relation that expresses a solution to an instance of the
knapsack problem in terms of solutions to its smaller subinstances. Let
us consider an instance defined by the first i items, 1 ≤ i ≤ n, with
weights w1, . . . , wi, values v1, . . . , vi, and knapsack
capacity j, 1 ≤ j ≤ W. Let F (i, j ) be the value of an optimal solution to
this instance, i.e., the value of the most valuable subset of the
first i items that fit into the knapsack of capacity j. We can divide all the
subsets of the first i items that fit the knapsack of capacity j into two
categories: those that do not include the ith item and those that do. Note
the following:

Among the subsets that do not include the ith item, the value of
an optimal subset is, by definition, F (i − 1, j ).
Among the subsets that do include the ith item (hence, j − wi ≥ 0), an
optimal subset is made up of this item and an optimal subset of the
first i − 1 items that fits into the knapsack of capacity j − wi. The value
of such an optimal subset is vi + F (i − 1, j − wi).

Thus, the value of an optimal solution among all feasible subsets of the
first i items is the maximum of these two values. Of course, if the ith
item does not fit into the knapsack, the value of an optimal subset
selected from the first i items is the same as the value of an optimal
subset selected from the first i − 1 items. These observations lead to the
following recurrence:

Our goal is to find F (n, W ), the maximal value of a subset of

the n given items that fit into the knapsack of capacity W, and an
optimal subset itself.

Figure 8.4 illustrates the values involved in equations (8.6) and (8.7).
For i, j > 0, to compute the entry in the ith row and the j th column, F (i,
j ), we compute the maximum of the entry in the previous row and the
same column and the sum of vi and the entry in the previous row
and wi columns to the left. The table can be filled either row by row or
column by column.
The dynamic programming table, filled by applying formulas (8.6) and
(8.7), is shown in Figure 8.5.

Thus, the maximal value is F (4, 5) = $37. We can find the composition
of an optimal subset by backtracing the computations of this entry in the
table. Since F (4, 5) > F (3, 5), item 4 has to be included in an optimal
solution along with an optimal subset for filling 5 − 2 = 3 remaining
units of the knapsack capacity. The value of the latter is F (3, 3). Since F
(3, 3) = F (2, 3), item 3 need not be in an optimal subset. Since F (2, 3) >
F (1, 3), item 2 is a part of an optimal selection, which leaves element F
(1, 3 − 1) to specify its remaining composition. Similarly, since F (1, 2)
> F (0, 2), item 1 is the final part of the optimal solution {item 1, item 2,
item 4}.
The time efficiency and space efficiency of this algorithm are both
in (nW ). The time needed to find the composition of an optimal
solution is in O(n). You are asked to prove these assertions in the
exercises.

Memory Functions

As we discussed at the beginning of this chapter and illustrated in

subsequent sections, dynamic programming deals with problems whose
solutions satisfy a recurrence relation with overlapping subproblems.
The direct top-down approach to finding a solution to such a recurrence
leads to an algorithm that solves common subproblems more than once
and hence is very inefficient (typically, exponential
or worse). The classic dynamic programming approach, on the other
hand, works bottom up: it fills a table with solutions to all smaller
subproblems, but each of them is solved only once. An unsatisfying
aspect of this approach is that solutions to some of these smaller
subproblems are often not necessary for getting a solution to the problem
given. Since this drawback is not present in the top-down approach, it is
natural to try to combine the strengths of the top-down and bottom-up
approaches. The goal is to get a method that solves only subproblems
that are necessary and does so only once. Such a method exists; it is
based on using memory functions.

This method solves a given problem in the top-down manner but, in

addition, maintains a table of the kind that would have been used by a
bottom-up dynamic programming algorithm. Initially, all the table’s
entries are initialized with a spe-cial “null” symbol to indicate that they
have not yet been calculated. Thereafter, whenever a new value needs to
be calculated, the method checks the correspond-ing entry in the table
first: if this entry is not “null,” it is simply retrieved from the table;
otherwise, it is computed by the recursive call whose result is then
recorded in the table.

The following algorithm implements this idea for the knapsack problem.
After initializing the table, the recursive function needs to be called
with i = n (the number of items) and j = W (the knapsack capacity).

ALGORITHM MFKnapsack(i, j )

//Implements the memory function method for the knapsack problem

//Input: A nonnegative integer i indicating the number of the first

items being considered and a nonnegative integer j indicating

the knapsack capacity

//Output: The value of an optimal feasible subset of the first i items

//Note: Uses as global variables input arrays W eight s[1..n], V
alues[1..n], //and table F [0..n, 0..W ] whose entries are initialized
with −1’s except for //row 0 and column 0 initialized with 0’s

if F [i, j ] < 0

if j < Weights[i]

value ← MFKnapsack(i − 1, j )
else

value ← max(MFKnapsack(i − 1, j ),

Values[i] + MFKnapsack(i − 1, j − Weights[i]))

F [i, j ] ← value return F [i, j ]

EXAMPLE 2 Let us apply the memory function method to the instance

consid-ered in Example 1. The table in Figure 8.6 gives the results. Only
11 out of 20 nontrivial values (i.e., not those in row 0 or in column 0)
have been computed

Just one nontrivial entry, V (1, 2), is retrieved rather than being
recomputed. For larger instances, the proportion of such entries can be
significantly larger.
In general, we cannot expect more than a constant-factor gain in using
the memory function method for the knapsack problem, because its time
efficiency class is the same as that of the bottom-up algorithm (why?). A
more significant improvement can be expected for dynamic
programming algorithms in which a computation of one value takes
more than constant time. You should also keep in mind that a memory
function algorithm may be less space-efficient than a space-efficient
version of a bottom-up algorithm.
Exercises 8.2
a. Apply the bottom-up dynamic programming algorithm to the
following instance of the knapsack problem:

How many different optimal subsets does the instance of part (a)
have?

In general, how can we use the table generated by the dynamic program-
ming algorithm to tell whether there is more than one optimal subset for
the knapsack problem’s instance?
a. Write pseudocode of the bottom-up dynamic programming
algorithm for the knapsack problem.

Write pseudocode of the algorithm that finds

the composition of an optimal subset from the table generated by the
bottom-up dynamic programming algorithm for the knapsack problem.

For the bottom-up dynamic programming algorithm for the

knapsack prob-lem, prove that
 its time efficiency is (nW ).

its space efficiency is (nW ).

the time needed to find the composition of

an optimal subset from a filled dynamic programming table is O(n).

a. True or false: A sequence of values in a row of the dynamic

programming table for the knapsack problem is always nondecreasing?

True or false: A sequence of values in a

column of the dynamic program-ming table for the knapsack problem is
always nondecreasing?

Design a dynamic programming algorithm for the version of the

knapsack problem in which there are unlimited quantities of copies for
each of the n item kinds given. Indicate the time efficiency of the
algorithm.

Apply the memory function method to the instance of the

knapsack problem given in Problem 1. Indicate the entries of the
dynamic programming table that are (i) never computed by the memory
function method, (ii) retrieved without a recomputation.
 Prove that the efficiency class of the memory function algorithm
for the knap-sack problem is the same as that of the bottom-up algorithm
(see Problem 3).

Explain why the memory function approach is unattractive for

the problem of computing a binomial coefficient by the formula C(n,
k) = C(n − 1, k − 1) +
C(n − 1, k).

Write a research report on one of the following well-known

applications of dynamic programming:

finding the longest common subsequence in

two sequences

optimal string editing

minimal triangulation of a polygon

Greedy Algorithm

The greedy method is one of the strategies like Divide and conquer used
to solve the problems. This method is used for solving optimization
problems. An optimization problem is a problem that demands either
maximum or minimum results. Let's understand through some terms.

The Greedy method is the simplest and straightforward approach. It is

not an algorithm, but it is a technique. The main function of this
approach is that the decision is taken on the basis of the currently
available information. Whatever the current information is present, the
decision is made without worrying about the effect of the current
decision in future.

This technique is basically used to determine the feasible solution that

may or may not be optimal. The feasible solution is a subset that
satisfies the given criteria. The optimal solution is the solution which is
the best and the most favorable solution in the subset. In the case of
feasible, if more than one solution satisfies the given criteria then those
solutions will be considered as the feasible, whereas the optimal solution
is the best solution among all the solutions.

Characteristics of Greedy method

The following are the characteristics of a greedy method:

o To construct the solution in an optimal way, this algorithm creates

two sets where one set contains all the chosen items, and another
set contains the rejected items.
o A Greedy algorithm makes good local choices in the hope that the
solution should be either feasible or optimal.

Components of Greedy Algorithm

The components that can be used in the greedy algorithm are:

o Candidate set: A solution that is created from the set is known as

a candidate set.
o Selection function: This function is used to choose the candidate
or subset which can be added in the solution.
o Feasibility function: A function that is used to determine whether
the candidate or subset can be used to contribute to the solution or
not.
 o Objective function: A function is used to assign the value to the
solution or the partial solution.
o Solution function: This function is used to intimate whether the
complete function has been reached or not.

Applications of Greedy Algorithm

o It is used in finding the shortest path.
o It is used to find the minimum spanning tree using the prim's
algorithm or the Kruskal's algorithm.
o It is used in a job sequencing with a deadline.
o This algorithm is also used to solve the fractional knapsack
problem.

Pseudo code of Greedy Algorithm

1. Algorithm Greedy (a, n)
2. {
3. Solution : = 0;
4. for i = 0 to n do
5. {
6. x: = select(a);
7. if feasible(solution, x)
8. {
9. Solution: = union(solution , x)
10. }
11. return solution;
12. }}

The above is the greedy algorithm. Initially, the solution is assigned with
zero value. We pass the array and number of elements in the greedy
algorithm. Inside the for loop, we select the element one by one and
checks whether the solution is feasible or not. If the solution is feasible,
then we perform the union.

Let's understand through an example.

Suppose there is a problem 'P'. I want to travel from A to B shown as

below:

P:A→B

The problem is that we have to travel this journey from A to B. There

are various solutions to go from A to B. We can go from A to B
by walk, car, bike, train, aeroplane, etc. There is a constraint in the
journey that we have to travel this journey within 12 hrs. If I go by train
or aeroplane then only, I can cover this distance within 12 hrs. There are
many solutions to this problem but there are only two solutions that
satisfy the constraint.

If we say that we have to cover the journey at the minimum cost. This
means that we have to travel this distance as minimum as possible, so
this problem is known as a minimization problem. Till now, we have
two feasible solutions, i.e., one by train and another one by air. Since
travelling by train will lead to the minimum cost so it is an optimal
solution. An optimal solution is also the feasible solution, but providing
the best result so that solution is the optimal solution with the minimum
cost. There would be only one optimal solution.

The problem that requires either minimum or maximum result then that
problem is known as an optimization problem. Greedy method is one of
the strategies used for solving the optimization problems.

Disadvantages of using Greedy algorithm

Greedy algorithm makes decisions based on the information available at

each phase without considering the broader problem. So, there might be
a possibility that the greedy solution does not give the best solution for
every problem.
It follows the local optimum choice at each stage with a intend of
finding the global optimum. Let's understand through an example.

Consider the graph which is given below:

We have to travel from the source to the destination at the minimum

cost. Since we have three feasible solutions having cost paths as 10, 20,
and 5. 5 is the minimum cost path so it is the optimal solution. This is
the local optimum, and in this way, we find the local optimum at each
stage in order to calculate the global optimal solution.

Characteristics of Greedy algorithm

For a problem to be solved using the Greedy approach, it must follow a
few major characteristics:
 There is an ordered list of resources(profit, cost, value, etc.)
 Maximum of all the resources(max profit, max value, etc.) are
taken.
 For example, in the fractional knapsack problem, the maximum
value/weight is taken first according to available capacity.
Introduction to Greedy Algorithm

Use of the greedy algorithm:-

The greedy algorithm is a method used in optimization problems where
the goal is to make the locally optimal choice at each stage with the
hope of finding a global optimum. It is called “greedy” because it tries
to find the best solution by making the best choice at each step,
without considering future steps or the consequences of the current
decision.
Some common use cases for the greedy algorithm include:

 Scheduling and Resource Allocation: The greedy algorithm can

be used to schedule jobs or allocate resources in an efficient
manner.
 Minimum Spanning Trees: The greedy algorithm can be used to
find the minimum spanning tree of a graph, which is the subgraph
that connects all vertices with the minimum total edge weight.
 Coin Change Problem: The greedy algorithm can be used to make
change for a given amount with the minimum number of coins, by
always choosing the coin with the highest value that is less than the
remaining amount to be changed.
 Huffman Coding: The greedy algorithm can be used to generate a
prefix-free code for data compression, by constructing a binary tree
in a way that the frequency of each character is taken into
consideration.
It’s important to note that not all optimization problems can be solved
by a greedy algorithm, and there are cases where the greedy approach
can lead to suboptimal solutions. However, in many cases, the greedy
algorithm provides a good approximation to the optimal solution and is
a useful tool for solving optimization problems quickly and efficiently.
All greedy algorithms follow a basic structure:
 Declare an empty result = 0.
 We make a greedy choice to select, If the choice is feasible add it to
the final result.
 return the result.
Why choose Greedy Approach?
The greedy approach has a few tradeoffs, which may make it suitable
for optimization. One prominent reason is to achieve the most feasible
solution immediately. In the activity selection problem (Explained
below), if more activities can be done before finishing the current
activity, these activities can be performed within the same time.
Another reason is to divide a problem recursively based on a
condition, with no need to combine all the solutions. In the activity
selection problem, the “recursive division” step is achieved by
scanning a list of items only once and considering certain activities.
Greedy Algorithm Example:
Some Famous problems that exhibit Optimal substructure
property and can be solved using Greedy approach are –
1) Job sequencing Problem:
Greedily choose the jobs with maximum profit first, by sorting the jobs
in decreasing order of their profit. This would help to maximize the
total profit as choosing the job with maximum profit for every time
slot will eventually maximize the total profit
2) Prim’s algorithm to find Minimum Spanning Tree:
It starts with an empty spanning tree. The idea is to maintain two sets
of vertices. The first set contains the vertices already included in the
MST, the other set contains the vertices not yet included. At every
step, it considers all the edges that connect the two sets and picks the
minimum weight edge from these edges. After picking the edge, it
moves the other endpoint of the edge to the set containing MST.
How does the Greedy Algorithm works?
When the choice to apply the greedy method is made without
conducting a thorough examination, the decision to utilize it can be
somewhat difficult and occasionally even result in failure. In some
cases taking the local best choice may lead to losing the global optimal
solution.
For example:
 One such example where the Greedy Approach fails is to find the
Maximum weighted path of nodes in the given graph.

Graph with weighted vertices

 In the above graph starting from the root node 10 if we greedily

select the next node to obtain the most weighted path the next
selected node will be 5 that will take the total sum to 15 and the
path will end as there is no child of 5 but the path 10 -> 5 is not the
maximum weight path.
 Greedy Approach fails

 In order to find the most weighted path all possible path sum must
be computed and their path sum must be compared to get the
desired result, it is visible that the most weighted path in the above
graph is 10 -> 1 -> 30 that gives the path sum 41.
 Correct Approach

 In such cases Greedy approach wouldn’t work instead complete

paths from root to leaf node has to be considered to get the correct
answer i.e. the most weighted path, This can be achieved by
recursively checking all the paths and calculating their weight.
Thus to use Greedy algorithm the problem must not
contain overlapping subproblems.

Prim's Algorithm

In this article, we will discuss the prim's algorithm. Along with the
algorithm, we will also see the complexity, working, example, and
implementation of prim's algorithm.
Before starting the main topic, we should discuss the basic and
important terms such as spanning tree and minimum spanning tree.

Spanning tree - A spanning tree is the subgraph of an undirected

connected graph.

Minimum Spanning tree - Minimum spanning tree can be defined as

the spanning tree in which the sum of the weights of the edge is
minimum. The weight of the spanning tree is the sum of the weights
given to the edges of the spanning tree.

ADVERTISEMENT

Now, let's start the main topic.

Prim's Algorithm is a greedy algorithm that is used to find the

minimum spanning tree from a graph. Prim's algorithm finds the subset
of edges that includes every vertex of the graph such that the sum of the
weights of the edges can be minimized.

Prim's algorithm starts with the single node and explores all the adjacent
nodes with all the connecting edges at every step. The edges with the
minimal weights causing no cycles in the graph got selected.

How does the prim's algorithm work?

Prim's algorithm is a greedy algorithm that starts from one vertex and
continue to add the edges with the smallest weight until the goal is
reached. The steps to implement the prim's algorithm are given as
follows -

o First, we have to initialize an MST with the randomly chosen

vertex.
o Now, we have to find all the edges that connect the tree in the
above step with the new vertices. From the edges found, select the
minimum edge and add it to the tree.
 o Repeat step 2 until the minimum spanning tree is formed.

The applications of prim's algorithm are -

o Prim's algorithm can be used in network designing.

o It can be used to make network cycles.
o It can also be used to lay down electrical wiring cables.

Example of prim's algorithm

Now, let's see the working of prim's algorithm using an example. It will
be easier to understand the prim's algorithm using an example.

ADVERTISEMENT

Suppose, a weighted graph is -

Step 1 - First, we have to choose a vertex from the above graph. Let's
choose B.
Step 2 - Now, we have to choose and add the shortest edge from vertex
B. There are two edges from vertex B that are B to C with weight 10 and
edge B to D with weight 4. Among the edges, the edge BD has the
minimum weight. So, add it to the MST.

Step 3 - Now, again, choose the edge with the minimum weight among
all the other edges. In this case, the edges DE and CD are such edges.
Add them to MST and explore the adjacent of C, i.e., E and A. So, select
the edge DE and add it to the MST.
Step 4 - Now, select the edge CD, and add it to the MST.

Step 5 - Now, choose the edge CA. Here, we cannot select the edge CE
as it would create a cycle to the graph. So, choose the edge CA and add
it to the MST.
 So, the graph produced in step 5 is the minimum spanning tree of the
given graph. The cost of the MST is given below -

Cost of MST = 4 + 2 + 1 + 3 = 10 units.

Algorithm
1. Step 1: Select a starting vertex
2. Step 2: Repeat Steps 3 and 4 until there are fringe vertices
3.

Step 3: Select an edge 'e' connecting the tree vertex and fringe vertex tha
t has minimum weight
4.

Step 4: Add the selected edge and the vertex to the minimum spanning tr
ee T
5. [END OF LOOP]
6. Step 5: EXIT
Complexity of Prim's algorithm

Now, let's see the time complexity of Prim's algorithm. The running time
of the prim's algorithm depends upon using the data structure for the
graph and the ordering of edges. Below table shows some choices -
 o Time Complexity

Data structure used for the minimum edge Time

weight Complexity

Adjacency matrix, linear searching O(|V|2)

Adjacency list and binary heap O(|E| log |V|)

Adjacency list and Fibonacci heap O(|E|+ |V| log |V|)

Prim's algorithm can be simply implemented by using the adjacency

matrix or adjacency list graph representation, and to add the edge with
the minimum weight requires the linearly searching of an array of
weights. It requires O(|V|2) running time. It can be improved further by
using the implementation of heap to find the minimum weight edges in
the inner loop of the algorithm.

The time complexity of the prim's algorithm is O(E logV) or O(V logV),
where E is the no. of edges, and V is the no. of vertices.

Implementation of Prim's algorithm

Now, let's see the implementation of prim's algorithm.

Program: Write a program to implement prim's algorithm in C

language.

1. #include <stdio.h>
2. #include <limits.h>
3. #define vertices 5 /*Define the number of vertices in the graph*/
4.

/* create minimum_key() method for finding the vertex that has minimu
m key-value and that is not added in MST yet */
5. int minimum_key(int k[], int mst[])
6. {
7. int minimum = INT_MAX, min,i;
8.
9. /*iterate over all vertices to find the vertex with minimum key-
value*/
10. for (i = 0; i < vertices; i++)
11. if (mst[i] == 0 && k[i] < minimum )
12. minimum = k[i], min = i;
13. return min;
14. }
15. /* create prim() method for constructing and printing the MST.
16.

The g[vertices][vertices] is an adjacency matrix that defines the graph fo

r MST.*/
17. void prim(int g[vertices][vertices])
18. {
19.

/* create array of size equal to total number of vertices for storing the
MST*/
20. int parent[vertices];
21.
 /* create k[vertices] array for selecting an edge having minimum weig
ht*/
22. int k[vertices];
23. int mst[vertices];
24. int i, count,edge,v; /*Here 'v' is the vertex*/
25. for (i = 0; i < vertices; i++)
26. {
27. k[i] = INT_MAX;
28. mst[i] = 0;
29. }
30. k[0] = 0; /*It select as first vertex*/
31. parent[0] = -1; /* set first value of parent[] array to -
1 to make it root of MST*/
32. for (count = 0; count < vertices-1; count++)
33. {
34.

/*select the vertex having minimum key and that is not added in the
MST yet from the set of vertices*/
35. edge = minimum_key(k, mst);
36. mst[edge] = 1;
37. for (v = 0; v < vertices; v++)
38. {
39. if (g[edge][v] && mst[v] == 0 && g[edge][v] < k[v])
40. {
41. parent[v] = edge, k[v] = g[edge][v];
42. }
43. }
44. }
45. /*Print the constructed Minimum spanning tree*/
46. printf("\n Edge \t Weight\n");
47. for (i = 1; i < vertices; i++)
48. printf(" %d <-> %d %d \n", parent[i], i, g[i][parent[i]]);
49.
50. }
51. int main()
52. {
53. int g[vertices][vertices] = {{0, 0, 3, 0, 0},
54. {0, 0, 10, 4, 0},
55. {3, 10, 0, 2, 6},
56. {0, 4, 2, 0, 1},
57. {0, 0, 6, 1, 0},
58. };
59. prim(g);
60. return 0;
61. }

Output

So, that's all about the article. Hope, the article will be helpful and
informative to you.

Kruskal's Algorithm

In this article, we will discuss Kruskal's algorithm. Here, we will also

see the complexity, working, example, and implementation of the
Kruskal's algorithm.
But before moving directly towards the algorithm, we should first
understand the basic terms such as spanning tree and minimum spanning
tree.

Spanning tree - A spanning tree is the subgraph of an undirected

connected graph.

Minimum Spanning tree - Minimum spanning tree can be defined as

the spanning tree in which the sum of the weights of the edge is
minimum. The weight of the spanning tree is the sum of the weights
given to the edges of the spanning tree.

Now, let's start with the main topic.

Kruskal's Algorithm is used to find the minimum spanning tree for a

connected weighted graph. The main target of the algorithm is to find
the subset of edges by using which we can traverse every vertex of the
graph. It follows the greedy approach that finds an optimum solution at
every stage instead of focusing on a global optimum.

How does Kruskal's algorithm work?

In Kruskal's algorithm, we start from edges with the lowest weight and
keep adding the edges until the goal is reached. The steps to implement
Kruskal's algorithm are listed as follows -

o First, sort all the edges from low weight to high.

o Now, take the edge with the lowest weight and add it to the
spanning tree. If the edge to be added creates a cycle, then reject
the edge.
o Continue to add the edges until we reach all vertices, and a
minimum spanning tree is created.

The applications of Kruskal's algorithm are -

 o Kruskal's algorithm can be used to layout electrical wiring among
cities.
o It can be used to lay down LAN connections.

Example of Kruskal's algorithm

Now, let's see the working of Kruskal's algorithm using an example. It

will be easier to understand Kruskal's algorithm using an example.

Suppose a weighted graph is -

The weight of the edges of the above graph is given in the below table -

Edge AB AC AD AE BC CD DE

Weight 1 7 10 5 3 4 2

Now, sort the edges given above in the ascending order of their weights.

Edge AB DE BC CD AE AC AD
Weight 1 2 3 4 5 7 10

Now, let's start constructing the minimum spanning tree.

Step 1 - First, add the edge AB with weight 1 to the MST.

Step 2 - Add the edge DE with weight 2 to the MST as it is not creating
the cycle.
Step 3 - Add the edge BC with weight 3 to the MST, as it is not creating
any cycle or loop.

Step 4 - Now, pick the edge CD with weight 4 to the MST, as it is not
forming the cycle.
Step 5 - After that, pick the edge AE with weight 5. Including this edge
will create the cycle, so discard it.

Step 6 - Pick the edge AC with weight 7. Including this edge will create
the cycle, so discard it.

Step 7 - Pick the edge AD with weight 10. Including this edge will also
create the cycle, so discard it.

So, the final minimum spanning tree obtained from the given weighted
graph by using Kruskal's algorithm is -

The cost of the MST is = AB + DE + BC + CD = 1 + 2 + 3 + 4 = 10.

 ADVERTISEMENT
ADVERTISEMENT

Now, the number of edges in the above tree equals the number of
vertices minus 1. So, the algorithm stops here.

Algorithm
1.

Step 1: Create a forest F in such a way that every vertex of the graph is a
separate tree.
2. Step 2: Create a set E that contains all the edges of the graph.
3.

Step 3: Repeat Steps 4 and 5 while E is NOT EMPTY and F is not spann
ing
4. Step 4: Remove an edge from E with minimum weight
5.

Step 5: IF the edge obtained in Step 4 connects two different trees, then
add it to the forest F
6. (for combining two trees into one tree).
7. ELSE
8. Discard the edge
9. Step 6: END
Complexity of Kruskal's algorithm

Now, let's see the time complexity of Kruskal's algorithm.

o Time Complexity
 The time complexity of Kruskal's algorithm is O(E logE) or O(V
logV), where E is the no. of edges, and V is the no. of vertices.

Implementation of Kruskal's algorithm

Now, let's see the implementation of kruskal's algorithm.

Program: Write a program to implement kruskal's algorithm in C++.

1. #include <iostream>
2. #include <algorithm>
3. using namespace std;
4. const int MAX = 1e4 + 5;
5. int id[MAX], nodes, edges;
6. pair <long long, pair<int, int> > p[MAX];
7. void init()
8. {
9. for(int i = 0;i < MAX;++i)
10. id[i] = i;
11. }
12. int root(int x)
13. {
14. while(id[x] != x)
15. {
16. id[x] = id[id[x]];
17. x = id[x];
18. }
19. return x;
20. }
21. void union1(int x, int y)
22. {
23. int p = root(x);
24. int q = root(y);
25. id[p] = id[q];
26. }
27. long long kruskal(pair<long long, pair<int, int> > p[])
28. {
29. int x, y;
30. long long cost, minimumCost = 0;
31. for(int i = 0;i < edges;++i)
32. {
33. x = p[i].second.first;
34. y = p[i].second.second;
35. cost = p[i].first;
36. if(root(x) != root(y))
37. {
38. minimumCost += cost;
39. union1(x, y);
40. }
41. }
42. return minimumCost;
43. }
44. int main()
45. {
46. int x, y;
47. long long weight, cost, minimumCost;
48. init();
49. cout <<"Enter Nodes and edges";
50. cin >> nodes >> edges;
51. for(int i = 0;i < edges;++i)
52. {
53. cout<<"Enter the value of X, Y and edges";
54. cin >> x >> y >> weight;
55. p[i] = make_pair(weight, make_pair(x, y));
56. }
57. sort(p, p + edges);
58. minimumCost = kruskal(p);
59. cout <<"Minimum cost is "<< minimumCost << endl;
60. return 0;
61. }

Output

So, that's all about the article. Hope the article will be helpful and
informative to you.

Dijkstra's Algorithm
Dijkstra's shortest path algorithm was invented in 1956 by the Dutch
computer scientist Edsger W. Dijkstra during a twenty minutes coffee
break, while out shopping with his fiancée in Amsterdam.
The reason for inventing the algorithm was to test a new computer called
ARMAC.

The Dijkstra algorithm is one of the most famous algorithms in the

world of computer science. It is a single-source shortest path algorithm
that authorises us to discover the shortest path between any two vertices
of a graph.
Here, the meaning of single-source is that we only have one source, and
we have to discover the shortest route from the source to all the nodes.

What is the Dijkstra Algorithm?

The Dijkstra is an iterative algorithm that finds the fastest path from a
particular origin node to all other nodes in a graph. It varies from the
least spanning tree in that the fastest distance between two vertices may
not involve all of the graph’s vertices.

It’s worth noting that Dijkstra’s algorithm is appropriate only when all
weights are favourable, as the weights of the edges are added to find the
shortest path during implementation.

History of Dijkstra Algorithm

This algorithm was designed and disseminated by Dr Edsger W.
Dijkstra, a shining Dutch computer scientist.

It was about 1959, he issued an article of about three pages. The title of
the page was: “A note on two problems in connexion with graphs”.
Here, he explained the whole algorithm.

Working of Dijkstra Algorithm

The algorithm works by maintaining a set of nodes, called the frontier,
which contains all the nodes that have been visited but not yet processed.
The algorithm then repeatedly expands the frontier by choosing the node
with the lowest weight and adding its neighbors to the frontier. This
process continues until the destination node is reached, at which point
the shortest path to the destination has been found.

Dijkstra’s algorithm is guaranteed to find the shortest path from the

source to the destination, if such a path exists. However, it is not
guaranteed to find the shortest path if there are multiple paths with the
same weight.

Advantages of Dijkstra’s Algorithm

 It is used in Google Maps.

 It is employed to identify the shortest path.
 It is very popular in the Geographical Maps.
 It is used to locate the points on the map that correspond to the
graph’s vertices.
 In order to identify the Open Shortest Path First, it is needed in IP
routing.
 The telephone network makes use of it.
Disadvantages of Dijkstra Algorithm

 It conducts a blind scan, which takes a lot of processing time.

 It is unable to manage sharp edges. As a result, acyclic graphs are
produced, and the ideal shortest path is frequently impossible to
find.
Applications of Dijkstra’s Algorithm

 To determine the quickest route

 In applications for social networking
 Within a phone network
 To locate the places on the map

Complexity of Dijkstra’s Algorithm

 The time complexity is: O(E Log V).

 Where E represents the edges and V indicates the vertices.
 The space complexity is: O(V)
Dijkstra's Algorithm

It differs from the minimum spanning tree because the shortest distance
between two vertices might not include all the vertices of the graph.

How Dijkstra's Algorithm works

Dijkstra's Algorithm works on the basis that any subpath B -> D of the
shortest path A -> D between vertices A and D is also the shortest path
between vertices B and D.

Each subpath is the shortest path

Djikstra used this property in the opposite direction i.e we overestimate

the distance of each vertex from the starting vertex. Then we visit each
node and its neighbors to find the shortest subpath to those neighbors.
The algorithm uses a greedy approach in the sense that we find the next
best solution hoping that the end result is the best solution for the whole
problem.

Example of Dijkstra's algorithm

It is easier to start with an example and then think about the algorithm.
Start with a weighted graph
Choose a starting vertex and assign infinity path values to all other
devices
Go to each vertex and update its path length
If the path length of the adjacent vertex is lesser than new path length,
don't update it
Avoid updating path lengths of already visited vertices
After each iteration, we pick the unvisited vertex with the least path
length. So we choose 5 before 7
Notice how the rightmost vertex has its path length updated twice
Repeat until all the vertices have been visited

Djikstra's algorithm pseudocode

We need to maintain the path distance of every vertex. We can store that
in an array of size v, where v is the number of vertices.

We also want to be able to get the shortest path, not only know the
length of the shortest path. For this, we map each vertex to the vertex
that last updated its path length.

Once the algorithm is over, we can backtrack from the destination vertex
to the source vertex to find the path.

A minimum priority queue can be used to efficiently receive the vertex

with least path distance.
function dijkstra(G, S)
for each vertex V in G
distance[V] <- infinite
previous[V] <- NULL
If V != S, add V to Priority Queue Q
distance[S] <- 0

while Q IS NOT EMPTY

U <- Extract MIN from Q
for each unvisited neighbour V of U
tempDistance <- distance[U] + edge_weight(U, V)
if tempDistance < distance[V]
distance[V] <- tempDistance
previous[V] <- U
return distance[], previous[]

Code for Dijkstra's Algorithm

The implementation of Dijkstra's Algorithm in C++ is given below. The

complexity of the code can be improved, but the abstractions are
convenient to relate the code with the algorithm.
Python
Java
C
C++
# Dijkstra's Algorithm in Python

import sys

# Providing the graph

vertices = [[0, 0, 1, 1, 0, 0, 0],
[0, 0, 1, 0, 0, 1, 0],
[1, 1, 0, 1, 1, 0, 0],
[1, 0, 1, 0, 0, 0, 1],
[0, 0, 1, 0, 0, 1, 0],
[0, 1, 0, 0, 1, 0, 1],
[0, 0, 0, 1, 0, 1, 0]]

edges = [[0, 0, 1, 2, 0, 0, 0],

[0, 0, 2, 0, 0, 3, 0],
[1, 2, 0, 1, 3, 0, 0],
[2, 0, 1, 0, 0, 0, 1],
[0, 0, 3, 0, 0, 2, 0],
[0, 3, 0, 0, 2, 0, 1],
[0, 0, 0, 1, 0, 1, 0]]

# Find which vertex is to be visited next

def to_be_visited():
global visited_and_distance
v = -10
for index in range(num_of_vertices):
if visited_and_distance[index][0] == 0 \
and (v < 0 or visited_and_distance[index][1] <=
visited_and_distance[v][1]):
v = index
return v

num_of_vertices = len(vertices[0])
visited_and_distance = [[0, 0]]
for i in range(num_of_vertices-1):
visited_and_distance.append([0, sys.maxsize])

for vertex in range(num_of_vertices):

# Find next vertex to be visited

to_visit = to_be_visited()
for neighbor_index in range(num_of_vertices):

# Updating new distances

if vertices[to_visit][neighbor_index] == 1 and \
visited_and_distance[neighbor_index][0] == 0:
new_distance = visited_and_distance[to_visit][1] \
+ edges[to_visit][neighbor_index]
if visited_and_distance[neighbor_index][1] > new_distance:
visited_and_distance[neighbor_index][1] = new_distance

visited_and_distance[to_visit][0] = 1

i=0

# Printing the distance

for distance in visited_and_distance:
print("Distance of ", chr(ord('a') + i),
" from source vertex: ", distance[1])
i=i+1

Dijkstra's Algorithm Complexity

 Time Complexity: O(E Log V)
where, E is the number of edges and V is the number of vertices.

Space Complexity: O(V)

Dijkstra's Algorithm Applications

 To find the shortest path

 In social networking applications

 In a telephone network

 To find the locations in the map

Huffman Coding | Greedy Algorithm

Huffman coding is a lossless data compression algorithm. The idea is
to assign variable-length codes to input characters, lengths of the
assigned codes are based on the frequencies of corresponding
characters.
The variable-length codes assigned to input characters are Prefix
Codes, means the codes (bit sequences) are assigned in such a way that
the code assigned to one character is not the prefix of code assigned to
any other character. This is how Huffman Coding makes sure that there
is no ambiguity when decoding the generated bitstream.
Let us understand prefix codes with a counter example. Let there be
four characters a, b, c and d, and their corresponding variable length
codes be 00, 01, 0 and 1. This coding leads to ambiguity because code
assigned to c is the prefix of codes assigned to a and b. If the
compressed bit stream is 0001, the de-compressed output may be
“cccd” or “ccb” or “acd” or “ab”.
See this for applications of Huffman Coding.
There are mainly two major parts in Huffman Coding
1. Build a Huffman Tree from input characters.
2. Traverse the Huffman Tree and assign codes to characters.

Algorithm:

The method which is used to construct optimal prefix code is

called Huffman coding.
This algorithm builds a tree in bottom up manner. We can denote this
tree by T
Let, |c| be number of leaves

|c| -1 are number of operations required to merge the nodes. Q

be the priority queue which can be used while constructing binary
heap.
Algorithm Huffman (c)
{
n= |c|

Q=c
for i<-1 to n-1

do
{
 temp <- get node ()

left (temp] Get_min (Q) right [temp] Get Min (Q)

a = left [templ b = right [temp]

F [temp]<- f[a] + [b]

insert (Q, temp)

return Get_min (0)

}
Steps to build Huffman Tree
Input is an array of unique characters along with their frequency of
occurrences and output is Huffman Tree.
1. Create a leaf node for each unique character and build a min heap of
all leaf nodes (Min Heap is used as a priority queue. The value of
frequency field is used to compare two nodes in min heap. Initially,
the least frequent character is at root)
2. Extract two nodes with the minimum frequency from the min heap.

3. Create a new internal node with a frequency equal to the sum of the
two nodes frequencies. Make the first extracted node as its left child
and the other extracted node as its right child. Add this node to the
min heap.
4. Repeat steps#2 and #3 until the heap contains only one node. The
remaining node is the root node and the tree is complete.
Let us understand the algorithm with an example:
character Frequency
a 5
b 9
c 12
d 13
e 16
f 45
Step 1. Build a min heap that contains 6 nodes where each node
represents root of a tree with single node.
Step 2 Extract two minimum frequency nodes from min heap. Add a
new internal node with frequency 5 + 9 = 14.

Illustration of step 2
Now min heap contains 5 nodes where 4 nodes are roots of trees with
single element each, and one heap node is root of tree with 3 elements
character Frequency
c 12
d 13
Internal Node 14
e 16
f 45
Step 3: Extract two minimum frequency nodes from heap. Add a new
internal node with frequency 12 + 13 = 25

Illustration of step 3

Now min heap contains 4 nodes where 2 nodes are roots of trees with
single element each, and two heap nodes are root of tree with more
than one nodes
character Frequency
Internal Node 14
e 16
Internal Node 25
f 45
Step 4: Extract two minimum frequency nodes. Add a new internal
node with frequency 14 + 16 = 30

Illustration of step 4

Now min heap contains 3 nodes.

character Frequency
Internal Node 25
Internal Node 30
f 45
Step 5: Extract two minimum frequency nodes. Add a new internal
node with frequency 25 + 30 = 55

Illustration of step 5

Now min heap contains 2 nodes.

character Frequency
f 45
Internal Node 55
Step 6: Extract two minimum frequency nodes. Add a new internal
node with frequency 45 + 55 = 100

Illustration of step 6

Now min heap contains only one node.

character Frequency
Internal Node 100
Since the heap contains only one node, the algorithm stops here.
Steps to print codes from Huffman Tree:
Traverse the tree formed starting from the root. Maintain an auxiliary
array. While moving to the left child, write 0 to the array. While
moving to the right child, write 1 to the array. Print the array when a
leaf node is encountered.

Steps to print code from HuffmanTree

The codes are as follows:

character code-word
f 0
c 100
 d 101
a 1100
b 1101
e 111
Recommended Problem
Huffman Encoding

Heap

priority-queue

+2 more

Morgan Stanley

Amazon

+3 more

Solve Problem
Submission count: 34K

import java.util.Comparator;

import java.util.PriorityQueue;

import java.util.Scanner;

class Huffman {

// recursive function to print the

// huffman-code through the tree traversal.

// Here s is the huffman - code generated.

public static void

printCode(HuffmanNode root, String s)

// base case; if the left and right are null

 // then its a leaf node and we print

// the code s generated by traversing the

tree.

if (root.left == null && root.right ==

null

&& Character.isLetter(root.c)) {

// c is the character in the node

System.out.println(root.c + ":" + s);

return;

// if we go to left then add "0" to the

code.

// if we go to the right add"1" to the

code.
 // recursive calls for left and

// right sub-tree of the generated tree.

printCode(root.left, s + "0");

printCode(root.right, s + "1");

// main function

public static void main(String[] args)

Scanner s = new Scanner(System.in);

// number of characters.

int n = 6;
 char[] charArray = { 'a', 'b', 'c', 'd', 'e', 'f'
};

int[] charfreq = { 5, 9, 12, 13, 16, 45 };

// creating a priority queue q.

// makes a min-priority queue(min-

heap).

PriorityQueue<HuffmanNode> q

= new
PriorityQueue<HuffmanNode>(

n, new MyComparator());

for (int i = 0; i < n; i++) {

// creating a Huffman node object

// and add it to the priority queue.

HuffmanNode hn = new
HuffmanNode();

hn.c = charArray[i];

hn.data = charfreq[i];

hn.left = null;

hn.right = null;

// add functions adds

// the huffman node to the queue.

q.add(hn);

// create a root node

HuffmanNode root = null;

 // Here we will extract the two
minimum value

// from the heap each time until

// its size reduces to 1, extract until

// all the nodes are extracted.

while (q.size() > 1) {

// first min extract.

HuffmanNode x = q.peek();

q.poll();

// second min extract.

HuffmanNode y = q.peek();

q.poll();
 // new node f which is equal

HuffmanNode f = new
HuffmanNode();

// to the sum of the frequency of the

two nodes

// assigning values to the f node.

f.data = x.data + y.data;

f.c = '-';

// first extracted node as left child.

f.left = x;

// second extracted node as the right

child.

f.right = y;
 // marking the f node as the root
node.

root = f;

// add this node to the priority-queue.

q.add(f);

// print the codes by traversing the tree

printCode(root, "");

// node class is the basic structure

// of each node present in the Huffman - tree.

class HuffmanNode {
 int data;

char c;

HuffmanNode left;

HuffmanNode right;

// comparator class helps to compare the

node

// on the basis of one of its attribute.

// Here we will be compared

// on the basis of data values of the nodes.

class MyComparator implements

Comparator<HuffmanNode> {

public int compare(HuffmanNode x,

HuffmanNode y)
 {

return x.data - y.data;

Output
f: 0
c: 100
d: 101
a: 1100
b: 1101
e: 111
Time complexity: O(nlogn) where n is the number of unique
characters. If there are n nodes, extractMin() is called 2*(n – 1) times.
extractMin() takes O(logn) time as it calls minHeapify(). So, the
overall complexity is O(nlogn).
If the input array is sorted, there exists a linear time algorithm. We will
soon be discussing this in our next post.
Space complexity :- O(N)
Applications of Huffman Coding:
1. They are used for transmitting fax and text.
2. They are used by conventional compression formats like PKZIP,
GZIP, etc.
3. Multimedia codecs like JPEG, PNG, and MP3 use Huffman
encoding(to be more precise the prefix codes).
It is useful in cases where there is a series of frequently occurring
characters.