DAA Unit 3
DAA Unit 3
Greedy method: General method, applications- knapsack problem, spanning trees, Job sequencing with deadlines, Minimum
cost spanning trees, Prim‟s Algorithm, Kruskal‟s Algorithms, An optimal randomized algorithm, Optimal Merge Patterns,
Single source shortest path problem.
General Method:
Greedy method is the most straight forward design technique. It is used to solve optimization
problems, and can be applied to wide variety of problems.
These problems have n inputs and require us to obtain a subset that satisfies some constraints.
Any subset that satisfies these constraints is called a feasible solution. A feasible solution that either
maximizes or minimizes a given objective function need to be find. A feasible solution that does this
is called an optimal solution.
An algorithm that works in stages can be devised by using greedy method. At each stage, a decision
is made whether a particular input is optimal solution or not. This is done by considering the inputs
in an order determined by some selection procedure.
The selection procedure is based on some optimization measure. This measure may be objective
function. This version of greedy technique is called subset paradigm.
The function select selects and removes an input from a[] and the input value is assigned to x.
The function Feasible determines whether x can be included into the solution or not.
The function union combines x with the solution and updates.
The above algorithm gives an abstract view of greedy technique. The functions select, feasible and
union need to be properly implemented for solving a particular problem.
Ordering Paradigm: In this paradigm, each decision is made using an optimization criterion that can be
computed using decisions already made.
1
Design and Analysis of Algorithms Unit-I
Knapsack problem:
We are given n objects and a knapsack. Object i has a weight wi and the knapsack has a capacity m.
If a fraction xi, 0 xi 1, of object i is placed into the knapsack, then a profit of pi xi is earned.
The objective is to obtain a filling of the knapsack that maximizes the total profit earned.
As the knapsack capacity is m, the total weight of all the chosen objects to be at most m.
Formally, the problem can be stated as
Maximize px i i ---- (1)
1in
Subject to w x
1in
i i m ---- (2)
A feasible solution is any set (x1, ....... ,xn ) , which satisfies (2) and (3).
An optimal solution is a feasible solution which maximizes px i i .
1in
Example: consider the following instances of the knapsack problem
n=3, m=20, (p1, p2, p3)=(25,24,15) and (w1, w2, w3)=(18,15,10)
Some feasible solutions are
(x1, x2 , x3 ) wi xi px i i
1 1 1
1 , , 9 + 5 + 2.5 = 16.5 12.5 + 8 + 3.75 = 24.25
2 3 4
2
1, ,0
2 18 + 2 + 0 = 20 25 + 3.2 + 0 = 28.2
15
2
0, , 1
3 0 + 10 + 10 = 20 0 + 16 +15= 31
3
1
0, 1,
4 0 + 15 + 5 = 20 0 + 24 + 7.5 = 31.5
2
Out of these 4 feasible solutions, solution 4 yields the maximum profit. Therefore this solution is optimal for
the given problem instance.
***In case the sum of all the weights is ≤ m, then xi=1, 1 ≤ i ≤ n is an optimal solution.
The knapsack problem calls for selecting a subset of objects and hence fits the subset paradigm. In addition,
this problem also involves the selection of xi for each object.
The greedy strategies to obtain feasible solutions are
Strategy 1:
Fill the knapsack by including next the object with largest profit. If the object doesn‟t fit, then a
fraction of it is included to fill the knapsack.
2
Design and Analysis of Algorithms Unit-I
Thus each time an object is included into the knapsack, we obtain the largest possible increase in
profit value. If we follow this method for the above example, solution 2 is formed. But this solution
is suboptimal.
This greedy method did not yield an optimal solution.
Strategy 2:
Fill the knapsack by including next the object with least weight.
If we follow this method in the above example, solution 3 results. This too is suboptimal.
Strategy 3:
This strategy strives to achieve a balance between the rate at which profit increases and the rate at
which capacity is used.
At each step, we include that object which has maximum profit per unit of capacity used. i.e., the
p
objects are considered in order of the ratio i . Solution 4 results, if we follow this approach.
wi
Following algorithm(GreedyKnapsack) obtains solutions corresponding to this strategy. The objects have to
p
be sorted in decreasing order of i .
wi
Algorithm GreedyKnapsack(m, n)
{
for i:=1 to m do x[i]:=0.0;
U:=m;
for i:=1 to n do
{
if (w[i] > U) then break;
x[i]:=1.0;
U:=U-w[i];
}
if ( i ≤ n ) then x[i]:=U/w[i];
}
If time to sort the objects is discarded, then this strategy requires O(n) time.
Exercise :
1. Find an optimal solution to the knapsack instance n=7, m=15, (p1,…,p7) = (10, 5, 15, 7, 6, 18, 3) and
(w1,….,w7) = (2, 3, 5, 7, 1, 4, 1)
3
Design and Analysis of Algorithms Unit-I
Since one job can be processed in a single machine. The other job has to be in its waiting state until the job
is completed and the machine becomes free.
So the waiting time and the processing time should be less than or equal to the dead line of the job.
Consider the jobs in the nonincreasing order of profits subject to the constraint that the resulting job
sequence J is a feasible solution.
To formulate the greedy algorithm for an optimal solution, we must formulate an optimization measure to
determine how the next job is chosen.
Choose the objective function pi as the optimization measure.
iJ
The next job to include is the one that increases pi the most, subject to the constraint that the
iJ
resulting J is a feasible solution.
This requires us to consider the jobs in the nonincreasing order of pi‟s .
Add one by one job to J and check whether it is feasible or not.
To determine whether the given J is feasible solution, check only permutation in which jobs are
ordered in nondecreasing order of deadlines.
4
Design and Analysis of Algorithms Unit-I
Ex 2: Let n = 5, (p1, . . ., p5) = (20, 15, 10, 5, 1) and (d1, . ., d5) = (2, 2, 1, 3, 3)
Job Assigned
Action J Profit
considered slots
1 Accept {1} (1,2) 20
2 Accept {1,2} (0,1)(1,2) 35
3 Reject {1,2} (0,1)(1,2) 35
4 Accept {1,2,4} (0,1)(1,2)(2,3) 40
5 Reject {1,2,4} (0,1)(1,2)(2,3) 40
Ex 3: Let n = 7, (p1, . . ., p7) = (3, 5, 20, 18, 1, 6, 30) and (d1, . ., d7) = (1, 3, 4, 3, 2, 1, 2)
Job Assigned
Action J Profit
considered slots
A high level description of the greedy algorithm for Job Sequencing with deadlines problem is shown in the
algorithm. This algorithm constructs an optimal set J of jobs that can be processed by their due times. The
selected jobs can be processed in the order given by the following theorem.
“ Let J be a set of k jobs and = (i1,i2,….ik) be a permutation of jobs in J such
that di1 ≤ di2 ≤…≤ dik. Then J is a feasible solution iff the jobs in J can be
processed in the order without violating any deadline.”
Algorithm GreedyJob(d, J, n)
// J is a set of Jobs that can be completed by their deadlines.
{
J := { 1 } ;
for i =2 to n do
{
if ( all jobs in J U { i } can be completed by their deadlines ) then J := J U { i };
}
}
5
Design and Analysis of Algorithms Unit-I
Complexity:
Complexity of Job Sequencing can be measured in terms of two parameters. They are n, the number
of jobs and S, the number of jobs included in the solution.
The while loop iterates for at most k times. Each iteration takes (1) .
If the if condition is true then the time taken to insert job I is (k r) .
Hence, the total time for each iteration of for loop is time (k) .
Loop iterates for (n – 1) times.
If s is the final value of k, then the time needed by algorithm JS is – (sn)
Since sn, the worst case time, as a function of n alone is - (n2 )
6
Design and Analysis of Algorithms Unit-I
Since the identification of minimum-cost spanning tree involves the selection of a subset of the
edges, this problems fits the subset paradigm.
A greedy method to obtain a minimum-cost spanning tree builds this tree edge by edge.
The next edge to include is chosen according to some optimization criterion.
The simplest such criterion is to choose an edge that result in a minimum increase in the sum of the
costs of the edges so far included.
There are two possible ways to interpret this criterion. Their respective algorithms are
1. Prim‟s algorithm
2. Kruskal‟s algorithm
Prim’s Algorithm:
The set of edges selected by this algorithm should form a tree.
Start from an arbitrary vertex and store it in A.
Thus, if A is the set of edges selected so far, then A forms a tree.
The next edge (u, v) to be included in A is a minimum-cost edge not in A with the property that
AU{(u,v)} is also a tree.
7
Design and Analysis of Algorithms Unit-I
Kruskal’s algorithm:
In this algorithm, the edges of the graph are considered in nondecreasing order of cost.
The set of t edges selected for the spanning tree be such that it is possible to complete t into a tree.
Thus t may not be a tree at all stages in the algorithm.
The set of edges selected are generally a forest since the set of edges t can be completed into a tree iff
there are no cycles in t.
This interpretation also results in a Minimum Spanning Tree.
8
Design and Analysis of Algorithms Unit-I
Example: following figure shows stages in kruskal‟s algorithm. It begins with no edges selected.
t := Ø;
while( ( t has less than n-1 edges ) and ( E ≠ Ø ) ) do
{
Choose an edge (v, w) from E of lowest cost;
Delete (v, w) from E;
If ((v, w) does not create a cycle in t) then
Add (v, w) to t;
else discard (v, w);
}
9
Design and Analysis of Algorithms Unit-I
Algorithm kruskal(E,cost,n,t)
//Eset of edges in G has „n‟ vertices.
//cost[u,v]cost of edge (u,v). tset of edges in minimum cost spanning tree
// the minimum cost is returned.
{
Construct a heap out of the edge costs;
for i:=1 to n do parent[i]:= -1;
i:=0; mincost:=0.0;
while((i<n-1)and (heap not empty)) do
{
delete a minimum cost edge (u,v) from the heap;
j:=find(u); k:=find(v);
if ( j ≠ k) then
{
i:=i+1;
t[i,1]:=u; t[i,2]:=v;
mincost=mincost+cost[u,v];
union(j,k);
}
}
if ( i ≠ n-1) then write(“No spanning tree”); else return minimum cost;
}
Analysis :
The time complexity of minimum cost spanning tree algorithm in worst case is O(|E|log|E|), where E is the edge
set of G.
An optimal randomized algorithm which takes O(|V| + |E|) can be devised as follows
1. Randomly sample m edges from G
2. Let G1 be the induced sub graph; i.e., G1 has V as its node set and the sampled edges in the
edge set. The subgraph G1 need not be connected. Recursively find Minimum Cost Spanning
Tree for each component of G1. Let F be the resultant minimum cost spanning forest of G1.
3. Using F eliminate certain edges (heavy edges) of G that cannot possibly in a Minimum Cost
Spanning Tree. Let G11 be the graph that results from G after elimination of F-heavy edges.
4. Recursively find Minimum Cost Spanning Tree for G11.
10
Design and Analysis of Algorithms Unit-I
Example :
Given input graph G is
Using F translate the given Graph G into G1 and remove the heavy edges.
11
Design and Analysis of Algorithms Unit-I
Eg: Consider three files x1, x2, x3 with record lengths 30, 20, 10.
The leaf nodes are drawn as squares and represent the given five files. These nodes are called
external nodes.
Remaining nodes are drawn as circles and are called internal nodes.
Each internal node has exactly two children.
The number in each node is the length (no.of records) of the file represented by that node.
12
Design and Analysis of Algorithms Unit-I
The external node x4 is at a distance of 3 from the root node z4. i.e., the records of file x4 are moved
three times, once to get z1, once to get z2 and finally to get z4.
Total number of record moves for the binary merge tree is ∑𝑛𝑖=1 𝑑𝑖 𝑞𝑖
o di is the distance from the root to the external node for file xi
o qi is the length of xi
This sum is the weighted external path length of the tree.
Input to this algorithm is list of n trees. Each node in a tree has 3 fields lchild, rchild, weight.
Initially, each tree in list has exactly one node.
Least(list) function finds a tree in list whose root has least weight.
Insert() function is used to insert a node into the list.
Analysis :
Main for loop is executed (n – 1) times
If list is kept in nondecreasing order according to weight value in the roots, then Least(list) requires
O(1) time
Insert(list, t) can be done in O(n) time.
Hence, the total time taken is O(n2)
Initially
13
Design and Analysis of Algorithms Unit-I
Exercise :
Find an optimal binary merge pattern for ten files whose lengths are
28, 32, 12, 5, 84, 53, 91, 35, 3, 11
14
Design and Analysis of Algorithms Unit-I
Let S denote the set of vertices (including v0) to which the shortest paths have already been
generated.
For w, not in S, let dist[w] be the length of the shortest path starting from v0, going through only
those vertices that are in S, and ending at w.
15
Design and Analysis of Algorithms Unit-I
Example : Use algorithm ShortestPaths to obtain in non-decreasing order the lengths of the shortest paths
from vertex 1to all remaining vertices in the digraph.
V0 =1
0 50 45 10 ∞ ∞
𝖥 ∞ 0 13 15 ∞ ∞1
Cost matrix = ∞ ∞ 0 ∞ 30 ∞
20 ∞ ∞ 0 15 ∞
I∞ 20 35 ∞ 0 ∞ I
L∞ ∞ ∞ ∞ 3 0
Vertex Distance
Iteration S
selected [1] [2] [3] [4] [5] [6]
Initially -- -- 0 50 45 10 ∞ ∞
1 {1} 4 0 50 45 10 25 ∞
2 {1, 4} 5 0 45 45 10 25 ∞
3 {1,4,5} 2 0 45 45 10 25 ∞
4 {1,2,4,5} 3 0 45 45 10 25 ∞
5 {1,2,3,4,5} 6 0 45 45 10 25 ∞
6 {1,2,3,4,5,6}
Example 2: Use algorithm ShortestPaths to obtain in non-decreasing order the lengths of the shortest paths
from city Boston to all remaining cities in the digraph.
16
Design and Analysis of Algorithms Unit-I
Length-adjacency matrix
Tracing
17