Scribd
Upload
2 views38 pages

Lecture 18 Greedy Algo

The document discusses dynamic programming (DP) algorithms, specifically the 0/1 Knapsack Problem, which aims to maximize the total value of items packed in a knapsack with a given weight capacity. It outlines the steps to solve the problem using DP, including characterizing the optimal solution, recursively defining its value, and constructing it from computed information. Additionally, it introduces the Greedy Algorithm and Huffman coding for data compression, highlighting their principles and applications.

Uploaded by

18-QADEER AHMAD
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
2 views38 pages

Lecture 18 Greedy Algo

The document discusses dynamic programming (DP) algorithms, specifically the 0/1 Knapsack Problem, which aims to maximize the total value of items packed in a knapsack with a given weight capacity. It outlines the steps to solve the problem using DP, including characterizing the optimal solution, recursively defining its value, and constructing it from computed information. Additionally, it introduces the Greedy Algorithm and Huffman coding for data compression, highlighting their principles and applications.

Uploaded by

18-QADEER AHMAD
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 38

Algorithms

Lecture 18
DP Algorithm
DP algorithm can broken into 4
steps
1. Characterize the structure of
optimal solution
2. Recursively define the value of
optimal solution
3. Compute the value of optimal
solution in bottom up manner
4. Construct an optimal solution
from computed information
0/1 Knapsack Problem
O Given a knapsack with maximum
capacity W, and a set S consisting
of n items
O Each item i has some weight wi
and value vi (all wi,vi and W are
integers)
O Problem: How to pack knapsack
to achieve maximum total value of
packed items ?
0/1 Knapsack Problem
Item(i) Weight(wi) Value(vi)
1 2 3
2 3 4
3 4 5
4 5 8
5 9 10

Knapsack can hold W=10


0/1 Knapsack Problem
O Mathematically, this problem is:

maximize ∑ vi
iεT

subject to ∑ wi≤W
iεT

O This problem is called 0/1 because


each item is entirely accepted or
rejected
Brute-force solution
O Since there are n item, there are 2n
possible combinations of the items
(an item is either chosen or not)
O We go through all combinations and
find the one with the most total
value and with total weight less or
equal to W
O The running time will be O(2n)
0/1 Knapsack Problem Using DP
Approach
O For each i≤n and w≤W, solve the knapsack
problem for the first i objects when capacity
is w
O We construct a matrix
V[0…n,0….w]
O For 1≤i≤n, and 0≤j≤W, V[i,j] will store the
maximum value of any set of object
{1,2,3….i} that can fit into a knapsack of
capacity j.
O V[n,w] will contain the maximum value of
all n objects that can fit into the entire
knapsack of weight W
0/1 Knapsack Problem
O To compute the entries of V we will
start from base case
O As a basis, V[0,j]=0 for 0≤j≤W since, if
we have no item than we have no
value
O Leave object i: If we chose to not
take object i, then the optimal value
will come about by considering how to
fill knapsack of capacity j with
remaining {1,2,…,i-1}
O This is just v[i-1,j]
0/1 Knapsack Problem
O Take object i:
If we take object i, then we gain
a value of vi
O But we use up wi of out capacity
with the remaining j-wi capacity in
the knapsack, we can fill it with
the best possible way with objects
{1,2,3,…i-1}
O This is vi+V[i-1,j-wi]
O This is only possible when wi≤j
Recursive Formulation
V[0,j]=0 if j≥0

V[i,j]=
v[i-1,j] if wi > j
max{v[i-1,j],vi+[i-1,j-wi]} if wi≤j

• A naive evaluation of this recursion


definition is exponential so, as usual we will
avoid this
Consider an example maximum weight which
knapsack can hold W is 10.There are five items to
choose from:

Weight limit
0 1 2 3 4 5 6 7 8 9 10
(j)
w0 =0 v0=0
w1 =1 v1=1
w2 =2 v2=6
w3 =5 v3=18
w4 =6 v4=22
w5 =7 v5=28
Weight limit
0 1 2 3 4 5 6 7 8 9 10
(j)
w0 =0 v0=0 0 0 0 0 0 0 0 0 0 0 0
w1 =1 v1=1 0
w2 =2 v2=6 0
w3 =5 v3=18 0
w4 =6 v4=22 0
w5 =7 v5=28 0
Weight limit
0 1 2 3 4 5 6 7 8 9 10
(j)
w0 =0 v0=0 0 0 0 0 0 0 0 0 0 0 0
w1 =1 v1=1 0 1 1 1 1 1 1 1 1 1 1
w2 =2 v2=6 0
w3 =5 v3=18 0
w4 =6 v4=22 0
w5 =7 v5=28 0
Weight limit
0 1 2 3 4 5 6 7 8 9 10
(j)
w0 =0 v0=0 0 0 0 0 0 0 0 0 0 0 0
w1 =1 v1=1 0 1 1 1 1 1 1 1 1 1 1
w2 =2 v2=6 0 1 6 7 7 7 7 7 7 7 7
w3 =5 v3=18 0
w4 =6 v4=22 0
w5 =7 v5=28 0
Weight limit
0 1 2 3 4 5 6 7 8 9 10
(j)
w0 =0 v0=0 0 0 0 0 0 0 0 0 0 0 0
w1 =1 v1=1 0 1 1 1 1 1 1 1 1 1 1
w2 =2 v2=6 0 1 6 7 7 7 7 7 7 7 7
w3 =5 v3=18 0 1 6 7 7 18 22 24 25 25 25

w4 =6 v4=22 0
w5 =7 v5=28 0
Weight limit
0 1 2 3 4 5 6 7 8 9 10
(j)
w0 =0 v0=0 0 0 0 0 0 0 0 0 0 0 0
w1 =1 v1=1 0 1 1 1 1 1 1 1 1 1 1
w2 =2 v2=6 0 1 6 7 7 7 7 7 7 7 7
w3 =5 v3=18 0 1 6 7 7 18 22 24 25 25 25

w4 =6 v4=22 0 1 6 7 7 18 24 29 29 29 29

w5 =7 v5=28 0 1 6 7 7 18 22 26 29 34 35
0/1 Knapsack Problem
O we entered the values using the
recursion V[i,j]=
O max{v[i-1,j],vi+[i-1,j-wi]} if
wi≤j
O V[3,7]=max{v[2,7],v3+v[2,2]}
=max{7,18+6}=24
O KnapSack(n,W)
1. for w0 to W
2. do V[0,w] 0
3. for i=0 to n
4. do v[i,0]0
5. for w=0 to w
6. do if(wi≤w & vi+v[i-1,w-wi]>v[i-1,w]
7. then v[i,w]vi+v[i-1,w-wi]
8. else v[i,w]v[i-1,w]
The time complexity of this algorithm is
clearly O(n.w)
Constructing the optimal
solution
O The algorithm for computing v[i,j]
does not keep the record of which
subset of items gives the optimal
solution
O To compute the actual subset we
can add an auxiliary Boolean
matrix keep[i,j] which is 1 if we
decide to take the ith element and
0 otherwise
Weight limit
0 1 2 3 4 5 6 7 8 9 10
(j)
w0 =0 v0=0 0 0 0 0 0 0 0 0 0 0 0
w1 =1 v1=1 0 1 1 1 1 1 1 1 1 1 1
w2 =2 v2=6 0 0 1 1 1 1 1 1 1 1 1
w3 =5 v3=18 0 0 0 0 0 1 1 1 1 1 1
w4 =6 v4=22 0 0 0 0 0 0 1 1 1 1 1
w5 =7 v5=28 0 0 0 0 0 0 0 1 1 1 1
Optimal solution
O How do we use all values to
determine optimal subset T of
items
O If keep[n,w] is 1 then nεT , we
can repeat this argument for
keep[n-1,w-wn]
O If keep[n,w] is 0 then n≠T and
we repeat the argument for
keep [n-1,W]
Weight limit
0 1 2 3 4 5 6 7 8 9 10
(j)
w0 =0 v0=0 0 0 0 0 0 0 0 0 0 0 0
w1 =1 v1=1 0 1 1 1 1 1 1 1 1 1 1
w2 =2 v2=6 0 0 1 1 1 1 1 1 1 1 1
w3 =5 v3=18 0 0 0 0 0 1 1 1 1 1 1
w4 =6 v4=22 0 0 0 0 0 0 1 1 1 1 1
w5 =7 v5=28 0 0 0 0 0 0 0 1 1 1 1

Optimal subset T={7,2,1}


Values=28+6+1=35
Greedy
Algorithms
Greedy Algorithm
O A Greedy Algorithms works in phases
O A Greedy Algorithm always make the
choice that looks best at the moment
without regard for future consequences
O It hopes that by choosing the local
optimum at each step, it will end up
with global optimum
O For some problems greedy strategy
always gets optimum, for others it
finds good but not always best, if so,
it is called greedy heuristic
O For still others greedy approach do
very poorly
Huffman coding
O Widely used data compression
technique
O Use in computer networks
O To transmit data to its destination
faster ,it is necessary to either
increase bandwidth
O Or the other way is to send data in
fact less data by means of data
compression
O JPEG uses this technique for
compression
O Mostly used for text document
Example
O Suppose we want to send message
“abacacacda”
O If we do not use compression than
this message will take 8n bits
which are 8.10=80 bits using ASCII
standard
O For compression first we build
table of frequencies
O Then we build a binary tree for
codes
Huffman Encoding
O This algorithm was introduced by
David Huffman as part of a course
assignment at MIT
O This algorithm uses a full binary to
represent variable-length codes
O In variable-length code frequent
words has short length code and
infrequent words has long length
codes
O The leaves of the tree represent the
character
A table for the string “abacacacda”
by scanning the string
Character a b c d
F 5 1 3 1
P=(Fi/∑F ) .5 .1 .3 .1
• Next step is to build a tree bottom-up
using the greedy approach
• Greedy approach is to always pick
the two less frequencies and build a
sub-tree and so on
• This tree holds the prefix property
What is prefix 10
property ? 0
1

a:5 5
a0 0 1
b  110
c:3 2
c  10 0 1
d  111
b:1 d:1

01100100100101110
abacacacda
Total no of 17 bits
Binary Character Code
 Suppose we want to send 100000 characters
 Which requires 800000 bits 11000101

300000

224000
Huffman Coding
O The tree holds the prefix code
property
O If C is the character set then
there are |C| leaves and |C|-1
internal nodes
O How we will build this tree
O We build it bottom-up using the
greedy approach
O HUFFMAN(C)
1. n |C|
2. QC // Q is a priority queue
3. for i 1 to n-1
4. do allocate a new node z
5. left[z]  x  Extract-Min(Q)
6. right[z]y Extract-Min(Q)
7. f[z]f[x]+f[y]
8. Insert(Q,z)
9. Return Extract-Min(Q) // return root
The codes are as
follows:
character code-word
f0
c 100
d 101
a 1100
b 1101
e 111

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy