Applied Algorithms

CSCI-B505 / INFO-I500

Lecture 7.
Amortized Analysis - 2

M. Oguzhan Kulekci
• Dynamic Array Allocation
• Amortized Dictionary Data Structure
 Dynamic Arrays

• Remember array is a contiguous block in memory.

• Thus, its size should be de nite at the time of creation.
• However, the size of an array can change frequently !
• Therefore, the dynamic arrays, whose size can be altered at run time is of our
interest. Notice that many modern programming languages make use of it.

• At the end we will see that, theoretically it is no di erent then static arrays !?
fi
ff
 Dynamic Array Allocation
Main Question: What to do when array is full, in other words, all cells are occupied,
and we need to add a new element to this full array.
13
1 2 3 4 5 6 7 8 9 10 11 12

Allocate a larger size in the memory

Move old elements to the new array

1 2 3 4 5 6 7 8 9 10 11 12

Append the new item to the new array

1 2 3 4 5 6 7 8 9 10 11 12 13
 Dynamic Array Allocation
Let’s consider incrementing the size by a xed amount each time we need to resize.

2
• E ort per item is O(n )/n = O(n)
1…k
Move k
items
• Not good, when compared to O(1)
1…k k+1 … 2k in the regular static array.
Move 2k
items 1…k k+1 … 2k 2k+1 … 3k
n
m=⌊ ⌋
Move mk k
items
1…k k+1 … 2k …………. mk+1 … n

k ⋅ m ⋅ (m − 1) 2 2
k(1 + 2 + … + (m − 1)) = ∈ O(k ⋅ m ) → O(n )
2
ff
fi
 Dynamic Array Allocation
Let’s consider doubling the current size each time we need to resize.

1 • Now, e ort per item is O(n)/n = O(1)

• Same with the regular static array
1 2 • But, where did our extra movements go? !
1 2 3 4 • In the hidden constant of O(1), which
is 3 on the dynamic case ?
1 2 3 4 5 6 7 8

k
2 n 2k+1

k
k
(1 + 2 + … + 2 ) = 2k+1 k
− 1 ∈ O(2 ) → O(n) k = ⌊log n⌋ → 2 ≈ n
ff
Dynamic Array Allocation
th
Aggregate analysis
ci is the cost of i insertion to the array
k
• If i = 2 + 1, for some k>0, then ci = i.
• Else, ci = 1.
n ⌊log n⌋
j
∑ ∑
ci ≤ n + 2
i=1 j=0
< n + 2n
< 3n
3n
• The cost per item is less than = 3
n
⌊log n⌋
• Therefore, O(1).
Dynamic Array Allocation
Accounting Method

$3 per each insert will guarantee to avoid bankruptcy.

k/2 has $0 k/2 has $2

If half of the elements have $2 on them, it is enough
1 2 ……. k to copy the old items. Since regular $1 cost will be
required per each, $3 will surely avoid bankruptcy.

$k is used to copy k elements

1 2 ……. k k+1 …….

Should have $2 left

on it after insertion
 Dynamic Array Allocation
Potential Method
ϕ(Di) = 2 ⋅ size(Di) − capacity(Di) ϕ(D0) = 0
# of elements Total space of the array

∀i, ϕ(Di) ≥ 0, since after each resize capacity is 2 times the number of elements.

Case 1: There is space in the capacity, so no resize is required.

• Actual cost: ci = 1
• ϕ(Di) − ϕ(Di−1) = 2 ⋅ size(Di) − capacity(Di) − 2 ⋅ size(Di−1) + capacity(Di−1) = 2, since
size(Di) = size(Di−1) + 1
capacity(Di) = capacity(Di−1)
• Amortized cost : cî = 1 + 2 = 3
 Dynamic Array Allocation
Potential Method
ϕ(Di) = 2 ⋅ size(Di) − capacity(Di) ϕ(D0) = 0
# of elements Total space of the array

∀i, ϕ(Di) ≥ 0, since after each resize capacity is 2 times the number of elements.

Case 2: There is NO space in the capacity, so resizing is required.

• Actual cost: ci = 1 + capacity(Di−1)
• ϕ(Di) − ϕ(Di−1) = 2 ⋅ size(Di) − capacity(Di) − 2 ⋅ size(Di−1) + capacity(Di−1)
= 2 − capacity(Di−1),
since size(Di) = size(Di−1) + 1 and capacity(Di) = 2 ⋅ capacity(Di−1)
• Amortized cost : cî = 1 + capacity(Di−1) + 2 − capacity(Di−1) = 3
Dynamic Array Allocation

• What if we want to support resize on delete operation ?

• When the number of items in the array become less, we shrink the array.
• What would be a good strategy?
• Halve the array when items become 1/2 ???
 Amortized Dictionary Data Structure
Dictionary structures are always in the heart of computing.
We have many alternatives. Here is one of those …
Problem: Given n items, provide an e cient way to support search and update.
Main idea:
• Instead of a single list, maintain a collection of arrays, say A0, A1, …, Ak−1
i
• Array Ai has either exactly 2 elements or empty with zero elements.
• Each array is sorted.
• There is no relation or order between the arrays.
Depending on the number n, how will we decide on the number of arrays (k=?), and
how will we decide which ones will be empty and full ?
ffi
 Amortized Dictionary Data Structure
Depending on the number n, how will we decide on the number of arrays (k=?), and
how will we decide which ones will be empty and full ?

Each integer can be written as the sum of the powers of 2.

Actually, this is its binary representation !
(n = 11) → n = 1011 indicates n=8.1+4.0+2.1+1.1
4 arrays A3, A2, A1, A0 with sizes 8,4,2,1, respectively. A2 is empty.
A0 : 5
• Given n, we maintain ⌈log(n + 1)⌉ arrays.
A1 : 1 9
• Each has its corresponding size.
A2 : • The ones with a 1 bit are full, others empty
A3 : 2 4 8 10 13 15 16 19
What do you think about the construction cost?
 Amortized Dictionary Data Structure
Searching for a key on this dictionary which maintains n keys in total ?

• Investigate each array one-by-one.

A0 :
A1 :
• We have k = ⌈log(n + 1)⌉ arrays.
A2 : • Search on a sorted array of t elements
A3 : is O(log t) via binary search.
• Longest array size ≤ n.
• At most k arrays will be searched.
Ak−1 : ………………………………..
• Each search is O(log n) time.
Then, overall cost of search is
k = ⌈log(n + 1)⌉
2
k ⋅ log n ∈ O(log n)
 Amortized Dictionary Data Structure
INSERTING a new key

• Put new element into array H of size 1.

A0 :
• i=0
A1 :
• Check Ai. If empty, copy H to Ai and stop. Else,
A2 :
H ← merge(Ai, H) and i ← i + 1 and repeat.
A3 :

A0 :
A1 :
A2 :
A3 :
 Amortized Dictionary Data Structure
INSERTING a new key
• Put new element into array H of size 1.
Merging two sorted list, each with size • i=0
ℓ requires less than 2ℓ comparisons ! • Check Ai. If empty, copy H to Ai and stop. Else,
H ← merge(Ai, H) and i ← i + 1 and repeat.

2 5 9 2 5 9 2 5 9 2 5 9

4 7 8 4 7 8 4 7 8 4 7 8

2 4 5 7 8 9
2 2 4 2 4 5
 Amortized Dictionary Data Structure
INSERTING a new key
k
• Worst case: We visit and merge all arrays in the dictionary, e.g. n = 2 − 1 elements in
k
the dictionary for some k, and we are adding the 2 th element
• What will be the cost of this worst case?
• Merging two sorted list, each with size ℓ requires less than 2ℓ comparisons !
k k+1
• Therefore, C = 2 + 4 + 8 + . . . . + 2 = 2 − 1. Since k ∈ O(log n), C ∈ O(n).

Once such a worst case happens, can it appear repeatedly ?

NO!
So, regular worst case analysis is not tight! We can try an amortized
approach by computing the cost of , say t, consecutive insert
operations.
 Amortized Dictionary Data Structure
INSERTING a new key has O(log n) amortized complexity.
i
The merge cost of Ai is at most 2 ⋅ 2 as merging two list each with ℓ costs 2ℓ.
During n insertion operations,
• A0 will be subject to merge n/2 times with a cost of 2.
n n n
• A1 will be subject to merge n/4 times with a cost of 4. ⋅ 2 + ⋅ 4 + ⋅ 8 + … ≈ n ⋅ log n
2 4 8
• A2 will be subject to merge n/8 times with a cost of 8.
• Totally O(log n) arrays will be subject to merge each with O(n) cost.
• Therefore, this makes total cost O(n log n) for n insertions, which makes amortized cost of
insertion O(log n).

This is exactly the same with the binary counter amortized analysis with one
th k
di erence as the cost of ipping k bit is 2 instead of a constant 1 unit.
ff
fl
 Amortized Dictionary Data Structure
DELETING a key
i
• Assume we will be deleting an item from the array Ai that includes 2 elements.
i−1 i−1 i
• Split Ai into small arrays of length 1,2,4,…,2 . Notice that 1 + 2 + 4 + … + 2 = 2 − 1,
which is exactly the number of remaining elements in Ai. Delete all items from Ai.
• For each of these small arrays, insert it into the dictionary again. Insert operation starts with
the corresponding list length, i.e., small array of size 1, start with A0, size 2 start with A1, and
continue accordingly.

There can be at most log n small arrays after deleting an element. The amortized cost of insertion
process is O(log n) as we showed previously. So the cost of deleting an element in the worst
2
case is O(log n) with the proposed method.
There might be other ways of deletion as well ?
Reading assignment

• Read chapter 17 Amortized Analysis from Cormen and also related chapters
from other text books or resources on the internet.

• Next week we will study recursions and divide-and-conquer type algorithms.