ALGORITHMS

SECTION1: INTRODUCTION

Eng. Dalia Elwi

ALGORITHM ANALYSIS TYPES

Statistical
Time
Types

Mathematical

Statistical
Space
Mathematical
ALGORITHM ANALYSIS TYPES

Statistical
Time
Types

Mathematical

Statistical
Space
Mathematical
 TIME ANALYSIS

• Mathematical Model:

Algorithm Running Time

(Time Complexity)
T(n)

• T(n) is determined by two factors:

1. Cost.
2. Frequency.

T(n) = ∑ cost * frequency

For all statements
 TIME ANALYSIS EXAMPLE 1

• Algorithm:
Int Count = 0;
For (int i = 0 ; i < N; i ++)
If (A[i] == 0)
Count ++;

• How to determine T(n):

Operation Frequency
Variable Declaration 2
Assignment Statement 2
Less than compare N+1
Equal to compare N
Array access N
Increment N – 2N
 TIME ANALYSIS EXAMPLE 1

• Algorithm:
Int Count = 0;
For (int i = 0 ; i < N; i ++)
If (A[i] == 0)
Count ++;

• How to determine T(n):

• Suppose that, the cost for each statement is equal to 1.
• So,

T(n) = 2+2+N+1+N+N+2N
= 5+6N
 SIMPLIFICATION

• Tilde Approximation and Order of Growth are two

methods to simplify the running time equation by
ignoring insignificant terms.
• Examples:
Time Equation Tilde Approximation Order of Growth
5 + 6𝑁𝑁 ~6𝑁𝑁 𝑁𝑁
1 3 ~1/6𝑁𝑁 3 𝑁𝑁 3
𝑁𝑁 + 20 𝑁𝑁 + 16
6
1 3 4 1 𝑁𝑁 3
𝑁𝑁 + 100𝑁𝑁 3 + 56 ~ 𝑁𝑁 3
6 6
log 𝑁𝑁 + 1 ~log 𝑁𝑁 log 𝑁𝑁
𝑁𝑁 2 N 𝑁𝑁 2 𝑁𝑁 2
− ~
2 2 2
3 ~3 1
 TIME ANALYSIS EXAMPLE 1

• Algorithm:
Int Count = 0;
For (int i = 0 ; i < N; i ++)
If (A[i] == 0)
Count ++;

• How to determine T(n):

• Suppose that, the cost for each statement is equal to 1.
• So,

T(n) = 2+2+N+1+N+N+2N
= 5+6N

Time Complexity T(n) = θ(N)

 TIME ANALYSIS EXAMPLE 2

• (2-Sum) Algorithm:
Int Count = 0;
For (int i = 0 ; i < N; i ++)
For (int j = 0 ; j < N; j ++)
If (A[i] + A[j] == 0)
Count ++;

Operation Frequency
Variable Declaration 2+N
Assignment Statement 2+N
Less than compare 1/2 (N+1) (N+1)
Equal to compare 1/2 (N+1) (N+1)
Array access (N+1) (N+1)
1
Increment 𝑁𝑁 2 – 𝑁𝑁 2
2
 TIME ANALYSIS EXAMPLE 2

• (2-Sum) Algorithm:
Int Count = 0;
For (int i = 0 ; i < N; i ++)
For (int j = 0 ; j < N; j ++)
If (A[i] + A[j] == 0)
Count ++;

• How to determine T(n):

• Suppose that, the cost for each statement is equal to 1.
• So,

Time Complexity T(n) = θ(𝑁𝑁 2 )

 TIME ANALYSIS EXAMPLE 3

• (3-Sum) Algorithm:
Int Count = 0;
For (int i = 0 ; i < N; i ++)
For (int j = 0 ; j < N; j ++)
For (int k = 0 ; k < N; k ++)
If (A[i] + A[j] + A[k] == 0)
Count ++;

Proxy Statement
• Cost Model:
• Use basic operations as a proxy for running time.
Proxy Statement Frequency = 0 - 1/6𝑁𝑁 3

Time Complexity T(n) = θ(𝑁𝑁 3 )

 ORDER OF GROWTH

𝟏𝟏 < log 𝑵𝑵 < 𝑵𝑵 < 𝑵𝑵 log 𝑵𝑵 < 𝑵𝑵𝟐𝟐 < 𝑵𝑵𝟑𝟑 < ⋯ < 𝟐𝟐𝑵𝑵 < 𝑵𝑵! < 𝑵𝑵𝑵𝑵
Name Order OF Growth Example
Constant 1 Without loops
Logarithmic log 𝑵𝑵 Binary Search
Linear 𝑵𝑵 Linear Search
Linearithmic 𝑵𝑵 log 𝑵𝑵 Merge Sort
Quick Sort
Quadratic 𝑵𝑵𝟐𝟐 2-Sum
Selection Sort
Insertion Sort
Cubic 𝑵𝑵𝟑𝟑 3-Sum
Exponential 𝟐𝟐𝑵𝑵 Extremely slow and
& Never used for large
Factorial problems
 MATH REVISION

• Summation formulas:
𝑼𝑼
• ∑𝒊𝒊=𝑳𝑳 𝟏𝟏 = 𝑼𝑼 − 𝑳𝑳 + 𝟏𝟏 ( 𝑳𝑳 ≤ 𝑼𝑼)

𝒏𝒏
• ∑𝒊𝒊=𝟏𝟏 𝟏𝟏 = 𝒏𝒏

𝒏𝒏 𝒏𝒏 𝒏𝒏+𝟏𝟏 𝟏𝟏
• ∑𝒊𝒊=𝟏𝟏 𝒊𝒊 = 𝟏𝟏 + 𝟐𝟐 + ⋯ + 𝒏𝒏 = ≅ 𝒏𝒏𝟐𝟐
𝟐𝟐 𝟐𝟐

𝒏𝒏 𝒏𝒏 𝒏𝒏+𝟏𝟏 𝟐𝟐𝟐𝟐+𝟏𝟏 𝟏𝟏
• ∑𝒊𝒊=𝟏𝟏 𝒊𝒊𝟐𝟐 = 𝟏𝟏𝟐𝟐 + 𝟐𝟐𝟐𝟐 + ⋯ + 𝒏𝒏𝟐𝟐 = ≅ 𝒏𝒏𝟑𝟑
𝟔𝟔 𝟔𝟔

𝒏𝒏 𝟏𝟏
• ∑𝒊𝒊=𝟏𝟏 𝒊𝒊𝒌𝒌 = 𝟏𝟏𝒌𝒌 + 𝟐𝟐𝒌𝒌 + ⋯ + 𝒏𝒏𝒌𝒌 ≅ 𝒏𝒏𝒌𝒌+𝟏𝟏
𝒌𝒌+𝟏𝟏 !
 MATH REVISION

• Summation formulas:
𝒏𝒏
• ∑𝒊𝒊=𝟎𝟎 𝒂𝒂𝒊𝒊 = 𝒂𝒂𝟎𝟎 + 𝒂𝒂𝟏𝟏 + 𝒂𝒂𝟐𝟐 + ⋯ + 𝒂𝒂𝒏𝒏
= 𝟏𝟏 + 𝒂𝒂 + 𝒂𝒂𝟐𝟐 + ⋯ + 𝒂𝒂𝒏𝒏 Important
𝒂𝒂𝒏𝒏+𝟏𝟏 −𝟏𝟏
= (𝒂𝒂 ≠ 𝟏𝟏)
𝒂𝒂−𝟏𝟏

𝒏𝒏
• ∑𝒊𝒊=𝟏𝟏 log 𝒊𝒊 ≅ 𝒏𝒏 log 𝒏𝒏
 MATH REVISION

• Summation Rules:
𝒖𝒖 𝒖𝒖
• ∑𝒊𝒊=𝒍𝒍 𝑪𝑪 𝑎𝑎𝑖𝑖 = 𝑪𝑪 ∑𝒊𝒊=𝒍𝒍 𝑎𝑎𝑖𝑖

𝒖𝒖 𝒖𝒖 𝒖𝒖
• ∑𝒊𝒊=𝒍𝒍(𝑎𝑎𝑖𝑖 ± 𝑏𝑏𝑖𝑖 ) = ∑𝒊𝒊=𝒍𝒍 𝑎𝑎𝑖𝑖 ± ∑𝒊𝒊=𝒍𝒍 𝑏𝑏𝑖𝑖

𝒖𝒖 𝒎𝒎 𝒖𝒖
• ∑𝒊𝒊=𝒍𝒍 𝑎𝑎𝑖𝑖 = ∑𝒊𝒊=𝒍𝒍 𝑎𝑎𝑖𝑖 ± ∑𝒊𝒊=𝒎𝒎+𝟏𝟏 𝑎𝑎𝑖𝑖 (𝒍𝒍 ≤ 𝒎𝒎 ≤ 𝒖𝒖)
 MATH REVISION

• Logarithmic Properties:
• log 𝒂𝒂 𝟏𝟏 = 𝟎𝟎

• log 𝒂𝒂 𝒂𝒂 = 𝟏𝟏

• log 𝒂𝒂 𝒙𝒙𝒚𝒚 = 𝒚𝒚 log 𝒂𝒂 𝒙𝒙

• log 𝒂𝒂 𝒙𝒙𝒙𝒙 = log 𝒂𝒂 𝒙𝒙 + log 𝒂𝒂 𝒚𝒚

𝒙𝒙
• log 𝒂𝒂 = log 𝒂𝒂 𝒙𝒙 − log 𝒂𝒂 𝒚𝒚
𝒚𝒚
 MATH REVISION

• Logarithmic Properties:
• 𝒂𝒂log𝒃𝒃 𝒙𝒙 = 𝒙𝒙log𝒃𝒃 𝒂𝒂 Important

• (log 𝒃𝒃 𝒂𝒂)𝒙𝒙 = log 𝒃𝒃 𝒂𝒂𝒙𝒙

log𝒃𝒃 𝒙𝒙
• log 𝒂𝒂 𝒙𝒙 = = log 𝒂𝒂 𝒃𝒃 . log 𝒃𝒃 𝒙𝒙
log𝒃𝒃 𝒂𝒂
 EXAMPLE

• Rank the following functions by order of growth:

• 𝑛𝑛 log 𝑛𝑛 • 𝑛𝑛2 •
3
( )𝑛𝑛
2
• log 𝑛𝑛 ! • log(𝑛𝑛)!
1 • log log 𝑛𝑛
• 𝑛𝑛log log 𝑛𝑛
• 𝑛𝑛 log 𝑛𝑛
• 2log 𝑛𝑛
• • 4log 𝑛𝑛
1 • 𝑛𝑛3
• 2𝑛𝑛
• log 𝑛𝑛 • log 𝑛𝑛log 𝑛𝑛
• 𝑛𝑛!
• 22𝑛𝑛+1 • 22𝑛𝑛 • ( 2)log 𝑛𝑛
• 𝑛𝑛 • log 𝑛𝑛 • 𝑛𝑛. 2𝑛𝑛
• 𝑛𝑛 + 1 ! • 𝑒𝑒 𝑛𝑛
 EXAMPLE

• Solution:
3 𝑛𝑛
2 2𝑛𝑛+1
> 2 > 𝑛𝑛 + 1 ! > 𝑛𝑛! > 𝑒𝑒 > 𝑛𝑛. 2 > 2 > ( ) > 𝑛𝑛log log 𝑛𝑛 =
2𝑛𝑛 𝑛𝑛 𝑛𝑛 𝑛𝑛
2
log 𝑛𝑛 log 𝑛𝑛
> log 𝑛𝑛 ! > 𝑛𝑛 > 4
3 log 𝑛𝑛
= 𝑛𝑛 >
2
𝑛𝑛 log 𝑛𝑛 ≅ log(𝑛𝑛)! >
2log 𝑛𝑛 = 𝑛𝑛 > ( 12)log 𝑛𝑛 = 𝑛𝑛 > log 𝑛𝑛log 𝑛𝑛 > log 𝑛𝑛 > log 𝑛𝑛 >
log log 𝑛𝑛 > 𝑛𝑛log 𝑛𝑛 = 1

• Hint:
• 𝑛𝑛log log 𝑛𝑛 = log 𝑛𝑛log 𝑛𝑛
• 4log 𝑛𝑛 = 𝑛𝑛log 4 = 𝑛𝑛2
• 2log 𝑛𝑛 = 𝑛𝑛log 2 = 𝑛𝑛
• ( 2)log 𝑛𝑛 = 2log 𝑛𝑛 = 𝑛𝑛log 2 = 𝑛𝑛
1
• 𝑛𝑛 log 𝑛𝑛 ≅ 𝑛𝑛0 = 1