Analysis of Algorithms

CSC – 333

Instructor
Dr.Eid Rehman
Faculty of Engineering and Information Technology
Department of Software Engineering
Foundation University Rawalpindi Campus
 Course Introduction 2

1 2 3
Foundations Sorting and Data Structures
Order Statistics

4 5 6
Advanced Design and Graph Algorithms Selected Topics
Analysis Techniques
1 – The Role of Algorithms in
Computing
 Contents 4

• The Role of Algorithms in Computing

• Algorithm
• Analysis of algorithms
• Why study algorithms?
• Why analyze algorithms?
• Algorithms as a technology
• Characteristics of algorithm
• Algorithm vs. Program
 Algorithm 5

• Set of steps to accomplish a task

• Step by step method to solve a problem
• Well-defined computational procedure
• Takes value/set of values as input and
• Performs a sequence of specified actions
• Produces value/set of values as output
An algorithm is a sequence of finite
number of computational steps
that transform the input into the output
 Algorithm 6

• Problems solved by algorithms

• Sorting
• Searching
• Matching
• Machine Learning
• Geometric
• Other
 Analysis of Algorithms 7

• Analysis
• Process of breaking a complex topic into smaller parts
• In order to gain a better understanding of it

Analysis of algorithms is the

theoretical study of
performance and resource usage of
the computer programs
 Analysis of Algorithms 8

• What is more important than performance?

• Correctness
• Functionality
• Simplicity
• Extensibility
• Modularity
• Reliability
• Maintainability
• User-friendliness
• Robustness
 Why Study Algorithms? 9

• Algorithms
• Important for all other branches of computer science
• Routing, Cryptography, Graphics, Databases, Bioinformatics
• Plays a key role in modern technological innovation
• Help us to understand scalability
• Language for talking about program behavior
• Performance
• Performance is the currency of computing
• Feasible vs. Impossible
• Speed is fun
• A real blend of precision and creativity
 Why Analyze Algorithms? 10

• Different algorithms can be designed to solve same problem

• Performance analysis of algorithm helps to determine:
• Time required to produce the output Best?
• Space required to execute
Algorithm 1
• Choose the best algorithm to solve the problem

Problem Algorithm 2

Algorithm 3
 Algorithms as a Technology 11

• Suppose computers are ideally fast and memory free

• Still need to study, design, learn, and use algorithms?
• Yes, it is still desirable that the solution method terminates and
• Produces the correct output
• Bounding resources for computing are:
• Computing time
• Space in memory
 Algorithms as a Technology 12

• Rapid advances in technologies such as:

• Computer architectures and Graphical User Interfaces
• Integrated Web technologies and Wireless networking
• Applications rely heavily on algorithms
• Does the application rely on hardware?
• Hardware design used algorithms
• Does the application rely on GUI?
• GUI design relies on algorithms
• Does the application rely on networking?
• Routing in networks relies heavily on algorithms
 Characteristics of Algorithm 13

1. Input
• An algorithm has zero or more inputs from a specified set of values
2. Output
• An algorithm has one or more inputs from a specified set of values
3. Definiteness
• Each step of an algorithm must be precisely defined
4. Finiteness
• An algorithm must always terminate after a finite number of steps
5. Effectiveness
• Each step of an algorithm must be performed in finite amount of time
6. Generality
• An algorithm should be applicable for all problems of a desired form
 Algorithm vs. Program 14

• Algorithm
• Defines specific steps required to solve a problem
• Program
• A computer program is the implementation of an algorithm
Algorithm Program
Design Implementation
Domain Knowledge Programming Knowledge
Any language Programming language
Hardware/OS Independent Hardware/OS Dependent
Analysis Testing
2 – Getting Started
 Contents 16

• Getting Started
• Insertion Sort
• Analyzing Algorithms
• Designing Algorithms
 Insertion Sort 17

• Input: A sequence of n numbers ⟨𝑎1, 𝑎2, … , 𝑎𝑛⟩

• Output: A permutation of ⟨𝑎’1, 𝑎’2, … , 𝑎’𝑛⟩ of the input
sequence such that 𝑎’1 ≤ 𝑎’2 ≤ ⋯ ≤ 𝑎’𝑛
• Solution Intuition
1 i j n
A=
key
Sorted
 Insertion Sort 18

• Pseudocode
INSERTIONSORT(A)
1 for j = 2 to A.length
2 key = A[i]
3 i = j – 1
4 while i > 0 and A[i] > key
5 A[i+1] = A[i]
6 i = i – 1
7 A[i+1] = key
19
20
 Analyzing Algorithms 21

• Analyzing refers to predict resources an algorithm requires

• Time taken by an algorithm grows with the size of input
• Running time of a program is a function of the size of its input
• Input Size
• Number of items in the input
• Running Time
• Number of primitive operations/steps executed
 Analyzing Algorithms 22

• Running time
• Depends on the input
• For instance, an already sorted sequence as input
• Depends on the input size
• For instance, a sequence of 6 elements vs. 6 × 109 elements
• Upper bounds are generally desired
• Guarantee to user
 Analyzing Algorithms 23

• Kinds of analysis
• Worst Case: (usually)
• 𝑇(𝑛) = maximum time of algorithm on any input of size 𝑛
• Provides an upper bound on running time
• Average Case: (sometimes)
• 𝑇(𝑛) = expected time of algorithm over all input of size 𝑛
• Need assumption of statistical distribution of inputs
• Provides an prediction about running time
• Best Case: (bogus)
• Cheat with a slow algorithm which works fast on some input
• Provides a lower bound on running time
 Analyzing Algorithms 24

• Insertion Sort – Analysis

INSERTIONSORT(A) COST TIMES where
1 for j = 2 to A.length - - - - - - - - - - - - - - c1 𝑛  𝑛 is the input size
 𝑡𝑗 is the number of
2 key ← A[j] - - - - - - - - - - - - - - - - - - - - - - - - - c2 𝑛−1 times the while
loop is executed for
3 i ← j – 1 - - - - - - - - - - - - - - - - - - - - - - - - - - c3 𝑛−1 that value of 𝑗
𝒏
4 while i > 0 and A[i] > key - - - - c4 ෍
𝒋=𝟐
𝑡𝑗

5 A[i+1] ← A[i] - - - - - - - - - - - - - - - - - c5 ෍
𝒋=𝟐
(𝑡𝑗 −1)

6 i ← i – 1 - - - - - - - - - - - - - - - - - - - - - - - c6 ෍
𝒋=𝟐
(𝑡𝑗 −1)

7 A[i+1] ← key - - - - - - - - - - - - - - - - - - - - - - c7 𝑛−1

 Analyzing Algorithms 25

• Insertion Sort – Analysis

𝑛
𝑇𝑜𝑡𝑎𝑙 𝐶𝑜𝑠𝑡 = σ𝑖=1(𝐶𝑜𝑠𝑡𝑖 × 𝑇𝑖𝑚𝑒𝑠𝑖 )
where n = number of steps

𝒏 𝒏 𝒏
𝑇(𝑛) = 𝑐1 × 𝑛 + 𝑐2 × 𝑛 − 1 + 𝑐3 × 𝑛 − 1 + 𝑐4 × ෍ 𝑡𝑗 + 𝑐5 × ෍ (𝑡𝑗 − 1) + 𝑐6 × ෍ (𝑡𝑗 − 1) + 𝑐7 × (𝑛 − 1)
𝒋=𝟐 𝒋=𝟐 𝒋=𝟐
 Analyzing Algorithms 26

• Insertion Sort – Analysis

• Best Case: Input is already sorted
• 𝑡𝑗 = 1 in line 4
𝑇 𝑛 = 𝑐1 × 𝑛 + 𝑐2 × 𝑛 − 1 + 𝑐3 × 𝑛 − 1 + 𝑐4 × 𝑛 − 1 + 𝑐7 × 𝑛 − 1

𝑇 𝑛 = 𝑐1 𝑛 + 𝑐2 𝑛 − 𝑐2 + 𝑐3 𝑛 − 𝑐3 + 𝑐4 𝑛 − 𝑐4 + 𝑐7 𝑛 − 𝑐7

𝑇 𝑛 = (𝑐1 + 𝑐2 + 𝑐3 + 𝑐4 + 𝑐7 )𝑛 − (𝑐2 − 𝑐3 − 𝑐4 − 𝑐7 )

𝑇 𝑛 = 𝐴𝑛 + 𝐵

Running Time Order of Growth

𝑻(𝒏) = 𝑨𝒏 + 𝑩 𝑻(𝒏) = 𝑶(𝒏)
 Analyzing Algorithms 27

• Insertion Sort – Analysis

• Worst Case: Input is reverse sorted
• 𝑡𝑗 = 𝑗 for 𝑗 = 2, 3, … , 𝑛 in line 4
𝑛(𝑛 + 1) 𝑛(𝑛 − 1) 𝑛(𝑛 − 1)
𝑇 𝑛 = 𝑐1 𝑛 + 𝑐2 × 𝑛 − 1 + 𝑐3 × 𝑛 − 1 + 𝑐4 × − 1 + 𝑐5 × + 𝑐6 × + 𝑐7 × 𝑛 − 1
2 2 2

𝑐4 𝑛2 𝑐4 𝑛 𝑐5 𝑛2 𝑐5 𝑛 𝑐6 𝑛2 𝑐6 𝑛
𝑇 𝑛 = 𝑐1 𝑛 + 𝑐2 𝑛 − 𝑐2 + 𝑐3 𝑛 − 𝑐3 + + + 𝑐4 + + + + + 𝑐7 𝑛 − 𝑐7
2 2 2 2 2 2

𝑐4 𝑐5 𝑐6 𝑐4 𝑐5 𝑐6
𝑇 𝑛 = + + 𝑛2 + 𝑐1 + 𝑐2 + 𝑐3 + + + 𝑛 + (𝑐2 + 𝑐3 + 𝑐4 + 𝑐7 )
2 2 2 2 2 2

𝑇 𝑛 = 𝐴𝑛2 + 𝐵𝑛 + 𝐶

Running Time Order of Growth

𝑻(𝒏) = 𝑨𝒏𝟐 + 𝑩𝒏 + 𝑪 𝑻(𝒏) = 𝑶(𝒏𝟐 )
3 – Growth of Functions
 Contents 29

• Growth of Functions
• Asymptotic Notations
• Standard Notations and Common Functions
 Asymptotic Notations 30

• Topic from Mathematics

• Because Algorithm Complexity express in term of

function with input

• So function from Math's

• To find the class of function

• Note; It does not denote complexity of algorithm

31
 Asymptotic Notations 32

• How do we compare algorithms?

• Compare execution time
• Not good – time is specific to a particular computer
• Count the number of statements executed
• Not good – number of statements vary with the programming language,
programming style, etc.
Then, How ?
Through Function with input
For function analysis we need

Asymptotic Analysis
 Asymptotic Notations 33

• Asymptotic Analysis
• In mathematics, also known as asymptotic
• Method of describing limiting behavior
• Asymptotic notations describe the class of function
• For example
• 𝑓(𝑛) = 𝑛2 + 3𝑛
• 𝑛 becomes very large, 3𝑛 becomes insignificant compared to 𝑛2
• Function 𝑓(𝑛) is said to be asymptotically equivalent to 𝑛2
𝑓(𝑛) ~ 𝑛2

𝑓(𝑛) is asymptotic to 𝑛2
 Asymptotic Notations 34

• Express running time as a function of input size 𝑛

• Asymptotic functions are independent of
• Machine time
• Programming language
• Programming style
• Define the complexity of an algorithm in terms of functions
• Function domains are the set of natural numbers 𝑁 = {1, 2, … }
• Describe behavior of functions in the form of limit
 Asymptotic Notations 35

• Classification of Time Functions

𝒇(𝒏) Classification
𝟏 Constant – run time is fixed, and does not depend upon 𝒏
𝒍𝒐𝒈 𝒏 Logarithmic – when 𝒏 increases, so does the run time, but much slower. Common
in programs which solve large problems by transforming them into smaller problems

𝒏 Linear – run time varies directly with 𝒏

𝒏 𝒍𝒐𝒈 𝒏 When 𝒏 doubles, the run time slightly more than doubles. Common in programs
which break a problem down into smaller sub-problems, solves them independently,
then combines solutions
𝒏𝟐 Quadratic – when 𝒏 doubles, the run time increases fourfold. Practical only for
small problems; typically the program processes all pairs of input (e.g., in a double
nested loop)
𝒏𝟑 Cubic – when 𝒏 doubles, the run time increases eightfold

𝟐𝒏 Exponential – when 𝒏 doubles, the run time squares. This is often the result of a
natural, “brute force” solution
 Asymptotic Notations 36

• Comparison of Time Functions

1 < log 𝑛 < 𝑛 < 𝑛 log 𝑛 < 𝑛2 < 𝑛3 < 2𝑛
𝒍𝒐𝒈 𝒏 𝒏 𝒏 𝒍𝒐𝒈 𝒏 𝒏𝟐 𝒏𝟑 𝟐𝒏
0 1 0 1 1 2
1 2 2 4 8 4
2 4 8 16 64 16
3 8 24 64 512 256
4 16 64 256 4096 65536
5 32 160 1024 32768 ~𝟏𝟎𝟗
 Asymptotic Notations 37

• Comparison of Time Functions

𝟐𝒏

𝒏𝟑

𝑻(𝒏) 𝒏𝟐

𝐥𝐨𝐠 𝒏

𝒏
 Asymptotic Notations 38

• Types of Notations
• Big Oh 𝑶 Upper Bound
• Theta () Average Bound
• Big Omega () Lower Bound
• Small Oh (𝒐)
• Small Omega (𝝎)
39
 Asymptotic Notations 40

• 𝑶-notation
• For a function f 𝑛 = 𝑶(𝒈(𝒏)) is defined as:
𝒇 𝒏 : ∃ 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠 𝑐 𝑎𝑛𝑑 𝑛0 , 𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡
𝑶 𝒈 𝒏 =
0 ≤ 𝑓 𝑛 ≤ 𝑐𝑔 𝑛 ∀ 𝑛 ≥ 𝑛0

• 𝑐 is the multiplicative constant, and 𝑛0 is the threshold

• Intuitively, the set of all functions whose rate of growth is the
same as or lower than that of 𝑔 𝑛
• 𝑔(𝑛) an is an asymptotic upper bound for 𝑓(𝑛)
 Asymptotic Notations 41

• 𝑶−notation

𝒇(𝒏) = 𝑶(𝒈(𝒏))

• Indicates 𝑓(𝑛) is a member of

the set 𝑂(𝑔 𝑛 )
• 𝑔 𝑛 includes all of the upper
bound functions
• The closest to the original
function is preferred
 Asymptotic Notations 42

• 𝑶−notation – Example
𝒇 𝒏 = 𝟐𝒏 + 𝟑, 𝑛0 = 1
2𝑛 + 3 = 𝑂(𝑔 𝑛 )
2𝑛 + 3 ≤ 10 𝑛 ∀𝑛≥1
𝑐 = 10 and 𝑔(𝑛) = 𝑛
OR write something such that LHS ≤ RHS

2𝑛 + 3 ≤ 7𝑛 ∀𝑛≥1
OR 𝑐 = 7 and 𝑔(𝑛) = 𝑛

2𝑛 + 3 ≤ 2𝑛 + 3𝑛
2𝑛 + 3 ≤ 5𝑛 ∀𝑛≥1
𝑐 = 5 and 𝑔(𝑛) = 𝑛
We can say that for that f(n)= 𝑂 (𝑛2 )
Because f(n)< all 1 < log 𝑛 < 𝑛 < 𝑛 log 𝑛 < 𝑛2 < 𝑛3 <
43
 8
Asymptotic Notations 44

• -notation
• For a function 𝑔(𝑛), the (𝒈(𝒏)) is defined as:
𝒇 𝒏 : ∃ 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠 𝑐 𝑎𝑛𝑑 𝑛0 , 𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡
 𝒈 𝒏 =
0 ≤ 𝑐𝑔 𝑛 ≤ 𝑓 𝑛 ∀ 𝑛 ≥ 𝑛0

• 𝑐 is the multiplicative constant, and 𝑛0 is the threshold

• Intuitively, the set of all functions whose rate of growth is the
same as or higher than that of 𝑔(𝑛)
• 𝑔(𝑛) an is an asymptotic lower bound for 𝑓(𝑛)

3 – Growth of Functions Malik Khizar Hayat

 8
Asymptotic Notations 45

• -notation

𝒇(𝒏) = (𝒈(𝒏))

• Indicates 𝑓(𝑛) is a member of

the set (𝑔 𝑛 )
• 𝑔 𝑛 includes all of the lower
bound functions
• The closest to the original
function is preferred

3 – Growth of Functions Malik Khizar Hayat

 8
Asymptotic Notations 46

• -notation – Example
𝒇 𝒏 = 𝟐𝒏 + 𝟑, 𝑛0 = 1
2𝑛 + 3 = (𝑔 𝑛 )
2𝑛 + 3 ≥ 1𝑛 ∀𝑛≥1 𝑐 = 1 and 𝑔(𝑛) = 𝑛
OR
2𝑛 + 3 ≥ 1 log 𝑛 ∀𝑛≥1 𝑐 = 1 and 𝑔(𝑛) = log 𝑛

3 – Growth of Functions Malik Khizar Hayat

47
 Asymptotic Notations 48

• -notation
• For a function 𝑔(𝑛), the (𝒈(𝒏)) is defined as:
𝒇 𝒏 : ∃ 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠 𝑐1 , 𝑐2 , 𝑎𝑛𝑑 𝑛0 , 𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡
 𝒈 𝒏 =
𝑐1 𝑔 𝑛 ≤ 𝑓 𝑛 ≤ 𝑐2 𝑔 𝑛 ∀ 𝑛 ≥ 𝑛0

• 𝑐1 , 𝑐2 are the multiplicative constants, and 𝑛0 is the threshold

• Intuitively, the set of all functions whose rate of growth is the
same as that of 𝑔(𝑛)
• 𝑔(𝑛) an is an asymptotic tight bound for 𝑓(𝑛)
 Asymptotic Notations 49

• -notation

𝒇(𝒏) = (𝒈(𝒏))

• Indicates 𝑓(𝑛) is a member of

the set (𝑔 𝑛 )
• 𝑔 𝑛 includes only the tight
bound functions
• Only the original function is
preferred
 Asymptotic Notations 50

• −notation – Example

𝒇 𝒏 = 𝟐𝒏 + 𝟑, 𝑛0 = 1
2𝑛 + 3 = (𝑔 𝑛 )
1𝑛 ≤ 2𝑛 + 3 ≤ 5𝑛 ∀𝑛≥1 𝑐1 = 1, 𝑐2 = 5 and 𝑔(𝑛) = 𝑛
 Asymptotic Notations 51

• 𝒐-notation
• For a function 𝑔(𝑛), the 𝒐(𝒈(𝒏)) is defined as:
𝒇 𝒏 : 𝑓𝑜𝑟 𝑎𝑛𝑦 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑐 > 0,
𝒐 𝒈 𝒏 = ∃ 𝑎 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑛0 > 0, 𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡
0 ≤ 𝑓 𝑛 < 𝑐𝑔 𝑛 ∀ 𝑛 ≥ 𝑛0

• 𝑐 is the multiplicative constant, and 𝑛0 is the threshold

• Intuitively, the 𝑓(𝑛) becomes insignificant relative to 𝑔(𝑛) when 𝑛
approaches to infinity
𝑓(𝑛)
lim =0
𝑛→∞ 𝑔 𝑛
 Asymptotic Notations 52

• 𝒐-notation – Example
• Used to denote an upper bound that is not asymptotically tight

2𝑛2 = 𝑶(𝑛2 ) is asymptotically tight

2𝑛 = 𝑶(𝑛2 ) is not asymptotically tight
Therefore
2𝑛 = 𝒐(𝑛2 ) but,
2𝑛2 ≠ 𝒐(𝑛2 )
 Asymptotic Notations 53

• 𝝎-notation
• For a function 𝑔(𝑛), the 𝝎(𝒈(𝒏)) is defined as:
𝒇 𝒏 : 𝑓𝑜𝑟 𝑎𝑛𝑦 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑐 > 0,
𝝎 𝒈 𝒏 = ∃ 𝑎 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑛0 > 0, 𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡
0 ≤ 𝑐𝑔 𝑛 < 𝑓 𝑛 ∀ 𝑛 ≥ 𝑛0

• 𝑐 is the multiplicative constant, and 𝑛0 is the threshold

• Intuitively, the 𝑓(𝑛) becomes arbitrarily large relative to 𝑔(𝑛)
when 𝑛 approaches to infinity
𝑓(𝑛)
lim =∞
𝑛→∞ 𝑔 𝑛
 Asymptotic Notations 54

• 𝝎-notation – Example
• Used to denote a lower bound that is not asymptotically tight
2𝑛2 = (𝑛2 ) is asymptotically tight
2𝑛2 = (𝑛) is not asymptotically tight
Therefore
2𝑛2 = 𝝎(𝑛) but,
2𝑛2 ≠ 𝝎(𝑛2 )
 Asymptotic Notations 55

• Comparison of Notations
Let 𝑎 belongs to 𝑓(𝑛) and 𝑏 to 𝑔(𝑛)
𝑓↔𝑔≈𝑎↔𝑏
𝑓(𝑛) = 𝑶(𝑔(𝑛)) ≈ 𝑎 ≤ 𝑏
𝑓 𝑛 = 𝑔 𝑛 ≈𝑎≥𝑏
𝑓 𝑛 = 𝑔 𝑛 ≈𝑎=𝑏
𝑓 𝑛 =o 𝑔 𝑛 ≈𝑎<𝑏
𝑓 𝑛 =w 𝑔 𝑛 ≈𝑎>𝑏
 Asymptotic Notations 56

• Limits of Notations
𝑓(𝑛)
0 < lim <∞ 𝑓 𝑛 ∈  𝑔 𝑛
𝑛→∞ 𝑔(𝑛)
𝑓(𝑛)
lim <∞ 𝑓(𝑛) ∈ 𝑶(𝑔 𝑛 )
𝑛→∞ 𝑔(𝑛)
𝑓(𝑛)
0< lim 𝑓(𝑛) ∈ (𝑔 𝑛 )
𝑛→∞ 𝑔(𝑛)
𝑓(𝑛)
lim =0 𝑓(𝑛) ∈ o(𝑔 𝑛 )
𝑛→∞ 𝑔(𝑛)
𝑓(𝑛)
lim =∞ 𝑓(𝑛) ∈ w(𝑔 𝑛 )
𝑛→∞ 𝑔(𝑛)
 Asymptotic Notations 57

• Properties of Notations
• Transitivity
𝑓 𝑛 = 𝑔 𝑛 and 𝑔 𝑛 =  ℎ 𝑛 imply f 𝑛 =  ℎ 𝑛
𝑓 𝑛 =𝑶 𝑔 𝑛 and 𝑔 𝑛 = 𝑶 ℎ 𝑛 imply f 𝑛 = 𝑶 ℎ 𝑛
𝑓 𝑛 = 𝑔 𝑛 and 𝑔 𝑛 =  ℎ 𝑛 imply f 𝑛 =  ℎ 𝑛
𝑓 𝑛 =𝒐 𝑔 𝑛 and 𝑔 𝑛 = 𝒐 ℎ 𝑛 imply f 𝑛 = 𝒐 ℎ 𝑛
𝑓 𝑛 =w 𝑔 𝑛 and 𝑔 𝑛 = w ℎ 𝑛 imply f 𝑛 = w ℎ 𝑛
 Asymptotic Notations 58

• Properties of Notations
• Transitivity – Example

𝑓 𝑛 = 𝑛 , 𝑔 𝑛 = 𝑛2 and ℎ 𝑛 = 𝑛3
𝑛 = 𝑶(𝑛2 ) and 𝑛2 = 𝑶(𝑛3 ) imply 𝑛 = 𝑶 𝑛3
 Asymptotic Notations 59

• Properties of Notations
• Reflexivity
𝑓 𝑛 = 𝑓 𝑛
𝑓 𝑛 =𝑶 𝑓 𝑛
𝑓 𝑛 = 𝑓 𝑛

• Example

𝑓 𝑛 = 𝑛2  𝑶(𝑛2 )
 Asymptotic Notations 60

• Properties of Notations
• Symmetric

𝑓 𝑛 = 𝑔 𝑛 if and only if 𝑔 𝑛 =  𝑓 𝑛

• Example

𝑓 𝑛 = 𝑛2 and 𝑔 𝑛 = 𝑛2 then
𝑓 𝑛 =  𝑛2 and 𝑔 𝑛 =  𝑛2
 Asymptotic Notations 61

• Properties of Notations
• Transpose Symmetric

𝑓 𝑛 =𝑶 𝑔 𝑛 if and only if 𝑔 𝑛 =  𝑓 𝑛
𝑓 𝑛 =𝒐 𝑔 𝑛 if and only if 𝑔 𝑛 = w 𝑓 𝑛

• Example

𝑓 𝑛 = 𝑛 and 𝑔 𝑛 = 𝑛2 then
𝑛 is 𝑶 𝑛2 and 𝑛2 =  𝑛 then
 Asymptotic Notations 62

• Relation between 𝑶 and 

• If the upper bound and the lower bound of a function is same
• Then, the tight of the function is also same
𝑓 𝑛 = 𝑔 𝑛 if and only if 𝑓 𝑛 = 𝑶 𝑔 𝑛 and 𝑓 𝑛 =  𝑔 𝑛

• Example
𝑓 𝑛 = 2𝑛2 + 3 and 𝑔 𝑛 = 𝑛2 then
𝑓 𝑛 = 𝑶 𝑛2 and 𝑓(𝑛) =  𝑛2 therefore, 𝑓 𝑛 =  𝑛2
 Asymptotic Notations 63

• Equations and Inequalities

• Asymptotic notations can be used within equations/inequalities
• For example,
2𝑛2 + 3𝑛 + 1 = 2𝑛2 +  𝑛

• Help eliminate inessential detail and clutter in equations

• When interested in asymptotic behavior
• No need to specify all the lower-order terms exactly
4 – Divide-and-Conquer
65
 Contents 66

• Divide-and-Conquer
• Recurrences
• Recurrences Solution Methods
• The Substitution Method
• The Recursion-tree Method
 Recurrences 67

• An equation or inequality that describes a function in terms

of its value on smaller input(s)
• An equation that recursively defines a sequence where the
next term is a function of the previous term(s)
• Example 1
𝑆 = 1, 4, 7, 10, 13, … Sequence
𝑆1 = 1 Initial Condition
𝑆𝑛 = 𝑆𝑛−1 + 3 Recurrence Relation
 Recurrences 68

• An equation or inequality that describes a function in terms

of its value on smaller input(s)
• An equation that recursively defines a sequence where the
next term is a function of the previous term(s)
• Example 2
𝑎𝑘 = 2𝑎𝑘−1 + 𝑘 ∀ 𝑘 ≥ 2 Recurrence Relation
𝑎1 = 1 Initial Condition
𝑎 = 1, 4, 11, 26, … Sequence
69
70
71
72
73