17-08-2023

Course Objectives
Bachelor of Technology
CSE (AIML)/ICT
The objective of this course is to
• Equip the students with the concepts of complexity and analysis of algorithms.

Analysis and Design of Algorithms • Understanding of advanced data structures.

Module 1 - Introduction • Provide an understanding of different techniques used in designing algorithms for a
variety of problems.

• Equip the students with the concepts of complexity classes of different problems.
Dr. Anuj Kr. Singh
Associate Professor
Computer Science & Engineering

Thursday, August 17, 2023 Dr. Anuj Kr. Singh, Assistant Professor (III) 2

Course Outcomes Introduction

On completion of this course, the students will be able to Analysis and Design of Algorithms
• State and explain the meaning of algorithm, analysis and design; apply the algorithmic Algorithm 01
concepts to analyze a given algorithm and compute its time complexity.

• Explain and apply the concept of Divide and Conquer approach in designing algorithms;
analyze a divide and conquer algorithm; explain and apply the concept of Greedy Approach in 02 Analysis
designing algorithms for optimization problems.

• Explain and apply the concept of Dynamic Programming in designing algorithms for
optimization problems; compare and contrast the Dynamic Programming with Greedy Design 03
approach and Divide and Conquer approach.

Thursday, August 17, 2023 Dr. Anuj Kr. Singh, Assistant Professor (III) 3 17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 4
 17-08-2023

Introduction Introduction

Algorithm 01 • No formal definition. Algorithm 01 A tool for solving a well-specified computational

• Informally, an algorithm is defined as: problem.
A sequence of FINITE computational steps that transform the
input into output.
A well defined computational procedure for achieving
the desired output for a given input.
Input Algorithm Output
0 or More 1 or More
Relation between input and output is defined by problem
statement.
A well-defined computational procedure that takes some value, or
set of values, as input and produces some value, or set of values,
as output.

17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 5 17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 6

Algorithm - Properties Example – Sorting Problem

Input: A sequence of n numbers [a1, a2, … , an].

Output: A permutation or reordering [a'1, a'2, … , a'n ] of the input sequence such that
1 2 3 4 5
a'1  a'2  …  a'n .

An instance of the Sorting Problem:

Finiteness Correctness Output Definiteness Effectiveness
Must terminate Halts only at correct Produces at least Every step must be Every step must be Input: A sequence of 6 number [31, 41, 59, 26, 41, 58].
within finite number output. one output. precisely defined basic enough so that
of steps. it can be carried out
using a paper and
Expected Output: The permutation of the input [26, 31, 41, 41, 58 , 59].
pencil.

17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 7 17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 8
 17-08-2023

Analysis of Algorithm Analysis of Algorithm

• Analyzing Algorithms • Analyzing algorithms means to find out the resources that an algorithm requires.

• Analyzing – General Meaning • Which resources?

CPU Time/
o CPU Time
Running Time/
• Consider the following three persons (A, B, and C) o Memory
Time Complexity
standing outside a room. (Each one is asked to o Processor
analyze the room) o Hardware
o A – Architect o Communication Bandwidth Analyzing algorithm means to find out the
o B – Interior Designer o & Many More Running Time/Time Complexity of the Algorithm.
o C – Civil Engineer

Guess the outcome of this analysis from each of

them…

17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 9 17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 10

Running Time of Algorithm RAM Model of Computation

Uniprocessor
• Running Time of algorithm can be defined as the Number of Key Steps or Operations executed? Environment
There is only one
processor
• Key Steps are the steps of the algorithm that depends upon input size.

Input Size: n

printf (“My First program”); // does not depend upon n Count: 1 • RAM – Random Access Machine
for (i=0, i<n, i++) // depends upon n Count: n+1
{
for (j=0, j<n, j++) // depends upon n Count: n(n+1)
{ RAM Model
printf (“I Like Programming); // depends upon n Count: n2 Two Assumptions

No Concurrent
} Running Time Execution
} T(n) = (n+1) + n(n+1) + n2 Instructions are
executed sequentially

17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 11 17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 12
 17-08-2023

Analysis of Algorithms Analysis of Algorithms

Two ways of
Exact Analysis 01 • Consider Sorting Problem (Algorithm for Sorting – Insertion Sort)
Analysis

Exact Approximate Step 1

Analysis Analysis
Understanding the
Logic
Only the order of running time. Step 2
Exact expression of running time. Eg. The order of T(n) is n2 or
Eg. T(n) = 3n2 + 4n + 5 Writing the
quadratic. Algorithm
Step 3
Very lengthy, cumbersome, and is Performing the
Easier and serve our purpose.
not required for our purpose. Analysis

17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 13 17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 14

Exact Analysis of Insertion Sort Exact Analysis of Insertion Sort

Logic ALGORITHM
Algorithm written in
• Consider a game of playing cards. INSERTION_SORT (A) pseudocode
• One player distributes the cards to all others.
1. for j ← 2 to length[A]
• Each player is required to sort the cards. 2. do key ← A[j]
A collection of accounts 3. A collection of accounts
i←j−1 • Just a syntax to write algorithm
or financial transactions or financial transactions • Very much like C, Pascal etc.
4. while i > 0 and A[i] > key
5. do A[i +1] ← A[i]
Left Hand Remaining Cards
(already sorted sub-sequence) (un-sorted sub-sequence) 6. i←i−1
Independent of any
7. A[i + 1] ← key programming language
Each player picks the card from Right Hand (un-sorted sub-sequence) one by one and INSERTS it at the correct
position in the left hand (already sorted sub-sequence) .

17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 15 17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 16
 17-08-2023

Exact Analysis of Insertion Sort Exact Analysis of Insertion Sort

Example Approach
• Each instruction executed in the algorithm exactly once, takes constant time (known).

• Count how many times the instruction is executed (frequency).

• Multiply the constant time and frequency of each instruction – to find out total time taken by
A collection of accounts A collection of accounts
or financial transactions or financial
each transactions
instruction.

• Add the total time of all the instructions – to find out the exact expression of running time.

17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 17 17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 18

Exact Analysis of Insertion Sort Exact Analysis of Insertion Sort

Approach
INSERTION_SORT (A) Cost No. of times
Instruction Constant Time Count (frequency) Total Time
1. for j ← 2 to length[A] C1 (n)
1 C1 n1 C1n1
2. do key ← A[j] C2 (n-1)
2 C2 n2 C2n2
3 C3 n3 C3n3 3. i←j−1 C3 (n-1)
A collection of accounts A collection of accounts
- transactions
or financial - - - 4. or financial transactions
while i > 0 and A[i] > key C4 ?
- - - - 5. do A[i +1] ← A[i] C5 ?
m Cm nm Cmnm 6. i←i−1 C6 ?
Exact expression of running time is T(n)= C1n1 + C2n2 + C3n3 + …………. + Cmnm
7. A[i + 1] ← key C7 (n-1)

Let us assume that for each value of j step no. 4 executes tj number of times.

17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 19 17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 20
 17-08-2023

Exact Analysis of Insertion Sort Exact Analysis of Insertion Sort

INSERTION_SORT (A) Cost No. of times

• The value of the unknown will depend upon the type of input.
1. for j ← 2 to length[A] C1 (n)
2. do key ← A[j] C2 (n-1) • Types of input - <given sequence/array A to be sorted>

3. i←j−1 tj C3 (n-1) THREE CASES

4. while i > 0 and A[i] > key C4 ∑ tj o Given array A is already sorted
A collection of accounts Unknown A collection of accounts
o Giventransactions
array A is reverse-sorted
5. or financial transactions
do A[i +1] ← A[i] C5 ∑ (tj−1) or financial
o Given array A is partially sorted
6. i←i−1 C6 ∑ (tj−1)
7. A[i + 1] ← key C7 (n-1)
Running time is
T(n)= C1 (n) + C2 (n-1) + C3 (n-1) + C4 ∑ tj + C5 ∑ (tj−1)+C6 ∑ (tj−1)+C7 (n-1) Equation I
17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 21 17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 22

Exact Analysis of Insertion Sort Exact Analysis of Insertion Sort

CASE 1 Given array A is already sorted Best Case Consider the Array
CASE 2 Given array A is Reverse Sorted Worst Case Consider the Array
12 10 9 6 4
INSERTION_SORT (A) INSERTION_SORT (A)
5 8 10 12 15
1. for j ← 2 to length[A] 1. for j ← 2 to length[A]
2. do key ← A[j] 2. do key ← A[j]
3. i←j−1 3. i←j−1
A collection of accounts
4. or financial transactions
while i > 0 and A[i] > key Inner loop is never iterated 4. while i > 0 and A[i] > key Inner loop is iterated maximum number of times.
5. do A[i +1] ← A[i] 5. do A[i +1] ← A[i]
For each value of j the control at step 4 comes exactly ONCE. For each value of j the control at step 4 comes exactly j times.
6. i←i−1 6. i←i−1
7. A[i + 1] ← key 7. A[i + 1] ← key
tj = 1 Now substitute this value in Equation I. tj = j Now substitute this value in Equation I.

T(n)= C1 (n-1) + C2 (n-1) + C3 (n-1) + C4 ∑ tj + C5 ∑ (tj−1)+C6 ∑ (tj−1)+C7 (n-1) T(n)= C1 (n-1) + C2 (n-1) + C3 (n-1) + C4 ∑ 𝑗 + C5 ∑ (j−1)+C6 ∑ (j−1)+C7 (n-1) T(n) = A(n2)+B(n)+C
T(n) = A(n)+B
(where A and B are constants) =C1 (n-1) + C2 (n-1) + C3 (n-1) + C4 (n-1) + ? + ? +C7 (n-1) (where A, B, and C are
=C1 (n-1) + C2 (n-1) + C3 (n-1) + C4 (n-1) + 0 + 0 +C7 (n-1)
[C4 ∑ 𝑗 = n(n+1)/2 -1] constants)
= (C1 + C2 + C3 + C4 +C7 ) n - (C1 + C2 + C3 + C4 +C7 )
17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 23 17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 24
 17-08-2023

Exact Analysis of Insertion Sort Approximate Analysis of Insertion Sort

CASE 3 Given array A is Partially Sorted Average Case Consider the Array
INSERTION_SORT (A) Cost Order No. of times
INSERTION_SORT (A) 6 10 9 3 12
1. for j ← 2 to length[A] C1 (n)
1. for j ← 2 to length[A] 2. do key ← A[j] C2 (n)
2. do key ← A[j]
3. i←j−1 C3 (n)
3. i←j−1
4. while i > 0 and A[i] > key C4 ?
4. while i > 0 and A[i] > key Inner loop is iterated average no. of times. 5. do A[i +1] ← A[i] C5 ?
5. do A[i +1] ← A[i] 6. i←i−1 C6 ?
For each value of j the control at step 4 comes (j+1)/2 number
6. i←i−1 of times – on average 7. A[i + 1] ← key C7 (n)
7. A[i + 1] ← key

tj = (j+1)/2 Now substitute this value in Equation I. Best Case Worst Case Average Case
T(n) = A(n2)+B(n)+C Step 4 executes min. no. of times Step 4 executes max. no. of times Step 4 executes avg. no. of times
(where A, B, and C are (n-1) (n-1)Xn (n-1)X(n+1)/2
constants) Order of Running Time is n. Order of Running Time is n2. Order of Running Time is n2.
17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 25 17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 26

Thanks

E-mail: anuj.singh@adaniuni.ac.in
URL: www.anujsingh.co.in
17-08-2023 Dr. Anuj Kr. Singh, Associate Professor 27