Course Coordinator

v Dr. Neetesh Kumar

v Assistant Professor,
v Department of Computer Science & Engineering (IIT-R)
v Write to me: neetesh@cs.iitr.ac.in
v Web link: https://www.iitr.ac.in/~CSE/Neetesh_Kumar
v Google Scholar: https://scholar.google.co.in/citations?user=Ut_l01AAAAAJ&hl=en
Why Should We Learn Design and Analysis of
Algorithms?

Ø Foundation of Computer Science

Ø Problem-Solving Skills
Ø Optimization and Efficiency
Ø Real-World Applications
Ø Interdisciplinary Relevance to Advanced Courses
Ø Technical Interviews and Competitions
 List of Books
Ø Cormen, T.H., Leiserson, C.E., Rivest, R.L. and Stein, C.,
“Introduction to Algorithms”, 5th Ed., Prentice-Hall of India.
Ø Sahni, S., “Data Structures, Algorithms and Applications in C++”,
WCB/McGraw-Hill.
Ø Mchugh, J.A., “Algorithmic Graph Theory”, Prentice-Hall.
Ø Quinn, M.J., “Parallel Computing Theory & Practice”, McGrawHill.
Ø Dasgupta, S., Papadimitriou, C. and Vazirani, U., “Algorithms”, Tata
McGraw-Hill.
 Source Contents Credit to:

v Dr. George Bebis (University of Nevada.)

v Dr. David Kauchak(Pomona College, Claremont, California , United
States.)
 Grading Scheme

Particular Weightage Marks (100) Remark

MTE 25% 25
ETE 40% to 50% 40 to 50
Assignments 20% to 25% 20 to 25 5 MARKS FOR
/Tutorials/ EXCEPTIONAL
Quiz
WHAT ABOUT ATTENDANCE?
 Prerequisite

Ø Data Structures
Ø Basic Mathematics
Ø C Programming or Principles of Programming
(Loops, Functions, Conditional statements, etc.)
 Design and Analysis of Algorithms
• Algorithm: a set of finite steps to solve a
particular problem
• Design: an art to design the algorithms which
minimize the cost
• Analysis: predict the cost of an algorithm in
terms of resources and performance
• Aim of this course: The theoretical study of
design and analysis of computer algorithms.
WHAT IS GOOD OR BAD ALGORITHM ?
 Our Machine Model
Generic Random Access Machine (RAM)
• Executes operations sequentially
• Set of primitive operations:
– Arithmetic. Logical, Comparisons, Function calls

• Simplifying assumption: all ops cost 1 unit

– Eliminates dependence on the speed of our computer,
otherwise impossible to verify and to compare
 ANY QUESTIONS ?

We shall begin from next session …

 The problem of sorting

Input: sequence áa1, a2, …, anñ of numbers.

Output: permutation áa'1, a'2, …, a'nñ such

that a'1 £ a'2 £ … £ a'n .

Example:
Input: 8 2 4 9 3 6
Output: 2 3 4 6 8 9
Structure of data
 Why Study Sorting Algorithms?
• There are a variety of situations that we can encounter
– Do we have randomly ordered keys?
– Are all keys distinct?
– How large is the set of keys to be ordered?
– Need guaranteed performance?

• Various algorithms are better suited to some of these

situations
 Some Definitions
• Internal Sort
– The data to be sorted is all stored in the computer’s main
memory.
• External Sort
– Some of the data to be sorted might be stored in some
external, slower, device.
• In Place Sort
– The amount of extra space required to sort the data is
constant with the input size.
 Stability
• A STABLE sort preserves relative order of records with equal
keys
Sorted on first key:

Sort file on second key:

Records with key value 3

are not in order on first
key!!
 Insertion Sort
• Idea: like sorting a hand of playing cards
– Start with an empty left hand and the cards facing down on
the table.
– Remove one card at a time from the table, and insert it into
the correct position in the left hand
• compare it with each of the cards already in the hand, from right to
left
– The cards held in the left hand are sorted
 Insertion Sort

To insert 12, we need to make

room for it by moving first 36
and then 24.
6 10 24 36

12
 Insertion Sort

6 10 24 36

12
 Insertion Sort

6 10 24 3
6

12
Example of insertion sort

8 2 4 9 3 6
Example of insertion sort

8 2 4 9 3 6
Example of insertion sort

8 2 4 9 3 6
2 8 4 9 3 6
Example of insertion sort

8 2 4 9 3 6
2 8 4 9 3 6
Example of insertion sort

8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
Example of insertion sort

8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
Example of insertion sort

8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
Example of insertion sort

8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
Example of insertion sort

8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
Example of insertion sort

8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
Example of insertion sort

8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
2 3 4 6 8 9 done
 Insertion Sort
input array

5 2 4 6 1 3
at each iteration, the array is divided in two sub-arrays:
left sub-array right sub-array

sorted unsorted
 INSERTION-SORT
Alg.: INSERTION-SORT(A) 1 2 3 4 5 6 7 8

for j ← 2 to n a1 a2 a3 a4 a5 a6 a7 a8

do key ← A[ j ] key
Insert A[ j ] into the sorted sequence A[1 . . j -1]

i←j-1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i–1
A[i + 1] ← key
• Insertion sort – sorts the elements in place
 Loop Invariant for Insertion Sort

Alg.: INSERTION-SORT(A)
for j ← 2 to n
do key ← A[ j ]
Insert A[ j ] into the sorted sequence A[1 . . j -1]

i←j-1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i–1
A[i + 1] ← key

Invariant: at the start of the for loop the elements in A[1 . . j-1] are in
sorted order
 Proving Loop Invariants
• Proving loop invariants works like induction
• Initialization (base case):
– It is true prior to the first iteration of the loop
• Maintenance (inductive step):
– If it is true before an iteration of the loop, it remains true
before the next iteration
• Termination:
– When the loop terminates, the invariant gives us a useful
property that helps show that the algorithm is correct
– Stop the induction when the loop terminates
 Loop Invariant for Insertion Sort
• Initialization:
– Just before the first iteration, j = 2:
the subarray A[1 . . j-1] = A[1], (the
element originally in A[1]) – is sorted
 Loop Invariant for Insertion Sort
• Maintenance:
– the while inner loop moves A[j -1], A[j -2], A[j -3], and so
on, by one position to the right until the proper position for
key (which has the value that started out in A[j]) is found
– At that point, the value of key is placed into this position.
 Loop Invariant for Insertion Sort
• Termination:
– The outer for loop ends when j = n + 1 Þ j-1 = n
– Replace n with j-1 in the loop invariant:
• the subarray A[1 . . n] consists of the elements originally in A[1 . . n],
but in sorted order
j-1 j

• The entire array is sorted!

Invariant: at the start of the for loop the elements in A[1 . . j-1] are in
sorted order
 Analysis of Insertion Sort
INSERTION-SORT(A) cost times
for j ← 2 to n c1 n
do key ← A[ j ] c2 n-1
Insert A[ j ] into the sorted sequence A[1 . . j -1] 0 n-1
i←j-1 c4 n-1
å
n
while i > 0 and A[i] > key c5 j =2 j
t
å
n
do A[i + 1] ← A[i] c6 j =2
(t j - 1)
å
n
i←i–1 c7 j =2
(t j - 1)
A[i + 1] ← key c8 n-1
tj: # of times the while statement is executed at iteration j

T (n) = c1n + c2 (n - 1) + c4 (n - 1) + c5 å t j + c6 å (t j - 1) + c7 å (t j - 1) + c8 (n - 1)
n n n

j =2 j =2 j =2
 Best Case Analysis
• The array is already sorted “while i > 0 and A[i] > key”
– A[i] ≤ key upon the first time the while loop test is run
(when i = j -1)

– tj = 1

• T(n) = c1n + c2(n -1) + c4(n -1) + c5(n -1) + c8(n-1)

= (c1 + c2 + c4 + c5 + c8)n - (c2 + c4 + c5 + c8)

= an - b = Q(n)
T (n) = c1n + c2 (n - 1) + c4 (n - 1) + c5 å t j + c6 å (t j - 1) + c7 å (t j - 1) + c8 (n - 1)
n n n

j =2 j =2 j =2
 Worst Case Analysis
• The array is in reverse sorted order“while i > 0 and A[i] > key”
– Always A[i] > key in while loop test
– Have to compare key with all elements to the left of the j-th
position Þ compare with j-1 elements Þ tj = j
n
n(n + 1) n
n(n + 1) n
n(n - 1)
using å
j =1
j=
2
=> å j =
j =2 2
- 1 => å ( j - 1) =
j =2 2
we have:

æ n( n + 1) ö n( n - 1) n( n - 1)
T ( n ) = c1n + c2 ( n - 1) + c4 ( n - 1) + c5 ç - 1÷ + c6 + c7 + c8 ( n - 1)
è 2 ø 2 2

= an 2 + bn + c a quadratic function of n

• T(n) = Q(n2) order of growth in n2

T (n) = c1n + c2 (n - 1) + c4 (n - 1) + c5 å t j + c6 å (t j - 1) + c7 å (t j - 1) + c8 (n - 1)
n n n

j =2 j =2 j =2
Comparisons and Exchanges in Insertion Sort

INSERTION-SORT(A) cost times

for j ← 2 to n c1 n
do key ← A[ j ] c2 n-1
Insert A[ j ] into the sorted sequence A[1 . . j -1] 0 n-1
i←j-1 » n2/2 comparisons c4 n-1
å
n
while i > 0 and A[i] > key c5 j =2 j
t

å
n
do A[i + 1] ← A[i] c6 j =2
(t j - 1)

exchanges c7 å
n
i←i–1 » n2/2 j =2
(t j - 1)

A[i + 1] ← key c8 n-1

 Insertion Sort - Summary
• Advantages
– Good running time for “almost sorted” arrays Q(n)
• Disadvantages
– Q(n2) running time in worst and average case
– » n2/2 comparisons and exchanges
 Bubble Sort
• Idea:
– Repeatedly pass through the array
– Swaps adjacent elements that are out of order

i
1 2 3 n

8 4 6 9 2 3 1
j

• Easier to implement, but slower than Insertion

sort
 Example
8 4 6 9 2 3 1 1 8 4 6 9 2 3
i=1 j i=2 j

8 4 6 9 2 1 3 1 2 8 4 6 9 3
i=1 j i=3 j

8 4 6 9 1 2 3 1 2 3 8 4 6 9
i=1 j i=4 j

8 4 6 1 9 2 3 1 2 3 4 8 6 9
i=1 j i=5 j

8 4 1 6 9 2 3 1 2 3 4 6 8 9
i=1 j i=6 j

8 1 4 6 9 2 3 1 2 3 4 6 8 9
i=1 j i=7
j
1 8 4 6 9 2 3
i=1 j
 Bubble Sort
Alg.: BUBBLESORT(A)
for i ¬ 1 to length[A]
do for j ¬ length[A] downto i + 1
do if A[j] < A[j -1]
then exchange A[j] « A[j-1]
i
8 4 6 9 2 3 1
i=1 j
 Bubble-Sort Running Time
Alg.: BUBBLESORT(A)
for i ¬ 1 to length[A] c1
do for j ¬ length[A] downto i + 1 c2
do if A[j] < A[j -1] c3
then exchange A[j] « A[j-1] c4
 Bubble-Sort Running Time
Alg.: BUBBLESORT(A)
for i ¬ 1 to length[A] c1
do for j ¬ length[A] downto i + 1 c2
Comparisons: » n2/2 do if A[j] < A[j -1] c3
Exchanges: » n2/2
then exchange A[j] « A[j-1] c4
n n n

T(n) = c1(n+1) + c2 å (n - i + 1) + c3 å (n - i ) + c4 å (n - i )
i =1
i =1 i =1
n
= Q(n) + (c2 + c2 + c4) å (n - i )
i =1
n n n
n ( n + 1) n 2
n
where å (n - i ) =å n - å i = n 2 - = -
i =1 i =1 i =1 2 2 2
Thus,T(n) = Q(n2)
Selection Sort
• Idea:
– Find the smallest element in the array
– Exchange it with the element in the first position
– Find the second smallest element and exchange it with
the element in the second position
– Continue until the array is sorted
• Disadvantage:
– Running time depends only slightly on the amount of
order in the file
 Example
8 4 6 9 2 3 1 1 2 3 4 9 6 8

1 4 6 9 2 3 8 1 2 3 4 6 9 8

1 2 6 9 4 3 8 1 2 3 4 6 8 9

1 2 3 9 4 6 8 1 2 3 4 6 8 9
 Selection Sort
Alg.: SELECTION-SORT(A)
n ← length[A] 8 4 6 9 2 3 1

for j ← 1 to n - 1
do smallest ← j
for i ← j + 1 to n
do if A[i] < A[smallest]
then smallest ← i
exchange A[j] ↔ A[smallest]
 Analysis of Selection Sort
Alg.: SELECTION-SORT(A) cost times
n ← length[A] c1 1
for j ← 1 to n - 1 c2 n
do smallest ← j c3 n-1
»n2/2 for i ← j + 1 to n c4 ånj=-11 (n - j + 1)
comparisons
do if A[i] < A[smallest] c5 å
n -1
j =1
(n - j )
»n
exchanges then smallest ← i c6 å
n -1
j =1
(n - j )

exchange A[j] ↔ A[smallest] c7 n-1

n -1 n -1 n -1
T ( n) = c1 + c2 n + c3 (n - 1) + c4 å (n - j + 1) + c5 å ( n - j ) + c6 å ( n - j ) + c7 (n - 1) = Q(n 2 )
j =1 j =1 j =2
Back UP
• vector<long long>
trial_division1(long long n) {
vector<long long> factorization; for
(long long d = 2; d * d <= n; d++) {
while (n % d == 0) {
factorization.push_back(d); n /= d;
} } if (n > 1)
factorization.push_back(n); return
factorization; }
We divide by each possible divisor dd. We can notice, that it is impossible that all prime factors of a composite
number nn are bigger than n−−√n. Therefore, we only need to test the divisors 2≤d≤n−−√2≤d≤n, which gives us the prime
factorization in O(n−−√)O(n).
The smallest divisor has to be a prime number. We remove the factor from the number, and repeat the process. If we
cannot find any divisor in the range [2;n−−√][2;n], then the number itself has to be prime.