Designn and Analysis of Algorithm Unit-I
Designn and Analysis of Algorithm Unit-I
UNIT 1 Introduction
SYLLABUS
input data.
•Output: The algorithm produces the desired output.
manageable steps.
•Optimizing solutions: Algorithms find the best or near-optimal
solutions to problems.
•Automating tasks: Algorithms can automate repetitive or complex
1. Analysis of Algorithms
Analysis of Algorithms is the process of evaluating the efficiency of
algorithms, focusing mainly on the time and space complexity. This
helps in evaluating how the algorithm's running time or space
requirements grow as the size of input increases.
2. Mathematical Algorithms
Mathematical algorithms are used for analyzing and optimizing data
structures and algorithms. Knowing basic concepts
like divisibility, LCM, GCD, etc. can really help you understand how
data structures work and improve your ability to design efficient
algorithms.
3. Bitwise Algorithms
Bitwise algorithms are algorithms that operate on individual bits of
numbers. These algorithms manipulate the binary representation of
numbers like shifting bits, setting or clearing specific bits of a
number and perform bitwise operations (AND, OR, XOR). Bitwise
algorithms are commonly used in low-level programming,
cryptography, and optimization tasks where efficient
manipulation of individual bits is required.
4. Searching Algorithms
Searching Algorithms are used to find a specific element or item in a
collection of data. These algorithms are widely used to retrieve data
efficiently from large datasets.
5. Sorting Algorithms
Sorting algorithms are used to arrange the elements of a list in
a specific order, such as numerical or alphabetical. It organizes the
items in a systematic way, making it easier to search for and access
specific elements.
6. Recursion
Recursion is a programming technique where a function calls
itself within its own definition. It is usually used to solve problems
that can be broken down into smaller instances of the same problem.
7. Backtracking Algorithm
Backtracking Algorithm is derived from the Recursion algorithm,
with the option to revert if a recursive solution fails, i.e. in case a
solution fails, the program traces back to the moment where it failed
and builds on another solution. So basically it tries out all the possible
solutions and finds the correct one.
9. Greedy Algorithm
Greedy Algorithm builds up the solution one piece at a time and
chooses the next piece which gives the most obvious and immediate
benefit i.e., which is the most optimal choice at that moment. So
the problems where choosing locally optimal also leads to the global
solutions are best fit for Greedy.
Given two algorithms for a task, how do we find out which one is better?
One naive way of doing this is – to implement both the algorithms and run the two
programs on your computer for different inputs and see which one takes less time. There
are many problems with this approach for the analysis of algorithms.
•It might be possible that for some inputs, the first algorithm performs better than the
second. And for some inputs second performs better.
•It might also be possible that for some inputs, the first algorithm performs better on
one machine, and the second works better on another machine for some other inputs.
Asymptotic Analysis is the big idea that handles the above issues in analyzing algorithms.
In Asymptotic Analysis, we evaluate the performance of an algorithm in terms of input size
(we don’t measure the actual running time). We calculate, order of growth of time taken
(or space) by an algorithm in terms of input size. For example linear search grows linearly
and Binary Search grows logarithmically in terms of input size.
For example, let us consider the search problem (searching a given item) in a sorted
array.
The solution to above search problem includes:
•Linear Search (order of growth is linear)
•Binary Search (order of growth is logarithmic).
To understand how Asymptotic Analysis solves the problems mentioned above in analyzing
algorithms,
•let us say:
We run the Linear Search on a fast computer A and
Binary Search on a slow computer B and
•For small values of input array size n, the fast computer may take less time.
•But, after a certain value of input array size, the Binary Search will definitely start taking
less time compared to the Linear Search even though the Binary Search is being run on a
slow machine. Why? After certain value, the machine specific factors would not matter as
the value of input would become large.
•The reason is the order of growth of Binary Search with respect to input size is logarithmic
while the order of growth of Linear Search is linear.
•So the machine-dependent constants can always be ignored after a certain
value of input size.
•Let’s say the constant for machine A is 0.2 and the constant for B is 1000 which means
that A is 5000 times more powerful than B.
Input Size Running time on A Running time on B
10 2 sec ~1h
Complexities of an Algorithm
The complexity of an algorithm computes the amount of time and spaces required
by an algorithm for an input of size (n). The complexity of an algorithm can be
divided into two types. The time complexity and the space complexity.
return false;
}
// Driver Code
int main()
{
// Given Input
int a[] = { 1, -2, 1, 0, 5 };
int z = 0;
int n = sizeof(a) / sizeof(a[0]);
// Function Call
if (findPair(a, n, z))
cout << "True";
else
cout << "False";
return 0;
}
Output
False
Assuming that each of the operations in the computer takes approximately constant time, let it be c.
The number of lines of code executed actually depends on the value of Z. During analyses of the
algorithm, mostly the worst-case scenario is considered, i.e., when there is no pair of elements with
sum equals Z. In the worst case,
•N*c operations are required for input.
•The outer loop i loop runs N times.
•For each i, the inner loop j loop runs N times.
So total execution time is N*c + N*N*c + c. Now ignore the lower order terms since the lower
order terms are relatively insignificant for large input, therefore only the highest order term is taken
(without constant) which is N*N in this case. Different notations are used to describe the limiting
behavior of a function, but since the worst case is taken so big-O notation will be used to represent
the time complexity.
Hence, the time complexity is O(N2) for the above algorithm. Note that the time complexity is
solely based on the number of elements in array A i.e the input length, so if the length of the array
will increase the time of execution will also increase.
Order of growth is how the time of execution depends on the length of the input. In the above
example, it is clearly evident that the time of execution quadratically depends on the length of the
array. Order of growth will help to compute the running time with ease.
Another Example: Let’s calculate the time complexity of the below algorithm:
count = 0
count++;
This is a tricky case. In the first look, it seems like the complexity is O(N * log N). N for the j
′s loop and log(N) for i′s loop. But it’s wrong. Let’s see why.
Think about how many times count++ will run.
•When i = N, it will run N times.
•When i = N / 2, it will run N / 2 times.
•When i = N / 4, it will run N / 4 times.
•And so on.
The total number of times count++ will run is N + N/2 + N/4+…+1= 2 * N. So the time complexity
will be O(N).
Some general time complexities are listed below with the input range for which they are accepted in
competitive programming:
Input Worst Accepted Time
Usually type of solutions
Length Complexity
// count frequencies
freq[arr[i]]++;
cout << x.first << " " << x.second << endl;
// Driver Code
int main()
// Given array
countFreq(arr, n);
return 0;
Output
5 1
20 4
10 3
Here two arrays of length N, and variable i are used in the algorithm so, the total space used is N * c
+ N * c + 1 * c = 2N * c + c, where c is a unit space taken. For many inputs, constant c is
insignificant, and it can be said that the space complexity is O(N).
There is also auxiliary space, which is different from space complexity. The main difference is
where space complexity quantifies the total space used by the algorithm, auxiliary space quantifies
the extra space that is used in the algorithm apart from the given input. In the above example, the
auxiliary space is the space used by the freq[] array because that is not part of the given input. So
total auxiliary space is N * c + c which is O(N) only.
Growth of Functions
Performance Measurements
Shell sort
Shell sort is mainly a variation of Insertion Sort. In insertion sort, we move elements only one position ahead. When an
element has to be moved far ahead, many movements are involved. The idea of ShellSort is to allow the exchange of far
items. In Shell sort, we make the array h-sorted for a large value of h. We keep reducing the value of h until it becomes
1. An array is said to be h-sorted if all sublists of every h’th element are sorted.
Algorithm:
Step 1 − Start
Step 2 − Initialize the value of gap size, say h.
Step 3 − Divide the list into smaller sub-part. Each must have equal intervals to h.
Step 4 − Sort these sub-lists using insertion sort.
Step 5 – Repeat this step 2 until the list is sorted.
Step 6 – Print a sorted list.
Step 7 – Stop.
int main()
{
int arr[] = {12, 34, 54, 2, 3}, i;
int n = sizeof(arr)/sizeof(arr[0]);
shellSort(arr, n);
return 0;
}
Output
Array before sorting:
12 34 54 2 3
Array after sorting:
2 3 12 34 54
Time Complexity: Time complexity of the above implementation of Shell sort is O(n2). In the
above implementation, the gap is reduced by half in every iteration. There are many other ways to
reduce gaps which leads to better time complexity. See this for more details.
Questions:
1. Which is more efficient shell or heap sort?
Ans. As per big-O notation, shell sort has O(n^{1.25}) average time complexity whereas, heap sort
has O(N log N) time complexity. According to a strict mathematical interpretation of the big-O
notation, heap sort surpasses shell sort in efficiency as we approach 2000 elements to be sorted.
Note:- Big-O is a rounded approximation and analytical evaluation is not always 100% correct, it
depends on the algorithms’ implementation which can affect actual run time.
Choice of Pivot
There are many different choices for picking pivots.
•Always pick the first (or last) element as a pivot. The below implementation is picks the last
element as pivot. The problem with this approach is it ends up in the worst case when array is
already sorted.
•Pick a random element as a pivot. This is a preferred approach because it does not have a pattern
for which the worst case happens.
•Pick the median element is pivot. This is an ideal approach in terms of time complexity as we can
find median in linear time and the partition function will always divide the input array into two
halves. But it is low on average as median finding has high constants.
Partition Algorithm
The key process in quickSort is a partition(). There are three common algorithms to partition. All
these algorithms have O(n) time complexity.
1.Naive Partition: Here we create copy of the array. First put all smaller elements and then all
greater. Finally we copy the temporary array back to original array. This requires O(n) extra space.
2.Lomuto Partition: We have used this partition in this article. This is a simple algorithm, we keep
track of index of smaller elements and keep swapping. We have used it here in this article because
of its simplicity.
3.Hoare’s Partition: This is the fastest of all. Here we traverse array from both sides and keep
swapping greater element on left with smaller on right while the array is not partitioned. Please
refer Hoare’s vs Lomuto for details.
Let us understand the working of partition algorithm with the help of the following example:
2/6
#include <bits/stdc++.h>
using namespace std;
int main() {
vector<int> arr = {10, 7, 8, 9, 1, 5};
int n = arr.size();
quickSort(arr, 0, n - 1);
Output
Sorted Array
1 5 7 8 9 10