UNIT 3

SEARCHING AND SORTING

A.Y 2023-2024
 Syllabus
 Searching: Search Techniques-Sequential
Search/Linear Search, Variant of Sequential Search-
Sentinel Search, Binary Search, Fibonacci Search,
and Indexed Sequential Search.
 Sorting: Types of Sorting-Internal and External
Sorting, General Sort Concepts-Sort Order, Stability,
Efficiency, and Number of Passes, Comparison
Based Sorting Methods-Bubble Sort, Insertion Sort,
Selection Sort, Quick Sort, Shell Sort,
 Non-comparison Based Sorting Methods-Radix Sort,
Counting Sort, and Bucket Sort, Comparison of All
Sorting Methods and their complexities
 What is an Array?
 An array is a variable which can store multiple values of
same data type at a time.
 An array is a collection of homogeneous (same type) data items
stored in contiguous memory locations.
 For example if an array is of type “int”, it can only store integer elements and
cannot allow the elements of other types such as double, float, char etc.
 An array can also be defined as follows...
 "Collection of similar data items stored in continuous

memory locations with single name".

 Array (ADT)
 The array is a basic abstract data type that holds an
ordered collection of items accessible by an integer index.
 Arrays are defined as ADT’s because they are capable of
holding contiguous elements in the same order. And they
permit access for the specific element via index or position .

 Following are the various operations supported by an

array.
 Traverse − print all the array elements one by one.
 Insertion − add an element at given index.
 Deletion − delete an element at given index.
 Search − search an element using given index or by
value.
 Update − update an element at given index.
 Create an empty array and display
(Python)
 Create an empty array as per user input create an
array and display the all array element on screen

Time Complexity : Output

Create an array : O(n)
Display array element: O(n)
 Insert operation in array
 Insertion in array perform by 2 ways
 Insert an Element to the End of an Array
 Insert an Array Element at a Predefined Location

Add an Element to the End of an Array

 element to insert at the end of the given array.
 Append function is used to add new data in the bottom
of array
 arrayname.append(data)
Add an Element to the specified position in Array
 element to insert at the specified position in array.
(position <=size_of_aray) (shifting concept is used)
 Insert function is used to add new data at specified
position. Insert function takes 2 argument (location and
Implementation Python..
 What is Searching
 Computer systems are often used to store
large amounts of data from which individual
records must be retrieved according to some
search criterion.

 Thus the efficient storage of data to facilitate

fast searching is an important issue.

 Searching involves deciding whether a search

key is present in the data. For example,
looking up a phone book or address book.
 Cont..
 Searching in data structure refers to the
process of finding location LOC of an
element in a list.

 Various algorithms have been defined to

find whether an element is present in the
collection of items or not.

 The searching algorithm includes:

 Sequential Search/Linear Search
 Binary Search
 Linear Search
 Linear search is a very simple search algorithm. In
this type of search, a sequential search is made over
all items one by one.

 Every item is checked and if a match is found then

that particular item is returned, otherwise the search
continues till the end of the data collection.

 In linear search, for searching any element in an

array, we have to start from beginning, scanning each
element of the array till end to see match found.
 If found then return true.
 If Not found after searching till then return false.
Searching Operation
 Let see the example
 Cont…
 In the sequential search, we start searching for the
target at the beginning of the list and continue
until we find the target.

 This approach gives us two possibilities:

 Successful Search
 unsuccessful Search

 It compares the element to be searched with all the

elements present in the array and when the element
is matched successfully, it returns the index of the
element in the array, else it return -1.
 Searching operation

The element
is matched succes
sfully, it returns the
index of the
 Unsuccessful Search

Element is
not
matched th
en it return -
1.
 Sequential Search
Algorithm
1. Take the input array arr[] from user.
2. Take element(x) you want to search in this array
from user.
3. Set flag variable as -1
4. LOOP : arr[start] -> arr[end]
1. if match found i.e arr[current_postion] == x then
1. Print “Match Found at position” current_position.
2. flag = 0
3. abort
5. After loop check flag variable.
1. if flag == -1
1. print “No Match Found”
6. STOP
 Sequential Search
Pseudocode
void LinearSearch(int arr[], int value, int n)
//arr[]=list of data, value=key to be search, n= total number of element
{ int found = 0;
for (int i = 0; i < n ; i++) {
if (value == arr[i] ) {
found = 1;
break;
}
}
if (found == 1)
printf("Element is present in the array at position %d “, i+1);
else
printf("Element is not present in the array.“);
}
 Cont..
Best case-
 In the best possible case,

 The element being searched may be found at the first position.

 In this case, the search terminates in success with just one

comparison.
 Thus in best case, linear search algorithm takes O(1) operations.

Worst Case-
 In the worst possible case,

 The element being searched may be present at the last position or

not present in the array at all.

 In the former case, the search terminates in success with n

comparisons.
 In the later case, the search terminates in failure with n comparisons.

 Thus in worst case, linear search algorithm takes O(n) operations.

 Linear Search (python)

 (linear searching) Searching of data : Best case: O(1)

 average and worst case: O(n)
Linear Search using function
Linear Searching using recursive algo
Complexity of Sequential Search

 Space complexity
 As linear search algorithm does not use any extra
space thus its space complexity = O(n) for an array
of n number of elements.
 Time Complexity
 Worst case complexity: O(n) – This case occurs
when the element to search is not present in the
array.
 Best case complexity: O(1) – This case occurs
when the first element is the element to be searched.
 Average complexity: O(n) – This means when an
element is present somewhere in the middle of the
array.
SENTINEL LINEAR SEARCH
 Sentinel Linear Search
 idea is to reduce the number of comparisons
required to find an element in a list.
 Here we replace the last element of the list
with the search element itself and run a while
loop to see if there exists any copy of the
search element in the list and quit the loop as
soon as we find the search element.
 This algorithm is faster than linear search
because it cuts down on the number of
comparisons
 Algorithm
 In sentinel search, we first insert the target at the end of
the list, and then we compare each item of the list until
we find the required item.

 Either the required item is in the list, in which case it will

be found before we reach the end of the list. Or the list
didn’t have the target, so the algorithm will reach the end
of the list and find the target item that we have inserted.

 Here, we only need to check if the item matches the

target, and we don’t need to check if the list is empty.
This is because we are going to find the target one way
or the other and break out of the loop.
 Cont..

 Here we see that the while loop makes only one comparison in each iteration and it is sure
that it will terminate since the last element of the list is the search element itself. So in the
worst case ( if the search element does not exists in the list ) then there will be at
most N+2 comparisons ( N comparisons in the while loop and 2 comparisons in the if
condition). Which is better than ( 2N+1 ) comparisons as found in Simple Linear Search.
 Take note that both the algorithms have time complexity of O(n).
Implementation
BINARY SEARCHING
 Binary Search
 The sequential search algorithm is very slow.
If we have an array of 1000 elements, we
must make 100 comparisons in the worst
case.

 Binary search is the most popular Search

algorithm. It is efficient and also one of the
most commonly used techniques that is
used to solve problems.

 Binary search algorithm requires already

 Cont..
 Binary search follows divide and conquer approach.

 In binary search, the list is divided into two halves and the item
is compared with the middle element of the list.

 If the match is found then, the location of middle element is

returned otherwise, we search into either of the halves
depending upon the result produced through the match.

 Once the target element found or collection is empty, then the

search is over.

 Binary Search divides the sorted collection in half until the

sought-for item is found, or until it is determined that the item
cannot be present in the smaller collection. Lets see the example
Algorithm to perform Binary Search
Binary Search Pseudocode
Binary Search using iterative method
 Best case Time complexity
 Consider the example. Here we have 7 elements
(nodes), and we want to find just one: i.e. 26

 Calculate mid index position first i.e

 Mid=(low+high)/2 i.e. mid=(0+6)/2 =3

 Compare 3rd index element from array is it equal to key

to be search.
 Yes

 Once element is found we not searching rest of array

and we terminate searching. Therefore time complexity
Best case Example
 Worst case and average case time
complexity
 Consider the example. Here we have 7 elements (nodes), and we
want to find just one: i.e. 12
 Cont…
 Binary tree is a case where a problem of size n is
divided into sub-problem of size n/2 until we reach a
problem of size 1
Cont..
Time complexity
 Worst-case performance:- O(log n)
 Best-case performance:- O(1)
 Average performance:- O(log n)
 Worst-case space complexity:- O(1)
Agenda
 Review Binary Search
 Recursive approach of binary search
 Time complexity analysis of binary
search
 Ternary Search
 Binary Search using
recursion
 How to implement binary search using recursion.
 In recursion, the function call itself until the base
condition is reached.
Recurrence Relation of Binary Search

 T(n) = T(n/2) + 1 is an example of a recurrence

relation
 A Recurrence Relation is any equation for a
function T, where T appears on both the left and
right sides of the equation.
 We always want to “solve” these recurrence
relation by getting an equation for T, where T
appears on just the left side of the equation
 Whenever we analyze the run time of a recursive
algorithm, we will first get a recurrence relation
 To get the actual run time, we need to solve the
recurrence relation
Recurrence Relation
Cont..
Cont..
Cont..
Cont..
Cont..
Solve T(n) = T(n/2) + 1
Cont..
Linear Search Vs Binary
Search
TERNARY SEARCHING
 Ternary Search
 Ternary search is a searching algorithm used
to find the position of a specific value within a
sorted array or list.
 It is an improvement over the binary search
algorithm and is particularly useful when
dealing with functions that are either strictly
increasing or strictly decreasing.
 Ternary search works by dividing the search
space into three parts instead of two in each
iteration.
 Ternary Search Algorithm
 Step 1: Start
 Step 2: Initialize left =0, right=n-1, mid1=0, and mid2=0.
 Step 3: Find mid1 and mid 2 using the formula:
 mid1 = left + (right - left) /3 mid2 = right - (right - left) / 3
 Step 4: If arr[mid1] = key Then return mid1, and print element
found.
 Step 5: If arr[mid2] = key Then return mid2, and print element
found.
 Step 6: If key < arr[mid1] Then set right = mid1-1 and go to step
3.
 Step 7: If key > array[mid2] Then set left = mid2+1 and go to step
3.
 Step 8: If key > arr[mid1] && key < arr[mid2]
 Then set left = mid1+1, right = mid2-1, and goto step 3.
 Step 9: Repeat steps 3 to 8 till Left <= Right.

Implementation non recursive
algorithm
Recursive Implementation
FIBONACCI SEARCHING
 Fibonacci search
 Fibonacci search is a search algorithm used to find an
element in a sorted array or list. It is a variation of
binary search that uses Fibonacci numbers to divide
the array into subarrays for comparison.
 Fibonacci search is not as commonly used as binary
search but can be useful in certain situations,
particularly when the data is large, and the
performance of binary search can be optimized
further.
Fibonacci search
Fibonacci search implementation
 Complexity
 Worst case time complexity: Θ(log n)
 Average case time complexity: Θ(log n)
 Best case time complexity: Θ(1)
 Space complexity: Θ(1)

 KeyPoint
 Fibonacci Search examines closer elements in few steps. So when
input array is big that cannot fit in CPU cache or in RAM, it is useful.
 On average, fibonacci search requires 4% more comparisons than
binary search
 Fibonacci search requires only addition and subtraction whereas
binary search requires bit-shift, division or multiplication operations.
 Fibonacci search can reduce the time needed to access an element
in a random access memory.
INDEXED SEQUENTIAL SEARCH.
 Indexed Sequential Search.
 Indexed Sequential Search, also known as Indexed
Sequential Access Method (ISAM), is a searching
technique used in data structures, particularly for
searching within large datasets stored in sequential
access files.

 It is based on sequential and random access searching

method. It search the elements according to the groups.

 This method combines the advantages of both

sequential search and binary search by creating an index
structure that helps in locating records efficiently. (Array
always in sorted manner)
Algorithm
SORTING
 What is sorting
 The arrangement of data in a preferred order is called sorting
in the data structure.
 Example: dictionary
 Sorting arranges data in a sequence which makes searching
easier.

 A sorting algorithm is just a series of orders or instructions. In

this, an array is an input, on which the sorting algorithm
performs operations to give out a sorted array.

 Here’s an example of what sorting does.

 Let’s suppose you have an array of strings: [h,j,k,i,n,m,o,l]
 Now, sorting would yield an output array in alphabetical order.
 Output: [h,i,j,k,l,m,n,o]
 Sorting Categories
 There are two different categories in
sorting:
 Internal Sorting: If all the data that is to be
sorted can be adjusted at a time in the main
memory, the internal sorting method is being
performed.

 External Sorting: When the data that is to be

sorted cannot be accommodated in the
memory at the same time and some has to be
kept in auxiliary memory such as hard disk,
floppy disk, magnetic tapes etc, then external
 What is stability in sorting?
 A sorting algorithm is said to be stable if two
objects with equal keys appear in the same
order in sorted output as they appear in the
input array to be sorted.
 Stable and Unstable Sorting
 If a sorting algorithm, after sorting the contents, does
not change the sequence of similar content in which they
appear, it is called stable sorting.

 Some examples of stable algorithms are Merge

Sort, Insertion Sort, Bubble Sort, and Binary Tree Sort
 Unstable Sorting
 If a sorting algorithm, after sorting the contents,
changes the sequence of similar content in which
they appear, it is called unstable sorting.

 Some examples of stable algorithms are Heap

Sort, and Selection sort are the unstable sorting
algorithm
 Sorting
 Sorting is a very classic problem of reordering
items (that can be compared, eg integers, floating-
point numbers, strings, etc) of an array (or a list)
in a certain order (increasing, non-decreasing,
decreasing, non- increasing, lexicographical, etc).

 There are many different sorting algorithms, each

has its own advantages and limitations

 Sorting is commonly used as the introductory

problem in various Computer Science classes to
showcase a range of algorithmic ideas.
 Different Sorting Algorithms
 There are many different techniques available for sorting,
differentiated by their efficiency and space requirements.
Following are some sorting techniques which we will be
covering in next few tutorials.
 Bubble Sort
 Insertion Sort
 Selection Sort
 Shell Sort
 Radix Sort
 Quick Sort
 Merge Sort
 Heap Sort

 Lets see one by one

Agenda
 Bubble Sort
 Selection Sort
 Difference between Bubble and Selection
sort
 Bubble Sort
 Bubble Sort is a simple algorithm which is used to
sort a given set of n elements provided in form of
an array with n number of elements.

 Bubble Sort compares all the element one by one

and sort them based on their values.

 bubble sort algorithm iterates through the list or

array that it is given, and compares each pair of
adjacent elements in the list by size. If they
elements are in the incorrect order, it swaps them,
and then moves on to the next pair of elements.
 Bubble Sort
 Def: A bubble sort algorithm walks through a collocation
(iterates) and compare two adjacent element at a time. If
the element are our of order, it swap them. It continue
these swapping until the entire collection become sorted
(Ascending or Descending Order).

 It is known as bubble sort, because with every complete

iteration the largest element in the given array, bubbles up
towards the last place or the highest index.

 Sorting takes place by stepping through all the elements

one-by-one and comparing it with the adjacent element
and swapping them if required.
 Steps to perform sorting
 Assume that A[] is an unsorted array of n elements. This array needs
to be sorted in ascending order.

 Step1: Starting with the first element(index = 0), compare the

current element with the next element of the array.
 Step2: Compare a pair of adjacent items (a, b),
 Step3: Swap that pair if the items are out of order (in this case, when
a> b),
 Step4: Repeat Step 2 and 3 until we reach the end of the array
(the last pair is the ( N -2) -th and ( N -1) -th items as we use 0-based
indexing
 Step5: By now, the largest item will be at the last position.
We then reduce N by 1 and repeat Step 1 until we have N = 1 .
Cont...

 Final Result after sorting

Example 1
Example 2
 here, we have a collection of unordered
numbers that need to be sorted:• 9, 7, 4,
Since is 9

1, 2. How would bubble sort handle this?

greater
than , the
7
algorithm
knows that it
should come
after 7.
• Since these
two numbers
are in the
incorrect
order relative
to one
another, it
will swap
them, which
will change
the order of
just those
 Cont..
 Okay, so that’s how the bubble sort algorithm
functions when comparing two elements at a time.
But how does it actually sort through the entire list?
 Cont…
 So as we can see in the representation above, after
the first iteration, 9 is placed at the last index,
which is the correct position for it.
 Similarly after the second iteration, 7 will be at the
second last index, and so on.
Original 9 7 4 1 2
Array
Pass1 7 4 1 2 9
Pass2 4 1 2 7 9
Pass3 1 2 4 7 9
Pass4 1 2 4 7 9

 In general, given a collection of n unsorted

elements, it takes (n-1) iterations through the list in
Example 3. sort the array 5,1,4,2,8
and Show each pass
 Bubble sort Implementation
void bubble_sort( int A[ ], int n ) {
int temp;
for(int k = 0; k< n-1; k++) {
// (n-k-1) is for ignoring comparisons of elements which have already been
compared in earlier iterations
for(int i = 0; i < n-k-1; i++) {
if(A[ i ] > A[ i+1] ) {
temp = A[ i ]; // here swapping of positions is being done.
A[ i ] = A[ i+1 ];
A[ i + 1] = temp;
}
}
}
}
 Time Complexity
 The complexity of bubble sort is O(n2) in
both worst and average cases, because the
entire array needs to be iterated for every
element.

 The best time complexity for Bubble Sort is

O(n)
 Optimized Bubble Sort
 There’s another issue, too: what if our list
was already sorted?
 A naive implementation of bubble sort would
iterate through the entire list, even if it was
sorted, and use a lot of time and memory to do
so.

 How we can optimized sorting through an

already sorted array or list?
 If we don’t make any swap in our first iteration,
then we can sure that our list is already sorted.
So we stop iteration process.
 Algo:

 The best time

complexity for
Bubble Sort is O(n)
Key points
 Total Passes required is n-1

 Total number of comparisons in bubble

sort is (n - 1) + (n - 2) + (n-3) +(n-4) +
(n-5) ….....(2) + (1) = n(n - 1)/2
Sort
Trace
Example
Trace the given implementation on the array below. Try to keep track of
how many comparisons and swaps are performed.

A 13 7 43 5 3 19 2 23 29

0 1 2 3 4 5 6 7 8
A 13 7 43 5 3 19 2 23 29 Original array
7 13 5 3 19 2 23 29 43 Pass 1
7 5 3 13 2 19 23 29 43 Pass 2
5 3 7 2 13 19 23 29 43 Pass 3
3 5 2 7 13 19 23 29 43 Pass 4
3 2 5 7 13 19 23 29 43 Pass 5
2 3 5 7 13 19 23 29 43 Pass 6
2 3 5 7 13 19 23 29 43 Pass 7
2 3 5 7 13 19 23 29 43 Pass 8
Selection sort
 Selection sort
 The Selection sort algorithm is based on the idea of
finding the minimum or maximum element in an
unsorted array and then putting it in its correct position
in a sorted array.

 Selection sort is an algorithm that selects the smallest

element from an unsorted list in each iteration and
places that element at the beginning of the unsorted
list.

 We divide the array into two parts: sorted and unsorted.

The left part is sorted subarray and the right part is
unsorted subarray. Initially, sorted subarray is empty
and unsorted array is the complete given array.
 Step to implement
 We perform the steps given below until the
unsorted subarray becomes empty:
 Pick the minimum element from the unsorted
subarray.
 Swap it with the leftmost element of the
unsorted subarray.
 Now the leftmost element of unsorted
subarray becomes a part (rightmost) of sorted
subarray and will not be a part of unsorted
subarray.
 Working

Step 1 find minimum element in array

Cont..
 Next element want to swap
Cont..
Cont..
Example
 Algorithm
1. Read the array from user and display before sorting array
elements
2. For the starting, select an [0] as the minimum element of the
array. As minimum = a[0].
3. Search the minimum element in the rest of the array.
4. If the current element is smaller then swap the minimum element
with the current element.
5. Move the minimum element variable to the next element of the
array.
6. Repeat the step 3 to 5 until the array becomes sorted.
 Pseudocode
1. FindMinIndex(Arr[], start, end
1. SelectionSort(Arr[], arr_size)
2. {
2. {
3. min_index = start
3. FOR i from 1 to arr_size:
4. FOR i from (start + 1) to
4. min_index = FindMinIndex(Arr, i,
end:
arr_size)
5. IF Arr[i] < Arr[min_ind
5. IF i != min_index:
6. swap(Arr[i], Arr[min_index])
6. min_index = i
7. END of IF
7. END of IF
8. END of FOR
8. END of FOR
9. }
9. Return min_index
10. }
Python Implementation
 Time Complexity
 Suppose, there are ‘n’ elements in the array.
Therefore, at worst case, there can be n iterations in
FindMinIndex() for start = 1 and end = n. No
auxiliary space used.
 Total iterations = (n – 1) + (n – 2) + . . . + 1 = (n * (n
– 1)) / 2 = (n2 – n) / 2
 Therefore,
 Time complexity: O(n2)
 Space complexity: O(1)
Difference between Bubble and
Selection sort
BUBBLE SORT
BASIS FOR COMPARISON SELECTION SORT

Basic Adjacent element is Largest element is

compared and swapped selected and swapped
with the last element (in
case of ascending order).

Best case time complexity O(n) O(n2)

Efficiency Inefficient Improved efficiency as

compared to bubble sort

Stable Yes No
Method Exchanging Selection
Speed Slow Fast as compared to
bubble sort
Agenda
 Insertion Sort
 Example of insertion sort
 Algorithm and time complexity
Inserti
on
sort
 Insertion sort
 Insertion sort is a simple sorting algorithm that
builds the final sorted array (or list) one item at a
time.

 Insertion sort is a simple sorting algorithm that

works the way we sort playing cards in our hands.

 Insertion sort iterates, consuming one input element

each repetition, and growing a sorted output list. At
each iteration, insertion sort removes one element
from the input data, finds the location it belongs
within the sorted list, and inserts it there. It repeats
until no input elements remain.
 Example
 Suppose we need to sort the following array.

 Step1 : The first element in the array is

assumed to be sorted. Take the second
element and store it separately in key.

Compare key with the first element. If the

first element is greater than key, then key is
placed in front of the first element.
 Step 2:Now, the first two elements are sorted.

 Take the third element and compare it with the

elements on the left of it. Placed it just behind the
element smaller than it. If there is no element smaller
than it, then place it at the beginning of the array.
 Similarly, place every unsorted element
at its correct position
Example 2
 Algorithm
1. Read the array from user and display before sorting array
elements
2. Consider the first element to be a sorted subarray and the rest
as an unsorted subarray
3. Sequentially iterate over the unsorted elements of the array and
move them to the sorted subarray.
4. For each unsorted element, compare the current element with
the elements before it
5. If the current element is greater than its preceding element,
leave it there, as it is already at the desired position. If not, keep
comparing it with the elements before it until:
1. A smaller or equal element is found: Insert the current element after it
2. All comparisons are made, and no smaller element is found: Insert the
current element at the beginning of the array
6. Repeat the above process for every element of the unsorted
subarray until the array is sorted
 Pseudocode
INSERTION-SORT(A)
Begin
for i := 1 to size-1 do
key := array[i]
j := i
while j > 0 AND array[j-1] > key do
array[j] := array[j-1];
j := j – 1
done
array[j] := key
done
End
Time complexity
 Time Complexity:
 O(n) for best case,
 O(n^2) for average and worst case
 Space Complexity: O(1)
Shell Sort
 Shell Sort
 In insertion sort, we could move the elements ahead only
by one position at a time. If we wish to move an element
to a faraway position in insertion sort, a lot of movements
were involved thus increasing the execution time.

 Shell sort overcomes this problem of insertion sort by

allowing movement and swap of far-away elements as
well.

 This sorting technique works by sorting elements in pairs,

far away from each other and subsequently reduces their
gap. The gap is known as the interval.
 gap = floor(n/2) where n = the number of elements in the
array.
 For Example
 Consider the array

 N = 10
 gap = floor(N/2) = floor(10/2) = 5
 Example 2
 X = [18, 32, 12, 5, 38, 33, 16, 2]
 No. of elements = 8
 gap = floor(N/2) = floor(8/2) = 4
 Algorithm
 Step 1: Divide the list into n/2 sublists.
 Step 2: Sort each sublist using insertion
sort.
 Step 3: Merge the sublists.
 Step 4: Halve the number of sublists.
 Step 5: Repeat steps 2 and 3 until the
number of sublists becomes “1”.
 Example
 X = [35, 33, 42, 10, 14, 19, 27, 44]
 No. of elements = 8
 gap = floor(N/2) = floor(8/2) = 4

 Make a virtual sub-list of all values located at the

interval of 4 positions. Here these values are {35, 14},
{33, 19}, {42, 27} and {10, 44}

We compare values in each

sub-list and swap them (if
necessary) in the original
array.
 Cont..
 After this step, the new array should look like
this

 Then, we take interval of 1 and this gap generates two

sub-lists - {14, 27, 35, 42}, {19, 10, 33, 44}

 We compare and swap the values, if required, in the

original array. After this step, the array should look like
this
 Cont..
 Finally, we sort the
rest of the array
using interval of
value 1. Shell sort
uses insertion sort
to sort the array.
Python Implementation
 Complexity
 Best case complexity: O(n*log n)
 Average case complexity: O(n*log n)
 Worst case complexity: O(n^2)

 Applications:
 The C standard library uses shell sort when dealing
with embedded systems.
 Compressors, such as bzip2, also use it to avoid
problems that could come when sorting algorithms
exceed a language’s recursion depth.
 It is also used in the Linux kernel because it does
not use the call stack.
Agenda
 What is Divide and Conquer approach
 Merge Sort
divide and conquer
 In “divide and conquer” algorithm where we
first divide the problem into subproblems and
then merge them together to conquer our
solution.

 Following algorithm work on this principals od

divide and conquer
 Binary Search
 Ternary Search
 Merge Sort
 Quick Sort
Merge Sort
 Why Merge Sort
 we learned about Bubble sort, Selection
Sort and Insertion Sort

 In these algorithm we have a average case

and worst-case running time of O(n2). As
the size of input grows, insertion and
selection sort can take a long time to run.

 Merge sort , on the other hand, runs

in O(n*log n) time in all the cases.
 Merge Sort
 Merge sort is one of the popular sorting
algorithms available and it follows a divide and
conquer approach. A problem is divided into sub-
problems and combined together to reach the
final solution!

 In Merge Sort, the given unsorted array

with n elements, is divided into n subarrays, each
having one element, because a single element is
always sorted in itself. Then, it repeatedly merges
these subarrays, to produce new sorted
subarrays, and in the end, one complete sorted
array is produced.
 Working
 The concept of Divide and
Conquer involves three steps:

 Divide the problem into

multiple small problems.

 Conquer the subproblems by

solving them. The idea is to
break down the problem into
atomic subproblems, where
they are actually solved.

 Combine the solutions of the

subproblems to find the
solution of the actual problem.
Example 2
Example 3
 Starting
index =1
 Steps
 In merge sort we follow the following steps:
 We take a variable p and store the starting index of our
array in this. And we take another variable r and store
the last index of array in it.
 Then we find the middle of the array using the
formula (p + r)/2 and mark the middle index as q, and
break the array into two subarrays, from p to q and
from q + 1 to r index.
 Then we divide these 2 subarrays again, just like we
divided our main array and this continues.
 Once we have divided the main array into subarrays
with single elements, then we start merging the
subarrays.
 Algorithm of merge sort
 Recursive method is used to implement
 Analysis of merge sort
 Suppose T(n) is the number of
comparisons needed to sort an
array of n elements by the
MergeSort algorithm.

 By splitting an array in two

parts we reduced a problem to
sorting two parts but smaller
sizes, namely n/2.

 Each part can be sort in T(n/2).

Finally, on the last step we
perform n-1 comparisons to
merge these two parts in one.

 All together, we have the

following equation T(n) =
2*T(n/2) + n - 1
calculation of time complexity using tree
pattern
Cont..
Solving
Cont..
 Cont..

 Worst Case Time Complexity [ Big-

O ]: O(n*log n)
 Best Case Time Complexity [Big-
omega]: O(n*log n)
 Average Time Complexity [Big-
theta]: O(n*log n)
 Space Complexity: O(n)
QUICK SORT
 Quick Sort

 Quick sort is based on the divide-and-conquer

approach based on the idea of choosing one element
as a pivot element and partitioning the array around
it such that: Left side of pivot contains all the
elements that are less than the pivot element Right
side contains all elements greater than the pivot
 It reduces the space complexity and removes the use
of the auxiliary array that is used in merge sort.
Selecting a random pivot in an array results in an
improved time complexity in most of the cases.
 Cont..
 In the implementation below, the following
components have been used: Here,
 A[] = array whose elements are to be sorted
 Start: Leftmost position of the array
 End: Rightmost position of the array
 i: Boundary between the elements that are less than
pivot and those greater than pivot
 J: Boundary between the partitioned and
unpartitioned part of array.
 Pivot:Pivot element
Quicksort Algorithm
Given an array of n elements (e.g.,
integers):
 If array only contains one element, return

 Else

 pick one element to use as pivot.

 Partition elements into two sub-arrays:
 Elements less than or equal to pivot
 Elements greater than pivot
 Quicksort two sub-arrays
 Return results
Example
We are given array of n integers to sort:
40 20 10 80 60 50 7 30 100
Pick Pivot Element
There are a number of ways to pick the pivot
element. In this example, we will use the first
element in the array:
40 20 10 80 60 50 7 30 100
Partitioning Array
Given a pivot, partition the elements of
the array such that the resulting array
consists of:
1. One sub-array that contains elements >=
pivot
2. Another sub-array that contains elements
< pivot

The sub-arrays are stored in the original

data array.

Partitioning loops through, swapping

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
 1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]

pivot_index = 4 7 20 10 30 40 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index
Partition Result

7 20 10 30 40 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

<= data[pivot] > data[pivot]

Recursion: Quicksort Sub-
arrays

7 20 10 30 40 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

<= data[pivot] > data[pivot]

Quick Sort example
Cont..
Repeat process for left and right subtree
Quick Sort Implementation
1. Int main()
2. {
3. Read total number of element want to
insert in array i.e. n
4. Read Unordered array from the user i.e
for i=0 to n-1
cin>>arr[i]
5. Pass array, starting_index and
end_index to quick_sort function i.e
Quicksort(a,0,n-1);
6. Print the sorrted array
1. int partition (int a[], int left,
if(i<j) {
int right){
swap(a[i],a[j]);
2. int i, j, pivot, temp;
}//end if
3. if(left<right) {
}//end while
pivot=left;
swap(a[pivot],a[j])
i=left;
} //end if
j=right;
return j;
while(i<j) {
}//end partition
function
while(a[pivot]>=a[i])
i++;
Quick Sort analysis
Example
Counting Sort
 Counting Sort
 A non in-place sorting algorithm that sorts the elements
of an array by counting the frequency of each unique
element.

 An auxiliary array is required to maintain those counts.

Then using those counts to compute every
element’s position in the final sorted array.

 Counting sort is an efficient algorithm that runs in O(n +

k) time (n is the number of elements in the array; k is the
range of values to be sorted).

 If the range of input array(ie. k) is very large, this

algorithm requires a lot of space and becomes inefficient.
 Working
 Counting sort algorithm sorts the elements of an array
by counting the number of times each element
appears and assign it into an auxiliary array. We
can fetch those counts to build up the final sorted
output.
 Auxiliary array — counts
 Map elements in the input array to counts
array indices.
working
Example
 Algorithm

 The time complexity of the Counting Sort algorithm is O(m+n) where m is the number of
elements in the input array and n is the range of input.
 In all the cases whether it be the best case, worst case, or average case the time
complexity of the algorithm is the same because the time complexity of the counting sort
algorithm is not dependent on how many elements we store in the array.
Bucket Sort
 BUCKET SORT
 Bucket sort is a starting algorithm mainly used when we have
data uniformly distributed over a range.

 As the name suggests, in bucket sort we have several groups

called buckets, each containing specified elements of the input
array.

 Each bucket is then sorted by either recursively applying the

same bucket algorithm or suitable sorting algorithms.

 Bucket sort algorithm assigns elements to several

buckets(or called bins) firstly. The elements are grouped in
different buckets and not sorted yet. Then
those buckets are sorted individually using any sorting
algorithm. Finally, gather sorted buckets to form a sorted
 Working steps
 step1:
a. get the maximum and minimum values in the unsorted
array
b. calculate required bucket size
bucket size = floor((maxValue - minValue) / len(array)) + 1
You can use different formula to determine bucket size based on your
application.
 step2: allocate elements into their
corresponding buckets
bucketIdx = floor((element - minValue) / len(array))
You can use different formula to assign elements into buckets based on
your application.
 step3: sort individual buckets (using insertion sort)
You can use any sorting algorithm to sort buckets.
 step4: combine sorted buckets to form a sorted array
Example
With –ve input
Algorithm bucketSort() {
1. create N buckets each of which can hold a range of values
2. for all the buckets
1. initialize each bucket with 0 values
3. for all the buckets
1. put elements into buckets matching the range
4. for all the buckets
1. sort elements in each bucket
5. gather elements from each bucket
Bucket Sort Pseudocode
Radix Sort
 Radix Sort
 A non in-place sorting algorithm that sorts numbers digit
by digit based on individual digit position.

 We can sort starting from the least significant digit(LSD) to

the most significant digit(MSD) or vice versa.

 Radix sort can incorporate different sorting algorithms to

sort digits of the same place value.

 Radix sort algorithm requires the number of passes which

are equal to the number of digits present in the largest
number among the list of numbers. For example, if the
largest number is a 3 digit number then that list is sorted
with 3 passes.
 Step by Step Process
 The Radix sort algorithm is performed using the following
steps...
 Step 1 - Define 10 queues each representing a bucket for
each digit from 0 to 9.
 Step 2 - Consider the least significant digit of each number
in the list which is to be sorted.
 Step 3 - Insert each number into their respective queue
based on the least significant digit.
 Step 4 - Group all the numbers from queue 0 to queue 9 in
the order they have inserted into their respective queues.
 Step 5 - Repeat from step 3 based on the next least
significant digit.
 Step 6 - Repeat from step 2 until all the numbers are
grouped based on the most significant digit.
Example
Algorithm
 Complexity
 Complexity of the Radix Sort Algorithm
 To sort an unsorted list with 'n' number of
elements, Radix sort algorithm needs the
following complexities...
 Worst Case : O(n)
Best Case : O(n)
Average Case : O(n)
Sorting Algorithm Time Complexity
Comparison
Oct 2022
FDS APR 2023