Sorting

Algorithms
 Motivation of Sorting
• The term list here is a collection of records.
• Each record has one or more fields.
• Each record has a key to distinguish one record with
another.
• For example, the phone directory is a list. Name,
phone number, and even address can be the key,
depending on the application or need.
 Two Common Categories
Sorting
Sorting
Algorithms
Algorithms
of O(N^2)
of
• Bubble Sort O(N log N)
• Selection
Heap SortSort
• Insertion
Merge Sort
Sort
• Quick Sort
 For small values of N
• It is important to note that all algorithms
appear to run equally as fast for small values
of N.

• For values of N from the thousands to the

millions, The differences between O(N^2)
and O(N log N) become dramatically
apparent
 O(N^2) Sorts
• Easy to program

• Simple to understand

• Very slow, especially for large values of N

• Almost never used in professional software

 Selection Sort

• More efficient than Bubble Sort.

• Works by finding the largest element in the

list and swapping it with the last element,
effectively reducing the size of the list by 1.
 Selection Sort Algorithm
void SelectionSort()
{
for (int i = 0; i < n-1; i++)
{
int indx=i, small = a[i];
for (int j = i+1; j < n; j++ )
{
if (a[j] < small)
{
small = a[j];
indx = j;
}
}
a[indx] = a[i];
a[i] = small;
}
}
 Insertion Sort
• while some elements unsorted:
– Using linear search, find the location in the sorted portion
where the 1st element of the unsorted portion should be
inserted
– Move all the elements after the insertion location up one
position to make space for the new element
45

38 45
60 60
66 45
66 79 47 13 74 36 21 94 22 57 16 29 81

the fourth iteration of this loop is shown here

An insertion sort partitions the array into two regions
An insertion sort of an array of five integers
 Insertion Sort Algorithm
void insertionSort()
{
for (int i = 1; i < n; i++)
{
int y = a[i];
// Shuffle up all sorted items > arr[i]
for (int pos = i-1; pos >=0 && y < a[pos]; pos--)
a[pos+1] = a[pos];
// Insert the current item
a[pos+1] = y;
}
}
 O(N log N) Sorts
• Fast
• Efficient
• Complicated, not easy to understand
• Most make extensive use of recursion and
complex data structures
 Heap Sort
• Slowest O(N log N) algorithm.

• Although the slowest of the O(N log N)

algorithms, it has less memory demands than
Merge and Quick sort.
 Heap Sort
Works by transferring items to a heap, which is
basically a binary tree in which all parent nodes
have greater values than their child nodes. The
root of the tree, which is the largest item, is
transferred to a new array and then the heap is
reformed. The process is repeated until the sort
is complete.
 Converting An Array Into A Max Heap

[1] 26 [1] 77

[2] 5 [3] 77 [2] 61 [3] 59

[4] 1 [5] 61 11 59 48 [4] [5] 19 11 26

[6] [7] [6] [7]

15 48 19 15 1 5
[8] [9] [10] [8] [9] [10]

(a) Input array (b) Initial heap

 Heap Sort Example

[1] 61 [1] 59

[2] 48 [3] 59 [2] 48 [3] 26

[4] 15 [5] 19 11 26 15 [5] 19 11 1

[6] [7] [6] [7]

5 1 5

[8] [9] [8]

Heap size = 9, Sorted = Heap size = 8, Sorted =

[77] [61, 77]
 Overview
• Divide and Conquer

• Merge Sort

• Quick Sort

17
 Divide and Conquer
1. Base Case, solve the problem directly if it is
small enough

2. Divide the problem into two or more similar

and smaller subproblems

3. Recursively solve the subproblems

4. Combine solutions to the subproblems

18
 Divide and Conquer - Sort
Problem:
• Input: A[left..right] – unsorted array of integers

• Output: A[left..right] – sorted in non-decreasing order

19
 Divide and Conquer - Sort
1. Base case
at most one element (left ≥ right), return
2. Divide A into two subarrays: FirstPart, SecondPart
Two Subproblems:
sort the FirstPart
sort the SecondPart
3. Recursively
sort FirstPart
sort SecondPart
4. Combine sorted FirstPart and sorted SecondPart

20
 Merge Sort: Idea

Divide into
A FirstPart SecondPart
two halves
Recursively sort

FirstPart SecondPart

Merge

A is sorted!

21
 Merge Sort: Algorithm
Merge-Sort (A, left, right)
if left ≥ right return
else
middle =(left+right)/2
Merge-Sort(A, left, middle) Recursive Call

Merge-Sort(A, middle+1, right)

Merge(A, left, middle, right)

22
Merge-Sort: Merge
Sorted

A:

merge
Sorted Sorted
FirstPart SecondPart

A:

A[left] A[middle] A[right] 23

 Merge-Sort: Merge Example

A: 25 35 7 28
15 8 30
1 46 5 14
10 6

L: R:
3 5 15 28 6 10 14 22

Temporary Arrays

24
 Merge-Sort: Merge Example

A:
31 5 15 28 30 6 10 14

k=0

L: R:
3
2 15
3 28
7 30
8 6
1 10
4 14
5 22
6

i=0 j=0
25
 Merge-Sort: Merge Example

A:
1 25 15 28 30 6 10 14

k=1

L: R:
3
2 5
3 15
7 28
8 6
1 10
4 14
5 22
6

i=0 j=1
26
 Merge-Sort: Merge Example

A:
1 2 3 28 30
15 6 10 14

k=2

L: R:
2 3 7 8 6
1 10
4 14
5 22
6

i=1 j=1
27
 Merge-Sort: Merge Example

A:
1 2 3 4 6 10 14

k=3

L: R:
2 3 7 8 6
1 10
4 14
5 22
6

i=2 j=1
28
 Merge-Sort: Merge Example

A:
1 2 3 4 5 6 10 14

k=4

L: R:
2 3 7 8 6
1 10
4 14
5 22
6

i=2 j=2
29
 Merge-Sort: Merge Example

A:
1 2 3 4 5 6 10 14

k=5

L: R:
2 3 7 8 6
1 10
4 14
5 22
6

i=2 j=3
30
 Merge-Sort: Merge Example

A:
1 2 3 4 5 6 7 14

k=6

L: R:
2 3 7 8 6
1 10
4 14
5 22
6

i=2 j=4
31
 Merge-Sort: Merge Example

A:
1 2 3 4 5 6 7 8
14

k=7

L: R:
3
2 5
3 15
7 28
8 6
1 10
4 14
5 22
6

i=3 j=4
32
 Merge-Sort: Merge Example

A:
1 2 3 4 5 6 7 8

k=8

L: R:
3
2 5
3 15
7 28
8 6
1 10
4 14
5 22
6

i=4 j=4

33
Merge(A, left, middle, right)
1. n1 = middle – left + 1
2. n2 = right – middle
3. create array L[n1], R[n2]
4. for i = 0 to n1-1 do L[i] = A[left +i]
5. for j = 0 to n2-1 do R[j] = A[middle+j]
6. k = i = j = 0
7. while (i < n1 & j < n2)
8. if ( L[i] < R[j])
9. A[k++] = L[i++]
10. else
11. A[k++] = R[j++]
12. while i < n1
13. A[k++] = L[i++]
14. while j < n2
15. A[k++] = R[j++]

34
 Merge-Sort(A, 0, 7)
Divide
A: 6 2 8 4 33 7 7 55 11

35
Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 3) , divide
A: 3 7 5 1

6 2 88 44

36
Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 1) , divide
A: 3 7 5 1

8 4

6 22

37
Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 0) , base case
A: 3 7 5 1

8 4

38
Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 0), return
A: 3 7 5 1

8 4

6 2

39
Merge-Sort(A, 0, 7)
Merge-Sort(A, 1, 1) , base case
A: 3 7 5 1

8 4

40
Merge-Sort(A, 0, 7)
Merge-Sort(A, 1, 1), return
A: 3 7 5 1

8 4

6 2

41
Merge-Sort(A, 0, 7)
Merge(A, 0, 0, 1)
A: 3 7 5 1

8 4

2 6

42
Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 1), return
A: 3 7 5 1

2 6 8 4

43
Merge-Sort(A, 0, 7)
Merge-Sort(A, 2, 3) , divide
A: 3 7 5 1

2 6

8 4

44
Merge-Sort(A, 0, 7)
Merge-Sort(A, 2, 2), base case
A: 3 7 5 1

2 6

45
Merge-Sort(A, 0, 7)
Merge-Sort(A, 2, 2), return
A: 3 7 5 1

2 6

8 4

46
Merge-Sort(A, 0, 7)
Merge-Sort(A, 3, 3), base case
A:

2 6

47
Merge-Sort(A, 0, 7)
Merge-Sort(A, 3, 3), return
A: 3 7 5 1

2 6

8 4

48
Merge-Sort(A, 0, 7)
Merge(A, 2, 2, 3)
A: 3 7 5 1

2 6

4 8

49
Merge-Sort(A, 0, 7)
Merge-Sort(A, 2, 3), return
A: 3 7 5 1

2 6 4 8

50
Merge-Sort(A, 0, 7)
Merge(A, 0, 1, 3)
A: 3 7 5 1

2 4 6 8

51
Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 3), return
A: 2 4 6 8 3 7 5 1

52
Merge-Sort(A, 0, 7)
Merge-Sort(A, 4, 7)
A: 2 4 6 8

3 7 5 1

53
Merge-Sort(A, 0, 7)
Merge (A, 4, 5, 7)
A: 2 4 6 8

1 3 5 7

54
Merge-Sort(A, 0, 7)
Merge-Sort(A, 4, 7), return
A: 2 4 6 8 1 3 5 7

55
 Quick Sort

• Divide:
• Pick any element p as the pivot, e.g, the first element
• Partition the remaining elements into
FirstPart, which contains all elements < p
SecondPart, which contains all elements ≥ p

• Recursively sort the FirstPart and SecondPart

• Combine: no work is necessary since sorting

is done in place

56
 Quick Sort
A: p
pivot Partition
FirstPart SecondPart

x<p p p≤x
Recursive call
Sorted Sorted
FirstPart SecondPart

x<p p p≤x

Sorted 57
 Quick Sort
Quick-Sort(A, left, right)
if left ≥ right return
else
middle = Partition(A, left, right)
Quick-Sort(A, left, middle–1 )
Quick-Sort(A, middle+1, right)
end if

58
 Partition
A: p

A: p x<p p≤x

A:
p x<p p p≤x

59
 Partition Example

A: 4 8 6 3 5 1 7 2

60
 Partition Example

i=0

A: 4 8 6 3 5 1 7 2

j=1

61
 Partition Example

i=0

A: 4 8 6 3 5 1 7 2

j=1

62
 Partition Example

i=0

A: 4 8 6 3 5 1 7 2

j=2

63
 Partition Example

i=0i=1

A: 4 8
3 6 83 5 1 7 2

j=3

64
 Partition Example

i=1

A: 4 3 6 8 5 1 7 2

j=4

65
 Partition Example

i=1

A: 4 3 6 8 5 1 7 2

j=5

66
 Partition Example

i=2

A: 4 3 61 8 5 16 7 2

j=5

67
 Partition Example

i=2

A: 4 3 1 8 5 6 7 2

j=6

68
 Partition Example

i=2i=3

A: 4 3 1 82 5 6 7 82

j=7

69
 Partition Example

i=3

A: 4 3 1 2 5 6 7 8

j=8

70
 Partition Example

i=3

A: 24 3 1 24 5 6 7 8

71
 Partition Example
pivot in
correct position

A: 2 3 1 4 5 6 7 8

x<4 4≤x

72
Partition(A, left, right)
1. x = A[left]
2. i = left
3. for j = left+1 to right
4. {
5. if (A[j] < x){
6. i = i + 1
7. swap(A[i], A[j])
8. }
9. }
10.swap(A[i], A[left])
11.return i

73
 Quick-Sort(A, 0, 7)
Partition
A: 42 3
8 61 34 55 16 77 8
2

74
Quick-Sort(A, 0, 7)
Quick-Sort(A, 0, 2) , partition
A: 4 5 6 7 8

21 23 31

75
Quick-Sort(A, 0, 7)
Quick-Sort(A, 0, 0) , base
returncase
4 5 6 7 8

1 2 3

76
Quick-Sort(A, 0, 7)
Quick-Sort(A, 2, 2) , base case
4 5 6 7 8

1 2

77
Quick-Sort(A, 0, 7)
Quick-Sort(A,2,0,2),
Quick-Sort(A, 2),return
return
1 2 3 4 5 6 7 8

1 2 3

78
Quick-Sort(A, 0, 7)
Quick-Sort(A,2,4,2),
Quick-Sort(A, 7) return
, partition
1 2 3 4

55 6 7 8

79
Quick-Sort(A, 0, 7)
Quick-Sort(A, 5, 7) , partition
1 2 3 4

6 77 88

80
Quick-Sort(A, 0, 7)
Quick-Sort(A, 6, 7) , partition
1 2 3 4

7 88

81
Quick-Sort(A, 0, 7)
Quick-Sort(A, 7, 7) returncase
, base
1 2 3 4

7 8

82
Quick-Sort(A, 0, 7)
Quick-Sort(A, 6, 7) , return
1 2 3 4

6 7 8

83
Quick-Sort(A, 0, 7)
Quick-Sort(A, 5, 7) , return
1 2 3 4

5 6 7 8

84
Quick-Sort(A, 0, 7)
Quick-Sort(A, 4, 7) , return
1 2 3 4 5 6 7 8

85
Quick-Sort(A, 0, 7)
Quick-Sort(A, 0, 7) , done!
1 2 3 4 5 6 7 8

86
 Radix Sort
Extra information: every integer can be
represented by at most k digits
– d1d2…dk where di are digits in base r
– d1: most significant digit
– dk: least significant digit
 Radix Sort
• Algorithm
– sort by the least significant digit first (counting sort)
=> Numbers with the same digit go to same bin
– reorder all the numbers: the numbers in bin 0
precede the numbers in bin 1, which precede the
numbers in bin 2, and so on
– sort by the next least significant digit
– continue this process until the numbers have been
sorted on all k digits
 Radix Sort
• Assume each key has a link field. Then the keys in the
same bin are linked together into a chain:
– f[i], 0 ≤ i ≤ r (the pointer to the first record in bin i)
– e[i], (the pointer to the end record in bin i)
– The chain will operate as a queue.
 Radix Sort Example
list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10]

179 208 306 93 859 984 55 9 271 33

e[0] e[1] e[2] e[3] e[4] e[5] e[6] e[7] e[8] e[9]

33 859

271 93 984 55 306 208 179

f[0] f[1] f[2] f[3] f[4] f[5] f[6] f[7] f[8] f[9]
list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10]

271 93 33 984 55 306 208 179 859 9

 Radix Sort Example (Cont.)

list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10]

271 93 33 984 55 306 208 179 859 9

e[0] e[1] e[2] e[3] e[4] e[5] e[6] e[7] e[8] e[9]

208 859 179

306 33 55 271 984 93

f[0] f[1] f[2] f[3] f[4] f[5] f[6] f[7] f[8] f[9]

list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10]

306 208 9 33 55 859 271 179 984 93

 Radix Sort Example (Cont.)

list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10]

306 208 9 33 55 859 271 179 984 93

e[0] e[1] e[2] e[3] e[4] e[5] e[6] e[7] e[8] e[9]

93

55

33 271

9 179 208 306 859 984

f[0] f[1] f[2] f[3] f[4] f[5] f[6] f[7] f[8] f[9]
list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10]

9 33 55 93 179 208 271 306 859 948