Analysis of Sorting

BIS
 The Sorting Problem

• Input:

– A sequence of n numbers a1, a2, . . . , an

• Output:

– A permutation (reordering) a1’, a2’, . . . , an’ of the

input sequence such that a1’ ≤ a2’ ≤ · · · ≤ an’

2
 Why Study Sorting Algorithms?
• There are a variety of situations that we can
encounter
– Do we have randomly ordered keys?
– Are all keys distinct?
– How large is the set of keys to be ordered?
– Need guaranteed performance?

• Various algorithms are better suited to some of

these situations

3
 Some Definitions
• Internal Sort
– The data to be sorted is all stored in the computer’s
main memory.
• External Sort
– Some of the data to be sorted might be stored in
some external, slower, device.
• In Place Sort
– The amount of extra space required to sort the data is
constant with the input size.

4
 Stability
• A STABLE sort preserves relative order of records with
equal keys
Sorted on first key:

Sort file on second key:

Records with key value

3 are not in order on
first key!!

5
 Bubble Sort
• Idea:
– Repeatedly pass through the array
– Swaps adjacent elements that are out of order

i
1 2 3 n

8 4 6 9 2 3 1
j

• Easier to implement, but slower than Insertion

sort

6
 Example
8 4 6 9 2 3 1 1 8 4 6 9 2 3
i=1 j i=2 j

8 4 6 9 2 1 3 1 2 8 4 6 9 3
i=1 j i=3 j

8 4 6 9 1 2 3 1 2 3 8 4 6 9
i=1 j i=4 j

8 4 6 1 9 2 3 1 2 3 4 8 6 9
i=1 j i=5 j

8 4 1 6 9 2 3 1 2 3 4 6 8 9
i=1 j i=6 j

8 1 4 6 9 2 3 1 2 3 4 6 8 9
i=1 j i=7
j
1 8 4 6 9 2 3
7
i=1 j
 Bubble Sort
Alg.: BUBBLESORT(A)
for i  1 to length[A]
do for j  length[A] downto i + 1
do if A[j] < A[j -1]
then exchange A[j]  A[j-1]
i
8 4 6 9 2 3 1
i=1 j

8
 Insertion Sort
• Idea: like sorting a hand of playing cards
– Start with an empty left hand and the cards facing
down on the table.
– Remove one card at a time from the table, and insert
it into the correct position in the left hand
• compare it with each of the cards already in the hand, from
right to left
– The cards held in the left hand are sorted
• these cards were originally the top cards of the pile on the
table

9
 Insertion Sort

To insert 12, we need to

make room for it by moving
first 36 and then 24.
6 10 24 36

12

10
 Insertion Sort

6 10 24 36

12

11
 Insertion Sort

6 10 24 3
6

12

12
 Insertion Sort
input array

5 2 4 6 1
3
at each iteration, the array is divided in two sub-arrays:

left sub-array right sub-array

sorted unsorted

13
Insertion Sort

14
 INSERTION-SORT
Alg.: INSERTION-SORT(A) 1 2 3 4 5 6 7 8

a1 a2 a3 a4 a5 a6 a7 a8
for j ← 2 to n
do key ← A[ j ] key
Insert A[ j ] into the sorted sequence A[1 . . j -1]

i←j-1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i–1
A[i + 1] ← key
• Insertion sort – sorts the elements in place
15
 Analysis of Insertion Sort
INSERTION-SORT(A) cost
for j ← 2 to n times
do key ← A[ j ] c1 n
c2
Insert A[ j ] into the sorted sequence A[1 . . j -1] n-1
i←j-1 0 n-1
 n-1
n
while i > 0 and A[i] > key c4 t
j 2 j


n
do A[i + 1] ← A[i] c5 j 2
(t j  1)

n
i←i–1 c6 j 2
(t j  1)

A[i + 1] ← key c7
tj: # of times the while statement is executed at iterationcj8 n-
n n 1
n
T (n) c1n  c2 (n  1)  c4 (n  1)  c5  t j  c6  t j  1 c7  t j  1 c8 (n  1)
j 2 j 2 j 2
16
 Best Case Analysis
• The array is already sorted “while i > 0 and A[i] > key”
– A[i] ≤ key upon the first time the while loop test is run
(when i = j -1)

– tj = 1

• T(n) = c1n + c2(n -1) + c4(n -1) + c5(n -1) +

c8(n-1) = (c1 + c2 + c4 + c5 + c8)n + (c2 + c4

+ c5 + c8) n n n
T (n) c1n  c2 (n  1)  c4 (n  1)  c5  t j  c6  t j  1 c7  t j  1 c8 (n  1)
= an + b = (n) j 2 j 2 j 2
17
 Worst Case Analysis
• The array is in reverse sorted order “while i > 0 and A[i] > key”
– Always A[i] > key in while loop test
– Have to compare key with all elements to the left of the j-th
position  compare with j-1 elements  tj = j
n
n(n  1) n
n(n  1) n
n(n  1)
using 
j 1
j
2
  j 
j 2 2
 1   ( j  1) 
j 2 2
we have:

 n ( n  1)  n ( n  1) n( n  1)
T ( n ) c1n  c2 ( n  1)  c4 ( n  1)  c5   1  c6  c7  c8 ( n  1)
 2  2 2

an 2  bn  c a quadratic function of n

• T(n) = (n2) order of growth in n 2

n n n
T (n) c1n  c2 (n  1)  c4 (n  1)  c5  t j  c6  t j  1 c7  t j  1 c8 (n  1)
j 2 j 2 j 2 18
 Comparisons and Exchanges in
Insertion Sort
INSERTION-SORT(A) cost times
c1 n
for j ← 2 to n
c2 n-1
do key ← A[ j ]
0 n-1
Insert A[ j ] into the sorted sequence A[1 . . j
-1] c4 n-1
n 2
/2 comparisons
c5
 t
n
i←j-1 j 2 j
c6
 (t
n
while i > 0 and A[i] > key j 2 j  1)
c7
do A[i + 1] ← A[i]  (t
n
 1)
n
2
/2 exchanges j 2 j
c8 n-1
i←i–1
A[i + 1] ← key 19
 Insertion Sort - Summary
• Advantages
– Good running time for “almost sorted” arrays (n)
• Disadvantages
 (n2) running time in worst and average case
  n2/2 comparisons and exchanges

20
Selection Sort
• Idea:
– Find the smallest element in the array
– Exchange it with the element in the first position
– Find the second smallest element and exchange it with
the element in the second position
– Continue until the array is sorted
• Disadvantage:
– Running time depends only slightly on the amount of
order in the file

21
 Example
8 4 6 9 2 3 1 1 2 3 4 9 6 8

1 4 6 9 2 3 8 1 2 3 4 6 9 8

1 2 6 9 4 3 8 1 2 3 4 6 8 9

1 2 3 9 4 6 8 1 2 3 4 6 8 9

22
 Selection Sort
Alg.: SELECTION-SORT(A)
n ← length[A] 8 4 6 9 2 3 1

for j ← 1 to n - 1
do smallest ← j
for i ← j + 1 to n
do if A[i] < A[smallest]
then smallest ← i
exchange A[j] ↔ A[smallest]

23
 Analysis of Selection Sort
Alg.: SELECTION-SORT(A) cost times

n ← length[A] c1 1

for j ← 1 to n - 1 c2 n

do smallest ← j c3 n-1
n2/2 c4 n 1

comparisons for i ← j + 1 to n  j 1
(n  j  1)
c5 n 1
do if A[i] < A[smallest]  j 1
(n  j )
n c6 n 1
exchanges then smallest ← i  j 1
(n  j )
c7 n-1
exchange A[j] ↔ A[smallest]
n 1 n 1 n 1
T ( n) c1  c2 n  c3 ( n  1)  c4  ( n  j  1)  c5  n  j   c6  n  j   c7 (n  1) (n 2 ) 24
j 1 j 1 j 2