DR. Gatot F. Hertono, MSc.

Design and Analysis of ALGORITHM

(Week 2)

Our Machine Model:

Assumptions

Generic Random Access Machine (RAM)

 Executes operations sequentially
 Set of basic operations:
 Arithmetic. Logical, Comparisons, Function
calls
 Simplifying assumption: all ops cost 1 unit

Notes:
 Eliminates dependence on the speed of our computer,
otherwise impossible to verify and to compare

1
 Running time
• The running time depends on the input.
Example: an already sorted sequence is easier to
sort.
• Major Simplifying Convention: Parameterize the
running time by the size of the input.
TA(n) = time of A on length n inputs
• Generally, we seek upper bounds on the running
time, to have a guarantee of performance.

Examples of Basic Operations

Algorithm Input Types Basic Operations

List Searching List with n elements Comparation
List Sorting List with n elements Comparation
Matrix Product n x n matrices Scalar Products
Prime Factorisation n digit numbers Scalar Division
Polynomial Evaluation n degree polynomial Scalar Products
Tree Traversal Tree with n nodes Visiting a node

Notes:
The running time of an algorithm is determined by its input size n

2
 Time Complexity
The complexity of an algorithm is determined by the number of
basic operations and how many time the algorithm computes those
basic operations.

Notes: The complexity analysis is machine independent.

Time complexity of an algorithm will determine the running time

depends on its input size, i.e. the time complexity is a function of
input size.
Time Complexity maps
“input size”
to
“time” T(n) executed.

Purpose
 To estimate how long a program will run.
 To estimate the largest input that can reasonably be
given to the program.
 To compare the efficiency of different algorithms.
 To help focus on the parts of code that are executed the
largest number of times.
 To choose an algorithm for an application.

3
 Time Complexity: an example

Best, Worst and Average Case

Sometimes, given two different inputs with a same size, an

algorithm can have different running time.

Example:
Suppose a sorting algorithm has some inputs with a same size
but different order:
In ascending order
-Input 1: 10, 5, 23, 45, 1, 100 Average case

-Input 2: 1,5,10, 23,45, 100 Best case

-Input 3: 100, 45, 23, 10, 5, 1 Worst case

Do those inputs give the same running time?

4
 Best, Worst and Average Case (cont.)

Best, Worst and Average Cases (cont.)

Let In denote a set of all input with size n of an algorithm and (i)
denote the number of primitive operations of the corresponding
algorithm when given input i.
Mathematically, we can define:

Best-case Complexity: is a function B(n)  B(n) = min{ (i)  i In }

Worst-case Complexity: is a function W(n)  W(n) = max{ (i)  i In }

Average-case Complexity: is a function A(n)  A(n) =  (i). p(i)

iI n

where p(i) is the probability of i occurs as an input of an algorithm.

5
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
2 3 4 6 8 9 done

Running Time of Insertion Sort

Insertion_Sort(A) Cost Times
1 for j  2 to length(A) c1 n
2 key  A(j) c2 n-1
3 i  j-1 c3 n-1
4 while i > 0 and A(i) > c4

n
t
j 2 j
key

n
5 A(i+1)  A(i) c5 t 1
j 2 j


n
t 1
6 ii–1 c6 j 2 j

7 A(i+1)  key c7 n-1

n = length(A)

tj = number of while loop execution for a certain value j

6
 Insertion Sort: an analysis
T (n)  c1n  c2 (n  1)  c3 (n  1)  c4  j 2 t j  c5  j 2 (t j  1) 
n n

c6  j 2 (t j  1)  c7 (n  1)
n

Best case: in an ordered list (i.e tj = 1, for j = 2, …, n)

T (n)  (c1  c2  c3  c4  c7 )n  (c2  c3  c4  c7 )

Worst case: in a reverse ordered list (i.e tj = j, for j = 2, …, n)

c4 c5 c6 2 c c c
T ( n)  (   )n  (c1  c2  c3  4  5  6  c7 )n 
2 2 2 2 2 2
(c2  c3  c4  c7 )

Time Complexity: a comparison

7
 Machine-independent time

What is insertion sort’s worst-case time?

BIG IDEAS:
• Ignore machine dependent constants,
otherwise impossible to verify and to compare algorithms

• Look at growth of T(n) as n → ∞ .

“Asymptotic Analysis”

Analysis

 Simplifications
 Ignore actual and abstract statement
costs
 Order of growth is the interesting
measure:
 Highest-order term is what counts
 Remember, we are doing asymptotic analysis
 As the input size grows larger it is the high
order term that dominates

8
Assignment 1
 In order to show that an algorithm is
not unique, design two different
algorithms of a specific problem.
 Design an algorithm and show its
time complexity to compute a product
of two n x n matrices (how is the time
complexity in the best and worst
cases?)

Performance & Speed