L1.

Lecture 1
Dr. Sifat Momen
 An algorithm is a finite set of precise
instructions for performing a computation or
for solving a problem.
 It must produce correct result
 It must finish in some finite time
 You can represent an algorithm using pseudocode, flowchart, or
even actual code

L1.
2
input Algorithm output
(optional)

Computational procedure for solving a problem

L1.
7
Can you come up with an algorithm to compute the GCD
(Greatest Common Divisor) of two integers.

L1.
8
Can you come up with an algorithm to take an input (very long)
and compute whether the input is divisible by 11 or not?

L1.9
The theoretical study of design and
analysis of computer algorithms
Basic goals for an algorithm:
• always correct
• always terminates
• performance
 Performance often draws the line between
what is possible and what is impossible.
L1.
10
• Analysis: predict the cost of an algorithm in
terms of resources and performance

• Design: design algorithms which minimize the

cost

L1.
11
Example:
Imagine two friends, Alice and Bob are given the
task of writing an algorithm that can sort 10 million
numbers
Alice writes an algorithm that takes 2N2 instructions
and implements using computer that executes 10
billion instructions per second.
Bob writes an algorithm that takes 50NlgN
instructions and implements using computer that
executes only 10 million instructions per second. L1.

Which one runs faster? 12

L1.
13
Input: sequence a1, a2, …, an of numbers.

Output: permutation a'1, a'2, …, a'n such

that a'1  a'2 …  a'n .
Example:
Input: 8 2 4 9 3 6
Output: 2 3 4 6 8 9
L1.
14
• There are various sorting algorithms
including bubble sort, insertion sort, quick
sort, merge sort, bucket sort, shell sort
etc…
• Can either be a stable or an unstable sorting
algorithm.
• A sorting algorithm is said to be stable if
two objects with equal keys appear in the
same order in sorted output as they
L1.
appear in the input array to be sorted. 15
L1.
16
 INSERTION-SORT (A, n) ⊳ A[1 . . n]
for j ← 2 to n
do key ← A[ j]
i←j–1
“pseudocode” while i > 0 and A[i] > key
do A[i+1] ← A[i]
i←i–1
A[i+1] = key
1 i j n
A:
key
sorted L1.
17
8 2 4 9 3 6

L1.
18
8 2 4 9 3 6

L1.
19
8 2 4 9 3 6
2 8 4 9 3 6

L1.
20
8 2 4 9 3 6
2 8 4 9 3 6

L1.
21
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6

L1.
22
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6

L1.
23
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6

L1.
24
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6

L1.
25
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6

L1.
26
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6

L1.
27
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
2 3 4 6 8 9 done
L1.
28
int main(){
int arr[] = {10, 6, 3, 2, 1, 8};
int l = sizeof(arr)/sizeof(*arr);
print(arr, l);
insertionSort(arr, l);
print(arr, l);

}
L1.
29
void insertionSort(int A[], int length){
int key, i;
for(int j = 1; j < length; j++){
key = A[j];
i = j - 1;
while(i > -1 && A[i] > key){
A[i+1] = A[i];
i = i - 1;
}
A[i+1] = key;
}
L1.
} 30
void print(int a[], int length){
for(int i = 0; i < length; i++)
cout << a[i] <<" ";
cout <<endl;
}

L1.
31
L1.
32
• The running time depends on the input: an
already sorted sequence is easier to sort.
• Major Simplifying Convention:
Parameterize the running time by the size of
the input, since short sequences are easier to
sort than long ones.
TA(n) = time of A on length n inputs
• Generally, we seek upper bounds on the
running time, to have a guarantee of
performance. L1.
33
Worst-case: (usually)
• T(n) = maximum time of algorithm
on any input of size n.
Average-case: (sometimes)
• T(n) = expected time of algorithm
over all inputs of size n.
• Need assumption of statistical
distribution of inputs.
Best-case: (NEVER)
• Cheat with a slow algorithm that
works fast on some input. L1.
34
INSERTION-SORT (A, n) ⊳ A[1 . . n]
for j ← 2 to n
dokey ← A[ j]
i←j–1
while i > 0 and A[i] > key
doA[i+1] ← A[i]
i←i–1
A[i+1] = key

What is the estimated running time?

Depends on arrangement of numbers in the input array. We are typically
interested in the runtime of an algorithm in the worst case scenario.
Because it provides us a guarantee that the algorithm won’t take any longer than
this for any type of input.

How can you arrange the input numbers so that this

35
algorithm becomes most inefficient (worst case)?
• Analyzing an algorithm means predicting the resources that
the algorithm requires.
• Typically time and memory
• Before we can analyze an algorithm, we need a model of
implementation technologies that we will use.
• Random Access Machine Model

36
• Single Processor
• Instructions are executed one after another, with no
concurrent operations
• The following instructions take a constant amount of time
• Arithmetic: add, subtract, multiply, divide, remainder, floor,
ceiling etc…
• Data movement: load, store, copy
• Control: conditional/unconditional branch, subroutine call,
return
• Data type: Integer and Float

37
Here tj = no. of times the condition of while loop is tested for the current value of j.
In the worst case (when input is reverse-sorted), in each iteration of the for loop, all the j-1
elements need to be right shifted and the key will be inserted in the front of them, i.e., tj=j.
Using this in the above equation, we get: T(n) = An2+Bn+C, where A, B, C are constants. 38
What is T(n) in the best case (when the input numbers are already sorted)?
 The best case is when the input is already in the sorted manner.
 Thus tj = 1

This can be expressed as T(n) = an + b, thus

T(n) = O(n) 39
 The worst case results when the array is in the reverse order (in
this case decreasing order)
 In this situation, we must compare each element A[j] with each
element in the entire sorted sub-array A[1… j-1]
 This results tj = j

L1.
40
L1.
41