CS-3208: Design and Analysis of Algorithms

Dr. Wajid Arshad Abbasi

Department of Computer and Information Sciences
University of AJ&K
https://sites.google.com/view/wajidarshad/home

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT
 History
 Procedures for solving geometric and arithmetic problems were formulated
by ancient Greeks

 Some two thousands years ago, the celebrated procedure for finding
greatest common divisor (gcd) was discovered by Euclid

 The word algorithm comes from the name of the 9th century Persian
mathematician Abu Abdullah Muhammad ibn Musa al-Khwarzimi

 His works introduced algebraic concepts

 He worked in Baghdad at the time when it was the centre of scientific

studies and trade.

 Al-Khwarzimi's name was translated into Latin, and eventually became

algorithm

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 2
 Algorithms today
 The word originally referred only to the rules of performing
arithmetic

 The word evolved to include all definite procedures for solving

problems or performing tasks

 In the mid-twentieth century D.E. Knuth undertook in depth

study and analysis of algorithms

 His work is embodied in his comprehensive book ‘The art of

computer programming’, which serves as a foundation for
modern study of algorithms

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 3
 Definition
 An algorithm is an orderly step-by-step procedure, which has the
characteristics:

1) It accepts one or more input value

2) It returns at least one output value
3) It terminates after finite steps

 An algorithm may also be viewed as a tool for solving a computational

problem

‘Determine whether the number x is in the list S of n numbers. The answer is

Yes if x is in S and No if it is not’
S = [5, 7, 11, 4, 9] n = 5 x = 9 is an instance of the problem
Solution to this instance is ‘Yes’

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 4
 Applications
 Algorithms have been developed to solve an enormous variety of problems in
many application domains. Some broad categories of algorithms are listed
below.

 Sorting Algorithms
 Searching Algorithms
 String Processing (Pattern matching, Compression, Cryptography)
 Image Processing (Compression, Matching, Conversion)
 Mathematical Algorithms (Random number generator, matrix operations)

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 5
 Applications…
 Some applications do not explicitly require algorithmic content at the
application level

 Even so, it may rely heavily upon algorithms

 Does the application require fast hardware?

 Hardware design uses algorithms

 Does the application rely on networking?

 Routing in networks relies heavily on algorithms

 Does it use a language other than machine code?

 Compilers, Interpreters make extensive use of algorithms

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 6
 Analysis of Algorithms
 Analyzing an algorithm has come to mean predicting the resources that it
requires

 The purpose of algorithm analysis is to determine:

 Time efficiency
 Performance in terms of running times for different input sizes

 Space utilization
 Requirement of storage to run the algorithm

 Correctness of algorithm
 Results are trustworthy, and algorithm is robust

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 7
 Algorithm Efficiency
 Time efficiency remains an important consideration when developing
algorithms

 Algorithms designed to solve the same problem may differ

dramatically in efficiency

 These differences can be much more significant than differences due

to hardware and software

 Example : Sequential search vs. Binary search

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 8
 Algorithm Efficiency…

 The number of comparisons done by sequential search and binary search

when x (value being searched) is larger than all array items

Array Size Number of comparisons - Number of comparisons -

Sequential search Binary search
128 128 8
1,024 1,024 11
1,048,576 1,048,576 21
4,294,967,296 4,294,967,296 33

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 9
 Algorithm Efficiency…
 Another comparison in terms of algorithm execution time

Execution time of Algorithm 1 Execution time of Algorithm 2

41 ns 1048 µs
61 ns 1s
81 ns 18 min
101 ns 13 days
121 ns 36 years
161 ns 3.8 * 107 years
201 ns 4 * 1013 years

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 10
 Approaches to Analysis
 Basically three approaches can be adopted to analyze algorithm
running time in terms of input size:
 Empirical Approach
 Running time measured experimentally

 Analytical Approach
 Running time estimated using mathematical modeling

 Visualization
 Performance is studied through animation for different data sets

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 11
 Empirical Approach
 The running time of algorithm is measured for different data sizes and time
estimates are plotted against the input. The graph shows the trend.

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 12
 Limitations
 Running time critically depends on:
 Hardware resources used
 (CPU speed, IO throughput, RAM size)

 Software environment
 (Compiler, Programming Language)

 Program design approach

 (Structured, Object Oriented)

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 13
 Analytical Approach
 We want a measure that is independent of the computer, programming
language, and complex details of the algorithm

 Usually this measure is in terms of how many times a basic operation is

carried out for each value of the input size

 Strategy: Running time is estimated by analyzing the primitive operations

which make significant contributions to the overall algorithm time. These
may broadly include:

 Comparing data items

 Computing a value
 Moving a data item
 Calling a procedure

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 14
 Algorithm Specification
 Plain natural language
 High level description

 Pseudo Code
 Low level to facilitate analysis and implementation

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 15
Specification Using Natural Language
 Preorder Tree Traversal Algorithm
Step1. Push tree root to stack
Step2. Pop the stack. If stack is empty exit, else process the node
Step3. Travel down the tree following the left most path, and
pushing each right child onto the stack
Step4. When leaf node is reached, go back to step 2.

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 16
Pseudo Code Convention

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 17
 Example

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 18
 Algorithm Design
 There are many approaches to designing algorithms:
 Divide-and-Conquer
 Greedy
 Dynamic Programming
 Brute Force
 Approximation

 Each has certain advantages and limitations

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 19
 Take home task
1. Read first chapter “The Role of Algorithms in
Computing” of Introduction to Algorithms

2. Follow this tutorial during the whole course.

– https://www.tutorialspoint.com/design_and_ana
lysis_of_algorithms/index.htm

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 20
 End of lecture

Design and Analysis of Algorithms Dr. Wajid Arshad Abbasi, Department of CS&IT 21