ANALYSIS AND DESIGN OF ALGORITHMS

Dr. J. GHAYATHRI
Associate Professor
Algorithms and Computing

A presentation on algorithms and computing, including

the definition of an algorithm, its importance and
examples on solving problems using algorithms
 What is an algorithm?

An algorithm is a sequence For any given input, an

of unambiguous algorithm should provide
instructions for solving a
problem. the required output in a
finite amount of time.

It is a set of steps or
procedures that lead to a
desired outcome or output
Algorithm

An algorithm is a sequence
of unambiguous instructions
for solving a problem, i.e.,
for obtaining a required
output for any legitimate
input in a finite amount of
time.
 Computers and algorithms

❖ The instructions in an algorithm imply that

there is a computer or device capable of
understanding and following them.
❖ Before the electronic computer was invented,
the word "computer" meant a human being
involved in performing numeric calculations.
❖ Nowadays, computers are ubiquitous
electronic devices that have become
indispensable in almost everything we do.
 Important points - Reg Algorithms

1. The non-ambiguity requirement for each step of an algorithm cannot

be compromised.
2. The range of inputs for which an algorithm works has to be specified
carefully.
3. The same algorithm can be represented in several different ways.
4. There may exist several algorithms for solving the same problem.
5. Algorithms for the same problem can be based on very different
ideas and can solve the problem with dramatically different speeds.
 GCD calculation

Euclid’s algorithm is based on applying repeatedly the

equality

gcd(m, n) = gcd(n, m mod n),

where m mod n is the remainder of the division of m by

n, until m mod n is equal to 0.

Since gcd(m, 0) = m (why?), the last value of m is also

the greatest common divisor of the initial m and n.

For example, gcd(60, 24) can be computed as follows:

gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12.

 Algorithm in pseudocode
Euclid’s algorithm for ALGORITHM Euclid(m, n)
computing gcd(m, n) //Computes gcd(m, n) by Euclid’s algorithm
//Input: Two nonnegative, not-both-zero integers
m and n
Step 1 If n = 0, return the value of m
as the answer and stop; otherwise, //Output: Greatest common divisor of m and n

proceed to Step 2. while n ≠ 0 do

Step 2 Divide m by n and assign the r ←m mod n
value of the remainder to r.
m←n
Step 3 Assign the value of n to m n←r
and the value of r to n. Go to Step 1.
return m
 Consecutive integer checking method for
Computing gcd(m,n)

Consecutive integer checking algorithm

 Step 1 Assign the value of min{m,n} to t

 Step 2 Divide m by t. If the remainder is 0,

go to Step 3; otherwise, go to Step 4

 Step 3 Divide n by t. If the remainder is 0, return t

and stop; Otherwise, go to Step 4

 Step 4 Decrease t by 1 and go to Step 2

 Middle-school method for gcd(m,n)

Middle-school procedure gcd(m,n)

Step 1 Find the prime factorization of m

Step 2 Find the prime factorization of n

Step 3 Find all the common prime factors

Step 4 Compute the product of all the common

prime factors and return it as gcd(m,n)
 Sieve of Eratosthenes
Input: Integer n ≥ 2
Output: List of primes less than or equal to n
for p ← 2 to n do A[p] ← p
for p ← 2 to n do
if A[p] != 0 //p hasn’t been previously eliminated from the list
j ← p* p
while j ≤ n do
A[j] ← 0 //mark element as eliminated
j←j+p
Example: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
2 3 5 7 9 11 13 15 17 19 21 23 25
2 3 5 7 11 13 17 19 23 25
2 3 5 7 11 13 17 19 23
 Examples of algorithms

Algorithms can be used to

solve different problems,
including computing the
greatest common divisor
01
of two integers.
02 There are many different
algorithms that can be
used for the same
problem, and they can
vary in terms of their
speed and efficiency.
The nonambiguity
requirement for each step
03
of an algorithm cannot be
compromised.
The importance of algorithm design

Poorly designed algorithms can be slow,

01 inefficient, and may even fail to produce the
correct output.

The design of an algorithm can have a

02 significant impact on its speed and
efficiency.

Algorithm design involves choosing the right

03
data structures and choosing the best
algorithm for a given problem.
 Practical applications of algorithms

● Algorithms are used in a wide range of applications,

including search engines, social media, and ecommerce.
● Optimization algorithms can be used to improve
performance and efficiency in various fields, including
finance, logistics, and transportation.
● Machine learning algorithms can be used to analyze and
interpret data, recognize patterns, and make predictions.
 Challenges and limitations of algorithms

Algorithms can also have ethical

Algorithms may have biases
and social implications, such as
or inaccuracies due to
privacy concerns, discrimination, or
factors such as incomplete
unintended consequences.
data or flawed assumptions.

Complex problems may require

massive amounts of computation,
which can be time-consuming and
resource-intensive.
Fundamentals of Algorithmic Problem Solving
A sequence of steps - in designing and analyzing an algorithm
1) Understanding the Problem
2) Ascertaining the Capabilities of the Computational Device
3) Choosing between Exact and Approximate Problem Solving
4) Algorithm Design Techniques
5) Designing an Algorithm and Data Structures
6) Methods of Specifying an Algorithm
7) Proving an Algorithm’s Correctness
8) Analyzing an Algorithm
9) Coding an Algorithm
Fundamentals of Algorithmic Problem solving
 1. Understanding the Problem

1. Read the problem’s description carefully

2. Ask questions if you have any doubts about the problem

3. Do a few small examples by hand

4. Think about special cases, and ask questions again if

needed.
 1. Understanding the Problem[Cont..]
 An input to an algorithm specifies an instance of the problem the
algorithm solves.
 It is very important to specify exactly the set of instances the
algorithm needs to handle.
 If you fail to do this, your algorithm may work correctly for a
majority of inputs but crash on some “boundary” value.
 A correct algorithm is not one that works most of the time, but one
that works correctly for all legitimate inputs.
 Do not skimp on this first step of the algorithmic problem-solving
process; otherwise, you will run the risk of unnecessary rework
2. Ascertaining the Capabilities of the Computational Device

Sequential algorithms :
Algorithms in use today are still meant to be programmed for a computer
closely resembling the von Neumann machine
The essence of this architecture is used by random-access machine (RAM).
In this, instructions are executed one after another, one operation at a time.
Algorithms designed to be executed on this RAM are called sequential
algorithms.
Parallel Algorithms:
Some newer computers that can execute operations concurrently, i.e., in
parallel. Algorithms using this capability are called parallel algorithms.

Still, studying the classic techniques for design and analysis of algorithms under the RAM model remains the
cornerstone of algorithmics for the foreseeable future
 3. Choosing between Exact and Approximate Problem Solving

Exact algorithm
Solving the problem exactly

Approximation algorithm
Solving problem approximately.
Why?
1. cannot be solved exactly for most of their instances
Example: Extracting square roots, solving nonlinear equations, and
evaluating definite integrals.
2. Available algorithms for solving a problem exactly can be
unacceptably slow because of the problem’s intrinsic complexity
 4. Algorithm Design Techniques

An algorithm design technique (or “strategy” or “paradigm”) is a

general approach to solving problems algorithmically that is
applicable to a variety of problems from different areas of
computing.
 5. Designing an Algorithm and Data Structures

Algorithms + Data Structures = Programs

In the new world of object-oriented programming,

data structures remain crucially important for
both design and analysis of algorithms
 6. Methods of Specifying an Algorithm

Two options to specify the Algorithms :

 In words (in a free and also a step-by-step form)

 Pseudo code

Pseudocode
 A mixture of a natural language and programming languagelike
constructs.
 Its usage often yields more crisp algorithm descriptions.
 7. Proving an Algorithm’s Correctness

 Once an algorithm has been specified, you have to prove its

correctness.
 Prove that the algorithm yields a required result for every legitimate
input in a finite amount of time.
 Example, the correctness of Euclid’s algorithm for computing the
greatest common divisor stems from the correctness of the equality
gcd(m, n) = gcd(n, m mod n)
 7. Proving an Algorithm’s Correctness [Cont]

 Use mathematical induction - because an algorithm’s

iterations provide a natural sequence of steps needed for
proofs.
 For exact Algorithm proof should be exact
 For an approximation algorithm, error produced by the
algorithm should not exceed a predefined limit
 8. Analyzing an Algorithm

 There are two kinds of algorithm Efficiency:

 Time efficiency, indicating how fast the algorithm runs,
 Space efficiency, indicating how much extra memory it uses.
 Another desirable characteristic of an algorithm is Simplicity
 Because simpler algorithms are easier to understand and easier
to program; consequently, the resulting programs usually
contain fewer bugs.
 8. Analyzing an Algorithm[Cont..]

Yet another desirable characteristic of an algorithm is generality.,

Two issues here:
Generality of the problem the algorithm solves
Set of inputs it accepts.
 9. Coding an Algorithm

Programming an algorithm presents both a peril and an opportunity.

The peril lies in the possibility of making the transition from an
algorithm to a program either incorrectly or very inefficiently.
As a practical matter, the validity of programs is still established by
testing. Testing of computer programs is an art rather than a
science, but that does not mean that there is nothing in it to learn.
Standard tricks as computing a loop’s invariant (an expression that
does not change its value) outside the loop, collecting common
subexpressions, replacing expensive operations by cheap ones,
and so on.
As a rule, a good algorithm is a result
of repeated effort and rework.
 Important Problem Types

 Sorting
 Searching
 String processing
 Graph problems
 Combinatorial problems
 Geometric problems
 Numerical problems
 SORTING

Rearranging the items of a given list in nondecreasing order.

To be meaningful, the nature of the list items must allow such an ordering.
(Mathematicians would say that there must exist a relation of total
ordering.)
Example: sort lists of numbers, characters from an alphabet, character
strings, and, most important, records of schools about their students,
libraries about their holdings, and companies about their employees.
KEY
In the case of records, we need to choose a piece of information to guide
sorting. That piece of information is called a key.
Example, Choose student records to sort in alphabetical order of names or by
student number or by student grade-point average.
 SORTING[Cont..]
Two properties : Stable and In-place
(i) Stable :
 Algorithm that preserves the relative order of any two equal elements in its
input
 Ie., if an input list contains two equal elements in positions i and j where i < j,
then in the sorted list they have to be in positions i and j such that i < j
 Example: To sort list of students alphabetically according to student GPA: a
stable algorithm will yield a list in which students with the same GPA will
still be sorted alphabetically
(ii) In-place :
 An algorithm does not require extra memory, except for few memory units
 SEARCHING

 It deals with finding a given value, called a search key, in a given set
 Straightforward sequential search to a spectacularly efficient but
limited binary search and algorithms based on representing the
underlying set in a different form more conducive to searching.
 No single algorithm that fits all situations best
 Searching has to be considered with two other operations:
 An addition to and deletion from the data set of an item.
 Choose balanced data structures and algorithms
 Organizing very large data sets for efficient searching poses special
challenges with important implications for real-world applications
 STRING PROCESSING

sequence of characters from an alphabet

Examples:
Text strings, which comprise letters, numbers, and special characters; bit
strings, which comprise zeros and ones; and gene sequences, which can
be modeled by strings of characters from the four-character alphabet {A,
C, G, T}.

Problem
string matching.
 Graph Problem
Collection of points called vertices, some of which are connected by line
segments called edges
Transportation, communication, social and economic networks, project
scheduling, and games.
Basic graph algorithms include
Graph-traversal algorithms (how to reach all the points in a network?),
Shortest-path algorithms (what is the best route between two cities?),
Topological sorting for graphs with directed edges (is a set of courses
with their prerequisites consistent or self-contradictory?).
 Graph Problem[Cont..]

Some graph problems are computationally very hard; the most well-known
examples are the traveling salesman problem and the graph-coloring
problem.
The traveling salesman problem (TSP)is the problem of finding the
shortest tour through n cities that visits every city exactly once.
In addition to obvious applications involving route planning, it arises in
such modern applications as circuit board and VLSI chip fabrication, X-
ray crystallography, and genetic engineering.
The graph-coloring problem the vertices of a graph so that no two adjacent
vertices are the same color : a solution to the graph-coloring problem
yields an optimal schedule.
 Combinatorial Problems
To find a combinatorial object—such as a Permutation, a combination
When the order doesn't matter, it is a Combination
When the order does matter it is a Permutation
or a subset—that satisfies certain constraints.
A desired combinatorial object may also be required to have some
additional property such as a maximum value or a minimum cost.
Combinatorial problems are the most difficult problems in computing
First, the number of combinatorial objects typically grows extremely fast
with a problem’s size, reaching unimaginable magnitudes even for
moderate-sized instances.
Second, there are no known algorithms for solving most such problems
exactly in an acceptable amount of time
 Geometric Problems
Deals with geometric objects such as points, lines, and polygons.
The ancient Greeks were very much interested in developing procedures
for solving a variety of geometric problems, including problems of
constructing simple geometric shapes—triangles, circles, and so on—
with an unmarked ruler and a compass.
Today people are interested in geometric algorithms with quite different
applications in mind, such as computer graphics, robotics, and
Tomography (x-ray technique)
Classic problems of computational geometry:
(i) Closest-pair problem (ii) convex-hull problem.
(i) Closest-pair problem : Given n points in the plane, find the closest pair
among them.
(ii) Convex-hull problem : Finds the smallest convex polygon that would
include all the points of a given set.
 Numerical Problems

 Large special area of applications,

 Problems that involve mathematical objects of continuous nature:
 solving equations and systems of equations, computing definite
integrals, evaluating functions, and so on.
 Mathematical problems can be solved only approximately for
problems typically require manipulating real numbers
 Lead to an accumulation of the round-off error
 ANALYSIS OF ALGORITHMS

How good is the algorithm?

• Correctness

• Time Efficiency – Time Complexity

• Space Efficiency – Space Complexity

Does there exist a better algorithm?

• Lower bounds

• Optimality
 Analysis of algorithms

• Issues:
– correctness
– Time efficiency
– Space efficiency
– Optimality

• Approaches:
– Theoretical analysis
– Empirical analysis
Theoretical analysis of time efficiency (Measuring Running Time)

Time efficiency is analyzed by determining the number of repetitions of the

basic operation as a function of input size

Basic operation: the operation that contributes most towards the running time
of the algorithm Cop - Execution Time of the
input size Algorithm

C(n) – Number of times the

T(n) ≈ copC(n) operation executed

T(n) – Running Time of a program

Running time Number of times

Execution time
basic operation is
for basic operation
executed
 Input size and basic operation examples

Problem Input size measure Basic operation

Searching for key in a

Number of list’s items, i.e. n Key comparison
list of n items

Multiplication of two Matrix dimensions or total Multiplication of two

matrices number of elements numbers

Checking primality of a n’size = number of digits

Division
given integer n (in binary representation)

Visiting a vertex or
Typical graph problem #vertices and/or edges
traversing an edge
Assume
Double the size of input
 Orders of growth

• Most important: Order of growth within a constant multiple as n→∞

• Example:
– How much faster will algorithm run on computer that is twice as fast?

– How much longer does it take to solve problem of double input size?
Orders of Growth (Constant Multiples for large-size inputs)
Values of some important functions as n  
 Empirical analysis of time efficiency

• Select a specific (typical) sample of inputs

• Use physical unit of time (e.g., milliseconds)

or
Count actual number of basic operation’s executions

• Analyze the empirical data

 Best-case, average-case, worst-case

For some algorithms efficiency depends on form of input:

• Worst case: Cworst(n) – maximum over inputs of size n

• Best case: Cbest(n) – minimum over inputs of size n

• Average case: Cavg(n) – “average” over inputs of size n

– Number of times the basic operation will be executed on typical input

– NOT the average of worst and best case

– Expected number of basic operations considered as a random variable

under some assumption about the probability distribution of all possible
inputs
Example: Sequential search
 Worst, Best and Average Efficiencies
Worst Case:
Best Case:
First matching element
Input of size n, for which the
happens to be the last one
algorithm runs the fastest
on the list, the algorithm
among all possible inputs of
makes the largest number
01 that size.
of key comparisons among
Best-case inputs for sequential
all possible inputs of size n: 02 search are lists of size n with
their first element equal to a
search key; accordingly,

03 for this algorithm.

Average Case
Algorithm’s behavior on a “typical” or “random” input.
To analyze assume about possible inputs of size.
 Average-Case Efficiency

Standard assumptions
(a) the probability of a successful search is equal to p (0 ≤ p ≤ 1)
(b) the probability of the first match occurring in the ith position of the list
is the same for every i.
We can find the average number of key comparisons Cavg(n) as follows.
● Case (a) Successful search, the probability of the first match occurring in the
ith position of the list is p/n for every i, and the number of comparisons made
by the algorithm in such a situation is obviously i.
● Case (b) Unsuccessful search, the number of comparisons will be n with the
probability of such a search being (1 − p). Therefore
 Average-Case Efficiency

If p=1
Asymptotic Notations and Basic Efficiency Classes

 (Big Oh) - Ο
 (Big Omega) - Ω
 (Big Theta) Θ
 t (n) and
 g(n) can be any nonnegative functions defined on the set of
natural numbers
 t(n) -- An algorithm’s running time
 g(n) -- Some simple function to compare the count with.
 NUMBER SYSTEM

 Natural Numbers: Range from 1 to infinity. Positive Numbers / Counting Numbers. Represented

by the symbol N.

Example: 1, 2, 3, 4, 5, 6, 7, and so on.

 Whole Numbers: Natural Numbers including ‘zero’. Represented by the symbol W.

Example: 0, 1, 2, 3, 4, and so on.

 Integers: Collection of Whole Numbers plus the negative values of the Natural Numbers. From

the Infinity at the Negative end and Infinity at the Positive end, including zero. Integers are

represented by the symbol Z.

Example: ...,-4, -3, -2, -1, 0, 1, 2, 3, 4,...

 Fractions: Written in the form of x/y, where, x belongs to Whole numbers and y belongs to

Natural Numbers, i.e., y can never be 0.

Example: 1/2, 3/7, 8/3, etc.

 NUMBER SYSTEM

 Rational Numbers: Represented in the fraction form i.e. x/y. Here, x and y both are integers and y≠0. All the
fractions are rational numbers but not all the rational numbers are fractions.

Example: -2/5, 0.54, 1/5, 13/4, ...

 Irrational Numbers: Irrational numbers are the numbers that can’t be represented in the form of fractions i.e.
they can not be written as x/y.

Example: √2, √3, √.434343, π...

 Real and Imaginary Numbers: Numbers represented in the decimal form. These numbers include whole
numbers, integers, fractions, etc. All the integers belong to Real numbers but all the real numbers do not belong to
the integers.

Imaginary Numbers are all those numbers that are not real numbers. These numbers when squared will result in a
negative number. The √-1 is represented as i. These numbers are also called complex numbers.

Example: √-2, √-5,...

 Prime Numbers and Composite Numbers: Numbers that do not have any factors other than 1 and the
number itself are termed as Prime Numbers. All the numbers other than Prime Numbers are termed as Composite
Numbers except 0. Zero is neither prime nor a composite number.

Example: 2, 3, 5, 7,... are prime numbers and

4, 6, 8, 9, 12,... are composite numbers

 Asymptotic order of growth

A way of comparing functions that ignores constant factors and

small input sizes

• O(g(n)) : Class of functions f(n) that grow no faster than g(n)

• Θ(g(n)) : Class of functions f(n) that grow at same rate as g(n)

• Ω(g(n)) : Class of functions f(n) that grow at least as fast as g(n)

 Informal Introduction

• Informally, O(g(n)) is the set of all functions with a lower or same

order of growth as g(n)

(to within a constant multiple, as n goes to infinity).

• Examples, the following assertions are all true:

These functions are linear & Lower order of
growth than g(n) = 𝒏𝟐

This is quadratic & the same order of growth as 𝒏𝟐 .

The functions 𝑛3 & 0.00001𝑛3 are both

cubic and have a higher order of
growth than 𝒏𝟐 ,
Same for fourth-degree polynomial
𝒏𝟒 + n + 1 also
 Ω(g(n))
Ω(g(n)) - Set of all functions with a higher or same order of growth as g(n)

(to within a constant multiple, as n goes to infinity).

For example,

But
 Θ(g(n))

 Θ(g(n)) is the set of all functions that have the same order
of growth as g(n)

(to within a constant multiple, as n goes to infinity).

 Every quadratic function a𝑛2 + bn + c with a > 0 is in Θ(𝑛2 ),

 Infinitely in,

𝑛2 + sin n

𝑛2 + log n.
 O – notation (FORMAL)

Definition:

A function t(n) is said to be in O(g(n)), denoted t (n) ∈ O(g(n))

if t (n) is bounded above by some constant multiple of g(n) for all

large n, i.e., if there exist some positive constant c and some non-negative
integer n0 such that
Big-oh O – notation (FORMAL)
 O – notation (FORMAL)

Prove : 100n + 5 ∈ O(𝑛2 )

Indeed, for Constants c=101 and n0 = 5,

100n + 5 ≤ 100n + n (for all n ≥ 5) = 101n ≤ 101 𝑛2

For c = 105 and n0 = 1.

100n + 5 ≤ 100n + 5n (for all n ≥ 1) = 105n

 Ω-notation
DEFINITION

A function t (n) is said to be in Ω(g(n)), denoted t (n) ∈ Ω(g(n)), if t (n) is

bounded below by some positive constant multiple of g(n) for all large
n, i.e., if there exist some positive constant c and some nonnegative
integer n0 such that

t (n) ≥ cg(n) for all n ≥ n0.

Big-omega Ω-notation
 Ω-notation

Proof for 𝑛3 ∈ Ω(𝑛2 ):

(Assume c = 1 and n0 = 0)
𝑛3 ≥ 𝑛2 for all n ≥ 0

Big Oh Left side smaller order of growth

Big Omega High order of growth
 Θ-notation
DEFINITION

• A function t (n) is said to be in Θ(g(n)), denoted t (n) ∈ Θ(g(n)),

if t (n) is bounded both above and below by some positive
constant multiples of g(n) for all large n, i.e., if there exist
some positive constants c1 and c2 and some nonnegative
integer n0 such that

c2g(n) ≤ t (n) ≤ c1g(n) for all n ≥ n0.

Big-theta Θ-notation
Useful Property Involving the Asymptotic Notations
 Some properties of asymptotic order of growth

 f(n)  O(f(n))

 f(n)  O(g(n)) iff g(n) (f(n))

 If f (n)  O(g (n)) and g(n)  O(h(n)) , then f(n)  O(h(n))

Note similarity with a ≤ b

 If f1(n)  O(g1(n)) and f2(n)  O(g2(n)) , then

f1(n) + f2(n)  O(max{g1(n), g2(n)})
 Establishing order of growth using limits

0 order of growth of T(n) < order of growth of g(n)

c > 0 order of growth of T(n) = order of growth of g(n)

lim T(n)/g(n) =
n→∞
∞ order of growth of T(n) > order of growth of g(n)

Examples:
• 10n vs. n2

• n(n+1)/2 vs. n2
 L’Hôpital’s rule and Stirling’s formula

L’Hôpital’s rule:
If limn f(n) = limn g(n) =  and the derivatives f´, g´ exist, then

Example: log n vs. n

Stirling’s formula

Example: 2n vs. n!
 Properties of Limits
 limx→a c = c, where c is a constant quantity.

 The value of limx→a x = a

 Value of limx→a bx + c = ba + c

 limx→a xn = an, if n is a positive integer.

 Value of limx→0+ 1/xr = +∞.

 limx→0− 1/xr = −∞, if r is odd, and

 limx→0− 1/xr = +∞, if r is even.

 Compare the orders of growth of

Since the limit is equal to a positive constant, the functions have the same order
of growth or, symbolically,
 Orders of growth of some important functions

• All logarithmic functions loga n belong to the same class

(log n) no matter what the logarithm’s base a > 1 is

• All polynomials of the same degree k belong to the same class:

aknk + ak-1nk-1 + … + a0  (nk)

• Exponential functions an have different orders of growth for different a’s

• order log n < order n (>0) < order an < order n! < order nn
 Basic Efficiency classes
Class Name Comments
1 Constant Short of Best-case

log n Logarithmic Linear running time

Algs scans list of size n

n Linear
(Sequential search )
Divide and conquer alg (average
n log n n-log-n
case)
𝑛2 Quadratic Two Nested loops (Sorting algs)
Three Nested loops (non trivial alg
𝒏𝟑 Cubic
from algebra)
All subsets (Larger order of
2𝑛 Exponential
growth) of n-elements set
All permutations of an n-element
n! Factorial
set
 Time efficiency of nonrecursive algorithms

General Plan for Analysis

• Decide on parameter n indicating input size

• Identify algorithm’s basic operation

• Determine worst, average, and best cases for input of size n

• Set up a sum for the number of times the basic operation is executed

• Simplify the sum using standard formulas and rules

 Useful summation formulas and rules

liu1 = 1+1+…+1 = u - l + 1
In particular, liu1 = n - 1 + 1 = n  (n)

1in i = 1+2+…+n = n(n+1)/2  n2/2  (n2)

1in i2 = 12+22+…+n2 = n(n+1)(2n+1)/6  n3/3  (n3)

0in ai = 1 + a +…+ an = (an+1 - 1)/(a - 1) for any a  1

In particular, 0in 2i = 20 + 21 +…+ 2n = 2n+1 - 1  (2n )

(ai ± bi ) = ai ± bi cai = cai liuai = limai + m+1iuai

 Mathematical Analysis of Nonrecursive Algorithms
Example 1: Maximum element
Problem of finding the value of the largest element in a list of n numbers
 Problem of finding the value of the largest element in a list of n numbers
Algorithm Metric Analysis
• Input’s size = Number of elements in the array - n.

• There are two operations in the loop’s body:

(i) Comparison A[i] > maxval

(ii) Assignment maxval ← A[i].

• Basic operation –Comparison is executed on each repetition of the loop

The assignment is not executed on each repetition of the loop

so comparison to be the algorithm’s basic operation.

• Number of comparisons same for all arrays of size n;

• So, no need to distinguish among the worst, average, and best cases here
• C(n) the number of times comparison is executed

• The algorithm makes one comparison on each execution of the loop, &
repeated for each value of the loop’s variable i within 1 and n − 1inclusive.

• Sum for C(n):

This is an easy sum to compute because it is nothing other than 1

repeated n − 1 times. Thus,
Frequently used two basic rules of sum manipulation

R1

R2
Two summation formulas
 Element uniqueness problem
check whether all the elements in a given array of n elements are distinct.
• The natural measure of the input’s size = n, (number of elements in the
array).

• Innermost loop contains a single operation - the comparison of two

elements, considered as the algorithm’s basic operation.

• Number of element comparisons depends on n and on whether there are

equal elements in the array (array positions they occupy).

• worst case only.

Two kinds of worst-case inputs :

(i) arrays with no equal elements

(ii) arrays in which the last two elements are the only pair of equal
elements.

• For such inputs, one comparison is made for each repetition of the
innermost loop, i.e., for each value of the loop variable j between

its limits i + 1 and n − 1;

• This is repeated for each value of the outer loop, i.e., for each value of the
loop variable i between its limits 0 and n − 2.
 Compute the sum faster using

The last equality is obtained by applying summation formula (S2).

Given two n × n matrices A and B,
Find the time efficiency of the definition-based algorithm for computing their
product C = AB.
By definition, C is an n × n matrix whose elements are computed as the scalar
(dot) products of the rows of matrix A and the columns of matrix B:
Example 3: Matrix multiplication
 Matrix multiplication
 Measure an input’s size by matrix order n.

 There are two arithmetical operations in the innermost loop here—

multiplication and addition—algorithm’s basic operation.

 On each repetition of the innermost loop each of the two is executed exactly

once.

 Consider multiplication as the basic operation

 sum for the total number of multiplications M(n) executed by the algorithm.

 One multiplication executed on each repetition of the algorithm’s innermost

loop, using the variable k ranging 0 to n − 1.

 Therefore, the number of multiplications made for every pair of specific

values of variables i and j is

• The total number of multiplications M(n) is expressed by the following
triple sum

• Compute this sum by using formula (S1) and rule (R1) given above.
Starting with the innermost sum which is equal to n, we get
• Running time of the algorithm on a particular machine

• cm is the time of one multiplication on the machine in question.

• More accurate estimate by taking into account of the time spent on the
additions too where ca is the time of one addition
 Example 4: Counting binary digits

It cannot be investigated the way the previous examples are.

• Most frequently executed operation here is not inside the while loop

• Comparison n > 1 determines loop’s body execution status

• Number of times the comparison executed is larger than the number of repetitions
of the loop’s body by exactly 1, the choice is not that important.

• The loop variable takes on only a few values between its lower and upper limits;

• So use an alternative way of computing the number of times the loop is executed.

• We could also get this answer by applying the analysis technique based on
recurrence relations; we discuss this technique in the next section because it is
more pertinent to the analysis of recursive algorithms
 General Plan for Analysis of Recursive Algorithms

• Decide on a parameter indicating an input’s size.

• Identify the algorithm’s basic operation.

• Check whether the number of times the basic op. is executed may vary on
different inputs of the same size.

(If it may, the worst, average, and best cases must be investigated
separately)

• Set up a recurrence relation with an appropriate initial condition

expressing the number of times the basic op. is executed.

• Solve the recurrence (or, at the very least, establish its solution’s order of
growth) by backward substitutions or another method.
 Example 1: Recursive evaluation of n!

Definition: n ! = 1  2  … (n-1)  n for n ≥ 1 and 0! = 1

Recursive definition of n!: F(n) = F(n-1)  n for n ≥ 1 and

F(0) = 1

Size:
Basic operation:
Recurrence relation:
• For simplicity, consider n as input size

• Basic operation of the algorithm is multiplication

• Number of executions is M(n).

• Function F (n) is computed using the formula

F(n) = F(n − 1) . n for n > 0

The number of multiplications M(n) must satisfy the equality

M(n − 1) multiplications are spent to compute F (n − 1), and one

more multiplication is needed to multiply the result by n.
Recurrence Relations (Recurrences)

• This equation defines M(n) not explicitly, i.e., as a function of n, but

implicitly as a function of its value at another point, namely n − 1.

• Recurrence relations used in

– (i)Analysis of algorithms

– (ii) some areas of applied mathematics.

• Goal : To solve the recurrence relation

M(n) = M(n − 1) + 1, i.e., to find an explicit formula for M(n) in terms of n only.
• Solve the recurrence relation M(n) = M(n − 1) + 1,

• Find an initial condition : The value with which the sequence starts.

• Obtain this value by inspecting the condition that makes the algorithm stop
its recursive calls:

if n = 0 return 1.

• This tells us two things.

(i) Calls stop when n = 0, the smallest value of n for the execution of
the algorithm and M(n) is 0.

(ii) By inspecting the pseudocode’s exiting line, when n = 0, the

algorithm performs no multiplications.
 Solving the recurrence for M(n)

Initial condition

Setting up the recurrence relation and initial condition for the algorithm’s
number of multiplications M(n):
Two Recursive functions used are
(i) Factorial function F(n) itself;
F(n) = F(n − 1).n for every n > 0, F (0) = 1.
(ii)The number of multiplications
M(n) needed to compute F(n) by the recursive algorithm

Method of backward substitutions.:

M(n) = M(n − 1) + 1
substitute M(n − 1) = M(n − 2) + 1 = [M(n − 2) + 1] + 1 = M(n − 2) + 2
substitute M(n − 2) = M(n − 3) + 1 = [M(n − 3) + 1] + 2 = M(n − 3) + 3

General formula for the pattern: M(n) = M(n − i) + i.

 Tower of Hanoi puzzle

• Initial : n disks of different sizes that can slide onto any of three pegs.

• Initially, all the disks are on the first peg in order of size, the largest on the
bottom and the smallest on top.

• Condition: (i) Move only one disk at a time,

(ii) it is forbidden to place a larger disk on top of a smaller one.

• Goal : To move all the disks to the third peg, using the second one as an
auxiliary, if necessary.
1) Move n > 1 disks from peg 1 to peg 3 - peg 2 auxiliary

2) Move recursively n − 1 disks from peg 1 to peg 2 - peg 3 auxiliary

3) Move the largest disk directly from peg 1 to peg 3,

4) Move recursively n − 1 disks from peg 2 to peg 3 - peg 1 auxiliary

5) if n = 1, move the single disk directly from the source peg to the destination

peg.
 The Tower of Hanoi Puzzle

Recurrence for number of moves:

• Apply the general plan.

• The number of disks n is the input’s size indicator,

• Basic operation : Moving one disk.

• The number of moves M(n) depends on n , the recurrence equation is:

M(n) = M(n − 1) + 1 + M(n − 1) for n > 1.

• With the obvious initial condition M(1) = 1, the recurrence relation for the
number of moves M(n):

M(n) = 2M(n − 1) + 1 for n > 1, (2.3) M(1) = 1

 Solving recurrence for number of moves

Solving this recurrence by the same method of backward substitutions:

M(n) = 2M(n − 1) + 1 sub. M(n − 1) = 2M(n − 2) + 1
= 2[2M(n − 2) + 1] + 1 = 22 M(n − 2) + 2 + 1 sub. M(n − 2) = 2M(n − 3) + 1
= 22 [2M(n − 3) + 1] + 2 + 1 = 23 M(n − 3) + 22 + 2 + 1.

Next one will be 24 M(n − 4) + 23 + 22 + 2 + 1 as per pattern

Generally, after i substitutions, we get

Initial condition n = 1, achieved for i = n − 1, formula for the solution to recurrence :

• It is the problem’s intrinsic difficulty that makes it so computationally hard.

• Careful with recursive algorithms because their succinctness may mask

their inefficiency.

• When a recursive algorithm makes more than a single call to itself, it can
be useful for analysis purposes to construct a tree of its recursive calls.

• In this tree, nodes correspond to recursive calls, and we can label them
with the value of the parameter of the calls.

• By counting the number of nodes in the tree, we can get the total number of
calls made by the Tower of Hanoi algorithm
Tree of calls for the Tower of Hanoi Puzzle
Counting #bits
• Set up a recurrence and an initial condition for the number of additions
A(n) made by the algorithm.

• The number of additions made in computing BinRec([n/2]) is A([n/2]),

plus one more addition is made by the algorithm to increase the
returned value by 1.

• This leads to the recurrence

• Since the recursive calls end when n is equal to 1 and there are no
additions made then, the initial condition is A(1) = 0.
• The presence of [n/2] in the function’s argument makes the method of
backward substitutions stumble on values of n that are not powers of 2.

• Therefore, the standard approach to solving such a recurrence is to solve it

only for n = 2𝑘 and then take advantage of the theorem called the
smoothness rule which claims that under very broad assumptions the order
of growth observed for n = 2𝑘 gives a correct answer about the order of
growth for all values of n.

• So let us apply this recipe to our recurrence, which for n = 2𝑘 takes the form
Now backward substitutions encounter no problems:

Thus, we end up with

or, after returning to the original variable n = 2k and hence k = log2 n,

• UNIT I