Theory of Algorithms 20 September 2021

Theory of Algorithms –
Decrease and Conquer

James Gain
Department of Computer Science
University of Cape Town
jgain@cs.uct.ac.za

Introduction
Strategy:
Solve smaller instance
Extend solution of smaller instance to obtain solution to original
problem
Also called inductive or incremental
3 Variants:
Decrease by a constant
Decrease by a constant factor
Variable size decrease

Theory of Algorithms 2

Page 1
Theory of Algorithms 20 September 2021

Variants Illustrated
A PROBLEM OF
SIZE n

SUBPROBLEM SUBPROBLEM SUBPROBLEM

OF SIZE n-c OF SIZE n/c OF smaller SIZE

A SOLUTION TO
SUBPROBLEM

A SOLUTION TO THE
ORIGINAL PROBLEM
Theory of Algorithms 3

Divide and Conquer is Different

Decrease by a Constant Divide and Conquer
Decrease
and Conquer Factor and Conquer
throws away A PROBLEM OF
A PROBLEM OF
half (or SIZE n
SIZE n
constant
factor) of the SUBPROBLEM SUBPROBLEM 1
OF SIZE n/2
SUBPROBLEM 2
OF SIZE n/2
OF SIZE n/2
work
Divide and A SOLUTION TO A SOLUTION TO
A SOLUTION TO SUBPROBLEM 1 SUBPROBLEM 2
Conquer SUBPROBLEM
does both
halves
A SOLUTION TO THE
A SOLUTION TO THE ORIGINAL PROBLEM
ORIGINAL PROBLEM
Theory of Algorithms 4

Page 2
Theory of Algorithms 20 September 2021

Map of Decrease and Conquer Algorithms

Decrease by a constant (often 1): Variable-size decrease:
Insertion sort Euclid’s algorithm
Graph Searching: DFS, BFS Interpolation Search
Generating permutations and Finding the k’th order statistic e.g.
subsets the median
Topological sort
Decrease by a constant factor
(usually 2):
Binary search
Fake-coin problem
Multiplication a la Russe

Theory of Algorithms 5

Strengths and Weaknesses

ü Strengths:
Can be implemented either top down (recursively) or bottom up
(without recursion)
Often very efficient (possibly Q(log n) )
Leads to a powerful form of graph traversal (Breadth and Depth
First Search)

✗Weaknesses:
Less widely applicable (especially decrease by a constant factor)

Theory of Algorithms 6

Page 3
Theory of Algorithms 20 September 2021

Theory of Algorithms –
Decrease by a Constant

Insertion Sort
Idea:
Assume an already sorted list of size n-1
Insert the remaining element in the correct position
Before doing this recursively perform insertion sort on smaller list
Recursive idea but better performed iteratively bottom up
Decrease by 1 and conquer
Worst case efficiency – Θ(n2), but works well on partially
sorted inputs

Theory of Algorithms 8

Page 4
Theory of Algorithms 20 September 2021

Insertion Sort Example

6 5 0 2 8 7 4
5 6|0 2 8 7 4
0 5 6|2 8 7 4
0 2 5 6|8 7 4
0 2 5 6 8|7 4
0 2 5 6 7 8|4
0 2 4 5 6 7 8

Theory of Algorithms 9

Insertion Sort Example

6|5 0 2 8 7 9 sort(1);insert(2nd)
5 6|0 2 8 7 9 sort(2);insert(3rd)
0 5 6|2 8 7 9 sort(3);insert(4th)
0 2 5 6|8 7 9 sort(4);insert(5th)
0 2 5 6 8|7 9 sort(5);insert(6th)
0 2 5 6 7 8|9 sort(6);insert(7th)
0 2 5 6 7 8 9 sort(7)

Theory of Algorithms 10

10

Page 5
Theory of Algorithms 20 September 2021

Topological Sort
Problem:
In a directed acyclic graph, list the vertices in an order such that edge
direction is respected
That is for any directed edge the source vertex must appear before the
destination in the list
Sometimes multiple possible orderings
Not solvable if there are cycles
Can use Depth-First Search (from last year) to solve
Applications:
Ordering a set of courses that have pre-requisites
Evaluating formulae with dependencies in a spreadsheet
Ordering tasks in a complex project
Theory of Algorithms 11

11

Topological Sorting Algorithm

0
1
b b
Solution: 0
2
2
Repeatedly a a
d
2 d 1
remove a source
c
vertex and its c

incident outgoing Remove a and edges Remove b and edges

to b and c. List {a} to c and d. List {a, b}
edges
Append to sorted
b b
list
0
If no source a
1
a
d
vertices exist, 0 d
either finished or c c
there is a cycle Remove c and edge Remove d.
Theory of Algorithms
to d. List {a, b, c} List {a, b, c, d} 12

12

Page 6
Theory of Algorithms 20 September 2021

Generating Permutations
Necessary as a component of Exhaustive Search
All possible orderings of numbers in range {1, …, n}
Can be used as indices for other objects
Solution:
Generate all (n-1)! permutations of {1, …, n-1}
Insert n into each possible position
During insert starting from the right or left alternately, satisfies
Minimal-Change requirement (next permutation obtained by
swapping two elements of previous)
Decrease by 1 and conquer
Theory of Algorithms 13

13

Generating Permutations Diagramatically

21 12

321 231 213 312 132 123

Example:
Start: 1
Insert 2: 12 21
Insert 3: 123 132 312 321 231 213
Theory of Algorithms 14

14

Page 7
Theory of Algorithms 20 September 2021

Johnson-Trotter Method
Alternative permutation generator that avoids
permutations of smaller lists
Use arrows to keep track of what permutation
comes next
An element k is mobile if its arrow points to an adjacent
element smaller than it
→ ← → ←
3 2 4 1
3 & 4 mobile. 1 & 2 not
Theory of Algorithms 15

15

Johnson-Trotter Algorithm

Initialize the first permutation with:

← ← ←
1 2 … n
while last permutation has mobile elem:
- find its largest mobile element k
- swap k with neighbour it points to
- reverse direction of elements > k
- add the new permutation to the list
Return the list of permutations

Theory of Algorithms 16

16

Page 8
Theory of Algorithms 20 September 2021

Johnson-Trotter Example
← ← ←
1 2 3
← ← ←
1 3 2 while last permutation
← ← ← has mobile elem:
3 1 2 - find its largest mobile element k
→ ← ← - swap k with neighbour it points to
3 2 1 - reverse direction of elements > k
← → ← - add the new permutation to the list
2 3 1
← ← →
2 1 3
Theory of Algorithms 17

17

Theory of Algorithms –
Decrease by a
Constant Factor

18

Page 9
Theory of Algorithms 20 September 2021

Fake Coin Problem

Problem:
Among n coins, one is fake (and weighs less)
We have a balance scale which can compare any two sets
Algorithm:
Divide into two size ën/2û piles (keeping a coin aside if n is odd)
If they weigh the same then the extra coin is fake
Otherwise proceed recursively with the lighter pile
Efficiency:
W(n) = W( ën/2û ) + 1 for n > 1
W(n) = ëlog2 nû = Q(log2 n)
Can we do better ?
Theory of Algorithms 19

19

Multiplication a la Russe
Solution:
n*m = (n/2) * (2m) } if n is even
((n-1)/2 ) * (2m) + m } if n is odd
Using doubling and halving instead of multiplication is
efficient in hardware (just shift operations!)
Decrease by a constant factor of 2 and conquer
Example:
50 * 20 = (25 * 40) = (12 * 80) + 40 = (6 * 160) + 40
= (3 * 320) + 40 = (1 * 640) + 320 + 40 = 1000
Theory of Algorithms 20

20

Page 10
Theory of Algorithms 20 September 2021

Theory of Algorithms –
Variable Size Decrease

21

Euclid’s GCD
Problem:
Greatest Common Divisor of two integers m and n is the largest
integer that divides both exactly
Euclid’s Solution:
gcd(m, n) = gcd(n, m mod n)
gcd(m, 0) = m
Right-side args are smaller by neither a constant size nor factor
Example:
gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12

Theory of Algorithms 22

22

Page 11
Theory of Algorithms 20 September 2021

Finding k-th Order Statistic

Problem:
p
Find the k’th smallest element in a list
Median is k = n/2
Sorting the list is inefficient A[i]≤p A[i]>p
Solution:
Exploit quicksort, noting that pivot is placed in the correct position
Partition as usual (≤ pivot | ≥ pivot)
Since the pivot ends up in its correct final position, we only need to
continue with one of the 2 partitions
(if pivot ends up ≥ k’th position, search the first partition, else search the
2nd partition)
Theory of Algorithms 23

23

Interpolation Search
Mimics the way humans search through a phone book (look
near the beginning for ‘Brown’)
Assumes that values between the leftmost (A[b]) and
rightmost (A[u]) elements increase linearly
value
Algorithm (key = v, find search index = i):
A[u]
Binary search with floating variable
v
at index i
Setup straight line through (b, A[b])
and (u, A[u]) A[b]

Find point P = (x, y) on line at y = v,

then i = x b i u index
Theory of Algorithms 24

24

Page 12