Divide and Conquer

1
Introduction
• Divide and Conquer algorithms consist of two parts:
• Divide: Smaller problems are solved recursively (except, of course, the base cases).
• Conquer: The solution to the original problem is then formed from the solutions to the
subproblems.
• Examples
• Binary Search
• Quick Sort
• Merge Sort
• Long Integer Multiplication (Karatsuba’s Multiplication]
• Maximum Sum Subarray
• Strassen’s Algorithm for Matrix Multiplication etc.
• Closest Pair Problem
• Min- max problem
2
Integer Multiplication
1011
x1111
• The standard integer multiplication routine of 2 n-digit 1011
numbers 10110
101100
• Involves n multiplications of an n-digit number by a single +1011000
digit 10100101
• Plus the addition of n numbers, which have at most 2 n digits

quantity time
1) Multiplication n-digit by 1-digit n O(n)
2) Additions 2n-digit by n-digit max n O(n)

Total time = n*O(n) + n*O(n) = 2n*O(n) = O(n2)

3
 Integer Multiplication 1011
x1111
1011
• Let’s consider a Divide and Conquer Solution 10110
101100
• Imagine multiplying an n-bit number by another n-bit number. +1011000
• We can split up each of these numbers into 2 halves. 10100101
• Let the 1st number be I, and the 2nd number J
• Let the “left half” of the 1st number by Ih and the “right half” be Il.

• So in this example: I is 1011 and J is 1111

• I becomes 10*22 + 11 where Ih = 10*22 and Il = 11.
• and Jh = 11*22 and Jl = 11

4
Integer Multiplication
• So for multiplying any n-bit integers I and J
• We can split up I into (Ih * 2n/2) + Il
• And J into (Jh * 2n/2) + Jl
• Then we get
• I x J = [(Ih x 2n/2) + Il] x [(Jh x 2n/2) + Jl]
• I x J = Ih x Jh x 2n + (Il x Jh + Ih x Jl) x 2n/2 + Il x Jl

• So what have we done?

• We’ve broken down the problem of multiplying 2 n-bit numbers into
• 4 multiplications of n/2-bit numbers plus 3 additions.
• T(n) = 4T(n/2) + (n)
• Solving this using the master theorem gives us…
• T(n) = (n2)
5
Integer Multiplication
• So we haven’t really improved that much,
• Since we went from a O(n2) solution to a O(n2) solution
• Can we optimize this in any way?
• We can re-work this formula using some clever choices…
• Some clever choices of:
P1 = (Ih + Il) x (Jh + Jl) = IhxJh + Ihx Jl + IlxJh + IlxJl
P2 = Ih x Jh , and
P3 = Il x Jl
• Now, note that
P1 - P2 – P3 = IhxJh + IhxJl + IlxJh + IlxJl - IhxJh - IlxJl
= IhxJl + IlxJh
• Then we can substitute these in our original equation:
IxJ = P2 x 2n + [P1 - P2 – P3]x 2n/2 + P3.
6
Integer Multiplication
IxJ = P2 x 2n + [P1 - P2 – P3]x 2n/2 + P3.

• Have we reduced the work?

• Calculating P2 and P3 – take n/2-bit multiplications.
• P1 takes two n/2-bit additions and then one n/2-bit multiplication.
• Then, 2 subtractions and another 2 additions, which take O(n) time.

• This gives us : T(n) = 3T(n/2) + θ(n)

• Solving gives us T(n) = θ(n(log23)), which is approximately T(n) = θ(n1.585), a solid
improvement.
7
Integer Multiplication
• Although this seems it would be slower initially because of some extra
pre-computing before doing the multiplications, for very large
integers, this will save time.

• Q: Why won't this save time for small multiplications?

• A: The hidden constant in the θ(n) in the second recurrence is much larger. It
consists of 6 additions/subtractions whereas the θ(n) in the first recurrence
consists of 3 additions/subtractions.

8
Integer Multiplication - Example

9
Integer Multiplication - Example

10
Integer Multiplication - Example

11
Integer Multiplication - Example

12
Integer Multiplication - Example

13
Pseudocode

14
Long Integer Multiplication
• Solve
• 1101 x 1011
• 1234 x 2345

15
Maximum Sum Subarray
• The maximum subarray problem is the task of finding
the largest possible sum of a contiguous subarray, within a
given one-dimensional array A[1…n] of numbers.
• The time complexity of the Naive method is O(n^2).
• Using Divide and Conquer approach, we can find the maximum
subarray sum in O(nLogn) time.

16
Maximum Sum Subarray

17
Solution #1 - Brute Force
• The naive approach is to generate all the possible subarray and print that
subarray which has maximum sum.
int MSS (int a[], int n) int MSS (int a[], int n)
{ int max=0 {
for(i=0 to n-1 do) int max=0
for(i=0 to n-1 do)
for(j=i to n-1 do) int s=0
int s=0 for (k=i to n do)
s=s+a[k]
for (k=i to j do)
if(s>max)
s=s+a[k] max= s
if(s>max) return max
}
max= s
return max } O(n2)
O(n3)
18
Solution #2 – Divide and
Conquer
/ Returns sum of maxium sum subarray in aa[l..h]
int maxSubArraySum(int arr[], int l, int h)
{
// Base Case: Only one element
if (l == h)
return arr[l];

// Find middle point

int m = (l + h) / 2;

/* Return maximum of following three possible

cases
a) Maximum subarray sum in left half
b) Maximum subarray sum in right half
c) Maximum subarray sum such that the
subarray
crosses the midpoint */
return max(maxSubArraySum(arr, l, m),
maxSubArraySum(arr, m + 1, h),
maxCrossingSum(arr, l, m, h));
19
}
Maximum Sum Subarray
// Find the maximum possible sum in arr[] auch that arr[m] is part of it
int maxCrossingSum(int arr[], int l, int m, int h)
{
// Include elements on left of mid.
int sum = 0;
int left_sum = INT_MIN;
for (int i = m; i >= l; i--) {
sum = sum + arr[i];
if (sum > left_sum)
left_sum = sum;
}
// Include elements on right of mid
sum = 0;
int right_sum = INT_MIN;
for (int i = m + 1; i <= h; i++) {
sum = sum + arr[i];
if (sum > right_sum)
right_sum = sum;
}
// Return sum of elements on left and right of mid returning only left_sum +

//right_sum will fail for [-2, 1]

return max(left_sum + right_sum, left_sum, right_sum);
20
}
Maximum Sum Subarray -
Example
• 2, -4, 1, 9, -6, 7, -3 Soln: 11

21
Maximum Sum Subarray -
Example
• 3, -5, 1, -2, 6, 4, -3 Soln: 10

22
Maximum Sum Subarray -
Example
• 2, -5, 6, -2, -3, 1, 5, -6 Soln: 7

23
Strassen’s Algorithm

24
Strassen’s Algorithm

25
Strassen’s Algorithm
• Strassen showed how two matrices can be multiplied using only 7 multiplications and 18
additions:
• Consider calculating the following 7 products:
• q1 = (a11 + a22) * (b11 + b22)
• q2 = (a21 + a22) * b11
• q3 = a11*( b12 – b22)
• q4 = a22 * (b21 – b11)
• q5 = (a11 + a12) * b22
• q6 = (a21 – a11) * (b11 + b12)
• q7 = (a12 – a22) * (b21 + b22)

• It turns out that

• c11 = q1 + q4 – q5 + q7
• c12 = q3 + q5
• c21 = q2 + q4
• c22 = q1 + q3 – q2 + q6
26
Strassen’s Algorithm

27
Example

28
Example

29
Example

30
Example

31
Example

32
Example

33
Example

34
Example

35
Strassen’s Algorithm

Mult Add Recurrence Relation Runtime

Regular 8 4 T(n) = 8T(n/2) + O(n2) O(n3)

Strassen 7 18 T(n) = 7T(n/2) + O(n2) O(n log27) = O(n2.81)

36
Strassen’s Algorithm
• No idea how Strassen came up with these combinations.
• He probably realized that he wanted to determine each element in the product
using less than 8 multiplications.
• From there, he probably just played around with it.

• If we let T(n) be the running time of Strassen's algorithm, then it

satisfies the following recurrence relation:
• T(n) = 7T(n/2) + O(n2)
• It's important to note that the hidden constant in the O(n 2) term is larger than the
corresponding constant for the standard divide and conquer algorithm for this problem.
• However, for large matrices this algorithm yields an improvement over the
standard one with respect to time.
37