Design and Analysis of Algorithm

Engr. Huma Sheraz

(Lecturer)
1
Matrix Chain Multiplication - DP
In mathematics, particularly in linear algebra, matrix multiplication is a binary
operation that produces a matrix from two matrices.
For matrix multiplication, the number of columns in the first matrix must be equal to
the number of rows in the second matrix.

Design and Analysis of Algorithm 2

 Matrix Chain Multiplication - DP
• If we see from this example here we:
• 2X3X3X2

• Here no. of Column of first matrix are equal to the no. of Rows of
second matrix.
• So, we talk about dimension it will be:
2x3x2

Design and Analysis of Algorithm 3

 Matrix Chain Multiplication - DP
• When ever we have matrix to take product of them very first
condition is to check feasibility.
• If we have 3 matrix for multiplication can we multiply at once ?
• No!
• So, we have to split the matrix in possible number of parenthesis.
• Like if we :
• A1XA2XA3
• It should be like 2x3forA1, 3X4 for A2 and 4X2 for A3

Design and Analysis of Algorithm 4

 Matrix Chain Multiplication - DP
• So, if we cannot multiply the 3 or more than 3 matrix at once than
what need to do?
• for that
A1XA2XA3

(A1XA2)XA3 A1X(A2XA3)

Design and Analysis of Algorithm 5

 Matrix Chain Multiplication - DP
• Now here comes a question , can both parenthesis patterns give the same
result?
• Yes, because they are associative in nature.
• Do they need same effort and resources require for multiplication?
• Lets consider, (A1XA2)XA3
(2x3X3x4)x0
2x3x4=24
First parenthesis have 2x4 dimension, for A3 we have 4x2 therefore:-
2x4x4x2
2x4x2
16
Design and Analysis of Algorithm 6
24 Matrix Chain Multiplication - DP
• In total,
24+0+16=40
• Similarly, consider other side and calculate it. Result for that side 36.
• Hence, different possible parenthesis reduce the effort required for
matric multiplication.
• Suppose if we have A1XA2XA3…………………………….A10
• Like if we have chain of problem for matrix multiplication than here
comes a point that which multiplication need to be done first so on so
for.

Design and Analysis of Algorithm 7

Example 1
Let us have 3 matrices, A1,A2,A3 of order (10 x 100), (100 x 5) and (5 x 50)
respectively.
Three Matrices can be multiplied in two ways:
1. A1,(A2,A3): First multiplying(A2 and A3) then multiplying and resultant
withA1.
2. (A1,A2),A3: First multiplying(A1 and A2) then multiplying and resultant
withA3.
No of Scalar multiplication in Case 1 will be:
(100 x 5 x 50) + (10 X 100 X 50 ) = 25000 + 50000 = 75000
No of Scalar multiplication in Case 2 will be:
(10 x 100 x 5) + ( 10 x 5 x 50 ) = 5000 + 2500 = 7500

Design and Analysis of Algorithm 8

 Matrix Chain Multiplication - DP
• Given following matrices {A1,A2,A3,...An} and we have to perform the matrix
multiplication, which can be accomplished by a series of matrix multiplication as :
A1 x A2 x A3 …………….. An .
• By this, we mean that we have to follow the above matrix order for multiplication but we
are free to parenthesize the above multiplication depending upon our need.

Design and Analysis of Algorithm 9

 Matrix Chain Multiplication - DP
• Suppose 4 matrix :-
A1XA2XA3XA4
d0 d1d1 d2d2 d3d3 d4

1. A1(A2(A3XA4))
2. A1((A2XA3)A4)
3. (A1XA3)(A3XA4)
4. (A1(A2XA3))A4
5. ((A1XA2)A3)A4
Design and Analysis of Algorithm 10
Matrix Chain Multiplication
• To find the best possible way to calculate the product, we could simply
parenthesis the expression in every possible fashion and count each
time how many scalar multiplication are required.
• Matrix Chain Multiplication Problem can be stated as "find the optimal
parenthesization of a chain of matrices to be multiplied such that the
number of scalar multiplication is minimized".
Number of ways for parenthesizing the matrices:
?

Design and Analysis of Algorithm 11

 Matrix Chain Multiplication - DP
• This is matrix chain multiplication where we need to decide which
kind of parenthesis should be done to get less resources and have
optimal result.
• For that we have a formulae:

Design and Analysis of Algorithm 12

Matrix Chain Multiplication
One possible answer to the first question for finding the best value
of 'k' is to check all possible choices of 'k' and consider the best
among them.
But that it can be observed that checking all possibilities will lead
to an exponential number of total possibilities.
It can also be noticed that there exists only O (n2 ) different
sequence of matrices, in this way do not reach the exponential
growth.

Design and Analysis of Algorithm 13

Matrix Chain Multiplication
Step1: Structure of an optimal parenthesization: Our first step in the
dynamic paradigm is to find the optimal substructure and then use it to
construct an optimal solution to the problem from an optimal solution to
subproblems.
• Let Ai....j where i≤ j denotes the matrix that results from evaluating the
product
Ai Ai+1....Aj.
• If i < j then any parenthesization of the product Ai Ai+1 ......Aj must split
that the product between Ak and Ak+1 for some integer k in the range i ≤
k ≤ j.
That is for some value of k, we first compute the matrices Ai.....k & Ak+1....j and then
multiply them together to produce the final product Ai....j. The cost of
computing Ai....k plus the cost of computing Ak+1....j plus the cost of multiplying them
together is the cost of parenthesization.
cost of parenthesization?
Design and Analysis of Algorithm 14
Matrix Chain Multiplication
Step 2: A Recursive Solution: Let m [i, j] be the minimum number
of scalar multiplication needed to compute the matrix Ai....j.
• If i=j the chain consist of just one matrix Ai....i=Ai so no scalar
multiplication are necessary to compute the product. Thus m [i, j] =
0 for i= 1, 2, 3....n.
• If i<j we assume that to optimally parenthesize the product we split
it between Ak and Ak+1 where i≤ k ≤j. Then m [i,j] equals the
minimum cost for computing the subproducts Ai....k and Ak+1....j+
cost of multiplying them together. We know Ai has dimension pi-1 x
pi, so computing the product Ai....k and Ak+1....j takes pi-1 pk pj scalar
multiplication, we obtain

Design and Analysis of Algorithm 15

Matrix Chain Multiplication
• So the minimum cost of parenthesizing the product
Ai Ai+1......Aj becomes
𝐦 𝐢,𝐣 = 𝐦𝐢𝐧 𝛏 𝐦 𝐢 , 𝐤 + 𝐦 𝐤 + 𝟏, 𝒋 + 𝐩𝐢 − 𝟏 𝐩𝐤 𝐩𝐣 } if i=j and i<j
To construct an optimal solution, let us define s [i,j] to be the value
of 'k' at which we can split the product Ai Ai+1 .....Aj To obtain an
optimal parenthesization i.e. s [i, j] = k such that

16
Design and Analysis of Algorithm
Matrix Chain Multiplication
Step 3: Computing Optimal Costs: let us assume that matrix
Ai has dimension pi-1x pi for i=1, 2, 3....n. The input is a sequence
(p0,p1,......pn) where length [p] = n+1. The procedure uses an
auxiliary table m [1....n, 1.....n] for storing m [i, j] costs an auxiliary
table s [1.....n, 1.....n] that record which index of k achieved the
optimal costs in computing m [i, j].

Design and Analysis of Algorithm 17

Matrix Chain Multiplication
• Example 1 : We are given the sequence {4, 10, 3, 12, 20, and
7}. The matrices have size 4 x 10, 10 x 3, 3 x 12, 12 x 20, 20 x 7.
We need to compute M [i,j], 0 ≤ i, j≤ 5. We know M [i, i] = 0 for all i.

Design and Analysis of Algorithm 18

Design and Analysis of Algorithm 19