1.

Multiplying row matrices and column matrices together

Let A be a 1 × 2 row matrix and B be a 2 × 1 column matrix:
 
  c
A= a b B=
d
The product of these two matrices is written AB and is the 1 × 1 matrix defined by:
 
  c
AB = a b × = [ac + bd]
d
Note that corresponding elements are multiplied together and the results are then added together.
For example
 
  6
2 −3 × = [12 − 15] = [−3]
5
This matrix product is easily generalised to other row and column matrices. For example if C is a
1 × 4 row matrix and D is a 4 × 1 column matrix:
 
3
   3 
C = 2 −4 3 2 B=  −2 


5
then we define the product of C with D as
 
3
   3 
CD = 2 − 4 3 2 ×   −2
 = [6 − 12 − 6 + 10] = [−2]

5
The only requirement is that the number of elements of the row matrix is the same as the number
of elements of the column matrix.

2. Multiplying two 2×2 matrices

If A and B are two matrices then the product AB is obtained by multiplying the rows of A with the
columns of B in the manner described above. This will only be possible if the number of elements
in the rows of A is the same as the number of elements in the columns of B. In particular, we
define the product of two 2 × 2 matrices A and B to be another 2 × 2 matrix C whose elements are
calculated according to the following pattern
     
a b w x aw + by ax + bz
× =
c d y z cw + dy cx + dz

A B = C
The rule for calculating the elements of C is described in the following Key Point:

16 HELM (2008):
Workbook 7: Matrices
 ®

Key Point 4
Matrix Product
AB = C
The element in the ith row and j th column of C is obtained
by multiplying the ith row of A with the j th column of B.

We illustrate this construction for the abstract matrices A and B given above:
     
  w   x
a b a b
     y z   
a b w x   aw + by ax + bz
× =    = cw + dy cx + dz

c d y z 
 

 w

  x 
c d c d
y z
For example
     
  2   4
     2 −1 2 −1
6    1
2 −1 2 4   −2 7
× =

   = −6 10

3 −2 6 1  
 
 2   4 
3 −2 3 −2
6 1

Task    
1 2 1 −1
Find the product AB where A = B=
3 4 −2 1

First write down row 1 of A, column 2 of B and form the first element in product AB:
Your solution

Answer  
−1
[1, 2] and ; their product is 1 × (−1) + 2 × 1 = 1.
1

Now repeat the process for row 2 of A, column 1 of B:

Your solution

Answer  
1
[3, 4] and . Their product is 3 × 1 + 4 × (−2) = −5
−2

HELM (2008): 17
Section 7.2: Matrix Multiplication
Finally find the two other elements of C = AB and hence write down the matrix C:

Your solution

Answer
Row 1 column 1 is 1 × 1 + 2 × (−2) = −3. Row 2 column 2 is 3 × (−1) + 4 × 1 = 1
 
−3 1
C=
−5 1

Clearly, matrix multiplication is tricky and not at all ‘natural’. However, it is a very important
mathematical procedure with many engineering applications so must be mastered.

3. Some surprising results

We have already calculated the product AB where
   
1 2 1 −1
A= and B =
3 4 −2 1
Now complete the following task in which you are asked to determine the product BA, i.e. with the
matrices in reverse order.

Task    
1 2 1 −1
For matrices A = and B = form the products of
3 4 −2 1
row 1 of B and column 1 of A row 1 of B and column 2 of A
row 2 of B and column 1 of A row 2 of B and column 2 of A

Now write down the matrix BA:

Your solution

Answer
row 1, column 1 is 1 × 1 + (−1) × 3 = −2 row 1, column 2 is 1 × 2 + (−1) × 4 = −2
row 2, column 1 is −2 × 1 + 1 × 3 = 1 row 2, column 2 is −2 × 2 + 1 × 4 = 0
 
−2 −2
BA is
1 0

It is clear that AB and BA are not in general the same. In fact it is the exception that AB = BA.
In the special case in which AB = BA we say that the matrices A and B commute.

18 HELM (2008):
Workbook 7: Matrices
 ®

Task
Calculate AB and BA where
   
a b 0 0
A= and B =
c d 0 0

Your solution

Answer  
0 0
AB = BA =
0 0

We call B the 2 × 2 zero matrix written 0 so that A × 0 = 0 × A = 0 for any matrix A.

Now in the multiplication of numbers, the equation

ab = 0
implies that either a is zero or b is zero or both are zero. The following task shows that this is not
necessarily true for matrices.

Task
Carry out the multiplication AB where
   
1 1 1 −1
A= , B=
1 1 −1 1

Your solution

Answer 
0 0
AB =
0 0

Here we have a zero product yet neither A nor B is the zero matrix! Thus the statement AB = 0
does not allow us to conclude that either A = 0 or B = 0.

HELM (2008): 19
Section 7.2: Matrix Multiplication
 Task    
a b 1 0
Find the product AB where A = and B =
c d 0 1

Your solution

Answer 
a b
AB = =A
c d
 
1 0
The matrix is called the identity matrix or unit matrix of order 2, and is usually denoted
0 1
by the symbol I. (Strictly we should write I2 , to indicate the size.) I plays the same role in matrix
multiplication as the number 1 does in number multiplication.
Hence
just as a × 1 = 1 × a = a for any number a, so AI = IA = A for any matrix A.

4. Multiplying two 3×3 matrices

The definition of the product C = AB where A and B are two 3 × 3 matrices is as follows
    
a b c r s t ar + bu + cx as + bv + cy at + bw + cz
C= d  e f   u v w  =  dr + eu + f x ds + ev + f y dt + ew + f z 
g h i x y z gr + hu + ix gs + hv + iy gt + hw + iz
This looks a rather daunting amount of algebra but in fact the construction of the matrix on the
right-hand side is straightforward if we follow the simple rule from Key Point 4 that the element in
the ith row and j th column of C is obtained by multiplying the ith row of A with the j th column of
B.
For example, to obtain the element in row 2, column 3 of C we take row 2 of A: [d, e, f ] and multiply
it with column 3 of B in the usual way to produce [dt + ew + f z].
By repeating this process we obtain every element of C.

20 HELM (2008):
Workbook 7: Matrices
 ®

  
Task 1 2 −1 2 −1 3
Calculate AB = 3
 4 0   1 −2 1 
1 5 −2 0 3 −2

First find the element in row 2 column 1 of the product:

Your solution

Answer  
2
Row 2 of A is (3, 4, 0) column 1 of B is  1 
0
The combination required is 3 × 2 + 4 × 1 + (0) × (0) = 10.

Now complete the multiplication to find all the elements of the matrix AB:
Your solution

Answer
In full detail, the elements of AB are:
 
1 × 2 + 2 × 1 + (−1) × 0 1 × (−1) + 2 × (−2) + (−1) × 3 1 × 3 + 2 × 1 + (−1) × (−2)
 3×2+4×1+0×0 3 × (−1) + 4 × (−2) + 0 × 3 3 × 3 + 4 × 1 + 0 × (−2) 
1 × 2 + 5 × 1 + (−2) × 0 1 × (−1) + 5 × (−2) + (−2) × 3 1 × 3 + 5 × 1 + (−2) × (−2)
 
4 −8 7
i.e. AB =  10 −11 13 
7 −17 12
 
1 0 0
The 3 × 3 unit matrix is: I =  0 1 0  and as in the 2 × 2 case this has the property that
0 0 1
AI = IA = A
 
0 0 0
The 3 × 3 zero matrix is  0 0 0 
0 0 0

HELM (2008): 21
Section 7.2: Matrix Multiplication
5. Multiplying non-square matrices together
So far, we have just looked at multiplying 2 × 2 matrices and 3 × 3 matrices. However, products
between non-square matrices may be possible.

Key Point 5

General Matrix Products

The general rule is that an n × p matrix A can be multiplied
by a p × m matrix B to form an n × m matrix AB = C.
In words:
For the matrix product AB to be defined the number
of columns of A must equal the number of rows of B.
The elements of C are found in the usual way:
The element in the ith row and j th column of C is obtained
by multiplying the ith row of A with the j th column of B.

Example 4  
  2 5
1 2 2
Find the product AB if A = and B =  6 1 
2 3 4
4 3

Solution
Since A is a 2 × 3 and B is a 3 × 2 matrix the product AB can be found and results in a 2 × 2
matrix.
     
  2   5
  1 2 2
  6  1 2 2  1  
 
  2 5  4 3   
1 2 2 
 = 22 13

AB = × 6 1 =
  
2 3 4      38 25
4 3 
   2   5  
 2 3 4  6  2 3 4  1  
4 3

22 HELM (2008):
Workbook 7: Matrices
 ®

Task    
1 −2 2 4 1
Obtain the product AB if A = and B =
2 −3 6 1 0

Your solution

Answer
AB is a 2 × 3 matrix.
       
  2   4   1
     1 −2 1 −2
6
1 −2  1 0
1 −2 2 4 1  
AB = × =
 
2 −3 6 1 0  
 
 2 
 
 4 
  
 1 
2 −3 2 −3 2 −3
6 1 0
 
−10 2 1
=
−14 5 2

6. The rules of matrix multiplication

It is worth noting that the process of multiplication can be continued to form products of more than
two matrices.
Although two matrices may not commute (i.e. in general AB 6= BA) the associative law always
holds i.e. for matrices which can be multiplied,
A(BC) = (AB)C.
The general principle is keep the left to right order, but within that limitation any two adjacent
matrices can be multiplied.
It is important to note
 that it is not always
 possible to multiply together
 any two given matrices.

1 2 a b c a + 2d b + 2e c + 2f
For example if A = and B = then AB = .
3 4 d e f 3a + 4d 3b + 4e 3c + 4f
  
a b c 1 2
However BA = is not defined since each row of B has three elements
d e f 3 4
whereas each column of A has two elements and we cannot multiply these elements in the manner
described.

HELM (2008): 23
Section 7.2: Matrix Multiplication
  
Task     1 4
1 3 5 1 2
Given A = , B= , C= 2 5 
2 4 6 3 4
3 6

State which of the products AB, BA, AC, CA, BC, CB, (AB)C, A(CB) is defined and state
the size (n × m) of the product when defined.
Your solution
AB
BA
AC
CA
BC
CB
(AB)C
A(CB)

Answer
A B B A
not possible possible; result 2 × 3
2×3 2×2 2×2 2×3
A C C A
possible; result 2 × 2 possible; result 3 × 3
2×3 3×2 3×2 2×3
B C C B
not possible possible; result 3 × 2
2×2 3×2 3×2 2×2
A (C B)
(AB)C not possible, AB not defined. possible; result 2 × 2
2×3 3×2
We now list together some properties of matrix multiplication and compare them with corresponding
properties for multiplication of numbers.

Key Point 6
Matrix algebra Number algebra
A(B + C) = AB + AC a(b + c) = ab + ac
AB 6= BA in general ab = ba
A(BC) = (AB)C a(bc) = (ab)c
AI = IA = A 1.a = a.1 = a
A0 = 0A = 0 0.a = a.0 = 0
AB may not be possible ab is always possible
AB = 0 does not imply A = 0 or B = 0 ab = 0 → a = 0 or b = 0

24 HELM (2008):
Workbook 7: Matrices
 ®

Application of matrices to networks

A network is a collection of points (nodes) some of which are connected together by lines (paths).
The information contained in a network can be conveniently stored in the form of a matrix.

Example 5
Petrol is delivered to terminals T1 and T2 . They distribute the fuel to 3 storage
depots (S1 , S2 , S3 ). The network diagram below shows what fraction of the fuel
goes from each terminal to the three storage depots. In turn the 3 depots supply
fuel to 4 petrol stations (P1 , P2 , P3 , P4 ) as shown in Figure 2:

T1 T2
0.4 0.3
0.4
0.5 0.2 0.2
S1 S2 S3
0.1 0.6
0.6 0.2
0.2
0.5 0.4
0.2 0.2

P1 P2 P3 P4

Figure 2
Show how this situation may be described using matrices.

Solution
Denote the amount of fuel, in litres, flowing from T1 by t1 and from T2 by t2 and the quantity being
received at Si by si for i = 1, 2, 3. This situation is described in the following diagram:

T1 T2
0.4 0.3
0.4
0.5 0.2 0.2
S1 S2 S3

From this diagram we see that

   
s1 = 0.4t1 + 0.5t2 s1 0.4 0.5  
t
s2 = 0.4t1 + 0.2t2 or, in matrix form: s2  = 0.4 0.2 1
t2
s3 = 0.2t1 + 0.3t2 s3 0.2 0.3

HELM (2008): 25
Section 7.2: Matrix Multiplication
 Solution (contd.)
In turn the 3 depots supply fuel to 4 petrol stations as shown in the next diagram:

S1 S2 S3
0.1 0.6
0.6 0.2
0.2
0.5 0.4
0.2 0.2

P1 P2 P3 P4

If the petrol stations receive p1 , p2 , p3 , p4 litres respectively then from the diagram we have:
   
p1 = 0.6s1 + 0.2s2 p1 0.6 0.2 0  
p2 = 0.2s1 + 0.5s2 p2  0.2 0.5 0  s1
or, in matrix form:  p3  = 0.2 0.2 0.4 s2
   
p3 = 0.2s1 + 0.2s2 + 0.4s3
s3
p4 = 0.1s2 + 0.6s3 p4 0 0.1 0.6
Combining the equations, substituting expressions for s1 , s2 , s3 in the equations for p1 , p2 , p3 , p4
we get:

p1 = 0.6s1 + 0.2s2
= 0.6(0.4t1 + 0.5t2 ) + 0.2(0.4t1 + 0.2t1 )
= 0.32t1 + 0.34t2

with similar results for p2 , p3 and p4 .

This is equivalent to combining the two networks. The results can be obtained more easily by
multiplying the matrices:

   
p1 0.6 0.2 0  
0.2 s1
p2 
  =  0.5 0  s2 
p3  0.2 0.2 0.4
s3
p4 0 0.1 0.6
 
0.6 0.2 0  
0.2 0.4 0.5  
0.5 0   t
0.4 0.2 1

= 
0.2 0.2 0.4 t2
0.2 0.3
0 0.1 0.6
   
0.32 0.34   0.32t1 + 0.34t2
0.28 0.20
 t1 = 0.28t1 + 0.20t2 
 
= 
0.24 0.26 t2 0.24t1 + 0.26t2 
0.16 0.20 0.16t1 + 0.20t2

26 HELM (2008):
Workbook 7: Matrices
 ®

Engineering Example 1

Communication network

Problem in words
Figure 3 represents a communication network. Vertices a, b, f and g represent offices. Vertices c, d
and e represent switching centres. The numbers marked along the edges represent the number of
connections between any two vertices. Calculate the number of routes from a and b to f and g

c
3 2
a 2 1 f
6
4 d
3
1
1 1
b g
3 e 2

Figure 3: A communication network where a, b, f and g are offices

and c, d and e are switching centres

Mathematical statement of the problem

The number of routes from a to f can be calculated by taking the number via c plus the number via
d plus the number via e. In each case this is given by multiplying the number of connections along
the edges connecting a to c, c to f etc. This gives the result:
Number of routes from a to f = 3 × 2 + 4 × 6 + 1 × 1 = 31.

The nature of matrix multiplication means that the number of routes is obtained by multiplying the
matrix representing the number of connections from ab to cde by the matrix representing the number
of connections from cde to f g.

Mathematical analysis
The matrix representing the number of routes from ab to cde is:
c d e
!
a 3 4 1
b 2 1 3
The matrix representing the number of routes from cde to f g is:

f g
c 2 1
 
6
d 3
e 1 2

HELM (2008): 27
Section 7.2: Matrix Multiplication
The product of these two matrices gives the total number of routes.
 
  2 1    
3 4 1  6 3 = 3 × 2 + 4 × 6 + 1 × 1 3 × 1 + 4 × 3 + 1 × 2 31 17
=
2 1 3 2×2+1×6+3×1 2×1+1×3+3×2 13 11
1 2

Interpretation
We can interpret the resulting (product) matrix by labelling the columns and rows.
f g
!
a 31 17
b 13 11
Hence there are 31 routes from a to f , 17 from a to g, 13 from b to f and 11 from b to g.

28 HELM (2008):
Workbook 7: Matrices
 ®

Exercises
     
1 2 5 6 0 −1
1. If A = B= C= find
3 4 7 8 2 −3
(a) AB, (b) AC, (c) (A + B)C, (d) AC + BC(e) 2A − 3C
 
cos θ sin θ
2. If a rotation through an angle θ is represented by the matrix A = and a
 − sin θ cos θ 
cos φ sin φ
second rotation through an angle φ is represented by the matrix B = show
− sin φ cos φ
that both AB and BA represent a rotation through an angle θ + φ.
   
1 2 3 2 4  
2 1
3. If A =  −1 −1 −1  , B =  −1 2  , C = , find AB and BC.
1 2
2 2 2 5 6
   
  1 2 3 0
1 2 −1
4. If A = , B= 5 0 0 , C =  1 ,
0 −1 2
1 2 −1 −2
verify A(BC) = (AB)C.
 
2 3 −1
5. If A =  0 1 2  then show that AAT is symmetric.
4 5 6


    0 1
11 0 0 1 2
6. If A = B= verify that (AB)T =  11 3  = B T AT
2 1 1 1 3
22 7

Answers
     
19 22 4 −7 16 −30
1. (a) AB = (b) AC = (c) (A + B)C =
43 50 8 −15 24 −46
   
16 −30 2 7
(d) AC + BC = (e)
24 −46 0 17
 
cos θ cos φ − sin θ sin φ cos θ sin φ + sin θ cos φ
2. AB =
− sin θ cos φ − cos θ sin φ − sin θ sin φ + cos θ cos φ
 
cos(θ + φ) sin(θ + φ)
=
− sin(θ + φ) cos(θ + φ)
which clearly represents a rotation through angle θ + φ. BA gives the same result.
   
15 26 8 10
3. AB =  −6 −12 , BC =  0 3 
12 24 16 17
 
−8
4. A(BC) = (AB)C =
8

HELM (2008): 29
Section 7.2: Matrix Multiplication

Powered by TCPDF (www.tcpdf.org)

Outline
1. Divide and Conquer

• Breaks a problem into independent subproblems,

• Review Strassen’s Algorithm solves them recursively, and combines the results.
• No overlapping subproblems; each subproblem is
solved independently.
• Examples: Merge Sort (O(nlogn)), Quick Sort,
Binary Search (O(logn)), Strassen’s Algorithm.
• Memory-efficient since past results are not stored.

• Detour – Matrix Squaring Divide and Conquer 2. Dynamic Programming (DP)

• Breaks a problem into overlapping subproblems

and stores results to avoid redundant calculations.
• Uses memoization (top-down) or tabulation

• Implementing Strassen’s Algorithm (bottom-up) for optimization.

• Examples: Fibonacci (O(n)), Knapsack (O(nW)),
Longest Common Subsequence (O(mn)).
• Higher memory usage due to caching, but faster for
problems with repeated calculations.

2
Review Strassen’s Algorithm
• Divide and Conquer at a High Level:
• Check for your base case.
• Divide your problem into multiple identical subproblems.
• Recursively solve each subproblem.
• Merge the solutions to your subproblems.
Review Strassen’s Algorithm
• Recall the matrix multiplication problem: we have two n by n matrices
X and Y, and we want to compute M = XY

M = X Y

By definition: !"# = ∑)&'( *"& +&# . So that gives us an O(n3)

iterative algorithm for free. What about recursion?
Review Strassen’s Algorithm

• Break each matrix up into four (n/2) by (n/2) sub-matrices as follows:

MA MB XA XB YA YB

=
MC MD XC XD YC YD

• Note
• MA = XAYA+ XBYC
• MB = XAYB+ XBYD There are 8 recursive
• MC = XCYA+ XDYC subproblems to solve!
• MD = XCYB+ XDYD
Review Strassen’s Algorithm
• Yields the recurrence T(n) = 8T(n/2) + O(n2). So T(n) = O(n3). No better
than the iterative algorithm!
• Strassen’s insight: The run time is dominated by the branching factor
of 8. What if we could reduce that? Let:
S1 = (XB - XD) (YC + YD)
S2 = (XA + XD) (YA + YD) MA = S1 + S2 – S4 + S6
S3 = (XA - XC) (YA + YB)
MB = S4 + S5
S4 = (XA + XB) (YD)
S5 = (XA) (YB - YD) MC = S6 + S7
S6 = (XD) (YC - YA) MD = S2 + S3 + S5 – S7
S7 = (XC + XD) (YA)
Review Strassen’s Algorithm
• We went from 8 matrix multiplications (recursive calls) and 4 matrix
additions (merge steps) to 7 matrix multiplications and 18 matrix
additions.
• T(n) = 7 T(n/2) + O(n2). So T(n) = O(nlg(7)) ~ O(n2.81).
• Does this matter? We’ll test that out in a minute.
Detour – Matrix Squaring Divide and Conquer
• Work on the following problem in groups. Let A be an n x n matrix.
We want to compute AA, the square of A.
1. Show that just five multiplications are sufficient to compute the
square of a 2 x 2 matrix.
2. Suppose we run Strassen’s algorithm but use 5 multiplications per
recursive step instead of 7 using our observation from part 1. If this
worked, what would be the asymptotic runtime?
3. Why does this not work?
4. *If you have time, try to give a reduction to prove that an O(nc) time
algorithm (for 2 ≤ # < 3) for matrix squaring implies an O(nc) time
algorithm for matrix multiplication.
Detour – Matrix Squaring Divide and Conquer
! "% !% + "# "(! + ))
1. Note that : =
# $ #(! + )) #" + ) %
2. The runtime would be log % 5 ≈ 2.32
3. Not all of the subproblems are matrix squaring problems! (Plus,
matrix multiplication, unlike scalar, is not commutative)
4. Suppose we have an O(nc) algorithm for matrix squaring, and we
want an O(nc) algorithm for matrix multiplication (say of n x n
0 5
matrices A and B). Define the 2n x 2n matrix 3 = . Then:
6 0
% 56 0
3 = , so we can read off the answer to AB.
0 65
• Review Strassen’s Algorithm

• Detour – Matrix Squaring Divide and Conquer

• Implementing Strassen’s Algorithm

• Does n2.81 really matter much compared to n3?
Implementing Strassen’s Algorithm
• Break into groups of ~ 3.
• Code up 3 simple matrix multiplication algorithms:
• Iterative algorithm by definition
• Naïve recursive algorithm
• Strassen’s recursive algorithm
• To test, generate random 32x32, 64x64, 128x128, and 256x256
matrices (in whatever way is convenient, use smallish integers).
• Time all of your algorithms, and try to explain your results.
• (ProTip – you may be able to improve your recursive algorithms by
using the iterative algorithm once you get to small matrices, maybe
8x8 or 16x16).
Implementing Strassen’s Algorithm
Run times in milliseconds
n Iterative Recursive Strassen R Library
32 67 78 149 0
64 552 552 342 1
128 4155 4101 2444 2
256 31730 34315 20071 15
Conclusion
• Many recursive divide and conquer algorithms can be sped up if you
can reduce the number of recursive calls, maybe at the expense of a
larger merge step.
• (But this improvement might not be large until you work with larger problem
sizes)
• There are tricks that matter in practice but not in theory. Examples:
• In many languages, basic operations like matrix multiplication, summing
vectors, etc., are heavily optimized, and you shouldn’t reinvent the wheel
(outside of this exercise).
• Combining recursive and iterative methods rather than recursing all the way
to the trivial base case often helps.