Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

MATH 4A - Linear Algebra with Applications

Lecture 7: Matrix operations

Substitute lecturer: Professor Christopher Ograin

15 April 2019

Reading: §2.1-2.3 from Lay, 5th ed.

Recommended problems from §2.1: 1, 3, 5, 7, 9, 11, 15, 16, 19,
21, 23
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Lecture plan

1 Sums and scalar multiples of matrices

2 Matrix multiplication

3 Other operations: power and transpose

Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Motivation
So far, we’ve done two types of things things with matrices:
1 Used them to solve linear systems.
2 Showed that every linear transformation has a matrix.
We still have two leftover issues, respectively:
1 What is going on geometrically with row operations? Note
that if all we wanted to do was solve linear systems, we’ve
succeeded at that wonderfully. We’ve even succeeded at
proving the row reduction method works. However, I would
say that at this point, it’s still unclear what the heck row
operations are doing intuitively/geometrically.
2 Use matrices to actually say something interesting about
linear transformations, beyond just that every linear
transformation has a matrix.
The algebra of matrix operations we develop today will allow us to
address these issues.
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Our first two operations generalize operations on column

vectors

Recall that last week we defined two algebraic operations on

column vectors: addition and scalar multiplication.

If we think of a matrix as just a bunch of column vectors next to

each other, then we can extend these operations to matrices, as we
now explain.
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Matrix addition: basic idea

Let A and B be two m × n matrices. We can write them each as

A = a1 a2 · · · an B = b1 b2 · · · bn

where a1 , a2 , · · · , an and b1 , b2 , · · · , bn are all column vectors in

Rm .

Then the sum of A and B is the m × n matrix

A + B = a1 + b1 a2 + b2 · · · an + bn

In words: the j th column of A + B is the column vector aj + bj .

Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Matrix addition: gorier details

An arbitrary m × n matrix A is of the form

a11 · · · a1j ···

 
a1n
 .. .. .. 
 . . . 
 
A =  ai1 · · · aij
 ··· ain 
 . .. .. 
 .. . . 
am1 · · · amj ··· amn

where aij is a scalar in R. We call aij the (i, j)-entry of A. It’s

located in the i th row and j th column.

Note: if we wanted to be more clear (and not confuse the

subscripts with multiplication of integers), the subscripts should
probably have commas—so we should probably write a1,1 , a1,j , ai,j
or am,n instead of a11 , a1j , aij or amn , for instance—but we aren’t
going to do this, because writing commas is tiresome.
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Matrix addition: gorier details

So, write
a11 · · · ··· b11 · · · ···
   
a1j a1n b1j b1n
 .. .. ..   .. .. .. 
 . . .   . . . 
   
A=  ai1 · · · aij ···  bi1 · · ·
ain  B =  bij ··· bin 


 . .. ..   . .. .. 
 .. . .   .. . . 
am1 · · · amj ··· amn bm1 · · · bmj ··· bmn
for appropriate scalars aij and bij . Then

a11 + b11 · · · a1j + b1j ···

 
a1n + b1n
 .. .. .. 

 . . . 

A + B =  ai1 + bi1 · · ·
 aij + bij ··· ain + bin 

 .. .. .. 
 . . . 
am1 + bm1 · · · amj + bmj ··· amn + bmn
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Caveat

In order to add two matrices A and B, they must have the same
size. For example, it does not make sense to add
   
1 2 3 4 0
A= and B = ,
0 0 1 3 2

so don’t even try.

Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

iClicker 1

What is the (3, 4) entry of the sum of the matrices

   
1 2 3 4 32 21 1 0
4 8 12 16 0 0 0 0
A=  and B =  ?
 8 16 24 32 97 32 1 −2
−1 −2 13 32 5 7 8 9

(a) 21
(b) 30
(c) 41
(d) 25
(e) 16
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Scalar multiplication of matrices

Just like we can scalar multiply column vectors, we can also scalar
multiply matrices. If
··· ···
 
a11 a1j a1n
 .. .. .. 

 . . . 


A = a1 a2 · · · an =   ai1 ··· aij ··· ain 
 . .. .. 
 .. . . 
am1 ··· amj ··· amn
and c is a scalar, then the scalar multiple cA is
ca11 · · · ca1j ···
 
ca1n
 .. .. .. 
 . . . 

 
cA = ca1 ca2 · · · can =  cai1 · · · caij
 ··· cain 
 . .. .. 
 .. . . 
cam1 · · · camj ··· camn
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Properties of scalar multiplication and matrix sums

Just as for column vectors, matrix addition and scalar

multiplication satisfy some useful elementary identities. First, some
notation: a zero matrix is any matrix all of whose entries are 0.

Unlike the zero column vector 0, we usually write a zero matrix

simply as 0, without any bold. However, just like the zero column
vector 0 in Rn , the size m × n of a zero matrix 0 depends on
context. That is, m and n will usually be provided.

Examples: all of the following are zero matrices:

 
0    


0 0 0 0 0
0 0 0 0
0 0 0 0 0
0
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Properties of scalar multiplication and matrix sums

Theorem
Let A, B and C be matrices of the same size, and let r and s be
two scalars. Then:
(a) (Commutativity of matrix addition) A + B = B + A
(b) (Associativity of matrix addition) (A + B) + C = A + (B + C )
(c) (0 matrix is an additive identity) A + 0 = A
(d) (Linearity of scalar multiplication) r (A + B) = rA + rB
(e) (Linearity of scalar multiplication) (r + s)A = rA + sB
(f) (Associativity of scalar multiplication) r (sA) = (rs)A

Proof sketch: all of the properties follow more-or-less immediately

from the fact that usual addition and multiplication of real
numbers satisfy similar properties.
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

iClicker 2

Let
   
1 2 3 4 32 21 1 0
4 8 12 16 0 0 0 0 
A= 
 8 16 24 32 and B =  
97 32 1 −2
−1 −2 13 32 5 7 8 9

What is the (4, 2)-entry of the matrix 2A − 3B?

(a) -59
(b) 70
(c) 8
(d) -25
(e) 32
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Matrices have a new operation that column vectors didn’t

have

Recall last week that we said there wasn’t a good (or rather,
useful) way to define multiplication of two column vectors in Rn .

Matrices, on the other hand, can sometimes be multiplied, if their

sizes match up correctly, as we now explain.
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Basic idea: matrix multiplication generalizes matrix-vector

multiplication

Let A be a m × n matrix and let B be a n × p matrix. Write

B = b1 b2 · · · bp

where each b1 , b2 , . . . , bp is a column vector in Rn . Then the

product AB is the m × p matrix with columns Ab1 , Ab2 , . . . , Abp .
That is,



AB = A b1 b2 · · · bp = Ab1 Ab2 · · · Abp

Intuitively: each column of AB is a linear combination of the

columns of A using weights from the corresponding column of B.
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Gory details: explicit formula for matrix multiplication

Let A be a m × n matrix and suppose B is a matrix such that AB

is defined. (Thus, B has n rows and p columns for some p.) Let
(AB)ij denote the entry of AB in the i th row and j th column. (So
1 ≤ i ≤ m and 1 ≤ j ≤ p.) Then

(AB)ij = ai1 b1j + ai2 b2j + · · · + ain bnj

Xn
= aik bkj
k=1
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

iClicker 3

What is the (2,3) entry of AB, where

   
1 4 5 3 2
A= and B = ?
9 0 0 4 8

(a) 19
(b) This question makes no sense.
(c) 34
(d) 27
(e) 18
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

  
1 4 5 3 2
AB =
9 0 0 4 8
 
1·5+4·0 1·3+4·4 1·2+4·8
=
9·5+0·0 9·3+0·4 9·2+0·8
 
5 19 34
=
45 27 18
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

iClicker 4

What is the (3,2) entry of BA, where

   
1 4 5 3 2
A= and B = ?
9 0 0 4 8

(a) 19
(b) This question makes no sense.
(c) 34
(d) 27
(e) 18
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Motivation: Matrix multiplication corresponds to

composition of transformations

Why in the world would that be our definition of matrix

multiplication???

Because it corresponds to composition of linear transformations.

Let T : Rn → Rm and S : Rm → Rp be two linear transformations.
Note that the codomain of T equals the domain of S. The
composition of S and T is the transformation

S ◦ T : Rn → Rp

that sends x in Rn to S(T (x)) in Rp .

Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Schematic: composition of transformations

Applying S ◦ T

Applying T Applying S

S(T (x))
x
T (x)
Rn Rm Rp
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Schematic: composition of transformations

Applying S ◦ T

S(T (x))
x
T (x)
Rn Rm Rp
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Motivation: Matrix multiplication corresponds to

composition of transformations

Given linear transformations T : Rn → Rm and S : Rm → Rp , let

B be the standard matrix for T and let A be the standard matrix
for S. (So A is p × m and B is m × n.) Then the standard matrix
for the composition S ◦ T is exactly the p × n matrix

AB.
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Identity matrix

We use the notation Im to denote the identity matrix which is the

m × m square matrix


Im = e1 e2 · · · em

where e1 , e2 , . . . , em denote the standard basis vectors of Rm .

For example:
 
  1 0 0 0
  1 0 0

1 0 0 1 0 0
I1 = 1 , I2 = , I3 = 0 1 0 , I4 =  
0 1 0 0 1 0
0 0 1
0 0 0 1
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Why is Im called the identity matrix?

Im is the identity matrix because

Im x = x

for all vectors x in Rm . Thus, Im corresponds to the identity linear

transformation T : Rm → Rm that sends every vector to itself:
T (x) = x.

Note: from the definition of matrix multiplication, we see that for

any m × p matrix A = B

Im B = B

and for any n × m matrix A

AIm = A.
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Properties of matrix multiplication

Theorem
Let A, B and C be matrices. Then, whenever the following matrix
products make sense, we have identities:
(a) (associativity of matrix multiplication) A(BC ) = (AB)C
(b) (left distributive law) A(B + C ) = AB + AC
(c) (right distributive law) (B + C )A = BA + CA
(d) (commutativity of scalar multiplication)
r (AB) = (rA)B = A(rB) for any scalar r
(e) (identity matrices are multiplicative identities) Im A = A = AIn
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Caveats

1 Matrix multiplication only works if the matrices sizes match

up correctly: if A is m × n and B is k × l, the product AB is
defined only if n = k, and the product BA is defined only if
l = m.
2 Matrix multiplication is not commutative: there exist matrices
A and B such that AB 6= BA.
3 You can’t “cancel out” (“divide out”) matrices: there exist
matrices A, B, C such that AB = AC but B 6= C .
4 If AB = 0, it is not necessarily true that either A = 0 or
B = 0.
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Example: matrix multiplication is not commutative

         
1 0 0 1 0 1 0 0 0 1 1 0
= 6= =
0 0 0 0 0 0 0 0 0 0 0 0
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Example: matrices can’t be canceled out

     
0 1 0 0 0 1 1 0
=
0 0 0 0 0 0 0 0
(because
  both matrix products are 0) but we can’t cancel out
0 1
because
0 0
   
0 0 1 0
6=
0 0 0 0
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Example: nonzero matrices can multiply to 0

  
0 1 1 0
=0
0 0 0 0
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Powers of a matrix

If A is a n × n square matrix, and if k is a positive integer, then Ak

denotes the product of k copies of A:

Ak = AA · · · A (k times)

Thus, if x is in Rn , then Ak x is the result of multiplying x by A

repeatedly k times. If k = 0, then A0 x should be x itself. Thus, we
define A0 = In , the identity matrix.
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Example: powers of a shear

Consider the shear transformation
 
1 1
A= .
0 1
Let’s compute its powers:
    
2 1 1 1 1 1 2
A = =
0 1 0 1 0 1
      
3 1 1 2 1 1 1 2 1 3
A = A = =
0 1 0 1 0 1 0 1
      
1 1 1 1 1 3 1 4
A4 = A3 = =
0 1 0 1 0 1 0 1
We could argue in general that
 
k 1 k
A = .
0 1
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Eigenvector sneak peek: why powers of a matrix are

interesting
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Eigenvector sneak peek: why powers of a matrix are

interesting
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Eigenvector sneak peek: why powers of a matrix are

interesting
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Eigenvector sneak peek: why powers of a matrix are

interesting
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Eigenvector sneak peek: why powers of a matrix are

interesting
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

What just happened?

As we repeatedly sheared the sheep, the picture flattens and

elongates.

Later we’ll see that as we take the number of times we shear—that

is, k—to infinity, the entire plane R2 in some sense converges to
the x1 -axis. This is because the vector e1 is the “eigenvector” of
the transformation  
1 1
0 1
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Transpose of a matrix

Given a m × n matrix A, the transpose of A is the n × m matrix,

denoted AT , whose columns are formed from the corresponding
rows of A.
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Examples of transpose

Let  
  4 2
a b

A= B = 9 1 C= 0 0 0
c d
0 3
Then
 
    0
a c 4 9 0
AT = BT = CT = 0

b d 2 1 3
0
Sums and scalar multiples of matrices Matrix multiplication Other operations: power and transpose

Properties of transpose

Theorem
Let A and B denote two matrices whose sizes are appropriate for
the following sums or products
(a) (AT )T = A
(b) (A + B)T = AT + B T
(c) For any scalar r , (rA)T = rAT
(d) (AB)T = B T AT

In words, (d) says: the transpose of a product of matrices equals

the product of transposes in the reverse order.