Orthogonal transformations (Matrices)

Warm-up (a) If M is a (real) matrix whose columns are orthonormal, what can you say about M T M ?

Solution. Let ~u1 , . . . , ~um be the columns of M ; we know they are orthonormal. Then,
 T  T
~u1 ~u1 · · · ~uT1 ~um

~u1
M T M =  ...  ~u1 · · · ~um =  ... .. .. 
    
. . 
~uTm ~uTm ~u1 · · · ~uTm ~um

That is, M T M is the m × m matrix whose (i, j) entry is ~uTi ~uj = ~ui · ~uj . But since ~u1 , . . . , ~um are
orthonormal, (
0 if i 6= j
~ui · ~uj =
1 if i = j

So, the diagonal entries of M T M are 1, and all other entries are 0; that is, M T M is the
identity matrix Im .

(b) If M is a square matrix whose columns are orthonormal, is M invertible? If so, what is its
inverse? Solution. If M is an n × n matrix with orthonormal columns, then its columns must
form a basis of Rn (n linearly independent vectors must form a basis of Rn , and orthonormal
vectors are linearly independent, so M must be invertible.
The fact that M T M = In then shows that the inverse of M is M T . A square matrix with
orthonormal columns (and real entries) is, rather confusingly, called an orthogonal matrix. There
is no special term for a square matrix with orthogonal columns.

1. (T/F)  
0 1 1 0 0 0
0
 −1 1 0 0 0

1 0 0 0 0 0
 
0 0 0 0 1 1
 
0 0 0 0 −1 1
0 0 0 1 0 0
is an orthogonal transformation.
Solution. False.

Orthogonal transformations preserve the length of vectors and the angles between them.

2. Which of the following is always an orthogonal transformation?

(a) Shears

(b) Projection onto a plane

1
 (c) Projection onto a line

(d) Reflection about a line

(e) Reflection about a plane

(f) Dilation

(g) Rotation

(h) Dilation Rotation

(i) The composition of two orthogonal transformations

(j) The inverse of an orthogonal transformation

Solution. (d) (e) (g) (i) (j)

3. If A is an n × m matrix, what is the relationship between rank(AT ) and rank(A)?

Solution. rank(A) = rank(AT )

4. Suppose M is an invertible matrix. Is M T necessarily invertible? If so, what can you say about its
inverse? If not, why not?

Solution. If M is invertible, then M T must be invertible; here’s one way to see this. We know that
an n × n matrix is invertible if and only if its rank is n.1 So, if M is an invertible n × n matrix, then
rank M = n. Then #3 tells us that rank(M T ) = n, so M T must also be invertible.
Now that we know M T is invertible, let’s find its inverse. Remember that the inverse of M T is the
one and only matrix that “undoes” M T , so it’s the sole matrix B with the property that BM T =
M T B = In . Since M is invertible, we know that M has an inverse M −1 , with the property that
M M −1 = M −1 M = In . Taking the transpose of this equation,

(M M −1 )T = (M −1 M )T = InT

The transpose of the identity matrix is just the identity matrix, and we can simplify the transpose of
the product using the rule (AB)T = B T AT . So, the previous equation becomes

(M −1 )T M T = M T (M −1 )T = In .

Therefore, the inverse of M T is (M −1 )T .

Caution: The above argument is more subtle than it may look at first. It is not true in general, that
if AB = In , then A and B are inverses of each other; can you think of a counterexample?

5. Suppose A is an n × m matrix and B is an m × p matrix. Decide whether each of the following

expressions makes sense; if so, what size is the matrix?
1 If this seems unfamiliar, you probably remember that an n × n matrix A is invertible if and only if rref(A) = I . But
n
rref(A) = In if and only if rref(A) has n leading ones; that is, rref(A) = In if and only if rank A = n.

2
 (a) (AB)T

Solution. This makes sense; the product AB is an n × p matrix, so the transpose (AB)T is a
p × n matrix.

(b) AT B T
Solution. This doesn’t make sense: AT is an m × n matrix and B T is a p × m matrix, so the
product AT B T is not defined (unless n happens to equal p).

(c) B T AT
Solution. This makes sense; B T is a p × m matrix and AT is an m × n matrix, so B T AT is a
p × n matrix. In fact, B T AT is equal to (AB)T .

6. Complete the following properties:

(a) (AB)T = B T AT

(b) (AT )−1 = (A−1 )T

(assuming A is invertble)

(c) (AT )T = A

7. Show that ~x · A~y = AT ~x · ~y

Solution. ~x · A~y = (~x)T A~y = AT ~x · ~y

One can use this property to prove that if A is a orthogonal matrix, then A~x · A~y = ~x · ~y . This implies
orthogonal transformations preserve the length of vectors and the angles between them.

Definition: A is symmetric if AT = A.

8. (T/F)

(a) If V is any subspace of Rn , then the matrix of proj V is symmetric.

True

(b) Any symmetric matrix is invertible.

False
   
0 3
1
9. Find the orthogonal projection P from R3 to the linear space spanned by ~v1 = 0, and ~v2 = 4
5
1 0

A matrix M whose columns are orthonormal has a special property: M T M is an identity matrix. We are
particularly interested in square matrices with orthonormal columns, so these get a special name.

3
Definition 1. An orthogonal matrix is a square matrix (with real entries) whose columns are orthonormal.

When M and M T are square matrices, the equation M T M = I tells us that M is invertible, and its inverse
is exactly M T .
Fact. If M is an orthogonal matrix, then M is invertible, and M −1 = M T .

This fact can save a lot of computational effort when working with orthogonal matrices, since taking a
transpose is much simpler than our usually method of calculating the inverse of a matrix.