Linear Algebra Recap

(Matrix Algebra)
 What is a matrix?
A rectangular array of numbers (we will concentrate on
real numbers). A nxm matrix has ‘n’ rows and ‘m’
columns
M11 M12 M13 M14  First row

M3x4  M21 M22 M23 M24  Second row 
M31 M32 M33 M34  Third row

First Second Third Fourth

column column column column
Row number
12 Column number
What is a vector?

A vector is an array of ‘n’ numbers

A row vector of length ‘n’ is a 1xn matrix

a1 a2 a3 a4 
A column vector of length ‘m’ is a mx1 matrix

 a1 
a 
 2
a 3 
Special matrices

Zero matrix: A matrix all of whose entries are zero

0 0 0 0
03x 4 
 0 0 0 0 
0 0 0 0
Identity matrix: A square matrix which has ‘1’ s on the
diagonal and zeros everywhere else.

1 0 0
I3x 3 
 0 1 0 
0 0 1
Matrix operations
Equality of matrices

If A and B are two matrices of the same size,

then they are “equal” if each and every entry of one
matrix equals the corresponding entry of the other.

 1 2 4 a b c 

A   3 0 7  
B  d e f  
 9 1 5  g h i 
a  1, b  2, c  4,
A  B  d  3, e  0, f  7,
g  9, h  1, i  5.
Matrix operations Addition of two
matrices

If A and B are two matrices of the same size,

then the sum of the matrices is a matrix C=A+B whose
entries are the sums of the corresponding entries of A and
B
 1 2 4   1 3 10 
  
A    3 0 7 B    3 1 0  
 9 1 5  1 0 6 
 0 5 14 

C  A  B   6 1 7  
 10 1 11
 Addition of of matrices
Matrix operations
Properties

Properties of matrix addition:

1. Matrix addition is commutative (order of
addition does not matter)
AB B A
2. Matrix addition is associative
A  B  C   A  B   C
3. Addition of the zero matrix
A0 0A  A
Matrix operations Multiplication by a
scalar

If A is a matrix and c is a scalar, then the product cA is a

matrix whose entries are obtained by multiplying each of
the entries of A by c

1 2 4

A   3 
0 7 c  3
 9 1 5
 3 6 12 

cA    9 0 21 
 27 3 15 
 Multiplication by a
scalar
Matrix operations
Special case

If A is a matrix and c =-1 is a scalar, then the product

(-1)A =-A is a matrix whose entries are obtained by
multiplying each of the entries of A by -1

 1 2 4

A    3 0 7  c  1
 9 1 5
1 2  4

cA  -A   3 0  7
  9 1  5
Matrix operations Subtraction

If A and B are two square matrices of the same

size, then A-B is defined as the sum A+(-1)B

 1 2 4   1 3 10 
  
A    3 0 7 B    3 1 0  
 9 1 5  1 0 6 
 2  1  6

C  A  B  0  1 7  
8 1  1
Note that A - A  0 and 0 - A  -A
 Transpose
Special
operations
If A is a mxn matrix, then the transpose of A is
the nxm matrix whose first column is the first
row of A, whose second column is the second
column of A and so on.

 1 2 4 1  3 9 
  
A    3 0 7   A   2 0 1
T 
 9 1 5 4 7 5
 Transpose
Special
operations

If A is a square matrix (mxm), it is called

symmetric if

AA T
Matrix operations Scalar (dot) product of
two vectors
If a and b are two vectors of the same size
 a1   b1 
 
a  a 2 ; b  b 
 2
a3  b3 
The scalar (dot) product of a and b is a scalar
obtained by adding the products of
corresponding entries of the two vectors
a b   a1b1  a 2 b 2  a 3 b3 
T
Matrix operations Matrix multiplication

For a product to be defined, the number of columns

of A must be equal to the number of rows of B.

A B = AB
mxr rxn mxn
inside

outside
 Matrix operations Matrix multiplication

If A is a mxr matrix and B is a rxn matrix, then the

product C=AB is a mxn matrix whose entries are
obtained as follows. The entry corresponding to row ‘i’
and column ‘j’ of C is the dot product of the vectors
formed by the row ‘i’ of A and column ‘j’ of B
 1 2 4  1 3 
A 3x3   3 0 7  B3x2   3 1 
 9 1 5   1 0 
T
 3 5  1  1
C3x2 
 AB  10  
9  notice  2    3  3
 
 7 28   4   1 
 Multiplication of
Matrix operations matrices
Properties

Properties of matrix multiplication:

1. Matrix multiplication is noncommutative
(order of addition does matter)
AB  BA in general
 It may be that the product AB exists but BA
does not (e.g. in the previous example
C=AB is a 3x2 matrix, but BA does not
exist)
 Even if the product exists, the products AB
and BA are not generally the same
 Multiplication of
Matrix operations matrices
Properties

2. Matrix multiplication is associative

ABC   AB C
3. Distributive law
AB  C  AB  AC
B  CA  BA  CA
4. Multiplication by identity matrix
AI  A; IA  A
5. Multiplication by zero matrix A0  0; 0A  0
6.
A B T
B A
T T
 Miscellaneous
Matrix operations properties

1. If A , B and C are square matrices of the

same size, and A  0 then AB  AC
does not necessarily mean that B  C
2. AB  0 does not necessarily imply that
either A or B is zero
 Definition
Inverse of a
matrix

If A is any square matrix and B is another

square matrix satisfying the conditions

A B  BA  I
Then
(a)The matrix A is called invertible, and
(b) the matrix B is the inverse of A and is
denoted as A-1.

The inverse of a matrix is unique

 Uniqueness
Inverse of a
matrix

The inverse of a matrix is unique

Assume that B and C both are inverses of A

AB  BA  I
AC  CA  I
(BA)C  IC  C
B(AC)  BI  B
BC
Hence a matrix cannot have two or more
inverses.
 Some properties
Inverse of a
matrix

Property 1: If A is any invertible square

matrix the inverse of its inverse is the matrix A
itself
A 
-1 1
A
Property 2: If A is any invertible square
matrix and k is any scalar then

1 -1
k A  1
 A
k
Inverse of a Properties
matrix
Property 3: If A and B are invertible square
matrices then
1
A B
1 -1
B A
(AB) AB 
1
I
-1
Premultiplying both sides by A
A-1(AB) AB   A 1
1

A ABAB 
-1 1
 A 1
BAB   A 1
1

-1
Premultiplying both sides by B
AB  1 1
B A 1
 What is a determinant?
The determinant of a square matrix is a number
obtained in a specific manner from the matrix.

For a 1x1 matrix:

A  a11  ; det( A)  a11
For a 2x2 matrix:

 a11 a12 
A  ; det( A)  a11a 22  a12a 21
a 21 a 22 

Product along red arrow minus product along blue arrow

Example 1
1 3
Consider the matrix A 
5 7 

Notice (1) A matrix is an array of numbers

(2) A matrix is enclosed by square brackets

1 3
det( A )   1  7  3  5  8
5 7

Notice (1) The determinant of a matrix is a number

(2) The symbol for the determinant of a matrix is
a pair of parallel lines
Computation of larger matrices is more difficult
Duplicate column method for 3x3 matrix
For ONLY a 3x3 matrix write down the first two
columns after the third column

a11 a12 a13   a11 a12 a13  a11 a12


A  a 21 a 22 a 23  a 
 21 a 22 a 23  a 21 a 22
a 31 a 32 a 33  a 31 a 32 a 33  a 31 a 32

Sum of products along red arrow

minus sum of products along blue arrow

det( A )  a11a 22a 33  a12a 23a 31  a13a 21a 32

 a13a 22a 31  a11a 23a 32  a12a 21a 33
This technique works only for 3x3 matrices
Example

2 4 - 3  2 4  3 2 4
A   1 0 4 

1 0 4 1 0
 2 - 1 2   
2  1 2  2 1

0 -8 8 0 32 3

Sum of red terms = 0 + 32 + 3 = 35

Sum of blue terms = 0 – 8 + 8 = 0
Determinant of matrix A= det(A) = 35 – 0 = 35
 Finding determinant using inspection
Special case. If two rows or two columns are proportional
(i.e. multiples of each other), then the determinant of the
matrix is zero

2 7 8
3 2 4 0
2 7 8

because rows 1 and 3 are proportional to each other

If the determinant of a matrix is zero, it is called a

singular matrix
 What is a cofactor?
Cofactor method
If A is a square matrix
a11 a12 a13 
A  a 21 a 22 a 23 
a 31 a 32 a 33 
The minor, Mij, of entry aij is the determinant of the submatrix
that remains after the ith row and jth column are deleted from A.
The cofactor of entry aij is Cij=(-1)(i+j) Mij

a 21 a 23 a 21 a 23
M12   a 21a 33  a 23a 31 C12   M12  
a 31 a 33 a 31 a 33
What is a cofactor?

Sign of cofactor  - 
-  - 
 

 - 
Find the minor and cofactor of a33
2 4 - 3 
A   1 0 4 
 Minor 2 4
 2 - 1 2  M 33   2  0  4  1  4
1 0

Cofactor C  ( 1) ( 33) M  M  4

33 33 33
 Cofactor method of obtaining the
determinant of a matrix

The determinant of a n x n matrix A can be computed by

multiplying ALL the entries in ANY row (or column) by
their cofactors and adding the resulting products. That is,
for each 1  i  n and 1  j  n

Cofactor expansion along the jth column

det( A)  a1jC1j  a 2jC 2j    anjCnj

Cofactor expansion along the ith row

det( A)  ai1Ci1  ai2Ci2    ainCin

 Example: evaluate det(A) for:
1 0 2 -3
A= 3 4 0 1
det(A) = a11C11 +a12C12 + a13C13 +a14C14
-1 5 2 -2
0 1 1 3

4 0 1 3 0 1 3 4 1
det(A)=(1) 5 2 -2 - (0) -1 2 -2 +2 -1 5 -2
1 1 3 0 1 3 0 1 3
3 4 0
- (-3) -1 5 2 = (1)(35)-0+(2)(62)-(-3)(13)=198
0 1 1
 Example : evaluate
1 5 -3
det(A)= 1 0 2
3 -1 2
By a cofactor along the third column

det(A)=a13C13 +a23C23+a33C33

1 0 1 5 1 5
det(A)= -3* (-1)4 +2*(-1)5 +2*(-1)6
3 -1 3 -1 1 0

= det(A)= -3(-1-0)+2(-1)5(-1-15)+2(0-5)=25
Quadratic form

The scalar Ud kd

T d  vector
k  square matrix
Is known as a quadratic form

If U>0: Matrix k is known as positive definite

If U≥0: Matrix k is known as positive semidefinite
Quadratic form

 d1   k11 k12 
Let d  k  
Symmetric
d 2  k 21 k 22  matrix

Then
 k11 k12   d1 
U  d k d  d1 d 2 
T
  
k12 k 22  d 2 
 k11d1  k12 d 2 
 d1 d 2  
k12 d1  k 22 d 2 
 d1 (k11d1  k12 d 2 )  d 2 (k12 d1  k 22 d 2 )
2 2
 k11d1  2k12 d1 d 2  k 22 d 2
Differentiation of quadratic form

Differentiate U wrt d1
U
 2k11d 1  2k12 d 2
d1
Differentiate U wrt d2
U
 2k12 d 1  2k 22 d 2
d 2
Differentiation of quadratic form

Hence

 U 
U  d1   k11 k12   d1 
   2   
 d  U  k12 k 22  d 2 
 d 2 
2k d