MA1513

Chapter 1: (Linear Algebra) Matrix and System

1.1 Systems of Linear Equations (P.1)
1.2 Solving System of Linear Equations (P.7)
1.3 Gaussian Elimination (P.14)
1.4 Matrices (P.18)
1.5 Matrix Inverses I (P.22)
1.6 Matrix Inverses II (P.25)
1.7 Determinants of Matrices (P.29)
1.8 Ranks of Matrices (P.33)

1.1 Systems of Linear Equations

In this section, we introduce the most fundamental object in Linear Algebra – System of Linear
Equations (or Linear System), which play an important role in the mathematical modelling of many
engineering problems and other real life applications.
Linear Equations (slide 2)
In general, a linear equation in n variables is given by this algebraic expression:
a1x1 + a2x2 + ··· + anxn = b
Here x1 to xn denote the variables, while a1 to an, and b represent some fixed numbers.
Specifically, a1 to an are called the coefficients corresponding to the variables, and b is called the
constant term.
Examples:
(a) x + 3y = 7
(b) y = x – 0.5z + 4.5
(c) x1 + x2 + ··· + xn = 1
(d) x1 + 2x2 + 2x3 + x4 = x5
(a) is a linear equation in two variables x and y. This represents a line in the xy plane.
(b) is a linear equation in three variables x, y, z. This represents a plane in the xyz space. (Note that
the x and z terms are on the right side of the equation. In any equation, we are free to move the
various terms between the left and right side. But the standard form is to place all the variables on
the left, and the constant term on the right as such: -x + y + 0.5z = 4.5.)
In (c), instead of representing the variables using regular letters like x, y, z, we use the indexed
notation x1, x2, …, xn. This is common when there are many variables in the equation.
(d) also uses indexed notation. There are 5 variables x1 to x5. Again, we can bring the variable x5 to
the left side to get the standard form.

The key feature of a linear equation is that different variables in the equation
can only be combined using addition or subtraction.

1
Non-linear Equations (slide 3)
Examples:
(a) xy = 2
(b) sin(θ) + cos (φ) = 0.2
(c) x12 + x22+ ··· + xn2 = 1
(d) x = ey

(a) is not linear because it involves the multiplication of the two variables x and y.
(b) is not linear in the variables θ and φ, as the sine and cosine functions are being applied to the
variables.
(c) involves the square of each variable x1 to xn, and hence is not linear.
(d) is not linear because the exponential function is applied to the variable y.

Geometrical Representation (slide 4)

If a linear equation has two variables x and y such as ax + by = c, then it represents a line in the xy-
plane.

(Note that in this course, when we use the word “line”, we always refer to straight line, rather than a
line that curves or bends.)

If a linear equation has three variables x, y and z such as ax + by +cz = d, then it does not represent a
line in the xyz-space. Rather, it represents a flat plane in the xyz-space.

A linear equation always represent objects that are straight or flat. Furthermore, such an equation
always represent an object that is “one dimension smaller” than that of the space it is sitting in.

If a linear equation ax + by + cz + dw = e has four variables (x, y, z and w), it does not have any
geometrical interpretation. This is because with 4 variables, it means we need a 4-dimensional space
to hold the three dimensional object. This is something beyond visualization.

Nevertheless, linear equations with more than 3 variables still have many real life applications in
math modelling.

2
System of Linear Equations (slide 5)

When a few linear equations are put together, they form a system of linear equations.
𝑎𝑎 𝑥𝑥 + 𝑎𝑎12 𝑥𝑥2 + … + 𝑎𝑎1𝑛𝑛 𝑥𝑥𝑛𝑛 = 𝑏𝑏1
⎧ 11 1
𝑎𝑎21 𝑥𝑥1 + 𝑎𝑎22 𝑥𝑥2 + … + 𝑎𝑎2𝑛𝑛 𝑥𝑥𝑛𝑛 = 𝑏𝑏2
⎨ ⋮ ⋮
⎩𝑎𝑎𝑚𝑚1 𝑥𝑥1 + 𝑎𝑎𝑚𝑚2 𝑥𝑥2 + … + 𝑎𝑎𝑚𝑚𝑚𝑚 𝑥𝑥𝑛𝑛 = 𝑏𝑏𝑚𝑚
The equations must have the same set of variables in common. Such a system is also called a linear
system.
From the indexed notation, we see that there are m linear equations in the system, and n common
variables x1 to xn.
Again, we denote the coefficients and constant terms by the a’s and the b’s respectively.
To better track the coefficients in the system, we have used a “double indexed” notation. For
example, a12 refers to the coefficient of x2 in the 1st equation, and so on.

Solutions of Linear System (slide 6)

Given a linear system, one of the problems we would like to solve is to find solutions to the system.
Specifically, for a system with n variables say x1 to xn, we want to find n (real) numbers, say s1, s2, ..,
sn that satisfy all the equations simultaneously. When this happen, we say s1 to sn is a solution of the
system and write it in the following way to specify which number correspond to which variable:
x1 = s1, x2 = s2, …, xn = sn
This is known as a particular solution of the linear system.
Example: Linear System with more than 1 solution (slide 7)
 4 x1 − x2 + 3x3 = −1
 3x + x + 9 x = −4
 1 2 3

x1 = 1, x2 = 2, x3 = –1 is a (particular) solution, since it satisfies both equations in the system.

x1 = 1, x2 = 8, x3 = 1 is not a solution, since it satisfies the first equation but not the second one.
A linear system need not have just one solution. You can check that x1 = 0, x2 = -1/4, x3 = –5/12
is another particular solution of the system.
In fact, we shall see later that when a linear system has more than 1 solution, it will have infinitely
many solutions.

Example: Linear System with no solution (slide 8)

With the constraint of each linear equation in a system, it is highly possible for a random system to
have no solution at all.
 x + y = 4
 2 x + 2y = 6

This system has no solution. Observe the second equation can be simplified as x + y = 3. Then we see
that the first equation requires the sum of the two variables to be 4, while the second equation
requires the sum to be 3. In other words, the two equations are contradicting each other, or not
consistent with each other, hence it is impossible to find a solution.
In general, when we have three or more equations in a linear system, it may not be easy to tell
whether the equations are inconsistent by observation. We will see later how to determine the
consistency of a general linear system.

3
Number of Solutions of Linear Systems (slide 9)
An important fact about the number of solutions that any linear system can have:
There are only three possibilities:
 no solution
 exactly one solution
 infinitely many solutions

For the first case, we call a system with no solution as an inconsistent system, as there are
inconsistencies among the equations in the system.
For the second and third cases, the system has at least one solution, and we say the system is
consistent.
In our earlier example
 4 x1 − x2 + 3x3 = −1
 3x + x + 9 x = −4
 1 2 3

we have found two solutions. Since the system cannot have just two solutions, we conclude that it
must belong to the third case, i.e. it has infinitely many solutions, even though we have not explicitly
found them. We shall see later how to find all the solutions for the infinite case.

2 by 2 Linear Systems (slide 10)

The reason that there are only three possible cases for the number of solutions for any system is
because it is linear. The 2 x 2 system with two equations in two variables will provide some insight to
this peculiar property.
 a1 x + b1y = c1 ( l1 )
a x +by =c ( l2 )
 2 2 2

Geometrically the two equations represent two lines 𝑙𝑙 1 and 𝑙𝑙 2 in the xy-plane.
There are three possible relative positions for the two lines:

In the first diagram, the two lines 𝑙𝑙 1 and 𝑙𝑙 2 are parallel, so they have no intersection. This means the
system has no solution.
In the second diagram, 𝑙𝑙 1 and 𝑙𝑙 2 are not parallel, so they will intersect at exactly one point. This
corresponds to exactly one solution for the system.
In the third diagram, the two lines overlap, so 𝑙𝑙 1 and 𝑙𝑙 2 represent the same line. This means every
point on the line is part of the intersection. Hence the system has infinitely many solutions.

4
Homogeneous Linear Systems (slide 11)
We say a system is homogeneous if the system looks like this:
𝑎𝑎11 𝑥𝑥1 + 𝑎𝑎12 𝑥𝑥2 + … + 𝑎𝑎1𝑛𝑛 𝑥𝑥𝑛𝑛 = 0
𝑎𝑎21 𝑥𝑥1 + 𝑎𝑎22 𝑥𝑥2 + … + 𝑎𝑎2𝑛𝑛 𝑥𝑥𝑛𝑛 = 0
�
⋮ ⋮
𝑎𝑎𝑚𝑚1 𝑥𝑥1 + 𝑎𝑎𝑚𝑚2 𝑥𝑥2 + … + 𝑎𝑎𝑚𝑚𝑚𝑚 𝑥𝑥𝑛𝑛 = 0
Notice that it looks almost like an ordinary linear system, except that the constant terms are all 0.
On the other hand, if some of the constant terms are not zero, we say the system is non-
homogeneous.

Trivial Solution of Homogeneous System (slide 12)

There is an obvious solution to the homogeneous system. When we substitute all the variables with
0, they clearly satisfy all the equations, and hence
x1 = 0, x2 = 0, …, xn = 0
is a solution of the system. We call this the trivial solution of the homogeneous system. In particular,
a homogeneous system is always consistent regardless of the coefficients.
It is possible for a homogeneous system to have only the trivial solution, or it might have other
additional solutions. These additional solutions are called non-trivial solutions. Here’s an example of
a homogeneous system that has more than one solution.
 x1 − x2 + x3 = 0

 x1 + x2 + 3x3 = 0

First of all, it has the trivial solution x1 = 0, x2 = 0, x3 = 0. Check that x1 = 2, x2 = 1, x3 = -1 is another

solution. So this is a non-trivial solution to the system, and we know this implies the system in fact
has infinitely many solutions.

Number of Solutions of Homogeneous Systems (slide 13)

Here are two properties about the number of solutions of homogeneous systems.

1. A homogeneous system of linear equations has either only the trivial solution or infinitely
many solutions in addition to the trivial solution.
2. A homogeneous system of linear equations with more variables than equations has
infinitely many solutions.

Examples:

The homogeneous system on the left has 2 equations and 3 variables; and the system on the right
has 3 equations and 4 variables. By the second property above, regardless of what their actual
coefficients are, we can conclude that both systems have infinitely many solutions.

5
Trivial Vs Non-trivial Solutions (slide 14)
Consider the 2 x 3 homogeneous system, with 2 equations and 3 variables.
 a1x + b1y + c1z = 0 (P1 )

 a2 x + b2 y + c 2 z =
0 (P2 )
Recall these two equations represent two planes in the xyz-space. The constant term being zero,
means the planes contain the origin. So the two planes intersect at least the origin, which
corresponds to the trivial solution.
But the property in the previous segment tells us that the system has more than just the trivial
solution.
There are two possible relative positions of the two planes as shown in the diagrams below.

P1

First possibility (left diagram) is the two planes intersect. We see that they must intersect at a line,
which consists of infinitely many points. They are the origin, which represent the trivial solution, and
all other points, representing the non-trivial solutions. Hence this homogeneous system has infinitely
many solutions.
The second possibility (right diagram) is when the two planes overlap each other completely. This
time we have the origin representing the trivial solution, and infinitely many other points on the
plane representing the non-trivial solutions. Hence there are again infinitely many solutions for the
homogeneous system.

6
1.2 Solving System of Linear Equations
When we are given a linear system, how do we determine whether it has a solution? If it does, how
do we find all the solutions in a systematically way? This section provides the first step towards
answering these questions.

Augmented Matrix (slide 2)

First, we use a slightly simpler notation to represent a linear system.
𝑎𝑎11 𝑥𝑥1 + 𝑎𝑎12 𝑥𝑥2 + … + 𝑎𝑎1𝑛𝑛 𝑥𝑥𝑛𝑛 = 𝑏𝑏1
𝑎𝑎21 𝑥𝑥1 + 𝑎𝑎22 𝑥𝑥2 + … + 𝑎𝑎2𝑛𝑛 𝑥𝑥𝑛𝑛 = 𝑏𝑏2
�
⋮ ⋮
𝑎𝑎𝑚𝑚1 𝑥𝑥1 + 𝑎𝑎𝑚𝑚2 𝑥𝑥2 + … + 𝑎𝑎𝑚𝑚𝑚𝑚 𝑥𝑥𝑛𝑛 = 𝑏𝑏𝑚𝑚
This is a system with m equations and n variables.
We take all the coefficients and constant terms, and arrange them in a rectangular array as shown.
𝑎𝑎11 𝑎𝑎12 … 𝑎𝑎1𝑛𝑛 𝑏𝑏1
𝑎𝑎21 𝑎𝑎22 … 𝑎𝑎2𝑛𝑛 𝑏𝑏2
� ⋮ ⋮ ⋮ � ⋮ �
𝑎𝑎𝑚𝑚1 𝑎𝑎𝑚𝑚2 … 𝑎𝑎𝑚𝑚𝑚𝑚 𝑏𝑏𝑚𝑚
This array has m horizontal rows, corresponding to the m equations of the system, and n+1 vertical
columns, corresponding to the n variables and the constant terms. Note that, to keep the notation
simple, the variables, the plus and equal signs are not displayed in the array. Instead, a vertical line is
drawn to separate the coefficients and the constant terms. We call this the augmented matrix of the
linear system.

Elementary Row Operations (slide 3)

Next we reduce the augmented matrix to a simpler form, so that it becomes easier to solve the linear
system. The way to do this is by performing some operations on the rows of the matrix. There are
three types of row operations, namely
• Multiplying a row by a nonzero constant;
• Interchanging two rows;
• Adding a multiple of a row to another row.
These are the three basic operations on the rows of a matrix, which we called the elementary row
operations.

Example: Elementary Row Operations (slide 4)

For the first type of row operation, multiply the first row of the left augmented matrix by the
number 3, we will get the augmented matrix on the right.
1 1 2 9  3 3 6 27 
   
 2 4 −3 1   2 4 −3 1 
 3 6 −5 0   3 6 −5 0 
   

Notice that all the entries in the first row get multiplied by 3.

7
For the second type of row operation, we interchange the second and third row of the left
augmented matrix to get the right.

1 1 2 9 1 1 2 9
   
 2 4 −3 1   3 6 −5 0 
 3 6 −5 0   2 4 −3 1 
   

For the third type of row operation, we want to add 2 times of the first row to the second row. First,
we multiply the first row of the left augmented matrix by 2 to get (2 2 4 | 18). Then we add this to
the second row of the same matrix, by adding the corresponding entries. The resulting matrix looks
like this.
1 1 2 9 1 1 2 9 
   
 2 4 −3 1   4 6 1 19 
 3 6 −5 0   3 6 −5 0 
   

Note that in the resulting matrix, only the second row is changed; the first row will remain the same
as the original matrix.

Notation of Elementary Row Operations (slide 5)

Let us denote the rows of an augmented matrix to be operated on using the indexed notation Ri. So
R1 represents the first row, R3 represents the third row and so on. With that, we can now represent
the three row operations as follow:
First type: cRi (multiplying the ith row by the constant c)
Second type: Ri ↔ Rj (interchanging the positions of the ith and jth rows)
Third type: Ri + cRj (adding c times the jth row to the ith row)
In the last notation, it means the ith row will be changed, but the jth row remains unchanged after the
operation.

Example: Solving Linear System (slide 6-9)

Take note of the action taken on the left hand side (the linear system), and the corresponding row
operations being performed on the right side (the augmented matrix).
In the first step below, we add -2 times of equation 1 to equation 2, and change equation 2 to
equation 4 as shown. Correspondingly, the second row of the augmented matrix is changed
accordingly by adding -2 times of first row of the augmented matrix to the second row as shown in
the diagram.

8
In this next step, we change equation 3 to equation 5. This correspond to the row operation of
adding -3 times of the first row of the matrix to the third row.

Next, we change equation 5 to equation 6. This corresponds to the row operation of adding 6/4
times of the second row of the matrix to the third row.

Let’s summarize how the original augmented matrix has been transformed under the operations:

1 1 3 0 1 1 3 0 1 1 3 0 1 1 3 0
       
 2 − 2 2 4  0 − 4 − 4 4 0 − 4 − 4 4
 0 − 4 − 4 4 
3 9 0 3  3 9 0 3  3  0 0 − 15 9 
0 6 − 9 
Observe that the augmented matrix now has a “staircase” of 0 at the bottom left corner of the array.
This is the simplified form of the augmented matrix that we are looking for. It is known as the row
echelon form (which will be defined more precisely later).

In the corresponding linear system, we observe the number of variables in each equation decreases
as we go down the row. This will facilitate us to solve the equations from bottom up, which we call
back substitution.

First, from (6), we can easily solve z = -3/5.

Substituting this value of z into (4), we get -4y − 4(-3/5) = 4 which gives y = -2/5.
Finally, substituting the values of y and z into (1), we get x −2/5 −9/5 = 0, which gives x = 11/5.
These three numbers give us a solution of the linear system. In fact, from the process, we see that
this is the only solution of the original linear system.

9
Row Echelon Form (R.E.F.) (slide 10)
 In a row echelon form, any zero rows must be below all the nonzero rows.
A zero row is one whereby all the entries are 0: (0 0 … 0 | 0); and a nonzero row is one with some
nonzero entries. For example, (0 0 3 0 1 | 0) and (0 0 0 0 0 | 2) are examples of non-zero rows.
If we go from left to right on each nonzero row, the first nonzero entry is called the leading entry. So
the leading entry of (0 0 3 0 1 | 0) is 3, and the leading entry of (0 0 0 0 0 | 2) is 2.
In the diagram below, the symbol ⊗ represents the leading entry of each non-zero row. (Note that
entries * on the right of the leading entry can be either zero or non-zero.)
0 ⊗ ∗ ∗ ∗ ∗ ∗ ∗
0 0 0 ⊗ ∗ ∗ ∗ ∗
⎛ ⎞
0 0 0 0 0 ⊗ ∗ �∗
⎜ ⎟
⎜0 0 0 0 0 0 ⊗� ∗ ⎟
0 0 0 0 0 0 0 0
⎝0 0 0 0 0 0 0 0⎠
The most important feature of a row echelon form is that
 the number of zeros on the left of the leading entries is strictly increasing as we go down the
row, creating a staircase of zeros at the bottom left of the matrix.
(Note that the width of each stair need not be the same.)
We are interested in the columns of a row echelon form that contain the leading entries. We call
these columns the pivot columns.
0 ⊗ ∗ ∗ ∗ ∗ ∗ ∗
0 0 0 ⊗ ∗ ∗ ∗ ∗
⎛ ⎞
0 0 0 0 0 ⊗ ∗ �∗
⎜ ⎟
⎜0 0 0 0 0 0 ⊗� ∗ ⎟
0 0 0 0 0 0 0 0
⎝0 0 0 0 0 0 0 0⎠
In the above augmented matrix, the pivot columns are the 2nd, 4th, 6th and 7th columns. The
remaining columns in the matrix are called the non-pivot columns.
The pivot and non-pivot columns in the row echelon form play an important role in determining the
number of solutions of a linear system.

Example: System with Infinitely Many Solutions (slide 11)

This is a system that is already reduced. In other words, its augmented matrix is in row echelon form.
This system has 4 variables and 3 equations. In other words, there are more variables than
equations. We need to set some variables as free parameters. The variables to be set as parameters
are those that correspond to non-pivot columns in the row echelon form.
In our example, the third column is a non-pivot column, and it corresponds to variable x3. So we shall
set x3 as a free parameter, say t, and write
x3 = t where t ∈ ℝ.
In other words, x3 can represent any real number.

10
We shall substitute all the variable x3 in the system with the parameter t and perform back-
substitution as before:
From (3), we solve x4 = 2.
Substituting x4 by 2 in the system, we can then solve (2) for x2 in terms of t: x2 = 1 – t.
Substituting x2 by 1 – t in the system, we can finally solve x1 = 2 + (1/2) t.
By putting all these together, we can write the solution in the following way:
 x1 = 2 + 12 t

 x2 = 1−t

 x3 = t
 x 4 = 2

The parameter t represents any real number, and therefore gives infinitely many solutions of the
linear system. In fact, this will give all the possible solutions of the system, and hence it is called a
general solution of the system.

Solutions of Linear Systems (slide 12)

We can determine how many solutions a system has from the row echelon form of the augmented
matrix of the system.
(i) When every column of the row echelon form is a pivot column, except the last column,
the system will have exactly one solution.
Here is an example we have seen earlier. (The columns in blue are pivot columns.)

(ii) When there is at least one non-pivot column in the row echelon form other than the last
column, the system will have infinitely many solutions.
This is an example we have seen in the previous segment. (The column highlighted is the non-pivot
column.)

Recall that the non-pivot columns (can be more than one) correspond to variables being set as free
parameters, and hence giving infinitely many solutions.
What about the case of no solution? Can we tell from the row echelon form? The answer is yes.
(iii) When the very last column of the row echelon form is a pivot column, the system will
have no solution.

Example: System with No Solution (slide 13)

Suppose a linear system has the augmented matrix on the left below and we perform some
elementary row operations to reduce it to a row echelon form.

11
Convert this back to equation form,
 3x1 + x2 = 4

 2 x2 = 1
 0 = 1

we see that the third equation is 0 = 1, which is impossible. This suggests that the system is
inconsistent without solution. The corresponding row in the row echelon form (rightmost
augmented matrix above) is (0 0 | 1), which means that 1 is a leading entry, and hence the last
column in the row echelon form is a pivot column as indicated in the diagram.

Non-homogeneous VS Homogeneous System (slide 14-15)

This is an earlier example of linear system, which we denote by (A).
 2x1 + 2x2 + x3 − 2x4 = 2
(A) 
 x2 + x3 + x 4 = 3
 2x4 = 4

We have obtained the general solution of (A). By setting t with any fix value, we get a particular
solution of the system. For example, if we let t = 0, then we get a particular solution as shown below.

Write down the associated homogeneous system of (A) as shown below, and denote it by (B):
 2x1 + 2x2 + x3 − 2x4 = 0
(B) 
 x2 + x3 + x 4 = 0
 2x4 = 0


Note that (A) and (B) are almost the same except that the constant terms in (B) are all 0.
By going through the same process (back substitution) as before, we can find the general solution of
the homogeneous system:

Compare the two general solutions, we notice the similarity of the parameter terms between the
two systems. Furthermore, we also observe the relations between the constant terms of the general
solution of (A) and the particular solution of (A) above.
From this example, we can formulate the following relationship:

(**) general solution of (A) = general solution of (B) + a particular solution of (A)

The general solution of the non-homogeneous system (A) is the sum of the general solution of the
associated homogeneous system (B) and a particular solution of (A).

12
In fact, given any non-homogeneous system and its associated homogeneous system, if the non-
homogeneous system is consistent, the same relationship (**) between their general solution holds.
Note that the particular solution in (**) can be obtained by substituting any value for the
parameters, i.e. it is not necessary to set t = 0.
In other words,
(∆∆) general solution of (A) * = general solution of (B) + another particular solution of (A)
* The general solution of (A) in (∆∆) may not look the same as the general solution of A in (**). But
both of them will give all possible solutions of (A).

13
1.3 Gaussian Elimination
This section illustrates a systematic approach to reduce the augmented matrix of the system to a row
echelon form. The method Gaussian Elimination (G.E.) that we are going to introduce is named after
the great mathematician Carl Friedrich Gauss. The process is a series of elementary row operations
performed according to a set of instruction. The objective is to reduce an augmented matrix to a row
echelon form. We will use the following two augmented matrices as examples to illustrate the steps.

Example A Example B
0 3 … … 0 3 1 …
�1 −2 … … � �0 2 −3 … �
4 0 … … 0 −1 6 …
Step 1 (slide 3)
 Locate the leftmost column that does not consist entirely of zero.
Example A Example B
0 3 … … 0 3 1 …
�1 −2 … … � �0 2 −3 … �
4 0 … … 0 −1 6 …

Step 2 (slide 4)
 Interchange the top row with another row, if necessary, to bring a nonzero entry to the top of the
column found in Step 1.
Example A Example B
0 3 … … 0 3 1 …
�1 −2 … … � �0 2 −3 … �
4 0 … … 0 −1 6 …

For example A, the top entry is 0, so we perform the second type of row operation, namely
interchanging the 1st row with another row with nonzero entry. In this case, we can interchange it
with the second row R1 ↔ R2. In fact, it is also possible to interchange the 1st row with the 3rd row.
For example B, as the top entry is nonzero, it is not necessary to interchange row.
Step 3 (slide 5)
 For each row below the top row, add a suitable multiple of the top row to it so that the entry
below the leading entry of the top row becomes zero.
Example A Example B
1−2 … … 0 3 1 …
�0 3 … … � �0 2 −3 … �
4 0 … … 0 −1 6 …

In example A, there is only one nonzero entry 4 in the third row. So only one row operation is
needed. We perform the third type of operation R3 – 4R1.

14
In example B, there are two nonzero entries 2 and -1 below the top entry. So we need two row
operations to turn them into 0. We perform R2 – (2/3)R1 and R3 + (1/3)R1.
(Remember, we are trying to build a staircase of zero at the bottom left corner of the matrices.)
Step 4 (slide 6)
 Now cover the top row in the matrix and begin again with Step 1 applied to the submatrix that
remains.
Example A Example B
1 −2 … … 0 3 1 …
�0 3 … …� �0 0 −11/3 …�
0 8 … … 0 0 19/3 …
In example A, after covering the top row, the next nonzero column is the second column. In example
B, after covering the top row, the next nonzero column is the third column. We repeat step 1 to step
3 to both matrices.
Final Step (slide 7)
 After each cycle, we cover one more row, and repeat the process, until the matrix is reduced to a
row-echelon form.
Example A Example B
1 −2 … … 0 3 1 …
�0 3 … …� �0 0 −11/3 …�
0 0 … … 0 0 0 …
Note that the final matrices that we get in both examples are in row echelon form. This completes the
Gaussian elimination.

Reduced Row Echelon Form (R.R.E.F.) (slide 8)

 A row echelon form is said to be a reduced row echelon form, if in each of its pivot columns, the
leading entry is 1 and all other entries are 0.
Let us illustrate this with our diagram. For a row echelon form, the leading entries can be any
nonzero numbers. But for a reduced row echelon form, the leading entries are all equal to 1.
Furthermore, in the pivot columns, all other non-leading entries are 0.
0 ⊗ ∗ ∗ ∗ ∗ ∗ ∗ 0 1 ∗ 0 ∗ 0 0∗
0 0 0 ⊗ ∗ ∗ ∗ ∗ 0 0 0 1 ∗ 0 0∗
⎛ ⎞ ⎛0
0 0 0 0 0 ⊗ ∗ �∗ 0 0 0 0 1 0� ∗ ⎞
⎜ ⎟ ⎜ ⎟
⎜0 0 0 0 0 0 ⊗� ∗⎟ ⎜0 0 0 0 0 0 1� ∗⎟
0 0 0 0 0 0 0 0 0 0 0 0 0 0 00
⎝0 0 0 0 0 0 0 0⎠ ⎝0 0 0 0 0 0 0 0⎠
Row echelon form Reduced row echelon form
In some applications, it is necessary for us to reduce a matrix to this specific reduced row echelon
form. In order to do that, we need to extend the Gaussian elimination to include additional steps.
This modified version of the procedure is called Gauss-Jordan elimination.

15
Gauss-Jordan Elimination (G.J. E.) (slide 9-11)
Step A: Multiply a suitable constant to each row so that all the leading entries becomes 1.
We use a different example here for illustration. This matrix is in row echelon form (the blank space
on the bottom left corner represent a staircase of zero entries), but some of the leading entries are
not 1.
2 −4 … … −8 …
⎛ 1 … … 3 …⎞
⎜ ⋱ … … …⎟
−3 −6 …
⎝ 1 …⎠

So we need to perform type 1 elementary row operations on those rows indicated. The resulting
augmented matrix is given below:
1 −2 … … −4 …
⎛ 1 … … 3 …⎞
⎜ ⋱ … … …⎟
1 2 …
⎝ 1 …⎠

Step B: Start from the rightmost pivot column and work “upward” − add a suitable multiples of
each row to the rows above to introduce 0 above the leading entries.
1 −2 … … −4
… Work
⎛ 1 … … 3
… ⎞ from
⎜ ⋱ … …… ⎟ bottom
1 2
… up
⎝ 1
…⎠
We work from “bottom up” starting with the entry 2 right above the leading entry. For this entry, we
add -2 times the bottom row to its row. Similarly, for the entry 3, we add -3 times of the bottom row
to its row, which is the second row. Finally, for the entry -4, we add 4 times of the bottom row to the
first row.
With that, we have completed the rightmost pivot columns. We then move on to the next pivot
columns to the left and repeat the same procedure.
1 −2 … … 0 …
⎛ 1 … … 0 …⎞
⎜ ⋱ … … …⎟
1 0 …
⎝ 1 …⎠

Work from right to left

When this next pivot column is done, we move to the next one on the left, and so on.
Remember, for these additional steps in Gauss-Jordan elimination,
we have to work from right to left, bottom to top.

16
Example: Solving a Linear System using G.E./G.J.E. (slide 12-13)
 2x3 + 4x4 + 2x5 = 8

 x1 + 2x2 + 4x3 + 5x4 + 3x5 = − 9
 − 2 x − 4 x − 5x − 4 x + 3x = 6
 1 2 3 4 5

Our objective is to make a comparison solve this linear system by using Gaussian elimination and
Gauss Jordan elimination. We start from the augmented matrix.
0 0 2 4 2 8
�1 2 4 5 3�−9�
−2 −4 −5 −4 3 6
After performing Gaussian elimination, we obtain a row echelon form:
𝑅𝑅1 ↔𝑅𝑅21 2 4 5 3 −9 𝑅𝑅3∓2𝑅𝑅1 1 2 4 5 3 −9 𝑅𝑅3−3/2𝑅𝑅2 1 2 4 5 3 −9
�⎯⎯⎯� � 0 0 2 4 2� 8 � �⎯⎯⎯⎯� �0 0 2 4 2� 8 � �⎯⎯⎯⎯⎯� �0 0 2 4 2� 8 �
−2 −4 −5 −4 3 6 0 0 3 6 9 −12 0 0 0 0 6 −24
If we continue with Gauss Jordan elimination, we will get a reduced row echelon form:
1
1/2𝑅𝑅2 ,1/6𝑅𝑅3 2 4 5 3 −9 𝑅𝑅2−𝑅𝑅3 1 2 4 5 3 −9 𝑅𝑅1−3𝑅𝑅3 1 2 4 5 0 3
�⎯⎯⎯⎯⎯⎯⎯⎯� �0 0 1 2 1� 4 � �⎯⎯⎯� �0 0 1 2 0� 8 � �⎯⎯⎯⎯� �0 0 1 2 0� 8 �
0 0 0 0 1 −4 0 0 0 0 1 −4 0 0 0 0 1 −4
𝑅𝑅1 −4𝑅𝑅2 1 2 0 −3 0 −29
�⎯⎯⎯⎯� �0 0 1 2 0� 8 �
0 0 0 0 1 −4
Put the row echelon form and reduced row echelon form respectively back in standard equation
forms, we get the systems:
 x1 + 2 x2 + 4 x3 + 5x4 + 3x5 = − 9  x1 + 2 x2 − 3x 4 = − 29
 
 2 x3 + 4 x4 + 2 x5 = 8  x3 + 2x4 = 8
 6 x = − 24  x5 = −4
 5 

By Gaussian elimination VS By Gauss-Jordan elimination

We observe that the system using G.J.E. is much simpler than that using G.E. Nevertheless, the
number of equations and variables in both systems are the same. In both cases, we need to
introduce parameters to two of the variables, since there are two more variables than equations.
If we look at the non-pivot columns (2nd and 4th columns) of the (reduced) row echelon form, the
variables that we need to set as parameters are x2 and x4, regardless of which method we use.
By setting x2 = s and x4 = t and substitute them into the systems, we can then solve the system in
terms of these two parameters.

For the system from G.E., we need to do back substitution to get the general solution. For the
simpler system from G.J.E., we get the general solution immediately by bringing the parameters to
the right hand side of the equations. We notice that the general solutions using either method are
the same:
 x1 = − 29 − 2s + 3t

 x2 = s
 x3 = 8 − 2t
x = t
 4
 x5 = −4

with parameters t and s representing any real numbers.

17
1.4 Matrices
We have come across the augmented matrix of a linear system in the earlier discussion. In this
section, we will formally introduce the concept of matrices and some of their operations.
The singular form is call Matrix; while the plural form is called Matrices. It is represented by a
rectangular array of numbers. Unlike the augmented matrix we have seen earlier, there is no
separating lines to separate the columns.
𝑎𝑎11 𝑎𝑎12 … 𝑎𝑎1𝑛𝑛
𝑎𝑎21 𝑎𝑎22 … 𝑎𝑎2𝑛𝑛
𝑨𝑨 = �
⋮ ⋮ ⋮ �
𝑎𝑎𝑚𝑚1 𝑎𝑎𝑚𝑚2 … 𝑎𝑎𝑚𝑚𝑚𝑚
A matrix has horizontal rows and vertical columns. From the indexed notation, we see that there are
m rows and n columns in this matrix A. We call A an m x n matrix and refer to m x n as the size of the
matrix.
The entry aij is called the ij-entry of the matrix. The double indices indicate the position of the entry,
i.e. it is on the ith row and jth column. For example, the (1,2)-entry a12 is on the first row and second
column, while the (2,1)-entry a21 is on the second row and first column.

Special Types of Matrices (slide 3-4)

This is an example of a 3 x 3 square matrix.

Diagonal entries of an n x n matrix refers to the (1,1)-

entry, (2,2)-entry, up to (n, n)-entry. In this example,
the diagonal entries are 1, 3, 2.
A scalar matrix is a special type of diagonal matrix.

We have a special notation In to denote the n x n

identity matrix. So the 3x3 identity matrix here is
denoted by I3.

Zero matrix need not be a square matrix. We can have

zero matrices of any sizes. We have a special notation
0mxn to denote the m x n zero matrix.
In our example, note that the first row consists of
entries 1 -1 0, and the first column also consists of 1 -1
0. Similarly for the other row-column pairs.
All entries on and “above” the diagonals have no
restriction, i.e. they can be zero or non-zero.
All entries on or “below” the diagonals can be zero or
non-zero.

18
Matrix Operations (slide 5)
Let A and B be matrices of the same size, and c a real number (known as a scalar). The table below
summarizes the four basic operations on matrices. The column on the right provides examples for
the 2 x 2 case.

Matrix Multiplication (slide 6)

A slightly less straight forward, but very useful operation, is matrix multiplication. We first illustrate
using a simple example.
1 1
1 2 3  
  2 3 
 4 5 6   −1 −2 
 
We represent matrix multiplication of two matrices by putting them side by side. If we denote the
two matrices by A and B, then the notation of the product is AB. Here we have a 2 x 3 matrix A on
the left, and a 3 x 2 matrix B on the right. After the multiplication, the resulting matrix AB is 2 x 2.
 1 × 1 + 2 × 2 + 3 × (−1) 1 × 1 + 2 × 3 + 3 × (−2)   2 1 
AB =   = 
 4 × 1 + 5 × 2 + 6 × (−1) 4 × 1 + 5 × 3 + 6 × (−2)   8 7 
For the (1,1)-entry, we take the first row of A and the first column of B and take the sum of the
product of their respective entries as shown. For the (1,2)-entry, we take the first row of A and the
second column of B and do the same. Similarly, for the (2,1)-entry, we use the second row of A and
the first column of B, and for the (2,2)-entry, we use the second row of A and the second column of
B. After simplifying, we get the resulting product on the right.

Multiply Matrices of Compatible Sizes (slide 7)

We can only multiply two matrices A and B (in the manner AB) when the number of columns of A
is equal to the number of rows of B. In other words, if A is of size m x p, then B must be of size p x n
in order to perform the multiplication AB. In this case, the size of the product AB will be m x n.

To illustrate with concrete examples, we are able to multiply the pair of 2x3 and 3x2 matrices, but
we cannot multiply the pair of 2x3 and 2x3 matrices.

Matrix multiplication Matrix multiplication

can be performed cannot be performed

19
Properties of Matrix Multiplication (slide 8)

1. Matrix multiplication is not commutative: AB ≠ BA in general, even if the product exist.

For example:

To distinguish between the product AB and BA, we say: for AB, A is pre-multiplied to B; and for
BA, A is post-multiplied to B.
2. AB = 0 does not imply A = 0 or B = 0.
i.e. two non-zero matrices can give a zero matrix after multiplication. See the example above.

Properties that are similar to ordinary numbers multiplication (slide 9)

3. A(BC) = (AB)C Associative property

4. A(B1 + B2) = AB1 + AB2 (Left) Distributive property
5. (C1 + C2) A = C1A + C2A (Right) Distributive property
6. c(AB) = (cA)B = A(cB) c is any scalar (real number)

Multiplication with Zero and Identity Matrices (slide 10)

Let A be an m x n matrix.
3 A0n x q = 0m x q and 0p x m A = 0p x n
Zero matrix behaves like number “0” in matrix multiplication
4 AIn = Im A = A
Identity matrix behaves like number “1” in matrix multiplication

Powers of Matrices (slide 11)

Let A be a square matrix and n be a nonnegative integer. We define An as follows:
𝑨𝑨𝑛𝑛 = 𝑨𝑨𝑨𝑨 … 𝑨𝑨 (n times)
𝑨𝑨0 = 𝑰𝑰
Matrix Transpose (slide 12)
Matrix Transpose of a matrix A, denoted by AT (or At) interchange the rows and columns of A.
For example,

If A is symmetric, then A = AT.

In other words, the transpose operator does not change a symmetric matrix. For example,

20
Some Properties of Transpose (slide 13)
Let A be an m x n matrix.
1. (AT)T = A
2. If B is an m x n matrix, then (A + B)T = AT + BT.
3. If a is a scalar, then (aA)T = aAT.
4. If B is an n x p matrix, then (AB)T = BTAT.
Note that, in property 4, the order of the multiplication is “reversed”: On the left side, we pre-
multiply A to B; while on the right hand side, we post multiply AT to BT.
Example:
1 1 1 𝑇𝑇 2 8
1 2 3 2 1 1 𝑇𝑇
(𝑨𝑨𝑨𝑨)𝑇𝑇 = �� �� 2 3 0�� = � � = �1 7�
4 5 6 8 7 4
−1 −2 0 1 4
𝑇𝑇
1 1 1 𝑇𝑇 1 2 −1 1 4 2 8
𝑇𝑇 𝑇𝑇 1 2 3
𝑩𝑩 𝑨𝑨 = � 2 3 0� � � = �1 3 −2� �2 5� = �1 7�
4 5 6
−1 −2 0 1 0 0 3 6 1 4

21
1.5 Matrix Inverses I
The concept of matrix inverse applies strictly to square matrices, and is closely linked to many
different topics in linear algebra.
Matrix Equation Form of Linear System (slide 2-3)
Look at the following linear system:

 4x + 5y + 6z = 1

 x − y = 2
 y − z = 3


We can rewrite the system in matrix equation form as given on the right.
In this form, we have a 3 x 3 matrix pre-multiply to a 3 x 1 column matrix. After performing the
matrix multiplication and equating with the right hand side, we recover the original linear system.
More generally, if we have a linear system with m equations and n variables, the matrix equation
form will be given as follow.
𝑎𝑎11 𝑥𝑥1 + 𝑎𝑎12 𝑥𝑥2 + … + 𝑎𝑎1𝑛𝑛 𝑥𝑥𝑛𝑛 = 𝑏𝑏1
𝑎𝑎21 𝑥𝑥1 + 𝑎𝑎22 𝑥𝑥2 + … + 𝑎𝑎2𝑛𝑛 𝑥𝑥𝑛𝑛 = 𝑏𝑏2
Standard form: �
⋮ ⋮
𝑎𝑎𝑚𝑚1 𝑥𝑥1 + 𝑎𝑎𝑚𝑚2 𝑥𝑥2 + … + 𝑎𝑎𝑚𝑚𝑚𝑚 𝑥𝑥𝑛𝑛 = 𝑏𝑏𝑚𝑚
Matrix Equation Form:
𝑎𝑎11 𝑎𝑎12 … 𝑎𝑎1𝑛𝑛 𝑥𝑥1 𝑏𝑏1
𝑎𝑎21 𝑎𝑎22 … 𝑎𝑎2𝑛𝑛 𝑥𝑥2 𝑏𝑏2
� ⋮ =
⋮ ⋮ �� ⋮ � � ⋮ �
𝑎𝑎𝑚𝑚1 𝑎𝑎𝑚𝑚2 … 𝑎𝑎𝑚𝑚𝑚𝑚 𝑥𝑥𝑛𝑛 𝑏𝑏𝑚𝑚
There are three matrices involved:
On the left we have the m x n matrices made up of all the coefficients of the linear system. We call it
the coefficient matrix.
In the middle we have the n x 1 column matrices consisting of the n variables of the system. We call
this the variable matrix.
On the right side of the matrix equation is an m x 1 column matrix, consisting of all the constant
terms. We call this the constant matrix.
A Concise Notation for Linear System (slide 4)
Matrix equation form allows us to represent linear system symbolically with letters. For example, if
we let
𝑎𝑎11 𝑎𝑎12 … 𝑎𝑎1𝑛𝑛 𝑥𝑥1 𝑏𝑏1
𝑎𝑎21 𝑎𝑎22 … 𝑎𝑎2𝑛𝑛 𝑥𝑥2 𝑏𝑏
𝑨𝑨 = � , 𝒙𝒙 = � ⋮ � , 𝒃𝒃 = � 2 � ,
⋮ ⋮ ⋮ � ⋮
𝑎𝑎𝑚𝑚1 𝑎𝑎𝑚𝑚2 … 𝑎𝑎𝑚𝑚𝑚𝑚 𝑥𝑥𝑛𝑛 𝑏𝑏𝑚𝑚
then we can represent the linear system as Ax = b.
We also have a simple way to represent solution of a linear system. Recall this is how we write a
solution for the system:
𝑥𝑥1 = 𝑢𝑢1 , 𝑥𝑥2 = 𝑢𝑢2 , … , 𝑥𝑥𝑛𝑛 = 𝑢𝑢𝑛𝑛 .

22
 𝑢𝑢1
𝑢𝑢2
In matrix equation form, the solution is represented by an n x 1 column matrix 𝒖𝒖 = � ⋮ �.
𝑢𝑢𝑛𝑛
Substituting each individual number u1 to un for each variable, can now be replaced collectively by
substituting the column matrix u for the variable matrix x.
Algebraically, we say:
u is a solution of the linear system Ax = b, if we have the equality Au = b.

Solving Matrix Equations (slide 5)

Let Ax = b be (the matrix equation of) a linear system. Can we solve the linear system in terms of A
and b?
Let’s recall how we solve an ordinary linear equation ax = b with a and b being numbers instead of
matrices. If a ≠ 0, then to solve the equation ax = b, we have x = b/a.
For our matrix equation Ax = b, we cannot write: x = b/A, as we do not have “division operation” for
matrices.
In the number version, we can rewrite the solution x = b/a = a-1 b where a-1 is called the inverse or
reciprocal of the number a.
Can we find an inverse A-1 for matrix A (which has the similar property as a-1)?
The answer is yes, provided A is a square matrix, and satisfies certain condition.

Inverse of a Square Matrix (slide 6)

Let A be an n x n square matrix. The n x n matrix A-1 is the inverse of A if the following conditions
hold:
AA-1 = I and A-1A = I.

Not every square matrix has an inverse. If an inverse of A exists, we say A is non-singular or
invertible. Otherwise, we say A is singular or non-invertible.

Inverses of 2 x 2 Matrices (slide 7)

If A is a 2 x 2 matrix, there is a simple way to determine whether it is non-singular and to find
its inverse.
A =  a b 
c d 
If ad - bc ≠ 0, then the matrix is non-singular, and the inverse of the matrix is given by this
expression:
 d −b

A −1 =  ad−−cbc ad a− bc 
 ad − bc ad − bc 
If ad - bc = 0, then A is singular. So the inverse of the matrix does not exist.

23
Example: 2 x 2 Non-Singular Matrix (slide 8)
 2 − 5
A =  
− 1 3 
We compute 2 x 3 – (-5) x (-1) ≠ 0. Therefore the matrix is non-singular.
Using the formula given above, we get the inverse of A:
3 5
A−1 =  
1 2 
You may verify that this matrix is indeed the inverse of A by checking AA-1 = I and A-1A = I.

Example: Solving a Matrix Equation (slide 9)

Using the matrix A above, let us set up a linear system Ax = b:
 2 − 5  x   4 
    =  
 − 1 3   y  0
This is the matrix equation form of a linear system with 2 equations and 2 variables.
Now pre-multiplying the inverse matrix A-1 = 35
 on both sides of the matrix equation:
1 2 

Hence the solution of the linear system: x = 12, y = 4.

3 5 4 12
We have seen this solution is obtained from A-1b = � � � � = � �.
1 2 0 4

24
1.6 Matrix inverses II
In this section, we discuss matrix inverse for general n x n square matrices. We will also see how the
singularity of a matrix links to other concepts we have introduced so far.

Example: Finding Inverse of 3 x 3 Matrix (slide 2-3)

First we form the 3 x 6 “augmented” matrix using the 3 x 3 identity matrix:

We shall now perform Gauss-Jordan elimination on this matrix:

Within the same column, the third type of
elementary row operations on different rows
can be performed simultaneously.

Though the matrix is already in row

echelon form, we need to continue with
the G.J.E. steps.

The matrix is now in reduced row

echelon form.

In the final matrix, the left half is the identity matrix, and the right half gives the inverse of the
original matrix A as shown below.

If we now just focus on the right half of all the augmented matrices in the G.J.E. steps, we observe is
that the series of elementary row operations that we performed above will transform the identity
matrix to the inverse of our given matrix. (See the next segment)

Finding Inverse of n x n Matrix (slide 4)

If a sequence of elementary row operations reduce A to I, the same sequence will also reduce I to
A–1.

This gives us a way to use Gauss-Jordan Elimination to find the inverse of any non-singular matrix.

25
If the square matrix is singular, i.e. does not have an inverse, we will see in the next segment that
there is no sequence of row operations that can reduce the matrix to the identity matrix.

Singularity and Row Echelon Form (slide 5)

Let A be any square matrix. We can check whether A is non-singular by looking at its row echelon
form:

(i) If the reduced row-echelon form of A is an identity matrix, then A is non-singular.

(ii) If any row-echelon form of A has a zero row, then A is singular.

Here’s an example.
1 0 1
 
1 0 0
 
After performing Gauss Jordan elimination, we obtain the identity
G. J.E .
 0 1 1   → 0 1 0  matrix. So we know this matrix is non-singular.
1 1 0 0 0 1
   

Let’s look at another example.

1 2 1 
 
1 2 1 
 
It is enough to just perform Gaussian elimination (instead of G.J.E.) to
G.E .
 0 1 1   → 0 1 1  get a row echelon form. As long as we get a zero row, we conclude that
1 1 0 0 0 0
    the matrix is singular. Note that when there is zero row in the matrix, it
is not possible to reduce it further to get the identity matrix.

Matrix Operations on Non-Singular Matrices (slide 6)

Let A, B be two non-singular matrices (same size) and 𝑎𝑎 is a non-zero scalar.

The table below includes four types of matrix operations: scalar multiplication, transpose, inverse
and matrix multiplication (of A and B).

This table answers two questions:

The first question (second column) is, if the original matrices A and B are non-singular, will the
resulting matrices be still non-singular after the matrix operations?
The answer is “yes” for each of the operations. All the resulting matrices have their inverses.
The second question (third column) is, whether we can express the inverses of the resulting matrices
in terms of the original ones?
The answer is again “yes”. The respective expression for each inverse is given in red in the third
column. Note that for matrix multiplication, the inverse of the product is equal to the product of the
inverses with the order reversed.

26
Example: Matrix Operations (slide 7)

with inverses

(i) (AT )–1 = (A–1)T

To find the inverse of AT, we can either do it directly using the formula,
or just use the fact that this is the same as the transpose of A-1.
(ii) (AB)–1 = B –1A–1

To find the inverse of AB, instead of working out its

product and take the inverse, we can use the product
of the inverses of B and A.

Multiplications of Non-Singular Matrices (slide 8)

Given A1, A2, …, Ak are all non-singular matrices of the same size.

The product A1A2…Ak is a non-singular matrix and the inverse of A1A2…An is

Note that the order of the matrices is reversed.

If A is non-singular, then we can define its negative power:

Singularity and Solutions of Linear System (slide 9)

If A is non-singular,

i. The linear system Ax = b has exactly one solution given by x = A-1b.

ii. The homogeneous system Ax = 0 has only the trivial solution x = 0.

If A is singular,

i. The linear system Ax = b has either no solution or infinitely many solutions.

ii. The homogeneous system Ax = 0 has both the trivial solution and non-trivial solutions.

27
Example: Solutions of Linear System (slide 10)

We use two earlier examples:

1 0 1 1 0 0 1 2 1  1 2 1 
       
A =  0 1 1  
G. J.E .
→ 0 1 0  B =  0 1 1  
G.E .
→ 0 1 1 
1 1 0 0 0 1 1 1 0 0 0 0
       
Recall that A is non-singular and B is singular.

Using A to form the homogeneous system Ax = 0, we can tell it has only

the trivial solution.

If A is the coefficient matrix of a non-homomgeneous system, we can tell

that it has exactly one solution.

1 2 1
 
0
 
Using B to form the homogeneous system Bx = 0, we can tell it has
0 1 1 X = 0 infinitely many solutions.
1 1 0 0
   

1 2 1
 
 5 
 
If B is the coefficient matrix of a non-homomgeneous system, we cannot
 0 1 1  X =  − 1tell the number of solutions: it can have either no solution or infinitely
1 1 0  3 
   many solutions (i.e. it cannot have only one solution).
In this case, we need to apply Gaussian elimination to the augmented matrix and look at the
row echelon form.

28
1.7 Determinants of Matrices
Determinant is another useful concept associated with a square matrix. We shall see in this section
how to compute determinant, some of its basic properties, and how it is related to the singularity of
a matrix.

Determinant of 2 x 2 Matrices (slide 2)

Let be a 2 x 2 matrix.

Then the determinant of A , denoted by det(A), is the number defined as det(A) = ad – bc.

An alternative notation for det(A) is: a b

c d

1 2 1 2
So if 𝑨𝑨 = � �, then det(A) = � � = 1 × 4 − 2 × 3 = −2.
3 4 3 4
From earlier section, we have seen that this number is related to the singularity of the matrix:

If det(A) ≠ 0, then A is non-singular.

If det(A) = 0, then A is singular.
In general, how do we find the determinant of an n x n square matrix? There are in fact various ways
to compute determinants. A systematic but tedious way to do it is by induction (expansion).

Determinant of 3 x 3 Matrices (slide 3)

 − 3 − 2 4
 
B= 4 3 1
 0 2 4 


We shall compute the 3 x 3 determinant of B in terms of 2 x 2 determinants:

3 1 4 1 4 3
det(B) = (−3) − (−2) +4
2 4 0 4 0 2

The 3 coefficients (-3, -2, 4) attached to the 2 x 2 determinants come from the first row of B.
The 2x2 determinants are all parts of the matrix B (we call them the submatrices of B):
3 1
The first one is obtained from B by removing its first row and first column.
2 4
4 1
The second one is obtained from B by removing its first row and second column.
0 4
4 3
The third one is obtained from B by removing its first row and third column.
0 2
Note that, on the right hand side, we are taking the alternating sum. In other words, we have the
first term minus the second term plus the third term. i.e. the terms alternate between + and – sign,
starting with a + sign.
We expand the 2 x 2 determinants further to obtain the final answer:

det(B) = −3(3 × 4 − 1 × 2) + 2(4 × 4 − 1 × 0) + 4(4 × 2 − 3 × 0) = 34.

29
Determinant of n x n Matrices (slide 4)

In general, an n x n determinant is computed in terms of (n-1) x (n-1) determinants. We shall

illustrate using a 4 x 4 matrix:
 0 −1 2 0
 
2 −3 3 −2 
C=
0 2 4 0
 
0 0 2 −1 

The determinant is given as the expansion of four 3 x 3 determinants:

−3 3 −2 2 3 −2 2 −3 −2 2 −3 3
det(C) = 0 2 4 0 − (−1) 0 4 0 + 2 0 2 0 − 0 0 2 4
0 2 −1 0 2 −1 0 0 −1 0 0 2

Like before, the coefficients indicated in red come from the first row of C. The 3 x 3 submatrices are
obtained by deleting the first row, and one of the 4 columns of C. Finally, we apply the alternating
sum to the 4 terms.

The four 3 x 3 determinants on the right hand side can be further expanded in terms of 2 x 2
determinants in a similar way. I will leave it to the readers to verify that the final answer is

det(C) = -16.

This inductive method is quite tedious when the size of the matrix is large. We shall see later how to
make use of some properties of determinant to simplify the computation.

Determinant and Singularity (slide 5)

The relation between determinant and singularity holds for any n x n square matrix A:

o if det(A) ≠ 0, then A is non-singular;

o if det(A) = 0, then A is singular.

We have seen before that det(B) = 34 ≠ 0, so B is non-singular.

det(C) = 0 (check!), so C is singular.

Determinants of Triangular Matrices (slide 6)

If A is a triangular matrix or diagonal matrix,

then the determinant of A is equal to the product of the diagonal entries of A.

Look at the three matrices below. They are upper triangular, lower triangular and diagonal matrices
respectively.

So we know that the first and third matrices are non-singular as their determinants are not 0, while
the second matrix is singular.

30
Determinants with Two Identical Rows (or Columns) (slide 7)

1. The determinant of a square matrix with two identical rows is zero.

2. The determinant of a square matrix with two identical columns is zero.

All the three matrices above have either two identical rows or two identical columns. So their
determinants are known to be 0 (without having to expand in terms of smaller determinants) and
they are all singular matrices.

Determinants and Elementary Row Operations (slide 8)

The table below summarises the effect of the three types of elementary row operations on
determinants.

An example for the first type of row operation:

Observe that B is obtained from A by multiplying the second row by 3,

i.e. 3R2.

An example for the second type of row operation:

Observe that B is obtained from A by interchanging first and third row of

A, i.e. R1 ↔ R3.

The third type of row operation does not change the determinant. We shall see an example in the
next segment.

Find Determinants by Gaussian Elimination (slide 9)

Instead of computing the 4 x 4 determinant below inductively, we are going to perform elementary
row operations to reduce the matrix to an upper triangular matrix.

The first and third row operations above are of the third type, and hence do not affect the
determinant. The second row operations are of the second type, and the determinants will differ by
a factor of -1.

31
We observe that the last determinant on the right is in row echelon form (an upper triangular
matrix). So the determinant can be obtained easily from the diagonal entries 3, 2, 1 and -1. Hence
the original determinant is given by (3 x 2 x 1 x -1) = 6.

Determinant and Matrix Operations (slide 10)

The last thing we want to discuss is the effect of some matrix operations on the determinants.

Let A and B be n x n square matrices, and c be a scalar.

(The power n in statement 1 is the size of A)

(The two sides in statement 5 are not equal in general.

For some special cases, such as one of the matrices is
zero, equality can still hold)

Example: Matrix Operations (slide 11)

Let’s demonstrate the above properties using our two previous matrices B and C:

32
1.8 Ranks of Matrices
In this short section, we introduce another commonly used concept: the rank of a matrix. We will
establish the link between rank of a matrix and some other concepts that we have discussed so far.
Rank of a Matrix (slide 2)
Let A be an m x n matrix and R be a row echelon form of A.

The rank of A is the number of nonzero rows of R .

Notation: rank(A)
It follows that
rank(A) = the number of leading entries in R = the number of pivot columns in R
Example: Rank of a Matrix (slide 3)
The rank of the following 6 x 4 matrix A is 3, i.e. rank(A) = 3, by counting the number of nonzero rows
or number of leading entries or number of pivot columns of the row echelon form on the right.

Rank of Zero and Identity Matrices (slide 4)

0 ⋯ 0
Zero matrix: 0 = � ⋮ ⋮�
0 ⋯ 0
rank(0) = 0, as there is no non-zero row.
In fact, zero matrices are the only matrices that have rank equal to 0.
1 ⋯ 0
Identity matrix: 𝑰𝑰𝑛𝑛 = � ⋮ ⋱ ⋮�
0 ⋯ 1
rank(In) = n, as every row is a non-zero row.
Note that both matrices are already in row echelon form. So there is no need to perform Gaussian
Elimination.
Largest Possible Rank of a Matrix (slide 5)
What is the largest possible rank of a 5 x 3 matrix A?
Since A has 3 columns, and the rank is given by the number of pivot columns in the row echelon form
of A, so the largest possible rank of A is 3.
What is the largest possible rank of a 3 x 5 matrix B?
Since B has 3 rows, and the rank is given by the number of nonzero rows in the row echelon form of
B, so the largest possible rank of B is 3.

33
Full Rank (slide 6)
From the above discussion, we notice that the possible rank of a matrix is bounded by the smaller of
the number of rows and the number of columns of the matrix. So for an m x n matrix A,

For example, if A is a 4 x 6 general matrix, the largest possible rank it can have is 4, and the possible
rank of A is 0, 1, 2, 3 and 4. Note that the rank can only be 0 if A is the 4 x 6 zero matrix.
When a matrix A achieves its largest possible rank, i.e. rank(A) = min(m , n), we say that A is a full
rank matrix.
For example, if a 4 x 6 matrix A has a rank of 4, then A is a full rank matrix.
The n x n identity matrix In has rank equal to n. So In is a full rank matrix.
1 0 0
On the other hand, the 3 x 3 matrix �0 0 0� has rank 1. So it is not full rank.
0 0 0
Rank and Determinant (slide 7)
The following discussion links the concept of rank with some of the results we have mentioned in
earlier sections.

An n x n square matrix A is full rank means rank(A) = n. This means the reduced row echelon form of
A has n pivot columns. So the reduced row echelon form of A must be the identity matrix In. This in
turn means A is non-singular and det(A) ≠ 0.

We say, any one of these conditions on the matrix A is equivalent to the other three.
 − 3 − 2 4
 
Here is a concrete 3 x 3 matrix:  4 3 1
 0 2 4 

The following diagram connects the various conditions on the matrix:

34