Basic Linear Algebra

Prof. Inder K. Rana

Department of Mathematics
Indian Institute of Technology - Bombay

Lecture – 06
Systems of Linear Equations III

We prove that every matrix has a REF. So let us say that or we are going to do on induction,
number of rows, 1 × 𝑚. There is only 1 row in the matrix. Can you say it is in the row echelon
form? No basically because if it is non-0, right, some 0's may be coming. First time there will be
some non-0 if it is a non-0 matrix. That is fine. There is only 1 row, so nothing to worry, right.

So the proof that a row matrix, a row vector is in row echelon form if it is non-0. This is okay? By
default, it is non-0. So it has to be in row echelon form. So induction starts, 𝑚 = 1, the proof is,
let us assume when the number of rows is something, it is okay. When number of rows become 1
more, how to do it, right? So that is a question. So let us look at that.
(Refer Slide Time: 01:31)

So let us say these are the rows of 𝑅1 , 𝑅2 , … , 𝑅𝑚 . m is non-0 matrix. So what one does is?
(Refer Slide Time: 01:41)
Look at the columns of 𝑀. We are given a matrix, right. Look at the first column. Possibly
everything is 0 in that column, fine. So what will the machine do? It will scan the first column. No
non-0 entry, go to the second column. Non-0 entry, so look at a column where the first time it spots
a non-0 entry in some place. We do not know where. So let 𝑝1 be the first column that has a non-
0 entry.

First column 0, second column 0. The first column where non-0 entry somewhere comes is the
column number 𝑝1. So non-0 entry has come at some place, that we should make it as the first one,
that should make as the row. So that row should be made as the first row. Because everything else
was 0 on the left side of it. Is it clear what we are doing? Look at the column 1. Column 1 is all 0.

Scan column 2. All 0. At the 𝑝1 column, the first time a column has a non-0 entry is 𝑝1. Now it
will come at some place. Say it is at the 𝑖th place. Everything else on the left side is 0, right.
Everything at the top is also 0. So what we do is, we take this row and make it as the first row. So
what will happen? Everything else is 0. The first non-0 entry is coming at the place 𝑝1, right.

So first stage is achieved. The first non-0 row as a non-0 entry at a place 𝑝1. Everything on the left
is 0. At the bottom, we do not know what it is, right. But if there is a non-0 entry coming somewhere
at the bottom, we can make that also 0 by row operations. So everything below that can be made
as 0, right. So what we have gotten? 𝑝1 non-0, everything below that is 0, everything on the left is
0.

Now again now starts scanning the remaining part of the matrix. The remaining part of the matrix,
what is the order of the remaining part? Already 1 row is gone. So 1 row less. So by induction, I
should have row echelon form for that. So proof is over, right. So it is done by scanning the
columns. Scan a column, non-0 entry, make it as the first row, right. Now go to the next columns.

Spot the non-0 entry. Make that as the second row. And everything below that keep on making it
0. So that will make, right, that is definite way of telling a machine that proceed this way, write
systematically and in the end, everything is over. Everything you scanned, all the columns are
scanned, your matrix will be in row echelon form.

So that is what is written here. So it says look at the columns of m. Let 𝑝1 be the first column that
has a non-0 entry. Say ith place. First column non-0 entry. So this is what the ith row will look
like. 0 0 0 first non-0 entry coming here. Now that should be made as a first one, right. So change
it.
(Refer Slide Time: 05:08)

So change it to the first one. Something is there at the bottom, we do not know. But on the left,
everything is 0. These may be non-0 entries here at the bottom. But what we can do is? By row
operations, I can make these entries as 0, right by a suitable multiplications. This is non-0, divide
by, right, multiply by the 1/of that number, that will become 1, negative of that, add it here, I can
make this as 0, make this as 0, this as 0, right.

So once a non-0 entry has come at a place, everything at the bottom can be made as, in that column,
can be made as 0 by elementary row operations. So I will have 0's here, I will have a non-0 entry
here and this will be all 0's. So what is left? Left is this part of my matrix and that has order less
than m, right. Number of rows has reduced, okay. So that will mean what? A number of, this has
reduced, that means by induction, I can have that as in the row echelon form.

I can go on applying this. I look at this column now and say non-0 entry will possibly will come
here and change it in this part. In this part, so I will be looking only this matrix now. So look at
this column here. Non-0 entry come again, I will shift it up and so on, okay. That is the induction
part, okay. So that is how.
(Refer Slide Time: 06:33)

So the second step was, this is non-0, everything else is made 0, right, which I said. Now look at
this part and do the same thing. Now repeat the process on this part. So in your computer language,
what you will say, do loop, right. This is a loop thing. On this matrix, do nothing. Because
everything is reduced, then that will be the remaining part and so on.
(Refer Slide Time: 06:56)
So this is a definite way of doing things which can be fed to a machine also, right, okay. So how
do we use this row echelon form to solve system of linear equations?
(Refer Slide Time: 07:14)

So we have a given system of linear equations 𝐴𝑋 = 𝑏, right. Then we will make the augmented
matrix because 𝑏 is going to important. So this is a new matrix which has got 1 more column added
to it and we will reduce it to row echelon form, right.
(Refer Slide Time: 07:39)
So I think let us give the video a break and try to solve this problem now. Then we will see the
solution. So look at the system where the augmented matrix is given, sorry, not, actually it is in
the row echelon form. So this example is already in that form, right. Is that okay? You are doing
so that this matrix comes in the, 𝐴 is the matrix. Let us discuss this example now. So this matrix
[𝐴|𝑏], 𝐴 is already in the echelon form. Now we use backward substitution to get the solutions.
When a variable is not determined, that is, it does not have a fixed value given by the system, it is
a free variable and it can attain any arbitrary value.

(Refer Slide Time: 14:08)

I am just repeating that solution again, right, last equation.
(Refer Slide Time: 14:11)

So that is what the solution.

(Refer Slide Time: 15:13)

So let us do one more illustration so that you understand. So this is a system given, okay. Once
again I already purposefully put in the row echelon form so that we can analyze what is the
backward substitution. Otherwise, you write 𝐴 and 𝑏 and reduce it to row echelon form, you will
get something like this, right. Here it is already done. That [𝐴̃ | 𝑏̃]. So last row does not give you
anything, is 0, forget about it, no information.
So in this system, there is only 1 solution possible, right. In this system, there is only 1 solution
possible, which we obtain by back substitution.
Here column 1 has 𝑝1, 2 has 𝑝2 . If the 𝑝1 , 𝑝2 , … , 𝑝𝑟 are in 𝑛 columns, 𝑟 = 𝑛 and each column has
to get a pivot. That means there will be a unique solution. If each column has a pivot, it is going
to be unique solution, right, that is what is happening in this. So these observations we are putting,
we will put them together.
(Refer Slide Time: 17:36)

So 𝑟 = 𝑛, the number of variables, right. So 𝑟 = 𝑛 means unique solution. If 𝑟 < 𝑛, then 𝑛 − 𝑟

variables get independent values, arbitrary values. Then infinite solutions come, right. If 𝑟 < 𝑛,
then some columns will not have pivots. That means those variables will be determined in terms
of the other one, okay.
(Refer Slide Time: 18:30)
Now let us look at this another one. That means when I reduce the matrix 𝐴, aim is keep 𝐴 and 𝑏
side by side. Aim is to make 𝐴 in the row echelon form. Go on changing 𝑏 accordingly. Do not
bother about it. So it gives you, last row is 0. But here comes a row in 𝐴 which is 0 but on the right
hand side, becomes 3. So what will this equation look like? 0=3, that means there is an absurd
equation.

One of the equations is never going to have a solution. That means the system cannot have a
solution.

So that gives you an inconsistent. So this is the way you will spot an inconsistency. So 3
possibilities arise. One the number of pivots, right, the 𝑟 is strictly less than 𝑛, the number of
variables. Then 𝑛 − 𝑟 variables will get arbitrary values and remaining 𝑟 will be determined in
terms of these 𝑛 − 𝑟.

Second, each column gets a pivot. 𝑛 = 𝑟. Then unique solution, right. And third, there is a row 0
= non-0, right. So in that case, no solution, inconsistent. So 3 possibilities. Again look at 2 variables
and 3 variables. 2 lines, either they are parallel, so no solution, they do not meet, right. Then 2
lines intersect.

They will intersect only at a unique point and then they are coincidental. So when the 2 lines are
coincidental, then in the row echelon form, 1 row will be 0; otherwise, relation between 2 variables.
So 1 variable will give you the other value. So infinite solutions, right. So perfect similarity comes,
okay.
(Refer Slide Time: 21:02)

(Refer Slide Time: 21:27)

So once again, let us, this row echelon form for solving a system of linear equations. Summarize
what we have done. So it says look at the system 𝐴𝑋 = 𝑏, a 𝑚 × 𝑛 system with augmented matrix
as 𝐴 along with 𝑏, call it as 𝑀, augmented matrix, we are calling it as 𝑀. So reduce the matrix to
the row echelon form and look at the non-0 rows, 𝑟 is a number of non-0 rows for this and pivots
are coming at 𝑝𝑖 th place, right.
(Refer Slide Time: 22:14)

Then what? The system is inconsistent if there is a pivot in that column, right, the b column. There
is a non-0. If the pivot comes there, that means all the previous must be 0, right. If in the augmented
matrix, there is a pivot lying in the column where b is coming that means there is a non-0 entry in
that, right. Everything else on the left side must be, what is the definition of a pivot? Everything
on the left is 0. So 0 = non-0 that is inconsistent, system is inconsistent. That is what we saw, right.
(Refer Slide Time: 22:50)

And if 𝑟 < 𝑛, then there are 𝑛 − 𝑟 parameters or variables, whatever you want to call them, which
will determine a general solution, right.
(Refer Slide Time: 23:00)

So what we have done is? For a system 𝐴𝑋 = 𝑏, right, we take 𝐴 and 𝑏 and reduce to row echelon
form, right. Now reducing a matrix to row echelon form is by scanning and interchanging or
multiplying, right. Now how does a computer knows, I cannot give a verbal command,
interchange. I have to give it in the form of something computational to the, you understand what
I am saying.

If you want to implement this command that interchange 2 rows. How does the computer
understand that command? It can understand only multiplication, division, addition, arithmetic,
right. So what we are going to do is, interchanging of 2 rows, that is one command. Other is
multiplying a matrix by, row by a non-0 scalar, second. And third is adding, right. Adding one row
to another.

These ones we want to make it executable commands so that a computer can understand, right.
(Refer Slide Time: 26:04)
(Refer Slide Time: 27:32)

(Refer Slide Time: 29:57)

For an elementary row operation to be done on A, premultiply it with the matrix obtained from
identity matrix by that operation. So what we Type equation here.are saying is to give a command
to a computer, how do I do that operation is take identity matrix, okay. Whatever operation you
want elementary row operation you want to be done on 𝐴, do it on identity first. We will get a
matrix. So store that matrix.

Once you have stored that matrix, multiply the matrix 𝐴 on the left side by that matrix. The
resulting matrix will be doing that operation on 𝐴. So if you want to interchange 2 rows, take
identity matrix, interchange its rows, you will get a matrix. Premultiply 𝐴 with it, the resulting
matrix will be the one where rows of 𝐴 are corresponding rows are interchanged.

If you want to add, do the same thing for identity matrix, add those corresponding rows, identity
matrix, you get a matrix, premultiply 𝐴 with this matrix, the corresponding rows will be added.
And similarly, right, if you want to multiply and add or interchange, same thing you do here and
apply it here. So for all 3 elementary row operations, what are the 3 elementary row operations?

One was interchange of rows. The other was multiplying row by a scalar. And the third one was
multiplying and adding. These 3 operations you can do on identity matrix and store them and
whenever you want to do these operations on 𝐴, premultiply 𝐴 with that matrix. So computer will
store in the computer those things and do that multiplications on the computer to do those
operations, right.

So these matrices are called elementary matrices. What are elementary matrices? These are the
matrices obtained from the corresponding identity matrix by doing elementary row operations on
them of 3 types, right. So there are 3 types of elementary matrices which are used to do elementary
row operations on any given matrix, right. So we will continue study of this elementary matrices
next time.