1-Matrices and Determinants
1-Matrices and Determinants
Chemical Engineering
DCHET, COET, MSU-IIT
7. Matrices,
Vectors, Determinants.
Linear Systems
▪ Linear algebra is a fairly extensive subject that covers vectors and
matrices, determinants, systems of linear equations, vector spaces
and linear transformations, eigenvalue problems, and other topics.
As an area of study, it has a broad appeal in that it has many
applications in engineering, physics, geometry, computer science,
economics, and other areas. It also contributes to a deeper
understanding of mathematics itself.
▪ Matrices, which are rectangular arrays of numbers or functions,
and vectors are the main tools of linear algebra. Matrices are
important because they let us express large amounts of data and
functions in an organized and concise form. Furthermore, since
matrices are single objects, we denote them by single letters and
calculate with them directly. All these features have made matrices
and vectors very popular for expressing scientific and
mathematical ideas.
The chapter keeps a good mix between applications and theory.
Chapter 7 is structured as follows:
▪ Sections 7.1 and 7.2 provide an intuitive introduction to matrices
and vectors and their operations, including matrix
multiplication.
▪ The next block of sections, that is, Secs. 7.3–7.5 provide the most
important method for solving systems of linear equations by
the Gauss elimination method. This method is a cornerstone of
linear algebra, and the method itself and variants of it appear in
different areas of mathematics and in many applications. It leads
to a consideration of the behavior of solutions and concepts such
as rank of a matrix, linear independence, and bases.
▪ We shift to determinants, a topic that has declined in
importance, in Secs. 7.6 and 7.7. Section 7.8 covers inverses of
matrices. The chapter ends with vector spaces, inner product
spaces, linear transformations, and composition of linear
transformations. Eigenvalue problems follow in Chap. 8.
7.1 Matrices, Vectors:
Addition and Scalar
Multiplication
ADDITION AND SCALAR MULTIPLICATION
Let us first take a leisurely look at matrices before we formalize our
discussion. A matrix is a rectangular array of numbers or functions
which we will enclose in brackets. For example,
(1)
are matrices. The numbers (or functions) are called entries or, less
commonly, elements of the matrix. The first matrix in (1) has two rows,
which are the horizontal lines of entries. Furthermore, it has three
columns, which are the vertical lines of entries. The second and third
matrices are square matrices, which means that each has as many
rows as columns— 3 and 2, respectively.
ADDITION AND SCALAR MULTIPLICATION
(1)
The entries of the second matrix have two indices, signifying their
location within the matrix. The first index is the number of the row
and the second is the number of the column, so that together the
entry’s position is uniquely identified. For example, 𝑎23 (read a two
three) is in Row 2 and Column 3, etc. The notation is standard and
applies to all matrices, including those that are not square.
ADDITION AND SCALAR MULTIPLICATION
Matrices having just a single row or column are called vectors. Thus,
the fourth matrix in (1) has just one row and is called a row vector.
The last matrix in (1) has just one column and is called a column
vector. Because the goal of the indexing of entries was to uniquely
identify the position of an element within a matrix, one index suffices
for vectors, whether they are row or column vectors. Thus, the third
entry of the row vector in (1) is denoted by 𝑎3 .
ADDITION AND SCALAR MULTIPLICATION
Example 1: Linear Systems, a Major Application of Matrices
We are given a system of linear equations, briefly a linear system,
such as
(2)
VECTORS
A vector is a matrix with only one row or column. Its entries are
called the components of the vector. We shall denote vectors by
lowercase boldface letters 𝐚, 𝐛, or by its general component in
brackets, 𝐚 = [𝑎𝑗 ] , and so on. Our special vectors in (1) suggest that a
(general) row vector is of the form
DEFINITION:
ADDITION AND SCALAR MULTIPLICATION OF MATRICES AND VECTORS
Example 2: Equality of Matrices
Let
Then
DEFINITION:
(3)
Here 𝟎 denotes the zero matrix (of size 𝑚 × 𝑛), that is, the 𝑚 × 𝑛
matrix with all entries zero. If 𝑚 = 1 or 𝑛 = 1, this is a vector, called a
zero vector.
ADDITION AND SCALAR MULTIPLICATION OF MATRICES AND VECTORS
Hence matrix addition is commutative and associative [by (3a) and
(3b)]. Similarly, for scalar multiplication we obtain the rules
(4)
7.2 Matrix
Multiplication
MATRIX MULTIPLICATION
Matrix multiplication means that one multiplies matrices by matrices.
Its definition is standard but it looks artificial. Thus you have to study
matrix multiplication carefully, multiply a few matrices together for
practice until you can understand how to do it. Here then is the
definition.
DEFINITION:
MATRIX MULTIPLICATION
The condition 𝑟 = 𝑛 means that the second factor, 𝐁, must have as
many rows as the first factor has columns, namely 𝑛. A diagram of
sizes that shows when matrix multiplication is possible is as follows:
The entry 𝑐𝑗𝑘 in (1) is obtained by multiplying each entry in the 𝑗th
row of 𝐀 by the corresponding entry in the 𝑘th column of 𝐁 and then
adding these 𝑛 products. For instance, 𝑐21 = 𝑎21 𝑏11 + 𝑎22 𝑏21 + ⋯ +
𝑎2𝑛 𝑏𝑛1 and so on. One calls this briefly a multiplication of rows into
columns. For 𝑛 = 3, this is illustrated by
MATRIX MULTIPLICATION
Let us illustrate the main points of matrix multiplication by some
examples. Note that matrix multiplication also includes multiplying a
matrix by a vector, since, after all, a vector is a special matrix.
Here
whereas
is undefined.
MATRIX MULTIPLICATION
Example 3: Products of Row and Column Vectors
(2)
provided 𝐀, 𝐁, and 𝐂 are such that the expressions on the left are
defined; here, 𝑘 is any scalar. (2b) is called the associative law. (2c)
and (2d) are called the distributive laws.
MATRIX MULTIPLICATION
Since matrix multiplication is a multiplication of rows into columns,
we can write the defining formula (1) more compactly as
(3)
Where 𝐚𝑗 is the 𝑗th row vector of 𝐀 and 𝐛𝑘 is the 𝑘th column vector of
𝐁, so that in agreement with (1),
MATRIX MULTIPLICATION
Example 5: Product in Terms of Row and Column Vectors
(4)
MATRIX MULTIPLICATION
Example 6: Computing Products Columnwise by (5)
To obtain
(6*)
and suffice to explain the idea. (For general 𝑛 they will be discussed
in Sec. 7.9.) For instance, (6*) may relate an 𝑥1 𝑥2 -coordinate system to
a 𝑦1 𝑦2 -coordinate system in the plane. In vectorial form we can write
(6*) as
(6)
(7)
MOTIVATION OF MULTIPLICATION BY LINEAR TRANSFORMATIONS
Then the 𝑦1 𝑦2 -system is related to the 𝑤1 𝑤2 -system indirectly via the
𝑥1 𝑥2 -system, and we wish to express this relation directly. Substitution
will show that this direct relation is a linear transformation, too, say,
(8)
This proves that 𝐂 = 𝐀𝐁 with the product defined as in (1). For larger
matrix sizes the idea and result are exactly the same. Only the
number of variables changes. We then have 𝑚 variables 𝑦 and 𝑛
variables 𝑥 and 𝑝 variables 𝑤. The matrices 𝐀, 𝐁, and 𝐂 = 𝐀𝐁 then
have sizes 𝑚 × 𝑛, 𝑛 × 𝑝 and 𝑚 × 𝑝, respectively. And the requirement
that 𝐂 be the product 𝐀𝐁 leads to formula (1) in its general form. This
motivates matrix multiplication.
TRANSPOSITION
We obtain the transpose of a matrix by writing its rows as columns (or
equivalently its columns as rows). This also applies to the transpose of
vectors. Thus, a row vector becomes a column vector and vice versa.
In addition, for square matrices, we can also “reflect” the elements
along the main diagonal, that is, interchange entries that are
symmetrically positioned with respect to the main diagonal to obtain
the transpose. Hence 𝑎12 becomes 𝑎21 , 𝑎31 becomes 𝑎13 and so forth.
Example 7 illustrates these ideas. Also note that, if 𝐀 is the given
matrix, then we denote its transpose by 𝐀T .
(12)
(13)
SPECIAL MATRICES
Example 10: Diagonal Matrix 𝐃. Scalar Matrix 𝐒. Unit Matrix 𝐈
7.3 Linear Systems
of Equations.
Gauss Elimination
LINEAR SYSTEM, COEFFICIENT MATRIX, AUGMENTED MATRIX
A linear system of 𝒎 equations in 𝒏 unknowns 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑛 is a set
of equations of the form
(1)
(2)
are column vectors. We assume that the coefficients 𝑎𝑗𝑘 are not all
zero, so that 𝐀 is not a zero matrix. Note that 𝐱 has 𝑛 components,
whereas 𝐛 has 𝑚 components.
LINEAR SYSTEM, COEFFICIENT MATRIX, AUGMENTED MATRIX
The matrix
Step 1. Elimination of 𝒙𝟏
Call the first row of 𝐀 the pivot row and the first equation the pivot
equation. Call the coefficient 1 of its 𝑥1 -term the pivot in this step.
Use this equation to eliminate 𝑥1 (get rid of 𝑥1 ) in the other equations.
For this, do:
Add 1 times the pivot equation to the second equation.
Add −20 times the pivot equation to the fourth equation.
GAUSS ELIMINATION AND BACK SUBSTITUTION
This corresponds to row operations on the augmented matrix as
indicated in BLUE behind the new matrix in (3). So the operations are
performed on the preceding matrix. The result is
(3)
Step 2. Elimination of 𝒙𝟐
The first equation remains as it is. We want the new second equation
to serve as the next pivot equation. But since it has no 𝑥2 -term (in fact,
it is 0 = 0, we must first change the order of the equations and the
corresponding rows of the new matrix. We put 0 = 0 at the end and
move the third equation and the fourth equation one place up. This is
called partial pivoting (as opposed to the rarely used total pivoting,
in which the order of the unknowns is also changed). It gives
GAUSS ELIMINATION AND BACK SUBSTITUTION
To eliminate 𝑥2 , do:
Add −3 times the pivot equation to the third equation.
The result is
(4)
GAUSS ELIMINATION AND BACK SUBSTITUTION
Back Substitution. Determination of 𝒙𝟑 , 𝒙𝟐 , 𝒙𝟏 (in this order)
Working backward from the last to the first equation of this
“triangular” system (4), we can now readily find 𝑥3 , then 𝑥2 , and then
𝑥1 :
where A stands for “amperes.” This is the answer to our problem. The
solution is unique.
ELEMENTARY ROW OPERATIONS. ROW-EQUIVALENT SYSTEMS
Elementary Row Operations for Matrices:
Interchange of two rows
Addition of a constant multiple of one row to another row
Multiplication of a row by a nonzero constant c
CAUTION! These operations are for rows, not for columns! They
correspond to the following
Theorem 1:
Because of this theorem, systems having the same solution sets are
often called equivalent systems. But note well that we are dealing with
row operations. No column operations on the augmented matrix are
permitted in this context because they would generally alter the
solution set.
GAUSS ELIMINATION: THE THREE POSSIBLE CASES OF SYSTEMS
Example 3: Gauss Elimination if Infinitely Many Solutions Exist
Solve the following linear system of three equations in four unknowns
whose augmented matrix is
(5)
Thus
GAUSS ELIMINATION: THE THREE POSSIBLE CASES OF SYSTEMS
Solution.
As in the previous example, we circle pivots and box terms of
equations and corresponding entries to be eliminated. We indicate
the operations in terms of equations and operate on both equations
and matrices.
Step 1. Elimination of 𝒙𝟏 from the second and third equations by
adding
This gives the following, in which the pivot of the next step is circled.
(6)
GAUSS ELIMINATION: THE THREE POSSIBLE CASES OF SYSTEMS
Step 2. Elimination of 𝒙𝟐 from the third equation of (6) by adding
times
This gives
(7)
Back Substitution.
From the second equation, 𝑥2 = 1 − 𝑥3 + 4𝑥4 . From this and the first
equation, 𝑥1 = 2 − 𝑥4 . Since 𝑥3 and 𝑥4 remain arbitrary, we have
infinitely many solutions. If we choose a value of 𝑥3 and a value of 𝑥4 ,
then the corresponding values of 𝑥1 and 𝑥2 are uniquely determined.
GAUSS ELIMINATION: THE THREE POSSIBLE CASES OF SYSTEMS
Example 4: Gauss Elimination if no Solution Exists
What will happen if we apply the Gauss elimination to a linear system
that has no solution? The answer is that in this case the method will
show this fact by producing a contradiction. For instance, consider
(8)
(9)
Here, 𝑟 ≦ 𝑚, 𝑟11 ≠ 0, and all entries in the blue triangle and blue
rectangle are zero.
7.7 Determinants.
Cramer’s Rule
DETERMINANTS. CRAMER’S RULE
A determinant of order 𝑛 is a scalar associated with an 𝑛 × 𝑛 (hence
square!) matrix 𝐀 = 𝑎𝑗𝑘 and is denoted by
(1)
(2)
DETERMINANTS. CRAMER’S RULE
A determinant of order 𝑛 is a scalar associated with an 𝑛 × 𝑛 (hence
square!) matrix 𝐀 = 𝑎𝑗𝑘 and is denoted by
(1)
(2)
DETERMINANTS. CRAMER’S RULE
For 𝑛 ≧ 2 by
(3a)
Or
(3b)
Here,
Note the following. The signs on the right are + − + . Each of the
three terms on the right is an entry in the first column of 𝐷 times its
minor, that is, the second-order determinant obtained from 𝐷 by
deleting the row and column of that entry; thus, for 𝑎11 delete the first
row and first column, and so on.
DETERMINANTS. CRAMER’S RULE
Example 1: Expansions of a Third-Order Determinant
This is the expansion by the first row. The expansion by the third
column is
Verify that the other four expansions also give the value −12.
DETERMINANTS. CRAMER’S RULE
Checkerboard pattern
then
CRAMER’S RULE
Cramer’s Rule for Linear Systems of Three Equations
is
and
7.8 Inverse of a
Matrix.
Gauss–Jordan
Elimination
INVERSE OF A MATRIX. GAUSS–JORDAN ELIMINATION
In this section we consider square matrices exclusively.
The inverse of an 𝑛 × 𝑛 matrix 𝐀 = 𝑎𝑗𝑘 is denoted by 𝐀−𝟏 and is an
𝑛 × 𝑛 matrix such that
(1)
Solution.
We apply the Gauss elimination (Sec. 7.3) to the following matrix,
where BLUE always refers to the previous matrix.
DETERMINATION OF THE INVERSE BY THE GAUSS–JORDAN METHOD
DETERMINATION OF THE INVERSE BY THE GAUSS–JORDAN METHOD
This is 𝐔 𝐇 as produced by the Gauss elimination. Now follow the
additional Gauss–Jordan steps, reducing U to I, that is, to diagonal
form with entries 1 on the main diagonal.
DETERMINATION OF THE INVERSE BY THE GAUSS–JORDAN METHOD
The last three columns constitute 𝐀−𝟏 . Check: