Chapter III

System of Equations
3.1 Direct Methods for Solving Linear Systems
3.1.1 Gauss Elimination Method
The method of Gaussian Elimination is based on the approach to the solution of a pair of
simultaneous equations whereby a multiple of one equation is subtracted from the other (
Ri → Ri −α R j , α=aik /akk ) to eliminate one of the two unknowns (a forward elimination step). The
resulting equation is then solved for the remaining unknown, and its value is substituted back into
the original equations to solve for the other (a back-substitution step).
Consider the linear system of equations: A x=b
where A is an (n x n) matrix, b is a column vector of constants, called the right-hand-side, and x
is the vector of (n) unknown solution values to be determined. This system can be written out as:

[ ][ ] [ ]
a11 a 12 a 13 … a 1n x1 b1
a21 a 22 a 23 … a 2 n x 2 = b 2
⋮ ⋮ ⋮ ⋱ ⋮ ⋮ ⋮
…
an 1 an 2 an 3 ann x n bn

Gaussian
The Gaussian elimination is of the following form: [ A∨b ] [ U∨c ], where [ A∨b ] is the
Elimination
augmented matrix of the system, U is upper triangular form and c is a column vector.
Example 1: Use Gauss elimination to solve

{
2 x 1 +4 x2 −6 x3 =−4
x 1+5 x 2 +3 x3 =10
x 1+3 x 2 +2 x 3 =5

Solution:
 Forward elimination: The augmented matrix of the system is given by

( |)
2 4 −6 −4
[ A∨b ] := 1 5 3 10
1 3 2 5

Applying elementary row operations[ A∨b ] , we have

( |) ( |) ( |)
2 4 −6 −4 2 4 −6 −4 2 4 −6 −4
R → R 2−1/2 R 1
1 5 3 10 2 0 3 6 12 R3 → R3 −1/3 R 2 0 3 6 12
1 3 2 5 R3 → R 3−1/2 R 1 0 1 5 7 0 0 3 3

{
2 x 1 +4 x2 −6 x3 =−4
Or, we have 3 x 2+ 6 x 3 =12
3 x 3=3

 Backward substitution: 3 x 3=3

1
 3 x 2+ 6 x 3 =12 x 2=2 and

2 x1 + 4 x 2−6 x 3=−4 x 1=−3

Therefore, the solution is {(−3 ,2 , 1 )T }

3.1.2 Gauss Elimination Method with partial pivoting

bi −( ∑ a ik x k )
k=i+1
x i=
In general we can write: aii

One computational difficulty can arise with the standard Gauss elimination technique. In the Gauss
forward elimination method, the elements a(1) (2) (3) (n)
11 , a22 , a33 … ann (assumed to be non-zero) on the main

diagonal after elimination are called pivot elements. For example, in the above example (1):
(1) (2) (3)
a 11 =2 , a22 =3 , a33 =3. Under certain circumstances, the diagonal element can become very small in
magnitude compared with the rest of the elements in the pivot row, as well as perhaps being quite
inaccurate. The computation step

' aik
a ij=aij − a
akk kj

is the source of round-off error in the algorithm, since a kj must be multiplied by a ik /a kk which will
be large if the pivot element a kk is small relative to a ik. Improvements to the basic Gauss Elimination
algorithm include careful selection of the pivot equation and pivot element.
Procedures:
In the first stage of elimination, the first column is searched for the largest element in magnitude and
brought as the first pivot by interchanging the first equation with the equation having the largest
element in magnitude. In the second elimination stage, the second column is searched for the largest
element in magnitude among the (n−1) element leaving the first element, and this element is
brought as the second pivot by an interchange of the second equation with the equation having the
largest element in magnitude. This procedure is continued until we arrive at the equation:
[ A∨b ] Gaussian [ U∨c ]
Elimination
The algorithm to find the pivot is as follows:
Choose j , the smallest integer for which |a jk |=max|aik | , k ≤ i≤ n and interchange rows k ∧ j .
(k) (k )

Remark: If the coefficient matrix A is diagonally dominant or real, symmetric and positive definite
then no pivoting is necessary.
Example 2: The linear system

{0.003000 x+59.14 y=59.17

5.291 x−6.130 y =46.78

has the exact solution: x=10.00 and y=1.000 . Solve the system using Gaussian elimination
performed on this system using four-digit arithmetic with rounding.

2
a. Without partial pivoting: The first element,a(1)
11 =0.003000 , is small, and is associated multiplier,

5.291
m21= =1763.6 6̇ ,
0.003000

rounds to the large number 1764. Performing elementary row operation and the appropriate rounding
gives

{0.003000 x+59.14 y=59.17

−104300 y=−104400
,

instead of the precise values

{−104309.37
0.003000 x+ 59.14 y=59.17
6̇ y=−104309.37 6̇

The disparity in the magnitudes of m21 a 13 and a 23 has introduced roundoff error, but the roundoff
error has not yet been propagated. Backward substitution yields y=1.001 , which is a close
approximation to the actual value, y=1.000 . However, because of the small pivot a 11=0.003000 ,
59.17− (59.14 ) (1.001) 59.17−59.20 −0.03000
x= = = =−10.00
0.003000 0.003000 0.003000

b. With partial pivoting: The pivoting procedure just described results in first finding;

max {|a 11 |,|a21 |}=max {|0.003000|,|5.291|}=5.291=|a 21|

(1) (1) (1)

Interchanging rows, we have

{ 5.291 x−6.130 y =46.78

0.003000 x+59.14 y=59.17
(1)
a21
The multiplier for this system is m21= =0.0005670
a(1)
11

Applying elementary row operation, we have

{5.29159.14
x−6.130 y =46.78
y=59.14

The four-digit answers resulting from the backward substitution are the correct values:

x=10.00∧ y=1.000

Example 3: Solve the following using Gauss elimination method with partial pivoting.

{
10 x− y +2 z=4
a. x +10 y−z=3 Ans. Diagonally dominant, x=(0.375 , 0.289 , 0.269)T
2 x +3 y +20 z=7
Ans. x=(3 ,1 , 2)T . The GEM process breaks down as soon as a zero
pivotal entry is computed. In such an event, we need to resort exchanging
rows of the system in such a way that non vanishing pivots are obtained.
3
 {
x+ y+ z=6
b. 3 x +3 y + 4 z=20
2 x+ y+ 3 z =13

3.1.3 Gauss-Jordan Elimination Method

Given the system of linear equations x=b , the Gauss-Jordan method is stated as:

Gauss
[ A∨b ] [ I ∨c ],
Jordan

where [ A∨b ] is the augmented matrix of the system, I is an identity matrix same dimension with A
and c is a column vector, which is the solution of the system. That is, x= {c } .

Example 4: Use Gauss-Jordan method to solve

{
2 x 1 +4 x2 −6 x3 =−4
x 1+5 x 2 +3 x3 =10
x 1+3 x 2 +2 x 3 =5

Solution: The augmented matrix of the system is given by

( |)
2 4 −6 −4
[ A∨b ] := 1 5 3 10
1 3 2 5

Applying elementary row operations[ A∨b ] , we have

( |) ( |)
2 4 −6 −4 2 4 −6 −4
R2 → R 2−1/2 R 1
1 5 3 10 0 3 6 12 R3 → R3 −1/3 R 2
1 3 2 5 R 3 → R 3 −1/2 R 1 0 1 5 7

( |) ( |)
2 4 −6 −4 R1 → 1/2 R 1 1 2 −3 −2
0 3 6 12 R 2 → 1/3 R 2 0 1 2 4 R1 → R 1−2 R 2
0 0 3 3 R 3 → 1/3 R 3 0 0 1 1

( | ) ( |)
1 0 −7 −10 1 0 0 −3
R1 → R1 +7 R3
0 1 2 4 0 1 0 2
0 0 1 1 R 2 → R 2 −2 R 3 0 0 1 1

¿ : [ I ∨c ]

Therefore, the solution is { c }= {(−3 , 2, 1 )T }

3.1.4 Gauss-Jordan Method for matrix inversion

For a given system of linear equations Ax=b , if A is invertible the solution is given by x= A−1 b .
Thus, by the Gauss-Jordan method is A−1obtained as:

4
 Gauss
[ A∨I ] [ I ∨A −1 ]
Jordan

Example 5: Use Gauss-Jordan method to find the inverse of the coefficient matrix A of the system
below.

{
2 x 1 +4 x2 −6 x3 =−4
x 1+5 x 2 +3 x3 =10
x 1+3 x 2 +2 x 3 =5

Solution: Applying elementary row operations on the augmented matrix, we have

( | )
2 4 −6 1 0 0
( A|I ) ≔ 1 5 3 0 1 0
1 3 2 0 0 1

( | )
2 4 −6 1 0 0
R2 → R 2−1/2 R 1
0 3 6 −1/2 1 0
R3 → R 3−1/2 R 1 0 1 5 −1/2 0 1

( | )
2 4 −6 1 0 0
R3 → R 3−1/3 R 2 0 3 6 −1/2 1 0
0 0 3 −1/3 −1 /3 1

( | )
R1 →1 /2 R1 1 2 −3 1/2 0 0
R2 →1 /3 R2 0 1 2 −1 /6 1 /3 0
R3 →1 /3 R3 0 0 1 −1 /9 −1 /9 1/3

( | )
1 0 −7 −5/6 −2/3 0
R1 → R1 −2 R2 0 1 2 −1/6 1/3 0
0 0 1 −1/9 −1/9 1 /3

( | )
1 0 0 1/18 −13/9 7 /3
R1 → R 1+7 R3
5 /9 −2/3 =: ( I| A )
−1
0 1 0 1/18
R2 → R2 −2 R 3 0 0 1 −1/9 −1/9 1/3

( ) ( )
1/18 −13 /9 7 /3 1 −26 42
−1 1
Now we have, A = 1/18 5 /9 2 /3 = 1 10 −12
18
−1 /9 −1 /9 1 /3 −2 −2 6

Therefore,

( )( ) ( )
1 −26 42 −4 −3
−1 1
x= A b= 1 10 −12 10 = 2 is the solution.
18
−2 −2 6 5 1

3.1.5 Matrix LU-decomposition (Factorization) Method

5
This method involves the decomposition of the coefficient matrix A of the system of equations
AX=B in upper (U) and lower (L) triangular matrices. This decomposition is not unique for a
matrix A. This method is called triangular decomposition and it includes: Doolittle, Crout and
Cholesky decomposition method. In these three cases, the decomposition is unique.

a. Doolittle Method

For the system of equations AX=B, we let A=LU , L is the Lower Unit Triangular form and U is
Upper Triangular form. Thus, AX=B

LUX =B

LY =B , where Y =UX and Y is a column vector.

[ ] [ ]
1 0 0 …0 U 11 U 12 U 13 … U 1 n
l 21 1 0 … 0 0 U 22 U 23 … U 2 n
Forms: L= and U =
⋮ ⋮ ⋮ ⋱⋮ ⋮ ⋮ ⋮ ⋱ ⋮
l n1 l n 2 l n 3 … 1 0 0 0 … U nn

The decomposition process is accomplished by applying the usual Gauss-Elimination method. In this
method U is simply the reduced triangular form and L is obtained by collecting α for l ijsuch that
Ri → Ri −α R j , α=aik /akk . In other words, when the i th row Ri is replaced by the difference of Ri and
multiple of R j.

( )( ) ( )
1 2 3 x1 0
Example 6: Solve the following system using LU-decomposition: 3 4 1 x 2 = 6
1 0 1 x3 1

( )
1 2 3
Solution: Here A= 3 4 1 . Appling Gauss-Elimination method, we have
1 0 1

( ) ( ) ( )
1 2 3 1 2 3 1 2 3
R → R2−3 R 1 ⇒ l 21=3 R → R3−R 2
A= 3 4 1 2 0 −2 −8 3 0 −2 −8 =U
1 0 1 R3 → R3 −R 1 ⇒l 31=1 0 −2 −2 ⇒ l 31=1 0 0 6

( ) ( )
1 0 0 1 2 3
Thus, L= 3 1 0 and U = 0 −2 −8
1 1 1 0 0 6

( ) ( )( ) ( ) ( ) ( )
y1 1 0 0 y1 0 y1 0
Let L Y =B, forY = y 2 . That is 3 1 0 y 2 = 6 ⇒ y 2 = 6
y3 1 1 1 y3 1 y3 −5

6
 ( )( ) ( ) ( ) ( )
1 2 3 x1 0 x1 11/6
But UX =Y ⇒ 0 −2 −8 x 2 = 6 ⇒ x 2 = 1/3
0 0 6 x3 −5 x 3 −5 /6

( )
T
11 1 5
Therefore, the solution is , ,−
6 3 6

b. Crout’s Method

Here for the system of equations AX=B, we let A=LU , L is the Lower Triangular form and U is
Upper Unit Triangular form. Thus, AX=B

LUX =B

LV =B , where V =UX and V is a column vector.

[ ] [ ]
l 11 0 0 … 0 1 U 12 U 13 … U 1n
l l 0…0 0 1 U 23 … U 2 n
Forms: L= 21 22 and U =
⋮ ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ ⋮ ⋱ ⋮
l n1 l n 2 l n 3 … l nn 0 0 0 … 1

The decomposition process is accomplished by applying the usual Gauss-Elimination method. In this
method L is simply the reduced triangular form and U is obtained by collecting α for uij such that
C i → Ci−α C j. In other words, when the i th column C i is replaced by the difference of C i and
multiple ofC j.

( )( ) ( )
1 1 1 x1 1
Example 7: Solve the following system using LU-decomposition: 4 3 −1 x 2 = 6
3 5 3 x3 4

( )
1 1 1
Solution: Here A= 4 3 −1 . Appling Gauss-Elimination method, we have
3 5 3

( ) ( ) ( )
1 1 1 1 0 0 1 0 0
C 2 →C 2−C 1 ⇒ u12=1 C 3 →C 3−5 C 2
A= 4 3 −1 4 −1 −5 4 −1 0 =L
3 5 3 C3 →C 3−C 1 ⇒ u13=1 3 2 0 ⇒ u 23=5 3 2 −10

( ) ( )
1 0 0 1 1 1
Thus, L= 4 −1 0 and U = 0 1 5
3 2 −10 0 0 1

() ( )( ) ( ) ( ) ( )
v1 1 0 0 v1 1 v1 1
Let L V =B, forV = v 2 . That is 4 −1 0 v 2
= 6 ⇒ v 2
= −2
v3 3 2 −10 v3 4 v3 −1/2

7
 ( )( ) ( ) ( ) ( )
1 1 1 x1 1 x1 1
But UX =V ⇒ 0 1 5 x 2 = −2 ⇒ x 2 = 1/2
0 0 1 x3 −1/2 x3 −1/2

( )
T
1 1
Therefore, the solution is 1 , ,−
2 2

c. Cholesky Method

This method is also known as the square-root method. If the coefficient matrix A is diagonally
dominant or real symmetric and positive definite, A can be decomposed as A=L LT , where
L=( l ij ) , l ij=0 ,i< j or L is a lower triangular matrix and LT its adjoint or transpose. Alternatively, A

can be decomposed as L=UU T , where U =( uij ) ,uij =0 ,i> j or U is an upper triangular matrix and U T
its adjoint or transpose.
For the case A=L LT , for the given system of equations Ax=b .
T
Ax=L L x=b
T
Lz=b for L x=z
The intermediate solution values z i ,i=0 ,1 , 2 ,… n are obtained as in (a) and (b). Similarly, the
values x i ,i=0 ,1 , 2 ,… n are obtained as in (a) and (b).
Alternatively, we can find only one inverse L−1 and obtain
−1
z=L b
and
T −1 −1 T
x=(L ) z=(L ) z
The inverse of A may be obtained from
−1 −1 T −1
A =( L ) L
The elements of the lower triangular matrix L may be obtained as

( )
i−1 1 /2
l ii = aii −∑ l 2ij ,i=1 , 2 , … , n
j=1

( )
j−1
1
l ij = a −∑ l l , i= j+1 , j+2 , j+3 , … , n
l jj ij k=1 jk ik
l ij =0 , i< j
where the summation is taken as zero if the upper limit is less than the lower limit.
For the case A=U U T , the elements uij are given by:
1 /2
unn=( ann )

8
 a¿
u¿ = , i=1 , 2, 3 , … , n−1
ann

( )
n
1
uij = a − ∑ u l , i=n−2 ,n−3 , n−4 , … , 1; j=i+1 ,i+2 , … , n−1
u jj ij k= j+1 ik jk

( )
n 1 /2
uii = aii − ∑ u 2
ik ,i=n−1 , n−2 , … , 1
k=i +1

uij =0 , i> j
Example 8: Solve the system of linear equations below using Cholesky’s LU-decomposition
method. Also, find A−1 using this method.

{ {
16 x 1 +4 x2=4 4 x 1−x 2=1
4 x 1+ 50 x 2 +14 x 3=8 −x 1+ 4 x 2−x 3=0
a. b.
14 x 2+ 29 x 3 +15 x 4=12 x 2 + 4 x3 −x 4=0
15 x 3 + 45 x 4 =0 −x 3+ 4 x 4 =0

Solution:

a. The system can be written as:

[ ][ ] [ ]
16 4 0 0 x1 4
4 50 14 0 x2
= 8
0 14 29 15 x3 12
0 0 15 45 x4 0

Let

[ ]
l 11 0 0 0
l l 0
L= 21 22 0
l 31 l 32 l 33 0
l 41 l 42 l 43 l 44

Then, we get

( )
0 1 /2
l 11= a 11−∑ l 21 j =( 16−0 )1 /2=4
j=1

( )
0
1 1
l 21= a21−∑ l 1 k l 2 k = ( 4−0 )=1
l 11 k=1 4

( )
1 1/ 2
l 22= a22−∑ l
1/ 2
2
1j =( a22−l 221) =( 50−1 )1 /2 =7
j=1

( )
0
1 a 0
l 31= a31−∑ l 1 k l 3 k = 31 = =0
l 11 k=1 l 11 4
9
 ( )
1
1 1 1
l 32= a32−∑ l 2 k l 3 k = ( 14−l 21 l 31) = ( 14−(−1 ) ( 0 ) ) =2
l 22 k=1 7 7

( )
2 1 /2
l 33= a33−∑ l 2
3j =¿ ¿
j=1

( )
0
1 1
l 41= a −∑ l l = ( 0−0 )=0
l 11 41 k=1 1 k 4 k 4

( )
1
1 1 1
l 42= a 42−∑ l 2 k l 4 k = ( 0−l 21 l 41) = ( 0−( 1 ) ( 0 ) )=0
l 22 k=1 7 7

( )
2
1 1
l 43= a43−∑ l 3 k l 4 k = ¿
l 33 k=1 5
1
¿ ¿
5

( )
3 1/ 2
l 44= a 44−∑ l 4 j
2
=¿ ¿
j=1

and
l ij =0 , i< j

[ ]
4000
L= 1 7 0 0
0250
0036
The solution of the system

[ ][ ] [ ]
4000 z1 4
z2
Lz=b 1 7 0 0 = 8
0250 z3 12
0036 z4 0

is
z 1=1 , z 2=1 , z 3=2 , z 4=−1
But

[ ][ ] [ ][ ] [ ] [ ]
4 100 x1 1 x1 1/4 0.2500
x2 x2
L x=z 0 7 2 0 = 1 0 = 0.0000
T
=
0 05 3 x3 2 x3 1/2 0.5000
0 0 06 x4 −1 x4 −1/6 0.1667

{( )}
T
1 1 1
The solution set is given by , 0 , ,−
4 2 6

10
Here, applying Gauss-Jordan method for matrix inversion on L,

[ | ]
4000 1000
[ L∨I ]= 1 7 0 0 0 1 0 0 R → R −1/4 R
2 2 1
0250 0010
0036 0001

[ | ]
4000 1 000
0 7 0 0 −1/4 1 0 0 R → R −2/7 R
3 3 2
0 25 0 0 0 1 0
0036 0 001

[ | ]
4000 1 0 00
0 7 0 0 −1/4 1 0 0 R → R −3/5 R
4 4 3
0 0 5 0 1/14 −2 /7 1 0
0036 0 0 01

[ | ]
4000 1 0 0 0 R1 →1/ 4 R1
0 7 0 0 −1/ 4 1 0 0 R2 →1 /7 R 2
0 0 5 0 1 /14 −2/7 1 0 R3 → 1/5 R3
0 0 0 6 −3/70 6/35 −3/5 1 R 4 →1 /6 R 4

[ | ]
1 0 0 0 1 /4 0 0 0
0 1 0 0 −1/28 1/7 0 0 ≔[I ∨L−1 ]
0 0 1 0 1/70 −2/35 1/5 0
0 0 0 1 −1/140 1/35 −1/10 1 /6

we get,

[ ] [ ]
1/ 4 0 0 0 35 0 0 0
1
L = −1/28 1 /7 0 0 = −5 20 0 0
−1

1/70 −2/35 1 /5 0 140 2 −8 28 0

−1/140 1 /35 −1 /10 1/6 −1 4 −14 23. 3̇

[ ]
0.2500 0 0 0
¿ −0.0357 0.1429 0 0
0.0143 −0.0571 0.2000 0
−0.0071 0.0286 −0.1000 0.1667
But
−1 T
A =( L ) ∗L
−1 −1

[ ]
35 −5 2 −1
0 20 −8 4 ∗1

[ ]
0 0 28 −14 35 0 0 0
1 0 0 0 23. 3̇ −5 20 0 0
¿
140 140 2 −8 28 0
−1 4 −14 23. 3̇
11
 [ ][ ]
125.5 −12 7 −7 /3 0.0640−0.0061 0.0036−0.0012
1 −12 40 −28 28/3 −0.0061 0.0245−0.0143 0.0048
¿ =
1960 7 −28 98 −98/3 0.0036−0.0143 0.0500−0.0167
−7 /3 28/3 −98 /3 490/9 −0.0012 0.0048−0.0167 0.0278

[ ][ ] [ ]
0.0640−0.0061 0.0036−0.0012 4 0.2500
−0.0061
−1 0.0245−0.0143 0.0048 ∗ 8 = 0.0000
Thus, x= A B= is the solution.
0.0036−0.0143 0.0500−0.0167 12 0.5000
−0.0012 0.0048−0.0167 0.0278 0 0.1667

b. The system can be written as:

[ ][ ] [ ]
4 −1 0 0 x1 1
−1 4 −1 0 x2
=0
0 −1 4 −1 x3 0
0 0 −1 4 x4 0

Let

[ ]
l 11 0 0 0
l l 0
L= 21 22 0
l 31 l 32 l 33 0
l 41 l 42 l 43 l 44

Then, we get

( )
0 1 /2
l 11= a 11−∑ l 21 j =( 4−0 )1 /2=2
j=1

( )
0
1 1 −1
l 21= a21−∑ l 1 k l 2 k = (−1−0 )=
l 11 k=1 2 2

( ) √
1/ 2

( )
1 1 /2
1 15
l 22= a22−∑ l 21 j
1/ 2
=( a22−l 221) = 4+ =
j=1 4 4

( )
0
1 a 0
l 31= a31−∑ l 1 k l 3 k = 31 = =0
l 11 k=1 l 11 2

( )√ √ ( ( ) ) √
1
1 4 4 −1 4
l 32= a32−∑ l 2 k l 3 k = (−1−l 21 l 31) = −1− ( 0 ) =−
l 22 k=1 15 15 2 15

( )
2 1 /2
l 33= a33−∑ l 23 j =¿ ¿
j=1

( )
0
1 1
l 41= a41−∑ l 1 k l 4 k = ( 0−0 )=0
l 11 k=1 2

12
 ( )√ √ ( ( ) )
1
1 4 4 −1
l 42= a 42−∑ l 2 k l 4 k = ( 0−l 21 l 41 )= 0− ( 0 ) =0
l 22 k=1 15 15 2

( )√
2
1 15
l 43= a43−∑ l 3 k l 4 k = ¿
l 33 k=1 56

¿
√ 15
56
¿

( )
3 1/ 2
l 44= a 44−∑ l 2
4j =¿ ¿
j=1

and
l ij =0 , i< j

[ ]
2 0 0 0
L= −1/2 √ 15/ 4
0 0
0 − √ 4/15 −√ 56/15 0
0 0 −√ 15/56 −√ 209/56

The solution of the system

[ ][ ] [ ]
2 0 0 0 z1 1
Lz=b −1/2 √ 15/ 4
0 0 z2
=0
0 −√ 4 /15 −√ 56/15 0 z3 0
0 0 −√ 15/56 −√ 209/56 z4 0

is
z 1=1/2 , z 2=1 / √ 60 , z 3=1/ √ 840 , z 4=1/ √ 11704
But

[ ][ ] [ ]
2 −1/2 0 0 x1 1 /2
1 / √ 60
L x=z 0 √ 15 /4 −√ 4 /15
T 0 x2
=
0 0 − √ 56 /15 − √ 15/56 x3 1 / √ 840
0 0 0 −√ 209/56 x4 1/ √ 11704

[ ][ ][ ]
x1 56/209 0.2679
x2
= 15/209 = 0.0718
x3 4 /209 0.0191
x4 1/209 0.0048

Applying Gauss-Jordan method for matrix inversion on L, we get

[ ][ ]
1 /2 0 0 0 0.5000 0 0 0
−1 1 / √ 60 √ 4/15 0 0 0.1291 0.5164 0 0
L = =
1 / √ 840 √ 2/105 √ 15/56 0 0.0345 0.1380 0.5176 0
1/ √ 11704 √ 2/1463 √ 15 /11704 √ 56/209 0.0092 0.0370 0.1387 0.5176

13
But
−1 T
A =( L ) ∗L
−1 −1

[ ][ ]
0.5000 0.1291 0.0345 0.0092 0.5000 0 0 0
¿ 0 0.5164 0.1380 0.0370 ∗ 0.1291 0.5164 0 0
0 0 0.5176 0.1387 0.0345 0.1380 0.5176 0
0 0 0 0.5176 0.0092 0.0370 0.1387 0.5176

[ ]
0.2679 0.0718 0.0191 0.0048
¿ 0.0718 0.2871 0.0766 0.0191
0.0191 0.0766 0.2871 0.0718
0.0048 0.0191 0.0718 0.2679
3.1.6 Tri-diagonal Matrix Method (Thomas algorithm for Tri-diagonal system)
Consider the linear system of equations: A x=b , where A is a tri-diagonal n × n square matrix.

[ ][ ] [ ]
d 1 u1 0 0 … 0 x1 b1
l 1 d2 u 2 0 … 0 x2 b2
0 l 2 d 3 u3 ⋱ 0 x3 = b3
⋮ ⋮ ⋮ ⋱⋱ ⋮ ⋮ ⋮
0 0 0 0 … dn xn bn
The system of equations given above is solved using Thomas Algorithm which is described in three
steps as shown below:

l i ui−1
Step 1: Set y 1=d1 and compute: y i=di − , i=2 ,3 , … , n
y i−1

b1 bi−l i z i−1
Step 2: Set z 1= and compute: z i= ,i=2, 3 , … , n
d1 yi

ui x i+ 1
Step 3: Set x i=z i− ,i=n−1 , n−2, … , 1 , where x n=z n
yi

Example 9: Solve the following equations by Thomas Algorithm.

{
3 x1−x 2=5
2 x 1−3 x 2 +2 x 3=5
x 2 +2 x3 +5 x 4 =10
x 3−x 4 =1

[ ][ ] [ ]
3 −1 0 0 x1 5
2 −3 2 0 x2
Solution: The system can be written as: = 5
0 1 2 5 x3 10
0 0 1 −1 x4 1

[ d 1 , d 2 , d 3 , d 4 ]= [ 3 ,−3 ,2 ,−1 ]

14
 [ u1 ,u 2 , u3 ]= [−1 , 2 ,5 ]
[ l2 ,l3 ,l 4 ]=[ 2 ,1 , 1 ]
[ b 1 , b2 , b3 , b4 ]=[ 5 ,5 , 10 , 1 ]
l i ui−1
Step 1: Set y 1=d1 and compute: y i=di − , i=2 ,3 , … , n
y i−1

l 2 u1 2 (−1 ) −7
y 1=3 i=2 , y 2=d 2− =−3− =
y1 3 3

l 3 u2 1 ( 2 ) 20
i=3 , y 3=d 3 − =2− =
y2 −7 7
3

l 4 u3 1 ( 5 ) −55
i=4 , y 4=d 4− =−1− =
y3 −20 20
7

b1 5 bi−ai z i−1
Step 2: Set z 1= = and z i= , i=2 , 3 , … ,n
d1 3 yi

i=2 , z 2=
b 2−l 2 z 1
=
5−2 ( 53 ) = −5
y2 −7 7
3

b −l z
i=3 , z 3= 3 3 2 =
10−1 ( −53 ) = 75
y3 20 20
7

i=4 , z 4=
b 4−l 4 z 3
=
1−1 ( 7520 ) =1
y4 −55
20

ui x i+ 1
Step 3: Set x i=z i− ,i=n−1 , n−2, … , 1 , wher e x n=z n
yi

u3 x 4 75 5(1)
i=3 , x 3=z 3− = − =2
x 4 =z 4=1 y 3 20 20
7

15
 u 2 x 3 −5 2(2)
i=2 , x 2=z 2− = − =1
y2 7 −7
3

u1 x 1 5 (−1 ) 1
i=1 , x1 =z1− = − =2
y1 3 3

Therefore, the solution is( 2 , 1, 2 , 1 )T .

3.2. Indirect Methods for Solving Linear Systems

Sometimes when solving science and engineering problems, systems of equations will results
which involve large numbers of equations and unknowns (100,000 s¿1,000,000 s ). For large systems
of equations, Gauss Elimination is inefficient and prone to large round off errors. In this case it is
often more convenient to use a solution method that involves a sequential process of generating
solutions that converge on the true solutions as the number of steps in the sequence increases.
A x=b

{
a11 x1 + a12 x 2+ a13 x 3 +…+ a1 n x n=b1
a 21 x 1+ a22 x 2+ a23 x3 +…+ a2 n x n=b2
or a 31 x 1+ a32 x 2+ a33 x 3 +…+ a2 n xn =b2
⋮
an 1 x1 +a n 2 x 2 +an 3 x 3+ …+a nn x n=b n

Iterative methods of solution, as distinct from direct methods such as Gauss Elimination, begin
by rearranging the system of equations so that one unknown is isolated on the left-hand side of
each equation

b1− ( a12 x 2 +a 13 x 3+ …+a 1n x n )

x 1=
a 11
b2− ( a21 x 1 +a 23 x 3+ …+a 2n x n )
x 2=
a22
⋮
b −(a n 1 x 1+ an 2 x2 +…+ an , n−1 x n−1)
x n= n
ann

bi−( )
In general, we have: x i=
aii

Now, if an initial guess x 01 , x 02 , … , x 0n for all the unknowns was available, we could substitute these
values into the right-hand side of the previous set of equations and compute an updated guess for the
unknowns x 11 , x 12 ,… , x 1n, . There are several ways to accomplish this, depending on how you use the
most recently computed guesses.

16
3.2.1 Gauss-Jacobi Method
In the Jacobi method, all of the values of the unknowns are updated before any of the new
information is used in the calculations. That is, starting with the initial guess x 01 , x 02 , … , x 0n , compute
the next approximation of the solution as
0 0 0
1 b1−(a12 x 2 + a13 x 3 +…+ a1 n x n )
x 1=
a 11
1 b2−(a21 x 1 + a23 x 03 +…+ a2 n x 0n )
0

x 2=
a 22
⋮
0 0 0
1 bn −(a n 1 x 1 +a n2 x 2+ …+a n ,n−1 x n−1 )
x n=
a nn

or, after k iterations of this process, we have

k k k
k+1 b 1−(a12 x 2 +a13 x3 + …+a1 n x n)
x1 =
a11
k+1 b2 −(a21 x 1 +a23 x3k + …+a 2n x kn)
k

x2 =
a22
⋮
k k k
k+1 b n−(an 1 x 1 + an 2 x 2 +…+ an , n−1 x n−1)
xn =
ann

n+1 b i−( )
More generally, xi =
a ii

Example 10: Consider the system of equations

{
2 x 1−x 2=1
−x 1+3 x 2−x 3=8
−x 2 +2 x3 =−5

A convenient initial guess is usually x 01=x 02=x 03=0. In this case we have
k k
1−(− x2 ) 1+ x 2
k +1
x =
1 =
2 2 k
−5+ x k2
k+1 −5−(−x 2 )
k k
8−(−x 1−x 3 ) x 3 = = ¿¿
k+1
x2 =
k k
=8+ x 1 + x 3 ¿ 2 2
3
¿
3

or, taking two iterations of the method we have

17
 0 1
1 1+ x 2 1+ 0 2 1+ x 2 1+2.667
x=
1 = =0.5 x 1= = =1.8333
2 2 −5+ 0 2 2
1 x 2 −5+ 0 2 −5+ x
1 0 0
x 2=8+ x 1 + x 3 ¿ =2.667 ¿ x 3= = =−2.5 ¿ ¿ x 22=8+ x 11 + x 13 ¿ =2¿ x 3=
=8+ 0+0 ¿ 2 2 =8+0.5−2.5 0¿ 2
3 3
3 3

Continuing this process ultimately leads, after 20 iterations, to: x 1=2 , x 2=3 , x 3=−1

3.2.2 Gauss- Seidel Method

When applying the Jacobi method, one may realize that information is being made available at each
step of the algorithm. That is the current approximation of one of the unknowns is available for use
after each step. This information could be used immediately in the calculation of the next unknown.
If we implement this, our method would look like
0 0 0
1 b 1−(a12 x 2 +a 13 x 3 +…+a 1 n x n )
x 1=
a 11
1 b2−(a21 x 1 +a 23 x 03+ …+a 2 n x 0n)
1

x 2=
a22
⋮
1 1 1
b −(a n 1 x 1+ an 2 x2 +…+ an , n−1 x n−1)
x 1n= n
ann

or, after k iterations of this process, we have

k k k
k+ 1 b1−(a12 x 2 +a 13 x 3 +…+ a1 n x n )
x1 =
a 11
k +1 b2−(a21 x1 +a23 x3k + …+a 2n x kn)
k+1

x2 =
a22
⋮
k +1 k+1 k+1
k+1 b n−(an 1 x 1 + an 2 x2 +…+ an ,n−1 xn −1 )
xn =
ann

¿ ¿¿
∑ ¿ j <i ¿
aii
More generally, j≠i ¿

Example 11: Consider again the system of equations:

2 x 1−x 2=1
−x 1+ 3 x 2−x 3=8
−x2 +2 x 3=−5

A convenient initial guess is usually x 01=x 02=x 03=0. Using the Gauss -Siedel method, we have

18
 k
1+ x 2
k +1
x = k+1
k+1 −5+ x 2
1
2 x = ¿¿
3
k+1 k +1 k
x2 =8+ x 1 + x3 ¿ 2
¿
3

or, taking two iterations of the method, we have

0 1
1 1+ x 2 1+0 2 1+ x 2 1+2.8333
x=
1 = =0.5 x1 = = =1.916
2 2 1
−5+ x2 −5+2.8333 2 2
1
1 1 0
x 2=8+ x 1 + x 3 ¿ =2.8333 ¿ x3 = = =−1.08333 ¿ ¿ x 22=8+ x 21 + x 13 ¿
=8+0.5+ 0 ¿ 2 2 =8+1.9167−1.08
3 3
3 3
x
Continuing this process ultimately leads, after 9 iterations, to 1 =2 , x 2 =3 , x 3 =−1
Remark:

 To apply Jacobi or Gauss-Seidel iterative methods, the equations can be re-arranged so that a ii is
as large as possible in order to speed convergence. Jacobi’s method required twice as many
iterations as Gauss- Seidel for the same accuracy. If the Jacobi method is convergent, the Gauss-
Seidel is too.
n

 If the coefficient matrix A is strictly diagonally dominant ( i .e . ∑ |aij|<|aii|, i=1 , 2 ,3 , … n ¿, then

j=1
j≠ i
∞
for any choice of X (0) , both Jacobi and Gauss-Seidel methods give sequences { X(k) } k=0 that
converge to the solution of Ax=B .
 Define ∆ X k =‖ X (k)−X‖ , k = 0, 1, 2, … and δ X k =‖ X (k+1)− X (k )‖, k = 0, 1, 2, 3, … where ∆ X k is
the norm of the error vector at iteration k, and δ X k is the norm of the solution vector change
from iteration k to k+1. A convergent process drives the values of ∆ X k and δ X k to zero as k
increases. The norm ∆ X k can’t be computed if the solution we are seeking is initially unknown,
where as we can compute the norm δ X k and use it to establish convergence. The stopping
‖ X k+1− X k‖
criterion is to iterate until <∈ , where k=0 , 1 , 2, 3 , …, and ∈ is the tolerance such that
‖ X k +1‖
it depends on precision level used, definition of the norm and magnitudes of the elements of the
solution vector.
3.2.3 Systems of non-linear equations using Newton’s method
Consider a system of n non-linear equations in n unknowns given by:

19
 {
f 1 ( x 1 , x 2 , x 3 , … , x n ) =0
f 2 ( x 1 , x 2 , x 3 , … , x n ) =0
⋮
f n ( x 1 , x 2 , x 3 , … , x n ) =0

The system can be written in a single expression using vectors, i.e.

f ( x )=0,
where the vector x contains the independent variables, and the vector f contains the functions f i ( x ):

[] [ ][ ]
x1 f 1 ( x 1 , x 2 , x 3 ,… , x n )=0 f 1(x)
x= x 2 , f ( x )= f 2 ( x 1 , x 2 , x 3 ,… , x n )=0 = f 2 ( x ) .
⋮ ⋮ ⋮
xn f n ( x 1 , x 2 , x 3 , … , x n )=0 f n(x)

A Newton-Raphson method for solving the system of linear equations requires the evaluation of a
matrix, known as the Jacobian of the system, which is defined as:

[ ]
∂ f 1 /∂ x 1 ∂ f 1 /∂ x2 ∂ f 1 /∂ x 3 … ∂ f 1 /∂ x n
∂ ( f 1 , f 2 , f 3 , … , f n)
J= = ∂ f 2 /∂ x 1 ∂ f 2 /∂ x2 ∂ f 2 /∂ x 3 … ∂ f 2 /∂ x n
∂ ( x1 , x2 , x3 , … , xn ) ⋮ ⋮ ⋮ ⋱ ⋮
∂ f n /∂ x 1 ∂ f n /∂ x 2 ∂ f n /∂ x 3 … ∂ f n /∂ x n

¿
[ ]
∂fi
∂ xj n ×n
, for i , j=0 , 1 , 2, … , n

If x=x 0 (a vector) represents the first guess for the solution, successive approximations to the
solution are obtained from
−1
x k+1 =x k −J . f (x ¿¿ k )=x k −∆ x k ¿ , with ∆ x k = x k+1−x k .
or

[ ][][ ][ ]
−1
x1
k+1
x1
k
∂ f 1 /∂ x1 ∂ f 1 /∂ x 2 ∂ f 1 /∂ x 3 … ∂ f 1 /∂ x n f 1 ( xk )
∗ f 2 ( xk )
k+1 k
x2 = x2 − ∂ f 2 /∂ x1 ∂ f 2 /∂ x 2 ∂ f 2 /∂ x 3 … ∂ f 2 /∂ x n
⋮ ⋮ ⋮ ⋮ ⋮ ⋱ ⋮ ⋮
k+1
xn xnk ∂ f n /∂ x 1 ∂ f n /∂ x 2 ∂ f n /∂ x 3
… ∂ f n /∂ xn x =x k
f n ( xk )

A convergence criterion for the solution of a system of non-linear equation could be, for example,
that the maximum of the absolute values of the functions f i ( x k ) is smaller than a certain tolerance ε,
i.e.,
max|f i (x k )|<ℇ
i

Another possibility for convergence is that the magnitude of the vector f (x ¿¿ k )¿ be smaller than the
tolerance, i.e.,

20
 ¿
We can also use as convergence criteria the difference between consecutive values of the solution,
i.e.
max|(x i)k +1−(x i )k|<ℇ
i

or,
‖∆ x k‖2=‖ x k+1−x k‖2 < ℇ .
The main complication with NR-method to solve a system of non-linear equations is having to
define all the functions∂ f i /∂ x j, fori , j=0 , 1 ,2 , … , n, included in the Jacobian. As the number of
equations and unknowns,n , increases, so does the number of elements in the Jacobian, n2 .
Example 12: Solve the system of non-linear equations given below using NR-method.

{
x 3−2 y−2=0
{x xy+ y=25=50
2 2
a. b. x3 −5 z2 +7=0
2
y z −1=0
Solution:
a. Here, the exact solutions are ( 5 , 5 )∧(−5 ,−5 ) .

[ ]
2 2
f ( x )= f 1 ( x , y )=x + y −50=0 and the Jacobian J is given by
f 2 ( x , y )=xy −25=0

[ ][
∂f1 ∂f1
J= ∂ x
∂f2
∂ y = 2x 2 y
∂f2 y x ]
∂x ∂y

J =
−1
[ 2yx 2xy ]= x 0.5− y [−xy
2 2
−2 y
2x ]
Now,
−1
x k+1 =x k −J . f (x ¿¿ k ) ¿

[ ]
( k+1 )
x k+1 = x (k +1)
y

[ ]( )[ ][ ]
(k)
(k) 2 (k) 2 (k)
(k)
0.5 x −2 y (x ) +( y ) −50 , k =0 , 1 ,2 …
¿ x (k) − (k) 2 (k) 2
∗ (k) (k)
∗ (k) (k )
y (x ) −( y ) −y 2x (x )( y )−25

[ 64 ], we have
Taking initial guess x 0=

[ 4] 36−16
x = 6 −(
1
0.5
)∗[−46 −812 ]∗[ 36+16−50
6 ( 4 )−25 ]

21
 []
4 [ ][ ]
¿ 6 −( 0.025 ) 20 = 5.5
−20 4.5

[ 4.5]
x = 5.5 −( 0.05 )∗
2 [−4.5
5.5 −9
11 ] [−0.25 ] [ 4.75 ]
∗ 0.5 = 5.25

[ 4.75] [−4.75
x = 5.25 −( 0.1 )∗
3
10.5 ] [−0.0625 ] [ 4.875 ]
5.25 −9.5
∗ 0.125 = 5.125

[ 4.875 ] [−4.875
x = 5.125 −( 0.2 )∗
4
10.25 ] [−0.015625 ] [ 4.9375 ]
5.125 −9.75
∗ 0.03125 = 5.0625

[ 4.9375] [−4.9375
x = 5.0625 −( 0.4 )∗
5
10.125 ] [−0.00390625 ] [ 4.96875 ]
5.0625 −9.875
∗ 0.0078125 = 5.03125

[ ]
2 2
f ( x5 )= f 1 ( 5.03125 , 4.96875 )=5.03125 + 4.96875 −50=0.00193125
f 2 ( 5.03125 , 4.96875 )=( 5.03125 ) ( 4.96875 ) −25=−0.0009765625

Here, one can observe that the iteration is converges to the exact solution ( 5 , 5 ) .

{
x 3−2 y−2=0
b. For the system x3 −5 z2 +7=0 the exact solutions are:
2
y z −1=0

[ ][√3 3 √3 3
][ ][ ]
1.44224957 1.44224957
1/2 = 0.50000000 ∧ 1 /2 = 0.50000000 .
√ 2 1.414213562 √ 2 1.414213562

[] [ ]
1 1
x = x
Taking initial guess 0 1 and 0 = 1 . Hint! The Jacobian J and its inverse J −1 is given
2 −2
by:

[ ] [ ]
3 x2 −2 0 10 z 2 4y 20
2 −1 1 2 2 2
J= 3 x 0 −10 z J = 2 2 2 −6 x y 6 x y 30 x
30 x z +12 x y
0 z2 2 yz 3 x 2 z −3 x2 z 6 x 2
3.2.4 ILL-CONDITIONED SYSTEM OF EQUATIONS
A system of equations AX=B is said to be ill-conditioned or unstable if it is highly sensitive to
small changes in A and B i.e., small change in A or B causes a large change in the solution of the
system. On the other hand if small changes in A and B give small changes in the solution, the system
is said to be stable, or, well conditioned. Thus in an ill-conditioned system, even the small round
off errors effect the solutions very badly. Unfortunately it is quite difficult to recognize an ill-
conditioned system.

Example 13: Consider the following two almost identical systems of linear equations:

a. {x−1.00001
x− y=1
y=0
b. {x−0.99999
x− y=1
y=0
22
Their respective solutions are:

[{ 100,001
100,000 }
] and { −100,000]}
[ −99,999

Obviously, the two solutions differ very widely. Therefore, the system is ill-conditioned.

Example 14: Show that the following system of linear equations is ill-conditioned:

{7 x−10 y=1
5 x 1 +7 y=0.7

Solution: On solving the given equations we get x=0∧ y=0.1 and now we make slight changes in
the given system of equations. The new system becomes:

{75x−10 y=1.01
x + 7 y=0.69
1

Here, we get x=−0.17∧ y =0.22. Hence the given system is ill-conditioned.

Definition:
‖∆ X‖ ‖∆ B‖
Let AX=B be a system of linear equations. Thus, ≤‖A −1‖‖ A‖ , where ∆ X∧∆ B are
‖ X‖ ‖B‖
changes in X and B, respectively. The quantity ‖ A−1‖‖ A‖ is the condition number of the matrix A
and denoted bycond ( A ) ∨k ( A). That is,
k ( A )=‖ A−1‖‖ A‖.
If the condition number k ( A ) is small, then small changes in A or B produce small changes in X . If
k ( A ) is large, then small relative changes in A or B produce large relative change in X and in this
case the system is said to be ill-conditioned. If k ( A ) is near unity (1), then the system AX=B is said
to be well-conditioned.
Example 15: Find the condition number of the system:

{
2.1 x 1 +1.8 x 2=2.1
6.2 x 1 +5.3 x 2=6.2
Solution: Here the coefficient matrix:
A= (
2.1 1.8
= ) (
1 21 18
6.2 5.3 10 62 53 )
and hence
A = (
−1 1 −530
3 620 −210
180
)
Now, ‖ A‖=8.6128∧‖ A−1‖=287.0927(Euclidean norm or max norm…) and thus:

k ( A )=‖ A−1‖‖ A‖=2472.7

23
Thus, the system is highly ill-conditioned and it is very sensitive to rounding off errors.
3.4 An Iterative Method to Approximate the Eigenvalues and Eigenvectors
3.4.0 Introduction

Definition: Let A be an n × n matrix. If there is a number λ and a column vector v ≠ 0 for which

Av=λv

then we say λ is an eigenvalue(characteristic value, proper value) of A and v is an associated

eigenvector(characteristic vector, proper vector).
A canonical (standardized) choice of eigenvector is the vector with unit length (there are in fact
two of these in the direction of v), but often it is easier to work with an eigenvector of integer values
(if there is one).
Note that λ is an eigenvalue iff| A−λI |=0.

Now, | A−λI | is expanded to a polynomial in λ of degreen, where A is an n × n matrix, and this

polynomial is called characteristic polynomial and the equation is called characteristic equation.

Example 16: For the following matrices find the eigenvalues and associated eigenvectors.

( )
−7 13 −16
a. A=
1 1
4 −2 ( ) b. A= 13 −10 13 .
−16 13 −7
Solution:

a. Here | A−λI |=0 (| 14 1

−2) ( )|
−
λ 0
0 λ
=0

(| 1−λ
4 −2−λ )|
1
=0

( 1− λ ) (−2−λ )−4=0 (Characteristic equation)

( λ−2 ) ( λ+3 )=0
λ=−3∨λ=2
To find the eigenvector,
i. λ 1=−3 ( A− λ1 I ) X 1=0, where X 1 =( x 1 , x 2) t

{
4 x 1 + x 2=0
4 x 1 + x 2=0
.
t
Take x 1=1∧we get x2 =−4 , X 1 =( 1,−4 ) is the eigenvector corresponding to
λ 1=−3
ii. λ 2=2 , similarly ( A−λ2 I ) X 2=0 X 2= (1 , 1 )t is the eigenvector corresponding to λ 2=2

( )
−7 13 −16
b. A= 13 −10 13
−16 13 −7

24
 | |
λ+7 −13 16
Now, |λI − A|= −13 λ +10 −13 =0
16 −13 λ+7
3 2
λ + 24 λ −405 λ+ 972=0
λ 1=−36 , λ2=9 , λ3=3
t
i. For λ 1=−36, to find the associated eigenvector X 1 =( x 1 , x 2 , x 3 ) , solve
( λI − A ) X 1=0
(−36 I − A ) X 1 =0

( )( ) ( ) {
−29 −13 16 x 1 0 −29 x1 −13 x2 +16 x 3=0
−13 −26 −13 x 2 = 0 −13 x1 −26 x 2 −13 x3 =0
16 −13 −29 x 3 0 16 x1 −13 x2 −29 x3 =0
If x 1=0 then the only solution is X 1 =0and we can choose x 1=1. Thus we have

{
−13 x 2+16 x 3=29
−26 x 2−13 x3 =13 x 2=−1 , x 3=1
−13 x2 −29 x 3=−16
t
Thus the eigenvector is X 1 =( 1,−1 ,1 ) , or any non-zero multiple of this vector.
t
ii. For λ 2=9, to find the associated eigenvector X 2 =( x 1 , x 2 , x 3 ) , solve
( λI − A ) X 2=0 (−36 I − A ) X 2 =0

( )( ) ( ){
16 −13 16 x 1 0 16 x 1 −13 x 2 +16 x 3=0
−13 19 −13 x 2 = 0 −13 x 1 19 x 2 −13 x 3=0
16 −13 16 x 3 0 16 x 1 −13 x 2 16 x 3=0
Taking x 1=−1, we have

{
−13 x2 +16 x 3=16
19 x 2−13 x 3=−13
−13 x2 +16 x 3=16
{
−247 x2 +304 x 3=304
247 x 2−169 x 3=−169
x 2=0 , x 3=1

t
Thus, the eigenvector is X 2 =(−1 ,0 , 1 ) , or any non-zero multiple of this vector.
t
iii. For λ 3=3, to find the associated eigenvector X 3 =( x 1 , x 2 , x 3 ) , solve
( λI − A ) X 3=0
(−36 I − A ) X 3 =0

( )( ) ( ){
10 −13 16 x 1 0 10 x 1 −13 x 2 +16 x 3=0
−13 13 −13 x 2 = 0 −13 x 1 13 x 2 −13 x 3=0
16 −13 10 x 3 0 16 x 1 −13 x 2 16 x 3=0
We arbitrarily choose x 1=1. This leads us to the system:

{
−13 x2 +16 x 3=16
13 x 2−13 x 3=−13 x 1=2 , x 2=1
−13 x 2+10 x 3=16
t
Thus, the eigenvector is X 3 =( 1 ,2 , 1 ) , or any non-zero multiple of this vector.
Remark:

25
  The eigenvalues of a matrix tell us, how the linear transform scales a given matrix, where
as the eigenvectors tell us, which directions are ‘’purely scaled’’.
 If v is an eigenvector, then any non-zero multiple of v is also an eigenvector for the same
eigenvalue λ .
 If λ is an eigenvalue of A, then λ nthe eigenvalue of An .
 The eigenvalues of a diagonal or upper triangular or lower triangular matrix are the
diagonal entries.
 The zero vector cannot be an eigenvector eventhough A ∙ 0=λ ∙ 0.
 An eigenvalue can be zero.
Definition: The largest magnitude of the eigenvalues is called the spectral radius ( ρ ¿.

 The spectral radius ( ρ ¿ for either the Jacobi or Gauss-Seidel method is related to the error
and vector change norms by:
∆ X k +1 δ X k +1
ρ≈ ≈ , for large k.
∆ Xk δ Xk
3.4.1 The Power Method
The power method is an iterative method used to find the largest eigenvalue (in magnitude) and the
corresponding eigenvector of a square matrix. Let A be a matrix whose eigenvalue and eigenvector
are required. And let X (0) be a non-zero initial vector, then we evaluate A X (0)and write as:
(0) (1) (1)
A X =λ X
where λ is the Euclidean norm (‖∙‖2) of the vector A X (0). Then λ(1) will be the first approximated
(1)

eigenvalue and X (1) will be the first approximated corresponding eigenvector. Similarly, we evaluate
(1)
A X and write the result as:
(1) (2) (2 )
A X =λ X
which gives the second approximation. Repeating this process till ‖ X(n+ 1)−X (n)‖, n=0 ,1 , 2 , … is negligible.
Then λ(n) will be the largest eigenvalue of A and X (n)is the corresponding eigenvector.
Example 18: Find the largest eigenvalue and the corresponding eigenvector of the matrix:

( )
1 3 −1
A= 3 2 4 , taking initial guess to the corresponding eigenvector X (0)=(0 , 0 , 1)T and the
−1 4 10
tolerance limit 0.01. The exact eigenvalues are:
λ 1=−2.501392314843656

λ 2=3.839401318248099

λ 3=11.661990996595559

and the exact eigenvectors are:

(1) T
X =(−0.669661446564720 , 0.690122014712989 ,−0.274381398409828)
(2) T
X =(0.742313122917803 , 0.610590011930241 ,−0.275954823974951)
(3) T
X =(0.022907957762868 , 0.388473219333732 , 0.921175218583100)
26
Solution:

First approximation

( )( ) ( ) ( ) ( )
1 3 −1 0 −1 −0.0924 −0.0924
(0) (1) (1)
A X = 3 2 4 0 = 4 =10.817 0.3700 λ =10.817 X = 0.3700
−1 4 10 1 10 0.9245 0.9245
Second approximation

( )( )( ) ( ) ( )
1 3 −1 −0.0924 0.0931 0.0080 0.0080
(1) (2) (2 )
AX = 3 2 4 0.3700 = 4.1608 =11.590 0.3590 λ =11.590 X = 0.3590
−1 4 10 0.9245 10.8174 0.9333 0.9333
Third approximation

( )( ) ( ) ( ) ( )
1 3 −1 0.0080 0.1517 0.0130 0.0130
(2) (3) (3)
A X = 3 2 4 0.3590 = 4.4752 =11.655 0.3840 λ =11.655 X = 0.3840
−1 4 10 0.9333 10.761 0.9233 0.9233
Fourth approximation

( )( ) ( ) ( ) ( )
1 3 −1 0.0130 0.2417 0.0207 0.0207
(3) (4) (4)
A X = 3 2 4 0.3840 = 4.5002 =11.662 0.3859 λ =11.662 X = 0.3859
−1 4 10 0.9233 10.756 0.9223 0.9223
The absolute error is | λ(4 )−λ(3)∨¿ 0.007 <0.01.

( )( ) ( ) ( ) ( )
1 3 −1 0.0207 0.2561 0.0451 0.0451
(4) (4) (4 )
A X = 3 2 4 0.3859 = 4.5231 =11.6618 0.3878 λ =11.6618 X = 0.3878
−1 4 10 0.9223 10.7459 0.9214 0.9214

Exercise 3
1. Use the different direct methods to solve the following systems of linear equations. Show all
steps in the computation. Be sure to substitute your answers into the original equation to check
your answers. The exact answers are given aside.

a. {
2 x 1−6 x 2=−18
−x1 +8 x 2=40
Ans . 9.6 6.2

b. {
0.77 x 1+ 6 x 2=14.25
1.2 x1 +1.7 x 2=20
Ans . 16.2579 0.2886

{
−12 x 1 + x 2−x 3=−20
c. −2 x 1−4 x 2+ 2 x 3=10 Ans .1 , 2 ,10
x 1 +2 x 2 +2 x 3=25

{
x 1 + x 2+ x3 =6
d. 3 x1 +2 x 2+ x 3=10 Ans .1 , 2 , 3
−2 x 1 +3 x 2−2 x3 =−2

{
3 x 1 +2 x 2+ 4 x 3=7
e. 2 x 1 + x 2+ x 3=4 Ans .2.25 ,−1.125 ,0.625
x 1 +3 x 2 +5 x 3=2

27
 {
−12 x 1 + x 2−8 x 3=−80
f. x 1−6 x 2 +4 x3 =13 Ans .0.6517 , 4.3166 , 9.5620
−2 x 1 −x2 +10 x 3=90
2. Use Gaussian elimination and thee-digit chopping arithmetic to solve the following linear systems, and
compare the approximations to the actual solution. The exact answers are given aside.

a. {0.03 x 1 +58.9 x 2=59.2

5.31 x 1−6.10 x 2=47.0
and {
58.9 x 1+0.03 x 2=59.2
−6.10 x 1+5.31 x 2=47.0
Ans .10.0 , 1.00 and Ans .1.00 , 10.0

{
3.03 x 1−12.1 x 2+ 14 x 3=−119
b. −3.03 x 1 +12.1 x2−7 x 3 =120 Ans .0.000 ,10.0 , 0.143
6.11 x 1−14.2 x 2+21 x 3=−139

{
3.3330 x 1+15,920 x 2 +10.333 x3 =7953
c. 2.2220 x1 +16.710 x 2+ 9.6120 7 x3 =0.965 Ans . 1.00 , 0.500 ,−1.00
−1.5611 x 1 +5.1792 x2 −1.6855 x 3=2.714

{
1.19 x 1 +2.11 x 2−100 x 3+ x 4=1.12
14.2 x1 −0.122 x 2 +12.2 x3 −x 4=3.44 Ans . 0.177 , 0.0127 ,−0.0206 ,−1.19
d.
100 x 2−99.9 x 3 + x 4=2.15
15.3 x 1 +0.110 x 2−13.1 x3 −x 4=4.16

{
π x 1−e x2 + √ 2 x3 −√ 3 x 4 =√ 11
2 2 3
π x 1+ e x 2−e x 3 + x 4 =0
7 Ans . 0.788 ,−3.12 , 0.168 , 4.56
e.
√ 5 x 1− √6 x 2 + x 3−√ 2 x 4 =π
1
π x 1 +e x 2− √ 7 x 3 + x 4= √ 2
3 2
9

3. Solve the following systems of equations by the Gauss-Siedel method. Use an accuracy of
ε =0.001. The exact answers are given aside.

{
2 x 1−x 2=124
a. −x 1+2 x 2−x 3=4 Ans . 98.5000 73.0000 43.5000
−x 2 +2 x 3=14

{
17 x 1−2 x 2−3 x 3=500
b. −5 x1 +21 x 2−2 x 3=200 Ans . 33.996318.8928 13.3839
−5 x1−5 x 2 +22 x 3 =30
4. Start from an initial guess of x 01=x 02=x 03=1.0, show the first 2 iterations of the Gauss- Seidel
method for the solution of this system of equations. Compute the error after the second
iteration. The exact answers are given aside.

28
 {
3 x1−0.1 x 2−0.2 x 3=7.85
0.1 x 1 +7 x 2−0.3 x 3=−19.3 Ans . 3.0000−2.5000 7.0000
0.3 x1 −0.2 x 2 +10 x 3=71.4
5. Solve the following set of equations using the Gauss-Seidel iterative method.
Use the starting values x(0) (0) (0)
1 =x 2 =x 3 =1.

{
5 x1 + x 2 +2 x 3=17
x1 +3 x 2+ x3 =8 Ans . 2 ,1 , 3
2 x 1 + x 2+ 6 x3 =23

6. Given the system of equations

{
−12 x 1 + x 2−x 3=−20
−2 x 1−4 x 2+ 2 x 3=10 Ans .1.0000 2.0000 10.0000
x 1 +2 x 2 +2 x 3=25
Make 2 iteration of the Gauss Seidel method to solve the system of equations taking an initial
guess of x 01=x 02=x 03=0. Compute the error after the second iteration.

7. Taking initial guess x 0= [−3

−2 ]
, approximate the other solution of the system in (Example 12a).

8. Solve the system of non-linear equations using NR-method:

{
3 2
x + y =0
a. 2 3 , The exact real solutions are ( 0 , 0 ) ∧(−1 ,1 ) .
x − y =0

{
3
x + y=1
b. 3 , The exact real solution is ( 1 , 0 ) .
y −x=−1

{ [ ]
−x 3
x +e + y =0 3
c. 2 2 , taking initial guess x 0= .
x +2 xy− y + tan x=0 −1.5

The exact real solution is ( 3.1324479604 ,−1.4699284012 )T .

{
1 3
4 x 2−20 x + y +8=0
d.
1
x y 2+ 2 x−5 y +8=0
4
, taking initial guess x 0= [ 00].
2

{(
sin (4 πxy )−2 y−x=0
e. 4 π −1 2 x
4π )
, x=0
( e −e ) + 4 e y 2−2ex =0 taking initial guess 0 0 . []

{
1
3 x−cos ( yz )− =0
2

[ ] [ ]
2 1 /10 0.50000000
2 1 , x = x =
f. x −81( y + ) +sin(z )+ 1.06=0 0 1 /10 5 0.00000000
10 −1 /10 −0.52359877
− xy 10 π −3
e + 20 z + =0
3

29
 { []
2
x + y −37=0 0
g. 2 ,
x − y −5=0 0 0x =
x+ y+ z −3=0 0

9. Find characteristic equations, eigenvalues and eigenvectors for

( )
123 4

( ) ( )
1 −3 3 a 1 0
A=(5 −1
1 3 )
B= 3 −5 3 C= 0 a 1 D= 0 2 8 −6
6 −6 4 0 0 a
0 0 3 −5
000 4
10. Find the first three iterations obtained by the power method applied to the following matrices and
compare your solutions with the exact one, given aside.

( ) ( ) ( )
1 −1 0 1 −1 0 2 −1 0
−2 4 −2 −2 4 −2 −1 2 −1
a. 0 −1 1 , b. 0 −1 2 , c. 0 −1 2
0 t 0 t 0 t
use x =( 1, 0 , 0 ) use x =( 1, 0 , 0 ) use x =( 1, 0 , 0 )

( ) ( )
4 1 1 1 4 1 −1 0
1 3 −1 1 1 3 −1 0
1 −1 2 0 −1−1 5 2
d. 1 1 0 2 e. 0 0 2 4
x 0=( 1 , 1 ,1 , 1 )t use x 0 =( 0 ,1 , 0 , 0 )t
Ans.

[ ]
0.2357
a. Eigenvalue λ=5.0000 and Eigenvector v = −0.9428
0.2357

[ ]
−0.2250
b. Eigenvalue λ=5.1249 and Eigenvector v = 0.9280
−0.2970

[ ]
−0.5000
c. Eigenvalue λ=3.4142 and Eigenvector v = 0.7071
−0.5000

[ ]
0.7795
0.4817
d. Eigenvalue λ=5.2361and Eigenvector v =
0.0920
0.3897

[ ]
0.4300
0.3280
e. Eigenvalue λ=6.2143and Eigenvector v =
−0.6242
−0.5638

30
 Chapter IV
FINITE DIFFERENCE OPERATORS
Consider a function y=f (x ) defined on(a ,b) . Here x and y are the independent and dependent
variables respectively. If the points x0 , x1 , x2 , … , xn are taken at equidistance i.e.,
x i=x 0 +ih, i=0 , 1 ,2 , … , n, then the value of y , when x=x i, is denoted as y i, where y i=f (x i) . Here,
the values of x are called arguments and the values of y are known as entries. The interval h is
called the difference interval. The differences y 1− y 0 , y 2− y 1 , y3 − y 2 ,… , y n− y n−1 are called the
first differences of the function y . They are denoted by ∆ y 0 , ∆ y 1 , ∆ y 2 , ∆ y 3… etc. That is
∆ y 0= y 1− y 0 , ∆ y 1= y 2− y1 , ∆ y 2= y 3 − y 2 , … , ∆ y n−1= y n− y n−1 …….. (1)
The symbol ∆ in Eq. (1) is called the difference operator.
A. Forward Differences
The forward difference or simply difference operator is denoted by ∆ and is defined as:
∆ f ( x )=f ( x +h ) −f (x ) …………………………….… (2)
or writing in terms of y , at x=x i, (2) becomes
∆ f ( x i )=f ( x i +h )−f ( x i ) ∨∆ y i= y i +1− y i , for i=0 , 1 , 2, … , n−1…….. (3)

31
The differences of the first differences are called the second differences and they are denoted by:
2 2 2 2
∆ y 0 , ∆ y 1 , ∆ y 2 , … , ∆ y n−2. Hence,
2
∆ y 0 =∆ y 1−∆ y 0= y 2− y 1−( y 1− y 0 )= y 2−2 y 1+ y 0 ,
2
∆ y 1=∆ y 2−∆ y 1= y 3− y 2−( y 2 − y 1 )= y 3−2 y 2 + y 1 ,
2
∆ y 2=∆ y 3−∆ y 2= y 4− y 3−( y 3− y 2) = y 4 −2 y3 − y 2 ,
…
2
∆ y n−2=∆ y n−1−∆ y n−2 = y n− y n−1−( y n−1 − y n−2 )= y n−2 y n−1 + y n−2
Similarly,
3 2 2
∆ y 0 =∆ y 1−∆ y 0 = y 3−2 y 2+ y 1−( y 2−2 y 1 + y 0 )= y 3−3 y 2+ 3 y1 − y 0
3 2 2
and ∆ y 1=∆ y 2−∆ y 1= y 4−2 y 3 + y 2−( y 3 −2 y 2 + y 1 )= y 4−3 y 3 +3 y 2− y 1
…
Generalizing, we have
∆ n+1 f ( x )=∆ [ ∆n f ( x ) ], That is, ∆ y i=∆ [ ∆ y i ] ,n=0 , 1 ,2 , …
n+1 n

f ( x )=∆ [ f ( x+ h )−f ( x ) ] =∆ f ( x +h )−∆ f ( x ) … ..(4 )

n+1 n n n
Also, ∆
and∆ n+1 y i=∆ n y i +1−∆ n y i , n=0 ,1 , 2 ,… … … … … … … … … … … … ..(5)
where ∆ 0 ≡identity operator i.e., ∆ 0 f ( x )=f (x) and ∆ 1 = Δ.
The forward differences for the six arguments x 0 , x 1 , x 2 , … , x 5 are shown in the table below (a).
Table below (a) is called a diagonal difference table or forward difference table. The first term in
the table is y 0 and is called the leading term. The differences ∆ y 0 , ∆ 2 y 0 , ∆3 y 0 , … are called the
leading differences. Similarly, the differences with fixed subscript are called forward differences.
B. Backward Differences
The backward difference operator is denoted by ∇ and it is defined as:
∇ f ( x )=f ( x )−f ( x−h ) ) ………………………..………………. (6)
Equation (6) can be written as
∇ y i= y i− y i−1 , for i=1 , 2 ,3 , … , n .………………….……….. (7)
or ∇ y 1= y 1− y 0 , ∇ y 2= y 2 − y 1 , ∇ y3 = y 3− y 2 , … , ∇ y n= y n − y n−1 ………….(8)
The differences in (8) are called first differences. The second differences are denoted by
2 2 2 2
∇ y 2 , ∇ y 3 , ∇ y 4 , … , ∇ y n.
2
Hence ∇ y 2 =∇ y 2−∇ y 1= y 2− y1 −( y 1− y 0 ) = y 2−2 y 1+ y 0

Similarly, y 3=∇ y 3−∇ y 2= y 3− y 2 −( y 2− y 1 )= y 3−2 y 2+ y 1 and so on.

32
Generalizing, we have
k k−1 k−1
∇ y i=∇ y i−∇ y i−1 ,i=1 , 2 ,3 , … , k , … , n−2, n−1 ,n …… (9)
where∇ 0 y i= y i , ∇1 y i=∇ y i .
The backward differences written in a tabular form (for the five arguments x 0 , x 1 , x 2 , … , x 5) is shown
in Table (b). In Table (b), the differences ∇ n y with a fixed subscript ‘i’ lie along the diagonal
upward sloping.
Table a: Forward difference table Table b: Backward difference table
x y ∆y 2
∆ y ∆ y
3
∆ y
4
∆ xy
5 y ∇y 2
∇ y
3
∇ y
4
∇ y
5
∇ y
x0 y0 x0 y0
∆ y0 ∇ y1
x1 y1 ∆ y0
2 x1 y1 2
∇ y2
∆ y1 ∆ y0
3 ∇ y2 3
∇ y3
x2 y2 ∆ y1
2
∆ y0
4 x2 y2 2
∇ y3
4
∇ y4
∆ y2 ∆ y1
3 5
∆ y0 ∇ y3 3
∇ y4
5
∇ y5
x3 y3 ∆ y2
2
∆ y1
4 x3 y3 2
∇ y4
4
∇ y5
∆ y3 ∆ y2
3 ∇ y4 3
∇ y5
x4 y4 ∆ y3
2 x4 y4 2
∇ y5
∆ y4 ∇ y5
x5 y5 x5 y5

C. Central Differences
The central difference operator is denoted by the symbol δ and is defined by
h
δ f ( x )=f x+ −f x− )
2
h
2 ( ) ( )
where h is the interval of differencing.
In terms of y, the first central difference is written as: δ yi = y 1 − y 1 ……………… (10) i+ i−
2 2

2
( h2 ) and y
where y i+ 1 =f x i + i−
h
2
.
1
2
=f x i− ( )
Hence δ y 1 = y 1− y 0 , δ y 3 = y 2− y 1 , … , δ y n− 1 = y n− y n−1
2 2 2

The second central differences are given by

2
δ y i=δ y 1 −δ y 1 =( y i +1− y i )−( yi − y i−1 )= y i+ 1−2 y i + y i−1
i+ i−
2 2
n n−1 n−1
Generalizing, δ yi =δ y
i+
1 −δ y
i−
1 …………………………………… (11)
2 2

The central difference table for the seven arguments x 0 , x 1 , x 2 , … , x 6 is shown in Table below.

33
x y δ δ
2
δ
3
δ
4
δ
5
δ
6

x0 y0
δy 1 Table: Central difference table
2
x1 y1 2
δ y1 It is noted in table above that all odd differences
δy 3 3
δ y3 have fraction suffices and all the even
2 2 differences are with integral suffices.
x2 y2 2
δ y2
4
δ y2
δy 5 3
δ y5
5
δ y5
2 2 2
x3 y3 2
δ y3
4
δ y3
6
δ y3
δy 7 3
δ y7
5
δ y7
2 2 2
x4 y4 2
δ y4
4
δ y4
δy 9 3
δ y9
2 2
x5 y5 2
δ y5
δy 11 D. Shift operator (E)
2
x6 y6 The shift operator is defined as: Ef ( x ) =f ( x +h )
or E y i= y i +1………. (12)
Hence, shift operator sifts the function value y i to the next higher value y i+ 1. The second shift
operator gives E2 f ( x )=E [ Ef ( x ) ] =E [ f ( x +h ) ] =f (x +2 h) …………….. (13)
E is linear and obeys the law of indices.
Generalizing, En f ( x )=f ( x+ nh )∨E n y i= y i+nh ………………..………… (14)
The inverse shift operator E–1 is defined as: E−1 f ( x )=f ( x−h ) ………… (15)
In a similar manner, second and higher inverse operators are given by
−2 −n
E f ( x )=f ( x−2 h )∧hence E f ( x ) =f ( x−nh )
The more general form of E operator is given by: Er f ( x ) =f ( x +rh ) , where r ∈ R .……(16)
Example 4:
(a) Construct the forward difference table and the backward x 1 2 3 4 5
difference table for the following data: y 4 6 9 12 17
(b) Construct a forward difference table for y = f (x) = x3 + 2x + 1 for x = 1, 2, 3, 4, 5.
Obtain the backward differences for the function f (x) = x3 from x = 1 to 1.05 to two decimals chopped.
Solution: (a) The forward and backward difference tables are shown in tables below.

34
x y ∆ y ∆2 y ∆3 y ∆4 y x y ∇ y ∇2 y ∇3 y ∇4 y
1 4 1 4
2 2
2 6 1 2 6 1
3 1 3 1
3 9 0 3 3 9 0 3
3 2 3 2
4 12 2 4 12 2
5 5
5 17 5 17

b) Forward difference table. (d) backward differences

5
x y ∆ y ∆2 y ∆3 y ∆4 y x y ∇y ∇ y
2 3
∇ y
4
∇ y ∇ y
1 4 1.00 1.000
9 0.030
2 13 12 1.01 1.030 0.001
21 6 0.031 -0.001
3 34 18 0 1.02 1.061 0.000 0.002
39 6 0.031 0.001 -0.003
4 73 24 1.03 1.092 0.001 -0.001
63 0.032 0.000
5 136 1.04 1.124 0.001
0.033
1.05 1.157

Relation between the Operators

In order to develop approximations to differential equations, following summary of operators is
useful. For linking different operators, we consider Taylor’s formula:
' 1 2 ''
f ( x +h )=f ( x ) +h f ( x )+ h f ( x ) +…
2!
where h is the difference interval.
Operators Notations
1. Shifting Ef ( x ) =f (x +h)
2. Forward Difference ∆ f ( x )=f ( x +h ) −f (x )
3. Backward Difference ∇ f ( x )=f ( x )−f ( x−h)
4. Central Difference δf ( x ) =f ( x +h/2 )−f (x−h/2)
5. Mean (Averaging) Operator μf ( x )=1/2 [ f ( x+ h/2 ) +f (x−h/2) ] *
'
6. Differential Operator Df ( x )=f (x) *
35
 x+h
7. Integral Operator
Jf ( x )= ∫ f ( x ) dx *
x
Here h is the difference interval. It can be shown that
r E−1 1/ 2 E−1
1. E f ( x )=f ( x+ r h ) , r ∈ R 2. E=∆+1 3. ∇= 4. δ E =∆5. δ= 1/ 2
E E

The relationships among the various operators are shown in Table below.
E ∆ ∇ δ

(√ 1+ 14 δ )
E E ∆ +1 (1−∇)
−1
1
1+ δ 2+ δ 2
2

√
∆ E−1 ∆ (1−∇) −1
−1

δ + δ (1+ δ )
1 2 1 2
2 4

√ 4
∇ −1 −1
∇
δ + δ (1+ δ )
1−E 1−(∆+ 1) −1 2 1 2
2
δ 1 /2
E −E
−1 /2
∆ (∆+ 1)
−1 /2
∇ (1−∇)
−1 /2
δ

Note: If f (x)=a0 +a 1 x +a2 x 2+ …+a n x n is a polynomial of degreen , ∆ n f ( x )=n ! an hn with∆ n f ( x )=0,

∀ n>n .

Example 5:

( )
2
∆ 2
1. Evaluate: a. x b. ∆ sin x c. ∆ log x
E
Solutions:

( ) ( ) ( )
2
∆ 2 2 ( E−1 ) 2 E2−2E+1 2
a. x= x= x
E E E
¿ ( E−2+ E−1 ) x 2=E x 2−2 x2 + E−1 x 2
2 2 2 2
¿ ( x +h ) −2 x + ( x−h ) =2 h

b. ∆ sin x=sin ( x +h )−sin x=2 cos

x +h+ x
2
sin (
x +h−x
2
h
=2 cos 1+ sin
2
h
2) ( ) ( ) ()
x +h h
c. ∆ log x=log ( x+ h )−log x=log =log (1+ )
x x

( ) ( )
2 2 x
∆ ∆ x Ee
4 Find the following: a . ∆ 2 ( 3 e x ) b. x c. e 2 x
E E ∆ e
Solution
a . ∆ 2 ( 3 e x ) =∆ ( ∆ 3 e x )=∆ ( 3 e x+h−3 e x )
¿ ( 3 e x +2h −3 e x )−( 3 e x+h−3 e x )
2
¿ 3 e ( e −2 e +1 ) =3 e ( e −1 )
x 2h h x h

( )
2
∆
2
( E−1 ) −1
b. x= x=Ex−2 x+ E x =( x+ h )−2 x+ ( x −h )=0
E E

36
 ( ) ( )
2
∆2 x E e x ( E−1 ) x e x+h
c. e 2 x= e
E ∆ e E x+h
∆(e −e )
x

x+h x+h x+ h −1 x+h

e E e −2 e + E e
¿ ( E−2+ E ) e
−1 x x
= e
( e −2 e + e )
x+2 h x+ h x x+2 h x+h
(e −2 e +e )
x

( )
2 x+2 h 2 x+h 2x x+2 h x+h x
e −2 e +e e −2e +e x x 2
∆ x Ee
x
¿ = e =e or simply, e =e
x
(e −2 e + e ) ( e x+2 h−2e x+h +e x )
x+2 h x+h x 2 x
E ∆ e
3
5 Find∆ ( 1−3 x ) ( 1−2 x ) (1−x ).
Solution: Let f ( x )= (1−3 x )( 1−2 x ) (1−x )=−6 x 3 +11 x 2−6 x+ 1.

Here, f ( x ) is a polynomial of degree three and the coefficient of x 3 is(−6). Hence

3
∆ f ( x ) =(−6 ) 3 !=−36

37