NUMERICAL METHODS

Solution of Systems of Linear Equations

Non Iterative Method
Department of Chemical Engineering
Faculty of Industrial Technology
Parahyangan Catholic University
March 2024
 Introduction

Roots of a single equation: f ( x)  0

A general set of
equations:
 f 1 ( x1 , x 2 ,  x n )  0

- n equations,
 f 2 ( x1 , x 2 ,  x n )  0
- n unknowns. 
  
 f n ( x1 , x 2 ,  x n )  0

2
Linear Algebraic Equations
a11 x1  a12 x2    a1n xn  b1

a 21 x1  a 22 x2    a 2 n xn  b2

 
a n1 x1  a n 2 x2    a nn xn  bn

Nonlinear Equations
a11 x1  a12 x1 x2    a1n ( xn ) 5  b1

a 21 ( x1 ) 3  a 22 e x2    a 2 n ( x2 ) 3 / xn  b2

 
a x  a x    a x  b
 n1 1 n2 2 nn n n
3
 Solving Small Numbers of Equations

There are many ways to solve a system of linear

equations:
• Graphical method
• Cramer’s rule For n ≤ 3
• Method of elimination
• Numerical methods for solving larger number of
linear equations:
- Gauss elimination
- LU decompositions and matrix inversion
4
 Gauss Elimination Method

• It is a formalized way of the previous elimination

technique to large sets of equations by developing a
systematic scheme or algorithm to eliminate
unknowns and to back substitute.

• As in the case of the solution of two equations, the

technique for n equations consists of two phases:
1. Forward elimination of unknowns.
2. Back substitution.

5
 Gauss Elimination Method
• Consider the following system of n equations.

a11x1 + a12x2 + ... + a1nxn = b1 (1)

a21x1 + a22x2 + ... + a2nxn = b2 (2)
...
an1x1 + an2x2 + ... + annxn = bn (n)

Form the augmented matrix of [A|B].

Step 1 : Forward Elimination: Reduce the system to an upper triangular

system.

1.1- First eliminate x1 from 2nd to nth equations.

- Multiply the 1st eqn. by a21/a11 & subtract it from the 2nd equation.
This is the new 2nd eqn.
- Multiply the 1st eqn. by a31/a11 & subtract it from the 3rd equation.
This is the new 3rd eqn.
...
- Multiply the 1st eqn. by an1/a11 & subtract it from the nth equation.
This is the new nth eqn.
6
Gauss Elimination Method (cont’d)
Note:
- In these steps the 1st eqn is the pivot equation and a11 is
the pivot element.
- Note that a division by zero may occur if the pivot
element is zero. Naive-Gauss Elimination does not check
for this.
a11 a12 a13  a1n   x 1  b1 
0 a22 a23  a2n   x 2  b2 
    
The modified system is  0 a32 a33  a3n   x 3   b3 
 
          
   
indicates that the  0 an2 an3  ann   x n  bn 
system is modified once.

7
Gauss Elimination Method (cont’d)

1.2- Now eliminate x2 from 3rd to nth equations.

 a 11 a 12 a 13  a 1n   x 1  b1 
 0 a 22 a 23  a 2 n   x  b  
The modified system is    2   2 
 0 0 a 33
  a 3n   x 3   b 3 
       
      
   
 0 0 a n3   
a nn  x n  b n 

Repeat steps (1.1) and (1.2) upto (1.n-1).

 a11 a12 a13  a1n   x 1   b1 
we will get this upper 0 
a 22 
a 23  a 2 n  x   b  
  2   2 
triangular system    a3n 
0 0 a33  x 3    b 3 
       
      
   ( n 1) 
 0 0 0 ( n 1) 
0 a nn   x n  b n 
8
 Gauss Elimination Method (cont’d)
Step 2 : Back substitution
Find the unknowns starting from the last equation.
1. Last equation involves only xn. Solve for it. ( n 1)
b
xn  n
( n 1)
a nn
2. Use this xn in the (n-1)th equation and solve for xn-1.
...
3. Use all previously calculated x values in the 1st eqn
and solve for x1.
n
bi(i 1)   ij x j
a i 1

j i 1
xi  ( i 1)
for i  n-1, n-2, ..., 1
a ii 9
Summary of Gauss Elimination Method

10
Gauss Elimination Method
Example 1

Solve the following system using Naive Gauss Elimination.

6x1 – 2x2 + 2x3 + 4x4 = 16

12x1 – 8x2 + 6x3 + 10x4 = 26
3x1 – 13x2 + 9x3 + 3x4 = -19
-6x1 + 4x2 + x3 - 18x4 = -34

Step 0: Form the augmented matrix

6 –2 2 4 | 16
12 –8 6 10 | 26 R2-2R1
3 –13 9 3 | -19 R3-0.5R1
-6 4 1 -18 | -34 R4-(-R1)
11
 Gauss Elimination Method
Example 1 (cont’d)
Step 1: Forward elimination

1. Eliminate x1 6 –2 2 4 | 16 (Does not change. Pivot is 6)

0 –4 2 2 | -6
0 –12 8 1 | -27 R3-6R2
0 2 3 -14 | -18 R4-(-0.5R2)

2. Eliminate x2 6 –2 2 4 | 16 (Does not change.)

0 –4 2 2 | -6 (Does not change. Pivot is-4)
0 0 2 -5 | -9
0 0 4 -13 | -21 R4-2R3

3. Eliminate x3 6 –2 2 4 | 16 (Does not change.)

0 –4 2 2 | -6 (Does not change.)
0 0 2 -5 | -9 (Does not change. Pivot is 2)
0 0 0 -3 | -3

12
 Gauss Elimination Method
Example 1 (cont’d)
Step 2: Back substitution

Find x4 x4 =(-3)/(-3) = 1

Find x3 x3 =(-9+5*1)/2 = -2

Find x2 x2 =(-6-2*(-2)-2*1)/(-4) = 1

Find x1 x1 =(16+2*1-2*(-2)-4*1)/6= 3

13
 Gauss Elimination Method Example 2
(Using 6 Significant Figures)
3.0 x1 - 0.1 x2 - 0.2 x3 = 7.85
0.1 x1 + 7.0 x2 - 0.3 x3 = -19.3 R2-(0.1/3)R1
0.3 x1 - 0.2 x2 + 10.0 x3 = 71.4 R3-(0.3/3)R1

Step 1: Forward elimination

3.00000 x1- 0.100000 x2 - 0.200000 x3 = 7.85000

7.00333 x2 - 0.293333 x3 = -19.5617
- 0.190000 x2 + 10.0200 x3 = 70.6150

3.00000 x1- 0.100000 x2 - 0.20000 x3 = 7.85000

7.00333 x2 - 0.293333 x3 = -19.5617
10.0120 x3 = 70.0843

14
 Gauss Elimination Method Example 2 (cont’d)

Step 2: Back substitution

x3 = 7.00003
x2 = -2.50000
x1 = 3.00000

Exact solution:
x3 = 7.0
x2 = -2.5
x1 = 3.0

15
 Pitfalls of Gauss Elimination Methods

1. Division by zero
2 x2 + 3 x3 = 8
a11 = 0
4 x1 + 6 x2 + 7 x3 = -3
(the pivot element)
2 x1 + x2 + 6 x3 = 5

It is possible that during both elimination and back-

substitution phases a division by zero can occur.
2. Round-off errors
In the previous example where up to 6 digits were kept
during the calculations and still we end up with close to
the real solution.
x3 = 7.00003, instead of x3 = 7.0

16
 Pitfalls of Gauss Elimination (cont’d)
3. Ill-conditioned systems
x1 + 2x2 = 10
1.1x1 + 2x2 = 10.4  x1 = 4.0 & x2 = 3.0

x1 + 2x2 = 10
1.05x1 + 2x2 = 10.4  x1 = 8.0 & x2 = 1.0

Ill conditioned systems are those where small changes in

coefficients result in large change in solution. Alternatively, it
happens when two or more equations are nearly identical,
resulting a wide ranges of answers to approximately satisfy the
equations. Since round off errors can induce small changes in the
coefficients, these changes can lead to large solution errors.
17
 Pitfalls of Gauss Elimination (cont’d)
4. Singular systems.
• When two equations are identical, we would loose one
degree of freedom and be dealing with case of n-1
equations for n unknowns.

To check for singularity:

• After getting the forward elimination process and
getting the triangle system, then the determinant for
such a system is the product of all the diagonal
elements. If a zero diagonal element is created, the
determinant is Zero then we have a singular system.
• The determinant of a singular system is zero.

18
 Techniques for Improving Solutions
1. Use of more significant figures to solve for the
round-off error.
2. Pivoting. If a pivot element is zero, elimination step
leads to division by zero. The same problem may arise,
when the pivot element is close to zero. This Problem
can be avoided by:
 Partial pivoting. Switching the rows so that the
largest element is the pivot element.
 Scaled partial pivoting
 Complete pivoting. Searching for the largest element
in all rows and columns then switching.
3. Scaling
 Solve problem of ill-conditioned system.
 Minimize round-off error 19
Example : Without Pivoting
 Pivoting strategy
• To determine the smallest such that

and perform

 Partial pivoting
dramatic enhancement!
Effect of partial pivoting
Example: Partial Pivoting

2x 2 x 4 0
2x 1 2x 2 3x 3 2 x 4  2
4x 1 3x 2 x 4  7
6x 1 x 2 6x 3 5x 4 6

a) Forward Elimination

0 2 0 1 0  6 1 6 5 6
   
2 2 3 2 2  2 2 3 2 2 

R 4
R 1

 4 3 0 1 7   4 3 0 1 7 
   
 6 1 6 5 6   0 2 0 1 0 
24
 Example: Partial Pivoting (cont’d)
 6 1 6 5 6
 
2 2 3 2 2  R 2  0.33333  R 1
 4 3 0 1 7  R 3  0.66667  R 1
 
 0 2 0 1 0 
6 1 6 5 6 
 
0 1.6667 5 3.6667 4 

R3
R 2 

0 3.6667 4 4.3333 11
 
0 2 0 1 0 
6 1 6 5 6 
 
 0 3.6667 4 4.3333 11
 0 1.6667 5 3.6667 4 
 
 0 2 0 1 0 
 Example: Partial Pivoting (cont’d)
6 1 6 5 6 
 
0 3.6667 4 4.3333 11
0 1.6667 5 3.6667 4  R 3  0.45455  R 2
 
0 2 0 1 0  R 4  0.54545  R 2

6 1 6 5 6 
 
0 3.6667 4 4.3333 11 
0 0 6.8182 5.6364 9.0001
 
0 0 2.1818 3.3636 5.9999  R 4  0.32000  R 3

6 1 6 5 6 
 
0 3.6667 4 4.3333 11 
0 0 6.8182 5.6364 9.0001
 
0 0 0 1.5600 3.1199 
26
 Example: Partial Pivoting (cont’d)

6 1 6 5 6 
 
 0 3.6667 4 4.3333 11 
0 0 6.8182 5.6364 9.0001
 
 0 0 0 1.5600 3.1199 

b) Back Substitution
3.1199
x4   1.9999
1.5600
9.0001  5.6364  1.9999 
x3   0.33325
6.8182
11  4.3333  1.9999   4  0.33325
x2   1.0000
3.6667
6  5  1.9999   6  0.33325   11.0000 
x1   0.50000 27
6
 1 0 0 0.04000 0.58000
 
0 1 0 0.27993 1.5599 
0 0 1 0.82667 1.3200 
 
0 0 0 1.5599 3.1197 
Divide the 4th row by 1.5599 and create zero above the diagonal in the fourth
column.

1 0 0 0 0.49999 
 
0 1 0 0 1.0001 
0 0 1 0 0.33326 
 
0 0 0 1 1.9999 

Note: Gauss-Jordan method requires almost 50 % more operations than

Gauss elimination; therefore it is not recommended to use it.
28