Laboratory Manual of Simplified Numerical Analysis (C++ Version)

Accessible through: https://zenodo.org/record/8264208

Cite as:
Ali, Amjad, Ishaq, Muhammad, Farooq, Hamayun, & Umar, Muhammad. (2023).
Laboratory Manual of Simplified Numerical Analysis (C++ Version). Zenodo.
https://doi.org/10.5281/zenodo.8264208

For availability of the codes, please visit:

GitHub - DrAmjadAli11/SimplifiedNumericalAnalysis
https://github.com/DrAmjadAli11/SimplifiedNumericalAnalysis

Principal Book

Simplified Numerical Analysis

Fourth Edition
www.TimeRenders.com.pk

Companion Books

Laboratory Manual of Simplified Numerical Analysis (C++ Version)

Laboratory Manual of Simplified Numerical Analysis (MATLAB® Version)
Laboratory Manual of Simplified Numerical Analysis (Python Version)
 Laboratory Manual
of
Simplified Numerical Analysis
(C++ Version)
Fourth Edition

Amjad Ali, Ph.D.

Bahauddin Zakariya University (BZU), Multan

Muhammad Ishaq, Ph.D.

COMSATS University Islamabad, Vehari Campus

Hamayun Farooq, Ph.D.

Government Degree College, Muzaffar Garh

Muhammad Umar, Ph.D.

University of Heidelberg, Heidelberg

Esteemed Panel of the Supporters

Ms. Aniqa Faizan, Bahauddin Zakariya University, Multan

Ms. Amna Waheed, Bahauddin Zakariya University, Multan
Dr. Zainab Bukhari, Times Institute, Multan
Ms. Tayyaba Saeed, Bahauddin Zakariya University, Multan
Ms. Syeda Zahra Kazmi, Bahauddin Zakariya University, Multan
Ms. Zoha Saleem, Bahauddin Zakariya University, Multan
Laboratory Manual of Simplified Numerical Analysis (C++ Version)

A Companion book of the principal book:

Simplified Numerical Analysis (Fourth Edition)
©2023, Amjad Ali, Ph.D. (The Principal Author)
ISBN: 978-969-7821-14-3

Typeset: Mostly personally by Dr. Amjad Ali (The Principal Author), also contributed by the supporters.

Title Design: Mr. Muhammad Rizwan Qadeer (mrizwanqadeer@gmail.com)

The proofreading is powered by various AI-driven proprietary software.

Publishers www.timerender.com.pk

Distribution Point:
238-B (PRIDE), Near Girls Comprehensive School, Gulgasht Colony, Multan, Pakistan.
Cell Phone: +923486981925, timerenderpublishers@gmail.com

ii
Table of Contents
Chapter 1: Preliminary Concepts in Numerical Analysis (1)
1.1 Introduction
1.2 Number Systems and Representations
1.3 The Round-off Error
1.4 The Truncation Error

Computing Resources ............................................................................................................................... 1

Chapter Summary ..................................................................................................................................... 6
Chapter Excercises .................................................................................................................................... 8

Chapter 2: Solution of a Nonlinear Equation in One Variable (9)

Corridor I: BASICS
2.1 Introduction
2.2 Bracketing Methods
2.2.1 The Bisection Method (or Bolzano Method)
2.2.2 The False-Position Method (or Regula-Falsi Method)
2.3 Open Methods
2.3.1 The Fixed-Point Iteration Method
2.3.2 The Newton-Raphson Method
2.3.3 The Secant Method
Corridor II: ANALYSIS
2.4 Convergence Analysis
The Bisection Method
The Regula-Falsi Method
The Secant Method
The Newton-Raphson Method
The Fixed-Point Iteration Method
2.5 Further Discussions
Corridor III: PROGRAMMING ARCADE
2.6 Algorithms and Implementations ................................................................................................. 11
The Newton-Raphson Method ......................................................................................... 11
The Fixed-Point Iteration Method .................................................................................... 16
The Secant Method ......................................................................................................... 18
The Bisection Method ..................................................................................................... 20
The Regula-Falsi Method ................................................................................................. 23
Chapter Summary ................................................................................................................................... 28
Chapter Excercises .................................................................................................................................. 32

iii
Chapter 3: Polynomial Interpolation (37)
Corridor I: BASICS
3.1 Introduction
3.2 The Newton’s Divided Difference Interpolation
3.3 The Lagrange Interpolation
3.4 Deriving the Lagrange Interpolation Formula from the Newton’s Divided-Difference Formula
3.5 Interpolation Formulas for Equally Spaced Nodes
3.6 Hermite Interpolation
3.7 Spline Interpolation
3.7.1 Linear Spline
3.7.2 Quadratic Spline
3.7.3 Cubic Spline
Corridor II: ANALYSIS
3.8 Error of Interpolation
Corridor III: PROGRAMMING ARCADE
3.9 Algorithms and Implementations ................................................................................................. 38
The Newton's Divided Difference Interpolation Formula .............................................. 39

Chapter Summary ................................................................................................................................... 42

Chapter Excercises .................................................................................................................................. 45

Chapter 4: Numerical Integration (49)

Corridor I: BASICS
4.1 Introduction
4.2 The Trapezoidal Rule
4.3 The Simpson’s 1/3 Rule
4.4 Generalized Closed Newton-Cotes Quadrature
Corridor II: ANALYSIS
4.5 Truncation Error of the Trapezoidal Rule
4.6 Truncation Error of the Simpson’s 1/3 Rule
4.7 Further Discussions
4.8 The Gaussian Quadrature
Corridor III: PROGRAMMING ARCADE
4.9 Algorithms and Implementations ................................................................................................. 53
The Composite Trapezoidal Rule .................................................................................... 53
The Composite Simpson’s 1/3 Rule ................................................................................ 56
The Composite Simpson’s 3/8 Rule ................................................................................ 58

Chapter Summary ................................................................................................................................... 61

Chapter Excercises .................................................................................................................................. 64

iv
Chapter 5: Numerical Differentiation (69)
5.1 Introduction
5.2 Finite Difference Approximations of Derivatives using the Taylor Series
5.2.1 First Order Derivatives
5.2.2 Second Order Derivatives
5.3 Listing of the Derivative Formulas

Chapter 6: Direct Linear Solvers (71)

Corridor I: BASICS
6.1 Introduction to Linear Systems
6.2 Solving Linear Systems using the Gaussian Elimination Method
6.3 Pivoting Strategies
Partial Pivoting
Scaled Partial Pivoting
Complete Pivoting
6.4 The Gauss-Jordan Method
6.5 Solving Linear Systems using the LU Factorization Method
6.5.1 The Doolittle’s Method
6.5.2 The Crout’s Method
6.5.3 The Cholesky’s Method
Corridor II: ANALYSIS
6.6 Operation Count Analysi
6.7 Matrix Inversion
Corridor III: PROGRAMMING ARCADE
6.8 Algorithms and Implementations ................................................................................................. 73
The Guassian Elimination Method with Partial Pivoting ................................................ 74
Solving AX = B using the Doolittle's Method ................................................................ 77
Solving AX = B using the Crout's Method .................................................................... 81
Solving AX = B using the Cholesky's Method ............................................................... 86
Performing Operation Count Analysis ........................................................................... 89

Chapter Summary ................................................................................................................................. 101

Chapter Excercises ................................................................................................................................ 103

Chapter 7: Iterative Linear Solvers (107)

Corridor I: BASICS
7.1 Vector Norms and Distances
7.2 Convergence Criteria for Linear Solvers
7.3 Basic Methods
7.3.1 The Jacobi Method
7.3.2 The Gauss-Seidel Method

v
 7.3.3 The SOR Method
Corridor II: ANALYSIS
7.4 Matrix Norms and Conditioning
7.5 Iteration Matrix and Matrix Form of a Solver
Corridor III: PROGRAMMING ARCADE
7.6 Algorithms and Implementations ............................................................................................... 108
The Jacobi Method ...................................................................................................... 109
Modification in the Jacobi Method's algorithm for the Gauss-Seidel Method ........... 110
Modification in the Jacobi Method's algorithm for the SOR Method ......................... 110

Chapter Summary ................................................................................................................................. 114

Chapter Excercises ................................................................................................................................ 118

Chapter 8: Eigenvalues and Eigenvectors (119)

Corridor I: BASICS
8.1 Basic Definitions and Concepts
8.2 General Approach of Finding Eigenvalues and Eigenvectors
8.3 Some Numerical Methods for Eigenvalues
The Power Method
The Householder Method
The QR Factorization Method
The Sturm Method
Corridor II: ANALYSIS
8.4 Further Discussions
The Power Theorem
The Gerschgorin Circle Theorems
The Singular Value Decomposition (SVD)
Corridor III: PROGRAMMING ARCADE
8.5 Algorithms and Implementations ............................................................................................... 120
The Power Method ...................................................................................................... 121

Chapter Summary ................................................................................................................................. 124

Chapter Excercises ................................................................................................................................ 125

Chapter 9: Numerical Solution of Ordinary Differential Equations (ODEs)

(127)
Corridor I: BASICS
9.1 Introduction
9.2 Solving IVPs using Single Step Methods and Multistep Methods
The Euler Method

vi
 The Mid-point Method (an RK2 method of Order 2)
The Modified/Improved Euler Method (an RK2 method of Order 2)
The RK Method of order 4 (RK4)
9.3 Solving IVPs using Predictor-Corrector Methods
The Adams-Bashforth-Moulton Method of Order 4
9.4 Solving Systems of ODEs and Higher Order ODEs
Using the Classical RK4 Method
9.5 Solving Linear BVPs using the Finite Difference Method
Corridor II: ANALYSIS
9.6 Some Theoretical Concepts and Error Analysis
Corridor III: PROGRAMMING ARCADE
9.7 Algorithms and Implementations ............................................................................................... 129
Euler method ................................................................................................................ 130
Mid-point method ........................................................................................................ 135
Modified/Improved Euler method ............................................................................... 138
RK method of order 4 (RK4) ........................................................................................ 142
Adams-Bashforth method of order 4 ........................................................................... 145
Adams-Bashforth-Moulton method of order 4 ........................................................... 149
RK4 method for a system of two ODEs ........................................................................ 154
RK4 method for a system of three ODEs ..................................................................... 156
RK4 method for Second Order ODE ............................................................................. 160
RK4 method for Third Order ODE ................................................................................ 163
Linear FDM for BVP ...................................................................................................... 167

Chapter Summary .................................................................................................................................. 172

Chapter Excercises ................................................................................................................................ 173

Bibliography .......................................................................................................................................... 177

The End ................................................................................................................................................. 178

vii
Numerical

Analysis

is

the

mathematics

of

Scientific

Computing

viii
 Chapter 1
Preliminary Concepts
in Numerical Analysis
1.1 Introduction
1.2 Number Systems and Representations
1.3 The Round-off Error
1.4 The Truncation Error

To unleash the topics of this Chapter, please delve into the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

Computing Resources
The numerical methods are devised just to be used on computers. It makes no sense to study a
numerical method without considering its practicality using some computing tools. A variety of
numerical computing tools, both freeware and proprietary, are available. The students are advised to
understand the algorithmic (step-by-step) style of the numerical methods they learn. This book
suggests the following resources for beginners.

(1) C++: The numerical methods can be programmed in any programming language, especially
C++, FORTRAN, and Python. The book discusses a wide variety of C++ programs of the
numerical methods in this book. One can modify the as per need. Several C++ IDEs
(Integrated Development Environments) are available, such as Dev-C++, and Code::Blocks for
Windows and GNU-C++ for Linux operating system. One can even find C++ Apps (apps is an
acronym for computer application software) for Android or iOS devices. Some online C++
IDEs are also available, which can be used for executing C++ programs without installing
them.
1
Preliminary Concepts in Numerical Analysis 2

(2) Python: There are several free Python IDEs available for the Desktop use (such as Spyder,
Jupyter, and PyCharm) or On-line use (such as Google Colab). It is quite a pertinent skill of the
day that the students of computational sciences are familiar with programming in Python.
The companion website of this book (www.timerender.com.pk) shares a Python Library
having a variety of codes for the numerical methods discussed in this book.

(3) MATLAB®: It is a proprietary software, by The MathWorks, Inc., available in both Desktop and
Online versions. MATLAB® offers a wide variety of built-in functions and programming
capabilities for mathematical computations (both symbolic and numeric, although more
suitable and expert for numeric computations), for all modern areas of science and
engineering. The book discusses a wide variety of MATLAB ® programs and MATLAB® built-in
functions for the numerical methods in this book.

(4) GNU-Octave: It is an open-source (and freeware) version of MATLAB®, available in both

Desktop and Online versions. Most of the MATLAB ® codes and built-in functions discussed in
this book can be executed in GNU-Octave and Octave-online.

(5) MATHEMATICA®: It is a proprietary software by Wolfram Research. It is one of the best

Computer Algebra Systems (CAS) available. It offers an extensive variety of built-in functions
and programming capabilities for mathematical computations (both symbolic and numeric),
for all modern areas of science and engineering.

(6) MAPLE®: It is a proprietary software by Maplesoft for mathematical computations (both

symbolic and numeric), for all modern areas of science and engineering. It is also one of the
best Computer Algebra Systems (CAS).

(7) Spread-Sheet: A spread-sheet software (such as Excel by Microsoft®) can be used for
computations involved in simple numerical methods. The companion website of this book
(www.timerender.com.pk) may shares a spread-sheet workbook having a variety of sheets
for most of the numerical methods discussed in this book.

(8) Various Math Solver Tools: Wolfram|Alpha, Symbolab, and Microsoft® Math Solver are
three of the advanced tools for math education to be used as calculators. These are
extensive, feature-rich, online tools, accessible both through the web browser and the
relevant android/iOS apps. These tools provide automated step by step solutions to algebra
and calculus problems covering from middle school through college. The premier versions of
these tools are freely available, whereas professional (pro) versions are not free.

(9) Various Other Online Tools/Websites: There are various other online tools and websites that
offer basic computing facilities for numerical and symbolic computations. Examples include:
• AtoZmath.com [https://atozmath.com/]
• CalculatorSoup® [https://www.calculatorsoup.com/].
• Keisan - CASIO® [https://keisan.casio.com/]
∎∎∎
3 Simplified Numerical Analysis

Question 06: What are the significant figures (or significant digits) of an approximate number?

Significant figures of a number (that approximates a true value) are the digits that are used to
express the number meaningfully. The significant figures are counted for a number that
approximates some other number to express the degree of precision in the approximate number.
The significant figures begin with the leftmost nonzero digit and end with the rightmost correct
digit. The rightmost zeros, which are exact, are also significant. That is,

• All the nonzero digits (i. e. , 1, 2, 3, ⋯ ,9) are significant.

• Zeroes appearing anywhere between two nonzero digits are significant (e.g., in
3005.00102 there are nine significant digits).

• Leading zeros (i.e., left to the first nonzero digit) are not significant (e.g., the number
0.000081 has only two significant digits, namely 8 and 1). The leading zeros are used to
fix the decimal place.

• Trailing zeroes are significant if they are exact with regard to some true value. Trailing
zeros may or may not be significant. It depends on the context; how the number is
approximated or obtained by rounding-off some other number.
∎
Remark: The significant figures of a number can easily be identified by using its normalized
scientific notation. The digits in the fractional part (or mantissa) are regarded as significant
figures. For example, each of the numbers 42.134, 6.0013, and 0.0015784 has five significant
figures, which can be identified easily by converting these numbers into their normalized
scientific notation as:

42.134 = 0.42134 × 102

6.0013 = 0.60013 × 10
0.0015784 = 0.15784 × 10−2
∎

Remarks:

• 6500 has 2 significant figures (i.e., the digits 6 and 5) if it has been obtained by rounding-off a
number to the nearest 100 (e.g., by rounding-off the numbers 6497 or 6543.88 to the nearest
hundred). In fact, any number in the interval (6450, 6550) gives 6500, when rounded to the
nearest 100.

• 6500 has 3 significant figures (i.e., the digits 6, 5, and the following 0) if it has been obtained
by rounding-off a number to the nearest 10 (e.g., by rounding-off the numbers 6497 or
6504.99 to the nearest ten). In fact, any number in the interval [6495, 6505] gives 6500,
when rounded to the nearest 10.
Preliminary Concepts in Numerical Analysis 4

• 6500 has 4 significant figures if it has been obtained by rounding-off a number to the nearest
whole number (e.g., by rounding-off the numbers 6499.8 or 6500.47 to the nearest whole
number). In fact, any number in the interval [6499.5, 6500.5] gives 6500, when rounded to
the nearest whole number.

• 70500 has at least 3 significant figures (i.e., the digits 7, 5, and the 0 in between these).
Depending upon the context, as just explained, it may have 3 to 5 significant figures.

• 0.00364300 has 4 significant figures (i.e., the digits 3, 6, 4, and 3) if it has been obtained by
rounding-off a number to 4 significant figures (e.g., by rounding-off the numbers
0.003642859 or 0.0036432099 to 4 significant figures). Usually, in that case, the
approximate number is written as 0.003643, without any non-significant trailing zero. In fact,
any number in the interval [0.0036426,0.0036435) gives 0.003643, when rounded to 4
significant figures.

• 0.00364300 has 5 significant figures (i.e., the digits 3, 6, 4, 3, and the following 0) if it has
been obtained by rounding-off a number to 5 significant figures (e.g., by rounding-off the
numbers 0.003642978001 or 0.003643049 to 5 significant figures). Usually, in that case, the
approximate number is written as 0.0036430, without any non-significant trailing zero. In
fact, any number in the interval [0.00364295,0.00364306) gives 0.0036430, when rounded
to 5 s.f.

• 0.00364300 has 6 significant figures (i.e., the digits 3, 6, 4, 3, and the following two 0s) if it
has been obtained by rounding-off a number to 6 significant figures (e.g., by rounding-off the
numbers 0.003642998001 or 0.003643001 to 6 significant figures). In fact, any number in
the interval [0.003642995,0.003643006) gives 0.00364300, when rounded to 6 significant
figures.
∎

Remark:

An approximation 𝑥 ∗ to a number 𝑥 is called accurate to 𝑡 significant figures if there are exactly 𝑡

digits in the mantissa of 𝑥 ∗ that agree with the first 𝑡 digits of the mantissa of 𝑥, where 𝑥 has the
same exponent as 𝑥 ∗ . Suppose that the number 𝑥 is represented in the following form

𝑥 = ±0. 𝑑1 𝑑2 𝑑3 ⋯ 𝑑𝑡 𝑑𝑡+1 ⋯ × 10𝑒

Then, the number 𝑥 ∗ is accurate to 𝑡 significant figures to the number 𝑥 if it can be written in the
following form

𝑥∗ ′
= ±0. 𝑑1 𝑑2 𝑑3 ⋯ 𝑑𝑡 𝑑𝑡+1 ⋯ × 10𝑒
∎
5 Simplified Numerical Analysis

Fig. (1.3): According to the IEEE 754 standard, single-precision floating point representation of a
binary real number 𝑥 = ±1. b2 b3 b4 ⋯ × 2e is (1 − 2s) × 2c−127 × (1 + f).

Fig. (1.4): According to the IEEE 754 standard, double-precision floating point representation of a
binary real number x = ±1. b2 b3 b4 ⋯ × 2e is (1 − 2s) × 2c−1023 × (1 + f).

Here, 𝑠 is used for the sign of the number (0 means positive, 1 means negative). 𝑐 in the exponent
is called the biased exponent. 𝑓 is the mantissa minus 1 (the hidden bit).

Fig. (1.5): Overflow/Underflow for single-precision floating-point representation

Fig. (1.6): Overflow/Underflow for double-precision floating-point representation

∎∎∎
Preliminary Concepts in Numerical Analysis 6

Chapter Summary

• The numerical methods obtain some approximate solution of the problems, usually in the numeric
form, in contrast to the analytic or exact methods, which obtain the exact solution of the problem.

• Numerical Analysis is the field of deriving, analyzing, and implementing the numerical methods.

• The most common approach followed by the numerical methods is the iterative approach. According to
this, choose an initial approximation or guess to the solution and apply a set of simple computational
steps to obtain a better approximation. Repeatedly apply the same set of steps to the better
approximations, ultimately obtaining a sufficiently accurate solution and then stop the repetition. Each
course of repetition of the set of computational steps is called iteration. Geometrically, a root of an
equation 𝑓(𝑥) = 0 is the point where the graph of 𝑓(𝑥) intersects the 𝑥-axis.

• For selecting a numerical method from several choices, the characteristics of accuracy,
efficiency, and robustness are taken into consideration.
• The numerical analysis may be regarded as the “mathematics of scientific computing”.

• Errors can be quantified as:

o Absolute Error = |True value − Approximate value|

absolute error |True value−Approximate value|
o Relative Error = |True value|
= |True value|

absolute error
o Percentage Relative Error = |True value|
× 100 %

• The errors can be categorized in three major categories in regard to their sources: Data Error or
Inherent Error (quite unrelated to the numerical methods; occur as blunders, mistakes, model
simplification, or data uncertainty), Round-off Error (occurs due to number approximation by humans
and computers), Truncation Error (occurs due to approximation of a mathematical procedure to avoid
insignificance), and Distcretization error (occurs due to approximation of a continuous function by a
set of discrete data points).

• Significant figures of a real number (which is an approximation of the true value) are the digits that
are used to express the number meaningfully. The significant digits begin with the leftmost nonzero
digit and end with the rightmost correct digit. The rightmost zeros, which are exact are also significant.

• An approximation 𝑥 ∗ to a number 𝑥 is called accurate to 𝑡 significant figures if there are exactly 𝑡 digits
in the mantissa of 𝑥 ∗ that agreed with the first 𝑡 digits of the mantissa of 𝑥 having the same exponent or
characteristics.

• Accuracy of an approximate value is a measure of how much the approximate value agrees with the
true value. Precision, on the other hand, has nothing to do with how much the approximate value
agrees with the true value. Precision is only concerned about the size of the number.

• The following four are the commonly used number systems, even supported by the computer
architectures.
 1. Decimal number system (base 10) 3. Octal number system (base 8)
2. Binary number system (base 2) 4. Hexadecimal number system (base 16)

• Any nonzero real decimal number 𝑥 can be represented in floating-point form: 𝑥 = ±0. 𝑑1 𝑑2 𝑑3 ⋯ × 10𝑒 .
Here 𝑑𝑖 , 𝑖 = 1, 2, ⋯ are digits from 0 to 9 with 𝑑1 ≠ 0, called most significant digit and 𝑒 is an integer
that might be positive, negative or zero, called an exponent or characteristic. The number 0. 𝑑1 𝑑2 𝑑3 ⋯,
may be denoted by 𝑚, is called the finite normalized mantissa. For numbers in the decimal system with
1 1
base 10, ≤ 𝑚 < 1. That is, 𝑚 ∈ [ , 1).
10 10

• For numbers in the binary system, the floating-point representation of a number 𝑥 can be given by,
1
𝑥 = ±0. 𝑏1 𝑏2 𝑏3 ⋯ × 2𝑒 = ±𝑚 × 2𝑒 , were each of 𝑏𝑖 is a bit, either 0 or 1, with 𝑏1 ≠ 0, and ≤ 𝑚 < 1.
2

• The numbers that are representable precisely in a computer are called machine numbers. The real
numbers with a non-terminating fractional part (such as 1/3) cannot be represented, precisely. So
many other numbers (for example, 0.01) also has not a precise representation in computer (i.e., a
machine number).

• If the number lies within the allowable range of the possible numbers according to the precision level of
the computer, then it is rounded to a nearby machine number (incurring the round-off error) for storing
it. The rounding options involve correct rounding (round to nearest machine number), rounding up,
rounding down or towards zero, etc.

• There are commonly two ways to terminate the mantissa of a number to obtain its nearest machine
number, namely, correct chopping and correct rounding. The chopping or rounding of the number to
the nearest machine number (representable in a computer) for representation in computers (for
storage or for using in computations) causes the error in a number called the round-off error.

• The floating-point form of a number 𝑥 representable in a computer can be regarded as consisting of the
three parts: 𝑥 = ±𝑚 × 𝛽𝑒 = 𝒔𝒊𝒈𝒏 × 𝒎𝒂𝒏𝒕𝒊𝒔𝒔𝒂 × (𝑏𝑎𝑠𝑒)𝒆𝒙𝒑𝒐𝒏𝒆𝒏𝒕
1
The sign is either positive (+) or negative (−), the finite normalized mantissa is from the interval [ , 1),
𝛽
and the integer exponent either positive, negative, or zero as a power of the base.

• An account on the IEEE Binary Floating-Point Arithmetic Standard 754-1985 for representing the
real numbers in computers can be found under Question 13 in this chapter.

• If a number 𝒙∗ is accurate to 𝒕 significant figures in approximating a number 𝒙 then the relative

|𝒙−𝒙∗ |
error is bounded above by 𝟓 × 𝟏𝟎−𝒕 . That is, |𝒙|
≤ 𝟓 × 𝟏𝟎−𝒕

• If an iterative process is to be stopped when the successive approximations become accurate to 𝑡

significant figures, the relative error bound might be set as 5 × 10−𝑡 . Thus, the relative error is
computed after every iteration using the result of the current iteration and that of the previous
iteration. If the relative error is smaller than the bound of 5 × 10−𝑡 , then it ensures that the
approximation the accurate to 𝑡 significant digits.

• Whenever two nearly equal numbers are subtracted, some loss of significance might occur. The risk of
loss of significance can be eliminated by avoiding the subtraction through some mathematical
manipulation.

7
Preliminary Concepts in Numerical Analysis 8

Chapter Exercises

Exercise 01: Compute the absolute error 𝐸𝑎 and relative error 𝐸𝑟 in an approximation of 𝑥 by 𝑥 ∗
(𝑖) 𝑥 = log10 2 , 𝑥 ∗ = 0.301 (𝑖𝑖) 𝑥 = 17⁄6 , 𝑥 ∗ = 2.8333

(𝑖𝑖𝑖) 𝑥 = √𝜋, 𝑥 ∗ = 1.77245 (𝑖𝑣) 𝑥 = 𝑒 −1 , 𝑥 ∗ = 0.36787

Exercise 02: Write the following numbers in floating-point form and identify their mantissa and exponent:
(𝑖) 𝑥 = −23.500128 (𝑖𝑖) 𝑥 = 658.000012 (𝑖𝑖𝑖) 𝑥 = 0.010023
1
(𝑖𝑣) 𝑥 = −0.0000782 (𝑣) 𝑥= (𝑣𝑖) 𝑥 = 541000
234.24
Exercise 03: Simplify the following expression by performing the computations

(a) Exactly
(b) Using four-digit chopping arithmetic
(c) Using four-digit rounding arithmetic
(d) Compute the relative errors
7 5 5 2 𝜋−1
(𝑖) − (𝑖𝑖) ( + 4) (𝑖𝑖𝑖)
4 3 4 3 4
3
432 − 0.0012 2 9
(𝑖𝑣) 10𝜋 − 2𝑒 + 1 (𝑣) ( ) (𝑣𝑖) ( ).( )
101 9 7
Consider 𝜋 and 𝑒 expressed with fifteen significant digits as the exact numbers.

Exercise 04: Calculate the roundoff error if chopping and rounding is used to write the following numbers
accurate to four decimal digits:
3
(𝑖) 355/113 (𝑖𝑖) √3/142 (𝑖𝑖𝑖) √ln 2

Exercise 05: We want to round-off each the following numbers to three decimal places. For which number,
the result of “round-off by chopping” and “round-off by rounding-rule” will be the same:

(A) 5.5555 (B) 3.3575 (C) 5.5565 (D) 4.4555

Exercise 06: Find the absolute and relative errors involved in rounding 4.9997 to 5.000.

Exercise 07: Suppose a real number 𝑥 is represented approximately by 0.6032 with the relative error is at
most 0.1%. What is 𝑥?

Exercise 08: Suppose that a number is accurate to 𝑛 significant figures and 𝑎1 is the first significant figure
1
than show that the relative error is bounded above by × 101−𝑛 .
𝑎1

1
Exercise 09: Show that if a number is rounded off to 𝑛 digits than the relative error is bounded by × 101−𝑛 .
2

∎∎∎
 Chapter 2
Solution of a Nonlinear Equation
in One Variable

Corridor I: BASICS
Let’s plan it

2.1 Introduction
2.2 Bracketing Methods
2.2.1 The Bisection Method (or Bolzano Method)
2.2.2 The False-Position Method (or Regula-Falsi Method)
2.3 Open Methods
2.3.1 The Fixed-Point Iteration Method
2.3.2 The Newton-Raphson Method
2.3.3 The Secant Method

To unleash the topics of this Corridor, please delve into the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

9
10 Simplified Numerical Analysis

Corridor II: ANALYSIS

Let’s think deep

2.4 Convergence Analysis

The Bisection Method
The Regula-Falsi Method
The Secant Method
The Newton-Raphson Method
The Fixed-Point Iteration Method
2.5 Further Discussions

To unleash the topics of this Corridor, please delve into the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

Corridor III: PROGRAMMING ARCADE

Let’s do it

2.6 Algorithms and Implementations

The Newton-Raphson Method
The Fixed-Point Iteration Method
The Secant Method
The Bisection Method
The Regula-Falsi Method

To cross-check the results/output of the computer programs you would execute, please delve into
the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎
Solution of a Nonlinear Equation in One Variable 11

2.6 Algorithms and Implementations

Question 36: Write down the algorithm (pseudo code) of the Newton’s method to solve 𝑓(𝑥) = 0. The
algorithm should perform a fixed number of iterations.

Algorithm: To solve 𝑓(𝑥) = 0 using the following iterative formula (given an initial approximation 𝑥0 ):
𝑓(𝑥𝑘−1 )
𝑥𝑘 = 𝑥𝑘−1 − , for 𝑘 = 1, 2, 3, ⋯
𝑓 ′ (𝑥𝑘−1 )
𝒙𝟎: a real value as the initial approximation 𝑥0 sufficiently close to the root
𝐈𝐍𝐏𝐔𝐓𝐒: {
𝑵: an integer as the maximum number of iterations
𝒙𝒏: a real value as the approximate solution
𝐎𝐔𝐓𝐏𝐔𝐓: {
(on completing 𝑵 iterations)

Step 1 Receive the inputs as stated above

Step 2 Set 𝒙𝒏 = 𝒙𝟎 (initialize 𝒙𝒏 with the initial approximation)

Step 3 for 𝒌 = 𝟏, 𝟐, 𝟑, ⋯ , 𝑵 perform Steps 4-6

𝒙𝒑 is to keep a copy of the approximation 𝒙𝒏,
Step 4 Set 𝒙𝒑 = 𝒙𝒏 {
because 𝒙𝒏 is going to be updated.

Step 5 Set 𝒇𝒙𝒑 as the value of 𝑓(𝒙𝒑)

Set 𝒅𝒇𝒙𝒑 as the value of 𝑓′(𝒙𝒑)
Step 6
𝒇𝒙𝒑 Computing a new
𝒙𝒏 = 𝒙𝒑 − {
𝒅𝒇𝒙𝒑 approximation to the root

end for (Go to Step 4 for the next iteration)

Step 7 Print the output: 𝒙𝒏

[Additionally, the initial approximation (𝒙𝟎), number of iterations (𝒌), and 𝑓(𝒙𝒏) can be printed]

STOP.

Remark: In the algorithm, it is assumed that neither any pitfall of the method will occur, nor 𝑓(𝑥) will be
equal to zero (or the machine-epsilon) in any iteration for the given problem and initial approximation.

Problem 18: Write a C++ program to find a real root of 𝑓(𝑥) = 4𝑥 + sin 𝑥 − 𝑒 𝑥 = 0 using the Newton-
Raphson method. Take initial approximation as 𝑥0 = 0. Here 𝑓′(𝑥) = 4 + cos 𝑥 − 𝑒 𝑥 . The program should
perform a fixed number of iterations.
12 Simplified Numerical Analysis

1 # include <iostream>
2 # include <cmath>
3 using namespace std ;
4 # define N 100 // maximum number of iterations
5
6 int main ( )
7 {
8 int k ;
9 double xn , xp , fxp , dfxp ;
10
11 cout<< "Enter the initial approximation x0: " ;
12 cin>> xn ;
13
14 //----------------------- Processing Section -------------------------//
15
16 for ( k=1 ; k<=N ; k++ )
17 {
18 xp = xn ;
19 fxp = 4*xp + sin(xp) – exp(xp) ;
20 dfxp = 4 + cos(xp) – exp(xp) ; 𝑓(𝑥𝑘−1 )
𝑥𝑘 = 𝑥𝑘−1 −
21 xn = xp – fxp / dfxp ; 𝑓 ′ (𝑥𝑘−1 )
22 }
23
24 //------------------------ Output Section ----------------------------//
25
26 cout<< endl << "An approximate root of the given function is " << x << endl ;
27 cout<< endl << N << " iterations completed." ;
28
29 }

Remark: In the program of Problem 18, the code segment of line 26 can be placed just before line 22 to print
the latest result on completion of each of the iterations.

Remark: In the program, it is assumed that neither any pitfall of the method will occur, nor 𝑓(𝑥) will be
equal to zero (or the machine-epsilon) in any iteration for the given problem and initial approximation.

Remark: In the program, it is assumed that neither any pitfall of the method will occur, nor 𝑓(𝑥) will be
equal to zero (or the machine-epsilon) in any iteration.

Remark: The algorithm in Question 36 (likewise 18) has a shortcoming that on completion of the given fixed
number of 𝑁 iterations the solutions might not have been converged (the desired accuracy might not have
been achieved). Moreover, the algorithm has a shortcoming if the convergence has been achieved (or
divergence has occurred) in few iterations, even then the iterations would not stop immediately; the
algorithm will complete the fixed number of iterations. These shortcomings in the algorithm can be
addressed by incorporating the two convergence criteria such that if the convergence is achieved
Solution of a Nonlinear Equation in One Variable 13

(i. e., error < tolerence), then no more iterations will be performed, however, the number of iterations
would not exceed the maximum limit on the number of iterations. Such an indispensable modification
regarding the stopping criteria is adopted throughout the subsequent part of the book.

Question 37: Write down the algorithm (pseudo code) of the Newton’s method to solve 𝑓(𝑥) = 0.

Algorithm: To solve 𝑓(𝑥) = 0 using the following iterative formula (given an initial approximation 𝑥0 ):
𝑓(𝑥𝑘−1 )
𝑥𝑘 = 𝑥𝑘−1 − , for 𝑘 = 1, 2, 3, ⋯
𝑓 ′ (𝑥𝑘−1 )

𝒙𝟎: a real value as the initial approximation 𝑥0 sufficiently close to the root
𝐈𝐍𝐏𝐔𝐓𝐒: {𝑻𝑶𝑳: a real value as the tolerance (permissible error)
𝑵: an integer as the maximum number of iterations

𝒙𝒏: a real value as the approximate solution

𝐎𝐔𝐓𝐏𝐔𝐓: {
(either on convergence or on completing 𝑵 iterations − whichever happens first)

Step 1 Receive the inputs as stated above

Step 2 Set 𝒙𝒏 = 𝒙𝟎 (initialize 𝒙𝒏 with the initial approximation)

Step 3 for 𝒌 = 𝟏, 𝟐, 𝟑, ⋯ , 𝑵 perform Steps 4-8

𝒙𝒑 is to keep a copy of the approximation 𝒙𝒏,
Step 4 Set 𝒙𝒑 = 𝒙𝒏 {
because 𝒙𝒏 is going to be updated.

Step 5 Set 𝒇𝒙𝒑 as the value of 𝑓(𝒙𝒑)

Set 𝒅𝒇𝒙𝒑 as the value of 𝑓′(𝒙𝒑)
Step 6
𝒇𝒙𝒑 Computing a new
𝒙𝒏 = 𝒙𝒑 − {
𝒅𝒇𝒙𝒑 approximation to the root

Step 7 Set 𝒆𝒓𝒓 = |𝒙𝒏 − 𝒙𝒑| (or 𝑒𝑟𝑟 = |𝑥𝑛 − 𝑥𝑝|/|𝑥𝑝|)

Step 8
This means that the consecutive
if (𝒆𝒓𝒓 < 𝑻𝑶𝑳) then
} approximations are nearly the same.
Exit/Break the loop
Therefore, stop iterations.
end for (Go to Step 4 for the next iteration)

Step 9 Print the output: 𝒙𝒏

[Additionally, the initial approximation (𝒙𝟎), number of iterations (𝒌), 𝑓(𝒙𝒏), and error (𝒆𝒓𝒓) can be printed]

if (𝒆𝒓𝒓 < 𝑻𝑶𝑳) OUTPUT (‘The desired accuracy achieved; Solution converged.’)

else OUTPUT (‘The number of iterations exceeded the maximum limit.’) because 𝒌 > 𝑵

STOP.
Remark: In the algorithm, it is assumed that neither any pitfall of the method will occur, nor 𝑓(𝑥) will be
equal to zero (or the machine-epsilon) in any iteration for the given problem and initial approximation.
14 Simplified Numerical Analysis

Problem 20: Write a C++ program to find a real root of 𝑓(𝑥) = 4𝑥 + sin 𝑥 − 𝑒 𝑥 = 0 using the Newton-
Raphson method. Take initial approximation as 𝑥0 = 0. Here 𝑓′(𝑥) = 4 + cos 𝑥 − 𝑒 𝑥 . The iterations of the
method should stop when either the approximation is accurate within 10−5 , or the number of iterations
exceed 100, whichever happens first.

1 # include <iostream>
2 # include <cmath>
3 using namespace std ;
4 # define TOL 0.000001 // error tolerance
5 # define N 100 // maximum number of iterations
6
7 int main ( )
8 {
9 int k ;
10 double x0, xn , xp , fxp , dfxp , err ;
11
12 cout<< "Enter the initial approximation x0: " ;
13 cin>> x0 ;
14 xn = x0 ;
15
16 //----------------------- Processing Section -------------------------//
17
18 for ( k=1 ; k<=N ; k++ )
19 {
20 xp = xn ;
21 fxp = 4*xp + sin(xp) – exp(xp) ;
22 dfxp = 4 + cos(xp) – exp(xp) ; 𝑓(𝑥𝑘−1 )
𝑥𝑘 = 𝑥𝑘−1 −
23 xn = xp – fxp / dfxp ; 𝑓 ′ (𝑥𝑘−1 )
24 err = abs(xn – xp) ; Error = |𝑥𝑘 − 𝑥𝑘−1 |
25
26 cout<< "After " << k << "iterations, the approximate root = " << xn ;
27 cout<< " f(x) = " << fxp << " Error = " << err << endl ;
28
29 if ( err < TOL ) break ;
30 }
31
32 if ( err < TOL )
33 cout<< "The desired accuracy achieved; Solution converged." ;
34 else
35 cout<< "The number of iterations exceeded the maximum limit." ;
36 }

Remark: This program is based on the assumption that neither any pitfall of the method will occur, nor 𝑓(𝑥)
will be equal to zero (or machine-epsilon) in any iteration for the given problem and data.
Solution of a Nonlinear Equation in One Variable 15

Problem 22: Write a C++ program to find a real root of the equation 𝑓(𝑥) = 4𝑥 + sin 𝑥 − 𝑒 𝑥 = 0 using the
Newton-Raphson method. Take initial approximation as 𝑥0 = 0. Here 𝑓′(𝑥) = 4 + cos 𝑥 − 𝑒 𝑥 . Use #define
directive to evaluate 𝑓(𝑥) and 𝑓′(𝑥) wherever required. The iterations of the method should stop when
either the approximation is accurate within 10−5 , or the number of iterations exceed 100, whichever
happens first.

1 # include <iostream>
2 # include <cmath>
3
4 using namespace std ;
5
6 # define TOL 0.000001 // error tolerance
7 # define N 100 // maximum number of iterations
8 # define fval(x) 4*x + sin(x) – exp(x) // Evaluating f(x)
9 # define dfval(x) 4 + cos(x) – exp(x) // Evaluating f’(x)
10
11 int main ( )
12 {
13 int k ;
14 double x0, xn , xp , fxp , dfxp , err ;
15 cout<< "Enter the initial approximation x0: " ;
16 cin>> x0 ;
17 xn = x0 ;
18
19 //----------------------- Processing Section -------------------------//
20
21 for ( k=1 ; k<=N ; k++ )
22 {
23 xp = xn ;
24 fxp = fval(xp) ;
25 dfxp = dfval(xp) ; 𝑓(𝑥𝑘−1 )
𝑥𝑘 = 𝑥𝑘−1 −
26 xn = xp – fxp / dfxp ; 𝑓 ′ (𝑥𝑘−1 )
27 err = abs(xn – xp) ; Error = |𝑥𝑘 − 𝑥𝑘−1 |
28
29 cout<< "After " << k << " iterations, the approximate root = " << xn ;
30 cout<< " f(x) = " << fxp << " Error = " << err << endl ;
31
32 if ( err < TOL ) break ;
33 }
34
35 if ( err < TOL ) cout<<"The desired accuracy achieved; Solution converged.";
36 else cout<< "The maximum number of iterations exceeded the limit." ;
37 }
16 Simplified Numerical Analysis

Question 38: Write down the algorithm (pseudo code) of the Fixed-Point Iteration method to solve 𝑓(𝑥) = 0.

Algorithm: To solve 𝑓(𝑥) = 0 ⟺ 𝑥 = 𝑔(𝑥), using the following iterative formula (given an initial
approximation 𝑥0 )
𝑥𝑘 = 𝑔(𝑥𝑘−1 ), for 𝑘 = 1, 2, 3, ⋯

𝒙𝟎: a real value as the initial approximation 𝑥0 sufficiently close to the root
𝐈𝐍𝐏𝐔𝐓𝐒: {𝑻𝑶𝑳: a real value as the tolerance (permissible error)
𝑵: an integer as the maximum number of iterations

𝒙𝒏: a real value as the approximate solution

𝐎𝐔𝐓𝐏𝐔𝐓: {
(either on convergence or on completing 𝑵 iterations − whichever happens first)

Step 1 Receive the inputs as stated above

Step 2 Set 𝒙𝒏 = 𝒙𝟎 (initialize 𝒙𝒏 with the initial approximation)

Step 3 for 𝒌 = 𝟏, 𝟐, 𝟑, ⋯ , 𝑵 perform Steps 4-7

𝒙𝒑 is to keep a copy of the approximation 𝒙𝒏,
Step 4 Set 𝒙𝒑 = 𝒙𝒏 {
because 𝒙𝒏 is going to be updated.

Step 5
Computing a new
Set 𝒙𝒏 as the value of 𝑔(𝒙𝒑) {
approximation to the root

Step 6 Set 𝒆𝒓𝒓 = |𝒙𝒏 − 𝒙𝒑| (or 𝑒𝑟𝑟 = |𝑥𝑛 − 𝑥𝑝|/|𝑥𝑝|)

Step 7
This means that the consecutive
if (𝒆𝒓𝒓 < 𝑻𝑶𝑳) then
} approximations are nearly the same.
Exit/Break the loop
Therefore, stop iterations.

end for (Go to Step 4 for the next iteration)

Step 8 Print the output: 𝒙𝒏

[Additionally, the initial approximation (𝒙𝟎), number of iterations (𝒌), 𝑓(𝒙𝒏), and error (𝒆𝒓𝒓) can be printed]

if (𝒆𝒓𝒓 < 𝑻𝑶𝑳) OUTPUT (‘The desired accuracy achieved; Solution converged.’)

else OUTPUT (‘The number of iterations exceeded the maximum limit.’)

STOP.
Solution of a Nonlinear Equation in One Variable 17

Problem 24: Write a C++ program to find a real root of the equation 𝑓(𝑥) = 4𝑥 + sin 𝑥 − 𝑒 𝑥 = 0 using the
1
Fixed-Point Iteration method. Take 𝑥 = 𝑔(𝑥) = (𝑒 𝑥 − sin 𝑥) and 𝑥0 = 0 as an initial approximation. The
4
iterations of the method should stop when either the approximation is accurate within 10−5 , or the number
of iterations exceeds 100, whichever happens first.

1 # include <iostream>
2 # include <cmath>
3 using namespace std ;
4
5 # define TOL 0.000001 // error tolerance
6 # define N 100 // maximum number of iterations
7
8 int main ( )
9 {
10 int k ;
11 double x0 , xp , xn , err ;
12
13 cout<< "Enter the initial approximation x0: " ;
14 cin>> x0 ;
15 xn = x0 ;
16
17 //----------------------- Processing Section -------------------------//
18
19 for ( k=1 ; k<=N ; k++)
20 {
21
22 xp = xn ;
23
24 xn = 0.25 * ( exp(xp) – sin(xp) ) ; // Computing g(x) at the current approx.
25
26 err = abs(xn – xp) ; Error = |𝑥𝑘 − 𝑥𝑘−1 |
27
28 if ( err < TOL )
29 break ;
30
31 }
32
33 if ( err < TOL )
34 cout<< "The desired accuracy achieved; Solution converged." ;
35 else
36 cout<< "The number of iterations exceeded the maximum limit." ;
37
38 }
18 Simplified Numerical Analysis

Question 39: Write down the algorithm (pseudo code) of the Secant method to solve 𝑓(𝑥) = 0.

Algorithm: To solve 𝑓(𝑥) = 0 using the iterative formula (given the root containing interval):
𝑓(𝑥𝑘−1 )(𝑥𝑘−1 − 𝑥𝑘−2 )
𝑥𝑘 = 𝑥𝑘−1 − , for 𝑘 = 2, 3, 4, ⋯
𝑓(𝑥𝑘−1 ) − 𝑓(𝑥𝑘−2 )

𝒂 and 𝒃: two real values as the initial approximations sufficiently close to the root
𝐈𝐍𝐏𝐔𝐓𝐒: {𝑻𝑶𝑳: a real value as the tolerance (permissible error)
𝑵: an integer as the maximum number of iterations

𝒙𝒏: a real value as the approximate solution

𝐎𝐔𝐓𝐏𝐔𝐓: {
(either on convergence or on completing 𝑵 iterations − whichever happens first)

Step 1 Receive the inputs as stated above

Step 2 Set 𝒙𝒏 = 𝒃 (initialize 𝒙𝒏 with any of the two endpoints)

Step 3 Set 𝒙𝟎 = 𝒂
Set 𝒙𝟏 = 𝒃
Set 𝒇𝒙𝟎 as the value of 𝑓(𝒙𝟎)
Set 𝒇𝒙𝟏 as the value of 𝑓(𝒙𝟏)

Step 4 for 𝒌 = 𝟐, 𝟑, ⋯ , 𝑵 + 𝟏 perform Steps 5-10

𝒙𝒑 is to keep a copy of the approximation 𝒙𝒏,
Step 5 Set 𝒙𝒑 = 𝒙𝒏 {
because 𝒙𝒏 is going to be updated.
Step 6
𝒇𝒙𝟏(𝒙𝟏 − 𝒙𝟎) Computing a new
𝒙𝒏 = 𝒙𝟏 − {
𝒇𝒙𝟏 − 𝒇𝒙𝟎 approximation to the root

Step 7 Set 𝒇𝒙𝒏 as the value of 𝑓(𝒙𝒏)

Step 8 Set 𝒆𝒓𝒓 = |𝒙𝒏 − 𝒙𝒑|/|𝒙𝒑| (or 𝑒𝑟𝑟 = |𝑥𝑛 − 𝑥𝑝|)

Step 9
This means that the consecutive
if (𝒆𝒓𝒓 < 𝑻𝑶𝑳) then
} approximations are nearly the same.
Exit/Break the loop
Therefore, stop iterations.
else
Set 𝒙𝟎 = 𝒙𝟏
Set 𝒇𝒙𝟎 = 𝒇𝒙𝟏 preparing two approximations
Set 𝒙𝟏 = 𝒙𝒏 for the next iteration
Set 𝒇𝒙𝟏 = 𝒇𝒙𝒏 }

end for (Go to Step 5 for the next iteration)

Step 10 Print the output: 𝒙𝒏

[Additionally, the initial approx. (𝒙𝟎 and 𝒙𝟏), number of iterations (𝒌 − 𝟏), 𝑓(𝒙𝒏), and error (𝒆𝒓𝒓) can be printed]

if (𝒆𝒓𝒓 < 𝑻𝑶𝑳) OUTPUT (‘The desired accuracy achieved; Solution converged.’)
else OUTPUT (‘The number of iterations exceeded the maximum limit.’)
STOP.
Solution of a Nonlinear Equation in One Variable 19

Problem 26: Write a C++ program to find a real root of the equation 𝑓(𝑥) = 4𝑥 + sin 𝑥 − 𝑒 𝑥 = 0 using the
Secant method. Take initial approximation as 𝑥0 = 0 and 𝑥1 = 1. The iterations of the method should stop
when either the approximation is accurate within 10−5 , or the number of iterations exceeds 100, whichever
happens first.

1 # include <iostream>
2 #include<cmath>
3 using namespace std ;
4 # define TOL 0.000001 // error tolerance
5 # define N 100 // maximum number of iterations
6
7 int main ( )
8 {
9 int k ;
10 double x0 , x1 , xn , xp , fx0 , fx1 , fxn , fxp , err ;
12 cout<< "Enter the first initial approximation: " ; cin>> a ;
13 cout<< "Enter the second initial approximation: " ; cin>> b ;
15
16 //----------------------- Processing Section -------------------------//
17 xn = b ;
18 x0 = a ;
19 x1 = b ;
20 fx0 = 4*x0 + sin(x0) – exp(x0) ; // Evaluating f(x) at x0
21 fx1 = 4*x1 + sin(x1) – exp(x1) ; // Evaluating f(x) at x1
22
23 for ( k=2 ; k<=N+1 ; k++)
24 {
25 xp = xn ;
26 xn = x1 – (fx1 * (x1 – x0)) / (fx1 – fx0) ; 𝑓(𝑥𝑘−1 )(𝑥𝑘−1 − 𝑥𝑘−2 )
𝑥𝑘 = 𝑥𝑘−1 −
27 fxn = 4*xn + sin(xn) – exp(xn) ; 𝑓(𝑥𝑘−1 ) − 𝑓(𝑥𝑘−2 )
28 err = abs(xn – xp)/abs(xn) ; |𝑥𝑘 − 𝑥𝑘−1 |
Error =
29 |𝑥𝑘 |
30
31 cout<< "After " << k–1 << " iterations, the approximate root = " << xn ;
32 cout<< " f(x) = " << fxn << " Error = " << err << endl ;
33
34 if ( err < TOL ) break ;
35 else
36 {
37 x0 = x1 ;
38 fx0 = fx1 ;
39 x1 = xn ;
40 fx1 = fxn ;
41 }
42 }
20 Simplified Numerical Analysis

43 if ( err < TOL )

44 cout<< "The desired accuracy achieved; Solution converged." ;
45 else
46 cout<< "The number of iterations exceeded the maximum limit." ;
47 }

Question 40: Write down the algorithm (pseudo code) of the Bisection method to solve 𝑓(𝑥) = 0.

Algorithm: To solve 𝑓(𝑥) = 0 using the iterative formula (given the root containing interval):

𝑥𝑘−1 − 𝑥𝑘−2
𝑥𝑘 = 𝑥𝑘−2 + , for 𝑘 = 2, 3, 4, ⋯
2

𝒂 and 𝒃: two real values as the initial approximations bracketing the root
𝐈𝐍𝐏𝐔𝐓𝐒: {𝑻𝑶𝑳: a real value as the tolerance (permissible error)
𝑵: an integer as the maximum number of iterations

𝒙𝒏: a real value as the approximate solution

𝐎𝐔𝐓𝐏𝐔𝐓: {
(either on convergence or on completing 𝑵 iterations − whichever happens first)

Step 1 Receive the inputs as stated above

Step 2 Set 𝒙𝒏 = 𝒃 (initialize 𝒙𝒏 with any of the two endpoints)

Step 3 Set 𝒙𝟎 = 𝒂
Set 𝒙𝟏 = 𝒃
Set 𝒇𝒙𝟎 as the value of 𝑓(𝒙𝟎)
Set 𝒇𝒙𝟏 as the value of 𝑓(𝒙𝟏)

Step 4 for 𝒌 = 𝟐, 𝟑, ⋯ , 𝑵 + 𝟏 perform Steps 5-10

𝒙𝒑 is to keep a copy of the approximation 𝒙𝒏,
Step 5 Set 𝒙𝒑 = 𝒙𝒏 {
because 𝒙𝒏 is going to be updated.

Step 6
𝒙𝟏 − 𝒙𝟎 Computing a new
𝒙𝒏 = 𝒙𝟎 + {
𝟐 approximation to the root
Step 7 Set 𝒇𝒙𝒏 as the value of 𝑓(𝒙𝒏)

Step 8 Set 𝒆𝒓𝒓𝟏 = |𝒙𝒏 − 𝒙𝒑|/|𝒙𝒏| (or 𝑒𝑟𝑟 = |𝑥𝑛 − 𝑥𝑝|)

Set 𝒆𝒓𝒓𝟐 = |𝒇𝒙𝒏|
Set 𝒆𝒓𝒓 = 𝒎𝒊𝒏(𝒆𝒓𝒓𝟏, 𝒆𝒓𝒓𝟐)

Step 9
This means that either 𝒇(𝒙𝒏) is the close to
if (𝒆𝒓𝒓 < 𝑻𝑶𝑳 )then
} zero, or the consecutive approximations are
Exit/Break the loop
nearly the same. Therefore, stop iterations.
Solution of a Nonlinear Equation in One Variable 21

else if (𝒇(𝒙𝟎)𝒇(𝒙𝒏) < 0) then

Set 𝒙𝟏 = 𝒙𝒏 Adjusting one endpoint
Set 𝒇𝒙𝟏 = 𝒇𝒙𝒏 of the interval such that
else half of the interval will be
Set 𝒙𝟎 = 𝒙𝒏 used in the next iteration
Set 𝒇𝒙𝟎 = 𝒇𝒙𝒏 }

end for (Go to Step 5 for the next iteration)

Step 10 Print the output: 𝒙𝒏

[Additionally, the starting interval [𝑎, 𝑏], number of iterations (𝒌 − 𝟏), 𝑓(𝒙𝒏), and error (𝒆𝒓𝒓) can be printed]

if (𝒆𝒓𝒓 < 𝑻𝑶𝑳) OUTPUT (‘The desired accuracy achieved; Solution converged.’)

else OUTPUT (‘The number of iterations exceeded the maximum limit.’)

STOP.
∎

Remark: While using a bracketing method, there might arise a situation in which the two
consecutive approximations to the roots are not sufficiently close to each other (i.e., the sequence
of successive approximations has not converged), but the function values at the approximations
are sufficiently close to zero (i.e., |𝑓(𝑥𝑘 )| < tolerence). Therefore, there is no point to proceed the
iterations further. The iterations should be stopped. Therefore, the algorithm of a bracketing
method (the Bisection, or Regula-Falsi method) should include both of the convergence criteria
of testing the convergence of the roots, and closeness of the function values to zero. The iterations
should be terminated on whichever criterion is met first, ensuring the convergence. To
accommodate this in the algorithm, the two kinds of errors are computed and the minimum of the
two errors is found to compare with the tolerance:

Set 𝑒𝑟𝑟1 = |𝑥𝑛 − 𝑥𝑝|/|𝑥𝑛| (or 𝑒𝑟𝑟 = |𝑥𝑛 − 𝑥𝑝|)

Set 𝑒𝑟𝑟2 = |𝑓𝑥𝑛|
Set 𝑒𝑟𝑟 = 𝑚𝑖𝑛(𝑒𝑟𝑟1, 𝑒𝑟𝑟2)
∎

Problem 28: Write a C++ program to find a real root of the equation 𝑓(𝑥) = 4𝑥 + sin 𝑥 − 𝑒 𝑥 = 0 in [0, 1]
using the Bisection method. The two function values at the endpoints of the interval have opposite signs. The
iterations of the method should stop when either the approximation is accurate within 10−5 , or the number
of iterations exceeds 100, whichever happens first.

1 # include <iostream>
2 # include <cmath>
3 using namespace std ;
4
22 Simplified Numerical Analysis

5 # define TOL 0.000001 // error tolerance

6 # define N 100 // maximum number of iterations
7
8 int main ( )
9 {
10 int k ;
11 double x0 , x1 , xn , xp , fx0 , fx1 , fxn , fxp , err , err1 , err2 ;
12 cout<< "Enter the left endpoint of the interval: " ; cin>> a ;
13 cout<< "Enter the right endpoint of the interval: " ; cin>> b ;
14
15 //----------------------- Processing Section -------------------------//
16
17 xn = b ;
18 x0 = a ;
19 x1 = b ;
20 fx0 = 4*x0 + sin(x0) – exp(x0) ; // Evaluating f(x) at x0
21 fx1 = 4*x1 + sin(x1) – exp(x1) ; // Evaluating f(x) at x1
22
23 for ( k=2 ; k<=N+1 ; k++)
24 {
25 xp = xn ;
26
27 xn = x0 + (x1 – x0) / 2 ; 𝑥𝑘−1 − 𝑥𝑘−2
𝑥𝑘 = 𝑥𝑘−2 +
28 fxn = 4*xn + sin(xn) – exp(xn) ; 2
29
30 err1 = abs(xn – xp)/abs(xn) ; Error 1 = |𝑥𝑘 − 𝑥𝑘−1 |/|𝑥𝑘 |
31 err2 = abs(fxn) ; Error 2 = |𝑓(𝑥𝑘 )|
32 err = min(err1, err2) ;
33
34 cout<< "After " << k–1 << " iterations, the approximate root = " << xn ;
35 cout<< " f(x) = " << fxn << " Error = " << err1 << endl ;
36
37 if ( err < TOL ) break ;
38 else if ( fx0*fxn < 0 )
39 {
40 x1 = xn ;
41 fx1 = fxn ;
42 }
43 else
44 {
45 x0 = xn ;
46 fx0 = fxn ;
47 }
48
49 }
Solution of a Nonlinear Equation in One Variable 23

50
51 if ( err < TOL )
52 cout<< "The desired accuracy achieved; Solution converged." ;
53 else
54 cout<< "The number of iterations exceeded the maximum limit." ;
55 }

Question 41: Write down the algorithm (pseudo code) of the Regula-Falsi method to solve 𝑓(𝑥) = 0.

Algorithm: To solve 𝑓(𝑥) = 0 using the iterative formula (given the root containing interval):
𝑓(𝑥𝑘−1 )(𝑥𝑘−1 − 𝑥𝑘−2 )
𝑥𝑘 = 𝑥𝑘−1 − , for 𝑘 = 2, 3, 4, ⋯
𝑓(𝑥𝑘−1 ) − 𝑓(𝑥𝑘−2 )
𝒂 and 𝒃: two real values as the initial approximations bracketing the root
𝐈𝐍𝐏𝐔𝐓𝐒: {𝑻𝑶𝑳: a real value as the absolute error tolerance
𝑵: an integer as the maximum number of iterations
𝒙𝒏: a real value as the approximate solution
𝐎𝐔𝐓𝐏𝐔𝐓: {
(either on convergence or on completing 𝑵 iterations − whichever happens first)

Step 1 Receive the inputs as stated above

Step 2 Set 𝒙𝒏 = 𝒃 (initialize 𝒙𝒏 with any of the two endpoints)

Step 3 Set 𝒙𝟎 = 𝒂
Set 𝒙𝟏 = 𝒃
Set 𝒇𝒙𝟎 as the value of 𝑓(𝒙𝟎)
Set 𝒇𝒙𝟏 as the value of 𝑓(𝒙𝟏)

Step 4 for 𝒌 = 𝟐, 𝟑, ⋯ , 𝑵 + 𝟏 perform Steps 5-10

𝒙𝒑 is to keep a copy of the approximation 𝒙𝒏,
Step 5 Set 𝒙𝒑 = 𝒙𝒏 {
because 𝒙𝒏 is going to be updated.

Step 6
𝒇𝒙𝟏(𝒙𝟏 − 𝒙𝟎) Computing a new
𝒙𝒏 = 𝒙𝟏 − {
𝒇𝒙𝟏 − 𝒇𝒙𝟎 approximation to the root

Step 7 Set 𝒇𝒙𝒏 as the value of 𝑓(𝒙𝒏)

Step 8 Set 𝒆𝒓𝒓𝟏 = |𝒙𝒏 − 𝒙𝒑|/|𝒙𝒏| (or 𝑒𝑟𝑟 = |𝑥𝑛 − 𝑥𝑝|)

Set 𝒆𝒓𝒓𝟐 = |𝒇𝒙𝒏|
Set 𝒆𝒓𝒓 = 𝒎𝒊𝒏(𝒆𝒓𝒓𝟏, 𝒆𝒓𝒓𝟐)

Step 9
24 Simplified Numerical Analysis

This means that either 𝒇(𝒙𝒏) is the close to

if (𝒆𝒓𝒓 < 𝑻𝑶𝑳 )then
} zero, or the consecutive approximations are
Exit/Break the loop
nearly the same. Therefore, stop iterations.
else if (𝒇(𝒙𝟎)𝒇(𝒙𝒏) < 0) then
Set 𝒙𝟏 = 𝒙𝒏 Adjusting one endpoint
Set 𝒇𝒙𝟏 = 𝒇𝒙𝒏 of the interval such that
else a shorter interval will be
Set 𝒙𝟎 = 𝒙𝒏 used in the next iteration
Set 𝒇𝒙𝟎 = 𝒇𝒙𝒏 }

end for (Go to Step 5 for the next iteration)

Step 10 Print the output: 𝒙𝒏

[Additionally, the starting interval [𝑎, 𝑏], number of iterations (𝒌 − 𝟏), 𝑓(𝒙𝒏), and error (𝒆𝒓𝒓) can be printed]

if (𝒆𝒓𝒓 < 𝑻𝑶𝑳) OUTPUT (‘The desired accuracy achieved; Solution converged.’)

else OUTPUT (‘The number of iterations exceeded the maximum limit.’)

STOP.

Problem 30: Write a C++ program to find a real root of the equation 𝑓(𝑥) = 4𝑥 + sin 𝑥 − 𝑒 𝑥 = 0 in [0, 1]
using the Regula-Falsi method. The two function values at the endpoints of the interval have opposite signs.
The iterations of the method should stop when either the approximation is accurate within 10−5 , or the
number of iterations exceeds 100, whichever happens first.

1 # include <iostream>
2 # include <cmath>
3 using namespace std ;
4
5 # define TOL 0.000001 // error tolerance
6 # define N 100 // maximum number of iterations
7
8 int main ( )
9 {
10 int k ;
11 double x0 , x1 , xn , xp , fx0 , fx1 , fxn , fxp , err , err1 , err2 ;
12 cout<< "Enter the left endpoint of the interval: " ; cin>> a ;
13 cout<< "Enter the right endpoint of the interval: " ; cin>> b ;
14
15 //----------------------- Processing Section -------------------------//
16
17 xn = b ;
18 x0 = a ;
19 x1 = b ;
20 fx0 = 4*x0 + sin(x0) – exp(x0) ; // Evaluating f(x) at x0
21 fx1 = 4*x1 + sin(x1) – exp(x1) ; // Evaluating f(x) at x1
22
Solution of a Nonlinear Equation in One Variable 25

23 for ( k=2 ; k<=N+1 ; k++)

24 {
25 xp = xn ;
26
27 xn = x1 – (fx1 * (x1 – x0)) / (fx1 – fx0) ; 𝑓(𝑥𝑘−1 )(𝑥𝑘−1 − 𝑥𝑘−2 )
𝑥𝑘 = 𝑥𝑘−1 −
28 fxn = 4*xn + sin(xn) – exp(xn) ; 𝑓(𝑥𝑘−1 ) − 𝑓(𝑥𝑘−2 )
29
30 err1 = abs(xn – xp)/abs(xn) ; Error 1 = |𝑥𝑘 − 𝑥𝑘−1 |/|𝑥𝑘 |
31 err2 = abs(fxn) ; Error 2 = |𝑓(𝑥𝑘 )|
32 err = min(err1, err2) ;
33
34 cout<< "After " << k–1 << " iterations, the approximate root = " << xn ;
35 cout<< " f(x) = " << fxn << " Error = " << err1 << endl ;
36
37 if ( err < TOL ) break ;
38 else if ( fx0*fxn < 0 )
39 {
40 x1 = xn ;
41 fx1 = fxn ;
42 }
43 else
44 {
45 x0 = xn ;
46 fx0 = fxn ;
47 }
48
49 }
50
51 if ( err < TOL )
52 cout<< "The desired accuracy achieved; Solution converged." ;
53 else
54 cout<< "The number of iterations exceeded the maximum limit." ;
55 }

Problem 32: Write a C++ program to find a real root of the equation 𝑓(𝑥) = 4𝑥 + sin 𝑥 − 𝑒 𝑥 = 0 in [0, 1]
using the Regula-Falsi method. The two function values at the endpoints of the interval have opposite signs.
Use #define directive to evaluate 𝑓(𝑥) wherever required. The iterations of the method should stop when
either the approximation is accurate within 10−5 , or the number of iterations exceeds 100, whichever
happens first.

1 # include <iostream>
2 # include <cmath>
3 using namespace std ;
4
5 # define TOL 0.000001 // error tolerance
26 Simplified Numerical Analysis

6 # define N 100 // maximum number of iterations

7 # define fval(x) 4 * x + sin(x) – exp(x) // Evaluating f(x)
8
9 int main ( )
10 {
11 int k ;
12 double x0 , x1 , xn , xp , fx0 , fx1 , fxn , fxp , err , err1 , err2 ;
13 cout<< "Enter the left endpoint of the interval: " ; cin>> a ;
14 cout<< "Enter the right endpoint of the interval: " ; cin>> b ;
15
16 //----------------------- Processing Section -------------------------//
17
18 xn = b ;
19 x0 = a ;
20 x1 = b ;
21 fx0 = fval(x0) ; // Evaluating f(x) at x0
22 fx1 = fval(x1) ; // Evaluating f(x) at x1
23
24 for ( k=2 ; k<=N+1 ; k++)
25 {
26 xp = xn ;
27
28 xn = x1 – (fx1 * (x1 – x0)) / (fx1 – fx0) ; 𝑓(𝑥𝑘−1 )(𝑥𝑘−1 − 𝑥𝑘−2 )
𝑥𝑘 = 𝑥𝑘−1 −
29 fxn = fval(xn) ; 𝑓(𝑥𝑘−1 ) − 𝑓(𝑥𝑘−2 )
30
31 err1 = abs(xn – xp)/abs(xn) ; Error 1 = |𝑥𝑘 − 𝑥𝑘−1 |/|𝑥𝑘 |
32 err2 = abs(fxn) ; Error 2 = |𝑓(𝑥𝑘 )|
33 err = min(err1, err2) ;
34
35 cout<< "After " << k–1 << " iterations, the approximate root = " << xn ;
36 cout<< " f(x) = " << fxn << " Error = " << err1 << endl ;
37
38 if ( err < TOL ) break ;
39 else if ( fx0*fxn < 0 )
40 {
41 x1 = xn ;
42 fx1 = fxn ;
43 }
44 else
45 {
46 x0 = xn ;
47 fx0 = fxn ;
48 }
49
50 }
Solution of a Nonlinear Equation in One Variable 27

51
52 if ( err < TOL )
53 cout<< "The desired accuracy achieved; Solution converged." ;
54 else
55 cout<< "The number of iterations exceeded the maximum limit." ;
56 }

∎
Remark: An interesting online calculator by CASIO ® at https://keisan.casio.com has the
following webpage to approximate the root of a non-linear equation using different methods.

https://keisan.casio.com/menu/system/000000001000

∎∎∎
28 Simplified Numerical Analysis

Chapter Summary

• The root-finding problem refers to find some appropriate value 𝑥 = 𝛼 in the domain of a function 𝑓 such
that 𝑓(𝛼) = 0. Every such possible value 𝛼 is called a root of the equation 𝑓(𝑥) = 0.

• Geometrically, a root of an equation 𝑓(𝑥) = 0 is the point where the graph of 𝑓 intersects the 𝑥-axis.

• An iterative numerical method to approximate the root starts with some appropriate or reasonable
estimation (also called initial approximation or guess) of the exact root and attempts to refine the
approximation, iteratively. The iterations are repeated until a desired level of accuracy is achieved.

• Let 𝑥0 denotes the initial approximation and 𝑥1 , 𝑥2 , 𝑥3 , ⋯ denote the successive iterative solutions to an
exact root 𝛼 of the equation 𝑓(𝑥) = 0. The sequence {𝑥𝑘 }∞ 𝑘=0 of the successive approximations is said to
converge to the exact root 𝛼, if the successive approximations approach 𝛼. In such a case, the iterative
method is also said to converge. In other words, the iterative method is said to be convergent for a
given initial approximation if the corresponding sequence of successive approximations is convergent
to the exact solution. Under certain conditions, it is possible for an iterative method that the sequence of
successive approximations might diverge from a desired exact root 𝛼.

• Stopping Criteria: The most common convergence criterion to stop the iterative process is based on
the comparison of the estimated error with the error tolerance. For this purpose, the current
approximation is considered as the true solution and the previous approximation is considered as the
approximate solution for estimating the error and any appropriate one of the following criteria is used,
(1) |𝑥𝑘 − 𝑥𝑘−1 | ≤ 𝜏

𝑥𝑘 − 𝑥𝑘−1
(2) | | ≤ 𝜏
𝑥𝑘

𝑥𝑘 − 𝑥𝑘−1
(3) | | × 100 ≤ 𝜏
𝑥𝑘

Here 𝑥𝑘 and 𝑥𝑘−1 denote the current and previous approximations, respectively, and 𝜏 denotes the
tolerance.

• Another Stopping Criterion: Note that the values of the function 𝑓 tend to zero with the progress of
the iterative process. Thus, falling of the difference between the function values and zero beyond a
certain level might also indicate convergence.

• The numerical methods of finding a root of 𝑓(𝑥) = 0 can be categorized as bracketing methods and
open methods.

• Bracketing methods start with an interval containing a root and squeeze down the interval, iteratively.
Two well-known root bracketing methods are the Bisection method and the Regula-Falsi (False-
Position) method.
Solution of a Nonlinear Equation in One Variable 29

• Open Methods are those who obtain successive single approximations irrespective of their location at
any side of the root. Some of the well-known open methods are the Fixed-Point Iteration method, the
Newton-Raphson method (Newton’s method), and the Secant method.

• A bracketing method for finding a root/zero of a continuous function 𝑓 starts with an interval [𝑎, 𝑏]
containing a root. The opposite signs of 𝑓(𝑎) and 𝑓(𝑏) ensure (due to the Intermediate value theorem)
that there exists a root 𝛼 of 𝑓(𝑥) = 0 in (𝑎, 𝑏). To get closer to the root 𝛼, first a point 𝑐 ∈ (𝑎 + 𝑏) is
chosen. If 𝑓(𝑐) = 0, then 𝑐 is the exact root. Otherwise, either of the intervals [𝑎, 𝑐] or [𝑐, 𝑏] is chosen as
the squeezed interval containing the root. The root lies in [𝑎, 𝑐] if 𝑓(𝑎)𝑓(𝑐) < 0, or in [𝑐, 𝑏] if 𝑓(𝑐)𝑓(𝑏) <
0. The selected interval is relabeled as [𝑎, 𝑏] and the process is repeated. This way, a sequence of points
𝑐1 , 𝑐2 , 𝑐3 , ⋯, is formed. The iterations are performed until the approximations of the root of 𝑓(𝑥) in two
consecutive iterations are sufficiently close to each other.

• The Bisection method selects 𝑐 ∈ (𝑎 + 𝑏), as the midpoint of the interval [𝑎, 𝑏], using the formula
(𝑎 + 𝑏)
𝑐=
2

• The Regula-Falsi method selects 𝑐 ∈ (𝑎 + 𝑏), as the point where the line segment joining 𝑓(𝑎) and
𝑓(𝑏) intersects the 𝑥-axis, using the formula
𝑓(𝑏)(𝑏 − 𝑎)
𝑐 = 𝑏−
𝑓(𝑏) − 𝑓(𝑎)

• For the Bisection method, the error-bound is given by,

𝑏−𝑎
|𝛼 − 𝑐𝑘 | ≤ , for 𝑘 = 1, 2, 3, ⋯,
2𝑘
𝑎𝑘−1 +𝑏𝑘−1
Here 𝛼 is the exact root of the equation 𝑓(𝑥) = 0 in (𝑎, 𝑏) and 𝑐𝑘 = is the midpoint of the
2
interval in 𝑘th iteration.

• The formula to determine the maximum number of iterations 𝑁 of the Bisection method after which the
error associated with any point in the squeezed interval is not greater than a given permissible absolute
error 𝜏𝑎 is as below:
log(𝑏 − 𝑎) − log(𝜏𝑎 )
𝑁≥
log(2)

This formula tells that, for an interval of unit length, it is sure that after 10, 14, 17, and 20 iterations the
length of the squeezed interval (or the absolute error) is not greater than 10−3, 10−4 , 10−5 , and 10−6 ,
respectively.

• The Fixed-Point Iteration method is an open method that approximates a root of the equation 𝑓(𝑥) =
0 by rearranging the equation 𝑓(𝑥) = 0 to get an appropriate form 𝑥 = 𝑔(𝑥) and generating a sequence
of successive approximations {𝑥𝑘 }∞
𝑘=1 by the iterative formula 𝑥𝑘 = 𝑔(𝑥𝑘−1 ), for 𝑘 = 1, 2, 3, ⋯ . The said
sequence may

o converge but could be different for different forms of 𝑥 = 𝑔(𝑥),

30 Simplified Numerical Analysis

o converge but could be different for different choices of the initial approximation 𝑥0 for a
particular form of 𝑥 = 𝑔(𝑥), or

o diverge for some unsuitable form of 𝑥 = 𝑔(𝑥) or an initial approximation 𝑥0 .

• Suppose that 𝑓 is a continuous function and the equation 𝑓(𝑥) = 0 has a real root 𝛼. Suppose that the
equation 𝑓(𝑥) = 0 can be rearranged in the form 𝑥 = 𝑔(𝑥) such that 𝛼 is a fixed-point of the function 𝑔,
and 𝑔 and 𝑔’ are continuous in some neighbourhood 𝐼 around 𝛼. If

|𝑔′(𝑥)| ≤ 𝐾 < 1, for all 𝑥 ∈ 𝐼,

then for any initial approximation 𝑥0 ∈ 𝐼, the sequence {𝑥𝑘 }∞𝑘=1 of successive approximations, generated
by the iterative formula 𝑥𝑘 = 𝑔(𝑥𝑘−1 ), for 𝑘 = 1, 2, 3, ⋯, converges to the solution 𝛼.

• To find a root of a non-linear equation 𝑓(𝑥) = 0 the Newton-Raphson method requires an initial
solution 𝑥0 and considers the 𝑥-intercept of the tangent line to the function 𝑓(𝑥) at 𝑥 = 𝑥0 as the new
approximation. Then, the 𝑥-intercept of the tangent line to the function at the new approximation is
considered as the next approximation. This way, the process is repeated with the successive
approximations until sufficient convergence is achieved. The formula to generate the sequence of
successive approximations based on the said approach is given by

𝑓(𝑥𝑘−1 )
𝑥𝑘 = 𝑥𝑘−1 − , for 𝑘 = 1, 2, 3, ⋯
𝑓 ′ (𝑥𝑘−1 )

• A sufficient condition of convergence for the Newton-Raphson method: Suppose that 𝛼 is a root of the
equation 𝑓(𝑥) = 0. Suppose that 𝐼 is a neighbourhood of 𝛼 such that 𝑓(𝑥), 𝑓′(𝑥) and 𝑓′′(𝑥) are
continuous on 𝐼. If |𝑓(𝑥)𝑓′′(𝑥)| ≤ |𝑓′(𝑥)|2 , for all 𝑥 ∈ 𝐼, then for an initial approximation 𝑥0 ∈ 𝐼, the
sequence {𝑥𝑘 }∞𝑘=1 of successive approximations, generated by the Newton’s formula, converges to the
solution 𝛼.

• The iterative formula of the Secant method for solving 𝑓(𝑥) = 0 (with 𝑥 = 𝑥0 and 𝑥 = 𝑥1 as the initial
approximations) is given by
𝑓(𝑥𝑘−1 )(𝑥𝑘−1 − 𝑥𝑘−2 )
𝑥𝑘 = 𝑥𝑘−1 − , for 𝑘 = 2, 3, 4, ⋯
𝑓(𝑥𝑘−1 ) − 𝑓(𝑥𝑘−2 )

• Comparison of the False-Position method and the Secant method:

o The False-Position method is a bracketing method, whereas the Secant method is an open method.

o The False-Position method keeps the root bracketed by selects out the root bracketing subintervals
out the two subintervals obtains in each of the iterations. On the other hand, the Secant method
selects the two most recent approximations out of the three available approximations in any
iteration to proceed to the next iteration.

o The False-Position method always converges, whereas the Secant method may not converge for
certain situations.
Solution of a Nonlinear Equation in One Variable 31

o If the Secant method is convergent, it converges faster than the False-Position method. That is, it
has a higher convergence rate than that of the False-Position method.

• The order/rate of convergence of the Bisection method is 1 (i.e., linear) and the asymptotic error
constant is (1⁄2)

• The order/rate of convergence of the False-Position or Regula-Falsi method is 1 (i.e., linear) and the
1 𝑓′′ (𝛼)
asymptotic error constant is − 𝜀
2 𝑓′ (𝛼) 0

• The order/rate of convergence of the Fixed-Point Iteration method is 1 (i.e., linear) and the asymptotic
error constant is the maximum value of the function 𝑔′(𝑥) in some neighbourhood around the
solution 𝛼.

• The order/rate of convergence of the Newton-Raphson method is 2 (i.e., quadratic) and the asymptotic
1 𝑓′′ (𝛼)
error constant is −
2 𝑓′ (𝛼)

• The order/rate of convergence of the Secant method is 1.62 (i.e., superlinear).

• The Newton-Raphson method may fail to converge to a root in different situations including where
𝑓′(𝑥) or 𝑓′′(𝑥) becomes zero at any approximation.

• The Newton-Raphson method converges to a multiple root very slowly (instead of exhibiting quadratic
convergence).

• The Aitken’s ∆𝟐 method offers a technique for accelerating the convergence of any sequence that is
linearly convergent. From the given sequence {𝑥𝑘 }∞ 𝑘=1 that linearly converges to 𝛼, another sequence
𝑘=1 that also converges to 𝛼 with possibly improved convergence rate is constructed by using the
{𝑥̿𝑘 }∞
Aitken’s acceleration formula given as

(∆𝑥𝑘 )2
𝑥̿𝑘 ≅ 𝑥𝑘 −
∆2 𝑥𝑘
∎∎∎
32 Simplified Numerical Analysis

Chapter Exercises

Exercise 01: Find a real root of the following equations using the Bisection method accurate to
four decimal places.
(i) log(𝑥) − cos 𝑥 = 0
(ii) 𝑒 −𝑥 − 10𝑥 = 0
(iii) 𝑥3 + 𝑥2 − 1 = 0

Exercise 02: Find a real root of the following equations using the Bisection method accurate to
three decimal places.
(i) 𝑥6 − 𝑥4 − 𝑥3 − 1 = 0
(ii) 𝑥 3 − sin 𝑥 + 1 = 0
(iii) 𝑥 log10 𝑥 = 4.77

Exercise 03: Approximate the solution of the following equations using the Regula-Falsi method
accurate to three decimal places.
(i) 3𝑥 + sin 𝑥 − 𝑒 𝑥 = 0
2
(ii) 4𝑥 3 − 1 − 𝑒 (𝑥 /2)
(iii) 𝑥 2 = (𝑒 −2𝑥 − 2)/𝑥

𝑒𝑥
Exercise 04: Find the approximation to a real root of the equation 2 sin 𝑥 − − 1 = 0 starting
4
with [−5, −3] using the Regula-Falsi method.

Exercise: Find a real root of each of the following equations using (𝑎) the Bisection method, (𝑏)
the Regula-Falsi method, (𝑐) the Newton’s method, (𝑑) the Secant method. Choose the initial
approximation/s in the given interval. Assume that the tolerance for the approximate root is
0.001. The numeric values should not be rounded to less than 5 decimal places. (𝑥 is in radians,
wherever applicable).

(𝑖) cos 𝑥 − 𝑥𝑒 𝑥 = 0, in [0, 1]

(𝑖𝑖) cos 𝑥 − 𝑥 + 2 = 0, in [1, 2]
(𝑖𝑖𝑖) 𝑒 𝑥 − 𝑥 − 3 = 0, in [1, 2]
(𝑖𝑣) ln(𝑥) + 𝑥 − 4 = 0, in [2, 3]
(𝑣) 4𝑥 + sin 𝑥 − 𝑒 𝑥 = 0, in [0, 1].

Exercise 06: Find a real root of the Chebyshev polynomial of degree four, 𝑇4 (𝑥) = 8𝑥 4 − 8𝑥 2 + 1
using the Newton’s method accurate to four decimal places.
Solution of a Nonlinear Equation in One Variable 33

Exercise 07: Find a root of the Laguerre polynomial of degree four, 𝐿4 (𝑥) = 𝑥 4 − 16𝑥 3 + 72𝑥 2 −
96𝑥 + 24 using the Newton’s method accurate to four decimal places.

Exercise 08: Find a root of the following equations using the Newton’s method accurate to 4
decimal places.
(i) 2𝑥 + 3 cos 𝑥 − 𝑒 𝑥 = 0,
(ii) 𝑥 2 − 4𝑥 + 4 − ln 𝑥 = 0
(iii) tan 𝑥 − 6 = 0

Exercise 09: Find the roots accurate to within 10−3 of the Legendre polynomial 𝑃4 (𝑥) = 𝑥 4 −
6 3
𝑥2 + on each interval, using the Secant method.
7 35

(i) [−1, −0.5]

(ii) [−0.5, 0]
(iii) [0, 0.5]
(iv) [0.5, 1]

3
Exercise 10: Approximate the value of √4 using the Secant method accurate to 10−4 .

Exercise 11: Find a real root of the following equations using the Secant method accurate to 10−3
.
(i) 𝑥 3 − 2𝑥 + 2 = 0
(ii) 10 − 2𝑥 + sin 𝑥 = 0
(iii) 2𝑒 −3 𝑥 + 1 − 3𝑒 −3𝑥 = 0

Exercise 12: Use the Fixed-Point method to find a root of the following, accurate to 3 decimal
places.
(i) 𝑒 𝑥 − 2𝑥 2 for 0 ≤ 𝑥 ≤ 2
(ii) 𝑥𝑒 𝑥 = 0 for 1 ≤ 𝑥 ≤ 2
(iii) 𝑥 2 − sin 𝑥 − 𝑥 = 0

Exercise 13: Find the solutions of the following equations using the fixed-point method accurate
to 10−3 .
(i) 𝑥 = tan 𝑥
(ii) 𝑥 = cos 𝑥
(iii) 𝑥 = sin(𝑥 + 2)

Exercise 14: Find the solution of the equation (relevant to the vibrating beam),
cos 𝑥 cosh 𝑥 = 1
34 Simplified Numerical Analysis

3
near 𝑥 = − 𝜋 using the Newton-Raphson method.
2

Exercise 15: The velocity 𝑉, in meters per second (𝑚/𝑠), of a free falling sky diver is expressed
as:
𝑔𝑚 −𝐷𝑐 𝑡
𝑉= (1 − 𝑒𝑥𝑝 ( ))
𝐷𝑐 𝑚
Here 𝑚 is the mass of the falling body in kilograms (𝑘𝑔), 𝐷𝑐 is the drag coefficient in kilogram per
second (𝑘𝑔/𝑠), 𝑡 is the time in seconds (𝑠), and 𝑔 = 9.8𝑚/𝑠 2 is the gravitation acceleration. If the
velocity of a body of mass 85𝑘𝑔 is 40𝑚/𝑠 after 5 seconds of free fall, then calculate the drag
coefficient.

Hint for the Solution:

Given 𝑚 = 80𝑘𝑔, 𝑉 = 40𝑚/𝑠, 𝑔 = 9.8𝑚/𝑠 2 , and 𝑡 = 5𝑠, the equation takes the form:

(9.8)(85) −5𝐷𝑐
40 = (1 − 𝑒𝑥𝑝 ( ))
𝐷𝑐 85
or
𝑓(𝐷𝑐 ) = 𝐷𝑐 + 17 ln(1 − 0.04802𝐷𝑐 ) = 0
Solve this equation for 𝐷𝑐 , using any appropriate iterative method. To obtain an initial guess of 𝐷𝑐 ,
a trick is to calculate 𝑉 for different assumed values of 𝐷𝑐 . The values of the 𝐷𝑐 , which produce
values of 𝑉 close to 40, can offer reasonable initial guess of 𝐷𝑐 . While using an iterative method,
approximate error should be calculated at each iteration. [𝑒𝑥𝑝(𝑥) = 𝑒 𝑥 ]

Exercise 16: The velocity 𝑉, in meters per second (𝑚/𝑠), of a free falling sky diver is expressed
as:
𝑔𝑚 −𝐷𝑐 𝑡
𝑉= (1 − 𝑒𝑥𝑝 ( ))
𝐷𝑐 𝑚
Here 𝑚 is the mass of the falling body in kilograms (𝑘𝑔), 𝐷𝑐 is the drag coefficient in kilogram per
second (𝑘𝑔/𝑠), 𝑡 is the time in seconds (𝑠), and 𝑔 = 9.8𝑚/𝑠 2 is the gravitation acceleration. If the
velocity of a falling body with drag coefficient of 18 𝑘𝑔/𝑠 is 50𝑚/𝑠 after 7 seconds of free fall, then
calculate the mass 𝑚 of the body, accurate to 0.0001. [𝑒𝑥𝑝(𝑥) = 𝑒 𝑥 ]

Hint for the Solution:

Given 𝐷𝑐 = 18𝑘𝑔/𝑠, 𝑉 = 50𝑚/𝑠, 𝑔 = 9.8𝑚/𝑠 2 , and 𝑡 = 7𝑠, the equation takes the form:

(9.8)𝑚 −126
50 = (1 − 𝑒𝑥𝑝 ( ))
18 𝑚
or
Solution of a Nonlinear Equation in One Variable 35

91.83673
𝑓(𝑚) = 𝑚 ln (1 − ) + 126 = 0
𝑚
Solve this equation for 𝑚, using any appropriate iterative method. To obtain an initial guess of 𝑚,
a trick is to calculate 𝑉 for different assumed values of 𝑚. The values of the 𝑚, which produce
values of 𝑉 close to 50, can offer reasonable initial guess of 𝑚. While using an iterative method,
approximate the error at each iteration.

Exercise 17: The volume 𝑉 of spherical water-tank in cubic meters can be calculated as:
𝜋𝐻2 (3𝑅 − 𝐻)
𝑉=
3
where 𝐻 denotes the height of water level in meters from the base of the tank, and 𝑅 denotes the
radius of the spherical tank in meters. If the radius 𝑅 of a tank is 2.5 meters, then how much water
level must be raised in the tank to hold 27 cubic meters of water.

Hint for the Solution:

Given 𝑅 = 2.5 and 𝑉 = 27, and taking 𝜋 = 3.14159 the equation takes the form

𝜋𝐻2 (7.5 − 𝐻)
27 =
3
or
𝑓(𝐻) = 3.14159𝐻3 − 23.56193𝐻2 + 81 = 0

Solve this equation for H, using any appropriate iterative method. Intuitively, appropriate initial
guesses for 𝐻 can be taken from [0,2𝑅]. While using an iterative method, approximate error
should be calculated at each iteration.

Exercise 18: Numerically, compare the convergence of the method:

𝑓(𝑥𝑘−1 )
𝑥𝑘 = 𝑥𝑘−1 − 2 , for 𝑘 = 1, 2, 3, ⋯
𝑓 ′ (𝑥𝑘−1 )

with the Newton-Raphson method on a function with a known double root.

Exercise 19: The ideal gas equation relates the volume (𝑉 in 𝐿), temperature (𝑇 in 𝐾), pressure
(𝑃 in 𝑎𝑡𝑚), and the amount of gas (number of moles 𝑛) by:
𝑛𝑅𝑇
𝑃=
𝑉
where 𝑅 = 0.08206 (𝐿 𝑎𝑡𝑚)/(mol 𝐾) is the gas constant.
The van der Waals equation gives the relationship between these quantities for a real gas by
𝑛2 𝑎
(𝑃 + ) (𝑉 − 𝑛𝑏) = 𝑛𝑅𝑇
𝑉2
36 Simplified Numerical Analysis

where 𝑎 and 𝑏 are constants that are specific for each gas.
Calculate the volume of 2 mol 𝐶𝑂2 at temperature of 50°C, and pressure of 6 𝑎𝑡𝑚. For 𝐶𝑂2 , 𝑎 =
3.59 (𝐿2 𝑎𝑡𝑚)/𝑚𝑜𝑙 2 , and b = 0.0427 L/ mol. Because 𝐶𝑂2 is a real gas, so we need to use the
second equation for the solution. But for solving the second equation for the volume, obtain an
appropriate guess of the volume from the first equation: ideal gas equation.

Exercise 20: Golden-ratio corresponds to the order of which method:

(A) Secant (B) Regula-Falsi (C) Fixed-Point Iteration (D) Newton-Raphson

Exercise 21: Which of the following methods, has an explicit formula that can be used to
determine the required number of iterations in advance for achieving a given accuracy:
(A) Bisection (B) Regula-Falsi (C) Fixed-Point Iteration (D) Newton-Raphson (E)
Secant

Exercise 22: The convergence rate of which of the following methods is highest:
(A) Bisection (B) Regula-Falsi (C) Fixed-Point Iteration (D) Newton-Raphson (E)
Secant

∎∎∎
 Chapter 3
Polynomial Interpolation

Corridor I: BASICS
Let’s plan it

3.1 Introduction
3.2 The Newton’s Divided Difference Interpolation
3.3 The Lagrange Interpolation
3.4 Deriving the Lagrange Interpolation Formula from the Newton’s Divided-Difference
3.5 Interpolation Formulas for Equally Spaced Nodes
3.6 Hermite Interpolation
3.7 Spline Interpolation
3.7.1 Linear Spline
3.7.2 Quadratic Spline
3.7.3 Cubic Spline

To unleash the topics of this Corridor, please delve into the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

37
38 Simplified Numerical Analysis

Corridor II: ANALYSIS

Let’s think deep

3.8 Error of Interpolation

To unleash the topics of this Corridor, please delve into the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

Corridor III: PROGRAMMING ARCADE

Let’s do it

3.9 Algorithms and Implementations

The Newton's Divided Difference Interpolation Formula

To cross-check the results/output of the computer programs you would execute, please delve into
the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

3.9 Algorithms and Implementations

Polynomial Interpolation 39

Question 21: Write down an algorithm (pseudo code) to interpolate or extrapolate the function
at a point using the 𝑛th-degree Newton’s Divided difference interpolating polynomial.

Algorithm: Given 𝒏 + 1 data points, approximate 𝑓(𝑥) at 𝑥 = 𝒙𝒑 with 𝑃𝒏 (𝒙𝒑) .

𝒏: an integer as the degree of interpolating polynomial

𝒙 , 0 ≤ 𝑖 ≤ 𝒏: real values as the aribrary nodes
𝐈𝐍𝐏𝐔𝐓𝐒: { 𝒊
𝒇𝒊 , 0 ≤ 𝑖 ≤ 𝒏: real values as the function values corresponding to 𝒙𝒊 nodes
𝒙𝒑: real values as the entries

𝐎𝐔𝐓𝐏𝐔𝐓: 𝒇𝒙𝒑: a real number as an interpolated value at 𝒙 = 𝒙𝒑

Step 1 Receive the inputs as stated above

Step 2 for 𝑖 = 0, 1, ⋯ , 𝒏
𝑑𝑑𝑓𝑖,0 = 𝒇𝒊 (Computing zeroth divided differences, 𝑓[𝒙𝒊 ] = 𝒇𝒊 )

Step 3 (Computing the divided differences of order 1 to 𝒏)

for 𝑗 = 1, 2, ⋯ , 𝒏
for 𝑖 = 0, 1, ⋯ , 𝒏 − 𝑗 𝑓[𝑥𝑖 , ⋯ , 𝑥𝑖+𝑗 ] =
[𝑑𝑑𝑓𝑖+1,𝑗−1 − 𝑑𝑑𝑓𝑖,𝑗−1 ]} (𝑓[𝑥𝑖+1 , ⋯ , 𝑥𝑖+𝑗 ] − 𝑓[𝑥𝑖 , ⋯ , 𝑥𝑖+𝑗−1 ])
𝑑𝑑𝑓𝑖,𝑗 =
[𝑥𝑖+𝑗 − 𝑥𝑖 ] 𝑥𝑖+𝑗 − 𝑥𝑖

Step 4 (Evaluating the interpolation polynomial at 𝒙𝒑)

Set 𝑝𝑟𝑜 = 1
Set 𝒇𝒙𝒑 = 𝑑𝑑𝑓0,0 𝑛 𝑘−1

for 𝑘 = 1, 2, ⋯ , 𝒏 (𝑃𝑛 = 𝑓[𝑥0 ] + ∑ [𝑓[𝑥0 , ⋯ , 𝑥𝑘 ] ∏(𝑥𝑝 − 𝑥𝑡 )])

𝑘=1 𝑡=0
𝑝𝑟𝑜 = 𝑝𝑟𝑜 × (𝑥𝑝 − 𝒙𝑘−1 )
𝒇𝒙𝒑 = 𝒇𝒙𝒑 + 𝑝𝑟𝑜 × 𝑑𝑑𝑓0,𝑘 }

Step 5 Print the output: 𝒇𝒙𝒑

STOP.
40 Simplified Numerical Analysis

Problem 14: Write a C++ program for the second order Newton’s Divided Difference
Interpolation.

1 # include <iostream>
2 using namespace std ;
3 # define n 2 // degree of interpolating polynomial
4 int main ( )
5 {
6 char quit ;
7 int i , j , k ;
8 double xp , fxp , pro ;
9
10 //------------------------- Input Section -----------------------------//
11 double ddf[n+1][n+1] , f[n+1] = {5, 10, 12} , x[n+1] = {0, 1, 3} ;
12 cout<< "The Divided Difference Interpolation. " << endl ;
13 cout<< "Enter a real value it which the interpolation is to be obtained: \n" ;
14 cin>> xp ;
15
16 //----------------------- Processing Section -------------------------//
17 // Computing zeroth divided differences,f[x_i ] = f_i
18 for ( i=0 ; i<=n ; i++ )
19 ddf[i][0] = f[i] ;
20
21 // Computing the divided differences of order 1 to n
22 for ( j=1 ; j<=n ; j++ )
23 for ( i=0 ; i<=n-j ; i++ )
24 ddf[i][j] = ( ddf[i+1][j-1] - ddf[i][j-1] ) / ( x[i+j] - x[i] ) ;
25
26 // Evaluating the interpolation polynomial at xp
27 pro = 1 ; fxp = ddf[0][0] ;
28 for ( k=1 ; k<=n ; k++ )
29 {
30 pro = pro * ( xp - x[k-1] ) ;
31 fxp = fxp + pro * ddf[0][k] ;
32 }
33
34 //---------------------- Output Section -------------------------//
35 cout<< "The interpolate or extrapolate value of function at x = xp is " << endl ;
36 cout<< fxp ;
37 cin>> quit ;
38 return 0 ; }
Polynomial Interpolation 41

Problem 16: Write a C++ program for the Newton’s Divided Difference Interpolation.

1 # include <iostream>
2 using namespace std ;
3 # define n 10 // degree of interpolating polynomial
4
5 int main ( )
6 {
7 char quit ;
8 int i , j , k ;
9 double ddf[n+1][n+1] , f[n+1] , x[n+1] ;
10 double xp , fxp , pro ;
11
12 //------------------------- Input Section -----------------------------//
13
14 cout<< "The Divided Difference Interpolation. " << endl ;
15 cout<< "Enter real values as the arbitrary nodes "<< endl ;
16 for ( i=0 ; i<=n ; i++ )
17 cin>> x[i] ;
18 cout<< endl ;
19 cout<< " Enter real values as the function values corresponding to x_i
20 nodes "<< endl ;
21 for ( i=0 ; i<=n ; i++ )
22 cin>> f[i] ;
23 cout<< "Enter a real value it which the interpolation is to be obtained: \n" ;
24 cin>> xp ;
25 //----------------------- Processing Section -------------------------//
26 // Computing zeroth divided differences,f[x_i ] = f_i
27
28 for ( i=0 ; i<=n ; i++ )
29 ddf[i][0] = f[i] ;
30 // Computing the divided differences of order 1 to n
31
32 for ( j=1 ; j<=n ; j++ )
33 for ( i=0 ; i<=n-j ; i++ )
34 ddf[i][j] = ( ddf[i+1][j-1] - ddf[i][j-1] ) / ( x[i+j] - x[i] ) ;
35
36 // Evaluating the interpolation polynomial at xp
37
38 pro = 1 ; fxp = ddf[0][0] ;
39 for ( k=1 ; k<=n ; k++ )
42 Simplified Numerical Analysis

40 {
41 pro = pro * ( xp - x[k-1] ) ;
42 fxp = fxp + pro * ddf[0][k] ;
43 }
44
45 //---------------------- Output Section -------------------------//
46
47 cout<< "The interpolate or extrapolate value of function at x = xp is " << endl ;
48 cout<< fxp ;
49
50 cin>> quit ;
51 return 0 ;
52
53 }
∎∎

Chapter Summary

• Curve fitting refers to the process of constructing a curve (a mathematical function) that reasonably
fits the given discrete data points along a continuum. The obtained curve offers a simpler alternative to
the original function (whose values at discrete points were given) that might be used to estimate the
data values at points between the given points (and sometimes beyond the given data points, as well).

• Regression and Interpolation are the two basic approaches for curve fitting. Regression is the process
of deriving a single curve that provides for the general trend of the data (and that curve is not required
to pass through any of the data points). Interpolation is the process of fitting a curve (a single function
or a piecewise function) that interpolates (passes through) each of the given data points.

• Suppose that the values of a function 𝑓 at different points 𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑛 are given. The points 𝑥𝑖 are
referred to as nodes or arguments and the 𝑛 + 1 ordered pairs (𝑥𝑖 , 𝑓(𝑥𝑖 )), 𝑖 = 0, 1, 2, ⋯ , 𝑛, are referred
to as data points of 𝑓. Interpolation (or, more precisely, polynomial interpolation) refers to the process
of approximating the value of 𝑓 at any intermediate point to the given data points.

• The interpolation process consists of determining the unique polynomial 𝑃𝑛 (𝑥) of degree at most 𝑛 that
interpolates (passes through) the given data points, i.e.,
𝑃𝑛 (𝑥𝑖 ) = 𝑓(𝑥𝑖 )

And then, the polynomial 𝑃𝑛 (𝑥) serves as the formula to approximate the function values at
intermediate points to the given data points and, thus, is referred to as interpolating polynomial. If the
polynomial 𝑃𝑛 (𝑥) is used approximate the function values at beyond the given data points, then the
process is called extrapolation.
Polynomial Interpolation 43

• Newton’s Divided Difference Interpolation: For 𝑛 + 1 arbitrarily spaced data points, (𝑥0 , 𝑓0 ), (𝑥1 , 𝑓1 ),
⋯ , (𝑥𝑛 , 𝑓𝑛 ), of a function 𝑓, the Newton’s Divided Difference interpolation formula for the interpolating
polynomial 𝑃𝑛 (𝑥) of degree at most 𝑛 is given by
𝑃𝑛 (𝑥) = 𝑓[𝑥0 ] + 𝑓[𝑥0 , 𝑥1 ](𝑥 − 𝑥0 ) + 𝑓[𝑥0 , 𝑥1 , 𝑥2 ](𝑥 − 𝑥0 )(𝑥 − 𝑥1 )
+ ⋯ + 𝑓[𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑛 ](𝑥 − 𝑥0 )(𝑥 − 𝑥1 ) ⋯ (𝑥 − 𝑥𝑛−1 )
or
𝑛

𝑃𝑛 (𝑥) = 𝑓[𝑥0 ] + ∑ 𝑓[𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑘 ](𝑥 − 𝑥0 )(𝑥 − 𝑥1 ) ⋯ (𝑥 − 𝑥𝑘−1 )

𝑘=1

Here the 𝒌th divided difference of the function 𝑓 with respect to the nodes 𝑥𝑖 , 𝑥𝑖+1 , ⋯ , 𝑥𝑖+𝑘 is denoted
by 𝑓[𝑥𝑖 , 𝑥𝑖+1 , ⋯ , 𝑥𝑖+𝑘 ] and is recursively defined by

𝑓[ 𝑥𝑖+1 , 𝑥𝑖+2 , ⋯ , 𝑥𝑖+𝑘 ] − 𝑓[𝑥𝑖 , 𝑥𝑖+1 , ⋯ , 𝑥𝑖+(𝑘−1) ]

𝑓[𝑥𝑖 , 𝑥𝑖+1 , ⋯ , 𝑥𝑖+𝑘 ] =
𝑥𝑖+𝑘 − 𝑥𝑖

with 𝑓[𝑥𝑖 ] = 𝑓(𝑥𝑖 ) = 𝑓𝑖 as the zeroth divided difference.

• Lagrange Interpolation: For 𝑛 + 1 arbitrarily spaced data points, (𝑥0 , 𝑓0 ), (𝑥1 , 𝑓1 ), ⋯ , (𝑥𝑛 , 𝑓𝑛 ), of a
function 𝑓, the Lagrange interpolation formula for the interpolating polynomial 𝑃𝑛 (𝑥) of degree at most
𝑛 is given by

𝑃𝑛 (𝑥) = 𝐿0 (𝑥)𝑓(𝑥0 ) + 𝐿1 (𝑥)𝑓(𝑥1 ) + ⋯ + 𝐿𝑛 (𝑥)𝑓(𝑥𝑛 )

= ∑ 𝐿𝑘 (𝑥) 𝑓(𝑥𝑘 )
𝑘=0

Here 𝐿𝑘 (𝑥) denotes the 𝒌th Lagrange coefficient (also called cardinal polynomial) and is defined by

𝑛
(𝑥 − 𝑥0 )(𝑥 − 𝑥1 ) ⋯ (𝑥 − 𝑥𝑘−1 )(𝑥 − 𝑥𝑘+1 ) ⋯ (𝑥 − 𝑥𝑛 ) 𝑥 − 𝑥𝑗
𝐿𝑘 (𝑥) = = ∏
(𝑥𝑘 − 𝑥0 )(𝑥𝑘 − 𝑥1 ) ⋯ (𝑥𝑘 − 𝑥𝑘−1 )(𝑥𝑘 − 𝑥𝑘+1 ) ⋯ (𝑥𝑘 − 𝑥𝑛 ) 𝑥𝑘 − 𝑥𝑗
𝑗=0
𝑗≠𝑘

and satisfies the Kronecker delta equation:

1 for 𝑥 = 𝑥𝑘
𝐿𝑘 (𝑥) = {
0 for all 𝑥, except 𝑥 = 𝑥𝑘

• First Theorem on Interpolation Error: If 𝑃𝑛 (𝑥) is the polynomial of degree at most 𝑛 that interpolates
a function 𝑓 at 𝑛 + 1 arbitrary nodes 𝑥0 , 𝑥1 , ⋯ , 𝑥𝑛 in an interval [𝑎, 𝑏] and if 𝑓 ∈ 𝐶 (𝑛+1) [𝑎, 𝑏], then for
each 𝑥 in [𝑎, 𝑏], there exists an 𝜉 in (𝑎, 𝑏) for which

𝑓 (𝑛+1) (𝜉)
𝐸(𝑥) = 𝑓(𝑥) − 𝑃𝑛 (𝑥) = (𝑥 − 𝑥0 )(𝑥 − 𝑥1 ) ⋯ (𝑥 − 𝑥𝑛 )
(𝑛 + 1)!

Here 𝐸(𝑥) is the truncation error of the polynomial interpolation.

44 Simplified Numerical Analysis

• A Lagrange interpolation formula can be obtained from the relevant Newton’s Divided Difference
interpolation formula, after some rearrangements.

• Suppose that 𝑛 + 1 data points, (𝑥0 , 𝑓0 ), (𝑥1 , 𝑓1 ), ⋯ , (𝑥𝑛 , 𝑓𝑛 ), of a function 𝑓 are given on the interval
[𝑎, 𝑏] for consecutively arranged and equispaced nodes 𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑛 , such that
𝑎 = 𝑥0 < 𝑥1 < 𝑥2 < ⋯ < 𝑥𝑛−1 < 𝑥𝑛 = 𝑏
with astep size of length ℎ = 𝑥𝑖 − 𝑥𝑖−1 , for 𝑖 = 1, 2, 3, ⋯ , 𝑛
and 𝑓(𝑥𝑖 ) = 𝑓𝑖

The Newton Forward-Difference Interpolation formula for the interpolating polynomial 𝑃𝑛 (𝑥) of
degree at most 𝑛 is given by

𝛼(𝛼 − 1) 2 𝛼(𝛼 − 1)(𝛼 − 2) ⋯ (𝛼 − (𝑛 − 1)) 𝑛

𝑃𝑛 (𝑥) = 𝑓0 + 𝛼∆𝑓0 + ∆ 𝑓0 + ⋯ + ∆ 𝑓0
2! 𝑛!
where
𝑥 − 𝑥0
𝛼 =
ℎ
Here the 𝒌th forward-difference of 𝑓 at 𝑥𝑖 is denoted by ∆𝑘 𝑓𝑖 and is recursively defined by

∆𝑘 𝑓𝑖 = ∆(∆𝑘−1 𝑓𝑖 ) = ∆𝑘−1 𝑓𝑖+1 − ∆𝑘−1 𝑓𝑖 for 𝑘 = 2, 3, ⋯ , 𝑛

with ∆𝑓𝑖 = 𝑓𝑖+1 − 𝑓𝑖

The Newton Backward-Difference Interpolation formula for the interpolating polynomial 𝑃𝑛 (𝑥) of
degree at most 𝑛 is given by
𝛽(𝛽 + 1) 2 𝛽(𝛽 + 1)(𝛽 + 2) ⋯ (𝛽 + (𝑛 − 1)) 𝑛
𝑃𝑛 (𝑥) = 𝑓𝑛 + 𝛽∇𝑓𝑛 + ∇ 𝑓𝑛 + ⋯ + ∇ 𝑓𝑛
2! 𝑛!
where
𝑥 − 𝑥𝑛
𝛽 =
ℎ
Here the 𝒌th backward-difference of 𝑓 at 𝑥𝑖 is denoted by ∇𝑘 𝑓𝑖 and is recursively defined by

∇𝑘 𝑓𝑖 = ∇(∇𝑘−1 𝑓𝑖 ) = ∇𝑘−1 𝑓𝑖 − ∇𝑘−1 𝑓𝑖−1 for 𝑘 = 2, 3, ⋯ , 𝑛

with ∇𝑓𝑖 = 𝑓𝑖 − 𝑓𝑖−1

• There are central difference interpolation formulas also available in the literature, which are more
suited for approximation of a function value around mid of the interval of interpolation. Following are
the examples of some well-known central difference interpolation formulas:

o Gauss Forward Difference Interpolation Formula

o Gauss Backward Difference Interpolation Formula
o Stirling’s Central Difference Interpolation Formula
o Bessel’s Central Difference Interpolation Formula
o Everrett’s Central Difference Interpolation Formula
∎∎∎
Polynomial Interpolation 45

Chapter Exercises

Exercise 01: Find the linear interpolating polynomial passing through the following set of pairs of the points.

(i) {(0.1, sin(0.1)), (0.2, sin(0.2))}

1 1
(ii) {(1.2, (1.2)2) , (1.4, (1.4)2)}

(iii) {(1, 7), (2, 4)}

1
(iv) {(1, 𝑒 −1 ), (1.5, 𝑒 −1.5 )}

Exercise 02: Construct the interpolating polynomial to approximate the following functions at 𝑥 = 0.25. Use
the arguments 𝑥0 = −0.3, 𝑥1 = 0, 𝑥2 = 0.4.

(i) 𝑓(𝑥) = ln(1 + 𝑥)

2
(ii) 𝑓(𝑥) = 𝑒 −𝑥

(iii) 𝑓(𝑥) = tan 𝑥 2

1
(iv) 𝑓(𝑥) =
√𝑥 2 −1

Exercise 03: Use the Lagrange Interpolating Polynomial and the Newton’s Divided Difference Interpolating
polynomial of the appropriate degree to interpolate the following:

(i) Compute 𝑓(1.5), given that, 𝑓(0.5) = 0.479, 𝑓(1.0) = 0.841, 𝑓(2.0) = 0.909
(ii) Compute 𝑓(3.6), given that 𝑓(3.0) = 0.1506, 𝑓(4.0) = 0.3001, 𝑓(4.5) = 0.2663, 𝑓(4.7) = 0.2346

(iii) Compute 𝑓(2/3), given that,

𝑓(1.1) = −0.071812, 𝑓(1.3) = −0.024750, 𝑓(1.7) = 0.334937, 𝑓(2.0) = 1.101000

Exercise 04: Find the missing value in the following table using the Newton’s Divided Difference
Interpolating polynomial.

𝑥 −1 1 2 3
𝑓(𝑥) −21 15 ? 3

Exercise 05: Find the missing value in the following table using Lagrange Interpolating Polynomial

𝑥 −2 0 2 4 6
𝑓(𝑥) 33 5 9 ? 113
46 Simplified Numerical Analysis

Exercise 06: Find, for what values of 𝑥, 𝑦 attained extreme values using the data given below

𝑥 3 4 5 6 7 8
𝑦 0.205 0.240 0.259 0.262 0.250 0.224

Exercise 07: Use Lagrange Interpolating Polynomial of the appropriate degree to complete the record of the
export of a certain commodity during six years

Year: 𝑥 1981 1982 1983 1984 1985 1986

Export: 𝑦 43 84 93 ? 105 157

Exercise 08: Use the Newton’s Divided Difference Interpolating Polynomial to obtain an interpolation that
passing through the following points

𝑥 0 0.1 0.3 0.4 0.7 0.8

𝑦 −1.5 −1.27 −0.98 −0.63 −0.22 0.25

Exercise 09: Find a bound for the error associated with linear polynomial interpolation for the following
function. Use the arguments 𝑥0 = 0, 𝑥1 = 0.4.

(i) 𝑓(𝑥) = ln(1 + 𝑥)

2
(ii) 𝑓(𝑥) = 𝑒 −𝑥

(iii) 𝑓(𝑥) = tan 𝑥 2

1
(iv) 𝑓(𝑥) =
√𝑥 2 −1

Exercise 10: Find a bound for the error associated with quadratic polynomial interpolation for the following
function. Use the arguments 𝑥0 = 0, 𝑥1 = 0.1, 𝑥2 = 0.4.

(i) 𝑓(𝑥) = sin 𝑥 + cos 𝑥

(ii) 𝑓(𝑥) = 𝑥 ln 𝑥

(iii) 𝑓(𝑥) = 𝑥 sin 𝑥 − 𝑥 3 + 2𝑥 − 1

(iv) 𝑓(𝑥) = √𝑥 − 𝑥 2

Exercise 11: Find a bound for the error associated with cubic polynomial interpolation for the following
function. Use the arguments 𝑥0 = 1, 𝑥1 = 1.3, 𝑥2 = 1.6, 𝑥3 = 2.0

(i) 𝑓(𝑥) = sin(𝑒 −𝑥 − 1)

(ii) 𝑓(𝑥) = ln 𝑥 − 𝑥 4 + 𝑥 2 − 1
2
(iii) 𝑓(𝑥) = 𝑥 2 𝑒 −𝑥
1
(iv) 𝑓(𝑥) =
√1+𝑥
Polynomial Interpolation 47

Exercise 12: Construct the Newton’s Forward and Backward Difference Interpolating polynomials passes
through the points (0.2, 0.9980), (0.4, 0.9686), (0.6, 0.8443), and (0.8, 0.5358).

Exercise 13: Construct the Newton’s Forward and Backward Difference Interpolating polynomials to
approximate the following functions at 𝑥 = 1.2 and 2.0. Use the arguments 𝑥0 = 1.1, 𝑥1 = 1.3, 𝑥2 = 1.5, 𝑥3 =
1.7, 𝑥4 = 1.9

(i) 𝑓(𝑥) = ln(1 + 𝑥)

2
(ii) 𝑓(𝑥) = 𝑒 −𝑥

(iii) 𝑓(𝑥) = tan 𝑥 2

1
(iv) 𝑓(𝑥) =
√𝑥 2 −1

Exercise 14: Some data of the speed (𝑉) versus drag coefficient (𝐶𝑑 ) of a cricket ball is given in the following
table: Estimate 𝐶𝑑 at 𝑉 = 150 𝑘𝑚/ℎ.

𝑉 in 𝑘𝑚/ℎ 𝐶𝑑
0 0.5
80 0.48
120 0.39
160 0.32

Exercise 15: The mileages covered by a car per liter of fuel at different speeds are shown is the table below:

Speed in 𝑘𝑚/ℎ Mileage covered in 𝑘𝑚/𝑙

60 14.2
75 16.1
90 14.8
105 13.7
120 11.5

Using interpolation, approximate the fuel efficiency of the car at the speed of 100 𝑘𝑚/ℎ.

Hint for the Solution: Use any interpolation formula, preferable the Newton’s Backward Difference
Interpolation formula.

Exercise 16: Some recorded data of number of deaths due to Novel Coronavirus (2019-nCoV) is given in the
table below. Use interpolation to determine number of deaths on January 29 and 31, 2020.
48 Simplified Numerical Analysis

Date Number of Deaths

Jan. 24 16
Jan. 26 24
Jan. 28 26
Jan. 30 43
Feb. 1 45

Hint for the Solution: The given data spans over 9 days. The function values are given for 𝑥 = 1, 3, 5, 7, 9. Find
an interpolating polynomial and use it to calculate value at 𝑥 = 6 and 𝑥 = 8 for the desired solutions.

Exercise 17: The census data of Pakistan is given in the following table (source: Pakistan Bureau of
Statistics):

Census Year Population in thousands

1951 33740
1961 42880
1972 65309
1981 84254
1998 132352
2017 207774

Use interpolation to determine the population for the year 2010.

Hint for the Solution: The given data spans over 67 years. The function values are given for 𝑥 =
1, 11, 22, 31, 48, 67. Find an interpolating polynomial and use it to calculate value at 𝑥 = 60 for the desired
solution.
Exercise 18: Suppose that a table lists the values of the tangent function for the angles ranging from 0o to
45o in increments of 5o. What is the largest error that we would introduce by performing linear interpolation
between successive values in this table?

∎∎∎
 Chapter 4
Numerical Integration

Corridor I: BASICS
Let’s plan it

4.1 Introduction
4.2 The Trapezoidal Rule
4.3 The Simpson’s 1/3 Rule
4.4 Generalized Closed Newton-Cotes Quadrature

To unleash the topics of this Corridor, please delve into the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

49
50 Simplified Numerical Analysis

Question 12: Tabulate Closed Newton-Cotes Integration formulas with relevant features, for both
the basic and the composite forms, separately.

Suppose that 𝑛 data points, (𝑥𝑗 , 𝑓𝑗 ), where 𝑓(𝑥𝑗 ) = 𝑓𝑗 , of the integrand 𝑓(𝑥) are given on the
interval [𝑎, 𝑏] = [𝑥0 , 𝑥1 ] for consecutively arranged and equispaced nodes 𝑥𝑗 such that ℎ =
𝑥
(𝑏 − 𝑎)/𝑛. The Closed Newton Cotes quadrature formulas for the definite integral = ∫𝑥 𝑛 𝑓(𝑥) 𝑑𝑥
0
are tabulated below.

Required
Numerical number of Interpolating
function polynomial used for
Integration Formula
values at integral evaluation
Method equidistant (to derive the formula)
points
I = ℎ(𝑓0 ) (starting-point rule) or

I = ℎ(𝑓1 ) (end-point rule) or

Rectangular Polynomial of degree 0
one
Rule I = ℎ(𝑓 )∗
(mid-point rule) (constant function)

𝑥0 +𝑥1
where 𝑓 ∗ = 𝑓 ( )
2

Trapezoidal ℎ Polynomial of degree 1

I= [𝑓 + 𝑓1 ] two
Rule 2 0 (linear polynomial)

Simpson’s 1/3 ℎ Polynomial of degree 2

I= [𝑓 + 4𝑓1 + 𝑓2 ] three
Rule 3 0 (quadratic polynomial)

Simpson’s 3/8 3ℎ Polynomial of degree 3

I= [𝑓 + 3(𝑓1 + 𝑓2 ) + 𝑓3 ] four
Rule 8 0 (cubic polynomial)

Boole’s Rule 2ℎ
I= [7𝑓 + 32𝑓1 + 12𝑓2 + 32𝑓3 + 7𝑓4 ] five Polynomial of degree 4
(Milne’s Rule) 45 0

5ℎ
I= [19𝑓0 + 75𝑓1 + 50𝑓2 + 50𝑓3
Six-Point Rule 288 six Polynomial of degree 5
+75𝑓4 + 19𝑓5 ]
ℎ
I= [41𝑓0 + 216𝑓1 + 27𝑓2 + 272𝑓3
Weddle’s Rule 140 seven Polynomial of degree 6
+27𝑓4 + 216𝑓5 + 41𝑓6 ]
Numerical Integration 51

Possible Interpolating
values of 𝒏 polynomial
Numerical Formula used for
(K represents
Integration (for 𝑛 + 1 data points, (𝑥𝑗 , 𝑓𝑗 ), 𝑗 = 0,1,2, ⋯ , 𝑛, the number of integral
Method and 𝑛 subintervals of equal length ℎ = (𝑥𝑛 − 𝑥0 )⁄𝑛) multiple evaluation
applications of (to derive the
the formula) formula)

I = ℎ[𝑓0 + 𝑓1 + 𝑓2 + ⋯ + 𝑓𝑛−1 ] (starting-point rule) or Piecewise

n = 1, 2, 3, ...
I = ℎ[𝑓1 + 𝑓2 + ⋯ + 𝑓𝑛−1 + 𝑓𝑛 ] (end-point rule) or polynomial of
Composite (i.e., n = K
degree 0
Rectangular I = ℎ[𝑓1∗ + 𝑓2∗ + ⋯ + 𝑓𝑛−1
∗
+ 𝑓𝑛∗ ] (mid-point rule) could be any
(piecewise-
Rule positive
where 𝑓𝑗∗ = 𝑓 (
𝑥𝑗−1 +𝑥𝑗 constant
) , for 𝑗 = 1,2,3, … , 𝑛 integer)
2 function)
𝑛 = 1, 2, 3, ⋯ Piecewise
Composite (i.e., n = K polynomial of
ℎ
Trapezoidal I = [𝑓 + 2(𝑓1 + 𝑓2 + ⋯ + 𝑓𝑛−1 ) + 𝑓𝑛 ] could be any degree 1
Rule 2 0 positive (piecewise-
integer) linear)
Piecewise
n = 2, 4, 6, ...
Composite ℎ polynomial of
I = [𝑓0 + 4(𝑓1 + 𝑓3 + ⋯ + 𝑓𝑛−1 ) (i.e., n = 2K,
Simpson’s 3 degree 2
where
1/3 Rule + 2(𝑓2 + 𝑓4 + ⋯ + 𝑓𝑛−2 ) + 𝑓𝑛 ] (piecewise-
K = 1, 2, 3, ...)
quadratic)
Piecewise
n = 3, 6, 9, ...
Composite 3ℎ polynomial of
I= [𝑓 + 3(𝑓1 + 𝑓2 ) + 2(𝑓3 ) + 3(𝑓4 + 𝑓5 ) + 2(𝑓6 ) (i.e., n = 3K,
Simpson’s 8 0 degree 3
where
3/8 Rule + ⋯ + 3(𝑓𝑛−2 + 𝑓𝑛−1 ) + 𝑓𝑛 ] (piecewise-
K = 1, 2, 3, ...)
cubic)
2ℎ
Composite I= [7𝑓 + 32(𝑓1 + 𝑓5 + 𝑓9 + ⋯ + 𝑓𝑛−3 )
45 0 n = 4, 8, 12, ...
Boole’s Rule Piecewise
+ 12(𝑓2 + 𝑓6 + 𝑓10 + ⋯ + 𝑓𝑛−2 ) (i.e., n = 4K,
(Composite polynomial of
+ 32(𝑓3 + 𝑓7 + 𝑓11 + ⋯ + 𝑓𝑛−1 ) where
Milne’s degree 4
K = 1, 2, 3, ...)
Rule) + 14(𝑓4 + 𝑓8 + 𝑓12 + ⋯ + 𝑓𝑛−4 ) + 7𝑓𝑛 ]
5ℎ
I= [19𝑓0 + 75(𝑓1 + 𝑓6 + 𝑓11 + ⋯ + 𝑓𝑛−4 )
288 n = 5, 10, 15, ...
Composite + 50(𝑓2 + 𝑓7 + 𝑓12 + ⋯ + 𝑓𝑛−3 ) Piecewise
(i.e., n = 5K,
Six-Point + 50(𝑓3 + 𝑓8 + 𝑓13 + ⋯ + 𝑓𝑛−2 ) polynomial of
where
Rule degree 5
+ 75(𝑓4 + 𝑓9 + 𝑓14 + ⋯ + 𝑓𝑛−1 ) K = 1, 2, 3, ...)
+ 38(𝑓5 + 𝑓10 + 𝑓15 + ⋯ + 𝑓𝑛−5 ) + 19𝑓𝑛 ]
ℎ
I= [41𝑓0 + 216(𝑓1 + 𝑓7 + 𝑓13 + ⋯ + 𝑓𝑛−5 )
140
+ 27(𝑓2 + 𝑓8 + 𝑓14 + ⋯ + 𝑓𝑛−4 ) n = 6, 12, 18, ...
Composite Piecewise
+ 272(𝑓3 + 𝑓9 + 𝑓15 + ⋯ + 𝑓𝑛−3 ) (i.e., n = 6K,
Weddle’s polynomial of
+ 27(𝑓4 + 𝑓10 + 𝑓16 + ⋯ + 𝑓𝑛−2 ) where
Rule degree 6
K = 1, 2, 3, ...)
+ 216(𝑓5 + 𝑓11 + 𝑓17 + ⋯ + 𝑓𝑛−1 )
+ 82(𝑓6 + 𝑓12 + 𝑓18 + ⋯ + 𝑓𝑛−6 ) + 41𝑓𝑛 ]
∎∎∎
52 Simplified Numerical Analysis

Corridor II: ANALYSIS

Let’s think deep

4.5 Truncation Error of the Trapezoidal Rule

4.6 Truncation Error of the Simpson’s 1/3 Rule
4.7 Further Discussions
4.8 The Gaussian Quadrature

To unleash the topics of this Corridor, please delve into the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

Corridor III: PROGRAMMING ARCADE

Let’s do it

4.9 Algorithms and Implementations

The Composite Trapezoidal Rule
The Composite Simpson’s 1/3 Rul
The Composite Simpson’s 3/8 Rule

To cross-check the results/output of the computer programs you would execute, please delve into
the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎
Numerical Integration 53

4.9 Algorithms and Implementations

Question 27: Write down the algorithm (pseudo-code) of the Composite Trapezoidal rule for
numerical integration of definite integrals.

𝑏
Algorithm: To approximate the definite integral 𝐼 = ∫𝑎 𝑓(𝑥) 𝑑𝑥 using the formula:
ℎ
𝐼 = [𝑓0 + 2(𝑓1 + 𝑓2 + 𝑓3 + ⋯ + 𝑓𝑛−1 ) + 𝑓𝑛 ]
2
𝒂 and 𝒃: two real values as the endpoints of the interval of integration
𝐈𝐍𝐏𝐔𝐓𝐒: {
𝒏: a positive integer as the number of subintervals
𝐎𝐔𝐓𝐏𝐔𝐓: 𝑰: a real number as an approximation to the integral

Step 1 Receive the inputs as stated above

Step 2 Set real number 𝒙𝟎 = 𝒂

Set real number 𝒙𝒏 = 𝒃
Set real number 𝒉 = (𝒙𝒏 − 𝒙𝟎)/𝒏
Set real number 𝒇𝒙𝟎 as the value 𝑓(𝒂)
Set real number 𝒇𝒙𝒏 as the value 𝑓(𝒃)
Step 3 Set 𝑰 = 𝒇𝒙𝟎 + 𝒇𝒙𝒏

Step 4 Set real number 𝒙𝒄 = 𝒙𝟎

Set real number 𝒔𝒖𝒎 = 𝟎
for 𝑗 = 1, 2, ⋯ , 𝒏 − 1
Set 𝒙𝒄 = 𝒙𝒄 + 𝒉
Set 𝒇𝒙𝒄 as the value 𝑓(𝒙𝒄)
Set 𝒔𝒖𝒎 = 𝒔𝒖𝒎 + 𝒇𝒙𝒄 (Forming 𝑓1 + 𝑓2 + ⋯ + 𝑓𝑛−1 )
end for

Step 5 Set 𝑰 = (𝒉⁄𝟐) × (𝑰 + 𝟐 × 𝒔𝒖𝒎)

Step 6 Print the output: 𝑰

STOP.

Problem 16: Write a C++ program to evaluate the integral of 𝑓(𝑥) = √𝑥 2 + 1 over [0, 2] with 12
subintervals using the Composite Trapezoidal rule.
54 Simplified Numerical Analysis

1 #include<iostream>
2 #include<cmath>
3 using namespace std ;
4
5 int main ( )
6 {
7
8 int n , j ;
9 double h , sum , I , x0 , xn , xc , fx0 , fxn , fxc ;
10
11 //------------------------- Input Section -----------------------------//
12 cout<< "The Composite Trapezoidal Rule." ;
13 cout<< "\nEnter the lower limit of the integral: " ;
14 cin>> x0 ;
15 cout<< "\nEnter the upper limit of the integral: " ;
16 cin>> xn ;
17 cout<< "\nEnter the number of subintervals n: ";
18 cin>> n ;
19
20 //----------------------- Processing Section -------------------------//
21
22 h = (xn - x0)/n ; // finding step size
23 fx0 = sqrt(x0*x0 + 1) ;
24 fxn = sqrt(xn*xn + 1) ;
25 I = fx0 + fxn ;
26 xc = x0 ;
27 sum = 0.0 ;
28
29 for ( j=1 ; j<=n-1 ; j++ )
30 {
31 xc = xc + h ;
32 fxc = sqrt(xc*xc + 1) ;
33 sum = sum + fxc ;
34 }
35
36 I = (h / 2.0) * (I + 2.0 * sum) ;
37
38 //------------------------ Output Section ----------------------------//
39 cout<< "The approximate integral = " << I << endl ;
40
41 }
Numerical Integration 55

Problem 18: Write a C++ program to evaluate the integral of 𝑓(𝑥) = √𝑥 2 + 1 over [0, 2] with 12
subintervals using the Composite Trapezoidal rule. Use #define directive for evaluating 𝑓(𝑥) at the different
nodes (i.e., for finding the values of 𝑓 at the different nodes).

1 #include<iostream>
2 #include<cmath>
3 using namespace std ;
4
5 # define fval(x) sqrt(x*x + 1) // evaluating f(x)
6
7 int main ( )
8 {
9 int n , j ;
10 double h , sum , I , x0 , xn , xc , fx0 , fxn , fxc ;
11
12 //------------------------- Input Section -----------------------------//
13 cout<< "The Composite Trapezoidal Rule." ;
14 cout<< "\nEnter the lower limit of the integral: " ;
15 cin>> x0 ;
16 cout<< "\nEnter the upper limit of the integral: " ;
17 cin>> xn ;
18 cout<< "\nEnter the number of subintervals n: ";
19 cin>> n ;
20
21 //----------------------- Processing Section -------------------------//
22
23 h = (xn - x0)/n ; // finding step size
24 fx0 = fval(x0) ;
25 fxn = fval(xn) ;
26 I = fx0 + fxn ;
27 xc = x0 ;
28 sum = 0.0 ;
29
30 for ( j=1 ; j<=n-1 ; j++ )
31 {
32 xc = xc + h ;
33 fxc = fval(xc) ;
34 sum = sum + fxc ;
35 }
36
37 I = (h / 2.0) * (I + 2.0 * sum) ;
56 Simplified Numerical Analysis

38
39 //------------------------ Output Section ----------------------------//
40 cout<< "The approximate integral = " << I << endl ;
41 }

Question 28: Write down the algorithm (pseudo-code) of the Composite Simpson’s 1/3 rule for
numerical integration of definite integrals.
𝑏
Algorithm: To approximate the definite integral 𝐼 = ∫𝑎 𝑓(𝑥) 𝑑𝑥 using the formula:
ℎ
𝐼 = [𝑓0 + 4(𝑓1 + 𝑓3 + ⋯ + 𝑓𝑛−1 ) + 2(𝑓2 + 𝑓4 + ⋯ + 𝑓𝑛−2 ) + 𝑓𝑛 ]
3
𝒂 and 𝒃: two real values as the endpoints of the interval of integration
𝐈𝐍𝐏𝐔𝐓𝐒: {
𝒏: a positive even integer as the number of subintervals

𝐎𝐔𝐓𝐏𝐔𝐓: 𝑰: a real number as an approximation to the integral

Step 1 Receive the inputs as stated above

Step 2 Set real number 𝒙𝟎 = 𝒂

Set real number 𝒙𝒏 = 𝒃
Set real number 𝒉 = (𝒙𝒏 − 𝒙𝟎)/𝒏
Set real number 𝒇𝒙𝟎 as the value 𝑓(𝒂)
Set real number 𝒇𝒙𝒏 as the value 𝑓(𝒃)

Step 3 Set 𝑰 = 𝒇𝒙𝟎 + 𝒇𝒙𝒏

Step 4 Set real number 𝒙𝒄 = 𝒙𝟎

Set real number 𝒔𝒖𝒎𝟏 = 𝟎
Set real number 𝒔𝒖𝒎𝟐 = 𝟎
for 𝑗 = 1, 2, ⋯ , 𝒏 − 1
Set 𝒙𝒄 = 𝒙𝒄 + 𝒉
Set 𝒇𝒙𝒄 as the value 𝑓(𝒙𝒄)
if 𝑗 is odd
Set 𝒔𝒖𝒎𝟏 = 𝒔𝒖𝒎𝟏 + 𝒇𝒙𝒄 (Forming 𝑓1 + 𝑓3 + ⋯ + 𝑓𝑛−1 )
else
Set 𝒔𝒖𝒎𝟐 = 𝒔𝒖𝒎𝟐 + 𝒇𝒙𝒄 (Forming 𝑓2 + 𝑓4 + ⋯ + 𝑓𝑛−2 )
end for

Step 5 Set 𝑰 = (𝒉⁄𝟑) × (𝑰 + 𝟒 × 𝒔𝒖𝒎𝟏 + 𝟐 × 𝒔𝒖𝒎𝟐)

Step 6 Print the output: 𝑰 ; STOP.

Numerical Integration 57

Problem 20: Write a C++ program to evaluate the integral of 𝑓(𝑥) = √𝑥 2 + 1 over [0, 2] with 12
subintervals using the Composite Simpson’s 1/3 rule.

1 #include<iostream>
2 #include<cmath>
3 using namespace std ;
4
5 int main ( )
6 {
7 int n , j ;
8 double h , sum1 , sum2 , I , x0 , xn , xc , fx0 , fxn , fxc ;
9
10 //------------------------- Input Section -----------------------------//
11
12 cout<< "The Composite Simpson's 1/3 Rule." ;
13 cout<< "\nEnter the lower limit of the integral: " ;
14 cin>> x0 ;
15 cout<< "\nEnter the upper limit of the integral: " ;
16 cin>> xn ;
17 cout<< "\nEnter the number of subintervals n: ";
18 cin>> n ;
19
20 //----------------------- Processing Section -------------------------//
21
22 h = (xn - x0)/n ; // finding step size
23 fx0 = sqrt(x0*x0 + 1) ;
24 fxn = sqrt(xn*xn + 1) ;
25 I = fx0 + fxn ;
26 xc = x0 ;
27 sum1 = 0.0 ; sum2 = 0.0 ;
28
29 for ( j=1 ; j<=n-1 ; j++ )
30 {
31 xc = xc + h ;
32 fxc = sqrt(xc*xc + 1) ;
33 if ( j%2 != 0)
34 sum1 = sum1 + fxc ;
35 else
36 sum2 = sum2 + fxc ;
37 }
38 I = (h / 3.0) * (I + 4 * sum1 + 2 * sum2) ;
58 Simplified Numerical Analysis

39
40 //------------------------ Output Section ----------------------------//
41 cout<< "The approximate integral = " << I << endl ;
42
43 }

Question 29: Write down the algorithm (pseudo-code) of the Composite Simpson’s 3/8 rule for
numerical integration of definite integrals.
𝑏
Algorithm: To approximate the definite integral 𝐼 = ∫𝑎 𝑓(𝑥) 𝑑𝑥 using the formula:
3ℎ
𝐼= [𝑓 + 3(𝑓1 + 𝑓2 + 𝑓4 + 𝑓5 + ⋯ + 𝑓𝑛−2 + 𝑓𝑛−1 ) + 2(𝑓3 + 𝑓6 + ⋯ + 𝑓𝑛−3 ) + 𝑓𝑛 ]
8 0
𝒂 and 𝒃: two real values as the endpoints of the interval of integration
𝐈𝐍𝐏𝐔𝐓𝐒: {
𝒏: a positive integer (multiple of 3) as the number of subintervals

𝐎𝐔𝐓𝐏𝐔𝐓: 𝑰: a real number as an approximation to the integral

Step 1 Receive the inputs as stated above

Step 2 Set real number 𝒙𝟎 = 𝒂
Set real number 𝒙𝒏 = 𝒃
Set real number 𝒉 = (𝒙𝒏 − 𝒙𝟎)/𝒏
Set real number 𝒇𝒙𝟎 as the value 𝑓(𝒂)
Set real number 𝒇𝒙𝒏 as the value 𝑓(𝒃)

Step 3 Set 𝑰 = 𝒇𝒙𝟎 + 𝒇𝒙𝒏

Step 4 Set real number 𝒙𝒄 = 𝒙𝟎
Set real number 𝒔𝒖𝒎𝟏 = 𝟎
Set real number 𝒔𝒖𝒎𝟐 = 𝟎
for 𝑗 = 1, 2, ⋯ , 𝒏 − 1
Set 𝒙𝒄 = 𝒙𝒄 + 𝒉
Set 𝒇𝒙𝒄 as the value 𝑓(𝒙𝒄)
if 𝑗 is divisible by 3
Set 𝒔𝒖𝒎𝟐 = 𝒔𝒖𝒎𝟐 + 𝒇𝒙𝒄 (Forming 𝑓3 + 𝑓6 + ⋯ + 𝑓𝑛−3 )
else
Forming 𝑓1 + 𝑓2 + 𝑓4 + 𝑓5
Set 𝒔𝒖𝒎𝟏 = 𝒔𝒖𝒎𝟏 + 𝒇𝒙𝒄 ( )
+ ⋯ + 𝑓𝑛−2 + 𝑓𝑛−1
end for

Step 5 Set 𝑰 = (𝟑 × 𝒉⁄𝟖) × (𝑰 + 𝟑 × 𝒔𝒖𝒎𝟏 + 𝟐 × 𝒔𝒖𝒎𝟐)

Step 6 Print the output: 𝑰 ; STOP.
Numerical Integration 59

Problem 22: Write a C++ program to evaluate the integral of 𝑓(𝑥) = √𝑥 2 + 1 over [0, 2] with 12
subintervals using the Composite Simpson’s 3/8 rule.

1 #include<iostream>
2 #include<cmath>
3 using namespace std ;
4
5 int main ( )
6 {
7 int n , j ;
8 double h , sum1 , sum2 , I , x0 , xn , xc , fx0 , fxn , fxc ;
9
10 //------------------------- Input Section -----------------------------//
11
12 cout<< "The Composite Simpson's 3/8 Rule." ;
13 cout<< "\nEnter the lower limit of the integral: " ;
14 cin>> x0 ;
15 cout<< "\nEnter the upper limit of the integral: " ;
16 cin>> xn ;
17 cout<< "\nEnter the number of subintervals n: ";
18 cin>> n ;
19
20 //----------------------- Processing Section -------------------------//
21
22 h = (xn - x0)/n ; // finding step size
23 fx0 = sqrt(x0*x0 + 1) ;
24 fxn = sqrt(xn*xn + 1) ;
25 I = fx0 + fxn ;
26 xc = x0 ;
27 sum1 = 0.0 ; sum2 = 0.0 ;
28
29 for ( j=1 ; j<=n-1 ; j++ )
30 {
31 xc = xc + h ;
32 fxc = sqrt(xc*xc + 1) ;
33 if ( j%3 == 0)
34 sum2 = sum2 + fxc ;
35 else
36 sum1 = sum1 + fxc ;
37 }
38 I = (3.0 * h / 8.0) * (I + 2 * sum2 + 3 * sum1) ;
60 Simplified Numerical Analysis

39
40 //------------------------ Output Section ----------------------------//
41 cout<< "The approximate integral = " << I << endl ;
42
43 }

Remark: Likewise the programs in the solutions of Problem 19, the programmer can modify the
programs in the solutions of Problems 20 and 22 to evaluate the function values at the desired
nodes through the use of user-defined function and inline function (using #define).
∎∎∎
Numerical Integration 61

Chapter Summary

• Numerical integration or quadrature refers to the process of numerically approximating the value of
𝑏
the integral 𝐼 = ∫𝑎 𝑓(𝑥) 𝑑𝑥, by using the values of 𝑓 at a finite number of sample points. The limits of
integration could be finite, semi-finite, or infinite.

• The integral is approximated by a numerical integration rule or quadrature formula, 𝑄𝑓 , which is a

linear combination of certain function values:
𝑏 𝑛

𝐼 = ∫ 𝑓(𝑥) 𝑑𝑥 ≅ 𝑄𝑓 = ∑ 𝜔𝑗 ∙ 𝑓(𝑥𝑗 )
𝑎 𝑗=0

Here 𝑥𝑖 are the ordered points, called the quadrature nodes (or simply nodes), taken usually within the
limits of integration at which the function values 𝑓(𝑥𝑗 ) are known and 𝜔𝑗 are called the weights of the
quadrature formula.

• The quadrature formula satisfies the property that

𝑏
𝐼 = ∫ 𝑓(𝑥) 𝑑𝑥 = 𝑄𝑓 + 𝐸𝑓 ,
𝑎

where 𝐸𝑓 is the truncation error (also called the error term) associated with the quadrature formula.

• The Newton-Cotes integration formulas are based on the approach that 𝑛 + 1 number of equispaced
and ordered nodes are chosen within the limits of integration and the integrand function is replaced by
an interpolating polynomial of degree at most 𝑛 by using the nodes, and then the analytic integration of
the polynomial is performed to obtain the formula. A Composite Newton-Cotes integration formula is
obtained by applying the relevant Newton-Cotes formula in each of the different consecutive segments
of the interval of integration and then summing the integrals over all the segments.

• The examples of Newton-Cotes integration formulas include Trapezoidal rule, Simpson’s 1/3 rule,
Simpson’s 3/8 rule, Boole’s rule, Six-Point rule, and Weddle’s rule.

• The Trapezoidal rule to numerically integrate the function 𝑓 over the interval [𝑎, 𝑏] is
𝑏
(𝑏 − 𝑎)
𝐼 = ∫ 𝑓(𝑥)𝑑𝑥 ≅ [𝑓(𝑎) + 𝑓(𝑏)]
2
𝑎
• The Composite Trapezoidal rule to integrate a function 𝑓 over the interval [𝑎, 𝑏] is given by,
𝑏
ℎ
𝐼 = ∫ 𝑓(𝑥)𝑑𝑥 ≅ [𝑓 + 2(𝑓1 + 𝑓2 + ⋯ + 𝑓𝑛−1 ) + 𝑓𝑛 ]
2 0
𝑎

𝑏 − 𝑎 𝑥𝑛 − 𝑥0
where ℎ= = , 𝑓𝑗 = 𝑓(𝑥𝑗 ) and 𝑥𝑗 = 𝑥0 + 𝑗ℎ for 𝑗 = 0, 1, 2, ⋯ , 𝑛
𝑛 𝑛
62 Simplified Numerical Analysis

• The Simpson’s 1/3 rule to integrate a function 𝑓 over the interval [𝑎, 𝑏] is given by,
𝑏
ℎ
𝐼 = ∫ 𝑓(𝑥)𝑑𝑥 ≅ [𝑓 + 4𝑓1 + 𝑓2 ]
3 0
𝑎
𝑏 − 𝑎 𝑥2 − 𝑥0
where ℎ= = , 𝑓𝑗 = 𝑓(𝑥𝑗 ) and 𝑥𝑗 = 𝑥0 + 𝑗ℎ for 𝑗 = 0, 1, 2
2 2

• The Composite Simpson’s 1/3 rule to integrate a function 𝑓 over the interval [𝑎, 𝑏] is given by,
𝑏
ℎ
𝐼 = ∫ 𝑓(𝑥)𝑑𝑥 ≅ [𝑓 + 4(𝑓1 + 𝑓3 + ⋯ + 𝑓𝑛−1 ) + 2(𝑓2 + 𝑓4 + ⋯ + 𝑓𝑛−2 ) + 𝑓𝑛 ]
3 0
𝑎
𝑏 − 𝑎 𝑥𝑛 − 𝑥0
where ℎ= = , 𝑓𝑗 = 𝑓(𝑥𝑗 ) and 𝑥𝑗 = 𝑥0 + 𝑗ℎ for 𝑗 = 0, 1, 2, ⋯ , 𝑛
𝑛 𝑛

• A comprehensive summary of the Newton-Cotes formulas and the Composite Newton-Cotes formulas
can be found under Question 12 (page 252).

𝑏
• The error term 𝐸𝑇 of order 𝒪(ℎ3 ) associated with the Trapezoidal rule in approximating 𝐼 = ∫𝑎 𝑓(𝑥) 𝑑𝑥
is given by,
1 3 ′′
𝐸𝑇 = − ℎ 𝑓 (𝜉),
12
for some appropriate point 𝜉 in (𝑎, 𝑏) and ℎ = 𝑏 − 𝑎.

• The error term 𝐸𝐶𝑇 of order 𝒪(ℎ2 ) associated with the Composite Trapezoidal rule in approximating 𝐼 =
𝑏
∫𝑎 𝑓(𝑥) 𝑑𝑥 is given by,
𝑏 − 𝑎 2 ′′
𝐸𝐶𝑇 = − ℎ 𝑓 (𝜂),
12
for some appropriate point 𝜂 in (𝑎, 𝑏) and ℎ = (𝑏 − 𝑎)/𝑛, where 𝑛 is the number of subintervals of
[𝑎, 𝑏].

• The error term 𝐸𝑆 of order 𝒪(ℎ5 ) associated with the Simpson’s 1/3 rule in approximating 𝐼 =
𝑏
∫𝑎 𝑓(𝑥) 𝑑𝑥 is given by,
1 5 (4)
𝐸𝑆 = − ℎ 𝑓 (𝜉),
90
for some appropriate point 𝜉 in (𝑎, 𝑏) and ℎ = (𝑏 − 𝑎)/2.

• The error term 𝐸𝐶𝑆 of order 𝒪(ℎ4 ) associated with the Composite Simpson’s 1/3 rule in approximating
𝑏
𝐼 = ∫𝑎 𝑓(𝑥) 𝑑𝑥 is given by,

𝑏 − 𝑎 4 (4)
𝐸𝐶𝑆 = − ℎ 𝑓 (𝜂)
180
for some appropriate point 𝜂 in (𝑎, 𝑏) and ℎ = (𝑏 − 𝑎)/𝑛, where 𝑛 is the number of subintervals of
[𝑎, 𝑏].
Numerical Integration 63

• Suppose 𝐼ℎ denotes the approximate integral using a quadrature formula with step size ℎ, and 𝐸ℎ
denotes the associated error. Then, the exact integral = 𝐼ℎ + 𝐸ℎ

Similarly, suppose 𝐼ℎ⁄2 denotes the approximate integral using the same quadrature formula with a step
size ℎ⁄2, and 𝐸ℎ⁄2 denotes the associated error. Then, the exact integral = 𝐼ℎ⁄2 + 𝐸ℎ⁄2

According to the interval halving method, for a Newton-Cotes integration formula with an error of order
𝒪(ℎ𝑁 ) an estimate of the error 𝐸ℎ⁄2 is given by,
1
𝐸ℎ⁄2 ≅ (𝐼 − 𝐼ℎ )
2 − 1 ℎ ⁄2
𝑁

This leads to a better approximation of the integral as below:

1
𝐼 ≅ 𝐼ℎ⁄2 + (𝐼 − 𝐼ℎ )
2 − 1 ℎ ⁄2
𝑁

This corresponds to a special process called Richardson Extrapolation, in which two estimates of the
solution are used to obtain a third approximation, which is a more accurate one. This approach for
numerical integration forms an initial stage of a relatively broader way of numerical integration, called
Romberg Integration. Recall that for the Composite Trapezoidal rule 𝑁 = 2, and for the Composite
Simpson’s 1/3 rule 𝑁 = 4.

• There could be several approaches for improving the estimates of the integrals:

o Using smaller step size (or larger number of subintervals)

o Using higher-order formula (e.g., using the Simpson’s rule instead of the Trapezoidal rule)

o Using Richardson’s extrapolation (i.e., using two less accurate estimates to obtain a more
accurate estimate).

• The degree of precision, also referred to as the order of accuracy, of a quadrature formula is 𝑝 if and
only if the associated truncation error is zero for all polynomials of degree less than or equal to 𝑝, and
the error is not zero for some polynomial of degree greater than 𝑝. Note that the Trapezoidal rule is
based on the interpolating polynomial of degree 1 (linear polynomial). Therefore, it produces the exact
result while integrating a polynomial of degree 1. Hence it has the degree of precision as 1. The
Simpson’s 1/3 rule might be expected to have a degree of precision as 2 because it is based on
interpolating polynomial of degree 2 (quadratic polynomial). However, it produces the exact result
while integrating a polynomial of degree 2, as well as degree 3. Hence, it has the degree of precision as 3.
This fact is also evident while deriving the error term for the Simpson’s 1/3 rule. This property,
together with certain other reasons, makes the Composite Simpson’s 1/3 rule often the best choice
among the Newton-Cotes integration formulas.

• A concise description of the error terms associated with the Newton-Cotes formulas and relevant
degrees of precision can be found under Question 23 (page 276).

• The Gaussian Quadrature is an advanced numerical integration technique in which the quadrature
nodes are selected in the interval of integration using the roots of some special polynomial to obtain an
optimal approximation of the integral.
∎∎∎
64 Simplified Numerical Analysis

Chapter Exercises

𝑏
Exercise 01: Approximate the integral ∫𝑎 𝑓(𝑥)𝑑𝑥 for the following functions over the interval [0, 1] using the
Trapezoidal, Simpson’s 1/3 and Simpson’s 3/8 rules.
1
(𝑖) 𝑓(𝑥) = 𝑥 2 + 𝑥 − 1 (𝑖𝑖) 𝑓(𝑥) = ln(1 + 𝑥) (𝑖𝑖𝑖) 𝑓(𝑥) =
1 + 𝑥2
𝑥 1
(𝑖𝑣) 𝑓(𝑥) = cos ( ) (𝑣) 𝑓(𝑥) =
𝜋 √𝑥 2 +4

𝑏
Exercise 02: Approximate the integral ∫𝑎 𝑓(𝑥)𝑑𝑥 for the following functions over the interval [0, 1] using the
Composite Trapezoidal, Simpson’s 1/3, and Simpson’s 3/8 rules with ℎ = 0.1.
1
(𝑖) 𝑓(𝑥) = 𝑥 2 + 𝑥 − 1 (𝑖𝑖) 𝑓(𝑥) = ln(1 + 𝑥) (𝑖𝑖𝑖) 𝑓(𝑥) =
1 + 𝑥2
𝑥 1
(𝑖𝑣) 𝑓(𝑥) = cos ( ) (𝑣) 𝑓(𝑥) =
𝜋 √𝑥 2 +4

Exercise 03: Approximate the integral

16
𝜋√𝑥
∫ sin ( ) 𝑑𝑥
4
4
using the Composite Trapezoidal rule with ℎ = 1 and five-digit rounding arithmetic.

2
Exercise 04: Find an approximate value of the integral ∫0 (2 + sin(2√𝑥))𝑑𝑥 using the Composite
Trapezoidal rule for 𝑛 = 10 and five-digit rounding arithmetic.

Exercise 05: Approximate the arc length of the following functions over the interval [0, 𝜋]
4+𝑥
(𝑖) 𝑓(𝑥) = sin2 𝑥 (𝑖𝑖) 𝑓(𝑥) = ln ( )
𝜋
𝜋
using the Composite Simpson’s 1/3 rule for ℎ = and four-digit rounding arithmetic.
6

8 2
Exercise 06: Find the approximate value of the integral ∫3 (𝑓(𝑥)) 𝑑𝑥 using the Composite Simpson’s 1/3
rule, given that

𝑥𝑗 3 4 5 6 7 8 9

𝑓(𝑥𝑗 ) 0.205 0.240 0.259 0.262 0.250 0.224 0.220

Exercise 07: Approximate the area of a surface of revolution of the following curves:
(𝑖) 𝑥 = 4𝑦, (𝑖𝑖) 𝑥 = tan 𝑦
Numerical Integration 65

about the 𝑦 − 𝑎𝑥𝑖𝑠 from 0 ≤ 𝑦 ≤ 1 using the Composite Simpson’s 3/8 rule for 𝑛 = 10 and four-digit
rounding arithmetic.

Exercise 08: Find the approximate value of the integral

3
𝑥
𝑓(𝑥) = ∫ 𝑑𝑥
𝑥2 + 3
0

using the Composite Boole’s rule with step size ℎ = 0.25 and five-digit rounding arithmetic.

Exercise 09: Find the approximate value of the integral

𝑓(𝑥) = ∫ ln(𝑥 − 1) 𝑑𝑥
2

using the Composite Six-Point rule with step size ℎ = 0.3 and five-digit rounding arithmetic.

Exercise 10: Find the approximate value of the integral

𝑓(𝑥) = ∫ sinh(𝑥 2 ) 𝑑𝑥
1

using the Composite Weddle’s rule with step size ℎ = 0.25 and five-digit rounding arithmetic.

Exercise 11: Suppose that 𝑓(0) = 1, 𝑓(0.5) = 2.5, 𝑓(1) = 2 and 𝑓(0.25) = 𝑓(0.75) = 𝛼. Find 𝛼 if the
1
Composite Trapezoidal rule with 𝑛 = 4 gives the value 1.75 for ∫0 𝑓(𝑥)𝑑𝑥 .

Exercise 12: Suppose that 𝑓(4) = 0.240, 𝑓(6) = 0.262, 𝑓(8) = 0.224, 𝑓(3) = 𝑓(5) = 𝑓(7) = 𝛼, and 𝑓(9) =
0.220 Find 𝛼 if the Composite Simpson’s 1/3 Rule gives the value 1.473 for
9

𝐼 = ∫ 𝑓(𝑥)𝑑𝑥
3

Exercise 13: Suppose that 𝑓(0.2) = 1.56, 𝑓(0.4) = 2.00, 𝑓(0.6) = 3.01, 𝑓(0.1) = 𝑓(0.3) = 𝑓(0.5) = 𝛼, and
𝑓(0.7) = 3.32 Find 𝛼 if the Composite Simpson’s 3/8 rule gives the value 1.30312 for
0.7

𝐼 = ∫ 𝑓(𝑥)𝑑𝑥
0.1

Exercise 14: To approximate the integral of 𝑓(𝑥) over the interval [0, 1] with an absolute error less than
1
× 10−4 , how many subintervals are needed, in case of (𝑎) the Composite Trapezoidal rule, (𝑏) the
2
Composite Simpson’s 1/3 rule, and (𝑐) the Composite Simpson’s 3/8 rule? Given that,
66 Simplified Numerical Analysis

1
(𝑖) 𝑓(𝑥) = 𝑥 2 + 𝑥 − 1 (𝑖𝑖) 𝑓(𝑥) = ln(1 + 𝑥) (𝑖𝑖𝑖) 𝑓(𝑥) =
1 + 𝑥2
𝑥 1
(𝑖𝑣) 𝑓(𝑥) = cos ( ) (𝑣) 𝑓(𝑥) =
𝜋 √𝑥 2 +4

Exercise 15: Suppose we wish to evaluate the integral

𝜋
𝑓(𝑥) = ∫0 sin(√𝑥)𝑑𝑥

numerically, with an error of magnitude less than 10−5 . How many subintervals are needed if we wish to use
the Composite Trapezoidal and Composite Simpson 1/3 rules?

Exercise 16: Find the number of subintervals 𝑛 or step length ℎ so that the error 𝐸𝑇𝐶 for the Composite
Trapezoidal rule and error 𝐸𝑆𝐶 for the Composite Simpson’s 1/3 rule is less than 5 × 10−4 for numerically
integrating the Legendre polynomial,

6 3
𝑃4 (𝑥) = 𝑥 4 − 𝑥 2 +
7 35

over the interval [−1, 1].

Exercise 17: Obtain an upper bound on the absolute error when the Chebyshev polynomial of degree four,

𝑇4 (𝑥) = 8𝑥 4 − 8𝑥 2 + 1

is integrated over the interval [−1, 1] by means of the Composite Simpson’s 3/8 rule.

Exercise 18: Obtain an upper bound on the absolute error when the Laguerre polynomial of degree four

𝐿4 (𝑥) = 𝑥 4 − 16𝑥 3 + 72𝑥 2 − 96𝑥 + 24

is integrated over the interval [−1, 1], by means of the Composite Simpson’s 3/8 rule.

Exercise 19: A car travels the loop of the racing track in 65 seconds. The speed of the car in meter/second is
recorded after every 5 seconds as shown in the following table:

Time 0 5 10 15 20 25 30 35 40 45 50 55 60 65

Speed 0 40 62 70 72 65 71 79 75 72 68 63 75 82

Estimate the length of the loop of the racing track?

Hint for the Solution:

Clearly, the speed say 𝑆 is shown to be a function of time, say 𝑡, and its values 𝑆(𝑡) for different time instants
65
𝑡 are given. Obtain the estimate of the integral distance = ∫0 𝑆(𝑡) 𝑑𝑡 using any appropriate numerical
integration rule with the data given in the Table.
Numerical Integration 67

Exercise 20: The prime number theorem states that the number of primes in an interval 𝑎 ≤ 𝑥 ≤ 𝑏 is
approximately
𝑏
1
∫ 𝑑𝑥
𝑎 ln 𝑥

Estimate the number of primes existing in [50,150].

Hint for the Solution: Numerically evaluate the given integral for 𝑎 = 50 and 𝑏 = 150 using different values
1
of 𝑓(𝑥) = at equispaced nodes in [50,150], separated by step length ℎ = 10 or 20.
ln 𝑥

Exercise 21: The depths D (in meters) of a 80 meters wide river at different horizontal distances 𝑠 from the
bank is given in the following table.

𝑠 0 10 20 30 40 50 60 70 80

𝐷 0 3.5 6 12 10 15 9 5 0

Estimate the area of the cross-section of the river.

Hint for the Solution: Clearly, 𝐷 is shown to be a function of 𝑠 and its values 𝐷(𝑠) for different points 𝑠 are
80
given. Obtain the estimate of the integral, 𝑎𝑟𝑒𝑎 = ∫0 𝐷(𝑠) 𝑑𝑠 using any appropriate numerical integration
rule with the data given in the Table.

Exercise 22: A rectangular swimming pool is 35 feet wide and 60 feet long. At different positions 𝑃 in feet
along the length of the pool, the depths 𝐷 in feet are shown in the following Table. Estimate the volume of
the pool using numerical integration.

𝑃 0 6 12 18 24 30 36 42 48 54 60

𝐷 3 3.5 4 4.5 5 5.5 6 6.5 7 7.5 8

Hint for the Solution:

Clearly, 𝐷 is shown to be a function of 𝑃 and its values 𝐷(𝑃) for different points 𝑃 are given. Obtain the
60
estimate of the integral 𝑤 = ∫0 𝐷(𝑃) 𝑑𝑃 using any appropriate numerical integration rule with the data
given in the Table. Note that 𝑤 is the estimated area of one side-wall of the pool along the length. Multiplying
it with the width of 35 feet will give the volume of the pool.

Exercise 23: We know that

1
1 𝜋
∫ 2
𝑑𝑥 = tan−1 𝑥|10 = tan−1 1 =
1+𝑥 4
0

This means that the value of 𝜋 can be obtained evaluating the above integral and then multiplying the
answer by 4. Suppose that we want to approximate 𝜋 to four decimal places. This means absolute error must
68 Simplified Numerical Analysis

be less than 5.0 × 10−5 . This means that the error in approximating the integral must be less than
1
× (5.0 × 10−5 ) = 1.25 × 10−5 . Use the Composite Simpson’s 1/3 rule to approximate the value of 𝜋. For
4
this, first determine that what should be the minimum number of subintervals that would keep the error less
than the tolerance.

Exercise 24: The number of subintervals required to apply the Composite Simpson’s 1/3 rule should be

(A) Multiple of 1 (B) Multiple of 2

(C) Multiple of 3 (D) unconditionally many (E) None of above

Exercise 25: The Simpson’s 1/3 rule is based on the integration of interpolating polynomial of degree 2. The
Simpson’s 1/3 rule can accurately integrate the polynomials of degree

(A) up to 1 (B) up to 2

(C) up to 3 (D) up to any (E) None of above

Exercise 26: The Gaussian quadrature is different from the Newton’s Cotes Integration in regards to

(A) selection of polynomial degree (B) selection of quadrature nodes

(C) problem dependence (D) None of above

∎∎∎
 Chapter 5
Numerical Differentiation

Corridor I: BASICS
Let’s plan it

5.1 Introduction
5.2 Finite Difference Approximations of Derivatives using the Taylor Series
5.2.1 First Order Derivatives
5.2.2 Second Order Derivatives
5.3 Listing of the Derivative Formulas

To unleash the topics of this Corridor, please delve into the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

69
70 Simplified Numerical Analysis

∎∎∎
 Chapter 6
Direct Linear Solvers

Corridor I: BASICS
Let’s plan it

6.1 Introduction to Linear Systems

6.2 Solving Linear Systems using the Gaussian Elimination Method
6.3 Pivoting Strategies
Partial Pivoting
Scaled Partial Pivoting
Complete Pivoting
6.4 The Gauss-Jordan Method
6.5 Solving Linear Systems using the LU Factorization Method
6.5.1 The Doolittle’s Method
6.5.2 The Crout’s Method
6.5.3 The Cholesky’s Method

To unleash the topics of this Corridor, please delve into the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

71
72 Simplified Numerical Analysis

Fig. (6.3): A classification chart of linear solvers.

Corridor II: ANALYSIS

Let’s think deep

6.6 Operation Count Analysis

6.7 Matrix Inversion

To unleash the topics of this Corridor, please delve into the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎
Direct Linear Solvers 73

Corridor III: PROGRAMMING ARCADE

Let’s do it

People have been communicating and interconnecting since the beginning, but in
this era the communications and interconnections without modern technologies
(like phones, networks, internet, radio, and television) stand nowhere in regards
to possibility or survival. Likewise, people have been doing mathematics since early
ages, but in this modern era the mathematical applicability without making use
of the computers stands nowhere. Let’s modernize “OUR” culture of doing
mathematics so that it can be useful for all the disciplines of science and
engineering. It’s time to lead the frontiers of the knowledge and its applicability,

Remark: Suggestion: Before this Section, study, Corridor III of Chapter 07 to cope the difficulty level.

6.8 Algorithms and Implementations

The Gaussian Elimination Method with Partial Pivoting
Solving AX = B using the Doolittle's Method
Solving AX = B using the Crout's Method
Solving AX = B using the Cholesky's Method
Performing Operation Count Analysis

To cross-check the results/output of the computer programs you would execute, please delve into
the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

6.8 Algorithms and Implementations

74 Simplified Numerical Analysis

Question 20: Write down an algorithm (pseudo code) to solve a linear system 𝑨𝑿 = 𝑩 using the Gaussian
Elimination method with partial pivoting.

Algorithm: To solve 𝑨𝑿 = 𝑩.

𝒏: an integer as the number of equations and unknowns

𝐈𝐍𝐏𝐔𝐓𝐒: {𝑨 = (𝒂𝒊𝒋 ), 1 ≤ 𝑖, 𝑗 ≤ 𝒏: a real valued square matrix as the coefficient matrix
𝑩 = [𝒃𝟏 , 𝒃𝟐 , ⋯ , 𝒃𝒏 ]𝑻 : a real valued vector as the vector of right hand side constants

𝑿 = [𝒙𝟏 , 𝒙𝟐 , ⋯ , 𝒙𝒏 ]𝑻 : a real valued vector as the solution vector

𝐎𝐔𝐓𝐏𝐔𝐓𝐒: {
or a message that the given system has no unique solution

Step 1 Receive the inputs as stated above

Step 2 (Forward Elimination Phase)

for 𝑖 = 1,2, ⋯ , 𝒏 − 𝟏
Set 𝑟 = 𝑖
for 𝑗 = 𝑖 + 1, ⋯ , 𝒏 Searching largest absolute coefficient
} ( )
in 𝑖th column for partial pivoting
if (|𝒂𝒓𝒊 | < |𝒂𝒋𝒊 |) 𝑟 = 𝑗

if (𝒂𝒓𝒊 = 0) OUTPUT (‘The given system has no unique solution’) and STOP

else
if (𝑟 ≠ 𝑖), then interchange the 𝑖𝑡ℎ row with 𝑟𝑡ℎ row, and 𝑏𝑖 with 𝑏𝑟

for 𝑘 = 𝑖 + 1, 𝑖 + 2, ⋯ , 𝒏
𝒂𝒌𝒊
𝑚𝑢𝑙𝑡𝑖𝑝𝑙𝑖𝑒𝑟 =
𝒂𝒊𝒊
row replacement in the
augmented matrix for
for 𝑗 = 𝑖 + 1, 𝑖 + 2, ⋯ , 𝒏 ( )
eliminating the coefficients
𝒂𝒌𝒋 = 𝒂𝒌𝒋 − 𝑚𝑢𝑙𝑡𝑖𝑝𝑙𝑖𝑒𝑟 × 𝒂𝒊𝒋
below the pivot
𝒃𝒌 = 𝒃𝒌 − 𝑚𝑢𝑙𝑡𝑖𝑝𝑙𝑖𝑒𝑟 × 𝒃𝒊 }

Step 3 if (𝒂𝒏𝒏 = 0) OUTPUT (‘The given system has no unique solution’) and STOP

else go to step 4

Step 4 (Back Substitution Phase)

𝒃𝒏
𝒙𝒏 =
𝒂𝒏𝒏
for 𝑖 = 𝒏 − 1, ⋯ ,2, 1
𝑠𝑢𝑚 = 0
for 𝑗 = 𝑖 + 1, 𝑖 + 2, ⋯ , 𝒏 𝑛
1
𝑠𝑢𝑚 = 𝑠𝑢𝑚 + 𝒂𝒊𝒋 × 𝒙𝒋 (𝑥𝑖 = [𝑏 − ∑ 𝑎𝑖𝑗 𝑥𝑗 ])
𝑎𝑖𝑖 𝑖
[𝒃𝒊 − 𝑠𝑢𝑚] 𝑗=𝑖+1
𝒙𝒊 =
𝒂𝒊𝒊 }

Step 5 Print the output: 𝑿 = [𝒙𝟏 , 𝒙𝟐 , … , 𝒙𝒏 ]𝑻 and STOP.

Direct Linear Solvers 75

Problem 16: Write a C++ program to solve the following linear system using the Gaussian Elimination
method with partial pivoting. For simplification, specify the linear system within the program.

1.7𝑥1 + 2.3𝑥2 − 1.5𝑥3 = 2.35

1.1𝑥1 + 1.6𝑥2 − 1.9𝑥3 = −0.94
2.7𝑥1 − 2.2𝑥2 + 1.5𝑥3 = 2.70

1 #include<iostream>
2 #include<cmath>
3 using namespace std;
4 # define n 3 i// number of unknowns
5
6 int main ( )
7 {
8 char quit ;
9 int i , j ,k,n,r;
10 double a[n][n] = {{1.7, 2.3, -1.5} , {1.1, 1.6, -1.9} , {2.7, -2.2, 1.5}} ;
11 double b[n] = {2.35, -0.94, 2.7}; // right-hand side constants
12 double x[n] ; // solution vector
13 double sum , temp , temp1 , multiplier ;
14
15 //----------------------- Processing Section -------------------------//
16
17 // Forward Elimination Phase
18 // Searching largest absolute coefficient in the ith column for partial pivoting
19
20 for ( i=0 ; i<n-1 ; i++ )
21 {
22 r=i;
23 for ( j=i+1 ; j<n ; j++ )
24 {
25 if ( abs( a[r][i] )<abs( a[j][i] ) )
26 r=j;
27 }
28 if ( a[r][i] == 0 )
29 {
30 cout<< "The given system has no unique solution" ;
31 break ;
32 }
33 else if ( r != i )
34 for( j=0 ; j<n ; j++ )
76 Simplified Numerical Analysis

35 {
36 temp = a[i][j] ;
37 a[i][j] = a[r][j] ;
38 a[r][j] = temp ;
39 }
40 temp1 = b[i] ;
41 b[i] = b[r] ;
42 b[r] = temp1 ;
43 for ( k=i+1 ; k<n ; k++ )
44 {
45 multiplier = a[k][i] / a[i][i] ;
46 for ( j=i+1 ; j<n ; j++ ) 𝑟𝑜𝑤 𝑟𝑒𝑝𝑙𝑎𝑐𝑒𝑚𝑒𝑛𝑡 𝑖𝑛 𝑡ℎ𝑒
a[k][j] = a[k][j] – multiplier * a[i][j] ; 𝑎𝑢𝑔𝑚𝑒𝑛𝑡𝑒𝑑 𝑚𝑎𝑡𝑟𝑖𝑥 𝑓𝑜𝑟
47
𝑒𝑙𝑖𝑚𝑖𝑛𝑎𝑡𝑖𝑛𝑔 𝑡ℎ𝑒 𝑐𝑜𝑒𝑓𝑓𝑖𝑐𝑖𝑒𝑛𝑡
48 𝑏𝑒𝑙𝑜𝑤 𝑡ℎ𝑒 𝑝𝑖𝑣𝑜𝑡
49 b[k] = b[k] - multiplier * b[i] ;
50 }
51 }
52
53 if ( a[n-1][n-1]==0 ) {
54 cout<< "The system has no unique solution. " ;
55 break ; }
56 else
57 x[n-1] = b[n-1] / a[n-1][n-1] ;
58
59 for ( i=n-2 ; i>=0 ; i-- )
60 {
61 sum = 0.0 ;
62 for( j=i+1 ; j<n ; j++ ) 𝑛
1
63 sum = sum + a[i][j] * x[j] ; (𝑥𝑖 = [𝑏 − ∑ 𝑎𝑖𝑗 𝑥𝑗 ])
𝑎𝑖𝑖 𝑖
64 𝑗=𝑖+1

65 x[i] = ( b[i] – sum ) / a[i][i] ;

66 }
67 //------------------------ Output Section ----------------------------//
68
69 cout<< "The solution of the given system is " <<endl ;
70 for ( i=0; i<n ; i++ )
71 cout<< x[i] << " " ;
72
73 cin>> quit ;
74 return 0 ;
75 }
Direct Linear Solvers 77

Remark: The C++ program in Problem 16 can be modified to receive the linear system at the
execution time (instead of fixing in the code). For this, lines 10 and 11 in the solution of Problem
16 should be replaced by the following code segment:

//------------------------- Input Section -----------------------------//

cout<<"\nEnter the coefficient matrix row-wise: "<<n<<" unknowns<<endl;

for ( i=0 ; i<n ; i++)
for ( j=0 ; j<n ; j++)
cin >> a[i][j] ;

cout<<"Enter the elements of constant vector B "<<endl ;

for ( i=0 ; i<n ; i++)
cin >> b[i] ;

Question 21: Write down an algorithm (pseudo code) to solve a linear system using the Doolittle’s method.

Algorithm: To solve a linear system 𝑨𝑿 = 𝑩, for which the factorization 𝑨 = 𝑳𝑼 is possible.

𝒏: an integer as the number of equations and unknowns
𝐈𝐍𝐏𝐔𝐓𝐒: {𝑨 = (𝒂𝒊𝒋 ), 1 ≤ 𝑖, 𝑗 ≤ 𝒏: a real valued square matrix as the coefficient matrix
𝑩 = [𝒃𝟏 , 𝒃𝟐 , ⋯ , 𝒃𝒏 ]𝑻 : a real valued vector as the vector of right hand side constants

𝑳 = (𝒍𝒊𝒋 ), 1 ≤ 𝑖, 𝑗 ≤ 𝒏: a real valued square matrix as the lower triangular matrix

𝐎𝐔𝐓𝐏𝐔𝐓𝐒: {𝑼 = (𝒖𝒊𝒋 ), 1 ≤ 𝑖, 𝑗 ≤ 𝒏: a real valued square matrix as the upper triangular matrix
𝑿 = [𝒙𝟏 , 𝒙𝟐 , ⋯ , 𝒙𝒏 ]𝑻 : a real valued vector as the solution vector

Step 1 (Formation of 𝑳 and 𝑼 as factors of 𝑨, i.e., 𝑨 = 𝑳𝑼)

for 𝑖 = 1, 2, ⋯ , 𝒏

Set 𝒍𝒊𝒊 = 1
For 𝑗 = 𝑖, 𝑖 + 1, ⋯ , 𝒏
𝑠𝑢𝑚 = 0 𝑖−1
for 𝑠 = 1, 2, ⋯ , 𝑖 − 1
𝑠𝑢𝑚 = 𝑠𝑢𝑚 + 𝒍𝒊𝒔 × 𝒖𝒔𝒋 } (𝑢𝑖𝑗 = 𝑎𝑖𝑗 − ∑ 𝑙𝑖𝑠 𝑢𝑠𝑗 )
𝑠=1
𝒖𝒊𝒋 = 𝒂𝒊𝒋 − 𝑠𝑢𝑚

for 𝑗 = 𝑖 + 1, 𝑖 + 2, ⋯ , 𝒏
𝑠𝑢𝑚 = 0
for 𝑠 = 1, 2, ⋯ , 𝑖 − 1 𝑖−1
𝑠𝑢𝑚 = 𝑠𝑢𝑚 + 𝒍𝒋𝒔 × 𝒖𝒔𝒊 1
(𝑙𝑗𝑖 = [𝑎 − ∑ 𝑙𝑗𝑠 𝑢𝑠𝑖 ])
[𝒂𝒋𝒊 − 𝑠𝑢𝑚] 𝑢𝑖𝑖 𝑗𝑖
𝑠=1
𝒍𝒋𝒊 =
𝒖𝒊𝒊 }
78 Simplified Numerical Analysis

Step 2 (Forward substitution phase for solving 𝑳𝒀 = 𝑩)

𝒚𝟏 = 𝒃 𝟏
for 𝑖 = 2, 3, ⋯ , 𝒏
𝑠𝑢𝑚 = 0
𝑖−1
for 𝑗 = 1, 2, ⋯ , 𝑖 − 1
𝑠𝑢𝑚 = 𝑠𝑢𝑚 + 𝒍𝒊𝒋 × 𝒚𝒋 (𝑦𝑖 = 𝑏𝑖 − ∑ 𝑙𝑖𝑗 𝑦𝑗 )
𝑗=1
𝒚𝒊 = 𝒃𝒊 − 𝑠𝑢𝑚 }

Step 3 (Back Substitution Phase for solving 𝑼𝑿 = 𝒀)

𝒚𝒏
𝒙𝒏 =
𝒖𝒏𝒏
for 𝑖 = 𝒏 − 1, ⋯ ,2, 1
𝑠𝑢𝑚 = 0
for 𝑗 = 𝑖 + 1, 𝑖 + 2, ⋯ , 𝒏 𝑛
1
𝑠𝑢𝑚 = 𝑠𝑢𝑚 + 𝒖𝒊𝒋 × 𝒙𝒋 (𝑥𝑖 = [𝑦 − ∑ 𝑢𝑖𝑗 𝑥𝑗 ])
𝑢𝑖𝑖 𝑖
[𝒚𝒊 − 𝑠𝑢𝑚] 𝑗=𝑖+1
𝒙𝒊 =
𝒖𝒊𝒊 }
STOP.

Problem 18: Write a C++ program to solve the following linear system using the Doolittle’s method. For
simplification, specify the linear system within the program.
1.7𝑥1 + 2.3𝑥2 − 1.5𝑥3 = 2.35
1.1𝑥1 + 1.6𝑥2 − 1.9𝑥3 = −0.94
2.7𝑥1 − 2.2𝑥2 + 1.5𝑥3 = 2.70

1 #include<iostream>
2 using namespace std ;
3 # define n 3
4
5 int main ( )
6 {
7 char quit ;
8 int i , j , s ;
9 double a[n][n] = {{1.7, 2.3, -1.5} , {1.1, 1.6, -1.9} , {2.7, -2.2, 1.5}} ;
10 double b[n] = {2.35, -0.94, 2.7} ; // right-hand side constant vector
11 double x[n] ;
12 double u[n][n] , l[n][n] , y[n] , sum ;
13
14 //---------------------- Processing Section ----------------------//
15 // Formation of L and U as factors of A, i.e., A=LU
16
Direct Linear Solvers 79

17 for ( i=0 ; i<n ; i++ )

18 for ( j=0 ; j<n ; j++ )
19 l[i][j] = u[i][j] = 0.0 ;
20
21 for ( j=0 ; j<n ; j++ )
22 {
23 u[0][j] = a[0][j] ;
24 l[j][0] = a[j][0] / u[0][0] ;
25 }
26 for ( i=1 ; i<n ; i++ )
27 {
28 l[i][i] = 1 ;
29 for ( j=i ; j<n ; j++ )
30 {
31 sum = 0.0 ;
𝑖−1
32 for ( s=0 ; s<=i-1 ; s++ )
(𝑢𝑖𝑗 = 𝑎𝑖𝑗 − ∑ 𝑙𝑖𝑠 𝑢𝑠𝑗 )
33 sum = sum + l[i][s] * u[s][j] ;
𝑠=1
34 u[i][j] = a[i][j] – sum ;
35 }
36
37 for ( j=i+1 ; j<n ; j++ )
38 {
39 sum = 0.0 ;
𝑖−1
40 for ( s=0 ; s<=i-1 ; s++ ) 1
(𝑙𝑗𝑖 = [𝑎 − ∑ 𝑙𝑗𝑠 𝑢𝑠𝑖 ])
41 sum = sum + l[j][s] * u[s][i] ; 𝑢𝑖𝑖 𝑗𝑖
𝑠=1
42 l[j][i] = ( a[j][i] - sum) / u[i][i] ;
43 }
44 }
45
46 // Forward substitution phase for solving LY=B
47
48 y[0] = b[0] ;
49 for ( i=1 ; i<n ; i++ )
50 {
51 sum = 0.0 ;
𝑖−1
52 for ( j=0 ; j<=i-1 ; j++ )
(𝑦𝑖 = 𝑏𝑖 − ∑ 𝑙𝑖𝑗 𝑦𝑗 )
53 sum = sum + l[i][j] * y[j] ;
𝑗=1
54 y[i] = b[i] – sum ;
55 }
56
57 // Back Substitution Phase for solving UX=Y
80 Simplified Numerical Analysis

58
59 x[n-1] = y[n-1] / u[n-1][n-1] ;
60 for ( i=n-2 ; i>=0 ; i-- )
61 {
62 sum = 0.0 ;
𝑛
63 for ( j=i+1 ; j<n ; j++ ) 1
(𝑥𝑖 = [𝑦 − ∑ 𝑢𝑖𝑗 𝑥𝑗 ])
64 sum = sum + u[i][j] * x[j] ; 𝑢𝑖𝑖 𝑖
𝑗=𝑖+1
65 x[i] = ( y[i] – sum ) / u[i][i] ;
66 }
67
68 //---------------------- Output Section ----------------------//
69
70 cout<< "The L matrix is" << endl ;
71 for ( i=0 ; i<n ; i++ )
72 {
73 for ( j = 0 ; j<n ; j++ )
74 {
75 cout<< l[i][j]<< " " ;
76 }
77 cout<< endl ;
78 }
79 cout<< endl ;
80 cout<< "The U matrix is" << endl ;
81 for ( i=0 ; i<n ; i++ )
82 {
83 for ( j = 0 ; j<n ; j++ )
84 {
85 cout<< u[i][j] << " " ;
86 }
87 cout<< endl ;
88 }
89 cout<< endl ;
90 cout<< "The required solution is" << endl ;
91 for ( i=0 ; i<n ; i++ )
92 cout<< x[i] << " " ;
93
94 cin>> quit ;
95 return 0 ;
96 }
∎
Direct Linear Solvers 81

Remark: Replace the lines 9 and 10 in the solution of Problem 18 with the following code
segment to receive the linear system at the execution time (instead of fixing in the code):

//------------------------- Input Section -----------------------------//

cout<<"\nEnter the coefficient matrix row-wise: "<<n<<" unknowns<<endl;

for ( i=0 ; i<n ; i++)
for ( j=0 ; j<n ; j++)
cin >> a[i][j] ;

cout<<"Enter the elements of constant vector B "<<endl ;

for ( i=0 ; i<n ; i++)
cin >> b[i] ;
∎

Question 22: Write down an algorithm (pseudo code) to solve a linear system using the Crout’s method.

Algorithm: To solve a linear system 𝑨𝑿 = 𝑩, for which the factorization 𝑨 = 𝑳𝑼 is possible.

𝒏: an integer as the number of equations and unknowns
𝐈𝐍𝐏𝐔𝐓𝐒: {𝑨 = (𝒂𝒊𝒋 ), 1 ≤ 𝑖, 𝑗 ≤ 𝒏: a real valued square matrix as the coefficient matrix
𝑩 = [𝒃𝟏 , 𝒃𝟐 , ⋯ , 𝒃𝒏 ]𝑻 : a real valued vector as the vector of right hand side constants

𝑳 = (𝒍𝒊𝒋 ), 1 ≤ 𝑖, 𝑗 ≤ 𝒏: a real valued square matrix as the lower triangular matrix

𝐎𝐔𝐓𝐏𝐔𝐓𝐒: {𝑼 = (𝒖𝒊𝒋 ), 1 ≤ 𝑖, 𝑗 ≤ 𝒏: a real valued square matrix as the upper triangular matrix
𝑿 = [𝒙𝟏 , 𝒙𝟐 , ⋯ , 𝒙𝒏 ]𝑻 : a real valued vector as the solution vector

Step 1 (Formation of 𝑳 and 𝑼 as factors of 𝑨, i.e., 𝑨 = 𝑳𝑼)

for 𝑖 = 1, 2, ⋯ , 𝒏

Set 𝒖𝒊𝒊 = 1
for 𝑗 = 𝑖, 𝑖 + 1, ⋯ , 𝒏
𝑠𝑢𝑚 = 0
𝑖−1
for 𝑠 = 1, 2, ⋯ , 𝑖 − 1
𝑠𝑢𝑚 = 𝑠𝑢𝑚 + 𝒍𝒋𝒔 × 𝒖𝒔𝒊 (𝑙𝑗𝑖 = 𝑎𝑗𝑖 − ∑ 𝑙𝑗𝑠 𝑢𝑠𝑖 )
𝑠=1
𝒍𝒋𝒊 = 𝒂𝒋𝒊 − 𝑠𝑢𝑚 }
for 𝑗 = 𝑖 + 1, 𝑖 + 2, ⋯ , 𝒏
𝑠𝑢𝑚 = 0
for 𝑠 = 1, 2, ⋯ , 𝑖 − 1 𝑖−1
𝑠𝑢𝑚 = 𝑠𝑢𝑚 + 𝒍𝒊𝒔 × 𝒖𝒔𝒋 1
(𝑢𝑖𝑗 = [𝑎𝑖𝑗 − ∑ 𝑙𝑖𝑠 𝑢𝑠𝑗 ])
[𝒂𝒊𝒋 − 𝑠𝑢𝑚] 𝑙𝑖𝑖
𝑠=1
𝒖𝒊𝒋 =
𝒍𝒊𝒊 }
82 Simplified Numerical Analysis

Step 2 (Forward Substitution Phase for solving 𝑳𝒀 = 𝑩)

𝒃𝟏
𝒚𝟏 =
𝒍𝟏𝟏
for 𝑖 = 2, 3, ⋯ , 𝒏
𝑠𝑢𝑚 = 0
for 𝑗 = 1, 2, ⋯ , 𝑖 − 1 𝑖−1
1
𝑠𝑢𝑚 = 𝑠𝑢𝑚 + 𝒍𝒊𝒋 × 𝒚𝒋 (𝑦𝑖 = [𝑏 − ∑ 𝑙𝑖𝑗 𝑦𝑗 ])
[𝒃𝒊 − 𝑠𝑢𝑚] 𝑙𝑖𝑖 𝑖
𝑗=1
𝒚𝒊 =
𝒍𝒊𝒊 }

Step 3 (Back Substitution Phase for solving 𝑼𝑿 = 𝒀)

𝒙𝒏 = 𝒚𝒏
for 𝑖 = 𝒏 − 1, ⋯ ,2, 1
𝑠𝑢𝑚 = 0 𝑛
for 𝑗 = 𝑖 + 1, 𝑖 + 2, ⋯ , 𝒏
} (𝑥𝑖 = 𝑦𝑖 − ∑ 𝑢𝑖𝑗 𝑥𝑗 )
𝑠𝑢𝑚 = 𝑠𝑢𝑚 + 𝒖𝒊𝒋 × 𝒙𝒋
𝑗=𝑖+1
𝒙𝒊 = 𝒚𝒊 − 𝑠𝑢𝑚

STOP.

Problem 20: Write a C++ program to solve the following linear system using the Crout’s method. For
simplification, specify the linear system within the program.

1.7𝑥1 + 2.3𝑥2 − 1.5𝑥3 = 2.35

1.1𝑥1 + 1.6𝑥2 − 1.9𝑥3 = −0.94
2.7𝑥1 − 2.2𝑥2 + 1.5𝑥3 = 2.70
1 #include<iostream>
2 using namespace std ;
3 # define n 3 // number of unknowns
4
5 int main ( )
6 {
7 char quit ;
8 int i , j , s ;
9 double a[n][n] = {{1.7, 2.3, -1.5} , {1.1, 1.6, -1.9} , {2.7, -2.2, 1.5}} ;
10 double b[n] = {2.35, -0.94, 2.7} ; // right hand side constant vector
11 double x[n] ;
12 double u[n][n] , l[n][n] , y[n] , sum ;
13
14 //---------------------- Processing Section ----------------------//
15 // Formation of L and U as factors of A, i.e., A=LU
16
Direct Linear Solvers 83

17 for ( i=0 ; i<n ; i++ )

18 for ( j=0 ; j<n ; j++ )
19 l[i][j] = u[i][j] = 0.0 ;
20
21 for ( j=0 ; j<n ; j++ )
22 {
23 l[j][0] = a[j][0] ;
24 u[0][j] = a[0][j] / l[0][0] ;
25 }
26
27 for ( i=1 ; i<n ; i++ )
28 {
29 u[i][i] = 1 ;
30 for ( j=i ; j<n ; j++ )
31 {
32 sum = 0.0 ;
𝑖−1
33 for ( s=0 ; s<=i-1 ; s++ )
(𝑙𝑗𝑖 = 𝑎𝑗𝑖 − ∑ 𝑙𝑗𝑠 𝑢𝑠𝑖 )
34 sum = sum + l[j][s] * u[s][i];
𝑠=1
35 l[j][i] = a[j][i] – sum ;
36 }
37
38 for ( j=i+1 ; j<n ; j++ )
39 {
40 sum = 0.0 ;
𝑖−1
41 for ( s=0 ; s<=i-1 ; s++ ) 1
(𝑢𝑖𝑗 = [𝑎 − ∑ 𝑙𝑖𝑠 𝑢𝑠𝑗 ])
42 sum = sum + l[i][s] * u[s][j] ; 𝑙𝑖𝑖 𝑖𝑗
𝑠=1
43 u[i][j] = ( a[i][j] – sum ) / l[i][i] ;
44 }
45 }
46
47 // Forward substitution phase for solving LY=B
48
49 y[0] = b[0] / l[0][0] ;
50 for ( i=1 ; i<n ; i++ )
51 {
52 sum = 0.0 ;
𝑖−1
53 for ( j=0 ; j<=i-1 ; j++ ) 1
(𝑦𝑖 = [𝑏𝑖 − ∑ 𝑙𝑖𝑗 𝑦𝑗 ])
54 sum = sum + l[i][j] * y[j] ; 𝑙𝑖𝑖
𝑗=1
55 y[i] = ( b[i] – sum ) / l[i][i] ;
56 }
57
84 Simplified Numerical Analysis

58 // Back Substitution Phase for solving UX=Y

59
60 x[n-1] = y[n-1] ;
61 for ( i=n-2 ; i>=0 ; i-- )
62 {
63 sum = 0.0 ;
𝑛
64 for ( j=i+1 ; j<n ; j++ )
(𝑥𝑖 = 𝑦𝑖 − ∑ 𝑢𝑖𝑗 𝑥𝑗 )
65 sum = sum + u[i][j] * x[j] ;
𝑗=𝑖+1
66 x[i] = y[i] – sum ;
67 }
68
69 //---------------------- Output Section ----------------------//
70
71 cout<< "The L matrix is" << endl ;
72 for ( i=0 ; i<n ; i++ )
73 {
74 for ( j = 0 ; j<n ; j++ )
75 {
76 cout<< l[i][j]<< " " ;
77 }
78 cout<< endl ;
79 }
80 cout<< endl ;
81 cout<< "The U matrix is" << endl ;
82 for ( i=0 ; i<n ; i++ )
83 {
84 for ( j = 0 ; j<n ; j++ )
85 {
86 cout<< u[i][j] << " " ;
87 }
88 cout<< endl ;
89 }
90 cout<< endl ;
91 cout<< "The required solution is" << endl ;
92 for ( i=0 ; i<n ; i++ )
93 cout<< x[i] << " " ;
94
95 cin>> quit ;
96 return 0 ;
97 }
∎
Direct Linear Solvers 85

Remark: Replace the lines 9 and 10 in the solution of Problem 20 with the following code
segment to receive the linear system at the execution time (instead of fixing in the code):

//------------------------- Input Section -----------------------------//

cout<<"\nEnter the coefficient matrix row-wise: "<<n<<" unknowns<<endl;

for ( i=0 ; i<n ; i++)
for ( j=0 ; j<n ; j++)
cin >> a[i][j] ;

cout<<"Enter the elements of constant vector B "<<endl ;

for ( i=0 ; i<n ; i++)
cin >> b[i] ;
∎
86 Simplified Numerical Analysis

Question 23: Write down an algorithm (pseudo code) to solve a linear system using the Cholesky’s method.

Algorithm: To solve a linear system 𝑨𝑿 = 𝑩, for which the factorization 𝑨 = 𝑳𝑳𝑻 is possible.
𝒏: an integer as the number of equations and unknowns
𝐈𝐍𝐏𝐔𝐓𝐒: {𝑨 = (𝒂𝒊𝒋 ), 1 ≤ 𝑖, 𝑗 ≤ 𝒏: a real valued square matrix as the coefficient matrix
𝑩 = [𝒃𝟏 , 𝒃𝟐 , ⋯ , 𝒃𝒏 ]𝑻 : a real valued vector as the vector of right hand side constants

𝑳 = (𝒍𝒊𝒋 ), 1 ≤ 𝑖, 𝑗 ≤ 𝒏: a real valued square matrix as the lower triangular matrix

𝐎𝐔𝐓𝐏𝐔𝐓𝐒: {
𝑿 = [𝒙𝟏 , 𝒙𝟐 , ⋯ , 𝒙𝒏 ]𝑻 : a real valued vector as the solution vector

Step 1 (Formation of 𝑳 as factors of 𝑨, i.e., 𝑨 = 𝑳𝑳𝑻 )

for 𝑖 = 1, 2, ⋯ , 𝒏
1
𝑠𝑢𝑚 = 0 𝑖−1 2
for 𝑘 = 1, 2, ⋯ , 𝑖 − 1 2
} 𝑙𝑖𝑖 = [𝑎𝑖𝑖 − ∑ 𝑙𝑖𝑘 ]
𝑠𝑢𝑚 = 𝑠𝑢𝑚 + 𝒍𝒊𝒌 × 𝒍𝒊𝒌
𝑘=1
𝒍𝒊𝒊 = 𝒔𝒒𝒓𝒕(𝒂𝒊𝒊 − 𝑠𝑢𝑚)
( )
for 𝑗 = 𝑖 + 1, 𝑖 + 2, ⋯ , 𝒏
𝑠𝑢𝑚 = 0
for 𝑘 = 1, 2, ⋯ , 𝑖 − 1 𝑖−1
𝑠𝑢𝑚 = 𝑠𝑢𝑚 + 𝒍𝒊𝒌 × 𝒍𝒋𝒌 1
(𝑙𝑗𝑖 = [𝑎𝑗𝑖 − ∑ 𝑙𝑖𝑘 𝑙𝑗𝑘 ])
[𝒂𝒋𝒊 − 𝑠𝑢𝑚] 𝑙𝑖𝑖
𝑘=1
𝒍𝒋𝒊 =
𝒍𝒊𝒊 }

Step 2 (Forward Substitution Phase for solving 𝑳𝒀 = 𝑩)

𝒃𝟏
𝒚𝟏 =
𝒍𝟏𝟏
for 𝑖 = 2, 3, ⋯ , 𝒏
𝑠𝑢𝑚 = 0
for 𝑗 = 1, 2, ⋯ , 𝑖 − 1 𝑖−1
1
𝑠𝑢𝑚 = 𝑠𝑢𝑚 + 𝒍𝒊𝒋 × 𝒚𝒋 (𝑦𝑖 = [𝑏𝑖 − ∑ 𝑙𝑖𝑗 𝑦𝑗 ])
[𝒃𝒊 − 𝑠𝑢𝑚] 𝑙𝑖𝑖
𝑗=1
𝒚𝒊 =
𝒍𝒊𝒊 }

Step 3 (Back Substitution Phase for solving 𝑳𝑻 𝑿 = 𝒀)

𝒚𝒏
𝒙𝒏 =
𝒍𝒏𝒏
for 𝑖 = 𝒏 − 1, ⋯ ,2, 1
𝑠𝑢𝑚 = 0
for 𝑗 = 𝑖 + 1, 𝑖 + 2, ⋯ , 𝒏 𝑛
1
𝑠𝑢𝑚 = 𝑠𝑢𝑚 + 𝒍𝒋𝒊 × 𝒙𝒋 (𝑥𝑖 = [𝑦 − ∑ 𝑙𝑗𝑖 𝑥𝑗 ])
[𝒚𝒊 − 𝑠𝑢𝑚] 𝑙𝑖𝑖 𝑖
𝑗=𝑖+1
𝒙𝒊 =
𝒍𝒊𝒊 }

STOP.
Direct Linear Solvers 87

Problem 22: Write a C++ program to solve the following positive definite linear system using the Cholesky’s
method. For simplification, specify the linear system within the program.

0.4𝑥1 + 0.12𝑥3 = 1.4

0.64𝑥2 + 0.32𝑥3 = 1.6
−0.12𝑥1 + 0.32𝑥2 + 0.56𝑥3 = 5.4

1 #include <iostream>
2 #include <cmath>
3 using namespace std ;
4 # define n 3 // number of unknowns
5
6 int main ( )
7 {
8 char quit ;
9 int i , j , k ;
10 double a[n][n] = {{0.4, 0, 0.12} , {0, 0.64, 0.32} , {-0.12, 0.32, 0.56}} ;
11 double b[n] = {1.4, 1.6, 5.4} ; // right hand side constant vector
12 double x[n] ;
13 double l[n][n] , y[n] , sum ;
14
15 //---------------------- Processing Section ----------------------//
16 // Formation of L and U as factors of A, i.e., A=LU
17
18 for ( i=0 ; i<n ; i++ )
19 for ( j=0 ; j<n ; j++ )
20 l[i][j] = 0.0 ;
21 l[0][0] = sqrt( a[0][0] ) ;
22
23 for ( j=1 ; j<n ; j++ )
24 l[j][0] = a[j][0] / l[0][0] ;
25
26 for ( i=1 ; i<n ; i++ )
27 {
28 sum = 0.0 ; 1
𝑖−1 2
29 for ( k=0 ; k<=i-1 ; k++ ) 2
𝑙𝑖𝑖 = [𝑎𝑖𝑖 − ∑ 𝑙𝑖𝑘 ]
30 sum = sum + l[i][k] * l[i][k] ; 𝑘=1
31 l[i][i] = sqrt( a[i][i] – sum ) ; ( )
32
88 Simplified Numerical Analysis

33 for ( j=i+1 ; j<n ; j++ )

34 {
35 sum = 0.0 ;
𝑖−1
36 for ( k=0 ; k<=i-1 ; k++ ) 1
(𝑙𝑗𝑖 = [𝑎𝑗𝑖 − ∑ 𝑙𝑖𝑘 𝑙𝑗𝑘 ])
37 sum = sum + l[i][k] * l[j][k] ; 𝑙𝑖𝑖
𝑘=1
38 l[j][i] = (a[j][i] – sum ) / l[i][i] ;
39 }
40 }
41
42 // Forward substitution phase for solving LY=B
43
44 y[0] = b[0] / l[0][0] ;
45 for ( i=1 ; i<n ; i++ )
46 {
47 sum = 0.0 ;
𝑖−1
48 for ( j=0 ; j<=i-1 ; j++ ) 1
(𝑦𝑖 = [𝑏 − ∑ 𝑙𝑖𝑗 𝑦𝑗 ])
49 sum = sum + l[i][j] * y[j] ; 𝑙𝑖𝑖 𝑖
𝑗=1
50 y[i] = ( b[i] – sum ) / l[i][i] ;
51 }
52
53 // Back Substitution Phase for solving L^T X=Y
54
55 x[n-1] = y[n-1] / l[n-1][n-1] ;
56 for ( i=n-2 ; i>=0 ; i-- )
57 {
58 sum = 0.0 ;
𝑛
59 for ( j=i+1 ; j<n ; j++ ) 1
(𝑥𝑖 = [𝑦 − ∑ 𝑙𝑗𝑖 𝑥𝑗 ])
60 sum = sum + l[j][i] * x[j] ; 𝑙𝑖𝑖 𝑖
𝑗=𝑖+1
61 x[i] = (y[i] – sum ) / l[i][i] ;
62 }
63
64 //---------------------- Output Section ----------------------//
65
66 cout<< "The L matrix is" << endl ;
67 for ( i=0 ; i<n ; i++ )
68 {
69 for ( j = 0 ; j<n ; j++ )
70 {
71 cout<< l[i][j]<< " " ;
72 }
73 cout<< endl ;
Direct Linear Solvers 89

74 }
75 cout<< endl ;
76 cout<< "The required solution is" << endl ;
77 for ( i=0 ; i<n ; i++ )
78 cout<< x[i] << " " ;
79
80 cin>> quit ;
81 return 0 ;
82 }

Remark: Following are some notations and formulas that might be useful in carrying out
operation count analysis of the algorithms.
𝑛 𝑛

∑ 𝑐𝑓(𝑝) = 𝑐 ∑ 𝑓(𝑝)
𝑝=1 𝑝=1
𝑛 𝑛 𝑛

∑[𝑓(𝑝) + 𝑔(𝑝)] = ∑ 𝑓(𝑝) + ∑ 𝑔(𝑝)

𝑝=1 𝑝=1 𝑝=1
𝑛

∑1 = 1 + 1+ ⋯+ 1 = 𝑛
𝑝=1
𝑛

∑1 = 𝑛−𝑘+1
𝑝=𝑘
𝑛
𝑛(𝑛 + 1) 𝑛2
∑𝑝 = 1 + 2+ 3 + ⋯+ 𝑛 = = + 𝒪(𝑛)
2 2
𝑝=1
𝑛
𝑛(𝑛 + 1)(2𝑛 + 1) 𝑛3
∑ 𝑝2 = 12 + 22 + ⋯ + 𝑛 2 = = + 𝒪(𝑛2 )
𝑝=1
6 3

Question 24: Perform the operation count analysis of the algorithm that involves the following
phases to solve an 𝑛 × 𝑛 linear system:

(1) Forward elimination to obtain the upper triangular form using the Gauss Elimination method.
(2) Back substitution to solve the upper triangular system.
90 Simplified Numerical Analysis

(1) The forward elimination phase occurs just after setting the inputs in the algorithm. This phase
contains three nested loops. The first loop, say 𝑖-loop (which ranges from 𝑖 = 1 to 𝑛 − 1),
corresponds to the 𝑛 − 1 elimination stages of the method. For each row 𝑖, the 𝑖th element is
considered a pivot element. The second loop, say 𝑘-loop (which ranges from 𝑘 = 𝑖 + 1 to 𝑛),
corresponds to the elements below the pivot element to make them zero. The third loop, say 𝑗-
loop (which ranges from 𝑗 = 𝑖 + 1 to 𝑛) corresponds to the columns after the pivot element.

Note that, for any loop with index ranging from 𝑖 + 1 to 𝑛, the number of passes/iterations will be
𝑛 − (𝑖 + 1) + 1 (or simply (𝑛 − 𝑖) passes). Therefore, each of the 𝑘-loop and 𝑗-loop has (𝑛 − 𝑖)
passes.

Each pass of 𝑘-loop will perform one division to obtain the multiplier, and one multiplication and
subtraction to update the right-hand side constant, 𝑏𝑘 . Moreover, in each pass of 𝑘-loop, (𝑛 − 𝑖)
multiplications and (𝑛 − 𝑖) subtractions will be performed in 𝑗-loop to update the relevant entries
of the coefficient matrix, 𝑎𝑘𝑗 . Thus, in each pass of 𝑘 -loop, the total number of
multiplications/divisions will be (1 + 1 + (𝑛 − 𝑖)) or (𝑛 − 𝑖 + 2) and the total number of
additions/subtractions will be (1 + 𝑛 − 𝑖).

As there are (𝑛 − 𝑖) passes of 𝑘-loop in each pass of 𝑖-loop, therefore there will be (𝑛 − 𝑖) × (𝑛 −
𝑖 + 2) multiplications/divisions and (𝑛 − 𝑖) × (𝑛 − 𝑖 + 1) additions/subtractions in each pass of 𝑖-
loop.

Hence, the total number of multiplications/divisions in 𝑛 − 1 passes of 𝑖-loop of the forward

elimination phase will be

𝑛−1 𝑛−1

∑(𝑛 − 𝑖)(𝑛 − 𝑖 + 2) = ∑(𝑛 − 𝑖)((𝑛 + 2) − 𝑖)

𝑖=1 𝑖=1
𝑛−1 𝑛−1
2]
= ∑[𝑛(𝑛 + 2) − 𝑛𝑖 − 𝑖(𝑛 + 2) + 𝑖 = ∑[𝑛(𝑛 + 2) − 2𝑖(𝑛 + 1) + 𝑖 2 ]
𝑖=1 𝑖=1
𝑛−1 𝑛−1 𝑛−1

= 𝑛(𝑛 + 2) ∑ 1 − 2(𝑛 + 1) ∑ 𝑖 + ∑ 𝑖 2
𝑖=1 𝑖=1 𝑖=1

(𝑛 − 1)𝑛 (𝑛 − 1)𝑛(2𝑛 − 1)
= 𝑛(𝑛 + 2)(𝑛 − 1) − 2(𝑛 + 1) [ ]+[ ]
2 6
𝑛 1
= 𝑛(𝑛 − 1) [𝑛 + 2 − 𝑛 − 1 + − ]
3 6
𝑛 5 𝑛3 𝑛2 5𝑛 𝑛3
= (𝑛2 − 𝑛) [ + ] = + − = + 𝒪(𝑛2 )
3 6 3 2 6 3
Direct Linear Solvers 91

Similarly, the total number of additions/subtractions in 𝑛 − 1 passes of 𝑖-loop of the forward

elimination phase will be

𝑛−1 𝑛−1

∑(𝑛 − 𝑖)(𝑛 − 𝑖 + 1) = ∑(𝑛 − 𝑖)((𝑛 + 1) − 𝑖)

𝑖=1 𝑖=1
𝑛−1

= ∑[𝑛(𝑛 + 1) − 𝑛𝑖 − 𝑖(𝑛 + 1) + 𝑖 2 ]
𝑖=1
𝑛−1

= ∑[𝑛(𝑛 + 1) − 𝑖(2𝑛 + 1) + 𝑖 2 ]
𝑖=1
𝑛−1 𝑛−1 𝑛−1

= 𝑛(𝑛 + 1) ∑ 1 − (2𝑛 + 1) ∑ 𝑖 + ∑ 𝑖 2
𝑖=1 𝑖=1 𝑖=1

(𝑛 − 1)𝑛 (𝑛 − 1)𝑛(2𝑛 − 1)
= 𝑛(𝑛 + 1)(𝑛 − 1) − (2𝑛 + 1) [ ]+[ ]
2 6
1 𝑛 1
= 𝑛(𝑛 − 1) [𝑛 + 1 − 𝑛 − + − ]
2 3 6
𝑛 1 𝑛3 𝑛 𝑛3
= (𝑛2 − 𝑛) [ + ] = − = + 𝒪(𝑛)
3 3 3 3 3

The summary of the operation count of the Gaussian Elimination phase is given as:

Operations
Total flops
Multiplications⁄divisions Additions⁄subtractions

𝑛3 𝑛2 5𝑛 𝑛3 𝑛 2𝑛3
Forward Elimination + − − + 𝒪(𝑛2 )
3 2 6 3 3 3

(2) The back substitution phase occurs after the forward elimination phase. This phase contains
two nested loops. The first loop, say 𝑖-loop (which ranges from 𝑖 = 𝑛 − 1 to 1), corresponds to
𝑛 − 1 of the components of the solution vector. The second loop, say 𝑗-loop (which ranges from
𝑗 = 𝑖 + 1 to 𝑛), corresponds to the columns after the diagonal elements.

Each pass of 𝑖-loop will perform one subtraction and one division to obtain the value of 𝑥𝑖 .
Moreover, in each pass of 𝑖-loop, the number of both of the multiplications and additions will be
𝑛 − (𝑖 + 1) + 1 (or simply (𝑛 − 𝑖)) in 𝑗-loop. Thus, in each pass of 𝑖-loop, the total number of both
of the multiplications/divisions and additions/subtractions will be (𝑛 − 𝑖 + 1).
92 Simplified Numerical Analysis

Hence, the total number of the multiplications/divisions in the back substitution phase will be

𝑛−1 𝑛−1 𝑛−1

(𝑛 − 1)𝑛
1 + ∑(𝑛 + 1 − 𝑖) = 1 + (𝑛 + 1) ∑ 1 − ∑ 𝑖 = 1 + (𝑛 + 1)(𝑛 − 1) − [ ]
2
𝑖=1 𝑖=1 𝑖=1
2
𝑛 𝑛 𝑛2 𝑛 𝑛2
= 1 + 𝑛2 − 1 − + = + = + 𝒪(𝑛)
2 2 2 2 2

Similarly, the total number of the additions/subtractions in the back substation phase will be

𝑛−1
𝑛2 𝑛 𝑛2
∑(𝑛 + 1 − 𝑖) = + −1 = + 𝒪(𝑛)
𝑖=1
2 2 2

Finally, the summary of the operation count of the complete algorithm (including the two phases)
is given as:

Operations
Total flops
Multiplications⁄divisions Additions⁄subtractions

𝑛3 𝑛2 5𝑛 𝑛3 𝑛 2𝑛3 𝑛2 7𝑛
Forward elimination + − − + −
3 2 6 3 3 3 2 6

𝑛2 𝑛 𝑛2 𝑛
Back Substitution + − 𝑛2
2 2 2 2

𝑛3 𝑛 𝑛3 n2 5𝑛 2𝑛3 3𝑛2 7𝑛
Totals + 𝑛2 − + − + −
3 3 3 2 6 3 2 6

Question 25: Perform the operation count analysis of the algorithm that involves the following
phases to solve an 𝑛 × 𝑛 linear system:

(1) Factorization of the coefficient matrix using the Doolittle’s method.

(2) Forward substitution to solve the lower triangular system.
(3) Back substitution to solve the upper triangular system.

(1) The factorization of the coefficient matrix 𝐴 into the product of the unit lower triangular 𝐿 and
the upper triangular 𝑈 matrices occurs just after setting the inputs in the algorithm. The
formulation of 𝐿 and 𝑈 as the factors of 𝐴 contains three nested loops. The first loop, say 𝑖-loop
(which ranges from 𝑖 = 1 to 𝑛), corresponds to the 𝑖th row and column of 𝑈 and 𝐿 respectively.
The second loop, say 𝑗-loop (ranges from 𝑗 = 𝑖 to 𝑛), corresponds to the column 𝑗 of 𝑈 and (ranges
Direct Linear Solvers 93

from 𝑗 = 𝑖 + 1 to 𝑛), corresponds to the row 𝑗 of 𝐿. The third loop, say 𝑠-loop (which ranges from
𝑠 = 1 to 𝑖 − 1), corresponds to the multiplication of the rows of 𝐿 and columns of 𝑈.

Note that, the 𝑗-loop corresponding to column 𝑗 of 𝑈 ranging from 𝑖 to 𝑛, the number of
passes/iterations will be 𝑛 − 𝑖 + 1 . Similarly, the number of passes/iterations in 𝑗 -loop,
corresponds to row 𝑗 of 𝐿 ranging from 𝑖 + 1 to 𝑛, will be 𝑛 − (𝑖 + 1) + 1 (or simply (𝑛 − 𝑖)).

Each pass of 𝑗-loop will perform one subtraction to obtain the entry 𝑢𝑖𝑗 of 𝑈. Moreover, in each
pass of 𝑗-loop, the number of both of the multiplications and additions will be (𝑖 − 1) − 1 + 1 (or
simply (𝑖 − 1) ) in s-loop. Thus, in each pass of j-loop, the total number of
multiplications/divisions will be (𝑖 − 1) and the total number of additions/subtractions will be
(1 + 𝑖 − 1) or (𝑖). Similarly, in each pass of 𝑗-loop, to obtain the entry 𝑙𝑗𝑖 of 𝐿, the total number of
both of the multiplications and additions will be (𝑖 − 1) + 1 (or simply (𝑖)).

As there are (𝑛 − 𝑖 + 1) passes of 𝑗-loop in each pass of 𝑖 -loop, therefore there will be
(𝑛 − 𝑖 + 1) × (𝑖 − 1) multiplications/divisions and (𝑛 − 𝑖 + 1) × (𝑖) additions/subtractions in
each pass of 𝑖-loop for the formulation of row 𝑖 of 𝑈. Similarly, in each pass of 𝑖-loop, there will be
(𝑛 − 𝑖) × (𝑖) multiplications/divisions and (𝑛 − 𝑖) × (𝑖) additions/subtractions in each pass of 𝑖-
loop for the formulation of column 𝑖 of 𝐿.

Hence, the total number of multiplications/divisions in 𝑛 passes of 𝑖-loop for the formulation of
upper triangular matric 𝑈 will be

𝑛 𝑛

∑(𝑛 − 𝑖 + 1)(𝑖 − 1) = ∑(𝑛 + 1 − 𝑖)(𝑖 − 1)

𝑖=1 𝑖=1
𝑛 𝑛

= ∑[(𝑛 + 1)𝑖 − (𝑛 + 1) − 𝑖 2 + 𝑖] = ∑[(𝑛 + 2)𝑖 − 𝑖 2 − (𝑛 + 1)]

𝑖=1 𝑖=1
𝑛 𝑛 𝑛

= (𝑛 + 2) ∑ 𝑖 − ∑ 𝑖 2 − (𝑛 + 1) ∑ 1
𝑖=1 𝑖=1 𝑖=1

𝑛(𝑛 + 1) 𝑛(𝑛 + 1)(2𝑛 + 1)

= (𝑛 + 2) [ ]−[ ] − (𝑛 + 1)𝑛
2 6
𝑛 𝑛 1 𝑛 1
= 𝑛(𝑛 + 1) [ + 1 − − − 1] = (𝑛2 + 𝑛) [ − ]
2 3 6 6 6
𝑛3 𝑛 𝑛3
= − = + 𝒪(𝑛)
6 6 6
94 Simplified Numerical Analysis

Similarly, the total number of additions/subtractions in 𝑛 passes of 𝑖-loop for the formulation of
upper triangular matric 𝑈 will be

𝑛
𝑛3 𝑛 𝑛3
∑(𝑛 − 𝑖 + 1)(𝑖) = + = + 𝒪(𝑛)
6 6 6
𝑖=1

Moreover, the total number of multiplications/divisions and additions/subtractions in 𝑛 passes of

𝑖-loop for the formulation of unit lower triangular matric 𝐿 will be

𝑛 𝑛 𝑛

∑(𝑛 − 𝑖)(𝑖) = ∑(𝑛 − 𝑖)(𝑖) = ∑(𝑛𝑖 − 𝑖 2 )

𝑖=1 𝑖=1 𝑖=1

𝑛 𝑛
𝑛(𝑛 + 1) 𝑛(𝑛 + 1)(2𝑛 + 1)
= 𝑛 ∑ 𝑖 − ∑ 𝑖2 = 𝑛[ ]−[ ]
2 6
𝑖=1 𝑖=1

𝑛 𝑛 1 𝑛 1
= 𝑛(𝑛 + 1) [ − − ] = (𝑛2 + 𝑛) [ − ]
2 3 6 6 6

𝑛3 𝑛 𝑛3
= − = + 𝒪(𝑛)
6 6 6

The summary of the operation count of the 𝐿𝑈-factorization is given as:

Operations
Total flops
Multiplication/Division Addition/Subtraction

Upper Triangular 𝑛3 𝑛 𝑛3 𝑛 𝑛3
Matrix 𝑈 − +
6 6 6 6 3

Lower Triangular 𝑛3 𝑛 𝑛3 𝑛 𝑛3 𝑛
Matrix 𝐿 − − −
6 6 6 6 3 3

𝑛3 𝑛 𝑛3 2𝑛3 𝑛
𝐿𝑈-factorization − −
3 3 3 3 3

(2) The forward substitution phase occurs after the formulation of 𝐿 and 𝑈 as factors of the
coefficient matrix for solving the lower triangular system. This phase contains two nested loops.
The first loop, say 𝑖-loop (which ranges from 𝑖 = 2 to 𝑛), corresponds to 𝑛 − 1 of the components
Direct Linear Solvers 95

of the intermediate vector 𝑌. The second loop, say 𝑗-loop (which ranges from 𝑗 = 1 to 𝑖 − 1),
corresponds to the columns before the diagonal elements.

Each pass of 𝑖-loop will perform one subtraction to obtain the value of 𝑦𝑖 . Moreover, in each pass
of 𝑖-loop, the number of both of the multiplications and additions will be (𝑖 − 1) − 1 + 1 (or
simply (𝑖 − 1)) in j-loop. Thus, in each pass of i-loop, the total number of multiplications/divisions
will be (𝑖 − 1) and the total number of additions/subtractions will be (1 + 𝑖 − 1) or (𝑖).

Hence, the total number of the multiplications/divisions in the forward substitution phase will be

𝑛 𝑛 𝑛
𝑛(𝑛 + 1)
∑(𝑖 − 1) = ∑𝑖 − ∑1 = [ − 1] − (𝑛 − 2 + 1)
2
𝑖=2 𝑖=2 𝑖=2

𝑛2 𝑛 𝑛2 𝑛 𝑛2
= + −1−𝑛+1 = − = + 𝒪(𝑛)
2 2 2 2 2

Similarly, the total number of the additions/subtractions in the forward substation phase will be

𝑛
𝑛2 𝑛 𝑛2
∑(𝑖) = + −1 = + 𝒪(𝑛)
𝑖=2
2 2 2

The summary of the operation count of the Unit Lower triangular system 𝐿𝑌 = 𝐵 is given as:

Operations
Total flops
Multiplication/Division Addition/Subtraction
Unit Lower
𝑛2 𝑛 𝑛2 𝑛
triangular system − + −1 𝑛2 − 1
𝐿𝑌 = 𝐵 2 2 2 2

(3) The back substitution phase occurs after the solution of the lower triangular system. This
phase contains two nested loops. The first loop, say 𝑖-loop (which ranges from 𝑖 = 𝑛 − 1 to 1),
corresponds to 𝑛 − 1 of the components of solution vector 𝑋. The second loop, say 𝑗-loop (which
ranges from 𝑗 = 𝑖 + 1 to 𝑛), corresponds to the columns after the diagonal elements.

Each pass of 𝑖-loop will perform one subtraction and one division to obtain the value of 𝑥𝑖 .
Moreover, in each pass of 𝑖-loop, the number of both of the multiplications and additions will be
𝑛 − (𝑖 + 1) + 1 (or simply (𝑛 − 𝑖)) in 𝑗-loop. Thus, in each pass of 𝑖-loop, the total number of both
of the multiplications/divisions and additions/subtractions will be (𝑛 − 𝑖 + 1).
96 Simplified Numerical Analysis

Hence, the total number of the multiplications/divisions in the back substitution phase will be

𝑛−1 𝑛−1 𝑛−1

1 + ∑(𝑛 + 1 − 𝑖) = 1 + (𝑛 + 1) ∑ 1 − ∑ 𝑖
𝑖=1 𝑖=1 𝑖=1
(𝑛 − 1)𝑛
= 1 + (𝑛 + 1)(𝑛 − 1) − [ ]
2
𝑛2 𝑛 𝑛2 𝑛 𝑛2
= 1 + 𝑛2 − 1 − + = + = + 𝒪(𝑛)
2 2 2 2 2

Similarly, the total number of the additions/subtractions in the back substation phase will be

𝑛−1
𝑛2 𝑛 𝑛2
∑(𝑛 + 1 − 𝑖) = + −1 = + 𝒪(𝑛)
𝑖=1
2 2 2

The summary of the operation count of the Upper triangular system 𝑈𝑋 = 𝑌 is given as:

Operations
Total flops
Multiplications/Divisions Additions/Subtractions

Upper triangular 𝑛2 𝑛 𝑛2 𝑛
+ + −1 𝑛2 + 𝑛 − 1
system 𝑈𝑋 = 𝑌 2 2 2 2

Question 26: Perform the operation count analysis of the algorithm that involves the following
phases to solve an 𝑛 × 𝑛 linear system:

(1) Factorization of the coefficient matrix using the Doolittle’s method

(2) Forward substitution to solve the lower triangular system.
(3) Back substitution to solve the upper triangular system.

(1) The factorization of the coefficient matrix 𝐴 into the product of the lower triangular 𝐿 and the
unit upper triangular 𝑈 matrices occur just after setting the inputs in the algorithm. The
formulation of 𝐿 and 𝑈 as the factors of 𝐴 contains three nested loops. The first loop, say 𝑖-loop
(which ranges from 𝑖 = 1 to 𝑛), corresponds to the 𝑖th column of 𝐿 and 𝑖th row of 𝑈 respectively.
The second loop, say 𝑗-loop (ranges from 𝑗 = 𝑖 to 𝑛), corresponds to the 𝑗th row of 𝐿 and (ranges
from 𝑗 = 𝑖 + 1 to 𝑛), corresponds to the 𝑗th column of 𝑈. The third loop, say 𝑠-loop (which ranges
from 𝑠 = 1 to 𝑖 − 1), corresponds to the multiplication of the rows of 𝐿 and columns of 𝑈.
Direct Linear Solvers 97

Note that, the 𝑗 -loop corresponding to row 𝑗 of 𝐿 ranging from 𝑖 to 𝑛 , the number of
passes/iterations will be 𝑛 − 𝑖 + 1 . Similarly, the number of passes/iterations in 𝑗 -loop,
corresponds to column 𝑗 of 𝑈 ranging from 𝑖 + 1 to 𝑛, will be 𝑛 − (𝑖 + 1) + 1 (or simply (𝑛 − 𝑖)).

Each pass of 𝑗-loop will perform one subtraction to obtain the entry 𝑙𝑗𝑖 of 𝐿. Moreover, in each
pass of 𝑗-loop, the number of both of the multiplications and additions will be (𝑖 − 1) − 1 + 1 (or
simply (𝑖 − 1) ) in s-loop. Thus, in each pass of j-loop, the total number of
multiplications/divisions will be (𝑖 − 1) and the total number of additions/subtractions will be
(1 + 𝑖 − 1) or (𝑖). Similarly, in each pass of 𝑗-loop, to obtain the entry 𝑢𝑖𝑗 of 𝑈, the total number of
both of the multiplications and additions will be (𝑖 − 1) + 1 (or simply (𝑖)).

As there are (𝑛 − 𝑖 + 1) passes of 𝑗-loop in each pass of 𝑖 -loop, therefore there will be
(𝑛 − 𝑖 + 1) × (𝑖 − 1) multiplications/divisions and (𝑛 − 𝑖 + 1) × (𝑖) additions/subtractions in
each pass of 𝑖-loop for the formulation of column 𝑖 of 𝐿. Similarly, in each pass of 𝑖-loop, there will
be (𝑛 − 𝑖) × (𝑖) multiplications/divisions and (𝑛 − 𝑖) × (𝑖) additions/subtractions in each pass of
𝑖-loop for the formulation of row 𝑖 of 𝑈.

Hence, the total number of multiplications/divisions in 𝑛 passes of 𝑖-loop for the formulation of
the lower triangular matric 𝐿 will be

𝑛 𝑛

∑(𝑛 − 𝑖 + 1)(𝑖 − 1) = ∑(𝑛 + 1 − 𝑖)(𝑖 − 1)

𝑖=1 𝑖=1

𝑛 𝑛
2
= ∑[(𝑛 + 1)𝑖 − (𝑛 + 1) − 𝑖 + 𝑖] = ∑[(𝑛 + 2)𝑖 − 𝑖 2 − (𝑛 + 1)]
𝑖=1 𝑖=1

𝑛 𝑛 𝑛
2
(𝑛 + 2) ∑ 𝑖 − ∑ 𝑖 − (𝑛 + 1) ∑ 1
=
𝑖=1 𝑖=1 𝑖=1

𝑛(𝑛 + 1) 𝑛(𝑛 + 1)(2𝑛 + 1)

= (𝑛 + 2) [ ]−[ ] − (𝑛 + 1)𝑛
2 6

𝑛 𝑛 1 𝑛 1
= 𝑛(𝑛 + 1) [ + 1 − − − 1] = (𝑛2 + 𝑛) [ − ]
2 3 6 6 6

𝑛3 𝑛 𝑛3
= − = + 𝒪(𝑛)
6 6 6

Similarly, the total number of additions/subtractions in 𝑛 passes of 𝑖-loop for the formulation of
the lower triangular matric 𝐿 will be
98 Simplified Numerical Analysis

𝑛
𝑛3 𝑛 𝑛3
∑(𝑛 − 𝑖 + 1)(𝑖) = + = + 𝒪(𝑛)
6 6 6
𝑖=1

Moreover, the total number of multiplications/divisions and additions/subtractions in 𝑛 passes of

𝑖-loop for the formulation of the unit upper triangular matric 𝑈 will be

𝑛 𝑛 𝑛

∑(𝑛 − 𝑖)(𝑖) = ∑(𝑛 − 𝑖)(𝑖) = ∑(𝑛𝑖 − 𝑖 2 )

𝑖=1 𝑖=1 𝑖=1

𝑛 𝑛
𝑛(𝑛 + 1) 𝑛(𝑛 + 1)(2𝑛 + 1)
= 𝑛 ∑ 𝑖 − ∑ 𝑖2 = 𝑛[ ]−[ ]
2 6
𝑖=1 𝑖=1

𝑛 𝑛 1 𝑛 1
= 𝑛(𝑛 + 1) [ − − ] = (𝑛2 + 𝑛) [ − ]
2 3 6 6 6

𝑛3 𝑛 𝑛3
= − = + 𝒪(𝑛)
6 6 6

The summary of the operation count of the 𝐿𝑈-factorization is given as:

Operations
Total flops
Multiplication/Division Addition/Subtraction

Lower Triangular 𝑛3 𝑛 𝑛3 𝑛 𝑛3
Matrix 𝐿 − +
6 6 6 6 3

Upper Triangular 𝑛3 𝑛 𝑛3 𝑛 𝑛3 𝑛
Matrix 𝑈 − − −
6 6 6 6 3 3

𝑛3 𝑛 𝑛3 2𝑛3 𝑛
𝐿𝑈-factorization − −
3 3 3 3 3

(2) The forward substitution phase occurs after the formulation of 𝐿 and 𝑈 as factors of the
coefficient matrix for solving the lower triangular system. This phase contains two nested loops.
The first loop, say 𝑖-loop (which ranges from 𝑖 = 2 to 𝑛), corresponds to 𝑛 − 1 of the components
of the intermediate vector 𝑌. The second loop, say 𝑗-loop (which ranges from 𝑗 = 1 to 𝑖 − 1),
corresponds to the columns before the diagonal elements.
Direct Linear Solvers 99

Each pass of 𝑖-loop will perform one subtraction and one division to obtain the value of 𝑦𝑖 .
Moreover, in each pass of 𝑖-loop, the number of both of the multiplications and additions will be
(𝑖 − 1) − 1 + 1 (or simply (𝑖 − 1)) in 𝑗-loop. Thus, in each pass of 𝑖-loop, the total number of both
of the multiplications/divisions and additions/subtractions will be (1 + 𝑖 − 1) or simply (𝑖).

Hence, the total number of the multiplications/divisions in the forward substitution phase will be

𝑛 𝑛
𝑛(𝑛 + 1)
1 + ∑(𝑖) = 1 + ∑𝑖 = 1+[ − 1]
2
𝑖=2 𝑖=2

𝑛2 𝑛 𝑛2 𝑛 𝑛2
= + = + = + 𝒪(𝑛)
2 2 2 2 2

Similarly, the total number of the additions/subtractions in the forward substation phase will be

𝑛
𝑛2 𝑛 𝑛2
∑(𝑖) = + −1 = + 𝒪(𝑛)
𝑖=2
2 2 2

The summary of the operation count of the Unit Lower triangular system 𝐿𝑌 = 𝐵 is given as:

Operations
Total flops
Multiplication/Division Addition/Subtraction
Lower
𝑛2 𝑛 𝑛2 𝑛
triangular system + + −1 𝑛2 + 𝑛 − 1
𝐿𝑌 = 𝐵 2 2 2 2

(3) The back substitution phase occurs after the solution of the lower triangular system. This
phase contains two nested loops. The first loop, say 𝑖-loop (which ranges from 𝑖 = 𝑛 − 1 to 1),
corresponds to 𝑛 − 1 of the components of solution vector 𝑋. The second loop, say 𝑗-loop (which
ranges from 𝑗 = 𝑖 + 1 to 𝑛), corresponds to the columns after the diagonal elements.

Each pass of 𝑖-loop will perform one subtraction to obtain the value of 𝑥𝑖 . Moreover, in each pass
of 𝑖-loop, the number of both of the multiplications and additions will be 𝑛 − (𝑖 + 1) + 1 (or
simply (𝑛 − 𝑖)) in j-loop. Thus, in each pass of i-loop, the total number of multiplications/divisions
will be (𝑛 − 𝑖) and the total number of additions/subtractions will be (𝑛 − 𝑖 + 1).
100 Simplified Numerical Analysis

Hence, the total number of the multiplications/divisions in the back substitution phase will be

𝑛−1 𝑛−1 𝑛−1

(𝑛 − 1)𝑛
∑(𝑛 − 𝑖) = 𝑛∑1−∑𝑖 = 𝑛(𝑛 − 1) − [ ]
2
𝑖=1 𝑖=1 𝑖=1
𝑛2 𝑛 𝑛2 𝑛 𝑛2
= 𝑛2 − 𝑛 − + = − = + 𝒪(𝑛)
2 2 2 2 2

Similarly, the total number of the additions/subtractions in the back substation phase will be

𝑛−1
𝑛2 𝑛 𝑛2
∑(𝑛 + 1 − 𝑖) = + −1 = + 𝒪(𝑛)
𝑖=1
2 2 2

The summary of the operation count of the Upper triangular system 𝑈𝑋 = 𝑌 is given as:

Operations
Total flops
Multiplications/Divisions Additions/Subtractions

Upper triangular 𝑛2 𝑛 𝑛2 𝑛
− + −1 𝑛2 − 1
system 𝑈𝑋 = 𝑌 2 2 2 2

∎∎∎
Direct Linear Solvers 101

Chapter Summary

• A system of linear equations (simply called as linear system) is a set or collection of two or more
linear equations with the same set of variables whose simultaneous solution satisfies all the equations.
Precisely, a linear system can be referred to as a set of simultaneous linear algebraic equations.

• If 𝑚 > 𝑛, where 𝑚 is the number of equations and 𝑛 is the number of unknowns, then the linear system
is called over-determined. If 𝑚 < 𝑛, then the linear system is called under-determined.

• ̅), and non-homogeneous

A linear system 𝐴𝑋 = 𝐵 is called homogenous if 𝐵 is a zero vector (i.e., 𝐵 = 𝟎
or inhomogeneous if 𝐵 ≠ 𝟎.̅

• A non-homogeneous linear system 𝐴𝑋 = 𝐵 is called consistent if it has a unique solution or infinitely

many solutions, and it is called inconsistent if it has no solution.

• If 𝐴−1 does not exist, then matrix 𝐴 is called singular or non-invertible. If 𝐴−1 exists then 𝐴 is called
non-singular matrix and is invertible.

• If det(𝐴) = 0, then 𝐴−1 does not exist and the system 𝐴𝑋 = 𝐵 does not have a unique solution; the
system either has no solution or infinitely many solutions.

• Although the steps of the algorithms for the solution of a linear system are elementary in nature, there
might be certain pitfalls. This raises the need of skillful selection and use of an appropriate algorithm for
obtaining the solution.

• In general, methods for the solution of linear systems (also called linear solvers) are evaluated based
on their accuracy, speed of convergence, and computer resource requirements (CPU-requirements,
memory requirements).

• A linear equation in two variables, say 𝑥 and 𝑦, represents a line in 𝑥𝑦-plane. If there exists a unique
solution of the system then it is the point where the two lines intersect.

• A linear equation in three variables, say 𝑥, 𝑦, and 𝑧, represents a plane in 𝑥𝑦𝑧-space. If there exists a
unique solution of such a system then it is the point where the three planes intersect.

• There are two broad categories of methods to solve linear systems: the direct (also called exact)
methods and iterative methods. The prominent features of these two categories can be found in
Question 5 (Section 6.1).

• An 𝑛 × 𝑛 square matrix 𝑈 = (𝑢𝑖𝑗 ) is called the upper triangular matrix if 𝑢𝑖𝑗 = 0 whenever 𝑖 > 𝑗. A
linear system 𝑈𝑋 = 𝑌 is said to be upper triangular system if it's coefficient matrix is an upper
triangular one. It has a unique solution if no diagonal element is zero (i.e., |𝑢𝑖𝑖 | ≠ 0, for 𝑖 = 1,2, ⋯ , 𝑛),
otherwise it has either no solution or infinitely many solutions. If there is a unique solution of an upper
102 Simplified Numerical Analysis

triangular system then the solution can easily be obtained by a so-called back substitution process. In
analogy, the said propositions also hold for a lower triangular matrix 𝐿 = (𝑙𝑖𝑗 ) for which 𝑙𝑖𝑗 = 0
whenever 𝑖 < 𝑗. The solution of a lower-triangular system can be obtained by a similar so-called
forward substitution process.

• To solve a linear system 𝐴𝑋 = 𝐵, the Gaussian Elimination method aims at obtaining an upper
triangular system 𝑈𝑋 = 𝑌, equivalent to 𝐴𝑋 = 𝐵. This process may be termed as forward elimination.
The upper triangular system can then be solved by back substitution.

• To guard against the pitfalls of the Gaussian Elimination method, the process of pivoting is performed
while using the method. The pivoting could be any of partial, scaled or complete.

• Pivoting refers to the interchanging of two rows of the augmented matrix so that the diagonal
coefficient (to be used as the pivot element) is of greatest magnitude among the possible ones for the
row under consideration.

• Pivoting must be performed if the main diagonal coefficient is zero (to make the triangular system non-
singular). Pivoting should be performed if the magnitude of the main diagonal element is a smaller one
(to prevent the propagation of the round-off error).

• The Gauss-Jordan method is a variant of the Gaussian Elimination method. It is based on the same
elementary row operations; however, it eliminates all the elements below as well as above the pivot
element (in the same column). Thus it does not produce an upper-triangular system for back-
substitution; rather it obtains a diagonal matrix in which the solution vector is almost readily available.

• The 𝐿𝑈 Factorization or 𝐿𝑈 Decomposition method is another direct solver. A concise description of this
method (and its variants) can be found in Question 12 (Section 6.5).

• The operation count analysis of an algorithm usually refers to the counting of the arithmetic operations
involved. This is useful in determining the execution time required by the algorithm. For numerical
computations, the operation count analysis is mostly considered as the counting of the floating-point
operations (simply called as flops) involved in the algorithm.

• The additions/subtractions are considered to be requiring less CPU-time (being lighter operations) as
compared to the multiplications/divisions. Therefore, it might be appropriate to count the two types of
operations separately for the operation count analysis.
∎∎∎
Direct Linear Solvers 103

Chapter Exercises

Exercise 01: Solve the following system using the Gaussian Elimination method with back substitution.
2𝑥1 − 3𝑥2 + 𝑥3 = −1
4𝑥1 + 4𝑥2 − 3𝑥3 = 3
−2𝑥1 + 3𝑥2 + 𝑥3 = 7

Exercise 02: Solve the following system using the Gaussian Elimination method with partial pivoting.
𝑥1 + 𝑥2 + 𝑥3 = 6
3𝑥1 + 3𝑥2 + 𝑥3 = 12
2𝑥1 + 𝑥2 + 5𝑥3 = 20

Exercise 03: Solve the following system using the Gaussian Elimination method with partial pivoting and
three-digit rounding arithmetic.
2.5𝑥1 − 3𝑥2 + 4.6𝑥3 = −1.05
−3.5𝑥1 + 2.6𝑥2 + 1.5𝑥3 = −14.46
−6.5𝑥1 + −3.5𝑥2 + 7.3𝑥3 = −17.735

Exercise 04: Solve the following system using the Gaussian Elimination method with scaled partial pivoting.
𝑥1 + 𝑥2 − 2𝑥3 = 3
4𝑥1 − 2𝑥2 + 𝑥3 = 5
3𝑥1 − 𝑥2 + 3𝑥3 = 8

Exercise 05: Solve the following system using the Gaussian Elimination method with scaled partial pivoting
and four-digit rounding arithmetic.
3.03𝑥1 − 12.1𝑥2 + 14𝑥3 = −119
−3.03𝑥1 + 12.1𝑥2 − 7𝑥3 = 120
6.11𝑥1 − 14.2𝑥2 + 21𝑥3 = −139

Exercise 06: Solve the following system using the Gaussian Elimination method with complete pivoting.
𝑥1 + 2𝑥2 + 2𝑥3 = 1
2𝑥1 + 6𝑥2 + 10𝑥3 = −2
3𝑥1 + 14𝑥2 + 28𝑥3 = −11

Exercise 07: Solve the following system using the Gaussian Elimination method with complete pivoting and
three-digit rounding arithmetic.

1.012𝑥1 − 2.132𝑥2 + 3.104𝑥3 = 1.984

−2.132𝑥1 + 4.096𝑥2 − 7.013𝑥3 = −5.049
3.104𝑥1 − 7.013𝑥2 + 0.014𝑥3 = −3.895
104 Simplified Numerical Analysis

Exercise 08: Solve the following system using the Gauss-Jordan method
𝑥1 + 2𝑥2 + 𝑥3 = 6
2𝑥1 + 3𝑥2 + 4𝑥3 = 12
4𝑥1 + 3𝑥2 + 2𝑥3 = 12

Exercise 09: Solve the following system using the Gauss-Jordan method and three-digit rounding arithmetic.

0.125𝑥1 + 0.201𝑥2 + 0.401𝑥3 = 2.306

0.375𝑥1 + 0.501𝑥2 + 0.601𝑥3 = 4.806
0.501𝑥1 + 0.301𝑥2 + 0.001𝑥3 = 2.91

Exercise 10: Solve the following linear system 𝐴𝑋 = 𝐵 using the Doolittle’s method.
𝑥1 + 𝑥2 + 𝑥3 = 3
2𝑥1 − 𝑥2 + 2𝑥3 = 16
3𝑥1 + 𝑥2 + 𝑥3 = −3

Exercise 11: Solve the following linear system 𝐴𝑋 = 𝐵 using the Doolittle’s method.
𝑥1 + 𝑥2 + 2𝑥3 + 2𝑥4 = 9
2𝑥1 + 4𝑥2 + 7𝑥3 + 3𝑥4 = 25
−𝑥1 − 5𝑥2 − 6𝑥3 + 2𝑥4 = −17
𝑥1 − 𝑥2 + 3𝑥3 + 8𝑥4 = 15

Exercise 12: Solve the following linear system 𝐴𝑋 = 𝐵 using the Crout’s method.

8𝑥1 + 𝑥2 − 𝑥3 = 8
2𝑥1 + 𝑥2 + 9𝑥3 = 12
𝑥1 − 7𝑥2 + 2𝑥3 = −4

Exercise 13: Solve the following linear system 𝐴𝑋 = 𝐵 using the Crout’s method.

𝑥1 + 𝑥2 + 0𝑥3 + 3𝑥4 = 9
2𝑥1 + 𝑥2 − 𝑥3 + 𝑥4 = 5
3𝑥1 − 𝑥2 + 𝑥3 + 2𝑥4 = 6
−𝑥1 + 2𝑥2 + 3𝑥3 − 𝑥4 = 4

Exercise 14: Solve the following linear system 𝐴𝑋 = 𝐵 using the Cholesky’s method.

2𝑥1 + 3𝑥2 + 4𝑥3 = 1

3𝑥1 + 8𝑥2 + 5𝑥3 = 6
4𝑥1 + 5𝑥2 + 10𝑥3 = −1

Exercise 15: Solve the given linear system 𝐴𝑋 = 𝐵 using the Cholesky’s method

4𝑥1 + 𝑥2 + 𝑥3 + 𝑥4 = 9
𝑥1 + 3𝑥2 − 𝑥3 + 𝑥4 = 4
𝑥1 − 𝑥2 + 2𝑥3 + 0𝑥3 = 4
𝑥1 + 𝑥2 + 0𝑥3 + 2𝑥4 = 6
Direct Linear Solvers 105

Exercise 16: The upward velocity of a rocket at three different times after its launching are given as follows:

Time, 𝑡 in (𝑠) Velocity, 𝑣 in (𝑚/𝑠)

6 115.7
9 182.5
12 295.6

The velocity data is approximated by a polynomial as

𝑣(𝑡) = 𝑎1 𝑡 2 + 𝑎2 𝑡 + 𝑎3 , 5 ≤ 𝑡 ≤ 12

Thus, the coefficients 𝑎1 , 𝑎2 and 𝑎3 for the above expression are given by

36 6 1 𝑎1 115.7
[ 81 9 1] [𝑎2 ] = [182.5]
144 12 1 𝑎3 295.6

Find the values of 𝑎1 ,𝑎2 and 𝑎3 using a linear solver. Then, calculate the velocity at 𝑡 = 7, 8, 10, and 11
seconds.

Exercise 17: A factory produces three products, say Prod1, Prod2, and Prod3, by using three kinds of raw
materials, say Raw1, Raw2, and Raw3. The units of each of the raw materials needed to produce one unit of
each of the products are shown the table below.

Sectors Raw1 Raw2 Raw3

Prod1 5 3 1
Prod2 4 4 3
Prod3 2 1 3

If 335 units of Raw1, 532 units of Raw2, and 440 units of Raw3 are available, then how much each of the
three products can be produced.

Hint for the Solution:

Assume that 𝑥1 , 𝑥2 and 𝑥3 represent the quantities of the products: Prod1, Prod2, and Prod3, respectively.
The problem can be represented by a linear system whose solution would provide the required values.

5𝑥1 + 4𝑥2 + 2𝑥3 = 335

3𝑥1 + 4𝑥2 + 𝑥3 = 532
𝑥1 + 3𝑥2 + 3𝑥3 = 440

Exercise 18: Assume that the economy of a country depends on the three sectors: Food, Cloth, and House.
The production of one unit of each of these needs certain units of each of these sectors, as shown in the
following table:
106 Simplified Numerical Analysis

Food Units Cloth Units House Units

Sectors
Needed Needed Needed
Food 0.45 0.18 0.15
Cloth 0.25 0.27 0.07
House 0.30 0.40 0.45

The consumer demand is as in the table below:

Sector worth in billion rupees

Food 220
Cloth 185
House 550

For satisfying the above demands, what total output is required from each of the sectors.
Hint for the Solution in MATLAB: Assume that 𝑥1 , 𝑥2 and 𝑥3 represent the total outputs in units from Food,
Cloth and House sectors, respectively. The problem can be represented by a linear system whose solution
would provide the required values.

Exercise 19: A bakery produces three products: Cake, Pastry, and Muffin. It uses three kinds of materials:
Flour, Milk, and Sugar. The units of each of the raw materials needed to produce one unit of each of the
bakery products are shown the table below.

Product -> Cake Pastry Muffin

Flour 6 5 3
Milk 4 5 2
Sugar 2 3 3

If 347 units of Flour, 604 units of Milk, and 502 units of Sugar are available, then how much each of the three
products can be produced.

Exercise 20: Pivoting is necessary with the Gaussian elimination if

(A) the coefficient matrix is singular (B) the linear system is homogenous

(C) the linear system is ill conditioned (D) None of above

Exercise 21: Cholesky decomposition for a linear system is not possible, if

(A) the linear system is ill conditioned (B) the linear system is homogenous

(C) the coefficient matrix is asymmetric (D) None of above

∎∎∎
 Chapter 7
Iterative Linear Solvers

Corridor I: BASICS
Let’s plan it

7.1 Vector Norms and Distances

7.2 Convergence Criteria for Linear Solvers
7.3 Basic Methods
7.3.1 The Jacobi Method
7.3.2 The Gauss-Seidel Method
7.3.3 The SOR Method

To unleash the topics of this Corridor, please delve into the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

Fig. (7.4): Explanation of the different types of distances between the two vectors.

107
108 Simplified Numerical Analysis

Corridor II: ANALYSIS

Let’s think deep

7.4 Matrix Norms and Conditioning

7.5 Iteration Matrix and Matrix Form of a Solver

To unleash the topics of this Corridor, please delve into the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

Corridor III: PROGRAMMING ARCADE

Let’s do it

7.6 Algorithms and Implementations

`The Jacobi Method
Modification in the Jacobi Method's algorithm for the Gauss-Seidel Method
Modification in the Jacobi Method's algorithm for the SOR Method

To cross-check the results/output of the computer programs you would execute, please delve into
the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

7.6 Algorithms and Implementations

Iterative Linear Solvers 109

Question 22: Write down an algorithm (pseudo code) to solve a linear system using the Jacobi method.

Algorithm: To solve 𝐴𝑋 = 𝐵, given an initial approximation 𝑋 (0) .

𝒏: an integer as the number of equations and unknowns

𝑨 = (𝒂𝒊𝒋 ), 1 ≤ 𝑖, 𝑗 ≤ 𝒏: a real valued square matrix as the coefficient matrix
𝑩 = [𝒃𝟏 , 𝒃𝟐 , ⋯ , 𝒃𝒏 ]𝑻 : a real valued vector as the vector of right hand side constants
𝐈𝐍𝐏𝐔𝐓𝐒:
𝑿 = [𝒙𝟏 , 𝒙𝟐 , ⋯ , 𝒙𝒏 ]𝑻 : a real valued vector (having initial approximation, 𝑋 (0) )
𝑻𝑶𝑳: a real value as the error tolerance
{𝑵: an integer as the maximum number of iterations

𝑿 = [𝒙𝟏 , 𝒙𝟐 , ⋯ , 𝒙𝒏 ]𝑻 : a real valued vector as the approximate solution

𝐎𝐔𝐓𝐏𝐔𝐓: {
(either on convergence, or on completing 𝑵 iterations − which ever happens first)

Step 1 Receive the inputs as stated above

Step 2 for 𝒌 = 𝟏, 𝟐, 𝟑, ⋯ , 𝑵 perform steps 3-6

𝑿𝑷 = [𝒙𝒑𝟏 , 𝒙𝒑𝟐 , ⋯ , 𝒙𝒑𝒏 ]𝑻 is to keep a copy of present

Step 3 for 𝑖 = 1, 2, ⋯ , 𝒏 Set 𝒙𝒑𝒊 = 𝒙𝒊 {
approximation 𝑿, because 𝑿 is going to be updated

Step 4 for 𝑖 = 1, 2, ⋯ , 𝒏 (compute the components of solution vector 𝑿)

𝑠𝑢𝑚 = 0
for 𝑗 = 1, 2, ⋯ , 𝒏 𝑛
(𝑘) 1 (𝑘−1)
𝑖𝑓 (𝒋 ≠ 𝒊) 𝑠𝑢𝑚 = 𝑠𝑢𝑚 + 𝒂𝒊𝒋 × 𝑿𝑷𝒋 𝑥𝑖 = 𝑏 − ∑ 𝑎𝑖𝑗 𝑥𝑗
𝑎𝑖𝑖 𝑖
[𝒃𝒊 − 𝑠𝑢𝑚] 𝑗=1
𝒙𝒊 = [ 𝑗≠𝑖 ])
𝒂𝒊𝒊 } (

Step 5 Compute 𝒆𝒓𝒓 = ‖𝑿 − 𝑿𝑷‖ (or 𝒆𝒓𝒓 = ‖𝑿 − 𝑿𝑷‖/‖𝑿‖) Here ‖∙‖ is any suitable norm.

Step 6
This means that the consecutive
if (𝒆𝒓𝒓 < 𝑻𝑶𝑳 )then
} approximations are nearly the same.
Exit/Break the loop
Therefore, stop iterations.

end for loop of Step 2 (Go to Step 3)

Step 7 Print the output: 𝑿 = [𝒙𝟏 , 𝒙𝟐 , … , 𝒙𝒏 ]𝑻

if (𝒆𝒓𝒓 < 𝑻𝑶𝑳) OUTPUT (‘The desired accuracy achieved; Solution converged.’)
else OUTPUT (‘The number of iterations exceeded the maximum limit.’)

STOP.
110 Simplified Numerical Analysis

Question 23: What modification a programmer needs to make in the algorithm (pseudo code) of
the Jacobi method (as given in the answer of Question 22) to convert it into the Gauss-Seidel
method for solving a linear system.

The algorithm (pseudo code) of the Jacobi method (as given in the answer of Question 22) can be
converted into the algorithm of the Gauss-Seidel method simply by replacing its Step 4 with the
following:

Step 4 for 𝑖 = 1, 2, ⋯ , 𝒏 (compute the components of solution vector 𝑿)

𝑠𝑢𝑚 = 0 𝑖−1 𝑛
for 𝑗 = 1, 2, ⋯ , 𝒏 (𝑘) (𝑘−1)
𝑠𝑢𝑚 = ∑ 𝑎𝑖𝑗 𝑥𝑗 + ∑ 𝑎𝑖𝑗 𝑥𝑗
𝑖𝑓 (𝒋 ≠ 𝒊) 𝑠𝑢𝑚 = 𝑠𝑢𝑚 + 𝒂𝒊𝒋 × 𝒙𝒋 𝑗=1 𝑗=𝑖+1
[𝒃𝒊 − 𝑠𝑢𝑚] (𝑘) 1
𝒙𝒊 = 𝑥 = [𝑏 − 𝑠𝑢𝑚]
𝒂𝒊𝒊 } ( 𝑖 𝑎𝑖𝑖 𝑖 )

Question 24: What modification a programmer needs to make in the algorithm (pseudo code) of
the Jacobi method (as given in the answer of Question 22) to convert it into the Gauss-Seidel
method with over-relaxation (i.e., the SOR method) for solving a linear system.

The algorithm (pseudo code) of the Jacobi method (as given in the answer of Question 22) can be
converted into the algorithm of the SOR method simply by taking one more input:

𝑾𝑭 = 1.3: a real value as the over − relaxation / weighting factor

And then replacing Step 4 with the following:

Step 4
for 𝑖 = 1, 2, ⋯ , 𝒏 (compute the components of solution vector 𝑿)

𝑠𝑢𝑚 = 0 𝑖−1 𝑛
for 𝑗 = 1, 2, ⋯ , 𝒏 (𝑘) (𝑘−1)
𝑠𝑢𝑚 = ∑ 𝑎𝑖𝑗 𝑥𝑗 + ∑ 𝑎𝑖𝑗 𝑥𝑗
𝑖𝑓 (𝒋 ≠ 𝒊) 𝑠𝑢𝑚 = 𝑠𝑢𝑚 + 𝒂𝒊𝒋 × 𝒙𝒋 𝑗=1 𝑗=𝑖+1
[𝒃𝒊 − 𝑠𝑢𝑚] (𝑘) 1
𝒙𝒊 = 𝑥 = [𝑏 − 𝑠𝑢𝑚]
𝒂𝒊𝒊 } ( 𝑖 𝑎𝑖𝑖 𝑖 )

𝒙𝒊 = 𝑾𝑭 × 𝒙𝒊 + (1 − 𝑾𝑭)𝑿𝑷𝒊 (apply over − relaxation)

∎
Iterative Linear Solvers 111

Problem 17: Write a C++ program to solve the following linear system using the Jacobi method. Take initial
approximate solution as: 𝑋 (0) = [0, 0, 0]𝑇 . The iterations of the method should stop when either the
approximation is accurate within 10−6 , or the number of iterations exceeds 200, whichever happens first.
5𝑥1 + 3𝑥2 + 2𝑥3 = 17, 3𝑥1 + 4𝑥2 − 𝑥3 = 8, −𝑥1 + 𝑥2 − 3𝑥3 = −8

1 # include <iostream>
2 # include <cmath>
3 using namespace std ;
4
5 # define n 3 // number of unknowns
6 # define TOL 0.000001 // error tolerance
7 # define N 200 // maximum number of iterations
8
9 int main ( )
10 {
11
12 int i , j ,k;
13 double a[n][n] = {{5, 3, 2}, {3, 4, -1}, {-1, 1,-3}} ; // coefficient matrix
14 double b[n] = {17, 8, -8} ; // right-hand side constants
15 double x[n] ; // the solution vector
16 double xp[n] , sum , err ;
17
18 cout<<"The Gauss-Jacobi Method for solving a system of "<<n<<" unknowns." ;
19
20 for ( i=0 ; i<n ; i++)
21 x[i] = 0.0 ; // setting initial approximation as zero vector
22
23 //----------------------- Processing Section -------------------------//
24
25 for ( k=1 ; k<=N ; k++ )
26 {
27 for ( i=0 ; i<n ; i++)
28 xp[i] = x[i] ;
29
30 for ( i=0 ; i<n ; i++)
31 { 𝑛
32 sum = 0 ; ∑ 𝑎𝑖𝑗 𝑥𝑗
(𝑘−1)

33 for ( j=0 ; j<n ; j++) 𝑗=1

34 if (j != i) sum = sum + a[i][j] * xp[j] ; 𝑗≠𝑖

35
36 x[i] = ( b[i] - sum) / a[i][i] ; (𝑘) 1
𝑥𝑖 = [𝑏 − 𝑠𝑢𝑚]
37 } 𝑎𝑖𝑖 𝑖
38
39 sum = 0.0 ;
40 for ( i=0 ; i<n ; i++) Computing
41 sum = sum + ( ( x[i] - xp[i] ) * ( x[i] - xp[i] ) ) ; 𝑙2 − 𝑛𝑜𝑟𝑚
42 err = sqrt(sum) ;
43
44 if ( err < TOL ) break ;
45 }
112 Simplified Numerical Analysis

46
47 //------------------------ Output Section ----------------------------//
48
49 cout << " The latest approximate solution vector is given by : " << endl;
50 for ( i=0 ; i<n ; i++)
51 cout << x[i] << "\t" ;
52
53 if ( err < TOL)
54 cout<< "\nThe desired accuracy achieved; Solution converged." ;
55 else
56 cout<< "\nThe number of iterations exceeded the maximum limit." ;
57
58 }
∎

Remark: Replacing xp[j] by x[j] in line 34 of the C++ program in Problem 17 would convert the
program for the Gauss-Seidel method, because it would then correspond to computing

𝑖−1 𝑛
(𝑘) (𝑘−1)
𝑠𝑢𝑚 = ∑ 𝑎𝑖𝑗 𝑥𝑗 + ∑ 𝑎𝑖𝑗 𝑥𝑗
𝑗=1 𝑗=𝑖+1

Remark: In the program of Problem 17, the code segment of lines 45-47 can be placed just before
line 45 to print the latest result on completion of each of the iterations.

∎
Problem 19: Write a C++ program to solve the following linear system using the Gauss-Seidel method with
over-relaxation (the SOR method). Take initial approximate solution as: 𝑋 (0) = [0, 0, 0]𝑇 and over-
relaxation factor as 1.2. The iterations of the method should stop when either the approximation is accurate
within 10−6 , or the number of iterations exceeds 200, whichever happens first.

5𝑥1 + 3𝑥2 + 2𝑥3 = 17, 3𝑥1 + 4𝑥2 − 𝑥3 = 8, −𝑥1 + 𝑥2 − 3𝑥3 = −8

1 # include <iostream>
2 # include <cmath>
3 using namespace std ;
4
5 # define n3 // number of unknowns
6 # define TOL 0.000001 // error tolerance
7 # define N 200 // maximum number of iterations
8 # define WF 1.2 // over-relaxation factor
9
10 int main ( )
11 {
12
Iterative Linear Solvers 113

13 int i , j ,k;
14 double a[n][n] = {{5, 3, 2}, {3, 4, -1}, {-1, 1,-3}} ; // coefficient matrix
15 double b[n] = {17, 8, -8} ; // right-hand side constants
16 double x[n] ; // the solution vector
17 double xp[n] , sum , err ;
18
19 cout<<"The Gauss-Seidel method with over-relaxation for solving a system.\n" ;
20
21 for ( i=0 ; i<n ; i++)
22 x[i] = 0.0 ; // setting initial approximation as zero vector
23
24 //----------------------- Processing Section -------------------------//
25
26 for ( k=1 ; k<=N ; k++ )
27 {
28 for ( i=0 ; i<n ; i++)
29 xp[i] = x[i] ;
30
31 for ( i=0 ; i<n ; i++)
32 { 𝑛
33 sum = 0 ; ∑ 𝑎𝑖𝑗 × (latest value of 𝑥𝑗 )
34 for ( j=0 ; j<n ; j++) 𝑗=1
35 if (j != i) sum = sum + a[i][j] * x[j] ; 𝑗≠𝑖

36
37 1
(𝑘)
38 x[i] = ( b[i] - sum) / a[i][i] ; 𝑥𝑖 = [𝑏 − 𝑠𝑢𝑚]
𝑎𝑖𝑖 𝑖
39
40
41 x[i] = WF * x[i] + (1 – WF) * xp[i] ; 𝑥𝑖 = 𝑊𝐹 × 𝑥𝑖 + (1 − 𝑊𝐹)𝑋𝑃𝑖
42
43 }
44
45 sum = 0.0 ;
46 for ( i=0 ; i<n ; i++) Computing
47 sum = sum + ( ( x[i] - xp[i] ) * ( x[i] - xp[i] ) ) ; 𝑙2 − 𝑛𝑜𝑟𝑚
48 err = sqrt(sum) ;
49
50 if ( err < TOL ) break ;
51 }
52
53 //------------------------ Output Section ----------------------------//
54
55 cout << " The latest approximate solution vector is given by : " << endl;
56 for ( i=0 ; i<n ; i++)
57 cout << x[i] << "\t" ;
58
59 if ( err < TOL)
60 cout<< "\nThe desired accuracy achieved; Solution converged." ;
61 else
62 cout<< "\nThe number of iterations exceeded the maximum limit." ;
114 Simplified Numerical Analysis

63
64 }
∎

Remark: Note that setting the weighting factor of over-relaxation (WF) as 1.0 in the solutions of
Problem 19 would make the programs for the Gauss-Seidel method.

Remark: In the program of Problem 19, the code segment of lines 55-57 can be placed just before
line 51 to print the latest result on completion of each of the iterations.
∎

Remark: The C++ programs in Problem 17 and 19 can be modified to receive the linear system at
the execution time (instead of fixing in the code). For this, the lines 13-14 in the program of
Problem 17 and the lines 14-15 in the program of Problem 19 should be replaced by the following
code segment:

cout<<"\nEnter the coefficient matrix row-wise: "<<n<<" unknowns<<endl;

for ( i=0 ; i<n ; i++)
for ( j=0 ; j<n ; j++)
cin >> a[i][j] ;

cout<<"Enter the elements of constant vector B "<<endl ;

for ( i=0 ; i<n ; i++)
cin >> b[i] ;
for i = 1:n
b(i) = input('Enter the element of constant vector: ') ;
end
∎∎∎

Chapter Summary

• The norm of a vector is a real-valued function that provides a measure of “size”, “length”, or
“magnitude” of the vector. Let ℝ denotes the set of real numbers, and ℝ𝑛 denotes the space of
n-dimensional real-valued column vectors. A norm of a vector on ℝ𝑛 is a function, ‖∙‖ ∶ ℝ𝑛 →
ℝ, with the following properties,
1. ‖𝑋‖ ≥ 0, for all 𝑋 ∈ ℝ𝑛
2. ‖𝑋‖ = 0, if and only if 𝑋 = 𝟎 in ℝ𝑛
3. ‖α𝑋‖ = |α|‖𝑋‖, for all α ∈ ℝ and 𝑋 ∈ ℝ𝑛
4. ‖𝑋 + 𝑌‖ ≤ ‖𝑋‖ + ‖𝑌‖, for all 𝑋, 𝑌 ∈ ℝ𝑛
Iterative Linear Solvers 115

• The vector norm definitions, as well as the concerning illustrations, can be found in Question
01 (Section 7.1).

• The norm of a vector gives a measure for the distance between an arbitrary vector and the
zero vector, just as the absolute value of a real number is its distance from 0.

• The distance between two vectors is defined as the norm of the “difference vector” of the
two vectors, just as the distance between two real numbers is the absolute value of their
difference. The definitions of different vector distances, as well as the concerning
illustrations, can be found in Question 02 (Section 7.1).

• To determine the convergence of an iterative solution, the norm of the difference vector of
every two consecutive approximations is ensured to be smaller than a pre-specified error
tolerance 𝜏, i.e.,

‖𝑋 (𝑘) − 𝑋 (𝑘−1) ‖ < 𝜏

• A square matrix, say 𝐴 = (𝑎𝑖𝑗 )𝑛×𝑛 , is said to be diagonally dominant if, for 𝑖 = 1, 2, ⋯ , 𝑛
𝑛

|𝑎𝑖𝑖 | ≥ ∑|𝑎𝑖𝑗 | ,
𝑗=1
𝑗≠𝑖

• A linear system is said to be diagonally dominant if its coefficient matrix is diagonally

dominant (i.e., the magnitude of the diagonal entry in a row is greater than or equal to the
sum of the magnitudes of all other entries in that row).

• If “≥” is replaced by “>” in the above equation, then 𝐴 is said to be strictly diagonally
dominant. A strictly diagonally dominant matrix is always non-singular.

• If a linear system is not diagonally dominant, then a rearrangement of its rows might make it
diagonally dominant.

• The Gauss-Jacobi, Gauss-Seidel, and SOR methods must converge if the linear system to be
solved is diagonally dominant.

• Suppose that 𝐴𝑋 = 𝐵 is a 𝑛 × 𝑛 linear system to be solved such that 𝐴 = (𝑎𝑖𝑗 )𝑛×𝑛 is the
coefficient matrix, 𝐵 = (𝑏𝑖 )𝑛×1 is the vector of right-hand side constants, and 𝑋 = (𝑥𝑖 )𝑛×1 is
the vector of unknowns.
➢ The Jacobi method can be written in a compact form as

𝑛
(𝑘) 1 (𝑘−1)
𝑥𝑖 = 𝑏 − ∑ 𝑎𝑖𝑗 𝑥𝑗 , for 𝑖 = 1, 2, ⋯ , 𝑛
𝑎𝑖𝑖 𝑖
𝑗=1
[ 𝑗≠𝑖 ]
116 Simplified Numerical Analysis

➢ The Gauss-Seidel method can be written in a compact form as

𝑖−1 𝑛
(𝑘) 1 (𝑘) (𝑘−1)
𝑥𝑖 = [𝑏 − (∑ 𝑎𝑖𝑗 𝑥𝑗 + ∑ 𝑎𝑖𝑗 𝑥𝑗 )] , for 𝑖 = 1, 2, ⋯ , 𝑛
𝑎𝑖𝑖 𝑖
𝑗=1 𝑗=𝑖+1

➢ The successive over-relaxation (SOR) method can be written in a compact form as

𝑖−1 𝑛
(𝑘) 1 (𝑘) (𝑘−1)
𝑥̿𝑖 = [𝑏 − (∑ 𝑎𝑖𝑗 𝑥𝑗 + ∑ 𝑎𝑖𝑗 𝑥𝑗 )] , for 𝑖 = 1, 2, ⋯ , 𝑛
𝑎𝑖𝑖 𝑖
𝑗=1 𝑗=𝑖+1

(𝑘) (𝑘) (𝑘−1)

𝑥𝑖 = 𝜔𝑥̿𝑖 + (1 − 𝜔)𝑥𝑖 (for 1 ≤ 𝜔 ≤ 2, usually the best is around 1.2 )
(𝑘)
Here 𝑘 = 1, 2, 3, ⋯, represents the iterations and 𝑥𝑖 represents the kth approximation of the
ith unknown. The iterative procedure is started with an initial approximation vector 𝑋 (0) =
(0) (0) (0) (0) 𝑻 ∞
[𝑥1 , 𝑥2 , 𝑥3 , ⋯ , 𝑥𝑛 ] and produces a sequence of successive approximations {𝑋 (𝑘) }𝑘=1 ,
(𝑘) (𝑘) (𝑘) (𝑘) 𝑻
such that 𝑋 (𝑘) = [𝑥1 , 𝑥2 , 𝑥3 , ⋯ , 𝑥𝑛 ] . The sequence is anticipated to refine/improve
the approximate solution gradually and ultimately converge to the exact solution vector
(theoretically). In practice, the iterations of the method are stopped when a sufficient level of
accuracy is achieved.

• The norm of a matrix is a real-valued function that provides a measure of “size”, “length”, or
“magnitude” of the matrix. Let ℝ denotes the set of real numbers, and 𝕄𝑛 denotes the set of
𝑛 × 𝑛 real-valued matrices. The norm of a matrix on 𝕄𝑛 is a function, ‖∙‖ ∶ 𝕄𝑛 → ℝ, with the
following properties,
1. ‖𝐴‖ ≥ 0, for all 𝐴 ∈ 𝕄𝑛
2. ‖𝐴‖ = 0, if and only if 𝐴 = 𝟎 in 𝕄𝑛
3. ‖α𝐴‖ = |α|‖𝐴‖, for all α ∈ ℝ and 𝐴 ∈ 𝕄𝑛
4. ‖𝐴 + 𝐵‖ ≤ ‖𝐴‖ + ‖𝐵‖, for all 𝐴, 𝐵 ∈ 𝕄𝑛
5. ‖𝐴𝐵‖ ≤ ‖𝐴‖‖𝐵‖, for all 𝐴, 𝐵 ∈ 𝕄𝑛

• The matrix norm definitions can be found in Question 11 (Section 7.4).

• The distance between two matrices 𝐴 and 𝐵 with respect to a certain norm ‖∙‖ is defined as
the norm of the “difference matrix” of the two matrices, i.e., ‖𝐴 − 𝐵‖.

• The condition number of a non-singular matrix 𝐴 with respect to a matrix norm ‖∙‖ is
defined as
𝒦(𝐴) = ‖𝐴‖‖𝐴−1 ‖, (and 𝒦(𝐴) ≥ 1)

• The condition number of a linear system is the condition number of its coefficient matrix.
Iterative Linear Solvers 117

• A computational problem is called ill-conditioned (or ill-posed) if small changes in the data
(the input) cause large changes in the solution (the output). On the other hand, a problem is
called well-conditioned (or well-posed) if small changes in the data cause only small changes
in the solution.

• The main issue while solving an ill-conditioned problem is that the round-off errors can cause
production of wide range worthless solutions (which appear to be original ones because they
approximately satisfy the given problem). Therefore, minimizing the round-off errors
becomes more relevant for the ill-conditioned problems.

• If 𝐴𝑋 = 𝐵 is an ill-conditioned linear system then the solution of its perturbed system (the
one which is obtained by making small changes in the original system, either through small
changes in 𝐴, or in 𝐵) is much different from that of the original linear system. In that case,
the matrix 𝐴 is said to be an ill-conditioned matrix. The determinant of an ill-conditioned
matrix 𝐴 is usually close to zero (NOT the zero). Remind that if the determinant is exactly
zero then a relevant linear system 𝐴𝑋 = 𝐵 has either no solution, or an infinite number of
solutions.

• There is no strict line between the well-conditioning and ill-conditioning of a system, as these
concepts are qualitative. A linear system whose condition number (i.e., the condition number
of its coefficient matrix) is close to 1 is well-conditioned, whereas a condition number
significantly larger than 1 indicates that the linear system is ill-conditioned. If the condition
number is below 100, it is usually not a reason for concern. However, a condition number of
more than 100 calls for caution. It may be noted that a coefficient matrix, having magnitudes
of diagonal elements larger than that of other elements in each of the rows, indicates well-
conditioning of the linear system.

• In general, an iterative linear solver involves a process that converts an 𝑛 × 𝑛 system 𝐴𝑋 = 𝐵

into an equivalent system of the form 𝑋 = 𝑇𝑋 + 𝐶 for some fixed matrix 𝑇 and vector 𝐶. After
the initial vector 𝑋 (0) is selected, the sequence of approximate solution vectors, 𝑋 (1) , 𝑋 (2) ,
𝑋 (3) , ⋯, is generated by computing

𝑋 (𝑘) = 𝑇𝑋 (𝑘−1) + 𝐶, for 𝑘 = 1, 2, 3, ⋯

The matrix 𝑇 is called the iteration matrix of the iterative method, and the relation is called
the matrix form of the iterative method.

• The iterative linear solvers for which the iteration matrix remains unchanged (or fixed)
during the iterative process are said to be stationary solvers, whereas the iterative linear
solvers for which the iteration matrix changes from iteration to iteration are referred to as
non-stationary solvers.

• Examples of stationary solvers include simple methods like the Jacobi, Gauss-Seidel, and SOR
methods. Examples of the non-stationary solvers include more sophisticated methods like the
118 Simplified Numerical Analysis

Krylov subspace methods: especially, Conjugate Gradient (CG) methods, Minimal Residual
methods (especially GMRES), and many more.
∎∎∎

Chapter Exercises

Exercise 01: Workout first three iterations of (𝑖) the Jacobi method, (𝑖𝑖) the Gauss-Seidel method, and (𝑖𝑖𝑖)
the Gauss-Seidel method with successive over-relaxation factor 𝜔 = 1.2 and 𝜔 = 1.5 for solving the
following systems for any initial approximation. Perform computations with a precision of 4 decimal digits,
at least. Assume the error tolerance as 0.0001.

(a)
𝑥1 − 0.25𝑥2 − 0.25𝑥3 = 9
−0.25𝑥1 + 𝑥2 − 0.25𝑥3 = 4
−0.25𝑥1 − 0.25𝑥2 + 𝑥3 = −1
(b)
4𝑥1 + 𝑥2 − 𝑥3 + 𝑥4 = 2.5
𝑥1 + 4𝑥2 − 𝑥3 − 𝑥4 = 0.5
−𝑥1 − 𝑥2 + 5𝑥3 + 𝑥4 = 5
𝑥1 − 𝑥2 + 𝑥3 + 3𝑥4 = 4
(c)
2𝑥1 − 𝑥2 + 𝑥3 = −3
2𝑥1 + 4𝑥2 + 2𝑥3 = 8
−𝑥1 − 𝑥2 + 2𝑥3 = 1
(d)
𝑥1 − 0.25𝑥2 − 0.25𝑥3 + 0𝑥4 = 11
−0.25𝑥1 + 𝑥2 + 0𝑥3 − 0.25𝑥4 = 7
−0.25𝑥1 + 0𝑥2 + 𝑥3 − 0.25𝑥4 = 3
0𝑥1 − 0.25𝑥2 − 0.25𝑥3 + 𝑥4 = −1
(e)
0.2𝑥1 + 0.3𝑥2 + 0𝑥3 = 0.1
0.3𝑥1 + 0𝑥2 + 0.2𝑥3 = 0.1
0𝑥1 + 0.2𝑥2 + 0.3𝑥3 = 0.8
(f)
8𝑥1 + 4𝑥2 + 0𝑥3 + 0𝑥4 = 10
4𝑥1 + 12𝑥2 + 2𝑥3 + 0𝑥4 = 12
0𝑥1 + 2𝑥2 + 7𝑥3 + 2.5𝑥4 = 9.25
0𝑥1 + 0𝑥2 + 2.5𝑥3 + 4.5𝑥4 = 4.75
∎∎∎
 Chapter 8
Eigenvalues and Eigenvectors

Corridor I: BASICS
Let’s plan it

8.1 Basic Definitions and Concepts

8.2 General Approach of Finding Eigenvalues and Eigenvectors
8.3 Some Numerical Methods for Eigenvalues
The Power Method
The Householder Method
The QR Factorization Method
The Sturm Method

To unleash the topics of this Corridor, please delve into the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)

Corridor II: ANALYSIS

Let’s think deep

8.4 Further Discussions

The Power Theorem
The Gerschgorin Circle Theorems
The Singular Value Decomposition (SVD)

To unleash the topics of this Corridor, please delve into the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)

119
120 Simplified Numerical Analysis

Corridor III: PROGRAMMING ARCADE

Let’s do it

8.5 Algorithms and Implementations

The Power Method

To cross-check the results/output of the computer programs you would execute, please delve into
the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

8.5 Algorithms and Implementations

Eigenvalues and Eigenvectors 121

Question 12: Write down an algorithm (pseudo code) to find dominant eigenvalue and a corresponding
eigenvector of a matrix using the Power method.

Algorithm: To approximate the dominant eigenvalue and associated eigenvector of an 𝑛 × 𝑛 matrix 𝐴, given
a nonzero normalized vector 𝑋 (i.e., having 1 as the largest component) as the initial approximation.

𝒏: an integer as the length of the vector 𝑋

𝑿 = [𝒙𝟏 , 𝒙𝟐 , ⋯ , 𝒙𝒏 ]𝑻 : a real valued vector (as a normalised initial approximation)
𝐈𝐍𝐏𝐔𝐓𝐒: 𝑨 = (𝒂𝒊𝒋 ), 1 ≤ 𝑖, 𝑗 ≤ 𝒏: a real valued square matrix whose eigenvalue is to be obtained
𝑻𝑶𝑳: 𝑎 real value as the tolerance
{𝑵: an integer as the maximum number of iterations
𝑩: a real value as the approximate eigenvalue
𝐎𝐔𝐓𝐏𝐔𝐓: {
𝑿 = [𝒙𝟏 , 𝒙𝟐 , ⋯ , 𝒙𝒏 ]𝑻 : 𝑎 normalized vector as the eigenvector corresponding to 𝑩

Step 1 Receive the inputs as stated above

Step 2 for 𝒌 = 𝟏, 𝟐, 𝟑, ⋯ , 𝑵 perform steps 3-6

𝑿𝑷 = [𝒙𝒑𝟏 , 𝒙𝒑𝟐 , ⋯ , 𝒙𝒑𝒏 ]𝑻 is to keep a copy of present

Step 3 for 𝑖 = 1, 2, ⋯ , 𝒏 Set 𝒙𝒑𝒊 = 𝒙𝒊 {
approximation 𝑿, because 𝑿 is going to be updated

Step 4 (Compute the vector such that 𝑋 (𝑘) = 𝐴𝑋 (𝑘−1) )

for 𝑖 = 1, 2, ⋯ , 𝒏
sum = 0 𝑛
for 𝑗 = 1, 2, ⋯ , 𝒏 (𝑘) (𝑘−1)
(𝑥𝑖 = ∑ 𝑎𝑖𝑗 𝑥𝑗 )
sum = sum + 𝒂𝒊𝒋 × 𝒙𝒑𝒋
𝑗=1
𝑥𝑖 = sum }

Step 5 (Approximate the eigenvalue 𝑩 and normalize the vector 𝑿)

set 𝑟 = 1
Finding the element of 𝑋 with
for 𝑖 = 1, 2, ⋯ , 𝒏
} (the largest absolute value )
if (|𝑥𝑖 | > |𝑥𝑟 |) 𝑟 = 𝑖
and then setting it as 𝑩
set 𝑩 = 𝑥𝑟
for 𝑖 = 1, 2, ⋯ , 𝒏
𝑥𝑖 = 𝑥𝑖 /𝑩 (Normalizing the vector 𝑋)
Step 6
This means that the consecutive
if (𝒆𝒓𝒓 < 𝑻𝑶𝑳 )then
} approximations are nearly the same.
Exit/Break the loop
Therefore, stop iterations.

end for loop of Step 2 (Go to Step 3)

Step 9 Print the output: eigenvalue 𝑩, and eigenvector 𝑿 = [𝒙𝟏 , 𝒙𝟐 , ⋯ , 𝒙𝒏 ]𝑻

if (𝒆𝒓𝒓 < 𝑻𝑶𝑳) OUTPUT (‘The desired accuracy achieved; Solution converged.’)
else OUTPUT (‘The number of iterations exceeded the maximum limit.’)
STOP.
122 Simplified Numerical Analysis

Problem 10: Write a C++ program to find the dominant eigenvalue of the following matrix using the Power
method. For simplification, specify the matrix within the program. Take 𝑋 (0) = [1, 1, 1]𝑇 as the initial
approximation. The iterations of the method should stop when either the approximation is accurate within
10−5 , or the number of iterations exceeds 100, whichever happens first.
4 1 0
𝐴 = [2 5 0]
7 2 1
1 # include <iostream>
2 # include <cmath>
3 using namespace std ;
4
5 # define n 3 // number of components
6 # define TOL 0.00001 // error tolerance
7 # define N 100 // maximum number of iterations
8
9 int main ( )
10 {
11 char quit ;
12 int i , j , r , k ;
13 double a[n][n] = { {4,1,0} , {2,5,0} , {7,2,1} } ; // the problem matrix
14 double x[n] = { 1, 1, 1 } ; // initial approx. to the dominant eigenvector
15 double xp[n] , sum , sum1 , B , err ;
16
17 //----------------------- Processing Section -------------------------//
18
19 for ( k=1 ; k<=N ; k++ )
20 {
21
22 for ( i=0 ; i<n ; i++ )
23 xp[i] = x[i] ;
24
25 // Computing the vector X^(k) = A * X^(k-1)
26 for ( i=0 ; i<n ; i++ )
27 {
28 sum = 0.0 ; 𝑛
29 for (j=0 ; j<n ; j++ ) (𝑘) (𝑘−1)
(𝑥𝑖 = ∑ 𝑎𝑖𝑗 𝑥𝑗 )
30 sum = sum + a[i][j] * xp[j] ;
𝑗=1
31 x[i] = sum ;
32 }
33
34 // Approximating the eigenvalue B and normalizing the vector X
35 r=0;
36 for ( i= 1 ; i<n ; i++ )
37 if ( abs(x[i]) > abs(x[r]) ) r=i;
38
39 B = x[r] ;
40
41 for ( i=0 ; i<n ; i++ )
42 x[i] = x[i] / B ;
Eigenvalues and Eigenvectors 123

43
44 // Computing the error as L2-norm
45 sum1 = 0.0 ;
46 for (i=0 ; i<n ; i++ ) Computing
47 sum1 = sum1 + ( x[i] -xp[i] ) * ( x[i] - xp[i] ) ; 𝑙2 − 𝑛𝑜𝑟𝑚
48 err = sqrt( sum1 ) ;
49
50 if ( err < TOL ) break ;
51
52 }
53
54 //---------------------- Output Section -------------------------//
55
56 cout<< "\nThe approximate dominant eigenvalue is " << B ;
57
58 cout<< "\nThe approximate corresponding eigenvector is " << endl ;
59 for ( i=0 ; i<n ; i++ ) cout<< x[i] << " " ;
60
61 if( err<TOL )
62 cout<< "\nThe desired accuracy achieved; Solution converged" ;
63 else
64 cout<< "\nThe number of iterations exceeded the maximum limit.";
65
66 return 0 ;
67
68 }

Remark: In the program of Problem 10, the code segment of lines 56-59 can be placed just before
line 52 to print the latest results on completion of each of the iterations.

Remark: The C++ program in Problem 10 can be modified to receive the square matrix and the
initial approximation of the Eigenvector at the execution time (instead of fixing in the code). For
this, the code segment at lines 13 and 14 in the program of Problem 10 should be replaced by the
following code segment:

cout<<"\nEnter the matrix row-wise: "<<endl;

for ( i=0 ; i<n ; i++)
for ( j=0 ; j<n ; j++)
cin >> a[i][j] ;

cout<<"Enter the initial approximation:"<<endl ;

for ( i=0 ; i<n ; i++)
cin >> b[i] ;

∎∎∎
124 Simplified Numerical Analysis

Chapter Summary

• An eigenvalue of a square matrix 𝐴 = (𝑎𝑖𝑗 ) is a number 𝜆 such that the vector equation
𝑛×𝑛

𝐴𝑋 = 𝜆𝑋

• has a non-zero solution vector 𝑋. The solution vector 𝑋 is then called an eigenvector of the matrix 𝐴
corresponding to the eigenvalue 𝜆. The set of all eigenvalues of a matrix is called the spectrum of the
matrix. An eigenvalue is also called a characteristic value or latent root. Likewise, an eigenvector is also
called a characteristic vector or latent vector.

• A concise account of the results and techniques relevant to the eigenvalues and eigenvectors is given in
Section 8.1.

• The theorem of the Power method: Suppose that an 𝑛 × 𝑛 matrix 𝐴 has 𝑛 eigenvalues 𝜆1 , 𝜆2 , ⋯ , 𝜆𝑛 and
associated 𝑛 linearly independent eigenvectors, 𝑉1 , 𝑉2 , ⋯ , 𝑉𝑛 . Further, suppose that 𝑋 (0) is a normalized
vector (i.e., a vector having maximum absolute value as 1) in the space of the said eigenvectors. The
∞
sequence of normalized vectors {𝑋 (𝑘) }𝑘=1 and the sequence of scalars {𝛽𝑘 }∞
𝑘=1 generated recursively by

1 (𝑘)
𝑋 (𝑘) = 𝑌 ,
𝛽𝑘
(𝑘) (𝑘)
where 𝑌 (𝑘) = 𝐴𝑋 (𝑘−1) , and 𝛽𝑘 = 𝑦𝑟 such that |𝑦𝑟 | = ‖𝑌 (𝑘) ‖∞ ,

will converge to the dominant eigenvector and eigenvalue, respectively.

∞
• In the Power method, both the sequences of the scalars {𝛽𝑘 }∞
𝑘=1 and the normalized vectors {𝑋
(𝑘)
}𝑘=1
converge linearly to the dominant eigenvalue 𝜆1 and a corresponding eigenvector 𝑉1 , respectively. Thus,
the order of convergence of the Power method is linear.

• Aitken’s ∆𝟐 method offers a technique for accelerating the convergence of any sequence that is linearly
convergent. Using a given sequence, say {𝛽𝑘 }∞ 𝑘=1 , which converges linearly to 𝜆1 , another sequence
∞
{𝛽̂𝑘 }
𝑘=1
(that also converges to 𝜆1 with possibly improved convergence rate) is constructed by using the
Aitken’s ∆2 process as:

(𝛽𝑘+1 − 𝛽𝑘 )2 (∆𝛽𝑘 )2
𝛽̂𝑘 = 𝛽𝑘 − = 𝛽𝑘 − , for 𝑘 = 0, 1, 2, ⋯
𝛽𝑘+2 − 2𝛽𝑘+1 + 𝛽𝑘 ∆2 𝛽𝑘

• Suppose that 𝜆 is a non-zero eigenvalue of a square matrix 𝐴 and 𝑋 is an eigenvector corresponding to 𝜆.

Then, 1⁄𝜆 is an eigenvalue of 𝐴−1 and the same 𝑋 is an eigenvector corresponding to 1⁄𝜆. Thus, the
reciprocal of all the non-zero eigenvalues of a square matrix 𝐴 are the eigenvalues of 𝐴−1 (having the
same set of eigenvectors). Hence, the largest of the absolute eigenvalues of 𝐴 is the smallest of the
Eigenvalues and Eigenvectors 125

eigenvalues of 𝐴−1 (and vice-versa). Thus, the Power method can be used to obtain the largest
eigenvalue of 𝐴−1 and then taking its reciprocal gives the smallest eigenvalue of 𝐴.

∎∎∎

Chapter Exercises

Exercise 01: Find all the eigenvalues and eigenvectors of the following matrices using the characteristic
equations. Also find the spectrum, spectral radius, trace, and determinant of the given matrix.

3 2 −1 3 −2 0.5 2 0 0
(i) [2 6 4] (ii) [−1 −2 1.5] (iii) [−6 8 −14]
−1 4 5 −4 0 4 0 0 −6

−15.5 −10 10 4.5 0 1.5

(iv) [ 3 4.5 −3 ] (v) [−6 9 6]
−17 −10 11.5 1.5 0 4.5

Exercise 02: Apply the Power method to find the dominant eigenvalue and corresponding eigenvector of the
given matrices.

3 2 −1 3 −2 0.5 2 0 0
(i) [2 6 4] (ii) [−1 −2 1.5] (iii) [−6 8 −14]
−1 4 5 −4 0 4 0 0 −6

−15.5 −10 10 4.5 0 1.5

(iv) [ 3 4.5 −3 ] (v) [−6 9 6]
−17 −10 11.5 1.5 0 4.5

Exercise 03: Apply the Power method to find the dominant eigenvalue and corresponding eigenvector of the
given matrices.

8 1 0 0 1 10 6 −6
0 7 0 0 0 −9 0 0
(i) [ ] (ii) [ ]
−2 1 10 0 −0.5 16.5 7.5 0.5
−4 −1 4 6 −6.5 10.5 6.5 1.5
126 Simplified Numerical Analysis

Exercise 04: Use Householder’s method to place the following matrices in tridiagonal form.

1 1 1 2 −1 −1
(i) [1 1 0] (ii) [−1 2 −1]
1 0 1 −1 −1 2

5 −2 −0.5 1.5 2 −1 −1 0
−2 5 1.5 −0.5 −1 3 0 0
(iii) [ ] (iv) [ ]
−0.5 1.5 5 −2 −1 0 4 1
1.5 −0.5 −2 5 0 −2 2 3

Exercise 05: Apply two iterations of the QR Factorization method without shifting the following matrices.

4 −1 0 3 1 0
(i) [−1 3 −1] (ii) [1 4 2]
0 −1 2 0 2 1

4 2 0 0 0.5 0.25 0 0
2 4 2 0 0.25 0.8 0.4 0
(iii) [ ] (iv) [ ]
0 2 4 2 0 0.4 0.6 0.1
0 0 2 4 0 0 0.1 1

Exercise 06: Determine a singular value decomposition for the following matrices.

1 1 0 2 1
(i) [−1 0 1] (ii) [−1 1]
0 1 −1 1 1

1 1 0 2 1
(iii) [1 0 1] (iv) [1 0 ]
0 1 1 0 1

∎∎∎
 Chapter 9
Numerical Solution of
Ordinary Differential Equations (ODEs)

Corridor I: BASICS
Let’s plan it

9.1 Introduction
9.2 Solving IVPs using Single Step Methods and Multistep Methods
The Euler Method
The Mid-point Method (an RK2 method of Order 2)
The Modified/Improved Euler Method (an RK2 method of Order 2)
The RK Method of order 4 (RK4)
9.3 Solving IVPs using Predictor-Corrector Methods
The Adams-Bashforth-Moulton Method of Order 4
9.4 Solving Systems of ODEs and Higher Order ODEs
Using the Classical RK4 Method
9.5 Solving Linear BVPs using the Finite Difference Method

To unleash the topics of this Corridor, please delve into the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

127
 128 Simplified Numerical Analysis

Corridor II: ANALYSIS

Let’s think deep

9.6 Some Theoretical Concepts and Error Analysis

To unleash the topics of this Corridor, please delve into the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)
∎∎∎

Figure: The connection between various terms related to MDE (Model Differential Equation/s -
ODE/PDE) and the related FDE (Finite Difference Equation/s).
 Numerical Solution of Ordinary Differential Equations (ODEs) 129

Corridor III: PROGRAMMING ARCADE

Let’s think deep

9.7 Algorithms and Implementations

Euler method
Mid-point method
Modified/Improved Euler method
RK method of order 4 (RK4)
Adams-Bashforth method of order 4
Adams-Bashforth-Moulton method of order 4
RK4 method for a system of two ODEs
RK4 method for a system of three ODEs
RK4 method for Second Order ODE
RK4 method for Third Order ODE
Linear FDM for BVP

To cross-check the results/output of the computer programs you would execute, please delve into
the principal book:
Simplified Numerical Analysis (Fourth Edition; by Dr. Amjad Ali; www.TimeRenders.com.pk)

∎∎∎

9.7 Algorithms and Implementations

 130 Simplified Numerical Analysis

Question 16: Write down an algorithm (pseudo code) to solve a first-order ODE using the Explicit
Euler’s method (the Taylor method of order 1).

Algorithm: To solve 𝑦 ′ = 𝑓(𝑥, 𝑦) , for 𝑎 ≤ 𝑥 ≤ 𝑏 and 𝑦(𝑎) = 𝛼 by approximating 𝑦 = 𝑦(𝑥) at

(𝑚 + 1) equispaced nodes 𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑚 , such that 𝑎 = 𝑥0 < 𝑥1 < 𝑥2 < ⋯ < 𝑥𝑚 = 𝑏, ℎ = (𝑏 −
𝑎)/𝑚 and 𝑦(𝑥𝑖 ) = 𝑦𝑖 using the Explicit Euler’s method (the Taylor method of order 1): For 𝑖 =
1,2,3, ⋯ , 𝑚
𝑤𝑖 = 𝑤𝑖−1 + ℎ × 𝑓(𝑥𝑖−1 , 𝑦𝑖−1 )
𝑎, 𝑏: real values as the endpoints of the interval: 𝑥 ∈ [𝑎, 𝑏]
𝑚: an integer as the number of nodes (other than 𝑎) in the 𝑥 direction
𝐈𝐍𝐏𝐔𝐓𝐒:
𝑎𝑙𝑝ℎ𝑎: a real value as the initial condition 𝑦(𝑎)
{A definition of the function 𝑓(𝑥, 𝑦) in an appropriate way
𝑊 = [𝑤0 , 𝑤1 , ⋯ , 𝑤𝑚 ]𝑇 : a real valued vector as the approximate solution 𝑤(𝑥𝑖 )
𝐎𝐔𝐓𝐏𝐔𝐓: {
at the nodes 𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑚
ℎ: a real value as the step length in 𝑥 direction such that ℎ = (𝑏 − 𝑎)/𝑚
Auxiliary Variables: {
𝑥 = (𝑥𝑖 ) for 𝑖 = 0, 1, ⋯ , 𝑚: a real valued vector to represent 𝑥𝑖 𝑠

Step 1 Receive the inputs as stated above

Step 2 Set ℎ = (𝑏 − 𝑎)/𝑚
Set 𝑥(0) = 𝑎
Set 𝑥(𝑚) = 𝑏
Step 3 for 𝑖 = 1, 2, ⋯ , 𝑚 − 1
Set 𝑥(𝑖) = 𝑥(0) + 𝑖 × ℎ (Constructing interior mesh points, 𝑥𝑖 )
end for
Step 4 Set 𝑤(0) = 𝑎𝑙𝑝ℎ𝑎 (Setting the initial condition)
Step 5 for 𝑖 = 1, 2, ⋯ , 𝑚
𝑓𝑣𝑎𝑙 = 𝑓(𝑋(𝑖 − 1), 𝑤(𝑖 − 1)) (Computing the value 𝑓(𝑥𝑖−1 , 𝑦𝑖−1 ) )
𝑤(𝑖) = 𝑤(𝑖 − 1) + ℎ × 𝑓𝑣𝑎𝑙
end for
Step 6 Print the output: 𝑊 = [𝑤0 , 𝑤1 , ⋯ , 𝑤𝑚 ]𝑇 ;

STOP.
 Numerical Solution of Ordinary Differential Equations (ODEs) 131

Problem 09: Write down a C++ program to solve the IVP, 𝑦 ′ = 4𝑦 + 4𝑥 2 + 3𝑥, for 0 ≤ 𝑥 ≤ 1, with
initial condition 𝑦(0) = 𝛼 = 1/2, using the Explicit Euler’s method (the Taylor method of order 1).
Computer the solution for 10 steps. At each step, compare the approximate solution with the exact
5 5 13
solution, to be obtained by 𝑦(𝑥) = −𝑥 2 − 4 𝑥 − 16 + 16 𝑒 4𝑥 , by finding the relative error between the
two solutions.

1 #include <iostream>
2 #include <cmath>
3 #include <iomanip>
4 using namespace std ;
5
6 #define a 0.0 // starting point of domain
7 #define b 1.0 // ending point of domain
8 #define alpha 0.5 // initial condition
9 #define m 10 // number of steps
10
11 #define fval(x,y) 4*y+4*x*x+3*x
12 #define fexact(x) -x*x-1.2*x-(5.0/16.0)+(13.0/16.0)*exp(4*x)
13
14 int main()
15 {
16 double h; // step size
17 double fv ; // RHS function value
18 int i ;
19 double x[m+1] ; // vector of nodes
20 double w[m+1] ; // vector of solution values
21 double sol , err ;
22
23 h = (b-a)/m ;
24
25 x[0] = a ;
26
27 for (i=1 ; i<=m ; i++)
28 {
29 x[i] = x[i-1] + h ; // x[i] = x[0] + i*h ;
30 }
31
32 w[0]= alpha ; // setting initial condition
33
 132 Simplified Numerical Analysis

34 // ------ Computing solutions with the Euler method ------ //

35 for (i=1 ; i<=m ; i++)
36 {
37 fv = fval(x[i-1], w[i-1]) ;
38 w[i] = w[i-1] + h*fv ;
39 }
40
41 // ----------------- Printing Solutions ----------------- //
42 for (i=0 ; i<=m ; i++)
43 {
44 cout << "Node= " << setw(2) << i ;
45 cout << "\tx= " << setw(8) << x[i] ;
46 cout << "\tw= " << setw(8) << w[i] ;
47 sol = fexact(x[i]) ;
48 cout << "\tExact sol= " << setw(8) << sol ;
49 err = fabs(sol-w[i])/fabs(sol) ;
50 cout << "\tRelative Error= " << err << endl ;
51 }
52
53 return 0;
54 }

The above program can be written in a better way that a C++ function for the Euler method is
formed to compute the solution. This makes the program better manageable and modular. The new
program is given as follows.

1 #include <iostream>
2 #include <cmath>
3 #include <iomanip>
4 using namespace std ;
5
6 #define a 0.0 // starting point of domain
7 #define b 1.0 // ending point of domain
8 #define alpha 0.5 // initial condition
9 #define m 10 // number of steps
10
11 #define fval(x,y) 4*y+4*x*x+3*x
12 #define fexact(x) -x*x-1.2*x-(5.0/16.0)+(13.0/16.0)*exp(4*x)
13
Numerical Solution of Ordinary Differential Equations (ODEs) 133

14 void euler(double [], double [], double) ;

15
16 int main()
17 {
18 double h ; // step size
19 double sol , err ;
20
21 int i ;
22 double x[m+1] ; // vector of nodes
23 double w[m+1] ; // vector of solution values
24
25 h = (b-a)/m ;
26
27 // forming vector of x-values or nodes
28 x[0] = a ;
29 x[m] = b ;
30
31 for (i=1 ; i<=m ; i++)
32 {
33 x[i] = x[i-1] + h ; // x[i] = x[0] + i*h ;
34 }
35
36 w[0]= alpha ; // setting initial condition
37
38 euler(x, w, h) ; // Call-by-reference to the function
39
40 // ----------------- Printing Solutions ----------------- //
41 for (i=0 ; i<=m ; i++)
42 {
43 cout << "Node= " << setw(2) << i ;
44 cout << "\tx= " << setw(8) << x[i] ;
45 cout << "\tw= " << setw(8) << w[i] ;
46 sol = fexact(x[i]) ;
47 cout << "\tExact sol= " << setw(8) << sol ;
48 err = fabs(sol-w[i])/fabs(sol) ;
49 cout << "\tRelative Error= " << err << endl ;
50 }
51
52 return 0;
53 }
54
 134 Simplified Numerical Analysis

55 // ----- User-defined function for the Euler method ----- //

56
57 void euler(double x[m+1], double w[m+1], double h)
58 {
59 double fv ; // RHS function value
60 int i ;
61
62 for (i=1 ; i<=m ; i++)
63 {
64 fv = fval(x[i-1], w[i-1]) ;
65 w[i] = w[i-1] + h*fv ; // computing next solution value
66 }
67 }

The results are shown in the following table.

Steps Numerical Exact solution Error of computer

Node 𝑥(𝑖) solution 𝑤𝑖 =
𝑖 𝑦𝑖 = 𝑦(𝑥𝑖 ) program solution
𝑤(𝑥𝑖 )

0 0 0.5 0.5 0

1 0.1 0.7 0.769608 0.0904455

2 0.2 1.014 1.21575 0.165948

3 0.3 1.4956 1.93509 0.227118

4 0.4 2.21984 3.07184 0.277358

5 0.5 3.29178 4.84111 0.320037

6 0.6 4.85849 7.56383 0.357669

7 0.7 7.12588 11.7188 0.391926

8 0.8 10.3844 18.0202 0.423855

9 0.9 15.0311 27.5336 0.45408

10 1.0 21.6376 41.8485 0.482954

 Numerical Solution of Ordinary Differential Equations (ODEs) 135

Question 17: Write down an algorithm ( pseudo code ) to solve a first-order ODE using the
Midpoint method (which is an RK method of order 2).

Algorithm: To solve 𝑦 ′ = 𝑓(𝑥, 𝑦) , for 𝑎 ≤ 𝑥 ≤ 𝑏 and 𝑦(𝑎) = 𝛼 by approximating 𝑦 = 𝑦(𝑥) at

(𝑚 + 1) equispaced nodes 𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑚 , such that 𝑎 = 𝑥0 < 𝑥1 < 𝑥2 < ⋯ < 𝑥𝑚 = 𝑏, ℎ = (𝑏 −
𝑎)/𝑚 and 𝑦(𝑥𝑖 ) = 𝑦𝑖 using the Midpoint method: For 𝑖 = 1,2,3, ⋯ , 𝑚
ℎ
𝑦̿𝑖 = 𝑦𝑖−1 + × 𝑓(𝑥𝑖−1 , 𝑦𝑖−1 )
2
ℎ
𝑦𝑖 = 𝑦𝑖−1 + ℎ × 𝑓 (𝑥𝑖−1 + , 𝑦̿)
2 𝑖

𝑎, 𝑏: real values as the endpoints of the interval: 𝑥 ∈ [𝑎, 𝑏]

𝑚: an integer as the number of nodes (other than 𝑎) in the 𝑥 direction
𝐈𝐍𝐏𝐔𝐓𝐒:
𝑎𝑙𝑝ℎ𝑎: a real value as the initial condition 𝑦(𝑎)
{A definition of the function 𝑓(𝑥, 𝑦) in an appropriate way
𝑊 = [𝑤0 , 𝑤1 , ⋯ , 𝑤𝑚 ]𝑇 : a real valued vector as the approximate solution 𝑤(𝑥𝑖 )
𝐎𝐔𝐓𝐏𝐔𝐓: {
at the nodes 𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑚

ℎ: a real value as the step length in 𝑥 direction such that ℎ = (𝑏 − 𝑎)/𝑚

Auxiliary Variables: {
𝑋 = (𝑥𝑖 ) for 𝑖 = 0, 1, ⋯ , 𝑚: a real valued vector to represent 𝑥𝑖 𝑠

Step 1 Receive the inputs as stated above

Step 2 Set ℎ = (𝑏 − 𝑎)/𝑚
Set 𝑥(0) = 𝑎
Set 𝑥(𝑚) = 𝑏
Step 3 for 𝑖 = 1, 2, ⋯ , 𝑚 − 1
Set 𝑥(𝑖) = 𝑥(0) + 𝑖 × ℎ (Constructing interior mesh points, 𝑥𝑖 )
end for

Step 4 Set 𝑤(0) = 𝑎𝑙𝑝ℎ𝑎 (Setting the initial condition)

Step 5 for 𝑖 = 1, 2, ⋯ , 𝑚
𝑓𝑣𝑎𝑙1 = 𝑓(𝑥(𝑖 − 1), 𝑤(𝑖 − 1)) (Computing the value 𝑓(𝑥𝑖−1 , 𝑦𝑖−1 ))
𝑎𝑢𝑥 = 𝑤(𝑖 − 1) + (ℎ/2) × 𝑓𝑣𝑎𝑙1
𝑓𝑣𝑎𝑙2 = 𝑓(𝑥(𝑖 − 1) + (ℎ/2), 𝑎𝑢𝑥) (Computing 𝑓(𝑥𝑖−1 + ℎ/2, 𝑎𝑢𝑥) )
𝑤(𝑖) = 𝑤(𝑖 − 1) + ℎ × 𝑓𝑣𝑎𝑙2
end for

Step 6 Print the output: 𝑊 = [𝑤0 , 𝑤1 , ⋯ , 𝑤𝑚 ]𝑇

STOP.
 136 Simplified Numerical Analysis

Problem 10: Write down a C++ program to solve the IVP, 𝑦 ′ = 4𝑦 + 4𝑥 2 + 3𝑥, for 0 ≤ 𝑥 ≤ 1, with
initial condition 𝑦(0) = 𝛼 = 1/2, using the Midpoint method (which is an RK method of order 2).
Computer the solution for 10 steps. At each step, compare the approximate solution with the exact
5 5 13
solution, to be obtained by 𝑦(𝑥) = −𝑥 2 − 4 𝑥 − 16 + 16 𝑒 4𝑥 , by finding the relative error between the
two solutions.

1 #include <iostream>
2 #include <cmath>
3 #include <iomanip>
4 using namespace std ;
5
6 #define a 0.0 // starting point of domain
7 #define b 1.0 // ending point of domain
8 #define alpha 0.5 // initial condition
9 #define m 10 // number of steps
10
11 #define fval(x,y) 4*y+4*x*x+3*x
12 #define fexact(x) -x*x-1.2*x-(5.0/16.0)+(13.0/16.0)*exp(4*x)
13
14 void eulermid(double [], double [], double) ;
15
16 int main()
17 {
18 double h ; // step size
19 double sol , err ;
20
21 int i ;
22 double x[m+1] ; // vector of nodes
23 double w[m+1] ; // vector of solution values
24
25 h = (b-a)/m ;
26
27 // forming vector of x-values or nodes
28 x[0] = a ;
29 x[m] = b ;
30
31 for (i=1 ; i<=m ; i++)
32 {
33 x[i] = x[i-1] + h ; // x[i] = x[0] + i*h ;
34 }
35
Numerical Solution of Ordinary Differential Equations (ODEs) 137

36 w[0]= alpha ; // setting initial condition

37
38 eulermid(x, w, h) ; // Call-by-reference to the function
39
40 // ----------------- Printing Solutions ----------------- //
41 for (i=0 ; i<=m ; i++)
42 {
43 cout << "Node= " << setw(2) << i ;
44 cout << "\tx= " << setw(8) << x[i] ;
45 cout << "\tw= " << setw(8) << w[i] ;
46 sol = fexact(x[i]) ;
47 cout << "\tExact sol= " << setw(8) << sol ;
48 err = fabs(sol-w[i])/fabs(sol) ;
49 cout << "\tRelative Error= " << err << endl ;
50 }
51
52 return 0;
53 }
54
55 // ----- User-defined function for the Euler Mid-point ----- //
56
57 void eulermid(double xa[m+1], double wa[m+1], double h)
58 {
59 double fv ; // RHS function value
60 int i ;
61 double whalf , xhalf ;
62
63 for (i=1 ; i<=m ; i++)
64 {
65 fv = fval(xa[i-1], wa[i-1]) ;
66 whalf = wa[i-1] + (h/2.0)*fv ;
67
68 xhalf = xa[i-1] + (h/2.0) ;
69 fv = fval(xhalf, whalf) ;
70
71 wa[i] = wa[i-1] + h*fv ;
72 }
73 }
 138 Simplified Numerical Analysis

The results are shown in the following table.

Steps Numerical Exact solution Error of computer

Node 𝑥(𝑖) solution 𝑤𝑖 =
𝑖 𝑦𝑖 = 𝑦(𝑥𝑖 ) program solution
𝑤(𝑥𝑖 )

0 0 0.5 0.5 0
1 0.1 0.756 0.769608 0.0176812
2 0.2 1.17968 1.21575 0.0296705
3 0.3 1.86113 1.93509 0.0382248
4 0.4 2.93367 3.07184 0.0449802
5 0.5 4.59463 4.84111 0.0509141
6 0.6 7.13605 7.56383 0.0565563
7 0.7 10.9902 11.7188 0.0621758
8 0.8 16.7966 18.0202 0.0678992
9 0.9 25.5022 27.5336 0.0737776
10 1.0 38.5081 41.8485 0.0798221

Question 18: Write down an algorithm (pseudo code) to solve a first-order ODE using the RK
method of order 2 (also known as the Modified or Improved Euler’s method).

Algorithm: To solve 𝑦 ′ = 𝑓(𝑥, 𝑦) , for 𝑎 ≤ 𝑥 ≤ 𝑏 and 𝑦(𝑎) = 𝛼 by approximating 𝑦 = 𝑦(𝑥) at

(𝑚 + 1) equispaced nodes 𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑚 , such that 𝑎 = 𝑥0 < 𝑥1 < 𝑥2 < ⋯ < 𝑥𝑚 = 𝑏, ℎ = (𝑏 −
𝑎)/𝑚 and 𝑦(𝑥𝑖 ) = 𝑦𝑖 using the Modified Euler’s method of order 2: For 𝑖 = 1,2,3, ⋯ , 𝑚
𝐾1 = ℎ × 𝑓(𝑥𝑖−1 , 𝑦𝑖−1 )
𝐾2 = ℎ × 𝑓(𝑥𝑖 , 𝑦𝑖−1 + 𝐾1 )
1
𝑦𝑖 = 𝑦𝑖−1 + × [𝐾1 + 𝐾2 ]
2

𝑎, 𝑏: real values as the endpoints of the interval: 𝑥 ∈ [𝑎, 𝑏]

𝑚: an integer as the number of nodes (other than 𝑎) in the 𝑥 direction
𝐈𝐍𝐏𝐔𝐓𝐒:
𝑎𝑙𝑝ℎ𝑎: a real value as the initial condition 𝑦(𝑎)
{A definition of the function 𝑓(𝑥, 𝑦) in an appropriate way
𝑊 = [𝑤0 , 𝑤1 , ⋯ , 𝑤𝑚 ]𝑇 : a real valued vector as the approximate solution 𝑤(𝑥𝑖 )
𝐎𝐔𝐓𝐏𝐔𝐓: {
at the nodes 𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑚

ℎ: a real value as the step length in 𝑥 direction such that ℎ = (𝑏 − 𝑎)/𝑚

Auxiliary Variables: {
𝑋 = (𝑥𝑖 ) for 𝑖 = 0, 1, ⋯ , 𝑚: a real valued vector to represent 𝑥𝑖 𝑠
 Numerical Solution of Ordinary Differential Equations (ODEs) 139

Step 1 Receive the inputs as stated above

Step 2 Set ℎ = (𝑏 − 𝑎)/𝑚
Set 𝑥(0) = 𝑎
Set 𝑥(𝑚) = 𝑏
Step 3 for 𝑖 = 1, 2, ⋯ , 𝑚 − 1
Set 𝑥(𝑖) = 𝑥(0) + 𝑖 × ℎ (Constructing interior mesh points, 𝑥𝑖 )
end for

Step 4 Set 𝑤(0) = 𝑎𝑙𝑝ℎ𝑎 (Setting the initial condition)

Step 5 for 𝑖 = 1, 2, ⋯ , 𝑚
𝑘1 = ℎ × 𝑓(𝑥(𝑖 − 1), 𝑤(𝑖 − 1))
𝑘2 = ℎ × 𝑓(𝑥(𝑖), 𝑤(𝑖 − 1) + 𝑘1)
𝑤(𝑖) = 𝑤(𝑖 − 1) + 0.5 × (𝑘1 + 𝑘2)
end for

Step 6 Print the output: 𝑊 = [𝑤0 , 𝑤1 , ⋯ , 𝑤𝑚 ]𝑇

STOP.

Problem 11: Write down a C++ program to solve the IVP, 𝑦 ′ = 4𝑦 + 4𝑥 2 + 3𝑥, for 0 ≤ 𝑥 ≤ 1, with
initial condition 𝑦(0) = 𝛼 = 1/2, using the RK method of order 2 (also known as the Modified or
Improved Euler’s method ) . Computer the solution for 10 steps. At each step, compare the
5 5 13
approximate solution with the exact solution, to be obtained by 𝑦(𝑥) = −𝑥 2 − 4 𝑥 − 16 + 16 𝑒 4𝑥 , by
finding the relative error between the two solutions.

1 #include <iostream>
2 #include <cmath>
3 #include <iomanip>
4 using namespace std ;
5
6 #define a 0.0 // starting point of domain
7 #define b 1.0 // ending point of domain
8 #define alpha 0.5 // initial condition
9 #define m 10 // number of steps
10
11 #define fval(x,y) 4*y+4*x*x+3*x
12 #define fexact(x) -x*x-1.2*x-(5.0/16.0)+(13.0/16.0)*exp(4*x)
13
140 Simplified Numerical Analysis

14 void eulerimp(double [], double [], double) ;

15
16 int main()
17 {
18 double h ; // step size
19 double sol , err ;
20
21 int i ;
22 double x[m+1] ; // vector of nodes
23 double w[m+1] ; // vector of solution values
24
25 h = (b-a)/m ;
26
27 // forming vector of x-values or nodes
28 x[0] = a ;
29 x[m] = b ;
30
31 for (i=1 ; i<=m ; i++)
32 {
33 x[i] = x[i-1] + h ; // x[i] = x[0] + i*h ;
34 }
35
36 w[0]= alpha ; // setting initial condition
37
38 eulerimp(x, w, h) ; // Call-by-reference to the function
39
40 // ----------------- Printing Solutions ----------------- //
41 for (i=0 ; i<=m ; i++)
42 {
43 cout << "Node= " << setw(2) << i ;
44 cout << "\tx= " << setw(8) << x[i] ;
45 cout << "\tw= " << setw(8) << w[i] ;
46 sol = fexact(x[i]) ;
47 cout << "\tExact sol= " << setw(8) << sol ;
48 err = fabs(sol-w[i])/fabs(sol) ;
49 cout << "\tRelative Error= " << err << endl ;
50 }
51
52 return 0;
53 }
54
 Numerical Solution of Ordinary Differential Equations (ODEs) 141

55 // ----- User-defined function for the RK 2----- //

56
57 void eulerimp(double xa[m+1], double wa[m+1], double h)
58 {
59 double fv, fvnext ; // RHS function value
60 int i ;
61 double wnext ;
62
63 for (i=1 ; i<=m ; i++)
64 {
65 fv = fval(xa[i-1], wa[i-1]) ;
66 wnext = wa[i-1] + h*fv ;
67
68 fvnext = fval(xa[i], wnext) ;
69
70 wa[i] = wa[i-1] + h*(fv+fvnext)/2.0 ;
71 }
72 }

The results are shown in the following table.

Steps Numerical Exact solution Error of computer

Node 𝑥(𝑖) solution 𝑤𝑖 =
𝑖 𝑦𝑖 = 𝑦(𝑥𝑖 ) program solution
𝑤(𝑥𝑖 )

0 0 0.5 0.5 0
1 0.1 0.757 0.769608 0.0163818
2 0.2 1.18216 1.21575 0.0276306
3 0.3 1.8658 1.93509 0.0358113
4 0.4 2.94158 3.07184 0.0424044
5 0.5 4.60734 4.84111 0.0482887
6 0.6 7.15586 7.56383 0.0539372
7 0.7 11.0205 11.7188 0.0595885
8 0.8 16.8425 18.0202 0.0653535
9 0.9 25.5711 27.5336 0.0712755
10 1.0 38.611 41.8485 0.0773618

∎
 142 Simplified Numerical Analysis

Question 19: Write down an algorithm (pseudo code) to solve a first-order ODE using the RK
method of order 4.
Algorithm: To solve 𝑦 ′ = 𝑓(𝑥, 𝑦) , for 𝑎 ≤ 𝑥 ≤ 𝑏 and 𝑦(𝑎) = 𝛼 by approximating 𝑦 = 𝑦(𝑥) at
(𝑚 + 1) equispaced nodes 𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑚 , such that 𝑎 = 𝑥0 < 𝑥1 < 𝑥2 < ⋯ < 𝑥𝑚 = 𝑏, ℎ = (𝑏 −
𝑎)/𝑚 and 𝑦(𝑥𝑖 ) = 𝑦𝑖 using the RK method of order 4: For 𝑖 = 1,2,3, ⋯ , 𝑚
𝐾1 = ℎ × 𝑓(𝑥𝑖−1 , 𝑦𝑖−1 )
𝐾2 = ℎ × 𝑓(𝑥𝑖−1 + 0.5ℎ, 𝑦𝑖−1 + 0.5𝐾1 )
𝐾3 = ℎ × 𝑓(𝑥𝑖−1 + 0.5ℎ, 𝑦𝑖−1 + 0.5𝐾2 )
𝐾4 = ℎ × 𝑓(𝑥𝑖 , 𝑦𝑖−1 + 𝐾3 )
1
𝑦𝑖 = 𝑦𝑖−1 + × [𝐾1 + 2𝐾2 + 2𝐾3 + 𝐾4 ]
6
𝑎, 𝑏: real values as the endpoints of the interval: 𝑥 ∈ [𝑎, 𝑏]
𝑚: an integer as the number of nodes (other than 𝑎) in the 𝑥 direction
𝐈𝐍𝐏𝐔𝐓𝐒:
𝑎𝑙𝑝ℎ𝑎: a real value as the initial condition 𝑦(𝑎)
{A definition of the function 𝑓(𝑥, 𝑦) in an appropriate way
𝑊 = [𝑤0 , 𝑤1 , ⋯ , 𝑤𝑚 ]𝑇 : a real valued vector as the approximate solution 𝑦(𝑥𝑖 )
𝐎𝐔𝐓𝐏𝐔𝐓: {
at the nodes 𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑚
ℎ: a real value as the step length in 𝑥 direction such that ℎ = (𝑏 − 𝑎)/𝑚
Auxiliary Variables: {
𝑋 = (𝑥𝑖 ) for 𝑖 = 0, 1, ⋯ , 𝑚: a real valued vector to represent 𝑥𝑖 𝑠
Step 1 Receive the inputs as stated above
Step 2 Set ℎ = (𝑏 − 𝑎)/𝑚
Set 𝑥(0) = 𝑎
Set 𝑥(𝑚) = 𝑏
Step 3 for 𝑖 = 1, 2, ⋯ , 𝑚 − 1
Set 𝑥(𝑖) = 𝑥(0) + 𝑖 × ℎ (Constructing interior mesh points, 𝑥𝑖 )
end for
Step 4 Set 𝑤(0) = 𝑎𝑙𝑝ℎ𝑎 (Setting the initial condition)
Step 5 for 𝑖 = 1, 2, ⋯ , 𝑚
𝑘1 = ℎ × 𝑓(𝑥(𝑖 − 1), 𝑤(𝑖 − 1))
𝑘2 = ℎ × 𝑓(𝑥(𝑖 − 1) + 0.5 × ℎ, 𝑤(𝑖 − 1) + 0.5 × 𝑘1)
𝑘3 = ℎ × 𝑓(𝑥(𝑖 − 1) + 0.5 × ℎ, 𝑤(𝑖 − 1) + 0.5 × 𝑘2)
𝑘4 = ℎ × 𝑓(𝑥(𝑖), 𝑤(𝑖 − 1) + 𝑘3)
𝑤(𝑖) = 𝑤(𝑖 − 1) + (𝑘1 + 2 × 𝑘2 + 2 × 𝑘3 + 𝑘4)/6
end for
Step 6 Print the output: 𝑊 = [𝑤0 , 𝑤1 , ⋯ , 𝑤𝑚 ]𝑇
STOP.
 Numerical Solution of Ordinary Differential Equations (ODEs) 143

Problem 12: Write down a C++ program to solve the IVP, 𝑦 ′ = 4𝑦 + 4𝑥 2 + 3𝑥, for 0 ≤ 𝑥 ≤ 1, with
initial condition 𝑦(0) = 𝛼 = 1/2, using the RK method of order 4. Computer the solution for 10
steps. At each step, compare the approximate solution with the exact solution, to be obtained by
5 5 13
𝑦(𝑥) = −𝑥 2 − 𝑥 − + 𝑒 4𝑥 , by finding the relative error between the two solutions.
4 16 16

1 #include <iostream>
2 #include <cmath>
3 #include <iomanip>
4 using namespace std ;
5
6 #define a 0.0 // starting point of domain
7 #define b 1.0 // ending point of domain
8 #define alpha 0.5 // initial condition
9 #define m 10 // number of steps
10
11 #define fval(x,y) 4*y+4*x*x+3*x
12 #define fexact(x) -x*x-1.2*x-(5.0/16.0)+(13.0/16.0)*exp(4*x)
13
14 void rk4(double [], double [], double) ;
15
16 int main()
17 {
18 double h ; // step size
19 double sol , err ;
20
21 int i ;
22 double x[m+1] ; // vector of nodes
23 double w[m+1] ; // vector of solution values
24
25 h = (b-a)/m ;
26
27 // forming vector of x-values or nodes
28 x[0] = a ;
29 x[m] = b ;
30
31 for (i=1 ; i<=m ; i++)
32 {
33 x[i] = x[i-1] + h ; // x[i] = x[0] + i*h ;
34 }
35
 144 Simplified Numerical Analysis

36 w[0]= alpha ; // setting initial condition

37
38 rk4(x, w, h) ; // Call-by-reference to the function
39
40 // ----------------- Printing Solutions ----------------- //
41 for (i=0 ; i<=m ; i++)
42 {
43 cout << "Node= " << setw(2) << i ;
44 cout << "\tx= " << setw(8) << x[i] ;
45 cout << "\tw= " << setw(8) << w[i] ;
46 sol = fexact(x[i]) ;
47 cout << "\tExact sol= " << setw(8) << sol ;
48 err = fabs(sol-w[i])/fabs(sol) ;
49 cout << "\tRelative Error= " << err << endl ;
50 }
51
52 return 0;
53 }
54
55 // ----- User-defined function for the RK 4----- //
56
57 void rk4(double x[m+1], double w[m+1], double h)
58 {
59 double k1 , k2 , k3 , k4 ;
60 int i ;
61 double p , q ;
62
63 for (i=1 ; i<=m ; i++)
64 {
65 k1 = h*(fval( x[i-1] , w[i-1] ));
66 k2 = h*(fval( (x[i-1]+(h/2.0)) , (w[i-1]+(k1/2.0) )));
67 k3 = h*(fval( (x[i-1]+(h/2.0)) , (w[i-1]+(k2/2.0) )));
68 k4 = h*(fval( x[i] , (w[i-1]+k3) ));
69
70 w[i] = w[i-1] + (k1+2*k2+2*k3+k4)/6.0 ;
71 }
72 }

The results are shown in the following table.

 Numerical Solution of Ordinary Differential Equations (ODEs) 145

Steps Numerical Exact solution Error of computer

Node 𝑥(𝑖) solution 𝑤𝑖 =
𝑖 𝑦𝑖 = 𝑦(𝑥𝑖 ) program solution
𝑤(𝑥𝑖 )

0 0 0.5 0.5 0
1 0.1 0.764547 0.769608 0.00657595
2 0.2 1.20556 1.21575 0.00838021
3 0.3 1.91966 1.93509 0.00797517
4 0.4 3.05096 3.07184 0.00679678
5 0.5 4.81444 4.84111 0.00550804
6 0.6 7.53081 7.56383 0.00436537
7 0.7 11.6785 11.7188 0.00343964
8 0.8 17.9711 18.0202 0.00272617
9 0.9 27.4731 27.5336 0.00219447
10 1.0 41.7728 41.8485 0.0018091

Question 20: Write down an algorithm (pseudo code) to solve a first-order ODE using the Adams-
Bashforth method of order 4.

Algorithm: To solve 𝑦 ′ = 𝑓(𝑥, 𝑦) , for 𝑎 ≤ 𝑥 ≤ 𝑏 and 𝑦(𝑎) = 𝛼 by approximating 𝑦 = 𝑦(𝑥) at

(𝑚 + 1) equispaced nodes 𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑚 , such that 𝑎 = 𝑥0 < 𝑥1 < 𝑥2 < ⋯ < 𝑥𝑚 = 𝑏, ℎ = (𝑏 −
𝑎)/𝑚 and 𝑦(𝑥𝑖 ) = 𝑦𝑖 . Having 𝑦(𝑥0 ) = 𝛼0 , 𝑦(𝑥1 ) = 𝛼1 , 𝑦(𝑥2 ) = 𝛼2 , and 𝑦(𝑥3 ) = 𝛼3 , compute 𝑦𝑖
using the 4-step explicit Adams-Bashforth method of order 4: For 𝑖 = 4,5,6, ⋯ , 𝑚

ℎ
𝑦𝑖 = 𝑦𝑖−1 + × [55𝑓(𝑥𝑖−1 , 𝑦𝑖−1 ) − 59𝑓(𝑥𝑖−2 , 𝑦𝑖−2 ) + 37𝑓(𝑥𝑖−3 , 𝑦𝑖−3 ) − 9𝑓(𝑥𝑖−4 , 𝑦𝑖−4 )]
24
𝑎, 𝑏: real values as the endpoints of the interval: 𝑥 ∈ [𝑎, 𝑏]
𝑚: an integer as the number of nodes (other than 𝑎) in the 𝑥 direction
𝐈𝐍𝐏𝐔𝐓𝐒:
𝑎𝑙𝑝ℎ𝑎: a real value as the initial condition 𝑦(𝑎)
{A definition of the function 𝑓(𝑥, 𝑦) in an appropriate way

𝑊 = [𝑤0 , 𝑤1 , ⋯ , 𝑤𝑚 ]𝑇 : a real valued vector as the approximate solution 𝑤(𝑥𝑖 )

𝐎𝐔𝐓𝐏𝐔𝐓: {
at the nodes 𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑚

ℎ: a real value as the step length in 𝑥 direction such that ℎ = (𝑏 − 𝑎)/𝑚

Auxiliary Variables: {
𝑋 = (𝑥𝑖 ) for 𝑖 = 0, 1, ⋯ , 𝑚: a real valued vector to represent 𝑥𝑖 𝑠

Step 1 Receive the inputs as stated above

 146 Simplified Numerical Analysis

Step 2 Set ℎ = (𝑏 − 𝑎)/𝑚

Set 𝑥(0) = 𝑎
Set 𝑥(𝑚) = 𝑏
Step 3 for 𝑖 = 1, 2, ⋯ , 𝑚 − 1
Set 𝑥(𝑖) = 𝑥(0) + 𝑖 × ℎ (Constructing interior mesh points, 𝑥𝑖 )
end for
Step 4 Set 𝑤(0) = 𝑎𝑙𝑝ℎ𝑎 (Setting the initial condition)
Step 5 Obtain or compute (using some other basic method for ODEs) the following:
𝑤(1) = 𝑎𝑙𝑝ℎ𝑎1
𝑤(2) = 𝑎𝑙𝑝ℎ𝑎2
𝑤(3) = 𝑎𝑙𝑝ℎ𝑎3

Step 5 for 𝑖 = 4, 5, 6, ⋯ , 𝑚
𝑓𝑣1 = 𝑓(𝑥(𝑖 − 1), 𝑤(𝑖 − 1)) (Computing the value 𝑓(𝑥𝑖−1 , 𝑦𝑖−1 ))
𝑓𝑣2 = 𝑓(𝑥(𝑖 − 2), 𝑤(𝑖 − 2)) (Computing the value 𝑓(𝑥𝑖−2 , 𝑦𝑖−2 ))
𝑓𝑣3 = 𝑓(𝑥(𝑖 − 3), 𝑤(𝑖 − 3)) (Computing the value 𝑓(𝑥𝑖−3 , 𝑦𝑖−3 ))
𝑓𝑣4 = 𝑓(𝑥(𝑖 − 4), 𝑤(𝑖 − 4)) (Computing the value 𝑓(𝑥𝑖−4 , 𝑦𝑖−4 ))
ℎ
𝑤(𝑖) = 𝑤(𝑖 − 1) + ( ) × (55𝑓𝑣1 − 59𝑓𝑣2 + 37𝑓𝑣3 − 9𝑓𝑣4)
24

end for
Step 6 Print the output: 𝑊 = [𝑤0 , 𝑤1 , ⋯ , 𝑤𝑚 ]𝑇
STOP.

Problem 13: Write down a C++ program to solve the IVP, 𝑦 ′ = 4𝑦 + 4𝑥 2 + 3𝑥, for 0 ≤ 𝑥 ≤ 1, with
initial condition 𝑦(0) = 𝛼 = 1/2, using the Adams-Bashforth method of order 4. Compute the
solution for 10 steps. For computing the approximate solution at the first three steps, use the RK4
method. At each step, compare the approximate solution with the exact solution, to be obtained by
5 5 13
𝑦(𝑥) = −𝑥 2 − 𝑥 − + 𝑒 4𝑥 , by finding the relative error between the two solutions.
4 16 16

1 #include <iostream>
2 #include <cmath>
3 #include <iomanip>
4 using namespace std ;
5
6 #define a 0.0 // starting point of domain
7 #define b 1.0 // ending point of domain
Numerical Solution of Ordinary Differential Equations (ODEs) 147

8 #define alpha 0.5 // initial condition

9 #define m 10 // number of steps
10
11 #define fval(x,y) 4*y+4*x*x+3*x
12 #define fexact(x) -x*x-1.2*x-(5.0/16.0)+(13.0/16.0)*exp(4*x)
13
14 void adamsb4(double [], double [], double) ;
15 void rk4(double [], double [], double) ;
16
17 int main()
18 {
19 double h ; // step size
20 double sol , err ;
21
22 int i ;
23 double x[m+1] ; // vector of nodes
24 double w[m+1] ; // vector of solution values
25
26 h = (b-a)/m ;
27
28 // forming vector of x-values or nodes
29 x[0] = a ;
30 x[m] = b ;
31
32 for (i=1 ; i<=m ; i++)
33 {
34 x[i] = x[i-1] + h ; // x[i] = x[0] + i*h ;
35 }
36
37 w[0]= alpha ; // setting initial condition
38
39 rk4(x, w, h) ; //Call-by-reference to function RK4
40 adamsb4(x, w, h) ; // Call-by-reference to function adamsb4
41
42 // ----------------- Printing Solutions ----------------- //
43 for (i=0 ; i<=m ; i++)
44 {
45 cout << "Node= " << setw(2) << i ;
46 cout << "\tx= " << setw(8) << x[i] ;
47 cout << "\tw= " << setw(8) << w[i] ;
48 sol = fexact(x[i]) ;
148 Simplified Numerical Analysis

49 cout << "\tExact sol= " << setw(8) << sol ;

50 err = fabs(sol-w[i])/fabs(sol) ;
51 cout << "\tRelative Error= " << err << endl ;
52 }
53
54 return 0;
55 }
56
57 // ---- User-defined function for Adams-Bashforth method ---- //
58
59 void adamsb4(double x[m+1], double w[m+1], double h)
60 {
61 double k1 , k2 , k3 , k4 , fv ;
62 double p , q ;
63 int i ;
64
65 for (i=4 ; i<=m ; i++)
66 {
67 k1 = fval( x[i-1] , w[i-1] );
68 k2 = fval( x[i-2] , w[i-2] );
69 k3 = fval( x[i-3] , w[i-3] );
70 k4 = fval( x[i-4] , w[i-4] );
71
72 w[i] = w[i-1] + (h/24.0)*(55*k1 - 59*k2 + 37*k3 - 9*k4) ;
73 }
74 }
75
76 // ----- User-defined function for the RK 4----- //
77
78 void rk4(double x[m+1], double w[m+1], double h)
79 {
80 double k1 , k2 , k3 , k4, fv ;
81 int i ;
82 double p , q ;
83
84 for (i=1 ; i<=3 ; i++)
85 {
86 k1 = h*(fval( x[i-1] , w[i-1] ));
87 k2 = h*(fval( (x[i-1]+(h/2.0)) , (w[i-1]+(k1/2.0) )));
88 k3 = h*(fval( (x[i-1]+(h/2.0)) , (w[i-1]+(k2/2.0) )));
89 k4 = h*(fval( x[i] , (w[i-1]+k3) ));
 Numerical Solution of Ordinary Differential Equations (ODEs) 149

90
91 w[i] = w[i-1] + (k1+2*k2+2*k3+k4)/6.0 ;
92 }
93 }

The results are shown in the following table.

Steps Numerical Exact solution Error of computer

Node 𝑥(𝑖) solution 𝑤𝑖 =
𝑖 𝑦𝑖 = 𝑦(𝑥𝑖 ) program solution
𝑤(𝑥𝑖 )
0 0 0.5 0.5 0
1 0.1 0.764547 0.769608 0.00657595
2 0.2 1.20556 1.21575 0.00838021
3 0.3 1.91966 1.93509 0.00797517
4 0.4 3.04469 3.07184 0.00883945
5 0.5 4.79306 4.84111 0.00992469
6 0.6 7.48205 7.56383 0.0108119
7 0.7 11.5814 11.7188 0.011726
8 0.8 17.7896 18.0202 0.0127952
9 0.9 27.1477 27.5336 0.0140137
10 1.0 41.2059 41.8485 0.0153558

Question 21: Write down an algorithm (pseudo code) to solve a first-order ODE using the Adams-
Bashforth-Moulton method of order 4.

Algorithm: To solve 𝑦 ′ = 𝑓(𝑥, 𝑦) , for 𝑎 ≤ 𝑥 ≤ 𝑏 and 𝑦(𝑎) = 𝛼 by approximating 𝑦 = 𝑦(𝑥) at

(𝑚 + 1) equispaced nodes 𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑚 , such that 𝑎 = 𝑥0 < 𝑥1 < 𝑥2 < ⋯ < 𝑥𝑚 = 𝑏, ℎ = (𝑏 −
𝑎)/𝑚 and 𝑦(𝑥𝑖 ) = 𝑦𝑖 . Having 𝑦(𝑥0 ) = 𝛼0 , 𝑦(𝑥1 ) = 𝛼1 , 𝑦(𝑥2 ) = 𝛼2 , and 𝑦(𝑥3 ) = 𝛼3 , compute 𝑦𝑖
using

(1) the 4-step explicit Adams-Bashforth method of order 4 as the predictor:

ℎ
𝑦𝑖 = 𝑦𝑖−1 + × [55𝑓(𝑥𝑖−1 , 𝑦𝑖−1 ) − 59𝑓(𝑥𝑖−2 , 𝑦𝑖−2 ) + 37𝑓(𝑥𝑖−3 , 𝑦𝑖−3 ) − 9𝑓(𝑥𝑖−4 , 𝑦𝑖−4 )]
24
(2) the 3-step implicit Adams-Moulton method of order 4 as the corrector:
ℎ
𝑦𝑖 = 𝑦𝑖−1 + × [9𝑓(𝑥𝑖 , 𝑦𝑖 ) + 19𝑓(𝑥𝑖−1 , 𝑦𝑖−1 ) − 5𝑓(𝑥𝑖−2 , 𝑦𝑖−2 ) + 𝑓(𝑥𝑖−3 , 𝑦𝑖−3 )]
24
for 𝑖 = 4, 5, 6, ⋯ , 𝑚.
 150 Simplified Numerical Analysis

𝑎, 𝑏: real values as the endpoints of the interval: 𝑥 ∈ [𝑎, 𝑏]

𝑚: an integer as the number of nodes (other than 𝑎) in the 𝑥 direction
𝐈𝐍𝐏𝐔𝐓𝐒:
𝑎𝑙𝑝ℎ𝑎: a real value as the initial condition 𝑦(𝑎)
{A definition of the function 𝑓(𝑥, 𝑦) in an appropriate way

𝑊 = [𝑤0 , 𝑤1 , ⋯ , 𝑤𝑚 ]𝑇 : a real valued vector as the approximate solution 𝑦(𝑥𝑖 )

𝐎𝐔𝐓𝐏𝐔𝐓: {
at the nodes 𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑚

ℎ: a real value as the step length in 𝑥 direction such that ℎ = (𝑏 − 𝑎)/𝑚

Auxiliary Variables: {
𝑋 = (𝑥𝑖 ) for 𝑖 = 0, 1, ⋯ , 𝑚: a real valued vector to represent 𝑥𝑖 𝑠

Step 1 Receive the inputs as stated above

Step 2 Set ℎ = (𝑏 − 𝑎)/𝑚
Set 𝑥(0) = 𝑎
Set 𝑥(𝑚) = 𝑏
Step 3 for 𝑖 = 1, 2, ⋯ , 𝑚 − 1
Set 𝑥(𝑖) = 𝑥(0) + 𝑖 × ℎ (Constructing interior mesh points, 𝑥𝑖 )
end for
Step 4 Set 𝑤(0) = 𝑎𝑙𝑝ℎ𝑎 (Setting the initial condition)
Step 5 Obtain or compute (using some other basic method for ODEs) the following:
𝑤(1) = 𝑎𝑙𝑝ℎ𝑎1
𝑤(2) = 𝑎𝑙𝑝ℎ𝑎2
𝑤(3) = 𝑎𝑙𝑝ℎ𝑎3

Step 5 for 𝑖 = 4, 5, 6, ⋯ , 𝑚
𝑓𝑣1 = 𝑓(𝑥(𝑖 − 1), 𝑤(𝑖 − 1)) (Computing the value 𝑓(𝑥𝑖−1 , 𝑦𝑖−1 ))
𝑓𝑣2 = 𝑓(𝑥(𝑖 − 2), 𝑤(𝑖 − 2)) (Computing the value 𝑓(𝑥𝑖−2 , 𝑦𝑖−2 ))
𝑓𝑣3 = 𝑓(𝑥(𝑖 − 3), 𝑤(𝑖 − 3)) (Computing the value 𝑓(𝑥𝑖−3 , 𝑦𝑖−3 ))
𝑓𝑣4 = 𝑓(𝑥(𝑖 − 4), 𝑤(𝑖 − 4)) (Computing the value 𝑓(𝑥𝑖−4 , 𝑦𝑖−4 ))
ℎ
𝑤(𝑖) = 𝑤(𝑖 − 1) + ( ) × (55𝑓𝑣1 − 59𝑓𝑣2 + 37𝑓𝑣3 − 9𝑓𝑣4)
24

𝑓𝑣 = 𝑓(𝑥(𝑖), 𝑤(𝑖)) (Computing the value 𝑓(𝑥𝑖−4 , 𝑦𝑖−4 ))

ℎ
𝑤(𝑖) = 𝑤(𝑖 − 1) + ( ) × (9𝑓𝑣 + 19𝑓𝑣1 − 5𝑓𝑣2 + 𝑓𝑣3)
24

end for
Step 6 Print the output: 𝑊 = [𝑤0 , 𝑤1 , ⋯ , 𝑤𝑚 ]𝑇
STOP.
 Numerical Solution of Ordinary Differential Equations (ODEs) 151

Problem 14: Write down a C++ program to solve the IVP, 𝑦 ′ = 4𝑦 + 4𝑥 2 + 3𝑥, for 0 ≤ 𝑥 ≤ 1, with
initial condition 𝑦(0) = 𝛼 = 1/2, using the Adams-Bashforth-Moulton method of order 4. Compute
the solution for 10 steps. For computing the approximate solution at the first three steps, use the
RK4 method. At each step, compare the approximate solution with the exact solution, to be obtained
5 5 13
by 𝑦(𝑥) = −𝑥 2 − 4 𝑥 − 16 + 16 𝑒 4𝑥 , by finding the relative error between the two solutions.

1 #include <iostream>
2 #include <cmath>
3 #include <iomanip>
4 using namespace std ;
5
6 #define a 0.0 // starting point of domain
7 #define b 1.0 // ending point of domain
8 #define alpha 0.5 // initial condition
9 #define m 10 // number of steps
10
11 #define fval(x,y) 4*y+4*x*x+3*x
12 #define fexact(x) -x*x-1.2*x-(5.0/16.0)+(13.0/16.0)*exp(4*x)
13
14 void adamsb4m3(double [], double [], double) ;
15 void rk4(double [], double [], double) ;
16
17 int main()
18 {
19 double h ; // step size
20 double sol , err ;
21
22 int i ;
23 double x[m+1] ; // vector of nodes
24 double w[m+1] ; // vector of solution values
25
26 h = (b-a)/m ;
27
28 // forming vector of x-values or nodes
29 x[0] = a ;
30 x[m] = b ;
31
32 for (i=1 ; i<=m ; i++)
33 {
34 x[i] = x[i-1] + h ; // x[i] = x[0] + i*h ;
35 }
152 Simplified Numerical Analysis

36
37 w[0]= alpha ; // setting initial condition
38
39 rk4(x, w, h) ; //Call-by-reference to function RK4
40 adamsb4m3(x, w, h) ; // Call-by-reference to function adamsb4
41
42 // ----------------- Printing Solutions ----------------- //
43 for (i=0 ; i<=m ; i++)
44 {
45 cout << "Node= " << setw(2) << i ;
46 cout << "\tx= " << setw(8) << x[i] ;
47 cout << "\tw= " << setw(8) << w[i] ;
48 sol = fexact(x[i]) ;
49 cout << "\tExact sol= " << setw(8) << sol ;
50 err = fabs(sol-w[i])/fabs(sol) ;
51 cout << "\tRelative Error= " << err << endl ;
52 }
53
54 return 0;
55 }
56
57 // ---- User-defined function for Adams-Bashforth-Moulten method ---- //
58
59 void adamsb4m3(double x[m+1], double w[m+1], double h)
60 {
61 double fv1 , fv2 , fv3 , fv4 , fv ;
62 double p , q ;
63 int i ;
64
65 for (i=4 ; i<=m ; i++)
66 {
67 fv1 = fval( x[i-1] , w[i-1] );
68 fv2 = fval( x[i-2] , w[i-2] );
69 fv3 = fval( x[i-3] , w[i-3] );
70 fv4 = fval( x[i-4] , w[i-4] );
71
72 w[i] = w[i-1] + (h/24.0)*(55*fv1 - 59*fv2 + 37*fv3 - 9*fv4) ;
73
74 fv = fval( x[i] , w[i]);
75
76 w[i] = w[i-1] + (h/24.0)*(9*fv + 19*fv1 - 5*fv2 + fv3);
 Numerical Solution of Ordinary Differential Equations (ODEs) 153

77 }
78 }
79
80 // ----- User-defined function for the RK 4----- //
81
82 void rk4(double x[m+1], double w[m+1], double h)
83 {
84 double k1 , k2 , k3 , k4, fv ;
85 int i ;
86 double p , q ;
87
88 for (i=1 ; i<=3 ; i++)
89 {
90 k1 = h*(fval( x[i-1] , w[i-1] ));
91 k2 = h*(fval( (x[i-1]+(h/2.0)) , (w[i-1]+(k1/2.0) )));
92 k3 = h*(fval( (x[i-1]+(h/2.0)) , (w[i-1]+(k2/2.0) )));
93 k4 = h*(fval( x[i] , (w[i-1]+k3) ));
94
95 w[i] = w[i-1] + (k1+2*k2+2*k3+k4)/6.0 ;
96 }
97 }

The results are shown in the following table.

Numerical
Steps Exact solution Error of computer
Node 𝑥(𝑖) solution 𝑤𝑖 =
𝑖 𝑦𝑖 = 𝑦(𝑥𝑖 ) program solution
𝑤(𝑥𝑖 )
0 0 0.5 0.5 0
1 0.1 0.764547 0.769608 0.00657595
2 0.2 1.20556 1.21575 0.00838021
3 0.3 1.91966 1.93509 0.00797517
4 0.4 3.05087 3.07184 0.00682605
5 0.5 4.81417 4.84111 0.00556428
6 0.6 7.53021 7.56383 0.0044448
7 0.7 11.6773 11.7188 0.00354036
8 0.8 17.9689 18.0202 0.00284706
9 0.9 27.4693 27.5336 0.00233489
10 1.0 41.7661 41.8485 0.00196874
∎
 154 Simplified Numerical Analysis

Problem 15: Write a C++ program to solve the following system of two ODEs for the functions 𝑦1 =
𝑦1 (𝑥) and 𝑦2 = 𝑦2 (𝑥), where 𝑥 ∈ [0,1]:
𝑦1′ = 𝑦1 𝑦2 − 2
𝑦2′ = 2𝑦1 − 𝑦23
With initial conditions:
𝑦1 (0) = 2.0
𝑦2 (0) = 0.3
Use the RK4 method of order 4 for 5 steps.
For 5 steps the domain is discretized as
𝑏 − 𝑎 1.0 − 0.0
ℎ= = = 0.2
𝑚 5
𝑥0 = 0, 𝑥1 = 0.2, 𝑥2 = 0.4, 𝑥3 = 0.6, 𝑥4 = 0.8, 𝑥5 = 1.0.
According to the initial conditions:
𝑤10 = 𝑦10 = 𝑦1 (𝑥0 ) = 𝑦1 (0) = 2.0
𝑤20 = 𝑦20 = 𝑦2 (𝑥0 ) = 𝑦2 (0) = 0.3
The problem is to find approximations 𝑤1𝑖 to 𝑦1𝑖 = 𝑦1 (𝑥𝑖 ) and 𝑤2𝑖 to 𝑦2𝑖 = 𝑦2 (𝑥𝑖 ), for 𝑖 = 1,2,3,4,5.

The C++ program for the solution is as follows.

1 #include <iostream>
2 #include <cmath>
3 #include <iomanip>
4 using namespace std ;
5
6 #define a 0.0 // starting point of domain
7 #define b 1.0 // ending point of domain
8 #define alpha1 2.0 //initial condition for first variable y1=y1(x)
9 #define alpha2 0.3 // initial condition for second variable y2=y2(x)
10 #define m 5 // number of steps
11
12 #define f1(x,y1,y2) y1*y2 - 2
13 #define f2(x,y1,y2) 2*y1 - y2*y2*y2
14
16 void rk4system2(double [], double [], double[], double) ;
17
18 int main()
19 {
20 double h ; // step size
21 double sol , err ;
Numerical Solution of Ordinary Differential Equations (ODEs) 155

22
23 int i ;
24 double x[m+1] ; // vector of nodes
25 double w1[m+1] ; // vector of solution values for y1=y1(x)
26 double w2[m+1] ; // vector of solution values for y2=y2(x)
27
28 h = (b-a)/m ;
29
30 // forming vector of x-values or nodes
31 x[0] = a ;
32 x[m] = b ;
33
34 for (i=1 ; i<=m ; i++)
35 {
36 x[i] = x[i-1] + h ; // x[i] = x[0] + i*h ;
37 }
38
39 w1[0] = alpha1 ; // setting initial condition for y1
40 w2[0] = alpha2 ; // setting initial condition for y2
41
42 // Call-by-reference to the solver of ODEs system of 2 equations
43 rk4system2(x , w1 , w2 , h) ;
44
45 // ----------------- Printing Solutions ----------------- //
46 for (i=0 ; i<=m ; i++)
47 {
48 cout<< "Node=" << setw(3) << i ;
49 cout << "\tx= " << setw(8) << x[i] ;
50 cout<< "\tw1=" << setw(8) << w1[i] ;
51 cout<< "\tw2=" << setw(8) << w2[i] << endl ;
52 }
53
54 return 0;
55 }
56
57 // --- User-defined function for ODE system of 2 equations using RK4 --- //
58
59 void rk4system2(double x[m+1], double w1[m+1], double w2[m+1], double h)
60 {
61 double k11, k12, k13, k14 ;
62 double k21, k22, k23, k24 ;
63 int i ;
64
65 for (i=1 ; i<=m ; i++)
66 {
 156 Simplified Numerical Analysis

67 k11 = h * (f1(x[i-1], w1[i-1], w2[i-1])) ;

68 k21 = h * (f2(x[i-1], w1[i-1], w2[i-1])) ;
69
70 k12 = h*(f1((x[i-1]+0.5*h), (w1[i-1]+0.5*k11), (w2[i-1]+0.5*k21))) ;
71 k22 = h * (f2((x[i-1]+0.5*h), (w1[i-1]+0.5*k11), (w2[i-1]+0.5*k21))) ;
72
73 k13 = h * (f1((x[i-1]+0.5*h), (w1[i-1]+0.5*k12), (w2[i-1]+0.5*k22))) ;
74 k23 = h * (f2((x[i-1]+0.5*h), (w1[i-1]+0.5*k12), (w2[i-1]+0.5*k22))) ;
75
76 k14 = h * (f1((x[i-1]+h), (w1[i-1]+k13), (w2[i-1]+k23))) ;
77 k24 = h * (f2((x[i-1]+h), (w1[i-1]+k13), (w2[i-1]+k23))) ;
78
79 w1[i] = w1[i-1] + (k11 + 2*k12 + 2*k13 + k14)/6.0 ;
80 w2[i] = w2[i-1] + (k21 + 2*k22 + 2*k23 + k24)/6.0 ;
81 }
82 }

The results are shown in the following table.

Numerical Numerical
Steps
Node 𝑥(𝑖) solution solution
𝑖
𝑤1𝑖 = 𝑤1 (𝑥𝑖 ) 𝑤2𝑖 = 𝑤2 (𝑥𝑖 )
0 0.0 2 0.3
1 0.2 1.815132 0.985522
2 0.4 1.90079 1.36485
3 0.6 2.08065 1.52571
4 0.8 2.38251 1.62306
5 1.0 2.85383 1.72393
∎

Problem 16: Write a C++ program to solve the following system of two ODEs for the functions 𝑦1 =
𝑦1 (𝑥), 𝑦2 = 𝑦2 (𝑥), and 𝑦3 = 𝑦3 (𝑥), where 𝑥 ∈ [0,1]:

𝑦1′ = 𝑦1 + 3𝑦2 − 3𝑦3 + 𝑒 −𝑥

𝑦2′ = 2𝑦2 + 𝑦3 − 3𝑒 −𝑥
𝑦3′ = 𝑦1 + 2𝑦2 + 𝑒 −𝑥
With initial conditions:
𝑦1 (0) = 2.5
𝑦2 (0) = −1.5
𝑦3 (0) = −1.0
 Numerical Solution of Ordinary Differential Equations (ODEs) 157

Use the RK4 method of order 4 for 10 steps.

For 10 steps, the domain is discretized as
𝑏 − 𝑎 1.0 − 0.0
ℎ= = = 0.1
𝑚 10
𝑥0 = 0, 𝑥1 = 0.1, 𝑥2 = 0.2, 𝑥3 = 0.3, 𝑥4 = 0.4, 𝑥5 = 0.5, 𝑥6 = 0.6, 𝑥7 = 0.7, 𝑥8 = 0.8, 𝑥9 = 0.9, 𝑥10 = 1.0.

According to the initial conditions:

𝑤10 = 𝑦10 = 𝑦1 (𝑥0 ) = 𝑦1 (0) = 2.5
𝑤20 = 𝑦20 = 𝑦2 (𝑥0 ) = 𝑦2 (0) = −1.5
𝑤30 = 𝑦30 = 𝑦3 (𝑥0 ) = 𝑦3 (0) = −1.0

The problem is to find approximations 𝑤1𝑖 to 𝑦1𝑖 = 𝑦1 (𝑥𝑖 ), 𝑤2𝑖 to 𝑦2𝑖 = 𝑦2 (𝑥𝑖 ), and 𝑤3𝑖 to 𝑦3𝑖 =
𝑦3 (𝑥𝑖 ), for 𝑖 = 1,2, ⋯ , 10.

The C++ program for the solution is as follows.

1 #include <iostream>
2 #include <cmath>
3 #include <iomanip>
4 using namespace std ;
5
6 #define a 0.0 // starting point of domain
7 #define b 1.0 // ending point of domain
8 #define alpha1 2.5 //initial condition for first variable y1=y1(x)
9 #define alpha2 -1.5 // initial condition for second variable y2=y2(x)
10 #define alpha3 -1.0 // initial condition for third variable y3=y3(x)
11 #define m 5 // number of steps
12
13 #define f1(x,y1,y2,y3) y1 + 3*y2 - 3*y3 + exp(-x)
13 #define f2(x,y1,y2,y3) 2*y2 + y3 - 3*exp(-x)
14 #define f3(x,y1,y2,y3) y1 + 2*y2 + exp(-x)
16
17 void rk4system3(double [], double [], double[], double [], double) ;
18
19 int main()
20 {
21 double h ; // step size
22 double sol , err ;
23
24 int i ;
25 double x[m+1] ; // vector of nodes
158 Simplified Numerical Analysis

26 double w1[m+1] ; // vector of solution values for y1=y1(x)

27 double w2[m+1] ; // vector of solution values for y2=y2(x)
28 double w3[m+1] ; // vector of solution values for y3=y3(x)
29
30 h = (b-a)/m ;
31
32 // forming vector of x-values or nodes
33 x[0] = a ;
34 x[m] = b ;
35
36 for (i=1 ; i<=m ; i++)
37 {
38 x[i] = x[i-1] + h ; // x[i] = x[0] + i*h ;
39 }
40
41 w1[0] = alpha1 ; // setting initial condition for y1
42 w2[0] = alpha2 ; // setting initial condition for y2
43 w3[0] = alpha3 ; // setting initial condition for y3
44
45 // Call-by-reference to the solver of ODEs system of 3 equations
46 rk4system3(x , w1 , w2 , w3 , h) ;
47
48 // ----------------- Printing Solutions ----------------- //
49 for (i=0 ; i<=m ; i++)
50 {
48 cout<< "Node=" << setw(3) << i ;
49 cout << "\tx= " << setw(8) << x[i] ;
50 cout<< "\tw1=" << setw(8) << w1[i] ;
51 cout<< "\tw2=" << setw(8) << w2[i] ;
52 cout<< "\tw3=" << setw(8) << w3[i] << endl ;
53 }
54
55 return 0;
56 }
57
58 // --- User-defined function for ODE system of 3 equations using RK4 --- //
59
60 void rk4system3(double x[m+1], double w1[m+1], double w2[m+1],
double w3[m+1], double h)
61 {
62 double k11, k12, k13, k14 ;
63 double k21, k22, k23, k24 ;
64 double k31, k32, k33, k34 ;
65 int i ;
66
 Numerical Solution of Ordinary Differential Equations (ODEs) 159

67 for (i=1 ; i<=m ; i++)

68 {
69 k11 = h * (f1(x[i-1], w1[i-1], w2[i-1], w3[i-1])) ;
70 k21 = h * (f2(x[i-1], w1[i-1], w2[i-1], w3[i-1])) ;
71 k31 = h * (f3(x[i-1], w1[i-1], w2[i-1], w3[i-1])) ;
72
73 k12 = h * (f1((x[i-1]+0.5*h), (w1[i-1]+0.5*k11),
74 (w2[i-1]+0.5*k21), (w3[i-1]+0.5*k31))) ;
75 k22 = h * (f2((x[i-1]+0.5*h), (w1[i-1]+0.5*k11),
76 (w2[i-1]+0.5*k21), (w3[i-1]+0.5*k31))) ;
77 k32 = h * (f3((x[i-1]+0.5*h), (w1[i-1]+0.5*k11),
78 (w2[i-1]+0.5*k21), (w3[i-1]+0.5*k31))) ;
79
80 k13 = h * (f1((x[i-1]+0.5*h), (w1[i-1]+0.5*k12),
81 (w2[i-1]+0.5*k22), (w3[i-1]+0.5*k32))) ;
82 k23 = h * (f2((x[i-1]+0.5*h), (w1[i-1]+0.5*k12),
83 (w2[i-1]+0.5*k22), (w3[i-1]+0.5*k32))) ;
84 k33 = h * (f3((x[i-1]+0.5*h), (w1[i-1]+0.5*k12),
85 (w2[i-1]+0.5*k22), (w3[i-1]+0.5*k32))) ;
86
87 k14 = h * (f1((x[i-1]+h), (w1[i-1]+k13),
88 (w2[i-1]+k23), (w3[i-1]+k33))) ;
89 k24 = h * (f2((x[i-1]+h), (w1[i-1]+k13),
90 (w2[i-1]+k23), (w3[i-1]+k33))) ;
91 k34 = h * (f3((x[i-1]+h), (w1[i-1]+k13),
92 (w2[i-1]+k23), (w3[i-1]+k33))) ;
93
94 w1[i] = w1[i-1] + (k11 + 2*k12 + 2*k13 + k14)/6.0 ;
95 w2[i] = w2[i-1] + (k21 + 2*k22 + 2*k23 + k24)/6.0 ;
96 w3[i] = w3[i-1] + (k31 + 2*k32 + 2*k33 + k34)/6.0 ;
97 }
98 }
The results are shown in the following table.

Numerical Numerical Numerical

Steps
Node 𝑥(𝑖) solution solution solution
𝑖
𝑤1𝑖 = 𝑤1 (𝑥𝑖 ) 𝑤2𝑖 = 𝑤2 (𝑥𝑖 ) 𝑤3𝑖 = 𝑤3 (𝑥𝑖 )
0 0.0 2.5 -1.5 -1.0
1 0.2 2.45262 -3.16688 -1.22125
2 0.4 1.43325 -5.68432 -2.39815
3 0.6 -0.713292 -9.79838 -5.20827
4 0.8 -4.26124 -16.8302 -10.7742
5 1.0 -9.74139 -29.1044 -21.007
 160 Simplified Numerical Analysis

Problem 17: Write a C++ program to find the numerical solution of the ODE, 𝑥𝑦 ′′ − 𝑦 ′ + 8𝑥 3 𝑦 3 =
0 with initial condition 𝑦(1) = 0.5 and 𝑦 ′ (1) = −0.5 for 𝑦(1.1). Consider the step size of ℎ = 0.1,
thus only step is required. i.e., 𝑚 = 1. Use the exact solution, 𝑦 = 1⁄(1 + 𝑥 2 ), to find the error in
the numerical solution.

For the solution, consider

𝑦′ = 𝑧
Then, the given ODE becomes
(𝑧 − 8𝑥 3 𝑦 3 )
𝑧′ =
𝑥
Thus, the second-order IVP is essentially converted to the problem of a first-order system of ODEs
of comprising the two equations subject to the initial conditions:
𝑤10 = 𝑦0 = 𝑦(𝑥0 ) = 𝑦(1) = 0.5
𝑤20 = 𝑧0 = 𝑧(𝑥0 ) = 𝑧(1) = −0.5
The problem is to find approximations 𝑤11 to 𝑦1 = 𝑦(𝑥1 ) and 𝑤21 to 𝑧1 = 𝑧(𝑥1 ).
The C++ program for the solution is as follows.

1 #include <iostream>
2 #include <cmath>
3 #include <iomanip>
4 using namespace std ;
5
6 #define a 1.0 // starting point of domain
7 #define b 1.1 // ending point of domain
8 #define alpha1 0.5 //initial condition for first variable y1=y1(x)
9 #define alpha2 -0.5 // initial condition for second variable y2=y2(x)
10 #define m 5 // number of steps
11
12 #define f1(x,y,z) z
13 #define f2(x,y,z) (z-8*x*x*x*y*y*y)/x
14
16 void rk4system2(double [], double [], double[], double) ;
17
18 int main()
19 {
20 double h ; // step size
21 double sol , err ;
Numerical Solution of Ordinary Differential Equations (ODEs) 161

22
23 int i ;
24 double x[m+1] ; // vector of nodes
25 double w1[m+1] ; // vector of solution values for y1=y1(x)
26 double w2[m+1] ; // vector of solution values for z=y'
27
28 h = (b-a)/m ;
29
30 // forming vector of x-values or nodes
31 x[0] = a ;
32 x[m] = b ;
33
34 for (i=1 ; i<=m ; i++)
35 {
36 x[i] = x[i-1] + h ; // x[i] = x[0] + i*h ;
37 }
38
39 w1[0] = alpha1 ; // setting initial condition for y
40 w2[0] = alpha2 ; // setting initial condition for z
41
42 // Call-by-reference to the solver of ODEs system of 2 equations
43 rk4system2(x , w1 , w2 , h) ;
44
45 // ----------------- Printing Solutions ----------------- //
46 for (i=0 ; i<=m ; i++)
47 {
48 cout<< "Node=" << setw(3) << i ;
49 cout << "\tx= " << setw(8) << x[i] ;
50 cout<< "\tw1=" << setprecision(10) << setw(13) << w1[i] ;
51 cout<< "\tw2=" << setprecision(10) << setw(13) << w2[i] <<
endl ;
52 }
53
54 return 0;
55 }
56
57 // --- User-defined function for ODE system of 2 equations using RK4 --- //
58
59 void rk4system2(double x[m+1], double w1[m+1], double w2[m+1],
double h)
60 {
 162 Simplified Numerical Analysis

61 double k11, k12, k13, k14 ;

62 double k21, k22, k23, k24 ;
63 int i ;
64
65 for (i=1 ; i<=m ; i++)
66 {
67 k11 = h * (f1(x[i-1], w1[i-1], w2[i-1])) ;
68 k21 = h * (f2(x[i-1], w1[i-1], w2[i-1])) ;
69
70 k12 = h*(f1((x[i-1]+0.5*h), (w1[i-1]+0.5*k11),
71 (w2[i-1]+0.5*k21))) ;
72 k22 = h * (f2((x[i-1]+0.5*h), (w1[i-1]+0.5*k11),
73 (w2[i-1]+0.5*k21))) ;
74 k13 = h * (f1((x[i-1]+0.5*h), (w1[i-1]+0.5*k12),
75 (w2[i-1]+0.5*k22))) ;
76
77 k14 = h * (f1((x[i-1]+h), (w1[i-1]+k13), (w2[i-1]+k23))) ;
78 k24 = h * (f2((x[i-1]+h), (w1[i-1]+k13), (w2[i-1]+k23))) ;
79
80 w1[i] = w1[i-1] + (k11 + 2*k12 + 2*k13 + k14)/6.0 ;
81 w2[i] = w2[i-1] + (k21 + 2*k22 + 2*k23 + k24)/6.0 ;
82 }
83 }

The results are shown in the following table.

Numerical Numerical
Steps
Node 𝑥(𝑖) solution solution
𝑖
𝑤1𝑖 = 𝑤1 (𝑥𝑖 ) 𝑤2𝑖 = 𝑤2 (𝑥𝑖 )

0 1 0.5 -0.5
1 1.02 0.4897000998 -0.5299800347
2 1.04 0.4788015942 -0.5598407786
3 1.06 0.4673080456 -0.5894651108
4 1.08 0.4552253218 -0.618739865
5 1.1 0.4425615 -0.6475575603

∎
 Numerical Solution of Ordinary Differential Equations (ODEs) 163

Problem 18: Solve the ODE 𝑦 ′′′ = −𝑦 ′′ + 3𝑦 ′ + 3𝑦 for 𝑦 = 𝑦(𝑥) in 𝑥 ∈ [0,2] with the initial
conditions: 𝑦(0) = 2.0, 𝑦 ′ (0) = −1.0, and 𝑦 ′′ (0) = 8.0. Solve it for 10 steps.
Given the equation,
𝑦 ′′′ = −𝑦 ′′ + 3𝑦 ′ + 3𝑦 − − −(1)
For 𝑦 = 𝑦(𝑥) in 𝑥 ∈ [0,2] with the initial conditions:
𝑦(0) = 2.0
𝑦 ′ (0) = −1.0
′′ (0)
𝑦 = 8.0
consider
𝑦′ = 𝑧1 − − −(2)
𝑦 ′′ = 𝑧1′ = 𝑧2 − − −(3)
Then, the given third-order Eq. (1) becomes
𝑧2′ = −𝑧2 + 3𝑧1 + 3𝑦 − − −(4)
Thus, the third-order IVP is essentially converted to the problem of a first-order system of ODEs of
comprising the three equations (2) - (4) subject to the initial conditions:
𝑦(0) = 2.0
𝑧1 (0) = −1.0
𝑧2 (0) = 8.0
For 10 steps, the domain is discretized as
𝑏−𝑎 2.0 − 0.0
ℎ = = = 0.2
𝑚 10
𝑥0 = 0, 𝑥1 = 0.2, 𝑥2 = 0.4, 𝑥3 = 0.6, 𝑥4 = 0.8, 𝑥5 = 1.0, 𝑥6 = 1.2, 𝑥7 = 1.4, 𝑥8 = 1.6, 𝑥9 = 1.8, 𝑥10 =
2.0.
According to the initial conditions:
𝑤10 = 𝑦0 = 𝑦(𝑥0 ) = 𝑦(0) = 2.0
𝑤20 = 𝑧10 = 𝑧1 (𝑥0 ) = 𝑧1 (0) = −1.0
𝑤30 = 𝑧20 = 𝑧2 (𝑥0 ) = 𝑧2 (0) = 8.0
The problem is to find approximations 𝑤1𝑖 to 𝑦𝑖 = 𝑦(𝑥𝑖 ) , 𝑤2𝑖 to 𝑧1𝑖 = 𝑧1 (𝑥𝑖 ) , and 𝑤3𝑖 to 𝑧2𝑖 =
𝑧2 (𝑥𝑖 ), for 𝑖 = 1,2, ⋯ , 10.

The C++ program for the solution is as follows.

164 Simplified Numerical Analysis

1 #include <iostream>
2 #include <cmath>
3 #include <iomanip>
4 using namespace std ;
5
6 #define a 0.0 // starting point of domain
7 #define b 2.0 // ending point of domain
8 #define alpha1 2.0 // initial condition: function value
9 #define alpha2 -1.0 // initial condition: first derivative value
10 #define alpha3 8.0 // initial condition: second derivative value
11 #define m 5 // number of steps
12
13 #define f1(x,y,z1,z2) z1
13 #define f2(x,y,z1,z2) z2
14 #define f3(x,y,z1,z2) -z2 + 3*z1 + 3*y
16
17 void rk4system3(double [], double [], double[], double [], double) ;
18
19 int main()
20 {
21 double h ; // step size
22 double sol , err ;
23
24 int i ;
25 double x[m+1] ; // vector of nodes
26 double w1[m+1] ; // vector of solution values for y=y(x)
27 double w2[m+1] ; // vector of solution values for z1 = y'
28 double w3[m+1] ; // vector of solution values for z2 = y"
29
30 h = (b-a)/m ;
31
32 // forming vector of x-values or nodes
33 x[0] = a ;
34 x[m] = b ;
35
36 for (i=1 ; i<=m ; i++)
37 {
38 x[i] = x[i-1] + h ; // x[i] = x[0] + i*h ;
39 }
40
41 w1[0] = alpha1 ; // setting initial condition for y
42 w2[0] = alpha2 ; // setting initial condition for z1
43 w3[0] = alpha3 ; // setting initial condition for z2
44
45 // Call-by-reference to the solver of ODEs system of 3 equations
Numerical Solution of Ordinary Differential Equations (ODEs) 165

46 rk4system3(x , w1 , w2 , w3 , h) ;
47
48 // ----------------- Printing Solutions ----------------- //
49 for (i=0 ; i<=m ; i++)
50 {
48 cout<< "Node=" << setw(3) << i ;
49 cout << "\tx= " << setw(8) << x[i] ;
50 cout<< "\tw1=" << setprecision(10) << setw(13) << w1[i] ;
51 cout<< "\tw2=" << setprecision(10) << setw(13) << w2[i] ;
52 cout<< "\tw3=" << setprecision(10) << setw(13) << w3[i] <<
endl ;
53 }
54
55 return 0;
56 }
57
58 // --- User-defined function for ODE system of 3 equations using RK4 --- //
59
60 void rk4system3(double x[m+1], double w1[m+1],
61 double w2[m+1], double w3[m+1], double h)
62 {
63 double k11, k12, k13, k14 ;
64 double k21, k22, k23, k24 ;
65 double k31, k32, k33, k34 ;
66 int i ;
67
68 for (i=1 ; i<=m ; i++)
69 {
70 k11 = h * (f1(x[i-1], w1[i-1], w2[i-1], w3[i-1])) ;
71 k21 = h * (f2(x[i-1], w1[i-1], w2[i-1], w3[i-1])) ;
72 k31 = h * (f3(x[i-1], w1[i-1], w2[i-1], w3[i-1])) ;
73
74 k12 = h * (f1((x[i-1]+0.5*h), (w1[i-1]+0.5*k11),
75 (w2[i-1]+0.5*k21), (w3[i-1]+0.5*k31))) ;
76
77 k22 = h * (f2((x[i-1]+0.5*h), (w1[i-1]+0.5*k11),
77 (w2[i-1]+0.5*k21), (w3[i-1]+0.5*k31))) ;
78 k32 = h * (f3((x[i-1]+0.5*h), (w1[i-1]+0.5*k11),
79 (w2[i-1]+0.5*k21), (w3[i-1]+0.5*k31))) ;
80
81 k13 = h * (f1((x[i-1]+0.5*h), (w1[i-1]+0.5*k12),
82 (w2[i-1]+0.5*k22), (w3[i-1]+0.5*k32))) ;
83 k23 = h * (f2((x[i-1]+0.5*h), (w1[i-1]+0.5*k12),
84 (w2[i-1]+0.5*k22), (w3[i-1]+0.5*k32))) ;
85 k33 = h * (f3((x[i-1]+0.5*h), (w1[i-1]+0.5*k12),
 166 Simplified Numerical Analysis

86 (w2[i-1]+0.5*k22), (w3[i-1]+0.5*k32))) ;
87
88 k14 = h * (f1((x[i-1]+h), (w1[i-1]+k13),
89 (w2[i-1]+k23), (w3[i-1]+k33))) ;
90 k24 = h * (f2((x[i-1]+h), (w1[i-1]+k13),
91 (w2[i-1]+k23), (w3[i-1]+k33))) ;
92 k34 = h * (f3((x[i-1]+h), (w1[i-1]+k13),
93 (w2[i-1]+k23), (w3[i-1]+k33))) ;
94
95 w1[i] = w1[i-1] + (k11 + 2*k12 + 2*k13 + k14)/6.0 ;
96 w2[i] = w2[i-1] + (k21 + 2*k22 + 2*k23 + k24)/6.0 ;
97 w3[i] = w3[i-1] + (k31 + 2*k32 + 2*k33 + k34)/6.0 ;
98 }
99 }
∎

The results are shown in the following table.

Numerical Numerical Numerical

Steps
Node 𝑥(𝑖) solution solution solution
𝑖
𝑤1𝑖 = 𝑤1 (𝑥𝑖 ) 𝑤2𝑖 = 𝑤2 (𝑥𝑖 ) 𝑤3𝑖 = 𝑤3 (𝑥𝑖 )

0 0.0 2 -1 8
1 0.4 2.2144 2.0592 7.984
2 0.8 3.75537152 5.92358144 12.16498688
3 1.2 7.317856436 12.5913347 22.55617331
4 1.6 14.62815413 25.4339096 44.2884481
5 2.0 29.29652369 50.88508966 88.1604031
∎
 Numerical Solution of Ordinary Differential Equations (ODEs) 167

Question 22: Write down an algorithm (pseudo code) to solve a second-order linear ODE (BVP)
with Dirichlet boundary condition using the finite difference method of second-order accuracy. The
algorithm should follow the Gauss-Seidel approach to solve the linear system resulted after
discretization of the model equation.

Algorithm: To solve 𝑦′′ = 𝑓(𝑥, 𝑦, 𝑦′) = 𝑝(𝑥)𝑦′ + 𝑞(𝑥)𝑦 + 𝑟(𝑥) , for 𝑎 ≤ 𝑥 ≤ 𝑏 subject to the
Dirichlet boundary conditions: 𝑦(𝑎) = 𝛼 and 𝑦(𝑏) = 𝛽 by approximating 𝑦 = 𝑦(𝑥) at (𝑚 + 2)
equispaced nodes 𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑚 , 𝑥𝑚+1 , such that 𝑎 = 𝑥0 < 𝑥1 < 𝑥2 < ⋯ < 𝑥𝑚 < 𝑥𝑚+1 = 𝑏, ℎ =
(𝑏 − 𝑎)/𝑚 and 𝑦(𝑥𝑖 ) = 𝑦𝑖 using the finite difference method based on the central difference of
second-order accuracy.

𝑎, 𝑏: real values as the endpoints of the interval: 𝑥 ∈ [𝑎, 𝑏]

𝑚: an integer as the number of interior nodes in the 𝑥 direction
𝑎𝑙𝑝ℎ𝑎: a real value as the boundary condition 𝑦(𝑎)
𝐈𝐍𝐏𝐔𝐓𝐒: 𝑏𝑒𝑒𝑡𝑎: a real value as the boundary condition 𝑦(𝑏)
𝑁: an integer as the maximum number of iterations
𝑇𝑂𝐿: a real value as the error tolerance
{Definitions of the functions 𝑝(𝑥), 𝑞(𝑥), and 𝑟(𝑥) in an appropriate way

𝑍 = [𝑧0 , 𝑧1 , ⋯ , 𝑧𝑚 , 𝑧𝑚+1 ]𝑇 : a real valued vector as the approximate values of 𝑦(𝑥𝑖 )

𝐎𝐔𝐓𝐏𝐔𝐓: {
at the nodes 𝑥0 , 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑚 , 𝑥𝑚+1

ℎ: a real value as the step length in 𝑥 direction: ℎ = (𝑏 − 𝑎)/(𝑚 + 1)

𝑋 = (𝑥𝑖 ) for 𝑖 = 0, 1, ⋯ , 𝑚, 𝑚 + 1: a real valued vector to represent 𝑥𝑖 𝑠
𝑇
𝑍𝑃 = [𝑧𝑝0 , 𝑧𝑝1 , ⋯ , 𝑧𝑝𝑚 , 𝑧𝑝𝑚+1 ] : a real valued vector to keep a copy of 𝑍
𝑒𝑟𝑟: a real number to hold the value of error norm in each iteration
Auxiliary Variables:
𝐵 = [𝑏0 , 𝑏1 , ⋯ , 𝑏𝑚 ]𝑇 : a real valued vector to hold right hand side constants
𝐷 = [𝑑0 , 𝑑1 , ⋯ , 𝑑𝑚 ]𝑇 : a real valued vector to hold diagonal entries
𝑈 = [𝑢0 , 𝑢1 , ⋯ , 𝑢𝑚 ]𝑇 : a real valued vector to hold upper diagonal entries
{𝐿 = [𝑙0 , 𝑙1 , ⋯ , 𝑙𝑚 ]𝑇 : a real valued vector to hold lower diagonal entries

Step 1 Receive the inputs as stated above

Step 2 Set ℎ = (𝑏 − 𝑎)/(𝑚 + 1)

Set 𝑥(0) = 𝑎
Set 𝑥(𝑚 + 1) = 𝑏

Step 3 for 𝑖 = 1, 2, ⋯ , 𝑚
Set 𝑥(𝑖) = 𝑥(0) + 𝑖 × ℎ (Constructing interior mesh points, 𝑥𝑖 )
end for
 168 Simplified Numerical Analysis

Step 4 (Applying the boundary conditions)

Set 𝑤(0) = 𝑎𝑙𝑝ℎ𝑎
Set 𝑤(𝑚 + 1) = 𝑏𝑒𝑒𝑡𝑎

Step 5 (Setting the initial conditions on interior nodes)

for 𝑖 = 1, 2, ⋯ , 𝑚
Set 𝑤(𝑖) = 0 (Constructing interior mesh points, 𝑥𝑖 )
end for

Step 6 for 𝑖 = 1, 2, ⋯ , 𝑚
Set 𝐵(𝑖) = −ℎ × ℎ × 𝑟(𝑥(𝑖)); end for

for 𝑖 = 1, 2, ⋯ , 𝑚
Set 𝐷(𝑖) = 2 + ℎ × ℎ × 𝑞(𝑥(𝑖)) ; end for

for 𝑖 = 1, 2, ⋯ , 𝑚
Set 𝑈(𝑖) = −1 + ℎ × 0.5 × 𝑝(𝑥(𝑖)) ; end for

for 𝑖 = 1, 2, ⋯ , 𝑚
Set 𝐿(𝑖) = −1 − ℎ × 0.5 × 𝑝(𝑥(𝑖)) ; end for

Step 7 for 𝑘 = 1, 2, 3, ⋯ , 𝑁 perform steps 8-11

Step 8

for 𝑖 = 1, 2, ⋯ , 𝑚 Set 𝑍𝑃(𝑖) = 𝑊 (keeping a copy of 𝑍 in 𝑍𝑃 for taking the norm)

Step 9
for 𝑖 = 1, 2, ⋯ , 𝑚 (compute the components of solution vector 𝑍)
𝐵(𝑖) − 𝐿(𝑖) × 𝑤(𝑖 − 1) − 𝑈(𝑖) × 𝑤(𝑖 + 1)
𝑤(𝑖) =
𝐷(𝑖)
end for
Step 10 Compute 𝑒𝑟𝑟 = ‖𝑊 − 𝑍𝑃‖
(or 𝑒𝑟𝑟 = ‖𝑋 − 𝑋𝑃‖/‖𝑋‖) Here ‖∙‖ is any suitable norm.
Step 11
This means that the consecutive
if (𝑒𝑟𝑟 < 𝑇𝑂𝐿 )then
} approximations are nearly the same.
Exit/Break the loop
Therefore, stop iterations.
end for loop of Step 7 (Go to Step 8)

Step 12 Print the output: 𝑊 = [𝑤0 , 𝑤1 , ⋯ , 𝑤𝑚 ]𝑇 ; STOP.

 Numerical Solution of Ordinary Differential Equations (ODEs) 169

Problem 19: Write a C++ program that uses a second-order accurate Finite Difference method to
solve the following boundary value problem:
𝜋
𝑦 ′′ = 𝑦 ′ + 2𝑦 + cos(𝑥) , for 𝑦 = 𝑦(𝑥), where 0 ≤ 𝑥 ≤
2
𝜋
subject to the following Dirichlet boundary conditions: 𝑦(0) = −0.3 and 𝑦 ( ) = −0.1.
2
𝜋
For domain discretization, take step sizes as ℎ = ∆𝑥 =
8

𝜋
To form the computational domain, the physical domain [0, ] is discretized by considering that it
2
consists of a number of equispaced discrete points or nodes, 𝑥𝑖 , for 𝑖 = 0, 1, 2, ⋯ , 𝑚 + 1. For the
given problem,

Number of interior nodes = 𝑚 = 3

𝑝(𝑥) = 1
𝑞(𝑥) = 2
𝑟(𝑥) = cos(𝑥)
The target is to obtain the approximations 𝑤𝑖 to the function values 𝑦𝑖 = 𝑦(𝑥𝑖 ) at the interior
nodes 𝑥𝑖 , for 𝑖 = 1, 2, 3. The values of the solution function are known at 𝑥0 and 𝑥4 due to Dirichlet
boundary conditions:
𝑤0 = 𝑦(𝑥0 ) = −0.3
𝑤4 = 𝑦(𝑥4 ) = −0.1

A C++ program that uses the Gauss-Seidel approach for the stated solution is as follows.

1 #include<iostream>
2 #include<cmath>
3 #include<iomanip>
4 using namespace std;
5
6 #define a 0.0
7 #define b M_PI_2
8 #define alpha -0.3
9 #define beta -0.1
10 #define m 3 //number of interior nodes
11 #define N 200 //maximum number of iterations
12 #define TOL 0.0000001
13
14 #define p(x) 1.0
15 #define q(x) 2.0
16 #define r(x) cos(x)
170 Simplified Numerical Analysis

17
18 void egs(double [], double [] , double [] , double [], double [], double h) ;
19
20 int main()
21 {
22 double h , err , sum ;
23 double x[m+2] , z[m+2] , zp[m+2] ;
24 double B[m+1] , D[m+1] , U[m+1], L[m+1] ;
25 int i , k ;
26
27 h = (b-a)/(m+1) ;
28 x[0] = a ;
29 x[m+1] = b ;
30
31 for(i=1 ; i<=m ; i++)
32 {
33 x[i] = x[i-1]+h ;
cout<< "\tnodes" << x[i] << endl ;
25 }
26
27 z[0] = alpha ;
28 z[m+1] = beta ;
29
30 for (i=1 ; i<=m ;i++)
31 B[i] = (-h)*(h)*(r(x[i]));
32
33 for (i=1 ; i<=m ; i++)
34 D[i] = 2+((h)*(h)*(q(x[i])));
35
36 for (i=1 ; i<=m ; i++)
37 U[i] = -1.0 + ((h*0.5)*(p(x[i])));
38
39 for (i=1 ; i<=m ; i++)
40 L[i] = -1.0-((h*0.5)*(p(x[i]))) ;
41
42 for(i=1 ; i<=m ; i++)
43 z[i] = 0.0 ;
44
45 k=0;
46 cout<< setw(4) << k << ": " ;
47 cout<< "z= " ;
Numerical Solution of Ordinary Differential Equations (ODEs) 171

48 cout<< setprecision(2) << setw(3) << z[0] << " " ;

49 for (i=1 ; i<=m ; i++)
50 {
51 cout<< setprecision(8) << setw(9) << z[i] << " " ;
52 }
53
54 cout<< setprecision(2) << setw(3) << z[m+1] << " " ;
55 cout << endl ;
56
57 egs(z, B, D, U, L, h) ;
58
59 return 0 ;
60 }
61
62 // --- User-defined function for an efficient Gauss-Seidel method --- //
63
64 void egs(double z[m+2], double B[m+1] , double D[m+1] ,
65 double U[m+1], double L[m+1], double h)
66
67 {
68 int i, k ;
69 double sum , err ;
70 double zp[m+2] ;
71
72 for (k=1 ; k<=N ; k++) // Iterations loop
73 {
74 // Making a copy of the solution vector before updating it
75 for (i=1 ; i<=m ; i++)
76 zp[i] = z[i] ;
77
78 // Updating the solution vector
79 for (i=1 ; i<=m ; i++)
80 {
81 z[i] = (B[i]-(L[i]*z[i-1])-(U[i]*z[i+1]))/D[i] ;
82 }
83
84 // Printing the latest solution vector
85 cout<< setw(4) << k << ": " ;
86 cout<< "z= " ;
87 cout<< setprecision(2) << setw(3) << z[0] << " " ;
172 Simplified Numerical Analysis

88 for (i=1 ; i<=m ; i++)

89 {
90 cout<< setprecision(8) << setw(9) << z[i] << " " ;
91 }
92 cout<< setprecision(2) << setw(3) << z[m+1] << " " ;
93 cout << endl ;
94
95 // Finding the error as the L2-norm
96 sum = 0.0 ;
97 for (i=1 ; i<=m ; i++)
98 sum = sum + (z[i]-zp[i])*(z[i]-zp[i])/(z[i]*z[i]) ;
99 err = sqrt(sum) ;
100
101 // Testing the convergence
102 if (err<TOL) break ;
103 }
104 }
∎

Chapter Summary

• The numerical solution of an ODE is not a definition of 𝑦 = 𝑦(𝑥). The numerical solution of
the ODE is a set of numbers 𝑤𝑖 that are approximations to the function values 𝑦(𝑥𝑖 ) at
some pre-specified discrete values 𝑥𝑖 ∈ [𝑎, 𝑏]. That is, 𝑤𝑖 ≅ 𝑦𝑖 = 𝑦(𝑥𝑖 ).

• To solve an initial-value problem consisting of a single first-order ODE in 𝑦 = 𝑦(𝑥) for 𝑎 ≤

𝑥 ≤ 𝑏 and an initial-value 𝑦(𝑎) = 𝛼 , first the domain [𝑎, 𝑏] is discretized by selecting
(𝑚 + 1) equispaced nodes 𝑥0 , 𝑥1 , 𝑥2 , ⋯, 𝑥𝑚 in [𝑎, 𝑏] such that 𝑎 = 𝑥0 < 𝑥1 < 𝑥2 < ⋯ <
𝑥𝑚 = 𝑏 , and ℎ = (𝑏 − 𝑎)/𝑚 . Then, approximations 𝑤𝑖 to the values 𝑦𝑖 = 𝑦(𝑥𝑖 ) for 𝑖 =
1,2, ⋯ , 𝑚 are obtained with 𝑤0 =𝑦(𝑎). For simplicity, 𝑦(𝑥𝑖 ) is denoted by 𝑦𝑖 .

• There is a wide variety of methods for finding numerical solutions of the ODEs involved in
initial value problems (IVPs) and boundary value problems (BVPs).

• Methods for IVPs include single step methods and multi-step methods, each category
having explicit and implicit methods. A hybrid method, i.e., predictor-corrector method,
involves a combination of explicit and implicit formulas.

• Methods for BVPs are so versatile and involve much richer mathematical constructs.
 Numerical Solution of Ordinary Differential Equations (ODEs) 173

• The accuracy of the approximate solution can be improved either by using a larger number
of steps (a smaller step size), or by using a better numerical method.

• The prime characteristics (or considerations) associated with a finite difference scheme to
determine its quality include

➢ Stability
➢ Local Truncation Error
➢ Consistency (Compatibility)
➢ Discretization Error
➢ Convergence
∎∎∎

Chapter Exercises

Exercise 01: Find the numerical solution of the ODE, 𝑦 ′ = 3 − 3𝑦 − 𝑒 −6𝑥 , for 0 ≤ 𝑥 ≤ 2, with initial condition
1
𝑦(0) = 1.0. Consider the step size of 0.5. Use the exact solution, 𝑦(𝑥) = 3 (𝑒 −6𝑥 − 𝑒 −3𝑥 + 3), to find the error
in the numerical solution.

Exercise 02: Find the numerical solution of the ODE, 𝑦 ′ = 1 + (𝑥 − 𝑦)2 , for 2 ≤ 𝑥 ≤ 3, with initial condition
𝑦(2) = 1.0. Consider the step size of 0.5. Use the exact solution, 𝑦(𝑥) = 𝑥 + 1⁄(1 − 𝑥), to find the error in the
numerical solution.

Exercise 03: Find the numerical solution of the ODE, 𝑦 ′ = 2 + (𝑥 − 𝑦)2 , for 2 ≤ 𝑥 ≤ 3, with initial condition
𝑦(2) = 1.5. Consider the step size of 0.5. Use the exact solution, 𝑦(𝑥) = 𝑥 − tan(−𝑥 + 2.463), to find the

Exercise 04: Find the numerical solution of the ODE, 𝑦 ′ = (1 + 𝑥)⁄(1 + 𝑦) , for 0 ≤ 𝑥 ≤ 1 , with initial
condition 𝑦(0) = 2.0. Consider the step size of 0.5. Use the exact solution, 𝑦(𝑥) = √𝑥 2 + 2𝑥 + 9 − 1, to find
the error in the numerical solution.

Exercise 05: For the functions 𝑦1 = 𝑦1 (𝑥) and 𝑦2 = 𝑦2 (𝑥), where 𝑥 ∈ [0,1], solve the following system of two
ODEs:

𝑦1′ = 𝑦1 𝑦2 − 2

𝑦2′ = 2𝑦1 − 𝑦23

With initial conditions:

 174 Simplified Numerical Analysis

𝑦1 (0) = 2.0

𝑦2 (0) = 0.3

Use the RK4 method of order 4 for 5 steps.

HINT: For 5 steps the domain is discretized as

𝑏−𝑎 1.0 − 0.0
ℎ = = = 0.2
𝑚 5
𝑥0 = 0, 𝑥1 = 0.2, 𝑥2 = 0.4, 𝑥3 = 0.6, 𝑥4 = 0.8, 𝑥5 = 1.0.

According to the initial conditions:

𝑤10 = 𝑦10 = 𝑦1 (𝑥0 ) = 𝑦1 (0) = 2.0

𝑤20 = 𝑦20 = 𝑦2 (𝑥0 ) = 𝑦2 (0) = 0.3

The problem is to find approximations 𝑤1𝑖 to 𝑦1𝑖 = 𝑦1 (𝑥𝑖 ) and 𝑤2𝑖 to 𝑦2𝑖 = 𝑦2 (𝑥𝑖 ), for 𝑖 = 1,2,3,4,5.

Exercise 06: For the functions 𝑦1 = 𝑦1 (𝑥), 𝑦2 = 𝑦2 (𝑥), and 𝑦3 = 𝑦3 (𝑥), where 𝑥 ∈ [0,1], solve the following
system of three ODEs:

𝑦1′ = 𝑦1 + 3𝑦2 − 3𝑦3 + 𝑒 −𝑥

𝑦2′ = 2𝑦2 + 𝑦3 − 3𝑒 −𝑥

𝑦3′ = 𝑦1 + 2𝑦2 + 𝑒 −𝑥

with initial conditions:

𝑦1 (0) = 2.5

𝑦2 (0) = −1.5

𝑦3 (0) = −1.0

Use the RK4 method of order 4 for 10 steps.

HINT: For 10 steps, the domain is discretized as

𝑏−𝑎 1.0 − 0.0
ℎ = = = 0.1
𝑚 10
𝑥0 = 0, 𝑥1 = 0.1, 𝑥2 = 0.2, 𝑥3 = 0.3, 𝑥4 = 0.4, 𝑥5 = 0.5, 𝑥6 = 0.6, 𝑥7 = 0.7, 𝑥8 = 0.8, 𝑥9 = 0.9, 𝑥10 = 1.0.

According to the initial conditions:

𝑤10 = 𝑦10 = 𝑦1 (𝑥0 ) = 𝑦1 (0) = 2.5

𝑤20 = 𝑦20 = 𝑦2 (𝑥0 ) = 𝑦2 (0) = −1.5

𝑤30 = 𝑦30 = 𝑦3 (𝑥0 ) = 𝑦3 (0) = −1.0

The problem is to find approximations 𝑤1𝑖 to 𝑦1𝑖 = 𝑦1 (𝑥𝑖 ), 𝑤2𝑖 to 𝑦2𝑖 = 𝑦2 (𝑥𝑖 ), and 𝑤3𝑖 to 𝑦3𝑖 = 𝑦3 (𝑥𝑖 ), for 𝑖 =
1,2, ⋯ , 10.
 Numerical Solution of Ordinary Differential Equations (ODEs) 175

Exercise 07: Find the numerical solution of the IVP, 𝑦 ′′ − 8𝑦 ′ + 7𝑦 = 16𝑒 −𝑥 for 0 ≤ 𝑥 ≤ 1 , with initial
condition 𝑦(0) = 4.0 and 𝑦 ′ (0) = 4.0. Also find 𝑦(1.1). Consider the step size of 0.1. Use the exact solution,
𝑦 = (1⁄3)(𝑒 7𝑥 + 8𝑒 𝑥 + 3𝑒 −𝑥 ), to find the error in the numerical solution.
HINT: Given the equation,
𝑦 ′′ − 8𝑦 ′ + 7𝑦 = 16𝑒 −𝑥 − − −(1)
For the solution, consider
𝑦′ = 𝑧 − − −(2)
Then, the given second-order Eq. (1) becomes
𝑧′ = 4𝑧 − 3𝑦 + 7𝑒 −𝑥 − − −(3)
Thus, the second-order IVP is essentially converted to the problem of a first-order system of ODEs of
comprising the two equations (2) and (3) subject to the initial conditions:
𝑦(0) = 3.0
𝑧(0) = 3.0
For 10 steps, the domain is discretized as
𝑏−𝑎 1.0 − 0.0
ℎ = = = 0.1
𝑚 10
𝑥0 = 0, 𝑥1 = 0.1, 𝑥2 = 0.2, 𝑥3 = 0.3, 𝑥4 = 0.4, 𝑥5 = 0.5, 𝑥6 = 0.6, 𝑥7 = 0.7, 𝑥8 = 0.8, 𝑥9 = 0.9, 𝑥10 = 1.0.
According to the initial conditions:
𝑤10 = 𝑦0 = 𝑦(𝑥0 ) = 𝑦(0) = 3.0
𝑤20 = 𝑧0 = 𝑧(𝑥0 ) = 𝑧(0) = 3.0
The problem is to find approximations 𝑤1𝑖 to 𝑦𝑖 = 𝑦𝑖 (𝑥𝑖 ) and 𝑤2𝑖 to 𝑧𝑖 = 𝑧(𝑥𝑖 ), for 𝑖 = 1,2, ⋯ , 10.

Exercise 08: Solve the ODE 𝑦 ′′ = 4𝑦 ′ − 3𝑦 + 7𝑒 −𝑥 for 𝑦 = 𝑦(𝑥) in 𝑥 ∈ [0,1] with the initial conditions: 𝑦(0) =
3.0 and 𝑦′(0) = 3.0. Solve it for 10 steps.

Exercise 09: Find the numerical solution of the BVP, 𝑦 ′′ − 9𝑦 ′ + 𝑦 = 𝑥 for 0 ≤ 𝑥 ≤ 1, with initial condition
𝑦(0) = 0.0 and 𝑦 ′ (1) = 6.0. Consider the step size of 0.1.

Exercise 10: Find the numerical solution of the ODE, 𝑥 2 𝑦 ′′ + 3𝑥𝑦 ′ + 3𝑦 = 0, with initial condition 𝑦(1) = 1
1 1
and 𝑦 ′(1) = −5 for 𝑦(1.1). The exact solution is, 𝑦 = (cos(√2 ln 𝑥) + ( − 5) sin(√2 ln 𝑥)).
𝑥 𝑥2

Exercise 11: Find the numerical solution of the ODE, 𝑦 ′′ − 6𝑦 ′ + 9𝑦 = 𝑥 2 𝑒 3𝑥 , with initial condition 𝑦(0) = 2
1
and 𝑦′(0) = 6 for 𝑦(1.1). The exact solution is, 𝑦 = 2𝑒 3𝑥 + 𝑥 4 𝑒 3𝑥 .
12

Exercise 12: Solve the ODE 𝑦 ′′′ = −𝑦 ′′ + 3𝑦 ′ + 3𝑦 for 𝑦 = 𝑦(𝑥) in 𝑥 ∈ [0,2] with the initial conditions:
𝑦(0) = 2.0, 𝑦 ′ (0) = −1.0, and 𝑦 ′′ (0) = 8.0. Solve it for 10 steps.

HINT: Given the equation,

𝑦 ′′′ = −𝑦 ′′ + 3𝑦 ′ + 3𝑦 − − −(1)

For 𝑦 = 𝑦(𝑥) in 𝑥 ∈ [0,2] with the initial conditions:

𝑦(0) = 2.0
 176 Simplified Numerical Analysis

𝑦′(0) = −1.0

𝑦′′(0) = 8.0

Solve it for 10 steps.

For the solution, consider

𝑦′ = 𝑧1 − − −(2)

𝑦 ′′ = 𝑧′1 = 𝑧2 − − −(3)

Then, the given third-order Eq. (1) becomes

𝑧2′ = −𝑧2 + 3𝑧1 + 3𝑦 − − −(3)

Thus, the third-order IVP is essentially converted to the problem of a first-order system of ODEs of comprising
the three equations (2) - (4) subject to the initial conditions:

𝑦(0) = 2.0

𝑧1 (0) = −1.0

𝑧2 (0) = 8.0

For 10 steps, the domain is discretized as

𝑏−𝑎 2.0 − 0.0
ℎ = = = 0.2
𝑚 10
𝑥0 = 0, 𝑥1 = 0.2, 𝑥2 = 0.4, 𝑥3 = 0.6, 𝑥4 = 0.8, 𝑥5 = 1.0, 𝑥6 = 1.2, 𝑥7 = 1.4, 𝑥8 = 1.6, 𝑥9 = 1.8, 𝑥10 = 2.0.

According to the initial conditions:

𝑤10 = 𝑦0 = 𝑦(𝑥0 ) = 𝑦(0) = 2.0

𝑤20 = 𝑧10 = 𝑧1 (𝑥0 ) = 𝑧1 (0) = −1.0

𝑤30 = 𝑧20 = 𝑧2 (𝑥0 ) = 𝑧2 (0) = 8.0

The problem is to find approximations 𝑤1𝑖 to 𝑦𝑖 = 𝑦(𝑥𝑖 ), 𝑤2𝑖 to 𝑧1𝑖 = 𝑧1 (𝑥𝑖 ), and 𝑤3𝑖 to 𝑧2𝑖 = 𝑧2 (𝑥𝑖 ), for 𝑖 =
1,2, ⋯ , 10.

Exercise 13: Using a second-order accurate Finite Difference method, solve the following BVP:

𝑦 ′′ = 9𝑦 ′ − 𝑦 + 𝑥, for 𝑦 = 𝑦(𝑥), where 0 ≤ 𝑥 ≤ 1

subject to the following Dirichlet boundary conditions: 𝑦(0) = 0 and 𝑦(1) = 6.

For domain discretization, take step sizes as ℎ = ∆𝑥 = 0.25.

Exercise 14l: Using a second-order accurate Finite Difference method, solve the following BVP:

𝑦 ′′ = −5𝑦 ′ − 8𝑦 + 𝑥 2 , for 𝑦 = 𝑦(𝑥), where 1≤𝑥≤2

subject to the following Dirichlet boundary conditions: 𝑦(1) = 0 and 𝑦(2) = 24. For domain discretization,
take step sizes as ℎ = ∆𝑥 = 0.25.

∎∎∎
 177

Bibliography
1. Richard L. Burden & J. Douglas Faires, (2011), Numerical Analysis, (9th Edition), USA,
Brooks/Cole Pub. Co.

2. Steven C. Chapra & Raymond P. Canale, (2006), Numerical Methods for Engineers, (5 th
Edition), NY, USA, McGraw-Hill Co.

3. David R. Kincaid & E. Ward Cheney, (2002), Numerical Analysis: Mathematics of Scientific
Computing, (3rd Edition), USA, Brooks/Cole Pub. Co.

4. E. Ward Cheney & David R. Kincaid, (2013), Numerical Mathematics and Computing, (7th
Edition), New-Delhi India, Cengage Learning India Pvt. Ltd.

5. Brian Bradie, (2005), A Friendly Introduction to Numerical Analysis, Pearson.

6. John H. Mathews & Kurtis D. Fink, (2015), Numerical Methods using MATLAB, (4 th
Edition), India, Pearson India Education Services Pvt. Ltd.

7. M. K. Jain, S. R. K. Iyengar & R. K. Jain, (2012), Numerical Methods for Scientific and
Engineering Computation, (6th Edition), New-Delhi India, New Age International Pvt. Ltd.

8. George R. Lindfield & John E. T. Penny, (2013), Numerical Methods using MATLAB, (3 rd
Edition), USA, Academic Press, An imprint of Elsevier.

9. Amos Gillat, (2011), MATLAB: An Introduction with Applications, (4 th Edition), USA, John
Wiley & Sons, Inc.

10. Laurene V. Fausett, (2009), Applied Numerical Analysis using MATLAB, (2 nd Edition),
India, PEARSON Education Inc.

11. Babu ram, (2010), Numerical Methods, India, PEARSON Education Inc.

12. Francis Schied, (1990), 2000 Solved Problems in Numerical Analysis, (International
Edition), NY, USA, McGraw-Hill Co.

13. P. Siva Ramakrishna Das & C. Vijayakumari, (2004), Numerical Analysis, (1st Edition),
India, Dorling Kindersley Pvt. Ltd.

14. Saeed Akhtar Bhatti & Naveed Akhtar Bhatti, (2008), A First Course in Numerical
Analysis with C++, (5th Edition), Lahore, Pakistan, A-ONE Publishers.
 178

15. Mohammad Iqbal, (1990), An Introduction to Numerical Analysis, Urdu Bazar Lahore,
Pakistan, Ilmi Kitab Khana.

16. Fiaz Ahmad & Muhammad Afzal Rana, (1995), Elements of Numerical Analysis,
Islamabad, Pakistan, National Book Foundation

17. Amjad Pervez, (1996), An Introduction to Numerical Analysis, Urdu Bazar Lahore,
Pakistan, A.H. Publishers.

18. Germund Dahlquist & Ake Bjö rck, (2003), Numerical Methods, New Jersey, USA, Prentice-
Hall Inc.

19. Erwin Kreyszig, (2011), Advanced Engineering Mathematics, (10th Edition), USA, John
Wiley & Sons, Inc.

20. S. S. Sastry, (2019), Introductory Methods of Numerical Analysis, (Fifth Edition), PHI
Learning Private Limited.

21. Curtis F. Gerald & Patrick O. Wheatley, (2003), Applied Numerical Analysis, (7th Edition),
India, PEARSON Education Inc.

22. K. Sankara Rao, (2009), Numerical Methods for Scientists and Engineers, (Third Edition),
PHI Learning Private Limited.

23. Lal Din Baig, (2014), Numerical Analysis, Ilmi Kitab Khana, Lahore.