MAT208: INTRODUCTION TO NUMERICAL ANALYSIS (3 UNITS)

Outline:
Introduction

Solution to Algebraic and Transcendental Equations

Numerical Differentiation and Integration

Numerical Solution of Polynomial Equations

1.0 Introduction:
Numerical Analysis is the branch of mathematics concerned with obtaining
numerical results for mathematical problems as well as performing operations on
tabulated data, with the operation of arithmetic and following the rules of
algebra.
Solution to problems like 3𝑥2 + 𝑥 − 2 = 0 and may be solved by applying
the rules governing algebra and calculus, where the solution obtained are
numerical values. Similarly, solution to differential equations like ,

(0) = 1 is obtainable numerical, where the solution to such problem is a function

of x.

Equation of the form 3𝑥5 + 𝑥 − 2 = 0 do not have a general closed form expression,
however, its graph shows that there exist at least one value of in the interval
(0.5, 1) for which the expression equals zero. The question then is, how this value
might be determined.

Again, for a non-linear differential problem 𝑦′ = −(𝑦 + 1) + 𝑥, some theories

guarantees that the solution exist, which is obtainable in a case-by-case basis.
The challenge then would be, the ease of obtaining such solution.

To circumvent these challenges, Numerical method is adopted to obtain estimated

solution, which are determined to be accurate, if such numerical solution is close
to an exact or analytic solution.

1.1 Solution to Algebraic and Transcendental Equations

1.1.1 Method of Finite Differences
Let the function 𝑦 = (𝑥) be given at discrete values of 𝑥 ∈ 𝑋, expressed as
𝑥0 𝑥1 𝑥2 … 𝑥𝑛
𝑓(𝑥) 𝑓(𝑥0) 𝑓(𝑥1) 𝑓(𝑥2) … 𝑓(𝑥𝑛)
Suppose the values are equally spaced so that 𝑥𝑖+1 − 𝑥𝑖 = ℎ and 𝑥𝑖+1 = 𝑥𝑖 + ℎ, so
that 𝑥𝑖+𝑘 = 𝑥𝑖 + 𝑘ℎ, where is the step size. We can then define the following
forward differences

A. First Differences

The first differences (𝛥𝑓(𝑥) = 𝑓(𝑥 + ℎ) − 𝑓(𝑥)) of the function 𝑓(𝑥) is derived
as follows (note that subscripts represents the order of 𝑥 − values as well as
𝑓(𝑥) −values and 𝑓(𝑥𝑖 ) = 𝑓𝑖 .
𝛥𝑓(𝑥0 ) = 𝑓(𝑥0 + ℎ) − 𝑓(𝑥0 ) = 𝑓(𝑥1 ) − 𝑓(𝑥0 ) = 𝑓1 − 𝑓0
𝛥𝑓(𝑥1 ) = 𝑓(𝑥1 + ℎ) − 𝑓(𝑥1 ) = 𝑓(𝑥2 ) − 𝑓(𝑥1 ) = 𝑓2 − 𝑓1
⋮
𝛥𝑓(𝑥𝑖 ) = 𝑓(𝑥𝑖 + ℎ) − 𝑓(𝑥𝑖 ) = 𝑓(𝑥𝑖+1 ) − 𝑓(𝑥𝑖 ) = 𝑓𝑖+1 − 𝑓
Thus, we have that

Δ𝑓0 = 𝑓1 − 𝑓0, Δ𝑓1 = 𝑓2 − 𝑓1, Δ𝑓2 = 𝑓3 − 𝑓2, … Δ𝑓𝑖 = 𝑓𝑖+1 − 𝑓𝑖 (1)

Example: Consider the function tabulated below, the first and second columns
represents 𝑥 and 𝑓(𝑥) respectively, while the third column represents the first
differences (written inline between preceding rows).
𝑓(𝑥) Δ𝑓
0.0 0.000
0.203
0.2 0.203
0.220
0.4 0.423
0.261
0.6 0.684
0.346
0.8 1.030
0.527
1.0 1.557
1.2 2.572 1.015

Clearly, Δ𝑓1 = 𝑓2 − 𝑓1 = 0.423 − 0.203 = 0.220

B. Second and Higher Differences.

The second differences are obtained as the difference of the first differences,
illustrated below.
Δ2𝑓0 = Δ(Δ𝑓0) = Δ(𝑓1 − 𝑓0)
= Δ𝑓1 − Δ𝑓0
= 𝑓2 − 𝑓1 − (𝑓1 − 𝑓0)
= 𝑓2 − 2𝑓1 + 𝑓0
Similarly,
 Δ2𝑓1 = 𝑓3 − 2𝑓2 + 𝑓1.

Thus, we can generalize the second differences as

Δ2𝑓𝑖 = Δ𝑓𝑖+1 − Δ𝑓𝑖 (2)

The third forward difference is computed in the same matter to obtain

Δ3𝑓0 = Δ(Δ2𝑓0) = Δ(Δ𝑓1 − Δ𝑓0)
= 𝛥2 𝑓1 − 𝛥2 𝑓0
= 𝑓3 − 2𝑓2 + 𝑓1 − (𝑓2 − 2𝑓1 + 𝑓0)
= 𝑓3 − 3𝑓2 + 3𝑓1 − 𝑓0

Δ3𝑓1 = Δ(Δ2𝑓1) = Δ(Δ𝑓2 − Δ𝑓1)

= 𝛥2 𝑓2 − 𝛥2 𝑓1
= 𝑓4 − 2𝑓3 + 𝑓2 − (𝑓3 − 2𝑓2 + 𝑓1 )
= 𝑓4 − 3𝑓3 + 3𝑓2 − 𝑓1

In general, higher differences is defined as

𝛥𝑛 𝑓𝑖 = 𝛥𝑛−1 𝑓𝑖+1 − 𝛥𝑛−1 𝑓𝑖 (3)

Equation (3) is called the nth backward difference.

Below is the standard tabular format for displaying forward finite differences
𝛥𝑓 𝛥2 𝑓 𝛥3 𝑓 𝛥4 𝑓
𝑥0 𝑓0
𝛥𝑓0
𝑥1 𝑓1 𝛥2 𝑓0
𝛥𝑓1 𝛥3 𝑓0
𝑥2 𝑓2 𝛥 𝑓1
2
𝛥4 𝑓0
𝛥𝑓2 𝛥 𝑓1
3
𝑥3 𝑓3 𝛥2 𝑓2
𝑥4 𝑓4 𝛥𝑓3

When 𝑓(𝑥) behaves like a polynomial, the number of data in the table has special
properties.

Example:

A function 𝑓(𝑥) = 𝑥 3 is tabulated on a domain 𝑥 = 1 to 𝑥 = 6 as tabulated below.

0 1 2 3 4 5 6

0 1 8 27 64 125 216
Build a forward difference table for this function.
Solution:
𝑥 𝑓 Δ𝑓 Δ2 𝑓 Δ3 𝑓 Δ4 𝑓
0 0
1
1 1 6
7 6
2 8 12 0
19 6
3 27 18 0
37 6
4 64 24 0
61 6
5 125 30
91
6 216

Observe that the difference became a constant value at the third difference,
thus fourth and higher differences are zero.

This implies that for a degree polynomial, the 𝑛𝑡ℎ difference will constant while
the (𝑛 + 1)𝑡ℎ and higher differences will be zeros.

Effect of Error in the Difference Table

Consider a function 𝑓(𝑥) = 0 with error introduced in one of the function values
as tabulated below
Δ𝑓 Δ2𝑓 Δ3𝑓 Δ4𝑓

0
0
0 0
0 𝑒
0 𝑒
𝑒 −4𝑒
𝑒 −3𝑒
−2𝑒 6𝑒
−𝑒 3𝑒
𝑒 −4𝑒
0 𝑒
0 𝑒
0 0
0 0
0 0
0
0
10

The effect of the error spreads fanwise, it also increases in magnitude as the
order of the differences increase. In the column of the differences, the
coefficients of corresponds to the coefficients in the binomial expansion of
(1 − 𝑒)𝑛 , thus the error can be located in the reverse direction and corrected.

Example:

Locate and correct the error in the tabulated values below

1 2 3 4 5 6 7 8 9 10 11 12
𝑓(𝑥) 4 11 22 37 56 79 108 137 172 211 254 301
Solution:

The 2nd differences in the table above are equal except for 3-terms. Also, the 3rd
differences are all zeros with the exception of a group of four terms. The
conclusion is that the tabulated data behaves like a polynomial of degree 2.

The error propagates symmetrically through the table, thus the error is traced
back to the entry of 𝑓(𝑥) at 𝑥 = 7.

The entry in the 4th difference is 2 times the set 1, −4, 6, −4, 1 and this implies
that 𝑓(7) is in error by 2.

Thus 𝑓(7) = 106 instead of 108 as in the table.

1.2 Interpolation with Uniformly Spaced 𝒙 −values

1.2.1 Newton Gregory Forward (NGF) Interpolating Polynomial
When a tabulated function behaves like a polynomial, we attempt to approximate
the function by a polynomial that resembles it. This is achieve by constructing an
𝑛𝑡ℎ degree polynomial that passes through (𝑛 + 1) pair of points (𝑥𝑖 , 𝑓𝑖 ),
𝑖 = 0,1, … , 𝑛. Such a polynomial is always unique.
Thus we have

where , and ℎ = 𝑥𝑖+1 − 𝑥𝑖 is the step size.

Note that is the value at which we want to estimate the function. Also, (𝑥𝑠)
matches all the data pair (𝑥𝑖, 𝑓𝑖), except for a small expected error.

So, if 𝑠 = 0, 𝑓(𝑥0 ) = 𝑓0 , if 𝑠 = 1, 𝑃(𝑥1) = 𝑓1 and if 𝑠 = 𝑛, 𝑃𝑛(𝑥𝑛) = 𝑓𝑛.

Example:
From the data tabulated below, estimate 𝑓(0.83) using the NGF polynomial of
degree 3
𝑥 0.6 0.8 1.0 1.2 1.4
𝑓(𝑥) 0.684 1.030 1.557 2.572 5.798
Solution:

A polynomial of degree 3 implies 𝑛 = 3

Hence,

To estimate 𝑓(0.83) ⇒ 𝑥 = 0.83 and ℎ = 0.2 from the table. Therefore

We form the forward difference table thus

𝛥𝑓 𝛥2 𝑓 𝛥3 𝑓
0.6 0.684
0.346
0.8 1.030 0.181
0.527 0.307
1.0 1.557 0.488
1.015 1.723
1.2 2.572 2.211
3.226
1.4 5.759

Hence,

Therefore, (0.83) ≈ 𝑃3(0.83) = 1.090

Remark:

The tabulated data is a function defined as 𝑓(𝑥) = 𝑡𝑎𝑛(𝑥) in radians,

⇒ 𝑓(0.83) = 𝑡𝑎𝑛(0.83) ≈ 1.093
Hence the error from the computation using NGF polynomial of degree 3 is

𝑒 = 1.093 − 1.090 = 0.003

1.2.2 Newton Gregory Backward (NGB) Interpolating Polynomial.
The NGB polynomial takes the form

𝑠(𝑠 + 1) 𝑠(𝑠 + 1)(𝑠 + 2) 𝑠(𝑠 + 1) … (𝑠 + 𝑛 − 1)

𝑃𝑛 (𝑥𝑠 ) = 𝑓0 + 𝑠∇𝑓−1 + ∇2 𝑓−2 + ∇3 𝑓−3 + ⋯ + ∇𝑛 𝑓−𝑛 (6)
2! 3! 𝑛!

The choice of 𝑥0 shall be stated in a given problem, with which the backward
difference table can be constructed, this is illustrated below.
𝑥 𝑓(𝑥) ∇𝑓 ∇2 𝑓 ∇3 𝑓 ∇4 𝑓
𝑥−2 𝑓−2
∇𝑓−2
𝑥−1 𝑓−1 ∇2 𝑓−2
∇𝑓−1 ∇3 𝑓−2
2
𝑥0 𝑓0 ∇ 𝑓−1 ∇4 𝑓−2
3
∇𝑓0 ∇ 𝑓−1
𝑥1 𝑓1 ∇2 𝑓0
∇𝑓1
𝑥2 𝑓2
Example:

The function (𝑥) = 𝑒−𝑥 is evaluated between 1.00 ≤ 𝑥 ≤ 2.00 and tabulated below.
Find 𝑒−1.9 using the NGB polynomial of degree 4, choosing 𝑥0 = 2.00

Solution:

We construct the difference table of the data thus;

𝑥 𝑓(𝑥) ∇𝑓 ∇2 𝑓 ∇3 𝑓 ∇4 𝑓
1.00 0.3679
−0.0814
1.25 0.2865 0.0180
−0.0634 −0.0039
1.50 0.2231 0.0141 𝟎. 𝟎𝟎𝟎𝟔
−0.0493 −𝟎. 𝟎𝟎𝟑𝟑
1.75 0.1739 𝟎. 𝟎𝟏𝟎𝟖
−𝟎. 𝟎𝟑𝟖𝟓
𝟐. 𝟎𝟎 𝟎. 𝟏𝟑𝟓𝟑

To find 𝑒−1.9 ⇒ (1.9) ⇒ 𝑥𝑠 = 1.9, thus,

So for a polynomial of degree 4,

Therefore,
Thus, we have that 𝑓(1.9) = 𝑒 −1.9 ≈ 𝑃4 (1.9) = 0.1496.

Comparing the result from NGB technique and the exact value of 𝑒−1.9 = 0.1496

This implies that NGB polynomial of degree 4 estimated this function accurately
to 4 decimal places.

1.2.3 Gauss Forward (GF) Interpolating Polynomial

The Gauss Forward interpolating polynomial is given as

𝑠(𝑠 − 1) 𝑠(𝑠 − 1)(𝑠 + 1) 𝑠(𝑠 − 1)(𝑠 + 1)(𝑠 − 2)

𝑃𝑛 (𝑥𝑠 ) = 𝑓0 + 𝑠Δ𝑓0 + Δ2 𝑓−1 + Δ3 𝑓−1 + Δ4 𝑓−2 + ⋯ (7)
2! 3! 4!

Example:

The function 𝑓(𝑥) = 𝑒 −𝑥 is evaluated on the interval 1.00 ≤ 𝑥 ≤ 2.00 and tabulated
below.
𝑥 1.00 1.25 1.50 1.75 2.00
𝑓(𝑥) 0.3679 0.2865 0.2231 0.1738 0.1353

Find 𝑒 −1.9 using GF interpolating polynomial with 𝑥0 = 1.50.

Solution:

The forward difference table for the data is presented as

𝑥 𝑓(𝑥) Δ𝑓 Δ2 𝑓 Δ3 𝑓 Δ4 𝑓
1.00 0.3679
−0.0814
1.25 0.2865 0.0180
−0.0634 −0.0039
𝟏. 𝟓𝟎 𝟎. 𝟐𝟐𝟑𝟏 𝟎. 𝟎𝟏𝟒𝟏 𝟎. 𝟎𝟎𝟎𝟔
−𝟎. 𝟎𝟒𝟗𝟑 −𝟎. 𝟎𝟎𝟑𝟑
1.75 0.1739 0.0108
−0.0385
2.00 0.1353
 𝑥 − 𝑥0 1.90 − 1.50 0.4
ℎ = 0.25, 𝑥0 = 1.50 ⇒𝑠= = = = 1.6
ℎ 0.25 0.25
Thus
𝑓(1.9) ≈ 𝑃4 (1.9)
1.6(0.6)(0.0141)
= 0.2231 + (1.6)(−0.0493) +
2!
1.6(0.6)(2.6)(−0.0033) 1.6(0.6)(2.6)(−0.4)(0.0006)
+ +
3! 4!
that is 𝑃4 (1.9) = 0.1494

1.2.4 Gauss Backward (GB) Interpolating Polynomial

The Gauss Backward interpolating polynomial is given as

𝑠(𝑠 + 1) 𝑠(𝑠 + 1)(𝑠 − 1) 𝑠(𝑠 − 1)(𝑠 + 1)(𝑠 + 2)
𝑃𝑛 (𝑥𝑠 ) = 𝑓0 + 𝑠∇𝑓−1 + ∇2 𝑓−1 + ∇3 𝑓−2 + ∇4 𝑓−2 + ⋯ (8)
2! 3! 4!

Example:

The function 𝑓(𝑥) = 𝑒 −𝑥 is evaluated on the interval 1.00 ≤ 𝑥 ≤ 2.00 and tabulated
below.
𝑥 1.00 1.25 1.50 1.75 2.00
𝑓(𝑥) 0.3679 0.2865 0.2231 0.1738 0.1353

Find 𝑒 −1.9 using GB interpolating polynomial with 𝑥0 = 1.50.

Solution:

The difference table for the data is presented as

𝑥 𝑓(𝑥) ∇𝑓 ∇2 𝑓 ∇3 𝑓 ∇4 𝑓
1.00 0.3679
−0.0814
1.25 0.2865 0.0180
−𝟎. 𝟎𝟔𝟑𝟒 −𝟎. 𝟎𝟎𝟑𝟗
𝟏. 𝟓𝟎 𝟎. 𝟐𝟐𝟑𝟏 𝟎. 𝟎𝟏𝟒𝟏 𝟎. 𝟎𝟎𝟎𝟔
−0.0493 −0.0033
1.75 0.1739 0.0108
−0.0385
2.00 0.1353

𝑥 − 𝑥0 1.90 − 1.50 0.4

ℎ = 0.25, 𝑥0 = 1.50 ⇒𝑠= = = = 1.6
ℎ 0.25 0.25
Hence
𝑓(1.9) ≈ 𝑃4 (1.9)
1.6(2.6)(0.0141) 1.6(2.6)(0.6)(−0.0039)
= 0.2231 + (1.6)(−0.0634) + +
2! 3!
1.6(2.6)(0.6)(3.6)(0.0006)
+
4!
Thus, 𝑃4 (1.9) = 0.1496

Clearly, NGF, NGB, GF and GB gave results that are accurate up to 4 decimal
places, when the results are compared with the exact value of 𝑒 −1.9 which is
0.1496 (to 4 decimal places).

1.3 Interpolation with non-uniformly spaced 𝒙 −values

1.3.1 The Lagrange Interpolation Formula

Suppose 𝑥 −values from an experimental observation are not equally spaced, we

can obtain an approximation for desired 𝑥 values using the Lagrange formula as
follows;
𝑛

𝑃𝑛 (𝑥) = ∑ 𝐿𝑖 (𝑥)𝑦𝑖 = 𝐿0 (𝑥)𝑦0 + 𝐿1 (𝑥)𝑦1 + ⋯ + 𝐿𝑛 (𝑥)𝑦𝑛 (9)

𝑖=0

𝐿𝑖 (𝑥𝑗 ) = 0 for 𝑖 ≠ 𝑗 and 𝐿𝑖 (𝑥𝑗 ) = 1 for 𝑖 = 𝑗.

𝐿𝑖 (𝑥) is defined as
𝑛
(𝑥 − 𝑥𝑗 )
𝐿𝑖 (𝑥) = ∏ , 𝑖≠𝑗
(𝑥𝑖 − 𝑥𝑗 )
𝑗=0

That is
(𝑥 − 𝑥0 )(𝑥 − 𝑥1 )(𝑥 − 𝑥2 ) … (𝑥 − 𝑥𝑖−1 )(𝑥 − 𝑥𝑖+1 ) … (𝑥 − 𝑥𝑛 )
𝐿𝑖 (𝑥) = (10)
(𝑥𝑖 − 𝑥0 )(𝑥𝑖 − 𝑥1 ) … (𝑥𝑖 − 𝑥𝑖−1 )(𝑥𝑖 − 𝑥𝑖+1 ) … (𝑥𝑖 − 𝑥𝑛 )
Example:

Write down the polynomial, which passes through (1, 5), (1.5, −3) and (3, 0).

Solution:

With the 3 given points (𝑥, 𝑦) = (1, 5), (1.5, −3) and (3, 0), we employ a polynomial
of degree 2. Thus
2

𝑃2 (𝑥) = ∑ 𝐿𝑖 (𝑥)𝑦𝑖 = 𝐿0 (𝑥)𝑦0 + 𝐿1 (𝑥)𝑦1 + 𝐿2 (𝑥)𝑦2

𝑖=0

(𝑥 − 𝑥1 )(𝑥 − 𝑥2 ) (𝑥 − 1.5)(𝑥 − 3)
𝐿0 (𝑥) = = = 𝑥 2 − 4.5𝑥 + 4.5
(𝑥0 − 𝑥1 )(𝑥0 − 𝑥2 ) (1 − 1.5)(1 − 3)
(𝑥 − 𝑥0 )(𝑥 − 𝑥2 ) (𝑥 − 1)(𝑥 − 3) 4
𝐿1 (𝑥) = = = − (𝑥 2 − 4𝑥 + 3)
(𝑥1 − 𝑥0 )(𝑥1 − 𝑥2 ) (1.5 − 1)(1.5 − 3) 3
(𝑥 − 𝑥0 )(𝑥 − 𝑥1 ) (𝑥 − 1)(𝑥 − 1.5) 1 2
𝐿0 (𝑥) = = = 𝑥 − 2.5𝑥 + 1.5
(𝑥2 − 𝑥0 )(𝑥2 − 𝑥1 ) (3 − 1)(3 − 1.5) 3
Therefore,
 4 1
𝑃2 (𝑥) = (𝑥 2 − 4.5𝑥 + 4.5)(5) + − (𝑥 2 − 4𝑥 + 3)(−3) + ( 𝑥 2 − 2.5𝑥 + 1.5)(0)
3 3
⇒ 𝑃2 (𝑥) = 9𝑥 2 − 38.5𝑥 + 34.5
Example:

Below is the tabulated values of 𝑦 = ln(𝑥) for 𝑥 = 1.0, 1.1 and 1.3,
𝑥 1.0 1.1 1.3
𝑦 0.0000 0.0953 0.2624
From the values, compute ln(1.2)

Solution:

Three data set requires a polynomial of degree two

2

⇒ 𝑃2 (𝑥) = ∑ 𝐿𝑖 (𝑥)𝑦𝑖 = 𝐿0 (𝑥)𝑦0 + 𝐿1 (𝑥)𝑦1 + 𝐿2 (𝑥)𝑦2

𝑖=0

Therefore,

𝑃2 (1.2) = 𝐿0 (1.2)𝑦0 + 𝐿1 (1.2)𝑦1 + 𝐿2 (1.2)𝑦2

(𝑥 − 𝑥1 )(𝑥 − 𝑥2 ) (𝑥 − 1.1)(𝑥 − 1.3) 𝑥 2 − 2.4𝑥 + 1.43
𝐿0 (𝑥) = = =
(𝑥0 − 𝑥1 )(𝑥0 − 𝑥2 ) (1.0 − 1.1)(1.0 − 1.3) 0.03

(1.2)2 − 2.4(1.2) + 1.43 1.44 − 2.88 + 1.43 1

⇒ 𝐿0 (1.2) = = =−
0.03 0.03 3

(𝑥 − 𝑥0 )(𝑥 − 𝑥2 ) (1.2 − 1)(1.2 − 1.3)

𝐿1 (𝑥) = ⇒ 𝐿1 (1.2) = =1
(𝑥1 − 𝑥0 )(𝑥1 − 𝑥2 ) (1.1 − 1)(1.1 − 1.3)

(𝑥 − 𝑥0 )(𝑥 − 𝑥1 ) (1.2 − 1.0)(1.2 − 1.1) 1

𝐿2 (𝑥) = ⇒ 𝐿2 (1.2) = =
(𝑥2 − 𝑥0 )(𝑥2 − 𝑥1 ) (1.3 − 1.0)(1.3 − 1.1) 3

1 1
𝑃2 (1.2) = − (0.0000) + 1(0.0953) + (0.2624) = 0.1828
3 3
Hence, using Lagrange interpolation formula, we estimated the value of ln(1.2) to
be 0.1828. This differs from the exact value of ln(1.2) with −0.0005, that is, the
estimate is in error by −0.0005,

1.4 Curve Fittings

1.4.1 Least Squares Method

This method is employed if the given data set is linear, and the approach minimizes
the sum of squares of expected computation errors.
The method determines the values of 𝑎 and 𝑏 of 𝑦(𝑥) = 𝑎𝑥 + 𝑏 subject to the
condition that the sum of square of the error(s) is minimum.

Hence, from a given data set, we determine the values of 𝑎 and 𝑏 as

𝑁∑𝑥𝑖 𝑦𝑖 − (∑𝑥𝑖 )(∑𝑦𝑖 )
𝑎=
𝑁 ∑ 𝑥𝑖2 − (∑𝑥𝑖 )2
(∑𝑦𝑖 )(∑ 𝑥𝑖2 ) − (∑𝑥𝑖 )(∑𝑥𝑖 𝑦𝑖 )
𝑏=
𝑁 ∑ 𝑥𝑖2 − (∑𝑥𝑖 )2

which are substituted in 𝑦(𝑥) = 𝑎𝑥 + 𝑏 to get an equation of fit to given linear

data. 𝑁 is the number of given data.

Example:

Approximate by least squares method, a curve that fits the following data for
𝑦(𝑥), hence find 𝑦(7) to the nearest whole number.
𝒙 3 4 6 8 9
𝒚 2 5 7 9 10
Solution:

A curve that fits this data using least squares method requires that we solve for
𝑎 and 𝑏 in 𝑦(𝑥) = 𝑎𝑥 + 𝑏, where 𝑁 = 5. Thus, we have
𝒙 𝒚 𝒙𝒚 𝒙𝟐
3 2 6 9
4 5 20 16
6 7 42 36
8 9 72 64
9 10 90 81
2
∑𝑥𝑖 = 30 ∑𝑦𝑖 = 33 ∑𝑥𝑖 𝑦𝑖 = 230 ∑𝑥 = 206
Therefore,
𝑁∑𝑥𝑖 𝑦𝑖 − (∑𝑥𝑖 )(∑𝑦𝑖 ) 5(230) − (30)(33) 1150 − 990 160 16
𝑎= = = = =
𝑁 ∑ 𝑥𝑖2 − (∑𝑥𝑖 )2 5(206) − (30)2 1030 − 900 130 13

and
(∑𝑦𝑖 )(∑ 𝑥𝑖2 ) − (∑𝑥𝑖 )(∑𝑥𝑖 𝑦𝑖 ) (33)(206) − (30)(230) 6798 − 6900 102 51
𝑏= 2 2
= 2
= =− =−
𝑁 ∑ 𝑥𝑖 − (∑𝑥𝑖 ) 5(206) − (30) 1030 − 900 130 65

Hence, the curve that fits the data is

𝑦(𝑥) = 𝑎𝑥 + 𝑏
16 51
⇒ 𝑦(𝑥) = 𝑥−
13 65
Hence,
16 51 509
𝑦(7) = (7) − = = 7.8307 ≈ 8
13 65 65
2.0 Numerical Differentiation and Integration

In this chapter, we study the technique that allow us to compute the derivative
of functions from tabulated data, using previous definitions of interpolating
polynomials.

2.1 First Derivative from NGF Polynomial

The derivative of a function at a point, say 𝑥0 , is estimated from the slope of the
interpolation polynomial, provided the function is suitably approximated.

Since 𝑓(𝑥) = 𝑃𝑛 (𝑥) + 𝑒𝑟𝑟𝑜𝑟 , the derivative 𝑓 ′ (𝑥0 ) of a function 𝑓(𝑥) from a
tabulated data is computed as
1 1 1 1
𝑓 ′ (𝑥0 ) ≈ (Δ𝑓0 − Δ2 𝑓0 + Δ3 𝑓0 − + − + ⋯ ± Δ𝑛 𝑓0 ) (11)
ℎ 2 3 𝑛
Example:

A function 𝑓(𝑥) is tabulated as presented below

𝑥 1.3 1.5 1.7 1.9 2.1 2.3 2.5
𝑓(𝑥) 3.669 4.482 5.474 6.686 8.166 9.974 12.182
Estimate the derivative of 𝑓(𝑥) at 1.7 using (i) 2 terms and (ii) 4 terms of
the approximating polynomial.

Solution:

Form a difference table for the given data,

𝑥 𝑓(𝑥) Δ𝑓 Δ2 𝑓 Δ3 𝑓 Δ4 𝑓 Δ5 𝑓 Δ6 𝑓
1.3 3.669
0.813
1.5 4.482 0.179
0.992 0.041
1.7 5.474 0.220 0.007
1.212 0.048 0.005
1.9 6.686 0.268 0.012 −0.005
1.480 0.060 0.000
2.1 8.166 0.328 0.012
1.808 0.072
2.3 9.974 0.400
2.208
2.5 12.182
The point at which the derivative of the function is desired is taken to be 𝑥0 , so

(i) Using 2 terms of the formula (equation (11)), we have

1 1
𝑓 ′ (𝑥0 ) ≈ (Δ𝑓0 − Δ2 𝑓0 )
ℎ 2
Thus,

1 1
𝑓 ′ (1.7) ≈ (1.212 − (0.268)) = 5.390
0.2 2
 (ii) Using 4 terms of the formula (equation (11)), we have that
1 1 1 1
𝑓 ′ (𝑥0 ) ≈ (Δ𝑓0 − Δ2 𝑓0 + Δ3 𝑓0 − Δ4 𝑓0 )
ℎ 2 3 4
That is,
1 1 1 1
𝑓 ′ (1.7) ≈ (1.212 − (0.268) + (0.060) − (0.012)) = 5.475
0.2 2 3 4

Remark: The table for this example is for the function 𝑓(𝑥) = 𝑒 𝑥 , the derivative
of this function remain itself, thus the derivative of the function at 𝑥 = 1.7 is
5.474.

The result obtained from the 4 term truncation is more accurate than the two
term variant.

2.2 Numerical Integration

The definite integral of some function of interest may be obtained from the
tabulated data values of such function.

𝑦
𝑓(𝑥)

𝑎 = 𝑥0 𝑥1 𝑥2 … 𝑥𝑛−1 𝑥𝑛 = 𝑏 𝑥
Figure 1: The graph of 𝑓(𝑥), indicating data points 𝑥𝑖 ′𝑠
𝑏
From the diagram above, ∫𝑎 𝑓(𝑥)𝑑𝑥 is evaluated by dividing the entire interval
[𝑎, 𝑏] into 𝑛 equal subintervals, and the integral becomes a weighted sum of the
𝑏−𝑎
area under the curve, with step size ℎ taken to be ℎ = 𝑛 .

Note: 𝑛 is 1 less than the total number of points 𝑥0 to 𝑥𝑛 presented in the table.

Three methods of integration are considered in what follows:

2.2.1 Trapezoidal Rule of Integration

The integral of a function within an interval [𝑎, 𝑏] is obtained using

𝑏
ℎ
∫ 𝑓(𝑥)𝑑𝑥 = (𝑓 + 2𝑓2 + 2𝑓3 + ⋯ + 2𝑓𝑛−1 + 𝑓𝑛 )
𝑎 2 1

Example:

Using Trapezoidal rule, approximate the integral of the function 𝑓(𝑥) tabulated
on the interval 2.0 to 3.0 as presented below
 𝑥 2.0 2.2 2.4 2.6 2.8 3.0
𝑓(𝑥) 7.389 9.025 11.023 13.646 16.445 20.086
Solution:

The integral of the function using trapezoidal rule is obtained using

𝑏
ℎ
∫ 𝑓(𝑥)𝑑𝑥 = (𝑓 + 2𝑓2 + 2𝑓3 + ⋯ + 2𝑓𝑛−1 + 𝑓𝑛 )
𝑎 2 1

where 𝑛 = 5, 𝑎 = 2.0 𝑏 = 3.0

𝑏 − 𝑎 3.0 − 2.0 1
⇒ℎ= = = = 0.2
𝑛 5 5
Therefore
3.0
0.2
∫ 𝑓(𝑥)𝑑𝑥 = (𝑓 + 2𝑓2 + 2𝑓3 + 2𝑓4 + 2𝑓5 + 𝑓6 )
1.0 2 1
0.2
= (7.389 + 2(9.025) + 2(11.023) + 2(13.646) + 2(16.445) + 20.086)
2
= 12.775

𝟏
2.2.2 Simpson’s Rule of Integration
𝟑

The integral of a function following this rule is given as

𝑏
ℎ
∫ 𝑓(𝑥)𝑑𝑥 = (𝑓 + 4𝑓2 + 2𝑓3 + 4𝑓4 + 2𝑓5 + ⋯ + 2𝑓𝑛−2 + 4𝑓𝑛−1 + 𝑓𝑛 )
𝑎 3 1

Example:

A function 𝑓(𝑥) is tabulated with 𝑥 values as

𝒙 0.000 0.125 0.250 0.375 0.500 0.625 0.750
𝒇(𝒙) 1.0000 0.9998 0.9961 0.9801 0.9412 0.8676 0.7596
1
Compute the integral of the function using Simpson’s 3 integration rule.

Solution:
1
Simpson’s rule is given defined as
3
𝑏
ℎ
∫ 𝑓(𝑥)𝑑𝑥 = (𝑓 + 4𝑓2 + 2𝑓3 + 4𝑓4 + 2𝑓5 + ⋯ + 2𝑓𝑛−2 + 4𝑓𝑛−1 + 𝑓𝑛 )
𝑎 3 1

where 𝑛 = 6, 𝑎 = 0.000, 𝑏 = 0.750, thus ℎ is calculated as

𝑏 − 𝑎 0.750 − 0.000
ℎ= = = 0.125
𝑛 6
therefore
 0.750
0.125
∫ 𝑓(𝑥)𝑑𝑥 = (𝑓1 + 4𝑓2 + 2𝑓3 + 4𝑓4 + 2𝑓5 + 4𝑓6 + 𝑓7 )
0.000 3
0.125
= (1.0000 + 4(0.9998) + 2(0.9961) + 4(0.9801) + 2(0.9412) + 4(0.8676)
3
+ 0.7596)
= 0.7093

𝟑
2.2.3 Simpson’s Rule of Integration
𝟖
3
The Simpson’s 8 rule is expressed as
𝑏
3ℎ
∫ 𝑓(𝑥)𝑑𝑥 = (𝑓 + 3𝑓2 + 3𝑓3 + 2𝑓4 + 3𝑓5 + 3𝑓6 + 2𝑓7 + ⋯ + 3𝑓𝑛−2 + 3𝑓𝑛−1 + 𝑓𝑛 )
𝑎 8 1

Example:
3
Apply Simpson’s rule in computing the integral of a function 𝑓(𝑥) tabulated on
8
an interval as presented thus
𝒙 0.000 0.125 0.250 0.375 0.500 0.625 0.750
𝒇(𝒙) 1.0000 0.9998 0.9961 0.9801 0.9412 0.8676 0.7596

Solution:
3
Simpson’s rule is given defined as
8
𝑏
3ℎ
∫ 𝑓(𝑥)𝑑𝑥 = (𝑓 + 3𝑓2 + 3𝑓3 + 2𝑓4 + 3𝑓5 + 3𝑓6 + 2𝑓7 + ⋯ + 3𝑓𝑛−2 + 3𝑓𝑛−1 + 𝑓𝑛 )
𝑎 8 1

where 𝑛 = 6, 𝑎 = 0.000, 𝑏 = 0.750, thus ℎ is calculated as

𝑏 − 𝑎 0.750 − 0.000
ℎ= = = 0.125
𝑛 6
therefore
0.750
3(0.125)
∫ 𝑓(𝑥)𝑑𝑥 = (𝑓1 + 3𝑓2 + 3𝑓3 + 2𝑓4 + 3𝑓5 + 3𝑓6 + 𝑓7 )
0.000 8
3(0.125)
= (1.0000 + 3(0.9998) + 3(0.9961) + 2(0.9801) + 3(0.9412) + 3(0.8676) + 0.7596)
8
= 0.7094
Remarks:
1
1. Simpson’s rule is applicable when 𝑛 is a multiple of 2
3
3
2. Simpson’s rule is employed when 𝑛 is a multiple of 3
8
1 3
3. Applications of Simpson’s and shows that they both integrated the
3 8
function with very close numerical results.
3 Numerical Solution to Higher Order Polynomials

Suppose the function 𝑓(𝑥) is a polynomial of degree greater than 2, to find an

approximate root of 𝑓(𝑥) = 0, we may perform some extrapolation, until success
values of iteration converges, subject to some properties of the function.

We thus, discuss three extrapolation approaches

3.1 The Newton-Raphson Method

Newton Raphson’s formula is defined as

𝑓(𝑥𝑘 )
𝑥𝑘+1 = 𝑥𝑘 − , 𝑘 = 1, 2, 3, …
𝑓 ′ (𝑥𝑘 )
The formula describes a procedure of iteration, which is applied successively until
convergence is achieved, where the convergence value is an approximate root of
a polynomial of interest.

Example:

Find the roots of 𝑥 4 − 𝑥 = 10 that lies near 2 (to 4 decimal places), using Newton
Raphson method.

Solution:
𝑥 4 − 𝑥 = 10 ⇒ 𝑥 4 − 𝑥 − 10 = 0
that is, 𝑓(𝑥) = 𝑥 4 − 𝑥 − 10 and 𝑓 ′ (𝑥) = 4𝑥 3 − 1

the Newton Raphson’s formula given as

𝑓(𝑥𝑘 )
𝑥𝑘+1 = 𝑥𝑘 − , 𝑘 = 1, 2, 3, …
𝑓 ′ (𝑥𝑘 )
Take 𝑥1 = 2

→ when 𝑘 = 1, 𝑓(2) = 24 − 2 − 10 = 4 and 𝑓 ′ (2) = 4(2)3 − 1 = 31, hence

𝑓(𝑥𝑘 ) 𝑓(𝑥1 ) 4
𝑥𝑘+1 = 𝑥𝑘 − ⇒ 𝑥2 = 𝑥1 − ⇒ 𝑥2 = 2 − = 1.870968 ≈ 1.8710
𝑓 ′ (𝑥𝑘 ) 𝑓 ′ (𝑥1 ) 31
→ when 𝑘 = 2,
𝑓(𝑥2 ) (1.8710)4 − 1.8710 − 10
𝑥3 = 𝑥2 − ⇒ 𝑥3 = 1.8710 − = 1.8558
𝑓 ′ (𝑥2 ) 4(1.8710)3 − 1
→ when 𝑘 = 3,
𝑓(𝑥3 ) (1.8558)4 − 1.8558 − 10
𝑥4 = 𝑥3 − ′ ⇒ 𝑥4 = 1.8558 − = 1.8556
𝑓 (𝑥3 ) 4(1.8558)3 − 1
→ when 𝑘 = 4,
𝑓(𝑥4 ) (1.8556)4 − 1.8556 − 10
𝑥5 = 𝑥4 − ⇒ 𝑥5 = 1.8556 − = 1.85558 ≈ 1.8556
𝑓 ′ (𝑥4 ) 4(1.8556)3 − 1
Hence, we conclude that a root of 𝑥 4 − 𝑥 = 10 that lies near 2 is 1.8556 (to 4 d p).

Advantage of Newton Raphson Method

i. The technique converges rapidly, if the function is convergent

Disadvantages of Newton Raphson Method

i. Convergence is not guaranteed if the function has a turning point, that is if

𝑓 ′ (𝑥) = 0.
ii. 𝑓 ′ (𝑥) may be difficult to obtain.

3.2 Bisection Method

The Bisection Method of finding a root of a function 𝑓(𝑥) is based upon the Mean
Value Theorem.

The iteration method starts with an initial guess of two points, say 𝑎 and 𝑏. Then
we check for the condition that 𝑓(𝑎) and 𝑓(𝑏) have different signs (that we check
if 𝑓(𝑎)𝑓(𝑏) < 0).

If 𝑓(𝑎)𝑓(𝑏) > 0, we proceed to guess another point 𝑐, until the condition that the
function 𝑓 at any two of these points have different signs, is satisfied.

The average of any two such points 𝑎, 𝑏 or 𝑐, is then considered for the next
bisection check.

Example:

Find the real roots of the equation 𝑓(𝑥) = 𝑥 3 − 3𝑥 − 3, that lies near 2 (to 2
decimal places), using the Bisection Method

Solution:

Suppose 𝑥1 = 2
𝑓(𝑥1 ) = 𝑓(2) = 23 − 3(2) − 3 = −1
We then 𝑓(𝑥) check for 𝑥 = 1
⇒ 𝑓(1) = 13 − 3(1) − 3 = −5
Clearly, 𝑓(2) and 𝑓(1) have the same signs and 𝑓(2)𝑓(1) > 0, so a root of 𝑓(𝑥)
cannot lie between 2 and 1.

Let 𝑥 = 3,
⇒ 𝑓(3) = 33 − 3(3) − 3 = 15
Since , 𝑓(2) and 𝑓(3) have different signs and 𝑓(2)𝑓(3) < 0, then there exist a
root of 𝑓(𝑥) that lies between 2 and 3.

So, we let 𝑥2 = 3 and compute 𝑥3 as

 2+3
𝑥3 = = 2.5
2
Then, we check the sign of 𝑓(𝑥3 ),
⇒ 𝑓(𝑥3 ) = 𝑓(2.5) = 2.53 − 3(2.5) − 3 = 5.125
Again, 𝑓(𝑥2 ) and 𝑓(𝑥3 ) have different signs and 𝑓(𝑥2 )𝑓(𝑥3 ) < 0, thus
2 + 2.5
𝑥4 = = 2.25
2
The bisection procedure continues in the manner

⇒ 𝑓(𝑥4 ) = 𝑓(2.25) = 2.253 − 3(2.25) − 3 = 1.6406, 𝑓(𝑥4 ) > 0

2 + 2.25
𝑥5 = = 2.125
2
𝑓(𝑥5 ) = 𝑓(2.125) = 2.1253 − 3(2.125) − 3 = 0.2207, 𝑓(𝑥5 ) > 0

Root must lie between 𝑥2 and 𝑥5 , thus

2 + 2.125
𝑥6 = = 2.063
2
𝑓(𝑥6 ) = 𝑓(2.063) = 2.0633 − 3(2.063) − 3 = −0.4089, 𝑓(𝑥6 ) < 0

Root must lie between 𝑥5 and 𝑥6 , thus

2.125 + 2.063
𝑥7 = = 2.094
2
𝑓(𝑥7 ) = 𝑓(2.094) = 2.0943 − 3(2.094) − 3 = −0.1002, 𝑓(𝑥7 ) < 0
2.125 + 2.094
𝑥8 = = 2.1095 ≈ 2.11
2
A close look at roots 𝑥4 to 𝑥8 suggests that

the best estimate for a root of 𝑓(𝑥) = 𝑥 3 − 3𝑥 − 3 is approximately 2.1

3.3 Fixed Point Iteration Method

Given an equation 𝑓(𝑥) = 0, we can approximate a root of the function via fixed
point iteration following the steps listed below

i. Find initial values 𝑎 and 𝑏 such that 𝑓(𝑎) and 𝑓(𝑏) have different signs, then,
take
𝑎+𝑏
𝑥0 =
2
ii. Transform the given function to become 𝑔(𝑥), such that |𝑔′ (𝑥0 )| < 1. This
guarantees convergence of the expression.
iii. Apply successive iteration to get approximate root, in the form
𝑥𝑛 = 𝑔(𝑥𝑛−1 ), 𝑛 = 1, 2, 3, …
Example:

Find an approximate root of the equation 2𝑥 3 − 2𝑥 − 5 = 0 using fixed point

iteration.

Solution:

Let 𝑓(𝑥) = 2𝑥 3 − 2𝑥 − 5

i. Take 𝑎 = 1 ⇒ 𝑓(1) = 2(1)3 − 2(1) − 5 = −5

and 𝑏 = 2 ⇒ 𝑓(2) = 2(2)3 − 2(2) − 5 = 7
Thus
1+2 3
𝑥0 = = = 1.5
2 2
ii. Expressing 2𝑥 3 − 2𝑥 − 5 = 0 in another form, we make 𝑥 the subject of the
formula, and check all possible expressions for the condition |𝑔′ (𝑥0 )| < 1.
Thus, we have
a. 2𝑥 3 − 2𝑥 − 5 = 0
2𝑥 = 2𝑥 3 − 5
2𝑥 3 − 5 2𝑥 3 − 5
𝑥= ⇒ 𝑔(𝑥) =
2 2
Therefore, 𝑔 ′ (𝑥)
= 3𝑥 2
and |𝑔′ (𝑥0 )| = 3(1.5)2 = 6.75
Since |𝑔′ (𝑥0 )| > 1, we find another expression for 𝑔(𝑥)

b. 2𝑥 3 − 2𝑥 − 5 = 0
2𝑥 3 = 2𝑥 + 5
1 1
2𝑥 + 5 3 2𝑥 + 5 3
𝑥=( ) ⇒ 𝑔(𝑥) = ( )
2 2
Thus,
2 2
1 1 1
𝑔′(𝑥) = (2𝑥 + 5)−3 and 𝑔′ (𝑥) = (2(1.5) + 5)−3 = that is |𝑔′ (𝑥0 )| < 1
3 3 12

We therefore, proceed to the iteration stage.

1
2𝑥+5 3
iii. Using 𝑔(𝑥) = ( 2 ), we have that 𝑥0 = 1.5 and
1
2𝑥𝑛−1 + 5 3
𝑥𝑛 = ( )
2
when 𝑛 = 1
1 1
2𝑥0 + 5 3 2(1.5) + 5 3
𝑥1 = ( ) =( ) = 1.5874
2 2
when 𝑛 = 2
1 1
2𝑥1 + 5 3 2(1.5874) + 5 3
𝑥2 = ( ) =( ) = 1.5989
2 2
 when 𝑛 = 3
1 1
2𝑥2 + 5 3 2(1.5989) + 5 3
𝑥3 = ( ) =( ) = 1.6003
2 2
when 𝑛 = 4
1 1
2𝑥3 + 5 3 2(1.6004) + 5 3
𝑥4 = ( ) =( ) = 1.6005
2 2
when 𝑛 = 5
1 1
2𝑥4 + 5 3 2(1.6005) + 5 3
𝑥5 = ( ) =( ) = 1.6006
2 2
when 𝑛 = 6
1 1
2𝑥5 + 5 3 2(1.6006) + 5 3
𝑥6 = ( ) =( ) = 1.6006
2 2
From the values of 𝑥3 through 𝑥6 , it is observed that the iteration converges to
1.6006, hence a root of 2𝑥 3 − 2𝑥 − 5 = 0 is 1.6006 (to 4 decimal place).
Appendices:

Appendix 1: Derivation of Formulae

Throughout the following derivations, we note that

𝑥𝑖 = 𝑥0 + 𝑖ℎ and 𝑥𝑖+𝑗 = 𝑥𝑖 + 𝑗ℎ

Thus, the following operators are defined;

1. Shift Operator
𝐸𝑓(𝑥) = 𝑓(𝑥 + ℎ) or 𝐸𝑓𝑖 = 𝑓𝑖+1 , and 𝐸 𝑛 𝑓𝑖 = 𝑓𝑖+𝑛 ∀ 𝑛

2. Forward Difference Operator

Δ𝑓(𝑥) = 𝑓(𝑥 + ℎ) − 𝑓(𝑥) or Δ𝑓𝑖 = 𝑓𝑖+1 − 𝑓𝑖 and Δ𝑛 𝑓𝑖 = 𝑓𝑖+𝑛 − 𝑓𝑖

3. Backward Difference Operator

∇𝑓(𝑥) = 𝑓(𝑥) − 𝑓(𝑥 − ℎ) or ∇𝑓𝑖 = 𝑓𝑖 − 𝑓𝑖−1 and ∇𝑛 𝑓𝑖 = 𝑓𝑖 − 𝑓𝑛−1

Appendix 2: Relationship between 𝑬, 𝚫 and 𝛁

a. Δ𝑓(𝑥) = 𝑓(𝑥 + ℎ) − 𝑓(𝑥) = 𝐸𝑓(𝑥) − 𝑓(𝑥) = (𝐸 − 1)𝑓(𝑥). Thus,

Δ = 𝐸 − 1.

b. ∇𝑓(𝑥) = 𝑓(𝑥) − 𝑓(𝑥 − ℎ) = 𝑓(𝑥) − 𝐸 −1 𝑓(𝑥) = (1 − 𝐸 −1 )𝑓(𝑥). Thus,

∇= 1 − 𝐸 −1

Appendix 3: 𝑬, 𝚫 and 𝛁 are Linear Operators

To show that these operators are linear, we adopts the definition of linearity

Definition: Let 𝛼 be an operator. 𝛼 is a linear operator if for any given functions

𝑓 and 𝑔, we have that

i. 𝛼(𝑓 + 𝑔) = 𝛼𝑓 + 𝛼𝑔 ii. 𝛼(𝑐𝑓) = 𝑐(𝛼𝑓) where c is a constant.

Hence, by the definition above, we describe the linearity of operators 𝐸, Δ and ∇

a. Linearity of 𝐸
Let 𝑓(𝑥) and 𝑔(𝑥) be any two functions, then

i. 𝐸(𝑓(𝑥) + 𝑔(𝑥)) = 𝑓(𝑥 + ℎ) + 𝑔(𝑥 + ℎ) = 𝐸𝑓(𝑥) + 𝐸𝑔(𝑥)

ii. 𝐸(𝑐𝑓(𝑥)) = 𝑐𝑓(𝑥 + ℎ) = 𝑐𝐸𝑓(𝑥)
 b. Linearity of Δ
Let 𝑓(𝑥) and 𝑔(𝑥) be any two functions, then

i. Δ(𝑓(𝑥) + 𝑔(𝑥)) = 𝑓(𝑥 + ℎ) + 𝑔(𝑥 + ℎ) − (𝑓(𝑥) + 𝑔(𝑥))

= 𝑓(𝑥 + ℎ) − 𝑓(𝑥) + 𝑔(𝑥 + ℎ) − 𝑔(𝑥) = Δ𝑓(𝑥) + Δ𝑔(𝑥)
ii. Δ(𝑐𝑓(𝑥)) = 𝑐𝑓(𝑥 + ℎ) − 𝑐𝑓(𝑥) = 𝑐(𝑓(𝑥 + ℎ) − 𝑓(𝑥)) = 𝑐Δ𝑓(𝑥)

c. Linearity of ∇
Let 𝑓(𝑥) and 𝑔(𝑥) be any two functions, then

i. ∇(𝑓(𝑥) + 𝑔(𝑥)) = (𝑓(𝑥) + 𝑔(𝑥) − (𝑓(𝑥 − ℎ) − 𝑔(𝑥 − ℎ))

= 𝑓(𝑥) − 𝑓(𝑥 − ℎ) + 𝑔(𝑥) − 𝑔(𝑥 − ℎ) = ∇𝑓(𝑥) + ∇𝑔(𝑥)
ii. ∇(𝑐𝑓(𝑥)) = 𝑐𝑓(𝑥) − 𝑐𝑓(𝑥 − ℎ) = 𝑐(𝑓(𝑥) − 𝑓(𝑥 − ℎ)) = ∇𝑓(𝑥)

Appendix 4: Exercise

Derive the NGF from the relation given by Δ = 𝐸 − 1.

Solution:

Δ=𝐸−1 ⇒ 𝐸 = 1 + Δ, it follows that 𝐸 𝑠 = (1 + Δ)s

Therefore,

𝑠(𝑠 − 1) 2 𝑠(𝑠 − 1)(𝑠 − 2) 3

𝐸 𝑠 𝑓0 = (1 + Δ)𝑠 𝑓0 = (1 + 𝑠Δ + Δ + Δ + ⋯ ) 𝑓0
2! 3!

𝑠(𝑠 − 1) 2 𝑠(𝑠 − 1)(𝑠 − 2) 3

⇒ 𝐸 𝑠 𝑓0 = 𝑓0 + 𝑠Δ𝑓0 + Δ 𝑓0 + Δ 𝑓0 + ⋯
2! 3!