Numerical Analysis 1

Tran Nguyen Nam Hung1

1 21TTH1TN

Department of Mathematics and CS

Ho Chi Minh University of Science

Analytics

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 1 / 42

Table of contents

1 Taylor expansion

2 Integral Approximation Method

3 Using numerical method to solve ODE

4 Using numerical method to solve single variable equation

5 The codes of some algorithms

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 2 / 42

Taylor expansion

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 3 / 42

Many ideas of approximation in numerical analysis originates from Taylor
expansion. In this part, we will make revision about this expansion and
introduces how to use it to estimate the error of certain calculations.
Taylor expansion comes from the following theorem that is mentioned in
Calculus 1A.
Theorem
Given a function f : (a, b) → R that has the (n + 1)-th derivative in (a, b).
Then, for all x, x0 ∈ (a, b), there exists θ ∈ (0, 1) such that
n
X f (k) (x0 ) f (n+1) (x0 + θ(x − x0 ))
f (x) = (x − x0 )k + . (x − x0 )n+1 .
k! (n + 1)!
k=0

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 4 / 42

The polynomial
n
X f (k) (x0 )
Pn (x) = (x − x0 )k
k!
k=0

is called Taylor polynomial, while

f (n+1) (x0 + θ(x − x0 ))

Rn (x) = (x − x0 )n+1
(n + 1)!

is called remainder.

Next slide: Turn to Integral Approximation Method

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 5 / 42

Introduction

We need to determine
Zb
I = f (x)dx.
a

If f is continuous on [a, b] and F (x) is an antiderivative of f (x), using the

Newton-Leibniz formula, we have

Zb
I = f (x)dx = F (b) − F (a).
a

In the real life, the information about f is given in form of datasheet, or

the formula for f is too complicated.

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 6 / 42

Idea

We have
Zb Zb Zb
f (x)dx = P(x)dx + R(x)dx
a a a
Zb n
X Zb
= Ln,i (x)f (xi )dx + R(x)dx.
a i=0 a
n
X
≈ f (xi )ai ,
i=0

Rb
in which ai = Ln,i (x)dx. This formula is called numberical quadrature
a
formula, and the number ai is called weight.

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 7 / 42

Trapezoidal rule

Using the Lagrange interpolating polynomial of degree 1 on [a, b], we can

Rb
approximate f (x)dx as follows
a

Zb
f (a) + f (b)
f (x)dx = (b − a) · .
2
a

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 8 / 42

Simpson rule

This is also called the parabola formula.

Zb n−1 n−1
!
h X X
f (x)dx ≈ f (a) + f (b) + 4 f (x2i+1 ) + 2 f (x2i ) ,
3
a i=0 i=1

where h = b−a
2n , x0 = a, x2n = b and xi+1 = xi + h for all i ∈ 0, 2n − 1.
The error of this approximation is

M(b − a)
ε< × h4 ,
180

where M = max |f (4) (x)|.

[a,b]

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 9 / 42

Midpoint Rule

Assume that f (x) is continuous on [a, b]. Let n be a positive integer and
∆x = b−an . If [a, b] is divided into n subintervals, each of length ∆x, and
mi is the midpoint of the ith interval, set
n
X
Mn = f (mi )∆x.
i=1

Rb
Then lim Mn = f (x)dx.
n→+∞ a

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 10 / 42

Using numerical method
to solve ODE

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 11 / 42

In numerical analysis, a branch of applied mathematics, the midpoint
method is a one-step method for numerically solving the differential
equation,
y ′ (t) = f (t, y (t)) , y (t0 ) = y0 .
The explicit midpoint method is given by the formula
 
h h
yn+1 = yn + hf tn + , yn + f (tn , yn ) ,
2 2

the implicit midpoint method by

 
h 1
yn+1 = yn + hf tn + , (yn + yn+1 )
2 2

for n = 0, 1, 2, . . .

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 12 / 42

Here, h is the step size – a small positive number, tn = t0 + nh, and yn is
the computed approximate value of y (tn ). The explicit midpoint method is
sometimes also known as the modified Euler method, the implicit
method is the most simple collocation method, and, applied to
Hamiltonian dynamics, a symplectic integrator.

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 13 / 42

Using numerical method
to solve single variable
equation

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 14 / 42

Introduction

We consider the paradigm of population growth in the short run. Let N(t)
be the number of individuals in the population at time t and λ be the
birth rate (which doesn’t depend on t) of the population.

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 15 / 42

Introduction

The population must satisfy the differential equation

dN(t)
= λN(t) + v ,
dt
where v represents the immigration rate. The root of this equation is
v  λt 
N(t) = N0 e λt + e −1 ,
λ
where N0 is the number of individuals in the population at t = 0.

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 16 / 42

Introduction

Assume that
The number of individuals in the population initially is
N(0) = 1000000.
The immigration rate in the first year is 435000.
After a year, the number of individuals is 1564000.
The equation we must solve is
435 λ
N(λ) = 1000e λ + (e − 1).
λ
It is impossible to find the root of this equation clearly, but we may make
an approximation.

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 17 / 42

Methods used

Bisection method
Fixed-Point method
Newton

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 18 / 42

Bisection method

Assume that f is a continuous function on [a, b] such that f (a), f (b) have
different signs. Firstly, let p = a+b
2 .
If f (a)f (p) < 0, then we continue our algorithm the interval [a, p].
Otherwise, we continue our algorithm with the interval [b, p].
As a result, the length of the interval we consider is decreased by a half.
Doing this n times, we may make an approximation of the root of the
equation f (x) = 0 with error b−a
2n , which can be very small.

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 19 / 42

Example

Examples
Prove that the equation f (x) = x 3 + 4x 2 − 10 = 0 has a root on [1, 2] and
using the bisection method to determine the approximate root, with error
10−4 .
Firstly, we have f (1.5) => 0, f (1) < 0, f (2) > 0. Therefore, we consider
the interval [1, 1.5].

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 20 / 42

Examples
Determine the number of loops needed to count the approximate root of
the equation f (x) = x 3 + 4x 2 − 10 = 0 with error ε = 10−3 , a = 1, b = 2.

Let n be the number of loops. Then the error is b−a 1

2n = 2n . For the error
to be 10−3 , we must have 21n ≤ 10−3 or 2n ≥ 103 , or n ≥ 10.
Therefore, the minimum number of loops needed is 10.

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 21 / 42

Pros and cons

Pros: The idea and algorithm is clear.

Cons:
The sequence (pn ) converges slowly. That is, the value of N might
become too large before the difference |p − pN | becomes sufficiently
small.

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 22 / 42

Fixed-Point method

Given a function g . A value p in the domain of g is called a fixed point

of g if g (p) = p.

Theorem
1 If g : R → R, g ∈ C ([a, b]) and g (x) ∈ [a, b], ∀x ∈ [a, b], then g has
a fixed point p on [a, b].
2 If g ∈ C 1 ([a, b]) and there exists 0 < k < 1 such that |g ′ (x)| ≤ k for
all x ∈ [a, b], then g has at most one fixed point.

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 23 / 42

Examples
x 2 −1
Prove that the function g (x) = 3 has a unique fixed point on [−1, 1].

We have g ′ (x) = 23 x, therefore |g ′ (x)| = 23 |x| ≤ 23 < 1 for all x ∈ [−1, 1].
Using the theorem above, we induce that g has at most one fixed point (*)
Moreover, we have g (x) ∈ [−1, 1] for all x ∈ [−1, 1]. Applying the first
claim of the theorem above, g has a fixed point (**).
Combine (*) and (**), we claim that g has a unique fixed point.

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 24 / 42

Idea

With the problem of finding the root of f , we might change into the
problem of finding the fixed point p of the function g (x) = f (x) + x (or
g (x) = x − f (x)). To find approximately the fixed point of a function g ,
we do as follow:
Firstly, we choose the initial approximate value p0 .
Then, we create the sequence {pn }∞
n=0 by using the recursive formula
pn = g (pn−1 ), ∀n ≥ 1.
If the sequence {pn }∞
n=0 converges to p and g is continuous, then p is a
fixed point of g .

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 25 / 42

Examples
Using the fixed point method, find (approximately) the solution to the
equation
f (x) = xe x − 1 = 0.

We may think of finding the fixed points of these three functions

1+x
g1 (x) = x − xe x + 1, g2 (x) = , g3 (x) = e −x .
1 + ex

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 26 / 42

Corollary
Assume that g satisfies the conditions mentioned in the two parts of the
theorem in slide 15. Then, we have

|pn − p| ≤ k n max{p0 − a, p0 − b}

and
kn
|pn − p| ≤ |p1 − p0 |, ∀n ≥ 1.
1−k

This corollary is used to determine which function g to choose.

The yellow 15: it is actually (slide) 23

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 27 / 42

Proof of the corollary

Note that from the condition |g ′ (x)| ≤ k, we have

|g (x) − g (y )| ≤ k|x − y |, ∀x, y ∈ [a, b].

The first statement trivially holds for n = 0. To make movement from n to

n + 1, we have

|pn+1 − p| = |g (pn ) − g (p)| ≤ k|pn − p|

≤ k · k n max{p0 − a, p0 − b}
= k n+1 max{p0 − a, p0 − b}.

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 28 / 42

Proof of the corollary

Also, for m > n, we have

|pn − pm | ≤ |pn − pn+1 | + |pn+1 − pn+2 | + · · · + |pm−1 − pm |

≤ |p0 − p1 |(k n + k n+1 + · · · + k m−1 )
1 − k m−n kn
= k n |p0 − p1 | ≤ |p0 − p1 |.
1−k 1−k
When m tends to infinity, |pn − pm | → |pn − p|. Thus, we have

kn
|pn − p| ≤ |p0 − p1 |.
1−k

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 29 / 42

Note

The corollary above is related to the speed of convergence of the sequence

(pn ).
The closer to 0 the number k is, the faster (pn ) converges to p.
The closer to 1 the number k is, the slower (pn ) converges to p.
The closer to 12 the number k is, the faster the sequence converges
when we use the bisection method.

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 30 / 42

Newton method

To find the root of the equation f (x) = 0, called p, after we find an

approximation value p0 of p (i.e. |p0 − p| is sufficiently small), possibly by
using other methods, we can come up with a better approximation by
using the Newton method. This method is based on the Taylor
expression. We have the Taylor expression
1
f (p) = f (p0 ) + (p − p0 )f ′ (p0 ) + (p − p0 )2 f ′′ (ζ).
2
As p is the root of the equation f (x) = 0, we have

1
f (p0 ) + (p − p0 )f ′ (p0 ) + (p − p0 )2 f ′′ (ζ).
2
Now that |p − p0 | is sufficiently small, we have (p − p0 )2 ≈ 0.

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 31 / 42

Newton method

This means that f (p0 ) + (p − p0 )f ′ (p0 ) ≈ 0 and

f (p0 )
p ≈ p0 − .
f ′ (p0 )

Examples
Given the equation f (x) = cos(x) − x = 0. Find the approximate root of
this equation, by using
a) Bisection method.
b) Newton method, with p0 = π4 .

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 32 / 42

Solving system of equations

Consider the following linear system of equations




 a11 x1 + a12 x2 + · · · + a1n xn = b1

a x + a x + · · · + a x = b
21 1 22 2 2n n 2


 . . .

a x + a x + · · · + a x = b
n1 1 n2 2 nn n n

Let A be the n × n matrix satisfying the scalar at the cell (i, j) is aij and
b = (b1 , b2 , . . . , bn )T , x = (x1 , x2 , . . . , xn )T . We must solve the equation
Ax = b. Write A = D − T , where D is an invertible matrix. The equation
is equivalent to

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 33 / 42

 Dx − Tx = b ⇒ D −1 (Dx − Tx) = D −1 b ⇒ x = D −1 Tx + D −1 b.

In other words, x is a fixed point of the mapping

f (x) = D −1 Tx + D −1 b.

Starting with a given column vector of size n, called x (0) , we construct the
sequence (x (k) )k∈N that satisfies the following recursive formula

x (k) = D −1 Tx (k−1) + D −1 b.

Whenever ||x (k) − x (k−1) ||∞ < TOL (or sometimes we may consider the
||x (k) −x (k−1) ||∞
condition ||x (k) ||∞
< TOL), we stop the process of construction.

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 34 / 42

Here,
||(x1 , x2 , . . . , xn )T ||∞ = max |xi |,
1≤i≤n

which is similar to the norm that you usually use in the subject of
Functional Analysis.

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 35 / 42

The codes of some
algorithms

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 36 / 42

Newton method

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 37 / 42

Bisection method

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 38 / 42

Fixed point method

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 39 / 42

Jacobi method

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 40 / 42

Lagrange Interpolating Polynomial – The case of a
single variable

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 41 / 42

Lagrange Interpolating Polynomial – The case of
two variables

Tran Nguyen Nam Hung (HCMUS) Analytics 2023 42 / 42