Numerical Methods

King Saud University

Aims

Chapter 2

Lecture #2

In this lecture, we will . . .

I Introduce the Fixed-Point Method
Fixed-Point Method

The basic idea of this method which is also called successive approximation
method or function iteration, is to rearrange the original equation

f (x) = 0, (1)

into an equivalent expression of the form

x = g(x). (2)

Any solution of (2) is called a fixed-point for the iteration function g(x) and hence
a root of (1).
Definition 1
(Fixed-Point of a Function)
A fixed-point of a function g(x) is a real number α such that α = g(α).
x2 − 4x + 8
For example, x = 2 is a fixed-point of the function g(x) = because
2
g(2) = 2. •
The fixed-point method essentially solves two functions simultaneously; y = x and
y = g(x). The point of intersection of these two functions is the solution to
x = g(x), and thus to f (x) = 0, see Figure 1.

y=x
y=g(x)

0
x5 x4 x3 x2 x1 x

Figure 1: Graphical Solution of Fixed-Point Method.

Definition 2
(Fixed-Point Method)
The iteration defined in the following

xn+1 = g(xn ); n = 0, 1, 2, . . . , (3)

is called the fixed-point method or the fixed-point iteration. •

The value of the initial approximation x0 is chosen arbitrarily and the hope is that
the sequence {xn}∞ n=0 converges to a number α which will automatically satisfies (1).
Moreover, since (1) is a rearrangement of (2), α is guaranteed to be a zero of f (x). In
general, there are many different ways of rearranging of (2) in (1) form. However,
only some of these are likely to give rise to successful iterations but sometime we
don’t have successful iterations. To describe such behaviour, we discuss the following
theorem.
Theorem 3
(Fixed-Point Theorem)
If g is continuously differentiable on the interval [a, b] and g(x) ∈ [a, b] for all
x ∈ [a, b], then
(a) g has at-least one fixed-point in the given interval [a, b].
Moreover, if the derivative g 0 (x) of the function g(x) exists on an interval [a, b]
which contains the starting value x0 , with

k ≡ max |g 0 (x)| < 1; for all x ∈ [a, b]. (4)

a≤x≤b

Then
(b) The sequence (3) will converge to the attractive (unique) fixed-point α in
[a, b].
(c) The iteration (3) will converge to α for any initial approximation.
(d) We have the error estimate
kn
|α − xn | ≤ |x1 − x0 |, for all n ≥ 1. (5)
1−k
(e) The limit holds:
α − xn+1
lim = g 0 (α). (6)
n→∞ α − xn
Example 0.1
Consider the nonlinear equation x3 = 2x + 1 which has a root in the interval
[1.5, 2.0] using fixed-point method with x0 = 1.5, take three different rearrangements
for the equation and discuss which one is convergent or not.
Solution. Let us consider the three possible rearrangement of the given equation
as follows:
(x3n − 1)
(i) xn+1 = g1 (xn ) = ; n = 0, 1, 2, . . . ,
2
1
(ii) xn+1 = g2 (xn ) = ; n = 0, 1, 2, . . . ,
(x2n − 2)
s
(2xn + 1)
(iii) xn+1 = g3 (xn ) = ; n = 0, 1, 2, . . . ,
xn

then the numerical results for the corresponding iterations, starting with the
initial approximation x0 = 1.5 with accuracy 5 × 10−2 , are given in Table 1.
Example 0.1
Consider the nonlinear equation x3 = 2x + 1 which has a root in the interval
[1.5, 2.0] using fixed-point method with x0 = 1.5, take three different rearrangements
for the equation discuss which one is convergent or not.
Solution. Let us consider the three possible rearrangement of the given equation
as follows:
(x3n − 1)
(i) xn+1 = g1 (xn ) = ; n = 0, 1, 2, . . . ,
2
1
(ii) xn+1 = g2 (xn ) = ; n = 0, 1, 2, . . . ,
(x2n − 2)
s
(2xn + 1)
(iii) xn+1 = g3 (xn ) = ; n = 0, 1, 2, . . . ,
xn

then the numerical results for the corresponding iterations, starting with the
initial approximation x0 = 1.5 with accuracy 5 × 10−2 , are given in Table 1.
 Table: Solution of x3 = 2x + 1 by fixed-point method

n xn xn+1 = g1 (xn ) xn+1 = g2 (xn ) xn+1

p = g3 (xn )
= (x3n − 1)/2 = 1/(x2n − 2) = (2xn + 1)/xn
00 x0 1.500000 1.500000 1.500000
01 x1 1.187500 4.000000 1.632993
02 x2 0.337280 0.071429 1.616284
03 x3 -0.480816 -0.501279 1.618001
04 x4 -0.555579 -0.571847 1.618037
05 x5 -0.585745 -0.597731 1.618034

We note that the first two considered sequences diverge and the last one
converges. This example asks the need for a mathematical analysis of the method.
The following theorem gives sufficient conditions for the convergence of the
fixed-point iteration. •
Now we come back to our previous Example 0.1 and discuss that why the first two
rearrangements we considered, do not converge but on the other hand, last
sequence has a fixed-point and converges.
Since, we observe that f (1.5)f (2) < 0, then the solution we seek is in the interval
[1.5, 2].
x3 − 1
(i) For g1 (x) = , we have g10 (x) =(3/2) x2, which is greater than unity
2
throughout the interval [1.5, 2]. So by Fixed-Point Theorem 3 this iteration
will fail to converge.
1 −2x
(ii) For g2 (x) = 2 , we have g20 (x) = 2 , and |g20 (1.5)| > 1, so from
x −2 (x − 2)2
Fixed-Point Theorem 3 this iteration will fail to converge.
r
2x + 1 √ the
(iii) For g3 (x) = g0 (x) = -x−3/2/2 2x + 1 < 1, for all x in
x , we have 3
given interval [1.5, 2]. Also, g3 is decreasing function of x, and
g3 (1.5) = 1.63299 and g3 (2) = 1.58114 both lie in the interval [1.5, 2]. Thus
g3 (x) ∈ [1.5, 2], for all x ∈ [1.5, 2], so from Fixed-Point Theorem 3 the
iteration will converge, see Figure 2. •
 2

1.9

1.8
y = [(2x + 1)/x]1/2
1.7
α
1.6

y 1.5

1.4

1.3

1.2
y = x
1.1

1
1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2
x

Figure 2: Graphical Solution of x =

p
(2x + 1)/x.
Note 1
From (5) Note that the rate of convergence of the fixed-point method depends on
kn
the factor ; the smaller the value of k, then faster the convergence. The
(1 − k)
convergence may be very slow if the value of k is very close to 1. •
Note 2
Assume that g(x) and g0(x) are continuous functions of x for some open interval I,
with the fixed-point α contained in this interval. Moreover assume that

|g 0 (α)| < 1, for α ∈ I,

then, there exists an interval [a, b], around the solution α for which all the
conditions of Theorem 3 are satisfied. But if

|g 0 (α)| > 1, for α ∈ I,

then the sequence (3) will not converge to α. In this case α is called a repulsive
fixed-point. If
|g 0 (α)| = 0, for α ∈ I,
then the sequence (3) converges very fast to the root α while if

|g 0 (α)| = 1, for α ∈ I,

then the convergence the sequence (3) is not guaranteed and if the convergence
happened, it would be very slow. Thus to get the faster convergence, the value of
|g0(α)| should be equal to zero or very close to zero. •
Example 0.2
2 − e x + x2
Find an interval [a, b] on which fixed-point problem x = will
3
converges. Estimate the number of iterations n within accuracy 10−5 .
2 − ex + x2
Solution. Since x = can be written as
3

f (x) = ex − x2 + 3x − 2 = 0,

and we observe that f (0)f (1) = (−1)(e1 ) < 0, then the solution we seek is in the
interval [0, 1].
2 − ex + x2 2x − ex
For g(x) = , we have g 0 (x) = < 1, for all x in the given
3 3
interval [0, 1]. Also, g is decreasing function of x and g(0) = 0.3333 and
3−e
g(1) = = 0.0939 both lie in the interval [0, 1]. Thus g(x) ∈ [0, 1], for all
3
x ∈ [0, 1], so from Fixed-Point Theorem 3 the g(x) has a unique fixed-point in
[0, 1]. Taking x0 = 0.5, we have

2 − ex0 + x20
x1 = g(x0 ) = = 0.2004.
3
Example 0.2
2 − e x + x2
Find an interval [a, b] on which fixed-point problem x = will
3
converges. Estimate the number of iterations n within accuracy 10−5 .
2 − ex + x2
Solution. Since x = can be written as
3

f (x) = ex − x2 + 3x − 2 = 0,

and we observe that f (0)f (1) = (−1)(e1 ) < 0, then the solution we seek is in the
interval [0, 1].
2 − ex + x2 2x − ex
For g(x) = , we have g 0 (x) = < 1, for all x in the given
3 3
interval [0, 1]. Also, g is decreasing function of x and g(0) = 0.3333 and
3−e
g(1) = = 0.0939 both lie in the interval [0, 1]. Thus g(x) ∈ [0, 1], for all
3
x ∈ [0, 1], so from Fixed-Point Theorem 3 the g(x) has a unique fixed-point in
[0, 1]. Taking x0 = 0.5, we have

2 − ex0 + x20
x1 = g(x0 ) = = 0.2004.
3
Also, we have

k1 = |g 0 (0)| = 0.3333 and k2 = |g 0 (1)| = 0.2394,

which give k = max{k1 , k2 } = 0.3333. Thus the error estimate (5) within the
accuracy 10−5 is

(0.3333)n
|α − xn | ≤ 10−5 , gives (0.2996) ≤ 10−5 ,
1 − 0.3333
and by solving this inequality, we obtain n ≤ 9.7507. So we need ten
approximations to get the desired accuracy for the given problem. •
Example 0.3
Show that the function g(x) = 3−x on the interval [0, 1] has at least one fixed-point
but it is not unique.

Solution. Given x = g(x) = 3−x, and it can be written as

x − 3−x = f (x) = 0.

So f (0)(1) = (−1)(2/3) < 0, so f (x) has a root in the interval [0, 1], see Figure 3. Note
that g is decreasing function of x and g(0) = 1 and g(1) = 0.3333 both lie in the
interval [0, 1]. Thus g(x) ∈ [0, 1], for all x ∈ [0, 1], so from Fixed-Point Theorem 3 the
function g(x) has at least one fixed-point in [0, 1]. Since the derivative of the function
g(x) is

g0(x) = −3−x ln 3,

which is less than zero on [0, 1], therefore, the function g is decreasing on [0, 1]. But g0
(0) = − ln 3 = −1.0986, so |g 0 (x)| > 1 on (0, 1).

Thus from Fixed-Point Theorem 3 the function g(x) has no unique fixed-point in
[0, 1]. •
Example 0.3
Show that the function g(x) = 3−x on the interval [0, 1] has at least one
fixed-point but it is not unique.
Solution. Given x = g(x) = 3−x , and it can be written as

x − 3−x = f (x) = 0.

So f (0)(1) = (−1)(2/3) < 0, so f (x) has a root in the interval [0, 1], see Figure 3.
Note that g is decreasing function of x and g(0) = 1 and g(1) = 0.3333 both lie in
the interval [0, 1]. Thus g(x) ∈ [0, 1], for all x ∈ [0, 1], so from Fixed-Point
Theorem 3 the function g(x) has at least one fixed-point in [0, 1]. Since the
derivative of the function g(x) is

g 0 (x) = −3−x ln 3,

which is less than zero on [0, 1], therefore, the function g is decreasing on [0, 1]. But
g 0 (0) = − ln 3 = −1.0986, so

|g 0 (x)| > 1 on (0, 1).

Thus from Fixed-Point Theorem 3 the function g(x) has no unique fixed-point in
[0, 1]. •
 1

0.9
y = 3−x
0.8
y=x

0.7

0.6
y
0.5

0.4

0.3

0.2

0.1

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
x

Figure 3: Graphical Solution of x = 3−x.

Example 0.4
√
Show that the function g(x) = 2x − 1 on the interval [0, 1] that satisfies none of
the hypothesis of Theorem 3 √
but still has a unique fixed-point on [0, 1].

Solution. Since x = g(x) = 2x − 1, it gives

x2 − 2x + 1 = (x − 1)2 = f (x) = 0.

Then x = α = 1 ∈ [0, 1] is the root of the nonlinear equation f (x) = 0 and the
fixed-point of the function g(x) as g(1) = 1. But notice that the function g(x) is
not continuous on the interval [0, 1] and the derivative of the function g(x)

1
g 0 (x) = √ ,
2x − 1

does not exist on the interval (0, 1). So all the conditions of Fixed-Point
Theorem 3 fail. •
Example 0.4
√
Show that the function g(x) = 2x − 1 on the interval [0, 1] that satisfies none of
the hypothesis of Theorem 3 √
but still has a unique fixed-point on [0, 1].
Solution. Since x = g(x) = 2x − 1, it gives

x2 − 2x + 1 = (x − 1)2 = f (x) = 0.

Then x = α = 1 ∈ [0, 1] is the root of the nonlinear equation f (x) = 0 and the fixed-
point of the function g(x) as g(1) = 1. But notice that the function g(x) is not
continuous on the interval [0, 1] and the derivative of the function g(x)

1
g 0 (x) = √ ,
2x − 1

does not exist on the interval (0, 1). So all the conditions of Fixed-Point
Theorem 3 fail. •
Example 0.5
Show that the fixed point form of the equation x = N 1/3 can be written as
x = N x−2 and the associated iterative scheme

xn+1 = N x−2
n , n ≥ 0,

will not successful (diverge) in finding the approximation of cubic root of the
positive number N .
Solution. Given x = N 1/3 and it can be written as
N
x3 − N = 0 or x= = N x−2 .
x2
It gives the iterative scheme

xn+1 = N x−2
n = g(xn ), n ≥ 0.

From this, we have

g(x) = N x−2 and g 0 (x) = −2N x−3 .

Since α = x = N 1/3 , therefore

g 0 (α) = −2N α−3 and g 0 (N 1/3 ) = −2N (N 1/3 )−3 = −2N N −1 = −2.

Thus
|g 0 (N 1/3 )| = | − 2| = 2 > 1,
which shows the divergence. •
Example 0.5
Show that the fixed point form of the equation x = N 1/3 can be written as
x = N x−2 and the associated iterative scheme

xn+1 = N x−2
n , n ≥ 0,

will not successful (diverge) in finding the approximation of cubic root of the
positive number N .
Solution. Given x = N 1/3 and it can be written as
N
x3 − N = 0 or x= = N x−2 .
x2
It gives the iterative scheme

xn+1 = N x−2
n = g(xn ), n ≥ 0.

From this, we have

g(x) = N x−2 and g 0 (x) = −2N x−3 .

Since α = x = N 1/3 , therefore

g 0 (α) = −2N α−3 and g 0 (N 1/3 ) = −2N (N 1/3 )−3 = −2N N −1 = −2.

Thus
|g 0 (N 1/3 )| = | − 2| = 2 > 1,
which shows the divergence. •
Example 0.6
One of the possible rearrangement of the nonlinear equation ex = x + 2, which has
root in [1, 2] is

xn+1 = g(xn ) = ln(xn + 2); n = 0, 1, . . . .

(a) Show that g(x) has a unique fixed-point in [1, 2].

(b) Use fixed-point iteration formula (3) to compute approximation x3 , using
x0 = 1.5.
(c) Compute an error estimate |α − x3 | for your approximation.
(d) Determine the number of iterations needed to achieve an approximation with
accuracy 10−2 to the solution of g(x) = ln(x + 2) lying in the interval [1, 2] by
using the fixed-point iteration method.

Solution. Since, we observe that f (1)f (2) < 0, then the solution we seek is in the
interval [1, 2].
(a) For g(x) = ln(x + 2), we have g 0 (x) = 1/(x + 2) < 1, for all x in the given
interval [1, 2]. Also, g is increasing function of x, and
g(1) = ln(3) = 1.0986123 and g(2) = ln(4) = 1.3862944 both lie in the interval
[1, 2]. Thus g(x) ∈ [1, 2], for all x ∈ [1, 2], so from fixed-point theorem the g(x)
has a unique fixed-point, see Figure 4.
(b) using the given initial approximation x0 = 1.5, we have the other
approximations as
x1 = g(x0 ) = 1.252763, x2 = g(x1 ) = 1.179505, x3 = g(x2 ) = 1.156725.
(c) Since a = 1 and b = 2, then the value of k can be found as follows
k1 = |g 0 (1)| = |1/3| = 0.333 and k2 = |g 0 (2)| = |1/4| = 0.25,
which give k = max{k1 , k2 } = 0.333. Thus using the error formula (5), we
have
(0.333)3
|α − x3 | ≤ |1.252763 − 1.5| = 0.013687.
1 − 0.333
(d) From the error bound formula (5), we have
kn
|x1 − x0 | ≤ 10−2 .
1−k
By using above parts (b) and (c), we have
(0.333)n
|1.252763 − 1.5| ≤ 10−2 .
1 − 0.333
Solving this inequality, we obtain
n ln(0.333) ≤ ln(0.02698), gives, n ≥ 3.28539.
So we need four approximations to get the desired accuracy for the given
problem. •
 3.5 2

1.9
3
1.8
2.5
1.7
2 y = ex − x − 2 1.6
y

y
1.5 1.5

1.4 y=x y = ln(x + 2)

1
1.3
0.5
α 1.2 α
0
1.1

−0.5 1
1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2
x x

Figure 4: Graphical Solution of ex = x + 2 Graphical solution of x = ln(x + 2).

Example 0.7
Example 0.7
Procedure
(Fixed-Point Method))

1. Choose an initial approximation x0 such that x0 ∈ [a, b].

2. Choose a convergence parameter  > 0.
3. Compute new approximation xnew by using the iterative formula (3).
4. Check, if |xnew − x0 | <  then xnew is the desire approximate root; otherwise
set x0 = xnew and go to step 3.
Summary

In this lecture, we ...

I Introduced the Fixed-Point Method