1

Chapter 4: Solving Nonlinear Equation

Vu Van Thieu, Dinh Viet Sang, Ban Ha Bang

SoICT
Hanoi University of Science and Technology

2
Introduction

1 Problem
Existence and uniqueness of solutions
Sensitivity and conditions for solving nonlinear equations
Iterative procedure
2 Bisection method
3 Chord method
4 Newton’s method
5 Secant method
6 Iterative method
7 Bairstow method

3
Solve nonlinear equations

Problem
Given the non-linear function f (x), we need to find x satisfying

f (x) = 0.

The solution x is the solution of the equation and is also called the (zero point) solution of the
function f (x). The problem of finding x is called the root finding problem.

4
Solve nonlinear equations

Examples of problems finding solutions of nonlinear equations

1 1 + 4x − 16x 2 + 3x 3 − 3x 4 = 0
√
x (2.1−0.5x)
2 √ − 369 = 0 with (0 < x < 1)
(1−x) (1.1−0.5x)
e x −e −x
3 tg (x) − tanh(x) = 0 where tanh(x) = e x +e −x

5
Solve nonlinear equations

Problem
If the equations f (x) are nonlinear then
it usually does not have an explicit formula solution
numerical methods that allow us to find solutions based on iterative procedure

6
Solve nonlinear equations

Solution interval
For function f : R 7→ R the interval [a, b] is called solution interval if function f has opposite
signs at both ends a, b, i.e. f (a)f (b) < 0.

Existence of solutions
If f is a continuous function on the interval [a, b] and f (a)(f (b) < 0 then there exists
x ∗ ∈ [a, b] such that f (x ∗ ) = 0.

7
Solve nonlinear equations

Examples of solutions to nonlinear equations

1 e x + 1 = 0 has no solution
2 e −x − x = 0 has a solution
3 x 2 − 4sin(x) = 0 has two solutions
4 x 3 − 6x 2 + 11x − 6 = 0 has three solutions
5 cos(x) = 0 has infinitely many solutions

8
Solve nonlinear equations

Conditions for solving equations

1
The absolute value of the Condition number x ∗ of the function f : R 7→ R is |f ′ (x ∗ )| .
A solution of x ∗ is said to be ill-conditioned (well) if the tangent line to the graph of the
function f(x) atx ∗ is almost horizontal (vertical).

9
Solve nonlinear equations

Iterative procedure
Nonlinear equations often do not have an explicit solution. Therefore, to find solution we often
have to use numerical methods based on iterative procedures.
Stop condition: |f (xk )| < ϵ or |x ∗ − xk | < ϵ where ϵ is the given precision and xk is the
approximate solution obtained at step k
Convergence rate: We denote error at iteration k as: ek = xk − x ∗ . The sequence {ek }
is said to converge to the degree r if

|ek+1 |
lim =C
k→∞ |ek |r

where C is a non-zero constant

10
 Solve nonlinear equations
Scientific Computation
2023-12-29
Iterative procedure

Problem Nonlinear equations often do not have an explicit solution. Therefore, to find solution we often
have to use numerical methods based on iterative procedures.
Stop condition: |f (xk )| < ϵ or |x ∗ − xk | < ϵ where ϵ is the given precision and xk is the

Iterative procedure approximate solution obtained at step k

Convergence rate: We denote error at iteration k as: ek = xk − x ∗ . The sequence {ek }
is said to converge to the degree r if

Solve nonlinear equations lim

k→∞
|ek+1 |
|ek |r
=C

where C is a non-zero constant

Name of the speed of convergence in some cases

• r = 1 - linear convergence rate
• r > 1 - linear convergence rate
• r = 2 - convergence rate squared
Question

11
1 Problem
Existence and uniqueness of solutions
Sensitivity and conditions for solving nonlinear equations
Iterative procedure

2 Bisection method

3 Chord method

4 Newton’s method

5 Secant method

6 Iterative method

7 Bairstow method

12
Bisection method

Iterative procedure
Assume that the solution interval [a, c] has only one solution
1 Reduce the size of the solution interval through division
(a+c)
2 The division to be performed is halving b = 2
If f (b) = 0 then b is the correct solution,
otherwise if f (b) ̸= 0 we have
▶ f (a)f (b) < 0 then the new solution interval is [a, b]
▶ Otherwise, the new interval is [b, c]
Steps 1-2 are repeated until [a, c] < ϵ (the given error)

13
 Bisection method
Scientific Computation
2023-12-29 Bisection method Iterative procedure
Assume that the solution interval [a, c] has only one solution
1 Reduce the size of the solution interval through division
(a+c)
2 The division to be performed is halving b = 2
If f (b) = 0 then b is the correct solution,
otherwise if f (b) ̸= 0 we have
▶ f (a)f (b) < 0 then the new solution interval is [a, b]

Bisection method ▶ Otherwise, the new interval is [b, c]

Steps 1-2 are repeated until [a, c] < ϵ (the given error)

If precision ϵ is given, the number of iterations is an integer n satisfying

ca
n ≥ log2
ϵ
because of
ca
<ϵ
2n
Bisection method
Example 1:
Find the solution of the equation e x − 2 = 0, given a solution interval [0.2] with precision
ϵ = 0.01
Loops a b c f(a) f(b) f(c) error
1 0.0000 1.0000 20000 -1.0000 0.7183 5.0389 20000
2 0.0000 0.5000 1,00000 -1.0000 -0.3513 0.7183 1,0000
3 0.5000 0.7500 1,0000 -0.3513 0.1170 0.7183 0.5000
4 0.5000 0.6250 0.7500 -0.3513 -0.1318 0.1170 0.2500
5 0.6250 0.6875 0.7500 -0.1318 -0.0113 0.1170 0.1250
6 0.6875 0.7188 0.7500 -0.0113 0.0519 0.1170 0.0625
7 0.6875 0.7031 0.7188 -0.0113 0.0201 0.0519 0.0313
8 0.6875 0.6953 0.7031 -0.0113 0.0043 0.0201 0.0156
9 0.6875 0.6914 0.6953 -0.0113 -0.00349 0.0043 0.0078

14
Bisection method
Example 2
Consider the solution of the equation f (x) = 1/(xe −x ), given a solution interval [0, 1] with
precision ϵ = 0.00001

Error: ek = max{x ∗ − ak , ck − x ∗ }
Horizontal axis: number of iterations
k
Vertical axis: − log2 (ek )
Apparently ek ≈ 2−k

15
Bisection method

Comment on the Bisection method

Strengths: Works even with non-analytic functions.
Weaknesses:
▶ Need to determine a solution interval and find only one solution.
▶ Could not find a double solution.
▶ When the function f has singularities, the bisection method can treat them as solutions.

16
Bisection method

20 16
10
14
15
12
5
10
10

5 8
−3 −2 −1 1 2 3
6
1 2 3 4 5 6
4 −5
−5
2

−10 a) 1 2 3 4 5 6 b) c)
Many solutions Double root Singularity

17
Bisection method

18
1 Problem
Existence and uniqueness of solutions
Sensitivity and conditions for solving nonlinear equations
Iterative procedure

2 Bisection method

3 Chord method

4 Newton’s method

5 Secant method

6 Iterative method

7 Bairstow method

19
Chord method

Iterative procedure
Assume that the solution interval [a, c] has only one solution
1 Reduce the size of the solution interval through division
ca af (c)−cf (a)
2 The division to be performed is b = a − f (c)−f (a) f (a) = f (c)−f (a) If f (b) = 0 then b is
the solution, Otherwise, if f (b) ̸= 0, we have:
▶ If f (a)f (b) < 0 then the new interval is [a, b]
▶ Otherwise, the new interval is [b, c]
Steps 1-2 are repeated until [a, c] < ϵ (given error).
So b is the intersection of the horizontal axis with the line segment connecting A(a,f(a)) to B(
c, f(c))

20
Chord method

30

B(c,f(c))
20

10

b1 b2
3.5 4 4.5 5 5.5 6 6.5

A(a,f(a))
−10

21
Chord method

Comment
Advantages: like bisection, we do not need the analytic form of the equation f
Disadvantages:
▶ Need to know the solution interval
▶ Single-sided convergence is slow, especially when the solution interval is large
▶ Can be improved by using together with the bisection method

22
Chord method

23
1 Problem
Existence and uniqueness of solutions
Sensitivity and conditions for solving nonlinear equations
Iterative procedure

2 Bisection method

3 Chord method

4 Newton’s method

5 Secant method

6 Iterative method

7 Bairstow method

24
Newton’s method

Description
The basic idea of the method is to replace the nonlinear equation f (x) = 0 with an
approximate, linear equation for x based on Taylor expansion.
Assuming f (x) is continuously differentiable to the order n + 1 then there exists ξ ∈ (a, b)

(b − a)2
f (b) = f (a) + (b − a)f ′ (a) + f ”(a) + · · ·
2!
(b − a)n (n) (b − a)(n+1) (n+1)
+ f (a) + f (ξ)
n! (n + 1)!

25
Newton’s method

Description (continued)
Taylor expansion of f (x) at the neighborhood of the original approximate solution x0 :

f (x) = f (x0 ) + f ′ (x0 )(x − x0 ) + O(h2 )

where h = x − x0 .
Solve the approximate equation for x:

f (x0 ) + f ′ (x0 )(x − x0 ) = 0

Obtained: x = x0 − ff ′(x 0)
(x0 )
x is an incorrect solution, but this solution will be closer to the correct solution than the initial
value x0 .

26
Newton’s method

Iterative procedure
1 Initialize with x0
2 Calculates with k > 0
f (xk )
xk+1 = xk −
f ′ (xk )
3 Repeat step 2 until |f (xk )| < ϵ where ϵ is the given precision

27
Newton’s method

20

15

10

x2 x1 x0
1 2 3 4 5 6 7

28
Newton’s method

Comment
Advantages:
▶ For a smooth enough function and we start from the point near the solution, the convergence
rate of the method is squared or r = 2
▶ No need to know the solution dissociation, just the initial point x0
Disdvantages:
▶ Need to calculate the first derivative f ′ (xk ), we can also approximate it with the formula
(xk −h)
f ′ (xk ) = f (xk +h)−f
2h where h is a very small value eg h = 0.001
▶ Not always converges

29
Newton’s method

Example 1
Use Newton’s method to find the solution of the equation

f (x) = x 2 − 4 sin(x) = 0

First derivative of f (x)

f ′ (x) = 2x − 4 cos(x)
The iterative formula of Newton’s method is
f (xk ) xk2 − 4 sin(xk )
xk+1 = xk − = xk −
f ′ (xk ) 2xk − 4 cos(xk )

Approximate starting point x0 = 3

30
Newton’s method

k xk f (xk )
0 3.000000 8.435520
1 2.153058 1.294773
2 1.954039 0.108439

31
Newton’s method

Example 2
Solve the equation f (x) = x 2 − 2 = 0 because f ′ (x) = 2x so the iterative formula will be
xk2 −2 √
∗ =x − 2
xk+1 = xk − 2x k
error e k = x k − x k

k xk ek
0 4,000000000 2.5857864376
1 2.250000000 0.8357864376
2 1.569444444 0.1552308821
3 1.421890364 0.0076768014
4 1.414234286 0.0000207236
5 1.414213563 0.0000000002

32
Newton’s method

Example 3
Solve the equation p
f (x) = sign(xa) |xa|
This equation satisfies:
f (x)
xa − = −(xa)
f ′ (x)
The zero point of the function is x ∗ = a.
If we draw a tangent to the graph at any point, it always intersects the horizontal axis at the
point of symmetry with the line x = a.
Newton’s method infinitely iterative, neither convergent nor divergent.

33
Newton’s method

34
Newton’s method

35
1 Problem
Existence and uniqueness of solutions
Sensitivity and conditions for solving nonlinear equations
Iterative procedure

2 Bisection method

3 Chord method

4 Newton’s method

5 Secant method

6 Iterative method

7 Bairstow method

36
Secant method

Iterative procedure
An improvement of Newton’s method, instead of using the derivativef ′ (x), we use an
approximate difference based on two successive iterations.
1 Starts with two starting points x0 and x1
2 With k ≥ 2, we iterate by the formula

f (xk ) − f (xk−1 )
sk =
xk − xk−1
f (xk )
xk+1 = xk −
sk

3 Repeat step 2 until |f (xk )| < ϵ (given error).

37
Secant method

20

15

10

x2 x1 x0
1 2 3 4 5 6 7

38
Secant method

Comment
Advantages:
▶ No need to know the solution interval, just two initial points x0 and x1
▶ No need to calculate first derivative f ′ (xk )
Disdvantages:
▶ Two initialization points are required √
1+ 5
▶ Convergence rate of method on linear 1 < r < 2, specifically golden ratio r ≈ 2 = 1.618

39
Secant method

Example 1
Solve the equation p
f (x) = sign(x − 2) |x − 2| = 0
with two starting points x0 = 4, x1 = 3 and ϵ = 0.001

f (xk ) − f (xk−1 )
sk =
xk − xk−1
p p
sign(xk − 2) |xk − 2| − sign(xk−1 − 2) |xk−1 − 2|
=
xk − xk−1
p
f (xn ) sign(xk − 2) |xk − 2|
xk+1 = xk − = xk −
sn sk

40
Secant method

k xk ek
0 4,000000000 2,000,000000000
1 3,000000000 1,000,000,000,000
2 0.585786438 1.4142135624
3 1.897220119 0.1027798813
.. .. ..
. . .
26 1.999989913 0.0000100868
27 1.999998528 0.0000014716
28 2,000003853 0.0000038528
29 2.000000562 0.0000005621

41
Secant method

Example 2
Solve the equation
f (x) = e −x/2 sin(3x) = 0
with two starting points x0 , x1 and precision ϵ entered from the keyboard

f (xk ) − f (xk−1 )
sk =
xk − xk−1
e −xk /2 sin(3xk ) − e −xk−1 /2 sin(3xk−1 )
=
xk − xk−first
f (xn ) e −xk /2 sin(3xk )
xk+1 = xk − = xk −
sn sk

42
Secant method

Two starting points x0 = 4

x1 = 5
Accuracy
ϵ = 0.001

43
Secant method

44
1 Problem
Existence and uniqueness of solutions
Sensitivity and conditions for solving nonlinear equations
Iterative procedure

2 Bisection method

3 Chord method

4 Newton’s method

5 Secant method

6 Iterative method

7 Bairstow method

45
Iterative method

Fixed point
Instead of writing the equation as f (x) = 0, we rewrite it as a problem

Find x satisfying x = g (x)

The point x ∗ is called fixed point of the function g (x) if x ∗ = g (x ∗ ), i.e. the point x ∗ is not
transformed by g . mapping

46
Iterative method

2.5

2 (x ∗ , g (x ∗ )
g (x ∗ )

1.5

0.5

x∗
0.5 1 1.5 2 2.5

47
Iterative method

Examples
Newton’s method, according to the formula xk+1 = xk − ff ′(x k)
(xk ) , the function g that we
need to find the fixed point x ∗ would be g (x) = x − f (x)/f ′ (x)
Find the solution f (x) = x − e −x ⇒ g (x) = e −x
√
Find the solution f (x) = x 2 − x − 2 ⇒ g (x) = x + 2 or g (x) = x 2 − 2
Find the solution f (x) = 2x 2 − x − 1 ⇒ g (x) = 2x 2 − 1

48
Iterative method

Iterative procedure
Approach to problem solving

xk+1 = g (xk ) with k = 1, 2, · · ·

The above iterative procedure is often called an iterative find fixed point with a given
starting point x1

49
Iterative method

Comment
Advantages:
▶ No need to know the solution interval
Disdvantages:
▶ does not always converge

50
Iterative method

Example 1
Finding the solution of the equation f (x) = x 2 − x − 2 = 0 can lead to finding a fixed point
√
g (x) = x + 2

Approximate starting point x1 = −1

51
Iterative method

Starting point
x1 = −1
Number of iterations
n=3

52
Iterative method

Example 2
Find the solution of the equation f (x) = x 2 − x − 2 by finding the point of disagreement of
the function
g (x) = x 2 − 2
The starting point x1 = 2.02 is very close to the solution

53
Iterative method

Starting point
x1 = 2.02
Number of iterations
n = 50

54
Iterative method

Convergence theorem of iterative methods

Theorem 1: Assume the function g (x) is continuous and the sequence repeats

xk+1 = g (xk ), k = 1, 2, · · ·

then if xk → x ∗ when k → ∞ then x ∗ is the fixed point of g .

Theorem 2: Suppose g ∈ C 1 and |g ′ (x)| < 1 in some interval containing the fixed point x ∗ .
If x0 is also in this range, the iterative sequence {xk } converges to x ∗ .
Theorem 3: If g is a co function then it has only one fixed point and the iterative sequence
{xk } converges to x ∗ for all points starting x0 .
Note: The function g is called a co function if there is a constant L < 1 such that for every
x, y we have: |g (x) − g (y )| < L(xy )).

55
Iterative method

56
1 Problem
Existence and uniqueness of solutions
Sensitivity and conditions for solving nonlinear equations
Iterative procedure

2 Bisection method

3 Chord method

4 Newton’s method

5 Secant method

6 Iterative method

7 Bairstow method

57
Bairstow method

Description
This is the method used to find the solution of a polynomial,:

y = a0 + a1 x + · · · + aN xN

It can be rewritten as a quadratic factor plus a remainder

y = (x 2 + px + q)G (x) + R(x)

Where:
p and q are arbitrary values.
G (x) is a polynomial of degree N − 2
R(x) is the remainder, usually a first degree polynomial.

58
Bairstow method

Description (continued)
So the polynomial G (x) and the remainder R(x) have the form

G (x) = b2 + b3 x + b4 x 2 + · · · + bN x N−2
R(x) = b0 + b1 x

The value of b0 and b1 depends on the choice of p and q, the goal is to find p = p ∗ and
q = q ∗ such that
b0 (p ∗ , q ∗ ) = b1 (p ∗ , q ∗ ) = 0 ⇒ R(x) = 0
(x 2 + p ∗ x + q ∗ ) ⇒ square factor of y

59
Bairstow method

Iterative procedure
1 Initializes p and q and calculates b0 and b1 (see formular in the Ref book)
2 Calculates the values (b0 )p ,(b1 )p and (b0 )q ,(b1 )q (see formular in the Ref book)
3 Find ∆p and ∆q when solving equation (9)
4 Obtained p ∗ and q ∗ by the formula p ∗ = p + ∆p and q ∗ = q + ∆q

60
Bairstow method

Comment
Advantages:
▶ method that converges to the quadratic factor (x 2 + px + q) regardless of the initialization
value p, q
▶ the coefficients of the polynomial G (x) are also automatically obtained
Disdvantages:
▶ the accuracy of the solution is not high
▶ to improve, you can use Newton’s method to recalculate each solution

61