Convergence of NM and Modified Newton Ralphon Method
Convergence of NM and Modified Newton Ralphon Method
Consider the problem of finding x ∗ , the solution of the equation: fx 0 for x in a, b . Assume that
′ ′
f x is continuous and f x ≠ 0 for x in a, b .
1. Newton’s Method:
Suppose that x ∗ is a simple zero of fx. Then we know fx x − x ∗ Qx where
lim∗ Qx ≠ 0.
x→x
In Section 2.2, the Newton Method is introduced for solving fx 0. Recall
Algorithm Newton Method: newton.m
Given fx, x 0 in a, b and , for n 1, 2, . . . ,
fx n−1
(1) compute x n x n−1 − ′
;
f x n−1
(2) if x n − x n−1 or fx n , then the algorithm is terminated and
x ∗ ≈ x n ; otherwise goto Step (1).
From Section 2.3, we know that the sequence of iterations x n n1 generated by Newton’s Method can
also be considered as iterations generated by the Fixed-Point Algorithm with function
fx
g newton x x − ′ .
f x
By the Fixed-Point Theorem, the sequence of iterations x n n1 converges to x ∗ if
g ′ x ≤ K 1 for all x in a, b
where x 0 in a, b , and the rate of convergence is OK n . We now investigate conditions on
fx under which the property g ′newton x 1 for x ∈ a, b is satisfied.
Observe that
′ ′ ′′ ′′ ′′
f xf x − fxf x fxf x fxf x
g ′newton x 1 − 1−1 .
′ 2 ′ 2 ′ 2
f x f x f x
′ ′ ′′
We know that f x ∗ 0. If f x ∗ ≠ 0, f x ≤ M 1 for all x in a, b and f x ≤ M 2 for all x
in a, b , then there exists an interval ā, b̄ ⊆ a, b such that
1
′′
fxf x
≤ K 1 and for x 0 in ā, b̄ , lim x x∗.
n→ n
′ 2
f x
Example Determine if the iterations generated by Newton’s Method for solving x 2 − 2 0 converges.
′
fx x 2 − 2, f x 2x, f ′′ x 2. Consider a, b 1, 2.
′′
′ fxf x 2x 2 − 2
g Newton x 1 1 − 22 ≤ g ′newton 1 1 K
f x
′ 2
2x 2 2 x 2
0.5
y
0.4
21 2
N for the number of iterations needed:
ln10 −8 1 − 0. 5/ 32 − 1
N 26. 575 425
ln0. 5
Let N 27.
Use the MatLab program newton.m to find an approximation.
fun@(x) x.^2-2;
2
fpfun@(x) 2*x;
[xsol, flag, ct]newton(fun, fpfun,1,10^(-8),100)
xsol
1.00000000000000
1.50000000000000
1.41666666666667
1.41421568627451
1.41421356237469
flag 1
ct 5
Example Find an approximation of the solution of the equation x 3 x 1 0 for x in −1, 0 within
10 −5 .
(1) Compute x n by (a) Newton’s Method with x 0 −1, (b) the Bisection Method and (c) the
Fixed point Iteration with gx −1 and p 0 −1.
x2 1
(2) Compare numerical results to determine which algorithm has the best performance for
this problem.
(3) Can we reach algebraically the same conclusion?
3
(c) Fixed-point Algorithm:
fixpt
input initial point p0 -1
input the tolerance for stopping criterion, ex:.0001,10^(-8) 10^(-5)
input the maximum number of iterations 100
Algorithm converges with number of iterations 25
fixed point p -0.68232442571947
(2) Compare the numbers of approximations generate by all three algorithms:
algorithm Newton Method Bisection Method Fixed-Point Iteration
Numerical of Approximations 5 17 25
The Newton Method has the best performance.
(3) (a) Newton Method:
0.8
g newton x x − x 2 x 1 , f x 6x,
3 ′′ y
3x 1
0.6
x 3 x 16x graphically
g ′newton x 0. 8
3x 2 1 2
0.4
x n x ∗ O0. 8 n
x 0 −1, x 1 g newton −1 − 34 0.2
−5
ln10 1 − 0. 8/ − 3
1
N 4
52. 59
ln0. 8 0.0
-1.0 -0.8 -0.6 -0.4 -0.2 0.0
Note that N 5 by the algorithm for 10 −5 . x
′
g x
(b) Bisection Method:
x n x ∗ O 21n x ∗ O0. 5 n
ln1 − 0 − ln10 −5
N 16. 609 640 5, N 17
ln2
(c) Fixed-Point Iteration:
4
Check the maximum value of |g ′ x| : y=0.7 y
0.6
g ′ x − 2x
2
x 1
2
0.4
2|x|
|g ′ x| 2
≤ 0. 7 K
x 2 1 0.2
n
x n x ∗ O0. 7
0.0
ln 10 −5 ln1 − 0. 7 -1.0 -0.5 0.0
N 35. 654 x
ln0. 7
Note that N 25 by the algorithm for 10 −5 . 2|x|
y |g ′ x|
2
x 1 2
Algebraically, we find the Bisection Method has the best performance so we cannot reach the same
conclusion as we have obtained from numerical results.
Order of convergence:
Theorem 2 Modified Fixed Point Theorem:
Let p be a fixed point of gx. If g ′ p 0 and g ′′ p ≠ 0, then p n → p quadratically ( 2)
with an asymptotic error constant 1 |g ′′ p|.
2
′ ′′
Proof: Assume that g p 0 but g p ≠ 0. By the Taylor Theorem, we have for x ≠ p,
g ′ p g ′′ cx
gx gp x − p x − p 2 , where cx is between x and p.
1! 2!
Then for x p n ,
g ′ p g ′′ cx n g ′′ cp n
gp n gp p n − p p n − p 2 gp p n − p 2
1! 2! 2!
where cp n−1 is between x and p n−1 .
g ′′ cp n−1
p n−1 − p 2 − gp 1 g ′′ cp n−1 p n−1 − p
2
p n − p gp n−1 − gp gp
2! 2
and
p n1 − p 1 g ′′ cp n 1 g ′′ lim cp n
lim
n→ 2
lim
n→ 2 n→
1 g ′′ p .
pn − p 2 2
So, the order of convergence is 2 and the asymptotic error constant 1 |g ′′ p|.
2
Theorem 3 Let x n be generated by the Newton Method. If x ∗ is a simple zero of fx, then x n → x ∗
′′
f x ∗
quadratically with an asymptotic error constant ′
.
2f x ∗
′′
fxf x
Proof: By the Modified Fixed-Point Theorem, we have g ′ x . Then
′ 2
f x
5
′ ′′ ′′′ ′ 2 ′′ ′
f xf x fxf x f x − fxf x 2f xf ′′ x
g ′′ x
′ 4
f x
′ ′′ ′′′ ′ ′′ 2
f xf x fxf x f x − 2fx f x
′ 3
f x
′ 2 ′′
fx ∗ 0 f x ∗ f x ∗ ′′
f x ∗
′′ ∗
g x ′
,
′ 3
f x ∗ f x ∗
′′
f x ∗
the asymptotic error constant is 1 |g ′′ x ∗ | ′
.
2 2f x ∗
2. Zero of Multiplicity m:
Definition A solution x ∗ of fx 0 is a zeros of multiplicity m of f if for x ≠ x ∗ , f can be expressed as
fx x − x ∗ m qx
where lim x→x ∗ qx ≠ 0.
Example Let fx e x − x − 1. f has a zero at x 0. Determine the multiplicity m of this zero.
Three ways to determine m.
′ ′′
a. Use Theorem 5: find m such that f 0 f 0 . . . f m−1 0 0 but f m 0 ≠ 0. Check:
′ ′ ′′ ′′
f x e x − 1, f 0 1 − 1 0. f x e x , f 0 1 ≠ 0. So, m 2.
b. Use the definition: find m such that fx x m qx where lim x→0 qx ≠ 0. That is
6
fx
lim ≠ 0.
x→0 xm
fx x x
lim x lim e − xx − 1 lim e − 1 0.
x→0 x→0 x→0 1
fx x x x
lim 2 lim e − x2 − 1 lim e − 1 lim e 1 ≠ 0
x→0 x x→0 x x→0 2x x→0 2 2
So, m 2.
c. Use the definition and Taylor series of fx.
1 x x x . . .
2 3
fx x −x−1
lim lim e − xm − 1 lim 2! 3!
x→0 xm x→0 x x→0 xm
x x . . .
2 3
lim 2! 3! 1 if m 2.
x→0 xm 2
So, m 2.
′ ′′′
f xf x fxf 4 x
1 1 lim∗ 1 0 1 1
2 2 xx ′′ 2 ′ ′′′ 2 2
f x f xf x
So, x n still converges to x ∗ . However, since lim xx ∗ g ′ x 1 ≠ 0, x n → x ∗ linearly instead of
′ ′′ ′′′
2
quadratically. If f x ∗ 0, f x ∗ 0 but f x ∗ ≠ 0, then
7
′ ′′′
f xf x fxf 4 x
lim∗ g x 1 1 lim∗
′
x→x 2 2 xx ′′ 2 ′ ′′′
f x f xf x
′′ ′′′
f xf x 2f ′ xf 4 x f xf 5 x
1 lim∗ ′′ ′′′ ′ 4
2 x→x 2 3f xf x f xf x
′′′ 2 ′′ 6
f x 3f ′′ xf 4 x 3f ′ xf 5 x f xf x
1 1 lim∗
2 2 x→x ′′′ 2 ′′ 4 ′ 5
3 f x 4f xf x f xf x
1 1 2
2 23 3
′ ′′ ′′′ 4
If f x ∗ 0, f x ∗ 0, f x ∗ 0 but f x ∗ ≠ 0, then
′′′ 2 ′′ 6
f x 3f ′′ xf 4 x 3f ′ xf 5 x f xf x
lim g ′ x 1 1 lim∗
x→x ∗ 2 2 x→x ′′′ 2 ′′ 4 ′ 5
3 f x 4f xf x f xf x
′′′
xf 4 x3f ′′′ xf 4 x3f ′′ xf 5 xf ′′′ xf 6 x3f ′′ xf 5 x3f ′ xf 6 xf ′′ xf 7 x
1 1 lim∗
2f
′′′
2 2 x→x 6f xf 4 x4f ′′′ xf 4 xf ′′ xf 5 x4f ′′ xf 5 xf ′ xf 6 x
′′′ ′′′ 6 ′′ 7
5f xf 4 x 6f ′′ xf 5 x f xf x 3f ′ xf 6 x f xf x
1 1 lim∗ ′′′ ′′ 5 ′ 6
2 2 x→x 10f xf 4 x 5f xf x f xf x
2 ′′′
5f 4 x 5f xf 5 x . . .
1 1
lim∗ 1 1 1 3
2 2 x→x 10f 4 x 2 10f ′′′ xf 5 x . . . 2 2 2 4
′ ′′ m m1
Theorem If f x ∗ 0, f x ∗ 0, … , f x ∗ 0 but f x ∗ ≠ 0, then
lim g ′ x m m 1.
x→x ∗
1. Example Consider solving the equation: lnx 1 − x 0 using Newton’s Method with x 0 1.
8
fun@(x) log(x1)-x;
fpfun@(x) 1./(x1)-1;
0.5
[xsol,flag,ct]newton(fun,fpfun,1,10^(-5),100)
xsol 0.48
1.00000000000000
0.38629436111989 0.46
0.17219218899218
0.44
0.08154040269002
0.03970514407998 0.42
0.01959491748522
0.4
0.00973408497441
0.00485132683348
0.38
1 2 3 4 5 6 7 8
0.00242175034192
x n1 −x ∗
flag 1 y x n −x ∗
ct 9
orderconv(xsol,0,9,1)
′ 1 − 1 0 when x 0 and f ′′ x − 1
Clearly, x n → 0 linearly. Since f x ≠0
1x 1 x 2
when x 0, x 0 is a zero of multiplicity 2 of f. Can we improve the convergence of Newton’s
Method for the problems where m 1?
9
g ′ x d 2g Newton x − x 2g ′Newton x − 1
dx
g ′ x ∗ 2g ′Newton x ∗ − 1 2 1 − 1 0,
2
x n converges to x ∗ and converges to x ∗ quadratically.
(2) This method can also be considered as solving x 0 using Newton’s Method where has a
simple zero at x ∗ .
′ ′
∗ fx 0
f x f x ∗
x lim
x→p
x lim
x→p ′
0
lim
x→p ′′
′′
0
f x L’Hopital f x f x ∗
′ 2 ′′
fx f x − fxf x fxf x
′′
x d
′
1−
dx f x
′
f x
′ 2
f x
′ 2
′′ ′′
fxf x fxf x
′ x ∗ lim∗ ′ x lim∗ 1 − 1 − lim∗
x→x x→x ′ 2 x→x ′ 2
f x f x
′ ′′ ′′′
0
f xf x fxf x
0
1 − lim∗ ′ ′′
x→x
L’Hopital 2f xf x
′ ′′ ′′′
f xf x fxf x
1− 1 1 lim∗ 1 −0 1 ≠ 0
2 x→x ′′ 2 ′ ′′′ 2 2
f x f xf x
Since ′ x ∗ ≠ 0, has a simple zero at x ∗ and x n → x ∗ quadratically.
Example Consider solving the equation: lnx 1 − x 0 using both modified Newton’s Methods with
x 0 1.
fun@(x) log(x1)-x;
fpfun@(x) 1./(x1)-1; 0.38
[xsol,flag,ct]newtonM(fun,fpfun,1,10^(-5),100,2) 0.36
xsol 0.34
1.00000000000000 0.32
(1) -0.22741127776022
0.3
-0.01951471341458
0.28
-0.00012819465259
0.26
flag 1
0.24
ct 4
0.22
orderconv(xsol,0,4,2)
1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3
x n1 −x ∗
y 2
x n −x ∗
10
and
f x − x 1 − 2x −
′′ 1 .
x 1 x 1 2
Define
fx lnx 1 − x lnx 1 − xx 1
x ′
−x − x
f x x1
and
′′
fxf x fx f ′′ x
′ x 1 − 1− ′ ′
′ 2
f x f x f x
fpfun@(x) 1(log(x1)-x).*(x1)./x.^2;
[xsol,flag,ct]newton(fun,fpfun,1,10^(-5),100) 0.3
xsol 0.25
1.00000000000000
0.11460991822207 0.2
0.00366562036799
0.15
0.00000445171110
flag 1 0.1
1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3
ct 4 x n1 −x ∗
y 2
orderconv(xsol,0,4,2) x n −x ∗
Example Use the Newton Method to solve fx e 3x − 27x 6 27x 4 e x − 9x 2 e 2x for x in 3, 5.
Determine the multiplicity of the zero of fx.
fx e 3x − 27x 6 27x 4 e x − 9x 2 e 2x
′
f x 3e 3x − 162x 5 274x 3 e x x 4 e x − 92xe 2x 2x 2 e 2x
3e 3x − 162x 5 27x 3 4 xe x − 18x1 xe 2x
fun@(x) exp(3*x)-27*x.^627*x.^4.*exp(x)-9*x.^2.*exp(2*x);
fpfun@(x) 3*exp(3*x)-162*x.^527*x.^3*(4x).*exp(x)-18*x.*(1x).*exp(2*x);
[xsol,flag,ct]newton(fun,fpfun,3,10^(-8),200)
ct24
xsol(ct)3.73315956142332
for k1:ct-2;
r1(k)abs(xsol(k1)-xsol(ct))/abs(xsol(k)-xsol(ct));
end
plot(r1)
11
title(’|x_{n1}-x*|/x_n-x*|’)
for k1:ct-2;
r2(k)abs(xsol(k1)-xsol(ct))/(abs(xsol(k)-xsol(ct)))^2;
end
plot(r2)
title(’|x_{n1}-x*|/x_n-x*|^2’)
2
|x -x*|/|x -x*| |x -x*|/|x -x*|
k+1 k k+1 k
0.75 4000
0.7 3500
0.65 3000
0.6 2500
0.55 2000
0.5 1500
0.45 1000
0.4
500
0.35
0 5 10 15 20 25 0
0 5 10 15 20 25
12
2
|x -x*|/x -x*| |x -x*|/x -x*|
n+1 n n+1 n
2.2 700
2
600
1.8
1.6 500
1.4
400
1.2
300
1
0.8 200
0.6
100
0.4
0.2 0
0 5 10 15 0 5 10 15
13
2
|x -x*|/x -x*| |x -x*|/x -x*|
n+1 n n+1 n
4 5
4.5
3.5
4
3
3.5
2.5
3
2 2.5
2
1.5
1.5
1
1
0.5
0.5
0 0
0 2 4 6 8 0 2 4 6 8
14
Exercises:
1. Consider solving the equation cosx − x 0 for x in 0, .
2
(1) Compare numerical results you generated in Problem 4 of Section 2.3 to determine which algorithm
among the Newton Method, the Bisection Method and the Fixed-Point Iteration has the best performance
for this problem.
(2) What can we say algebraically about the performances of these three methods for this problem?
Show your work in detail to support your answer.
5. The sum of two real numbers is 20. If each number is added to its square root, the product of the two
sums is 155.55. Determine the two numbers to within 10 −8 by the Newton Method.
15