§2.

3 Newton’s Method for solving f (p) = 0

▶ Suppose that f ∈ C 2 [a, b].
▶ Let p0 ∈ [a, b] be an approximation to p with

f ′ (p0 ) ̸= 0, and |p − p0 | ”small”.

▶ Taylor expand f (x) at x = p:

(p − p0 )2 ′′
0 = f (p) = f (p0 ) + (p − p0 )f ′ (p0 ) + f (ξ(p)).
2
”Solve” for p:

f (p0 ) (p − p0 )2 ′′
p = p0 − − f (ξ(p))
f ′ (p0 ) 2f ′ (p0 )
f (p0 ) def
≈ p0 − ′ = p1 .
f (p0 )
f (pk )
Newton’s Method: pk+1 = pk − f ′ (pk ) , k = 0, 1, · · ·
1/48
Newton’s Method for solving f (p) = 0

f (p0 ) (p − p0 )2 ′′
p = p0 − − f (ξ(p))
f ′ (p0 ) 2f ′ (p0 )
f (p0 ) def
≈ p0 − ′ = p1 .
f (p0 )

▶ If p0 ”close to” p, we can expect fast convergence.

▶ Best hope in practice: p0 ”not too far from” p. Newton’s
method may or may not converge.
▶ If Newton’s method converges, it converges quickly.

2/48
Geometry of Newton’s Method

▶ Taylor expand f (x) at x = p:

(p − p0 )2 ′′
0 = f (p) = f (p0 )+(p−p0 )f ′ (p0 )+ f (ξ(p)).
2
▶ Replace f (x) by a straight line:

f (p0 ) + (p − p0 )f ′ (p0 ) ≈ 0.

▶
f (p0 )
p ≈ p0 − is
f ′ (p0 )
horizontal intercept of straight line

y = f (p0 ) + (x − p0 )f ′ (p0 )

3/48
4/48
5/48
Newton’s Method as Fixed Point Iteration
Assume f (p) = 0 is a root, and x close to p. Would like a fixed
point iteration

pk+1 = g (pk ), k = 0, 1, · · · ,

where g (x) = x − f (x) h(x). How to choose h(x)?

6/48
Newton’s Method as Fixed Point Iteration
Assume f (p) = 0 is a root, and x close to p. Would like a fixed
point iteration

pk+1 = g (pk ), k = 0, 1, · · · ,

where g (x) = x − f (x) h(x). How to choose h(x)? Since

g ′ (x) = 1 − f ′ (x) h(x) − f (x) h(x)′ ≈ 1 − f ′ (x) h(x).

”optimal choice”: g ′ (x) ≈ 0, so that

1
h(x) =
f ′ (x)

and so
f (pk )
pk+1 = pk − , k = 0, 1, · · · .
f ′ (pk )

6/48
Newton Method for f (x) = e x − (2 + 2x 2 )
Newton Method with 8 Points
8

−2

−4
2.6 2.8 3 3.2 3.4 3.6 3.8

7/48
Computing square root with Newton’s Method

√
▶ Given a > 0, p def
= a is positive root of equation
def
f (x) = x 2 − a = 0.

▶ Newton’s Method

p2 − a
 
1 a
pk+1 = pk − k = pk + , k = 0, 1, 2, · · · ,
2pk 2 pk
▶ Newton’s Method is well defined for any p0 > 0.

8/48
Computing square root with Newton’s Method: Analysis

Assume that p0 > 0, then for k = 0, 1, 2, · · · ,

▶ Newton’s Method satisfies
√ √ √
2
 
1 a 1
pk+1 − a = pk + − a= pk − a ≥ 0,
2 pk 2 pk
√
▶ It follows that pk − a ≥ 0 for all k ≥ 1.
▶ Newton’s Method converges for any p0 > 0:
√ 1 √
1 √

0 ≤ pk+1 − a≤ pk − a ≤ · · · ≤ k p1 − a −→ 0.
2 2

9/48
Newton Method for square root

10/48
Newton Method: Think 5 (if it works)

11/48
12/48
Proof of Theorem 2.6
▶ Newton’s method is fixed point iteration
pn = g (pn−1 ), g (x) = x − ff ′(x)
(x) .
′
▶ Since f (p) ̸= 0, there exists an interval
[p − δ1 , p + δ1 ] ⊂ [a, b] on which f ′ (x) ̸= 0. Thus, g (x) is
defined on [p − δ1 , p + δ1 ].
▶
f ′ (x) f ′ (x) − f (x) f ′′ (x) f (x) f ′′ (x)
g ′ (x) = 1− = ∈ C [p−δ1 , p+δ1 ].
(f ′ (x))2 (f ′ (x))2
▶ Since g ′ (p) = 0, there exists 0 < δ < δ1 so that
1
|g ′ (x)| ≤ κ (= ), for all x ∈ [p − δ, p + δ].
2
▶ If x ∈ [p − δ, p + δ], then
|g (x)−p| = |g (x)−g (p)| = |g ′ (ξ)(x −p)| ≤ κ|x −p| ≤ |x −p|.
Therefore g (x) ∈ [p − δ, p + δ].
▶ {pn } converges to p by Fixed Point Theorem.
13/48
Newton Method Divergence Example: f (x) = x 1/3

1/3
f (xk ) xk
xk+1 = xk − ′
= xk − −2/3
= xk − 3 xk = −2 xk
f (xk ) 1/3 xk
14/48
Secant Method: Poor man’s Newton Method
Motivation

▶ Newton method style of fast convergence

▶ Avoid need for derivative calculations.

Approach
f (pn )
▶ Newton method: pn+1 = pn − f ′ (pn ) .
▶ Replace f ′ (pn ) by its cheap approximation

f (pn ) − f (x) f (pn ) − f (pn−1 )

f ′ (pn ) = lim ≈ .
x→ pn − x pn − pn−1
▶ Secant method
f (pn )(pn − pn−1 )
pn+1 = pn − , n = 1, 2, · · · .
f (pn ) − f (pn−1 )

15/48
Secant Method: Geometry

▶ ”Approximate” f (x) by a straight line

(x − p0 )f (p1 ) − (x − p1 )f (p0 )
f (x) ≈ .
p1 − p0

Both f (x) and straight line go through

points (p0 , f (p0 )), (p1 , f (p1 )).
▶ Let approximate root p2 be the
x-intercept of the straight line

p0 f (p1 ) − p1 f (p0 ) f (p1 )(p1 − p0 )

p2 = = p1 − .
f (p1 ) − f (p0 ) f (p1 ) − f (p0 )

16/48
Fixed point for f (x) = e x − (2 + 2x 2 )
SecantMethod with 11 Points
25

20

15

10

−5
2.5 3 3.5 4

17/48
Newton Method for f (x) = e x − (2 + 2x 2 )
Newton Method with 8 Points
8

−2

−4
2.6 2.8 3 3.2 3.4 3.6 3.8

18/48
Newton Method for f (x) = e x − (2 + 2x 2 )
Newton Method with 8 Points
8

−2

−4
2.6 2.8 3 3.2 3.4 3.6 3.8

Fewer iterations, and therefore less work?

18/48
§2.4 Number of iterations vs. error in the solution

▶ Function to be considered

g (x) = log (2 + 2x 2 ), f (x) = x − g (x) = x − log (2 + 2x 2 ).

▶ Root p of f (i.e., f (p) = 0)

p = 2.98930778246493e + 00.

Method comparison:
▶ Bisection Method: 52 iterations
▶ Fixed Point Iteration: 66 iterations
▶ Newton’s Method: 8 iterations
▶ Secant Method: 11 iterations

19/48
§2.4 Number of iterations vs. error in the solution

▶ Function to be considered

g (x) = log (2 + 2x 2 ), f (x) = x − g (x) = x − log (2 + 2x 2 ).

▶ Root p of f (i.e., f (p) = 0)

p = 2.98930778246493e + 00.

Method comparison:
▶ Bisection Method: 52 iterations
▶ Fixed Point Iteration: 66 iterations
▶ Newton’s Method: 8 iterations
▶ Secant Method: 11 iterations
Even in convergence, rates differ

19/48
Bisection Method Order of Convergence
Convergence Rate, Bisection Method
0
10

−2
10

−4
10

−6
10

−8
10

−10
10

−12
10

−14
10

−16
10
0 10 20 30 40 50 60

20/48
Fixed Point Iteration Order of Convergence
Convergence Rate, Fixed Point Method
0
10

−5
10

−10
10

−15
10
0 10 20 30 40 50 60 70

21/48
Secant Method Order of Convergence
Convergence Rate, Secant Method
2
10

0
10

−2
10

−4
10

−6
10

−8
10

−10
10

−12
10
1 2 3 4 5 6 7 8 9 10

22/48
Newton Method Order of Convergence
Convergence Rate, Newton Method
0
10

−2
10

−4
10

−6
10

−8
10

−10
10

−12
10

−14
10
1 2 3 4 5 6 7

23/48
Linear and Quadratic Order of convergence

Definition Suppose {pn }∞ n=1 is a sequence that converges to p

with pn ̸= p for all n. If positive constants λ and α exist with

|pn+1 − p|
lim = λ,
n→∞ |pn − p|α

then {pn }∞
n=1 converges to p of order α, with asymptotic error
constant λ.

24/48
Linear and Quadratic Order of convergence

Definition Suppose {pn }∞ n=1 is a sequence that converges to p

with pn ̸= p for all n. If positive constants λ and α exist with

|pn+1 − p|
lim = λ,
n→∞ |pn − p|α

then {pn }∞
n=1 converges to p of order α, with asymptotic error
constant λ.
▶ If α = 2, then {pn }∞
n=1 converges quadratically.
▶ If α = 1 and λ < 1, {pn }∞
n=1 converges linearly.
▶ If α = 1 and λ = 0, {pn }∞
n=1 converges super-linearly.

24/48
Recall and contrast: rate of convergence, the Big O

the Big O() = rate of convergence

25/48
Linear and Quadratic Order of convergence (I)
▶ Suppose that {pn }∞
n=1 is linearly convergent to 0,

|pn+1 | |pn+1 |
lim = 0.5, or roughly ≈ 0.5,
n→∞ |pn | |pn |

hence pn ≈ (0.5)n |p0 |.

▶ Suppose that {p̃n }∞
n=1 is quadratically convergent to 0,

|p̃n+1 | |p̃n+1 |
lim = 0.5, or roughly ≈ 0.5.
n→∞ |p̃n |2 |p̃n |2

But now

26/48
Linear and Quadratic Order of convergence (II)

n
Consider pn ≈ (0.5)n and p̃n ≈ (0.5)2 .

27/48
Linear and Quadratic Order of convergence (III)

n
Consider pn ≈ (0.5)n and p̃n ≈ (0.5)2 .

28/48
Linear and Quadratic Order of convergence (IV)

Definition Suppose {pn }∞ n=1 is a sequence that converges to p

with pn ̸= p for all n. If positive constants λ and α exist with

|pn+1 − p|
lim = λ,
n→∞ |pn − p|α

then {pn }∞
n=1 converges to p of order α, with asymptotic error
constant λ.
▶ If α = 2, then {pn }∞
n=1 converges quadratically.
▶ If α = 1 and λ < 1, {pn }∞
n=1 converges linearly.
▶ If α = 1 and λ = 0, {pn }∞
n=1 converges super-linearly.

29/48
Linear and Quadratic Order of convergence (IV)

Definition Suppose {pn }∞ n=1 is a sequence that converges to p

with pn ̸= p for all n. If positive constants λ and α exist with

|pn+1 − p|
lim = λ,
n→∞ |pn − p|α

then {pn }∞
n=1 converges to p of order α, with asymptotic error
constant λ.
▶ If α = 2, then {pn }∞
n=1 converges quadratically.
▶ If α = 1 and λ < 1, {pn }∞
n=1 converges linearly.
▶ If α = 1 and λ = 0, {pn }∞
n=1 converges super-linearly.

α ≥ 3 or higher does not typically work better than quadratic

29/48
Order of convergence: Fixed Point Iteration (I)
Given initial approximation p0 , Fixed Point Iteration (FPI) is

pn = g (pn−1 ), n = 1, 2, · · · ,

Assume that
▶ FPI converges to fixed point p.
▶ g (x) is continuously differentiable with 0 < |g ′ (p)| < 1.
Theorem: FPI converges linearly,

|pn+1 − p|
lim = |g ′ (p)|. (1)
n→∞ |pn − p|

30/48
Order of convergence: Fixed Point Iteration (I)
Given initial approximation p0 , Fixed Point Iteration (FPI) is

pn = g (pn−1 ), n = 1, 2, · · · ,

Assume that
▶ FPI converges to fixed point p.
▶ g (x) is continuously differentiable with 0 < |g ′ (p)| < 1.
Theorem: FPI converges linearly,

|pn+1 − p|
lim = |g ′ (p)|. (1)
n→∞ |pn − p|

Proof:

pn+1 − p = g (pn ) − g (p) = g ′ (ξn ) (pn − p),

where ξn is between pn and p, and therefore converges to p.

(1) follows immediately.
30/48
Order of convergence: Bisection Method (I)

31/48
Order of convergence: Bisection Method (II)

32/48
Order of convergence: Bisection Method (II)

In FPI, |pn − p| ≤ k|pn−1 − p| ≤ k 2 |pn−2 − p| ≤ · · · ≤ k n |p0 − p|.

Bisection Method is ”considered” linearly convergent.

32/48
(Review) Newton’s Method as Fixed Point Iteration
Assume f (p) = 0 is a root, and x close to p. Would like a fixed
point iteration
pk+1 = g (pk ), k = 0, 1, · · · ,

where g (x) = x − f (x) h(x). How to choose h(x)?

33/48
(Review) Newton’s Method as Fixed Point Iteration
Assume f (p) = 0 is a root, and x close to p. Would like a fixed
point iteration
pk+1 = g (pk ), k = 0, 1, · · · ,

where g (x) = x − f (x) h(x). How to choose h(x)? Since

g ′ (x) = 1 − f ′ (x) h(x) − f (x) h(x)′ ≈ 1 − f ′ (x) h(x).

”optimal choice”: g ′ (x) ≈ 0, so that
1
h(x) =
f ′ (x)
and so
f (pk )
pk+1 = pk − , k = 0, 1, · · · .
f ′ (pk )

33/48
(Review) Newton’s Method as Fixed Point Iteration
Assume f (p) = 0 is a root, and x close to p. Would like a fixed
point iteration
pk+1 = g (pk ), k = 0, 1, · · · ,

where g (x) = x − f (x) h(x). How to choose h(x)? Since

g ′ (x) = 1 − f ′ (x) h(x) − f (x) h(x)′ ≈ 1 − f ′ (x) h(x).

”optimal choice”: g ′ (x) ≈ 0, so that
1
h(x) =
f ′ (x)
and so
f (pk )
pk+1 = pk − , k = 0, 1, · · · .
f ′ (pk )

Newton’s Method satisfies g ′ (p) = 0

33/48
(Review) Order of convergence: Fixed Point Iteration (I)
Given initial approximation p0 , Fixed Point Iteration (FPI) is

pn = g (pn−1 ), n = 1, 2, · · · ,

Assume that
▶ FPI converges to fixed point p.
▶ g (x) is continuously differentiable with 0 < |g ′ (p)| < 1.
Theorem: FPI converges linearly,
|pn+1 − p|
lim = |g ′ (p)|. (1)
n→∞ |pn − p|

34/48
(Review) Order of convergence: Fixed Point Iteration (I)
Given initial approximation p0 , Fixed Point Iteration (FPI) is

pn = g (pn−1 ), n = 1, 2, · · · ,

Assume that
▶ FPI converges to fixed point p.
▶ g (x) is continuously differentiable with 0 < |g ′ (p)| < 1.
Theorem: FPI converges linearly,
|pn+1 − p|
lim = |g ′ (p)|. (1)
n→∞ |pn − p|
Proof: pn+1 − p = g (pn ) − g (p) = g ′ (ξn ) (pn − p),

where ξn is between pn and p, and therefore converges to p.

(1) follows immediately.

34/48
(Review) Order of convergence: Fixed Point Iteration (I)
Given initial approximation p0 , Fixed Point Iteration (FPI) is

pn = g (pn−1 ), n = 1, 2, · · · ,

Assume that
▶ FPI converges to fixed point p.
▶ g (x) is continuously differentiable with 0 < |g ′ (p)| < 1.
Theorem: FPI converges linearly,
|pn+1 − p|
lim = |g ′ (p)|. (1)
n→∞ |pn − p|
Proof: pn+1 − p = g (pn ) − g (p) = g ′ (ξn ) (pn − p),

where ξn is between pn and p, and therefore converges to p.

(1) follows immediately.

superlinear convergence if g ′ (p) = 0

34/48
Order of convergence: Fixed Point Iteration (II)
Given initial approximation p0 , Fixed Point Iteration (FPI) is
pn = g (pn−1 ), n = 1, 2, · · · ,
Assume that
▶ FPI converges to fixed point p.
▶ g ′′ (x) is continuously differentiable with g ′ (p) = 0.
Theorem: FPI converges at least quadratically,
|pn+1 − p| 1
lim = |g ′′ (p)|. (2)
n→∞ |pn − p|2 2

35/48
Order of convergence: Fixed Point Iteration (II)
Given initial approximation p0 , Fixed Point Iteration (FPI) is
pn = g (pn−1 ), n = 1, 2, · · · ,
Assume that
▶ FPI converges to fixed point p.
▶ g ′′ (x) is continuously differentiable with g ′ (p) = 0.
Theorem: FPI converges at least quadratically,
|pn+1 − p| 1
lim = |g ′′ (p)|. (2)
n→∞ |pn − p|2 2
Proof: By second order Taylor expansion,
1
g (pn ) = g (p) + g ′ (p) (pn − p) + g ′′ (ξn ) (pn − p)2 ,
2
where ξn is between pn and p, and converges to p. So
1
pn+1 − p = g (pn ) − g (p) = g ′′ (ξn ) (pn − p)2 ,
2
(2) follows immediately.
35/48
Order of convergence: Fixed Point Iteration (II)
Given initial approximation p0 , Fixed Point Iteration (FPI) is
pn = g (pn−1 ), n = 1, 2, · · · ,
Assume that
▶ FPI converges to fixed point p.
▶ g ′′ (x) is continuously differentiable with g ′ (p) = 0.
Theorem: FPI converges at least quadratically,
|pn+1 − p| 1
lim = |g ′′ (p)|. (2)
n→∞ |pn − p|2 2
Proof: By second order Taylor expansion,
1
g (pn ) = g (p) + g ′ (p) (pn − p) + g ′′ (ξn ) (pn − p)2 ,
2
where ξn is between pn and p, and converges to p. So
1
pn+1 − p = g (pn ) − g (p) = g ′′ (ξn ) (pn − p)2 ,
2
(2) follows immediately. Newton’s method satisfies g ′ (p) = 0.
35/48
Order of convergence: Newton Method (again)
Given initial approximation p0 , Newton Method is
f (pn )
pn+1 = pn − ′ , n = 0, 1, 2, · · · ,
f (pn )
Assume that
▶ Newton Method converges to root p.
▶ f ′′ (x) is continuous with f ′ (p) ̸= 0.
Theorem: Newton Method converges at least quadratically,
|pn+1 − p| |f ′′ (p)|
lim = . (3)
n→∞ |pn − p|2 2 |f ′ (p)|

36/48
Order of convergence: Newton Method (again)
Given initial approximation p0 , Newton Method is
f (pn )
pn+1 = pn − ′ , n = 0, 1, 2, · · · ,
f (pn )
Assume that
▶ Newton Method converges to root p.
▶ f ′′ (x) is continuous with f ′ (p) ̸= 0.
Theorem: Newton Method converges at least quadratically,
|pn+1 − p| |f ′′ (p)|
lim = . (3)
n→∞ |pn − p|2 2 |f ′ (p)|
Proof: By second order Taylor expansion,
1
f (p) = f (pn ) + f ′ (pn ) (p − pn ) + f ′′ (ξn ) (p − pn )2 ,
2
where ξn is between pn and p, and converges to p. So
f (pn ) (pn − p) f ′ (pn ) − f (pn ) f ′′ (ξn )
pn+1 − p = pn − p − = = (p − pn )2 ,
f ′ (pn ) f ′ (pn ) 2 f ′ (pn )
and therefore (3) follows immediately.
36/48
Order of convergence: Secant Method (I)
Given initial approximations p0 , p1 , Secant Method is
f (pn )(pn − pn−1 )
pn+1 = pn − , n = 1, 2, · · · .
f (pn ) − f (pn−1 )
Assume that
▶ Secant Method converges to root p.
▶ f ′′ (x) is continuous with f ′ (p) ̸= 0.

37/48
Order of convergence: Secant Method (I)
Given initial approximations p0 , p1 , Secant Method is
f (pn )(pn − pn−1 )
pn+1 = pn − , n = 1, 2, · · · .
f (pn ) − f (pn−1 )
Assume that
▶ Secant Method converges to root p.
▶ f ′′ (x) is continuous with f ′ (p) ̸= 0.
By Secant Method,
f (pn )(pn − pn−1 )
pn+1 − p = pn − p −
f (pn ) − f (pn−1 )
(pn − p) (f (pn ) − f (pn−1 )) − f (pn )(pn − pn−1 )
=
f (pn ) − f (pn−1 )
F (pn ) − F (pn−1 )
= (pn − p) (pn−1 − p),
f (pn ) − f (pn−1 )
def f (x)−f (p) def
where F (x) = x−p is differentiable with F (p) = f ′ (p).
37/48
Order of convergence: Secant Method (II)
Assume that
▶ Secant Method converges to root p.
▶ f ′′ (x) is continuous with f ′ (p) ̸= 0.
√
1+ 5
Theorem: Secant Method converges at order α = 2 ,

|pn+1 − p|
lim = constant. (4)
n→∞ |pn − p|α

38/48
Order of convergence: Secant Method (II)
Assume that
▶ Secant Method converges to root p.
▶ f ′′ (x) is continuous with f ′ (p) ̸= 0.
√
1+ 5
Theorem: Secant Method converges at order α = 2 ,

|pn+1 − p|
lim = constant. (4)
n→∞ |pn − p|α
def f (x)−f (p)
Proof: With F (x) = x−p ,
F (pn ) − F (pn−1 )
pn+1 − p = (pn − p) (pn−1 − p),
f (pn ) − f (pn−1 )
′′
we can show that ratio converges to 2f f ′(p)
(p) and that α satisfies
α (α − 1) = 1, therefore
  −(α−1)
|pn+1 − p|  F (pn ) − F (pn−1 )  |pn − p|
=   
|pn − p|α f (pn ) − f (pn−1 )  |pn−1 − p|α
38/48
   −(α−1)
|pn+1 − p|  F (pn ) − F (pn−1 )  |pn − p|
= 
 
|pn − p|α f (pn ) − f (pn−1 )  |pn−1 − p|α

 ′′  1
|pn+1 −p| |pn+1 −p|
=  2f f ′(p)
α
Claim: limn→∞ exists =⇒ limn→∞

|pn −p|α |pn −p|α (p) 
39/48
Simple root: f (p) = 0, f ′ (p) ̸= 0
▶ Definition: A solution p of f (x) = 0 is a simple root if
f (p) = 0 and f ′ (p) ̸= 0.
▶ Theorem: The function f ∈ C 1 [a, b] has a simple root at p in
(a, b) if and only if
f (x) = (x − p) q(x), where lim q(x) ̸= 0.
x→p

40/48
Simple root: f (p) = 0, f ′ (p) ̸= 0
▶ Definition: A solution p of f (x) = 0 is a simple root if
f (p) = 0 and f ′ (p) ̸= 0.
▶ Theorem: The function f ∈ C 1 [a, b] has a simple root at p in
(a, b) if and only if
f (x) = (x − p) q(x), where lim q(x) ̸= 0.
x→p

Proof of ”only if”: Assume f (p) = 0 and f ′ (p) ̸= 0. Define for

x ̸= p,
f (x) − f (p) f (x)
q(x) = = ,
x −p x −p
f (x) − f (p)
then lim q(x) = lim = f ′ (p) ̸= 0.
x→p x→p x −p

40/48
Simple root: f (p) = 0, f ′ (p) ̸= 0
▶ Definition: A solution p of f (x) = 0 is a simple root if
f (p) = 0 and f ′ (p) ̸= 0.
▶ Theorem: The function f ∈ C 1 [a, b] has a simple root at p in
(a, b) if and only if
f (x) = (x − p) q(x), where lim q(x) ̸= 0.
x→p

Proof of ”only if”: Assume f (p) = 0 and f ′ (p) ̸= 0. Define for

x ̸= p,
f (x) − f (p) f (x)
q(x) = = ,
x −p x −p
f (x) − f (p)
then lim q(x) = lim = f ′ (p) ̸= 0.
x→p x→p x −p
Proof of ”if”: Assume f (x) = (x − p) q(x). Then f (p) = 0 and
f (x) − f (p)
f ′ (p) = lim = lim q(x) ̸= 0.
x→p x −p x→p

40/48
Multiple roots: f (p) = 0, f ′ (p) = 0

▶ Definition: A solution p of f (x) = 0 is a root of multiplicity

m (with integer m) of f if for x ̸= p, we can write

f (x) = (x − p)m q(x), where lim q(x) ̸= 0.

x→p

▶ Theorem: The function f ∈ C m [a, b] has a root of

multiplicity m at p in (a, b) if and only if

0 = f (p) = f ′ (p) = f ′′ (p) = · · · = f (m−1) (p), but f (m) (p) ̸= 0.

▶ simple root (m = 1): f satisfies f (p) = 0, but f ′ (p) ̸= 0.

41/48
Multiple roots: f (p) = 0, f ′ (p) = 0

▶ Definition: A solution p of f (x) = 0 is a root of multiplicity

m (with integer m) of f if for x ̸= p, we can write

f (x) = (x − p)m q(x), where lim q(x) ̸= 0.

x→p

▶ Theorem: The function f ∈ C m [a, b] has a root of

multiplicity m at p in (a, b) if and only if

0 = f (p) = f ′ (p) = f ′′ (p) = · · · = f (m−1) (p), but f (m) (p) ̸= 0.

▶ simple root (m = 1): f satisfies f (p) = 0, but f ′ (p) ̸= 0.

42/48
(Recall) Quadratic Convergence for Newton Method
Given initial approximation p0 , Newton Method is

f (pn )
pn+1 = pn − , n = 0, 1, 2, · · · ,
f ′ (pn )

Assume that
▶ Newton Method converges to root p.
▶ f ′′ (x) is continuous with f ′ (p) ̸= 0.
Theorem: Newton Method converges at least quadratically,

|pn+1 − p| |f ′′ (p)|
lim = . (1)
n→∞ |pn − p|2 2 |f ′ (p)|

43/48
(Recall) Quadratic Convergence for Newton Method
Given initial approximation p0 , Newton Method is

f (pn )
pn+1 = pn − , n = 0, 1, 2, · · · ,
f ′ (pn )

Assume that
▶ Newton Method converges to root p.
▶ f ′′ (x) is continuous with f ′ (p) ̸= 0.
Theorem: Newton Method converges at least quadratically,

|pn+1 − p| |f ′′ (p)|
lim = . (1)
n→∞ |pn − p|2 2 |f ′ (p)|

Right hand side in (1) not defined if f ′ (p) = 0.

43/48
Example I: f (x) = e x − x − 1, f (0) = f ′ (0) = 0, f ′′ (0) = 1

Function Newton’s Method

Much worse than typical Newton method convergence behavior

44/48
Example II: f (x) = 1 − cos x, f (0) = f ′ (0) = 0, f ′′ (0) = 1
Number of iterations vs. Magnitude in function values

Newton Method for function f(x) = 1−cos(x) with 25 Points Convergence Rate, Newton Method
0
0.25 10

−1
10

0.2
−2
10

−3
10
0.15

−4
10

0.1 −5
10

−6
10
0.05

−7
10

−8
0 10
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0 5 10 15 20 25

45/48
Modified Newton Method

m f (pk )
pk+1 = pk − , for k = 1, 2, · · · , (must know multiplicity m of root)
f ′ (pk )

▶ m = 2 for function f (x) = e x − x − 1,

▶ |f (pk )| drops to O(10−9 ) only, far from machine precision

O(10−16 ).
46/48
Modified Newton Method is Quadratically Convergent
m f (x)
pk+1 = g (pk ), g (x) = x − , for k = 1, 2, · · · ,
f ′ (x)
Proof of quadratic convergence:
▶ Let f (x) = (x − p)m q(x), q(p) ̸= 0.
▶ Write g (x) in terms of q(x):
m f (x) m (x − p)q(x)
g (x) = x − ′
=x− .
f (x) mq(x) + (x − p)q ′ (x)

47/48
Modified Newton Method is Quadratically Convergent
m f (x)
pk+1 = g (pk ), g (x) = x − , for k = 1, 2, · · · ,
f ′ (x)
Proof of quadratic convergence:
▶ Let f (x) = (x − p)m q(x), q(p) ̸= 0.
▶ Write g (x) in terms of q(x):
m f (x) m (x − p)q(x)
g (x) = x − ′
=x− .
f (x) mq(x) + (x − p)q ′ (x)

▶ Terms cancel in g ′ (x):

 ′
′ m q(x) q(x)
g (x) = 1 − − m (x − p)
mq(x) + (x − p)q ′ (x) mq(x) + (x − p)q ′ (x)
′
(x − p)q ′ (x)

q(x)
= − m(x − p)
mq(x) + (x − p)q ′ (x) mq(x) + (x − p)q ′ (x)

47/48
Modified Newton Method is Quadratically Convergent
m f (x)
pk+1 = g (pk ), g (x) = x − , for k = 1, 2, · · · ,
f ′ (x)
Proof of quadratic convergence:
▶ Let f (x) = (x − p)m q(x), q(p) ̸= 0.
▶ Write g (x) in terms of q(x):
m f (x) m (x − p)q(x)
g (x) = x − ′
=x− .
f (x) mq(x) + (x − p)q ′ (x)

▶ Terms cancel in g ′ (x):

 ′
′ m q(x) q(x)
g (x) = 1 − − m (x − p)
mq(x) + (x − p)q ′ (x) mq(x) + (x − p)q ′ (x)
′
(x − p)q ′ (x)

q(x)
= − m(x − p)
mq(x) + (x − p)q ′ (x) mq(x) + (x − p)q ′ (x)
▶ g ′ (p) = 0, hence quadratic convergence
(by fixed point iteration convergence theorem.)
47/48
Modified Newton Method in textbook

f (x) f ′ (x)
pk+1 = g (pk ), g (x) = x− , for k = 1, 2, · · · ,
(f ′ (x))2 − f (x) f ′′ (x)

▶ Also quadratic convergence.

▶ Even less practical since second order derivative is involved.

48/48