W EEK

5
Numerical Differentiation

Study Organiser

Before you begin this note, please check through your study organiser given below. It shows
the topics that we’ll be covering, the skills you need to acquire (the learning outcomes) and
the resources and activities you are given to help you acquire these skills.

Topics Learning Outcomes Learning Learning

Resources Activities
Estimating Derive and use a numerical derivative Readings - Section 5.1 Tutorial 5
derivatives formula and state the error involved. Examples 5.1-5.6 Quiz 5
Richardson Show how the Richardson Extrapolation Readings - Section 5.2 Homework 5
Extrapolation can be used to improve the accuracy of Examples 5.7-5.9 Glossary 5
numerical derivative formulas. Test 1

5.1 Estimating derivatives

Consider the table given below. What would be f ′ (1)?

x 0 0.5 1 1.5
f (x) 1 -1 2 -3

5.1.1 Estimating derivatives using Interpolation

Let Pn (x) denote the polynomial of degree at most n that interpolates f (x) at n + 1 nodes
(x0 , x1 , . . . , xn ), Then to calculate f ′ (x) at some point x = c, we can use

f ′ (c) ≈ Pn′ (c).

2 W EEK 5

Example 5.1. Use an interpolating polynomial to estimate f ′ (x) for each of the x value given in the
table.

x 1 3 4 6
f (x) 2 -8 11 7

Solution
Verify that the Newton form of the interpolating polynomial is

P3 (x) = 2 − 5(x − 1) + 8(x − 1)(x − 3) − 3(x − 1)(x − 3)(x − 4)

= 67 − 94x + 32x2 − 3x3

So P3′ (x) = −94 + 64x − 9x2 , Thus

f ′ (1) ≈ P3′ (1) = −39

f ′ (3) ≈ P3′ (3) = 17
f ′ (4) ≈ P3′ (4) = 18
f ′ (6) ≈ P3′ (6) = −34

5.1.2 Estimating derivatives using Taylor Series

Consider the method of obtaining f ′ (x) based on the definition:

f (x + h) − f (x)
f ′ (x) ≈
h
What is the error involved in this formula? To find this out, we use Taylor series:

h2 ′′ h3
f (x + h) = f (x) + hf ′ (x) + f (x) + f ′′′ (x) + · · ·
2! 3!
Rearranging this equation gives

f (x + h) − f (x) h h2
f ′ (x) = − f ′′ (x) − f ′′′ (x) − · · ·
h 2! 3!
So the error is given by

h h2
Error = − f ′′ (x) − f ′′′ (x) − · · ·
2! 3!
h ′′
= − f (ξ) for ξ ∈ (x, x + h)
2!
= O(h)
f (x+h)−f (x)
The above equation shows that as h → 0, f (x) − h → 0 at the same rate as h → 0.
i.e. O(h).
W EEK 5 3

The precision of numerical differentiation formulas is often judged simply by the power of
h in the error term. The higher the power of h the better the approximation because h is
always a small number. A superior formula is
f (x + h) − f (x − h)
f ′ (x) ≈
2h
To derive this, we again use the Taylor series

h2 ′′ h3
f (x + h) = f (x) + hf ′ (x) + f (x) + f ′′′ (x) + · · ·
2! 3!
h 2 h3
f (x − h) = f (x) − hf ′ (x) + f ′′ (x) − f ′′′ (x) + · · ·
2! 3!
2h 3
f (x + h) − f (x − h) = 2hf ′ (x) + f ′′′ (x) + · · ·
3!
2h3 ′′′
2hf ′ (x) = f (x + h) − f (x − h) − f (x) − · · ·
3!
f (x + h) − f (x − h) h2 ′′′
f ′ (x) = − f (x) − · · ·
2h 3!
Thus
f (x + h) − f (x − h)
f ′ (x) ≈
2h
and the error is given by

h2 ′′′
Error = − f (x) − · · ·
3!
h2
= − f ′′′ (ξ) for ξ ∈ (x − h, x + h)
3!
= O(h2 )

Example 5.2. Derive the formula

f (x) − f (x − h)
f ′ (x) ≈
h
and establish the error term.

Solution
Note that
h2 ′′ h3
f (x − h) = f (x) − hf ′ (x) + f (x) − f ′′′ (x) + · · ·
2! 3!
Rearranging this equation gives

f (x) − f (x − h) h h2
f ′ (x) = + f ′′ (x) − f ′′′ (x) + · · ·
h 2! 3!
Thus
f (x) − f (x − h)
f ′ (x) ≈
h
4 W EEK 5

and the error is given by

h ′′ h2
Error = f (x) − f ′′′ (x) + · · ·
2! 3!
h ′′
= f (ξ) for ξ ∈ (x − h, x)
2!
= O(h)

Example 5.3. Derive the formula

4f (x + h) − 3f (x) − f (x + 2h)
f ′ (x) ≈
2h
and establish the error term.

Solution
Note that

h2 ′′ h3
f (x + h) = f (x) + hf ′ (x) + f (x) + f ′′′ (x) + · · ·
2! 3!
4h 2 8h3 ′′′
f (x + 2h) = f (x) + 2hf ′ (x) + f ′′ (x) + f (x) + · · ·
2! 3!
4h3 ′′′
4f (x + h) − f (x + 2h) = 3f (x) + 2hf ′ (x) + 0 − f (x) + · · ·
6
4h3 ′′′
2hf ′ (x) = 4f (x + h) − f (x + 2h) − 3f (x) + f (x) − · · ·
6
4f (x + h) − f (x + 2h) − 3f (x) 2h2 ′′′
f ′ (x) = + f (x) − · · ·
2h 6
Thus
4f (x + h) − f (x + 2h) − 3f (x)
f ′ (x) ≈
2h
and the error is given by

2h2 ′′′
Error = f (x) − · · ·
6
2h2 ′′′
= f (ξ) for ξ ∈ (x, x + 2h)
6
= O(h2 )

Example 5.4. Derive the formula

3f (x) − 4f (x − h) + f (x − 2h)
f ′ (x) ≈
2h
and establish the error term.
W EEK 5 5

Solution
Note that

h2 ′′ h3
f (x − h) = f (x) − hf ′ (x) + f (x) − f ′′′ (x) + · · ·
2! 3!
4h 2 8h3 ′′′
f (x − 2h) = f (x) − 2hf ′ (x) + f ′′ (x) − f (x) + · · ·
2! 3!
4h3 ′′′
−4f (x − h) + f (x − 2h) = −3f (x) + 2hf ′ (x) + 0 − f (x) + · · ·
6
4h3 ′′′
2hf ′ (x) = 3f (x) − 4f (x − h) + f (x − 2h) + f (x) − · · ·
6
3f (x) − 4f (x − h) + f (x − 2h) 2h2 ′′′
f ′ (x) = + f (x) − · · ·
2h 6
Thus
3f (x) − 4f (x − h) + f (x − 2h)
f ′ (x) ≈
2h
and the error is given by

2h2 ′′′
Error = f (x) − · · ·
6
2h2 ′′′
= f (ξ) for ξ ∈ (x − 2h, x)
6
= O(h2 )

Example 5.5. Derive the formula

f (x + 2h) − 2f (x) + f (x − 2h)

f ′′ (x) ≈
4h2
and establish the error term.

Solution
Note that

h2 ′′ h3 h4
f (x + h) = f (x) + hf ′ (x) + f (x) + f ′′′ (x) + f (4) (x) + · · ·
2! 3! 4!
2
4h ′′ 3
8h ′′′ 16h4 (4)
f (x + 2h) = f (x) + 2hf ′ (x) + f (x) + f (x) + f (x) + · · ·
2! 3! 24
4h2 ′′ 8h3 ′′′ 16h4 (4)
f (x − 2h) = f (x) − 2hf ′ (x) + f (x) − f (x) + f (x) + · · ·
2! 3! 24
4h4 (4)
f (x + 2h) + f (x − 2h) = 2f (x) + 0 + 4h2 f ′′ (x) + 0 + f (x) + · · ·
3
4h4 (4)
4h2 f ′′ (x) = f (x + 2h) + f (x − 2h) − 2f (x) − f (x) − · · ·
3
f (x + 2h) + f (x − 2h) − 2f (x) h2 (4)
f ′′ (x) = − f (x) − · · ·
4h2 3
6 W EEK 5

Thus
f (x + 2h) − 2f (x) + f (x − 2h)
f ′′ (x) ≈
4h2
and the error is given by

h2 (4)
Error = − f (x) − · · ·
3
h2
= − f (4) (ξ) for ξ ∈ (x − 2h, x + 2h)
3
= O(h2 )

f (x + h) − f (x − h)
Example 5.6. 1. Derive the approximation f ′ (x) ≈ and establish its error
2h
term.
ln(x2 + 1) − x2
2. Use the result of part (a) to approximate f ′ (2.5) if f (x) = . Take h = 0.001.
sinh(x + 2) + x3

Solution:

1. To derive this, we use the Taylor series

h2 ′′ h3
f (x + h) = f (x) + hf ′ (x) + f (x) + f ′′′ (x) + · · ·
2! 3!
h 2 h3
f (x − h) = f (x) − hf ′ (x) + f ′′ (x) − f ′′′ (x) + · · ·
2! 3!
2h 3
f (x + h) − f (x − h) = 2hf ′ (x) + f ′′′ (x) + · · ·
3!
2h3 ′′′
2hf ′ (x) = f (x + h) − f (x − h) − f (x) − · · ·
3!
f (x + h) − f (x − h) h2 ′′′
f ′ (x) = − f (x) − · · ·
2h 3!
f (x+h)−f (x−h) 2
Thus f ′ (x) ≈ 2h and the error is given by − h3! f ′′′ (x) − · · · = O(h2 ).

f (2.501) − f (2.499)
2. f ′ (2.5) ≈ ≈ 0.00296028.
0.002

5.2 Richardson Extrapolation

We will see how a procedure known as Richardson Extrapolation can be used to improve
the accuracy of numerical formulas. We have seen in the previous lecture that

f (x + h) − f (x − h) h2 ′′′ h4
f ′ (x) = − f (x) − f (5) (x) − · · ·
2h 3! 5!
W EEK 5 7

This equation has the form

L = φ(h) + a2 h2 + a4 h4 + · · · (5.1)
f (x+h)−f (x−h)
where L stands for f ′ (x) and φ(h) stands 2h . For each h > 0, the error is given by
a2 h2 + a4 h4 + · · · = O(h2 ).

We see that the first term a2 h2 in the error is greater than the other terms when h is suf-
ficiently small. We shall therefore look for a way of eliminating this dominant term, a2 h2 .
One way to this is to first replace h by h/2 in equation (5.1) to get
1 1
L = φ(h/2) + a2 h2 + a4 h4 + · · ·
4 16
1
2
4L = 4φ(h/2) + a2 h + a4 h4 + · · · (5.2)
4
Equation (5.2)−Equation (5.1) gives
3
3L = 4φ(h/2) − φ(h) − a4 h4 − · · ·
4
4φ(h/2) − φ(h) 1
L= − a4 h4 − · · ·
3 4
Thus
4φ(h/2) − φ(h)
L≈
3
with error= O(h4 ). It shows that a simple combination of φ(h) and φ(h/2) gives a more
accurate estimate of L.

Example 5.7. If L = φ(h) + c1 h + c2 h2 + c3 h3 + · · · , then what combination of φ(h) and φ(h/2)

would give an estimate of L with a precision of O(h2 )?

Solution
Given L = φ(h) + c1 h + c2 h2 + c3 h3 + · · · . Replacing h by h/2, we have

c1 h c2 h2 c3 h3
L = φ(h/2) + + + + ···
2 4 8
Multiplying the latter by 2 and subtracting the former, we obtain
1 3
L = 2φ(h/2) − φ(h) − c2 h2 − c3 h3 − · · · .
2 4

Example 5.8. If L = φ(h)+c1 h1/2 +c2 h+c3 h3/2 +· · · , then what combination of φ(h) and φ(h/2)
would give an estimate of L with a precision of O(h)?
8 W EEK 5

Solution
Given L = φ(h) + c1 h1/2 + c2 h + c3 h3/2 + · · · . Replacing h by h/2, we have

c1 h1/2 c2 h c3 h3/2
L = φ(h/2) + √ + + 3/2 + · · ·
2 2 2
√
Multiplying the latter by 2 and subtracting the former, we obtain
 
√ √ 1 1
( 2 − 1)L = 2φ(h/2) − φ(h) + √ − 1 c2 h − c3 h3/2 − · · · .
2 2

Thus √
2φ(h/2) − φ(h) 1 1
L= √ − √ c2 h − √ c3 h3/2 − · · · .
2−1 2 2( 2 − 1)

Example 5.9. If L − f (h) = c6 h6 + c9 h9 + · · · , then what combination of f (h) and f (h/2) should
give an accurate estimate of L?

Solution
Given L = f (h) + c6 h6 + c9 h9 + · · · . Replacing h by h/2, we have

c6 h6 c9 h9
L = f (h/2) + + 9 + ···
26 2
Multiplying the latter by 26 and subtracting the former, we obtain

7
63L = 64f (h/2) − f (h) − c9 h9 − · · · .
8
Thus
64f (h/2) − f (h) 1
L= − c9 h9 − · · · .
63 72