Section 4

Numerical Differentiation and Integration

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 129
Numerical differentiation

Recall the definition of derivative is

0 f (x0 + h) − f (x0 )
f (x0 ) = lim
h→0 h
We can approximate f 0 (x0 ) by

f (x0 + h) − f (x0 )
, for some small h
h

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 130
 Numerical differentiation
Approximate f 0 (x0 ) by

f (x0 + h) − f (x0 ) 4.1 Numerical Differentiation 175

, for some small h
h
Figure 4.1
y
Slope f !(x 0)
f (x0 " h) # f (x 0)
Slope
h

x0 x0 " h x

How to quantify the error of this approximation?

with h = 0.1 gives
Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 131
Numerical differentiation

If f ∈ C 2 , then Taylor’s theorem says ∃ ξ ∈ (x0 , x0 + h) s.t.

1 00
0
f (x0 + h) = f (x0 ) + f (x0 )h + f (ξ)h2
2
0 f (x0 + h) − f (x0 ) 1 00
⇐⇒ f (x0 ) = − f (ξ)h
h 2
If ∃ M > 0 s.t. |f 00 (x)| ≤ M for all x near x0 , then

0 f (x0 + h) − f (x0 ) 1 00 Mh
Error = f (x0 ) − = f (ξ)h ≤
h 2 2

So the error is of order “O(h)”.

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 132
Example

Example (Error of numerical differentiations)

Let f (x) = ln(x) at x0 = 1.8. Use h = 0.1, 0.05, 0.01 to
approximate f 0 (x0 ). Determine the approxiamtion errors.
Solution. We compute for h = 0.1, 0.05, 0.01 that

f (1.8 + h) − f (1.8) ln(1.8 + h) − ln(1.8)

=
h h
1 1
Then |f 00 (x)| = | − x2
| ≤ 1.82
=: M for all x > 1.8. Error is
bounded by Mh 2 .

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 133
Numerical differentiation

Example (Error of numerical differentiations)

Let f (x) = ln(x) at x0 = 1.8. Use h = 0.1, 0.05, 0.01 to
approximate f 0 (x0 ). Determine the approxiamtion errors.
Solution (cont.)
f (1.8+h)−f (1.8) Mh
h h 2
0.10 0.5406722 0.0154321
0.05 0.5479795 0.0077160
0.01 0.5540180 0.0015432
1
The exact value is f 0 (1.8) = 1.8 = 0.555̄.

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 134
Three-point endpoint formula

Recall the Lagrange interpolating polynomial for x0 , . . . , xn is

n
X (x − x0 ) · · · (x − xn ) (n+1)
f (x) = f (xk ) Lk (x) + f (ξ(x))
(n + 1)!
k=0

Suppose we have x0 , x1 , x0 + h, x2 , x0 + 2h, then

2
X (x − x0 )(x − x1 )(x − x2 ) (3)
f (x) = f (xk ) Lk (x) + f (ξ(x))
6
k=0

where ξ(x) ∈ (x0 , x0 + 2h).

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 135
Three-point endpoint formula

Take derivative w.r.t. x of

2
X (x − x0 )(x − x1 )(x − x2 ) (3)
f (x) = f (xk ) Lk (x) + f (ξ(x))
6
k=0

and set x = x0 yields4 the Three-point endpoint formula:

0 1   h2 (3)

f (x0 ) = −3f (x0 ) + 4f (x0 + h) − f (x0 + 2h) + f ξ(x0 )
2h 3
where ξ(x0 ) ∈ (x0 , x0 + 2h).

4 (x−x0 )(x−x1 )(x−x2 ) df (3) (ξ(x))

Note that 6 dx
|x=x0 = 0.
Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 136
Three-point midpoint formula

Suppose we have x−1 = x0 − h, x0 , x1 , x0 + h, then

1
X (x − x−1 )(x − x0 )(x − x1 ) (3)
f (x) = f (xk ) Lk (x) + f (ξ1 )
6
k=−1

where ξ1 ∈ (x0 − h, x0 + h).

Take derivative w.r.t. x, and set x = x0 yields Three-point

midpoint formula:

0 1   h2 (3)
f (x0 ) = f (x0 + h) − f (x0 − h) − f (ξ1 )
2h 6

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 137
 Three-point midpoint formula
Numerical Differentiation and Integration

Illustration of Three-point midpoint formula:

e 4.2
y

Slope f #(x 0)

1
Slope [ f (x0 ! h) " f (x 0 " h)]
2h

x0 " h x0 x0 ! h x

Five-Point Formulas
Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 138
Five-point midpoint formula

We can also consider xk = x0 + kh for k = −2, −1, 0, 1, 2, then

2 Q2
X k=−2 (x − xk )
f (x) = f (xk ) Lk (x) + f (5) (ξ0 )
5!
k=−2

where ξ0 ∈ (x0 − 2h, x0 + 2h).

Show that you can get the Five-point midpoint formula:

0 1  
f (x0 ) = f (x0 − 2h) − 8f (x0 − h) + 8f (x0 + h) − f (x0 + 2h)
12h
h4 (5)
+ f (ξ0 )
30

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 139
Five-point endpoint formula

We can also consider xk = x0 + kh for k = 0, 1, . . . , 4, then

4
X Q4
k=0 (x − xk )
f (x) = f (xk ) Lk (x) + f (5) (ξ0 )
5!
k=0

where ξ0 ∈ (x0 , x0 + 4h).

Show that you can get the Five-point endpoint formula:


0 1
f (x0 ) = − 25f (x0 ) + 48f (x0 + h) − 36f (x0 + 2h)
12h

h4 (5)
+ 16f (x0 + 3h) − 3f (x0 + 4h) + f (ξ0 )
5

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 140
Example

Example (3-point and 5-point formulas)

Use the values in the table to find f 0 (2.0):

x f (x)
1.8 10.889365
1.9 12.703199
2.0 14.778112
2.1 17.148957
2.2 19.855030

Compare your result with the true value f 0 (2) = 22.167168.

Hint: Use three-point midpoint formula with h = 0.1, 0.2, endpoint
with h = ±0.1, and five-pint midpoint formula with h = 0.1.

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 141
Second derivative midpoint formula

Expand f in a third Taylor polynomial about a point x0 and

evaluate at x0 + h and x0 − h:
1 00
0 1 000 1 (4)
f (x0 + h) = f (x0 ) + f (x0 ) h + f (x0 ) h2 + f (x0 ) h3 + f (ξ1 ) h4
2 6 24
0 1 00 1 000 1 (4)
f (x0 − h) = f (x0 ) − f (x0 ) h + f (x0 ) h2 − f (x0 ) h3 + f (ξ−1 ) h4
2 6 24
where ξ±1 is between x0 and x0 ± h.
h i
Adding the two and using IVT f (4) (ξ) = 21 f (4) (ξ1 ) + f (4) (ξ−1 )
(assuming f ∈ C 4 ) yield:

00 1   h2 (4)
f (x0 ) = 2 f (x0 − h) − 2f (x0 ) + f (x0 + h) − f (ξ)
h 12
where x0 − h < ξ < x0 + h.

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 142
Roundoff error instability

Recall we have three-point midpoint approximation

1   h 2
f 0 (x0 ) = f (x0 + h) − f (x0 − h) − f (3) (ξ1 )
2h 6
for ξ1 ∈ (x0 − h, x0 + h).

Will we get better accuracy as h → 0? Not necessarily.

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 143
Round-off error instability

In numerical computations, round-off error is inevitable:

f (x0 + h) = f˜ (x0 + h) + e (x0 + h)

f (x0 − h) = f˜ (x0 − h) + e (x0 − h)

f˜(x0 +h)−f˜(x0 −h)

Hence we’re approximating f 0 (x0 ) by 2h with error:
˜ (x0 + h) − f˜ (x0 − h)
f e (x 0 + h) − e (x 0 − h) h 2
f 0 (x0 ) − = − f (3) (ξ1 )
2h 2h 6
Suppose |e(x)| ≤ ε, ∀x, then the error bound is:

˜ (x0 + h) − f˜ (x0 − h)
f ε h 2
f 0 (x0 ) − ≤ + M
2h h 6

So the error does not go to 0 as h → 0, due to the round-off error.

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 144
Richardson’s extrapolation

Goal: generate high-accuracy results by low-order formula.

Suppose we have formula N1 (h) to approximate M with 5

M = N1 (h) + K1 h + K2 h2 + K3 h3 + · · ·

with some unknown K1 , K2 , K3 , . . . .

For h small enough, the error is dominated by K1 h, then

 
h h h2 h3
M = N1 + K1 + K2 + K3 + · · ·
2 2 4 8

f (x0 +h)−f (x0 )

5
E.g., M = f 0 (x0 ) and N1 (h) = h
.
Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 145
Richardson’s extrapolation

Therefore
  "   # ! !
h h h2 h3
M = N1 + N1 − N1 (h) +K2 − h2 +K3 − h3 +· · ·
2 2 2 4

Define   "   #
h h
N2 (h) = N1 + N1 − N1 (h)
2 2

then M can be approximated by N2 (h) with order O(h2 ):

K2 2 3K3 3
M = N2 (h) − h − h − ···
2 4

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 146
Example

Example (Richardson’s extrapolation)

Let f (x) = ln(x). Approximate f at x = 1.8 with forward difference
using h = 0.1 and h = 0.05. Then approximate using N2 (0.1).
Solution. We know the forward difference is O(h), and
(
f (x0 + h) − f (x0 ) 0.5406722, for h = 0.1
N1 (h) = f (x0 ) − =
h 0.5479795, for h = 0.05

N2 (0.1) = N1 (0.05) + (N1 (0, 05) − N1 (0.1)) = 0.555287.

Formula N1 (0.1) N1 (0.05) N2 (0.1)

Error 1.5 × 10−2 7.7 × 10−3 2.7 × 10−4

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 147
Richardson’s extrapolation

Suppose M = N1 (h) + K1 h2 + K2 h4 + K3 h6 + . . . , then for

j = 2, 3, . . . , we have O(h2j ) approximation:
 
h Nj−1 (h/2) − Nj−1 (h)
Nj (h) = Nj−1 +
2 4j−1 − 1

We can show the order of generating these Nj (h) 6 :

O(h2 ) O(h4 ) O(h6 ) O(h8 )

1: N1 (h)
2: N1 ( h2 ) 3: N2 (h)
4: N1 ( h4 ) 5: N2 ( h2 ) 6: N3 (h)
7: N1 ( h8 ) 8: N2 ( h4 ) 9: N3 ( h2 ) 10: N4 (h)

6
Exercise: write a computer program for Richardson’s extrapolation.
Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 148
Example

Example (Richardson’s extrapolation)

Consider approximation of f 0 (x0 ):

0 1   h2 000 h4 (5)
f (x0 ) = f (x0 + h) − f (x0 − h) − f (x0 ) − f (x0 ) − · · ·
2h 6 120

Find the approximation errors of order O(h2 ), O(h4 ), O(h6 ) for

f 0 (2.0) when f (x) = xe x and h = 0.2.
Solution. We have O(h2 ) approximation

0 h2 000 h4 (5)
f (x0 ) = N1 (h) − f (x0 ) − f (x0 ) − · · ·
6 120
1
 
where N1 (h) = 2h f (x0 + h) − f (x0 − h) . Then compute
N1 (h), N1 ( h2 ), N2 (h), N1 ( h4 ), N2 ( h2 ), . . . in order.

Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 149