Chapter 3 Numerical Differentiation & Differential Equations

1. Theory summary
1.1. Numerical Differentiation
Finite-divided-difference approximations of derivatives are derived from Taylor
series expansions:
f ' ' (x i) 2
f (x i+1 )=f (xi )+f ' (x i). h+ . h + .. .
2
High-accuracy divided-difference formulas can be generated by including
additional terms from the Taylor series expansion. For examples, for first order
derivatives:
Forward finite-divided-difference formulas:
f ( x i+1 )−f (xi )
f ' (x i)=
h
−f (x i+2 )+ 4 f (x i+1 )−3 f ( xi )
f ' (x i)=
2h
Centered finite-divided-difference formulas:
f ( x i+1 )−f (xi −1 )
f ' (x i)=
2h
−f (x i+2 )+ 8 f (x i +1)−8 f (x i−1)+ f (x i−2)
f ' (x i)=
12 h
Backward finite-divided-difference formulas:
f ( x i)−f (x i−1)
f ' (x i)=
h
3 f (x i)−4 f (x i−1)+f ( x i−2 )
f ' (x i)=
2h
 Higher order derivatives can also be derived. For instance, for second order:
Forward finite-divided-difference formulas:
f (x i+2 )−2 f ( x i +1)+ f (x i)
f ' ' (x i)= 2
h

Centered finite-divided-difference formulas:

f (x i+1 )−2 f ( x i )+ f (x i−1 )
f ' ' (x i)= 2
h

Backward finite-divided-difference formulas:

f (x i)−2 f ( x i−1)+ f (x i−2)
f ' ' (x i)= 2
h

1.2. Differential Equations

1.2.1. Euler’s method
dy
This part aims at solving ordinary differential equations of the form =f ( x , y ).
dx

The method was of the general form:

New value = old value + slope x step size
Or, in mathematical terms: y i+ 1= y i+ φh
According to this equation, the slope estimates of φ is used to extrapolate from an
old value yi to a new value yi+1 over a distance h.
 All one-step methods can be expressed in this general form, with the only
difference being the manner in which the slope is estimated. As in the falling parachutist
problem, the simplest approach is to use the differential equation to estimate the slope in
the form of the first derivative at xi. In other words, the slope at the beginning of the
interval is taken as an approximation of the average slope over the whole interval. This
approach, called Euler’s method.
The first derivative provides a direct estimate of the slope at xi:
φ=f ( x i , y i )

Then : y i+ 1= y i+ f ( x i , y i ) . h
1.2.2. Heun’s method
One method to improve the estimate of the slope involves the determination of
two derivatives for the interval—one at the initial point and another at the end point. The
two derivatives are then averaged to obtain an improved estimate of the slope for the
entire interval. This approach, called Heun’s
method.
The slope at the beginning of an interval is
similar to Euler’s method:
'
y i=f ( x i , y i )

which is used to extrapolate linearly to y i+ 1

:
0
y i+ 1= y i+ f ( x i , y i ) . h

This predictor equation provides an

estimate of y i+ 1 that allows the calculation of an
estimated slope at the end of the interval :
 y 'i+ 1=f ( x i+ 1 , y 0i+1 )

The average slope is then used to extrapolate linearly from y ito y i+ 1:

f ( xi , y i ) +f ( xi +1 , y i+ 1)
0

y i+ 1= y i+ .h
2

1.2.3. Runge-Kutta methods

Runge-Kutta (RK) methods achieve the accuracy of a Taylor series approach
without requiring the calculation of higher derivatives, generally written in form:
y i+ 1= y i+ φ ( x i , y i , h ) . h

Where φ ( x i , y i , h ) is called an increment function:

φ=a 1 . k 1 +a 2 . k 2 +. . .+ an . k n

Where the a’s are constant and k’s are:

k 1=f ( x i , y i )

k 2=f ( x i + p1 . h , y i +q11 . k 1 . h )

…
k n=f ( x i + pn−1 .h , y i +q n−1 ,1 . k 1 . h+q n−1 ,2 .k 2 . h+. . .+q n−1 , n−1 . k n −1 . h )

(q’s and p’s are constant)

For instances: The third order RK method:
1
y i+ 1= y i+ (k 1+ 4 k 2+ k 3 ). h
6

With k 1=f ( x i , y i )

1
2( 1
k 2=f xi + . h , y i+ . k 1 . h
2 )
k 3=f ( x i +h , y i−k 1 . h+2 k 2 h )

The fourth order RK method:

 1
y i+ 1= y i+ (k 1+ 2 k 2+ 2 k 3+ k 4 ). h
6

With k 1=f ( x i , y i )

1
2 ( 1
k 2=f xi + . h , y i+ . k 1 . h
2 )
1
2 ( 1
k 3=f xi + h , y i + . k 2 .h
2 )
k 4=f ( x i+ h , y i +k 3 h )

2. Numerical examples
2.1. Numerical differentiation
*Example 1 (Polynomial): Evaluate the first and the second derivatives of
4 3 2
f (x)=−x −2 , 5 x −5 x −4 , 5 x+3

at xi = 1, step size h = 0,25

Solution
Analytically:
' 3 2
f ❑( x )=−4 x −7 , 5 x −10 x−4 ,5
' 3 2
f ❑( 1 )=−4.1 −7 ,5 .1 −10 x−4 , 5=−26
'' 2
f ❑( x )=−12 x −15 x−10
'' 2
f ❑( 1 )=−12.1 −15.1−10=−37

Numerically:
Centered finite-divided-difference formula with 3 points:
f ( x i+1 )−f (xi −1 )
f ' (x i)=
2h
4 3 2
f ( x i ) =f ( 1)=−1 −2 , 5 .1 −5 .1 −4 ,5.1+3=−10
4 3 2
f ( x i +1 )=f ( 1+0 ,25 )=f ( 1 , 25 )=−1 , 25 −2 ,5 .1 , 25 −5 .1 ,25 −4 ,5.1 , 25+3
 4547
¿−
256
4 3 2
f ( x i−1 )=f ( 1−0 , 25 ) =f ( 0 , 75 )=−0 ,75 −2 , 5 .0 ,75 −5.0 ,75 −4 , 5.0 , 75+3

1167
¿−
256
−4547 1167
+
f (1 , 25)−f (0.75) 256 256
f ' (1)= = =−26,40625
2h 2.0 , 25

Error: ε = |−26,40625−(−26)
−26
.100 %=1,5625 % |
''
f ( xi +1 )−2 f ( x i ) + f ( xi−1 )
f ❑( x i )= 2
h

−4547 1167
−2.(−10)−
'' f (1 , 25 )−2 f (1 )+ f ( 0 , 75 ) 256 256
f ❑( 1 )= 2
= 2
=−37,125
h 0 ,25

Error: ε = |−37.125−(−37)
−37 |
.100 %=0,3378 %

*What if the step size h=0,25 is reduced to h=0,1

4 3 2
f ( x i +1 )=f ( 1+0 ,1 )=f ( 1 ,1 ) =−1 ,1 −2 ,5 .1 , 1 −5 .1 ,1 −4 , 5.1 ,1+3

¿−12,7916
4 3 2
f ( x i−1 )=f ( 1−0 , 1 )=f ( 0 , 9 )=−0 ,9 −2 , 5 .0 , 9 −5 .0 , 9 −4 , 5.0 , 9+ 3

¿−7,5786
f (1 , 1 )−f ( 0 , 9 ) −12,7916 +7,5786
f ❑' ( 1 )= = =−26,065
2h 2.0 ,1

Error: ε = |−26,065−(−26)
−26 |
.100 %=0 ,25 %

Centered finite-divided-difference formula with 5 points:

−f (x i+2 )+ 8 f (x i +1)−8 f (x i−1)+ f (x i−2)
f ' (x i)=
12 h
 4 3 2
f ( x i +2 )=f (1+2.0 , 25)=f (1 ,5)=−1 , 5 −2 ,5 .1 , 5 −5 .1 ,5 −4 ,5.1 , 5+3=−28 ,5
4 3 2
f ( x i−2 )=f (1−2.0 , 25)=f (0 ,5)=−0 ,5 −2 , 5 .0 ,5 −5 .0 ,5 −4 ,5.0 , 5+3=−0,875

−f ( 1, 5 ) +8 f (1 , 25 )−8 f ( 0 , 75 ) +f ( 0 , 5 )
f ❑' ( 1 )=
12 h

¿
−(−28 ,5 )+ 8. ( −4547
256 )
−8. (
256 )
−1167
−0,875

12.0 , 25
¿−26

Error: ε = |
−26−(−26)
−26
.100 %=0 %|
−f (x i+2 )+ 16 f (x i +1)−30 f (x i)+16 f (x i−1)−f (x i−2)
f ' ' (x i)= 2
12 h
−f (1 , 5)+ 16 f (1 , 25)−30 f (1)+16 f (0 , 75)−f (0 , 5)
f ' ' (1)=
12 .0 , 252

''
f ❑( 1 )=
−(−28.5 )+ 16. (−4547
256 )−30. (−10 ) +16. (
256 )
−1167
−(−0,875 )
=−37
2
12 .0 ,25

Error: ε = |
−37−(−37)
−37
.100 %=0 %|
*Example 2 (Not a polynomial): Evaluate the first derivatives of
2

f (x)=e−x sin2 x

at x=1, step size h=0,5

Solution
Analytically
2

f ' (x)=e−x (−2 xsin2 x +2 cos 2 x )

2

f ' (1)=e−1 (−2 sin 2+ 2cos 2)=−0,9752073898

Using centered finite-divided-difference formula with 5 points

2

f ( x i ) =f ( 1 )=e−1 sin 2=0,3345118292

 2

f ( x i +1 )=f ( 1 , 5 )=e−1 , 5 sin 3=0,01487393942

2

f ( x i−1 )=f ( 0 , 5 )=e−0 , 5 sin 1=0,6553382619

2

f ( x i +2 )=f ( 2 )=e−2 sin 4=−0,01386132121

2

f ( x i−2 )=f ( 0 ) =e−0 sin 0=0

−f ( 2 )+ 8 f ( 1 ,5 )−8 f ( 0 , 5 ) + f ( 0 )
f ❑' ( 1 )= =−0,8516422098
12 h

Error: ε = | −0,8516422098−(−0,9752073898)
−0,9752073898 |
.100 %=12, 67 %

2.2. Differential Equations

dy 3 2
*Example: Solve =f (x)=−x +10 x −15 x+7 ,5
dx

With step size h=0.25 and the initial condition at x=0, y=2
Solution
Analytically
−1 4 10 3 15 2
y= x + x − x +7 , 5 x+C
4 3 2
−1 4 10 3 15 2
y ( 0 )=2 ⇒ C=2 ⇒ y = x + x − x +7 ,5 x +2
4 3 2
−1 4 10 3 15 2
 y ( 0 , 25 )= .0 , 25 + .0 , 25 − .0 , 25 +7 , 5.0 , 25+2=3,457356771
4 3 2

Euler’s method
y i+ 1= y i+ f ( x i , y i ) . h

f ( x i , y i )=f ( 0 )=7 , 5

y (0 , 25)= y (0)+7 ,5.0 , 25=2+ 1,875=3,875

Error: ε = |3,875−3,457356771
3,457356771 |.100 % ≈ 12 , 08 %
Heun’s method
0
y i+ 1= y i+ f ( x i , y i ) . h
 0
¿> y (0 , 25)= y (0)+7 , 5.0 ,25=2+1,875=3,875
279
f ( xi +1 , y i +1) =f (0 , 25)=−0 , 25 +10 .0 , 25 −15.0 , 25+7 , 5=
0 3 2
64

f ( xi , y i ) +f ( xi +1 , y i+ 1)
0

y i+ 1= y i+ .h
2
279
7 ,5+
 f (0)+f (0 , 25) 64
y (0 , 25)= y (0)+ .0 ,25=2+ .0 , 25=3,482421875
2 2

Error: ε = |3,482421875−3,457356771
3,457356771 |.100 % ≈ 0 , 72 %
Runge-Kutta method (4th order)
k 1=f ( x i , y i ) =f (0)=7 , 5

( 1 1
)
k 2=f xi + . h , y i+ . k 1 . h =f (0+
2 2
0 , 25
2
)=f (0,125)=−0,1253 +10 .0,1252−15.0,125+7 ,5=
2959
512

k =f ( x + h , y + . k .h )=f (0,125)=
1 1 2959
3 i i 2
2 2 512

3 2 279
k 4=f ( x i+ h , y i +k 3 h )=f (0 , 25)=−0 , 25 +10 .0 , 25 −15.0 , 25+7 , 5=
64
1
y i+ 1= y i+ (k 1+ 2 k 2+ 2 k 3+ k 4 ). h
6

 y ( 0 , 25 )= y ( 0 ) +
1
6(7 , 5+2.
2959
512
+2.
2959 279
+
512 64 )
.0 , 25=2+1,457356771

¿ 3,457356771

Error: ε = |3,457356771−3,457356771
3,457356771 |.100 %=0 %
3. Solving using software (Octave-7.3.0)
3.1. Numerical differentiation
*Example 1 (Polynomial): Evaluate the first and the second derivatives of
4 3 2
f (x)=−x −2 , 5 x −5 x −4 , 5 x+3
at xi = 1, step size h = 0,25
 *Example 2 (Not a polynomial): Evaluate the first derivatives of
2

f (x)=e−x sin2 x

at x=1, step size h=0,5

3.2. Differential Equations

dy 3 2
*Example: Solve =f (x)=−x +10 x −15 x+7 ,5
dx
With step size h=0.25 and the initial condition at x=0, y=2
4. Estimating the errors
Error calculations are already included in 2. Numerical examples
5. Conclusion
 About errors:
- For numerical calculation of differentiation as well as differential equation, the
smaller the step size h is, the better the results.
- For Differentiation, the more terms added to the formulas, the smaller the
errors.
- In Differential Equations, the Runge-Kutta method shows the best result while
Euler’s method gets the largest error among the 3 methods mentioned.
 Agreement between solutions obtained by hand and by software:
- In case the function is a polynomial, software (Octave) solution is the same as
the one solved analytically by hand. (The results in example 1 of Numerical
Differentiation; The graphs shown in 3.2 coincides with each other)
- For a non-polynomial function, software shares the same result with the
numerically manual one. However, in such cases, the accuracy of software’s
solutions depends on the formula input (the method used). For instance, a
better result be obtained if the five-point formula is applied instead of the
three-point one or, Runge-Kutta formulas are input instead of Heun’s.
 Conclusion:
In real life, especially in the technical field, it is more often to jump into
complex non-polynomial functions, hence, numerical methods seem to be more
useful. Besides, software has high accuracy, quick calculation, which have been an
irreplaceable tool for engineer.