CH 5 – Numerical Diff.

& Integration

Motivation/guiding questions:
1. Given a function defined by a discrete data table, how can I
approximate the value of at a point and ∫ (x)dx ?

2. Given a function ( I know nothing of its derivative /antiderivative or

impossible to find ), how can I approximate value of by linear
combinations of values of ?
Example: It is impossible to evaluate the following integrals

∫ ∫ 1+
 CH 5 – Numerical Diff. & Integration
5.1 Numerical Differentiations
5.1.1 Finite Difference Approximation
Numerical Approximations of nth order derivatives
( of a given function at a point).
 the approximations are derived from Taylor
Polynomial (Numerical Methods in Engineering
with Python, Jaan Kiusalaas, 2013)
 Taylor Polynomial: The value of the function
a distance ℎ away from is

ℎ ℎ
+ℎ = +ℎ ′ + ′′ + ′′′ +⋯
2! 3!
117
 CH 5 – Numerical Diff. & Integration
 The derivation of the finite difference
approximation for the derivatives of f(x) is
based on forward & backward Taylor Series
expansion of f(x) about x, such as

(a) +ℎ = +ℎ + + + +⋯
! ! !

(b) −ℎ = −ℎ + − + −⋯
! ! !

( ) ( ) ( ) ( )
(c) + 2ℎ = + 2ℎ + + + +
! ! !

⋯
( )
(d) − 2ℎ = − 2ℎ + − + −
! ! !
118
 CH 5 – Numerical Diff. & Integration
 The sum and difference of the previous
equations yield

(e) + ℎ + ( − ℎ) = 2 +ℎ + +⋯

(f) + ℎ − ( − ℎ) = 2ℎ + +⋯
!

( )
(c) + 2ℎ + ( − 2ℎ) = 2 + 4ℎ ′′ + +⋯

(d) + 2ℎ − ( − 2ℎ) = 4ℎ + +…

Eqns.(a) – (h) can be viewed as simultaneous equations

that can be solved for various derivatives of f(x).

119
 CH 5 – Numerical Diff. & Integration
 Thenumber of equations involved & the number
of terms kept in each equation depends on the
order of the derivative and the desired degree
of accuracy.
[i] First central difference approximation
Solving eqn.(f) for f’(x) yields:
+ ℎ − ( − ℎ) ℎ
f x = − −⋯
2ℎ 6
Or
( )
(5.1) = + (ℎ )
Similarly , eqn.(e) gives the first central
120
difference approx. for :
 CH 5 – Numerical Diff. & Integration
+ ℎ − 2 ( ) + ( − ℎ) ℎ ( )
f x = − −⋯
ℎ 12
Or
( )
(5.2) = + (ℎ )
Central difference approximations for other
higher derivatives can be obtained from
eqns.(a)-(h) in the same manner.
Eliminating f’(x) from eqn.(f) and (h) and
solving for f’’(x) yields:
( )
(5.3) = + (ℎ )
121
 CH 5 – Numerical Diff. & Integration
Similarly from eqns.(e) and (g) after
eliminating f’’(x) we get as
( ) ( )
(5.4) = + (ℎ )
Table 5.1 Coefficients of Central Finite Difference
Approximation of ( )
( − ) ( − ) ( ) + ( + )
( ) -1 0 1
( ) 1 -2 1
( ) -1 2 0 -2 1

( )
( ) 1 -4 6 -4 1
122
 CH 5 – Numerical Diff. & Integration
[ii] First non central Finite difference
approximation
 Central finite difference approximations are
not applicable for functions defined at the
n-discrete points x0,x1,…, xn as it is
impossible to compute the derivates at x0 and
xn.
 This necessitate an alternative finite
difference expression that require
evaluations of the function only on one side
of x.
Forward difference approximations:
Solving eqn.(a) for f’(x) , we get
+ℎ − ( ) ℎ ℎ ℎ 123
= − − − −⋯
ℎ 2 6 4!
 CH 5 – Numerical Diff. & Integration
Keeping only the 1st term on the RHS yields the 1st
forward difference approximation:
( )
(5.5) = + (ℎ)
Similarly, eqn.(b) yields the first backward
difference approximation:
( )
(5.6) = + (ℎ)
Note that the truncation error is now O(h) , is not
as good as O(ℎ ) in central difference approximations.
Following similar approach discussed before, we get (
from Eqns.(a) & (c))
( )
(5.7) = + (ℎ)
The higher order derivatives ( , … ) involves Taylor
series expansions for f(x+ h), f(x+2h), f(x+3h) etc.
124
 CH 5 – Numerical Diff. & Integration
 First Non-central Finite Difference
Approximation
 applicable to condition that require evaluations
of function only on one side of x.
Table 5.2a Coefficients of Forward Finite Difference
Approximation of (ℎ)
( ) ( + ) ( + ) + ( + )
( ) -1 1
( ) 1 -2 1
( ) -1 3 -3 1

( )
( ) 1 -4 6 -4 1
125
 CH 5 – Numerical Diff. & Integration

Table 5.2b Coefficients of Backward Finite

Difference Approximation of (ℎ)
( − ) ( − ) ( − ) − ( )
( ) -1 1
( ) 1 -2 1
( ) -1 3 -3 1

( )
( ) 1 -4 6 -4 1

126
 CH 5 – Numerical Diff. & Integration
[iii] Second non central Finite difference
approximation
To get a better approximation for f’(x), f’’(x),…,
etc. we need to retain more terms in the Taylor
series.
As an example, let us derive an improved expression
for f’(x).
Starting with Eqns.(a) and (c) , we get
ℎ ℎ ℎ
+ℎ = +ℎ + + + +⋯
2 6 24
4ℎ 2ℎ
+ 2ℎ = + 2ℎ + 2ℎ + + +⋯
3 3
st
Eliminating ′′( ) by multiplying the 1 eqn. by 4 and
subtracting it from the 2nd eqn. yields:
127
 CH 5 – Numerical Diff. & Integration
2 ℎ
+ 2ℎ − 4 + ℎ = −3 − 2ℎ + ℎ + +⋯
3 2
( )
(5.8) = + (ℎ )
Refer the Table 5.3 for higher order derivative
approximation(of error O(ℎ ))
Table 5.3a Coefficients of Forward Finite Difference
Approximation of (ℎ2)

( ) ( + ) ( + ) + ( + ) ( + )

2 ( ) -3 4 -1

( ) 2 -5 4 -1

2 ( ) -5 18 -24 14 -3

( )
( ) 3 -14 26 -24 11 -2 128
 CH 5 – Numerical Diff. & Integration
Error in Finite difference approximations
 A closer look at all finite difference expressions
reveals that the sum of the coefficients to be
ZERO. And this has profound effect on the round off
error.
 If ℎ is very small, the values of , ±ℎ , ( ±
2ℎ), and so on , will be approximately equal. And
when they are multiplied by the coefficients and
added, several significant figures can be lost. Yet
we can not make ℎ too large as truncation error
would become excessive.
Remedy:
 Use double precision arithmetic
 employ finite difference formulas that are
accurate at least (ℎ ) 129
 CH 5 – Numerical Diff. & Integration

Table 5.4 ( )′′ at x = 1 from central Finite difference approx. Eqn.(5.2) versus
1 = = 0.36787944

Six-digit precision Eight-digit precision

0.64 0.380610 0.38060911
0.32 0.371035 0.37102939
0.16 0.368711 0.36866484
0.08 0.368281 0.36807656
0.04 0.37 0.36783125
0.02 0.37 0.3679
0.01 0.38 0.3679
0.005 0.40 0.3679
0.0025 0.48 0.3680
130
0.00125 1.25 0.3712
 CH 5 – Numerical Diff. & Integration

Example 5.1 Implement the numerical approx. of

(ℎ ) for the 1st derivatives of f(x) given
below, and show the effect of step size on the
approximation.

= . sin( )
Use steps size of:
ℎ = 0.1 To use scipy built-in
ℎ = 0.01 derivative check:
ℎ = 0.001 scipy.misc.derivative
ℎ = 0.0001
131
ℎ = 1.0 10
CH 5 – Numerical Diff. & Integration

132
 CH 5 – Numerical Diff. & Integration

Example 5.2 Given the evenly spaced data

points: 0 0.1 0.2 0.3 0.4
0 0.0819 0.1341 0.1646 0.1797

compute ( ) and ( ) at = 0 and 0.2 using

finite difference approximation of (ℎ )
Ans: 0.2 = 0.4235, 0.2 = −2.17

Note: It is impossible to compute

0.2 using the given
data & forward difference with approximation with
(ℎ )

5.1.2 Derivatives by Interpolation

133
[for unevenly spaced – Reading Assignment]
 CH 5 – Numerical Diff. & Integration
5.2 Numerical Integrations
5.2.1 Trapezoidal Rule
 Composite Trapezoidal Rule
 Recursive Trapezoidal Rule
5.2.2 Simpson’s Rule
5.2.3 Romberg Integration* ( Trapezoidal
Rule + Richardson extrapolation)
5.2.4 Integration with unequal segments
5.2.5 Multiple Integral*

134
 CH 5 – Numerical Diff. & Integration
5.2.1 Trapezoidal Rule

(5.1)
ℎ
= [ ( ) + ( )]
2

Figure 5.1 Trapezoidal Rule

 Composite Trapezoidal Rule (5.2)

ℎ
= =[ +2 +⋯+2 + ]
2

135
Figure 5.2 Composite Trapezoidal Rule
 CH 5 – Numerical Diff. & Integration
Example 5.3
Evaluate the integral:
( )
∫
Using composite trapezoidal rule.
Q: How to determine the # of trapeziums?
A: As a thump rule ( basic rule) , if we double
the number of trapeziums and get the same
answer within 1 in 1000000, the answer is
probably correct.
136
Solution:
 CH 5 – Numerical Diff. & Integration
Solution:

137
 CH 5 – Numerical Diff. & Integration
Solution(with stopping criteria):

138
 CH 5 – Numerical Diff. & Integration
 Recursive Trapezoidal Rule
• Let be the integral evaluated with the
Composite Trapezoidal Rule using 2 panels.
Let H = b - a
• Eqn.(5.2) yields the following results for k
= 1, 2 & 3
k = 1 ( one panel):
(5.3)
= [ ( ) + ( )]
2
k = 2 ( two panels):
= [ ( ) + 2 ( + ) + ( )] = + ( + )
139
 CH 5 – Numerical Diff. & Integration
 Recursive Trapezoidal Rule
k = 3 ( four panels):

=[ ( )+2 ( + )+2 + +2 ( + ) + ( )]

= +[ + + ( + )]
• For arbitrary K>1 we have
(5.4a)
= + ∑ + , = 2,3, …
( Recursive Trapezoidal Rule)
• Recursive Trapezoidal rule allows us to monitor
convergence & terminate the process when the difference
b/n and becomes sufficiently small. 140
 CH 5 – Numerical Diff. & Integration
 Recursive Trapezoidal Rule
• A form of (5.4a) that is easier to remember
is:
1 (5.4b)
ℎ = 2ℎ + ℎ ( )
2
• The algorithm computes ( ), given ( )
5.2.2 Simpson’s 1/3 Rule [Even # of intervals]

+ ℎ (5.5)
ℎ =[ +4 + ( )]
2 3
(the area under the parabola)
141
Figure 5.3 Simpson’s Rule
 CH 5 – Numerical Diff. & Integration
 Composite Simpson’s 1/3 Rule

Figure 5.4 Composite Simpson’s Rule

• The integration range(a,b) is divided into
panels( even) of width ℎ = ( − )/ each.
• Applying Eq.(5.5) to two adjacent panels, we
have (5.6)
ℎ
≈[ +4 + ]
3 142
 CH 5 – Numerical Diff. & Integration
• Substituting Eq.(5.6) into
∫ = ∫ =∑ , ,… [∫ ]
Yields

∫ ≈ =[ +4 +2 +4 +⋯

…+ 2 +4 + ( )]

• If the # of panels ISNOT EVEN, we can

integrate over the first(or last) three
panels with Simpson’s 3/8 rule
3ℎ (5.7)
=[ +3 +3 + ] 143
8
 CH 5 – Numerical Diff. & Integration
Example 5.4
.
Estimate ∫ from the following data:
0 0.5 1.0 1.5 2.0 2.5

( ) 1.5000 2.0000 2.0000 1.6364 1.2500 0.9565

144
 CH 5 – Numerical Diff. & Integration
5.2.3 Romberg Integration
• Combines the Recursive Trapezoidal Rule with
Richardson extrapolation.
Example 5.5:
Use Romberg integration to evaluate ∫ 2 cos

145
 CH 5 – Numerical Diff. & Integration
5.2.4 Integration with UNEQUAL SEGMENT
• Experimentally derived data are often of this
type.
• One method is to apply the Trapezoidal rule to
each segment and sum the results:
• The independent input vector, x, should be
MONOTONICALLY ASCENDING
+ ( ) + ( ) + ( )
I=ℎ +ℎ + ⋯+ ℎ
2 2 2

146
 CH 5 – Numerical Diff. & Integration
Example 5.6: Trapezoidal Rule for Unequal
segments
Table 5.4 Data for = 0.2 + 25 − 200 + 675 − 900 + 400 with
unequally spaced values of x.

147
CH 5 – Numerical Diff. & Integration

148
 5.5.2 Multiple Integral
A general equation to compute the average of a
two-dimensional function can be written as
∫ (∫ , )
̅=
( − )( − )
The numerator is called a double integral.

149
Figure 4.5 Double Integral as the area under the function surface
 CH 5 – Numerical Diff. & Integration
Example 5.7 Suppose that the temperature of a
rectangular heated plate is described by the
following function:
, =2 +2 − −2 + 72
If the plate is 8 m long ( x dimension) and 6 m
wide ( y dimension), compute the average
temperature.
Solution:

150
 Unit-VI: Numerical Solution of ODEs

6.1 Overview of Ordinary Differential

Equations
6.2 Initial value problem
6.2.1 Euler’s Method
6.2.2 Heun’s Method
6.2.3 Runge-Kutta Method
 Higher-order Runge-Kutta Method
 Higher Order ODEs & Systems
6.4 Adaptive Runge-Kutta Method
151
 Unit-VI: Numerical Solution of ODEs
6.1 Overview of Ordinary Differential
Equations
 Modeling of dynamic processes,i.e.the behavior
of physical, chemical, or biological systems
that have not reached equilibrium, is done
using mathematical formulations that contain
one or more differential terms such as or
⁄ . Such equations are called differential

equations.
152
 Unit-VI: Numerical Solution of ODEs
6.1 Overview of Ordinary Differential
Equations
Table 6.1 Examples of fundamental laws that are written in
terms of the rate of change of variables(t = time, and x =
position)
Law Mathematical Variables and parameters
Expression
Newton’s 2nd law of motion Velocity (v) , force (F) , and mass
=
(m)
Fourier’s heat law Heat flux (q) , thermal
=−
conductivity ( ) and temperature
( T)
Fick’s law of diffusion Mass flux ( J), diffusion
=−
coefficient (D), and concentration
(c)
Faraday’s law ( voltage drop Voltage drop ( ∆ ), inductance 153
∆ =
across and inductor) (L) and current (i)
 6.2 Initial value problems
 An ordinary differential equation of order
( ) = ( , , ,…, ) (6.1)

can always be transformed into 1st order

equations. Using the notation
( ) (6.2)
= = = … =
 The Equivalent 1st order equations are (6.3a)
= = = … = ( , , ,…, )

 The solution requires the knowledge of n auxiliary

conditions. If these conditions are specified at
154
the same value of x – Initial value problem.
 6.2 Initial value problems
 Then the auxiliary conditions, called initial
conditions, have the form
=∝ =∝ … =∝ (6.3b)

 If are specified at different values of t,

the problem is called a boundary value
problem.
 Using vector notation, we can rewrite
Eqns.(6.3) as
= ( , ) =∝ (6.4a)

.
where , = .
. 155
( , )
 6.2.1 Euler Method
Given ̇ = , , = , we want to determine
X(t) for t>

156
 6.2.2 Heun’s Method
Given ̇ = , , = , we want to determine
X(t) for t>

157
6.2.2 Heun’s Method

158
6.2.3 Runge Kutta Methods

159
6.2.3 Runge Kutta Methods

160
6.2.3 Runge Kutta Methods

161
 Unit-VI: Numerical Solution of ODEs
 Higher Order Runge-Kutta Method

162
 Unit-VI: Numerical Solution of ODEs
 Higher Order ODEs & Systems

163
 Unit-VI: Numerical Solution of ODEs
6.4 Adaptive RK Method

164
 Unit-VI: Numerical Solution of ODEs
( )
Example 6.1: Solve the ODE = −2. . ,
with 0 = 1 using Forward Euler Method.

165
 Unit-V: Numerical Solution of ODEs
Example 6.2: Simulate logistic growth of a
population governed by ODE
( )
= ( ) 1−

Using alpha( ) = 0.2 and = 1.0

166
 Unit-VI: Numerical Solution of ODEs
Example 6.3:The 2nd order differential
equation for the angle of pendulum acted on by
gravity with friction is given by
+ . + . sin ( ) = 0, 0 = /2, 0 =0

167
 Unit-VI: Numerical Solution of ODEs
Example 6.4: Coupled 1st Order ODEs
(apmonitor.com/pdc)
Simulate HIV Infection
The human immunodeficiency virus (HIV) infection spreads and can
develop into acquired immunodeficiency syndrome (AIDS). AIDS can
lead to immune system failure and eventual inability to defend the body
against infection or cancer. Without treatment with antiretroviral drugs,
survival time after infection with HIV is about 9 to 11 years, depending
on a number of factors. Antiretroviral drugs such as TDF (tenofovir),
either 3TC (lamivudine) or FTC (emtricitabine), and EFV (efavirenz) are
recommended by the World Health Organization as soon as HIV
infection is diagnosed. This simulation predicts the spread of HIV
infection in a body with an initial infection.
The spread of HIV in a patient is approximated with balance equations
168
on (H)ealthy, (I)nfected, and (V)irus population counts.
 Unit-VI: Numerical Solution of ODEs
Initial Conditions
H(0) = healthy cells = 1,000,000
I(0) = infected cells = 0
V(0) = virus = 100
Equations
The equations are transient balances on the number of healthy cells (H),
infected cells (I), and virus count (V). Terms on the right side with a
positive sign (blue) increase that corresponding number of cells or virus.
Likewise, terms with a negative sign (red) decrease the number of cells
or virus.
Simulate the healthy, infected, and virus count over the course of 15
years if no treatment such as an anti-retroviral is taken. 169
 Unit-VI: Numerical Solution of ODEs

There are six parameters (kr1..6) in the model that provide the rates of
cell death, infection spread, virus replication, and other processes that
determine the spread of HIV in the body.
Parameters
kr1 = 1e5 = new healthy cells per year
kr2 = 0.1 = death rate of healthy cells
kr3 = 2e-7 = healthy cells converting to infected cells
kr4 = 0.5 = death rate of infected cells
kr5 = 5 = death rate of virus 170
kr6 = 100 = production of virus by infected cells
 Unit-VI: Numerical Solution of ODEs
Example 6.5: 2nd Order Coupled ODEs

The differential equations for this system are:

171