Numerical Differentiation

by Pushpendra Gupta
27-03-2025

1 Introduction
Numerical differentiation is a technique used to approximate the derivative of a function when an analytical
solution is not feasible. The three primary methods discussed are the Forward Difference, Backward Difference,
and Central Difference methods. Implement them in the MATLAB.

2 Forward Difference
The forward difference approximation for the first derivative is given by:
f (x + h) − f (x)
f ′ (x) = lim (1)
h→0 h
For a small step size h, we approximate:
f (x + h) − f (x)
f ′ (x) ≈ (2)
h
The error in this approximation is of the order O(h), meaning that the accuracy improves linearly as h decreases.

3 Backward Difference
The backward difference formula is given by:
f (x) − f (x − h)
f ′ (x) = lim (3)
h→0 h
For small h, the numerical approximation is:
f (x) − f (x − h)
f ′ (x) ≈ (4)
h
This method also has an error of order O(h).

4 Central Difference
A more accurate approximation is provided by the central difference formula:
f (x + h) − f (x − h)
f ′ (x) = lim (5)
h→0 2h
Approximating for small h:
f (x + h) − f (x − h)
f ′ (x) ≈ (6)
2h
2
This method has an error of order O(h ), meaning it is more accurate than the forward and backward difference
methods for the same step size.

5 Error Analysis and Log-Log Plot

If we vary h from 10−1 to 10−5 and plot the numerical error on a log-log scale, we can visualize how the
error decreases as h becomes smaller. A log-log plot is useful for visualizing data that spans multiple orders of
magnitude, revealing the rate of convergence more clearly.
However, choosing a very small h can lead to an increase in error due to the combined effect of truncation
error and round-off error. While reducing h decreases truncation error, round-off errors due to floating-point
arithmetic may grow, leading to an optimal choice of h rather than an arbitrarily small one. Try to visualize it.

1
6 Second-Order Differentiation
The second derivative of a function can be approximated using the central difference method as follows:

f (x + h) − 2f (x) + f (x − h)
f ′′ (x) ≈ (7)
h2
This method has an error of order O(h2 ), similar to the first derivative central difference approximation.
Try implementing this in MATLAB, but ensure that you correctly compute the true value, as now you are
dealing with the second derivative of that function.

2
 Numerical Integration
by Pushpendra Gupta
27-03-2025

1 Introduction
Numerical integration is used to approximate the definite integral of a function when an analytical solution is
not feasible. The three commonly used methods discussed are the Trapezoidal Rule, Simpson’s 1/3 Rule, and
Simpson’s 3/8 Rule. Implement them in MATLAB as discussed in the Class.

2 Trapezoidal Rule
The Trapezoidal Rule approximates the integral of a function by assuming the function behaves like a straight
line between two points. Given a function f (x), the integral from a to b (OR a + h) can be approximated as:
Z b
h
f (x)dx ≈ [f (a) + f (a + h)] (1)
a 2

where h = b − a. This method has an error order of O(h2 ), meaning the accuracy improves quadratically as h
decreases.

2.1 Dividing it in Multiple Subintervals

[This is not in the Syllabus] For a general interval [a, b], we divide it into n subintervals of equal width

b−a
h=
n
and apply the rule iteratively to improve accuracy over large intervals or for functions where considering only
the first and last points may result in significant errors.:
Z b " n−1
#
h X
f (x)dx ≈ f (a) + 2 f (xi ) + f (b) (2)
a 2 i=1

In general, the points in the interval are chosen as:

x0 = a, x1 = a + h, x2 = a + 2h, x3 = a + 3h, ..., xn = b

which can be expressed as:

xi = a + ih, for i = 0, 1, 2, . . . , n.
This is the same for all the subsequent methods given below.

3 Simpson’s 1/3 Rule

Simpson’s 1/3 Rule approximates the function using a quadratic polynomial that passes through three points,
namely a, a + h, and a + 2h. Instead of integrating the function directly, we integrate the quadratic polynomial
over the given interval. Given three points, the integral from a to b is approximated as:
Z b
h
f (x)dx ≈ [f (a) + 4f (a + h) + f (b)] (3)
a 3

where h = b−a 4
2 . This method has an error order of O(h ), making it significantly more accurate than the
Trapezoidal Rule.

1
3.1 Dividing it into multiple Even Segments
[Not in the Syllabus] For an interval divided into even segments, the extended Simpson’s 1/3 Rule is given
by:      
Z b n−1 n−2
h X X
f (x)dx ≈ f (a) + 4 ×  f (xi ) + 2 ×  f (xi ) + f (b) (4)
a 3 i=2, even
i=1, odd

4 Simpson’s 3/8 Rule

Simpson’s 3/8 Rule approximates the function using a cubic polynomial that passes through four points from
a to a + 3h. The integral from a to b is approximated as:
Z b
3h
f (x)dx ≈ [f (a) + 3f (a + h) + 3f (a + 2h) + f (b)] (5)
a 8

where h = b−a 5
3 . This method also has an error order of O(h ), making it comparable in more accuracy to
Simpson’s 1/3 Rule. The number of subintervals should be a multiple of three.

4.1 Dividing into multiples of three Segments

[This is also not in Syllabus] For a general interval divided into multiples of three segments (e.g n =
3 or 6 or 9 etc.), the extended Simpson’s 3/8 Rule is:
 
Z b n−1 n−3
3h  X X
f (x)dx ≈ f (a) + 3 f (xi ) + 2 f (xi ) + f (b) (6)
a 8
i=1, mod 3̸=0 i=3, mod 3=0

The first summation includes all function values at indices that are not multiples of 3, multiplied by 3. The
second summation includes all function values at indices that are multiples of 3 (except the endpoints), multiplied
by 2.