CHAPTER 1

Numerical Methods for Ordinary Differential

Equations

In this chapter we discuss numerical method for ODE . We will

discuss the two basic methods, Euler’s Method and Runge-Kutta
Method.

1. Numerical Algorithm and Programming in Mathcad

1.1. Numerical Algorithm. If you look at dictionary, you will
the following definition for algorithm,

1. a set of rules for solving a problem in a finite

number of steps; 2. a sequence of steps designed for
programming a computer to solve a specific problem.
A numerical algorithm is a set of rules for solving a problem in finite
number of steps that can be easily implemented in computer using any
programming language. The following is an algorithm for compute the
root of f (x) = 0,
Input f , a, N and tol .
Output: the approximate solution to f (x) = 0 with
initial guess a or failure message.
• Step One: Set x = a
• Step Two: For i=0 to N do Step Three - Four
Step Three: Compute x = x − ff0(x) (x)
Step Four: If f (x) ≤ tol return x
• Step Five return ”failure”.
In analogy, a numerical algorithm is like a cook recipe that specify the
input — cooking material, the output—the cooking product, and steps
of carrying computation — cooking steps.
In an algorithm, you will see loops (for, while), decision making
statements(if, then, else(otherwise)) and return statements.
• for loop: Typically used when specific number of steps need
to be carried out. You can break a for loop with return or
break statement.
1
2 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

• while loop: Typically used when unknown number of steps

need to be carried out. You can break a while loop with a
break statement or a return statement.
• if – else(otherwise): Used when condition(s) must meet be-
fore carrying out some steps. The else(otherwise) provide al-
ternative steps if any.

1.2. Programming in Mathcad. To translate a numerical al-

gorithm into a program in Mathcad , you need to define a function
name with function arguments and function body. The function
name can be any sequence of alphabetic letter begin with an English
letter, such any ”myFunction”, ”Myfunction1” etc. The function ar-
guments specify the input to the function, typically the items specified
in the input statement of an algorithm and is an comma(,) separated
list enclose by () after function name, such as myFunction(f, a, N, tol),
myOne(a,b). Notice the function argument is optional, you can define
a function with out argument as myFunction(). The function body
implements the step section of an algorithm and the output statement.
In Mathcad the function body is on the right side of := with vertical
bars as grouping symbol. The entire function body is considered as
a group, inside, it might has many subgroups. For the algorithm of
finding f (x) = 0, there is an subgroup that contains step three and
four, as shown in the following screen shot, the screen shot also shows
how to get the programming toolbar from the math toolbar.

Figure 1. mySolver and Programming toolbar

 1. NUMERICAL ALGORITHM AND PROGRAMMING IN Mathcad 3

To enter code in Mathcad ,

(1) First enter the function name with arguments list and assignment operator
:=, with sequence of typing,
mySolver(f,a,N,tol):
(2) Click the Add Line button or press ] to get a vertical bar and two (you
should add more by press ] several times).
(3) enter x and hold [Shift] type ] to get the local assignment ← or click on the
programming toolbar, at the type a. Notice x ← a reads as ”assign a to x.”
(4) Click for on the programming menu or [Shift][Ctrl][’] to get
for ∈

in after for enter i, in after ∈, enter ”1;N”, and in under for, press left
bracket ] to get a vertical line and several .
(5) To get if operator, you need either click on the toolbar or press [Shift][Ctrl][
] ]. you will get
if
enter condition in after if and other statements in before if.
(6) To get return operator, you need either click on the toolbar or press
[Shift][Ctrl][ ], you get
return
in type any information you want to return.
After defining a function, we call it with concrete input data, as
shown in the screen shot, that call the mySolver function to find zero
for both f (x) = x2 − 1, with initial guess x = 2 and g(t) = t3 − 3t2 − 2t,
with initial guess t = −2. Notice there is two ways to call a function,
one way is to define all arguments before calling the function, another
is to define some arguments and call the function with concrete values
for the undefined arguments.
As in all programming language, Mathcad allows you to call one
function from within another. For example, we could create a function
called myDerivative which will compute derivative of a given func-
tion f (x) at a given number x and step size h > 0 using the forward
difference formula: f 0 (s) ≈ f (x+h)−f
h
(x)
. The following screen shot shows
the call of myDerivative inside mySolver,

Figure 2. Call another function

4 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

The algorithm for this function is very simple (here, we break down
the computation in three steps),
Input f , x, and h .
Output: the approximation to f 0 (x) .
• Step One: Set w = f (x + h)
• Step Two: Set d = f (x)
• Step Three: return w−d h

2. Euler’s Method
2.1. Euler’s Method. Euler’s method is the simplest method in
find approximate solutions to first order equations. From the forward
difference formula
f (x + h) − f (x)
f 0 (x) ≈ ,
h
we have
(1) f (x + h) ≈ f (x) + f 0 (x)h
Now if x0 (t) = f (t, x) is a first order differential equations, apply (1),
we have
x(t + h) ≈ x(t) + f (t, x(t))h.
Suppose we want to find approximate solution over interval [a, b] with
initial value x(a) = x0 , we divided the interval into n subintervals
each with length h = b−a n
, the ends of the subintervals is t0 = a, t1 =
a + h, t2 = a + 2h, · · · , tn = a + nh = b. Start with x0 we can compute
x1 = x0 + f (t0 , x0 )h
x2 = x1 + f (t1 , x1 )h
..
.xn = xn−1 + f (tn−1 , xn−1 )h
Example 2.1. Find approximate to x(1) if x0 (t) = t2 −ex sin(t), x(0) =
1 with h = 0.25.

Solution Here the interval is [0, 1], so a = 0, b = 1. Since h =

b−a
n
= 0.25 we have n = 4 and we need to compute x1 , x2 , x3 , x4 starting
with x0 = x(0) = 1.
x1 = x0 + f (t0 , x0 )h = 1 + f (0, 1) ∗ 0.25 = 0.15853
x2 = x1 + f (t1 , x1 )h = 0.15853 + f (0.25, 0.15853) ∗ 0.25 = 0.01833
x3 = x2 + f (t2 , x2 )h = 0.01833 + f (0.5, 0.01833) ∗ 0.25 = 0.23811
x4 = x3 + f (t3 , x3 )h = 0.23811 + f (0.75, 0.15853) ∗ 0.25 = 0.30128
So the approximation to x(1) is x(1) ≈ x4 = 0.30128. Also x(0.25) ≈
x1 = 0.15853, x(0.5) ≈ x2 = 0.01833, x(0.75) ≈ x3 = 0.23811. a
 2. EULER’S METHOD 5

2.2. Mathcad implementation of Euler’s Method. First, we

have the following simple algorithm for the Euler’s method,
Input f , a, b, x0 n.
Output: the approximate solution to x0 = f (t, x)
with initial guess x0 over interval [a, b].
• Step One: Initialization
Set h = b−a
n
Set x0 = x0
Set t0 = a
• Step Two: For i=1 to n do Step Three

Step Three: Set xi = xi−1 − f (ti−1 , xi−1 ) ∗ h

Set ti = ti−1 + h
• Step Four return x.
Notice, algorithm return an array of values, the ith element of the
return array is an approximations of x(t) at t = a + ih.
The following screen shot shows Mathcad code for implementing
the algorithm. Notice, we also create a function tMesh(a, b, N) to
compute the ending of subinterval for graphing purpose.

Figure 3. Mathcad code for Euler’s Method

Notice the line to line corresponding between the Mathcad and the
algorithm. Since Mathcad programming language is a scripting lan-
guage, the translation between algorithm and code is straight forward,
and you don’t need to worry about the variable type, io, etc. Also,
without explicit return statement, the result of last is, by default, will
be returned.
6 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

The next screen shot shows a call to the myEuler and tMesh for
the equation x0 = t2 x + t2 sin(t3 ). It also shows the graph of approxi-
3
mate solution comparing with the exact solution x(t) = − 10 cos(t3 ) −
1 1
3 10 t 3
10
sin(t3 ) + 10 e

Figure 4. Mathcad Euler’s Approximation to x0 = t2 x + t2 sin(t3 )

2.3. Error analysis of Euler’s Method. There are two source

of error in every numerical method used to approximate a solution of
x0 (t) = f (t, x),
• Local Truncation Error
• The roundoff error.
The local truncation error is due to the method and roundoff error
is due to computer that is used. For Euler’s method, we use xi =
xi−1 + f (ti−1 , xi−1 ) ∗ h to approximate the value of x(ti ), the difference,
assuming no rounding is introduced, is |xi − x(ti )|, and is called the
local truncation error. The following theorem shows how the local
truncation error is depending on f (t, x) and h,
Theorem 2.1. Suppose f (t, x), ∂f∂t (t,x)
, and ∂f∂x
(t,x)
are continuous
b−1
on [a, b], and h = n is the step size. Furthermore let x(t) be the
solution of initial value problem x0 = f (t, x), x(a) = x0 and xi = xi−1 +
f (ti−1 , xi−1 ) ∗ h be the approximate of x(ti ), and let ei = x(ti ) − xi be
 3. RUNGE-KUTTA METHOD 7

the local truncation error, then

1
ei+1 ≤ (1 + hK)|ei | + h2 M, i = 1, ...n.
2
∂f (t,x) 00
, where | ∂x | ≤ K, and |x (t)| ≤ M.
Theorem 2.2 indicates that, without roundoff error, the smaller h
gives better approximate solution. However, due the roundoff error
introduced in each step of √ computation, (such as compute can only
give approximate value for 2) we can’t choose h arbitrarily small.
The following result gives the optimal step size h,
Theorem 2.2. With the same assumption of Theorem 2.2,and sup-
pose at each step the roundoff error is bounded by ² then the optimal
value for the step length is
r
2²
hopt =
M

3. Runge-Kutta Method
3.1. Runge-Kutta Method. There are several Runge-Kutta meth-
ods, but the fourth order Runge-Kutta method is most popular. Sup-
pose x0 (t) = f (t, x), x(a) = x0 . and h = b−a
n
is the step length, the
fourth order Runge-Kutta method is given below,
k1 = hf (ti , xi )
k2 = hf (ti + 12 h, xi + 12 k1 )
k3 = hf (ti + 12 h, xi + 12 k2 )
k4 = hf (ti + h, xi + k3 )
xi+1 = xi + 16 (k1 + 2k2 + 2k3 + k4 )
In general Runge-Kutta method gives more accurate result than Eu-
ler’s method at the same step length. However, Runge-Kutta method
is more expensive to implement, that is, at each step, Runge-Kutta
method requires more computation than Euler’s method( four func-
tion evaluations compare one in Euler’s method).
Example 3.1.
Example 3.2. Find approximate to x(1) if x0 (t) = t2 −ex sin(t), x(0) =
1 with h = 0.25.
Solution Here the interval is [0, 1], so a = 0, b = 1. Since
b−a
h = n = 0.25 we have n = 4 and we need to compute x1 , x2 , x3 , x4
starting with x0 = x(0) = 1, t0 = 0
8 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

x1 :

k1 = hf (t0 , x0 ) = 0.25 ∗ f (0, 1) = 0

k2 = hf (t0 + 12 h, x0 + 12 k1 ) = f (0.125, 1) = −0.373276
k3 = hf (t0 + 12 h, x0 + 12 k2 )
= 0.25f (0.125, 1 + 0.5(−0.373276)) = −0.373276
k4 = hf (t0 + h, x0 + k3 ) = 0.25f (0.25, 1 − 0.373276) = −0.309854
x1 = x0 + 61 (k1 + 2k2 + 2k3 + k4 )
= 1 + 16 (0 + 2 ∗ (−0.373276) + 2 ∗ (−0.373276) − 0.309854)
= 0.923911
t1 = t0 + h = 0 + 0.25 = 0.25

x2 :

k1 = hf (t1 , x1 ) = 0.25 ∗ f (.25, 0.923911) = −0.560741

k2 = hf (t1 + 21 h, x1 + 12 k1 )
= f (0.125, 0.923911 + 0.5(−0.560741)) = −0.719601
k3 = hf (t1 + 21 h, x1 + 12 k2 )
= 0.25f (0.125, 0.923911 + 0.5(−0.719601)) = −0.702688
k4 = hf (t1 + h, x1 + k3 )
= 0.25f (0.5, 0.923911 − 0.702688) = −0.763158
x2 = x1 + 16 (k1 + 2k2 + 2k3 + k4 )
= 0.923911 + 61 (−0.560741 + 2 ∗ (−0.719601) + 2 ∗ (−0.702688) − 0.763158)
= 0.750224
t2 = t1 + h = 0.25 + 0.25 = 0.5

x3 :

k1 = hf (t2 , x2 ) = 0.25 ∗ f (.5, 0.750224) = −0.765171

k2 = hf (t2 + 21 h, x2 + 12 k1 )
= f (0.625, 0.750224 + 0.5(−0.765171)) = −0.735295
k3 = hf (t2 + 21 h, x2 + 12 k2 )
= 0.25f (0.625, 0.750224 + 0.5(−0.735295)) = −0.739508
k4 = hf (t2 + h, x2 + k3 )
= 0.25f (0.75, 0.750224 − 0.739508) = −0.637224
x3 = x2 + 61 (k1 + 2k2 + 2k3 + k4 )
= 0.750224 + 61 (−0.765171 + 2 ∗ (−0.735295) + 2 ∗ (−0.739508) − 0.637224)
= 0.568891
t3 = t2 + h = 0.5 + 0.25 = 0.75
 3. RUNGE-KUTTA METHOD 9

x4 :
k1 = hf (t3 , x3 ) = 0.25 ∗ f (0.75, 0.568891) = −0.641483
k2 = hf (t3 + 21 h, x3 + 12 k1 )
= f (0.875, 0.568891 + 0.5(−0.641483)) = −0.485628
k3 = hf (t3 + 21 h, x3 + 12 k2 )
= 0.25f (0.875, 0.568891 + 0.5(−0.485628)) = −0.510243
k4 = hf (t3 + h, x3 + k3 )
= 0.25f (1, 0.568891 − 0.510243) = −0.308297
x4 = x3 + 61 (k1 + 2k2 + 2k3 + k4 )
= 0.568891 + 61 (−0.641483 + 2 ∗ (−0.485628) + 2 ∗ (−0.510243) − 0.308297)
= 0.446327
t4 = t3 + h = 0.75 + 0.25 = 1.0
So the approximation to x(1) is x(1) ≈ x4 = 0.446327. a

3.2. Mathcad implementation. First, we have the following sim-

ple algorithm for the Euler’s method,
Input f , a, b, x0 n.
Output: the approximate solution to x0 = f (t, x)
with initial guess x0 over interval [a, b].
• Step One: Initialization
Set h = b−a
n
Set x0 = x0
Set t0 = a
• Step Two: For i=1 to n do Step Three
Step Three:
Set k1 = hf (ti−1 , xi−1 )
Set k2 = hf (ti−1 + 12 h, xi−1 + 12 k1 )
Set k3 = hf (ti−1 + 12 h, xi−1 + 12 k2 )
Set k4 = hf (ti−1 + h, xi−1 + k3 )
Set xi = xi + 16 (k1 + 2k2 + 2k3 + k4 )
Set ti = ti−1 + h
• Step Four return x.
This algorithm returns an array of values, the ith element of the
return array is an approximations of x(t) at t = a + ih.
The following screen shot shows Mathcad code for implementing
the algorithm. Again notice the line to line corresponding between the
Mathcad and the algorithm.
10 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

Figure 5. Mathcad code for Euler’s Method

The next screen shot shows a call to the myRungeKutta and tMesh
for the equation x0 = t2 x+t2 sin(t3 ). It also shows the graph of approx-
3
imate solution comparing with the exact solution x(t) = − 10 cos(t3 ) −
1 1
3 10 t 3
10
sin(t3 ) + 10 e , and with the approximate solution using Euler’s
method. In this case Runge-Kutta provides much superior result, an
almost exact match!

Figure 6. Mathcad Runge-Kutta’s Approximation to x0 = t2 x + t2 sin(t3 )

 4. NUMERICAL METHOD FOR SYSTEM OF EQUATIONS 11

4. Numerical Method for System of Equations

Both Euler’s method and Runge-Kutta method can be used to find
the approximate solution to the system of first order differential equa-
tions. In fact, the Mathcad codes for system of first differential equa-
tions are exactly the same as Mathcad does not differentiate scalar and
vectors when performs most computations. The only thing needs to be
taken care is that at each step the result is vector instead of a scalar.
Suppose x(t) is a vector-valued function that satisfies
x0 (t) = F (t, x(t)),
where F (t, x) is also vector-valued function, ti = a+ih then the Euler’s
method compute the ith approximate,
(2) xi = xi−1 + F (ti−1 , xi−1 ).
Example 4.1. Let
x0 (t) = t2 sin(x(t)) + et cos(y(t))
y 0 (t) = 2tx(t) + ey(t)
x(0) = 1, y(0) = −1
Find approximate solution for x(t), y(t) over interval [0, 1] with h =
0.1
· ¸ · 2 ¸
x(t) t sin(x) + et cos(y)
Solution Set x(t) = and F (t, x) =
y(t) 2tx + ey
The equation can then be written, in vector form,
x0 (t) = F (t, x(t)),
· ¸
1
x(0) = From (2) we can find, with t0 = 0,
−1
x1 = ·
x0 + h¸∗ F·(t0 , x0 ) ¸
1 t20 sin(x0 ) + et0 cos(y0 )
= +
· −1 ¸ 2t0 x0 + ey0
cos(−1)
=
e−1
The following screen shot shows the results obtain by calling our
Mathcad implementation of Euler’s method. Notice that in defining
the vector-valued function F (t, X), for Mathcad to know that X is an
vector, in the definition, we use subscript to access the component of
X, as shown in the screen shot.
12 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

Figure 7. Euler’s method for system of equations

a
 4. NUMERICAL METHOD FOR SYSTEM OF EQUATIONS 13

Similarly, we can use the same Mathcad code of Runge-Kutta method

to find approximate solution as shown in the following screen shot for
the above system of equations,

Figure 8. Runge-Kutta’s method for system of equations

4.1. Numerical Method for Higher Order Equations. To

find numerical approximate for solutions of higher order differential
equations using either Euler’s method or Runge-Kutta method, we
first transform an higher order differential equations into a system of
first order differential equations and then apply the methods.
Example 4.2. Rayleigh Equation In modeling the oscillations
of a clarinet reed. Lord Rayleigh introduced an equation of the form
mx00 + kx = ax0 − b(x0 )3
Find approximate solution for m = 1, k = 1, a = 2, b = 3. and initial
conditions x(0) = 1, x0 (0) = −2
14 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

Solution Suppose y(t) = x0 (t) is the velocity, then Rayleigh

equation becomes
x0 = y
y 0 = 2y − 3y 3 − 2x
· ¸
y
So we apply the numerical method for F = and
2y − 3y 3 − 2x
· ¸ · ¸
x(t) = x(t) 1
x0 (t) = F (t, x) with , x(0) = . The following
y(t) −2
screen shot gives approximate solution using Runge-Kutta method,

Figure 9. Runge-Kutta’s method for Rayleigh’s equation

a
we also plot the solution in xy-plane, which is called the velocity-
position phase plane, with y-axis represent the velocity and x-axis is the
 4. NUMERICAL METHOD FOR SYSTEM OF EQUATIONS 15

position. The velocity-position plane shows that all velocity-position

solution pairs are eventually being attracted to an closed orbit, the
closed orbit is called the global attractor.