Chapter 2

M O D EL ING, C O M PU T ERS AN D ER R O R AN ALYSIS

1
NUMERICAL MODELING

3
 Non-Computer Methods
• If we do not have computers (Pre-computer
era) 1
Nonlinear 𝑏 ⋅ 𝑥 + 𝑐 ⋅ exp 𝑥 2 + 𝑑 ⋅ (ln 𝑥 − cos(𝑥 3 − 𝑥)) = 𝑞
o Analytical solutions can be searched.
▪ Usually works for linear (may be
Linear 𝑏 ⋅ 𝑥 = 𝑞 (if we assume 𝑐, 𝑑 = 0)
too hard for nonlinear)

▪ Can be derived for a small class of Quadratic: 𝑐 + 𝑏 ⋅ 𝑥 + 𝑎 ⋅ 𝑥 2 = 0

problems. −𝑏± 𝑏2 −4𝑎𝑐
Analytic solution: 𝑥 =
2𝑎

▪ Real problems are nonlinear and Even being linear, the solution of the system
high dimensional! 𝑩⋅𝒙=𝒒
might be challenging.

4
 Non-Computer Methods
• If we do not have computers (Pre-
computer era)

o Graphical solutions
▪ Tedious for complex processes
▪ Not precise
▪ Without computers, awkward
to implement (requires effort)

Solve,

𝑏 ⋅ 𝑥 + 𝑐 ⋅ exp 𝑥 2 + 𝑑 ⋅ (ln 𝑥 − cos(𝑥)) = 0

for 𝑥.

5
 Non-Computer Methods
• If we do not have computers (Pre-computer era)

o Calculators, slide-rules
▪ Works for complex processes
▪ Not like computers (speed, memory)

https://upload.wikimedia.org/wikipedia/commons/thumb/6/62/Keuffel_and_Esser-Model_4181-1_Log_log_Duplex_Decitrig_slide_rule-IMG_5821-
white_%28cropped%29.jpg/1280px-Keuffel_and_Esser-Model_4181-1_Log_log_Duplex_Decitrig_slide_rule-IMG_5821-white_%28cropped%29.jpg 6
 Non-Computer Methods
• The three phases of Engineering Problem
Solving: Formulation
Deep expressions of relationships
of problem to fundamental laws

Solution
Easy implementation of computer
methods

Interpretation
Calculations let in-depth analysis of
the results and develop intuitions
for a better solution

7
 Non-Computer Methods
• The Reasons for to study numerical methods (according to Chapra) :

• Many daily life problems are high dimensional, nonlinear, discrete. Not possible to solve
analytically. Numerical methods supply approaches to solve these problems with arithmetic
operation in finite time.

• Many techniques are packed together in professional programs, then knowledge on the theory let
the coder to analyze bottlenecks or numerical errors.

• Numerical methods are an efficient way to learn computers, and vice-versa. If you want to learn a
technique, just try to write its code.

• Numerical methods is a solid approach to understand mathematics. It is an algebraic operations

approach to cumbersome math problems. Even math proofs are benefiting from trials of the codes.

8
NUMERICAL MODELING IN REAL LIFE APPLICATIONS

9
 Numerical Integration- 𝑓(𝑥)
Computer Methods Differentiation Fitting
• What kind of basic 𝑦
problems we encounter
in Numerical Methods: 𝐼 𝑥

𝑓(𝑥) interpolation
𝐼 = න 𝑓 𝑥 𝑑𝑥 𝑥
𝑓(𝑥)

𝑓 𝑥0 = 0 𝑏11 ⋯ 𝑏1𝑛 𝑥0 𝑞0
⋮ ⋱ ⋮ × ⋮ = ⋮ 𝑥
𝑥 𝑏𝑛1 ⋯ 𝑏𝑛𝑛 𝑥𝑛 𝑞𝑛
Root finding
𝑩⋅𝒙 =𝒒
System of Equations

10
 Computer Methods: Surface Reconstruction Technologies

• Surface

• interpolation of point cloud, NURBS

• algorithms: automatic remeshing, noise
removal and smoothing
• sources for 3D printing

source: https://www.meshlab.net/#description 11
 Computer Methods: Surface Reconstruction Technologies

Knots

Surface elements

Splines
Volume elements
12
 Computer Methods: Surface Reconstruction Technologies

Linear

Quadratic

source: https://www.meshlab.net/#description 13
 Computer Methods: Physically Based Models

Source: https://www.youtube.com/watch?v=LBMIzr0aPI8 (SIMULIA) https://www.youtube.com/watch?v=KkoaKgv2fhc

https://www.youtube.com/watch?v=Scp38PWbE0A&list=PLBcXmzqlnOg05RHAhbKH5cjw-R1djkBTs&index=6 https://www.youtube.com/watch?v=GBuLxoCs65E 14
 Computer Methods: Physically Based Models

PySPH library for

physical models

Equations

Numerical Schemes can be

derived from these equations.
https://en.wikipedia.org/wiki/Smoothed-
https://pysph.readthedocs.io/en/latest/tutorial/circular_patch_simple.html
15
particle_hydrodynamics#:~:text=Smoothed%2Dparticle%20hydrodynamics%20(SPH),1977%2C%20initially%20for
https://pysph.readthedocs.io/en/latest/design/overview.html %20astrophysical%20problems.
NUMERICAL MODELING

16
Numerical modeling
• Mathematical Modeling:

𝑦 = 𝑓 𝑥, 𝑐, 𝑃

𝐷𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡 𝑣𝑎𝑟𝑖𝑎𝑏𝑙𝑒𝑠 = 𝑓(𝑖𝑛𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡 𝑣𝑎𝑟𝑠, 𝑝𝑎𝑟𝑎𝑚𝑒𝑡𝑒𝑟𝑠, 𝑓𝑜𝑟𝑐𝑖𝑛𝑔 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛𝑠)

Formulation
Deep expressions of relationships of problem to
fundamental laws

Problem Definition Solution

Which variables are effecting the nature of the Easy implementation of computer methods
problem and its outcomes. Which physical processes
are involved and why are we solving for.
Interpretation
Calculations let in-depth analysis of the results
and develop intuitions for a better solution

17
Numerical modeling
• A simple mathematical model:

𝐹 =𝑚⋅𝑎

𝐷𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡 𝑣𝑎𝑟𝑖𝑎𝑏𝑙𝑒𝑠 = 𝑓(𝑖𝑛𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡 𝑣𝑎𝑟𝑠, 𝑝𝑎𝑟𝑎𝑚𝑒𝑡𝑒𝑟𝑠, 𝑓𝑜𝑟𝑐𝑖𝑛𝑔 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛𝑠)

• Forcing function is 𝐹, the net force acting on the body, which is a constant function.

• 𝑚 is the mass of the object. This is a constant parameter of the model.

• Acceleration 𝑎 is the dependent variable, since 𝑎 is not the reason but the result of the
forcing function, then
1
𝑎= 𝐹
𝑚
18
Numerical modeling
• Acceleration 𝑎 is the dependent variable, since 𝑎 is not the reason but the result of the
forcing function, then
1
𝑎= 𝐹
𝑚
• Idealized model
• Can be used for prediction
• Can be coded easily

• We can create a new form of the equation by using the definition of acceleration:

𝑑𝑣 𝑑𝑣 1 Δ𝑣 1 𝑣(𝑡𝑖+1 ) − 𝑣(𝑡𝑖 ) 1
𝑎= → = 𝐹 → = 𝐹 → = 𝐹
𝑑𝑡 𝑑𝑡 𝑚 Δ𝑡 𝑚 𝑡𝑖+1 − 𝑡𝑖 𝑚
Analytical Numerical

19
 Numerical modeling
• Generally, this approach is called the Euler Method:

𝑑𝑦 𝑦(𝑡𝑖+1 ) − 𝑦(𝑡𝑖 )
= 𝑓 𝑦, 𝑡 → = 𝑓 𝑦𝑖 , 𝑡𝑖
𝑑𝑡 𝑡𝑖+1 − 𝑡𝑖

𝑦(𝑡𝑖+1 ) = 𝑦 𝑡𝑖 + 𝑓(𝑦𝑖 , 𝑡𝑖 )(𝑡𝑖+1 − 𝑡𝑖 )

𝑦(𝑡𝑖+1 )

𝑦(𝑡𝑖 )

𝑡𝑖 𝑡𝑖+1 20
Numerical modeling
• What is 𝐹?

• Can be decomposed into 𝐹𝐺 and 𝐹𝐷𝑟𝑎𝑔

Gravitational force

𝐹𝐺 = 𝑚 𝑔

Drag force (due friction, linear approach)

𝐹𝐷𝑟𝑎𝑔 = −𝑐 𝑣

Net force acting on a falling object

𝐹 = 𝐹𝐺 + 𝐹𝐷𝑟𝑎𝑔 = 𝑚𝑔 − 𝑐𝑣
21
Numerical modeling
• A much more complex model can be given,

1
𝐹𝐷𝑟𝑎𝑔 = 𝑐 𝜌 𝑣 2𝐴
2
𝑐 is drag coefficient
𝜌 is the atmospheric density
𝑣 is the speed
𝐴 is the cross-sectional area of the object

• For the simplified model the solution of the model

𝑑𝑣 1 𝑔𝑚 𝑐
= (𝑚𝑔 − 𝑐𝑣) → 𝑣 𝑡 = (1 − exp( 𝑡))
𝑑𝑡 𝑚 𝑐 𝑚

22
Numerical modeling
• For the simplified model the analytical solution of the model

𝑑𝑣 1 𝑔𝑚 𝑐
= (𝑚𝑔 − 𝑐𝑣) → 𝑣 𝑡 = (1 − exp( 𝑡))
𝑑𝑡 𝑚 𝑐 𝑚

• For the simplified model the numerical solution of the model

𝑣(𝑡𝑖+1 ) − 𝑣(𝑡𝑖 ) 𝑐 𝑣(𝑡𝑖+1 ) − 𝑣(𝑡𝑖 ) 𝑐

= 𝑔− 𝑣 → = 𝑔 − 𝑣(𝑡𝑖 )
𝑡𝑖+1 − 𝑡𝑖 𝑚 𝑡𝑖+1 − 𝑡𝑖 𝑚

𝑐
𝑣(𝑡𝑖+1 ) = 𝑣 𝑡𝑖 + 𝑔 − 𝑣(𝑡𝑖 ) ⋅ (𝑡𝑖+1 − 𝑡𝑖 )
𝑚

23
Numerical modeling
• For the simplified model the numerical solution of the model

𝑐
𝑣(𝑡𝑖+1 ) = 𝑣 𝑡𝑖 + 𝑔 − 𝑣(𝑡𝑖 ) ⋅ (𝑡𝑖+1 − 𝑡𝑖 )
𝑚

𝑣(𝑡𝑖+1 )
If we know the speed at time 𝑡𝑖
we can predict the speed at 𝑡𝑖+1
𝑣(𝑡𝑖 )

𝑡𝑖 𝑡𝑖+1

24
Numerical modeling
• For the more complex model, the numerical solution is

𝑐 2
𝑣(𝑡𝑖+1 ) = 𝑣 𝑡𝑖 + 𝑔 − 𝜌 𝑣 𝑡𝑖 𝐴 ⋅ (𝑡𝑖+1 − 𝑡𝑖 )
2𝑚

If we know the speed at time 𝑡𝑖

we can predict the speed at 𝑡𝑖+1
𝑣(𝑡𝑖+1 )

This is the simplest approach to

𝑣(𝑡𝑖 )
solve a dynamical system

𝑡𝑖 𝑡𝑖+1

25
Numerical modeling
Example
If the numerical velocity model of an object entering the atmosphere is given by

𝜌 𝑐𝐴 2
𝑣(𝑡𝑖+1 ) = 𝑣 𝑡𝑖 + 𝑔 − 𝑣 𝑡𝑖 ⋅ (𝑡𝑖+1 − 𝑡𝑖 )
2 𝑚
then calculate the speed of the entering object after 60 seconds and 120 seconds. The parameters are:

𝑘𝑔
𝑚 = 1000𝑘𝑔, 𝑐 = 1, 𝐴 = 50.3𝑚2 , 𝜌 = 1.225 𝑚3 , 𝑔 = 9.8 𝑚/𝑠 2
𝑣0 = 7300 𝑚/𝑠

26
Numerical modeling
Problem
The evaporation rate of a spherical droplet is given by the equation below. Here 𝑉 is the volume and 𝐴 is
the surface area. Calculate the contact area of the droplet after 10 and 30 minutes by a numerical
approach. When 𝑅 < 0.001𝑚𝑚. then it is assumed that the droplet is completely vaporized. Calculate
the time required for the droplet to vaporize completely.
mm
The initial volume of the droplet is 𝑉0 = 2.5 𝑚𝑚. and 𝑘 = 0.08 min . The volume of a sphere is calculated
4
as 𝑉 = 3 𝜋𝑅3 and the surface area of the droplet is half the area of a sphere, then 𝐴 = 2𝜋𝑅2 .

surface 𝑑𝑉
= −𝑘 𝐴
𝑑𝑡
𝑅

27
MEASURING ERRORS

28
Measuring errors
• Let 𝑥ො be a computed quantity of 𝑥. Then,

𝐓𝐫𝐮𝐞 𝐀𝐛𝐬𝐨𝐥𝐮𝐭𝐞 𝐄𝐫𝐫𝐨𝐫 𝜀𝑇𝑎𝑏𝑠 = 𝑥ො − 𝑥

𝑥ො − 𝑥
𝐓𝐫𝐮𝐞 𝐑𝐞𝐥𝐚𝐭𝐢𝐯𝐞 𝐄𝐫𝐫𝐨𝐫 𝜀𝑇𝑟𝑒𝑙 =
𝑥

if the quantity exist. If the exact quantity 𝑥 is not known, then, as an approximation in iterative
algorithms, the current estimation can be used for the ratio.

𝑥ො𝑐𝑢𝑟𝑟𝑒𝑛𝑡 − 𝑥ො𝑝𝑟𝑒𝑣
𝐀𝐩𝐩𝐫𝐨𝐱𝐢𝐦𝐚𝐭𝐞 𝐑𝐞𝐥𝐚𝐭𝐢𝐯𝐞 𝐄𝐫𝐫𝐨𝐫 𝜀𝑟𝑒𝑙
Ƹ =
𝑥ො𝑐𝑢𝑟𝑟𝑒𝑛𝑡

29
Measuring errors
• Let 𝑓𝑙(𝑥) be the IEEE machine arithmetic model of 𝑥. Then the relative rounding error of 𝑥
is no more than one-half machine epsilon:

𝑓𝑙𝑟𝑜𝑢𝑛𝑑 (𝑥) − 𝑥 1
𝐑𝐞𝐥𝐚𝐭𝐢𝐯𝐞 𝐑𝐨𝐮𝐧𝐝𝐢𝐧𝐠 𝐄𝐫𝐫𝐨𝐫 = ≤ 𝜖𝑚𝑎𝑐ℎ
𝑥 2

𝑓𝑙𝑐ℎ𝑜𝑝 (𝑥) − 𝑥
𝐑𝐞𝐥𝐚𝐭𝐢𝐯𝐞 𝐂𝐡𝐨𝐩𝐩𝐢𝐧𝐠 𝐄𝐫𝐫𝐨𝐫 = ≤ 𝜖𝑚𝑎𝑐ℎ
𝑥

30
APPROXIMATIONS AND ROUND -OFF ERRORS

31
Approximations and round-off errors
• We discussed that a number that exceeds representation capability of a floating point
system can be rounded to the nearest value. Thus, this number will always be shifted from
its original value. Such as 𝜋;

𝝐𝒎𝒂𝒄𝒉
𝝅 = 𝒇𝒍𝒓𝒐𝒖𝒏𝒅 𝝅 + 𝜺, 𝜺≤
𝟐

𝝅 ≈ 𝒇𝒍𝒓𝒐𝒖𝒏𝒅 𝝅

• Then;
Case example
Radar pulse 1Hz

𝐓𝐫𝐮𝐞 𝐕𝐚𝐥𝐮𝐞 = 𝐀𝐩𝐩𝐫𝐨𝐱𝐢𝐦𝐚𝐭𝐢𝐨𝐧 + 𝐄𝐫𝐫𝐨𝐫

Sample Case from Real Life:

These are called round-off errors. Ground unit missed the

software problem in the
computer. This problem
calculation that became
operated.
attacking missile because of a
ground system’s weapon control
led to an inaccurate tracking
worse the longer the system
500m

32
LOSS OF SIGNIFICANCE

33
Loss of significance
• Loss of significance is encountered when two floating point descriptions 𝑝, 𝑞 are nearly
equal with respect to their number of significant digits.

• Assume a computer can store three decimal digits. For example it transforms 𝑞 = 3.1415 to
𝑞 = 3.14, and 𝑞 = 123.1 to 𝑞 = 123.

Example
Solve the quadratic expression for 𝑥.
𝑥 2 + 912 𝑥 = 3

−𝑏± 𝑏2 −4𝑎𝑐 −912 + 924 −4⋅3

Since 𝑥 = , then, 𝑥1 = and Python gives:
2𝑎 2

34
Loss of significance
−𝑏± 𝑏2 −4𝑎𝑐
• Since 𝑥 = , this problem occurs for the second root of the equation where,
2𝑎

𝑏≈ 𝑏 2 − 4𝑎𝑐
are nearly equal in their FP representation.

• To eliminate such problems, a mathematical approach should be developed. To eliminate

the subtraction from the numerator multiply both sides by 𝑏 + 𝑏2 − 4𝑎𝑐 :

𝑏 + 𝑏2 − 4𝑎𝑐 −𝑏 + 𝑏2 − 4𝑎𝑐 −2𝑐

× →
2
𝑏 + 𝑏 − 4𝑎𝑐 2𝑎 𝑏 + 𝑏2 − 4𝑎𝑐

Correct result 35
TRUNCATION ERRORS AND TAYLOR SERIES

36
Truncation errors and Taylor series
• Polynomials are the basic representation forms to describe any function.

Theorem
(Weierstrass Approximation Theorem, 1885)

Let 𝑓: 𝑏, 𝑑 → ℜ, −∞ < 𝑏 ≤ 𝑑 < ∞ be a continuous function. For a real 𝛿 > 0, for every 𝑥 ∈ [𝑏, 𝑑]
there exists a polynomial 𝑝(𝑥) with real coefficients, satisfying 𝑓 𝑥 − 𝑝(𝑥) < 𝛿.

37
Truncation errors and Taylor series
• Polynomials are the basic representation forms to describe any function.
Taylor’s Theorem

′′′ 𝑞 𝑛
𝑓′′ 𝑞 𝑓 𝑓 𝑞
𝑓 𝑥 = 𝑓 𝑞 + 𝑓′ 𝑞 𝑥 − 𝑞 + 𝑥−𝑞 2+ 𝑥−𝑞 3 + ⋯+ 𝑥−𝑞 𝑛 + 𝑅𝑛
2! 3! 𝑛!

𝑥
𝑛
𝑥−𝑡 𝑛+1
𝑅𝑛 = න 𝑓 𝑡 𝑑𝑡
𝑛!
𝑞

𝑓 𝑛+1 𝜉 𝑛+1
= 𝑥−𝑞 𝑞<𝜉≤𝑥
(𝑛 + 1)!

38
 Truncation errors and Taylor series
• Polynomials are the basic representation forms to describe any function.

Linear
• Zeroth Order (Constant) Approximation quadratic

𝑓 𝑥 ≈𝑓 𝑞 𝑓(𝑥)

• First Order (Linear) Approximation

𝑓 𝑥 ≈ 𝑓 𝑞 + 𝑓′ 𝑞 𝑥 − 𝑞
constant
• Second Order (Quadratic) Approximation 𝑓(𝑞)

𝑓′′ 𝑞
𝑓 𝑥 ≈ 𝑓 𝑞 + 𝑓′ 𝑞 𝑥 − 𝑞 + 𝑥−𝑞 2
𝑞 𝑥
2!

39
Truncation errors and Taylor series
• Polynomials are the basic representation forms to describe any function.

Linear

quadratic

𝑓(𝑥)

𝑓(𝑞) constant

𝑞 𝑥

40
 Truncation errors and Taylor series: Code

exp(𝑥)

2nd order

1th order

0th order

Source: NUMERICAL METHODS IN ENGINEERING WITH PYTHON 3 Jaan Kiusalaas, GPL licence code 41
 Truncation errors and Taylor series
• Polynomials are the basic representation forms to describe any function.
Example
• Use Zeroth and Second Order approaches to approximate the polynomial at 𝑞 = 1. Try to predict the function’s value
at 𝑥 = 1.5. What is the absolute error of these approximations.

𝑃 𝑥 = 𝑥4 − 𝑥 + 1

Solution: (The real value evaluated at 1.5 is 𝑷 𝟏. 𝟓 = 𝟒. 𝟓𝟔𝟐𝟓)

Zeroth order -> 𝑓 𝑥 ≈ 𝑓 𝑞 , then assume 𝑃 1.5 ≈ 𝑃 1 calculate 𝑃 1 = 14 − 1 + 1 = 1, then 𝑃 1.5 ≈ 1

𝑓′′ 𝑞
Second order approximation -> 𝑓 𝑥 ≈ 𝑓 𝑞 + 𝑓′ 𝑞 𝑥 − 𝑞 + 𝑥−𝑞 2
2!

First, we need first and second derivatives of 𝑃 𝑥

42
 Truncation errors and Taylor series
• Polynomials are the basic representation forms to describe any function.
𝑓′′ 𝑞
Second order approximation -> 𝑓 𝑥 ≈ 𝑓 𝑞 + 𝑓′ 𝑞 𝑥 − 𝑞 + 𝑥−𝑞 2
2!

The function is: 𝑃 𝑥 = 𝑥4 − 𝑥 + 1

Its derivatives are:

𝑃′ 𝑥 = 4𝑥 3 − 1
𝑃′′ 𝑥 = 12𝑥 2

𝑃′′ 𝑞
𝑃 𝑥 ≈𝑃 𝑞 + 𝑃′
𝑞 𝑥−𝑞 + 𝑥 − 𝑞 2 = 𝑃 𝑞 + 4𝑥 3 − 1 𝑥 − 𝑞 + 6𝑥 2 𝑥 − 𝑞 2
2!
Then 𝑃 1.5 ≈ 1 + 4 ⋅ 1 − 1 1.5 − 1 + 6 ⋅ 1.5 1.5 − 1 2 = 𝟒. 𝟕𝟓
(close to the real value 4.5625)

43
 Truncation errors and Taylor series
• Polynomials are the basic representation forms to describe any function.
Example
𝜋 𝜋
• Use first and second order approaches to find sin( 4 ) by using 𝑞 = 6 .

𝑓 𝑥 = sin(𝑥)
𝑓(𝑥)

𝑥
𝑞 𝜋 𝜋 𝜋
4 2

44
Truncation errors and Taylor series
• Sometimes when it is hard to evaluate the differential form of an arbitrary function, the
Taylor form can be used to obtain an approximate derivative at the defined point. Since;

𝑓′′ 𝑞 𝑓 ′′′ 𝑞 𝑓 𝑛 𝑞
𝑓 𝑥 = 𝑓 𝑞 + 𝑓′ 𝑞 𝑥 − 𝑞 + 𝑥−𝑞 2+ 𝑥−𝑞 3 + ⋯+ 𝑥−𝑞 𝑛 + 𝑅𝑛
2! 3! 𝑛!

And from the first order definition we can estimate 𝑓(𝑥𝑖+1 ) by using the quantities at 𝑞 = 𝑥𝑖 .

𝑓 𝑥𝑖+1 = 𝑓 𝑥𝑖 + 𝑓 ′ 𝑥𝑖 𝑥𝑖+1 − 𝑥𝑖

as a result a first order approximation to derivative is:

𝑓 𝑥𝑖+1 − 𝑓 𝑥𝑖
= 𝑓 ′ 𝑥𝑖
𝑥𝑖+1 − 𝑥𝑖

45
Truncation errors and Taylor series
Order of Truncation

𝑓 𝑥 + ℎ = 𝑓 𝑥 + 𝑓 ′ 𝑥 ℎ + . . . + 𝑅𝑛

𝑓 𝑛+1 𝜉 𝑛+1
𝑅𝑛 = ℎ
(𝑛 + 1)!

Then the order of the error for a truncation operation is said to be 𝑂 ℎ𝑛+1 . Here ℎ is said to be the step
size of the approximation.

46
 Truncation errors and Taylor series
• Polynomials are the basic representation forms to describe any function.
Example
• Calculate the first derivative of the polynomial at 𝑥𝑖 = 1, 𝑥𝑖+1 = 1.2 (ℎ = 0.2) with a first order approach.

𝑃 𝑥 = 𝑥2 − 𝑥

Solution:
The analytical derivative is
𝑃′ 𝑥 = 2𝑥 − 1
𝑷′ 𝟏 = 𝟏

The numerical result is;

𝑃 𝑥𝑖+1 − 𝑃 𝑥𝑖 0.24 − 0
𝑃′ 𝑥𝑖 ≈ = = 𝟏. 𝟐
𝑥𝑖+1 − 𝑥𝑖 0.2

47
 Truncation errors and Taylor series
• Polynomials are the basic representation forms to describe any function.
Example
• Calculate the first derivative of the polynomial at 𝑥𝑖 = 1, 𝑥𝑖+1 = 10 ℎ = 9 with a first order approach.

𝑃 𝑥 = 𝑥2 − 𝑥

Solution:
The analytical derivative is 𝑥𝑖+1 should be close
𝑃′ 𝑥 = 2𝑥 − 1 to 𝑥𝑖 , the expected
𝑷′ 𝟏 = 𝟏 error is 𝑂(ℎ).

The numerical result is;

𝑃 𝑥𝑖+1 − 𝑃 𝑥𝑖 90 − 0
𝑃′ 𝑥𝑖 ≈ = = 𝟏𝟎
𝑥𝑖+1 − 𝑥𝑖 9

48
 Truncation errors and Taylor series
Example
The following Taylor polynomial can be used to approximate 𝑒 𝑥

1 1 1
𝑒 𝑥 = 1 + 𝑥 + 2! 𝑥 2 + 3! 𝑥 3 + ⋯ + 𝑛! 𝑥 𝑛

If we know that 𝑒 0 = 1, then calculate the transcendental real 𝑒 by using the above formula.
0
1 2 1 3 1 𝑛
𝑒 = 1 + 0 + 0 + 0 + ⋯+ 0
2! 3! 𝑛!
Solution
𝑒 1 = 1 + 1 + 𝑂 ℎ2 = 2
1 2
𝑒 = 1 + 1 + 1 + 𝑂 ℎ3 = 2.5
1
2
1 2 1 3
𝑒 = 1 + 1 + 1 + 1 + 𝑂 ℎ4 = 2.5 + 0.16 = 2.66
1
2 6

49
 Truncation errors and Taylor series
• The Taylor expression under summation operator is as follows:
𝑛
𝑓 (𝑘) 𝑥 𝑘
𝑓 𝑥+ℎ = ෍ ℎ + 𝑂 ℎ𝑛+1
𝑘!
𝑘=0

Example
Calculate the truncation order of the sum of the Taylor polynomials for

1 1
𝑒 ℎ = 1 + ℎ + 2! ℎ2 + 3! ℎ3 + 𝑂(ℎ4 )
and
1 1 4 1 6
sin ℎ = 1 − ℎ2 + ℎ − ℎ + 𝑂(ℎ7 )
2 24 720

50
 Truncation errors and Taylor series
Example
The following Taylor polynomial can be used to approximate 𝑒 𝑥

1 1 1
𝑒 𝑥 = 1 + 𝑥 + 2! 𝑥 2 + 3! 𝑥 3 + ⋯ + 𝑛! 𝑥 𝑛

Then calculate the integral by using the approximation 𝑒 𝑥 which has order of truncation error 𝑂 𝑥 3 .

න 𝑒 𝑥 𝑑𝑥 =?
0

51
PROPAGATION OF ERRORS

52
Propagation of errors
• Errors may propagate through function evaluations. Let 𝑔(𝑥1 , 𝑥2 ) be a function:

𝑔 𝑥1 , 𝑥2 = 𝑥1 + 𝑥2

and due to numerical round of errors,

𝑥1 = 𝑓𝑙𝑟𝑜𝑢𝑛𝑑 (𝑥1 ) + 𝜀1 = 𝑥ො1 + 𝜀1

Then for a computer presentation, yields,

𝑔 𝑥1 , 𝑥2 = 𝑥ො1 +𝑥ො2 + (𝜀1 + 𝜀2 )

53
Propagation of errors
• Errors may propagate through function evaluations. Let 𝑔(𝑥1 , 𝑥2 ) be a function:

𝑔 𝑥1 , 𝑥2 , … , 𝑥𝑛 = 𝑥1 + 𝑥2 + ⋯ + 𝑥𝑛

and due to numerical round of errors,

𝑥1 = 𝑓𝑙𝑟𝑜𝑢𝑛𝑑 (𝑥1 ) + 𝜀1 = 𝑥ො1 + 𝜀1

Then for a computer presentation, yields,

𝑔 𝑥1 , 𝑥2 = 𝑥ො1 +𝑥ො2 + ⋯ + 𝑥ො𝑛 + (𝜀1 + 𝜀2 + ⋯ +𝜀𝑛 )

𝜖 𝑛𝜖
Since 𝜀𝑗 ≤ , the max error propagated by 𝑔() is limited above by ≤ .
2 2

54
Propagation of errors
• Let 𝑔(𝑥1 , 𝑥2 ) be a function:

𝑔 𝑥1 , 𝑥2 = 𝑥1 ⋅ 𝑥2

the error propagated by the function is,

𝑥1 ⋅ 𝑥2 = 𝑥ො1 + 𝜀1 ⋅ 𝑥ො2 + 𝜀2 = 𝑥ො1 ⋅ 𝑥ො2 + 𝑥ො2 ⋅ 𝜀1 + 𝑥ො1 ⋅ 𝜀2 + 𝜀1 ⋅ 𝜀2

The relative error in the product is ≈0

𝑥1 ⋅ 𝑥2 − 𝑥ො1 ⋅ 𝑥ො2 𝜀1 𝜀2 𝜀1 ⋅ 𝜀2
𝜀𝑟𝑒𝑙 = ≈ + + → 𝜀𝑟𝑒𝑙1 + 𝜀𝑟𝑒𝑙2
𝑥1 ⋅ 𝑥2 𝑥1 𝑥2 𝑥1 ⋅ 𝑥2

For the products , propagation of the relative errors is additive

55
Propagation of errors
• How can we measure the propagation rate through function evaluations? If 𝑓(𝑥) is a
function and 𝑥ො is an approximation of 𝑥. Then what will be the effect of the difference
between 𝑥, 𝑥ො on the outcomes of 𝑓 𝑥 ?. This can be symbolized as,

how
𝑥 − 𝑥ො
effects the difference
𝑓 𝑥 − 𝑓(𝑥)
ො

Lets define the differences as,

Δ𝑥ො = 𝑥 − 𝑥ො

Δ𝑓 𝑥ො = 𝑓 𝑥 − 𝑓(𝑥)
ො
56
Propagation of errors
• Lets define the differences as,
Δ𝑥ො = 𝑥 − 𝑥ො

Δ𝑓 𝑥ො = 𝑓 𝑥 − 𝑓(𝑥)
ො
• From Taylor’s expansion
′
𝑓′′ 𝑞 2
𝑓 𝑥 =𝑓 𝑞 +𝑓 𝑞 𝑥−𝑞 + 𝑥−𝑞 +⋯
2!

The first order analysis yields,

𝑓 𝑥 = 𝑓 𝑥ො + 𝑓 ′ 𝑥ො 𝑥 − 𝑥ො

𝑓 𝑥 − 𝑓 𝑥ො = 𝑓 ′ 𝑥ො 𝑥 − 𝑥ො

ෝ = 𝒇′ 𝒙
𝜟𝒇 𝒙 ෝ 𝚫ෝ
𝒙
57
Propagation of errors: Stability and Condition
• Stability and Condition is related to the • The relative error of 𝑥 is :
propagation of Errors:
𝑥 − 𝑥ො
From the first order analysis: 𝑥ො

𝑓 𝑥 = 𝑓 𝑥ො + 𝑓 ′ 𝑥ො 𝑥 − 𝑥ො • The ratio of the relative error in 𝑓(𝑥) and the relative

error in 𝑥 is:
𝑓 𝑥 − 𝑓 𝑥ො = 𝑓 ′ 𝑥ො 𝑥 − 𝑥ො
𝑓 ′ 𝑥ො
𝑥 − 𝑥ො 𝒇′ 𝒙
ෝ
𝑓 𝑥ො
ෝ
→𝒙
The relative error in propagation is: 𝑥 − 𝑥ො 𝒇 𝒙ෝ
𝑥ො
𝑓 𝑥 − 𝑓 𝑥ො 𝑓 ′ 𝑥ො 𝑓 ′ 𝑥ො 𝒇′ 𝒙
ෝ
= 𝑥 − 𝑥ො ≈ 𝑥 − 𝑥ො ෝ
𝐂𝐨𝐧𝐝𝐢𝐭𝐢𝐨𝐧 𝐍𝐮𝐦𝐛𝐞𝐫 = 𝒙
𝑓 𝑥 𝑓 𝑥 𝑓 𝑥ො 𝒇 𝒙ෝ

58
Propagation of errors
• This coefficient is very important to evaluate the characteristics of a numerical problem.
Functions with very high condition numbers are said to be ill-conditioned and unstable.

𝒇′ 𝒙
ෝ
ℎ(𝑏) ෝ
𝜿=𝒙
ℎ(𝑥) 𝒇 𝒙ෝ 𝑔(𝑥)

𝑏 𝑔(𝑏) 𝑏
𝜿≫𝟏 𝜿≈𝟏

𝑏෠ ෠
𝑔(𝑏) 𝑏෠

෠
ℎ(𝑏)

ill-conditioned (errors are greatly amplified) 59

 Truncation errors and Taylor series
Example
𝜋
Calculate the condition number of sin(𝑥 2 ) at 𝑥ො = 4 .
Solution
𝑓 ′ 𝑥ො
𝜅 = 𝑥ො
𝑓 𝑥ො
𝑓 𝑥 = sin(𝑥 2 ) and 𝑓′ 𝑥 = 2𝑥 cos(𝑥 2 )

Then,
𝜋 2 𝜋 2
𝜅 = 2𝑥ො 2 cot(𝑥ො 2 ) = 2 cot = 1.739
4 4

60
 Truncation errors and Taylor series
Example
Calculate the condition number of 𝑓 𝑥 = 𝑥 100 − 1 at 𝑥ො = 2 .

Solution
𝑓 ′ 𝑥ො
𝜅 = 𝑥ො
𝑓 𝑥ො
𝑓 𝑥 = 𝑥 100 − 1 and 𝑓′ 𝑥 = 100𝑥 99

Then
100 ⋅ 299
𝜅 = 2 ⋅ 100 = 100
2 −1

Steepness of the function and its derivative resulted in a very high condition number. If the function evaluations around
𝑥ො = 2 are used iteratively, then the results will be unstable.

61
TOTAL NUMERICAL ERROR

62
Total numerical error
• Total numerical error is the sum of truncation and round-off errors.

total numerical error = truncation error + round-off errors

• Round-off errors increases by the loss of significance.

• Round-off errors increases when the number of algebraic operations increases. The
dimension of the problem makes it worse.

• Truncation errors can be minimized by decreasing the step size ℎ.

63
Total numerical error
• Total numerical error is:
total numerical error = truncation error + round-off errors
𝜀∑ = 𝜀𝑡𝑟𝑢𝑛𝑐 + 𝜀𝑟𝑜𝑢𝑛𝑑−𝑜𝑓𝑓

(log error)
𝜀𝑡𝑟𝑢𝑛𝑐
𝜀∑
log(𝜀∑ )

𝜀𝑟𝑜𝑢𝑛𝑑−𝑜𝑓𝑓

log(ℎ) (log step size)

64
Total numerical error
• Error Analysis of Numerical Differentiation (first order)

See the slide: Truncation errors and Taylor series

2
𝑓 𝜉
𝑓 𝑥𝑖 + ℎ = 𝑓 𝑥𝑖 + 𝑓′ 𝑥𝑖 ℎ + ℎ2 𝜀𝑡𝑟𝑢𝑛𝑐
2!
𝜀∑

𝑓 𝑥𝑖 + ℎ − 𝑓 𝑥𝑖 1 𝑓2 𝜉 2 log(𝜀∑ )
𝑓′ 𝑥𝑖 = − ⋅ ℎ
ℎ ℎ 2!
2
𝜀𝑟𝑜𝑢𝑛𝑑−𝑜𝑓𝑓
𝑓 𝑥𝑖 + ℎ − 𝑓 𝑥𝑖 𝑓 𝜉
𝑓′ 𝑥𝑖 = − ℎ
ℎ 2! log(ℎ)
Finite-difference
approximation Truncation error

65
Total numerical error
• Error Analysis of Numerical Differentiation (first order)

𝑓 𝑥𝑖 + ℎ − 𝑓 𝑥𝑖 𝑓2 𝜉
𝑓 ′ 𝑥𝑖 = − ℎ
ℎ 2!

Computer representations will inevitably include the effects of rounding, then

𝑓 𝑥𝑖 + ℎ = 𝑓መ 𝑥𝑖 + ℎ + 𝜀𝑖+ℎ
𝑓 𝑥𝑖 = 𝑓መ 𝑥𝑖 + 𝜀𝑖
Then the total error is:

𝑓መ 𝑥𝑖 + ℎ − 𝑓መ 𝑥𝑖 𝜀𝑖+ℎ − 𝜀𝑖 𝑓መ 2 𝜉
𝑓መ ′ 𝑥𝑖 = + − ℎ
ℎ ℎ 2!

66
Total numerical error
𝜀𝑖+ℎ − 𝜀𝑖 𝑓መ 2 𝜉
𝜀Σ = − ℎ
ℎ 2!

since the max value of 𝜀𝑖+ℎ − 𝜀𝑖 in a floating point operation is limited by machine epsilon from above,

|𝜀𝑖+ℎ − 𝜀𝑖 | ≤ 𝜖

𝜖 𝑀
𝜀Σ = + ℎ
ℎ 2

Then a minimum of 𝜀Σ can be obtained for the optimal step size ℎ𝑜𝑝𝑡 ,

𝑑𝜀Σ 𝜖 𝑀 2𝜖
=− 2+ →0 ℎ𝑜𝑝𝑡 =
𝑑ℎ ℎ 2 𝑀

67
Total numerical error
Example
A centered difference formula for the first derivative is defined as follows:

𝑓 𝑥 + ℎ − 𝑓 𝑥 − ℎ 𝑓 3 𝜉
𝑖 𝑖
𝑓 ′ 𝑥𝑖 − ℎ = − ℎ2
2ℎ 3!

Determine optimal step size ℎ = ℎ𝑜𝑝𝑡 that minimizes the total numerical error based on a single
precision (32-bit) system.

68
Total numerical error

69
Total numerical error
𝜋
sin′
3

loss of precision

70
PROBLEM SESSION

71
Problems

Problem
Sometimes loss of significance can be avoided by rearranging the terms. Find equivalent formula for the
functions to avoid this problem.

a) 𝑥 2 − 1 − 𝑥, for large 𝑥

b) ln 𝑥 + 1 − ln 𝑥 , for large 𝑥
𝜋
c) cos2 𝑥 − sin2 𝑥 , for 𝑥 ≈ 4 (by using the formula for cos(2𝑥))

72
Problems
Problem
Find condition numbers for the functions.

a) 𝑓 𝑥 = exp −𝑥 , for 𝑥 = 1.00001

b) 𝑥 2 − 1 − 𝑥, for 𝑥 = 300

c) sin(𝑥)/(1 + cos(𝑥)), for 𝑥 = 1.0001𝜋

73
Problems
Problem
Find the Taylor approximation 𝑞ො 𝑥 to the given 𝑞(𝑥) function at 𝑥0 = 0 on the order of 𝑂(𝑥 2 ). Then
determine the value of the integral according to this approximation. Calculate the true relative error by
using your evaluation.
𝑒

𝑞 𝑥 = ln(𝑥 2 ) න ln(𝑥 2 ) 𝑑𝑥 = 2
1

𝑞ො 𝑥 =? න 𝑞ො 𝑥 𝑑𝑥 =?
1

74