INTERPOLATION & APPROXIMATION

Arvind Deshpande
Curve Fitting
 Interpolation - The estimation of values between well-known discrete
points
 The data is known to be very precise, the basic approach is to fit a
curve or a series of curves that pass directly through each of the points.
 Such data usually originates from tables. Examples are values for the
density of water or for the heat capacity of gases as a function of
temperature.
 Approximation - To derive a single curve that represents the general
trend of the data.
 The data exhibits a significant degree of error or “noise”.
 Because any individual data point may be incorrect, no effort is made to
intersect every point. Rather, the curve is designed to follow the pattern
of the points taken as a group.

9/11/2019 Arvind Deshpande (VJTI) 2

9/11/2019 Arvind Deshpande (VJTI) 3
APPROXIMATION
Statistics
 Arithmetic Mean of a sample
y
 y i

 y 
2
 Standard Deviation y
sy  i

n 1

 Sum of squares of residuals  

S t   yi  y
2

 Variance
s y2 
  y  y
i
2

n 1

s 
2  i  i /n
y 2
 y 2
 
n 1
y

9/11/2019 Arvind Deshpande (VJTI) 5

Linear Regression
 Where substantial error is
associated with data, polynomial
interpolation is inappropriate and
may yield unsatisfactory results
when used to predict intermediate
values.
 Experimental data is often of this
type.

y  a0  a1 x  e
 e  y  a0  a1 x
9/11/2019 Arvind Deshpande (VJTI) 6
Criteria for best fit

n n
a)  ei   yi  a0  a1 xi 
i 1 i 1
n n
b)  | ei | | yi  a0  a1 xi |
i 1 i 1

c) min max  yi  a0  a1 xi 

9/11/2019 Arvind Deshpande (VJTI) 7

Least-Squares Fit of a Straight Line

S r   e   yi ,m easured  yi ,model  0   yi   a0  a1 xi
n n
2 2
i

0   yi xi   a0 xi  a1 x i2
i 1 i 1
n
   yi  a0  a1 xi  na0   xi a1   yi
2

i 1

S r
 2  yi  a0  a1 xi 
 x a   x a   x y
i 0
2
i 1 i i
a0
n x y   x  y
S r a  i i i i

 2  yi  a0  a1 xi xi  n x   x 
1 2 2

a1 i i

a0  y  a1 x
9/11/2019 Arvind Deshpande (VJTI) 8
Quantification of error of linear regression
Sr
Standard error of the estimate, s y / x 
n2
S  Sr
Coefficien t of determinat ion, r 2  t where St    yi  y 
2

St
n xi yi   xi  yi 
Correlatio n Coefficien t, r 
n xi2  xi  n yi2  yi 
2 2

9/11/2019 Arvind Deshpande (VJTI) 9

 Linearization of
Nonlinear
relationship

9/11/2019 Arvind Deshpande (VJTI) 10

Polynomial Regression
y  a0  a1 x  a2 x 2  e 0   yi   a0  a1 xi   a x 2
2 i

S r   e   yi ,m easured  yi ,model 
n n

0   yi xi   a0 xi  a1 xi2   a2 xi3
2 2
i
i 1 i 1

0   yi xi2   a0 xi2  a1 xi3   a2 xi4

 
n
  yi  a0  a1 xi  a x
2 2

 x a   y
na0   xi a1 
2 i
i 1 2
i 2 i
S r
a0

 2 yi  a0  a1 xi  a2 xi2   x a   x a   x a   x y
2 3
i 0 i 1 i 2 i i

S r

 2 yi  a0  a1 xi  a2 xi2 xi    x a   x a   x a   x y
2
i 0
3
i 1
4
i 2
2
i i
a1
S r
a2

 2 yi  a0  a1 xi  a2 xi2 xi2  
9/11/2019 Arvind Deshpande (VJTI) 11
Polynomial Regression

 n

x x i
2
i
  a0    yi 
   
  xi x x 2
i
3
i a
  
1   xi y i 
 xi2 x x 3 4  a2   xi2 yi 
 i i   
Sr
sy / x 
n3
9/11/2019 Arvind Deshpande (VJTI) 12
Multiple Linear Regression
y  a0  a1 x  a2 x2  e
0   yi   a0  a1 x1i   a2 x2i
S r   ei2   yi ,m easured  yi ,model 
n n

0   yi x1i   a0 x1i  a1 x12i   a2 x1i x2i

2

i 1 i 1
n
0   yi x2i   a0 x2i  a1 x1i x2i   a2 x22i
   yi  a0  a1 x1i  a2 x2i 
2

i 1
na0   x1i a1   x2i a2   yi
S r
a0
 2  yi  a0  a1 x1i  a2 x2i   x a   x a   x x a   x y
1i 0
2
1i 1 1i 2 i 2 1i i

S r
 2  yi  a0  a1 x1i  a2 x2i x1i 
 x a   x x a   x a   x y
2i 0 1i 2 i 1
2
2i 2 2i i
a1
S r
 2  yi  a0  a1 x1i  a2 x2i x2i 
a2
9/11/2019 Arvind Deshpande (VJTI) 13
Multiple Linear Regression

 n

x 1i x 2i   a0    y i 
   
  x1i x 2
1i x x 1i 2 i   a1     x1i yi 
  x2 i x x x  2 i i 
2   
 1i 2 i

2i   2 
a  x y
Sr
sy / x 
n3
9/11/2019 Arvind Deshpande (VJTI) 14
Multiple Linear Regression

Q  a0 D S
a1 a2

Taking logarithm
logQ  log a0  a1 log D  a 2 log S
In this form, the equation is suited for
multiple linear regression because
log Q is a linear function of log S and log D.

9/11/2019 Arvind Deshpande (VJTI) 15

INTERPOLATION
Interpolation
 The most common method used
for this purpose is polynomial
interpolation.
 The general formula for an nth-
order polynomial is
f x  a0  a1 x  a2 x 2  .....  an x n
 For n + 1 data points, there is one
and only one polynomial of order n
that passes through all the points.

9/11/2019 Arvind Deshpande (VJTI) 17

Newton’s divided-difference interpolating polynomial

 Linear Interpolation
f  x   f  x0  f  x1   f  x0 

x  x0 x1  x0
f  x1   f  x0 
f  x   f  x0   x  x0 
x1  x0

 Finite-divided-difference approximation
of the first derivative

9/11/2019 Arvind Deshpande (VJTI) 18

Quadratic Interpolation

f x   b0  b1 x  x0   b2  x  x0  x  x1 
b0  f x0 
f x1   f x0 
b1 
x1  x0 Finite divided
f x2   f x1  f x1   f x0  difference of the
 second derivative =
x2  x1 x1  x0 difference of two
b2  finite divided
x2  x0 difference of the first
derivatives
9/11/2019 Arvind Deshpande (VJTI) 19
General Form of Newton’s Interpolating Polynomial of
nth order
f  x   b0  b1  x  x0   b2  x  x0  x  x1   .....  bn  x  x0  x  x1 .... x  xn 1 
b0  f  x0 
b1  f x1 , x0 
b2  f x2, x1 , x0 
.
.
.
bn  f xn , xn 1 ,......., x1 , x0 
f  x   f  x0   f x1 , x0  x  x0   f x2, x1 , x0  x  x0  x  x1 
 .....  f xn , xn 1 ,......., x1 , x0  x  x0  x  x1 .... x  xn 1 
9/11/2019 Arvind Deshpande (VJTI) 20
 Errors of Newton’s Interpolating Polynomials
f n 1  
Rn  xi 1  xi n1 .....Truncation error for Taylor series(ξ in between i and i  1)
n  1!
f n 1  
Rn  x  x0 x  x1 ...... x  xn  [Similar relation for n th order interpolat ing polynomial ]
n  1!
Rn  f x, xn , xn 1 ,...., x0  x  x0  x  x1 ......  x  xn 
Rn  f xn 1 , xn , xn 1 ,...., x0  x  x0  x  x1 ......  x  xn 

9/11/2019 Arvind Deshpande (VJTI) 21

Newton’s Interpolating Polynomials

 Points should be centred around and as close as possible to

the unknown value.

9/11/2019 Arvind Deshpande (VJTI) 22

Lagrange Interpolating polynomials
n
f n  x    Li  x  f  xi 
i 0

x  xj
Li  x   
n
( denotes product of)
j 0
j i
xi  x j
n 1

f1  x  
x  x1  f x   x  x0  f x 
x0  x1  0 x1  x0  1
Reformulat ion of Newton' s interpolat ing polynomial
f1  x   b0  b1  x  x0 
f  x1   f  x0 
f1  x   f  x0   x  x0 
x1  x0

f1  x   f  x0  
f  x1 
x  x0   f x0  x  x0 
x1  x0 x0  x1

f1  x  
x  x1  f x   x  x0  f x 
x0  x1  0 x1  x0  1
9/11/2019 Arvind Deshpande (VJTI) 23
Lagrange Interpolating polynomials

n  2 (Second order)

f 2 x  
 x  x1 x  x2 
f  x0  
x0  x1 x0  x2 
x  x0 x  x2  f x  
x1  x0 x1  x2  1

x  x0 x  x1  f x 
x2  x0 x2  x1  2

9/11/2019 Arvind Deshpande (VJTI) 24

Newton and Lagrange polynomials

 For cases where the order of the polynomial is unknown, the Newton
method has advantages because of the insight it provides into the
behaviour of the different order formulas.
 In addition, the error estimate represented can usually be integrated
easily into the Newton computation because the estimate employs a
finite difference. Thus, for exploratory computations, Newton’s method is
often preferable.
 When only one interpolation is to be performed, the Lagrange and
Newton formulations require comparable computational effort. However,
the Lagrange version is somewhat easier to program. Because it does
not require computation and storage of divided differences, the
Lagrange form is often used when the order of the polynomial is known
a priori.
9/11/2019 Arvind Deshpande (VJTI) 25
Spline Interpolation
 Higher order polynomials can lead
to erroneous results because of
round-off error and overshoot.
 An alternative approach is to apply
lower-order polynomials to
subsets of data points. Such
connecting polynomials are called
spline functions.
 Spline - Thin flexible strip

9/11/2019 Arvind Deshpande (VJTI) 26

Linear Spline

f x   f x0   m0 x  x0  x0  x  x1
f x   f x1   m1 x  x1  x1  x  x2
.
.
.
f x   f xn 1   mn 1 x  xn 1  xn 1  x  xn
f  xi 1   f xi 
mi 
xi 1  xi
9/11/2019 Arvind Deshpande (VJTI) 27
Quadratic Spline

fi(x) = aix2+bix+ci
Conditions
1. The function values of adjacent polynomials must be equal at the
interior knots.
2. The first and last functions must pass through the end points.
3. The first derivatives at the interior knots must be equal.
4. Assume that the second derivative is zero at the first point.

9/11/2019 Arvind Deshpande (VJTI) 28

Cubic Spline

 Each pair of knots is connected by x  xi x  xi 1

f i "  x   f i "  xi 1   f i "  xi 
a cubic, the second derivative xi 1  xi xi  xi 1
within each interval is a straight Rewriting first term
line. xi  x x  xi 1
f i "  x   f i "  xi 1   f i "  xi 
xi  xi 1 xi  xi 1
 The second derivatives can be
Integratin g
represented by a first-order
f i "  xi 1  f i "  xi 
Lagrange interpolating polynomial f i ' x    xi  x   x  xi 1 2  C1
2

2 xi  xi 1  2xi  xi 1 
Integratin g again
f i "  xi 1  f i "  xi 
f i x   xi  x   x  xi 1 3  C1 x  C2
3

6 xi  xi 1  6 xi  xi 1 

9/11/2019 Arvind Deshpande (VJTI) 29

Cubic Spline

Invoking the function - equality conditions — f (x) must equal f (x i-1 ) at

x i-1 and f (x) must equal f (x i ) at x i .
f i "  xi 1 
f i  xi 1   xi  xi 1 2  C1 xi 1  C2
6
f i "  xi 
f i  xi   xi  xi 1   C1 xi  C2
2

6
f i  xi   f i  xi 1   f i "  xi   f i "  xi 1  
C1   xi  xi 1 
xi  xi 1   6 

C2 
 
xi f i xi 1   xi 1 f i  xi  xi f i "  xi 1   xi 1 f i "  xi   xi  xi 1 

xi  xi 1  6
9/11/2019 Arvind Deshpande (VJTI) 30
Cubic Spline
f i "  xi 1  f i "  xi 
f i x   xi  x   x  xi 1 3
3

6 xi  xi 1  6 xi  xi 1 
f i  xi   f i  xi 1   f i "  xi   f i "  xi 1  
 x   xi  xi 1 x
xi  xi 1   6 


 
xi f i  xi 1   xi 1 f i  xi  xi f i "  xi 1   xi 1 f i "  xi   xi  xi 1 

xi  xi 1  6
f i "  xi 1  f i "  xi 
f i x   xi  x   x  xi 1 3
3

6 xi  xi 1  6 xi  xi 1 
 fi  xi 1  f i "  xi 1  xi  xi 1  
    xi  x 
 x 
 i i 1x  6 
 f i  xi  f i "  xi xi  xi 1  
    x  xi 1 
 x 
 i i 1x  6 

9/11/2019 Arvind Deshpande (VJTI) 31

Cubic Spline
f i "  xi 1  f i "  xi 
f i ' x   xi  x  . 3  x  xi 1 2 3
2

6 xi  xi 1  6 xi  xi 1 
 fi  xi 1  f i "  xi 1  xi  xi 1  
-  
 x 
 i i 1x  6 
 f i  xi  f i "  xi  xi  xi 1  
  
  xi  xi 1  6 
At x  x i
f i "  xi   fi  xi 1  f i "  xi 1  xi  xi 1  
f i '  xi   xi  xi 1 2 3 -  
6 xi  xi 1   
 i i 1
x x  6 
 f i  xi  f i "  xi  xi  xi 1  
  
 x 
 i i 1x  6 
f i "  xi 1  xi  xi 1  2 f i "  xi  xi  xi 1  f i  xi   fi  xi 1 
f i '  xi    
6 6 xi  xi 1 
9/11/2019 Arvind Deshpande (VJTI) 32
 Cubic Spline
f i "1  xi  f i "1  xi 1 
f i 1 '  x   xi 1  x  . 3  x  xi 2 3
2

6 xi 1  xi  6 xi 1  xi 
 fi 1 xi  f i "1  xi  xi 1  xi  
-  
 x 
 i 1 ix  6 
 f i 1  xi 1  f i "1  xi 1  xi 1  xi  
  

 i 1 i
x  x  6 
At x  x i
f i "1  xi   fi 1 xi  f i "1  xi  xi 1  xi  
f i 1 '  xi   xi 1  x  . 3 -  
2

6 xi 1  xi   
 i 1 i
x x  6 
 f i 1  xi 1  f i "1  xi 1  xi 1  xi  
  

 i 1 i
x  x  6 
 f i "1  xi 1  xi 1  xi  2 f i "1  xi  xi 1  xi  f i 1  xi 1   f i 1  xi 
f i 1 '  xi    
6 6 xi 1  xi 
9/11/2019 Arvind Deshpande (VJTI) 33
Cubic Spline

Invoking the condition that the first derivative s at the knots must be continuous
f i '  xi   f i ' 1  xi 
xi  xi 1  f " xi 1   2xi 1  xi 1  f " xi   xi 1  xi  f " xi 1 

6
 f xi 1   f xi   6
 f xi 1   f xi 
xi 1  xi  xi  xi 1 
n  1 simultaneo us equations
Second derivative s at the end knots are zero (Natural cubic spline)
Can be solved with TDMA

9/11/2019 Arvind Deshpande (VJTI) 34

Multidimensional Interpolation

Bilinear Interpolation

xi  x2 x x
f xi , y1   f x1 , y1   i 1 f x2 , y1 
x1  x2 x2  x1
xi  x2 x x
f xi , y2   f  x1 , y2   i 1 f  x2 , y2 
x1  x2 x2  x1
yi  y 2 y  y1
f xi , yi   f xi , y1   i f xi , y2 
y1  y2 y2  y1

9/11/2019 Arvind Deshpande (VJTI) 35

Bilinear Interpolation

Bilinear Interpolation

f xi , yi 
yi  y2 xi  x2 y  y2 xi  x1
 f x1 , y1   i f x2 , y1  
y1  y2 x1  x2 y1  y2 x2  x1
yi  y1 xi  x2 y  y1 xi  x1
f x1 , y2   i f  x2 , y 2 
y2  y1 x1  x2 y2  y1 x2  x1

9/11/2019 Arvind Deshpande (VJTI) 36

Newton’s Forward difference

x f Δf Δ2 f Δ3 f
x0 f0 Δf0=f1-f0 Δ2f0=Δf1-Δf0 Δ3f0=Δ2f1-Δ2f0
Δ2f0=f2-2f1+f0 Δ3f0=f3-3f2+3f1+f0
x1 f1 Δf1=f2-f1 Δ2f1=Δf2-Δf1
Δ2f1=f3-2f2+f1
x2 f2 Δf2=f3-f2
x3 f3
x 0 , x1 , x 2 , x 3 are equally spaced (h spacing between points).
x-x 0

h
f x   f 0  f 0 (First order)
   1
f x   f 0  f 0  2 f 0 (Second Order)
2!
   1 2    1  2 3
f x   f 0  f 0   f0   f 0 (Third Order)
2! 3!
9/11/2019 Arvind Deshpande (VJTI) 37
Newton’s Backward difference

x f ᐁf ᐁ 2f ᐁ 3f
x0 f0
x1 f1 ᐁf1=f1-f0
x2 f2 ᐁf2=f2-f1 ᐁ2f2=ᐁf2-ᐁf1
ᐁ2f2=f2-2f1+f0
x3 f3 ᐁf3=f3-f2 ᐁ2f3=ᐁf3-ᐁf2 ᐁ3f3=ᐁ2f3-ᐁ2f2
ᐁ2f3=f3-2f2+f1 ᐁ3f3=f3-3f2+3f1+f0
x 0 , x1 , x 2 , x 3 are equally spaced (h spacing between points).
x-x n

h
f x   f n  f n (First order)
   1
f  x   f n   f n   2 f n (Second Order)
2!
   1 2    1  2 3
f  x   f n   f n   fn   f n (Third Order)
2! 3!
9/11/2019 Arvind Deshpande (VJTI) 38