Curve-fitting and interpolation

Unit III
• Curve fitting
Curve-Fitting and Interpolation – linear least squares fitting problem
– transformations to linearize nonlinear problems
– three solution techniques:
• normal equations
• QR decomposition
• SVD
• Interpolation
– polynomial interpolation
– piecewise polynomial interpolation
– cubic splines

Unit III - Curve fitting and interpolation 1 Unit III - Curve fitting and interpolation 2

Curve fitting Linear curve fitting

• finding an analytical function which approximates a • consider a single variable problem initially
set of data points • start with m data points (xi,yi), i=1,...,m
• statistics can quantify the relationship between the • choose n basis functions f1(x),..., fn(x)
fit and errors in data points (e.g. from experiment) • define a fit function F(x) = c1f1(x) +...+ cnfn(x) as a
• numerical methods is concerned with the linear combination of the basis functions
calculation of the fitting curve itself • the problem:
– find the unknown coefficients c1,..., cn so that F(xi) ! yi
– provide a means to quantify the fit

Unit III - Curve fitting and interpolation 3 Unit III - Curve fitting and interpolation 4

Linear curve fitting Least squares problem

• in general m > n • quality of fit can be measured by sum of squares of

– large number of data points the residuals ! = "ri2 = " [yi - (c1xi + c2)]2
– (much) smaller number of coefficients/basis functions
– easy to calculate the fit coefficients
• so F(xi) = yi exactly is not possible to achieve – agrees with statistical expectations derived from data
• fitting to a line uses two basis functions analysis
– f1(x) = x • minimizing ! with respect to c1 and c2 provides the
– f2(x) = 1 least-squares fit
• the fitting function is F(x) = c1x + c2 – data points xi and yi are known in expression for !
• m residuals are defined by – only c1 and c2 are unknowns

ri = yi - F(xi) = yi - (c1xi + c2)

Unit III - Curve fitting and interpolation 5 Unit III - Curve fitting and interpolation 6
 Least squares problem Geometry or algebra?

• to solve the least squares fitting problem we can ...

1. minimize the sum of squares (geometry) OR
2. solve the over-determined system (algebra)
• the equations you get are ...
– mathematically the same, but
– have significantly different numerical properties

Unit III - Curve fitting and interpolation 7 Unit III - Curve fitting and interpolation 8

Geometric: Minimize the residual Minimize the residual

• to minimize we get
• to minimize ! = !(c1,c2) we must have the partial
derivatives

• differentiating gives (sums are i=1,...,m) • organizing in matrix form gives the normal
equations for the least squares fit:

Unit III - Curve fitting and interpolation 9 Unit III - Curve fitting and interpolation 10

Algebraic: Over-determined system Over-determined system

• formally write the equation of the line through all • a compromise solution can be found by minimizing
the points the residual r = y - Ac (as defined in unit I)
• find the minimum of ! = ||r||22
= rTr = (y - Ac)T(y - Ac)
= yTy - yTAc - cTATy + cTATAc
= yTy - 2yTAc + cTATAc [why?]
• to minimize ! requires that #!/#c = 0
• Ac = y is an over-determined system – differentiation with respect to a vector c
• it has an exact solution only if all the data points lie – means a column vector of partial derivatives with respect
on a line, i.e. y is in the column space of A to c1, c2

Unit III - Curve fitting and interpolation 11 Unit III - Curve fitting and interpolation 12
 Digression: vector derivatives Over-determined system
• now evaluate: #!/#c = 0
• how to differentiate with respect to a vector? • #!/#c = #/#c [yTy - 2yTAc + cTATAc]
• we need some properties of vector derivatives ... = -2ATy + [ATAc + (ATA)Tc]
– #(Ax)/#x = AT
– #(xTA)/#x = A = -2ATy + 2ATAc
– #(xTx)/#x = 2x =0
– #(xTAx)/#x = Ax + ATx
• the notation convention for vector derivative #/#x is
NOT standardized with respect to transposes:
– disagrees with Jacobian matrix definition

Unit III - Curve fitting and interpolation 13 Unit III - Curve fitting and interpolation 14

Normal equations again Linearizing a nonlinear relationship

• a line can be fit to an apparently nonlinear
• these give the normal equations again in a slightly relationship by using a transformation
different disguise: • example: to fit y = c1exp(c2x)
ATAc = ATy – take ln of both sides: ln y = ln c1+ c2x
• these are the same equations as the normal – put v = ln y, # = c2, $ = ln c1 to linearize the problem
equations derived previously from geometric – you get v = # x+ $
reasoning – fit the transformed data points: (xi, vi)
• to fit data to a line you can solve the normal • this procedure minimizes the residuals of the
equations for c transformed data fitted to a line, NOT the residuals
• Matlab example: linefit(x,y) of the original data
– fits data points by solving the normal equations using
backslash

Unit III - Curve fitting and interpolation 15 Unit III - Curve fitting and interpolation 16

Generalizing to arbitrary basis functions Generalizing to arbitrary basis functions

• y = F(x) = c1f1(x) +...+ cnfn(x) is the general linear fit • ... and derive the same normal equations as for line
equation fit
• the basis functions fj .... ATAc = ATy
– are independent of the coefficients cj • some examples of basis functions:
– may be individually nonlinear – {1, x} .... the line fit from previously
• F(x) itself is a linear combination of the fj – {1, x, x2, x3, ..., xk} ..... a polynomial fit
• as before we can write the general over-determined – {x-2/3, x2/3}
system Ac = y: – {sin x, cos x, sin 2x, cos 2x, ...} .... Fourier fit
• the solution c to the normal equations gives the
coefficients of the fitting function
• inverse (ATA)-1 gives useful statistical information
– covariances of the fitting parameters c
– variances "2(cj) on the diagonal
Unit III - Curve fitting and interpolation 17 Unit III - Curve fitting and interpolation 18
 Arbitrary basis functions: example Solving the normal equations
• fit data with the basis functions {x-1,x} • ATAc = ATy is a linear system
• Matlab example: – Gaussian elimination is sometimes ok and provides the
– load (xi,yi) data from xinvpx.dat inverse (ATA)-1 for statistical information
– design matrix a = [1./x x] – LU decomposition is numerically equivalent, and more
efficient if the covariance matrix not required
– solve the normal equations for (c1,c2)
– Cholesky also an option since ATA is positive definite
– can use the mfile fitnorm(x,y,Afun) with ...
– inline function Afun=inline(‘[a./x x]’)
• all solution methods based on normal equations are
inherently susceptible to roundoff error and other
numerical problems
• a better approach is to apply decomposition
techniques directly to the ‘design matrix’ A
– QR decomposition is a stable algorithm
– SVD is best and completely avoids numerical problems

Unit III - Curve fitting and interpolation 19 Unit III - Curve fitting and interpolation 20

Normal equations: Numerical issues Normal equations: example

• normal equations may be close-to-singular • find the least squares solution to the system Ac=y
• very small pivots giving very large cj’s that nearly where A = [1 -1; 3 2; -2 4], y = [4 1 3]T
cancel when F(x) is evaluated • ATA = [14 -3; -3 21]
– data doesn’t match the chosen basis functions • ATy = [1 10]T
– two of them may be an equally good or equally bad fit • so normal equations ATAc = ATy are
– ATA cannot distinguish between the functions so they get
similar very large weights
• ‘under-determined’ due to these ambiguities
• SVD can resolve these problems automatically
• exact LS solution is c1 = 17/95, c2 = 143/285

Unit III - Curve fitting and interpolation 21 Unit III - Curve fitting and interpolation 22

Normal equations: another example Solving the over-determined system

• find the least squares solution to Ac=y with • fit a line through three points (1,1), (2,3), (3,4)
• the over-determined system Ac = y is

• normal equations are ATAc=ATy with

• can solve the normal equations ATAc = ATy and get
the fitting function y = 1.5x - 0.3333 y
• what about applying Gaussian elimination directly
• Cholesky factorization is to the over-determined system?
• you get an inconsistent
• solution is augmented system ....

Unit III - Curve fitting and interpolation 23 Unit III - Curve fitting and interpolation 24
 Why QR factorization? Why QR factorization?
• why doesn’t this approach give the LS solution? • however an orthogonal matrix Q preserves norms:
• the elementary row ops applied to A are ||Qx||2 = [(Qx)T(Qx)]1/2 = [xTQTQx]1/2 = [xTx]1/2 = ||x||2
• so to minimize ||y - Ac||2 we can ...
– look for an orthogonal Q so that ....
– minimizing ||QTy - QTAc||2 is an easy problem to solve

• problem is multiplication by M doesn’t preserve the • factorize the m%n matrix A = QR

– m%n orthogonal Q
L2-norm of the residual:
– n%n upper triangular R
• first how do we find the factorization? ....

• so the ‘solution’ walks away from the LS solution

Unit III - Curve fitting and interpolation 25 Unit III - Curve fitting and interpolation 26

QR factorization QR Factorization

• can apply the Gramm-Schmidt process to the columns • each u can be expressed using the orthonormal basis as:
of A ui = (ui·q1)q1 + (ui·q2)q2 + ...+ (ui·qn)qn i = 1,2,...,n
– converts a basis {u1,...,un} into an orthonormal basis {q1,...,qn} • in matrix form this is
– the most direct approach to finding QR
– not numerically stable though
– other more sophisticated algorithms are used in practice
• consider a full rank n%n matrix A
• the columns of A are
– linearly independent • A = Q R
– form a basis of the column space of A
• this is the QR factorization required
• what is the relationship between
• why is R upper triangular?
A = [u1|u2|...|un] and Q = [q1|q2|...|qn]? – for j$2 the G-S ensures that qj is orthogonal to all u1, u2, ... uj-1

Unit III - Curve fitting and interpolation 27 Unit III - Curve fitting and interpolation 28

QR factorization: example Skinny QR factorization

• suppose A is m%n and has rank n (linearly independent
• find the QR decomposition of
columns)
• previous method still works and gives A=QR with
• apply Gramm-Schmidt to the columns of A to get columns of Q: – Q m%n
– R n%n
• this is the economy (skinny) QR factorization
– contains all the information necessary to reconstruct A

• then

• so

Unit III - Curve fitting and interpolation 29 Unit III - Curve fitting and interpolation 30
 Full vs skinny QR factorization QR Factorization in Matlab
• can also have a full QR factorization with extra stuff • the Matlab function [Q,R] = qr(A) provides the QR
– Q m%m factorization of A
– R m%n • the function [Q,R] = qr(A,0) provides the skinny QR
• last m-n rows of R are zero factorization
• first n columns of Q span the column space of A – we’ll always use this one
• last m-n columns of Q span the orthogonal complement • example:
of the column space of A – calculate the skinny QR factorization of the A in the simple
– this is the nullspace of A 3-point fitting example on slide 23
– not needed to solve the LS problem – check that the residuals are small
– so the skinny QR factorization is good enough for LS problems
• in general if A has rank k<n only the first k columns of Q
are needed

Unit III - Curve fitting and interpolation 31 Unit III - Curve fitting and interpolation 32

QR solution to the LS problem QR solution to LS problem

• a solution to the LS problem Ac = y is obtained by • so the 2-norm of the LS residual is
minimizing the 2-norm of the residual
• some simple calculation and apply the QR factorization:
||y - Ac||2 = ||QT(y - Ac)||2
• magnitude of the residual consists of two parts:
= ||QTy - QTAc||2
– the first part can be zero by selecting a suitable c vector
= ||QTy - Rc||2 – the second part cannot be influenced by the choice of c so is
• write the full QR factorization in block matrix form: irrelevant to the LS problem
– that’s why only the skinny QR factorization is needed
– we’ll drop the Q2 part of Q
• to solve the LS problem Ac = y choose c so that
• then the column vector residual above can be written
QTy = Rc
– where A = QR is the skinny QR factorization

Unit III - Curve fitting and interpolation 33 Unit III - Curve fitting and interpolation 34

QR, LS, and Matlab QR vs Cholesky: example

• solve the simple 3-point line fit example on slide
23 using Matlab and the skinny QR factorization • minimize ||Ax - b|| with
• the backslash operator in Matlab A\y
– if A rectangular (over-determined) Matlab automatically
• the normal equations ATAx = ATb
applies the QR factorization and finds the LS solution
– Matlab assumes the LS solution is desired when
backslash in encountered under this circumstance
– Q is only used to evaluate QTy have solution x1 = x2 = 1
– can avoid storing the Q matrix by careful algorithm • compare solution to this LS problem with
• for a standard system notated as Ax=b the LS Cholesky and QR keeping 8 significant digits ....
solution becomes: QTb = Rx

Unit III - Curve fitting and interpolation 35 Unit III - Curve fitting and interpolation 36
 QR vs Cholesky: example solution
QR vs normal equations
• in the example Cholesky solution of the normal
equations fails due to rounding error
• normal equations can be very inaccurate for ill-
conditioned systems where cond(ATA) is large
• but when m » n the normal equations involve
– half the arithmetic as compared to QR
– less storage than QR
• if A is ill-conditioned and ...
– residual is small then the normal equations are less
accurate than QR
– residual is large then both methods give an inaccurate LS
solution
• choosing the right algorithm is not an easy decision
Unit III - Curve fitting and interpolation 37 Unit III - Curve fitting and interpolation 38

A third choice: SVD and LS problem SVD and LS problem

• written in blocks this is
• we’ll use the standard notation for the LS problem
Ax = b with A m%n
• apply the SVD to A ...
• the norm of the residual is ...
||r|| = ||Ax - b||
= ||USVTx - b||
= ||SVTx - UTb|| • to choose z so that ||r|| is minimal requires (k=1,...,n)
• minimizing the residual is equivalent to minimizing zk = dk / "k "k % 0
||Sz - d|| where z=VTx and d=UTb zk = 0 "k = 0
• we have d = UTb and z = VTx

Unit III - Curve fitting and interpolation 39 Unit III - Curve fitting and interpolation 40

SVD and LS problem: pseudo-inverse

SVD and LS problem: sales pitch
• write S+ to denote the transpose of the matrix • the SVD method is ....
obtained by replacing each non-zero "k by its – powerful
reciprocal – convenient
– this is NOT a regular matrix inverse since S is not square – intuitive
• the minimization condition can be written as – numerically advantageous
VTx = S+UTb so ... • problems with ill-conditioning and large residuals
• the LS solution to Ax = b (with A = USVT) is can be circumvented automatically
x = V S+UTb • the SVD can solve problems for which both the
• the matrix A+ = VS+UT is called the pseudo-inverse normal equations and QR factorization fail
of the coefficient matrix A
– behaves for rectangular matrices like the normal inverse
for square matrices

Unit III - Curve fitting and interpolation 41 Unit III - Curve fitting and interpolation 42
 Polynomial curve fitting
SVD and LS problem: zeroing
• consider curve-fitting with mononomial basis
• use a zero entry in S+ if "j = 0 (to machine functions {1, x, x2, ..., xk}
precision) • can solve the least-squares problem with any of the
• this forces a zero coefficient in the linear previous techniques (normal equations, QR, SVD)
combination of basis functions that gives the fitting • the simplicity of the basis functions allows some
equation .... trickery when setting up the equations
• ... instead of a random large coefficient that has to – to fit known data in column vectors x and y to the quadratic
y = c1x2 + c2x + c3 requires solving the over-determined
delicately cancel with another one system
• if the ratio "j / "1 ~ n&m then zero the entry in the
pseudo-inverse matrix since the value is probably
corrupted by roundoff anyway

Unit III - Curve fitting and interpolation 43 Unit III - Curve fitting and interpolation 44

Polynomial curve fitting in Matlab

Interpolation
• these types of coefficient matrices are called
Vandermonde matrices • general remarks
– but see interpolation section of this unit ..... • polynomial interpolation
• in Matlab: A = [x.^2 x ones(size(x)) ] – mononomial basis functions
– the least squares solution can then be found with x = A\y – Lagrange basis
• Matlab can automate this with the polyfit function – Newton basis ... divided differences
– fits data to an arbitrary nth order polynomial by ... – polynomial wiggle
– constructing the design matrix A and ... • piecewise polynomial interpolation
– obtaining the least squares solution by QR factorization
– cubic splines
– vector p = polyfit(x,y,n) contains the coefficients of the fitting
polynomial in descending order – Bezier curves and B-splines
– the polynomial can be evaluated at xf by yf = polyval(p,xf) • Matlab comments
– it’s also possible to analyse residuals and uncertainties with
optional parameters to polyfit and polyval functions
Unit III - Curve fitting and interpolation 45 Unit III - Curve fitting and interpolation 46

Basic ideas
Interpolation
• given: support points (xi,yi) i = 1, ... ,n that are
• interpolation is the ‘exact’ form of curve-fitting
supposed to result from a function y = f(x)
• for given data points (xi,yi)
• to find: an interpolating function y = F(x) valid for a
– curve-fitting finds the solution to the over-determined
system that is closest to the data (least-squares solution) range of x values that includes the xi’s
– interpolation finds a function that passes through all the • requirements
points and ‘fills in the spaces’ smoothly – F(xi) = f(xi), i=1, ...,n
• piece-wise linear interpolation is the simplest, but – F(x) should be a good approximation of the f(x)
lacks smoothness at the support points (xi,yi) • the function f(x) may
• to approximate a function outside the range of – not be known at all
values of xi use extrapolation – not be easy (or possible) to express symbolically
– not easy to evaluate
– known only in tabular format
Unit III - Curve fitting and interpolation 47 Unit III - Curve fitting and interpolation 48
 Basis functions Mononomial basis functions
• n basis functions '1, '2, ... , 'n can be used to • n basis functions are x0, x1, x2, ... , xn-1
define the interpolating function • there is a unique polynomial of degree n-1 passing
F(x) = a1'1(x)+ a2'2(x)+ ... + an'n(x) through n support points
• polynomial basis functions are common • so no matter how we get the interpolating
F(x) = a1+ a2x2+ ... + anxn polynomial it will be unique
• .....so are Fourier interpolations • however it can be expressed in multiple ways with
different coefficients multiplying the basis functions
F(x) = a1+ a2eix + ... + ane(n-1)ix
– for instance the offset monomials (x-a)0, (x-a)1, (x-a)2, ... ,
• polynomials are easy to evaluate but there are (x-a)n-1 could be used
numerical issues ... – we develop techniques to use alternative basis functions
for the interpolating polynomial (Lagrange, Newton)
• but first look at simple mononomials....
Unit III - Curve fitting and interpolation 49 Unit III - Curve fitting and interpolation 50

Vandermonde systems Vandermonde systems

• this is the simple, direct approach to the problem • [c1 c2 c3] is supposed to be
• example – the exact, unique solution to the 3%3 non-singular
– quadratic interpolating polynomial y = c1x2 + c2x + c3 system in interpolation
– support points (x1,y1), (x2,y2) and (x3,y3) – the least squares solution to an m%3 over-determined
– sub the points into the interpolating function and you get system in curve-fitting
the Vandermonde system: • there is a Matlab function to evaluate the
Vandermonde matrix: vander( [y1 y2 y3] )
• but...Vandermonde systems often
– are ill-conditioned
– have solutions with very large order of magnitude
differences in the coefficients
• compare this system to that for curve fitting....

Unit III - Curve fitting and interpolation 51 Unit III - Curve fitting and interpolation 52

Vandermonde systems: example Vandermonde systems: example

• yearly prices • coefficients vary over 16 orders of magnitude

year = [1986 1988 1990 1992 1994 1996]’; • adding and subtracting very large quantities is
price = [133.5 132.2 138.7 141.5 137.6 144.2]’;
supposed to give a delicate balance, but the result
A = vander(year);
is mostly roundoff corruption
c = A\price;
y = linspace(min(year),max(year)); • a simple re-scaling using offset mononomials can
p = polyval(c,y); sometimes fix the problem
plot(year,price,’o’,y,p,’-’); ys = year - mean(year)
• cond(A) ~ 1031... not good "" A = vander(ys)

– serious problems with roundoff error in calculating the • system not ill-conditioned now !!
coefficients [see the oscillations in the plot] • coefficients vary over only 5 orders of magnitude
– no spurious oscillations now from roundoff

Unit III - Curve fitting and interpolation 53 Unit III - Curve fitting and interpolation 54
 Lagrange polynomials Lagrange polynomials
• now the interpolating polynomial is expressed as
• an alternative basis for polynomials of degree n p1(x) = y1 L1(x) + y2 L2(x)
• first degree polynomial first.... • the basis functions L1(x) & L2(x) are called first-
• the (linear) poly passing through (x1,y1) & (x2,y2) is: degree Lagrange interpolating polynomials
• continuing to second degree we get...

L1 L2
• find this representation by ...
– writing the Vandermonde system
L1(x) L2(x) L3(x)
– solving for c1 and c2 and ...
– re-arranging the polynomial in the form above

Unit III - Curve fitting and interpolation 55 Unit III - Curve fitting and interpolation 56

Lagrange polynomials Lagrange polynomials

• in general for nth order interpolation • no system of equations has to be solved to get the
pn-1(x) = y1 L1(x) + y2 L2(x) + ... + yn Ln(x) interpolating polynomial with Lagrange polynomial
where.... basis functions
• the nth degree Lagrange interpolating polynomials • not susceptible to roundoff error problems
are • illustrative Matlab function lagrint shows an efficient
implementation of Lagrange polys

• notation warning: the meaning of the symbol ‘Lj(x)’

depends on the degree of interpolating polynomial
being used
Unit III - Curve fitting and interpolation 57 Unit III - Curve fitting and interpolation 58

Lagrange polynomials
Lagrange polynomials
• both Vandermonde and Lagrange can be
improved on
– too much arithmetic involved
– data points cannot be added or deleted without starting
the calculations again from scratch
– we don’t know what degree polynomial to use up front
and we can’t adjust that later without starting again
• an alternative is ....

Unit III - Curve fitting and interpolation 59 Unit III - Curve fitting and interpolation 60
 Newton basis functions Newton basis functions
• Newton basis functions are 1, (x-x1), (x-x1)(x-x2), • first consider the quadratic version...
(x-x1)(x-x2)(x-x3), ..., (x-x1)(x-x2)...(x-xn) p2(x) = c1 + c2(x-x1) + c3(x-x1)(x-x2)
• express the interpolating polynomial pn(x) in terms • apply the support point constraints pn(xi) = yi
of this basis:

• and apply forward elimination, first to get

• Newton basis functions
– are computationally efficient
– have important relevance in numerical integration
– have good numerical properties

Unit III - Curve fitting and interpolation 61 Unit III - Curve fitting and interpolation 62

....First order divided differences ....Second order divided differences

• this can be written compactly in the form

• where f[x1,x2,x3] is a second order divided difference

• where f[x1,x2] is a first order divided difference

• continuing with the elimination we get... • in general, for nth degree poly interpolation we
get...

Unit III - Curve fitting and interpolation 63 Unit III - Curve fitting and interpolation 64

Divided differences
....nth order divided differences
• find the 3rd degree Newton interpolating poly for
first four data points in the table below

• where f[x1,x2,... , xn] is an nth order divided difference

• divided differences can be calculated as shown

• notation warning: the indices usually start at zero,
but here they are adjusted to be consistent with
Matlab convention .... array indices cannot be zero
Unit III - Curve fitting and interpolation 65 Unit III - Curve fitting and interpolation 66
 Adding a data point is easy Wiggle
• to find the 4th degree poly that fits all the five points • more support points ( higher degree
in the table interpolating polynomial F(x) is required
– calculate the extra divided difference(s) and get the last
• F(xi) = yi is OK since the interpolant passes
coefficient (see the table on previous slide)
through all the support points
– you start with p3(x) and add one term to it
• but higher order polynomials will exhibit rapid
oscillations between the support points
• data points can easily be added or deleted • this polynomial wiggle is an extraneous artifact of
the interpolation method
• a better form for computation is with nested
multiplication (Horner’s rule): • limits the applicability of higher order polynomial
interpolation to improve accuracy
• suggests piecewise methods....

Unit III - Curve fitting and interpolation 67 Unit III - Curve fitting and interpolation 68

Piecewise polynomial interpolation

Wiggle
• instead of finding a single higher order
interpolating polynomial passing through all the
support points...
• ...find multiple lower order polynomials going
through subsets of support points
• the joints where these fit together are called
breakpoints or knots
• desire for global performance raises complexity
by demanding constraints on how the local
interpolants relate to their neighbours

Unit III - Curve fitting and interpolation 69 Unit III - Curve fitting and interpolation 70

Continuity constraints Searching for sub-intervals

• can demand f(x), f'(x) and/or f''(x) to be • pj(x) is the interpolating polynomial for the jth
continuous at the breakpoints sub-interval (xj ,xj+1)
• fundamental issue: are neighbouring • how to locate the sub-interval which brackets a
interpolating polynomials constrained with given test point x?
respect to each other? ... • assume the data is ordered or manage
– leads to large, sparse systems to solve (cubic splines) appropriate book-keeping to order it
• ... or with respect to the original function? • two techniques that work:
– leads to de-coupled equations to solve, but you need – incremental search ... look through the data in
more information from the data points (Hermite) sequence until the bracketing interval is found
– de-coupled polynomials require identification of the – binary search ... use bisection to locate the sub-interval
appropriate sub-interval for evaluation that x lies in

Unit III - Curve fitting and interpolation 71 Unit III - Curve fitting and interpolation 72
 Piecewise interpolants Why cubic splines?
• piecewise linear on sub-intervals • piecewise linear interpolation provides....
– solve for coefficients using Lagrange interpolating – a smooth y'(x) everywhere and...
polynomials – zero y''(x) inside the sub-intervals but...
• piecewise quadratic: conic splines – undefined (or infinite) y''(x) at the breakpoints xj
– exact representation of lines, circle, ellipses, parabolas and
hyperbolas
• piecewise cubic: cubic splines
• what’s a spline?
• cubic splines improve the behaviour of y''(x) ....
– carries some element of slope or curve shape constraint
– represent the curve of minimum strain energy (abstraction of
beam theory, e.g. f''''= zero at break points)
– drafting device from old times

Unit III - Curve fitting and interpolation 73 Unit III - Curve fitting and interpolation 74

Constructing cubic splines Constructing cubic splines

• suppose (blindly for now) that we have tabulated
• consider n tabulated support points values for the second derivative at each
yj= y(xj), j = 1,2,...,n breakpoint yj''(xj)
• first use linear interpolation on the jth sub-interval • add to the RHS of eqn [1] a cubic polynomial so
(xj, xj+1) – y = Ayj + Byj+1+ p(x)
• the local interpolant is • to ensure that this doesn’t alter continuity at the
y = F(x) = Ayj + Byj+1 ..... eqn 1 breakpoints we have to have
– p(xj) = p(xj+1) = 0
with coefficients
• to ensure that the second derivative is
continuous at the breakpoints we have to have
– y'' varying linearly (i.e. IS linear) from yj'' to yj+1''
derived from Lagrange interpolating polynomials • can we do all this?

Unit III - Curve fitting and interpolation 75 Unit III - Curve fitting and interpolation 76

Constructing cubic splines Cconstructing cubic splines

• YES ... and to make it work we have to have • calculate the derivatives and check that it works

• this eqn [2] comes from the derivatives

• the A, B, C, D factors depend on x:

– A, B are linearly dependent on x (from the previous
expressions) and ...
– C, D are cubicly dependent on x (through A, B)

Unit III - Curve fitting and interpolation 77 Unit III - Curve fitting and interpolation 78
 Cconstructing cubic splines Constructing cubic splines
• the second derivative is • we now get an equation to solve for these
unknown (assumed) yj'' values
• the trick is to demand continuity of the first
derivative at the breakpoints
• use eqn [2] to evaluate it at
• A = 1 and B = 0 when x=xj and opposite for xj+1 – x = xj the right endpoint of (xj-1,xj), and
so y''(xj) = yj'' and y''(xj+1) = yj+1'' – x = xj the left endpoint of (xj,xj+1) and
• this ensures that – equate these two for continuity
– the second derivative actually agrees with the tabulated • after some [messy] simplification you get the n-2
assumed values cubic spline equations (j= 2, ...,n-1) [eqn 3]
– the continuity condition for the second derivatives at the
breakpoints is satisfied
• so now what do we do about our assumed yj'' ??

Unit III - Curve fitting and interpolation 79 Unit III - Curve fitting and interpolation 80

Cubic spline equations! Cubic spline equations

• these are n-2 equations in the n unknown
breakpoint second derivatives yj'', j = 1, ...., n
• there is a two parameter family of solutions
unless we specify additional constraints
• for a unique solution we can specify boundary
conditions at the endpoints x1 and xn
– defines the behaviour of the interpolating function at the
global endpoints
• three choices are common for cubic spline
boundary conditions ....
•
is the size of the jth interval

Unit III - Curve fitting and interpolation 81 Unit III - Curve fitting and interpolation 82

Cubic splines: boundary conditions

Cubic splines: boundary conditions
• y1'' = yn'' = 0, i.e. a smooth flow away from the
two global endpoints (these called natural cubic
splines) OR
• set y1'' and/or yn'' to values calculated from eqn
[2] so that y1' and/or yn' have desired values
(specified endpoint slopes)
• force continuity of the third derivative at the first
and last breakpoints, e.g. y2''' from first or second
interval is to be the identical value, similarly yn-1'''
– this is called the not-a-knot condition
– it effectively makes the first two interpolating cubics
identical

Unit III - Curve fitting and interpolation 83 Unit III - Curve fitting and interpolation 84
 Cubic splines: computational issues Cubic splines: example
• have to solve a linear system for the n unknowns • find the equations for the four natural cubic
yj'' consisting of splines which interpolate the data points:
– n-2 spine equations (-2,4), (-1,-1), (0,2), (1,1), (2,8)
– two boundary conditions
• all hj = xj+1 - xj = 1 for this data [nice]
• this is a (sparse) tri-diagonal linear system • the spline equations
• only the main diagonal and its neighbours have
non-zero entries
• this reflects the fact that each sub-interval in the
interpolation problem is coupled only to its two are (j = 2,3,4):
nearest neighbours
• special efficient algorithms are available for
solving tri-diagonal systems
Unit III - Curve fitting and interpolation 85 Unit III - Curve fitting and interpolation 86

Cubic splines: example (cont) Cubic splines: example (cont)

• written out this system is • step #1 is complete now we have the second
derivatives at the breakpoints
• now use these values to obtain the cubic spline
equation coefficients A,B,C,D [formulas on slides
75&77], with j=1,...,4
• natural splines have so the system • first j = 1:
simplifies to:
solution

Unit III - Curve fitting and interpolation 87 Unit III - Curve fitting and interpolation 88

Cubic splines: example (cont) Cubic splines: example (cont)

• the spline equation for the first interval 2 < x < -1
is then given by
....S2

....S1 • for the third interval 0 < x < 1 we have j = 3:

• j = 2 gives the spline equation for the second

interval -1 < x < 0

and the third spline equation ....

Unit III - Curve fitting and interpolation 89 Unit III - Curve fitting and interpolation 90
 Cubic splines: example (cont) Cubic splines: example (cont)

..... S3 .... S4

• finally, for the 4th interval 1 < x < 2 we have j = 4: • these equations (S1-S4) are the four natural
cubic spline equations for y over the interval
-2 < x < 2
• to evaluate y(x) you decide which interval x lies
in and use the applicable equation

and the fourth spline equation ....

Unit III - Curve fitting and interpolation 91 Unit III - Curve fitting and interpolation 92

Font wars Interpolation: Matlab implementation

• proportional fonts are rendered at any size using • the basic Matlab functions you need to know are
curves determined by interpolation through – interp1: one-dimensional interpolation with piecewise
specified points polynomials
• the curves are rasterized for display following – spline: one-dimensional interpolation with cubic splines
using not-a-knot or fixed-slope end conditions
‘hints’ to keep the fonts realistic in small sizes
• there are others too
• Type I fonts, Adobe 1985
– multi-dimensional interpolations
– used in postscript and acrobat
– a sophisticated spline toolbox
– based on cubic splines as developed above
• TrueType fonts, Apple 1988 + now MS
– conic splines
– a subset of cubic splines with simpler interpolation
– faster rendering but require more ‘hints’

Unit III - Curve fitting and interpolation 93 Unit III - Curve fitting and interpolation 94

Interpolation: Matlab implementation

• ytest = interp1(x,y,xtest,method)
– y = tabulated function values
– xtest = test x value to evaluate the interpolant
– x = tabulated x values for interpolation (1:n default)
– method = ‘nearest’ [nearest neighbour constant],
‘linear’, ‘cubic’ [cubic polys s.t. interpolant & first
derivative are continuous at the breakpoints], or ‘spline’
[cubic splines, same as spline function]
– ytest = calculated interpolated value corresponding to
xtest
• default for splines is not-a-knot end conditions
• if size(y) = size(x)+2, first and last elements are
used for y'(x1) and y'(xn) end slopes
Unit III - Curve fitting and interpolation 95