Numerical Integration

Numerical integration is used for definite integrals. Most

techniques apply to low dimensional integration - we look at these
today. Integration in many-dimensional space typically relies on
Monte Carlo techniques (next semester).

We want to find a numerical approximation to

f(xi+1)
b

a f (x) dx f(xi)

h
a xi xi+1 b

Newton-Cotes methods - use equidistant points:

b
a
x i = a + ih i = 0,...,n h=
n
Winter Semester 2006/7 Computational Physics I Lecture 10 1
 Lagrange Polynomials

For n+1 points (xi,yi), with

i = 0,1,
, n xi
x j
i

there is a unique interpolating polynomial of degree n with

p(xi ) = yi i = 0,1,
, n
Can construct this polynomial using the Lagrange polynomials,
defined as:
(x
x0 )
(x
xi
1 )(x
xi +1 )
(x
xn )
Li (x) =
(xi
x0 )
(xi
xi
1 )(xi
xi +1 )
(xi
xn )
Degree n (denominator is constant), and
Li (xk ) =
i, k

Winter Semester 2006/7 Computational Physics I Lecture 10 2

 Lagrange Polynomials

The Lagrange Polynomials can be used to form the interpolating

polynomial:
n n n
x
xk
p(x) =  yi Li (x) =  yi 
i=0 i=0 k = 0, k i xi
x k

In our application, y i = f (x i )

Integration of the polynomial yields:

n n x
xk n n s
k
b
a p(x) dx =  f (x i ) ab  dx =  f (x i ) 0 
n
hds
i= 0 k = 0,k  i x i
x k i= 0 k = 0,k  i i
k

where x = a + sh
n s
k
Note that
n
0  hds =  i h
k = 0,k  i i
k
So
i is just a number depending on i,n
Winter Semester 2006/7 Computational Physics I Lecture 10 3
 Newton-Cotes Formulas
n
So,
b
a p(x) dx = h  f (x i )
i
i= 0

Look at some specific examples:

x
x1 x
x0
n =1 p(x) = f (x 0 ) + f (x1 )
x 0
x1 x1
x 0
1 s
k s
1 12
1
 0 = 0 
1
ds = 0
1
ds =
+ 1 =
k = 0,k  i i
k 0
1 2 2
1 s
k s
0 12
1
 1 = 0 
1
ds = 0
1
ds = =
k = 0,k  i i
k 1
0 2 2
n 1 1 h
b
a p(x) dx = h  f (x i )
i = hf (x 0 ) + hf (x1 ) = ( f (x 0 ) + f (x1 ))
i= 0 2 2 2
Trapezoidal rule
Winter Semester 2006/7 Computational Physics I Lecture 10 4
 Trapezoidal Rule

Graphically: f(x1)
A=f(x0)h+1/2[f(x1)-f(x0)]h
f(x0)
h
x0 x1
Of course, we can apply this to many subdivisions (composite
trapezoidal rule) :
m
b
a f (x) dx =  zz f (z) dz where z0 = a, zm +1 = b
i
i +1

i= 0

 f (x 0 ) f (x1 ) f (x1 ) f (x 2 ) f (b
h) f (b)

b
a f (x) dx  h + + + +
+ +
 2 2 2 2 2 2 
 f (x 0 ) f (b)
a f (x) dx  h  + f (x1 ) +
+ f (b
h) +
b
2 2 
Winter Semester 2006/7 Computational Physics I Lecture 10 5
 Error Order
To find the error order of a particular approximation, we consider a
series approximation. For the numerical simulation of derivatives,
we compared to a Taylor series. Here we compare to the Euler-
Maclaurin summation formula:

[ ]
n
1 1  B
 F(i) =
n
0 F(s) ds
[ F(0) + F(n)] +  2k F (2k
1) (n)
F (2k
1) (0)
i=1 2 k =1 (2k)!

where
F (k ) is the k - th derivative of F and the B2k are the Bernoulli numbers
n Bn
0 1
1 /2 = 
.5 Bn=0 for n
3, odd
2 1/6
0.1667
4 /30

.0333
6 1/42
0.02381
8 /30

.0333
10 5/66
0.07576
12 91/2730

.2531
14 7/6
1.1667

Winter Semester 2006/7 Computational Physics I Lecture 10 6

 Error Order

Make the transformation: x = sh + a

[ ]
n
1 1 b 1  B
 f (x i ) = a f (x) dx
[ f (a) + f (b)] +  2k f
(2k
1)
(b)
f ( 2k
1) (a) h 2k

i=1 h 2 k =1 (2k)!

or
n
1 2
h h
a f (x) dx = h  f (x i ) + [ f (a) + f (b)]
[ f  (b)
f  (a)] + O(h )
b 4
i=1 2 12
Trapezoidal Rule

We see that the error from the trapezoidal rule is of order h2.

The precision can be improved by setting h smaller and smaller

but we will eventually hit rounding errors and computing time
limitations. Look for methods with higher order of the error.

Winter Semester 2006/7 Computational Physics I Lecture 10 7

 Simpson’s Rule
n f (x 0 ) + 4 f (x1 ) + f (x 2 )
n=2 a p(x) dx = h  f (x i )
i = 2h
b
i= 0 6
f(x2)
f(x1) Error order h4
(interpolating
f(x0) polynomial of
2h
2nd order)
x0 x1 x2
Composite Simpson’s rule:
 f (a) + 4 f (a + h) + f (a + 2h) +  Note: range divided
b 2h  f (a + 2h) + 4 f (a + 3h) + f (a + 4h) +  into an even
 f (x) dx   
a 6 
+  multiple of h
 f (b
2h) + 4 f (b
h) + f (b) 

h
= [ f (a) + 4 f (a + h) + 2 f (a + 2h) + 4 f (a + 3h) +
+ 4 f (b
h) + f (b)]
3
Winter Semester 2006/7 Computational Physics I Lecture 10 8
 Midpoint Rule

It is sometimes preferable to sample the function at the midpoint

of the interval rather than at the ends (e.g., if the function has a
singularity at the endpoint). Simplest example:
f((a+b)/2)
 a + b
b
a f (x) dx  hf h = b
a
 2 
h

The composite midpoint rule gives: a b

b
a f (x) dx  2h ( f (a + h) + f (a + 3h) +
+ f (b
h))

The error order is h2

Winter Semester 2006/7 Computational Physics I Lecture 10 9

 Extrapolation Method

Let Rij be our estimate of the integral when 2i-1 intervals are used
and j is the order of the interpolating polynomial (previously n).
E.g., j=1 is Trapezoidal Rule, j=2 is Simpson’s Rule.
Look at the error formula:
b
a f (x) dx = TS (h) +  2 h 2 +  4 h 4 +

h
where TS (h) is e.g. the Trapezoidal rule. Now take h 
2
a f (x) dx = TS (h /2) +  2 h /4 +  4 h /16 +

b 2 4

so
4TS (h /2)
TS (h)
b
a f (x) dx
=
 4 h 4 /4 +

3
i.e., can get a higher order error by combining results from
different step sizes (recall extrapolation method for derivatives).
Winter Semester 2006/7 Computational Physics I Lecture 10 10
 Extrapolation Method

There is a general formula for calculating the higher order terms -

Richardson Formula:

Ri+1, j
Ri, j
Ri+1, j +1 = Ri+1, j + See Scherer lecture notes
4
1
j
for derivation

So we can gain higher order of error at the same time as making

the step size smaller (technique called Romberg integration).

R1,1 So, e.g., can get up h6

precision by extrapolating
R2,1 R2,2
the trapezoidal rule
R3,1 R3,2 R3,3 calculations with spacing

 h,h/2,h/4
Winter Semester 2006/7 Computational Physics I Lecture 10 11
 Example
Let’s investigate a concrete example:

K = 0 / 2 1
k 2 sin 2  d

We know the exact value for some k, e.g., k=1
K=1.0

Romberg Integration (3 step)

i
1 0.785398126
2 0.94805944 1.00227988
3 0.987115741 1.00013447 0.999991417
j 1 2 3

Trapezoidal rule used 3 times

Winter Semester 2006/7 Computational Physics I Lecture 10 12

 Example

Next example:
K = 02 4
x 2 dx = 

Convergence much
slower because of infinite
slope at x=2

d 4
x 2 1
2x
x
= = x
2

dx 2 4
x 2
4
x 2

Winter Semester 2006/7 Computational Physics I Lecture 10 13

 Magnetic Field
x
Current I flowing in wire.
Biot-Savart Law:



μ 0 I dz
 r
r
dB =
4
r 3




dz  r = rdz sin
yˆ
z
-L dz +L

L μ I r sin  L μ I x
B = Bˆy, B =  0
dz = 
0
dz

L 4
L 4 (x 2 + z 2 ) 2
3 3
r

μ0I
Consider L
, x small: Ampere’s Law gives B = where r is
2
r
the distance from the wire. Try our numerical calculations.
Winter Semester 2006/7 Computational Physics I Lecture 10 14
 Magnetic Field

Note that Ampere’s law applied to an infinite length wire. To

reproduce the result numerically, need to have L>>r.

 x   1  x   1 
r = x = 1,  3 2 = 3 2 , 3 2 = 3 2
 x + z min  1 + L  x + z max  1 + 0
2 2 2

 x 
3 2
 x + z 2 min  1 
= 3 2
 x   1 + L2
3 2
 x + z 2 max

23
13
for this ratio  , L > 
1  

For L = 1000,  = 10
9

Winter Semester 2006/7 Computational Physics I Lecture 10 15

 Magnetic Field
Step too big
Take x=1, L=1000

Rounding errors
Winter Semester 2006/7 Computational Physics I Lecture 10 16
 Magnetic Field
Try double precision: L=10000

Winter Semester 2006/7 Computational Physics I Lecture 10 17

 Gaussian Quadrature

We now move away from the requirement of equidistant points

for the integration. General technique called Gaussian
quadrature which yields polynomial accuracy of order (2n-1)
using nth order polynomials.

Consider the scalar product of two functions in the interval [-1,1]

fg = 
1
1
f (x)g(x) dx
Now consider an orthogonal set of polynomials on this interval,
such that
Pi P j =
ij
E.g., Legendre polynomials (1,x,x2-1/3,…)

Winter Semester 2006/7 Computational Physics I Lecture 10 18

 Gaussian Quadrature
Assume we have a polynomial p(x) of order 2n-1. We can
interpolate this polynomial at n points xi using our usual Lagrange
method of order n-1: n
p˜ (x) =
L j (x) p(x j )
j =1

We can rewrite p(x) as

p(x) = p˜ (x) + (x
x1 )(x
x 2 )
(x
x n )q(x)
where q(x) is a polynomial of degree n -1
If we pick the x i as the roots of an n th order polynomial
from an orthogonal set (e.g., Legendre Polynomials), then
Pn (x) = (x
x1 )(x
x 2 )
(x
x n )
n
1
1

1 Pn (x)q(x) =0 since q(x) =  a j P j (x)
j=0

Winter Semester 2006/7 Computational Physics I Lecture 10 19

 Gaussian Quadrature
n n
So,
1

1 p(x) dx = 
1
1
p˜ (x) dx = 
1
1
 L j (x) p(x j ) dx =  w j p(x j )
j =1 j =1

where the wj are determined by the choice of xj which in turn

come from the choice of orthogonal set of polynomials.

w j = 
1
1
L j dx
Note what we have achieved: 2n-1 accuracy using a polynomial of
degree n by picking sampling points in special way.
Procedure (Legendre polynomials):
1. Choose which order you want to use (n)
2. Find the n roots of Pn (look them up in a table)
3. Find the corresponding Lagrange Polynomials
4. Calculate the weight factors
6. Evaluate the integral
Winter Semester 2006/7 Computational Physics I Lecture 10 20
 Gaussian Quadrature

E.g., use Legendre polynomials, n=2

1 1
P2 (x) = x
with roots x1 =
2
3 x2 =
13
3
The Lagrange polynomials are
(x
x0 )
(x
xi
1 )(x
xi +1 )
(x
xn )
Li (x) =
(xi
x0 )
(xi
xi
1 )(xi
xi +1 )
(xi
xn )
x+ 1 x+ 1
(x
x 2 ) 3 3
L1 = = = w1 = 
1
1
L1 dx = 1
(x1
x 2 ) (
1

1
3 3
2 1
)3
x
13
L2 = w 2 = 
1
1
L2 dx = 1

2 1 3
Note that these can be used for any integrand you want to
evaluate
Winter Semester 2006/7 Computational Physics I Lecture 10 21
 Gaussian Quadrature
So, this gives the 2-point rule: 1

1 ( ) ( 1 3)
f (x) dx  f
1 3 + f

For different integration limits, make a change of variables:

a+b b
a
x= + u
2 2
a + b b
a b
a
a f (x) dx = 
1 f  +
b 1
u du
2 2 2
for our example
b
a a + b b
a 1 a +b b
a 1
b
a f (x) dx   f
+ f +
2   2 2 3  2 2 3

 /2  /2    /2  /2 1   /2  /2 1
e.g., K = 1
sin 2  d  cos
+ cos + 3 
0
2  2 2 3  2 2

= [0.9454092 + 0.3258856] = 0.9984726
4
Winter Semester 2006/7 Computational Physics I Lecture 10 22
 Gaussian Quadrature
Of course, can apply composite Gaussian Quadrature, or use a
higher order Legendre polynomial.
n xi wi
1 0 2
2 ±
1/3 1
0 8/9
3
±
3/5 5/9
±0.339981 0.652145
4
±0.861136 0.347855
0 0.568889
5 ±0.538569 0.478629
±0.906180 0.236927

Winter Semester 2006/7 Computational Physics I Lecture 10 23

 Gaussian Quadrature
One can also choose different sets of orthogonal polynomials by
breaking up the integral as follows
ab w(x) f (x) dx

Interval w(x) Orthogonal Polynomials

[-1,1] 1 Legendre

(-1,1) (1-x)
(1-x)
,>-1 Jacobi

(-1,1) 1/(1-x2) Chebyshev (first kind)

[-1,1] (1-x2) Chebyshev (second kind)

[0,) e-x Laguerre

a w(x)PN (x)PM (x) dx = 0 M
N

b
Orthogonality:
The weights are given by w i =
a w(x)Li (x) dx
b

Winter Semester 2006/7 Computational Physics I Lecture 10 24

 Field due to Current Loop

Example: the magnetic field due to a current loop:

z
. x,y,z


L



r
r )



Magnetic field can be calculated analytically for points along the

axis, but not for other points. Magnetic field will have components
in all directions. Need to calculate them one at a time.

Winter Semester 2006/7 Computational Physics I Lecture 10 25

 Field due to a Current Loop

Calculation of the magnetic field due to a current loop at a point P

off the axis of the loop. Following integrals appear:
x 5 Note singularity
I= 1

1 3
dx take a =
(a
x) 2 1
x 2 4 at endpoints !
1
Note that 2 already appears in integral, so we use this as
1
x
the weight function and use the roots of the Chebyshev
polynomials of the first kind. For n=4, these are
x1 = cos 8 , x 2 = cos 3 8 , x 3 =
cos 3 8 , x 4 =
cos 8
giving
w1 = w 2 = w 3 = w 4 =
4

so that I
w1 f (x1 ) + w 2 f (x 2 ) + w 3 f (x 3 ) + w 4 f (x 4 ) = 5.02

x
where f (x) = 3 Correct answer
5.33
Winter Semester 2006/7 (a
x) 2 Computational Physics I Lecture 10 26
 Exercizes

1. Compare the Trapezoidal rule, Simpson’s rule, the Romberg

extrapolation method, and Gaussian quadrature with
Legendre Polynomials for the following integral (try different
values of
m) for different step sizes:

d
8 0 m
cos
cos m

Winter Semester 2006/7 Computational Physics I Lecture 10 27