Variational Principles

Mark Jackson, Jan 10

0. Preliminaries
Problem 1
Find the curve of shortest length
going from to .
Problem 2
Find curve such that
, of length which
gives the maximum area.
We will develop a systematic way of obtaining solutions to such problems by showing that any
solution must satisfy a specific differential equation.
Analogous to the relationship in calculus between stationary points of

, given by
, and the minimum and maximum values of .
In problem 2 the area is

and the length is

is a functional, i.e. a function on a space of functions.
This class of problems is called the calculus of variations.
0.2 Direct method
Problem 3
Show that there exists such that

, for any
.
Find such that

. By the
intermediate value theorem, there exists a solution .
Alternative method;
Define

Claim; such that

. Given this claim, we know that

.
Proof of claim.
Step 1;

. On the
other hand, let

; then

But

So if . This shows the minimum value is attained inside
if anywhere.
Step 2; By Analysis I, with

.
This is called the direct method for variational problems.
1. Functions on

1.1 Functions

Let

Then

is linear if for ,

. Also

where

.
Definition 1.1.1. A function

is differentiable at if it can be well approximated by a

linear function close to in the sense that

where

.
i.e. , such that

Relation with difference quotients (
,

In Definition 1.1.1, put

, then identical argument to the case shows that if is

differentiable at then

exists and is finite, and is equal to

. This is because if such that

when , this implies

because linear.
Proposition 1.1.2. (i) If is differentiable at , then the partial derivatives

exist and the linear map (in definition 1.1.1) is given by

(ii) If all partial derivatives exist

exist and are continuous on

, then is differentiable
at each

and is given by

We will define

to be the set of real valued continuous functions on

all of whose partial

derivatives are continuous on

is the set of real valued continuous functions on

all of whose partial derivatives

up to order are continuous on

.
The definition of differentiability can now be written as

In Analysis II, youll see a proof that

implies that is differentiable at all

.
Lemma 1.1.3. If

, then . Also, if

, then
. This is a 1
st
order necessary condition for max or min.
Remark. This is also true if it holds only for in some ball around , i.e.
.
Terminology. If then is a stationary or critical point. If is a maximum or minimum
then it is an extremum (or extreme value).
1.2 Second-order conditions for extreme values
Definition. A symmetric matrix

(positive) if

for all

non-
zero. Similarly,

(non-negative) if

for all

non-zero.
Exercise. Show that if a symmetric matrix is , then all its eigenvalues are , and the same
with signs.
Theorem 1.2.1. If

and , then
(i) is a local minimum

, and a local maximum

(ii)

is a strict local minimum, and

is a strict local
maximum. Strict means that, around the point which is a minimum, the function is strictly greater
than at (or smaller, if is a maximum).
Imagine

which has a strict local minimum at

. Then

and

.
Assume

is the only stationary point, i.e.

. Is

a global minimum?
Yes - by Rolles theorem, because otherwise,

with

, and then Rolles

theorem would imply

with

, contradicting the assumption that there is only one

stationary point.
In

, , there are functions with only one stationary point which is a strict local minimum
but not a global minimum.
Exercise. Find such a function for .
1.3 Convexity
Definition. A set

is convex if, given

, we have

.
A function

is convex if

i.e. the line between two points on the curve lies above the curve.
Remark. Define the epigraph

Then

convex convex.
Proof (). Let

. Let

and

be in

, so

and

. For

Hence,

as was to be proved.
Proof (). Exercise.
Remark. Since the definition of convexity involves only line segment joining two points, to check
a function is convex you can check the functions are convex .
Proposition 1.3.1. If

then the following are equivalent;

(i) is convex
(ii)
(iii)
Proof that (i) (ii). Let . Then
and

By the chain rule,

, which is (ii).
Proof that (ii) (i). Let and .
Then


Choosing gives the definition of convexity.
Proof that (i) (iii). We have and
. Adding, as
required.
Proof that (iii) (i). Exercise.

To visualise the second condition, work in

.
is graph of
Normal vector

Tangent plane at is

To have a picture for (iii) consider ;

This says that

is monotone non-decreasing. Thus

Proposition 1.3.1 (in words). is convex lies above all its tangent planes is
monotone.
1.3.4 Strict convexity
Definition.

is strictly convex if

whenever and .
Proposition 1.3.2. If

, the
following are equivalent;
(i) strictly convex
(ii) for
(iii) for
Corollary to 1.3.1. If

is convex with stationary point , then is a global minimiser

for , i.e. we have .
Proof. and use (ii).
Corollary to 1.3.2. If

is strictly convex and is a stationary point, then it is a strict

global minimiser for , i.e. .
Recall that to solve you can hope to achieve this by minimising .
Corollary to 1.3.2. If

is strictly convex, then the equation has at most one

solution.
Proof. Assume and , so by
1.3.2(ii).
Remark. There is a corresponding notion of concavity, with the
inequalities reversed, i.e.

etc. etc. E.g.

on .
Lemma 1.3.3. If

then
(i)

(ii)

Exercise. Think of an example to show that the converse of (ii) is false.
Proof of (i) ().

Note this is using the chain rule in the form

Warning when we say a symmetric matrix

is , we mean

Example. Show that

is a concave function on the set of all

with

and

(a
probability distribution on ).
Note that if are both probability distributions on this set of points, then so is
, since

.
Calculate the matrix of second partial derivatives;

since

This is a negative definite matrix. QED.
1.4 Constraints and Lagrange multipliers
Example. Maximise

subject to the constraint set

Answer;

, attained at .

But notice is perpendicular to (unit circle) at these points. To see why, solve the problem
by parametrising as . Then

At maximum values we must have . Apply the chain rule;

Since

is the tangent vector to , this means when is normal to .

General situation; if we have a constraint set

, then assume admits

local parametrisation

. If we find a point

where a

function has
a maximum/minimum on , then

Then

has an unconstrained maximum at

, so

is perpendicular to the tangent vectors

Alternatively,

, i.e. such that

the Lagrange multiplier. Use function .
Example. Find the rectangle in the plane inscribed in the unit circle
which has the largest possible area.
Area .

Augmented function

where is the Lagrange multiplier.

Stationary points of ;

Assume neither nor are , then

i.e. stationary point is a square.
If say , we just have a line inside the circle, so and this is a minimum value of
amongst inscribed rectangles.
For

, recall that

. We will assume , and that we

can write in parametric form, so

Example.

To search for stationary points of a function

constrained to , we look at

written as

.
First-order necessay condition for stationary point;

where the

are tangent vectors to .

If is perpendicular to all tangent vectors, it is to , i.e. such that at a
stationary point.
For ,

Assume

is a stationary point at which

. Then

But

using the summation convention. Thus

Theorem 1.4.1. Let be

functioms and . Let ; then if

has maximum at

, then

for some , and

If

has minimum, then the second inequality has a . Here we use the notion of a symmetric
matrix being defined above.
Moral do usual calculus on .
To solve problems with two or more constraints,

, do the same thing with

Examples. (i) Find the probability distribution

on which maximises

Constraint is

, so

, so

(ii) For

, find on the unit circle and on the

line to minimise/maximise

. satisfies

, so

Then

.
Using

, we get

As always,

give the constraints

.
In vector form we have

Notice

cannot be zero, as this would imply , contradicting constraints since

and , .
From the first pair

But . Since

, we get . Clearly gives

the minimum value.
Example of 2
nd
derivative test. Let ,

and
. As we proved, we need to look at

We want to check this is a negative definite matrix at the solution

so the matrix of 2
nd
partial derivatives is




Find eigenvalues of , i.e.

which satisfies the necessary 2
nd
order condition for being a maximum.
1.5 The Legendre transform
For , given a

function, we define its Legendre transform

by

This is only defined for such that this is finite.
Examples. (i) Let

, , then

is defined for all and

attained at where

At that point

Now define

If is convex, then

.
(ii)

, , then

where this is finite. Here the domain of

is because the is nowhere defined.
(iii) . Then , so the domain of is with .
(iv) . Then when defined, so the domain of is
with .
Proposition 1.5.1.

is convex (on its domain).

Proof.

So

So if

lie in the domain of , then so does

, and

Theorem 1.5.2. If is convex, then

.
Proof. Prove for

with

. Then
is attained for all at unique
such that

. (Recall the corollary to 1.3.2.)

Given , there is a unique point where the
tangent line to has slope .

Consider, for given , the tangent line to
with slope . It has equation
or
Recall that a convex function lies above its tangent lines
(or planes). Call . For given , the line lies
below the graph of , and the supremum is attained when

, and supremum is , i.e.

i.e.

. QED.
Definition. An affine function is one of the form

Corollary. If convex, then it is a supremum of a family of affine
functions.
Example. Consider

. Then

Domain of is ; supremum is attained when

Then

(Youngs inequality). In fact we always have .
For

we define the Legendre transform

by

Exercise. Calculate the Legendre transform of

where

is a positive symmetric matrix.

Applications
(i) Classical mechanics. The Lagrangian

Hamiltonian

To calculate; supremum attained when

Exercise. Newton equations are equivalent to

(ii) Economics. A company buys imports

and produces a product which is sold with

revenue . Let

be the price of a unit of the th good. Then the total cost is and the profit
is . To maximise profit, they try to choose to get

(iii) Thermodynamics. An engine has internal energy where is the volume and
is the entropy. In idealised reversible change, the heat flow . Also
where is the temperature and the pressure. Then

(Maxwell relations).
Suppose the system is immersed in a constant temperature reservoir. In this case the system is
described by a function . This is called the Helmholtz free energy, and is defined by

The supremum is attained at such that

This defines , and we can substitute in to get




In Helmholtz description

(Maxwell relation).
Remark. The entropy is determined implicitly by

This determines uniquely as long as

Constant volume heat capacity

heat input needed to increase temperature at fixed

volume

So is a convex function of you need heat to raise the temperature.
2. Calculus of variations
2.1 Differentiating functionals
A functional is a function any space of functions or .
Examples. (i) Let and consider

, the Dirac functional.

(ii) Let

functions with . Then

Recall. If

, then the directional derivative at in the direction of is

We will apply this to the functionals;

To define something analogous to , we use

or for complex functions we will often use

Now

. In this case, we call

the functional derivative. More

generally, for any functional we have

when such a function exists. Lets try to apply this to

In Methods we will write

Example.

because

Therefore

Example 3. Let . Then

Now . When we calculate a derivative, we consider smooth with ,
and look at

since vanishes at the end points of the integral; . So

using the notation e.g.

Problem. Let . Minimise

amongst all smooth -periodic functions .
Solution. Assume such a minimising function does exist. Obtain a condition which determines it.
Well then show that there exists a function satisfying this condition, then finally well show that this
function minimises .
Assume

is a solution, then

for any , where is the set of smooth -

periodic functions. In particular, if

for any , then .

, i.e.

.
Exercise. Show that

So if

does minimise , then

Lemma 2.1.1. If (i)

for all smooth functions , with if is outside some sub-
interval ;
(ii)

then in .
Proof. Assume, to the contrary, that

with

.
Without loss of generality, take ; since is continuous,
such that on

.
Consider the function

Now

, because

.
Consider

Then

which is a contradiction.
Exercise. Find

(for the previous example) such that

.
Jargon.
We define closure of set where .
We say is properly supported in or if .
Problem 1. Minimise

over

set of smooth -periodic functions, and .

Solution. 1
st
stage (last time); assume there is such a minimising function

, then

for all and

.
Therefore

for all and

By general formula ( ),

Therefore

By Lemma 2.1.1,

, i.e. if

exists it solves this equation. This is called the

Euler-Lagrange equation.
When we get, with

periodic,

Remark. To prove there is only one -periodic solution of

, imagine is
another, where

with

. Subtract

.
Multiply by and integrate;

by parts. Middle term is by periodicity and then we get , i.e.

.
Final stage of minimisation problem; consider

Integrating the middle term by parts we get

for any

, . So

really minimises over smooth -periodic

functions.
Problem 2. Find the curve of shortest length, joining the two points in the plane,
and .
Mathematically

We need to minimise amongst such curves.
Assume

is a minimising curve, then exactly as before, at

. Now

The Euler-Lagrange equation is therefore

constantm so

. Fix so that

and

. This is our
candidate minimising curve.
Claim that

is strictly convex

By lemma 1.3.2,

. Now

which if and . So if is another -curve joining to it has
greater length than the straight line.
2.3 Multi-dimensional integrals
Moral. Integration by parts in one dimension converts to using Green identities in more than
one dimension.
Consider

where some region in

. Then

To achieve this we recall

Assume vanishes on , then

i.e.

but so we get Poissons equation;

. This gives the minimum value of

energy, . This gives gravitational potential of mass distribution , or electrostatic
potential of charge.
Problem. Tourist El Cheapo wants to drive 3000 miles from New York to San Francisco in car,
taking 100 days. Car uses gallons per mile, with . Fuel costs cents
per gallon. How should he choose his speed to spend least money?
For we have with and .
Assume he pays for his fuel as he uses it (continuous ), and that .
Cost to go a distance at time is . So total cost is

A minimising will be a stationary point, so .
Note that the integrand

so the functional derivative is

already, and then choose such that .
Notice that if is independent of then is constant, by the Euler-Lagrange equation.
Concepts in theoretical physics; an action in

would be of the form

with

and

. Writing so that

, the Euler-
Lagrange equation becomes

using the summation convention.
Derivation. Let be a smooth variation function, with near the boundary of . Then

since near the boundary of . Alternative form;

Example. Let an action be

so that and

. Then

Euler-Lagrange equation;

the wave equation.
2.4 Lagrange multipliers
DIDO problem; find a curve such that the area under the curve and above the -axis is
maximised, for a fixed length. I.e. maximise

subject to

Consider augmented function where is the Lagrange multiplier. Then

Substitute so that

, i.e.

which is a
circle. Therefore the solution is is arc of circle passing through of length .
Remark. The formulation of the problem disallows curves that go back on themselves in the -
direction, because this is not a graph ! To get over this, we can instead consider
parametrised curves

. Then integral functional is

even with

.
Euler-Lagrange equations;

for .
Exercise. Find closed curve in the plane such that to maximise

the area inside the curve, with

Finite number of constraints

with . Consider

as
for finite-dimensional problems.
Infinite number of constraints need Lagrange multiplier function. E.g. in fluid mechanics. A
velocity vector field on

subject to an infinite number of constraints at each point;

.
Minimise

subject to (incompressible). Then

Exercise. Work out .
Remark. For

the notation

and

Constraint for each means there is a Lagrange multiplier function .

Green identities

by Green, assuming for large

so 2 lines above

1
st
term =

using the second Green identity;

Therefore

So the Euler-Lagrange equation is

. Now

Final construction; stationary for with if

Static Navier-Stokes equation, then approximate by putting all

terms to zero, you get

this equation where is the pressure.
3. Scientific examples
Fermats principle when light
reflects in a mirror, the angle of reflection

is equal to the angle of incidence

.
This can be explained by postulating that
light rays take the shortest time possible
to get from A to B.
Time as path through

where speed of light.

This implies

and

Typical problem; imagine the speed of light
is . Then consider a light path
.
Time is

This is the type of integral

we know about. A minimising path would solve

2. Mechanics
A particle of mass has position , moving in a potential . The force so

This equation is the Euler-Lagrange equation for the action

where

Most fundamental equations of classical physics come in this form, eg. Maxwells.
3. Geodesics
A geodesic is a curve of shortest length, or more generally a stationary point for the length
.
(i) In the plane, curves from to ,

The only solution is the straight line. We can also write the curve and then

Since

by the change of variables and

Simplest to choose so that

constant, i.e. arc length.

Using these parametrisations, geodesics are curves which make stationary

For

, the Euler-Lagrange equation is (put , in the second

example.) This is exactly what you get by putting constant in .

Recall. We defined a geodesic curve to be a curve which makes stationary either the length

or the integral

The length is independent of parametrisation;

The second definition gives curves parametrised so that is constant. This is usually easier to
work with.
Problem. Find geodesic curves on a cylinder

Solution 1. Use cylindrical coordinates; , , . Then

For a curve on , is fixed, and , . Therefore

A geodesic curve on is a curve which makes stationary

The Euler-Lagrange equations are

Conclusion; geodesic curves are helicoidal curves. (See example sheet 2,
question 9 geodesics are great circles.)
Solution 2. Introduce

as a constraint, and use the

Lagrange multiplier function , and

The Euler-Lagrange equations are

To find and we must use constraints;

Find ; But by Euler-Lagrange,

Write

, then we have

and

. But

constant constant.
Solutions are therefore , . Again, we get helicoidal
curves.
Problem. (i) Consider

(no -dependence!) and show that a curve which makes stationary satisfies

(ii) Find the curve such that a bead moving without friction on the curve takes the shortest
possible time to go from to .
Solution. (i) Since makes stationary,

The last two terms come about by the chain rule, since . This is equal to

(ii) The bead moves under gravity, so the speed

satisfies

Euler-Lagrange;

This is very hard to solve! Instead we use (i), to say

Choose

; then

The curve is a cycloid.
4. Conservation laws (Noether theorem)

Let be a smooth solution of

Theorem 4.1. (i) If

(no dependence) then

constant.
(ii) If

(no dependence) then

constant.
Proof. (i)

constant by Euler-Lagrange equation.

(ii) See last lecture.
Special case from mechanics;

Newtons law is Euler-Lagrange equation for

Conservation laws; (i) if does not depend on then

constant. We sometimes call a cyclic coordinate.
(ii)

since does not depend explicitly on .
5. The second variation
When is a solution of the Euler-Lagrange equation

a (local) minimiser of

Possible methods; (i) use convexity. See 2.2 for example (shortest curve between two points).
(ii) look for condition analogous to the 2
nd
order condition in calculus.
If

, then Taylors theorem gives

This tells us that (i)

for all sufficiently small (strict local minimum).
(Positive symmetric means

(ii) a local minimum and

is non-negative (i.e.

).
To extend this to a functional , let be a smooth function (or with .
( is analogous to .) Assume is smooth, then;

For all , such that

then

In this case we get

Terminology.

Definition. is a weak local minimum for if for

sufficiently
small. Call it strict if inequality is strict for such .
Theorem 5.1. (i) If

, and

for some , then is a strict weak local minimum for .
(ii) If is a weak local minimum for , then

and

.
Remark. A space of functions is an infinite-dimensional vector space, so we have to be careful
which norm we use on a space of functions!
Example.

Euler-Lagrange equation;

There is a solution

is this a (weak) local minimum?

Exercise. The second variation is

Now

satisfies the criterion with . Therefore for

is a strict weak local minimum. Then

consider

Recall . Try , then

, so

since

. Conclusion;

is not a weak local minimum for

.
Remark. In general the second variation can be written

Associated to this is the Sturm-Liouville operator

This has an infinite sequence of eigenvalues,

where

is an eigenfunction. So if

then