14

Optimization
 “Take the derivative and set it
equal to zero!”
-Fermat’s
Theorem

Pierre de Fermat
1607- 1665
Pierre de Fermat was actually a lawyer before he was a
mathematician….perhaps he was trying to figure out
how to maximize his legal fees!

But, what’s a derivative and why

set it equal to zero?
 The derivative is the
Imagine calculating result of the distance
the slope between two between those two
distinct points on a points approaching
function zero
f  x  x   f  x   f  x  x   f  x  
slope  slope  lim    f '  x
x x  0
 x 
f  x
f  x
Now, let those two points
f  x  x  get closer and closer to
each other

f '  x

f  x
f  x

x x
x x  x x
Let’s try one numerically… 2

f  x  x   f  x 
36  4 slope 
slope  8 x
4
x f  x  x  Slope
f  x 4 36 8
2
x 2 16 6
1 9 5
36
.5 6.25 4.5
.25 5.0625 4.25
.1 4.41 4.1

4 .05 4.2025 4.05

.01 4.0401 4.01
x .001 4.004001 4.001
2 6

x
The slope gets closer and
closer to 4
 Or, in general… slope 
f  x  x   f  x 
x

2
 x  x   x2
slope 
x
f  x
x2

f  x  x  x 2  2 xx  x 2  x 2
slope 
x

2xx  x 2
f  x slope 
x

x
x slope 2 x
x  x
slope 2 x  x f '  2  4
x Let the change in x go
to 0
Some useful derivatives
Exponents
Linear Functions
f ( x)  Ax n  f ' ( x) nAx n  1
f ( x)  Ax  f ' ( x)  A

Example: f ( x) 4 x Example: f ( x) 3x 5

f '  x  4 f '  x  15 x 4

Logarithms
A
f ( x)  A ln( x)  f ' ( x) 
x
Example: f ( x) 12 ln  x 
12
f '  x 
x
A necessary condition for a maximum or a minimum is
that the derivative equals zero

f  x f  x
f '  x  0

OR
f '  x  0

x *
x
x * x

But how can we tell which is

which?
While the first derivative measures the slope (change in the value of the function),
the second derivative measures the change in the first derivative (change in the
slope)
f '  x  0
f  x f  x
f '  x  0

f '  x  0

f '  x  0
x x
x* x*

As x increases, As x
the slope is f ''  x   0 increases, the
f ''  x   0
decreasing slope is
increasing
PARTIAL DERIVATIVES

To maximize,
a volume function V = xyz subject to
the constraint 2xz + 2yz + xy = 12—which
expressed the side condition that the surface
area was 12 m2.
PARTIAL DERIVATIVES

Here, we present Lagrange’s method for

maximizing or minimizing a general function
f(x, y, z) subject to a constraint
(or side condition) of the form g(x, y, z) = k.
LAGRANGE MULTIPLIERS

It’s easier to explain the geometric basis

of Lagrange’s method for functions of two
variables.
 The method of Lagrange multipliers is a strategy
for finding the local maxima and minima of a
function subject to equality constraints.

Joseph-Louis
Lagrange
1736-1813

Harold Kuhn and Albert Tucker

later extended this method to
inequality constraints

Harold W. Kuhn Albert W. Tucker

1925 - 2014 1905 - 1995
 The Lagrange method writes the constrained
optimization problem in the following form
Objectiv
e
max f  x, y  Constrai
Choice x, y
nt(s)
variables
subject to g  x,y  0

The problem is then rewritten as follows

  f  x, y    g  x, y 

Multiplier (assumed greater or

equal to zero)
Here’ s how this would look in two dimensions…we’ve created
a new function that includes the constraints. This new function
coincides
f  x 
with the original objective at one point – the
maximum of both!

 g  x  0

  f  x    g  x 

f  x
x
x*
x 0 The new function is
Therefore, at the maximized
maximum, two thigs g  x  0 The new function
have to be true coincides with the
original objective
 So, we have our Lagrangian
function….
  f  x, y    g  x, y 

We need the derivatives with respect o both ‘x’ and ‘y’ to be zero

x  f x  x, y    g x  x, y  0
 y  f y  x, y    g y  x, y  0

And then we have the “multiplier conditions”

 0 g  x, y  0 g  x, y  0
LAGRANGE MULTIPLIERS—TWO VARIABLES

So, we start by trying to find the extreme

values of f(x, y) subject to a constraint of
the form g(x, y) = k.

 In other words, we seek the extreme values

of f(x, y) when the point (x, y) is restricted to lie
on the level curve g(x, y) = k.
LAGRANGE MULTIPLIERS—TWO VARIABLES

The figure shows this curve together with

several level curves of f.

 These have the

equations f(x, y) = c,
where c = 7, 8, 9,
10, 11
LAGRANGE MULTIPLIERS—TWO VARIABLES

To maximize f(x, y) subject to g(x, y) = k

is to find:

 The largest value of c

such that the level
curve f(x, y) = c
intersects g(x, y) = k.
LAGRANGE MULTIPLIERS—TWO VARIABLES

It appears that this happens when these

curves just touch each other—that is, when
they have a common tangent line.

 Otherwise, the value

of c could be
increased further.
LAGRANGE MULTIPLIERS—TWO VARIABLES

This means that the normal lines at

the point (x0 , y0) where they touch are
identical.

 So the gradient vectors are parallel.

 That is,
f ( x 0 , y0 )  g ( x 0 , y0 )
for some scalar λ.
LAGRANGE MULTIPLIERS—THREE VARIABLES

This kind of argument also applies to the

problem of finding the extreme values of
f(x, y, z) subject to the constraint g(x, y, z) = k.

 Thus, the point (x, y, z) is restricted to lie on

the level surface S with equation g(x, y, z) = k.
LAGRANGE MULTIPLIERS—THREE VARIABLES

Instead of the level curves in the previous

figure, we consider the level surfaces
f(x, y, z) = c.
 We argue that, if the maximum value of f is
f(x0, y0, z0) = c, then the level surface f(x, y, z) = c
is tangent to the level surface g(x, y, z) = k.

 So, the corresponding gradient vectors are parallel.

LAGRANGE MULTIPLIERS—THREE VARIABLES

This intuitive argument can be

made precise as follows.
LAGRANGE MULTIPLIERS—THREE VARIABLES

Suppose that a function f has an extreme

value at a point P(x0, y0, z0) on the surface S.

 Then, let C be a curve with vector equation

r(t) = <x(t), y(t), z(t)> that lies on S and passes
through P.
LAGRANGE MULTIPLIERS—THREE VARIABLES

If t0 is the parameter value corresponding to

the point P, then
r(t0) = <x0, y0, z0>

The composite function

h(t) = f(x(t), y(t), z(t))
represents the values that f
takes on the curve C.
LAGRANGE MULTIPLIERS—THREE VARIABLES

f has an extreme value at (x0, y0, z0).

So, it follows that h has an extreme value at t0.

 Thus, h’(t0) = 0.
LAGRANGE MULTIPLIERS—THREE VARIABLES

However, if f is differentiable, we can use

the Chain Rule to write:

0 h '  t0 
 f x  x0 , y0 , z0  x '  t0   f y  x0 , y0 , z0  y '  t0 
 f z  x0 , y0 , z0  z '  t0 
f  x0 , y0 , z0  r '  t0 
LAGRANGE MULTIPLIERS—THREE VARIABLES

This shows that the gradient vector

f  x0 , y0 , z0  is orthogonal to the tangent
vector r’(t0) to every such curve C.
 LAGRANGE MULTIPLIERS—THREE VARIABLES

However, as we know that the gradient vector

of g, , g  x0 , y0 , z0 
is also orthogonal to r’(t0) for every such
curve.

 This means that the gradient vectors f  x0 , y0 , z0 

and g x0 , y0 , z0 must be parallel.
 
LAGRANGE MULTIPLIER

Therefore, if g  x0 , y0 , z0  0 , there is
a number λ such that:

f  x0 , y0 , z0  g  x0 , y0 , z0 

 The number λ in the equation is called

a Lagrange multiplier.

 The procedure is as follows.

LAGRANGE MULTIPLIERS—METHOD

To find the maximum and minimum values

of f(x, y, z) subject to the constraint
g(x, y, z) = k [assuming that these extreme
values exist and g 0 on the surface
g(x, y, z) = k], we proceed as follows.
LAGRANGE MULTIPLIERS—METHOD

a. Find all values of x, y, z, and λ such that

f  x, y, z  g  x, y, z 
and g  x, y, z  k

b. Evaluate f at all the points (x, y, z) that

result from step a.
 The largest of these values is the maximum value of f.
 The smallest is the minimum value of f.
LAGRANGE’S METHOD

In deriving Lagrange’s method, we

assumed that g 0.

 In each of our examples, you can check that

g 0 at all points where g(x, y, z) = k.
LAGRANGE’S METHOD

If we write the vector equation f  g

in terms of its components, then the equations
in step a become:
fx = λgx fy = λgy fz = λgz g(x, y, z) = k

 This is a system of four equations in the four unknowns

x, y, z, and λ.
 However, it is not necessary to find explicit values for λ.
LAGRANGE’S METHOD

For functions of two variables,

the method of Lagrange multipliers is
similar to the method just described.
LAGRANGE’S METHOD

To find the extreme values of f(x, y) subject

to the constraint g(x, y) = k, we look for values
of x, y, and λ such that:
f  x, y  g  x, y  and g  x, y  k

 This amounts to solving three equations

in three unknowns:
fx = λgx fy = λgy g(x, y) = k
LAGRANGE’S METHOD

Our first illustration of Lagrange’s method

is to reconsider the problem.
LAGRANGE’S METHOD Example 1

A rectangular box without a lid is to be

made from 12 m2 of cardboard.

 Find the maximum volume of such a box.

LAGRANGE’S METHOD Example 1

As in Example, we let
x, y, and z be the length, width, and height,
respectively, of the box in meters.

 Then, we wish to maximize V = xyz

subject to the constraint

g(x, y, z) = 2xz + 2yz +xy = 12

LAGRANGE’S METHOD Example 1

Using the method of Lagrange multipliers,

we look for values of x, y, z, and λ
such that:

V g and g ( x, y, z ) 12

LAGRANGE’S METHOD Example 1

This gives the equations

Vx = λgx
Vy = λgy
Vz = λgz
2xz + 2yz + xy = 12
LAGRANGE’S METHOD

The equations become:

yz = λ(2z + y)

xz = λ(2z + x)

xy = λ(2x + 2y)

2xz + 2yz + xy = 12
LAGRANGE’S METHOD Example 1

There are no general rules for solving

systems of equations.

 Sometimes, some ingenuity is required.

LAGRANGE’S METHOD Example 1

In this example, you might notice that

if we multiply Equation 2 by x, Equation 3 by y,
and Equation 4 by z, then left sides of
the equations will be identical.
LAGRANGE’S METHOD

Doing so, we have:

xyz = λ(2xz + xy)

xyz = λ(2yz + xy)

xyz = λ(2xz + 2yz)

LAGRANGE’S METHOD Example 1

We observe that λ ≠ 0 because λ = 0

would imply yz = xz = xy = 0 from Equations
2, 3, and 4.

This would contradict.

LAGRANGE’S METHOD Example 1

Therefore, from Equations,

we have
2xz + xy = 2yz + xy
which gives xz = yz.

 However, z ≠ 0 (since z = 0 would give V = 0).

 Thus, x = y.
LAGRANGE’S METHOD Example 1

From Equations,
we have
2yz + xy = 2xz + 2yz
which gives 2xz = xy.

 Thus, since x ≠ 0, y = 2z.

LAGRANGE’S METHOD Example 1

If we now put x = y = 2z in Equation,

we get:
4z2 + 4z2 + 4z2 = 12

 Since x, y, and z are all positive, we therefore

have z = 1, and so x = 2 and y = 2.

 This agrees with our answer.