CHAPTER 16

Differentiable Functions of Several Variables

16.1. The Differential and Partial Derivatives

Let w f x y z be a function of the three variables x y z. In this chapter we shall explore how to evaluate the change in w near a point x0 y0 z0 , and make use of that evaluation. For functions of one variable, this led to the derivative: dw dx is the rate of change of w with respect to x. But in more than one variable, the lack of a unique independent variable makes this more complicated. In particular, the rates of change may differ, depending upon the direction in which we move. We start by using the one variable theory to dene change in w with respect to one variable at a time. Denition 16.1 Suppose we are given a function w f x y z. The partial derivative of f with respect to x is dened by differentiating f with respect to x, considering y and z as being held constant. That is, at a point x0 y0 z0 , the value of the partial derivative with respect to x is (16.1)

f x y z x 0 0 0

d f x y0 z0 dx

h 0

lim

f x0 h y0 z0 f x0 y0 z0 h

Similarly, if we keep x and z constant, we dene the partial derivative of f with respect to y by (16.2)

f y

d f x0 y z0 dy

and by keeping x and y constant, we dene the partial derivative of f with respect to z by (16.3)

f z

d f x0 y0 z dz x1 xy2 .

Example 16.1 Find the partial derivatives of f x y Thinking of y as a constant, we have (16.4)

f x

2 1 xy x21 xyy

1 xy1 3xy

235

Chapter 16

Differentiable Functions of Several Variables

236

Now, we think of x as constant and differentiate with respect to y: (16.5)

f y

x21 xyx

2x2 1 xy

Example 16.2 The partial derivatives of f x y z (16.6)

xyz are xz

f x

yz

f y

f z

zy

Of course, the partial derivatives are themselves functions, and when it is possible to differentiate the partial derivatives, we do so, obtaining higher order derivatives. More precisely, the partial derivatives are found by differentiating the formula for f with respect to the relevant variable, treating the other variable as a constant. Apply this procedure to the functions so obtained to get the second partial derivatives: (16.7)

2 f x2

f x x

2 f y x

f y x

2 f x y

f x y

2 f y2

f y y

Example 16.3 Calculate the second partial derivatives of the function in example 1. We have f x y x1 xy2 , and have found (16.8)

f x

1 xy1 3xy

f y

2x2 1 xy

Differentiating these expressions, we obtain (16.9)

2 f x2 2 f y x

1 xy3y y1 3xy

4y 6xy2

(16.10)

1 xy3x x1 3xy

4x 6x2y

(16.11)

2 f x y

4x1 xy 2x2y

4x 6x2y

(16.12)

2 f y2

2x2 x

2x3

Notice that the second and third lines are equal. This is a general fact: the mixed partials (the middle terms above) are equal when the second partials are continuous: (16.13)

2 f y x

2 f x y

16.1

The Differential and Partial Derivatives

237

This is not easily proven, but is easily veried by many examples. Thus 2 f x y can be calculated in whatever is the most convenient order. Finally, we note an alternative notation for partial derivatives; (16.14) fx

f x

fy

f y

fxx

2 f x2

fxy

2 f x y

fyy

2 f y2

etc

Example 16.4 Let f x y y tan x x secy. Show that fxy fyx . We calculate the rst partial derivatives and then the mixed partials in both orders: (16.15) (16.16) fx fyx y sec2 x secy sec2 x secy tan y fy fxy tan x x secy tan y sec2 x secy tan y

The partial derivatives of a function w f x y z tell us the rates of change of w in the coordinate directions. But there are many directions at a point on the plane or in space: how do we nd these rates in other directions? More generally, if two or three variables are changing, how do we explore the corresponding change in w? The answer to these questions starts with the generalization of the idea of the differential as linear approximation. For a function of one variable, a function w f x is differentiable if it is can be locally approximated by a linear function (16.17) w w0 mx x0

or, what is the same, the graph of w f x at a point x0 y0 is more and more like a straight line, the closer we look. The line is determined by its slope m f x0 . For functions of more than one variable, the idea is the same, but takes a little more explanation and notation. Denition 16.2 Let w f x y z be a function dened near the point x0 y0 z0 . We say that f is differentiable if it can be well- approximated near x0 y0 z0 by a linear function (16.18) w w0 ax x0 by y0 cz z0

In this case, we call the linear function the differential of f at x0 y0 z0 , denoted d f x0 y0 z0 . It is important to keep in mind that the differential is a function of a vector at the point; that is, of the increments x x0 y y0 z z0 . If f x y is a function of two variables, we can consider the graph of the function as the set of points x y z such that z f x y. To say that f is differentiable is to say that this graph is more and more like a plane, the closer we look. This plane, called the tangent plane to the graph, is the graph of the approximating linear function, the differential. For a precise denition of what we mean by well approximated, see the discussion in section 16.3. The following example illustrates this meaning. Example 16.5 Let f x y x2 y. Find the differential of f at the point (1,3). Find the equation of the tangent plane to the graph of z f x at the point. We have x0 y0 1 3, and z0 f x0 y0 4. Express z 4 in terms of x 1 and y 3: (16.19) z4 x2 y 4
1 x

12 3 y 3 4

Chapter 16

Differentiable Functions of Several Variables 1 2x 1 x 12 3 y 3 z4

238

(16.20) (16.21)

simpling to

2x 1 y 3 x 12

Comparing with (16.18), the rst two terms give the differential. x 12 is the error in the approximation. The equation of the tangent plane is (16.22) z4 2x 1 y 3 or z 2x y 1

If we just follow the function along the line where y y0 z z0 , then (16.18) becomes just w w0 ax x0 ; comparing this with denition 16.1, we see that a is the derivative of w in the x-direction, that is a w x. Similarly b w y and c w z. Finally, since the variables x y z are themselves linear, we have that dx is x x0 ,and so forth. This leads to the following restatement of the denition of differentiability: Proposition 16.1 Suppose that w (16.23) f x y z is differentiable at x0 y0 z0 . Then dw

f f f dx dy dz x y z

There are a variety of ways to use formula (16.23), which we now illustrate. Example 16.6 Let (16.24) z f x y x2 xy y3

Find the equation of the tangent plane to the graph at the point (2,-1). At x0 y0 2 1, we have z0 f x0 y0 6. We calculate (16.25) so, at (2,-1), f x (16.26) 5 f y z6

f x

2x y

f y

x 3y2
5x y 3

1. Substituting these values in (16.18) we obtain 5x 2 y 1 or z

An alternative approach is to differentiate equation (16.24) implicitly: (16.27) dz 2xdx xdy ydx 3y2dy Evaluating at (2,-1), we have z0 6, and dz 4dx 2dy dx 3dy. This is the equation of the tangent plane, with the differentials dx dy dz replaced by the increments x 2 y 1 z 5: (16.28) z6 4x 2 2y 1 x 2 3y 1 x2 xy y at

which is the same as (16.26). Example 16.7 Find the equation of the tangent plane to the graph of the function z (2,-1, 1).

16.1

The Differential and Partial Derivatives

239

First, we calculate the differential (16.29) and then evaluate it at the point: (16.30) dz 4dx 2dy dx dy 3dx dy dz 2xdx xdy ydx dy

We now get the equation of the tangent plane by replacing the differentials by the increments: (16.31) z1 3x 2 y 1 or z 3x y 4

Example 16.8 Find the points at which the graph of z f x y x2 2xy y has a horizontal tangent plane. The horizontal plane through the point x0 y0 z0 has the equation z z0 0. Thus our points are those where d f 0; i.e., solutions of the pair of equations (16.32) Calculating, we get 2x 2y 0

f x

0 0, so x

f y

0 1 2 and our point is 1 2 1 2.

2x 1

1 2 y

Example 16.9 Given the function z x2 xy y3 , in what direction, at the point (1,1,1) is the rate of change of z equal to zero? The differential of z is dz 2x ydx x 3y2dy, so at (1,1,1), we have dz dx 2dy. This is zero for the direction in which dx 2dy; that is along the line of slope -1/2. Thus the answer is given by a vector in that direction, for example: 2I J. Example 16.10 Suppose that we have designed a cylindrical silo of base radius 6 meters and height 10 meters, and we are asked to increase the radius by .25 m and the height by .2 m. By (approximately) how much do we increase the volume? The volume of a cylinder of radius r and height h is V r2 h. To answer this question, we consider the linear approximation of volume, so we take the differential of V : (16.33) Now, in our case r (16.34) dV 5 h 10 dr dV 25 dh 2 rhdr r2dh 2, so we calculate

2 510 25 52 2

25 5

30

cubic meters

By looking at gure 1, we can identify the two terms in the increment of volume: the rst is the volume of the shell of width dr around the cylinder, and the second is the volume of the cap of height dh. The negligible part is the volume 2 drdh of the washer at the top of width dr and height dh.

Chapter 16

Differentiable Functions of Several Variables

240

Figure 16.1

dh

dr

Proposition 16.2 (The Chain Rule). Let w f x y z be a function dened in a region R in space. Suppose that is a curve in R given parametrically by x xt y yt z zt , with t 0 corresponding to x0 y0 z0 . Then, considering w f xt yt zt as a function of t along , we have (16.35) dw dt

w dx w dy w dz x dt y dt z dt

That is, the rate of change of w with respect to t along is given by (16.35). We shall give an explanation of this formula in section 3. Example 16.11 Let w f x y z t 2 z lnt. Find dw dt at t 2. Differentiating, (16.36) xy y2 z. Consider the curve given parametrically by x t y

w x
dx dt dw dt 2 y dw dt

w y
dy dt

x 2yz

w z
dz dt 1 t

y2

(16.37) so (16.38) At t 2 we calculate x

2t

y1 x 2yz2t y2 ln 2, giving

1 t

4 and z

(16.39)

4 2 8 ln24 16 2

20 32 ln2

Example 16.12 Let z x cost y 2 sint.

x2 y2 . Find the maximum value of z on the ellipse given parametrically by

16.1

The Differential and Partial Derivatives

241

We need to nd the points at which dz dt (16.40) and thus (16.41) dz dt

0. Now dx dt

z x

2x

z y

2y

sint

dy dt

2 cost

2x sint 4y cost

Since x cost and y 2 sint, this gives dz dt 2 sint cost 8 sint cost Set this equal to zero to obtain 6 sint cost 0, or 3 sin2t 0, which has the solutions t 0 2 , and x 1 y 0 or x 0 y 2. The corresponding values of z are thus 1 and 4, so 1 is the minimum value and 4 is the maximum value on the ellipse. We can think of an equation of the form f x y z 0 as dening z implicitly as a function of x and y, in the sense that we could solve for z, given specic values of x and y. However, just as in one dimension, we need not solve for z to nd the partial derivatives. If we take the differential of the dening equation f x y z 0 we get (16.42) fx dx fy dy fz dz 0 so that dz

ffx dx ffy dy
z z

The coefcient of dx is thus z x, and the coefcient of dy is z y. Of course if fz dened. But if fz 0, then this method works.

0, these are not

Proposition 16.3 Suppose that f is a differentiable function of x y z near the point x0 y0 z0 , and that fz x0 y0 z0 0. Then the equation f x y z 0 denes z implicitly as a function of x y and (16.43)

z x

ffx
z

and

z x

ffy
z

Example 16.13 Given f x y z z3 3xz2 y2 z, nd expressions for z x and z y where z is dened implicitly as a function of x y by the equation f x y z 5. Evaluate these at the point (1,1,1). First we calculate the partial derivatives: (16.44) so that (16.45)

f x

3z2

f y

2yz

f z

3z2 6zx y2

z x

3z2 3z y2 6zx 3
10 z x

z y

3z2 2yz y2 6zx

10.

The values at (1,1,1) are z x

Chapter 16

Differentiable Functions of Several Variables

242

16.2. Gradients and Vector methods

Let w f x y z, where f is a differentiable function. To put the formula for the differential,(16.23), in vector form, we introduce the gradient of the function f : (16.46) f

f f f I J K x y z

and the vector differential dX dxI dyJ dzK. We interpret dX as a small change in the vector X. Then (16.17) can be rewritten as (16.47) dw

f f f dx dy dz x y z

dX

This leads to the following vectorial form of the chain rule. f x y z be a function dened Proposition 16.4 (The Gradient Form of the Chain Rule). Let w in a region R in space. Suppose that is a curve in R given parametrically by X Xt , with t 0 corresponding to X0 . Then, considering w f Xt as a function of t along , we have (16.48) evaluated at X0 . The partial derivatives tell us the rate of change of the function f in the coordinate directions. Using the gradient, we can calculate the rate of change in any direction. Denition 16.3 Let w f x y z be differentiable in a neighborhood of X0 . For any vector V, let Xt X0 tV parametrize the line through X0 in the direction V. The derivative of f along V is (16.49) D V f X 0
t

dw dt

dX dt

lim
0

f X0 tV f X0 t

Propostion 16.5. Given the differentiable function f and a vector V, we have (16.50) D V f X 0 f V

The right hand side of (16.49) is the derivative of f along the line in the direction of V. That line is parametrized by Xt X0 tV, so dX dt V. Now, by the chain rule (16.51) DV f X0 d f Xt dt

f dx f dy f dz x dt y dt z dt

dX dt

f V

If we replace V by a unit vector U, then the parameter t represents distance along the line, since Xt X0 tU t. We say that the line is parametrized by arc length, and refer to DU f as the directional derivative of f in the direction U. Example 16.14 Let f x y x3 3x2 xy 7 and U 0 6I 0 8J. Find DU f 1 2. We have fx 3x2 6x y fy x. Evaluating at (1,-2), we have f 1 2 5I J. Thus (16.52) DU f 1

f 1

2 U 3 0 8 3 8

16.2

Gradients and Vector methods

243

Example 16.15 For f as above, nd the direction U at (1,-2) in which DU f Let U aI bJ. We must solve (16.53) This gives b do Thus (16.54) f 1

0. 0 1

2 U

5I J aI bJ 5a b
26a2 I 5J 26

5a. Since U is a unit vector, we have a2 b2

1, so a

26 b

26 will

We also have the answer U. Notice that both these vectors are unit vectors in the direction of f . Example 16.16 Let be parametrized by Xt t 2 I lntJ tK, and let w along . What is the rate of change of w with respect to t at the point t 2? To use (16.48), we calculate (16.55) so that (16.56) since x t 2, y lnt and z dw dt
f

f x y

xyz. Find dw dt

yzI xzJ xyK

dX dt

1 2tI J K t

dX dt

2tyz

xz xy t

3t 2 lnt 1

t. At t

2, we get dz dt

12ln 2 1. x2 2xy. Find

Example 16.17 Let Xt costI sintJ parametrize the unit circle, and let f x y the maximum value of f on the unit circle. The function z f Xt has a maximum when dz dt 0. We calculate: (16.57) f
2x 2yI 2xJ

2cost sint I 2 costJ

(16.58) so that (16.59) To solve dz dt (16.60) dz dt

f

dX dt

sintI costJ
2cost sint sint 2 cos2 t

dX dt

0 we use double angle formulas:

2cost sint sint 2 cos2 t

2 cost sint 2cos2 t sin2 t sin2t 2 cos2t

sint are

which is zero when tan2t 2, or t 31 7 211 7. The corresponding values of x cost y x 526 y 851. Calculating the values of z at these points gives the maximum 1.172.

For a function w f x y z of three variables dened near the point X0 : x0 y0 z0 , let w0 f x0 y0 z0 . The equation w w0 is the level surface S of w at x0 y0 z0 . For f differentiable at a point

Chapter 16

Differentiable Functions of Several Variables

244

X0 , the fact that f can be approximated by a linear function implies that the surface S looks more and more like a plane, the closer we look. This plane, given by the equation d f X0 0, is the tangent plane to S at X0 . We now note that the gradient of f is the normal to this surface, and points in the direction of maximum increase of f . Proposition 16.5 Let f be a function differentiable in a neighborhood of the point X0 . a) f X0 points in the direction of maximum increase of the function f at X0 . b) f X0 is the normal to the tangent plane of the level set of f through X0 . To show a), start with a unit vector U. From (16.50) we have (16.61) D U X 0 f U f cos

where is the angle between f and U (since U 1). This takes its greatest value when cos 0, that is U f . To show b), let V be a vector on the tangent plane. By denition, d f X0 V 0, so (16.62) f X 0 V d f X0 V 0

Thus f X0 is orthogonal to every vector in the tangent plane, so can be taken to be its normal. Now, a point X lies in the tangent plane if and only if the vector X X0 lies on the tangent plane, or (16.63) f X0 X X0 0

which is thus the equation of the tangent plane. Example 16.18 Let f x y x3 3x2 y2 2y. Find the equation of the line tangent to the curve f x y 9 at (2,-1). The above discussion for three dimensions holds just as well in two dimensions. Thus, by proposition 16.6, the normal to the tangent line to the curve is f .We calculate f 3x2 6xy2 I 6x2 y 2J; which at x 2 y 1 is the vector 24I 22J. Now, the equation of the tangent line is given by (16.63), where X0 2I J is the vector to the point (2,-1): (16.64)

22J x 2I y 1J which simplies to 24x 22y 70.

24I

or 24x 2 22y 1

Example 16.19 Let f x y z xyz. Find the gradient of f . Find the equation of the tangent plane to the level surface f x y z 2 at the point X0 : 1 2 1. We calculate: (16.65) At X0 , f (16.66) f

f f f I J K x y z

yzI xzJ xyK 0:

2I J 2K, so the equation of the tangent plane is f X X0 2x 1 y 2 2z 1 0 or 2x y 2z 5

Example 16.20 Let w x xy yz2 . Find the equation of the tangent plane to the surface w 2 at (3,1,2). We calculate w 1 yI x z2J 2zyK. At the given point (3,1,2), w 2I J 6K. This is the normal to the tangent plane, at X0 3I J 2K, so the equation of that plane is (16.67) w X X0 2x 3 y 1 6z 2 0

16.2
or 2x y 6z 7 0.

Gradients and Vector methods

245

Example 16.21 Let S be the sphere x2 y2 z2 a2 a 0. Show that at any point X on the sphere, the vector X is orthogonal to the sphere. Let w x2 y2 z2 , so that S is the level set w a2 . Then w is normal to S at X xI yJ zK. But (16.68) w 2xI 2yJ 2zK 2X

Example 16.22 Let S1 be the sphere x2 y2 z2 4 and S2 the cylinder x2 y2 1. Let X xI yJ zK be a point on the curve of intersection of the surfaces S1 and S2 . Find a vector tangent to at X. Let w1 x2 y2 z2 w2 x2 y2 , so that is the intersection of the level sets w1 4 and w2 1. Then w1 and w2 are both orthogonal to the tangent to , so w1 w2 points in the direction of the tangent to . We calculate: (16.69) w1 w2 2xI yJ zK 2xI yJ 4yzI zxJ

In the above, we have considered a surface as a graph or as a level set of a function. Surfaces can also be given parametrically. Let u and v be the variables in a region R of the plane, and let Xu v xu vI yu vJ zu vK be a vector-valued function on R. Then the set of values of Xu v, as u v ranges over R describes a surface in space. Example 16.23 Consider the function Xu v coordinates, this is given by the equations uv y
u

vI u vJ uvK dened in u v space. In

uv

(16.70)

uv z

We can solve for u and v in terms of x and y; (16.71) u xy 2 v

x y ;
2

putting these in the formula for z we have (16.72) z uv x y x y 2 2 y2 x2 .

x2 y2
4

so the surface is the hyperbolic paraboloid 4z

Now, in general it may not be so easy (or simple) to realize a parametric surface as a level set; however, we can use the parametric equations to , for example, nd the tangent plane to the surface at a point. Proposition 16.6 Let Xu v region in R-space. Dene (16.73) xu vI yu vJ zu vK be a vector-valued function dened on a

Xu

x y z I J K u u u

Chapter 16

Differentiable Functions of Several Variables

246

x y z I J K v v v a) The vector Xu Xv is normal to the surface. b) If w f x y z is a function dened near the surface, we can consider it as a function of u and v by writing w f xu v yu v zu v. Then
(16.74) Xv (16.75)

w u

w Xu

w v

w Xv

To see this, x a point u0 v0 . If we set v by (16.76) Xu

v0 and let u vary, we get the curve C given parameterically

xu v0 I yu v0J zu v0 K

The tangent vector to this curve is Xu , and since the curve lies in the surface, its tangent vector lies in the tangent plane. Similarly, considering the curve u u0 , we see that the vector Xv also lies in the tangent plane. Thus Xu Xv is normal to the tangent plane. Part b) follows directly from the chain rule, applied to the curves u u0 and v v0 . Example 16.24 Find the equation of the tangent plane to the surface of example 23 at the point (-2,4,3). From (16.71), this point corresponds to the values u 1 v 3. Now, we differentiate the function dening the surface, obtaining (16.77) The values at u is N Xu Xv (16.78) Xu I J vK Xv 1 v 3 are Xu I J 3K Xv I J K. Thus, a normal to the tangent plane 2I 4J 2K, and the equation of the tangent plane is

I J uK
z

2x 2 4y 4 2z 3

0 or

x 2y 3

Example 16.25 Consider the surface given parametrically by (16.79) Xu v 3 cos u cosvI 4 cosu sin vJ 5 sinuK

Find the normal to the tangent plane ant the point corresponding to u Differentiate: (16.80) (16.81) Xu

3 v

6.

3 sin u cosvI 4 sin u sin vJ 5 cosuK

Xv

3 cosu sin vI 4 cosu cos vJ 9I 4

5 3J K 2

Evaluating at the given point, we have

(16.82)

Xu

1 3 23 23 I 4 23 1 J 5 2 K 2 Xv

(16.83) A normal to the plane is N

1 1 3 2 1 I 4 2 23 J 3 I 3J 2 4 Xu Xv 5 2 3I 15 8J 3 3K.

16.3

Theoretical considerations

247

16.3. Theoretical considerations

In order to make the intuitive concept of linear approximation, as used above, more precise we start with the idea of closeness in the space itself. We measure the nearness of two points by the length of the line segment joining the points. Thus, in vectorial terms, the distance between X and X0 is X X0 , that is, the square root of the sum of the squares of the components. We dene limits in terms of this distance. Denition 16.4 The ball of radius c centered at X0 (denoted BX0 c) is the set of all points of distance less than c from X0 . A neighborhood of X0 is any set which contains some ball centered at X0 . Denition 16.5 Suppose that f is a function dened in a neighborhood of X0 . We say that (16.84)
X X0

lim f X

if we can insure that f X L can be made as small as we please by taking X close enough to X0 . We say that f is continuous at X0 if (16.85)
X X0

lim f X

f X0

Just as in one variable, we are assured that all functions which can be expressed by polynomials in the coordinates are continuous. Denition 16.6 A linear function is a function of the form LX C X, for some vector C. In coordinates we have Lx y z ax by cz, where we have written C aI bJ cK and X xI yJ zK. Its level surface through X0 is the plane ax x0 by y0 cz z0 0, or C X X0 0. Now we dene differentiability at X0 of a function f : that it can be well- approximated by a linear function. This is the direct generalization of the denition of the derivative in one dimension. Denition 16.7 Suppose that f is a function dened in a neighborhood of X0 . We say that f is differentiable at X0 if there is a linear function L such that (16.86) lim f X f X0 LX X0 X X0 0 f

X X0

In this case, we call L the differential of f at X0 , denoted d f X0 . We can write L X X0 X0, where f is the gradient of f .

Now, as we have seen, the calculation of differentials amounts to calculating partial derivatives. To see this in terms of the above denition, lets look at the situation in two variables, writing X xI yJ. Suppose that f is differentiable at X0 , and its differential there is Lx x0 y y0 ax x0 by y0. First we see what happens to equation (16.86) along the line y y0 . Then X X0 x x0 I, and we get (16.87) lim

x x0

f x y0 f x0 y0 ax x0 x x0

f x y 0 f x 0 y 0 a x x0

Chapter 16

Differentiable Functions of Several Variables

248

or (16.88)

f x

x x0

lim

f x y0 f x0 y0 x x0

In the same way, we see that b f y. Now we turn to an argument for the chain rule in two dimensions. The Chain Rule. Let w f x y be a differentiable function dened in a region R. Suppose that is a differentiable curve in R given parametrically by X Xt , with t 0 corresponding to X0 . Then, considering w f Xt as a function of t along , we have (16.89) dw dt
f

dX dt

evaluated at X0 . We start with the denition of differentiability. Let (16.90)

f Xt f X0 L Xt X0

where we have written L for the gradient of w evaluated at X0 . By (16.86), (16.91) Now, by continuity Xt (16.92) X0 as t lim
0
t

Xt

lim
X0

t Xt X0

0, and thus lim Xt X0 t lim Xt X0 t 0 t 0

Xt X0

by (16.91), and the assumption of differentiability of Xt , which assure that the second limit on the right exists. Now, by the denition of : (16.93) f Xt f X0 0: lim L
0

L Xt X0 t Xt X0 t lim t 0 t t

Divide by t, and take the limit as t (16.94) dw dt lim

0

f Xt f X0 t

dX dt

16.4. Optimization
Now we turn to the technique for nding maxima and minima of a function z f x y of two variables.

Denition 16.8 If f x0 y0 f x y is at least as large as its value at all nearby points, we say that f has a local maximum at x0 y0 . More precisely, if, for some a 0 we have (16.95) f x0 y0 f x y

16.4

Optimization

249

for all x y within a distance a of x0 y0 , then x0 y0 is a local maximum point for f . Similarly, if instead we have (16.96) f x0 y0 f x y

for all x y sufciently close to x0 y0 , then x0 y0 is a local minimum point for f . The rst derivative test for functions of one variable gives us the following criterion: Proposition 16.7 Suppose that X0 is a local maximum (or minimum) for f . Then f To see this, pick a vector V and consider the line given by the equation Xt has a maximum at t 0, so (16.97) This can only be true for all V if f Denition 16.9 If f x0 y0 f V 0.
d f Xt 0 dt

0.

X0 tV. Then f Xt

0 we say that x0 y0 is a critical point.

Thus, to nd the local maxima or minima of a function in a given region, one must look among the critical points. Example 16.26 Find the critical points of the function f x y We calculate the components of the gradient: (16.98) x3 xy y2 x.

f x

3x2 y 1

f y

x 2y

Now, we set these equal to zero and solve. The second equation gives x rst gives 12y2 y 1 0, which has the roots (16.99) y

2y; substituting that in the

1 48 24

or y

1 4

1 3

Thus, the critical points are 1 2 1 4 2 3 1 3. But now, how can we tell whether or not we have a local maximum or a local minimum at either of these points? In fact, we may have neither; there is a third possibility: that along certain lines through the critical point, the value is a local maximum, and along other lines, the value is a local minimum. Such a point is a saddle point. Example 16.27 Let z x2 y2 . Then the origin is a critical point for z. Since z x2 along the line y 0, z has a minimum at the origin on this line, but on the line x 0, we have z y2 which has a maximum at the origin along this line. We distinguish among these points by using the second derivative test in one variable. In order to make clear what the criterion is, we rst consider the case of a quadratic function. Example 16.28 Let z be a quadratic function of the variables u v: z au2 2buv cv2. The origin is a critical point. By completing the square we can discover what kind of critical point it is: (16.100) z b2 b2 b au2 2 uv 2 v2 c v2 a a a b ac b2 2 v au v2 a a

Chapter 16

Differentiable Functions of Several Variables

250

Thus if both terms have positive coefcients, the origin is a minimum; if both terms have negative coefcients, the origin is a maximum, and if the signs differ, the origin is a saddle point. We call the expression D ac b2 the discriminant of the quadratic function dening z. Notice that if D 0, that the coefcients of (16.102) have the same sign, and if also a 0, we have a minimum. and if a 0, a maximum. If D 0, the coefcients have different signs and we have a saddle point. This example leads us directly to the general criterion by applying the second derivative test along each line through the critical point. For the function z f x y, let fxx fx y fyy represent the second partial derivatives of f . Proposition 16.8 Suppose that f x0 y0 0, that is x0 y0 is a critical point. Then (evaluating at x0 y0 ): If D fxx fyy fxy 2 0 at a point x0 y0 , the f has a saddle point there. If D fxx fyy fxy 2 0 and fxx 0, at a point x0 y0 , then f has a local minimum there. If D fxx fyy fxy 2 0 and fxx 0, at a point x0 y0 , then f has a local maximum there. If D 0, we can conclude nothing. We note that when D 0 the second derivative along all lines has the same sign, so we could check whether fyy is greater or less than 0 instead, if that were easier. To see this, choose a vector V uI vJ and consider the function fV t tv. Differentiating we nd, by the chain rule, (16.101) d f dt V u fx v fy d2 f dt 2 V d u f x v f y dt u d fx dt f X0 tV d fy dt f x0 tu y0

We compute the second derivative by applying the chain rule to the functions fx fy : (16.102) d2 f dt 2 V uu fxx v fyx vu fxy v fyy u2 fxx 2uv fxy v2 fyy

If this is positive, then the function fV has a minimum; that is the function f has a minimum along the line in the direction of V. If this holds for all directions V; that is, for all values of u v, then f has a local minimum at x0 y0 . But, referring back to example 16.28, this is true if D 0 fxx 0. Similarly, if D 0 fxx 0, then f has a maximum along all lines through x0 y0 , so f has a local maximum there. However, if D and fxx have different signs, then f has a local maximum in some directions, and a local minimum in others, so we have a saddle point. Example 16.29 We continue with example 16.26. We found critical points at P1 2 1 4, Q2 3 Differentiating the rst partials (see (16.98)), we get (16.103) Thus (16.104) at P : D 6 1 2 2 1 2 4 fxx 6x fxy 1 fyy 2

3.

and at Q :

2 2 2 1 3

so P is a saddle point, and since fxx

0, Q is a local minimum.

Example 16.30 Let f x y x2 2y4 xy 4x 2y. Find the local maxima and minima of z. Does f have a global maximum or minimum?

16.4
First we nd the critical points: (16.105) fx

Optimization

251

2x y 4

fy

8y3 x 2

To nd the points where both are zero, we obtain x the rst,we get (16.106) 28y3 2 y 4 0

8y3 2 from the second equation. Putting this in

or 8 1 4, R15 8 1 4.

This has the solutions y We now calculate the second derivatives: (16.107) fxx

16y3 y 0 0 1 4, so the critical points are P2 0 Q17

2 fxy 1 fyy 24y2

Then D 48y2 1, which is positive at all of these points. Since fxx is everywhere positive, these are all local minima. To determine the global minimum, we evaluate: f P 4 f Q 4 0078 f R 4 0078. Thus the global minimum is -4.0078, attained at both Q and R. Everywhere else the function has a direction in which it is increasing, so it has no global maximum. Notice, in these problems we have to solve several equations simultaneously, and usually they are not linear. There are no universal algorithms for solving such systems of equations, and we have to follow our intuition. Usually the technique of substitution works (although in the above problem, with other constants the cubic equation in (16.106) would be much more difcult). So, in general the procedure to follow is to look at the given equations to see if, in one of the equations one of the variables can be easily written in terms of the other. If so, substitute that expression in the other equation.

16.4.1

The Method of Lagrange Multipliers

Let C be a curve in the plane, not going through the origin. Lets nd the point on C which is closest to the origin. This amounts to nding the minimum value of f x y x2 y2 on the curve C. If C is given parametrically by the equations x xt y yt , we know what to do: differentiate f xt yt and set the derivative equal to zero. But, if the curve is given implicitly by an equation gx y c, we dont want to solve the equation explicitly, and we dont have to. Looking at the condition (16.108) d f xt yt dt 0 as f dX dt 0

we see that the requirement is that f is orthogonal to the tangent to the curve at the minimizing point. But g is orthogonal to its level set C everywhere, so at the minimizing point we have that f and g are collinear; that is, they are multiples of each other. Thus, we can solve the problem by inding the solution of the system (16.109) f

gx y

Example 16.31 Find the point on the line 3x 2y

This gives three scalar equations in three unknowns, which, in principle, can be solved. Of course the value of is not of interest, but is useful as an auxiliary to nding the values of x y. 1 which is closest to the point (4,7).

Chapter 16

Differentiable Functions of Several Variables 3x 2y

252

Given the constraint gx y gradients are (16.110) f

1, we want to minimize f x y 3I 2J

42 y 72. The

2x 4I 2y 7J and g 2 y 7

These gradients are collinear at the minimizing point, so we have to solve the equations (16.111) 2x 4 4x 4 3 2 and 3x 2y 1

We can eliminate from the rst two equations: (16.112) 6 6y 7 so that 4x 6y

26

Now we have simultaneous linear equations in x and y which we can solve, getting the point (16,47/2). We note that the Lagrangian equations (16.109) just say that the line from this point to (4,7) has to be orthogonal to the given line; something we knew from geometry. Example 16.32 Find the maximum value of f x y xy on the ellipse x2 4y2 1. Let gx y x2 4y2 . We calculate the gradients: f yI xJ and g 2xI 8yJ. At the point on the ellipse at which we have the maximum, we have f orthogonal to the tangent to the ellipse, so is collinear with g. Thus we have the equation f g for some . This gives the scalar equations (16.113) y 2 x x 8 y x2 8y2 1

We can eliminate by dividing the rst equation by the second: (16.114) y x 2 x 8 y x 4y giving x2 4y2

Substituting that in the last equation gives 4y2 4y2 (16.115) The possible values of f x y minimum is 1 4. x2 4y2 4 8

1, so that y 1

1
2

2. Then

so that x

xy at these points are 1 4, so the maximum value of f is 1/4, and its

The parameter , called the Lagrange multiplier, serves the purpose of nding a relation between x and y which is a consequence of the optimization. The value of is not important, but in some cases it may make the problem easier to rst determine . To summarize: given the problem: minimize (or maximize) a function f x y subject to a constraint gx y c. We observe that the chain rule tells us that, at the optimizing point, f is orthogonal to the tangent to the level set of g. But so is g, so we must have f g for some . Solve this equation in conjunction with gx y c to nd the point. This method (of Lagrange multipliers) works in three dimensions as well. Proposition 16.9 Suppose that w f x y z is a differentiable function, and we wish to nd its maxima and minima subject to a constraint gx y z c. At an optimizing point P there is a such that (16.116) f P

gP

gx y z

16.4

Optimization

253

These equations give a system of four equations in four unknowns which, in typical circumstances, have only a nite number of solutions. The maximum (minimum) of the function must occur at one of these points. To see why this is true, we follow the two dimensional argument. Let S be the level surface gx y z c. Let C be a curve through P lying in the surface S. Then f is optimized along C, so that the derivative of f along the curve is zero at P. But this just says that f P is orthogonal to the tangent to the curve. Since every vector in the tangent plane to S is the tangent vector to such a curve, f P is orthogonal to the tangent plane to S. But so is gP, so f P and gP must be colinear. Example 16.33 Find the point on the plane 2x 3y z 1 closest to the point 1 1 0. Here the constraint is gx y z 2x 3y z 1 and the function to be minimized is f x y z x 12 y 12 z2 . Taking the gradients and introducing the Lagrange multipier, we are led to the equations (16.117) 2x 1 2 2 y 1 3 2z

2x 3y z

We use the rst three equations to express the variables in terms of , and then use the last to solve for : (16.118) so that (16.119) This gives 2 1 3 3 2 2 2 1 x

3 2 2

1 7. Substituting into equations 16.118), we nd the desired point to be (1/7, -11/14, 1/14).

Example 16.34 Farmer Brown wishes to enclose a rectangular coop of 1000 square feet. He will build three sides of brick, costing $25 per linear foot, and the fourth of chain link fence, at $ 15 per linear foot. What should the dimensions be to minimize the cost? Let x and y be the dimensions of the coop, where x represents the sides, both of which are to be of brick. The constraint is gx y xy 1000, and the cost function is C 252x y 15y. We have C 50I 40J, and g yI xJ. The equations to solve are: (16.120) so (16.121) or 2
5040

50

40

x xy

1000

1000 1000 2, giving

xy

50 40 40

2. Then x

20 2 y

50

25 2.

Many problems involve nding the maximum or minimum of a function of many variables subject to many constraints The technique of Lagrange multipliers works in this general context, but - of course - is much more difcult to employ. To give a sense of the general procedure, we state the proposition in the case of a function of three variables with two constraints.

Chapter 16

Differentiable Functions of Several Variables

254

Proposition 16.10 To nd the extreme values of a function f x y z subject to two constraints (say along a curve), gx y z c hx y z d, we have to solve the ve equations in the ve unknowns x y z : (16.122) f P

gP h

gx y z

hx y z