Chapter 8: Performance Surfaces and Optimum Points

Brandon Morgan

1/14/2021

E8.2
We are given the following vector function

2 2
F (x) = e2x1 +2x2 +x1 −5x2 +10

The second-order Taylor series approximation for a vector function, F (x), is given by:

1
F (x) = F (x∗ ) + ∇F (x)|x=x∗ (x − x∗ ) + ∇2 F (x)|x=x∗ (x − x∗ )2
2
∂ ∂
Where ∇F (x) is known as the gradient, by [ ∂x 1
F (x)| ∂x 2
F (x)| . . . | ∂x∂n F (x)], and where ∇2 F (x) is known
as the Hessian given by equation 8.11 in book.
The gradient of our function is calcualted to be:

2 2 2 2
∇F (x) = [ (4x1 + 1)e2x1 +2x2 +x1 −5x2 +10 | (4x2 − 5)e2x1 +2x2 +x1 −5x2 +10 ]

A Hessian matrix is given by:

∂2 ∂2
" #
∂x21
F (x) ∂x1 ∂x2 F (x)
∇2 F (x) = ∂2 ∂2
∂x2 ∂x1 F (x) ∂x22
F (x)
Where it can be shown from our example that

∂2 2 2 2 2 2 2
F (x) = 16e2x1 +2x2 +x1 −5y+10 x22 − 40e2x1 +2x2 +x1 −5x2 +10 x2 + 29e2x1 +2x2 +x1 −5x2 +10
∂x21
∂2 2 2 2 2 2 2

2 F (x) = 16e2x1 +2x2 +x1 −5x2 +10 x21 + 8e2x1 +2x2 +x1 −5x2 +10 x1 + 5e2x1 +2x2 +x1 −5x2 +10
∂x2
∂2 2 2
F (x) = (4x + 1) e2x +2y +x−5y+10 (4y − 5)
∂x1 ∂x2
∂2 2 2
F (x) = (4y − 5) e2x +2y +x−5y+10 (4x + 1)
∂x2 ∂x1

1
From the above working, we can now find our second order Taylor series approximation for F (x) about the
point x = [0, 0]t .
We get the following working:

1
Vector Function

F [0, 0] = 22026.4658

Gradient

∂
F ([0, 0]) = 22026.46579
∂x1
∂
F ([0, 0]) = −110132.329
∂x2

 
22026.46579 −110132.329

Hessian

∂2
F ([0, 0]) = 638767.508
∂x21
∂2
F ([0, 0]) = 110132.329
∂x22
∂2
F ([0, 0]) = −110132.329
∂x1 ∂x2
∂2
F ([0, 0]) = −110132.329
∂x2 ∂x1
 
638767.508 −110132.329
−110132.329 110132.329

Combined

Together, we get the following second order Taylor approximation from the form F (x) = F (x∗ )+∇F (x∗ )(x−
x∗ ) + 12 (x − x∗ )T ∇2 F (x∗ )(x − x∗ )T , where x∗ = [0, 0]:
 
  1 T 638767.508 −110132.329
F (x) = 22026.458 + 22026.46579 −110132.329 (X) + (X) (X)
2 −110132.329 110132.329
Which reduces down to:

F (x) = 22026.458 + 22026.46579x − 110132.329y + 0.5(638767.508x2 − 220264.658xy + 110132.329y 2 )

The stationary point of a function is found by any set of points that satisfy Eq. (8.27), ∇F (x)|x=x∗ = 0.
Thus any set of points that makes the gradient equal to zero is considered stationary.
For our second order taylor series approximation, we need to compute and find the gradient. It’s gradient
can be found by finding its partial derivates, as described earlier; however, this time we have our function in
matrix form. Our Taylor series can be represented
 by F (x) = c + dT x + 21 xT Ax, where c = 22026.458, dT =
  638767.508 −110132.329
22026.46579 −110132.329 , and A = . From Eq. 8.38, ∇F (x) = Ax + d.
−110132.329 110132.329

2
Therefore, we will get the following gradient:
   
638767.508 −110132.329 22026.46579
∇F (x) = X+
−110132.329 110132.329 −110132.329
 
638767.508x1 − 110132.329x2 + 22026.46579
∇F (x) =
−110132.329x1 + 110132.329x2 − 110132.329

The reduced form was given earlier as well, one could have taken the partial derivatives with respect to each
value of X and computed the same answer; however, matrix differentiation is a little bit more fun!
Solving for when these two equations are set to zero will result in x = [1/6, 7/6]. The process can be done
algebraically because the first entry contains −110132.329x2 and the bottom entry contains 110132.329x2 ;
thus, adding these two equations together will yield 528635.18x1 − 88108.86 = 0. Then one can solve for x1
and plug it back into either equation to solve for x2 .

3
2 2 2 2
For our original function, the gradient is ∇F (x) = [(4x1 +1)e2x1 +2x2 +x1 −5x2 +10 | (4x2 −5)e2x1 +2x2 +x1 −5x2 +10 ].
We can set these two partial derivates equal to each other and solve for x1 and x2 . The exponential term
on both sides is the same so it reduces down to 4x1 + 1 = 4x2 − 5 = 0, which can be solved algebraically to
find x1 = −0.25 and x2 = 1.25.
The stationary point for our function is x = [−0.25, 1.25] as ∇F ([−0.25, 1.25]) = [0, 0].

In a scalar function, the points that make the first derivative equal to zero were known as extrema, where
furthur tests were necessary to determine if these were maxima or minima. Whenever the first derivative is
equal to zero, that set of points indicate where the function has no slope.
In our multidimensional case, the first derivative is known as the gradient, where points that make it equal
to zero are known as stationary points. In part 3, we found the actual stationary points of our function, the
points where possible minima or maxima are observed; whereas in part 2 we found the stationary points of
our second order Taylor series centered about the position x = [0, 0].
Figure 1 shows our original function from bounds x1 = [−0.249, −0.251] and x2 = [1.249, 1.259]. We can see
that there appears to be a minima at the bottom of the sub plot, which is shown by the point. In Figure
2 we heave the same plot again but this time the view is inverted to give the reader another perspective of
the graph.
Figure 3 shows our second order Taylor series function from bounds x1 = [0.1656̄, 0.1676̄] and x2 =
[1.1656̄, 1.1676̄]. We can see that there appears to be a minima at the floor of the sub plot, which is
shown by the point. In Figure 2 we heave the same plot again but this time the view is inverted to give the
reader another perspective of the graph.

The code to create these plots is down below:

Use the following libraries:

library(purrr) # used for map2_dbl function

library(plotly) # used for 3D plotting

3
Figure 1: Original Function with stationary point

4
Figure 2: Original Function with stationary point (inverted view)

5
Figure 3: Taylor Series Expansion with stationary point

6
Figure 4: Taylor Series Expansion with stationary point (inverted view)

7
## Warning: package ’plotly’ was built under R version 3.6.3

## Loading required package: ggplot2

## Warning: package ’ggplot2’ was built under R version 3.6.3

##
## Attaching package: ’plotly’

## The following object is masked from ’package:ggplot2’:

##
## last_plot

## The following object is masked from ’package:stats’:

##
## filter

## The following object is masked from ’package:graphics’:

##
## layout

library(ggplot2) # used for plotting

Our functions

fun = function(x, y) { # our original function

exp(2*x^2+2*y^2+x-5*y+10)
}

taylor = function(x, y) { # our taylor expansion

X = matrix(c(x, y), nrow=2)
A = matrix(c(638767.508, -110132.329, -110132.329, 110132.329), ncol=2, byrow=TRUE)
d = matrix(c(22026.46579, -110132.329), ncol=2)
2206.458+d%*%X+0.5*t(X)%*%A%*%X
}

Code used to create the 3D plots

Original Function

# BOUNDS USED FOR 3D PLot

# pointsX = seq(-0.249, -.251, length=200)
# pointsY = seq(1.249, 1.251, length=200)

# BOUNDS USED FOR CONTOUR

pointsX = seq(-1, 0, length=200)# create a 200x1 vector of values for x axis
pointsY = seq(1, 2, length=200) # create a 200x1 vector of values for y axis
myGrid = expand.grid(pointsX, pointsY) # create 200x200 grid of points
z = map2_dbl(myGrid$Var1, myGrid$Var2, ~fun(.x, .y)) # maps all possible combinations

8
# of the grid through the function
x = myGrid$Var1
y = myGrid$Var2
myPlot = plot_ly(x=~x, y=~y, z=~z, type="mesh3d", intensity=~z,
colors = colorRamp(c("black","red","yellow","chartreuse3")))
myGrid$z = z
myPlot = add_trace(p=myPlot, z = c(fun(-0.25, 1.25)), x=c(-0.25), y=c(1.25), type="scatter3d")
# add another layer onto the 3D rendering, in this case a scatterplot
# of a single point, the stationary point, AKA minima
#myPlot

v = ggplot( myGrid,aes(x, y, z=z)) +geom_contour_filled()

v + geom_point(aes(x=-0.25,y=1.25), colour="red")

2.00

level
1.75 (0, 1000]
(1000, 2000]
(2000, 3000]
(3000, 4000]
1.50
y

(4000, 5000]
(5000, 6000]
(6000, 7000]
(7000, 8000]
1.25
(8000, 9000]

1.00

−1.00 −0.75 −0.50 −0.25 0.00

x

Here we have the contour plot of our original function color filtered by level with the global minimum
highlighted by the red point.

Taylor Series Expanson

# BOUNDS USED FOR 3D PLot

# pointsX = seq(1/6-0.001, 1/6+0.001, length=200)
# pointsY = seq(7/6-0.001, 7/6+0.001, length=200)

# BOUNDS USED FOR CONTOUR

pointsX = seq(-2, 2, length=200)# create a 200x1 vector of values for x axis

9
pointsY = seq(-2, 2, length=200) # create a 200x1 vector of values for y axis
myGrid = expand.grid(pointsX, pointsY) # create 200x200 grid of points
z = map2_dbl(myGrid$Var1, myGrid$Var2, ~taylor(.x, .y)) # maps all possible combinations
# of the grid through the function
x = myGrid$Var1
y = myGrid$Var2
myPlot = plot_ly(x=~x, y=~y, z=~z, type="mesh3d", intensity=~z,
colors = colorRamp(c("black","red","yellow","chartreuse3")))
myGrid$z = z
# add another layer onto the 3D rendering, in this case a scatterplot
# of a single point, the stationary point, AKA minima
myPlot = add_trace(p=myPlot, z = c(taylor(1/6, 7/6)), x=c(1/6), y=c(7/6), type="scatter3d")
# myPlot

v = ggplot( myGrid,aes(x, y, z=z)) + geom_contour_filled()

v + geom_point(aes(x=1/6,y=7/6), colour="red")

2
level
(−200000, 0]
(0, 200000]
1 (200000, 400000]
(400000, 600000]
(600000, 800000]
(800000, 1000000]
0
y

(1000000, 1200000]
(1200000, 1400000]
(1400000, 1600000]
(1600000, 1800000]
−1
(1800000, 2000000]
(2000000, 2200000]
(2200000, 2400000]

−2

−2 −1 0 1 2
x

Here we have the contour plot of our Taylor Series function color filtered by level with the global minimum
highlighted by the red point.

10