Indr 460

Operations Research
Applications

Nonlinear Programming
Models

Koc University – Spring 2019

1
Nonlinear Programs (NLP)

2
How Nonlinear Terms Arise
p Engineering Considerations

p Risk – Variance of returns

p Increasing or decreasing returns to scale

p Demand dependent on price

p Utility functions

3
Example: Producing Tires
p Tires are made by combining rubber, oil, and
carbon.
p Tires must have a “hardness” of between 25 and 35.
p Tires must have an “elasticity” of at least 16
p Tires must have a “tensile strength” of at least 12
p To make a set of four tires, we require 100 pounds
of total material (rubber, oil, and carbon).
◦ At least 50 pounds of carbon
◦ Between 25 and 60 pounds of rubber

4
Example: Producing Tires
p Chemical Engineers tell you that the tensile strength,
elasticity, and hardness of tires made of r pounds of
rubber, o pounds of oil, and c pounds of carbon are
as below

n Tensile Strength
n Elasticity
n Hardness =

p The cost of rubber is $.04/pound, oil costs $.01 per

pound, and carbon costs $.07/pound
5
Model

Min: !. !#$ + !. !&' + !. !()

st: 0.1, + 0.06. − 0.31 + 0.01,. + 0.005. 3 + 0.0011 3 ≤ 1
0.1, + 0.06. − 0.31 + 0.01,. + 0.005. 3 + 0.0011 3 ≥ −9
0.35, + 0.04. − 0.002, 3 ≥ −1
0.1. + 0.001. 3 ≤ 0.5
,, ., 1 ≥ 0

6
Graphical Analysis

7
Optimal Solution

8
Graphical Analysis

9
Local vs. Global Optima

10
Convex Functions

11
Concave Functions

12
Convex or Concave

13
What functions are convex?

14
Convex functions
p If f and g are convex functions, then so are
m(x) = max{f(x),g(x)} and h(x) = f(x) + g(x).

p If f and g are convex functions and if g is non-

decreasing, then h(x) = g(f(x)) is convex.

15
Checking convexity of functions
p A differentiable function of one variable is
convex on an interval if and only if its derivative
is monotonically non-decreasing on that interval

p A twice differentiable function of one variable is

convex on an interval if and only if its second
derivative is non-negative there

p More generally, a continuous, twice

differentiable function of several variables is
convex on a convex set if and only if its Hessian
matrix is positive semidefinite on the interior of
the convex set. 16
Convex functions and sets
p For a convex function ƒ, the sublevel sets
{x | ƒ(x) < a} and {x | ƒ(x) ≤ a} with a ∈ R are
convex sets.
p However, a function whose sublevel sets are
convex sets may fail to be a convex function.
p A function whose sublevel sets are convex is
called a quasiconvex function (ex: S shape
functions)

17
Convex Functions and Convex Sets

For example:

18
Recognizing Convex Feasible Regions

19
Local Maximum (Minimum)
Property

20
Verifying Optimality
p We are interested in determining conditions under
which we can verify that a solution is optimal.

p Algorithms for nonlinear programming work to find

points that satisfy these conditions

p When faced with a problem that you don’t know how

to handle, write down the optimality conditions

p Often you can learn a lot about a problem, by

examining the properties of its optimal solutions.

21
Optimizing functions of a single
variable
Consider optimizing the following function (for a
variable

What is necessary for a point x to be an optimal

solution?

22
Example:

23
Is that all we need?

24
Example

25
Sufficiency

26
Constrained Optimization
Now we consider the following problem for a variable

p There are three cases for where an optimal solution

might be

27
Breaking it down
p If 0 < x < u, then the necessary and sufficient
conditions for optimality are the same as the
unconstrained case

p Remembering that the derivative is the rate of

change of the function, we can easily see that

28
KKT Conditions
p How do these conditions generalize to optimization
problems with more than one variable?

p The intuition: If a constraint holds with equality (is

binding) at an optimal point, then the gradient of the
objective function at that point must be pointing in a
way that would improve the objective in the direction
of the gradient of the constraint.

p Formally: At the optimal point, the gradient of the

objective function must be a linear combination of the
gradients of the binding constraints.

p Note: “KKT” stands for Karush-Kuhn-Tucker.

29
30
The Canonical Problem

31
KKT Conditions
p if ( ) is an optimal solution, then

p If a constraint is not binding, it’s “weight” must be 0.

32
KKT Conditions

33
Returning to the example

34
KKT Conditions
Primal Feasible:

Dual Feasible:

p Complementary Slackness:

35
Checking Optimal Solutions
p Let’s check whether or not the necessary (KKT)
conditions for optimality are satisfied by

p Primal Feasible

p Dual Feasible

36
Checking Optimal Solutions
p Complementary Slackness

p is potentially an optimal solution

37
Why “Potentially”?
p The KKT Conditions are only necessary (under
some constraint qualifications) – not sufficient.

p The sufficiency conditions (for maximization):

n for global optimality, f(x) must be concave,

and must be convex for all

(The objective function must have its one and

only “hump” in the right direction, and the
solution region must be a convex set).

38
Finance Applications
p Portfolio selection (quadratic programming)
p Financial planning (multi-period)
p Capital budgeting (integer programming)
p Capital asset pricing (nonlinear optimization,
regression)
n predict return on a stock
p Options pricing
p Stock hedging

39
Portfolio Selection
p Given a sum of money to invest,
p Decide on how to invest the money among a
portfolio of shares and stocks
- In asset allocation models returns are typically
represented by random variables
p Objectives:
n Maximize expected rate of return from the investment
n Minimize risk, that is the variance of the return
p Constraints: Investment guidelines

40
Portfolio Selection
p Two approaches:

n Minimize Risk (Variance of the return from the

portfolio)
s.t. Expected Return >= bound

n Maximize Expected Return

s.t. Risk <= bound

41
Portfolio Selection
p Let 1…n be assets considered for investment
p Let xi be percent of total fund invested in asset i
p Ri, annual return from each asset i is a random
variable with given E(Ri)=µi and Var(Ri)=σi2
p R = R1x1 + … +Rnxn , total return is also a r.v.
p E(R)= µ1 x1 + … + µn xn
p Var(R)= σ12 x12 + … + σn2 xn2 +covariance
terms
Var( R) = å xi2s i2 + å x x s s CORR(i, j)
i j i j
i i , j ,i ¹ j
42
Example
From Bertsimas and Freund

p Marathon Investments is assembling a portfolio of stocks

in Advent Communications, General Space Systems (GSS),
and Digital Devices. Estimated data:

Asset Annual Std Correlation

Name Expected Dev
Return of Advent GSS Digital
Return

Advent 11% 4.00%

GSS 14% 4.69% 0.160

Digital 7% 3.16% -0.395 0.067
43
Example
p Marathon would like to determine what fraction
of its investment dollars to invest in each of the
stocks so that
p Expected return of the portfolio is at least 11%
p The risk of the portfolio measured by the std
deviation of the total return is minimized

44
Model
A= fraction of marathon’s investment in Advent
G= fraction of marathon’s investment in GSS
D= fraction of marathon’s investment in Digital

E(R)= 0.11A + 0.14G + 0.07 D

Var(R)=16A2+ 22G2 + 10D2 + 6AG + 2GD – 10AD

45
Model
p Objective
n !"# 16&' + 22 * ' + 10 , ' + 6&* + 2*, − 10&,
p Target Return constraint
n 0.11 & + 0.14 * + 0.07 , ≥ 0.11(& + * + ,)
p Budget limit constraint
n &+*+, ≤1
p Non-negativity
n &, *, , ≥ 0

46
Solution with Excel Solver
Marathon Investment Example

Asset Expected Annual Correlation

Name Return % Std Dev % Advent GSS Digital
Advent 11.00 4.00
GSS 14.00 4.69 0.16
Digital 7.00 3.16 -0.395 0.067

Decision Variables Advent GSS Digital

0.376935 0.356037 0.267028

Total Expected Return 11

Standard Deviation 2.400819

Constraints
LHS RHS
fractions 1 1
47
target return 11 11
Sensitivity Report
Adjustable Cells
Final Reduced
Cell Name Value Gradient
$E$12 Advent 0.37693498 0
$F$12 GSS 0.356037145 0
$G$12 Digital 0.267027875 0

Constraints
Final Lagrange
Cell Name Value Multiplier
$D$20 fractions LHS 1 -2.893748581
$D$21 target return LHS 10.99999993 0.48132437

48
Notes on Solver Output
p When Solver says it found a solution, it is a
local optimum
p Lagrange Multiplier column in the Sensitivity
Report lists the shadow price of each constraint
p If RHS changes by Δ, then optimal objective
value increases approximately by Δ times the
Lagrange Multiplier, for small Δ.
p No range information is available

49
Conclusions
Be cautious with nonlinear functions:

p Recognize nonlinearity in functions

p Check if nonlinear terms can be linearized
p Recognize convexity/concavity of the objective
function and the convexity of the region
p Be aware of local/global optimality
p Be aware of the limitations of the commercial
solvers and what they can provide

50