A short introduction to

robust optimization
Erick Delage
Dept. of decision sciences, HEC Montréal
CRC in decision making under uncertainty
 A production problem
• A company produces two kinds of drugs, DrugI and DrugII,
containing a speci c agent A, which is extracted from raw
materials purchased on the market. Its goal is to nd the
production plan that maximizes the pro t of the company.

2 /27
fi
fi
fi
 Linear programming
formulation
maximize 6200DI + 6900DII (100RI + 199.90RII + 700DI + 800DII)
RI,RII,DI,DII
subject to RI + RII  1000 (Storage)
90DI + 100DII  2000 (Manpower)
40DI + 50DII  800 (Equipment)
100RI + 199.9RII + 700DI + 800DII  100000 (Budget)
0.01RI + 0.02RII 0.5DI 0.6DII 0 (Agent A)
RI 0, RII 0, DI 0, DII 0,

• Optimal solution :
RI* = 0, RII* = 438 kg, DI* = 17 552 packs, DII* = 0 pack, optimal pro t = 8820$
• Optimal solution relies heavily on extracting 8,78 g of agent A and using all of it
in the production of Drug I (no left-over).
• If there is a 2% estimation error in conversion rate for this raw material, then we
are missing 0,18g of agent A and can only produce 17201 packs of Drug I.
Pro t drops to 6889$ (i.e. -22%)
3 /27
fi
fi
 Robust formulation
• If there is a 2% estimation error in conversion rate
for Raw II and 0.5% for Raw I, then it would be more
robust to replace the (Agent A) constraint with:
(0.995 · 0.01)RI + (0.98 · 0.02)RII 0.5DI 0.6DII 0

• The optimal solution would then be :

RI* = 878 kg, RII* = 0 kg, DI* = 17 467 packs, DII* =
0 pack, guaranteed pro t = 8295$ (-6%)
• This solution can be considered immuned to
estimation error of the conversion rates of Raw I and
Raw II
4 /27
fi
 Generalized need for robust optimization
416 416 Aharon Ben-Tal, Arkadi
Aharon
Nemirovski
Ben-Tal, Ar
Ben-Tal & Nemirovski. Robust solutions of linear programming problems contaminated with
uncertain data. Mathematical Programming, Seriesproblems
Table 2. NETLIB A,Table
88(3):411–424,
2. NETLIB
with bad nominal 2000.
problems
solutions
with bad nominal solutions

Problem a)
SizeProblem Sizea)
! = 0.01% ! !==0.01%
0.1% !=
!=0.1%
1% !
« small perturbations of Nbadb) Indexc) Nbadb)
Nbad Indexc)
Index Nbad
Nbad Index
Index Nbad

obviously uncertain data

80BAU3B
25FV47
2263 ×80BAU3B
9799
822 ×25FV47
1571
37 2263 × 9799
84
14 822 × 1571
16
177
37
28
14
842
84
162
16
177
364
28 35
842
8,420
162
1,620
364
35

coef cients can make the

ADLITTLE
AFIRO
57 ×ADLITTLE
97
28 ×AFIRO
32
57 × 97
28 × 32
2
1
6
5
27
12
658
550
7
2
2325 ×BNL2 2325 × 3489
nominal optimal solution
BNL2
BRANDY
3489
221 ×BRANDY
249 221 × 249
24
1
34
5
24
1
272 ×CAPRI 272 × 353
heavily infeasible andCYCLE
thus
CAPRI 353
1904 ×CYCLE
2857 2 1904 × 110
2857
10
52
39
1,100
110
1014
56
39
390
1,100
11,000
14
6
2172 ×D2Q06C 1072172 ×1,150
practically meaningless
E226 »
D2Q06C 5167
224 ×E226
282
5167
224 × 282
134
107 11,500
1,150 134
168
2
11,500
115,000
15
168
2
FFFFF800 525 ×FFFFF800
854 525 × 854 6 8 6
• Nbad : number of FINNIS
GREENBEA
498 ×FINNIS
614
2393 ×GREENBEA
5405
12 498 × 61410
13 2393 × 116
5405
63
12
30
13
104
10
1,160
116
63
30
97
37
104
1,040
1,160
11,600
97
37
constraints where there KB2
MAROS
44 ×KB2
41
847 ×MAROS
1443
5 44 × 41 27
3 847 × 14436
65
38
3
268
27
57
6
610
38 73
268
2,680
57
566
10
73
is more than 2% chance NESM
PEROLD
751 ×NESM
2923
626 ×PEROLD
1376
751 × 2923
6 626 × 1376
34 266 339
34
37
26 58
20
339
3,390
37
58
that violation is > 5% PILOT
PILOT4
1442 ×PILOT
3652
411 ×PILOT4
1000
16 1442 × 3652
50
42 411 ×210,000
1000
185
16
63
42
498
50
2,100,000
210,000
185
379
6375
498
4,980
2,100,000
21,000,000
379
75
PILOT87 2031 ×PILOT87
4883 86 2031 × 130
4883 433
86 1,300
130 433
990 1,300
13,000 990
• Index: maximum PILOTJA 941 ×PILOTJA
1988 4 941 × 1988
46 204 463
46 20 59 463
4,630 59
PILOTNOV 976 ×PILOTNOV
2172 4 976 × 2172
69 134 694
69 13 47 694
6,940 47
violation in % among all PILOTWE
SCFXM1
723 ×PILOTWE
2789
331 ×SCFXM1
457
61 723 ×12,200
2789
1 331 × 45795
69
61
31
122,000
12,200
946
95
69 69
311
122,000
1,220,000
946
9,460
69
11
constraints SCFXM2
SCFXM3
661 ×SCFXM2
914
991 ×SCFXM3
1371
2 661 × 91495
3 991 × 1371
95
62
93
946
95
946
95
621
932
946
9,460
946
9,460
21
32
SHARE1B 118 ×SHARE1B
225 1 118 × 257
225 1 2,570
257 11 5 /27
2,570
25,700 1
fi
« In applications of LP, there exists a real need
of a technique capable of detecting cases
when data uncertainty can heavily affect the
quality of the nominal solution, and in these
cases to generate a reliable solution, i.e. one
which is immuned against uncertainty. »

-Ben-Tal & Nemirovski, 2000

6 /27
History of robust optimization
• « maximin » introduced by Abraham Wald in 1945 to
manage risk and is in uenced by zero-sum games
(von Neumann & Morgenstern, 1944)
• Allen Soyster (1973) proposed robustifying
constraints of a linear program by considering
interval uncertainty.
• Idea was considered too conservative for a long time
• In 1998, Ben-Tal & Nemirovski gave a second life to
the framework in context of convex optimization by
exploiting ellipsoidal uncertainty

7 /27
fl
Rebirth of robust optimization

8 /31
 Why ellipsoidal sets are less
conservative than boxes?
• In high dimension, appoint in the • In high dimension, vectors
box can be up to n times composed of i.i.d. components
further from zero than any point lie with high probability in the
in the ellipsoid that covers each following hyper-orbit
[-B, B] interval
(B, ..., B)
B
✓ ◆ ( ,... )
B B
p , ..., p
n n

-B B
B p
B n

-B p p p p
n o( n) n + o( n)
9 /27
 Possible reasons for rebirth
of robust optimization
• Characterizing uncertainty with a set of possible
realization can be easier than with a multivariate
distribution
• Some uncertainty sets have the following
properties:
• they reduce conservativeness
• they lead to reformulations that are easier to solve than SP
• they provide statistical guarantees.
• The development of more powerful computers and
in particular of fast interior point methods.

10 /27
 De ning robust optimization

(Nominal problem) maximize h(x, z)

x
subject to gj (x, z)  0 , 8 j = 1, ..., J ,

(Robust counterpart) maximize min h(x, z)

x z2Z
subject to gj (x, z)  0 , 8 z 2 Z, , 8 j = 1, ..., J .

11 /27
fi
 Reformulating a robust optimization
problem using duality theory
• ⇢
Consider the robust LP
>
RLP := maximize min c(x) z + d(x) ,
x2X z:P zq
• Reformulating inner problem
⇢ using Lagrangian
RLP ⌘ maximize min max c(x)> z + d(x) >
(q P z)
x2X z 0
• LP strong duality ⇢
> >
RLP ⌘ maximize max min c(x) z + d(x) (q P z)
x2X 0 z
• Proper book-keeping leads to compact reformulation
RLP ⌘ maximize d(x) q> ,
x2X , 0
>
subject to P + c(x) = 0
12 /27
 Robust production problem
maximize 6200DI + 6900DII (100RI + 199.90RII + 700DI + 800DII)
RI,RII,DI,DII
subject to RI + RII  1000
90DI + 100DII  2000
40DI + 50DII  800
100RI + 199.9RII + 700DI + 800DII  100000
0.01(1 + 0.005z1 )RI + 0.02(1 + 0.02z2 )RII 0.5DI 0.6DII 0 , 8z 2 Z
RI 0, RII 0, DI 0, DII 0,
• Box uncertainty would assume:
2
Z := {z 2 R | max |zi |  1}
i
• Ellipsoidal uncertainty would assume: X
Z := {z 2 R2 | zi2  1}
i
• Budgeted uncertainty would assume:
X
Z( ) := {z 2 R2 | max |zi |  1 , |zi |  }13 /27
i
 Robust production problem
maximize 6200DI + 6900DII (100RI + 199.90RII + 700DI + 800DII)
RI,RII,DI,DII
subject to RI + RII  1000
90DI + 100DII  2000
40DI + 50DII  800
100RI + 199.9RII + 700DI + 800DII  100000
0.01(1 + 0.005z1 )RI + 0.02(1 + 0.02z2 )RII 0.5DI 0.6DII 0 , 8z 2 Z
RI 0, RII 0, DI 0, DII 0,
• Box uncertainty leads to :
0.01RI + 0.02RII 0.5DI 0.6DII 0.00005|RI| + 0.0004|RII|
Ellipsoidal uncertainty leads to :p
•
2 2 2 2
0.01RI + 0.02RII 0.5DI 0.6DII 0.00005 RI + 0.0004 RII
• Budgeted uncertainty leads to (with new « t » variable):
0.01RI + . . . 0.6DII t + max(0; |0.00005RI| t) + max(0; |0.0004RII| t)
14 /27
RSOME implementation (see Google Colab)
Chen et al. Robust Stochastic Optimization Made Easy with RSOME. MS, 2020.

15 /31
RSOME implementation (see Google Colab)
Chen et al. Robust Stochastic Optimization Made Easy with RSOME. MS, 2020.

Box uncertainty set

16 /31
RSOME implementation (see Google Colab)
Chen et al. Robust Stochastic Optimization Made Easy with RSOME. MS, 2020.

Ellipsoidal
uncertainty set

17 /31
RSOME implementation (see Google Colab)
Chen et al. Robust Stochastic Optimization Made Easy with RSOME. MS, 2020.

Budgeted
uncertainty set

18 /31
 Distributionally robust
optimization
• Robust optimization can also be applied on top of a
stochastic program when one is not comfortable about
making a distribution assumption:
maximize min EF [h(x, z)]
min
x2X 2D
FF2D

where « D » is the set of all plausible distributions for « z ».

• For example, one might consider the following distribution
set: 8 9
< PF (z 2 Z) = 1 =
>ˆ 1
D := F (EF [z] µ̂) ⌃ (EF [z] µ̂)  1
: ˆ ;
EF [(z µ̂)(z µ̂)> ] 2 ⌃
19 /27
 SP vs. DRO : a portfolio
selection case study
100
Boeing
80 Motorola
stock prices

Dow Chemical Company

60
Merck & Co., Inc.
40

20
• We use both methods to select portfolios of four
0
stocks
1992 to be traded
1994 1996 daily
1998 between
2000 2001 2004
2002 and 20082006 2008 2
year
• Both methods maximize expected daily returns and
use the last 30 days returns to build an uncertainty
model for next daily return (distribution vs. distribution
set based on moments)
• 300 experiments are performed
20 /27
 SP vs. DRO : a portfolio
selection case study
• Below we present statistics of cumulated returns obtained
using both methods
• Curves are the average cumulated wealth while bars
indicate 5th and 95th percentiles
1.4
Distributionally robust optimization
1.2 Stochastic programming

1.5
1

1.25
Wealth

0.8

Wealth
1
0.6
0.75

0.4
0.5

0.2
2001 2002 2003 2004 2004 2005 2006 2007
Year Year

*) Delage & Ye, Operations Research, 2010. 21 /27

Preference robust optimization

• Expected utility theory states that rational risk

averse decision makers should optimize:

maximize E[u(h(x, z))] ,

x2X

where u( ) is a non-decreasing concave function

• In practice, how do we know what utility function to

use?

22 /27
Can we use risk assessment surveys?
Grable & Lytton, Financial Services Review (1999)

• You have just nished saving for a « once-in-a-lifetime »

vacation. Three weeks before you plan to leave, you
lose your job. You would:
• Cancel the vacation
• Take a much more modest vacation
• Go as scheduled, reasoning that you need the time
to prepare for a job search
• Extend your vacation, because this might be your
last chance to go rst-class

• You are on a TV game show and can choose one of the

following. Which would you take?
• $1,000 in cash
• A 50% chance at winning $ 5000
• A 25% chance at winning $ 10,000
• A 5% chance at winning $100,000
23 /27
fi
fi
Preference robust optimization
• Pairwise comparisons of gambles can be used to
construct an uncertainty set of plausible utility
functions:
8 9
>
> u(·) is non-decreasing >
>
< =
u(·) is concave (i.e. risk aversion)
U := u : R ! R 0
>
> u (·) is convex (i.e. prudence) >
>
: ;
E[u(Wk )] E[u(Yk )] 8 k = 1, . . . , K

• Based on this set, different u( ) leads to different

expected utility measurement, and different certainty
equivalent amounts:
1
u(·) 2 U ! E[u(h(x, z))] ! u (E[u(h(x, z))])
24 /27
Preference robust optimization
• One can try to maximize the worst-case certainty
equivalent:
maximize min u 1(E[u(h(x, z))]) ,
x2X u2U

• When h(x,z) is concave in x, objective function is

quasiconcave and reduces to:
1
maximize t subject to u (E[u(h(x, z))]) t 8u 2 U
x2X ,t

• Equivalently one can solve:

max t subject to max min E[u(h(x, z))] u(t) 0
t x2X u2U
*) Armbruster & Delage, Management Science, 2015.
Delage & Li, Management Science, 2017. 25 /27
A new tool for interacting with investors

26 /27
 MATH80624A : Quantitative risk
management using robust optimization
• Coming in Winter 2023
• Applications:
• Logistics • Electrical engineering
• Finance • Aerospace
• Marketing • Data mining
• etc.
• Topics:
• Robust counterpart of Linear Programs
• Data driven uncertainty set design
• Robust nonlinear programming
• Adjustable robust linear programming
• Distributionally robust optimization
• Globalized robust counterparts
• Pareto ef ciency in robust optimization

• More details available in Lecture Notes (url)

27 /27
-­
fi
 Optimization days 2022:
Sessions on Optimization under Uncertainty

• Optimization days is held in person May 16-18

• Sessions that I am organizing:
• Frontiers of optimization under uncertainty
• Applications of distributionally robust optimization
• Contextual optimization under uncertainty
• Applications of stochastic programming
• More info on the conference :
symposia.gerad.ca/jopt2022/fr
• Deadline to submit talks: April 8th

28 /27