Operations Research

Primal/Dual LP Problems (Main Ideas and Examples)

Assume that all primal constraints are equations with non-negative right-hand side,
and all the variables are non-negative. Then, we have the following rules for constructing
the dual problem

Dual problem
Primal Problem objective
Objective Constraints type Variable sign
maximization minimization unrestricted
minimization maximization  unrestricted

Key ideas

- Assign a dual variable for each primal (equality) constraint.

- Construct a dual constraint for each primal variable.

- The (column) constraint coefficients and the objective coefficient of the jth primal
variable respectively define the left-hand an the right-hand sides of the jth dual con-
straint.

- The dual objective coefficients equal the right-hand sides of the primal constraint
equations.

Example 1

• Primal problem • Primal in equation form

maximize z = 5x1 + 12x2 + 4x3 maximize z = 5x1 + 12x2 + 4x3 + 0x4

subject to x1 + 2x2 + x3  10 subject to x1 + 2x2 + x3 + x4 = 10
2x1 x2 + 3x3 = 8 2x1 x2 + 3x3 + 0x4 = 8
x 1 , x2 , x3 0 x 1 , x2 , x 3 , x4 0

• Dual • Dual problem

minimize w = 10y1 + 8y2 minimize w = 10y1 + 8y2

subject to y1 + 2y2 5 subject to y1 + 2y2 5
2y1 y2 12 2y1 y2 12
y1 + 3y2 4 y1 + 3y2 4
y1 + 0y2 0 y1 0
y1 , y2 unrestricted y2 unrestricted

RSA 1
Operations Research

Example 2

• Primal problem • Primal in equation form

minimize z = 15x1 + 12x2 minimize z = 15x1 + 12x2 + 0x3 + 0x4

subject to x1 + 2x2 3 subject to x1 + 2x2 x3 + 0x4 = 3
2x1 4x2  5 2x1 4x2 + 0x3 + x4 = 5
x 1 , x2 0 x 1 , x 2 , x3 , x4 0

• Dual • Dual problem

maximize w = 3y1 + 5y2

subject to y1 + 2y2  15 maximize w = 3y1 + 5y2
2y1 4y2  12 subject to y1 + 2y2  15
y1 + 0y2 0 2y1 4y2  12
0y1 + y2 0 y1 0
y 1 , y2 unrestricted y2  0

Example 3

• Primal problem • Primal equation form (here x1 = x1 x+

1)

maximize z = 5x1 + 6x2 maximize z = 5x1 5x+ 1 + 6x2

subject to x1 + 2x2 = 5 subject to x1 x+ 1 + 2x2 = 5
x1 + 5x2 3 +
x1 + x1 + 5x2 x3 = 3
4x1 + 7x2  8 4x1 4x+ 1 + 7x2 + x4 = 8
x1 unrestricted, , x2 0 x 1 , x+1 , x2 , x 3 , x4 0

• Dual • Dual problem

minimize z = 5y1 + 3y2 + 8y3

subject to y1 y2 + 4y3 5 minimize z = 5y1 + 3y2 + 8y3
y1 + y2 4y3 5 subject to y1 y2 + 4y3 = 5
2y1 + 5y2 + 7y3 6 2y1 + 5y2 + 7y3 6
y2 0 y2  0
y3 0 y3 0
y1 , y2 , y3 unrestricted y1 unrestricted

RSA 2
Operations Research

Following the rules listed above, we can use matrix-vector notation to easily find the dual
of any linear programming problem (written in standard form).

Primal problem Dual problem

maximize cT x minimize bT y
subject to Ax = b subject to AT y c
x 0

Here A is a m ⇥ n matrix, c, x 2 Rn , and Here y 2 Rm (dual variable)

b 2 Rm .
m constraints and n decision variables. n constraints and m decision variables.

Primal problem Dual problem

minimize cT x maximize bT y
subject to Ax = b subject to AT y  c
x 0

Some useful properties

1. Any feasible solution to the dual problem gives a bound on the optimal objective function
value in the primal problem.

2. Understanding the dual problem leads to specialized algorithms for some important
classes of linear programming problems. Examples include the transportation simplex
method, the Hungarian algorithm for the assignment problem, and the network simplex
method.

3. The dual can be helpful for sensitivity analysis. Changing the primal’s right-hand side
constraint vector or adding a new constraint to it can make the original primal optimal
solution infeasible. However, this only changes the objective function or adds a new
variable to the dual, respectively, so the original dual optimal solution is still feasible
(and is usually not far from the new dual optimal solution).

4. The dual variables give the shadow prices for the primal constraints. Suppose you have
a profit maximization problem with a resource constraint i. Then the value yi of the
corresponding dual variable in the optimal solution tells you that you get an increase
of yi in the maximum profit for each unit increase in the amount of resource i.

5. Sometimes the dual is easier to solve. A primal problem with many constraints and
few variables can be converted into a dual problem with few constraints and many
variables (the fewer the constraints, the fewer computations required in each iteration
of the simplex method).

RSA 3
Operations Research

6. The dual can be used to detect primal infeasibility. If the dual is a minimization
problem whose objective function value can be made as small as possible, and any
feasible solution to the dual gives an upper bound on the optimal objective function
value in the primal, then the primal problem cannot have any feasible solutions.

MATLAB and Optimization

1. If you are not familiar with MATLAB please click the following links and watch the
tutorial videos:

(a) Video 1 : Getting Started with MATLAB

http://www.mathworks.com/videos/getting-started-with-matlab-68985.html
(b) Video 2 : Writing a MATLAB Program
http://www.mathworks.com/videos/writing-a-matlab-program-69023.html

2. Recall the Klee and Minty linear programming problem. In its general form, this
LP problem is given by

max 2n 1 x1 + 2n 2 x2 + ... + 2xn 1 + 1xn

subject to 1x1 + +2n 1 x2 + ... + 44xn 1 + 1xn  5
4x1 + 1x2 +4x2 4xn 1 + 4xn 1 + 1xn  52
8x1 + 4x2 + 1x3 +4xn 1 4xn 1 + 1xn  53
.. n 1 .
.+2 x2 + ... + 4xn 1 + 1xn ..
2n x1 + 2n 1 x2 + ... + 4xn 1 + 1xn  5n
xi 0, i = 1, ..., n

The Tableau Simplex Method, starting at x = (0, 0, ...0)T , is known to visit all ex-
treme points in this LP.
We can use the MATLAB built-in solver for linear programming (LP) problems (this
is part of the Optimization Toolbox by MathWorks).
http://www.mathworks.com/help/optim/ug/linear-programming-algorithms.html
We use the function linprog in this example.

3. Some things to try:

(a) Run the MATLAB file (.m file) and verify the solution to the Klee and Minty
problem for n = 4.
(b) Compare the running time and number of iterations for both methods, simplex
and interior-point: in the .m file, modify the line that declares the method to
be used

RSA 4
% Rei Sanchez-Arias. WIT
% Spring 2016. MATH 3700 OR
% Linear Programming Example(LP)
% Goal: use MATLAB command linprog to solve the Klee and Minty problem
% Type help linprog for instructions or visit
% http://www.mathworks.com/help/optim/ug/linprog.html
%
% Test Problem: (n = 4)
% maximize 8*x_1 + 4*x_2 + 2*x_3 + x_4
% subject to x_1 <= 5
% 4*x_1 + x_2 <= 25
% 8*x_1 + 4*x_2 + x_3 <= 125
% 16*x_1 + 8*x_2 + 4*x_3 + x_4 <= 625
% x_1 , x_2 , x_3 , x_4 >= 0

% Cost function: recall linprog solves a minimization problem

f = [-8 ; -4; -2; -1];
% Right-hand side:
b = [5; 25; 125; 625];
% Matrix A (constraints) nxn matrix for this LP
A = [1 0 0 0; 4 1 0 0; 8 4 1 0; 16 8 4 1];
% Lower-bounds (non-negativity constraint)
lb = zeros(4,1);

% Optimization solver options: ('simplex' or 'interior-point')

method = 'interior-point';
options = optimoptions(@linprog,'Algorithm',method);

% Find solution x, f(x), and number of iterations

ub = [];
Aeq = [];
beq = [];
x0 = [];

% Start timer
tic
[x,fopt,exitflag, output,lambda] = linprog(f,A,b,Aeq,beq,lb,ub,x0,options);
% Stop timer
total_time = toc;
x

fprintf('***************************************\n\n');
message = strcat(['Optimal point x found. Method used: ', ' ',method]);
fprintf(strcat(message, ' algorithm\n'));
fprintf('f(x) = %f, after %d iterations \n', -fopt,output.iterations)
% Notice that the solution that MATLAB returns must be multiplied by -1
% for our maximization problem
fprintf('Time : %f seconds\n', total_time);
fprintf('***************************************\n\n');
Introduction to MATLAB • Logical Operators

& and ~ not | or

• Constants
In Matlab you don’t need type declaration. To create a constant just type
name=value where name is the name of the constant and value is the value you
MATLAB stands for MATrix LABoratory. It is developed by The Mathworks, Inc. want for it. To use a constant you created just type the name of the constant.
(http://www.mathworks.com). Matlab is an interactive, integrated, environment To change the value of the constant just type name=new value
for numerical computations, symbolic computations, and scientific visualizations.
It is a high-level programming language. Examples:
A = 3 s = ’hi’ f = 1e-4 alpha = 5.647

Also Matlab is case sensitive; if you try to use for example the constant f but you
type F it will tell you that the variable does not exist.

• Quitting Matlab • Operations with Constants

To quit Matlab just type quit in the command window. Caution: if you do this You can do operation with constants just like you do them in a calculator. You
everything that you had typed in the command window will be lost. only need to have the constants created before you want to do the operations.
• Runaway or Endless Computation Matlab does not give you an error when you want to perform operations on
A runaway or endless computation happens when you have a program that would different type of constants. You can also save the result in a new constant.
not stop of that got stuck. To stop programs like this just use ctrl + c.
Example
• Help D = A + alpha in this case D = 8.6470
To get help just type help in the command window and you will have a list of the E = A*alpha in this case E = 16.9410
topics inside help. If you want help in a specific function type help
function_name and it will give you a short description of the function. (for • Vectors
example, help factorial). If you feel you need more help click the icon ? on To create a vector just type name=[V1; V2; … Vn] where name can be any
the Matlab window. name you want for your vector and V1, V2, …, Vn are the values of your
vector. You can also create vector by typing
• Matlab Special Characters name=[lower:increment:upper] which creates a vector with values from
; Suppress printing * Multiplication the lower to upper limits. To access an element in a vector just type
% Comments / Division name(index) where index is the location of the element. To add elements or
+ Addition \ Solution to A*x=b change a value from the vector just type name(index)=new value
- Subtraction … Continue statement on next line
Examples
• Relational Operators
If V=[1;2;3;4] then V(2) will return 2 or V(2:4) will return the elements 2
< less than >= greater than to 4 in this case 2 3 4
> greater than == equal
<= less than or equal ~= not equal Let T=1:10 (if you don’t declare any increment the increment will be of one). T
will have the numbers from one to ten.
If U=10:-1:1 then U will have the numbers from ten to 1
1
• Operations with Vectors • Operations with Matrices
For any operation you want to perform between vectors the vectors must have
the same dimensions, in other words, the same number of rows and columns. For any operation you want to perform between matrices the matrices must have
To add or subtract two vectors just type vector1 + or – vector2. the appropriate dimensions (as defined in matrix algebra). To add or subtract two
Example matrices just type matrix1 + or – matrix2.
If V1=[1;2;3;4] V2=[2;4;5;7] V3=V1+V2 V4=V1-V2 then Example
V3=[3;6;8;11] and V4=[-1;-2;-2;-3] If m1=[1 2; 3 4] m2=[2 4; 5 7] m3=m1+m2 m4=m1-m2
Then m3=[ 3 6 and m4=[ -1 -2
The symbol ‘ works as the transpose operator. Then since, V1 is a column vector, 8 11] -2 -3]
V1’ will be a row vector. To multiply vectors remember that their dimensions To multiply two matrices just type matrix1 * matrix2.
must agree. Example
If m1=[1 2; 3 4] m2=[2 4; 5 7] m5=m1*m2
Example Then m5=[ 12 18
26 40 ]
If V1=[1;2] V2=[2;4] V5=V1*V2’ then V5=[ 2 4
4 8] • If Statement
and V1’*V2 = 10 The if statement checks if the conditional statement is true or false, if true it will
execute the commands if false the statement will not be executed and the
• Matrices program will go to the elseif clause of to the else clause if neither of this two are
You can create matrices in different ways. You can put a colon or a space in present it will go to the end. The format for an if statement is:
between elements and you can put a semicolon or hit the return button to
indicate a new row. You always have to end and start a matrix with a bracket. if condition (condition must include a relational operator)
Statements
Example end (every if statement must have an end)
A=[1 2 3 or A=[1,2,3;4,5,6] or A [1 2 3; 4 5 6] or A=[1,2,3
4 5 6] 4,5,6] Example
To access an element in a matrix just type the name of the matrix and the index of n=3 n=3
the element. if n<6 if n<6
Example x = n^2; OR x = n^2;
A(1,1)=1, A(2,3)=6 n = 5; n = 5;
end elseif n > 10
To access an entire row of a matrix just type name(row number, : )
x = 1;
Example n = 0;
B=A(1, : ) B = [1 2 3] will have row one else
To access an entire column just type name( : , column number) x = 0;
end
Example
C=A( : ,3) then C = [3 6] will have the third column • For Loop
To obtain a part of the matrix just type name(row indexes, column
indexes) The for loop repeats the group of statements a predetermined fixed number of
Example times. The format for a for loop is:
D=A(1:2,2:3) D = [ 2 3
5 6 ]

2
 for i=limit1:inc:limit2 (where increment (inc) can be
positive ornegative. If inc is not defined, the -------------------------------------------------------------------------------------------------------------
default increment is one) % Example Simple Newton Method to find x such that
Statements (It will repeat executing f(x)=0
end the statements until limit1 reaches limit2)
x = 3; % initial point
f = (x^2)-1; % Original function
If limit1 is equal to limit2 the statement will still be executed once.
df= 2*x; % First derivative
Example iter = 0;
for i=1:5 for i=5:-.5:n while iter<100 % maximum number of iterations
x=2^i; OR x=2^i;
end end if abs(f)<1e-6,
break
end % you got the solution
• While Loop
The while loop executes the statements while the condition is true. If the deltax = -f/df; % solving the Newton step
condition is false the statements will not be executed. When doing a while loop x = x + deltax; % update
always make sure that inside the loop there is a statement that will eventually
make the condition false, else you will have a runaway computation. The format iter = iter+1; % updating iterations
for the while loop is: f = x^2-1; % Evaluate the function at the
% current step
while condition df = 2*x; % Evaluate the first derivative
Statements
End end

x % To get the last value of x

Example
-------------------------------------------------------------------------------------------------------------
n = 5;
while n <= 15
x = 2*n; This program will be saved as newton1.m, to run the program just type
n = n+1; % this ensure that n will eventually be greater than 15 newton1 at the command window. Make sure the directory in the command
end % making the condition false window is the same as the directory where you saved your program.

(anything you write right after a % sign will be considered a comment) • Storing Data

• M-Files When doing a program it may be necessary to store the value of different
variables at iterations.
An M-file is a file where you can put a sequence of statements and save them on a To do this you can use fid = fopen( ‘filename’ ,’ Permission’ )
disk. They are called M-files because they must have the file type ".m" as the last opens the file filename in the mode specified by permission. Permission can be:
part of their filename. M-files are useful when you need to execute a series of
statements at the same time and when you need to edit multiple commands. 'r' read 'r+' read and write (do not create)
Inside an m-file you can have if statements, loops and graphs among other things. 'w' write (create if necessary) 'w+' create for read and write
'a' append (create if necessary) 'a+' read and append

3
This will create and open the file where you will be storing the data. To store the 2 1.133333 0.284444
data you will use fprintf(fid, ‘ format’ ,variables) where fid 3 1.007843 0.0157478
has been initialized before to be the file you will be using, variables will be the 4 1.000031 6.1037e-005
name of the variables you want to save and format can be: 5 1.000000 9.31323e-010

%c Single character %f Fixed-point notation • Functions

%i integer notation %g more compact of %e or %f. A function is a type of M-file and has the format:
%e Exponential notation % String of characters
function [output1,…,outputn] = filename(input1,…, inputn)
You can also specify the spacing you want with:
Functions are useful when you want to make a program more general by being
\b Backspace \n New line \t Horizontal tab able to change some parameters when you execute the program instead of having
to change the context of the program. Also a function can call another function.
The name of the function has to be the same as the name of the file.
------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------
%Example Newton Method with data storing % Example general Newton method

x = 3; % Initial point function [iter, sol] = newton3(x)

f = (x^2)-1; % Original function % input :
% x the initial guess
df= 2*x; % First derivative
% output :
iter = 0; % iter : the number of iterations it took to converge
fid= fopen('results.txt', 'w'); % sol : the approximate solution
% fid is the name of the file Iter = 0;
fprintf(fid, 'iter\t x\t\t f(x)\n'); [f,df] = func(x);
% title of the columns in your table fid = fopen('results.txt', 'w');
while iter<100 % maximum number of iterations fprintf(fid, 'iter\t x\t\t f(x)\n');
if abs(f)<1e-6, break ,end % you got the solution %the title of the columns in your table
deltax = - f/df; % solving the Newton step while iter<100 %maximum number of iterations
x = x + deltax; % update if abs(f)<1e-6, break ,end
iter = iter+1; % updating iterations %you got the solution
f = x^2-1; deltax = -f/df; %solving the Newton step
% Evaluate function at current x x = x+deltax; %update
df=2*x; % Evaluate first derivative at x iter = iter+1; %updating iterations
fprintf(fid, '%i\t %f\t %g\n', iter, x, f); [f,df] = func(x);
end %Evaluate function and 1st derivative at x
fclose(fid); fprintf(fid, '%i\t %f\t %g\n', iter, x, f);
------------------------------------------------------------------------------------------------------------- end
sol = x;
You can save this program as newton2.m, to run the program just type newton2 fclose(fid);
in the command window. After you run the program your results.txt file will -------------------------------------------------------------------------------------------------------------
look something like this:
This Newton method is getting the function and derivative from an outside
iter x f(x) function called func. The function func will be like this:
1 1.666667 1.77778
4
-------------------------------------------------------------------------------------------------------------
function [f,df] = func(x) • Clearing a Plot
f = x.^2 – 1; % Original function When you use the command plot a figure is created. To clear this figure just type
df = 2*x ; % First derivative of the function clf and the current figure will be cleared.
end
------------------------------------------------------------------------------------------------------------- • Multiple Plots
This function needs to be saved under the name func.m in the same directory as There are three different options for multiple plots. The first option is to have all
your main program. To run your main program just type the plots in one figure. This is a good idea when the two graphs are related and
[iter, sol] = newton3(x)
you want to see for example if they intersect. To use this option, type hold on
where x can be any value you want to use as the initial guess. You will get the
after the first plot command then type the next plot command. Also you can type
same table of results as in the previous example. grid on to add grid lines.

• Plotting Example:
To draw a two dimensional graph just type plot(X,Y,S) where X is the plot(x,y)
independent variable, Y is the dependent variable and S is the format of the hold on, grid on
graph. S can be any of the following characters or a combination of the different plot(x,y,'gd')
columns.
y yellow * star - solid
k black o circle : dotted
b blue x x-mark -. dash dot
r red + plus - - dashed
g green d diamond

To see more plotting options type help plot

Example:

If x = [1:.5:10] and y=2*x then

plot(x,y) plots a solid blue line. (Blue is the default color for plotting)
plot(x,y,'gd') plots green diamonds at each data point but does not The second option is to divide the figure in to several subfigures, in other words to
connect them. have several small graphs in the same paper. To use this option just type
subplot(m,n,p) this will divide the figure into a mxn matrix and p will be the
current plot.

Example:

subplot(2,2,1); plot(x,y)
subplot(2,2,2); plot(x,x.^2,'rd-')
subplot(2,2,3); plot(x,x,'b*')
subplot(2,2,4); plot(x,x.^3, 'g')

5
 Example:

x =[1:.5:5];

plot(x,x.^2)

hold on

plot(x,x.^2,'gd')

legend('graph', 'data points')

xlabel('independent variable')
ylabel('dependent variable')

title('Example of a plot')

Example of a plot
The last option is to have different graphs in different figures, in other words you 25
will have each graph in a different paper. To do this just type figure(n) where graph
data points
n will be the number of the figure you are using currently.
20
Example:

figure(1)

dependent variable
15

plot(x,y)

figure(2) 10

plot(x,y,'g+')

5
• Axis Labels, Titles and Legend

To add axis labels just type xlabel(‘label’) for the x-axis and
ylabel(‘label’) for the y-axis. To add a title to your graph just type 0
1 1.5 2 2.5 3 3.5 4 4.5 5
title(‘title’). A legend can be added when you are plotting several independent variable
graphs on the same plot. Just type
legend(‘legend1’, ‘legend2’, …, ‘legendn’).