lOMoARcPSD|49462241

MA3252 Cheatsheet: Intro to Linear Programming Concepts

Linear and Network Optimisation (National University of Singapore)

Scan to open on Studocu

Studocu is not sponsored or endorsed by any college or university

Downloaded by Joe Mama (iancheah24@gmail.com)
 lOMoARcPSD|49462241

3252 Cheatsheet (a) P does not contain a line. i. Take BFS of phase I to start phase II.
(b) P has a BFS. ii. Use cost coefficients of original LP to compute
1 Intro to LP (c) P has at least n linearly independent constraints. reduced costs.
1. max cT x = − min −cT x 8. If P has at least one BFS and one optimal solution, then iii. Apply simplex method to original LP.
1.1 Graphical Representation at least one optimal solution is a BFS. 2. Big-M Method
1. In Rn , {x : aT x = b} is a hyperplane with normal vector (a) Similar
Pmto above, but cost is original cost
3 Simplex Method +M y where M >> 0 is treated as an
a. Replace = with ≤ to get a halfspace. 3.1 Developing the method i=1 i
2. c corresponds to direction of increasing cT x. algebraic variable.
1. For a polyhedron P and a point x ∈ P , vector d is a (b) If original LP feasible and finite optimal value, then
3. A set of inequalities is the intersection of halfspaces. feasible direction if x + θd ∈ P for some θ > 0.
4. To minimise cT x, move line in the direction −c as much all artificial variables are driven to 0 and original cost
2. Let x = (xB , xN ), xB ≥ 0, xN = 0 be a BFS. A direction is minimised.
as possible while staying in feasible region. d moving from x to an adjacent BFS is of the form
1.2 Possibilities for LP (c) If original LP infeasible, then auxiliary LP has at
dj = (djB , djN ) for some j ∈ N , where djN = ej and least one positive artificial variable at optimality.
1. Unique solution
djB = −A−1 B Aj . 3.3 Special cases
2. Multiple optimal solution
3. Let x be a basic solution and c = (cB , cN ). For each 1. Degeneracy:
(a) Set of optimal solutions may be bounded or
j = 1, 2, . . . n, the reduced cost c̄j of xj is (a) A BFS with one or more zero basic variables is
unbounded. −1
(b) But still only one finite optimal objective value. c̄j = cj − cT B AB Aj .
degenerate.
3. Optimal cost unbounded; −∞ (a) For j ∈ B, c̄j = 0. (b) A tie in the ratio test leads to degeneracy.
4. Infeasible; ∞ (b) A direction dj is an improving direction iff c̄j < 0. (c) A degenerate solution may not be optimal.
x
1.3 Conversion into standard form 4. Let θ̄j = min{ xij : i ∈ B, dji < 0} = lj . Then x + θ̄j dj 2. Alternative optima:
−di −d (a) An LP has multiple optimal solutions when the
l
1. aiT x ≤ bi becomes aiT x + si = bi , si ≥ 0 is a BFS. Ifdj ≥ 0, then θ̄j = ∞. objective function is parallel to a binding constraint.
2. xi ≤ 0 becomes x− i ≥ 0, xi = −xi
−
Basic x Solution (b) Optimal set of multiple solutions, if bounded, is the
+ − + − −1 −1
3. xi free becomes xi − xi , xi , xi ≥ 0. 5. c̄ cT − cT T convex hull of the optimal solutions.
B AB A−cB AB b
1.4 Convexity xB A−1 A A −1 (c) Unbounded optimal set occurs if at optimality
B B b
1. A set S ⊆ Rn is convex if for all x, y ∈ S and λ ∈ [0, 1], (reduced costs ≥ 0), for some j, djB = −A−1
6. Update the tableau: B Aj ≥ 0.
λx + (1 − λ)y ∈ S. (a) Check for negative reduced cost; use as entering Then xB + θdjB is feasible and optimal for θ ≥ 0.
2. The feasible region of an LP is convex. variable. 3. Unbounded solution
3. x ∈ Rn is a convex combination of x(1) , x(2) , . . . x(k) ∈ Rn
Pk (b) Select leaving variable. (a) Unbounded objective value occurs when some djB ≥ 0
if x = λ x(i) , λi ∈ [0, 1].
i=1 i (A−1 b)i and c̄j < 0.
i. θ̄ = min{ B
: i ∈ B, (A−1
B Aj )i > 0} =
4. The convex hull of x(1) , x(2) , . . . x(k) ∈ Rn is the set of (A−1
B
Aj )i 4. Infeasibility
convex combinations of x(1) , x(2) , . . . x(k) . (A−1
B
b)l (a) Detect using the Big-M method. Infeasible when
, xl leaving variable.
5. f is a convex function if for all (A−1
B
Aj )l c̄ ≥ 0 (stopped) but B still contains nonzero auxiliary
x, y ∈ Rn , λ ∈ [0, 1], f (λx + (1 − λ)y) ≤ λf (x) + (1 − λ)f (y). (c) Perform row operations to zero elements. variable, i.e., M blows up objective value. Read from
−1
f is concave if −f is convex. i. Normalise row and change leaving variable to −cT B AB b.
6. The maximum of n convex functions is convex. In entering variable.
particular, the max of n affine functions is convex.
4 Duality Theory
ii. Perform row operations to zero the other entries
4.1 Dual Problem
2 Geometry of LP of the column.
1. Primal problem (P ) and equivalent dual (D) is defined as
(d) Repeat until reduced costs ≥ 0.
2.1 Polyhedra and Extreme Points
7. A BFS is degenerate if some basic variable is 0. (P ) min cT x (D) max pT b
1. A polyhedron is a set of the form {x : Ax ≤ b}. It is an
8. Consider a BFS x. s.t. Ax = b s.t. p free
intersection of halfspaces. A feasible region is a
(a) If c̄ ≥ 0, then x is optimal. x≥0 pT A ≤ cT
polyhedron.
(b) If x is optimal and non-degenerate, then c̄ ≥ 0.
2. x∗ ∈ P is a basic feasible solution if n linearly independent
3.2 Starting the Simplex Algorithm 2. Equivalent dual problem (D) to (P ) is
constraints are active at x∗ .
1. Two-Phase Method: 3. (D) is derived from (P ) with the general Lagrange dual
3. x∗ is rank k if the span of its active constraints is
(a) Phase I: Find BFS using auxiliary LP function.
dimension k.
i. Multiply constraints by -1 to ensure b ≥ 0 as 4. (P ) to (D) conversion rules
4. x∗ is a basic solution if it is rank n.
required. (P ) (D) CS
5. x∗ is a basic solution of the standard form LP iff
ii. Add artificial variables y1 , y2 , . . . , ym to min cT x max pT b
(a) Ax∗ = b and
constraints without positive slack. aT
(b) There exist indices B(k), k = 1, 2, . . . m s.t. i x ≥ bi pi ≥ 0
iii. Apply
Pm simplex method on auxiliary LP with cost aT pi (aT
i. Columns AxB(k) are linearly independent and i x ≤ bi pi ≤ 0 i x − bi ) = 0,
y.
i=1 i
T ∀i
ii. xi∗ = 0 if i ̸= B(k). ai x = bi pi free
iv. If optimal cost to auxiliary LP is 0, BFS to xj ≥ 0 p T A j ≤ c j
x∗ is a BFS if it is a basic solution and xB ∗ ≥ 0.
original LP found. If positive, then original LP is
6. A polyhedron P contains a line if for some x∗ ∈ P, d ∈ Rn xj ≤ 0 pT Aj ≥ cj (cj − pT Aj )xj = 0,
infeasible.
nonzero, x∗ + λd ∈ P for λ ∈ R. xj free pT Aj = cj ∀j
(b) Phase II: Solve original LP
7. TFAE: 5. Dual of dual (DD) is equivalent to primal (P )

Downloaded by Joe Mama (iancheah24@gmail.com)

 lOMoARcPSD|49462241

6. Duals (Di ) of equivalent form primals (Pi ) are equivalent. 5 Sensitivity Analysis is connected to only one other node.
(a) Introducing slack s ≥ 0. 5.1 Changes in RHS vector. 9. A graph is bipartite if nodes can be divided into two sets
− −
(b) Replacing xi free with x+ +
i − xi where xi , xi ≥ 0. 1. Change b component to bi + δi . s.t. the ends of each edge lie in different sets.
4.2 Duality Theorems 2. (Feasibility) Require x∗B + δ(A−1 B ei ) ≥ 0.
10. Network flow formulation:
1. (Weak Duality) If x is feasible in (P ) and p is feasible in 3. (Optimality) Unaffected. (a) A network is a directed graph G = (V, E) with
(D), then cT x ≥ pT b. Thus, 4. If old BFS infeasible, apply dual simplex. additional info on the nodes and edges.
5. If old BFS feasible, marginal cost is p∗i = CB T A−1 e . (b) For each arc (i, j) ∈ E, xij is the amount of flow
B i
sup pT b ≤ inf cT x through the arc (i, j).
5.2 Changes in cost vector
1. Change c component to cj + δj . (c) (Capacity constraint) Flows are bounded by
over p dual feasible x primal feasible. 0 ≤ xij ≤ uij . If uij = ∞, then the problem is
2. (Corollary) If cT x = pT b then x and p are primal and dual 2. (Feasibility) Unaffected.
−1 uncapacitated. Else, it is capacitated.
optimal. 3. (Optimality) Optimality condition is cT − cT B AB A ≥ 0.
(a) If j ∈ N , then c̄new = c̄j + δ j . Therefore, x B optimal
(d) cij is the cost per unit flow on the arc.
3. (Corollary) Unboundedness in (P ) implies infeasibility in j
(e) For each node i ∈ V, bi is the external
(D) and vice versa. iff δj ≥ −c̄j . If not optimal, apply primal simplex.
(b) If j ∈ B, then c̄new = c̄N − δeT −1 supply/demand.
4. (Strong duality) If an LP has an optimum, so does it dual N j AB AN . Only need (f) If bi < 0/bi > 0/bi = 0, then node i is a
and both optimal objective values are equal. to consider nonbasic.
5.3 Changes in nonbasic column of A supply/demand/transshipment node.
(a) If B is an optimal basis for (P ), and there exists basis (g) (Flow balance constraint) Flow out - Flow in =
B0 s.t. AB0 = I then pT = cT T 1. Change entry aij to aij + δ where Aj is nonbasic column.
B0 − c̄B0 . Supply.
−1 2. (Feasibility) Unaffected.
(b) If B is an optimal basis for (P ) then pT = cT B AB is 11. A flow is feasible if it satisfies the capacity constraint and
3. (Optimality) Only consider c̄new j = c̄j − δp∗i ≥ 0. If not
an optimal dual solution. the flow balance constraint.
optimal, apply primal simplex. P
5. (Complementary slackness) Suppose that x and p are 12. If a network has feasible flow, then b = 0.
5.4 Add a new variable i∈V i
primal and dual feasible. Then x and p are optimal iff 13. A network flow problem is represented by the node-arc
1. Add variable xn+1 , with cost cn+1 , coefficients An+1 , and
(a) pi (aT
i x − bi ) = 0 for all i, incident matrix A ∈ Rn×m , where n = |V |, m = |E| and
constraint xn+1 ≥ 0.
(b) (cj − pT Aj )xj = 0 for all j. 2. (Feasibility) Unaffected by setting xn+1 = 0. (a) Each row corresponds to a node.
See above table. (Optimality) If c̄n+1 = cn+1 − cT −1 (b) Each column corresponds to an arc.
3. B AB An+1 ≥ 0, then
6. (Consequences of CS) Suppose x is feasible. Then x is (c) The column corresponding to arc (i, j) has +1 in row
optimal. If not, then add column with c̄n+1 and A−1 B An+1
primal optimal iff there exists dual feasible p satisfying CS. i and −1 in row j, 0 otherwise.
to the old tableau apply primal simplex.
4.3 Dual Simplex 14. All network flow problems can be represented as the LP
5.5 Add a new constraint
1. Idea is to start from dual feasible (c̄ ≥ 0) and work toward
primal feasible A−1 1. New constraint aT m+1 x ≤ bm+1 added to original LP.
B b ≥ 0. min cT x
Let B be a basis s.t. c̄ ≥ 0, i.e., pT = cT −1 2. (Feasibility) Check aT m+1 xB ≤ bm+1 .
2. B AB is feasible. s.t. Ax = b,
Write the tableau for the primal simplex. 3. (Optimality) If feasible in new LP, then also optimal.
4. If not feasible, add new constraint with slack variable 0 ≤ x ≤ u,
3. Starting the algorithm:
(a) Cost vector c must be nonnegative. Basic xB xN xn+1 Solution
−1 for cost vector c, node-arc incident matrix A,
(b) Transform the LP s.t. all constraints are ’≤’ so that c̄ 0 c̄N 0 −cT B AB b
xn+1 in old tableau. −1 −1 supply-demand vector b and capacity upper bound u.
the slack variables si give us AB = I. Then c̄ = c ≥ 0. xB I AB A 0 AB b
4. Update the tableau: xn+1 aB T aNT 1 bm+1 This is also the uncapacitated minimum flow problem.
(a) Check for negative variable xi in B; use as leaving T s.t. (x , x 6.2 Shortest path problem
(a) Zero out the row aB B n+1 ) is basic solution.
variable. 1. The problem of finding the shortest path from node s ∈ V
(b) bm+1 becomes negative after zeroing so apply dual
(b) Select entering variable to node t ∈ V may be formulated as a network flow
simplex.
c̄ problem with bi = +1, −1 if i = s, t respectively, and
i. θ = min{ −1j : j ∈ N, (A−1
B Aj )i < 0} = bi = 0 otherwise, and x ∈ {0, 1}n . If no negative cycles,
|(AB Aj )i | 6 Network Optimisation
c̄j 6.1 Intro to Network Flow use x ≥ 0 instead.
, xj leaving.
(A−1
B
Aj )i 1. An (un)directed graph G = (V, E) consists of a set of 2. The optimal solution x∗ij = 0 or 1. The optimal path is
ii. This is to ensure that c̄ ≥ 0 after row operations. vertices V and a set of edges E. {(i, j) : x∗ij = 1}.
(c) Perform row operations to zero elements. 2. An edge {i, j} for i, j ∈ V, i ̸= j connects i and j. If graph 3. Dynamic lot sizing:
i. Normalise row and change leaving variable to is directed, an edge is (i, j), i outgoing and j incoming. (a) Problem: Given demands di for periods i = 1, . . . , T ,
entering variable. 3. A path is a finite sequence of nodes i1 , . . . , im s.t. we produce xi in period i and/or draw from
ii. Perform row operations to zero the other entries {ik , ik+1 } ∈ E, with no nodes repeated. inventory Ii−1 carried over from period i − 1 to i to
of the column. 4. A cycle is a path with the additional node {im , i1 }. It has meet demand di . Let ci be the per unit production
(d) Repeat until A−1 B b ≥ 0, i.e., the basic solution is at least 3 nodes. cost and hi the per unit holding cost in period i. In
primal feasible. 5. A directed path/cycle has only forward arcs. period i, a setup cost of Ki is incurred if production
5. Dual simplex detects primal infeasibility: 6. An (un)directed graph is connected if there exists a path is carried out. We want to find a production plan
(a) If there is (A−1 T −1
B b)i < 0 s.t. ei AB A ≥ 0 then (D) is between every pair of nodes. Otherwise, it is disconnected. (x1 , . . . , xT ) to meet demands (d1 , . . . , dT ) and
unbounded. 7. A graph is acyclic if it contains no cycle. minimise total cost.
(b) This can occur when doing the ratio test. 8. A tree is a connected acyclic graph. A leaf is a node that (b) Ii = Ii−1 + xi − di . Holding cost in period i is hi Ii .
(c) By weak duality, (P ) is infeasible. I0 = IT = 0.

Downloaded by Joe Mama (iancheah24@gmail.com)

 lOMoARcPSD|49462241

P
(c) In the optimal solution, in any period, either produce 6. The capacity of a cut is u(S, S̄) = (i,j)∈(S,S̄)
uij , uij is 7. Reduced costs and dual vector; Suppose the row
or carry forward inventory, not both, and each the capacity of (i, j). corresponding to node n is removed.
Pj
xi = d for some j > i, i.e. each xi meets 7. A minimum s − t cut is one with minimum capacity. (a) c̄ij = ci j − (pi − pj ) for (i, j) ∈ E.
k=1 k
demands exactly from i to j. 8. The capacity of any cut is an upper bound on the max (b) To compute dual vector, set c̄ij = 0 for (i, j) ∈ T .
(d) Problem reduces to finding shortest path from 1 to flow from S to S̄. Then pi − pj = cij and pn = 0.
T + 1, vertices {1, . . . , T + 1} and edges 9. The dual of the max flow problem is (c) Using calculated pi , compute reduced costs for
{(i, j) : 1 ≤ i < j ≤ T + 1}. Cost of edge (i, j) is X (i, j) ∈
/ T.
c(i, j) = total cost from periods i to j. min uij zij 6.5 Algorithm
Pj−1 Pj−2 Pj−1 1. Network Simplex Method


(e) c(i, j) = Ki + ci k=i k
d + h
k=i k l=k+1 l
d . (i,j)∈E
(a) Start with a spanning tree T , feasible solution x.
4. Project management s.t. yi − yj ≤ zij , zij ≥ 0, (i, j) ∈ E, ys − yt = 1. (b) Compute dual vector p and c̄ij for all arcs (i, j) ∈ / T.
(a) Artificial source node s and sink node t introduce to
10. By strong duality, the max flow is equal to the capacity of (c) If c̄ij ≥ 0 for (i, j) ∈ E, x is optimal and we are done.
signify start and end of project. Add arcs from s to
a min cut. If not, choose some (i, j) s.t. c̄ij < 0.
every node with no predecessor, arcs from every node
11. The cut given by the optimal solution is (d) Apply flow update scheme to get leaving arc (p, q)
with no successor to t.
S = {i : yi = 1}, S̄ = {i : yi = 0}. and updated flow x.
(b) In forward pass, take max value of incoming nodes.
6.4 Basic Properties (e) Update T ← [T {(p, q)}] ∪ {(i, j)}.
(c) In backward pass, take min value.
Consider the uncapacitated minimum flow problem. Assume (f) Return to step 2.
(d) Slack values (bottom - top) say how much a job can P
b = 0 for feasibility and graph G connected, or else 2. Flow update scheme
be delayed without delaying the project. i∈V i
network can be decomposed into smaller networks to be solved (a) Entering arc (i, j) forms a unique undirected cycle
(e) The longest path from start to end node (critical
independently. with some arcs in T. Orientate cycle so that (i, j) is a
path) is the time to complete the project. On this
1. Construct truncated node-arc incident matrix à by taking forward arc. Let Cf and Cb be the sets of forward
path, all slack values are 0. Activities on this path
any n − 1 rows of A. and backward arcs in the cycle.
are critical activities.
2. A spanning tree of a graph G is a tree containing all nodes (b) If Cb ̸= ∅, set θ∗ = min(k,l)∈Cb xkl = xpq . Then xpq
(f) A project management network is acyclic.
of G. leaves the basis. If Cb = ∅, then θ∗ = ∞ so objective
6.3 Maximum flow problem
3. A flow vector x is a tree solution of G if is −∞.
1. Let d = (1, 0, . . . , −1). The maximum flow problem can be
(a) Ãx = b̃, and (c) Update x ← x̂, where x̂kl = xkl + θ∗ if (k, l) ∈ Cf ,
written as
(b) there exists a spanning tree of G with edges T ⊆ E xkl − θ∗ if (k, l) ∈ Cb , xkl otherwise.
max v s.t. xij = 0 for (i, j) ∈
/ T. 3. Two-phase method to find initial BFS
4. A feasible tree solution x is a tree solution with x ≥ 0. (a) Phase I: For any i ∈ V \ {n}, if bi ≥ 0/bi < 0 and
s.t. Ax = dv, 0 ≤ x ≤ u.
5. The columns corresponding to n − 1 arcs form a basis of à (i, n)/(n, i) ∈
/ E, create artificial arc (i, n)/(n, i).
2. The dual of a max flow problem is a min cut problem. iff these arcs form a spanning tree. (b) Initial basis = {(i, n) if bi ≥ 0 or (n, i) if
3. A cut is a partition of V into subsets S and S̄ = V \ S. 6. Implication bi < 0 : i ∈ V \ {n}}.
4. (S, S̄)/(S̄, S) denotes the set of forward/backward arcs (a) Basic (feasible) solution ↔ (Feasible) tree solution (c) Initial flow xin = bi if bi ≥ 0, xni = −bi if bi < 0.
from nodes in S to nodes in S̄. (b) xij is a basic variable ↔ (i, j) is an arc in tree (d) Perform network simplex method until you have a
5. An s − t cut is a cut s.t. s ∈ S and t ∈ S̄. solution basis without artificial arcs.

Downloaded by Joe Mama (iancheah24@gmail.com)