Introduction to OR and Linear Algebra

Part-II

Prasun Das
Indian Statistical Institute
SQC & OR Unit
Kolkata, India
dasprasun@rediffmail.com

08-12-2020 1
 Some Results of Linear Algebra

Result-1: A set of vectors in En is linearly dependent (l.d.) iff at least one of this
can be expressed as a linear combination (l.c.) of the others.
Proof: use the definition of l.d. set of vectors. [Hint]
Result-2: A linearly independent set of vectors cannot contain zero vectors.
Proof: Try yourself
Result-3: Subset of a linearly independent set is linearly independent (l.i.).
Proof: Let, S = {a1,a2,…,ap} be a set of l.i. vectors and T = {a1,a2,…,an} (n<p)
be any subset of S.
Hint: Try by contradiction theory on the set T and show that S is l.d.
Result-4: Any superset of a linearly dependent set is linearly dependent.
Proof: Try yourself

08-12-2020 2
 Some Results of Linear Algebra

Result-5: Representation of any vector in terms of a given set of basis vectors is

unique.

Proof. Let B = {a1,a2,…,ap} be a set of basis vectors and let xEn.

Then, x = iai =iai i = 1,2,…,p
Hint: Now, use the definition of Basis (property of l.i.)

08-12-2020 3
 Some Results of Linear Algebra
Result-6: Replacement of a basis vector forms another basis.
Proof: Let, B = {a1,a2,…,an} be a set of basis vectors in En and 0bEn.
To show that, B1 = {a1,a2,…,an-1, b} is also a basis.
Condition-1: B1 is a l.i. set of vectors.
If possible, suppose B1 is l.d., then iai + .b = 0 for i = 1,2,…,n-1 and with at
least one i  0. Now   0, otherwise, a1,a2,…,an-1 are l.d. which is impossible
being subset of a l.i. set (Result 3). Thus,
1a1 + 2a2 + … + n-1an-1 + .iai = 0 (since, b = iai , for i = 1,2,…,n with
n  0, say)
or, (i +.i)ai + (.n) an = 0 where .n  0 (since, 0, n  0), i = 1,2,…,n-1
This shows, a1,a2,…,an are l.d. which is contradictory !!!
Hence, B1 is a l.i. set.
08-12-2020 4
 Some Results of Linear Algebra

Result-6 [contd…]: Replacement of a basis vector forms another basis.

Condition-2: B1 spans En
Let, xEn. Then, x = iai (Since, B is a basis of En), i = 1,2,…,n
So, x =  iai + nan, i = 1,2,…,n-1 ……………..(1)
Now, b =  iai + nan, i = 1,2,…,n-1
or, an = (1/n) [b -  iai ], i = 1,2,…,n-1 and n  0
Replacing an in (1), we get
x =  iai + n.(1/n) [b - iai ], i = 1,2,…,n-1
=  [i – (n./n).i]ai + (n./n).b, i = 1,2,…,n-1
This shows, x is a l.c. of elements of B1. Thus B1 spans En.
Hence the proof.

08-12-2020 5
 Some Results of Linear Algebra

Home Work:
Result-7: Hyperplane is a Convex Set.
Result-8: Half Spaces are Convex Sets.
Result-9: Intersection of two convex sets is Convex.
[Is it true for Union of two convex sets? Comment]

08-12-2020 6
 Some Results of Linear Algebra

Result-10: Convex Polyhedran is a Convex Set.

Proof: Let S = {x: x = ixi, i  0, i = 1} and u1, u2 S.
Then, u1 = ixi, i  0, i = 1 and u2 = ixi, i  0, i = 1
If u = (1-)u1 + u2, 01, then
Show that,
(i) u is expressed as convex combination of x1, x2,…, xp, say, u = ixi,
(ii) i  0, i = 1.
Then, u S and hence, S is convex.

08-12-2020 7
 Some Results of Linear Algebra

Result-11: In the general LPP, the set of all feasible solutions is a Convex Set.
Proof: Let K denote the set of all f.s. and x1, x2K.
Then, x1 0, x2 0, Ax1= b, Ax2= b
Let, x = (1-)x1 + x2, 01
Then, prove…. x  0 and Ax = b so that, xK.
Therefore, K is Convex.

08-12-2020 8
 Some Results of Linear Algebra

Result-12: If the set of all feasible solutions K of a LPP is a Convex Polyhedron, then at
least one extreme point must be obtained which is optimal.
Proof: Let, K be the convex polyhedron with finite number of extreme points x1, x2,…,
xk which generate it. Let, xK
Then, x = ixi, i  0, i = 1, i = 1,…,k
Now ctx = ct [ixi] = i (ctxi)
Let, max (ctx1, ctx2,…, ctxk) = ctxm [considering a maximization problem]
Then, i (ctxi)  i (ctxm) [since, i  0]
or, i (ctxi)  (ctxm) i
or, ctx  ctxm [since, i = 1]
This shows xm is optimal which is one of the k extreme points. So, at least one extreme
point must be obtained which is optimal.

08-12-2020 9
 B.F.S, Extreme Point & Optimal Soln.

Every b.f.s is an extreme point of K and vice-versa. Thus the number of

extreme points is finite (since, no. of b.f.s is finite). Earlier we found the result
that if there is optimal, at least one extreme point must be optimal. Thus, the
search for optimal solution can be restricted to among the finite number of
extreme points (b.f.s) and hence the infinite number of feasible solutions in K
need not be examined for optimal. This is the motivation for Simplex Method
where we start from an extreme point (b.f.s) and move from one extreme point
to another extreme point till optimality is reached or unbounded indication is
obtained..
obtained

08-12-2020 10
 Linear Programming
A company manufactures hockey sticks and chess sets, with gross margins of $2 and $4,
respectively. Producing a hockey stick requires 4 hrs. of processing at m/c. A and 2 hrs. at
m/c B. A chess set requires 6hrs. at m/c. A, 6hrs. at m/c. B and 1hr at m/c C. Machines A, B,
and C have 120 hrs, 72 hrs, and 10 hrs of processing capacity per day.
Find the optimal product mix that maximizes profit.

H = # of hockey sticks produced per day

Decision
C = # of chess sets produced per day variables

max Z  2H  4C Objective function: maximize

profit
subject to:
4H  6C  120 Machine A capacity constraint

2H  6C  72 Machine B capacity constraint

C  10 Machine C capacity constraint

H ,C  0 Non-negativity constraints

08-12-2020 11
 More LP Problems on Formulation
Ex.-2: A firm manufactures two types of products A & B. For A, per unit profit is Rs. 3/-
and for B per unit profit is Rs. 4/-. Each type of product needs to be processed on two
machines I and II. Per unit processing time in minutes for A & B is given below:

Product A Product B
M/c. I 2 3
M/c. II 5 2

Each machine is available for 60 hours a week. [Determine optimal weekly production
schedule for profit maximization.]
Ex.-3: A diet is being prepared for the students staying in a hostel. The objective is to
feed the students at the least cost, but the diet must have between 1,800 and 3,600
calories. No more than 1,400 calories can be starch, and no fewer than 400 can be
protein. The varied diet is to be made of two foods: A and B. Food A costs $0.75 per
pound and contains 600 calories, 400 of which are protein and 200 starch. No more
than two pounds of food A can be used per resident. Food B costs $0.15 per pound and
contains 900 calories, of which 700 are starch, 100 are protein, and 100 are fat.

08-12-2020 12
 General LP Problem Formulation
Given a set of m linear inequalities (or, equalities) in n variables x1, x2,…,xn. We
wish to find non-negative values of these variables which will satisfy all the m
linear inequalities and which maximizes (or, minimizes) some linear functions of
these variables. Such problem is called L.P.P. In symbols,

Objective Function:
Max / Min f(x1, x2,…,xn) = Z = c1x1+ c2x2 +…+cnxn =
Linear Constraints:
a11x1+ a12x2 +…+a1nxn  =  b1 In matrix notation,
a21x1+ a22x2 +…+a2nxn  =  b2 Max/Min Z = ctx
. . . . subject to Ax  =  b
. . . . A = Coefficient matrix;
. . . . b = Boundary restriction;
am1x1+ am2x2 +…+amnxn  =  bm x = Decision variables; and
Non-negative Constraints: ci = cost/profit/loss coefficients.
x1, x2,…,xn  0

08-12-2020 13
 Important Terms in Mathematical Optimization:

Decision variables: variables to be determined.

Objective function: linear function that has to be maximised or minimised
over decision variables.
Constraints: linear equalities and inequalities that decision variables should
satisfy sign restrictions.
Binding constraint: inequality constraint holds with equality, otherwise non-
binding.
Feasible region: set of all points (x1,…,xn) which satisfy constraints and sign
restrictions.

08-12-2020 14
 Parameters of LPP
Resource Usage Per Unit of Activity
Parameters of LPP
Activity (n) Amount of
Resource Available
Resource (m) 1 2 … n
1 a11 a12 … a1n b1
2 a21 a31 … a2n b2
. . .
. . .
. . .
m am1 am2 … am bm

Z/unit of activity c1 c2 … cn
Level of activity x1 x2 … xn

The resources are needed to perform these activities, but the amount available of each resource is
limited, so a careful allocation of resources to activities must be made. Determining the
allocation involves choosing the levels of the activities (the values of the decision variables) that
achieve the best possible value of the overall measure of performance Z.

08-12-2020 15
 SMLP and Implicit Assumptions
SMLP (Standard maximum LPP)
It is an LPP in which the objective function has to be maximized and the linear
constraints will have less than or equal to () inequalities.

Implicit Assumptions of LPP:

Proportionality: individual activities are considered independent of the others. The
overall measure of performance along with usage of each resource are directly
proportional to the level of each activity.

Additivity: the total function value can be obtained by adding the individual
contributions from the respective activities.

Divisibility: activity units can be divided into any fractional levels, so that non-integer
values for the decision variables are permissible.

Certainty: all the parameters of the model are known constants. In real problems, this
assumption is seldom satisfied. Hence, sensitivity analysis is required to identify the
sensitive parameters, if any.

08-12-2020 16
 LPP: About its Solution
Solution: It is a set of values x1,x2,…,xn which satisfy the linear constraints.
Feasible Solution: It is a solution, which satisfies the non-negativity restriction
as well.
Optimal Solution: This is a feasible solution, which optimizes
(maximizes/minimizes) the linear objective function.
Optimal Value: It is the value of the objective function corresponding to the
optimal solution

The general method of solving LPP is given by Simplex Method (G.B. Dantzig,
1947), which is an algebraic, iterative procedure tending to exact solution in a
finite number of steps. Simplex method for LP is an extension of G-J Elimination
method (C.F. Gauss & W. Jordan, 1888) to:
a) find nonnegative sol. if one exists, or establish that there is no nonnegative
sol. So, this method can solve systems of linear inequalities.
b) minimize a linear obj. function subject to linear inequalities.

08-12-2020 17
 Special Cases in LPP

• Infeasibility

• Unboundedness

• Redundancy

• Degeneracy(alternate optimal)

08-12-2020 18
 Infeasibility: Geometric Interpretation

Conflicting Constraints Example:

x1 + 2x2  6
2x1 + x2  8
x1 7

08-12-2020 19
 Unbounded: Geometric Interpretation

Possible Reasons:
Example:
Maximize Profit = 3x1 + 5x2 Subset of decision variables and/or
Sub to objective function value can be
x1 + 2x2  10 made infinitely large, thereby
x1 5 feasible region becomes open
x2  10 ended (unboundedness property)

08-12-2020 20
 Redundancy: Geometric Interpretation

Example: Explanation: A redundant constraint is

Maximize Profit = x1 + 2x2 simply one that does not affect the
Sub to feasible solution region. Redundancy
2x1 + x2  30 causes no major difficulties in solving
x1 + x2  20 LP problems graphically.
x1  25

08-12-2020 21
 Degeneracy: Geometric Interpretation

Example:
Maximize Profit = 3x1 + 2x2 Possible Reasons:
When the objective function’s
Sub to
isoprofit/isocost line runs perfectly
6x1 + 4x2  24 parallel to one of the problem’s
x1 3 constraints. Then, any point along that
line provides an optimal solution.

Q.: Identify its merit/demerit to a decision maker.

08-12-2020 22
 Sensitivity Analysis
Purpose: To see the effect of dynamic real-life conditions (price,
profit, resource, time etc.) onto the Optimal solutions obtained
under deterministic assumptions.

Three Areas to see the effect of change:

a) Changes in the Objective function coefficient
b) Changes in the Technological coefficients
c) Changes in the Resources

Approach: Analytic Postoptimality method

08-12-2020 23
Original Problem:

A Music System Company manufactures CD players and Stereo Receivers. Each of these
products requires a certain amount of skilled artisanship, of which there is a limited
weekly supply. The firm formulates the following LP problem in order to determine the
best production mix of CD players (X1) and Receivers (X2):

Formulation:

Maximize Profit = 50x1 + 120x2

Sub to

2x1 + 4x2  80 [hours of electricians’ time available]

3x1 + x2  60 [hours of technicians’ time available]

x1, x2  0

08-12-2020 24
Graphical Solution:

08-12-2020 25
 Case 1: Changes in the Objective function coefficient

In real-life problems, contribution rates (profit / cost) in the objective function

fluctuate periodically. Graphically, this means that although the feasible region
remains exactly the same, the slope of isoproft/isocost line will change.

Q. How much can the objective function coefficient change before

another corner point becomes optimal?

08-12-2020 26
 Case 2: Changes in the Technological coefficients

If fewer or more resources are needed to produce a product, coefficients in the

constraint equations will change. These changes will have no effect on the
objective function, but they can produce a significant change in the shape of
the feasible solution region, and hence in the optimal profit or cost.

Problem:
What happens, if the technological constraints
a) 3x1 + x2  60 changes to 2x1 + x2  60; and
b) 2x1 + 4x2  80 changes to 2x1 + 5x2  80

08-12-2020 27
 Case 3: Changes in the Resources

The r.h.s of the constraints often represent resources (labour hours; m/c. time,
money; prodn. matls. etc.) available. Higher profit could be realized with
additional resources, if available but to which extent?

If the r.h.s of a constraint is changed, the feasible region will change (unless
the constraint is redundant), and often the optimal solution will change.

Problem:
What happens, if the available electricians’ hours are
a) increased to 100 hours from 80 hours; and
b) decreased to 60 hours from 80 hours;

08-12-2020 28
 Case 3: Changes in the Resources

a) New optimal solution (0,25)

b) Extra 20 hours of electricians’ time results an
increase of profit of $600 (or, $30 per hour)

a) New optimal solution (0,15)

b) Reducing electricians’ time by 20 hours results in
decrease of profit of $600 (or, $30 per hour)

a) Dual/Shadow Price: Increase in obj. fn. value that

results from an one-unit increase in the r.h.s of a
constraint. In this case, it is $30/hr. which is
relevant only within limits. For ex.,
b) Available 240 hrs. electricians’ time generates
optimal solution (0,60) [obj. fn. value = $7,200]
with same dual price. More availability returns
back same optimal soln. with no further increase in
dual price.

08-12-2020 29