Optimal Control

Prof. Dr. Goshaidas Ray

Department of Electrical Engineering
Indian Institute of Technology, Kharagpur

Lecture - 21
Relationship between Primal and Dual Variables

(Refer Slide Time: 00:30)

So, last class we have discussed the primal and dual problem. The standard primal
problem is like this, way make maximum a linear function, which is where x 1 x 2 dot
dot x n are the design variables which you can write it vector form like this way. Subject
to a set of linear inequality constraints, which you can write it a x less than or equal to e.
So, this type of form of linear equations either in objective function, or in and in what is
called in inequality constraint, if it is like this way then it is called standard primal
problem.
(Refer Slide Time: 01:05)

And dual of this problem is the structure is like this way minimize the function f of d,
whatever the right hand side of this equation was there inequality equation right hand
side of the this will come, as a coefficient objective function linear objective function of
the dual problem. The e 1 y 1 e 2 y 2, so how many variables will be there equal to the
number of inequality constraints in final problems. And subject to that this that this type
of inequality conditions, inequality condition in on form of linear equations that is we
when you are converting into dual problems and we know to how to solve this one.

(Refer Slide Time: 01:51)

If the final problem is given then convert into a standard first convert into dual problem
then dual problem you convert into a standard L P problem, then you solve it by either
phase 1 method or phase 1 and 2 both methods are depending upon the type of inequality
involved in the set of linear equations. And we have worked out the simple example and
we have seen that the values

(Refer Slide Time: 02:24)

That is y 1 y 2 dot dot y 6 variables which is dual variables values are there then question
is how to go back to our final variables, which is the problem is original problem is given
that way now. Today's lecture is we have to find out the relationship between the primal
and dual variables or what is the relationship between primal and dual problems? So,
first relationship you can say if one problem has a feasible solution, if one problem is
feasible solution and unbounded, what is if one problem has a feasible solution and has a
bounded objective unction value then other problem should have a feasible solution, this
is one thing. Next is if one problem is next, next point, if one problem has a feasible
solution.
(Refer Slide Time: 03:27)

If is this is feasible solution and unbounded objective function value and then other
function is also do not have any feasible solution, feasible solution. That means, if one
problem has a feasible solution, but unbounded objective function that other problem
having a infeasible solution, this is a second observation you can see and the third one. If
one problem has no feasible solution and other problem other do not take feasible
solution or they have what is called unbounded objective function value. So, this...

Now, next we will see it how can convert the feasible variables, how one can convert
that our feasible variables let us call what is called dual variables, how one can convert
dual variables in the final variables and vice vice versa, before that we will study the first
see the few theorems, that theorem if x having a dimension n cross 1 is a feasible
solution to the to the primal problem, and y is our m cross n feasible a feasible solution
for the dual problem. Then one can write y transpose e which is objective function of the
dual problem y transpose z or e transpose y is value objective value function is greater
than equal to x transpose d.

The x you write it d and you know the dimension the dimension e you know what is
dimension of e, e is dimension is m cross 1 that primal problem how many inequality
constraints are there, there are n constraint are there. And same number of variables
presence present in the dual what is dual problems, so this is a scalar quantity and this is
also scalar quantity. So, our theorem tells if x is a feasible solution of a final problem and
y is a feasible solution of a dual problem, then the objective function of the dual problem
is always greater than or equal to objective function value of primal problem. That is one
lets us see the proof of this one.

So, since x is a feasible solution is a feasible solution of the primal problem, feasible
solution then what is the kind it must satisfy the our inequality constraints of primal
problems. That means A into x to the dimension is n cross variables are there A is less
than or equal to e, that is dimension e m cross 1 and what is the variables in dual
problems? Y is greater than or equal to 0 and also that x or I write it x is greater than or
equal to 0 for this one.

Similarly, in dual problems the feasible solution of the dual problem is y m cross 1.
Similarly, if the feasible solution of dual problem is y then we can write it immediately
we can write it A transpose y is greater than or equal to our d, and d dimension n cross 1
and y and dual variables values are all greater than 0. Now, see from this one this
equation if I multiply by both side by x transpose then one can write it if I multiply the
both sides x transpose for previous equation then we can write it.

(Refer Slide Time: 08:55)

x transpose a transpose y is greater than equal to x transpose d this is a scalar quantity

this is a vector column vector we have multiplying this which is nothing but a case is d,
if you see it and this is a vector of dimension if what is dimension n cross 1 and this is
the dimension of 1 cross n x transpose. So, this is a scalar quantity if you take a transpose
both sides because scalar quantity turns for both sides remains same. So, I can write it y
transpose A x is greater than or equal to d transpose x, which is nothing but x transpose d
same because scalar transform scalar value same values is there.

Note you can note that that A x in the final problem this A x is based on is less than or
equal to e. So, if I replace this 1 by greater number this inequality sign still valid. So, I
can write y transpose e is greater than or equal to x transpose d and what is this one, this
is nothing but a objective function value of the dual problems. So, this I can write it f
dual problem objective function which is a function of y is greater than or equal to the
objective function value of the primal problem. That is function of x, this always proves
for all x, x components is x 1 x 2 x 3 dot dot x n, n for all y 1 y 2 dot dot y n. So, this
proves the theorem of this one proved.

So, if you tell one thing if x is the feasible solution of the primal problem and y is the
feasible solution for dual problem then we can write it the objective function of the dual
problem is always greater than or equal to the objective function value of the primal
value. That is the our conclusion which we have proved here. Next we will see that
another theorem that is theorem.

(Refer Slide Time: 11:46)

And if you remind the just writing it here primal problem is what I am writing maximize
z transpose of x which is nothing but a we have written it z of p is equal to this one,
subject to A of x is less than or equal to e. That e is number inequality constraints in the
dual primal problem is correspondence in dual problem is what, minimize f of d and
what is the right hand side of the primal variables that will come as the coefficient of the
objective function in the dual problem.

So, it will be a if you see this one that is nothing but y transpose, y transpose e or you can
write it e transpose this. Subject to A transpose y is greater than or equal to d, what is
coefficient in the primal problem are coming in the objective function that will come that
right hand side of the inequality constraint of dual problem. That is dimension is that is
our basic statement of the primal dual problems.

The next theorem is this suppose x bar and y bar are feasible solution solutions to the
primal and dual problems to the primal and dual problems. That means x bar y bar are
the solution of the primal or dual problems, if the feasible solution of the dual problems,
then if d transpose x bar equal to if it is equal to that y transpose or e transpose y, if it is
this y bar that are feasible solution of the primal improvement. If this is equal to this then
x bar which we have considered feasible solution that will be x star, means optimal
solution of the primal problem, x bar and y bar will be equal to y star.

That means, optimal solution of the dual problem means maximization of the function
variables, that optimal points will get it here if this equal to this as this, this indicates this
is the optimal point of the dual and this is optimal point of the primal problems. These
are optimal solution to their respective problems, so please try to understand what is this
if x bar and y bar are the two feasible solution, if x bar are the two feasible solution. If
you get the objective function of d objective function value of the dual problem value
and the primal problem objective function value are same at that two feasible points, then
we call that two feasible points are the optimal solutions of their respective problems.

Then let us see the proof of this theorem, so let y be an arbitrary feasible solution, y is an
arbitrary feasible solution of dual problem this arbitrary feasible solution not optimal it is
at this moment to the to the dual problem, because x bar because x bar you see is a
feasible solution to the primal problem, then one can write immediately we can write it.
(Refer Slide Time: 17:25)

e transpose y objective function of this one is greater than or equal to that d transpose x
bar, this we can write it in a. Therefore, if e transpose y bar there are values of y bar
some feasible solution if this is equal to that d transpose x bar, this indicates then this
employs e transpose y bar is equal to d transpose x bar, x bar is less than equal to less
than equal to e transpose y, you see d transpose x bar is less than d transpose is there. So,
some other than y is a there is a feasible point, which equal to this, this indicates this
quantity of objective function value of primal and dual is less than e transpose of that
one. So, what does that mean?

This employees that our y star y bar this employees y bar is nothing but a our optimal
solution of the dual problem is the optimal solution for the dual problem, optimal means
optimal solution dual problem, dual problem is minimization of the problem
minimization. Similarly, on the other hand on the other end with the same reason on the
other hand, if let x on the other hand, let x enclose 1 be an arbitrary feasible solution,
arbitrary feasible solution x greater to the primal problem because according to the
statement of the problem because y bar is the feasible solution, is the is a feasible
solution of the dual problem to the dual problem.

Immediately you can write it that y transpose e transpose y bar is greater than or equal to
d transpose x this is a objective function of the dual problem of the dual problem, where
y bar is the solution in the feasible solution in the dual problem is better than this one.
So, this is greater than this one.

Therefore, if e transpose there are some choice of x is some of x some value let us call it
is a d transpose x bar some choice of x is equal to x bar, this objective function is same.
And this is this quantity will be greater than or less than or this quantity will be that this
is a quantity d bar of this will be greater than or equal to d transpose x, e of transpose this
and now telling their exist let us call some feasible solution x bar for which this value is
increased, objective function is it and it becomes equal to the e transpose y bar.

And since e transpose y bar value is greater than d bar of x this indicates that x bar this
employs, this employees that x bar is equal to x star is the optimal solution of the primal
problem. Optimal means, there are maximum that optimal if x is the optimal solution of
the corresponding to the maximum value of the function x star will give you, and that
proves the theorem. So, there are 2 theorems just told you If x is if x bar is a feasible
solution or primal problem, and y bar is a feasible solution of a dual problem.

(Refer Slide Time: 23:35)

If this that means, what is this objective function of x of this one if x, x bar transpose into
d is equal to that y bar transpose y bar transpose into e if it is same or vice verse d
transpose x is equal to e transpose y because it is a scalar quantity, I can take transpose
same then this indicates x bar at y bar at the optimal solution of the dual, what is called
primal problem. Optimal corresponding to the optimal solution of the dual problem and x
bar which is equal to x star the optimal solution of the primary problem, if their objective
functions are value is same.

So, next is very which stated theorem, which will give you the relationship between the
primal variables and the dual variables the theorem will tell you, how they are related to
primal variables x and dual variables y, how they are related the theorem. And keeping in
mind that primal standard primal problems statement and the dual corresponding dual
problem statement keeping in the mind, we can write it now the feasible solution. The
feasible solution x and y to a dual pair of the problems are optimal if and only if, if and
only if necessary and sufficient conditions.

The feasible solution x, x corresponding to the primal problem and y corresponding to

the dual problems there is a dual pair are optimal if and only if, 1 that x transpose then
you write the inequality constraints associate to the problems. That means a transpose y
minus d and that dimension n plus 1 similarly, this dimension is what should be the
dimension of this one m cross 1, this is m cross 1 this dimension is n cross 1 this equal to
0. Another condition is that we write y transpose then the dual problems we can write the
inequality constraints, that you got it what is called in the primal problem that A into x
minus e is equal to 0, this is scalar quantity this is scalar quantity. And this dimension is
m cross 1, this dimension is n cross 1.

Similarly, you can find out dimension as you earlier so our theorem tells if x and y are
the feasible solution of the dual problems, this are optimal this dual problems are dual
problems x 1 x 2 are the dual problems are optimal, if and only if this condition is
satisfy. That this indicates that if you know the solution of the dual problems one can
find out what is the solution of in primal problems, when a original given into a if it is
primal problems, what is the solution that we can find out using these two relationship.
So, let us see the proof of this 1 proof is very easy to see to how to get it this proof, prove
it this one.
(Refer Slide Time: 28:04)

If the if two solution are optimal then by theorem, the optimality theorem what we gain d
transpose x must be equal to y transpose or e transpose y, agree d transpose x is equal to
the, this is the function of the dual problems, dual problem objective function objective
function of dual problem. This is the objective function of primal problem, so if the
solutions are optimal then only this equal to this optimal, this equal to this we can write
it. We know that if x is the optimal solution of this one that means x is a feasible as a
feasible solution that because we can write it because x of k A into x with the is less than
or equal to e. Since, x is a feasible solution x is also optimal and feasible solution and
then is to not only this in the dual problem this y is greater than or equal to this.

Now we have now we have let us see x transpose A transpose y minus d, what is what
we get it this equation? This equal to you just expand this bracket you open it, then x
transpose d plus x transpose A transpose y, since it is a scalar quantity I can write it this
inverse y transpose A and x minus x transpose d, if you take x transpose d or d transpose
x is nothing but y transpose d. So, you can write it first term as it is second term I can
write it easily y transpose e, if you take y transpose common then this is x transpose of e.
So, x transpose a transpose is nothing but y transpose x is y, now look at this expression
individually the left hand side and right hand side then what we will get it see this one
left hand side.
(Refer Slide Time: 31:24)

x transpose A transpose y minus d, what we can write it x is the all the decision in the
primal problem and then in that values each element of x there are n variables are all are
greater than or equal to 0, whereas the equality constraint in dual problems, this quantity
if you see the inequality constraints this quantity is greater than or equal to 0. So, this
employees the results of this one is result of this one is becoming less than or greater
than or equal to 0.

That means I can write it that x transpose A transpose y, y minus d is equal to greater
than or equal to 0 the left hand side. Now, see the right hand side y transpose A x minus
e, what is this one? This is dual problem variables this values each elements of this
values vector y is greater than or equal to 0, and whatever the constraint in the primal
problems a x minus a is always less than or equal to 0. So, this employees that this
quantity is greater than or equal to 0 or less than equal to 0, this indicate that results is
less than or equal to 0. Means, y transpose A x minus e is less than or equal to 0, just
now we have already seen 2 equivalent equal and one is it is telling that is equal to
greater than 0.

Another case it is telling this equal to less than 0, but both are equal value so this cannot
be true this is only if is 2 only if x transpose A transpose y minus d equal to y transpose
A x minus e equal to 0. In other words we can write x transpose individually this two
terms y of d is equal to 0, and y transpose A x minus e is equal to 0. So, this is proved to
the theorem.

So, using this theorem one can get the values of dual values dual variables values from
the primal variable values and vice versa. I can get the primal variables values after
solving the dual problem by using this theorem and vice versa. So, this is the theorem let
us workout one simple example and see how 1 can solve such type of problems let us see
that example.

(Refer Slide Time: 34:54)

Consider the primal dual problem pair given by just here we will solve here, we know
the solution of dual problem how to get the solution of primal to solving the what is the
primal problem in solving using L P problems simplex method. So, let us call our primal
problem is maximize z of p is given 3 x 1 plus 2 x 2 subject to x 1 plus x 2 is less than
80. So, this is the inequality constraints of this type if you do not have at this type, you
have to convert into this type this is equal to this type of form by some manipulations,
we have to do even if you have a equality sign, equality sign then we have to convert into
this form that you know we have already discussed in the details earlier.

So, that equals this part we have written is e 1 if you recollect then we have a another
equations inequality conditions to x 1 plus plus x 2 is less than or equal to hundred is e 2
this is the our e 2 right hand side of the inequalities. And we have one equation are there
x 1 is less than 40. That means it e 3 40 is equal to right hand side of equality is 40, so if
just write it in terms of our notation that is d, d transpose of this is 3 2 I can write x 1 x 2.
So, this is nothing but a our g transpose this is nothing but our x so this set of equation I
can write it into matrix form, that is A into x is less than equal to vector e whose
dimension is 3 cross 1. And now identify what is our A, A is in our case if from this 3
equation one can write 1 1 2 1 1 0 this is our A, and this is what is e, e is our if you see
that 80, then 100, then our 40 is our e. So, it is a standard L P problem, what is dual what
is primal problems?

(Refer Slide Time: 38:13)

Now the corresponding dual problems, dual problem minimize f of d now you see what
is this? This will come as a coefficient in the dual problems of the objective functions.
So, that is 80 y 1 plus 100 y 2 how many variables will be there in the dual problems
saying as the number of inequalities present in the primal problems, plus 40 into y 3 and
subject to so this coefficient first column of this coefficient multiplied by y 1 y 2 y 3. So,
it will be a y 1 plus twice y into plus y 3 is greater than or equal to first coefficient of the
primal objective function 3. That is then next is y 1.

Similarly, I can write it y 1 plus y 2 greater than or equal to 2 that we denote it by d 1, d

1 coefficient is this is d 2 and then we have a because we have a two coefficients see here
so two inequality constraints will be there in dual problems, and y 1 y 2 and y 3 will
greater than or equal to 0. And what is our A transpose if you write in this set of
dependent is nothing but A transpose y is greater than equals to our d, and then you see
our a transpose is nothing but A what is A got it that transpose 1 to this matrices, if you
write this matrices y into form is 2 1 1 1 0 and it is nothing but a what are the A is got it
that transpose, agree? Than our y we know y 1 y 2 y 3 and d our case is these again case
is 3 2 and it is nothing but a we can write it this one, if is see this one we can write it e
transpose y, agree? So if you see the what e is this 1.

So you can solve this formula and by suppose this is here given primal problems and if
you are asked to solved, show that whether I can say show that, that x transpose x which
dimension is 2 cross 1 is equal to 20, 60 transpose is an optimal solution of the primal
problem, that is you show it. Once in a optimal solution of the primal problem that x 1 x
2 x 1 is 20, x 2 is 60 must be feasible solution of the primal problems.

So, let us see how one can solve this problem by that is first you solve be to this one.
Suppose, you want go back to from one solution to the what is call the I want to go this is
the final solution you have solve it that is I have to show it, so end no the solution.
Suppose, I want to know the what is dual problem solution that means y 1 y 2 y 3 start,
what is it solution of this one that I can once I know the solution of final problem, I can
get the solution of dual problem by using the our that theorem plus theorem, we have
discuss or if you have know the solution of the dual problem. We can go back with the
solution of our final problem by using that theorem.

(Refer Slide Time: 43:19)

Now, see our the complementary condition that implies conditions is relationship
between the dual variables and the primal variables are this two equation. If you see a y
transpose ((Refer Time: 43:22)) A into x minus e is equal to 0. And this is 1 cross 1 and
another condition is x transpose A y of this minus d is equal to 0, this is 1 cross 1. So,
this now let us see that once you know this one, how can you find out y start that means,
if you know the final solution how you get the dual problem solution y 1 y 2 y 3, let us
see. So, you can write it that y has a 3 components y 1 y 2 y 3 is a called as transpose is a
becoming a row vector.

And the theorem tells if you see this theorem it tells that if you have a what is called here
if you x and the solution of the dual problem, or primal or optimal then this equal to this
optimal solution. And our problem is given show that the optimal solution is that the find
out dual problems y 1 y 2. So, I can write it this or I minute like this is in star of y star,
star indicates the optimal solution of this one, multiplied by A x minus e and that already
we have seen our problem this A x minus e problem if you see this is A x minus if take
this is that side is that that one only.

So, x 1 plus x 2 minus 80 so x 1 star plus x 2 star minus 80 is 1, first equation then
second equation 2 x 1 you see 2 x 1 plus x 2 star 2 x 1 star plus x 2 star minus 100 then
the third 1 x 1 star minus 40, this equal to 0, 1 cross 1. Now, look this one if you expand
that that 1 y 1 star multiplied by this into this x 1 star plus x 2 star minus 80 plus y 2 star
2 x 1 star plus x 2 star minus 100 plus y 3 star plus into x 1 star minus 40 is equals to 0
look at this expression y star value is what? We expect that value must be equal to get to
an equal to 0.

And what is this values this values, if you see from the what is called in equality constant
of primal problem x 1 plus x 2 minus 80 is less than equal to 0. So, this quantity is less
that equal to 0, so whatever resultant of this one we will get tell me, greater than equal to
0 and this quantity is great than is less than equal to 0. So, the results will be less than
this products of this one is less than is equal to 0, this one. Now, what about this one
similarly, this is greater that equal to 0 and this is 2 x 1 plus x 1 minus 100 is less than
equal to 0 less than equal to 0.

So, resultant of this one will be product of this will be less than equal to 0 similarly, this
one in the same logic you can write y 1 this y 3 is greater than equal to 0, and this one
less than equal to 0 and the product is less than equal to 0. So, whole thing is 0, so there
is no possibility of cancelling one and other either to be negative if was what situation, if
all be negative which does not satisfy this right in this side. So, this only satisfy in each
component of this one is 0.

So, I can write it therefore y 1 star into x 1 star plus x 2 star minus 80 is equal to 0, and
then y 2 star is 2 x 1 star plus x 2 star that component minus 100 is equal to 0. And the
last 1 is your y 3 star x 1 star minus 40 is equals to 0, and this set of equation I got it
from this equations, this condition and this condition is getting from what is called the
theorem that if x and y are the 2 optimal solutions of the dual pair, then you can write
this and this equalities here only then if it is rare.

Now, look at this expression we have what solution is given and final solution is given x
star is 20 and 60. That means x 1 is 20, so in this equation if you put x 1 is 20. That
means this quantity is not equal to 0 then it denote what to make this is 0, y 3 must be 0
so y 3 star is equal to 0. Since, x 1 star is not equal to 40 again our case in your case x 1
star is what? X 1 star is 20, that why we got it x 3 is the dual problem solution x 3 star is
30 agree? But now look at this expression that x 1, x 2 now from the set of equation this
set of equation what you can write is from b conditions.

(Refer Slide Time: 50:30)

From b conditions similarly with the same logic If you expand that one you will get it
from b condition you will get it x 1 star, y 1 star plus twice y 2 star plus y 3 star, minus 3
is equal to 0. You see I am just applying that x 1 x 1 star is x 1 stands and x 1 transpose
then x 2 x 3 and then multiplied by A transpose y minus d you will no A y transpose
minus d that is our A y transfer minus d is this equation. So, I am writing this equation
now, so x 1 star into twice y 2 star plus minus plus y 3 minus 3 is equals to 0, because if
you expand in the similar manner with same logic, I can say that in the individual
component must be 0. So, this is 0 agree? And another condition I am getting it here x 2
star into that equation A transpose minus y minus d that one that y 1 star plus y 2 star
minus 2 is equals to 0, agree?

So you the value of x 1 is 20, this value is 20 primal solution and this value is our 60 or
you see this 1 is given that problem is given 20 and 60, so this is 60 so this implied that y
1 star plus twice y 2 star plus y 3 star minus 3 is equals to 0. And this implies this since it
is 60 cannot be only possible that y 1 star plus y 2 star minus 2 is equal to 0. And now
this value just now you have found out from condition A is 0, so we have to solve the
two equations now y 1 star plus y 2 star minus 3 is equal to 0, another is y 1 star plus
twice y 2 star minus 2 is equal to 0. So, solve this equations we get we get y 1 star is
equal 1, y 2 star is equal to 2. So, our solution in the solution on the dual variables the
optimal solution of the dual variables, we got it y 1 star is equal 1, y 2 is equal to 2, y 3 is
equal to 0.

And clearly if you see the graphical representation of this problem it is a feasible
corresponding function values is what? You can get it value in this equation, in which
equation we have to put in this value that f of d our nothing but e transpose y our e is
what 30 into 18 into 1, 100 into if you see y 1 value this is the our dual problem 18 into
1, 100 into 2. So, 200, 280 and y 3 value is 0 similarly, if you in the primal problems
here, primal problems our solution is just now we have assumed the solution is 20 n, 20 n
60 what is this problems, just a minute.

So, this our this is our primal problem is that one, if you just, if you just apply to use the
20 and 60 is an optimal solution of the our primal problems this in the objective function,
if you put the objective function x is equal to 20 and 60, you will the same objective
value function as you got of the dual problems.

So, I will just show you this results now here, here you see here you see optimal solution
of this one what do you got 20, 20 into 3 20 into 60 and this is 60, 60 into 2, 120, 120
and 60 180 and dual problems. What the results we got it, x 1 y 1 is our 1 y 2 is 2 and our
that, that value just if you see this one that value will got also 180 just now we told it you
got it 180 in the dual problems here 80 y 1 is 80 y 2 is 2, y 280 here you are getting 60
into 2, 120 into 60 180 you will get it the same as you did in the both cases. So, we will
stop it here ok.