Approximation Algorithms

Introduction
Approximation algorithms are used to get a
solution close to the (optimal) solution of an
optimization problem in polynomial time
For example, Traveling Salesman Problem (TSP) is
an optimization problem (the route taken has to
have minimum cost).
Since TSP is NP-Complete, approximation
algorithms allow for getting a solution close to the
solution of an NP problem in polynomial time.
 Definition
An algorithm is an α-approximation algorithm for an optimization
problem Π if
1. The algorithm runs in polynomial time
2. The algorithm always produces a solution that is within a factor of α of
the optimal solution
Minimization – α < 1
Maximization – α > 1
Set Cover (SC)
A set cover of a set T is any collection of subsets of T whose union is T.
The set cover problem: given a weight for each subset, find the set
cover which minimizes the total weight
Example of Set Cover
Set T = { 1, 2, 3, 4, 5, 6, 7, 8 }
Available subsets:
1. S1 = { 1, 2, 3 } w1 = 1
2. S2 = { 2, 7, 8 } w2 = 2
3. S3 = { 4, 5, 6, 7 } w3 = 3
4. S4 = { 4, 5, 6, 8 } w4 = 4
Solutions: C = { S1, S2, S3 }, C = { S1, S2, S4 }
Optimal Solution is C = { S1, S2, S3 }
Weighted Vertex Cover
(WVC)
Here, we try to find a collection of vertices of a graph such that each
edge of the graph contains at least one vertex from the collection
Each vertex has a weight and we try to find the collection that
minimizes the total weight
Example of Vertex
Cover
Possible Solutions The Graph
are:
1. { 1, 3, 4 } 1
2. { 1, 7, 5 } 2
7
If the weights are the
vertex numbers, then 3
the optimal solution 6
is: {1, 3, 4} 4
5
Vertex Cover as Set
Cover
Vertex Cover is a special case of Set Cover
To convert a problem from vertex cover to set
cover:
1. Make T = the set of edges E
2. Make the subsets correspond to each vertex,
with each subset containing the set of edges
that touch the corresponding vertex
 Example of VC as SC
T = { (1, 3), (3,7), (1, 7), (4, 5), The Graph
(4, 6) }
Subsets: 1
1. S1 = { (1, 3), (1, 7) } w = 1 2
7
2. S3 = { (1, 3), (3, 7) } w = 3
3. S7 = { (1, 7), (3, 7) } w = 7 3
4. S4 = { (4, 5), (4, 6) } w = 4 6
5. S6 = { (4, 6) } w = 6 4
5
6. S5 = { (4, 5) } w = 5
Algorithm for
Unweighted Set Cover
I0
while T 0 
pick ti  T
I  I   j : ti  S j 
TT S
jI
j
Proof of Algorithm
Elements are deleted from T only when they are covered, and we
delete at least one each time in the loop, at the end of which, no
elements are left in T.
Therefore, the algorithm returns a set cover.
Definition of f
Define f max ti T  j : ti  S j 
In other words, f is the number of subsets in the set cover that cover
that element that is in the most subsets in the set cover
For VC problem, f = 2 because each element in the set is covered always
by 2 subsets (representing the two endpoints of the edge).
Algorithm is f-
approximation (1)
The Algorithm runs in polynomial time
During each iteration of the loop, a ti chosen is
covered by at least one distinct subset that is in the
optimal solution.
This can be shown by contradiction: If there was a
tk chosen in a later iteration that is also in that
same Sj then it would have to remain in T after we
chose tj. But when tj was chosen all sets that
contain it were chosen too and all their elements
were removed from T including tk.
Algorithm is f-
approximation (2)
Therefore, if there are n iterations through the loop, n
is less or equal to the number of sets in the optimal
solution (OPT).
Each iteration in the loop we pick at most f sets to add
to I.
Therefore, |I| ≤ f * n ≤ f * OPT.
Therefore, the algorithm is an f-approximation.
However, this algorithm is not designed to solve the
weighted problem, which is what we wish to solve.
Linear Programming
(LP)
Linear programming is the problem of optimizing a
linear function subject to linear inequality
constraints.
The function being optimized is called the objective
function.
The function with the constraints is called the
Linear Program.
Any assignment of variables that satisfies the
constraints is called a feasible solution.
An example of LP
Minimize 7x1 + x2 + 5x3
Constraints:
◦ x1 - x2 + 3x3 ≥ 10
◦ 5x1 + 2x2 - x3 ≥ 6
◦ x1 ≥ 0
◦ x2 ≥ 0
◦ x3 ≥ 0
Standard Form of LP
All constraints are of type greater or equal in minimization LP, and less
or equal in maximization LP.
All variables are constrained to be non negative.
By a simple transformation any linear program can be written as a
standard minimization or maximization LP.
Upper Bound for
Minimization
Any upper bound (for the minimization) can be checked by simply
finding a solution that satisfies the equation in less than the upper
bound.
(2, 1, 3) is such an example for upper bound 30.
Lower Bound for
Minimization
To find a lower bound, we can note that 7x1 + x2 + 5x3 ≥ x1 - x2 + 3x3 ≥ 10,
because all coefficients are greater than the corresponding coefficients
and all variables are non-negative.
An even better lower bound can be obtained by doing: 7x1 + x2 + 5x3 ≥
(x1 - x2 + 3x3) + (5x1 + 2x2 - x3) ≥ 16
Maximizing Lower
Bound
The minimum solution of the objective function will be obtained when
we find the maximum lower bound for the function.
Particularly, it can be formulated as a linear program:
Maximization LP of
Lower Bound
Maximize 10y1 + 6y2
Constraints:
◦ y 1 + y2 ≤ 7
◦ -y1 + 2y2 ≤ 1
◦ 3y1 - y2 ≤ 5
◦ y1 ≥ 0
◦ y2 ≥ 0
 Min-Max (Primal-Dual)
Equivalency
Primal Problem Dual Problem

n m
minimize c x
j 1
j j maximize b y i i
i 1
n m
i(1,,m )  aij x j bi  j(1,,n )  aij yi c j
subject to j 1
subject to i 1
 j(1,,n ) x j 0 i(1,,m ) yi 0
Relation between the
Min/Max LPs
The first LP will be called the Primal Program and
the second LP will be called the Dual Program.
There is a systematic way of finding the Dual of any
Primal. Furthermore, the Dual of the Dual of X is X
itself.
By construction, every feasible solution to the dual
gives a lower bound on the optimum of the primal.
Also, every feasible solution to the primal gives an
upper bound on the optimum of the dual.
LP-Duality (1)
Furthermore, if the primal solution is equal to the
dual solution (say, some value, x) then that means:
◦ x is a lower bound on optimum of primal
◦ x is an upper bound on optimum of dual
◦ If the optimum of the dual was lower than x, then that
optimum is a feasible solution which means that
optimum is a lower bound on the primal (and is less than
x). This is a contradiction.
◦ Similarly, by contradiction, the optimum of the primal
can’t be larger than x.
◦ In other words, x is the optimal solution in this case.
LP-Duality (2)
This is a central theorem of Linear Programming called the LP-Duality
Theorem.
In fact, not only will this happen if the optimum of the dual is equal to
that of the primal but if there is a solution, then it will be equal to both
functions.
LP-Duality Theorem
Refer to Primal-Dual Equivalency Definitions
The primal program has finite optimum iff the dual has finite optimum.
Moreover, if x* and y* are optimal solutions for the primal and dual
respectively, then

n n

 c x  b y
j 1
j

j
i 1
i

i
Weak Duality Theorem
If x is a feasible solution for the primal and y is a feasible solution for the
dual, then

n n

 c x  b y
j 1
j j
i 1
i i
Integer Programming
(IP)
Integer Programming is simply Linear Programming with an added
condition:
◦ All variables must be integers

Many problems can be stated as Integer Programs.

n
minimize  w x
j 1
j j

, x
ti T j 1
subject to : j:ti S j
i(1,,m ) , x j  0,1
Relaxed LP for WSC
We can write the IP to the WSC in a more relaxed
form: n
minimize  w x
j 1
j j

, x
ti T j 1
subject to : j:ti S j
i(1,,m ) , x j 0
Now, if ZLP is the optimum of this LP, then ZLP ≤ OPT,
because any feasible solution for the IP is also
feasible for the LP. So the optimal LP will not be
greater than the optimal IP.
Using Relaxed LP as
Approximation
Given the LP, the optimum is ZLP.
If there is a solution of cost no more than α• ZLP, then the cost is no
more than α• OPT.
Therefore, if the LP is solved for a feasible solution in polynomial time
that is α times the optimum LP value, then that is also an α-
approximation of the IP, and corresponding Set Cover problem.
Basic Technique
1. Write the IP describing the problem.
2. Relax the IP to get an LP.
3. Find the optimal solution to the LP.
4. Find integral values for the (linear) variables such that the solution
is at most α of the optimal solution to the IP.
5. Reformulate the integral values as solution to the problem.
Overview of Basic
Technique
Steps 1 and 2 can be performed in polynomial time,
easily.
Step 5 can also be performed in polynomial time.
Step 3 can be done in polynomial time too, but
sometimes it’s easier to connect it to step 4.
The tricky part is really step 4, and four ways will be
shown to accomplish it.
The methods shown will solve the Set Cover, in
particular, but can be used for any IP.
Method 1: Rounding
This implements step 4 only:

1. I  0
2. for each S j
 1
3. if x j
f
4. I  I   j
Proof: Rounding Method
Produces Set Cover
Assume by contradiction that there is an element ti such that

Then,
ti  S j
And therefore:
 1 ji

Since,
 xj 
S j :ti S j f 1

But this violates the LP constraints.

j x   j : ti  S j  1
j:ti S j f
 j : t  S   f max  j : t  S 
i j ti T i j
Proof: Rounding is f-
approximation
Algorithm
The algorithm is a polynomial time algorithm for step 4 in the Basic
Technique.
Furthermore,
 wj
jI
 w j wj f
j

 f  w j wj
j
The first inequality holds, since  f OPT

j  I only if x f 1 j
Method 2: Dual-LP
This second method is based on the dual solution.
By weak duality, if there is a feasible solution y, then

y
i
i  z LP OPT
Dual LP-Algorithm
First, solve the dual linear program to get an optimal solution y*
Then, apply the following algorithm:

1. I  0
2. for each S j
3. if  yi w j


i:t i S j
4. I  I   j
Proof: Dual-LP Produces
Set Cover
Assume by contradiction that there is an element ti that is not part of
the final result.
In that case,
Method 3: Primal-Dual
Method 4: Greedy
Algorithm
I0
~
j , S j  S j
while T S j
jI
wj
l  arg min ~ ~
j:S j 0
Sj
I  I  l
~ ~ wl
ti  Sl , yi  ~ (used in proof)
Sl H g
~ ~
j , S j  S j  Sl
Greedy Algorithm
Define 1 1 1 1
H n 1       ln n
2 3 4 n
g max S j
j