NP-completeness

Pedro Ribeiro

DCC/FCUP

2018/2019

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 1 / 32

Computationally Hard Problems

Good news: Almost all the algorithms we’ve studied thus far can be
solved ”quickly”, i.e., they are polynomial-time algorithms: on
inputs of size n, their worst-case running time is O(nk ), for some
(typically small) constant k.

Bad news: Not all problems are like these....

I There are problems for which there is no solution: halting problem
I There are problems for which there is a solution, but not in polynomial
time...

This lecture will focus on this last set of problems. We want evidence
that some problems are intrinsically hard. We will be particularly
interested in NP-complete problems.

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 2 / 32

Halting Problem
The Halting Problem
Input: A program (turing-complete model) and an input
Output: Yes/No Answer: does the program halt (stop) when run?

For instance, consider the two following programs:

I while (true) continue
Continues always in an infinite loop and never halts
I write "Hello World!"
Does halt
On simple programs like these it is easy to decide, but on more
complex program this is really problematic...
We could run the program for a certain number of steps and see if it
stops. But if the program does not halt, how do we really know if will
eventually halt or continue forever?
This is an historically important problem because one of the first to
be proved as undecidable (by Alan Turing in 1936)
Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 3 / 32
Computationally Hard Problems
Why should we care?

NP-complete problems really come up all the time. Knowing

they’re hard lets us stop ”beating our head against a wall” trying to
solve them optimally, and instead:
I Use an heuristic: if I can’t quickly solve the problem with a good
worst case time, maybe I can come up with a method for solving a
reasonable fraction of the common cases.
I Use an approximation: a lot of the time it is possible to come up with
a fast algorithm, that doesn’t solve the problem exactly but comes up
with a solution we can prove is close (enough) to the optimal.
I Use an exponential solution: If you really have to solve the problem
exactly, you can settle down to writing an exponential time algorithm
and stop worrying about finding a better solution.
I Use a better abstraction: the hardness of the problem may come
from ignoring some of the seemingly unimportant details of a more
complicated real world problem. This can make the difference between
what we can and can’t solve.
Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 4 / 32
Computationally Hard Problems
Some examples

Sometimes, a problem may seem very similar to one we know that we

can solve polynomially, but still they are really hard...

Shortest vs Longest Path:

Given a weighted graph, finding the shortest path between two nodes
u and v can be solved polynomially (ex: Dijkstra and Bellman-Ford
algorithms). However, if we want to discover the longest path
between to nodes, we have an hard problem...

Eulerian Path vs Hamiltonian Path:

Given a graph, finding a path that traverses all edges exactly once (an
eulerian path) can be solved polynomially. However, finding a path
that traverses all nodes (hamiltonian path) is an hard problem...

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 5 / 32

Computationally Hard Problems
Some examples

Graph 2-coloring vs 3-coloring:

Given a graph, a k-coloring is a way to assign k colors to each node
such that no two adjacent nodes have the same color. Deciding
whether a graph admits a 2-coloring can be solved polynomially, while
deciding if it admits a 3-coloring is an hard problem...

2-CNF satisfiability vs 3-CNF satisfiability:

A k-CNF (conjunctive normal formula) is an AND of clauses of ORs,
each with with k boolean variables or their negations. For instance,
(x1 ∨ ¬x2 ) ∧ (¬x1 ∨ x3 ) ∧ (¬x2 ∨ ¬x3 ) is in 2-CNF form. Knowing if
2-CNF is satisfiable (if there is an attribution of values to the
variables that causes the formula to evaluate to true) can be solved
in polynomial time, but 3-CNF satisfiability is an hard problem...

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 6 / 32

Decision vs Optimization/Search Problems
Many problems of interests are optimization problems: there are
many valid solutions and we wish to find the best solution.
Here we will however (mainly) deal with decision problems, in which
the answer is simply a boolean value : YES or NO.
Nevertheless, usually we can produce a decision version of any
optimization problem which is related to it, in the sense that if the
optimization problem is ”easy”, than so would be the decision
problem. Hence, if we can show evidence that the decision problem is
”hard”, we are also showing that the optimization version is ”hard”.
Ex: Computing the shortest path between 2 nodes (SHORTEST-PATH)
is an optimization problem. The related decision problem (PATH) is:
given a graph G , nodes u and v and a constant k, is there a path
between u and v of at most cost k? If we have an efficient solution
for SHORTEST-PATH, than solving PATH becomes simply running
that solution and checking if the answer is ≤ k.
Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 7 / 32
Classes of Problems
A first informal notion

During the lecture we will be mainly referring to three classes of problems:

P (polynomial time). The set of decision problems that can be solved

in polynomial time. Intuitively, P is the set of problems that can be
solved ”quickly”.

NP (nondeterministic polynomial time). The set of decision problem

with the following property: if the answer is YES, then there is a proof
of this fact that can be checked in polynomial time. Intuitively, NP is
the set of problems where we can verify a YES answer quickly, if we
have the answer in front of us.

coNP is essentially the opposite of NP. if the answer is NO, then there
is a proof of this fact that can be checked in polynomial time.

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 8 / 32

Classes of Problems
Some examples

Let’s look again at some of the problems we referred to:

SHORT: knowing if there is a path of cost at most k between two

nodes is NP because if we have the path itself, than checking if it is
correct is just traversing it seeing if sum of the edges is ≤ k.
HAMILTON: knowing if there is an hamiltonian path is NP because
if we have the path itself, than checking if it is correct is just
traversing it and making sure we don’t visit any nodes more than once
3-COLOR: knowing if there is 3-coloring of a graph is NP because if
we have the coloring, than checking it is just going trough all the
edges and checking they do no not connect nodes of the same color
2-CNF: knowing if 2-CNF is satisfiable is NP because if we have the
values that satisfy the formula, than checking it is just (linearly)
applying the ORs and ANDs and check if the result is indeed true.
Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 9 / 32
P vs NP

Every problem in P is also in NP and in coNP

I If a problem is in P, we can verify YES answers in polynomial time
recomputing the answer from scratch! Similarly, every problem in P is
also in co-NP.

What we don’t know is if every problem in NP is also in P...

P = NP?
I Checking if a solution is correct, is easier than solving the problem
from scratch, no? But nobody knows how to prove it!
I This is one of the most important unanswered questions in Computer
Science (if not from all sciences!)
I P = NP is one of the (7) Millenium Prize problems. The Clay
Mathematics Institute offers an US 1, 000, 000 for anyone solving it.
This reward was posted in 2000 and in fact nobody solved it (yet?)
Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 10 / 32
Classes of Problems
What I (we?) think about it

(image by Jeff Erickson, UIllinois)

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 11 / 32

Classes of Problems
A typical misconception

It is a very common mistake that NP corresponds to non-polynomial.

This is really incorrect, because:
I We are not really sure if NP problems can or cannot be solved in
polynomial time ( P = NP?)
I There are much more harder classes of problems than NP. For example:
F PSPACE. Problems that can be solved using a polynomial amount of
space.
F EXPTIME. Problems that can be solved in exponential time.
F EXPSPACE. Problems that can be solved using an exponential
amount of space
P ⊆ NP ⊆ PSPACE ⊆ EXPTIME ⊆ EXPSPACE
F Undecidable. Problems we cannot solve.

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 12 / 32

Classes of Problems
Continuing the informal notions

There are two other important classes of problems we should know:

NP-hard (or NPH). A problem that is as hard as any problem in NP,
that is, if it can be solved in polynomial time, than every NP problem
can also be solved in polynomial time.
NP-complete (or NPC). A problem that is both NP-hard and in NP.
Note that an NP-hard problem needs not to be a decision problem, nor it
needs to have a way of checking solutions in polynomial time

(image by Jeff Erickson, UIllinois)

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 13 / 32

Problem Reduction
Consider we want to solve a problem A in polynomial time.
Suppose we know how to solve a problem B in polynomial time.
Suppose we have a procedure to transform any instance α of problem
A into an instance β of problem B, so that
I The transformation takes polynomial time
I The answers are the same (the answer to α is YES iff the answer for β
is YES).
We now have a way of solving problem A in polynomial time!
I Given any instance α of A, transform it into an instance β of B
I Run the polynomial time algorithm for B on the instance β
I Use the answer for β as the answer for α

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 14 / 32

Problem Reduction

What we did is called a problem reduction. We reduced problem A

to problem B!
Reduction
A Problem A is poly-time reducible to problem B (written as A ≤p B) if
we can solve problem A in polynomial time given a black-box algorithm for
problem B.

Recall we defined NP-hard as ”a problem that is as hard as any

problem in NP”. What does this mean? We now have a possible
answer:
Q is NP-hard if for any other problem X in NP, X ≤p Q!
If I can solve Q in polynomial time, than I can solve any problem in
NP in polynomial time!

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 15 / 32

Problem Reduction

This is very useful in order to prove a problem is NP-complete!

I Suppose we have a problem A that we know is NP-hard
I We want to show that problem B is NP-complete
I Besides proving that B is in NP (usually ”easy”), all we need to do is
to show that A ≤p B (A is reducible to B).
I A polynomial time algorithm for problem B would imply a polynomial
solution for problem A! So B is as hard as A
I Note that this is not valid in the ”other” direction, ie, with B ≤p A )

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 16 / 32

A first NP-complete problem

All we are missing now is a ”root” problem that we know is

NP-complete! All other problems can ”become” NP-complete by
reducing to this problem...

The Cook-Levin Theorem

SAT (Boolean Satisfiability) is NP-complete

SAT - Is there an ”interpretation” that satisfies a general a boolean

formula written? (in other versions SAT concerns only CNF formulas)
Is there an assignment of values TRUE or FALSE to the variables in such a
way that the formula evaluates do TRUE?
We’ll give a brief sketch of a proof of why SAT is NP-complete

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 17 / 32

CIRCUIT-SAT

Consider the problem CIRCUIT-SAT: determining if a boolean circuit is

satisfiable:

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 18 / 32

SAT and CIRCUIT-SAT

To prove SAT is NP-complete we need to prove:

1) SAT is in NP
2) SAT is NP-hard

1) SAT is in NP means we have a proof of satisfiability we can check in

polynomial time.
Imagine we have a valid assignment that satisfies a formula. To check it is
correct we just need to ”evaluate” the formula! And we can (trivially) do
it in polynomial time

2) To prove SAT is NP-hard we will use CIRCUIT-SAT. Let’s start by

reducing CIRCUIT-SAT to SAT (CIRCUIT-SAT ≤p SAT). If we can show
this, then all we need afterwards is to show that CIRCUIT-SAT is NP-hard
(in fact it is NP-complete, like SAT)

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 19 / 32

Reducing CIRCUIT-SAT to SAT

Given a boolean circuit, we can transform it into a boolean formula by

creating new output variables for each gate, and then just writing down
the list of gates separated by ANDs. For example:

(image by Jeff Erickson, UIllinois)

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 20 / 32

NP-hardness of CIRCUIT-SAT
Now we still need to prove that CIRCUIT-SAT is NP-hard (and this will be
our ”root”). A very brief sketch of a proof, giving the intuition:

The ”simple but big idea”: every program is essentially a circuit!

Consider a problem A which is in NP. Then for every instance x we have a

proof that we can verify with algorithm A in polynomial time.

We can convert the verification algorithm A into a circuit C that simulates

the algorithm step by step.

Since we know that ”everything” (including the number of steps A takes)

is polynomially bounded by the size of x, then this circuit can be
constructed in polynomial time.

We end up with a circuit C that is satisfiable if and only if the input to our
problem has an YES answer!
Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 21 / 32
3SAT is NP-complete

We now know that CIRCUIT-SAT and SAT are NP-complete. We can use
it as the basis for proving any other problem is also NP-complete.

Let’s start with a more ”restricted” version of SAT:

3SAT: is a 3-CNF formula satisfiable?

(recall CNF is conjunctive normal form)

Again, what we need to prove is the following:

1) 3SAT is in NP
2) 3SAT is NP-hard

1) is ”trivial”. If we have the assignment that evaluates to TRUE, than we

can just (linearly) evaluate the formula.

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 22 / 32

3SAT is NP-complete

Now we have two choices for problem we can reduce to 3SAT: SAT and its
”circuit equivalent”: CIRCUIT-SAT. Let’s choose this last one.

A trivial algorithm could end up having an exponential number of clauses.

But we can do the following:

1) Make sure every gate has only two inputs at most. If any gate has
k > 2 inputs, replace it with a binary tree of k − 1 input gates.

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 23 / 32

3SAT is NP-complete
2) Replace every gate by a CNF formula. Using the three types of gate
NOT, AND, OR (we could have more gates, but they can essentially be
represented by these):

3) Now transform every clause with one or two literals into a clause with
three literals, introducing new variables:

(images by Jeff Erickson, UIllinois)

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 24 / 32

3SAT is NP-complete

In the end we get a 3CNF formula in which each gate corresponds to at

most 5 clauses! Hence, 3SAT is NP-complete. For instance, the formula
for the circuit given before would become:

(image by Jeff Erickson, UIllinois)

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 25 / 32

IndSet is NP-complete (from 3SAT)
We now have 3 NP-complete problems: CIRCUIT-SAT, SAT and 3SAT

Let’s continue our ”journey”, but now going to the graphs realm.

Independent Set (IndSet): consider an unweighted graph G . An

independent set in G is a subset of vertices of G with no edges between
them.
Optimization version: What is the size of the largest independent set in G ?
Decision version: Does G contain an independent set of size ≥ k?

Example of an independent set of size 3:

IndSet is in NP, since we can check if a solution is valid by just iterating

trough all nodes and checking if there is no connection to any other node.
Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 26 / 32
IndSet is NP-complete (from 3SAT)
Let’s now describe a reduction from 3SAT. We need to describe a way
to (polynomially) transform a 3CNF formula into a graph that has an
independent set of a certain size only if the formula is satisfiable
Construct a graph of where we have one node for each literal of each
clause. Two nodes are connected if: (1) they correspond to literals in the
same clause; (2) they correspond to a variable and its negation.
For instance, the formula (a ∨ b ∨ c) ∧ (b ∨ ¬c ∨ ¬d) ∧ (¬a ∨ c ∨ d) ∧ (a ∨ ¬b ∨ ¬d)
is transformed into the following graph:

(image by Jeff Erickson, UIllinois)

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 27 / 32
IndSet is NP-complete (from 3SAT)

Now, suppose that the initial formula had k clauses. Then the formula is
satisfiable if and only if the graph has an independent set of size k
IndSet → Satisfying: if the graph has an independent set of k vertices, then
each vertex comes from a different clause. To obtain a satisfying assignment, we
assign TRUE to each literal in the independent set. Since contradictory literals
are connected by edges, this is consistent. There may be variables that have no
literal in the independent set; we can set these to any value we like. The resulting
assignment satisfies the original 3CNF formula.

Satisfying → IndSet: if we have a satisfying assignement, then we can choose

at least one literal in each clause that is TRUE. Those literals form an
independent set in the graph.

MaxIndSet is NP-complete. 

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 28 / 32

VERTEX-COVER is NP-complete (from IndSet)

VERTEX-COVER: a vertex cover is a set of nodes such that each edge it

incident/adjacent to at least one of them
Optimization version: What is the size of the smallest vertex cover of G ?
Decision version: Does G contain a vertex cover of size ≤ k?

Example of a vertex cover of size 4:

VERTEX-COVER is in NP, since we can check if a solution is valid by

just checking for all edges it they are covered by one of the selected nodes

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 29 / 32

VERTEX-COVER is NP-complete (from IndSet)
Let’s now describe a reduction from IndSet.
If C is a vertex cover of graph G with vertex set V , then V − C is an
independent set. Also, if S is an independent set, then V − S is a vertex
cover.

So the reduction from IndSet to VERTEX-COVER is very simple: given an

instance (G , k) for IndSet, produce the instance (G , n − k) for
VERTEX-COVER, where n = |V |. In other words, to solve the question
”is there an independent set of size ≥ k, just solve the question ”is there a
vertex cover of size ≤ n − k.

VERTEX-COVER is NP-complete. 
Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 30 / 32
CLIQUE is NP-complete (from IndSet)

Clique (CLIQUE): a clique is a complete graph, that is, a graph where

every pair of nodes is connected by an edge.
Optimization version: What is the size of the largest subgraph of G which
is a clique?
Decision version: Does G contain a clique of size ≥ k?

Example of a clique of size 4:

Clique is in NP, since we can check if a solution is valid by just checking

if all pairs of nodes are connected.

Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 31 / 32

CLIQUE is NP-complete (from IndSet)
Let’s now describe a reduction from IndSet.
Let H be the complement or inverse of a graph G (it has the same
nodes such that two distinct are connected if and only if they are not
connected in G ). If S is a vertex cover of graph G , then S is a clique in H.

So the reduction from IndSet to CLIQUE is very simple: given an instance

(G , k) for IndSet, produce the instance (H, k) for CLIQUE. In other words,
to solve the question ”is there an independent set of size ≥ k, just solve
the question ”is there a clique of size ≥ k in the complement graph”.

CLIQUE is NP-complete. 
Pedro Ribeiro (DCC/FCUP) NP-completeness 2018/2019 32 / 32