6.

045: Automata, Computability, and

Complexity (GITCS)

Class 15
Nancy Lynch

Today: More Complexity Theory

Polynomial-time reducibility, NP-completeness,
and the Satisfiability (SAT) problem
Topics:

Introduction (Review and preview)

Polynomial-time reducibility, p
Clique p VertexCover and vice versa
NP-completeness
SAT is NP-complete

Reading:
Sipser Sections 7.4-7.5

Next:
Sipser Sections 7.4-7.5

Introduction

Introduction
P = { L | there is some polynomial-time deterministic Turing
machine that decides L }
NP = { L | there is some polynomial-time nondeterministic
Turing machine that decides L }
Alternatively, L NP if and only if ( V, a polynomial-time
verifier ) ( p, a polynomial ) such that:
x L iff ( c, |c| p(|x|) ) [ V( x, c ) accepts ]
certificate

To show that L NP, we need only exhibit a suitable

verifier V and show that it works (which requires saying
what the certificates are).
P NP, but its not known whether P = NP.

Introduction
P = { L | poly-time deterministic TM that decides L }
NP = { L | poly-time nondeterministic TM that decides L }
L NP if and only if ( V, poly-time verifier ) ( p, poly)
x L iff ( c, |c| p(|x|) ) [ V( x, c ) accepts ]
Some languages are in NP, but are not known to be in P (and
are not known to not be in P ):
SAT = { < > | is a satisfiable Boolean formula }
3COLOR = { < G > | G is an (undirected) graph whose
vertices can be colored with 3 colors with no 2 adjacent
vertices colored the same }
CLIQUE = { < G, k > | G is a graph with a k-clique }
VERTEX-COVER = { < G, k > | G is a graph having a
vertex cover of size k }

CLIQUE
CLIQUE = { < G, k > | G is a graph with a k-clique }
k-clique: k vertices with edges between all pairs in
the clique.
In NP, not known to be in P, not known to not be in
P.
b

c
f

3-cliques: { b, c, d }, { c, d, f }
Cliques are easy to verify, but may be hard to find.

CLIQUE
CLIQUE = { < G, k > | G is a graph with a k-clique }
b
a

c
f

Input to the VC problem: < G, 3 >

Certificate, to show that < G, 3 > CLIQUE, is { b, c,
d } (or { c, d, f }).
Polynomial-time verifier can check that { b, c, d } is a
3-clique.

VERTEX-COVER
VERTEX-COVER = { < G, k > | G is a graph with a
vertex cover of size k }
Vertex cover of G = (V, E): A subset C of V such
that, for every edge (u,v) in E, either u C or v C.
A set of vertices that covers all the edges.

In NP, not known to be in P, not known to not be in

P.
b
a

c
f

d
3-vc: { a, b, d }
e
Vertex covers are easy to verify, may be hard to find.

VERTEX-COVER
VERTEX-COVER = { < G, k > | G is a graph with a
vertex cover of size k }
b
a

c
f

Input to the VC problem: < G, 3 >

Certificate, to show that < G, 3 > VC, is { a, b, d }.
Polynomial-time verifier can check that { a, b, d } is a
3-vertex-cover.

Introduction

Languages in NP, not known to be in P, not known to not be in P:

SAT = { < > | is a satisfiable Boolean formula }
3COLOR = { < G > | G is a graph whose vertices can be colored with
3 colors with no 2 adjacent vertices colored the same }
CLIQUE = { < G, k > | G is a graph with a k-clique }
VERTEX-COVER = { < G, k > | G is a graph with a vc of size k }

There are many problems like these, where some structure

seems hard to find, but is easy to verify.
Q: Are these easy (in P) or hard (not in P)?
Not yet known. We dont yet have the math tools to answer
this question.
We can say something useful to reduce the apparent diversity
of such problems---that many such problems are reducible
to each other.
So in a sense, they are the same problem.

Polynomial-Time Reducibility

Polynomial-Time Reducibility
Definition: A * is polynomial-time reducible to
B *, A p B, provided there is a polynomial-time
computable function f: * * such that:
(w) [ w A if and only if f(x) B ]
*

*
f

B
f

Extends to different alphabets 1 and 2.

Same as mapping reducibility, m , but with a
polynomial-time restriction.

Polynomial-Time Reducibility
Definition: A * is polynomial-time reducible to B *,
A p B, provided there is a polynomial-time computable
function f: * * such that:
(w) [ w A if and only if f(x) B ]
Theorem: (Transitivity of p)
If A p B and B p C then A p C.
Proof:
Let f be a polynomial-time reducibility function from A to B.
Let g be a polynomial-time reducibility function from B to C.
*
A

*
C

Polynomial-Time Reducibility
Definition: A p B, provided there is a polynomial-time
computable function f: * * such that:
(w) [ w A if and only if f(w) B ]
Theorem: If A p B and B p C then A p C.
Proof:
Let f be a polynomial-time reducibility function from A to B.
Let g be a polynomial-time reducibility function from B to C.
*
A

*
C

Define h(w) = g(f(w)).

Then w A if and only if f(w) B if and only if g(f(w)) C.
h is poly-time computable:
h(w)

Polynomial-Time Reducibility
Theorem: If A p B and B p C then A p C.
Proof:
Let f be a polynomial-time reducibility function from A to B.
Let g be a polynomial-time reducibility function from B to C.
*
A

*
C

Define h(w) = g(f(w)).

h is poly-time computable:
|f(w)| is bounded by a polynomial in |w|.
Time to compute g(f(w)) is bounded by a polynomial in |f(w)|,
and therefore by a polynomial in |w|.
Uses the fact that substituting one polynomial for the variable in
another yields yet another polynomial.

Polynomial-Time Reducibility
Definition: A p B, provided there is a polynomial-time
computable function f: * * such that:
(w) [ w A if and only if f(x) B ]
Theorem: If A p B and B P then A P.
Proof:
Let f be a polynomial-time reducibility function from A to B.
Let M be a polynomial-time decider for B.
To decide whether w A:
Compute x = f(w).
Run M to decide whether x B, and accept / reject accordingly.
Polynomial time.

Corollary: If A p B and A is not in P then B is not in P.

Easiness propagates downward, hardness propagates
upward.

Polynomial-Time Reducibility
Can use p to relate the difficulty of two problems:
Theorem: If A p B and B p A then either both A and B are
in P or neither is.
Also, for problems in NP:
Theorem: If A p B and B NP then A NP.
Proof:
Let f be a polynomial-time reducibility function from A to B.
Let M be a polynomial-time nondeterministic TM that decides B.
Poly-bounded on all branches.
Accepts on at least one branch iff and only if input string is in B.
NTM M to decide membership in A:
On input w:
Compute x = f(w); |x| is bounded by a polynomial in |w|.
Run M on x and accept/reject (on each branch) if M does.
Polynomial time-bounded NTM.

Polynomial-Time Reducibility
Theorem: If A p B and B NP then A NP.
Proof:

Let f be a polynomial-time reducibility function from A to B.

Let M be a polynomial-time nondeterministic TM that decides B.
NTM M to decide membership in A:
On input w:
Compute x = f(w); |x| is bounded by a polynomial in |w|.
Run M on x and accept/reject (on each branch) if M does.
Polynomial time-bounded NTM.
Decides membership in A:
M has an accepting branch on input w
iff M has an accepting branch on f(w), by definition of M,
iff f(w) B,
since M decides B,
iff w A,
since A p B using f.
So M is a poly-time NTM that decides A, A NP.

Polynomial-Time Reducibility
Theorem: If A p B and B NP then A NP.
Corollary: If A p B and A is not in NP, then B is
not in NP.

Polynomial-Time Reducibility
A technical result (curiosity):
Theorem: If A P and B is any nontrivial language
(meaning not , not *), then A p B.
Proof:

Suppose A P.
Suppose B is a nontrivial language; pick b0 B, b1 Bc.
Define f(w) = b0 if w A, b1 if w is not in A.
f is polynomial-time computable; why?
Because A is polynomial time decidable.
Clearly w A if and only if f(w) B.
So A p B.

Trivial reduction: All the work is done by the decider for A,

not by the reducibility and the decider for B.

CLIQUE and VERTEX-COVER

CLIQUE and VERTEX-COVER

Two illustrations of p.
Both CLIQUE and VC are in NP, not known to be
in P, not known to not be in P.
However, we can show that they are essentially
equivalent: polynomial-time reducible to each
other.
So, although we dont know how hard they are, we
know they are (approximately) equally hard.
E.g., if either is in P, then so is the other.

Theorem: CLIQUE p VC.

Theorem: VC p CLIQUE.

CLIQUE and VERTEX-COVER

Theorem: CLIQUE p VC.
Proof:
Given input < G, k > for CLIQUE, transform to input
< G, k > for VC, in poly time, so that:
< G, k > CLIQUE if and only if < G, k > VC.

Example:
G = (V, E), k = 4

G = (V, E), k = n k = 3

Other n k = 3
vertices
Clique of size k = 4

k vertices

Size n k
Vertex cover

CLIQUE and VERTEX-COVER

< G, k > CLIQUE if and only if < G, k > VC.
Example: G = (V, E), k = 4, G = (V, E), k = n k = 3

Other n k = 3
vertices
Clique of size k = 4

k vertices

Size n k
Vertex cover

E = (V V) E, complement of edge set

G has clique of size 4 (left nodes), G has a vertex cover of
size 7 4 = 3 (right nodes).
All edges between 2 nodes on left are in E, hence not in E,
so right nodes cover all edges in E.

CLIQUE and VERTEX-COVER

Theorem: CLIQUE p VC.
Proof:
Given input < G, k > for CLIQUE, transform to input < G, k > for
VC, in poly time, so that < G, k > CLIQUE iff < G, k > VC.
General transformation: f(< G, k >), where G = (V, E) and |V| = n,
= < G, n-k >, where G = (V, E) and E = (V V) E.
Transformation is obviously polynomial-time.
Claim: G has a k-clique iff G has a size (n-k) vertex cover.
Proof of claim: Two directions:
Suppose G has a k-clique, show G has an (n-k)-vc.
Suppose C is a k-clique in G.
V C is an (n-k)-vc in G:
Size is obviously right.
All edges between nodes in C appear in G, so all are
missing in G.
So nodes in V-C cover all edges of G.

CLIQUE and VERTEX-COVER

Theorem: CLIQUE p VC.
Proof:
Given input < G, k > for CLIQUE, transform to input < G, k > for
VC, in poly time, so that < G, k > CLIQUE iff < G, k > VC.
General transformation: f(< G, k >), where G = (V, E) and |V| = n,
= < G, n-k >, where G = (V, E) and E = (V V) E.
Claim: G has a k-clique iff G has a size (n-k) vertex cover.
Proof of claim: Two directions:
Suppose G has an (n-k)-vc, show G has a k-clique.
Suppose D is an (n-k)-vc in G.
V D is a k-clique in G:
Size is obviously right.
All edges between nodes in V-D are missing in G, so must
appear in G.
So V-D is a clique in G.

CLIQUE and VERTEX-COVER

Theorem: VC p CLIQUE.
Proof: Almost the same.
Given input < G, k > for VC, transform to input < G, k >
for CLIQUE, in poly time, so that:
< G, k > VC if and only if < G, k > CLIQUE.

Example:
G = (V, E), k = 3

G = (V, E), k = 4

3-VC
4-clique

CLIQUE and VERTEX-COVER

< G, k > VC if and only if < G, k > CLIQUE.
Example: G = (V, E), k = 3, G = (V, E), k = 4

3-VC
4-clique

E = (V V) E, complement of edge set

G has a 3-vc (right nodes), G has clique of size 7 3 = 4
(left nodes).
All edges between 2 nodes on left are missing from G, so
are in G, so left nodes form a clique in G.

CLIQUE and VERTEX-COVER

Theorem: VC p CLIQUE.
Proof:
Given input < G, k > for VC, transform to input < G, k >
for CLIQUE, in poly time, so that < G, k > VC iff < G,
k > CLIQUE.
General transformation: Same as before.
f(< G, k >), where G = (V, E) and |V| = n,
= < G, n-k >, where G = (V, E) and E = (V V) E.
Claim: G has a k-vc iff G has an (n-k)-clique.
Proof of claim: Similar to before, LTTR.

CLIQUE and VERTEX-COVER

We have shown:
Theorem: CLIQUE p VC.
Theorem: VC p CLIQUE.
So, they are essentially equivalent.
Either both CLIQUE and VC are in P or
neither is.

NP-Completeness

NP-Completeness
p allows us to relate problems in NP, saying
which allow us to solve which others efficiently.
Even though we dont know whether all of these
problems are in P, we can use p to impose some
structure on the class NP:
NP
A B here means A p B.
P

Sets in NP P might not be

totally ordered by p: we
might have A, B with neither
A p B nor B p A:

NP
A

B
P

NP-Completeness
Some languages in NP are hardest, in the sense that
every language in NP is p-reducible to them.
Call these NP-complete.
NP
Definition: Language B is NP-complete if both
of the following hold:
(a) B NP, and
P
(b) For any language A NP, A p B.
Sometimes, we consider languages that arent, or might
not be, in NP, but to which all NP languages are reducible.
Call these NP-hard.
Definition: Language B is NP-hard if, for any language A
NP, A p B.

NP-Completeness
Today, and next time, well:
Give examples of interesting problems that are NPcomplete, and
Develop methods for showing NP-completeness.

Theorem: B, B is NP-complete.
There is at least one NP-complete problem.
Well show this later.

Theorem: If A, B, are NP-complete, then A p B.

Two NP-complete problems are essentially equivalent
(up to p).

Proof: A NP, B is NP-hard, so A p B by

definition.

NP-Completeness
Theorem: If some NP-complete language is in P,
then P = NP.
That is, if a polynomial-time algorithm exists for any NPcomplete problem, then the entire class NP collapses
into P.
Polynomial algorithms immediately arise for all
problems in NP.

Proof:

Suppose B is NP-complete and B P.

Let A be any language in NP; show A P.
We know A p B since B is NP-complete.
Then A P, since B P and easiness propagates
downward.
Since every A in NP is also in P, NP P.
Since P NP, it follows that P = NP.

NP-Completeness
Theorem: The following are equivalent.
1. P = NP.
2. Every NP-complete language is in P.
3. Some NP-complete language is in P.

Proof:
1 2:
Assume P = NP, and suppose that B is NP-complete.
Then B NP, so B P, as needed.

2 3:
Immediate because there is at least NP-complete language.

3 1:
By the previous theorem.

Beliefs about P vs. NP

Most theoretical computer scientists believe P NP.
Why?
Many interesting NP-complete problems have been
discovered over the years, and many smart people have
tried to find fast algorithms; no one has succeeded.
The problems have arisen in many different settings,
including logic, graph theory, number theory, operations
research, games and puzzles.
Entire book devoted to them [Garey, Johnson].
All these problems are essentially the same since all NPcomplete problems are polynomial-reducible to each other.
So essentially the same problem has been studied in many
different contexts, by different groups of people, with
different backgrounds, using different methods.

Beliefs about P vs. NP

Most theoretical computer scientists believe P NP.
Because many smart people have tried to find fast
algorithms and no one has succeeded.
That doesnt mean P NP; this is just some kind of
empirical evidence.
The essence of why NP-complete problems seem hard:
They have NP structure:
x L iff ( c, |c| p(|x|) ) [ V( x, c ) accepts ],
where V is poly-time.
Guess and verify.
Seems to involve exploring a tree of possible choices, exponential
blowup.

However, no one has yet succeeded in proving that they

actually are hard!
We dont have sharp enough methods.
So in the meantime, we just show problems are NP-complete.

Satisfiability is NP-Complete

Satisfiability is NP-Complete
SAT = { < > | is a satisfiable Boolean formula }
Definition: (Boolean formula):
Variables: x, x1, x2, , y,, z,
Can take on values 1 (true) or 0 (false).

Literal: A variable or its negated version: x, x, x1,

Operations:
Boolean formula: Constructed from literals using
operations, e.g.:
= x ( ( y z ) (y z ) ) ( x z )

Definition: (Satisfiability):
A Boolean formula is satisfiable iff there is an
assignment of 0s and 1s to the variables that makes the
entire formula evaluate to 1 (true).

Satisfiability is NP-Complete
SAT = { < > | is a satisfiable Boolean formula }
Boolean formula: Constructed from literals using
operations, e.g.:
= x ( ( y z ) (y z ) ) ( x z )

A Boolean formula is satisfiable iff there is an

assignment of 0s and 1s to the variables that
makes the entire formula evaluate to 1 (true).
Example: above
Satisfiable, using the assignment x = 1, y = 0, z = 0.
So SAT.

Example: x ( ( y z ) (y z ) ) ( x z )
Not in SAT.
x must be set to 1, so z must = 0.

Satisfiability is NP-Complete

SAT = { < > | is a satisfiable Boolean formula }

Theorem: SAT is NP-complete.
Lemma 1: SAT NP.
Lemma 2: SAT is NP-hard.
Proof of Lemma 1:
Recall: L NP if and only if ( V, poly-time verifier ) ( p, poly)
x L iff ( c, |c| p(|x|) ) [ V( x, c ) accepts ]
So, to show SAT NP, its enough to show ( V) ( p)

SAT iff ( c, |c| p(|x|) ) [ V( , c ) accepts ]

We know: SAT iff there is an assignment to the variables such
that with this assignment evaluates to 1.
So, let certificate c be the assignment.
Let verifier V take a formula and an assignment c and accept
exactly if with c evaluates to true.
Evaluate bottom-up, takes poly time.

Satisfiability is NP-Complete
Lemma 2: SAT is NP-hard.
Proof of Lemma 2:
Need to show that, for any A NP, A p SAT.
Fix A NP.
Construct a poly-time f such that
w A if and only if f(w) SAT.
A formula, write it as w.

By definition, since A NP, there is a nondeterministic

TM M that decides A in polynomial time.
Fix polynomial p such that M on input w always halts, on
all branches, in time p(|w|); assume p(|w|) |w|.
w A if and only if there is an accepting computation
history (CH) of M on w.

Satisfiability is NP-Complete
Lemma 2: SAT is NP-hard.
Proof, contd:
Need w A if and only if f(w) (= w) SAT.
w A if and only if there is an accepting CH of M on w.
So we must construct formula w to be satisfiable iff there
is an accepting CH of M on w.
Recall definitions of computation history and accepting
computation history from Post Correspondence Problem:
# C0 # C1 # C2
Configurations include tape contents, state, head position.

We construct w to describe an accepting CH.

Let M = ( Q, , , , q0, qacc, qrej ) as usual.
Instead of lining up configs in a row as before, arrange in
( p(|w|) + 1 ) row ( p(|w|) + 3 ) column matrix:

Proof that SAT is NP-hard

w will be satisfiable iff there is an accepting CH of M on w.
Let M = ( Q, , , , q0, qacc, qrej ).
Arrange configs in ( p(|w|) + 1 ) ( p(|w|) + 3 ) matrix:
# q0 w1 w2 w3 wn -- -- -- #
#
#
#
#
#
#
Successive configs, ending with accepting config.
Assume WLOG that each computation takes exactly p(|w|)
steps, so we use p(|w|) + 1 rows.
p(|w|) + 3 columns: p(|w|) for the interesting portion of the
tape, one for head and state, two for endmarkers.

Proof that SAT is NP-hard

w is satisfiable iff there is an accepting CH of M on w.
Entries in the matrix are represented by Boolean variables:
Define C = Q { # }, alphabet of possible matrix entries.
Variable xi,j,c represents the entry in position (i, j) is c.

Define w as a formula over these xi,j,c variables, satisfiable

if and only if there is an accepting computation history for w
(in matrix form).
Moreover, an assignment of values to the xi,j,c variables that
satisfies w will correspond to an encoding of an accepting
computation.
Specifically, w = cell start accept move , where:

cell : There is exactly one value in each matrix location.

start : The first row represents the starting configuration.
accept : The last row is an accepting configuration.
move : Successive rows represent allowable moves of M.

cell
For each position (i,j), write the conjunction of two formulas:

c C xi,j,c : Some value appears in position (i,j).

c, d C, c d (xi,j,c

xi,j,d ): Position (i,j) doesnt contain

two values.

cell: Conjoin formulas for all positions (i,j).

Easy to construct the entire formula cell given w input.
Construct it in polynomial time.
Sanity check: Length of formula is polynomial in |w|:
O( (p(|w|)2 ) subformulas, one for each (i,j).
Length of each subformula depends on C, O( |C|2 ).

start
The right symbols appear in the first row:
# q0 w1 w2 w3 wn -- -- -- #
start: x1,1,# x1,2,q0 x1,3,w1 x1,4,w2
x1,n+2,wn x1,n+3,--
x1,p(n)+2,-- x1,p(n)+3,#

accept
For each j, 2 j p(|w|) + 2, write the formula:
xp(|w|)+1,j,qacc
qacc appears in position j of the last row.
accept: Take disjunction (or) of all formulas for all j.
That is, qacc appears in some position of the last
row.

move
As for PCP, correct moves depend on
correct changes to local portions of
configurations.
Its enough to consider 2 3 rectangles:
If every 2 3 rectangle is good, i.e.,
consistent with the transitions, then the
entire matrix represents an accepting CH.
For each position (i,j), 1 i p(|w|), 1 j
p(|w|)+1, write a formula saying that the
rectangle with upper left at (i,j) is good.
Then conjoin all of these, O(p(|w|)2) clauses.
Good tiles for (i,j), for a, b, c in :

move
Other good tiles are defined in terms of the
nondeterministic transition function .
E.g., if (q1, a) includes tuple (q2, b, L), then
the following are good:

Represents the move directly; for any c:

Head moves left out of the rectangle; for any c, d:
Head is just to the left of the rectangle; for any c, d:
Head at right; for any c, d, e:
And more, for #, etc.

Analogously if (q1, a) includes (q2, b, R).

Since M is nondeterministic, (q1, a) may
contain several moves, so include all the
tiles.

q1

q2

q1

q1

q2

q2

move
The good tiles give partial constraints on the computation.
When taken together, they give enough constraints so that
only a correct CH can satisfy them all.
The part (conjunct) of move for (i,j) should say that the
rectangle with upper left at (i,j) is good:
It is simply the disjunction (or), over all allowable tiles, of
the subformula:
a1

a2

a3

b1

b2

b3

xi,j,a1 xi,j+1,a2 xi,j+2,a3 xi+1,j,b1 xi+1,j+1,b2 xi+1,j+2,b3

Thus, move is the conjunction over all (i,j), of the
disjunction over all good tiles, of the formula just above.

move
move is the conjunction over all (i,j), of the
disjunction over all good tiles, of the given sixterm conjunctive formula.
Q: How big is the formula move?
O(p(|w|)2) clauses, one for each (i,j) pair.
Each clause is only constant length, O(1).
Because machine M yields only a constant number of
good tiles.
And there are only 6 terms for each tile.

Thus, length of move is polynomial in |w|.

w = cell start accept move , length also poly in |w|.

move
w = cell start accept move , length poly in |w|.
More importantly, can produce w from w in time that is
polynomial in |w|.
w A if and only if M has an accepting CH for w if and
only if w is satisfiable.
Thus, A p SAT.
Since A was any language in NP, this proves that SAT is
NP-hard.
Since SAT is in NP and is NP-hard, SAT is NP-complete.

Next time
NP-completeness---more examples
Reading:
Sipser Sections 7.4-7.5

MIT OpenCourseWare
http://ocw.mit.edu

6.045J / 18.400J Automata, Computability, and Complexity

Spring 2011

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.