R and RE Languages

Announcements

Problem Set 6 due now.

Problem Set 7 out, due next Friday, November
16th.

Play around with Turing machines, the R and

RE languages, and their limits.
Some problems require Monday's lecture; if you
want to get a jump on that, look at Sipser,
Chapter 4.
Late days don't cross Thanksgiving break.
No checkpoint, even though the syllabus says
there's one.

Important Ideas for Today

The material from today will lay the

groundwork for the next few weeks.
Key concepts:

High-level specifications

R and RE languages.

Encodings.

Universal machines.

Nondeterministic Turing machines.

There is a lot of material today; please do not

hesitate to ask questions!

High-Level Descriptions

The Church-Turing Thesis

The Church-Turing thesis states that all

effective models of computation are
equivalent to or weaker than a Turing
machine.
As as a result, we can start to be less
precise with our TM descriptions.

High-Level Descriptions

A high-level description of a Turing machine is a

description of the form
M = On input x:
Do something with x.

Example:
M = On input x:
Repeat the following:
If |x| 1, accept.
If the first and last symbols of x aren't
the same, reject.
Remove the first and last characters of x.

High-Level Descriptions

A high-level description of a Turing machine is a

description of the form
M = On input x:
Do something with x.

Example:
M = On input x:
Construct y, the reverse of x.
If x = y, accept.
Otherwise, reject.

High-Level Descriptions

A high-level description of a Turing machine is a

description of the form
M = On input x:
Do something with x.

Example:
M = On input x:
If x is a palindrome, accept.
Otherwise, reject.

High-Level Descriptions

A high-level description of a Turing machine is a

description of the form
M = On input x:
Do something with x.

Example:
M = On input x:
Check that x has the form 0n1m2p.
If not, reject.
If nm = p, accept.
Otherwise, reject.

High-Level Descriptions

A high-level description of a Turing machine is a

description of the form
M = On input x:
Do something with x.

Example:
M = On input x:
Check that x has the form p?t,
where p, t {0, 1}*.
If not, reject.
If so, if p is a substring of t, accept.
Otherwise, reject.

Formatted Input

Many languages require the input to be in some

particular format.

(Think about ADD or SEARCH from the problem

sets).

We can encode this directly into our TMs:

M = On input p?t, where p, t {0, 1}*
If p is a substring of t, accept.
Otherwise, reject.

Machines of this form implicitly reject any inputs that

don't have the right format.

Formatted Input

Many languages require the input to be in some

particular format.

(Think about ADD or SEARCH from the problem

sets).

We can encode this directly into our TMs:

M = On input 0m1n2p:
If mn = p, accept.
Otherwise, reject.

Machines of this form implicitly reject any inputs that

don't have the right format.

What's Allowed?

General rule of thumb:

You can include anything in
a high-level description,
as long as you could write
a computer program for it.

A few exceptions: no user input, no

randomness, etc.
This is a consequence of the Church-Turing
thesis.

R and RE Languages

Some Important Terminology

A TM accepts a string w if it enters an accept state.

A TM rejects a string w if it enters a reject state.

A TM loops infinitely (or just loops) on a string w if

neither of these happens.
A TM does not accept a string w if it either rejects w or
loops infinitely on w.
A TM does not reject a string w if it either accepts w or
loops infinitely on w.
A TM halts if it accepts or rejects.
does not reject
does not accept

Accept
Loop
Reject

halts

Recall: Language of a TM

The language of a Turing machine M, denoted

(M), is the set of all strings that M accepts:
(M) = { w * | M accepts w }

For any w (M), M accepts w.

For any w (M), M does not accept w.

It might loop forever, or it might explicitly reject.

A language is called recognizable or recursively

enumerable if it is the language of some TM.
Notation: RE is the set of all recognizable
languages.
L RE iff L is recognizable

Why Recognizable?

Given TM M with language (M), running M on a

string w will not necessarily tell you whether w(M).
If the machine is running, you can't tell whether

It is eventually going to halt, but just needs more time,

and
It is never going to halt.

However, if you know for a fact that w (M), then the

machine can confirm this (it eventually accepts).
The machine can't decide whether or not w (M), but
it can recognize strings that are in the language.
We sometimes call a TM for a language L a recognizer
for L.

Deciders

Some Turing machines always halt; they

never go into an infinite loop.
Turing machines of this sort are called
deciders.
For deciders, accepting is the same as
not rejecting and rejecting is the same as
not accepting.
Accept
does not reject
halts (always)
does not accept

Reject

Decidable Languages

A language L is called decidable iff there is a

decider M such that (M) = L.

Given a decider M, you can learn whether or not a

string w (M).

These languages are also sometimes called

recursive.

Run M on w.
Although it make take a staggeringly long time, M
will eventually accept or reject w.

The set R is the set of all decidable languages.

w R iff w is decidable

Why R Matters

If a language is in R, there is an algorithm that can

decide membership in that language.

If there is an algorithm that can decide membership in

a language, that language is in R.

Run the decider and see what it says.

By the Church-Turing thesis, any effective model of

computation is equivalent in power to a Turing machine.
Thus if there is any algorithm for deciding membership in
the language, there must be a decider for it.
Thus the language is in R.

A language is in R iff there is an algorithm for

deciding membership in that language.

R RE

Every decider is a Turing machine, but not

every Turing machine is a decider.

Thus R RE.

Hugely important theoretical question:

Is R = RE?

That is, if we can verify that a string is in a

language, can we decide whether that string is
in the language?
We will need to build up some more formalisms
to answer this question.

Encodings

An Interesting Observation

WB programs give us a way to check whether

a string is contained within a given RE
language.
WB programs themselves are strings.
This means that we can run WB programs,
taking other WB programs as input!
Major questions:

Is this specific to WB programs?

What can we do with this knowledge?

There is a subtle flaw in this reasoning:

Because WB programs are strings,
WB programs can be run on the
source code of other WB programs.
What is it?

The Alphabet Problem

WB programs are described using multiple

characters: letters, digits, colons, newlines, etc.,
plus potentially all tape symbols being used.
Not all WB programs are written for languages
over this alphabet; in fact, most do not.
We cannot directly write the source of a WB
program onto the input tape of another WB
program.
Can we fix this?

A Better Encoding

We will restrict ourselves to talking about languages

over alphabets containing at least two symbols.
It's always possible to encode a string in any alphabet
using just two symbols.

This is how real computers work.

01000100 01001001 01001011 01000100 01001001 01001011

D
I
K
D
I
K

Notation for Encodings

If P is a WB program, we will denote its

binary encoding as P.
Don't worry too much about the details
about how exactly you would compute
P; the important part is that there's at
least one way to do it.

Encoding Other Automata

Using similar techniques, we can encode

all the other automata we've seen so far
as binary strings.
For example, if M is a Turing machine,
M refers to a binary encoding of it.
More generally, if we have any object O
we want to encode as a binary string, we
can write it out as O.

Encoding Multiple Objects

Suppose that we want to provide an encoding

of multiple objects.

Several Turing machines.

A Turing machine and a string.

A graph and a path in the graph.

A rock and a hard place.

Guns and Roses.

Repeal and Replace.

I just met you and this is crazy.

There are many ways that we can do this.

One Encoding Scheme

X1

X2

0 1 0

1 0

0 0 1 1 0 0 0 1 1 0
X1, X2

Encoding Multiple Objects

Given several different objects O1, O2, ,

On, we can represent the encoding of those
n objects as O1, O2, , On.
Example:

If M is a Turing machine and w is a string,

then M, w is an encoding of that program and
that string.
If G is a context-free grammar and P is a PDA,
then G, P is an encoding of that grammar and
that PDA.

We can now encode TMs as strings.

TMs can accept strings as input.
What can we do with this knowledge?

Universal Machines

Universal Machines and Programs

Theorem: There is a Turing machine UTM called the

universal Turing machine that, when run on M, w,
where M is a Turing machine and w is a string, simulates
M running on w.
As a high-level description:
UTM = On input M, w, where M is a TM and w *
Run M on w.
If M accepts w, accept.
If M rejects w, reject.
If
loops
If MM loops
loops on
on w,
w, then
then UUTM
loops
TM
as
as well.
well. This
This isis usually
usually omitted
omitted
from
from the
the description.
description.

Sketch of the Universal WB Program

It is a bit easier to understand the

universal WB program than the universal
TM.
We will write the universal WB program
in WB6 (finite variables, multiple tracks,
multiple tapes, multiple stacks) for
simplicity, and can then compile it
down into normal WB.

Sketch of the Universal WB Program

To run a WB program, at each point in time we

need to track three pieces of information:

The contents of the tape.

The location of the read head.

The index of the current instruction.

Our program will enter a loop and repeatedly do

the following:

Find the next instruction to execute.

Simulate that execution on the simulated tape.

Sketch of the Universal WB

Program
Simulated tape of the program being executed.

> 0 1 x 0 0 A 0 <

...

Program tape holding the program being executed.

> 0 :

M o v e

l e f t . 1 :

G o

...

Scratch tape for intermediate computation.

...

Variables for intermediate storage.

Instr

Letter

The Language of UTM

From a formal language perspective, what is the

language of a the universal TM UTM?
The universal Turing machine accepts all strings
of the form M, w, where M is a Turing machine
that accepts string w.
This language is called ATM:
ATM = { M, w | M is a TM that accepts w. }

Since (UTM) = ATM, we know ATM is recursively

enumerable.

Why This Matters (Philosophically)

As a historically significant achievement.

The universal Turing machine might be the very

first complicated program ever designed for a
computer.
Motivation for the stored-program model of
computers.

As a justification for the Church-Turing thesis.

All sufficiently powerful models of computation can

simulate one another.

Why This Matters (Practically)

Turing machines can run other Turing machines.

This lets TMs base their behaviors on the behavior of

other TMs.

As a stepping stone to building elaborate models of

computation.

A TM can use the universal TM as a subroutine to

simulate the execution of other TMs.

More on that later today.

As a stepping stone to finding unsolvable problems.

More on that on Monday.

RE is Closed Under Intersection

w Recognizer
w

Recognizer
for L1

for L2
M2

M1
M' = On input w:
Run M1 on w.
If M1 accepts:
Run M2 on w.
If M2 accepts w, accept.
Same
Same
If M2 rejects w, reject.
here.
here.
If M1 rejects w, reject.

If
If MM1 1 loops
loops on
on w,
w, we
we will
will
never
never get
get past
past this
this line.
line.

When
When running
running aa TM
TM as
as aa
subroutine,
subroutine, make
make sure
sure you
you
remember
remember to
to account
account for
for
this!
this!

Theorem: RE is closed under intersection.

Proof: Consider any L1, L2 RE. We will prove that L1 L2 RE
by constructing a TM M such that (M) = L1 L2.
Let M1 and M2 be recognizers for L1 and L2, respectively. Then
construct the machine M as follows:
M = On input w:
Run M1 on w.
If M1 accepts w:
Run M2 on w.
If M2 accepts w, accept.
If M2 rejects w, reject.
If M1 rejects w, reject.
We show that (M) = L1 L2 by proving that M accepts w iff
w L1 L2. To see this, note that M accepts w iff both M1
accepts w and M2 accepts w. M1 accepts w iff w L1 and M2
accepts w iff w L2. Thus M accepts w iff w L1 and w L2,
so M accepts w iff w L1 L2. Thus (M) = L1 L2. Since
(M) = L1 L2, we have that L1 L2 RE, as required.

R is Closed Under Complement

If L R, then L R as well.

Decider
for L
M

M' = On input w:
Run M on w.
If M accepts w, reject.
If M rejects w, accept.

Will
Will this
this work
work if
if MM isis
aa recognizer,
recognizer, rather
rather
than
than aa decider?
decider?

Theorem: R is closed under complementation.

Proof: Consider any L R. We will prove that L R by constructing a
decider M' such that (M') = L.
Let M be a decider for L. Then construct the machine M' as follows:
M' = On input w *:
Run M on w.
If M accepts w, reject.
If M rejects w, accept.
We need to show that M' is a decider and that (M') = L.
To show that M' is a decider, we will prove that it always halts.
Consider what happens if we run M' on any input w. First, M' runs
M on w. Since M is a decider, M either accepts w or rejects w. If
M accepts w, M' rejects w. If M rejects w, M' accepts w. Thus M'
always accepts or rejects, so M' is a decider.
To show that (M') = L, we will prove that M' accepts w iff w L.
Note that M' accepts w iff w * and M rejects w. Since M is a
decider, M rejects w iff M does not accept w. M does not accept w iff
w (M). Thus M' accepts w iff w * and w (M), so M' accepts
w iff w L. Therefore, (M') = L.
Since M' is a decider with (M') = L, we have L R, as required.

An Important Result

Suppose that L RE and L RE.

Let M and M be recognizers for L and L,
respectively.
M' = On input w,
Run M and M on w in parallel.
If M accepts w, accept.
If M accepts w, reject.

0 1 2 x
Simulate M

A B x 1
Simulate M

M'
M' is
is aa
decider!
decider!

Theorem: If L RE and L RE, then L R.

Proof: Let M be a recognizer for L and M be a recognizer for L.
Consider the TM M' defined as follows:
M' = On input w:
Run M and M on w in parallel.
If M accepts w, accept.
If M accepts w, reject.
We prove that (M') = L and that M' is a decider. To see that M'
is a decider, consider any string w *. Either w L or w L,
but not both. If w L, then w L. Thus either w L or w L,
but not both. If w L, then M will eventually accept w and M'
halts. If w L, then M will eventually accept w and M' halts.
Thus M' halts on all inputs.
To see that (M') = L, note that M' accepts w iff M accepts w
and M does not accept w. Since M accepts w iff w L and M
accepts w iff w L, if M accepts w, M does not accept w. Thus
M' accepts w iff M accepts w iff w L. Thus (M') = L.
Since (M') = L and M' is a decider, we have L R, as
required.

Nondeterministic Turing Machines

Nondeterministic Turing Machines

A nondeterministic Turing machine

(abbreviated NTM) is a Turing machine
in which there may be multiple
transitions defined for a particular
state/input combination.
If any possible set of choices causes the
machine to accept, it accepts.

Nondeterminisic TMs

An NTM accepts a string w if it enters an

accept state on some path.
An NTM rejects a string w if it enters a reject
state on every path.
An NTM loops on a string w if neither of these
happen (it doesn't accept on any path and
doesn't reject on every path).
Accept
Loop
Reject

Nondeterministic Algorithms

A natural number greater than 1 is composite if it is not prime.

Let = { 1 } and consider the language

COMPOSITE = { 1n | n * is composite }

We can build a multitape, nondeterministic TM for

COMPOSITE as follows:
M = On input 1n:

Nondeterministically write out q 1s on a second tape (2 q < n)

Nondeterministically write out r 1s on a third tape (2 r < n)

Deterministically check if qr = n.

If so, accept.

Otherwise, reject

Applications of NTMs

Consider the following question:

Given a TM M, does M
accept any strings?

Equivalently:
Given a TM M, is (M)?

As a language question:
LNE = {M | M is a TM and (M) }

Question: Is LNE RE?

Nondeterminism to the Rescue!

Consider the following NTM:

N = On input M, where M is a Turing machine:
Nondeterministically guess a string x *.
Deterministically run M on x.
If M accepts x, accept.
If M rejects x, reject.

Is this a legal nondeterministic Turing machine?

If so, how would we prove (M) = LNE?

Does this say whether LNE RE?

Guessing an Arbitrary String

0, R
1, R
start

, R

Guessing an Arbitrary String

0, R
1, R
start

, R

Guessing an Arbitrary String

0

0, R
1, R
start

, R

Guessing an Arbitrary String

0 1

0, R
1, R
start

, R

Guessing an Arbitrary String

0 1 1

0, R
1, R
start

, R

Guessing an Arbitrary String

0 1 1

0, R
1, R
start

, R

Proofs on NTMs

Given a nondeterministic TM M and a

language L, how do we prove that (M)=L?
Prove the following:
For any w *, w L iff there
is a series of choices M can make
such that M accepts w.

Note the biconditional and the existential.

N = On input M, where M is a Turing machine:

Nondeterministically guess a string x *.
Deterministically run M on x.
If M accepts x, accept.
If M rejects x, reject.
LNE = { M | M is a TM and (M) }
Theorem: (N) = LNE.
Proof: We will prove that N accepts w iff w LNE. To do this,
note that by construction, N accepts w iff w = M for
some TM M and there is some choice of x such M accepts x.
Also note that M accepts x iff x (M), so N accepts w iff
w = M for some TM M and there is some choice of x such
that x (M). Finally, note that there is some choice of x
such that x (M) iff (M) . This means that N accepts
w iff w = M for some TM M and (M) . Thus by
definition of LNE, we have that N accepts w iff w LNE.

The Power of Nondeterminism

In the case of finite automata, we saw that the

DFA and NFA had equivalent power.
In the case of pushdown automata, we saw that
the DPDA was strictly weaker than the NPDA.
What is the relative power of TMs and NTMs?
Nondeterministic TuringRecognizable Languages

Turing-Recognizable
Languages

A Rather Remarkable Theorem

Theorem: A language is recursively

enumerable iff it is accepted by a
nondeterministic Turing machine.
How is this possible?

Instantaneous Descriptions

An instantaneous description (or ID) of the execution

of a program (TM, WB program, etc.) is a string
encoding of a snapshot of that program at one instant in
time.
For Turing machines, it contains

The contents of the tape,

Where the tape head is, and

What state the machine is in.

For WB programs, it contains

The contents of the tape,

Where the tape head is, and

What line of code is next to be executed.

IDs and Universal Machines

There is a close connection between an

ID and universal machines.
The universal machine UTM works by
repeatedly reading the ID of the machine
being simulated, then executing one step
of that machine.

An ID for WB Programs
Simulated tape of the program being executed.

> 0 1 1 0 0 0 0 <

...

Program tape holding the program being executed.

> 0 :

M o v e

> 0 1 1

l e f t . 1 :

G o

...

0 0 0 0 < 1

This
This means
means the
the tape
tape head
head
isis under
under the
the next
next symbol.
symbol.

We
We write
write the
the line
line
number
number at
at the
the end
end
of
of the
the ID.
ID.

Manipulating IDs

Because IDs are strings (just like

machine or program encodings), we can
perform all sorts of operations on them:

Copy them to other tapes for later use.

Inspect them to see the state of the machine
at any instant in time.
Transform them to represent making
changes to the program as it is running.

The Key Idea

Store IDs of all of

branches of
computation and
execute them in a
breadth-first search.
Uses the tree
computation
interpretation of
nondeterminism.
If there is an accepting
computation, we will
eventually find it.

Simulating an NTM

Given an NTM M, we can simulate it with a

deterministic TM as follows:
On input w:

Put the initial ID of M running on w into a

separate tape.
While that tape contains IDs:

Move the first ID to a work tape.

If this ID is in an accepting state, accept.

If the ID is not in a rejecting state:

For all possible next steps, use the universal TM to simulate

the NTM making that choice, then append the resulting ID to
the other tape.

Reject.

Simulated tape of the program being executed.

> 0 1 0 0 0 1 0 <

...

Program tape holding the program being executed.

> 0 :

M o v e

l e f t . 1 :

G o

...

Stored IDs

> 0 1 < 1 3 # > 1 1 < 2 4 #

...

Scratch tape for intermediate computation.

...

Variables for intermediate storage.

Instr

Letter

Why This Matters

NTMs make it easier to solve a variety of

problems.
Theorem: LNE RE.

This is incredibly difficult to prove

without nondeterminism.
You will see more applications of
nondeterminism in the problem set.

Next Time

Unsolvable Problems

What languages are not in R or RE?

What problems are provably impossible to
solve?
Does R = RE?