Turing Machines

Fall 2005 Costas Busch - RPI 1

 Turing Machines

(At last!)
Here Arrived
The Big Daddy of all the
machines
 The Language Hierarchy

n n n
a b c ? ww ?

Context-Free Languages
n n R
a b ww
Regular Languages
a* a *b *
Fall 2005 Costas Busch - RPI 3
 Languages accepted by
Turing Machines
n n n
a b c ww

Context-Free Languages
n n R
a b ww
Regular Languages
a* a *b *
Fall 2005 Costas Busch - RPI 4
 Turing Machines (TM)

Generalize the class of CFLs:

Non-Recursively Enumerable Languages

Recursively Enumerable Languages

Recursive Languages

Context-Free Languages

Regular Languages

5
Another Part of the Hierarchy:

Non-Recursively Enumerable Languages

Recursively Enumerable Languages

Recursive Languages

Context-Sensitive Languages

Context-Free Languages - ε

Regular Languages - ε

6
Recursively enumerable languages are also
known as type 0 languages.

Context-sensitive languages are also known as

type 1 languages.

Context-free languages are also known as

type 2 languages.

Regular languages are also known as type 3

languages.

7
 A Turing Machine
Tape
...... ......

Read-Write head

Fall 2005 Costas Busch - RPI 8

 The Tape

No boundaries -- infinite length

...... ......

Read-Write head

The head moves Left or Right

Fall 2005 Costas Busch - RPI 9

 ...... ......

Read-Write head

The head at each time step:

1. Reads a symbol
2. Writes a symbol
3. Moves Left or Right
Fall 2005 Costas Busch - RPI 10
 The Turing Machine

Each move by a Turing Machine results in

• Change of state;
• Writing a tape symbol in the cell just
scanned; and
• Moving the tape head left or right.
Example:
Time 0
...... a b a c ......

Time 1
...... a b k c ......

1. Reads a
2. Writes k
3. Moves Left
Fall 2005 Costas Busch - RPI 12
 Time 1
...... a b k c ......

Time 2
...... a f k c ......

1. Reads b
2. Writes f
3. Moves Right
Fall 2005 Costas Busch - RPI 13
 The Input String

Input string Blank symbol

......   a b a c    ......

head

Head starts at the leftmost position

of the input string
Fall 2005 Costas Busch - RPI 14
 Input string Blank symbol

......   a b a c    ......

head

Fall 2005 Costas Busch - RPI 15

 The Turing Machine

Infinite tape
1 1 1 0 1 0 b b -----
Read/write head ---
Internal
states/program

16
IPQI-2010-Anu Venugopalan
 The deterministic Turing Machine
- the four main elements of the DTM are

1. Finite State Control

2. Tape

3. A read/write tape head

4. Program

IPQI-2010-Anu Venugopalan 17
 DTM- Finite State Control (FSC)
- Finite State Control

The FSC for a TM can be visualized as a stripped down

microprocessor which coordinates the other operations
of the machines

A finite set of m internal states: q1 , q2, ……….. qm

qs : starting state

qh: halting state

IPQI-2010-Anu Venugopalan 18
 The Tape
- the tape is a one dimensional strip – the tape squares
are labelled and each contains one symbol drawn from
some alphabet

e.g., 0,1 and b (blank)

marks the left hand edge of the tape

IPQI-2010-Anu Venugopalan 19
 The read/write head
- the read/write tape head identifies a single
square on the DTM tape as the square that is
being currently accessed by the machine

IPQI-2010-Anu Venugopalan 20
 Differences between finite automata and
Turing machine
1. A Turing machine can both write on the tape and read
from it.
2. The read-write head can move both to the left and to
the right.
3. The tape is infinite.
4. The special states for rejecting and accepting take
immediate effect.
5. The Turing machine may not read all of its input.
 Three Possible Outcomes
TM may halt and accept the input
TM may halt and reject the string
TM may not halt at all
 States & Transitions

Read Write
Move Left

q1 a  b, L q2

Move Right

q1 a  b, R q2
Fall 2005 Costas Busch - RPI 23
Example:
Time 1
......   a b a c    ......

q1
current state

q1 a  b, R q2
Fall 2005 Costas Busch - RPI 24
 Time 1
......   a b a c    ......

q1

Time 2
......   a b b c    ......

q2

q1 a  b, R q2
Fall 2005 Costas Busch - RPI 25
Example:
Time 1
......   a b a c    ......

q1

Time 2
......   a b b c    ......

q2

q1 a  b, L q2
Fall 2005 Costas Busch - RPI 26
Example:
Time 1
......   a b a c    ......

q1

Time 2
......   a b b c g   ......

q2

q1   g, R q2
Fall 2005 Costas Busch - RPI 27
 Deterministic Turing Machines
Deterministic Turing Machines

Allowed Not Allowed

a  b, R q2 a  b, R q2

q1 q1
q3 a  d, L q3
b  d, L

No lambda transitions allowed

Fall 2005 Costas Busch - RPI 28
 Partial Transition Function
Example:

......   a b a c    ......

q1

a  b, R q2 Allowed:

q1 No transition
for input symbol c
b  d, L q3
Fall 2005 Costas Busch - RPI 29
 Halting

The machine halts if there is

no possible transitions to follow

Fall 2005 Costas Busch - RPI 30

Halting Example 1:

......   a b a c    ......

q1

q1 No transition from q1
HALT!!!

Fall 2005 Costas Busch - RPI 31

Halting Example 2:

......   a b a c    ......

q1

a  b, R q2
No possible transition
q1 from q1 and symbol c
b  d, L q3 HALT!!!
Fall 2005 Costas Busch - RPI 32
 Accepting States

q1 q2 Allowed

q1 q2 Not Allowed

•Accepting states have no outgoing transitions

•The machine halts and accepts

Fall 2005 Costas Busch - RPI 33

 Acceptance

If machine halts
Accept Input
in an accept state

If machine halts
in a non-accept state
Reject Input or
If machine enters
an infinite loop
Fall 2005 Costas Busch - RPI 34
 Turing Machine Example

Accepts the language: a *

a  a, R

  , L
q0 q1

Fall 2005 Costas Busch - RPI 35

 Time 0   a a a  

q0

a  a, R

  , L
q0 q1

Fall 2005 Costas Busch - RPI 36

 Time 1   a a a  

q0

a  a, R

  , L
q0 q1

Fall 2005 Costas Busch - RPI 37

 Time 2   a a a  

q0

a  a, R

  , L
q0 q1

Fall 2005 Costas Busch - RPI 38

 Time 3   a a a  

q0

a  a, R

  , L
q0 q1

Fall 2005 Costas Busch - RPI 39

 Time 4   a a a  

q1

a  a, R Halt & Accept

  , L
q0 q1

Fall 2005 Costas Busch - RPI 40

 Rejection Example

Time 0   a b a  

q0

a  a, R

  , L
q0 q1
Fall 2005 Costas Busch - RPI 41
 Time 1   a b a  

q0
No possible Transition
a  a, R Halt & Reject

  , L
q0 q1
Fall 2005 Costas Busch - RPI 42
 Infinite Loop Example
A Turing machine
for language

b  b, L
a  a, R

  , L
q0 q1

Fall 2005 Costas Busch - RPI 43

 Time 0   a b a  

q0

b  b, L
a  a, R

  , L
q0 q1

Fall 2005 Costas Busch - RPI 44

 Time 1   a b a  

q0

b  b, L
a  a, R

  , L
q0 q1

Fall 2005 Costas Busch - RPI 45

 Time 2   a b a  

q0

b  b, L
a  a, R

  , L
q0 q1

Fall 2005 Costas Busch - RPI 46

 Time 2   a b a  
q0
Time 3   a b a  

Infinite loop
q0
Time 4   a b a  
q0

Time 5   a b a  
q0
Fall 2005 Costas Busch - RPI 47
 Because of the infinite loop:

•The final state cannot be reached

•The machine never halts

•The input is not accepted

Fall 2005 Costas Busch - RPI 48

 Another Turing Machine Example
n n
Turing machine for the language {a b }
n 1

q4 y  y, R y  y, L
y  y, R a  a, R a  a, L
  , L
y  y, R a  x, R b  y, L
q3 q0 q1 q2
x  x, R
Fall 2005 Costas Busch - RPI 49
 Basic Idea:
Match a’s with b’s:
Repeat:
replace leftmost a with x
find leftmost b and replace it with y
Until there are no more a’s or b’s

If there is a remaining a or b reject

Fall 2005 Costas Busch - RPI 50

 Time 0  a a b b  

q0

q4 y  y, R y  y, L
y  y, R a  a, R a  a, L
  , L
y  y, R a  x, R b  y, L
q3 q0 q1 q2
x  x, R
Fall 2005 Costas Busch - RPI 51
 Time 1  x a b b  

q1

q4 y  y, R y  y, L
y  y, R a  a, R a  a, L
  , L
y  y, R a  x, R b  y, L
q3 q0 q1 q2
x  x, R
Fall 2005 Costas Busch - RPI 52
 Time 2  x a b b  

q1

q4 y  y, R y  y, L
y  y, R a  a, R a  a, L
  , L
y  y, R a  x, R b  y, L
q3 q0 q1 q2
x  x, R
Fall 2005 Costas Busch - RPI 53
 Time 3  x a y b  

q2

q4 y  y, R y  y, L
y  y, R a  a, R a  a, L
  , L
y  y, R a  x, R b  y, L
q3 q0 q1 q2
x  x, R
Fall 2005 Costas Busch - RPI 54
 Time 4  x a y b  

q2

q4 y  y, R y  y, L
y  y, R a  a, R a  a, L
  , L
y  y, R a  x, R b  y, L
q3 q0 q1 q2
x  x, R
Fall 2005 Costas Busch - RPI 55
 Time 5  x a y b  

q0

q4 y  y, R y  y, L
y  y, R a  a, R a  a, L
  , L
y  y, R a  x, R b  y, L
q3 q0 q1 q2
x  x, R
Fall 2005 Costas Busch - RPI 56
 Time 6  x x y b  

q1

q4 y  y, R y  y, L
y  y, R a  a, R a  a, L
  , L
y  y, R a  x, R b  y, L
q3 q0 q1 q2
x  x, R
Fall 2005 Costas Busch - RPI 57
 Time 7  x x y b  

q1

q4 y  y, R y  y, L
y  y, R a  a, R a  a, L
  , L
y  y, R a  x, R b  y, L
q3 q0 q1 q2
x  x, R
Fall 2005 Costas Busch - RPI 58
 Time 8  x x y y  

q2

q4 y  y, R y  y, L
y  y, R a  a, R a  a, L
  , L
y  y, R a  x, R b  y, L
q3 q0 q1 q2
x  x, R
Fall 2005 Costas Busch - RPI 59
 Time 9  x x y y  

q2

q4 y  y, R y  y, L
y  y, R a  a, R a  a, L
  , L
y  y, R a  x, R b  y, L
q3 q0 q1 q2
x  x, R
Fall 2005 Costas Busch - RPI 60
 Time 10  x x y y  

q0

q4 y  y, R y  y, L
y  y, R a  a, R a  a, L
  , L
y  y, R a  x, R b  y, L
q3 q0 q1 q2
x  x, R
Fall 2005 Costas Busch - RPI 61
 Time 11  x x y y  

q3

q4 y  y, R y  y, L
y  y, R a  a, R a  a, L
  , L
y  y, R a  x, R b  y, L
q3 q0 q1 q2
x  x, R
Fall 2005 Costas Busch - RPI 62
 Time 12  x x y y  

q3

q4 y  y, R y  y, L
y  y, R a  a, R a  a, L
  , L
y  y, R a  x, R b  y, L
q3 q0 q1 q2
x  x, R
Fall 2005 Costas Busch - RPI 63
 Time 13  x x y y  

q4
Halt & Accept

q4 y  y, R y  y, L
y  y, R a  a, R a  a, L
  , L
y  y, R a  x, R b  y, L
q3 q0 q1 q2
x  x, R
Fall 2005 Costas Busch - RPI 64
 Observation:

If we modify the
n n
machine for the language {a b }

we can easily construct

n n n
a machine for the language {a b c }

Fall 2005 Costas Busch - RPI 65

 Formal Definitions
for
Turing Machines

Fall 2005 Costas Busch - RPI 66

 Transition Function

q1 a  b, R q2

 (q1, a )  (q2 , b, R )

Fall 2005 Costas Busch - RPI 67

 Transition Function

q1 c  d, L q2

 (q1, c)  (q2 , d , L)

Fall 2005 Costas Busch - RPI 68

Turing Machine:

Input Tape
alphabet alphabet
States

M  (Q, , ,  , q0 , , F )

Transition Accept
function states
Initial blank
Fall 2005
stateCostas Busch - RPI 69
 Def:
A Turing Machine is a 7-tuple (Q, Σ, Γ, S, q0, qaccept, qreject),
where Q, Σ, Γ are all finite sets and
1. Q: set of states,
2. Σ: the input alphabet NOT containing the special blank
symbol B,
3. Γ: the tape alphabet, where {B} Γ and Σ Γ,
4. δ:Q×Γ→Q×Γ×{L,R} ~ transition func.
5. q0Q is the start state,
6. qacceptQ is the accept state, and
7. qrejectQ is the reject state, where qreject ≠ qaccept
 The Turing Machine
Hopcroft and Ullman define a one-tape Turing machine
formula: M=(Q, Γ, b, ∑, δ, q0, F) where:

Q is a finite set of states

Γ is a finite set of the tape alphabet/symbols
b is the blank symbol (the only symbol allowed to
occur on the tape infinitely often at any step during
the computation) - Delimiter
Σ, a subset of Γ not including b is the set of input
symbols
δ = Q x Γ  Q x Γ x {L,R} is a partial function called
the transition function, where L is left shift, R is
right shift.
q0 is the initial state
F is the set of final or accepting states
As a Turing machine computes, it may halt with ‘accept’ or
‘reject’, or it may never halt!
During computation, changes occur in
1. the current state,
2. the current tape contents, and
3. the current head location.
The above three items form a “configuration” of the Turing
machine.
As a Turing machine computes, it may halt with ‘accept’ or
‘reject’, or it may never halt!
During computation, changes occur in
1. the current state,
2. the current tape contents, and
3. the current head location.
The above three items form a “configuration” of the Turing
machine.
 halting configurations
A Turing machine M accepts input w if a seq. of
configurations C1, C2, …, Ck exists where
1. C1 is the start conf. of M on input w,
2. each Ci yields Ci+1, and
3. Ck is an accepting configuration.
The collection of strings that M accepts is the
language of M, denoted L(M).
Def: A language is Turing-recognizable if some
Turing machine recognizes it.
(or recursively enumerable)
r.e.
 Configuration

  c a b a  

q1

Instantaneous description: ca q1 ba

Fall 2005 Costas Busch - RPI 76

 Time 4 Time 5
 x a y b    x a y b  

q2 q0

A Move: q2 xayb  x q0 ayb

(yields in one mode)

Fall 2005 Costas Busch - RPI 77

 Time 4 Time 5
 x a y b    x a y b  

q2 q0

Time 6 Time 7
 x x y b    x x y b  

q1 q1

q2 xayb  x q0 ayb  xx q1 yb  xxy q1 b

Fall 2005 Costas Busch - RPI 78
q2 xayb  x q0 ayb  xx q1 yb  xxy q1 b


Equivalent notation: q2 xayb  xxy q1 b

Fall 2005 Costas Busch - RPI 79

 Initial configuration: q0 w

Input string
w

 a a b b  

q0

Fall 2005 Costas Busch - RPI 80

 PRESENT TAPE SYMBOL TAPE SYMBOL TAPE SYMBOL
STATE # a b

(q1,a,L) ----- (q0,b,R)

q0

(starting State)
q1 (q2, #,R) (q1,a,L) (q1,b,L)
q2 ---- (q4,#,R) (q3,#,R)

q3 (q4,b,R) (q3,a,R) (q3,a,R)

q4
(q1,b,L) ---- ----

(Final State)

Fall 2005 Costas Busch - RPI 81

 Moves of Instantaneous Description
For the input string bb we have the following
moves

q0bb# ├ bq0b# ├bbq0# ├bq1ba

├q1bba ├q1#bba ├ #q2bba ├##q3ba
├##bq3a ├##baq3# ├##babq4 ├##baq1bb
├##bq1abb ├##q1babb ├#q1#babb ├##q2babb
├###q3abb ├###aq3bb ├###abq3b
├###abbq3# ├###abbbq4# ├###abbq1bb
├###abq1bbb ├###aq1bbbb ├###q1abbbb
├#q1#abbbb ├###q2abbbb ├####q4bbbb
Fall 2005 Costas Busch - RPI 82
 The Accepted Language

For any Turing Machine M


L( M )  {w : q0 w  x1 q f x2 }

Initial state Final state

Fall 2005 Costas Busch - RPI 83

 Standard Turing Machine

The machine we described is the standard:

• Deterministic

• Infinite tape in both directions

•Tape is the input/output file

Fall 2005 Costas Busch - RPI 84

 Computing Functions
with
Turing Machines

Fall 2005 Costas Busch - RPI 85

 A function f (w) has:

Domain: D Result Region: S

f (w)
w D f ( w)  S

Fall 2005 Costas Busch - RPI 86

 A function may have many parameters:

Example: Addition function

f ( x, y )  x  y

Fall 2005 Costas Busch - RPI 87

 Integer Domain

Decimal: 5

Binary: 101

Unary: 11111

We prefer unary representation:

easier to manipulate with Turing machines

Fall 2005 Costas Busch - RPI 88
Definition:

A function f is computable if
there is a Turing Machine M such that:

Initial configuration Final configuration

 w   f (w) 

q0 initial state q f final state

For all w D Domain

Fall 2005 Costas Busch - RPI 89
In other words:
A function f is computable if
there is a Turing Machine M such that:


q0 w  q f f ( w)

Initial Final
Configuration Configuration

For all w D Domain

Fall 2005 Costas Busch - RPI 90
 Example

The function f ( x, y )  x  y is computable

x, y are integers

Turing Machine:

Input string: x0 y unary

Output string: xy 0 unary

Fall 2005 Costas Busch - RPI 91

 x y

Start  1 1  1 0 1  1 

q0
initial state

The 0 is the delimiter that

separates the two numbers

Fall 2005 Costas Busch - RPI 92

 x y

Start  1 1  1 0 1  1 

q0 initial state

x y

Finish  1 1  1 1 0 

q f final state
Fall 2005 Costas Busch - RPI 93
 The 0 helps when we use
the result for other operations

x y

Finish  1 1  1 1 0 

q f final state
Fall 2005 Costas Busch - RPI 94
Turing machine for function f ( x, y )  x  y

1  1, R 1  1, R 1  1, L

q0 0  1, R q1   , L q 1  0, L q3
2

  , R
q4
Fall 2005 Costas Busch - RPI 95
 Execution Example: Time 0
x y
x  11 (2)
 1 1 0 1 1 
y  11 (2) q0

Final Result
x y
 1 1 1 1 0 

q4
Fall 2005 Costas Busch - RPI 96
 Time 0  1 1 0 1 1 

q0

1  1, R 1  1, R 1  1, L

q0 0  1, R q1   , L q 1  0, L q3
2

  , R
q4
Fall 2005 Costas Busch - RPI 97
 Time 1  1 1 0 1 1 

q0

1  1, R 1  1, R 1  1, L

q0 0  1, R q1   , L q 1  0, L q3
2

  , R
q4
Fall 2005 Costas Busch - RPI 98
 Time 2  1 1 0 1 1 

q0

1  1, R 1  1, R 1  1, L

q0 0  1, R q1   , L q 1  0, L q3
2

  , R
q4
Fall 2005 Costas Busch - RPI 99
 Time 3  1 1 1 1 1 

q1

1  1, R 1  1, R 1  1, L

q0 0  1, R q1   , L q 1  0, L q3
2

  , R
q4
Fall 2005 Costas Busch - RPI 100
 Time 4  1 1 1 1 1 

q1

1  1, R 1  1, R 1  1, L

q0 0  1, R q1   , L q 1  0, L q3
2

  , R
q4
Fall 2005 Costas Busch - RPI 101
 Time 5  1 1 1 1 1 

q1

1  1, R 1  1, R 1  1, L

q0 0  1, R q1   , L q 1  0, L q3
2

  , R
q4
Fall 2005 Costas Busch - RPI 102
 Time 6  1 1 1 1 1 

q2

1  1, R 1  1, R 1  1, L

q0 0  1, R q1   , L q 1  0, L q3
2

  , R
q4
Fall 2005 Costas Busch - RPI 103
 Time 7  1 1 1 1 0 

q3

1  1, R 1  1, R 1  1, L

q0 0  1, R q1   , L q 1  0, L q3
2

  , R
q4
Fall 2005 Costas Busch - RPI 104
 Time 8  1 1 1 1 0 

q3

1  1, R 1  1, R 1  1, L

q0 0  1, R q1   , L q 1  0, L q3
2

  , R
q4
Fall 2005 Costas Busch - RPI 105
 Time 9  1 1 1 1 0 

q3

1  1, R 1  1, R 1  1, L

q0 0  1, R q1   , L q 1  0, L q3
2

  , R
q4
Fall 2005 Costas Busch - RPI 106
 Time 10  1 1 1 1 0 

q3

1  1, R 1  1, R 1  1, L

q0 0  1, R q1   , L q 1  0, L q3
2

  , R
q4
Fall 2005 Costas Busch - RPI 107
 Time 11  1 1 1 1 0 

q3

1  1, R 1  1, R 1  1, L

q0 0  1, R q1   , L q 1  0, L q3
2

  , R
q4
Fall 2005 Costas Busch - RPI 108
 Time 12  1 1 1 1 0 

q4

1  1, R 1  1, R 1  1, L

q0 0  1, R q1   , L q 1  0, L q3
2

  , R
HALT & accept q4
Fall 2005 Costas Busch - RPI 109
 Another Example

The function f ( x)  2 x is computable

x is integer

Turing Machine:

Input string: x unary

Output string: xx unary

Fall 2005 Costas Busch - RPI 110
 x

Start  1 1  1 

q0 initial state

2x

Finish  1 1  1 1 1 

q f final state
Fall 2005 Costas Busch - RPI 111
Turing Machine Pseudocode for f ( x)  2 x

• Replace every 1 with $

• Repeat:
• Find rightmost $, replace it with 1

• Go to right end, insert 1

Until no more $ remain

Fall 2005 Costas Busch - RPI 112

Turing Machine for f ( x)  2 x

1  $, R 1  1, L 1  1, R

q0   , L q1 $  1, R q2
  , R
  1, L
q3
Fall 2005 Costas Busch - RPI 113
 Example
Start Finish
 1 1   1 1 1 1 
q0 q3
1  $, R 1  1, L 1  1, R

q0   , L q1 $  1, R q2
  , R
  1, L
q3
Fall 2005 Costas Busch - RPI 114
 Another Example

1 if x y
The function f ( x, y ) 
0 if x y
is computable

Fall 2005 Costas Busch - RPI 115

 Turing Machine for

1 if x y
f ( x, y ) 
0 if x y

Input: x0 y

Output: 1 or 0
Fall 2005 Costas Busch - RPI 116
Turing Machine Pseudocode:

• Repeat

Match a 1 from x with a 1 from y

Until all of x or y is matched

• If a 1 from x is not matched

erase tape, write 1 ( x  y)
else
Fall 2005
erase tape, write 0
Costas Busch - RPI
( x  y) 117
 Combining Turing Machines

Fall 2005 Costas Busch - RPI 118

 Block Diagram

Turing
input output
Machine

Fall 2005 Costas Busch - RPI 119

 Example: x  y if x  y
f ( x, y ) 
0 if x  y

x, y
Adder x y
x, y x y
Comparator

x y Eraser 0

Fall 2005 Costas Busch - RPI 120