18CSC301T – FORMAL

LANGUAGE AND AUTOMATA

UNIT-IV
Turing Machines
 Introduction
• TM’s described in 1936
– Well before the days of modern computers but remains a
popular model for what is possible to compute on today’s
systems
– Advances in computing still fall under the TM model, so
even if they may run faster, they are still subject to the
same limitations
• A TM consists of a finite control (i.e. a finite state
automaton) that is connected to an infinite tape.

unit-4/Turning Machine 3
 Turing Machine
• The tape consists of cells where each cell holds a symbol from the tape
alphabet. Initially the input consists of a finite-length string of symbols
and is placed on the tape. To the left of the input and to the right of the
input, extending to infinity, are placed blanks. The tape head is initially
positioned at the leftmost cell holding the input.

Finite control

… B B X 1 X2 … Xi Xn B B …

unit-4/Turning Machine 4
 Turing Machine Details
• In one move the TM will:
– Change state, which may be the same as the current state
– Write a tape symbol in the current cell, which may be the
same as the current symbol
– Move the tape head left or right one cell
• Formally, the Turing Machine is denoted by the
7-tuple:
– M = (Q, ∑, Γ, δ, q0, B, F)

unit-4/Turning Machine 5
 Turing Machine Description
• Q = finite states of the control
• ∑ = finite set of input symbols, which is a subset of Γ below
• Γ = finite set of tape symbols
• δ = transition function. δ(q,X) are a state and tape symbol X.
– The output is the triple, (p, Y, D)
– Where p = next state, Y = new symbol written on the tape, D =
direction to move the tape head
• q0= start state for finite control
• B = blank symbol. This symbol is in Γ but not in ∑.
• F= set of final or accepting states of Q.

unit-4/Turning Machine 6
 Instantaneous Description
• Sometimes it is useful to describe what a TM does in terms of
its ID (instantaneous description), just as we did with the
PDA.

• The ID shows all non-blank cells in the tape, pointer to the

cell the head is over with the name of the current state
– use the turnstile symbol ├ to denote the move.
– As before, to denote zero or many moves, we can use ├*.

unit-4/Turning Machine 7
 A Turing Machine
Tape
...... ......

Read-Write head
Control Unit

unit-4/Turning Machine 8
 The Tape
No boundaries -- infinite length
...... ......

Read-Write head

The head moves Left or Right

unit-4/Turning Machine 9
 ...... ......

Read-Write head

The head at each time step:

1. Reads a symbol
2. Writes a symbol
3. Moves Left or Right
unit-4/Turning Machine 10
Example:
Time 0
...... a b a c ......

Time 1
...... a b k c ......

a
1. Reads

2. Writes
k unit-4/Turning Machine 11
 Time 1
...... a b k c ......

Time 2
...... a f k c ......

1. Reads b
2. Writes f
3. Moves Right unit-4/Turning Machine 12
 The Input String
Input string Blank symbol

...... a b a c ......

head
Head starts at the leftmost position
of the input string
Are treated as left and right brackets for the
input written on the tape.
unit-4/Turning Machine 13
States & Transitions
Read Write
Move Left

q a b, L
q2
1
Move Right

q a b, R
q2
unit-4/Turning Machine 14
Example:
Time 1
...... a b a c ......

q1
current state

q a b, R
q2
1 unit-4/Turning Machine 15
 Time 1
...... a b a c ......

q1

Time 2
...... a b b c ......

q2

q a b, R
q2
unit-4/Turning Machine 16
Example:
Time 1
...... a b a c ......

q1

Time 2
...... a b b c ......

q2

q a b, L
q2
unit-4/Turning Machine 17
Example:
Time 1
...... a b a c ......

q1

Time 2
...... a b b c g ......

q2

q g,
q2
R unit-4/Turning Machine 18
 Determinism
Turing Machines are deterministic

Allowed Not
Allowed
a b, R q2 q2
a b, R
q q

1 b d, L q3 1 a d, L q3

unit-4/Turning Machine 19
 Partial Transition
Function
Example:
...... a b a c ......

q
1
a b, R Allowed:
q2
q No transition
for input symbol
c
1 b d, L q3
unit-4/Turning Machine 20
The machine halts if there are
Halting
no possible transitions to follow
Example:

...... a b a c ......

q
1
a b, R q2
No possible transition
q
HALT!!!
1 b d, L q3
unit-4/Turning Machine 22
 Final States
q q2 Allowed
1

q q2 Not
Allowed
1

• Final states have no outgoing transitions

• In a final state the machine halts

unit-4/Turning Machine 23
 Turing Machines as Acceptors
• A Turing machine halts when it no longer has
any available moves. If it halts in a final state,
it accepts its input; otherwise, it rejects its
input.

unit-4/Turning Machine 24
 Turing Machines as Acceptors
• We said a Turing machine accepts its input if it
halts in a final state. There are two ways this
could fail to happen:
• The Turing machine could halt in a nonfinal
state, or The Turing machine could never stop
(in which case we say it is in an infinite loop. )
• If a Turing machine halts, the sequence of
configurations leading to the halt state is
called a computation.

unit-4/Turning Machine 25
 Acceptance
If machine halts
Accept Input
in a final state

If machinehalts
in a non-final state
Reject Input or
If machine enters
an infinite loop
unit-4/Turning Machine 26
 Turing Machine Example
A Turing machine that accepts the language:
aa *

a a, R

, q
q0
L
1
unit-4/Turning Machine 27
Time 0 a a a

q0

a a, R

, q
q0
L
1
unit-4/Turning Machine 28
Time 1 a a a

q0

a a, R

, q
q0
L
1
unit-4/Turning Machine 29
Time 2 a a a

q0

a a, R

, q
q0
L
1
unit-4/Turning Machine 30
Time 3 a a a

q0

a a, R

, q
q0
L
1
unit-4/Turning Machine 31
Time 4 a a a

q
1

a a, R Halt & Accept

, q
q0
L
1
unit-4/Turning Machine 32
Rejection Example

Time 0 a b a

q0

a a, R

, q
q0
L
unit-4/Turning Machine 33
Time 1 a b a

q0

No possible Transition
a a, R Halt & Reject

, q
q0
L
unit-4/Turning Machine 34
Infinite Loop Example

b b, L
a a, R

, q
q0
L
1
unit-4/Turning Machine 35
Time 0 a b a

q0

b b, L
a a, R

, q
q0
L
1
unit-4/Turning Machine 36
Time 1 a b a

q0

b b, L
a a, R

, q
q0
L
1
unit-4/Turning Machine 37
Time 2 a b a

q0

b b, L
a a, R

, q
q0
L
1
unit-4/Turning Machine 38
Time 2 a b a

q0
Time 3 a b a

q0
Time 4 a b a

q0
Time 5 a b a

... Infinite Loop q0

unit-4/Turning Machine 39
Because of the infinite loop:

•The final state cannot be reached

•The machine never halts

•The input is not accepted

unit-4/Turning Machine 40
 Another Turing Machine
Example n n
Turing machine for the language {a b
}

q4 y y, R y y, L
y y, R a a, R a a, L
,
L
y y, R 0 a x, b y,
q3 q q q2
R L
x 1 x,
unit-4/Turning Machine 41
• aabb
• Xabb
• XaYb
• XXYb
• XXYY

unit-4/Turning Machine 42
 Time 0 a a b b

q0

q4 y y, R y y, L
y y, R a a, R a a, L
,
L
y y, R 0 a x, b y,
q3 q q q2
R L
x 1 x,
unit-4/Turning Machine 43
 Time 1 x a b b

q
1

q4 y y, R y y, L
y y, R a a, R a a, L
,
L
y y, R 0 a x, b y,
q3 q q q2
R L
x 1 x,
unit-4/Turning Machine 44
 Time 2 x a b b

q
1

q4 y y, R y y, L
y y, R a a, R a a, L
,
L
y y, R 0 a x, b y,
q3 q q q2
R L
x 1 x,
unit-4/Turning Machine 45
 Time 3 x a y b

q2

q4 y y, R y y, L
y y, R a a, R a a, L
,
L
y y, R 0 a x, b y,
q3 q q q2
R L
x 1 x,
unit-4/Turning Machine 46
 Time 4 x a y b

q2

q4 y y, R y y, L
y y, R a a, R a a, L
,
L
y y, R 0 a x, b y,
q3 q q q2
R L
x 1 x,
unit-4/Turning Machine 47
 Time 5 x a y b

q0

q4 y y, R y y, L
y y, R a a, R a a, L
,
L
y y, R 0 a x, b y,
q3 q q q2
R L
x 1 x,
unit-4/Turning Machine 48
 Time 6 x x y b

q
1

q4 y y, R y y, L
y y, R a a, R a a, L
,
L
y y, R 0 a x, b y,
q3 q q q2
R L
x 1 x,
unit-4/Turning Machine 49
 Time 7 x x y b

q
1

q4 y y, R y y, L
y y, R a a, R a a, L
,
L
y y, R 0 a x, b y,
q3 q q q2
R L
x 1 x,
unit-4/Turning Machine 50
 Time 8 x x y y

q2

q4 y y, R y y, L
y y, R a a, R a a, L
,
L
y y, R 0 a x, b y,
q3 q q q2
R L
x 1 x,
unit-4/Turning Machine 51
 Time 9 x x y y

q2

q4 y y, R y y, L
y y, R a a, R a a, L
,
L
y y, R 0 a x, b y,
q3 q q q2
R L
x 1 x,
unit-4/Turning Machine 52
 Time 10 x x y y

q0

q4 y y, R y y, L
y y, R a a, R a a, L
,
L
y y, R 0 a x, b y,
q3 q q q2
R L
x 1 x,
unit-4/Turning Machine 53
 Time 11 x x y y

q3

q4 y y, R y y, L
y y, R a a, R a a, L
,
L
y y, R 0 a x, b y,
q3 q q q2
R L
x 1 x,
unit-4/Turning Machine 54
 Time 12 x x y y

q3

q4 y y, R y y, L
y y, R a a, R a a, L
,
L
y y, R 0 a x, b y,
q3 q q q2
R L
x 1 x,
unit-4/Turning Machine 55
 Time 13 x x y y

q4

Halt & Accept

q4 y y, R y y, L
y y, R a a, R a a, L
,
L
y y, R 0 a x, b y,
q3 q q q2
R L
x 1 x,
unit-4/Turning Machine 56
Observation:

If we modify the
machine for the language n n
{a b }

we can easily construct

n n n
a machine for the language {a b c }

unit-4/Turning Machine 57
 Turing Machines as Language
Acceptors- Example 2
• Design a TM to recognize the language of
strings of the form 0n1n2n

unit-4/Turning Machine 58
• Construct a TM that accepts the palindrome of
the string over the alphabets {a,b}
b a b b a b B

unit-4/Turning Machine 59
unit-4/Turning Machine 60
Computing Functions
with
Turing Machines
A function has:

Domain: Result Region:

unit-4/Turning Machine 62
A function may have many parameters:

Example: Addition function

unit-4/Turning Machine 63
 Integer Domain

Decimal: 5

Binary: 101

Unary: 11111

We prefer unary representation:

easier to manipulate with Turing machines

unit-4/Turning Machine 64
Definition:

A function is computable if
there is a Turing Machine such that:

Initial configuration Final configuration

initial state accept state

For all Domain

unit-4/Turning Machine 65
In other words:
A function is computable if
there is a Turing Machine such that:

Initial Halts at Final

Configuration Configuration

For all Domain

unit-4/Turning Machine 66
 Example
The function is computable

are integers

Turing Machine:

Input string: unary

Output string: unary

unit-4/Turning Machine 67
Start

initial state

The 0 is the delimiter that

separates the two numbers

unit-4/Turning Machine 68
Start

initial state

Finish

final state
unit-4/Turning Machine 69
 The 0 here helps when we use
the result for other operations

Finish

final state
unit-4/Turning Machine 70
Turing machine for function

unit-4/Turning Machine 71
Execution Example: Time 0

(=2)

(=2)

Final Result

unit-4/Turning Machine 72
Time 0

unit-4/Turning Machine 73
Time 1

unit-4/Turning Machine 74
Time 2

unit-4/Turning Machine 75
Time 3

unit-4/Turning Machine 76
Time 4

unit-4/Turning Machine 77
Time 5

unit-4/Turning Machine 78
Time 6

unit-4/Turning Machine 79
Time 7

unit-4/Turning Machine 80
Time 8

unit-4/Turning Machine 81
Time 9

unit-4/Turning Machine 82
Time 10

unit-4/Turning Machine 83
Time 11

unit-4/Turning Machine 84
Time 12

HALT & accept

unit-4/Turning Machine 85
 Another Example
The function is computable

is integer

Turing Machine:

Input string: unary

Output string: unary

unit-4/Turning Machine 86
Start

initial state

Finish

accept state
unit-4/Turning Machine 87
Turing Machine Pseudocode for

• Replace every 1 with $

• Repeat:
• Find rightmost $, replace it with 1

• Go to right end, insert 1

Until no more $ remain

unit-4/Turning Machine 88
Turing Machine for

unit-4/Turning Machine 89
 Example
Start Finish

unit-4/Turning Machine 90
 Back to binary!
• How do we add two binary numbers on a Turing
machine?

• Instead of dealing with it directly, let us divide

the problem into 2 parts
• How to subtract 1 from a binary number
• How to add 1 to a binary number

• HW – think about how to do this without

subroutines/functions/methods

unit-4/Turning Machine 91
 How to add 1 to a binary number?
•What is 11000 + 1 = ?

•What is 10111 + 1 = ?

•The pattern that emerges is …..

unit-4/Turning Machine 92
 Multitape Turing Machines
• A multitape Turing machine is like an ordinary TM
but it has several tapes instead of one tape.
• Initially the input starts on tape 1 and the other
tapes are blank.
• The transition function is changed to allow for
reading, writing, and moving the heads on all the
tapes simultaneously.
– This means we could read on multiples tape and move in
different directions on each tape as well as write a
different symbol on each tape, all in one move.

unit-4/Turning Machine 93
 Multitape Turing Machine
• Theorem: A multitape TM is equivalent in power to an
ordinary TM. Recall that two TM’s are equivalent if they
recognize the same language. We can show how to convert
a multitape TM, M, to a single tape TM, S:
• Say that M has k tapes.
– Create the TM S to simulate having k tapes by interleaving the
information on each of the k tapes on its single tape
– Use a new symbol # as a delimiter to separate the contents of each
tape
– S must also keep track of the location on each of the simulated heads
• Write a type symbol with a * to mark the place where the head on the
tape would be
• The * symbols are new tape symbols that don’t exist with M
• The finite control must have the proper logic to distinguish say, x* and x
and realize both refer to the same thing, but one is the current tape
symbol.

unit-4/Turning Machine 94
 Multitape Machine

Equivalent Single Tape Machine:

unit-4/Turning Machine 95
 Nondeterministic TM
• Replace the “DFA” part of the TM with an “NFA”
– Each time we make a nondeterministic move, you can think of this as a
branch or “fork” to two simultaneously running machines. Each
machine gets a copy of the entire tape. If any one of these machines
ends up in an accepting state, then the input is accepted.
• Although powerful, nondeterminism does not affect the
power of the TM model
• Theorem: Every nondeterministic TM has an equivalent
deterministic TM.
– We can prove this theorem by simulating any nondeterministic TM, N,
with a deterministic TM, D.

unit-4/Turning Machine 96
 Nondeterministic TM
• Visualize N as a tree with branches whenever we fork off to two (or more)
simultaneous machines.
– Use D to try all possible branches of N, sequentially.
– If D ever finds the accept state on one of these branches, then D accepts.
– It is quite likely that D will never terminate in the event of a loop if there is no
accepting state.
• Search be done in a breadth-first rather than depth-first manner.
– An individual branch may extend to infinity, and if we start down this branch
then D will be stuck forever when some other branch may accept the input.
• We can simulate N on D by a tape with a queue of ID’s and a scratch tape
for temporary storage.
– Each ID contains all the moves we have made from one state to the next, for
one “branch” of the nondeterministic tree.
– From the previous theorem, we can make as many multiple tapes as we like and
this is still equivalent to a single tape machine. Initially, D looks like the
following:

unit-4/Turning Machine 97
 Nondeterministic TM
• ID1 is the sequence of moves we make from the start state. The *
indicates that this is the current ID we are executing.
• We make a move on the TM. If this move results in a “fork” by following
nondeterministic paths, then we create a new ID and copy it to the end of
the queue using the scratch tape.

Scratch Tape
D

… ID1* Queue of ID’s

unit-4/Turning Machine 98
 Nondeterministic TM
• For example, say that in ID1 we have two nondeterministic
moves, resulting in ID2 and ID3:

Scratch Tape
D

… ID1* # ID2 # ID3

unit-4/Turning Machine 99
 Nondeterministic TM
• After we’re done with a single move in ID1, which
may result in increasing the length of ID1 and storing
it back to the tape, we move on to ID2:

Scratch Tape
D

… ID1 # ID2* # ID3

unit-4/Turning Machine 100

 Nondeterministic TM
• If any one of these states is accepting in an ID, then the
machine quits and accepts.
• If we ever reach the last ID, then we repeat back with the first
ID.

• Note that although the constructed deterministic TM is

equivalent to accepting the same language as a
nondeterministic TM, the deterministic TM might take
exponentially more time than the nondeterministic TM.
– It is unknown if this exponential slowdown is necessary. We’ll come
back to this in the discussion of P vs. NP.

• Theorem: Since any deterministic Turing Machine is also

nondeterministic (there just happens to be no
nondeterministic moves), there exists a nondeterministic TM
for every deterministic TM.

unit-4/Turning Machine 101

 Equivalence of TM’s and Computers
• In one sense, a real computer has a finite amount of
memory, and thus is weaker than a TM.
• But, we can postulate an infinite supply of tapes,
disks, or some peripheral storage device to simulate
an infinite TM tape. Additionally, we can assume
there is a human operator to mount disks, keep
them stacked neatly on the sides of the computer,
etc.
• Need to show both directions, a TM can simulate a
computer and that a computer can simulate a TM

unit-4/Turning Machine 102

THANK YOU!!!

unit-4/Turning Machine 103

Unit 5
 Unit 5
Properties of
Recursive and
Undecidability Recursively Introduction to Computational Complexity
enumerable
languages

Examples of Post Time and Space Complexity

Decidability vs
Undecidable Rice Theorem Correspondence complexity of classes: Class P, NP hardness NP Completeness
Undecidability
Problem Problem TMs Class NP

Undecidable
problems about
Turing Machine-
Post’s
Correspondence
Problem
Undecidability
Decidable Languages
• A language is decidable,
• if there is a Turing machine (decider)
• that accepts the language and
• halts on every input string
Decidability
Example
Theorem
Theorem
Theorem
Theorem
Undecidable Languages
• undecidable language = not decidable language
• There is no decider:
• there is no Turing Machine which accepts the language and makes a
decision (halts) for every input string
• (machine may make decision for some input strings)
• there is no Turing Machine (Algorithm) that gives an answer (yes or
no) for every input instance
• (answer may be given for some input instances)
Undecidable problem
Halting Problem
Turing Machines are countable
Recap : Turing machines and computability
1. Definition of Turing machines: high level and low-level descriptions

2. Variants of Turing machines

3. Decidable and Turing recognizable languages

4. Church-Turing Hypothesis UNDECIDABILITY

5. Undecidability and a proof technique by diagonalization

Unit 5 / SRMIST / KTR 20

 Undecidable problems

• is empty?

• is regular?

• has size 2?

This can be generalized to all non-trivial properties of Turing-acceptable languages

Unit 5 / SRMIST / KTR 21

 Rice’s theorem (1953)

1. Any non-trivial property of R.E languages is undecidable!

2. Property P ≡ set of languages satisfying P
3. Property of r.e languages: membership of M in P depends only on
the language of M.
If L(M) = L(M ’), then <M> ∈ P if <M’> ∈ P
4. Non-trivial: It holds for some but not all TMs.

Unit 5 / SRMIST / KTR 22

Properties of RE Languages

Unit 5 / SRMIST / KTR 23

Property : Set of language
Just a subset of r.e. languages. Thus, L satisfies a property P if L ∈ P.

Examples:
1. Set of regular languages I
2. Set of context-free languages I
3. Set of all languages I
4. {∅} I
5. ∅

• So decidability of property P is decidability of language L P .

Unit 5 / SRMIST / KTR 24

Type of Property

Unit 5 / SRMIST / KTR 25

Non-trivial property

Unit 5 / SRMIST / KTR 26

 Non-trivial property:

A property possessed by some Turing-acceptable languages but not

all
Example: : is empty?
YES
NO

NO

Unit 5 / SRMIST / KTR 27

More examples of non-trivial properties

Unit 5 / SRMIST / KTR 28

 Trivial property:
A property possessed by ALL
Turing-acceptable languages

Examples: : has size at least 0?

True for all languages

: is accepted by some
Turing machine?
True for all
Turing-acceptable languages
Unit 5 / SRMIST / KTR 29
We can describe a property as the set
of languages that possess the property

If language has property then

Example: : is empty?
YES
NO

NO
Unit 5 / SRMIST / KTR 30
Example: Suppose alphabet is

: has size 1?
NO
YES
NO
NO

Unit 5 / SRMIST / KTR 31

 Non-trivial property problem
Input: Turing Machine

Question: Does have the non-trivial

property ?

Corresponding language:

Unit 5 / SRMIST / KTR 32

 Can u apply rice theorem ?

For each of No answers above, is the language decidable?

What do you do when Rice’s theorem does not apply? Fall back on reductions!

Unit 5 / SRMIST / KTR 33

Rice Theorem ( Part 1 )

Unit 5 / SRMIST / KTR 34

Rice Theorem – The reduction

Unit 5 / SRMIST / KTR 35

Rice Theorem ( Part 2 )

Unit 5 / SRMIST / KTR 36

Rice Theorem (Part 2) – The reduction

Unit 5 / SRMIST / KTR 37

POST’s CORRESPONDENCE PROBLEM
Post Correspondence Problem

• The Post Correspondence Problem (PCP), introduced by Emil Post in

1946, is an undecidable decision problem.
• The PCP problem over an alphabet ∑ is stated as follows −
• Given the following two lists, M and N of non-empty strings over ∑ −
• M = (x1, x2, x3,………, xn)
• N = (y1, y2, y3,………, yn)
• We can say that there is a Post Correspondence Solution, if for some
i1,i2,………… ik, where 1 ≤ ij ≤ n, the condition xi1 …….xik =
yi1 …….yik satisfies.
Example 1

Find whether the lists

M = (abb, aa, aaa) and N = (bba, aaa, aa)
have a Post Correspondence Solution?
Solution for Example 1

Here,
x2x1x3 = ‘aaabbaaa’
and y2y1y3 = ‘aaabbaaa’
We can see that
x2x1x3 = y 2y 1y 3
Hence, the solution is i = 2, j = 1, and k = 3.
Example 2

• Find whether the lists M = (ab, bab, bbaaa) and N = (a, ba, bab) have
a Post Correspondence Solution?
Solution for Example 2
• General Objective:

– to understand the concept of PCP

• Specific Objectives

– State the objective of PCP

– Define PCP problem

– Check whether the PCP instance have a solution or

not.

– Define MPCP

– Illustrate the conversion of TM to MPCP with an

example
 Post Correspondence Problem
• It involves with strings

• Goal:
–
To prove this problem about strings to
be undecidable
 Post Correspondence Problem
• FORMAL DEFINITION

Given two lists of strings A and B ( equal length)

A = w1, w2, …, wk B = x1, x2, …, xk

The problem is to determine if there is a

sequence
one of integers i1, i2, …, im such
or more
that:

wi1wi2…wim = xi1xi2…xim
 Example
A B
i wi xi
1 1 111
2 10111 10
3 10 0
This PCP instance has a solution: 2, 1, 1, 3:
w2w1w1w3 = x2x1x1x3 = 101111110
Does this PCP instance have a solution?

A B
i wi xi
1 110 110110
2 0011 00
3 0110 110
This PCP instance has a solution: 2,3,1
w2w3w1 = x2x3x1 = 00110110110

One more solution:

2,1,1,3,2,1,1,3
 Modified Post Correspondence Problem (MPCP

Definition:
first pair in the A and B lists must be the first pair in

the solution, i.e., the problem is to determine if

there is a sequence of zero or more integers i1, i2,
…, im such that:

w1wi1wi2…wim = x1xi1xi2…xim
Modified Post
Correspondence
Problem (MPCP)
List A List
i wi B
xi
1 10 10
2 110 11
3 11 011
This MPCP instance has a
solution: 1,2,3

w1 w2 w3 = x 1 x2 x3
10 110 11 = 10 11 011
 Undecidability of PCP
To show that PCP is undecidable, we will
reduce the universal language problem (Lu)
to MPCP and then to PCP:

Universal
Language A mapping A mapping PCP
Problem (Lu ) MPCP

If PCP can be solved, Lu can also be solved.

Lu is undecidable, so PCP must also be
undecidable.
 Reducing MPCP to PCP
• This can be done by inserting a special
symbol (*) to the strings in list A and B of
to make sure that the first pair will
always go first in any solution.
• List A : * follows the symbols of ∑
• List B : * precedes the symbols of ∑
w = $ ; x = *$
• k+1 k+1
 Mapping MPCP to PCP
Suppose the original MPCP
instance is:
A B
i wi xi
1 1 111
2 10111 10
3 10 0
 Mapping MPCP to PCP
The mapped PCP instance will be:
A B
i wi xi
0 *1* *1*1*1
1 1* *1*1*1
2 1*0*1*1*1* *1*0
3 1*0* *0
4 $ *$
 Mapping Lu to
MPCP
• Turing machine M and an input w, we
want to determine if M will accept w.
• the mapped MPCP instance should

have a solution if and only if M

accepts w.
 Mapping Lu to
MPCP
Lu instance MPCP instance
Construct an
Given: MPCP instance Two lists:
(T,w) A and B

If T accepts w, the two lists can be

matched. Otherwise, the two lists cannot be
matched.
 Rules of Reducing Lu to
MPCP
• We summarize the mapping as follows.
Given T and w, there are five types of
strings in list A and B:
• Starting string (first pair):
List A List B
# #q0w#
where q0 is the starting state of T.
• Strings for
copying: List
List B
A X X
where X#is any tape symbol
# (including
the blank).
# is a separator can be appended to both
the lists
• Strings from the transition function
δ: List AList B
qX Yp from δ(q,X)=(p,Y,R)
ZqX pZY from δ(q,X)=(p,Y,L)
q# Yp# from δ(q,#)=(p,Y,R)
Zq# pZY# from δ(q,#)=(p,Y,L)
where Z is any tape symbol except the
blank.
• Strings for consuming the tape
symbols at the end:
List AList B
Xq q
qY q
Xq q
where q Yis an accepting state, and
each X and Y is any tape symbol
except the blank.
• Ending
string: List
List B #
A
where q is an accepting state.
q##
• Using this mapping, we can show that
the original Lu instance has a solution if
and only if the mapped MPCP instance
has a solution.
 PCP is undecidable
• Theorm: Post’s Correspondence Problem is
undecidable.
• We have seen the reduction of MPCP to
PCP
• now we see how to reduce Lu to MPCP.
– M accepts w if and only if the constructed MPCP instance
has a solution.

– As Lu is undecidable, MPCP is also undecidable.

Recursive and Recursively Enumerable Language
- Properties
Recursive Enumerable (RE) or Type -0
Language
• RE languages or type-0 languages are generated by type-0
grammars.
• An RE language can be accepted or recognized by Turing
machine which means it will enter into final state for the strings
of language
• May or may not enter into rejecting state for the strings which are not
part of the language.
• It means TM can loop forever for the strings which are not a
part of the language.
• RE languages are also called as Turing recognizable
languages.
Recursive Enumerable (RE) or Type -0
Language
• RE languages or type-0 languages are generated by type-0
grammars.
• An RE language can be accepted or recognized by Turing
machine which means it will enter into final state for the strings
of language
• May or may not enter into rejecting state for the strings which are not
part of the language.
• It means TM can loop forever for the strings which are not a
part of the language.
• RE languages are also called as Turing recognizable
languages.
Recursive Language (REC)

• A recursive language (subset of RE)

• Decided by Turing machine
• It will enter into final state for the strings of language and
rejecting state for the strings which are not part of the language.
• e.g.; L= {anbncn|n>=1} is recursive because we can construct a turing
machine which will move to final state if the string is of the form anbncn
• else move to non-final state.
• So the TM will always halt in this case.
• REC languages are also called as Turing decidable languages.
Closure Properties of Recursive
Languages
Introduction to Computational
Complexity
• Computational Problems that can be
modelled and solved by
computer
• Complexity how much of some
resource (time, space) a
problem takes up when being
solved
• The amount of resources required for executing a particular
(computation or) algorithm is the computational complexity of that
algorithm.
• In general, when we talk about complexity we are talking about time
complexity and space complexity.
Turing Machines - the Role
• Basic tool for complexity theory.
• An abstract model of computation machine: conceptually simple .
• can execute any computation possible on “known computers”
• resources consumption models well “real” computers
TIME COMPLEXITY-Turing Machine

• How long computation takes to execute

• For a Turing machine, the time complexity refers to the measure of
the number of times the tape moves when the machine is initialized
for some input symbols.
• i.e Number of machine cycles
• Time complexity of Turing Machine M on an input word w, denoted
as T(M,w) is the number of steps done by the machine before it halts.
If it does not, we set T(M,w) = ∞
• We can also define time complexity of a machine itself as follows:
• Function f : N → N is time complexity of M iff ∀n∈N :
• f (n) = max{T(M,w) : w ∈ Σ n} (assuming it halts)
SPACE COMPLEXITY-Turing Machine
• How much storage is required for computation.
• In Turing machine It is number of cells used.
• No. of bytes used
• Space used by TM M on input word w, denoted as S(M,w) is defined
as the number of tape cells that were visited by the head before M
halted.
• If M does not halt, it is not defined. We say that f : N → N is space
complexity of M (with stop property) iff ∀n∈N : f (n) = max{S(M,w) :
w ∈ Σ n}
EXAMPLE
0 0 0 1 1 1
Length n

Time complexity
x€0n1n

•Each scan takes n Steps

•The overall running time is O(n2)
•Optimal solution for one tape Turing Machine O(nlogn)
EXAMPLE
Read only Tape
0 0 0 1 1 1
Length n

Space complexity
x€0n1n
•Make the input tape as read only tape
•How much space it require for computation?
•If it uses tape as a stack it requires O(n) cells
•Add Binary counter logic to the TM so If n=64,it takes
log2(n)=7 bits in binary to keep the count

•So we can solve it using O(logn) space.

NP HARD AND NP COMPLETE
-
P and NP problems
• P:P problems refer to problems where an algorithm would take a polynomial
amount of time to solve, or where Big-O is a polynomial (i.e. O(1), O(n), O(n²),
etc).
• These are problems that would be considered ‘easy’ to solve, and thus do not
generally have immense run times.
• Examples:
• a. Linear Search
• b. Binary Search
• c. Merge Sort
• d. Quick sort etc.,
P and NP problems
• Can you guess the approximate time complexity of the given problem.
NP-Hard and Complete-Need

• Np Hard and complete problems gives a base for a research topics.

• Most of the algorithms are categorized into polynomial time (linear time),
exponential time (2^n)algorithms.

• Reason for looking other algorithms when we already have an algorithm

• Reducing the run time is the major challenge!

• Improve Efficiency until we have constant time Algorithm ie.O(1)

Polynomial Vs Exponential Algorithms
Polynomial Algorithms Exponential Algorithms

Linear Search - n 0/1 Knapsack Problem -2n

Binary Search -log n Travelling Salesman Problem-2n

Insertion sort -n 2 Sum of Subsets-2n

Merge Sort -n log n Hamiltonian cycle-2n

Exponential Algorithms are much more time consuming than Polynomial Algorithms.
The framework for converting Exponential to Polynomial Algorithms is called NP
Hard and Np Complete problems.
P and NP problems
• Non deterministic Polynomial time solving.
• Problem which can't be solved in polynomial time like TSP( travelling salesman problem)
• An easy example of this is subset sum: given a set of numbers, does there exist a subset whose sum is zero?.
• but NP problems are checkable in polynomial time means that given a solution of a problem , we can check
that whether the solution is correct or not in polynomial time.
NP and P Problems
• P -Deterministic polynomial time Algorithms
• NP-Non Deterministic Polynomial Algorithms

For eg.Merge Sort from Non deterministic to Deterministic Algorithm

NP
P
NP Hard and NP Complete
• Before knowing about NP-Hard and NP-Complete we need to know
about reducibility:
• Reduction:
• Consider there are two problems, A and B, and we know that the
problem A is NP class problem and problem B is a P class problem.
• If problem A can be reduced, or converted to problem B, and this
reduction takes a polynomial amount of time, then we can say that A
is also a P class problem (A is reducible to B).
NP Hard and NP Complete
• A problem is classified as NP-Hard when an algorithm for solving it
can be translated to solve any NP problem.
• Then we can say, this problem is at least as hard as any NP problem,
but it could be much harder or more complex.
• NP-Complete problems are problems that live in both the NP and
NP-Hard classes.
• This means that NP-Complete problems can be verified in polynomial
time and that any NP problem can be reduced to this problem in
polynomial time.
How to convert Exponential Algorithms to Polynomial
Algorithms?
This can be done in two ways
• By finding Deterministic and Non Deterministic Polynomial Algorithms
• Deterministic –finite solution for every run of the program
• Non Deterministic-random solution for every run of the program(every state has
n number of possible next states)

Or

• By solving a base problem(which includes all Exponential problems)by finding out

the relationship among them
Non Deterministic Algorithm(Magic into procedure in
future)
Problem Statement : Search an element x on A[1:n] where n>=1, on successful search return j if a[j] is equals to x otherwise return 0.
• Non-deterministic Algorithm for this problem :
1.j= choice(a, n)------------------------O(1)
2.if(A[j]==x) then
{
write(j);
success();-----------------------------O(1)
}
3.write(0);
failure();------------------------------------O(1)
Non Deterministic Methods
choice(X) : chooses any value randomly from the set X.
failure() : denotes the unsuccessful solution.
success() : Solution is successful and current thread terminates.
Satisfiability Problem(SAT)
• Definition. A formula A is satisfiable if there exists an assignment of values to its
variables that makes A true.
A=(¬x ∨ y) ∧(~y V z)^(x V ~z V y)-Three variables hence we have 23 input so this is similar to other
exponential Algorithms
It take values from 000 to 111 (different combination of binary input for each variable x,y,z)
We have to try 8 values ie.23. For n variables it will be 2n
State space tree x=1 x=0

y=1 y=0
1 0

z=1 0 1 0 1 0 1 0

If this problem is solved all other exponential algorithms can be solved in polynomial time!
Np Hard-Reduction
• SAT is a Np-Hard Problem

• SAT 0/1 Knapsack Problem(Reduction!)

• Then 0/1 knapsack is also a NP Hard Problem if the reduction is in polynomial

time

• SAT L1,then L1 is also Np-Hard

• SAT L1,L1 L2 then L2 is also NP Hard hence transitive property holds true.
 Np Complete
If a problem comes under Np Hard and if it has a non deterministic algorithm then it is
also called as a Np Complete problem

SAT L Non Deterministic NP Complete Problem

Cook’s Theorem
• Cook’s Theorem states that

• Any NP problem can be converted to SAT in polynomial time

• SAT will come into P if and only if P=NP, it will become deterministic.
Clique to Vertex Cover
Summary
• Polynomial and Exponential Algorithms

• Framework/Guidelines for working on Exponential Algorithms

• NP-Hard

• Np-Complete

• Cook’s Theorem
Hope all our Non Deterministic Algorithms will turn into Polynomial Deterministic
Algorithm i.e.P=NP☺