CSE 204- Theory of

Computation
Introduction Lecture

Course Level: UG
Credits: 4 (L-3 T-0 P-0)

By: Dr. Pronaya Bhattacharya

Disclaimer: All the slides are prepared from standard reference books/IIT
NPTEL/Web Links with appropriate permissions taken.
About the Subject Faculty
Dr. Pronaya Bhattacharya (Associate Professor and Head of Department-CSE ASETK, and
Member-Research and Innovation Cell, Amity University, Kolkata)
PhD in Optical Networks (CSE)- AKTU, Lucknow, UP (2021) (State Govt. Univ.)
M. Tech (Hons., Gold Medal) in Network Security (IT), Karnataka State Univ., Mysore,
Karnataka (2012) (State Govt Univ.)
B. Tech in Computer Science and Engineering- AKTU, Lucknow, UP (2008). (State Govt Univ.)
Teaching experience of more than 12 years and industry experience of 2 years.
More than 170 research papers in leading SCI/SCIE/Scopus journals and conferences of repute.
Listed as Top 2% Scientists Worldwide as per Stanford University List, published by Elsevier.
UGC-NET (2017), and GATE qualified (2011,2012)
Google Scholar- 4143, H-index- 34, I10-index-75, Scopus-2689, H-index: 28
12 Best Research paper awards for outstanding research work.
Winner of Hackathon Coding Contest organized by IIT Kharagpur funded by HRD, Govt. of
India (2014)
Research Domains- Optical Networks, Blockchain Technology, Network Communications,
Artificial Intelligence
Syllabus of Module 1 (25%)

Module I: Introduction to Languages and Automata 25

Formal Grammars and Chomsky Hierarchy, Regular Expression
Deterministic and Nondeterministic Finite Automata, Regular
Expression, Regular Expression to Finite Automata and vice versa,
Arden’s Theorem , Two way Finite Automata, Finite Automata with
output, Properties of regular sets, pumping lemma for regular sets, My-
Hill-Nerode Theorem .
Reference Books
1. Introduction to theory of computation by Michael sipser.
2. An introduction to formal languages and Automata by Peter Linz,
Narosa Publication Papadimitrou, C. and Lewis, C.L., “Elements of
the Theory of Computation”, PHI
Course Learning Outcomes:
• Prerequisites: Discrete Mathematics, Basic Mathematical Fundamentals,
Proof Strategies.
• At the end of this course, the student will be able to
• Familiarity with basics of Formal Languages, Grammars, Automata and
their relationship.
• Design regular expressions and context-free grammars accepting or
generating a certain language.
• Transform between equivalent deterministic and non-deterministic finite
automata, and regular expressions.
• Design Push Down Automata, Equivalence of PDA to CFG and vice-versa.
• Designing of Turing machines to solve problems
• Classification of a Problem as decidable and undecidable.
• Have an overview of how the theoretical study in this course is applicable
to an engineering application like designing the compilers
Assessment Scheme
A general computer operation
z = 2*2 = 4
temporary memory
f ( x) = z * 2 = 8
input
f ( x) = x 3

x=2
CPU
f ( x) = 8
Program memory output
compute xx
compute x x
2
Automaton-Mathematical Model of General Computer

temporary memory

Automaton
input
CPU
output

Program memory
 Automaton
temporary memory

Automaton
input

output
transition

state
What is Automata Theory?
• Study of abstract computing devices, or “machines”
• Automaton = an abstract computing device
• Note: A “device” need not even be a physical
hardware!
• A fundamental question in computer science:
• Find out what different models of machines can do and
cannot do
• The theory of computation
• Computability vs. Complexity

10
 Different Kinds of Automata
Automata are distinguished by the temporary memory

• Finite Automata: no temporary memory

• Pushdown Automata: stack

• Turing Machines: random access memory

 Finite Automaton

temporary memory

input
Finite
Automaton
output

Example: Elevators, Vending Machines

(small computing power)
 Pushdown Automaton
Temp.
memory Stack Push, Pop

Pushdown input

Automaton
output

Example: Compilers for Programming Languages

(medium computing power)
 Turing Machine

Temp.
memory Random Access Memory

input
Turing
Machine
output
Examples: Any Algorithm
(highest computing power)
 Power of Automata
Simple More complex Hardest
problems problems problems

Finite Pushdown Turing

Automata Automata Machine

Less power More power

Solve more
computational problems
Turing Machine is the most powerful
computational model known

Question: Are there computational

problems that a Turing Machine
cannot solve?

Answer: Yes (unsolvable problems)

Time Complexity of Computational Problems:

NP-complete problems
Believed to take exponential
time to be solved

P problems
Solved in polynomial time
Alan Turing (1912-1954) (A pioneer of automata theory)

• Father of Modern Computer Science

• English mathematician
• Studied abstract machines called
Turing machines even before
computers existed
• Heard of the Turing test?

18
Theory of Computation: A Historical
Perspective
1930s • Alan Turing studies Turing machines
• Decidability
• Halting problem

1940-1950s • “Finite automata” machines studied

• Noam Chomsky proposes the
“Chomsky Hierarchy” for formal
languages

1969 Cook introduces “intractable” problems

or “NP-Hard” problems

1970- Modern computer science: compilers,

computational & complexity theory evolve

19
Languages & Grammars
• Languages: “A language is a collection of
Or “words” sentences of finite length all constructed
from a finite alphabet of symbols”
• Grammars: “A grammar can be regarded as
a device that enumerates the sentences of a
language” - nothing more, nothing less

• N. Chomsky, Information and Control, Vol 2,

1959

Image source: Nowak et al. Nature, vol 417, 2002 20

The Chomsky Hierachy
•A containment hierarchy of classes of formal languages

Regular Context-
(DFA) free Context-
Recursively-
(PDA) sensitive
enumerable
(LBA)
(TM)

21
Alphabet
An alphabet is a finite, non-empty set of symbols
• We use the symbol ∑ (sigma) to denote an alphabet
• Examples:
• Binary: ∑ = {0,1}
• All lower case letters: ∑ = {a,b,c,..z}
• Alphanumeric: ∑ = {a-z, A-Z, 0-9}
• DNA molecule letters: ∑ = {a,c,g,t}
• We use capital Greek letter Σ to designate the alphabets and Γ (pronounced as
gamma) to designate the typewriter font for symbols
• Examples:
• Σ1 = {0, 1}
• Σ2 = {a, b, …, z}, the set of all lower case letters
• The set of all ASCII characters or the set of all printable ASCII characters
• Γ = {0, 1, x, y, z}

22
Strings
A string or word is a finite sequence of symbols chosen from ∑
• Empty string is  (or “epsilon”)

• Length of a string w, denoted by “|w|”, is equal to the number of (non-

) characters in the string
• E.g., x = 010100 |x| = 6
• x = 01  0  1  00  |x| = ?

• xy = concatentation of two strings x and y

23
Some definitions
• If Σ1 = {0, 1} then 01001 is a string over the alphabet Σ1.
• If Σ2 = {a, b, c,…, z} then abracadabra is a string over Σ2.
• Length of String: If w is a string over Σ then the length of w is written as |w| which is the
number of symbols that it contains.
• The string of length zero is called the empty string. It is written as ε. The empty string plays the
role of 0 in a number system.
• If w has length n then we can write w = w1, w2, …, ws where each wi Є Σ.
• Reverse: The reverse of w written as wR is the string obtained by writing it in the
opposite order (i.e. wn, wn-1, … , w1).
• Substring: String z is a substring of w if z appears consecutively within w. For example
cad is a substring of abracadabra.
• Concatenation: If we have string x of length m and string y of length n , the
concatenation of x and y written as xy is the string obtained by appending y to the end of
x as in x1 . . . xmy1 . . . yn .
• Lexicographic Ordering: The lexicographic ordering of strings is the same as the familiar
dictionary ordering except that shorter strings precede longer strings.
• For example lexicographic ordering of all strings over the alphabet {0, 1} is (ε, 0, 1, 00, 01,
10, 11, 000, 0001, …).
Powers of an alphabet
Let ∑ be an alphabet.
Σ0 = ε, no matter what the alphabet Σ is. In other words ε is the only string of
length 0.

• ∑k = the set of all strings of length k

• ∑* = ∑0 U ∑1 U ∑2 U …

• ∑+ = ∑1 U ∑2 U ∑3 U …

25
Example
• Σ ={0,1}
• Σ1= {0,1} ( 21=2)
• Σ2= {00,01,10,11} (22=4)
• Σ3= {000,001,010,011,100,101,110,111} (23= 8)
• Σ*= {0,1}* ={ ε,0,1,00,01,10,11,………}
• Therefore, Σ*= Σ0U Σ1U Σ2U Σ3…………. With ε symbol
• The set of strings over an alphabet Σ excluding ε is usually denoted by
Σ+(Kleene plus) For instance, Σ+={0,1}+ ={0,1,00,10,01,11,…………}
Therefore, Σ+= Σ*- { ε}
Problems:
1. What if Σ ={a,b}. Compute Σ* and Σ+.
2. If Σ = {a, b, c}. Compute Σ2 and Σ3
Languages
L is a said to be a language over alphabet ∑, only if L  ∑*
➔ this is because ∑* is the set of all strings (of all possible length including 0) over the given
alphabet ∑
Examples:
1. Let L be the language of all strings consisting of n 0’s followed by n 1’s:
L = {, 01, 0011, 000111,…}
2. Let L be the language of all strings of with equal number of 0’s and 1’s:
L = {, 01, 10, 0011, 1100, 0101, 1010, 1001,…}

Canonical/Lexicographic ordering of strings in the language

Definition: Ø denotes the Empty language

• Let L = {}; Is L=Ø? NO

27
Some more examples
• The language of all strings consisting of n 0’s followed by n 1’s for some
n≥0; { ε, 01, 0011, 000111, . . . }.
• The set of strings of 0’s and 1’s with an equal number of each : {ε, 01, 10,
0011, 0101, 1001, . . .}
• The set of binary values whose value is prime: {10, 11, 101, 111, 1011,. . .}
• Σ* is a language for any alphabet Σ.
• ɸ the empty language is a language over any alphabet.
• {ε} the language consisting of only the empty string, is also a language over
any alphabet.
• Note: ɸ ≠ {ε}; the former has no strings and the later has one string.
• If L1 and L2 are two languages then we can generate a new language L1L2
which is defined as follows: L1L2 = {xy | x Є L1 and y Є L2}
The Membership Problem
Given a string w ∑*and a language L over ∑, decide whether or not w
L.
Example:
Let w = 100011
Q) Is w  the language of strings with equal number of 0s and 1s?

29
Real-Life Example
• The controller for an automatic door is one example of such a device
which is often found at supermarket entrances and exits.
• The controller is in either of two states “OPEN” and “CLSOED”,
representing the corresponding condition of the door.
• FSM-
Finite Automata : Examples
• On/Off switch action

state

• Modeling recognition of the word “then”

Start state Transition Intermediate Final state

state

31
Finite Automata
• Some Applications
• Software for designing and checking the behavior of digital circuits
• Lexical analyzer of a typical compiler
• Software for scanning large bodies of text (e.g., web pages) for pattern finding
• Software for verifying systems of all types that have a finite number of states
(e.g., stock market transaction, communication/network protocol)

32
Formal Definition of Finite Automaton
• A finite automaton is defined as a 5 – tuple
(Q, Σ, δ, q0, F) where
• 1. Q is the finite set called the states.
• 2. Σ is the finite set called the alphabet.
• 3. δ: Q x Σ → Q is the transition function.
• 4. q0 Є Q is the start state, and
• 5. F ⊆ Q is the set of accept states.
• By setting F to be the empty set ɸ yields 0
accepts states which is allowable. It means
machine does not accept/validate any
input.