Unit 1-Theory of Computation
Unit 1-Theory of Computation
T.Y. B.Tech
Theory of Computation
(CET2008B)
1/7/2024 1
Course Objective & Course Outcomes
• Course Objectives:
By participating in and understanding all facets of this Course a student will be able:
1. To learn Automata theory, Regular Expression from the perspective of formal
languages
2. To learn Context Free Grammar, Pushdown Automata, Turing Machine and
Complexity Theory
3. To Aquent with various applications of Automata Theory
4. To realize Industry relevance of Automata
5. To apply automata theory concepts in design and c implementation of Compilers.
6. To apply automata theory concepts in various domains.
• Course Outcomes:
After successful completion of this course students will be able:
1. To construct Finite Automata to solve problems in computing
2. To build regular expressions and understand regular language
3. To construct context-free grammar and Push Down Automata
4. To design computational models and classify the problems of decidability
1/7/2024 2
Text Books & Reference Books
• Text Books
1. Vivek Kulkarni, Theory of Computation, Oxford University
Press, ISBN-13: 978-0-19-808458-7
2. K.L.P Mishra, N. Chandrasekaran, Theory of Computer Science
(Automata, Languagesand Computation), Prentice Hall India,
2nd Edition.
• Reference Books
1. Introduction to the Theory of Computation, Michael Sipser.
2. Introduction to Languages and the Theory of Computation, John Martin.
3. Computers and Intractability: A Guide to the Theory of NP Completeness,
M. R. Garey and D. S. Johnson
1/7/2024 3
Important Instructions
• Be ready Be Attentive
• Add your profile picture and use only MITWPU email-id for TOC communications and
submissions.
• Basic Instructions:
⮚Submission of Theory Assignments and relevant components on Ruled pages
⮚Before uploading label your pages with your Rollno., Name, Component Name.
⮚You maybe asked to keep your video ON anytime during the Sessions.
1/7/2024 4
Unit I
• Introduction to Automata: Computability and Complexity theory:
• Concepts of Automata Theory: Alphabet, languages and grammars, productions
and derivation,
• Introduction to Finite Automata,
• Simplified notation: State transition graph, Transition table, Acceptance of a
string, Acceptance of a Language,
• Deterministic finite Automata: (DFA)-Formal Definition,
• Non Deterministic Finite Automata(NFA)-Formal Definition,
• Non-Deterministic Finite Automata (NFA) with epsilon transition, Equivalence
of NFA and DFA, Conversion from NFA to DFA, Conversion from NFA with
epsilon transition to DFA, Minimization of finite automata.
• Finite Automata with output : Moore and Mealy Machine, Moore to Mealy
conversion, Mealy to Moore conversion
1/7/2024 5
Planner - Lectures
Lecture No Topics Covered
1 Introduction to TOC, Introduction to Automata: Computability and Complexity theory,
Basic concepts: Alphabet, languages and grammars, productions and derivation
9 Finite Automata with output : Moore and Mealy Machine, Moore to Mealy conversion
1/7/2024 6
About TOC
• Course is about models of computation, their power, and relationships.
• Hierarchy of models of increasing power:
-DFA < PDA < TM.
For each model, we study:
-What can be computed.
-What cannot be computed.
1/7/2024 7
History of TOC
1/7/2024 8
Complexity theory and
Computability theory
- Computability theory is concerned with what can be computed
versus what cannot.
1/7/2024 9
Automata theory
• Automata are used as theoretical models for computing machines, and are
used for proofs about computability.
1/7/2024 10
Basic Definitions
1/7/2024
11
Basic Definitions
• Alphabet - a finite set of symbols.
✓ An alphabet is a finite set of symbols. These symbols are the basic elements
from which strings are formed.
✓ In the context of automata theory, an alphabet is used to define the input and
output symbols for a machine or a formal language.
✓ Notation: Σ .
✓ E.g.: Binary alphabet {0,1},
English alphabet {a,...,z,!,?}
1/7/2024 12
Basic Definitions (cont…)
• Languages:
✓ In automata theory, a language is a set of strings over a given alphabet.
✓ Languages can be classified into various types, such as regular languages,
context-free languages, context-sensitive languages, and recursively
enumerable languages, based on the complexity of the rules defining
them.
• Language over alphabet Σ - a set of strings over Σ.
Notation: L.
E.g:{0, 00, 000, ...} is an "infinite" language over the binary alphabet.
{a, b, c} is a "finite" language over the English alphabet.
• Empty Language: Empty set of strings.
Notation: Ф
1/7/2024 13
Basic Definitions (cont…)
• Regular Language- a finite set of symbols.
L is called regular Language if there is some automaton that accepts L.
E.g.Σ* , Even length of strings, strings containing abba are regular Languages.
• Grammars:
✓ A grammar is a set of production rules that describe how strings from a
language can be generated.
✓ Formal languages are often defined by grammars. Different types of
grammars are used to generate different classes of languages.
1/7/2024 14
Basic Definitions (cont…)
• Productions:
✓ Productions are the rules in a grammar that specify how symbols can be replaced by
other symbols or strings.
✓ In the context of context-free grammars, a production typically takes the form of a
non-terminal symbol being replaced by a sequence of terminal and/or non-terminal
symbols.
• Derivation:
✓ Derivation is the process of applying production rules to generate strings in a
language.
✓ Starting with an initial symbol, successive applications of production rules lead to
the generation of strings in the language defined by the grammar.
1/7/2024 15
Basic Machine and Finite State
Machine
• Basic Machine:
• A machine that recognizes input set I and produces
output set O,
where I & O are finite.
• Machine Function(MAF): I-> O
• Deals with what output is generated, not how output
is generated
E.g.: Switch
1/7/2024 16
Basic machine
It is the most primitive machine with a set of inputs I and set of output O.
E.g. AND,OR,NOR… gates , Vending machine, Decimal to binary converter,
Electric fan…
Weighing machine : I/p- Coin
O/p- Printed weight ticket
It just maps input set to output set.
Mapping function is called Machine function MAF I -> O
(0,1) 0
(1,0) 0
(1,1) 1
Limitations:
Cannot remember the intermediate states.
1/7/2024 17
Finite State machine(FSM)
In FSM the internal state of machine changes when it
receives I/P.
It has a pair of functions:
Machine Fn MAF : I x S -> 0
For a given i/p there will be always only one final state.
1/7/2024 18
Finite Automata
• 5-tuple:
• M= (Q,Σ, 𝛿, qo, F)
where,
• Q: Finite set of states
• Σ: Finite input alphabet
• 𝛿: STF that maps Q X Σ to Q, i.e. Q X Σ Q
• qo: Initial state of FA, qo ∈ Q
• F: Set of final states, F ⊆ Q
1/7/2024 19
Finite Automata (cont…)
Input
String
Output
Finite
Automaton Accept/Reject
1/7/2024 20
Block diagram of FA
0 0 1 Input Tape
Tape Head
Finite Control
q0
1/7/2024 21
Acceptance of a String
M= (Q,Σ, 𝛿, qo, F)
if 𝛿 (qo,x) = p,
1/7/2024
22
Acceptance of a Language
1/7/2024
23
Finite Automata (cont…)
1/7/2024
24
Preferred Notation for FA
❑Transition Diagram
❑Transition Table
1/7/2024 25
Transition Diagram
q0
⮚There is arrow into start state.
qf
• Given a state Sj, the same input symbol does not cause the FA to
move into more than one state-there is always a unique next state
for an input symbol
0
0
S0 S1
1 1
0 S2
1/7/2024 28
Designing FA
a
q0 q1 a,b
1/7/2024 29
Finite Automata Example
FA for language over {1,0} in which every string ends
with 10.
0
1
1 0
Q0 Q1 Qf
0 1
1/7/2024 30
Deterministic finite automata
(D.F.A)- Formal definition
▪ A deterministic finite automaton M is a 5-tuple, (Q, Σ, δ, q0, F)
consisting of-
• a finite set of states (Q)
• a finite set of input symbols called the alphabet (Σ)
• a transition function (δ : Q × Σ → Q)
• a start state (q0∈ Q)
• a set of accept states (F ∈ Q)
▪ Let w = a1a2 ... an be a string over the alphabet if a sequence of
states, r
▪ r0 = q0
▪ ri+1 = δ(ri, ai+1), for i = 0, ..., n-1
▪ rn∈ F.
1/7/2024 31
FA(DFA & NFA) Examples
1. Construct a FA to recognize language containing strings starting
with ‘10’
2. Construct a FA to recognize language containing strings ending
with ‘101’
3. Construct a FA to recognize language of strings with exactly two
0s anywhere
4. Construct a FA to recognize language containing strings starting
with ‘a’
5. Construct a FA to identify string containing even number of ‘a’s
over ∑ ={a}
1/7/2024
32
Non-Deterministic Finite Automata
• We can always convert an NFA to a DFA, but the latter may have exponentially
more states than the NFA (a rare case)
• The difference between the DFA and the NFA is the type of transition function δ
• For a NFA δ is a function that takes a state and input symbol as arguments
(like the DFA transition function), but returns a set of zero or more states
(rather than returning exactly one state, as the DFA must)
1/7/2024 33
NFA Definition
• 5-tuple:
• M= (Q,Σ, 𝛿, qo, F)
where,
• Q: Finite set of states
• Σ: Finite input alphabet
• 𝜹: STF that maps Q X 𝜮 to Q, i.e. Q X 𝜮 2Q
• (Here the power set of Q (2Q) has been taken because
in case of NDFA, from a state, transition can occur to
any combination of Q states)
• qo: Initial state of FA, qo ∈ Q
• F: Set of final states, F ⊆ Q
1/7/2024 34
1/7/2024 34
Example : pair of 0’s or pair of 1’s
0/1 0/1
Q = {q0, q1, q2 , q3 , q4}
Σ = {0, 1} 0 0 q4
Start state is q0 q0 q3
F = {q2, q4}
1 0/1
δ: 0 1
q0 {q0, q3} {q0, q1} 1
q1 q2
q1 {} {q2}
{q2} {q2}
q2
q3 {q4} {}
{q4} {q4}
q4
1/7/2024 35
NFA
1/7/2024 36
NFA Examples
1/7/2024
37
NFA
1
q0 0 q1 q2
1/7/2024 38
Equivalence of NFA and DFA
For every NFA, there exists an equivalent DFA.
NFA and DFA have equal powers
• Example: NFA vs DFA
1/7/2024 39
Conversion of NFA to DFA
(Method I)
1/7/2024 40
Conversion of NFA to DFA
(Method I) (Cont..)
1/7/2024 41
Conversion of NFA to DFA
(Method I) (Cont..)
Example:
Convert the NFA : M = ({q0, q1}, {0, 1}, 𝛿, q0,
{q1}) to its equivalent DFA
State Transition Table 𝜹 for example NFA
0 1
0,1
q0 q1
1/7/2024 42
Conversion of NFA to DFA
(Method I) (Cont..)
State Transition table for resultant NFA State Transition Graph for resultant NFA
1/7/2024 43
Conversion of NFA to DFA
(Method II)
1/7/2024 44
Conversion of NFA to DFA
(Method II) (Cont..)
1/7/2024 45
Conversion of NFA to DFA
(Method II) (Cont..)
• Example:
– Convert the NFA: M = ({q0, q1}, {0, 1}, 𝛿, q0,
{q1}) to its equivalent DFA
State Transition Table 𝜹 for example NFA Transition Graph for example NFA
1/7/2024 46
Conversion of NFA to DFA
(Method II) (Cont..)
DFA construction from the given NFA:
(a) Step 1 (b) Step 2 (c) Step 3 (d) Step 4 (e)Final DFA
1/7/2024 47
DFA Minimization
• Identify equivalent states and keep any one of them.
1/7/2024 48
DFA Minimization (cont..)
• Equivalent states:
– go to same next state on reading the same input
symbol
– same type (final or non-final)
• Unreachable states:
– Can not be reached from initial state on reading any
input symbol
• Dead (or Trap) states:
– Have no outgoing transitions, except to themselves
1/7/2024 49
DFA Minimization Rules
1/7/2024 50
DFA Minimization Rules (cont…)
• Replacing state A by state B means deleting all entries related to state A
i.e. deleting all the transitions for state A from the state transition table.
• If more equivalent states are found after applying all the rules,
repeat the same five steps to further reduce the DFA.
Example
𝜮 0 1
Q
p p,q p
q r r
r s -
s s s
1/7/2024 51
DFA Minimization (cont…)
State Transition Table of the DFA to be minimized
1/7/2024 52
DFA Minimization (cont…)
Minimized DFA
1/7/2024 53
DFA Minimization (cont…)
Further minimized DFA
1/7/2024 54
NFA with 𝜀 − Transitions
• 5-tuple:
• M= (Q,Σ, 𝛿, qo, F) where,
• Q: Finite set of states
• Σ: Finite input alphabet
• 𝜹: STF that maps Q x (𝜮∪ { 𝜺} ) to 2Q
i.e. Q X (𝜮 ∪{𝜺 }) 🡪 2Q
1/7/2024 55
NFA with 𝜀 − Transitions (cont..)
• Example:
Transition Graph for example NFA with 𝛆 − 𝐭𝐫𝐚𝐧𝐬𝐢𝐭𝐢𝐨𝐧𝐬
1/7/2024 56
NFA with ℇ moves Example
1/7/2024 57
NFA with 𝜀 − Transitions (Examples)
• Example:
Transition Graph for example NFA with 𝛆 − 𝐭𝐫𝐚𝐧𝐬𝐢𝐭𝐢𝐨𝐧𝐬
1/7/2024 58
NFA with 𝜀 − Transitions
(Examples)
• Example:
Convert the NFA with 𝜀 −transitions given in the figure to its equivalent DFA
State Transition table for example NFA with State Transition Graph for example NFA
𝛆 −transitions with 𝛆 −transitions
59
1/7/2024
NFA with 𝜀 − Transition (Examples)
• Example:
Transition Graph for example NFA with 𝛆 − 𝐭𝐫𝐚𝐧𝐬𝐢𝐭𝐢𝐨𝐧𝐬
1/7/2024 60
Significance of NFA with
𝜀 − Transitions
• Helps to split complex language acceptance problems into smaller
once.
• Solution to these problems can be integrated with the help of ϵ -
transition.
• Concatenation ,parallel path can be connected with the ϵ -
transition.
1/7/2024 61
C Conversion of NFA with
𝜀 − Transitions to DFA
• Indirect Conversion Method
• Direct Conversion Method
NFA with 𝜀 moves
Indirect Method
Direct Method
NFA without 𝜀
moves
Equivalent DFA
1/7/2024 62
Indirect Conversion Method
1/7/2024 63
Indirect Conversion Method (cont…)
• 𝜀-closure of a state:
• Set of all states p, such that there is a path from state q to
state p labeled ‘𝜀’, is known as 𝜀-closure (q)
• Set of all the states having distance zero from state q
• Every state is at distance zero from itself
• Denoted by 𝛿^
1/7/2024 64
Indirect Conversion Method (cont…)
• Example:
Convert the NFA with 𝜀 −transitions given in the figure to its equivalent
DFA
State Transition table for example NFA with State Transition Graph for example NFA
𝛆 −transitions with 𝛆 −transitions
1/7/2024 65
Indirect Conversion Method (cont…)
1/7/2024 66
Indirect Conversion Method (cont…)
1/7/2024 67
Indirect Conversion Method (cont…)
2. Conversion of NFA without 𝛆 −transitions to an equivalent DFA:
State Transition table for the equivalent DFA Reduced State Transition table for the
equivalent DFA
1/7/2024 68
Direct Conversion Method
• Begin with the initial state
• Go on adding the states as & when required to the diagram
• Follow the same process until there exists no state without
having the transitions specified
• Each state label consists of:
• Name of state label
• Combination of all the state symbols, which are reachable
from the given state
• Start building an equivalent DFA with the transition diagram of
given NFA
• Find the transition, one state at a time
• If the next state of a given transition is a new combination, add
that to the set of states for the resultant DFA
1/7/2024 69
Direct Conversion Method (cont…)
1/7/2024 70
1/7/2024 71
1/7/2024 72
1/7/2024 73
1/7/2024 74
1/7/2024 75
1/7/2024 76
Convert Є-NFA to DFA example
a b c Є
Solution Є-closure(p) = { p}
NFA Є-closure(p) = { p,q}
a b c Є-closure(p) ={ p,q,r}
1/7/2024 77
Convert Є-NFA to DFA example
0 1 Є
A {A} {B}
B {C} {D}
C {B} {D}
*D {D} {D}
Solution
NFA
0 1 DFA using direct method
A {A,B,C,D} {D} 0 1
B {C,D} {D} [A] [ABCD] [D]
[ABCD] [ABCD] [BD]
C --- {B,D}
*[D] -[D] [D]
*D {D} {D} [BD} [CD] [D]
[CD} [D] [BD]
1/7/2024 78
Applications and Limitations of FA
• Finite Automata Applications:
• Software for designing and checking behavior of digital circuits
• The “Lexical Analyzer” of a compiler
• Software for scanning large bodies of text
E.g. to find occurrences of words in collections of web
pages
• Software for verifying systems of all types that have a finite
number of distinct states
E.g. communication Protocols
• Used in Text Editor and Spell Checkers.
1/7/2024 79
Applications and Limitations of FA
➢ Limitations:
• Cannot recognize Palindrome of string.
• Set of strings over "(" and ")" & have balanced
parenthesis.
• No Memory, Only memory it has is State and State.
1/7/2024 80
Applications of TOC
1/7/2024 81
Application of TOC (cont…)
• Turing Machine:
• Prototype/Abstract of modern computer
• To find out what a computer can do and what it can
not do
• Help us understand what we can expect from our
software
• To decide, whether we can solve the problem, which
requires more time to solve, in less amount of time.
1/7/2024 82
Thank You…!!!
1/7/2024 83