Department of Computer and Science Engineering (CSE)

UNIVERSITY INSTITUTE OF
ENGINEERING
COMPUTER SCIENCE ENGINEERING
Bachelor of Engineering
Theory of Computation (CST-353)

Topic: Introduction DISCOVER . LEARN .

University Institute of Engineering (UIE) EMPOWER
 Department of Computer and Science Engineering (CSE)

Learning Objectives & Outcomes

Objective:
• To understand the basics to automata.

Outcome:
• Student will understand the
 Basics of automata

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

What is Automata Theory?

• Study of abstract computing devices, or
“machines”
• Automaton = an abstract computing device
• A fundamental question in computer science:
– Find out what different models of machines can do and
cannot do
– The theory of computation
• Computability vs. Complexity

3
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

(A pioneer of automata theory)

Alan Turing (1912-1954)

• Father of Modern Computer

Science

• English mathematician

• Studied abstract machines called

Turing machines even before
computers existed

• Heard of the Turing test?

4
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Theory of Computation: A Historical Perspective

1930s • Alan Turing studies Turing machines

• Decidability
• Halting problem

1940-1950s • “Finite automata” machines studied

• Noam Chomsky proposes the
“Chomsky Hierarchy” for formal
languages
1969 Cook introduces “intractable” problems
or “NP-Hard” problems
1970- Modern computer science: compilers,
computational & complexity theory evolve
5
University Institute of Engineering (UIE)
 Languages & Grammars
• Languages: “A language is a
Or “words” collection of sentences of finite
length all constructed from a
finite alphabet of symbols”
• Grammars: “A grammar can
be regarded as a device that
enumerates the sentences of a
language” - nothing more,
nothing less

• N. Chomsky, Information and

Control, Vol 2, 1959

Image source: Nowak et al. Nature, vol 417, 2002

University Institute of Engineering (UIE) 6
 Department of Computer and Science Engineering (CSE)

Alphabet

An alphabet is a finite, non-empty set of symbols

• We use the symbol ∑ (sigma) to denote an
alphabet
• Examples:
– Binary: ∑ = {0,1}
– All lower case letters: ∑ = {a,b,c,..z}
– Alphanumeric: ∑ = {a-z, A-Z, 0-9}
– DNA molecule letters: ∑ = {a,c,g,t}
– …

7
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Powers of an alphabet
Let ∑ be an alphabet.

– ∑k = the set of all strings of length k

– ∑* = ∑0 U ∑1 U ∑2 U …

– ∑+ = ∑1 U ∑2 U ∑3 U …

8
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Strings

A string or word is a finite sequence of symbols

chosen from ∑
• Empty string is  (or “epsilon”)

• Length of a string w, denoted by “|w|”, is equal to

the number of (non- ) characters in the string
– E.g., x = 010100 |x| = 6
– x = 01  0  1  00  |x| = ?

– xy = concatentation of two strings x and y

9
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Kleene Closure (*)

• Kleene Closure (*) gives an infinite set of all possible strings

of any length, including epsilon (ε), over input values of sigma
(Σ). It is also known as Kleene Star and denoted by Σ*. The
following is a mathematical notation of Kleene Clouser.
• Σ* = Σ0 U Σ1 U Σ2 ………. .
• Where Σ0 represents epsilon (ε) and gives infinite language.
• Example: Suppose Σ input value is “(0,1)”, applying Kleene
Closure (*) over Σ input which gives the following result
• {0,1}* =( 0∪1)* = {ε,0,1,11,00, 01,10 ,0101,1010, ………….,
n} // where n is infinity

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Positive Closure

• Positive Closure is similar to Kleene Closure except epsilon (ε).

It also gives an infinite set of all possible strings of any
length, excluding epsilon (ε) over input values of sigma (Σ).
Σ+ denotes it. The following is the mathematical notation of
Positive Closure.
• Σ+ = Σ*– {ε}
• Suppose Σ input value is “(0,1)”, applying Positive
Closure (+) over Σ input which gives the following result
• {0,1}+ = (0∪1)+ = {0,1,11,00, 01,10 ,0101,1010, …………., n}
// where n is infinity
• Result (0,1)+ will give all possible combinations of “0” and
“1” of any length excluding epsilon (ε). So, we can say
Positive Closure always gives infinite language.

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Examples of Positive Closure

All examples of positive Closure are similar to Kleene closure

except epsilon (ε). The following are the various examples of
Positive Closure over sigma (Σ) input values.
Example 01: Kleene Closure applied to an empty set, the
result is an empty set
∅+ = {}
Example 02: Kleene Closure applied to single character
{a}+ = {a, aa, aaa, aaaa, …………….} // all combination of “a”
with any length

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Difference Between Σ* and Σ+

• Σ* contains the epsilon (ε) sting along with other N

strings, but Σ+ is just like the Σ* but does not hold the
epsilon (ε) sting.
• We can say
• Σ* = Σ+ + Σ0 . Where Σ0 represent epsilon (ε)
• So, Σ* holds the identity(single) element called Absalon,
but Σ+ does not have the identity element called
epsilon (ε)

University Institute of Engineering (UIE)

 Grammer

• It is a formal system that defines a set of rules for

generating valid strings within a language.
• It serves as a blueprint for constructing
syntactically correct sentences or meaningful
sequences in a formal language.

University Institute of Engineering (UIE) 14

 Representation of Grammar

Any Grammar can be represented by 4 tuples – <N, T,

P, S>
•N – Finite Non-Empty Set of Non-Terminal Symbols.
•T – Finite Set of Terminal Symbols.
•P – Finite Non-Empty Set of Production Rules.
•S – Start Symbol (Symbol from where we start
producing our sentences or strings).

University Institute of Engineering (UIE) 15

 Parts of Grammar
1. Terminal Symbols: Terminal symbols are those that
are the components of the sentences generated using
grammar and are represented using small case letters
like a, b, c, etc.
2. Non-Terminal Symbols: Non-terminal symbols are
those symbols that take part in the generation of the
sentence but are not the component of the sentence.
Non-Terminal Symbols are also called Auxiliary Symbols
and Variables. These symbols are represented using a
capital letters like A, B, C, etc.

University Institute of Engineering (UIE) 16

 3. Production Rules in Grammer

• A production or production rule in computer science is a

rewrite rule specifying a symbol substitution that can be
recursively performed to generate new symbol sequences. It is
of the form α-> β
where α is a Non-Terminal Symbol which can be replaced by β
which is a string of Terminal Symbols or Non-Terminal
Symbols. Consider Grammar G1 = <N, T, P, S>
T = {a,b} #Set of terminal symbols
P = {A->Aa,A->Ab,A->a,A->b,A-> ϵ} #production rules
S = {A} #Start Symbol

University Institute of Engineering (UIE) 17

 Types of Grammer

University Institute of Engineering (UIE) 18

 The Chomsky Hierachy
• A containment hierarchy of classes of formal languages

Regular Context-
(DFA) Context- Recursively-
free
sensitive enumerable
(PDA)
(LBA) (TM)

University Institute of Engineering (UIE) 19

The Chomsky Hierachy

University Institute of Engineering (UIE) 20

The Chomsky Hierachy

University Institute of Engineering (UIE) 21

 Department of Computer and Science Engineering (CSE)

Type-0 Grammar (RE)

• Type-0 grammars generate recursively enumerable
languages.
• The productions have no restrictions. They are any phase
structure grammar including all formal grammars.
• They generate the languages that are recognized by a
Turing machine.
• The productions can be in the form of α → β where α is a
string of terminals and non terminals with at least one
non-terminal and α cannot be null. β is a string of
terminals and non-terminals.
• For example: S → ACaB
Bc → acB CB → DB aD → Db

University Institute of Engineering (UIE) 22

 Department of Computer and Science Engineering (CSE)

Type-1 Grammar(CSG)
• Type-1 grammars generate context-sensitive languages. The
productions must be in the form
• αAβ→αγβ
• where A ∈ N (Non-terminal)
• and α, β, γ ∈ (T ∪ N)* (Strings of terminals and non-
terminals)
• The strings α and β may be empty, but γ must be non-empty.
• The rule S → ε is allowed if S does not appear on the right side
of any rule. The languages generated by these grammars are
recognized by a linear bounded automaton.
• Example
• AB → AbBc A → bcA B → b
University Institute of Engineering (UIE)
 Department of Computer and Science Engineering (CSE)

Type-2 grammar (CFG)

• Type-2 grammars generate context-free languages.
• The productions must be in the form A → γ
• where A ∈ N (Non terminal)
• and γ ∈ (T ∪ N)* (String of terminals and non-terminals).
• These languages generated by these grammars are be
recognized by a non-deterministic pushdown automaton.
Example
• S→Xa
• X→a
• X → Ax
• X → abc
• X→ε

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Type 3 Grammar(Regular Grammar)

• Type-3 grammars generate regular languages. Type-3
grammars must have a single non-terminal on the left-
hand side and a right-hand side consisting of a single
terminal or single terminal followed by a single non-
terminal.
• The productions must be in the form X → a or X → aY
• where X, Y ∈ N (Non terminal)
• and a ∈ T (Terminal)
• The rule S → ε is allowed if S does not appear on the right
side of any rule.
• Example
• X→ε X → a | aY Y→b

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

Comparison of all Grammars

University Institute of Engineering (UIE)

 Department of Computer and Science Engineering (CSE)

References :
• Martin J.C., “Introduction to Languages and Theory of
Computation”, Tata McGraw-Hill Publising Company
Limited, 3rd Edition.
• https://youtu.be/S3cOulqSAmU
• Https://en.wikipedia.org/wiki/Finite-state_machine
• https://www.safaribooksonline.com
• https://nptel.ac.in/courses/106/103/106103070/

University Institute of Engineering (UIE)

THANK YOU

University Institute of Engineering (UIE)