IBADAN DISTANCE LEARNING SERIES

CSC 351
(Formal Languages and Automata Theory)
By
Ibiyinka Temilola AYORINDE (PhD)
Department of Computer Science
University of Ibadan
Ibadan, Nigeria.
FORMAL LANGUAGES AND AUTOMATA THEORY
General Introduction and Course Objective
The objective of this course is to introduce undergraduate students to the theoretical aspects of
computer science, and helps them define infinite languages in finite ways, construct algorithms
for related problems and decide whether a string is in a language or not. In this course, students
will learn how mathematical models like the finite automaton and context-free grammar play
critical roles in several applied arears of computer science. While the finite automaton is used in
text processing, compilers and hardware design, the context-free grammar is used in
programming languages and artificial intelligence.

Table of Content
Study Session 1: Introduction
1.1 Formal Language
1.2 Words/Strings
1.3 Languages
Study Session 2: Regular Expressions (REs) and Regular Languages (RLs)
2.1 Regular Expressions - (REs)
2.2 Regular Languages - (RLs)
2.3 Building Regular Expressions
2.4 Algebraic Laws of RE
Study Session 3: Finite Automata (FA)
3.1 Finite Automata
3.2 Formal Definition of a Finite Automata
Study Session 4: Designing Finite Automata
4.1 Formulate the Idea to Use to Design Automaton
4.2 Design an Automaton
Study Session 5: Nondeterministic Finite Automata (NFA)
5.1 Basic Concept of NFA
5.2 Computation with NFA
5.3 Formal Definition of NFA
5.4 Epsilon NFA (ε-NFA)
 5.5 Equivalence of DFA and ε-NFA
Study Session 6: Regular Expressions (REs) Versus Finite Automata s(FA)
6.1 Definitions
6.2 Interchangeability of FA and RE
6.3 The Pumping Lemma for RLs
6.4 Closure Properties of RLs
Study Session 7: Decision Properties of RLs
7.1 Decidability Problems
7.2 Equivalence and Minimization of DFAs
Study Session 8: The Chomsky Hierarchy
8.1 Relationship Between the different Hierarchies
8.2 Closure Properties Under Union, Intersection and Complement
Study Session 9: Context Free Languages (CFLs) and Context Free Grammars (CFGs)
9.1 Overview
9.2 Creating A String From a CFG
9.3 Formal Definition of a CFG
9.4 Syntactic Expressions in Programming Languages
9.5 String Membership
9.6 Leftmost and Rightmost Derivation Styles
9.7 Parse Trees
9.8 Ambiguity in CFGs
9.9 Applications of CFGs
9.10 Closure Properties of CFL
Study Session 10: Normal Forms
10.1 Chomsky Normal Form (CNF)
10.2 Simplification of CFG
10.3 Algorithms to Convert CFG into CNF
10.4 Greibach Normal Form (GNF)
10.5 Pumping Lemma for CFG
sStudy Session 11: Pushdown Automata (PDA)
11.1 Basic Structure of PDA
11.2 Pushdown Automata Acceptance
11.3 PDA and CFG
Study Session 1: Introduction
Expected Duration: 1 Week

Introduction
This chapter discusses several important concepts including definition of a formal language, sets,
symbols, strings and languages.

Learning Outcomes for Study Session 1

At the end of this lecture, you should be able to:
1. Understand the essence of a formal language
2. Explain basic concepts like word, language, sets and operations of a language
3. Explain the relationship between the concepts in (2) above.

1.1 Formal Language

Formal Language and automata theory is about mathematical models of computation i.e. ways of
thinking about how theory can drive practice. Theory is relevant to practice in the following
ways:
1. Theory provides conceptual tools that practitioners use in computer engineering.
2. It shows a new, simpler and more elegant side of computers which are normally
considered to be complicated machines.
3. The study of theory expands the mind i.e. it gives you the ability to think, express
yourself clearly and precisely, to solve problems and to know when you haven’t solved a
problem.

1.2 Words/Strings
A word (or string) is a finite sequence of items, so called symbols or letters chosen from a
specified finite set called the alphabet. E.g. Common alphabets (a – z), interword space,
punctuation marks and the bits 0 and 1. A word of length one is identified with its only symbol.
A special word is the empty word or null word having no symbols, denoted by λ or ε. In this
course, we will choose the notation ε o represent empty word. The length of the word, w, is the
number of symbols in it, denoted by |w|. The length of the empty word is 0. If there are k
symbols in the alphabet, then there are kn words of length n. Thus there are
n
∑Ki = Kn+1 - 1
i=0 K-1
words of length at most n, if k > 1, and n + 1 words, if k = 1. The set of all words is denumerably
infinite, that is, they can be given as an infinite list, say, by ordering the words first according to
their length.

The basic operation of words is concatenation, that is, writing words as a compound. The
concatenation of the words w1 and w2 is denoted simply by w1w2. Examples of
concatenations in the alphabet {x, y, z} is shown below:
Let w1 = xxzyyx and, w2 = zxxz. Then, w1w2 = xxzyyxzxxz
Let w1 = xxzyyx and w2 = ε. Then, w1w2 = w1 = xxzyyx
Let w1 = ε and, w2 = zxxz. Then, w1w2 = w2 = zxxz
Concatenation is associative in nature i.e., w1(w2w3) = (w1w2)w3. As a con sequence of this,
repeated concatenations can be written without parentheses. On the other hand, concatenation is
usually not commutative, As a rule, w1w2 ≠ w2w1, but not always, and in the case of a unary
alphabet(i.e where we have a single word, like only w1), concatenation is obviously
commutative.
The nth (concatenation) power of the word w is
wn = ww…w (n copies)
Especially w1 = w and w0 = ε. Also, ε n = ε.

The mirror image (or reversal) of the word w = x1x2 · · · xn is the word
wR = xn · · · x2x1,
The reversal of ε , denoted as ε R = ε. Also, we have the reversal of (w1w2)R = (w2)R(w1)R.
A word u is a prefix of the word w, if w = uv.
A word u is a subword or segment of the word w, if w = v1uv2 for some words v1 and v2.
A word u is a scattered subword of the word w, if w = w1u1w2u2 · · ·wnunwn+1
where u = u1u2 · · · un, for some n and some words w1,w2, . . . ,wn+1 and u1u2 · · · un.
1.3 Languages
A language is a collection of sentences of finite length, all constructed from a finite alphabet of
symbols. It can also be defined as a set of sentences. A language is also a set of words over some
alphabets. Examples of languages are:
1. Finite Languages: These languages have only a finite number of words.
2. Cofinite Languages: These are languages that have only a finite number of words missing
3. Empty Language: This is denoted as ∅. It has no words.
4. A Singleton Language {w}: This is identified with its only word w, and the language is
denoted simply by w.
Set-theoretic notations are used for languages. Examples of such notations or symbols are:
1. ⊆ (inclusion or subset)
2. ⊂ (proper inclusion or proper subset)
3. ∪ (union)
4. ∩ (intersection)
5. − (difference) and
6. ′ or ‾ (complement against the set of all words over the alphabet).
Membership: Belonging of a word w in the language L is denoted by w ∈ L.

Note also the negated relations of the above symbols: ⊄, ⊈, ⊉ and

The language of all words over the alphabet ∑, in particular ε, is denoted by ∑∗. The
language of all nonempty words over the alphabet ∑ is denoted by ∑+. Hence, L′ = ∑∗ − L
and ∑+ = ∑∗ − {ε}.

In formal language theory, defining languages and investigating languages via their definitions is
paramount. There are many other operations of languages in addition to the set-theoretic ones
above. The concatenation of the languages L1 and L2 is given as:
L1L2 = {w1w2 | w1 ∈ L1 and w2 ∈ L2}.
The nth (concatenation) power of the language L is:
Ln = { w1w2 · · ·wn | w1, w2, . . . , wn ∈ L},
Note the following:
i. L1 = L
ii. L0 = {ε}.
iii. ∅0 = {ε}
iv. The concatenation closure or Kleenean star of the language L is
g

L* is the set obtained by concatenating words of L in all possible ways, including the empty
string ε. Similarly, L+ is the set obtained by concatenating words of L in all possible ways,
excluding the empty string. Hence, ∅∗ = {ε}, but ∅+ = ∅. Note that: L+ = L*L = LL*
Example 1.1
Let L = {1, 00}
L0 = {ε}
L1 = {1, 00}
L2 = {1, 00}{1 , 00} = {11,100,001,0000}
L3 = {1, 00}{11,100,001,0000}
= {111, 1100, 1001, 10000, 000000, 00001, 00100, 0011}
L* = L0 ∪ L1 ∪ L2 ∪ L3 ∪ L4 ∪…
L+ = L1 ∪ L2 ∪ L3 ∪ L4 ∪…

The left and right quotients of the languages L1 and L2 are defined as
L1\L2 = {w2 | w1w2 ∈ L2 for some word w1 ∈ L1}
(remove from words of L2 prefixes belonging in L1 in all possible ways) and
L1/L2 = {w1 | w1w2 ∈ L1 for some word w2 ∈ L2}
(remove from words of L1 suffixes belonging in L2 in all possible ways). Note that the
prefix or the suffix can be empty. The mirror image (or reversal) of the language L is the
language L′ = { w′ | w ∈ L}.

Fundamental machines used in defining languages are:

1. Grammars: This generates words of the language
2. Automata: This recognizes words of the language.
3. Regular expressions: This is used to define a language.

Summary of Study Session 1

This chapter has given the preliminaries that are needed in teaching formal language and the
automata theory. The concepts taught in this chapter form the basics on which other concepts
will be built. Hence, the student is advised to know them very well.

Self-Assessment Questions (SAQs) for Study Session 1

1. Define a word and an alphabet.
2. What is the relationship between a word and an alphabet.
3. Describe the three kinds of languages you know.
4. Mention three machines that can be used to define a language.
5. Given that w1 = dddccchh and w2 = uuuuoikj, what is the value of w1w2
6. Given that w1 = uv =dddccchh where v = chh, what is the value of the substring u?

References
1. Sipser M. (2006). Introduction to the Theory of Computation. Second Edition. Thomson
Course Technology. ISBN: 0-534-95097-3
2. Jaeger H. (2011). Formal Languages and Logics – Lecture Notes. Jacobs University
Bremen. Fall 2011
Study Session 2: REGULAR EXPRESSIONS (REs) AND REGULAR LANGUAGES
(RLs)
Expected Duration: 1 Week
Introduction
This chapter treats one of the machines that can be used to define languages. Students will learn
how regular expressions can be used to define regular languages. The knowledge of regular
expressions is essential as they are are useful tools to know because most programming
languages are written as plain text and REs are used in search engines, search and replace dialogs
of word processors and text editors, in text processing utilities like analysis.

Learning Outcomes for Study Session 2

At the end of this lecture, you should
1. Learn the basic concepts of regular expressions
2. Know how REs can be used to define a language.
3. Know the algebraic laws of regular expression.

2.1 Regular Expressions (REs)

A regular expression is a formula which defines a language using set-theoretical union, denoted
here by +, concatenation and concatenation closure.

These operations are combined according to a set of rules, adding parentheses ( and ) when
necessary. The atoms of the formula are symbols of the alphabet, the empty language ∅, the
empty word ε and the braces { } indicating sets. Example of the three language operators
described above is given as follows:
Example 2.1

The order of precedence of the language operators are:

1. * (Star Operatoron)
2. ∙ (Concatenation Operator)
3. ∪ (Union Operator)

2.2 Regular Languages (RLs)

Languages defined by regular expressions are called regular languages. Given that the family of
regular languages over the alphabet ∑ by R∑, or simply by R if the alphabet is clear by the
context. R is the family of languages satisfying the following conditions:
1. The language ∅ is in R and the corresponding regular expression is ∅.
2. The language {ε} is in R and the corresponding regular expression is ε.
3. For each symbol x, the language {x} is in R and the corresponding regular expression
is x.
4. If L1 and L2 are languages in R, and r1 and r2 are the corresponding regular expressions,
then
(a) the language L1∪ L2 is in R and the corresponding regular expression is (r1+r2).
(b) the language L1 L2 is in R and the corresponding regular expression is (r1r2).
5. If L is a language in R and r is the corresponding regular expression, then L* is in
R and the corresponding regular expression is (r∗).
6. Only languages obtainable by using the above rules 1.–5. are in R.
In order to avoid overly long expressions, certain customary abbreviations are used, e.g.
(rr) =denote (r2) , (r(rr)) =denote= (r3) and (r(r∗)) =denote (r+).
On the other hand, the rules produce fully parenthesized regular expressions. The order
of precedence is: ∗ , concatenation , +. This means that * has the highest priority, followed by
concatenation and finally, +.

2.3 Building Regular Expressions

Let E be a regular expression and the language represented by E is L(E). Then,
1. (E) = E
2. L(E + F) = L(E) ∪ L( F)
3. L(EF) = L(E)L( F)
4. L(E*) = (L(E))*

2.4 Algebraic Laws of Regular Expression

1. Commutative: E+ F = F + E
2. Associative: (E+ F) + G = E+ (F + G ), (E F)G = E(FG )
3. Idntity: E+∅ = E, εE = E ε = E
4. Annihilator: ∅E = E∅ = ∅
5. Distributive: E(F+G) = EF + EG, (F+G)E = FE + GE
6. Idempotent Law: E+E=E
7. Involving Kleen Closures:
a. (E*)* = E*
b. ∅* = ε
c. ε* = ε
d. E+ = EE*
e. E? = ε+E
f. E+ + ε = E*

Example 2.2
Kleen closure (the * operator) of a given language L:
1. L0 = {ε}
2. L1 = {w| w ∈ L}
3. L2 = {w1w2 | w1 ∈ L, w2 ∈ L} – duplicates allowed.
4. Li = {w1w2wi | all w’s chosen are ∈ L} – (duplicates allowed) where i refers
to how many strings to concatenate from the parent language L to
produce strings in the language Li. Note that the choice of each wi is
independent.
Example 2.3

Example 2.4
1. ((R*)*)* = R*
2. (R+S)* = R* + S*
3. (RS + R*)RS = RSRS +R*RS

Summary of Study Session 2

In this chapter, we have been able to see the basic concepts of REs and how they can be used to
build RLs. The laws guiding the concepts were also treated.

Self-Assessment Questions (SAQs) for Study 2

Question 1.
Question 2:

References
1. Jaeger H. (2011). Formal Languages and Logics – Lecture Notes. Jacobs University
Bremen. Fall 2011
2. Sipser M. (2006). Introduction to the Theory of Computation. Second Edition. Thomson
Course Technology. ISBN: 0-534-95097-3
3. softwareengineering.stackexchange.com › questions › is-it.
4. en.wikipedia.org › wiki › Regular_expression
Study Session 3: Finite Automata (FA)

Expected Duration: 1 Week

Introduction
Abstraction and modelling play a vital role in Computer Science. Computer programs are
designed to replace manual procedures. Hence, computer programs are models of these manual
procedures. Finite automata are good computational models for computer with an extremely
limited memory. One may wonder what such memories could do but they have been very useful
in the design of controller for the automatic door, the timer of a washing machine and the on and
off switches of bulbs among others. Automata are used to recognize the words/strings of a
language. An automaton processes a word and afterwards decides whether or not the word is in
the language.

Learning Outcomes for Study Session 3

1. Understand formal models in computer science
2. Understand the basic principles of an automaton
3. Know the formal definition of an automaton
4. Describe several finite state automata

3.1 Finite Automata

A finite automaton (FA) is also known as finite acceptor (FA) or finite-state machine (FSM) or
finite-state automaton (FSA). It accepts or rejects a given string of symbols, by running through
a state sequence uniquely determined by the string.

It is probably the simplest and most widely used computer science model,. Intuitively, a FA is a
system that can enter a finite set of different states. As a consequence of some input, which may
range over a finite setoff possible values, the FA operates a transition from one state to another.
A graph provides an immediate representation of a FA. Nodes denote states and edges denote
transitions. For example, the graph shown in Figure 3.1 represents an automaton that models the
behavior of a light switch.
`3.1.1 Modelling the Behavior of a light switch with State Transitions
As earlier on said, nodes denote states and edges denote transitions.

Figure 3.1: State Diagram of a Light Switch

The transition of the two sates (OFF and ON) with the two input signals (Turn knob right and
Turn knob left) is shown n Figure 3.2 below:

Input Signals
States Turn knob left Turn knob right
OFF ON OFF
ON ON OFF

Figure 3.2: State Transition Table for A Light Switch

While the Light is “OFF”, on turning the knob to the left, the state changes to “ON”. While it is
on and the knob is still turned left, the state still remains “ON”. Also, while the Light is “ON”,
and the knob is turned to the right, the state changes to “OFF”. While it is “OFF” and the knob is
still turned right, the state still remains “OFF”.

3.1.2 Modelling the Behavior of an Automatic Door Controller

An automatic door has a pad in front to detect the presence of a person about to walk through the
doorway. Another pad is located to the rear of the doorway so that the controller can hold the
door open long enough for the person to pass all the way through and also so that the door does
not strike someone standing behind it as it opens. The state diagram shown in Figure 3.3 and the
state transition shown in Figure 3.4 both show the operation of the automatic door controller as a
finite state automaton.
 Figure 3.3: State Diagram of an Automatic Door Controller

Figure 3.4: Transition Table of an Automatic Door Controller

The controller has two states: "OPEN" or "CLOSED," representing the corresponding condition
of the door. As shown Figures 3.3 and 3.4 respectively, there are four possible input conditions:
"FRONT" (meaning that a person is standing on the pad in front of the doorway), "REAR"
(meaning that a person is standing on the pad to the rear of the doorway), "BOTH" (meaning that
people are standing on both pads), and "NEITHER" (meaning that no one is standing on either
pad).

The controller moves from one state to another, depending on the input it receives. When in the
CLOSED state and receiving input NEITHER or REAR, it remains in the CLOSED state. In
addition, if the input BOTH is received, it stays CLOSED because opening the door risks
knocking someone over on the rear pad. But if the input FRONT arrives, it moves to the OPEN
state. In the OPEN state, if input FRONT, REAR, or BOTH is received, it remains in OPEN. If
input NEITHER arrives, it returns to CLOSED.

The automatic door controller as a finite automaton suggests standard ways of representation.
This controller is a computer that has just a single bit of memory, capable of recording which of
the two states the controller is in. Other common devices have controllers with somewhat larger
memories. For instance, in an elevator controller a state may represent the floor the elevator is on
and the inputs might be the signals received from the buttons. This computer might need several
bits to keep track of this information. Controllers for various household appliances such as
dishwashers and electronic thermostats, as well as parts of digital watches and calculators, are
additional examples of computers with limited memories. The design of such devices requires
keeping the methodology and terminology of finite automata in mind.

We will now take a closer look at finite automata from a mathematical perspective. We will
develop a precise definition of a finite automaton, terminology for describing and manipulating
finite automata, and theoretical results that describe their power and limitations. Besides giving
you a clearer understanding of what finite automata are and what they can and cannot do, this
theoretical development will allow you to practice and become more comfortable with
mathematical definitions, theorems, and proofs in a relatively simple setting.

3.1.3 A Finite Automaton with Three States.

In beginning to describe the mathematical theory of finite automata, we do so in the abstract,
without reference to any particular application. The following figure depicts a finite automaton
called T1

Figure 3.5: An Automaton with Tree States

Figure 1.4 is called the state diagram of T1. It has three states, labeled ql, q2, and q3. The start
state, q1, is indicated by the arrow pointing at it from nowhere. The accept state, q2, is the one
with a double circle. The arrows going from one state to another are called transitions.

When this automaton receives an input string such as 1101, it processes that string and produces
an output. The output is either accept or reject. We will consider only this yes/no type of output
for now to keep things simple. The processing begins in T1's start state. The automaton receives
the symbols from the input string one by one from left to right. After reading each symbol, T1
moves from one state to another along the transition that has that symbol as its label. When it
reads the last symbol, T1 produces its output. The output is accept if T1 is now in an accept state
and reject if it is not. For example, when we feed the input string 1101 to the machine M1 in
Figure 1.4, the processing proceeds as follows.
1. Start in state ql.
2. Read 1, follow transition from qi to q2.
3. Read 1, follow transition from q2 to q2.
4. Read 0, follow transition from q2 to q3.
5. Read 1, follow transition from q3 to q2-
6. Accept because Ml is in an accept state q2 at the end of the input.
Experimenting with this machine on a variety of input strings reveals that it
accepts the strings 1, 01, 1 1, 101, and 0101010101 among others. In fact, Ml accepts any string
that ends with a 1, as it goes to its accept state q2 whenever it reads the symbol 1. In addition, it
accepts strings 100, 0100, 1100, 110000, and 0101000000, and any string that ends with an even
number of 0s following the last 1. It rejects other strings, such as 0, 10, 101000. With this
understanding in mind, the language of all strings that T1 accepts can be described. This will be
shown in the next section where we will give a formal definition of the automaton T1.

3.2 Formal Definition of a Finite Automaton

Although state diagrams are easier to grasp intuitively, we need the formal definition, too, for
two specific reasons. First, a formal definition is precise. It resolves any uncertainties about what
is allowed in a finite automaton. If you were uncertain about whether finite automata were
allowed to have 0 accept states or whether they must have exactly one transition exiting every
state for each possible input symbol, you could consult the formal definition and verify that the
answer is yes in both cases. Second, a formal definition provides notation. Good notation helps
you think and express your thoughts clearly.

The formal definition says that a finite automaton is a list of five objects that are active in the
automaton: (i) set of states, (ii) input alphabet, (iii) rules for moving, (iv) start state, and (v)
accept states. In mathematical language a list of five elements is often called a 5-tuple. Hence we
define a finite automaton to be a 5-tuple consisting of these five parts. We use something called a
transition function, frequently denoted as δ, to define the rules for moving. If the finite
automaton has an arrow from a state x to a state y labeled with the input symbol 1, that means
that, if the automaton is in state x when it reads a 1, it then moves to state y. We can indicate the
same thing with the transition function by saying that δ(x, 1) = y. This notation is a kind of
mathematical shorthand. Putting it all together we arrive at the formal definition of finite
automata as follows:
A finite automaton is a 5-turple (Q, ∑, δ, q0, F) where
1. Q is a finite set called states
2. ∑ is a finite set called alphabet
3. δ: Q x ∑ → Q is the transition function
4. q0 ∈ Q is the start state and
5. F ⊆ Q is the set of accept states.
Now, we can use the above formal definition to define the finite automaton T1 shown in Figure
3.5which is also repeated below:

Figure 3.5 Repeated (Automaton T1)

The definition will now be:
T1 = (Q, ∑, δ, q0, F) where
1. Q = {q1, q2, q3}
2. ∑ = {0,1}
3. δ is described as:

0 1
q1 q1 q2
q2 q3 q2
q3 q2 q2
4. q1 is the start state and
5. F = {q2}

If A is the set of all strings that machine T accepts, we say that A is the language of machine T
and it is written as L(T) = A. We say that T recognizes A or that T accepts A. Because the term
accept has different meanings when we refer to machines accepting strings and machines
accepting languages, we prefer the term recognize for languages in order to avoid confusion. A
machine may accept several strings, but it always recognizes only one language. If the machine
accepts no strings, it still recognizes one language namely, the empty language ∅. In our
example, let
A = {w| w contains at least one 1 and an even number of 0s follow the last 1}.
Then, L(T1) = A, or equivalently, T1 recognizes A.

3.2.1 Example 3.1

Figure 3.6 shows a finite automaton T2. Give the definition of the automaton and describe its
language.

Figure 3.6: State Diagram of Two-State Automaton T2

Solution: The formal definition of T2 is give as follows:
T2 = (Q, ∑, δ, q0, F) where
1. Q = {q1, q2}
2. ∑ = {0,1}
3. δ is described as:

0 1
q1 q1 q2
q2 q1 q2
4. q1 is the start state and
5. F = {q1}
Looking critically at T2, the start state is also the accept state. Hence, it can accept the empty
string ε. In addition to the empty string, the machine accepts any string that ends with 0. Hence,
its language can be described as follows:
L(T2) = {w | w is the empty string ε or ends with 0}.

3.2.2 Example 3.
Figure 3.7 shows a five-state finite automaton T3. Give the definition of the automaton and
describe its language.

Figure 3.7: A State Diagram of a Five-Sate Finite Automaton

Solution: The formal definition of T2 is give as follows:

T3 = (Q, ∑, δ, q0, F) where
1. Q = {S, q1, q2, r1, r2}
2. ∑ = {a, b}
3. δ is described as:

a b
S q1 r1
q1 q1 q2
q2 q1 q2
r1 r2 r1
r2 r2 r1
4. S is the start state and
5. F = {q1, r1}
Machine T3 has two accept states, q1 and r1 , and operates over the alphabet = {a, b}. Some
experimentation shows that it accepts strings a, b, aa, bb, and bab, but not strings ab, ba, or bbba.
This machine begins in state S, and after it reads the first symbol in the input, it goes either left
into the q states or right into the r states. In both cases it can never return to the start state (in
contrast to the previous examples), as it has no way to get from any other state back to s. If the
first symbol in the input string is a, then it goes left and accepts when the string ends with an a.
Similarly, if the first symbol is a b, the machine goes right and accepts when the string ends in b.
So T3 accepts all strings that start and end with a or that start and end with b. In other words, T3
accepts strings that start and end with the same symbol.

Summary of Study Session 3

This session has explained what finite automata are and their importance to designs in the area of
computing. Also, the formal definition is also given. Hence, the student should be able to
formally define any given finite automaton and be able to define the language it recognizes. Note
that all the finite automata treated in this session are also called Deterministic Finite Automata
(DFA). They are called deterministic because the next state can easily be determined by the
transition δ.

Self-Assessment Questions (SAQs) for Study 3

1. Give the formal definition of the automaton F1 shown below and describe its language.

Automaton F1.
2. Answer the following questions in relation to the finite automaton F2 shown below:
 Automaton F2
a. What is the start state?
b. What is the set of accept states?
c. What sequences of states does the machine go through on input abb?
d. Does the machine accept aaabb?
e. Does the machine accept the string ε?
3. The formal description of a DFA, F3 is ({q1, q2, q3, q4, q5}, {c, d}, δ, q3, {q3}), where
δ is given by the following table. Give the state diagram of the machine.

c d
q1 q1 q2
q2 q1 q3
q3 q2 q4
q4 q3 q5
q5 q4 q5

References
1. Sipser M. (2006). Introduction to the Theory of Computation. Second Edition. Thomson
Course Technology. ISBN: 0-534-95097-3
2. Xavier S. P. U. (2005). Theory of Automata, Formal Languages and Computation.
Published by New Age International (P) Ltd., Publishers. ISBN (10): 81-224-2334-5.
ISBN (13): 978-81-224-2334-1
3. Mandrioli, D. and Ghezzi, C. (1987). Theoretical Foundations of Computer Science. John
Wiley and Sons, Inc.
4. Ayorinde I. T. (2014). Formal Languages and Automata Theory - Lecture Notes.
Study Session 4: Designing an Automaton
Expected Duration: 1 Week

Introduction
The design of an automaton is a creative process. Hence, it cannot be reduced to a simple recipe
or formula. However, you might find a particular approach helpful when designing various types
of automata. That is, put yourself in the place of the machine you are trying to design and then
see how you would go about performing the machine's task. Pretending that you are the machine
is a psychological trick that helps engage your whole mind in the design process.

Learning Outcomes for Study Session 4

At the end of this lecture, you should be able to:
1. Formulate the idea to use to design an automaton
2. Design an automaton

4.1 Formulate the idea to use to design an automaton

Let's design a finite automaton using the "reader as automaton" method. Suppose that you are
given a language and want to design a finite automaton that recognizes it. Pretending to be the
automaton, you receive an input string and must determine whether it is a member of the
language the automaton is supposed to recognize. You get to see the symbols in the string one by
one. After each symbol you must decide whether the string seen so far is in the language. The
reason is that you, like the machine, don't know when the end of the string is coming, so you
must always be ready with the answer.

4.2 Design an automaton

Let the alphabet be {0,1} and that the language consists of all strings with an odd number of 1s.
You want to construct a finite automaton T4 to recognize this language. Pretending to be the
automaton, you start getting an input string of 0s and 1s symbol by symbol. You don’t need to
remember the entire string seen so far in order to determine whether the number of 1s is odd.
Simply remember whether the number of 1s seen so far is even or odd and keep track of this
information as you read new symbols. If you read a 1, flip the answer, but if you read a 0, leave
the answer as it is. But how does this help you design T4? Once you have determined the
necessary information to remember about the string as it is being read, you represent this
information as a finite list of possibilities. In this instance, the possibilities would be:
1. even so far, and
2. odd so far.
Then you assign a state to each of the possibilities. These are the states of T4, as shown below in
Figure 4.1

Figure 4.1: The two states qeven and qodd

Next, you assign the transitions by seeing how to go from one possibility to another upon reading
a symbol. So, if state qeven represents the even possibility and state qodd represents the odd
possibility, you would set the transitions to flip state on a 1 and stay put on a 0, as shown in
Figure 4.2.

Figure 4.2: Transitions telling how the possibilities rearrange

Next, you set the start state to be the state corresponding to the possibility associated with having
seen 0 symbols so far (the empty string ε). In this case the start state corresponds to state even
because 0 is an even number. Last, set the accept states to be those corresponding to possibilities
where you want to accept the input string. Set qodd to be an accept state because you want to
accept when you have seen an odd number of 1s. These additions are shown in Figure 3.9.

Figure 4.3: Adding the start and accept states

Summary of Study Session 4
In this chapter, we've learnt how to formulate the ideas needed to design an automaton and how
to eventually design one.

Self-Assessment Questions (SAQs) for Study Session 4.

1. Each of the following languages is the intersection of two simpler languages. In each
part, construct deterministic finite automata (DFAs) for simpler languages, then combine
them to give the state diagram of a DFA for the language given. In all parts, ∑ = {a, b}.
a. {w | w has at least three a’s and at least two b’s}
b. {w | w has exactly two a’s and at least two b’s}
c. {w | w has an odd number of a’s and ends with ab}

References
1. Sipser M. (2006). Introduction to the Theory of Computation. Second Edition. Thomson
Course Technology. ISBN: 0-534-95097-3
2. Ayorinde I. T. (2014). Formal Languages and Automata Theory - Lecture Notes.
Study Session 5: Nondeterministic Finite Automata (NFA)
Expected Duration: 1 Week

Introduction
Unlike deterministic finite automata (DFA), where the next state is known or determined, the
next state is not totally known or determined in nondeterministic finite automata (NFA).
Nondeterminism is a generalization of determinism, so every deterministic finite automaton is
automatically a nondeterministic finite automaton.

Learning Outcomes for Study Session 5

At the end of this lecture, you should be able to:
1. Know the additional features of an NFA
2. Know the difference between DFA, NFA and ε-NFA
3. Know the formal definition of NFA
4. Convert NFA to DFA

5.1 Basic Concept of NFA

Every state of a DFA always has exactly one exiting transition arrow for each symbol in the
alphabet. But in considering the state diagram of a nondeterministic finite automaton (NFA),
shown in Figure 5.1 below, that rule is violated i.e. there are more than one existing transition for
each symbol in the alphabet. State q1 has one exiting arrow for 0, but it has two for 1; q2 has
one arrow for 0, but it has none for 1. In an NFA, a state may have zero, one, or many exiting
arrows for each alphabet symbol.

Figure 5.1: The nondeterministic finite automaton R1

Also, in a DFA, labels on the transition arrows are symbols from the alphabet. This NFA has an
arrow with the label ε (empty string). In general, an NFA may have arrows labeled with
members of the alphabet or ε. Zero, one, or many arrows may exit from each state with the label
ε.
5.2 Computation with NFA
Suppose that we are running an NFA on an input string and come to a state with multiple ways to
proceed. For example, say that we are in state q1 in NFA R1shown in Figure 5.1 and that the
next input symbol is a 1. After reading that symbol, the machine splits into multiple copies of
itself and follows all the possibilities in parallel. Each copy of the machine takes one of the
possible ways to proceed and continues as before. If there are subsequent choices, the machine
splits again. If the next input symbol doesn't appear on any of the arrows exiting the state
occupied by a copy of the machine, that copy of the machine dies, along with the branch of the
computation associated with it.

Finally, if any one of these copies of the machine is in an accept state at the end of the input, the
NFA accepts the input string. If a state with an ε symbol on an exiting arrow is encountered,
something similar happens. Without reading any input, the machine splits into multiple copies,
one following each of the exiting ε-labeled arrows and one staying at the current state. Then the
machine proceeds nondeterministically as before.

Nondeterministic computation can be viewed is as a tree of possibilities. The root of the tree
corresponds to the start of the computation. Every branching point in the tree corresponds to a
point in the computation at which the machine has multiple choices. The machine accepts if at
least one of the computation branches ends in an accept state, as shown in Figure 5.2.

Figure 5.2: Deterministic and nondeterministic computations with an accepting branch

Let's consider some sample runs of the NFA R1 shown in Figure 5.1. The computation of R1 on
input 010110 is depicted in Figure 5.3.

Figure 5.3: The computation of R1 on input 010110

On input 010110 start in the start state qi and read the first symbol 0. From qi there is only one
place to go on a 0-namely, back to q1, so remain there. Next read the second symbol 1. In q, on a
1 there are two choices: either stay in q, or move to q2. Nondeterministically, the machine splits
in two to follow each choice. Keep track of the possibilities by placing a finger on each state
where a machine could be. So you now have fingers on states q1 and q2. An ε arrow exits on
state q2 so the machine splits again; keep one finger on q2, and move the other to q3. You now
have fingers on q1, q2, and q3.
When the third symbol 0 is read, take each finger in turn. Keep the finger on q1 in place, move
the finger on q2 to q3, and remove the finger that has been on q3. That last finger had no 0 arrow
to follow and corresponds to a process that simply "dies." At this point you have fingers on states
q1 and q3. When the fourth symbol 1 is read, split the finger on q1 into fingers on states qi and
q2, then further split the finger on q2 to follow the ε arrow to q3, and move the finger that was
on q3 to q4. You now have a finger on each of the four states.

When the fifth symbol 1 is read, the fingers on q1 and q3 result in fingers on states q1, q2, q3,
and q4, as you saw with the fourth symbol. The finger on state q2 is removed. The finger that
was on q4 stays on q4. Now you have two fingers on q4, so remove one, because you only need
to remember that q4 is a possible state at this point, not that it is possible for multiple reasons.
When the sixth and final symbol 0 is read, keep the finger on qi in place, move the one on q2 to
q3, remove the one that was on q3, and leave the one on q4 in place. You are now at the end of
the string, and you accept if some finger is on an accept state. You have fingers on states q1, q3,
and q4, and as q4 is an accept state, R1 accepts this string.

What does R1 do on input 010? Start with a finger on q1. After reading the 0 you still have a
finger only on q1, but after the 1 there are fingers on q1, q2, and q3 (don't forget the ε arrow).
After the third symbol 0, remove the finger on q3, move the finger on q2 to q3, and leave the
finger on q1 where it is. At this point you are at the end of the input, and as no finger is on an
accept state, N, rejects this input. By continuing to experiment in this way, you will see that R1
accepts all strings that contain either 101 or 11 as a substring.

Nondeterministic finite automata are useful in several respects. As we will show, every NFA can
be converted into an equivalent DFA, and constructing NFAs is sometimes easier than directly
constructing DFAs. An NFA may be much smaller than its deterministic counterpart, or its
functioning may be easier to understand. Nondeterminism in finite automata is also a good
introduction to nondeterminism in more powerful computational models because finite automata
are especially easy to understand. Now we turn to several examples of NFAs.

5.3 Formal Definition of NFA

The formal definition of a nondeterministic finite automaton is similar to that of a deterministic
finite automaton. Both have states, an input alphabet, a transition function, a start state, and a
collection of accept states. However, they differ in their transition function. In a DFA, the
transition function takes a state and an input symbol and produces the next state. In an NFA the
transition function takes a state and an input symbol or the empty string and produces the set of
possible next states. In order to write the formal definition, we need to set up some additional
notation. For any set Q we write P(Q) to be the collection of all subsets of Q. Here P(Q) is called
the power set of Q. For any alphabet ∑, we write ∑ε to be ∑ ∪ {ε}. Now we can write the formal
description of the type of the transition function in an NFA as δ: Q x ∑ε → P(Q).
A nondeterministic finite automaton is a 5-turple (Q, ∑, δ, q0, F) where
1. Q is a finite set of states
2. ∑ is a finite set alphabet
3. δ: Q x ∑ε → P(Q) is the transition function
4. q0 ∈ Q is the start state and
5. F ⊆ Q is the set of accept states.

Consider the state diagram of the NFA (R1 Repeated) in Figure 5.4 below. The formal definition
is given thus:

Figure 5.4:
The definition will now be:
R1 = (Q, ∑, δ, q1, F) where
1. Q = {q1, q2, q3, q4}
2. ∑ = {0,1}
3. δ is described as:

5.4 Epsilon Nondeterministic Finite Automata (ε-NFA)

The NFA with epsilon-transition is a finite state machine in which the transition from one state to
another state is allowed without any input symbol i.e. empty string ε. It helps to easily show the
equivalences between DFA, NFA and regular expressions (REs). More of this will be studied in
subsequent chapters.

5.5 Equivalence of DFA and NFA

Every DFA has an equivalent NFA and every NFA has an equivalent ε-NFA. For example,
Let X = (Qx, ∑, δx, q0, Fx) be an NFA which accepts the language L(X). We have to design an
equivalent DFA Y = (Qy, ∑, δy, q0, Fy) such that L(Y) = L(X). The following procedure converts
the NDFA to its equivalent DFA −
Algorithm
Input − An NFA
Output − An equivalent DFA
Step 1 − Create state table from the given NFA.
Step 2 − Create a blank state table under possible input alphabets for the equivalent DFA.
Step 3 − Mark the start state of the DFA by q0 (Same as the NFA).
Step 4 − Find out the combination of States {Q0, Q1,... , Qn} for each possible input alphabet.
Step 5 − Each time we generate a new DFA state under the input alphabet columns, we have to
apply step 4 again, otherwise go to step 6.
Step 6 − The states which contain any of the final states of the NFA are the final states of the
equivalent DFA
Example
Let us consider the NFA shown in the figure below.
q δ(q,0) δ(q,1)

a {a,b,c,d,e} {d,e}

b {c} {e}

c ∅ {b}

d {e} ∅

e ∅ ∅

Using the above algorithm, we find its equivalent DFA. The state table of the DFA is shown in
below.

q δ(q,0) δ(q,1)

[a] [a,b,c,d,e] [d,e]

[a,b,c,d,e] [a,b,c,d,e] [b,d,e]

[d,e] [e] ∅

[b,d,e] [c,e] [e]

[e] ∅ ∅

[c, e] ∅ [b]

[b] [c] [e]

[c] ∅ [b]

The state diagram of the DFA is as follows −

Table 5.1 shows the difference between a DFA and a NFA

DFA NFA

The transition from a state is to a single particular The transition from a state can be to multiple
next state for each input symbol. Hence it is next states for each input symbol. Hence it is
called deterministic. called non-deterministic.

Empty string transitions are not seen in DFA. NFA permits empty string transitions.

Backtracking is allowed in DFA In NFA, backtracking is not always possible.

Requires more space. Requires less space.

A string is accepted by a DFA, if it transits to a A string is accepted by a NFA, if at least one

final state. of all possible transitions ends in a final state.
Summary of Study Session 5
In this session students have learnt the features of NFA and ε-NFA and the differences between
DFA, NFA and ε-NFA. Also, the equivalence of NFA and DFA was shown.

Self-Assessment Questions (SAQs) for Study Session 5.

1. Briefly describe the difference between the transition of a DFA and a NFA.
2. What is the significance of ε-NFA?
3. How can you convert a DFA to NFA and vice versa?

References
1. Sipser, M. (2006). Introduction to the Theory of Computation. Second Edition. Thomson
Course Technology. ISBN: 0-534-95097-3
2. Ayorinde, I. T. (2014). Formal Languages and Automata Theory - Lecture Notes.
3. www.quora.com › What-is-epsilon-NFA
4. https://www.javatpoint.com/automata-conversion-from-nfa-with-null-to-dfa
5.https://www.tutorialspoint.com/automata_theory/non_deterministic_finite_automaton.htm
Study Session 6: REGULAR EXPRESSIONS (REs) VERSUS FINITE AUTOMATA
(FA)
Expected Duration: 1 Week

Introduction
For any regular expression, one can always construct a corresponding automaton, and for
any finite automaton there is always a corresponding regular expression.

Learning Outcomes for Study Session 6

At the end of this lecture, you should be able to:
1. Know the relationship between regular expressions and finite automata.
2. Be able to convert regular expression to FA and vice versa.

6.1 Definitions
1. A language is regular if and only if some regular expressions describe it.
2. A language is regular if we are able to construct one of the following: DFA, NFA or
ε-NFA.
3. A language is not regular if we can show that no FA ca be built for it. This can be shown
through the pumping lemma (discussed in session 6.3).

Figure 6.1 shows the formal language classes.

Figure 6.1: Formal Language Classes

Note the following:
1. RE offers a declarative way to express the pattern of any string we want to accept. E. g.
01* + 10*
2. Automata is more machine like. <input: string, output: [accept/reject]>
3. REs are more program syntax like.
4. Unix environments heavily use REs e.g bash shell, grep, vi and other editors.
5. Perl scripting is good for string processing

6.2 Interchangeability of FA and RE

To show that they are interchangeable, consider the following theorems:
Theorem 1:
For every DFA A, there exists a regular expression R such that L(R) = L(A)
Theorem 2:
For every RE R, there exists an ε-NFA such that L(E) = L(R).

6.2.1 DFA to RE Construction

Figure 6.2: Theorem 1: Conversion of DFA to RE

Figure 6.2 illustrate theorem 1 i.e. the transition or conversion of a DFA to RE. Informally, trace
all distinct paths (traversing cycles only once) from the start state to each of the final states and
enumerate all the expressions along the way. Example is shown in Figure 6.3.

Figure 6.3: Conversion of a DFA to RE

What is the language of the above regular expression?
Note that the value of a RE is a language. Hence, if E = 1*00*1(1 + 0)*, then, the language of E
is L(E) = {w | w consists of all possible strings of 0s and 1s that stars with 1}

6.2.2 Conversion of RE to ε-NFA

Figure 6.4 shows theorem 2.

Figure 6.4: Theorem 2: Conversion of RE to ε-NFA

Example: To convert the regular expression (ab ∪ a)* to an NFA, we go through a sequence of
stages, from the smallest sub-expression to larger sub-expressions until we have an NFA for the
original expression as shown in Figure 6.5.

Figure 6.5: Building an NFA from the Regular Expression (ab ∪ a)*

6.3 The Pumping Lemma for Regular Languages

The pumping lemma is a technique that is used to show that a given language is not regular.
Theorem: For any regular language L there exists an integer n, such that for all x ∈ L with |x| ≥ n,
there exist u, v, w ∈ Σ* , such that:
(1) x = uvw
(2) |uv| ≤ n
(3) |v| ≥ 1
(4) for all i ≥ 0: uvi w ∈ L.
Note: The clause should hold for all regular languages. From the definition, n is called the
pumping lemma.
Proof:
Let M be a DFA for L. Take n be the number of states of M plus 1. Since M has at most n − 1
states, some state must be visited twice or more in the first n steps of the path. Take any x ∈ L
with |x| ≥ n. Consider the path (from start state to an accepting state) in M that corresponds to x.
The length of this path is |x| ≥ n.

Example 6.1
The language L = {0m1m | m ≥ 0} is not regular. Show this by the pumping lemma as follows:
Assume that L is regular. Let n be the "pumping lemma constant" assured by the pumping
lemma. Consider the word w = 0n1n. By the lemma, it can be split into w = xyz, with |xy| ≤ n, that
is, xy is made only from 0's. By the lemma, y is non-empty, say it is 0k, k > 0. Again by the
lemma, also 0n+k 1n ∈ L would follow, contradiction.

Example 6.2
Problem
Prove that L = {aibi | i ≥ 0} is not regular.
Solution
 At first, we assume that L is regular and n is the number of states.
 Let w = anbn. Thus |w| = 2n ≥ n.
 By pumping lemma, let w = xyz, where |xy| ≤ n.
 Let x = ap, y = aq, and z = arbn, where p + q + r = n, p ≠ 0, q ≠ 0, r ≠ 0. Thus |y| ≠ 0.
 Let k = 2. Then xy2z = apa2qarbn.
 Number of as = (p + 2q + r) = (p + q + r) + q = n + q
 Hence, xy2z = an+q bn. Since q ≠ 0, xy2z is not of the form anbn.
 Thus, xy2z is not in L. Hence L is not regular.
6.4 Closure Properties of Regular Languages (RLs)
If a set of regular languages are combined using an operator, then, the resulting language is also
regular. Note that this is different from Kleene closure. Hence, a regular language is closed under
the following operations:
1. Union, Intersection, Complement and Difference
2. Reversal
3. Kleene Closure
4. Concatenation
5. Homomorphism
6. Inverse Homomorphism

Summary of Study Session 6

In this session, the student has learnt the relationship between regular expressions and finite
automata. Also, conversion of regular expression to FA and vice versa was also discussed.

Self-Assessment Questions (SAQs) for Study Session 6

1. Give state diagrams of DFAs recognizing the following languages. In all parts, the
alphabet is {0,1}.
a. {w | w begins with a 1 and ends with a 0}
b. {w | w contains at least three 1s}
2. Convert the following regular expressions to ε-NFA:
a. (0 ∪ 1)*000(0 ∪ 1)*
b. (((00)*(11)) ∪ 01)*

References
1. Sipser, M. (2006). Introduction to the Theory of Computation. Second Edition. Thomson
Course Technology. ISBN: 0-534-95097-3
2. Ayorinde, I. T. (2014). Formal Languages and Automata Theory - Lecture Notes.
3. www.quora.com › What-is-epsilon-NFA
4. Jaeger H. (2011). Formal Languages and Logics – Lecture Notes. Jacobs University
Bremen. Fall 2011
Study Session 7: DECISION PROPERTIES OF REGULAR LANGUAGES
Expected Duration: 2 Weeks

Introduction
Given two deterministic finite automata A and B, it is decidable whether they accept the same
language. As a consequence, using the closure properties, the following problems are also
decidable for arbitrarily given deterministic finite automata A and B, with accepted
languages LA and LB, respectively: Emptiness problem, Membership problem, finiteness problem
and equality problem.
Learning Outcomes for Study Session 7
At the end of this lecture, you should be able to:
1. Know whether or not a given DFA is finite, empty, a member of another language or
equal to another one.
2. Be able to determine any of these properties in a given DFA or two DFAs.

7.1 Decidability Problems

Nearly every characterization problem is algorithmically decidable for regular languages.
The most common ones are the following (where L or L1 and L2 are given regular languages)

7.1.1 Membership Test

Decision Problem: Given L, is w in L?
Possible Answer: Yes or No
Approach:
1. Build a DFA for L
2. Input w to the DFA
3. If the DFA ends in an accepting state, then, yes; otherwise, no.

7.1.2 Emptiness Test

Decision Problem: Is L = ∅ ?
Approach: On a DFA for L:
1. From the start state, run a reachability test which returns
 a. Success if there is at least one final state that is reachable from the start state.
b. Failure if otherwise
2. L = ∅ if and only if the reachability test fails.

7.1.3 Finiteness Test

Decision Problem: Is L finite or infinite?
Approach: On a DFA for L:
1. Remove all states unreachable from the start state.
2. Remove all states that cannot lead to any accepting state.
3. After removal, check for cycles in the resulting FA
4. L is finite if there are no cycles. Otherwise, it is infinite.

7.2 Equivalence and Minimization of DFAs

7.2.1 Equivalence
Two states p and q are said to be equivalent iff:
1. Any string w accepted by starting at p is also accepted by starting at q.
2. Any string w rejected by starting at p is also rejected by starting at q.

7.2.2 Minimization
Steps to minimize DFAs:
1. Remove all states unreachable from the start state
2. Identify and condense equivalent states into one.
DFA Minimization using Myphill-Nerode Theorem
Algorithm
Input − DFA
Output − Minimized DFA
Step 1 − Draw a table for all pairs of states (Qi, Qj) not necessarily connected directly [All are
unmarked initially]
Step 2 − Consider every state pair (Qi, Qj) in the DFA where Qi ∈ F and Qj ∉ F or vice versa
and mark them. [Here F is the set of final states]
Step 3 − Repeat this step until we cannot mark anymore states −
If there is an unmarked pair (Qi, Qj), mark it if the pair {δ (Qi, A), δ (Qi, A)} is marked for some
input alphabet.
Step 4 − Combine all the unmarked pair (Qi, Qj) and make them a single state in the reduced
DFA.
Example
Let us use Algorithm 2 to minimize the DFA shown below.

Step 1 − We draw a table for all pair of states.

a b c d e f

Step 2 − We mark the state pairs.

a b c d e f
a

c ✔ ✔

d ✔ ✔

e ✔ ✔

f ✔ ✔ ✔

Step 3 − We will try to mark the state pairs, with green colored check mark, transitively. If we
input 1 to state ‘a’ and ‘f’, it will go to state ‘c’ and ‘f’ respectively. (c, f) is already marked,
hence we will mark pair (a, f). Now, we input 1 to state ‘b’ and ‘f’; it will go to state ‘d’ and ‘f’
respectively. (d, f) is already marked, hence we will mark pair (b, f).

a b c d e f

c ✔ ✔

d ✔ ✔

e ✔ ✔

f ✔ ✔ ✔ ✔ ✔

After step 3, we have got state combinations {a, b} {c, d} {c, e} {d, e} that are unmarked.
We can recombine {c, d} {c, e} {d, e} into {c, d, e}
Hence we got two combined states as − {a, b} and {c, d, e}
So the final minimized DFA will contain three states {f}, {a, b} and {c, d, e}
DFA Minimization using Equivalence Theorem
If X and Y are two states in a DFA, we can combine these two states into {X, Y} if they are not
distinguishable. Two states are distinguishable, if there is at least one string S, such that one of δ
(X, S) and δ (Y, S) is accepting and another is not accepting. Hence, a DFA is minimal if and
only if all the states are distinguishable.
Algorithm 3
Step 1 − All the states Q are divided in two partitions − final states and non-final states and
are denoted by P0. All the states in a partition are 0th equivalent. Take a counter k and initialize
it with 0.
Step 2 − Increment k by 1. For each partition in Pk, divide the states in Pk into two partitions if
they are k-distinguishable. Two states within this partition X and Y are k-distinguishable if
there is an input S such that δ(X, S) and δ(Y, S) are (k-1)-distinguishable.
Step 3 − If Pk ≠ Pk-1, repeat Step 2, otherwise go to Step 4.
Step 4 − Combine kth equivalent sets and make them the new states of the reduced DFA.
Example
Let us consider the following DFA −
q δ(q,0) δ(q,1)

a b c

b a d

c e f

d e f

e e f

f f f

Let us apply the above algorithm to the above DFA −

 P0 = {(c,d,e), (a,b,f)}
 P1 = {(c,d,e), (a,b),(f)}
 P2 = {(c,d,e), (a,b),(f)}
Hence, P1 = P2.
There are three states in the reduced DFA. The reduced DFA is as follows −
Q δ(q,0) δ(q,1)

(a, b) (a, b) (c,d,e)

(c,d,e) (c,d,e) (f)

(f) (f) (f)

Summary of Study Session 7

In this session, you have learnt about the decision properties of a finite automaton. Given an
automaton, you would by now be able to show whether it is finite, empty, equal to another one or
a string belongs to it.

Self-Assessment Questions (SAQs) for Study Session 7

1. With the aid of any given DFA of your choice, show whether it if finite or not.
2. With the aid of any given DFA of your choice and a string w, show whether or not the
string belongs to the DFA.

References
1. Ruohonen K. (2009). Formal Languages.
2. Jaeger H. (2011). Formal Languages and Logics – Lecture Notes. Jacobs University
Bremen. Fall 2011
3. Ayorinde, I. T. (2014). Formal Languages and Automata Theory - Lecture Notes.
Study Session 8: THE CHOMSKY HIERARCHY
Expected Duration: 1 Week

Introduction
In this session, you will learn about the Chomsky hierarch, which describes a hierarchy of
languages, ranging from the very simple regular languages up through the very general
recursively enumerable languages. The hierarchy of languages can be described as a hierarchy of
languages (sets), a hierarchy of grammars or a hierarchy of machines.

Learning Outcomes for Study Session 8

At the end of this lecture, you should be able to:
1. Know the different hierarchies we have.
2. Understand the relationship and the differences between the different hierarchies
3. Know the significance and application of each hierarchy.

8.1 Relationship Between The Different Hierarchies

The relationship between the various hierarchies is shown in Figure 8.1

Figure 8.1: Chomsky Hierarchy

Also, Table 8.1 shows the relationship between the Chomsky hierarchy in a tabular form and
their examples.
Table 8.1: Chomsky Hierarchy
Languages Grammars Machines
Recursively Enumerable Type 0 Turing Machines
Languages Unrestricted Grammars (Deterministic or
Nondeterministic TMs)
Context Sensitive Languages Type 1 Linear Bounded Automata
Context Sensitive Grammars (Nondeterministic)
Context Free Languages Type 2 Pushdown Automata
Context Free Grammars (Nondeterministic)
Regular Languages Type 3 Finite Automata
Right Linear Grammars (Deterministic or
Nondeterministic)

Regular languages are a subclass of the context-free languages, which are a subclass of the
context-sensitive languages, which are included in the recursive languages, which are a subclass
of the recursively enumerable languages. Figure 8.1 shows this clearly.

8.2 Closure Properties under Union, Intersection and Complement

Table 2 shows the closure properties of each of the languages under different operators like the
union, intersection and complement.
Table 2: Closure Properties of Languages
Union Intersection Complement
Recursively Yes Yes No
Enumerable
Languages (RELs)
Recursive Languages Yes Yes Yes
(RLs)
Context Sensitive Yes Yes Yes
Languages (CSLs)
Context Free Yes No No
Languages (CFLs)
Regular Languages Yes Yes Yes
(RLs)

Summary of Study Session 8

In this session, you have learnt about the chomsky hierarchy, which describes a hierarchy of
languages, ranging from the very simple regular languages up through the very general
recursively enumerable languages.

Self-Assessment Questions (SAQs) for Study Session 8

1. Give example(s) of Type 0 and Type 3 machines
2. What is the difference between regular languages (or grammars) and context free
languages (or grammars)

References
1. Sipser M. (2006). Introduction to the Theory of Computation. Second Edition. Thomson
Course Technology. ISBN: 0-534-95097-3
2. Tinkham N. L. (2010). The Chomsky Hierarch – Section 12.7
3. https://www.javatpoint.com/automata-chomsky-hierarchy
Study Session 9: Context Free Languages (CFLs) and Context Free Grammars (CFGs)
Expected Duration: 3 Weeks

Introduction
Many formal languages of practical interest are not regular especially, programming languages.
Regular languages cannot "count" within words; hence, any (programming) language that needs
some counting/bookkeeping, e.g. for matching parentheses, cannot be regular. Regular languages
can be generalized such that they include a "counting" mechanism, arriving at the context-free
languages (CFLs). CFLs can be described by a generalization of DFAs, pushdown automata.
Context free languages support natural, recursive notation called “Context Free Grammars”.

Learning Outcomes for Study Session 9

At the end of this lecture, you should be able to:
1. Recognize a context free language/grammar
2. Know the significance of context free grammar
3. Prove that a language is context free

9.1 Overview
Context-free grammars (CFGs) are used to describe context-free languages. A context-free
grammar is a set of recursive rules used to generate patterns of strings. A context-free grammar
can describe all regular languages and more, but they cannot describe all possible languages.
Context-free grammars are studied in fields of theoretical computer science, compiler design,
and linguistics. CFG’s are used to describe programming languages and parser programs
in compilers can be generated automatically from context-free grammars.
Context-free grammars can generate context-free languages. They do this by taking a set of
variables which are defined recursively, in terms of one another, by a set of production rules.
Context-free grammars are named as such because any of the production rules in the grammar
can be applied regardless of context—it does not depend on any other symbols that may or may
not be around a given symbol that is having a rule applied to it.

Context-free grammars have the following components:

1. A set of terminal symbols which are the characters that appear in the language/strings
generated by the grammar. Terminal symbols never appear on the left-hand side of the
production rule and are always on the right-hand side.
2. A set of nonterminal symbols (or variables) which are placeholders for patterns of
terminal symbols that can be generated by the nonterminal symbols. These are the
symbols that will always appear on the left-hand side of the production rules, though they
can be included on the right-hand side. The strings that a CFG produces will contain only
symbols from the set of nonterminal symbols.
3. A set of production rules which are the rules for replacing nonterminal symbols.
Production rules have the following form: variable \right arrow→ string of variables and
terminals.
4. A start symbol which is a special nonterminal symbol that appears in the initial string
generated by the grammar.

For comparison, a context-sensitive grammar can have production rules where both the left-
hand and right-hand sides may be surrounded by a context of terminal and nonterminal symbols.

9.2 Creating a String from a CFG

The following is an example of a context-free grammar, which we call G1.
A → OA1
A → B
B → #
A grammar consists of a collection of substitution rules, also called productions. Each rule
appears as a line in the grammar, comprising a symbol and a string separated by an arrow. The
symbol is called a variable. The string consists of variables and other symbols called terminals.
The variable symbols often are represented by capital letters. The terminals are analogous to the
input alphabet and often are represented by lowercase letters, numbers, or special symbols. One
variable is designated as the start variable. It usually occurs on the left-hand side of the topmost
rule. For example, grammar G1 contains three rules. G1 's variables are A and B, where A is the
start variable. Its terminals are 0, 1, and #.
You use a grammar to describe a language by generating each string of that language in the
following manner.
1. Write down the start variable. It is the variable on the left-hand side of the top rule, unless
specified otherwise.
2. Find a variable that is written down and a rule that starts with that variable. Replace the
written down variable with the right-hand side of that rule.
3. Repeat step 2 until no variables remains.

Example 9.1
For example, grammar G1 generates the string 000#111. The sequence of substitutions to obtain
a string is called a derivation. A derivation of string 000#111 in grammar GI is:
A → 0A1 X4 → 00A11 X → 000A111 → 00B111 → 000#111
You may also represent the same information pictorially with a parse tree. An example of a
parse tree is shown in Figure 9.1

Figure 9.1: Parse Tree for 000#111

All strings generated in this way constitute the language of the grammar. We write L(G1) for the
language of grammar G1 . Some experimentation with the grammar G1 shows that the language
of grammar G1, represented as L(G1 ) is {On#1n | n ≥ 0}. Any language that can be generated
by some context-free grammar is called a context-free language (CFL). For convenience when
presenting a context-free grammar, we abbreviate several rules with the same left-hand variable,
such as A → 0A1 and A → B, into a single line A → 0A1 | B, using the symbol " | " as an "or."
Example 9.2
The following is a another example of a context-free grammar, called G2 , which describes a
fragment of the English language.

Grammar G2 has 10 variables (the capitalized grammatical terms written inside brackets); 27
terminals (the standard English alphabet plus a space character); and 18 rules. Strings in L(G2)
include
a boy sees
the boy sees a flower
a girl with a flower likes the boy
Each of these strings has a derivation in grammar G2 . The following is a derivation
of the first string on this list.
9.3 Formal Definition of a Context-Free Grammar
A Grammar G = (S, N, T, P) where
S - Is the Start symbol
N - Is a set of Non-terminal symbols
T - Is a set of Terminal symbols
P - Is a set of Productions

Note that an input string belongs to a CFL (i.e. is accepted) iff it can be generated by the CFG.
Example 9.3
Let w = 01110
And G3 is given as:
A → 0A0 | 1A1 | 0 | 1 | ε
G3 can be generated as follows:
A → 0A0
→ 01A10
→ 01110

9.3.1 CFG Conventions

1. Terminal Symbols a, b, c, ….
2. Non-terminal symbols A, B, C,….
3. Terminal or non-terminal symbols X, Y, Z
4. Terminal strings w, x, y, z
5. Arbitrary strings of terminals and non-terminals α, β, γ, #, ….

9.4 Syntactic Expressions in Programming Languages

Given that:
Result = a * b + mark + 15 * width + c
Terminals are: a, *, b, +, +, 15, *, +, c
Variables are: Result, mark, width
Note that Operators are also terminals.
9.5 String Membership
String membership can be done in two ways:
(i). Derivation (Head to Body)
(ii). Recursive Inference (Body to Head)
For example, using the string generated for G3 in example 9.3, we have

Recursive Inference
A → 0A0
Derivation

→ 01A10
→ 01110

Figure 9.2: Top-Down and Bottom-Up Derivation

9.5.1 Generalization of Derivation

(i). Derivation is Head → Body
(ii). A→X means (A derives X in a single step)
(iii). A → *GX means (A derives X in multiple steps)
(iv). Transitivity
If A → *GB and B → *GC
Then A → *GC

9.5.2 Context Free Language (CFL) Definition

The language of a context free grammar (CFG) G = (S, N, T, P), denoted by L(G) is the set of
terminal strings that have a derivation from the start variable S.
E.g. L(G) = {W in T*| S → *GW}
9.6 Leftmost and Rightmost Derivation Styles
Given that: G:
E => E+E | E*E | (E) | F
F => aF | bF | 0F | 1F | ε
Derive the string a*(ab+10) from G:
i.e. E → *G a*(ab+10)
Solution:
For leftmost derivation, always substitute leftmost variable as shown in example 9.4 below:
Example 9.4: Leftmost Derivation of the string: a*(ab+10)
E
 E*E
 aF * E
 a*E
 a * (E)
 a * (E + E)
 a * (F + E)
 a * (aF + E)
 a * (abF + E)
 a * (ab + E)
 a * (ab + F)
 a * (ab + 1F)
 a * (ab + 10F)
 a * (ab + 10)

For rightmost derivation, always substitute rightmost variable as shown in example 9.5 below:
Example 9.5: Rightmost Derivation of the string: a*(ab+10)

E
 E*E
 E * (E)
 E * (E + E)
  E * (E + F)
 E * (E + 1F)
 E * (E + 10F)
 E * (E + 10)
 E * (F + 10)
 E * (aF + 10)
 E * (abF + 10)
 E * (ab + 10)
 F * (ab + 10)
 aF * (ab + 10)
 a * (ab + 10)

Note that for every leftmost derivation, there is a rightmost derivation. Every word generated by
a CFG has a leftmost and rightmost derivation. There can be words which have more than one
leftmost derivation (called ambiguity in CFG) or rightmost derivation, depending on the
grammar.

9.6.1 Generation of Derivation Tree

A derivation tree or parse tree is an ordered rooted tree that graphically represents the semantic
information a string derived from a context-free grammar.
Representation Technique
 Root vertex − Must be labeled by the start symbol.
 Vertex − Labeled by a non-terminal symbol.
 Leaves − Labeled by a terminal symbol or ε.
If S → x1x2 …… xn is a production rule in a CFG, then the parse tree / derivation tree will be as
follows −
There are two different approaches to draw a derivation tree −
Top-down Approach −
 Starts with the starting symbol S
 Goes down to tree leaves using productions
Bottom-up Approach −
 Starts from tree leaves
 Proceeds upward to the root which is the starting symbol S
Derivation or Yield of a Tree
The derivation or the yield of a parse tree is the final string obtained by concatenating the labels
of the leaves of the tree from left to right, ignoring the Nulls. However, if all the leaves are Null,
derivation is Null.
Example
Let a CFG {N,T,P,S} be
N = {S}, T = {a, b}, Starting symbol = S, P = S → SS | aSb | ε
One derivation from the above CFG is “abaabb”
S → SS → aSbS → abS → abaSb → abaaSbb → abaabb
Sentential Form and Partial Derivation Tree
A partial derivation tree is a sub-tree of a derivation tree/parse tree such that either all of its
children are in the sub-tree or none of them are in the sub-tree.
Example
If in any CFG the productions are −
S → AB, A → aaA | ε, B → Bb| ε
the partial derivation tree can be the following −
If a partial derivation tree contains the root S, it is called a sentential form. The above sub-tree
is also in sentential form.
Leftmost and Rightmost Derivation of a String
 Leftmost derivation − A leftmost derivation is obtained by applying production to the
leftmost variable in each step.
 Rightmost derivation − A rightmost derivation is obtained by applying production to
the rightmost variable in each step.
Example
Let any set of production rules in a CFG be
X → X+X | X*X |X| a
over an alphabet {a}.
The leftmost derivation for the string "a+a*a" may be −
X → X+X → a+X → a + X*X → a+a*X → a+a*a
The stepwise derivation of the above string is shown as below −
The rightmost derivation for the above string "a+a*a" may be −
X → X*X → X*a → X+X*a → X+a*a → a+a*a
The stepwise derivation of the above string is shown as below −
Left and Right Recursive Grammars

9.7 PARSE TREES

Each CFG can be represented using a parse tree.
1. Each internal node is labelled in by a variable in N.
2. Each leaf is a terminal symbol.
3. For a production, A => X1, X2,…Xk, any internal node labelled A has k children which
are labelled X1, X2,…Xk from left to right.
Parse tree for the above production is shown in Figure 9.3.
 Figure 9.3: Production of a grammar

Example 9.6: Parse Tree of a + b

Given that a grammar G has the following productions, draw the parse tree of a + b
E => E + E | E * E | (E) | F
F => aF | bF | 0F | 1F | 0 | 1 | a | b
The parse tree of a + b is shown in Figure 9.4.

Figure 9.4: Parse Tree of a + b

9.8 AMBIGUITY IN CFGs
Sometimes a grammar can generate the same string in several different ways. Such a string will
have several different parse trees and thus several different meanings. This result may be
undesirable for certain applications, such as programming languages, where a given program
should have a unique interpretation. If a grammar generates the same string in several different
ways, we say that the string is derived ambiguously in that grammar. If a grammar generates
some string ambiguously we say that the grammar is ambiguous.
For example, consider grammar G shown below:

This grammar generates the string a+ax a ambiguously. The following figure
shows the two different parse trees.

Figure 9.5: The two parse trees for the string a+axa in grammar G
This grammar doesn't capture the usual precedence relations and so may group the + before the x
or vice versa. In contrast grammar G4 generates exactly the same language, but every generated
string has a unique parse tree. Hence G4 is unambiguous, whereas G5 is ambiguous. Now we
formalize the notion of ambiguity. When we say that a grammar generates a string ambiguously,
we mean that the string has two different parse trees, not two different derivations. Two
derivations may differ merely in the order in which they replace variables yet not in their overall
structure. To concentrate on structure we define a type of derivation that replaces variables in a
fixed order. A derivation of a string w in a grammar G is a leftmost derivation if at every step the
leftmost remaining variable is the one replaced.

9.8.1 Definition of Ambiguity

A string w is derived ambiguously in context-free grammar G if
it has two or more different leftmost derivations. Grammar G is
ambiguous if it generates some string ambiguously.

9.8.2 Inherently Ambiguous Context Free Languages (CFLs)

Sometimes when we have an ambiguous grammar we can find an unambiguous grammar that
generates the same language. Some context-free languages, however, can be generated only by
ambiguous grammars. Such languages are called inherently ambiguous. In inherently ambiguous
CFLs, it may not be possible to remove ambiguity. Hence, a CFL is said to be inherently
ambiguous if every CFG that describes it is ambiguous.
Example:
L = {anbncmdm | n, m ≥ 1} ∪ {anbmcmdn | n, m ≥ 1}
L is inherently ambiguous. Input string: anbncndn

9.8.3 Removing Ambiguity in Context Free Languages (CFLs)

It may be possible to remove ambiguity for some CFLs. For example, In a CFG, for expression
evaluation, imposing rules and restrictions such as the introduction of operators precedence
could be done. This would imply rewriting the grammar.
Order of Precedence: ( ), *, +
An example of an ambiguous version is shown below:
E => E+ E | E * E | (E) | a | b | c | 0 | 1
Modified unambiguous version is shown below:
E => E + T | T
T => T * F | F
F => K | (E)
K => a | b | c | 0 | 1
In summary, ambiguity can be removed through the following steps:
1. Associativity of operator (e.g 9-5+2)
2. Precedence of operators
3. Use of separate rules.
9.8.4 Examples
Problem
Check whether the grammar G with production rules −
X → X+X | X*X |X| a
is ambiguous or not.
Solution
Let’s find out the derivation tree for the string "a+a*a". It has two leftmost derivations.
Derivation 1 − X → X+X → a +X → a+ X*X → a+a*X → a+a*a
Parse tree 1 −

Derivation 2 − X → X*X → X+X*X → a+ X*X → a+a*X → a+a*a

Parse tree 2 −
Since there are two parse trees for a single string "a+a*a", the grammar G is ambiguous.

9.9 Applications of CFGs

1. Context-free grammars can be used in programming languages – In block statements like
If-Then- Else statements or Do…While statements. It can also be used in recognizing
expressions like identifiers and constants.
2. Usage in compilers: One of the key steps in a compiler is figuring out what a program
“means.” This is usually done by defining a grammar showing the high-level structure of
a programming language. There are certain classes of grammars (LL(1) grammars, LR(1)
grammars, LALR(1) grammars, etc.) for which it's easy to figure out how a particular
string was derived. Tools like yacc or bison automatically generate parsers from these
grammars.
3. Usage in Natural Language Processing: By building context-free grammars for actual
languages and applying statistical inference, it's possible for a computer to recover the
likely meaning of a sentence. CFGs were first called phrase-structure grammars and were
introduced by Noam Chomsky in his seminal work Syntactic Structures. They were then
adapted for use in the context of programming languages, where they were called
BackusNaur forms.
9.10 Closure Property of CFL
Context-free languages are closed under −
 Union
 Concatenation
 Kleene Star operation
Union
Let L1 and L2 be two context free languages. Then L1 ∪ L2 is also context free.
Example
Let L1 = { anbn , n > 0}. Corresponding grammar G1 will have P: S1 → aAb|ab
Let L2 = { cmdm , m ≥ 0}. Corresponding grammar G2 will have P: S2 → cBb| ε
Union of L1 and L2, L = L1 ∪ L2 = { anbn } ∪ { cmdm }
The corresponding grammar G will have the additional production S → S1 | S2
Concatenation
If L1 and L2 are context free languages, then L1L2 is also context free.
Example
Union of the languages L1 and L2, L = L1L2 = { anbncmdm }
The corresponding grammar G will have the additional production S → S1 S2
Kleene Star
If L is a context free language, then L* is also context free.
Example
Let L = { anbn , n ≥ 0}. Corresponding grammar G will have P: S → aAb| ε
Kleene Star L1 = { anbn }*
The corresponding grammar G1 will have additional productions S1 → SS1 | ε
Context-free languages are not closed under −
 Intersection − If L1 and L2 are context free languages, then L1 ∩ L2 is not necessarily
context free.
 Intersection with Regular Language − If L1 is a regular language and L2 is a context
free language, then L1 ∩ L2 is a context free language.
 Complement − If L1 is a context free language, then L1’ may not be context free.
Summary of Study Session 9
In this session the student must have learnt what is meant by context free grammar and context
free language and the relationship between the two. Also, formal definition of CFG, derivations,
parse trees and ambiguity of CFLs were also learnt among others.

Self-Assessment Questions (SAQs) for Study Session 9

1. List the 4 components of a context free grammar
2. Describe the relationship between terminals, non-terminals, and productions
3. Define ambiguity
4. Describe the language accepted by the following grammar:
a. S => abS | a
b. S => aSb | ε
5. Write a grammar for each of the following:
(i). a x b y , where x = 2y S
(ii). a x b y a z , where z = x+y
6. For the following grammar, list all parse trees for the string “true and true or true”
S => S and S | S or S | true

References
1. Jaeger H. (2011). Formal Languages and Logics – Lecture Notes. Jacobs University
Bremen. Fall 2011
2. Moore K., Chumbley A. and Khim J. (). Ontext Free Grammars. Retrieved September 1,
2020 from https://brilliant.org/wiki/context-free-grammars/
3. Nelson, R. Context-Free Grammars. Retrieved June 11, 2016,
from https://www.cs.rochester.edu/~nelson/courses/csc_173/grammars/cfg.html
4. , J. Leftmostderivations. Retrieved June 14, 2016,
from https://en.wikipedia.org/wiki/File:Leftmostderivations_jaredwf.png
5. Sipser M. (2006). Introduction to the Theory of Computation. Second Edition. Thomson
Course Technology. ISBN: 0-534-95097-3
6. https://www.tutorialspoint.com/automata_theory/
Study Session 10: NORMAL FORMS
Expected Duration: 1 Week

Introduction
If all productions of the grammar could be expressed in the same form(s), then it becomes easier
to design algorithms that use the grammar. It also becomes easy to show proofs and properties.
Hence, expressing grammars in this form is called normal forms. Different kinds of normal
forms like Chomsky normal form and Greibach normal form will be treated in this session.

Learning Outcomes for Study Session 10

At the end of this lecture, you should be able to:
1. Give a formal definition of Chomsky normal form (CNF)
2. Give a formal definition of Greibach normal form (GNF)
3. Convert a grammar to CNF
4. Prove whether a grammar is is CNF of GNF

10.1 Chomsky normal form (CNF)

Let G be a CFG for some L – {ε}
Definition:
G is said to be in Chomsky normal form (CNF) if all its productions are in one of the following
two forms:
1. A => BC where A, B, C are variables OR
2. A => a where a is a terminal.
From the above, the following conclusions can be made:
1. G has no useless symbols
2. G has no unit symbol
3. G has no ε – productions.

10.2 Simplification of CFG

In a CFG, it may happen that all the production rules and symbols are not needed for the
derivation of strings. Besides, there may be some null productions and unit productions.
Elimination of these productions and symbols is called simplification of CFGs. Simplification
essentially comprises of the following steps −
 Reduction of CFG
 Removal of Unit Productions
 Removal of Null Productions
Reduction of CFG
CFGs are reduced in two phases −
Phase 1 − Derivation of an equivalent grammar, G’, from the CFG, G, such that each variable
derives some terminal string.
Derivation Procedure −
Step 1 − Include all symbols, W1, that derive some terminal and initialize i=1.
Step 2 − Include all symbols, Wi+1, that derive Wi.
Step 3 − Increment i and repeat Step 2, until Wi+1 = Wi.
Step 4 − Include all production rules that have Wi in it.
Phase 2 − Derivation of an equivalent grammar, G”, from the CFG, G’, such that each symbol
appears in a sentential form.
Derivation Procedure −
Step 1 − Include the start symbol in Y1 and initialize i = 1.
Step 2 − Include all symbols, Yi+1, that can be derived from Yi and include all production rules
that have been applied.
Step 3 − Increment i and repeat Step 2, until Yi+1 = Yi.
Problem
Find a reduced grammar equivalent to the grammar G, having production rules, P: S → AC | B,
A → a, C → c | BC, E → aA | e
Solution
Phase 1 −
T = { a, c, e }
W1 = { A, C, E } from rules A → a, C → c and E → aA
W2 = { A, C, E } U { S } from rule S → AC
W3 = { A, C, E, S } U ∅
Since W2 = W3, we can derive G’ as −
G’ = { { A, C, E, S }, { a, c, e }, P, {S}}
where P: S → AC, A → a, C → c , E → aA | e
Phase 2 −
Y1 = { S }
Y2 = { S, A, C } from rule S → AC
Y3 = { S, A, C, a, c } from rules A → a and C → c
Y4 = { S, A, C, a, c }
Since Y3 = Y4, we can derive G” as −
G” = { { A, C, S }, { a, c }, P, {S}}
where P: S → AC, A → a, C → c
Removal of Unit Productions
Any production rule in the form A → B where A, B ∈ Non-terminal is called unit production..
Removal Procedure −
Step 1 − To remove A → B, add production A → x to the grammar rule whenever B →
x occurs in the grammar. [x ∈ Terminal, x can be Null]
Step 2 − Delete A → B from the grammar.
Step 3 − Repeat from step 1 until all unit productions are removed.
Problem
Remove unit production from the following −
S → XY, X → a, Y → Z | b, Z → M, M → N, N → a
Solution −
There are 3 unit productions in the grammar −
Y → Z, Z → M, and M → N
At first, we will remove M → N.
As N → a, we add M → a, and M → N is removed.
The production set becomes
S → XY, X → a, Y → Z | b, Z → M, M → a, N → a
Now we will remove Z → M.
As M → a, we add Z→ a, and Z → M is removed.
The production set becomes
S → XY, X → a, Y → Z | b, Z → a, M → a, N → a
Now we will remove Y → Z.
As Z → a, we add Y→ a, and Y → Z is removed.
The production set becomes
S → XY, X → a, Y → a | b, Z → a, M → a, N → a
Now Z, M, and N are unreachable, hence we can remove those.
The final CFG is unit production free −
S → XY, X → a, Y → a | b
Removal of Null Productions
In a CFG, a non-terminal symbol ‘A’ is a nullable variable if there is a production A → ε or
there is a derivation that starts at A and finally ends up with
ε: A → .......… → ε
Removal Procedure
Step 1 − Find out nullable non-terminal variables which derive ε.
Step 2 − For each production A → a, construct all productions A → x where x is obtained
from ‘a’ by removing one or multiple non-terminals from Step 1.
Step 3 − Combine the original productions with the result of step 2 and remove ε -
productions.
Problem
Remove null production from the following −
S → ASA | aB | b, A → B, B → b | ∈
Solution −
There are two nullable variables − A and B
At first, we will remove B → ε.
After removing B → ε, the production set becomes −
S→ASA | aB | b | a, A ε B| b | &epsilon, B → b
Now we will remove A → ε.
After removing A → ε, the production set becomes −
S→ASA | aB | b | a | SA | AS | S, A → B| b, B → b
This is the final production set without null transition.
10.3 Algorithm to Convert CFG into Chomsky Normal Form
Step 1 − If the start symbol S occurs on some right side, create a new start symbol S’ and a new
production S’→ S.
Step 2 − Remove Null productions. (Using the Null production removal algorithm discussed
earlier)
Step 3 − Remove unit productions. (Using the Unit production removal algorithm discussed
earlier)
Step 4 − Replace each production A → B1…Bn where n > 2 with A → B1C where C →
B2 …Bn. Repeat this step for all productions having two or more symbols in the right side.
Step 5 − If the right side of any production is in the form A → aB where a is a terminal and A,
B are non-terminal, then the production is replaced by A → XB and X → a. Repeat this step for
every production which is in the form A → aB.
Problem
Convert the following CFG into CNF
S → ASA | aB, A → B | S, B → b | ε
Solution
(1) Since S appears in R.H.S, we add a new state S0 and S0→S is added to the production set
and it becomes −
S0→S, S→ ASA | aB, A → B | S, B → b | ∈
(2) Now we will remove the null productions −
B → ∈ and A → ∈
After removing B → ε, the production set becomes −
S0→S, S→ ASA | aB | a, A → B | S | ∈, B → b
After removing A → ∈, the production set becomes −
S0→S, S→ ASA | aB | a | AS | SA | S, A → B | S, B → b
(3) Now we will remove the unit productions.
After removing S → S, the production set becomes −
S0→S, S→ ASA | aB | a | AS | SA, A → B | S, B → b
After removing S0→ S, the production set becomes −
S0→ ASA | aB | a | AS | SA, S→ ASA | aB | a | AS | SA
A → B | S, B → b
After removing A→ B, the production set becomes −
S0 → ASA | aB | a | AS | SA, S→ ASA | aB | a | AS | SA
A→S|b
B→b
After removing A→ S, the production set becomes −
S0 → ASA | aB | a | AS | SA, S→ ASA | aB | a | AS | SA
A → b |ASA | aB | a | AS | SA, B → b
(4) Now we will find out more than two variables in the R.H.S
Here, S0→ ASA, S → ASA, A→ ASA violates two Non-terminals in R.H.S.
Hence we will apply step 4 and step 5 to get the following final production set which is in CNF
−
S0→ AX | aB | a | AS | SA
S→ AX | aB | a | AS | SA
A → b |AX | aB | a | AS | SA
B→b
X → SA
(5) We have to change the productions S0→ aB, S→ aB, A→ aB
And the final production set becomes −
S0→ AX | YB | a | AS | SA
S→ AX | YB | a | AS | SA
A → b A → b |AX | YB | a | AS | SA
B→b
X → SA
Y→a

10.4 Greibach normal form (GNF)

A CFG is in Greibach Normal Form if the Productions are in the following forms −
A→b
A → bD1…Dn
S→ε
where A, D1,....,Dn are non-terminals and b is a terminal.
Algorithm to Convert a CFG into Greibach Normal Form
Step 1 − If the start symbol S occurs on some right side, create a new start symbol S’ and a new
production S’ → S.
Step 2 − Remove Null productions. (Using the Null production removal algorithm discussed
earlier)
Step 3 − Remove unit productions. (Using the Unit production removal algorithm discussed
earlier)
Step 4 − Remove all direct and indirect left-recursion.
Step 5 − Do proper substitutions of productions to convert it into the proper form of GNF.
Problem
Convert the following CFG into CNF
S → XY | Xn | p
X → mX | m
Y → Xn | o
Solution
Here, S does not appear on the right side of any production and there are no unit or null
productions in the production rule set. So, we can skip Step 1 to Step 3.
Step 4
Now after replacing
X in S → XY | Xo | p
with
mX | m
we obtain
S → mXY | mY | mXo | mo | p.
And after replacing
X in Y → Xn | o
with the right side of
X → mX | m
we obtain
Y → mXn | mn | o.
Two new productions O → o and P → p are added to the production set and then we came to
the final GNF as the following −
S → mXY | mY | mXC | mC | p
X → mX | m
Y → mXD | mD | o
O→o
P→p

10.5 Pumping Lemma for CFG

Lemma
If L is a context-free language, there is a pumping length p such that any string w ∈ L of
length ≥ p can be written as w = uvxyz, where vy ≠ ε, |vxy| ≤ p, and for all i ≥ 0, uvixyiz ∈ L.
Applications of Pumping Lemma
Pumping lemma is used to check whether a grammar is context free or not. Let us take an
example and show how it is checked.
Problem
Find out whether the language L = {xnynzn | n ≥ 1} is context free or not.
Solution
Let L is context free. Then, L must satisfy pumping lemma.
At first, choose a number n of the pumping lemma. Then, take z as 0n1n2n.
Break z into uvwxy, where
|vwx| ≤ n and vx ≠ ε.
Hence vwx cannot involve both 0s and 2s, since the last 0 and the first 2 are at least (n+1)
positions apart. There are two cases −
Case 1 − vwx has no 2s. Then vx has only 0s and 1s. Then uwy, which would have to be in L,
has n 2s, but fewer than n 0s or 1s.
Case 2 − vwx has no 0s.
Here contradiction occurs.
Hence, L is not a context-free language.
Summary of Study Session 10
In this session, you have learnt the importance of, writing context free languages in normal CNF
and GNF.

Self-Assessment Questions (SAQs) for Study Session 10

1. How can you recognize a language in CNF
2. How can you recognize a language in GNF
3. How can you eliminate empty productions?
4. How can you eliminate unit productions?
5. Given a context free language with the following productions:
S → aSa | bSb | aT b | bT a
T → aT a | aT b | bT a | bT b | ε | a | b.
Generate two nonpalindromes over {a,b}.

References
1. https://www.tutorialspoint.com/automata_theory/
Study Session 11: PUSHDOWN AUTOMATA (PDA)
Expected Duration: 1 Week

Introduction
Pushdown Automata is a finite automata with extra memory called stack which helps Pushdown
automata to recognize Context Free Languages. Pushdown automata are used in theories about
what can be computed by machines. They are more capable than finite-state machines but less
capable than Turing machines. Deterministic pushdown automata can recognize all deterministic
context-free languages while nondeterministic ones can recognize all context-free languages,
with the former often used in parser design. The term "pushdown" refers to the fact that
the stack can be regarded as being "pushed down" like a tray dispenser at a cafeteria, since the
operations never work on elements other than the top element

Learning Outcomes for Study Session 11

At the end of this lecture, you should be able to:
1. Know the formal definition of a pushdown automaton (PDA)
2. Understand how PDAs work
3. Know the difference between PDA, CFG and Regular languages

11.1 Basic Structure of PDA

A pushdown automaton is a way to implement a context-free grammar in a similar way we
design DFA for a regular grammar. A DFA can remember a finite amount of information, but a
PDA can remember an infinite amount of information.
Basically a pushdown automaton is −
"Finite state machine" + "a stack"
A pushdown automaton has three components −
 an input tape,
 a control unit, and
 a stack with infinite size.
The stack head scans the top symbol of the stack.
A stack does two operations −
  Push − a new symbol is added at the top.
 Pop − the top symbol is read and removed.
A PDA may or may not read an input symbol, but it has to read the top of the stack in every
transition.

A PDA can be formally described as a 7-tuple (Q, ∑, S, δ, q0, I, F) −

 Q is the finite number of states
 ∑ is input alphabet
 S is stack symbols
 δ is the transition function: Q × (∑ ∪ {ε}) × S × Q × S*
 q0 is the initial state (q0 ∈ Q)
 I is the initial stack top symbol (I ∈ S)
 F is a set of accepting states (F ∈ Q)
The following diagram shows a transition in a PDA from a state q1 to state q2, labeled as a,b →
c−
This means at state q1, if we encounter an input string ‘a’ and top symbol of the stack is ‘b’,
then we pop ‘b’, push ‘c’ on top of the stack and move to state q2.
11.1.1 Terminologies Related to PDA
Instantaneous Description
The instantaneous description (ID) of a PDA is represented by a triplet (q, w, s) where
 q is the state
 w is unconsumed input
 s is the stack contents
Turnstile Notation
The "turnstile" notation is used for connecting pairs of ID's that represent one or many moves of
a PDA. The process of transition is denoted by the turnstile symbol "⊢".
Consider a PDA (Q, ∑, S, δ, q0, I, F). A transition can be mathematically represented by the
following turnstile notation −
(p, aw, Tβ) ⊢ (q, w, αb)
This implies that while taking a transition from state p to state q, the input symbol ‘a’ is
consumed, and the top of the stack ‘T’ is replaced by a new string ‘α’.
Note − If we want zero or more moves of a PDA, we have to use the symbol (⊢*) for it.

11.2 Pushdown Automata Acceptance

There are two different ways to define PDA acceptability.
Final State Acceptability
In final state acceptability, a PDA accepts a string when, after reading the entire string, the PDA
is in a final state. From the starting state, we can make moves that end up in a final state with
any stack values. The stack values are irrelevant as long as we end up in a final state.
For a PDA (Q, ∑, S, δ, q0, I, F), the language accepted by the set of final states F is −
L(PDA) = {w | (q0, w, I) ⊢* (q, ε, x), q ∈ F}
for any input stack string x.
Empty Stack Acceptability
Here a PDA accepts a string when, after reading the entire string, the PDA has emptied its stack.
For a PDA (Q, ∑, S, δ, q0, I, F), the language accepted by the empty stack is −
L(PDA) = {w | (q0, w, I) ⊢* (q, ε, ε), q ∈ Q}
Example
Construct a PDA that accepts L = {0n 1n | n ≥ 0}
Solution

This language accepts L = {ε, 01, 0011, 000111, ............................. }

Here, in this example, the number of ‘a’ and ‘b’ have to be same.
 Initially we put a special symbol ‘$’ into the empty stack.
 Then at state q2, if we encounter input 0 and top is Null, we push 0 into stack. This may
iterate. And if we encounter input 1 and top is 0, we pop this 0.
 Then at state q3, if we encounter input 1 and top is 0, we pop this 0. This may also
iterate. And if we encounter input 1 and top is 0, we pop the top element.
 If the special symbol ‘$’ is encountered at top of the stack, it is popped out and it finally
goes to the accepting state q4.
Example
Construct a PDA that accepts L = { wwR | w = (a+b)* }
Solution
Initially we put a special symbol ‘$’ into the empty stack. At state q2, the w is being read. In
state q3, each 0 or 1 is popped when it matches the input. If any other input is given, the PDA
will go to a dead state. When we reach that special symbol ‘$’, we go to the accepting state q4.

11.3 PDA and Context Free Grammar

If a grammar G is context-free, we can build an equivalent nondeterministic PDA which accepts
the language that is produced by the context-free grammar G. A parser can be built for the
grammar G.
Also, if P is a pushdown automaton, an equivalent context-free grammar G can be constructed
where
L(G) = L(P)
In the next two topics, we will discuss how to convert from PDA to CFG and vice versa.
Algorithm to find PDA corresponding to a given CFG
Input − A CFG, G = (V, T, P, S)
Output − Equivalent PDA, P = (Q, ∑, S, δ, q0, I, F)
Step 1 − Convert the productions of the CFG into GNF.
Step 2 − The PDA will have only one state {q}.
Step 3 − The start symbol of CFG will be the start symbol in the PDA.
Step 4 − All non-terminals of the CFG will be the stack symbols of the PDA and all the
terminals of the CFG will be the input symbols of the PDA.
Step 5 − For each production in the form A → aX where a is terminal and A, X are
combination of terminal and non-terminals, make a transition δ (q, a, A).
Problem
Construct a PDA from the following CFG.
G = ({S, X}, {a, b}, P, S)
where the productions are −
S → XS | ε , A → aXb | Ab | ab
Solution
Let the equivalent PDA,
P = ({q}, {a, b}, {a, b, X, S}, δ, q, S)
where δ −
δ(q, ε , S) = {(q, XS), (q, ε )}
δ(q, ε , X) = {(q, aXb), (q, Xb), (q, ab)}
δ(q, a, a) = {(q, ε )}
δ(q, 1, 1) = {(q, ε )}
Algorithm to find CFG corresponding to a given PDA
Input − A CFG, G = (V, T, P, S)
Output − Equivalent PDA, P = (Q, ∑, S, δ, q0, I, F) such that the non- terminals of the grammar
G will be {Xwx | w,x ∈ Q} and the start state will be Aq0,F.
Step 1 − For every w, x, y, z ∈ Q, m ∈ S and a, b ∈ ∑, if δ (w, a, ε) contains (y, m) and (z, b, m)
contains (x, ε), add the production rule Xwx → a Xyzb in grammar G.
Step 2 − For every w, x, y, z ∈ Q, add the production rule Xwx → XwyXyx in grammar G.
Step 3 − For w ∈ Q, add the production rule Xww → ε in grammar G

Example 11.1
Construct a PDA for language L = {0n1m2m3n | n>=1, m>=1}
Approach used in this PDA –
First 0’s are pushed into stack. Then 1’s are pushed into stack.
Then for every 2 as input a 1 is popped out of stack. If some 2’s are still left and top of stack is a 0
then string is not accepted by the PDA. Thereafter if 2’s are finished and top of stack is a 0 then for
every 3 as input equal number of 0’s are popped out of stack. If string is finished and stack is
empty then string is accepted by the PDA otherwise not accepted.
 Step-1: On receiving 0 push it onto stack. On receiving 1, push it onto stack and goto next
state
  Step-2: On receiving 1 push it onto stack. On receiving 2, pop 1 from stack and goto next
state
 Step-3: On receiving 2 pop 1 from stack. If all the 1’s have been popped out of stack and
now receive 3 then pop a 0 from stack and goto next state
 Step-4: On receiving 3 pop 0 from stack. If input is finished and stack is empty then goto
last state and string is accepted

Examples:
Input : 0 0 1 1 1 2 2 2 3 3
Result : ACCEPTED

Input : 0 0 0 1 1 2 2 2 3 3
Result : NOT ACCEPTED

Problem:
Construct a PDA for language L = {0n1m | n >= 1, m >= 1, m > n+2}
Approach used in this PDA –
First 0’s are pushed into stack.When 0’s are finished, two 1’s are ignored. Thereafter for every 1 as
input a 0 is popped out of stack. When stack is empty and still some 1’s are left then all of them are
ignored.
  Step-1: On receiving 0 push it onto stack. On receiving 1, ignore it and goto next state
 Step-2: On receiving 1, ignore it and goto next state
 Step-3: On receiving 1, pop a 0 from top of stack and go to next state
 Step-4: On receiving 1, pop a 0 from top of stack. If stack is empty, on receiving 1 ingore
it and goto next state
 Step-5: On receiving 1 ignore it. If input is finished then goto last state

Examples:
Input : 0 0 0 1 1 1 1 1 1
Result : ACCEPTED

Input : 0 0 0 0 1 1 1 1
Result : NOT ACCEPTED

Summary of Study Session 11

At the end of this session, you have learnt the structure of a pushdown automata and how it
accepts a set of strings. Also, relationship between CFG and PDA is discussed.
Self-Assessment Questions (SAQs) for Study Session 11
1. Give the transition function δ of a DFA
2. With the aid of any given example of your choice, illustrate minimization of DFAs
3. Briefly describe the behaviour of a PDA during execution

References
1. https://www.geeksforgeeks.org/construct-pushdown-automata-given-languages/
2. https://www.tutorialspoint.com/automata_theory/