CSC 428

Formal Languages and Automata Theory

Automata theory is the study of abstract computing
devices, or “machines”. Before the invention of
computers, some abstract machines that have all the
capabilities of today’s computers were introduced. For
example, Alan Turing, introduced an abstract machine in
1930s. Though limited in some areas of computation, at
least it could go as far in what it could compute.
What is an algorithm?
A finite sequence of simple instructions that is guaranteed
to halt in a finite amount of time or a detailed step-by-
step method for solving a problem.
This is a very abstract definition, since the nature of this
simple instructions was not specified.
For example an instruction can be “increment a
number by one” or “Calculate the triple integral”
Which entity can execute these instructions?
For example, is this entity a person or a computer?
If it is a computer, what is the processor type? How
much memory does it have?
Application Areas in Computer Science
 Analysis of algorithms
 Complexity Theory (difficult idea that explains

something)
 Cryptography (the art of writing or solving codes)
 Compilers (translators of source codes)
 Circuit design
BRIEF HISTORY OF FINITE AUTOMATA
1936 Alan Turing invented the Turing machine, and proved that there
exists an unsolvable problem.
1940’s Stored-program computers were built.

1943 McCulloch and Pitts invented finite automata.

1956 Kleene invented regular expressions and proved the

equivalence of regular expression and finite automata.
1956 Chomsky defined Chomsky hierarchy, which organized
languages recognized by different automata into hierarchical classes.
1959 Rabin and Scott introduced nondeterministic finite automata
and proved its equivalence to (deterministic) finite automata.
1950’s-1960’s More works on languages, grammars, and compilers.
DEFINITION OF CONCEPTS
i) An alphabet is a finite, non-empty set of symbols.
{0,1 } is a binary alphabet.
{ A, B, …, Z, a, b, …, z } is an English alphabet.
ii) A string over an alphabet  is a sequence of any
number of symbols from .
0, 1, 11, 00, and 01101 are strings over {0, 1 }.
Cat, CAT, and compute are strings over the English
alphabet.
iii) An empty string, denoted by , is a string containing no
symbol.
 is a string over any alphabet.
iv) The length of a string x, denoted by length(x), is the
number of positions of symbols in the string.
Let Σ = {a, b, …, z}
length(automata) = 8
length(computation) = 11
length(ε) = 0
x(i), denotes the symbol in the ith position of a string
x, for 1 i  length(x).

String Operations
a) Concatenation
b) Substring
c) Reversal
(a) The concatenation of strings x and y, denoted
by xy or xy, is a string z such that:
z(i) = x(i) for 1  i  length(x)
z(i) = y(i) for length(x)<ilength(x)+length(y)
Example:
automatacomputation = automatacomputation
The concatenation of string x for n times, where n0, is
denoted by xn
x0 = 
x1 = x
x2 = x x
x3 = x x x
…
(b) Substrings :- Let x and y be strings over an alphabet Σ
The string x is a substring of y if there exist strings w
and z over Σ such that y = w x z.
ε is a substring of every string.
For every string x, x is a substring of x itself.

Example:
ε, comput and computation are substrings of
computation.
(c) Reversal
Let x be a string over an alphabet Σ
The reversal of the string x, denoted by xr, is a string such
that:
if x is ε, then xr is ε.
If a is in Σ, y is in Σ* and x = ay, then xr = yra.
Example:
(automata)r
= (utomata)r a
= (tomata)r ua
= (omata)r tua
= (mata)r otua
= (ata)r motua
= (ta)r amotua
= (a)r tamotua
= ()r atamotua
= atamotua
What is a Language?
A language over an alphabet Σ is a set of strings over Σ.
Let Σ = {0, 1} be the binary alphabet.
Le = {Σ* | the number of 1’s in  is even}.
, 0, 00, 11, 000, 110, 101, 011, 0000, 1100, 1010, 1001,
0110, 0101, 0011, … are in Le
Operations on Languages

Complementation, Union, Intersection, Concatenation

Reversal, Closure, positive closure, etc.
Closure (*)
The set of strings created from any number (0 or 1
or …) of symbols in an alphabet  is denoted by *.
That is, * denotes the set {∑n: n≥0}
E.g. 1: Let  = {0, 1}.
* = {, 0, 1, 00, 01, 10, 11, 000, 001, 010, 011, … }.
E.g. 2: a* = {, a, aa, aaa, aaaa, …}
A* denotes the set of string including the empty string,
which can be formed from the symbols contained in A.
Exercise 1: if A = {a,b,c}, then what is A*?
Positive Closure (Σ+)
The set of strings created from at least one symbol (1 or 2
or …) in an alphabet  is denoted by +.
That is, + denotes the set {an:n≥1}
E.g. (1): Let  = {0, 1}.
+ = {0, 1, 00, 01, 10, 11, 000, 001, 010, 011, … }.

E.g. (2): if A = {a,b,c} then

A+ = {a,b,c,aa,ab,ac,ba,bb,bc,ca,cb,cc,aaa,…}
Note:
 * and + are infinite sets.
 * is called the free monoid over the alphabet A.
 + is called the free semi-group over the alphabet A
 Finite State Automata (FSA)
A finite automaton (FA) is a simple idealized machine
used to recognize patterns within input taken from some
character set (or alphabet) C. The job of an FA is to accept
or reject an input depending on whether the pattern
defined by the FA occurs in the input. A finite
automaton consists of: a finite set S of N states.
Automata theory is the study of abstract machines, as
well as the computational problems that can be solved
using them.
It is a theory in theoretical computer science, under
discrete mathematics (a subject of study in both
mathematics and computer science).
An NFA is a Nondeterministic Finite Automaton.
Nondeterministic means it can transition to, and be in,
multiple states at once (i.e. for some given input). A DFA is
a Deterministic Finite Automaton. Deterministic means
that it can only be in, and transition to, one state at a time
(i.e. for some given input).
Deterministic Finite Automaton (DFA)
In DFA, for each input symbol, one can determine the state
to which the machine will move. Hence, it is
called Deterministic Automaton. As it has a finite number
of states, the machine is called Deterministic Finite
Machine or Deterministic Finite Automaton.
Formal Definition of a DFA
A DFA can be represented by a 5-tuple (Q, ∑, δ, q0, F)
where −
Q is a finite set of states.
∑ is a finite set of symbols called the input alphabet.
δ is the transition function where δ: Q × ∑ → Q
q0 is the initial state from where any input is processed

(q0 ∈ Q).
F is a set of final state/states of Q (F ⊆ Q).
Graphical Representation of a DFA
A DFA is represented by digraphs called state diagram.
 The vertices represent the states.
 The arcs labeled with an input alphabet show the
transitions.
 The initial state is denoted by an empty single
incoming arc.
 The final state is indicated by double circles.
Let a deterministic finite automaton be →
•Q = {a, b, c},
•∑ = {0, 1},
•q0 = {a},
•F = {c}, and

Transition function δ as shown by the

following table −
Present State Next State for Next State for
Input 0 Input 1
a a b
b c a
c b c
Its graphical representation would be as follows:
 Turing Machine
It is a conceptual model for general purpose computers
proposed by Alan Turing in 1936. It has an unlimited and
unrestricted amount of memory. A Turing machine can do
everything a real computer can do. Nevertheless there are
problems that a Turing machine cannot solve. In a real sense, these
problems are beyond the theoretical limits of computations.
Definition:- A Turing machine uses an infinite two-way tape, six-
tuples (six parts), and a halt state. Every effective computation or
algorithm can be carried out by a Turing Machine.
Components of Turing Machine:
1. An unlimited length tape of discrete cells.
2. A head which reads and writes on tape.
3. A control device with a finite number of states which can
a) Instruct the head to read the symbol on the tape
currently under head.
b) Instruct the head to write a symbol on the cell of the tape
currently under tape.
c) Move the head one cell to left (L), right (R) or No
movement (N).
d) Change its current state.
Conceptual View of a Turning Machine
 The Turing machine M may be viewed as a read/write tape
head which moves back and forth along an infinite tape. The tape is
divided lengthwise into squares (cells), and each square may be
blank or hold one tape symbol. At each point in time, the Turing
machine M is in a certain internal state si scanning one of the tape

symbols aj on the tape.

E.g. a 1 a2 a1 a1

s2

α = a1 s2 a2B a1 a1

The figure above is a picture of a TM in a state s2 scanning the

Turing Machine Instructions
Instructions of Turing Machine have the following format (Current
State, Current Symbol, Write, Move L/R or No move, New State)

Ex: Show the visualization of the following TM instructions:`

(i) (2, 0, 1, L, 3)
(ii) (3, 1, blank, N, 4)
(iii) (1, #, 0, R, 7)

The interpretation of the TM (Turing Machine) instructions in

example (i) (2, 0, 1, L, 3) is as follows:

When Turing machine (the control unit of TM) is at state 2 and the
current tape symbol is 0, write symbol 1 at current tape cell and go
to state 3.
The visualization of TM instruction (2, 0, 1, L, 3)
Turing Machine Conventions
(a) We always use state 1 as the initial state. (That is the
execution of the algorithm or program begins with stating of the
TM being 1.

(b) The tape is used for recording input and output, one symbol
per cell. Initially, the string to serve as input to our computation
is recorded beginning from the leftmost tape cell.

(c) Initially, the position of head is at left most cell.

Transition Diagram
A Transition diagram of the next-move functions δ of a TM
is a graphical representation consisting of a finite number
of nodes and directed-labelled arcs between the nodes.
Each node represents a state of the TM and a label on an
arc from one state (e.g. q1) to a state (say q2) represents
the information about the required input symbol say ‘a’ for
the transition from q1 to q2 to take place and the action on
the part of the control of TM.
Example 1: Let TM = {Q, ∑, ꭈ, δ, q0, h}
where:
Q = {q0, q1, q2, h}
∑ = {0, 1}
ꭈ = {0, 1, #}
and δ be given by the following table:

δ 0 1 #
q0 - - (q2, #, R)
q1 (q2, 0, R) (q1, #, R) (h, #, N)
q2 (q2, 0, L) (q2, 1, R) (h, #, N)
h - - -

The transition diagram for the above TM will be drawn on

the board.
Exercise 1: Design a TM that accepts the language of all
strings which contain aba as a substring.
Let TM = {Q, ∑, ꭈ, δ, q0, h}
where:
Q = {q0, q1, q2, h}
∑ = {a, b}
ꭈ = {a, b, #}
q0 is the initial state
and δ be given by the following table:
δ a b #
q0 (q1, a, L) (q0, b, L) *
q1 (q1, a, L) (q2, b, L) *
q2 (h, a, L) (q0, b, L) (h, #, N)
h * * Accept
 COMPUTABLE FUNCTION
The basic characteristic of a computable function is that there
must be a finite procedure (an algorithm) telling how to compute
the function. The models of computation listed above give
different interpretations of what a procedure is and how it is used,
but these interpretations share many properties. The fact that
these models give equivalent classes of computable functions
stems from the fact that each model is capable of reading and
mimicking a procedure for any of the other models, much as a
compiler is able to read instructions in one computer language
and emit instructions in another language.
Computable functions are the basic objects of study in
computability theory. Computable functions are the formalized
analogue of the intuitive notion of algorithm. They are used to
discuss computability without referring to any concrete model of
computation such as Turing machine or register machines. Any
definition, however, must make reference to some specific model
of computation but all valid definitions yield the same class of
functions. Particular models of computability that give rise to the
set of computable functions are the Turing-computable functions
and the µ-recursive functions.
COMPUTATIONALLY TRACTABLE AND INTRACTABLE PROBLEMS
A problem is said to be computationally tractable, if there exists an
algorithm of polynomial complexity for all instances and solves the
problem. While intractable problems or computationally hard
problems refer to an instance where the optimal algorithm for
solving a problem cannot solve a given problem in polynomial time.
These algorithms are said to take exponential time to resolve.

The basic idea for problem classification lies on whether there

exists a polynomial-time (computer) algorithm that can provide a
solution to a problem. A polynomial time is the time required for a
computer to solve a problem, where this time is a simple polynomial
function of the size of the input.