Introduction to the Theory of

Computation

Part I: Automata and Languages

1. Regular Languages
 1.1 Finite Automata
 Deterministic FA
 Non-Deterministic FA
 -NFA
 1.2 Regular Expressions
 RE = FA
 1.3 Properties of Regular Languages
 Pumping lemma
 Closure properties
 Decision properties
1.1 Finite Automata
 Deterministic FA
 Non-Deterministic FA
 NFA = DFA
 -NFA
 -NFA = NFA
Finite Automata
 The simplest computational model, with
very limited memory, but highly useful
 Example: a finite automaton modeling an
on/off switch

push
Start
off on

push
State Diagram of FA

transitions states
M1:
0 1
1 0

q1 q2 q3

0, 1

start state accept state

M1 Cont’d
0 1
1 0

q1 q2 q3

0, 1

on input “0110”, the machine goes:

q1q1q2q2q3 = “reject”
M1 Cont’d
0 1
1 0

q1 q2 q3

0, 1

on input “101”, the machine goes:

q1q2q3q2 = “accept”
M1 Cont’d
What is the
language accepted
0 1
by M1?
1 0

q1 q2 q3

0, 1
010: reject
11: accept
0110100: accept
010000010010: reject
Formal Definition of FA
 A finite automaton is defined by a 5-
tuple (Q, Σ, , q0, F)
 Q: finite set of states
 Σ: finite alphabet
 : transition function, : Q x ΣQ, takes
a state and input symbol as arguments,
and returns a state
 q0Q: start state
 FQ: set of accept states
M1’s Formal Definition
 M1 = (Q, Σ, , q0, F),
where 0 1
 Q = {q1, q2, q3} 1 0
 Σ = {0, 1}
q1 q2 q3
 (q1,0)=q1, (q1,1)=q2,
(q2,0)=q3, (q2,1)=q2, 0, 1
(q3,0)=q2, (q3,1)=q2 0 1
 q1 is the start state q1 q1 q2
 F = {q2}
q2 q3 q2
q3 q2 q2
Extension of  to Strings
 Intuitively, an FA accepts a string w =
a1a2…an if there is a path in the state
diagram that:
1. Begins at the start state,
2. Ends at an accept state, and
3. Has sequence of labels a1, a2, …, an.

 Formally, the transition function  can be

extended to *(q, w), where w is any
string of input symbols.
 Basis: *(q, ) = q
 Induction: *(q, wa) =  (*(q, w), a)
Language of an FA
 An FA M = (Q, Σ, , q0, F) accepts a
string w if *(q0, w) F.
 The language recognized by an FA
M= (Q, Σ, , q0, F) is
L(M) = {w | *(q0, w) F}.
 A language is called a regular
language if some finite automaton
recognizes it.
Designing Finite Automata
 Given some language, design a FA
that recognizes it.
 Pretending to be a FA,
 You see the input symbols one by one
 You have finite memory, i.e. finite set of
states, so remember only the crucial
information (finite set of possibilities)
about the string seen so far.
 Example
 Σ ={0,1}, L = {w | w has odd
number of 1s}, design a FA to
recognize L.
 What is the necessary
information to remember? --- Is
the number of 1s seen so far
even or odd? Two possibilities.
 Example
 Σ ={0,1}, L = {w | w has odd
number of 1s}, design a FA to
recognize L.
 What is the necessary
information to remember? --- Is
the number of 1s seen so far
even or odd? Two possibilities. qeven qodd
 Assign a state to each possibility.
 Example
 Σ ={0,1}, L = {w | w has odd
number of 1s}, design a FA to
recognize L. 0 0
 What is the necessary
information to remember? --- Is 1
the number of 1s seen so far
even or odd? Two possibilities. qeven qodd
 Assign a state to each possibility.
 Assign the transitions from one
1
possibility to another upon
reading a symbol.
 Example
 Σ ={0,1}, L = {w | w has odd
number of 1s}, design a FA to
recognize L. 0 0
 What is the necessary
1
information to remember? --- Is
the number of 1s seen so far
even or odd? Two possibilities. qeven qodd
 Assign a state to each
possibility. 1
 Assign the transitions from one
possibility to another upon
reading a symbol.
 Set the start and accept states.
Exercise
 Σ ={0,1}, L = {w | w has even
number of 0s and even number of
1s}, design a FA to recognize L.
 What to remember?
 How many possibilities?
Exercise
1

q00 q01

0 0 1 0 0

q10 q11

1
Example 1.9
 Σ ={0,1}, L = {w | w contains 001 as
a substring}, design a FA to recognize
L.
 four possibilities:
1. haven’t just seen any symbols of 001 ---
q
2. have just seen a 0 --- q0
3. have just seen a 00 --- q00
4. have seen the entire pattern 001 --- q001
Example 1.9
0
1 0, 1
0
0
1
q q0 q00 q001

1
1.1 Finite Automata
 Deterministic FA
 Non-Deterministic FA
 NFA = DFA
 -NFA
 -NFA = NFA
Nondeterministic Finite
Automata
 Deterministic: At any point when the machine is in
a state with an input symbol, there is a unique
next state to go.
 Non-Deterministic: There can be more than one
next state for each state-input pair.
 Example: an automaton that accepts all strings
ending in 01.

0, 1
M2: 0 1
q0 q1 q2
 How an NFA computes?
0, 1

0 1
q0 q1 q2

q0 q0 q0 q0 q0 q0

q1 q1 q1
(die)
q2 q2
(die)
Input: 0 0 1 0 1
Formal Definition of NFA
 An NFA can be in several states at
once, or viewed in another way, it
can “guess” which state to go next.
 Formally, an NFA is a 5-tuple N = (Q,
Σ, , q0, F), where all is as DFA, but
 : Q x ΣP (Q) is a transition function
from Q x Σ to the power set of Q.
 M2’s Formal Definition
 M2 = (Q, Σ, , q0, F),
where 0 1
 Q = {q0, q1, q2} q0 {q0,q1} {q0}
 Σ = {0, 1} q1  {q2}
 (q0,0)={q0,q1},
q2  
(q0,1)={q0}, (q1,1)={q2}
 q0 is the start state 0, 1
 F = {q2}
0 1
q0 q1 q2
Language of an NFA
 Extension of  to *(q, w):
 Basis: *(q, ) = {q}
 Induction: *(q, wa) = ∪p *(q, w)(p, a)
 Formally, the language recognized by
N = (Q, Σ, , q0, F) is
L(M) = {w | *(q0, w) ∩ F ≠ }.
(i.e. if any path from the start state to an
accept state is labeled w.)
1.1 Finite Automata
 Deterministic FA
 Non-Deterministic FA
 NFA = DFA
 -NFA
 -NFA = NFA
Equivalence of NFA and DFA
 NFA’s are usually easier to “program”
in.
 Surprisingly, for each NFA there is an
equivalent (recognizes the same
language) DFA.
 But the DFA can have exponentially
many states.
Subset Construction
 Given an NFA N=(QN, Σ, N, q0, FN),
we will construct an DFA D=(QD, Σ,
D, {q0}, FD), such that L(D) = L(N).
 Subset construction:
 QD = {S | S  QN}, i.e. power set of QN
 D(S, a) = ∪p  S N(p, a)
 FD = {S | S ∩ F ≠ , S ∈ QD}
Construct DFA from M2
 Subset 0 1
construction:   
{q0} {q0,q1} {q0}
8 possible subsets, {q1}  {q2}
3 accessible: {q2}  
1 0 {q0,q1} {q0,q1} {q0,q2}
{q0,q2} {q0,q1} {q0}
0 1
{q0} {q0,q1} {q0,q2} {q1,q2}  {q2}
{q0,q1 ,q2} {q0,q1} {q0,q2}
0
1
Example NFA
 Design an NFA to accept strings over alphabet {1, 2,
3} such that the last symbol appears previously,
without any intervening higher symbol, e.g., …11, …
21112, …312123.

1, 2, 3 q
1 1

2 2
p r t
3 1 3

s
1, 2
Equivalent DFA 1 2 3
p pq pr ps
 32 possible pq
pqt
pqt
pqt
pr
pr
ps
ps
subsets, 15 pr pqr prt ps
accessible: prt pqr prt ps
ps pqs prs pst
 DFA is much pst pqs prs pst
larger than NFA prs pqrs prst pst
prst pqrs prst pst
pqs pqst prs pst
pqst pqst prs pst
pqr pqrt prt ps
pqrt pqrt prt ps
pqrs pqrst prst pst
pqrst pqrst prst pst
Proof: L(D)=L(N)
 Induction on |w| to show that
*D({q0}, w) = *N(q0, w)
 Basis: w=, the claim follows from the def.
 Induction: *D({q0}, wa) = D(*D({q0}, w), a)
= D(*N(q0, w), a)
= ∪p* (q0, w)N(p, a)
N

= *N(q0, wa)
 Then it follows that L(D) = L(N), why?
1.1 Finite Automata
 Deterministic FA
 Non-Deterministic FA
 NFA = DFA
 -NFA
 -NFA = NFA
Finite Automata with -
Transitions
 Allow  to be a label on arcs.
 Formally the transition function  of
an -NFA is from Q x Σ∪{} to P (Q).
 Nothing else changes: acceptance of
w is still the existence of a path from
the start state to an accept state with
label w. But  can appear on arcs, and
means the empty string (i.e., no
visible contribution to w).
Example of an -NFA
0

1 
q r s

0 

“001” is accepted by the path

qsrqrs with label 001 = 001
1.1 Finite Automata
 Deterministic FA
 Non-Deterministic FA
 NFA = DFA
 -NFA
 -NFA = NFA
Elimination of -Transitions
 -transitions are a convenience, but do not
increase the power of FA's. To eliminate -
transitions:
1. Compute the transitive closure of the -arcs
only.
2. If a state p can reach state q by -arcs, and
there is a transition from q to r on input a (not
), then add a transition from p to r on input a.
3. Make state p an accept state if p can reach
some accept state q by -arcs.
4. Remove all -transitions.
-CLOSURE
 -CLOSURE(q): all states reachable from q
by a sequence …
 q-CLOSURE(q);
 p-CLOSURE(q) and r(p, )  r-CLOSURE(q)
Example
1. -CLOSURE(q)={q}, -CLOSURE(r)={r,s},
-CLOSURE(s)={r,s}

1 
q r s
0 

1
Example
1. -CLOSURE(q)={q}, -CLOSURE(r)={r,s},
-CLOSURE(s)={r,s}
2. Add (s, 0)={q}, (r, 1)={q}

1 
q r s
0, 1 

0, 1
Example
1. -CLOSURE(q)={q}, -CLOSURE(r)={r,s},
-CLOSURE(s)={r,s}
2. Add (s, 0)={q}, (r, 1)={q}
3. Add r into F as an accept state

1 
q r s
0, 1 

0, 1
Example
1. -CLOSURE(q)={q}, -CLOSURE(r)={r,s}, -CLOSURE(s)={r,s}
2. Add (s, 0)={q}, (r, 1)={q}
3. Add r into F as an accept state
4. Remove (s, )={r}, (r, )={s}

1
q r s
0, 1

0, 1
Summary of Finite Automata
 DFA, NFA, and -NFA are all
equivalent.

-NFA  NFA  DFA

-elimination subset construction

Example 1.21

b a,b
1 2 3

a

Assignment 1
 Exercise 1.4: (a)(b)(g)(i)(j)
 Exercise 1.5: (b)(c)
 Exercise 1.12