Finite Automata

Reading: Chapter 2

1
Finite Automaton (FA)
 Informally, a state diagram that comprehensively
captures all possible states and transitions that a
machine can take while responding to a stream or
sequence of input symbols
 Recognizer for “Regular Languages”

 Deterministic Finite Automata (DFA)

 The machine can exist in only
y one state at anyy given
g time
 Non-deterministic Finite Automata (NFA)
 The machine can exist in multiple states at the same time

2
Deterministic Finite Automata
- Definition
 A Deterministic Finite Automaton (DFA)
consists of:
 Q ==> a finite set of states
 ∑ ==> a finite set of input symbols (alphabet)
 q0 ==>
> a start state
 F ==> set of accepting states
 δ ==> a transition function, which is a mapping
b t
between Q x ∑ ==> Q
 A DFA is defined by the 5-tuple:
 {Q ∑ , q0,F,
{Q, F δ }

3
What does a DFA do on
reading an input string?
 Input: a word w in ∑*
 Question: Is w acceptable by the DFA?
 Steps:
 Start at the “start state” q0
 F every input
For i t symbol
b l iin th
the sequence w d
do
 Compute the next state from the current state, given the
current input symbol in w and the transition function
 If after all symbols in w are consumed, the current
state is one of the accepting states (F) then accept
w;
 Otherwise, reject w.
4
Regular Languages
 Let L(A) be a language recognized by a
DFA A.
 Then L(A) is called a “Regular Language”.

 Locate regular languages in the

Ch
Chomskyk Hi
Hierarchy
h

5
The Chomsky Hierachy
• A containment
t i t hierarchy
hi h off classes
l off fformall llanguages

Regular
g Context-
(DFA) Context-
C t t
free Recursively-
sensitive
(PDA) enumerable
(LBA) (TM)

6
Example #1
 Build a DFA for the following language:
 L = {w | w is a binary string that contains 01 as a substring}
 Steps for building a DFA to recognize L:
 ∑ = {0,1}
 Decide on the states: Q
 D i
Designate
t start
t t state
t t and d final
fi l state(s)
t t ( )
 δ: Decide on the transitions:
 “Final” states == same as “accepting states”
 Other states == same as “non-accepting states”

7
 Regular expression: (0+1)*01(0+1)*

DFA for strings containing 01

• What makes this DFA deterministic? • Q = {q0,qq1,qq2}

1 0 0,1 • ∑ = {0,1}
• start state = q0
start 0 1
q0 q1 q2 • F = {q2}
Accepting • Transition table
state symbols
0 1
q0 q1 q0
stattes

• What if the language

g g allows q1 q1 q2
empty strings? *q2 q2 q2

8
Example #2
Clamping Logic:
 A clamping circuit waits for a ”1” input, and turns on forever.
However to avoid clamping on spurious noise
However, noise, we’ll
we ll design
a DFA that waits for two consecutive 1s in a row before
clamping on.
 Build a DFA for the following language:
L = { w | w is a bit string which contains the
substring 11}
 State Design:
 q0 : start state (initially off), also means the most recent input
was not a 1
 q1: has
h never seen 11 b butt th
the mostt recentt iinputt was a 1
 q2: has seen 11 at least once
9
Example #3
 Build a DFA for the following language:
L = { w | w is a binary string that has even
number of 1s and even number of 0s}
 ?

10
Extension of transitions ((δ)) to
Paths (δ)
 δ (q,w) = destination state from state q
on input string w

 δ (q,wa) = δ (δ(q,w), a)

 Work out example #3 using the input

sequence w=10010, a=1:

 δ (q0,wa) = ?
11
Language of a DFA
A DFA A accepts string w if there is a
path from q0 to an accepting (or final)
state that is labeled by w

 i.e., L(A) = { w | δ(q0,w)  F }

 I.e., L(A) = all strings that lead to an

accepting
accept g state from
o q0

12
Non-deterministic Finite
Automata (NFA)
 A Non-deterministic Finite Automaton
(NFA)
 is of course “non-deterministic”
 Implying that the machine can exist in more
than one state at the same time
 Transitions could be non-deterministic

1 qj
qi … • Each transition function therefore
1 aps to a set o
maps of states
qk
13
Non-deterministic Finite
Automata (NFA)
 A Non-deterministic Finite Automaton (NFA)
consists of:
 Q ==> a finite set of states
 ∑ ==> a finite set of input symbols (alphabet)
 q0 ==>
> a start state
 F ==> set of accepting states
 δ ==> a transition function, which is a mapping
b t
between Q x ∑ ==> subset
b t off Q
 An NFA is also defined by the 5-tuple:
 {Q ∑ , q0,F,
{Q, F δ}

14
How to use an NFA?
 Input: a word w in ∑*
 Question: Is w acceptable by the NFA?
 St
Steps:
 Start at the “start state” q0
 For every input symbol in the sequence w do
 Determine all possible next states from all current states, given
the current input symbol in w and the transition function
 If after all symbols in w are consumed and if at least one of
the current states is a final state then accept w;
 Otherwise, reject w.

15
 Regular expression: (0+1)*01(0+1)*

NFA for strings containing 01

Why is this non-deterministic?

• Q = {q0,qq1,qq2}
0,1 0,1 •  = {0,1}
• start state = q0
start 0 1
q0 q1 q2 • F = {q2}
Final • Transition table
state symbols
0 1
What will happen if at state q1 q0 {q0,q1} {q0}
ates

an input
a put o
of 0 is
s received?
ece ed q1 Φ {q2}
sta

*q2 {q2} {q2}

16
Note: Omitting to explicitly show error states is just a matter of design convenience
(one that is generally followed for NFAs), and
i.e., this feature should not be confused with the notion of non-determinism.

What is an “error state”?

 A DFA for recognizing the key word
“while”
while w h i l e
q0 q1 q2 q3 q4 q5

Any other input symbol

qerr
Any symbol

 An NFA for the same purpose:

w h i l e
q0 q1 q2 q3 q4 q5

Transitions into a dead state are implicit 17

Example #2
 Build an NFA for the following language:
L = { w | w ends in 01}
 ?
 Other examples
 Keyword recognizer (e.g., if, then, else,
while for,
while, for include
include, etc
etc.))
 Strings where the first symbol is present
somewhere later on at least once
18
Extension of δ to NFA Paths
 Basis: δ (q,) = {q}

 Induction:
 Let δ (q0,w) = {p1,p2…,pk}
 δ (p
( i,a)) = Si f i=1,2...,k
for i 12 k

 Then, δ (q0,wa) = S1 U S2 U … U Sk

19
Language of an NFA
 An NFA accepts w if there exists at
least one path from the start state to an
accepting (or final) state that is labeled
by w
 L(N) = { w | δ(q0,w) ∩ F ≠ Φ }

20
Advantages & Caveats for NFA
 Great for modeling regular expressions
 String processing - e.g., grep, lexical analyzer

 Could a non-deterministic state machine be

implemented in practice?
 Probabilistic models could be viewed as extensions of non-
deterministic state machines
(e.g., toss of a coin, a roll of dice)
 They are not the same though

 Ap parallel computer
p could exist in multiple
p “states” at the same time

21
Technologies for NFAs
 Micron’s Automata Processor (introduced in 2013)
 2D array of MISD (multiple instruction single data)
fabric w/ thousands to millions of processing
elements.
 1 input symbol = fed to all states (i.e.,
(i e cores)
 Non-determinism using circuits
 http://www.micronautomata.com/
p

22
But, DFAs and NFAs are equivalent in their power to capture langauges !!

Differences: DFA vs. NFA

 DFA  NFA
1. All transitions are 1. Some transitions could be
deterministic non-deterministic
 Each transition leads to  A transition could lead to a
exactly one state subset of states
2. For each state, transition on 2. Not all symbol transitions
all possible symbols need to be defined explicitly (if
((alphabet)
p ) should be defined undefined will g go to an error
3. Accepts input if the last state state – this is just a design
visited is in F convenience, not to be
4. Sometimes harder to confused with “non-
construct because of the determinism”)
number of states 3. A
Acceptst input
i t if one off the
th last
l t
5. Practical implementation is states is in F
feasible 4. Generally easier than a DFA
to construct
5. P ti l implementations
Practical i l t ti
limited but emerging (e.g.,
Micron automata processor)
23
 Equivalence of DFA & NFA
 Theorem:
Should be  A language L is accepted by a DFA if and only if
true for
any L
it iis accepted
t dbby an NFA
NFA.
 Proof:
1
1. If part:
 Prove by showing every NFA can be converted to an
equivalent DFA (in the next few slides…)

2. Only-if part is trivial:

 Every DFA is a special case of an NFA where each
state
t t hhas exactly
tl one ttransition
iti ffor every iinputt symbol.
b l
Therefore, if L is accepted by a DFA, it is accepted by
a corresponding NFA.
24
Proof for the if-part
 If-part: A language L is accepted by a DFA if
it is accepted by an NFA
 rephrasing…
 Given any NFA N, we can construct a DFA D
such that L(N)=L(D)
 How to convert an NFA into a DFA?
 Observation: In an NFA
NFA, each transition maps to a
subset of states
 Idea: Represent:
eachh “subset
“ b t off NFA
NFA_states”
t t ”  a single
i l “DFA
“DFA_state”
t t ”
Subset construction
25
NFA to DFA by subset construction
 Let N = {QN,∑,δN,q0,FN}
 Goal: Build D={Q
{QD,∑,
,∑,δD,{q0},
},FD} s.t.
L(D)=L(N)
 Construction:
1. QD= all subsets of QN (i.e., power set)
2. FD=set of subsets S of QN s.t. S∩FN≠Φ
3. δD: for each subset S of QN and for each input
symbol a in ∑:
 δD(S,a)
(S a) = U δN(p,a)
(p a)
p in s

26
 Idea: To avoid enumerating all of
power set, do
“lazy creation of states”

NFA to DFA construction: Example

 L = {w | w ends in 01} 1 0
NFA: DFA: 0 1
01
0,1 [q0] [q0,q
q 1] [q0,q
q 2]
0
0 1 1
q0 q1 q2

δD 0 1 δD 0 1
δN 0 1
Ø Ø Ø [q0] [q0,q1] [q0]
q0 {q0,q1} {q0} [q0] {q0,q1} {q0} [q0,q1] [q0,q1] [q0,q2]
q1 Ø {q2} [q1] Ø {q2} *[q0,q2] [q0,q1] [q0]
*q2 Ø Ø *[q2] Ø Ø
[q0,q1] {q0,q1} {q0,q2}
*[q0,q2] {q0,q1} {q0} 0. Enumerate all possible subsets
*[q1,q
q2] Ø {q2} 1. Determine transitions
*[q0,q1,q2] {q0,q1} {q0,q2} 2. Retain only those states
reachable from {q0}
27
 NFA to DFA: Repeating the example
using LAZY CREATION
 L = {w | w ends in 01} 1 0
NFA: DFA: 0 1
01
0,1 [q0] [q0,q
q 1] [q0,q
q 2]
0
0 1 1
q0 q1 q2

δN 0 1 δD 0 1
q0 {q0,q1} {q0} [q0] [q0,q1] [q0]
q1 Ø {q2} [q0,q1] [q0,q1] [q0,q2]
*q2 Ø Ø *[q0,q2] [q0,q1] [q0]

Main Idea:
Introduce states as you go
(on a need basis)
28
Correctness of subset construction
Theorem: If D is the DFA constructed
from NFA N by subset construction,
then L(D)=L(N)
 Proof:

 Show that δD({q0},w) ≡ δN(q0,w} , for all w

 Using induction on w’s
w s length:
 Let w = xa
 δD({q0},xa)
} xa) ≡ δD( δN(q0,x},
x} a ) ≡ δN(q0,w}
w}

29
A bad case where
#states(DFA)>>#states(NFA)
 L = {w | w is a binary string s.t., the kth symbol
from its end is a 1}

 NFA has k+1 states

 But an equivalent DFA needs to have at least 2k

states

(Pigeon hole principle)

 m holes and >m pigeons
 => at least one hole has to contain two or more pigeons
30
Applications
 Text indexing
 inverted indexing
 For each unique word in the database, store all
locations that contain it using an NFA or a DFA
 Fi d pattern
Find tt P in
i ttextt T
 Example: Google querying
 E t
Extensions
i off this
thi id
idea:
 PATRICIA tree, suffix tree

31
A few subtle p
properties
p of
DFAs and NFAs
 The machine never really terminates.
 It is always waiting for the next input symbol or making

transitions.
transitions
 The machine decides when to consume the next symbol from
the input and when to ignore it.
 (but the machine can never skip a symbol)

 => A transition can happen even without really consuming an

input symbol (think of consuming  as a free token) – if this
happens then it becomes an -NFA
happens,  NFA (see next few slides).
slides)
 A single transition cannot consume more than one (non-)
symbol.

32
FA with -Transitions
 We can allow explicit -transitions in finite
automata
 i.e., a transition from one state to another state
without consuming any additional input symbol
 E li it -transitions
Explicit t iti b t
between different
diff t states
t t
introduce non-determinism.
 Makes
a es itt eas
easier
e sometimes
so et es to coconstruct
st uct NFAs s
Definition:  -NFAs are those NFAs with at
least one explicit
p -transition defined.
  -NFAs have one more column in their
transition table 33
 Example of an -NFA
L = {w | w is empty, or if non-empty will end in 01}
0,1

0 1
 -closure of a state q,
q0 q1 q2 ECLOSE(q), is the set

of all states (including
start q’0 itself) that can be
reached from q by
δE 0 1
 repeatedly making an
*q’0 Ø Ø {q’0,q0}
ECLOSE(q’0) arbitrary number of -
q0 {q0,q
q 1} {q0} {q0} ECLOSE(q0) transitions
transitions.
q1 Ø {q2} {q1} ECLOSE(q1)
*q2 Ø Ø {q2}
ECLOSE(q2) 34
 To simulate any transition:
Step 1) Go to all immediate destination states.
Step 2) From there go to all their -closure states as well.

Example of an -NFA
L = {w | w is empty, or if non-empty will end in 01}
0,1

0 1
Simulate for w=101:
q0 q1 q2
 q0’
 
start q’0
q0’ q0
1 1
Ø q0
δE 0 1
 x 0
ECLOSE(q’0)
*q’0 Ø Ø {q’0,q0} q1
q0 {q0,q
q 1} {q0} {q0} ECLOSE(q0) 1
q1 Ø {q2} {q1} q2
*q2 Ø Ø {q2}
35
 To simulate any transition:
Step 1) Go to all immediate destination states.
Step 2) From there go to all their -closure states as well.

Example of another -NFA

0,1 Simulate for w=101:
q0
0
q1
1
q2 ?
  1

start q’0 q3

δE 0 1

*q’
q0 Ø Ø {q’0,q
{q q0,q
q 3}
q0 {q0,q1} {q0} {q0,q3}
q1 Ø {q2} {q1}
*q
q2 Ø Ø {q2}
q3 Ø {q2} {q3}

36
Equivalency of DFA, NFA, -NFA
 Theorem: A language L is accepted by
some -NFA
NFA if and only if L is accepted by
some DFA

 Implication:
 DFA ≡ NFA ≡ -NFA
 (all accept Regular Languages)

37
Eliminating -transitions
Let E = {QE,∑,δE,q0,FE} be an -NFA
Goal: To build DFA D={QD,∑,δD,{qD},FD} s.t. L(D)=L(E)
C
Construction:
t ti
1. QD= all reachable subsets of QE factoring in -closures
2. qD = ECLOSE(q0)
3. FD=subsets S in QD s.t. S∩FE≠Φ
4. δD: for each subset S of QE and for each input symbol
a∑:
 Let R= U δE(p,a) // go to destination states
p in s

 , ) = U ECLOSE(r)
δD((S,a) () // from there,, take a union
r in R of all their -closures

Reading: Section 2.5.5 in book 38

 Example: -NFA  DFA
L = {w | w is empty, or if non-empty will end in 01}
0,1

0 1
q0 q1 q2

start q’0

δE 0 1
 δD 0 1

*q’0 Ø Ø {q’0,q0} *{q’0,q0}

q0 {q0,q
q 1} {q0} {q0} …
q1 Ø {q2} {q1}
*q2 Ø Ø {q2}
39
 Example: -NFA  DFA
L = {w | w is empty, or if non-empty will end in 01}
0,1 0
0
0 1 {q0,q1}
q0 q1 q2 0 1 {q0,q2}
 0
1
start start {q’0, q0} 1
q’0 q0 1
ECLOSE union
δE 0 1
 δD 0 1

*q’0 Ø Ø {q’0,q0} *{q’0,q0} {q0,q1} {q0}

q0 {q0,q
q 1} {q0} {q0} {q0,q
q 1} {q0,q
q 1} {q0,q
q 2}
q1 Ø {q2} {q1} {q0} {q0,q1} {q0}
*q2 Ø Ø {q2} *{q0,q2} {q0,q1} {q0}
40
Summary
 DFA
 Definition

 Transition diagrams & tables

 Regular language
 NFA
 Definition

 Transition diagrams & tables

 DFA vs. NFA

 NFA to DFA conversion using subset construction
 Equivalency of DFA & NFA
 Removal of redundant states and including dead states
 -transitions in NFA
 Pigeon hole principles
 Text searching applications 41