Week4 Chapter2 Automata
Week4 Chapter2 Automata
MATHEMATICAL MODELING
Methods and Applications
Chapter 2
AUTOMATA THEORY-
A. BASIC TERMS
B. AUTOMATA
MATHEMATICAL MODELING
Methods and Applications
3
COURTESY:
Some material of chapter 1 (Logic Theory) and 2 (Automat Theory) are borrowed from texts
1. REF. 1: MICHAEL HUTH and MARK RYAN. LOGIC IN COMPUTER SCIENCE - Modelling
and Reasoning about Systems, Chapter 1,2,4 [2nd ed., Cambridge University Press 2004]
2. REF. 2: MICHAEL HUTH and MARK RYAN. Solutions Manual - LOGIC IN COMPUTER
MATHEMATICAL MODELING
Methods and Applications
2.1. PART A. BASIC TERMS 4
1. Computer science a very diverse (with many important topics), but in spite of this diversity,
there are some common underlying principles. To study these basic principles, we construct
2. Why study automata theory? Automata theory provides such abstract model, its ideas
have some important applications, both software (programming languages, and compilers
3. An automaton 1 : a construct that possesses all the features of a digital computer, or any
process. It accepts input, produces output, may have some storage, and can make decisions
in transforming the input into the output, see Figure 2.1, in which we use notations below.
MATHEMATICAL MODELING
Methods and Applications
2.1. PART A. BASIC TERMS 5
MATHEMATICAL MODELING
Methods and Applications
2.1. PART A. BASIC TERMS 6
The set of all lower-case letters: Σ = {𝑎, 𝑏, 𝑐, . . . , 𝑧} ... And other sets.
Σ consists of almost all available characters (lowercase and capital letters, numbers, sym-
bols and special characters as space or Greek characters 𝛼, 𝛽...).
MATHEMATICAL MODELING
Methods and Applications
2.1. PART A. BASIC TERMS 7
Question: How about {𝜀}, {0, 00, 001}, are they languages over Σ = {0, 1} ?
guages.
(ii) formation rules by which these symbols are combined into entities called sentences.
5. Operations on languages:
Because languages are sets, hence set operators as the union, intersection, difference ...
MATHEMATICAL MODELING
Methods and Applications
2.1. PART A. BASIC TERMS 8
The reverse of a language is the set of all string reversals, that is,
𝐿𝑅 = {𝑤𝑅 : 𝑤 ∈ 𝐿.}
♦ EXAMPLE 2.1.
b) 𝐿1 ∩ 𝐿2 =? 𝐿1 ∖ 𝐿2 =?
c) 𝐿1 𝐿2 =? 𝐿1𝐿2 = {𝑎𝑏𝑏, 𝑎𝑎𝑏, 𝑏𝑏, 𝑎𝑏𝑐𝑎, 𝑎𝑎𝑐𝑎, 𝑏𝑐𝑎, 𝑎𝑏𝑏𝑎𝑐, 𝑎𝑎𝑏𝑎𝑐, 𝑏𝑏𝑎𝑐},
d) 𝐿2 𝐿1 =? Now newly define 𝐿 = {𝑎𝑏, 𝑎𝑎, 𝑏, 𝑐𝑎, 𝑏𝑎𝑐}, find 𝐿2 =?
PRACTICE 2.1.
1. Prove by induction that: If 𝑢 and 𝑣 are strings, then the length of their concatenation is the
MATHEMATICAL MODELING
Methods and Applications
2.2. Determining a language 10
PRACTICE 2.2.
Let Σ = {𝑎, 𝑏, 𝑐}. Give at least 5 strings for each of the following languages
MATHEMATICAL MODELING
Methods and Applications
2.2. Determining a language 11
b) Using a property, such that all words of the language have this property but other words
have not.
(it is in fact 𝐿3 = {𝜀, 𝑎𝑏, 𝑎𝑎𝑏𝑏, 𝑎𝑎𝑎𝑏𝑏𝑏, . . .} in PRACTICE 2.1 above by enumeration).
Also using property on string length gives 𝐿6 = {𝑢 ∈ Σ* | 𝑛𝑎 (𝑢) = 𝑛𝑏 (𝑢)}, where 𝑛𝑐 (𝑢)
MATHEMATICAL MODELING
Methods and Applications
2.2. Determining a language 12
𝑥 → 𝑦, 𝑥 ∈ (𝑉 ∪ 𝑇 )+ and 𝑦 ∈ (𝑉 ∪ 𝑇 )*.
We say that string 𝑤 derives string 𝑧 (or that 𝑧 is derived from 𝑤).
MATHEMATICAL MODELING
Methods and Applications
2.2. Determining a language 13
ELUCIDATION
1. Briefly, the production rules of grammar 𝐺 specify how the grammar transforms one string
into another, and they define a language associated with the grammar.
2. Successive strings are generally derived by applying such productions of the grammar 𝐺 in
MATHEMATICAL MODELING
Methods and Applications
2.2. Determining a language 14
♦ EXAMPLE 2.3.
MATHEMATICAL MODELING
Methods and Applications
2.2. Determining a language 15
*
We see 𝑆 ⇒ 𝑎𝑆𝑏 ⇒ 𝑎𝑎𝑆𝑏𝑏 = 𝑎2 𝑆𝑏2 ⇒ 𝑎2 𝑏2 ... then 𝑆 ⇒ 𝑎𝑎𝑏𝑏 = 𝑎2 𝑏2 .
♦ EXAMPLE 2.4.
MATHEMATICAL MODELING
Methods and Applications
2.2. Determining a language 16
Or 𝑆 ⇒ 𝑀 𝑏 ⇒ 𝑎𝑀 𝑏𝑏 ⇒ 𝑎𝑎𝑀 𝑏𝑏𝑏? ■
Often it is not so easy to find a grammar for a language described in an informal way, or to
cally that a given language 𝐿 is indeed generated by a certain grammar 𝐺, we must be able
to show
(a) that every word 𝑤 ∈ 𝐿 can be derived from the start variable 𝑆 using 𝐺 and
a
Theorem (REF. 3, p 17): Not all languages can be generated by grammars.
a
Antal Iványi. Algorithms of Informatics, AnTonCom Budapest (2010).
MATHEMATICAL MODELING
Methods and Applications
2.2. Determining a language 17
PRACTICE 2.3. Denote 𝑛𝑐 (𝑢) the number of letter ’c’ in any word 𝑢.
PRACTICE 2.4.
MATHEMATICAL MODELING
Methods and Applications
2.3. Regular Expressions 18
Definition 2.3
Regular expressions over some given alphabet Σ involve a combination of ♣
(︀)︀
♦ strings of symbols from Σ, parentheses , and
MATHEMATICAL MODELING
Methods and Applications
2.3. Regular Expressions 19
But (𝑎 + 𝑏 +) is not a regular expression, since there is no way it can be constructed from
MATHEMATICAL MODELING
Methods and Applications
2.3. Regular Expressions 20
representing language 𝐿.
The language 𝐿(𝑟) ⊂ Σ* associated with expression 𝑟 is defined by the following rules.
MATHEMATICAL MODELING
Methods and Applications
2.3. Regular Expressions 21
spectively. Then (𝑟+𝑠), (𝑟·𝑠), 𝑟 * are regular expressions, (so they) respectively representing
𝑌 , 𝑋 𝑌 , and 𝑋 *. Explicitly,
⋃︀
languages 𝑋
⋃︀
4. 𝐿(𝑟 + 𝑠) ≡ 𝐿(𝑟) + 𝐿(𝑠) = 𝑋 𝑌
6. 𝐿((𝑟)) ≡ 𝐿(𝑟) = 𝑋
)︀*
7. 𝐿(𝑟 * ) ≡ 𝐿(𝑟) = 𝑋 *.
(︀
The language 𝐿(𝑎* · (𝑎 + 𝑏)) = 𝐿(𝑎* ) 𝐿(𝑎 + 𝑏) = ...? = {𝑎, 𝑎𝑎, 𝑎𝑎𝑎, . . . , 𝑏, 𝑎𝑏, 𝑎𝑎𝑏, . . .}
𝑟 = (𝑎 + 𝑏)* represents all the (regular) strings of of 𝑎’s and 𝑏’s, any order.
)︀*
So language 𝐿(𝑟) = 𝐿(𝑎 + 𝑏) = Σ* ?
(︀
■
MATHEMATICAL MODELING
Methods and Applications
2.3. Regular Expressions 22
We also employ the following properties, applied for regular expressions 𝑥, 𝑟 and 𝑠.
(𝑥 + 𝑟) + 𝑠 ≡ 𝑥 + (𝑟 + 𝑠);
(𝑥𝑟)𝑠 ≡ 𝑥(𝑟𝑠)
(𝑥 + 𝑟)* ≡ (𝑥*𝑟*)*
(𝑟*)* ≡ 𝑟*
𝑥*𝑥 ≡ 𝑥𝑥*;
𝑥𝑥* + 𝜀 ≡ 𝑥*
MATHEMATICAL MODELING
Methods and Applications
2.3. Regular Expressions 23
PROBLEM 2.1.
On the alphabet Σ = {𝑎, 𝑏, 𝑐}, give at least 5 words for each language represented by the
a) 𝐸1 = 𝑎* + 𝑏* ,
Hint for c): You will prove firstly 𝐿2 = 𝐿(𝑠) = 𝐿(𝑎* · (𝑎 + 𝑏)) = {𝑎𝑛+1 , 𝑎𝑚 𝑏 | 𝑛, 𝑚 ≥ 0}.
MATHEMATICAL MODELING
Methods and Applications
2.3. Regular Expressions 24
PROBLEM 2.2.
1. 𝐸1 = 𝜀 + 𝑎𝑏 + 𝑎𝑏𝑎𝑏(𝑎𝑏)* ,
MATHEMATICAL MODELING
Methods and Applications
PART B. AUTOMATA
[[GoogleMap]]
2.4. PART B. FINITE AUTOMATA 26
the vertices give the initial states and one or several final/accepting states, and the edges
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 27
𝑤
A string 𝑤 = 𝑎1 𝑎2 · · · 𝑎𝑛 ∈ Σ* is accepted by an automaton 𝑀 if symbolically 𝑞 ⇝ 𝑞𝑤 ∈
♦ EXAMPLE 2.8.
(︀ )︀
This state diagram shows the two-state finite automaton 𝑀 = {𝑞1 , 𝑞2 }, {0, 1}, 𝑞1 , 𝛿, {𝑞2 } ,
where 𝐹 = {𝑞2 } (unique accepting state), the transition function 𝛿 is given by table below,
that is 𝛿(𝑞1 , 0) = 𝑞1 , 𝛿(𝑞1 , 1) = 𝑞2 and so on. Final states are drawn with a double circle.
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 28
0 1
𝑞1 𝑞 1 𝑞2
𝑞2 𝑞 1 𝑞2
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 29
But string 𝑠 = 110 leaves 𝑀 in state 𝑞1 , so it is rejected. You would see that
𝑀 accepts all strings that end in a 1. Thus the language of 𝑀 , 𝐿(𝑀 ) = {𝑤| 𝑤 ends in a 1 }.
𝑎 = 1, 𝑏 = 0 on the machine 𝑀 then we get the same conclusion, for 𝑟 = 𝑎𝑎𝑏𝑎 ∈ 𝑎*(𝑏 + 𝑎).
■
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 30
𝑏
𝑎, 𝑏
𝑞0 𝑞1
PROBLEM 2.3.
1) 𝑎3 𝑏,
2) 𝑎𝑏𝑎2 𝑏,
3) 𝑎4 𝑏2 𝑎𝑏3 𝑎,
4) 𝑎4 𝑏𝑎4 ,
5) 𝑎𝑏4 𝑎9 𝑏,
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 31
𝑎 𝑏
𝑏
𝑞0 𝑞1 𝑎 𝑞2
𝑏
𝑎
II) Do Exercises on page 4.2x, for 𝑥 = 4, 5, 6 of REF. 1.
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 32
(𝑄, Σ, 𝐼, 𝛿, 𝐹 ) where ♣
ELUCIDATION
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 33
Rather than prescribing a unique move in each situation, we allow a set of possible moves.
An NFA is a directed, labeled graph, named the transition graph 𝑇 (𝑀 ) = (𝑉, 𝐸) of automaton 𝑀 ,
whose node set 𝑉 ≡ 𝑄 and 𝐸 = {(𝑝, 𝑎, 𝑞)}, consisting of transitions - directed edges [la-
Initial states are marked by a small arrow entering the corresponding vertex,
while the final states are marked with double circles [as 𝑞4 in the next figure].
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 34
♦ EXAMPLE 2.10. Fix 𝑄 = {𝑞1, 𝑞2, 𝑞3, 𝑞4} and Σ = {0, 1, 𝜀}.
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 35
* The transition graph 𝑇 (𝑀 ) of 𝑀 has edges (𝑞1 , 𝑞1 ) and (𝑞1 , 𝑞2 ) with label 𝑎 = 1.
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 36
We need two more concepts for the next definition (language accepted by NFA).
We say that
if this word is the label of a productive walk (that is, there is some sequence of possible
moves that will put the machine in a final state 𝑞𝑛 at the end of the string.)
The set of words accepted by an NFA is called the language accepted by this NFA.
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 37
𝑤
Explicitly this means for each string 𝑤 ∈ 𝐿(𝑀 ) there is a walk labeled 𝑞 −→ 𝑝 from an
So instead of using
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 38
The language accepted by an NFA so is defined by the extended transition 𝛿 * (𝑞, 𝑤) [Def.
2.8].
(Kleene)].
Let 𝑟 be a regular expression. Then there exists some nondeterministic finite automaton
(NFA) that accepts the language 𝐿(𝑟). Consequently, 𝐿(𝑟) is a regular language.
We discuss more on the equivalence connection between regular languages and regular
1. Regular Expressions define regular languages: how construct an NFA 𝑀 from a given reg-
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 39
2. Regular Languages give regular expressions: from a regular language 𝐿, find a regular
PROBLEM 2.4.
(I) Consider the set of strings on Σ = {𝑎, 𝑏} in which every 𝑎𝑎 is followed immediately by 𝑏.
For example 𝑎𝑎𝑏, 𝑎𝑎𝑏𝑎, 𝑎𝑎𝑏𝑎𝑎𝑏𝑏𝑎𝑎𝑏 are in the language, but 𝑎𝑎𝑎𝑏 and 𝑎𝑎𝑏𝑎𝑎 are not.
𝑎 𝑏
𝑏
𝑞0 𝑞1 𝑎 𝑞2
𝑏
𝑎
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 40
𝑀 = (𝑄, Σ, 𝑞0, 𝛿, 𝐹 )
with
4. 𝛿 : 𝑄 × Σ → 𝑄 is a transition function,
ELUCIDATION
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 41
|𝛿(𝑞, 𝑎)| ≤ 1, ∀𝑞 ∈ 𝑄, ∀𝑎 ∈ Σ
in Defi. 2.7.
The fact |𝛿(𝑞, 𝑎)| = 0 says 𝛿(𝑞, 𝑎) = ∅ (nonfinal trap state in the DFA):
Given a walk 𝑟 = 𝑎𝑏𝑐 . . . 𝑠 (series of accepted arrows/ letters/ actions), and a state 𝑞 ,
the automaton 𝑀 starts with the transition function, called an application 𝛿(𝑞, 𝑎) = 𝑝.
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 42
𝑎 𝑏 𝑠
𝑞 −→ 𝑝 −→ · · · −→ 𝑝𝑛
The input mechanism then advances one position to the right, as a result
The transition graph 𝑇 (𝑀 ) of finite automaton 𝑀 has an edge (𝑞, 𝑝) with label 𝑎...
the string is accepted if the automaton is in state 𝑝𝑛 ∈ 𝐹 (one of its final states):
𝑎 𝑏 s
𝑞 −→ 𝑝 −→ · · · −→ pn.
♦ EXAMPLE 2.11.
The figure shows the transition graph 𝑇 (𝑀 ) of a DFA = 𝑀 = (𝑄, Σ, 𝑞0 , 𝛿, 𝐹 ) with 𝐹 = {𝑞1 }.
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 43
ii/ Check 𝑀 recognizes the set 𝐴 of strings which contain an odd number of 𝑎, that is
𝐿(𝑀 ) = 𝐴.
iii/ Determine a new machine 𝑀𝐵 accepts the set 𝐵 of strings which contain an even number
of 𝑎, that is 𝐵 = 𝐿(𝑀𝐵 ).
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 44
HINT: Obviously 𝑄 = {𝑞0 , 𝑞1 }, the alphabet Σ = {𝑎, 𝑏}, and the set 𝐹 = {𝑞1 } of accepting
states.
a b
→ 𝑞0 𝑞1 𝑞0
𝑞1 𝑞0 𝑞1
ii/: Need to write the pattern of strings 𝑟 ∈ 𝐴 which contain an odd number of 𝑎, as
𝑟 = 𝑏*𝑎𝑛𝑏*, 𝑛 = 2𝑘 + 1?
Then check the equality 𝛿(𝑞0 , 𝑟) = 𝑞1 ∈ 𝐹 if and only if 𝑟 has the above pattern only.
𝑠 = 𝑏*𝑎𝑚𝑏*, 𝑚 = 2𝑘 =⇒ 𝐵 = 𝐿(𝑀𝐵 )
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 45
Proposition 2.1
Transition graphs provide convenient ways for working with FA. ♠
Give a DFA that accepts all words that contain a number of 0 multiple of 3.
Define the states 𝑄, the transition 𝛿 and set 𝐹 of accepting states properly.
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 46
Then define a transition 𝛿 such that 𝛿(𝑞𝑜 , 𝑤) ∈ 𝐹 for any word 𝑤 satisfying that 𝑛0 (𝑤) mod 3 = 0
1 1
0
𝑞0 𝑞1
0
iii) What if we redefine states 𝑄 = {𝑞0 , 𝑞1 , 𝑞2 }, and 𝐹 = {𝑞1 , 𝑞2 }? Can we redraw the graph
as
1 1 1
0 0
𝑞0 𝑞1 𝑞2
0 0
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 47
MATHEMATICAL MODELING
Methods and Applications
2.4. PART B. FINITE AUTOMATA 48
WHAT NEXT?
1. Construction of a DFA and NFA from a given language 𝐿, or a given regular expression 𝐸 ,
2. Equivalence of automata: Build a DFA from a given NFA, see Section 2.6.1;
Why? For an NFA, for a single input given to a single state, the machine can go to more
than one state. So, for an NFA, it is hard to decide for a string to be accepted by the NFA or
MATHEMATICAL MODELING
Methods and Applications