Introduction

Formal Language and Automata Theory

1
Course Introduction

2
Formal Language and Automata Theory

• Pre-requisite: Basic logic and mathematics skills.

• Main topics of this course is Formal Languages and Automata Theory (FLAT).
• We will also deal with computational complexity related topics.
• Schedule: Tuesday 6~9 (14:00~17:00)
• This is important since we will have a quiz and exams.
• If there is no issue, exam schedule will follow the course schedule.

3
Contacts

• Contacts
• In person - Room 331, Mirae Hall, please check w/ me in advance.
• Use e-class messages.
• Do not use e-mails. All enquires through e-class.
• Work - Life Balance
• Please make your enquires before 9 PM during week days.
• Otherwise, I'll answer them in the next day, or on Monday for the late questions
or weekend questions.
4
Lecture Type and Teaching Methods

• Lecture topics will be according to syllabus, but it could be exible.

• Week 4: we will have a quiz.
• Week 11: there will be a programming assignment.
• Weeks 8 and 15
• Mid-term and Final exams.

fl
Lecture Type and Teaching Methods
Textbook

• Each lecture will explain topics and contents in the textbook.

• Peter Linz, An Introduction to Formal Languages and Automata, 6th Edition,
Jones & Bartlett

• You may want to use Korean version of the textbook.

• Peter Linz , , ,

• But you have to use the technical terms in the original textbook, so I
recommend to read the English version.

• You don't need to buy the book, but I strongly recommend you to read it.
6
Lecture Type and Teaching Methods
Lectures

• Each lecture covers some sections of the textbook.

• So lectures will be coherent to the textbook.
• Mostly I'll use the examples on the textbook, and give detailed explanation about them.
• For some cases, I'll use exercise questions and explain how to solve/answer the
questions.

• Occasionally, I'll provide some contents outside of the textbook.

• Since they can be also appeared on quizzes and exams, you need to check lecture
slides carefully.

• There are some typos and errors in the textbook, so be careful!

7
Recommended Study Methods

• I strongly recommend you to read the textbook before/after the lectures.

• For each week, we will cover one chapter of the textbook mostly.
• Each chapter consists of 25~30 pages.
• I guess it will take 2~3 hours to read and understand the contents.
• Most of the key contents will be covered during the lectures.
• Some details might not be covered, but still that could appear in the
quizzes and exams.

8
Recommended Study Methods

• There are two options for reading the textbook,

• If you think it's not di cult or want to spend more time on this course,
• Read it before the lecture, and try to answer some questions by yourself.
• If it is di cult to read and understand the contents,
• Read it after the lecture, and check if you understand the contents
correctly.

• In either way, do not simply skim de nitions and examples only.

9
ffi
ffi
fi
Recommended Study Methods

• I also recommend you to try some of the questions in exercises.

• Since this course is a theory course, and objectives are to understand those
theories.

• You can more rmly understand Automata and Formal Language by solving
those problems.

• More importantly, they will be on the quizzes and exams!

• Even if they won't, you can see how the quizzes and exams will be like.

10
fi
Evaluation
Attendance + Quizzes + Exams

• Attendance - only for pass / fail.

• Quiz + Assignment 20% - 10% for each.
• Exams
• Mid-term Exam 40%
• Final Exam 40%
• Note that if you missed more than 1/3 of the attendances, you will get an F for
this course (university regulation).

• Please remind that the quiz and exams will be also counted as attendances.
11
Quiz, Assignment and Exams

• Quiz
• Several problem solving questions.
• Questions will be about contents you've learned so far.
• Assignment
• A programming assignment in Python to simulate some automata we will
discuss during lectures.

• Questions will be given in English, but you can answer them either in English
or Korean.
12
Quizzes and Exams

• Exams
• Mid-term Exam - Weeks 1 ~ 7
• Final Exam - Weeks 1 ~ 14, but mostly from Weeks 9 ~ 14.
• Questions will be from various sources, even from the previous ones.
• Quiz - 10% / Exercises - 40%
• Unknown Mysterious Boxes - 50%
• Sometimes I'll ask simple things I mentioned during the lectures.
• e.g.) Who is this guy?
• Answers for the quiz and exercises will not be given.
13
Q&A

14
Introduction and Preliminaries

15
Topics

• Introduction of Formal Language and Automata Theory

• Mathematical Preliminaries and Notations.
• Three basic concepts in the theory of computation.
• Formal Language
• Grammar
• Automaton

16
Introduction to Formal Language
and Automata Theory

17
Why Do We Need to Study FLAT?

• Actually, you don't.

• Most of the topic in this course will not directly improve your skills in
programming.

• Might not be useful for getting a job within a year.

• This is a course to expand your way of thinking,
• and hopefully to be smarter!
• However, you need to put a lot of e ort to achieve that.

18
ff
Key Topics
Syntax Definition for
Programming Languages

Formal Languages Context-Free Computational

& Grammars Languages Complexity

Algorithm &
Problem Difficulties

Regular Languages
Turing Machine
String Pattern & Finite Automata
Matching in
Programming

19
What will be discussed in this course?
and what you should know after taking this course.

• There are three key concepts we will discuss in this course.

• Automaton
• models the hardware of a computer.
• Formal Language
• is related to programming language.
• Mechanical Computation
• is related to which can be done and cannot be done by a computer.
20
Automaton

• Automaton (pl., automata)

• a construct that possesses all the indispensable features of a digital
computer.

• accepts input, produces output.

• may have some temporary storage.
• can make decisions.

21
Formal Language and Grammar

• Formal language
• an abstraction of the general characteristics of programming languages.
• consists of a set of sentences, formed from a set of symbols, according to formal
rules.

• Grammar
• a set of rules for generating sentences in a formal language.
• We can learn some essential features of programming languages from formal languages.
• How to read and process strings (or code) written in a certain languages.

22
Mechanical Computation

• Algorithm
• describes how mechanical computation can be done to solve problems.
• We will discuss the kinds of problems, which can be solved by mechanical
means.

• What can be done by a computer?

• Or what cannot be done by a computer?
• How complex is it to be solved?

23
Mathematical Preliminaries and
Notations

24
Sets

• A set is a collection of elements with membership.

• Membership
• x ∈ S: x is an element of S.
• x ∉ S: x is not in S.
• S = {0, 1, 2} or S = {2, 4, 6, ...}
• More explicit notation
• S = { i : i > 0, i is even }
25
 Sets
Set operations

• Union (∪) • Universal Set U and Empty set ϕ

C
• S1 ∪ S2 = {x : x ∈ S1 or x ∈ S2} • Complementation (S )
C
• Intersection (∩) • S = {x : x ∈ U, x ∉ S}
S
• 1 ∩ S2 = {x : x ∈ S1 and x ∈ S2 } • DeMorgan's laws
C C C
• Di erence (−) • (S1 ∪ S2) = S1 ∩ S2
C C C
• S1 − S2 = {x : x . ∈ S1 and x ∉ S2} • (S1 ∩ S2) = S1 ∪ S2

26
ff
Sets

• Subset, Proper subset

S
• S1 ⊆ S2, S1 ⊂ S2. • Powerset (2 )
• Disjoint • The set of all subsets of a set S.
• if S1 ∩ S2 = ϕ, then S1 and S2 are disjoint. • S = {a, b, c}
• Finite set S S
• 2 = { , {a}, {b}, {c}, {a, b}, {a, c}, {b,
c}, {a, b, c} }.
• S contains a nite number of elements.
S |S| 3
• |2 | = 2 = 2 = 8.
• | S | = the size of S, i.e., the number of
elements in S.

27
𝜙
fi
Sets

• Cartesian product (×)

• S = S1 × S2 = {(x, y) : x ∈ S1, y ∈ S2}
• e.g.) S1 = {2,4} and S2 = {2,3,5,6}.
• S1 × S2 = {(2,2), (2,3), (2,5), (2,6), (4,2), (4,3), (4,5), (4,6)}.
• Note that its elements are ordered pairs, not sets.
• General version
• S1 × S2 × … × Sn = {(x1, x2, …, xn) : xi ∈ Si}.
28
Sets

• Subsets of S, S1, S2, …, Sn, are Partition of S if the following holds.

• The subsets S1, S2, …, Sn are mutually disjoint;
• S1 ∪ S2 ∪ … ∪ Sn = S;
• None of Si is empty.
• Simply saying, non-empty subsets S1 ∼ Sn are partition of S,
• if there is no overlap and combining them produces S.
29
Functions and Relations
Function

• A function is a rule that assigns

elements of one set to elements
of another set. S1 S2
f
• It's a rule of connecting
elements between two sets.

• f : S1 → S2
• If the domain of f is all of S1, then domain range
f is a total function on S1;
otherwise it's a partial function.
30
Functions and Relations
Order-of-magnitude Notation

• Let f(n) and g(n) be functions whose domain is a subset of positive integers.
• If there exists a positive constant c, s.t. for all su ciently large n
• f(n) ≤ c|g(n)|, we say that f has order at most g.
• f(n) = O( g(n) )
• Similarly, if |f(n)| ≥ c|g(n)|, then f has order at least g,
• f(n) = ( g(n) )
• if c1|g(n)| ≤ |f(n)| ≤ c2|g(n)|, then f and g have the same order of magnitude,
• f(n) = ( g(n) )
31
𝛩
𝛺
ffi
Functions and Relations
Order-of-magnitude Notation
f(n) ≤ c|g(n)| => 2n2 + 3n ≤ c|n3|
• Let
n = 100?
• f(n) = 2n2 + 3n,
• g(n) = n3, 2*100*100 + 3*100 ≤ c*1,000,000

• h(n) = 10n2 - 100. |g(n)| ≥ c|h(n)| => |n3| ≥ c|10n2 - 100|

• Then n = 2?
• f(n) = O( g(n) ), |2*2*2| ≥ c*|10*2*2 - 100| = c*60
It is about the growth rate,
not about the exact numbers.
• g(n) = ( h(n) ), n = 20?
• f(n) = ( h(n) ). |20*20*20| ≥ c*|10*20*20 - 100|
32
𝛩
𝛺
Functions and Relations
Relation

• Some functions can be represented by a set of pairs,

• {(x1, y1), (x2, y2), …},
• where xi is an element in the domain, and yi is the corresponding value in the range.
• Each xi can occur at most once.
• There is only one output for an input value. e.g.) f(1) = 2.
• Otherwise, the set is called relation.
• e.g.) f(1) = 2, f(1) = 3.

33
 Functions and Relations
Relation

• Equivalence: generalization of equality.

• A pair (x, y) in an equivalence relation:
• x≡y
• A relation is considered as an equivalence relation, if it satis es re exivity,
symmetry, transitivity rules.

• Re exive relation: x ≡ x for all x.

• Symmetric relation: if x ≡ y, then y ≡ x.
• Transitive relation: if x ≡ y and y ≡ z, then x ≡ z.
34
fl
fi
fl
Functions and Relations
Relation

• Example 1.4
• For non-negative integers, we can de ne a relation x ≡ y, i . x mod 3 = y mod 3.
• Is this an equivalence relation?
• Obviously 2 ≡ 2.
• 2 ≡ 5 ⟷ 5 ≡ 2, since 2 mod 3 = 5 mod 3 = 2.
• if 3 ≡ 6 and 6 ≡ 9, then 3 ≡ 9.
• How about for all integers, x ≡ y, i . x2 = y2?
35
ff
fi
ff
Graphs and Trees
Graph

• A graph is a construct consisting of two nite sets: vertices and edges.

• Vertices V = {vi, v2, …, vn}
• Edges E = {e1, e2, …, en}, where ei = (vj, vk) is an edge from vj to vk.
• It is actually a directed graph since edges have directions.
• A Graph G w/ V = {v1, v2, v3}, E = {(v1, v3), (v3, v1), (v3, v2), (v3, v3)}.

36
fi
Graphs and Trees
e2 e4

• A walk is a sequence of edges. e3

• A path is a walk without

repeated edges. e1

• A path is simple, if no vertex is e1, e2, e1, e3 is a walk from v1 to v2

repeated.
e1, e3 is a path from v1 to v2
• A cycle is a path from vi to vi. e1, e2 is a cycle
• A loop is an edge from vi to vi. e4 is a loop

37
Graphs and Trees
Trees

• A tree is a particular type of graph.

• a directed graph w/ no cycles.
• there is exactly one path from
the root to every other vertex.

• Parent, Child, Leaf nodes.

• level(v) = the number of edges
from the root.

• height(T) = max(level(vi)), vi ∈ T
38
Proof Techniques
Proof by Induction

• Suppose we have a sequence of statements P1, P2, ... - may be in nite - and
we want to prove that they are true.

• We can use induction like the following.

• For k ≥ 1, P1, P2, …, Pk (basis) are true.
• Assume that for n ≥ k, P1, P2, …, Pn are true. (inductive assumption)
• Then Pn+1 is also true. (inductive step)

39

fi
Proof Techniques
Proof by Induction

• The key is that we can prove each statement incrementally from the start.
• P1 (basis) is true and inductive assumption holds,
• then "proving inductive step is correct" automatically proves that
• P2 is true, then P3 is true, ...
• For k ≥ 1, P1, P2, …, Pk. (basis)

• Assume that for n ≥ k, P1, P2, …, Pn are true. (inductive assumption)

• Then Pn+1 is also true. (inductive step).

40
Proof Techniques
Example 1.5

n
• Prove that a binary tree of height n has at most 2 leaves.
• l(n): the maximum number of leaves of binary tree w/ height n.
n
• Then we need to show that l(n) ≤ 2 .
0
• Basis: l(0) = 1 = 2 .
k
• Inductive Assumption: l(k) ≤ 2 , for k = 0, 1, ..., n.
n n+1
• Inductive Step: Now we need to check if l(n) ≤ 2 is true, then l(n + 1) ≤ 2
is also true.

41
Proof Techniques
Example 1.5

n
• Inductive Step: If l(n) ≤ 2 is true, then
l(n + 1) ≤ 2n+1 is also true.
• We know that it's a binary tree, so every
node (and leaf) has at most 2 child nodes. height n

• i.e., l(n + 1) ≤ 2 × l(n). ............

• From inductive assumption, .......

n n+1
• l(n + 1) ≤ 2 × l(n) ≤ 2 × 2 = 2 .

• Thus, if inductive assumption is true for How many more nodes can be
n, then it is also true for n+1. attached to the current leaves?
n
• This proves l(n) ≤ 2 for all n.
42
 Proof Techniques
Proof by Contradiction

• Suppose we want to prove P is true.

• We rst assume that P is false, then show that it contradicts to Q which we
already know.

• Hence to resolve this contradiction, P must be true.

• This is quite useful when it is di cult to nd examples for proof,
• but you can easily nd some counter examples.

43
fi
fi
ffi
fi
Proof Techniques
Example 1.7

• Show that 2 is irrational.

• A rational number can be expressed as n/m, for n, m are two integers w/ no

common factor. A real number which is not rational is irrational.
n
• Assume that 2 is rational. Then we can write 2 = , where n, m are two
m
integers w/o common factors.
2 2
• Multiply m and take squares on both side gives 2m = n .

44
Proof Techniques
Example 1.7

2 2
• 2m = n (1).
• Then n must be even, so that we can write n = 2k.
2 2
• Substituting this to n from eq. (1), we get m = 2k .
• Now m is also even, which contradicts to the assumption that n and m have
no common factors.

• Hence such n, m cannot exist, and 2 is irrational.

45
Summary

• What you should check from this lecture.

• Sets and related operations and notations.
• What is the function and relation.
• Terms for graphs and trees.
• Understand and practice the two proof techniques.
• Check out how many "Show that" and "Prove that" questions in exercises.
n
• Can you prove that n! ≤ n for all positive integer n?
46
Three Basic Concepts

47
Topics

• Introduction for the key concepts in this course.

• Mathematical Preliminaries and Notations.
• Three basic concepts in the theory of computation.
• Formal Language
• Grammar
• Automaton

48
Languages

• Natural Languages
• Korean, English, Arabic, Chinese, Vietnamese, Spanish, Russian, German, and
many more.

• But, what is a "language"?

• It is about the system of communication which people use.
• This is not a precise de nition we can use for studying formal languages.
• How can we distinguish one language from other languages?
• How can we say that something belongs to a language and others don't?
49
fi
String

• Alphabet (Σ): a nite, non-empty set of symbols.

• String: a sequence of symbols from the alphabet.
• e.g.) Σ = {a, b}, then abb, abab, aabbaa are all strings on Σ.
• Lowercase letters a, b, c, ... will be used for symbols and u, v, w for string
names in the textbook.

• e.g.) w = abaaa, v = abcba, etc.

50
fi
String

• Concatenation of two strings w • Reverse of a string w.

and v.
R
• w = an⋯a2a1
• If w = a1a2⋯an and
v = b1b2⋯bm, • Length of a string w = |w|.
• the concatenation is the string, • Empty string λ: | λ | = 0
• wv = a1a2⋯anb1b2⋯bm. • Also, λw = wλ = w.

51
String

• Substring, Pre x, and Su x.

• w = vu
• v and u are substrings of w.
• v is a pre x of w.
• u is a su x of w.
• |w| = |u| + |v|, for any u and v.
• Check Example 1.8 of the textbook for proof.
52
ffi
fi
fi
ffi
String

• Length of a string
• | a | = 1, (single character)
• | wa | = | w | + 1, for all a ∈ Σ and w, any string on Σ.
n
• w : a string obtained by repeating w for n times.
0
• w = λ, for all w.

53
Sentence

• Σ*: strings obtained by concatenating zero or more symbols on Σ.

+
• Σ = Σ* − {λ}.
• The set Σ* is always in nite.
• A language de ned on Σ is a subset of Σ*.
• A string in a language L is called a sentence of L.

54
fi
fi
Language
Example 1.9

• Let Σ = {a, b}.

• Σ* = {λ, a, b, aa, ab, ba, bb, ⋯}.
• The set {a, aa, aab} is a nite language on Σ.
• The set
n n
• L = {a b : n ≥ 0} is an in nite language on Σ.
• aabb ∈ L, aab ∉ L.
• Note that languages are sets, and sentences are their elements.
55
fi
fi
Language

• We can use set operations (e.g., union, intersection, etc.) w/ languages too.
• Complement of L
c
• L = Σ* − L.
• Reverse of L
R R
• L = {w : w ∈ L}.
• Concatenation
• L1L2 = {xy : x ∈ L1, y ∈ L2}.
56
Language

n
• L : L concatenated n times.
0
• L = {λ} - a language has an empty string as its only sentence.
1
• L = L.
• Star-closure
0 1 2
• L* = L ∪ L ∪ L ⋯.
• Positive closure
+ 1 2
• L = L ∪ L ⋯ = L* − {λ}.
57
 Grammar
De nition 1.1

• A grammar G is de ned as a quadruple

• G = (V, T, S, P), where
• V is a nite set of variables,
• T is a nite set of terminal symbols,
• S ∈ V is the start symbol,
• P is a nite set of productions (or production rules).
58
fi
fi
fi
fi
fi
Grammar
Production Rules

• P. rule: x → y • w derives z, or z is derived from w.

+ • Multiple derivation is also possible.
• where x ∈ (V ∪ T) ,
y ∈ (V ∪ T)*.
• w1 ⇒ w2 ⇒ ⋯ ⇒ wn.
• Apply it to string w = uxv gives a
new string, • Then we say w1 derives wn.

• z = uyv. • It can be written as

• It can be written as w ⇒ z. • w1 ⇒* wn.

59
 Language generated by Grammar
De nition 1.2

• Let G = (V, T, S, P) be a grammar.

• Then the set
• L(G) = {w ∈ T* : S ⇒* w}
• is the language generated by G.
• The sentences of language L(G) can be derived from S.
• It only contains terminal symbols of G.

60
fi
Language and Grammar
Example 1.12

• Find a grammar that generates

n n+1
• L = {a b : n ≥ 0}
• Need to nd a grammar, which have production rules to concatenate the
same number of a/b, and add an extra b at the end.

• G = (V, T, S, P)

61
fi
Language and Grammar
Example 1.12

n n+1
• L = {a b : n ≥ 0}
• G = ({S, A}, {a, b}, S, P) with productions P is the following.
• S → aSb
• S → Ab
• A→λ
• S ⇒* aabbb?
62
Equivalence

• Normally, a given language has many grammars which generate it.

• If two grammars G1 and G2 generate the same language, then we say that
these two grammars are equivalent.

• L(G1) = L(G2).
• e.g.) the grammar we've found in the previous slide is equivalent to the
grammar of Example 1.12 in the textbook.

63
Equivalence
Example 1.14

• Consider the grammar G1 = ({S, A}, {a, b}, S, P1), with P1 consisting of productions
• S → aAb | λ
• A → aAb | λ
• This grammar is equivalent to the grammar G = ({S}, {a, b}, S, P) with P is given by
• S → aSb | λ
• Both of them generate
n n
• L(G) = L(G1) = {a b : n ≥ 0}.
64
Automata

• An automaton is an abstract model of a digital

computer.

• The input is a string over a given alphabet,

written on read-only Input le.

• It can produce an output.

• It may have storage, which can hold
symbols.

• It also has control unit, which can be in

any on of a nite number of internal states.

65
fi
fi
 Automata

• An automaton is assumed to operate in a discrete timeframe.

• The internal state of the control unit can change for di erent time steps.
• it is determined by the next-state or transition function.
• Con guration: a particular state of automaton.
• Move: the transition of the automaton from one con guration to the next.
• Deterministic: each move is uniquely determined by the current con guration.
• Nondeterministic: there are several possibilities.
• Accepter - yes or no, Transducer - produces a string.
66
fi
fi
ff
fi
Automata
Example 1.16

• Fig. 1.6 is an automaton

accepting all legal C identi ers.

• Each vertex represents an

internal state.

• An input string is read from left

to right one by one.

• Edges represent transitions

between internal states, based Fig. 1.6
on input symbols.

67
fi
Summary

• You have to get familiar with the notations for languages and grammars.
• We will use them a lot in future lectures.
• It takes a huge part of what you have to write during quizzes and exams.
• You need to understand the connections between languages and grammars.
• If you see a grammar, you can see a language generated by it.
• If you see a language, you can infer a grammar to derive it.

68