AUTOMATA AND COMPLEXITY THEORY

Automata theory is a branch of computer science fundamental limits and capabilities of

that deals with the study of abstract machines computation. It provides a theoretical
and their computational abilities. It focuses on foundation for designing and analyzing
understanding and characterizing the behavior of algorithms and systems.
these machines, called automata, which can be
2. Algorithm design and optimization:
thought of as mathematical models for
Complexity theory guides us in developing
computation. Automata theory provides a formal
efficient algorithms by identifying the
framework for describing and analyzing various
best-known algorithms for various
computational processes, including algorithms,
problems. It helps us analyze the time
programming languages, and systems.
and space requirements of algorithms and
Automata are often classified into different types optimize them for practical use.
based on their capabilities. For example, finite
3. Problem classification: Complexity theory
automata are machines with a limited amount of
provides a classification scheme for
memory that can recognize patterns in strings or
problems based on their inherent
languages. Pushdown automata can handle more
difficulty. This classification helps us
complex languages by using a stack as additional
understand which problems are feasible to
memory. Turing machines, the most powerful
solve within reasonable time and resource
type of automaton, can simulate any algorithmic
constraints.
process and serve as a theoretical foundation for
studying the limits of computation. 4. Practical applications: Automata theory
and complexity theory have practical
Complexity theory, on the other hand, deals with
applications in various fields, including
the study of the inherent complexity of
computer science, mathematics,
computational problems. It aims to understand
linguistics, artificial intelligence,
the resources (such as time and space) required
cryptography, and optimization. They
to solve problems as the input size grows.
provide essential tools and concepts for
Complexity theory provides a framework for
solving real-world problems efficiently.
classifying problems into different complexity
classes based on their difficulty and the resources Overall, automata theory and complexity theory
needed to solve them. form the basis of theoretical computer science
and play a crucial role in understanding
One of the fundamental concepts in complexity
computation, designing efficient algorithms, and
theory is the notion of computational complexity,
classifying and solving problems in various
which measures the efficiency of algorithms. This
domains.
is often expressed using big-O notation, which
characterizes the worst-case behavior of an FINITE STATE MACHINES
algorithm in terms of its input size. Complexity
theory helps us analyze and compare different A finite state machine (FSM), also known as a
algorithms, identify optimal or near-optimal finite automaton, is a mathematical model used
solutions, and determine the feasibility of solving to describe the behavior of systems that can be
problems within practical limits. in a limited number of states. In simpler terms, it
is like a machine that goes through a series of
We need automata theory and complexity theory
states based on the input it receives.
for several reasons :-
To understand FSMs, imagine a vending machine.
1. Understanding computation: Automata
It has different states: idle, waiting for coins,
theory helps us understand the
selecting a product, dispensing the product, etc.
These states represent the different stages the up of letters from an alphabet. For
machine can be in. The transitions between states example, the string "hello" consists of the
are triggered by specific events, such as inserting symbols 'h', 'e', 'l', 'l', and 'o' from the
coins, pressing buttons, or completing a English alphabet.
transaction.
4. Language: In automata theory, a language
In an FSM, you have a set of states, a set of is a set of strings formed from an
possible inputs or events, and a set of rules that alphabet. It represents a collection of
determine how the machine transitions from one words or sentences. For instance, the
state to another based on the current state and language of all valid email addresses can
the input it receives. These rules are often be represented by a set of strings like
represented by a diagram called a state transition "john@example.com,"
diagram or a state transition table. "alice@gmail.com," etc.
The FSM starts in an initial state and processes
5. Power of Sigma: The power of Sigma
inputs one at a time, transitioning from one state
refers to the number of symbols in an
to another according to the defined rules. The
alphabet. Sigma is often used to represent
machine can also have final or accepting states,
an alphabet. So, the power of Sigma
which indicate that a certain condition or goal
indicates the size or cardinality of the
has been reached.
alphabet. For example, if Sigma is {0, 1},
FSMs are used to model and solve problems in the power of Sigma is 2(Σ2) because there
various areas, such as software engineering, are two symbols in the alphabet.
control systems, natural language processing, and
Σ^1: This represents the power of Σ (the
protocol design. They provide a simple yet
alphabet) raised to the exponent 1. In
powerful framework for describing the behavior
simpler terms, it refers to the set of all
of systems with a finite number of states and
individual symbols in the alphabet. For
well-defined transitions between them.
example, if Σ is {a, b, c}, then Σ^1 is
CORE TERMINOLOGY OF AUTOMATA {a, b, c} itself.

Σ^2: This notation represents the power

Here are some core terminologies of automata of Σ raised to the exponent 2. It refers to
defined in simpler terms: the set of all possible combinations of two
1. Symbol: A symbol is a basic unit of symbols from the alphabet. For example,
information used in automata. It can be if Σ is {0, 1}, then Σ^2 would include
any individual character, number, or strings like {00, 01, 10, 11}.
object. For example, in a language, Σ*: This represents the Kleene closure or
symbols can be letters of the alphabet, the star operation on Σ. In simpler terms,
digits, or punctuation marks. it refers to the set of all possible strings
2. Alphabet: An alphabet is a finite set of that can be formed using symbols from
symbols from which strings are formed. It the alphabet, including the empty string.
represents the collection of all possible For example, if Σ is {a, b}, then Σ*
symbols that can be used in an would include strings like {ε (empty
automaton. For instance, the alphabet of string), a, b, aa, ab, ba, bb, aaa, ...}.
English language text would consist of the Σ^+: This notation represents the positive
letters A to Z. closure of Σ. It is similar to Σ*, but it
3. String: A string is a sequence of symbols excludes the empty string. In other words,
from an alphabet. It is like a word made it refers to the set of all non-empty
 strings that can be formed using symbols 7) State: A state represents a particular
from the alphabet. For example, if Σ is condition or situation in an automaton. It
{0, 1}, then Σ^+ would include strings indicates the current "mode" or "state" of
like {0, 1, 00, 01, 10, 11, 000, ...}. the machine. For instance, in a traffic
light system, the states can be "green,"
6) Grammar: A grammar is a set of rules
"yellow," and "red."
or guidelines for constructing valid
sentences or strings in a language. It tells 8) Transition: A transition defines a
us how to arrange symbols to form change from one state to another in an
meaningful structures. automaton based on an input symbol. It
specifies how the machine moves from
For example, imagine you have a grammar for
one condition to another. For example,
creating sentences in a made-up language called
when a coin is inserted into a vending
"FunSpeak." The grammar might have rules
machine, it transitions from the "idle"
like :-
state to the "waiting for coins" state.
1. A sentence starts with a noun.
9) Accepting State/Final State: An
2. A noun can be followed by a verb.
accepting state, also known as a final
3. A verb can be followed by an object.
state, is a designated state in an
With these rules, you can create valid sentences automaton that indicates the successful
in FunSpeak by following the grammar. For completion or acceptance of a string. If
instance: the machine reaches an accepting state, it
means the input string is recognized or
• Noun: "Dog"
accepted by the automaton.
• Verb: "runs"
• Object: "fast" Combining them according These are some key terminologies in automata
to the grammar, you can create the theory. Understanding these concepts helps in
sentence "Dog runs fast." analyzing and designing automata to solve
various computational problems.
Grammars are commonly used in languages,
programming, and linguistics to describe the DETERMINISTIC FINITE AUTOMATA
structure and rules of a language. They provide a
framework for generating and understanding valid A deterministic automaton, also known as a
sentences or strings. deterministic finite automaton (DFA), is a type of
finite state machine that follows a set of clear
In summary :-
and unambiguous rules for state transitions. In
• Σ^1 represents the individual symbols in simpler terms, it is a machine that always knows
the alphabet. exactly what to do based on the current state and
• Σ^2 represents all possible combinations the input it receives.
of two symbols from the alphabet.
Here's a breakdown of what makes a
• Σ* represents all possible strings formed
deterministic automaton:
using symbols from the alphabet,
including the empty string. 1. States: A DFA has a set of states, each
• Σ^+ represents all non-empty strings representing a particular condition or
formed using symbols from the alphabet. mode of the machine. These states are
distinct and well-defined.
These notations are commonly used in formal
languages and automata theory to describe and 2. Transition Rules: The DFA has precise
analyze the sets of strings that can be recognized rules that determine how it transitions
or generated by automata. from one state to another based on the
current state and the input symbol. These
 rules are deterministic, meaning that for a Example :-
given state and input, there is only one
possible next state.

3. Deterministic Transition Function: The

DFA uses a transition function, often
denoted as δ (delta), which takes a state
and an input symbol as arguments and
returns the next state. The transition
function always produces a unique result.

4. Deterministic Behavior: When the DFA

receives an input symbol, it follows the The Five Tuples
transition rules and moves from one state In a finite state machine (FSM), also known as a
to another, updating its current state. It finite automaton, the behavior of the machine is
does not require any guessing or defined by a five-tuple. Each component of the
backtracking. tuple provides essential information about the
5. Accepting States: A DFA can have one or FSM. Here's a breakdown of the five
more accepting states. If the machine components :-
reaches an accepting state after processing 1. Q (Set of States) :- Q represents the set of
a sequence of input symbols, it indicates all states in the FSM. States are distinct
that the input is accepted or recognized conditions or modes in which the machine
by the DFA. can be. For example, in a traffic light
system, the set of states can be {Green,
Yellow, Red}.

2. Σ (Alphabet/Input Symbols): Σ refers to

the alphabet or set of input symbols that
the FSM can accept. These symbols are
the inputs that drive the transitions
between states. For instance, in a simple
coin-operated vending machine, Σ might
be {Coin, Button}.

3. δ (Transition Function) :- δ defines the

The key characteristic of a deterministic
transition function that maps a state and
automaton is that it operates deterministically,
an input symbol to the next state. It
meaning it always makes precise and
specifies how the machine transitions from
unambiguous decisions based on the current state
one state to another based on the input it
and input symbol. This deterministic behavior
receives. For example, δ(Green, Coin) =
makes the DFA predictable and efficient for
Yellow indicates that if the FSM is in the
processing inputs.
state Green and receives a Coin input, it
Deterministic automata are widely used in various transitions to the state Yellow.
applications, such as pattern matching, lexical
4. q₀ (Initial State) :- q₀ represents the initial
analysis in compilers, regular expression
state of the FSM. It indicates the starting
matching, and protocol design. They provide a
condition or mode of the machine. When
simple and efficient model for recognizing and
the FSM begins processing input, it starts
validating strings in a deterministic manner.
from this initial state. For example, q₀ =
 Green indicates that the FSM initially 3. No memory: DFAs have a limited memory
starts in the Green state. capacity. They don't remember what
happened in the past or keep track of
5. F (Set of Accepting States/Final States) :-
long sequences of inputs. They only focus
F represents the set of accepting or final
on the current state and input.
states in the FSM. These states indicate
successful completion or acceptance of a 4. Limited states: DFAs have a finite number
particular string or input sequence. If the of states. It's like having a few different
FSM reaches any state in F after rooms to be in. Each state represents a
processing an input string, it means the specific situation or condition.
input is accepted. For instance, F = {Red}
5. Well-defined transitions: DFAs have clear
signifies that the Red state is the only
rules for transitioning between states.
accepting state in the FSM.
When they receive an input, they follow a
predetermined path to move from one
state to another. It's like following a set
of arrows to go from one room to
another.

6. Accepting or rejecting: DFAs can tell you

if an input belongs to a particular
language or not. If they reach a
designated accepting state after processing
an input, they accept it. Otherwise, they
reject it. It's like a machine that says
Together, the five-tuple (Q, Σ, δ, q₀, F) provides "Yes" or "No" depending on whether
a complete specification of a finite state machine. something meets the rules.
It defines the set of states, the input alphabet,
7. DFA has only one unique state :- In a
the transition function, the initial state, and the
deterministic finite automaton (DFA), for
accepting states, enabling the machine to process
a given current state and input symbol,
inputs and progress through different states based
there is only one unique next state. This
on the specified rules.
is one of the defining characteristics of
DFAs.

PROPERTIES OF DETERMINISTIC FINITE AUTOMATA When the DFA receives an input symbol while in
a particular state, it follows a specific transition
Here are some properties of deterministic finite rule that leads it to a single, predetermined next
automata (DFAs) explained in simpler terms: state. The transition rules of a DFA are
deterministic, meaning they leave no room for
1. Clear rules: DFAs follow straightforward
ambiguity or multiple possible outcomes.
and easy-to-understand rules. It's like a
game where you know exactly what This property of having a unique next state based
moves to make based on the current on the current state and input symbol makes
situation. DFAs easy to understand and analyze. It ensures
that the behavior of the DFA is well-defined and
2. Predictable behavior: DFAs always behave
predictable for any given input sequence.
in a predictable manner. They don't make
random choices or guesses. Every time These properties make DFAs simple and easy to
they receive an input, they know exactly understand. They operate in a step-by-step
which state to transition to. manner, always knowing what to do next based
on the input and their current state. DFAs are languages are useful in various tasks, like
often used to recognize and validate patterns in searching for words or patterns in text.
strings, making them useful in various
In simpler terms, a regular language is a set of
applications, such as text processing, compilers,
words or strings that can be described or
and language recognition.
recognized by a special type of machine called a
finite state machine (FSM) or a regular
expression.
REGULAR LANGUAGES
Think of a regular language as a collection of
patterns or rules that define which strings are
Imagine a regular language as a special club that
considered part of the language. These patterns
only allows certain words or strings to be part of
can be simple or complex, depending on the
it. This club has specific rules or patterns that
language. For example, the regular language of
determine whether a word is allowed or not.
all words that start with the letter "a" and end
Here are some key points about regular languages with the letter "b" can be described by the
1. Words that follow a pattern: A regular pattern "a...b", where the dots represent any
language is like a group of words that sequence of characters in between.
follow a specific pattern. Think of it like Regular languages are characterized by their
a secret code that only some words know simplicity and can be recognized by deterministic
how to follow. finite automata (DFAs) or non-deterministic finite
2. Simple patterns: These patterns are not automata (NFAs). These automata are machines
too complicated. They can be as simple as with a limited amount of memory that can
"words that start with 'a' and end with 't'" process input strings and determine if they belong
or "words that have exactly three letters." to the regular language.

3. Special machines: There are special Regular languages have many practical
machines called finite state machines that applications, such as in text processing, pattern
can check if a word follows the pattern. matching, and lexical analysis in compilers. They
These machines are like detectives that provide a fundamental framework for describing
examine each letter of the word to see if and working with patterns in strings, allowing for
it matches the pattern. efficient and concise representations of language
constraints.
4. Language membership: If a word matches
the pattern and is allowed in the club, we OPERATION ON REGULAR LANGUAGES
say it belongs to the regular language. It's Operations on regular languages are ways to
like being a member of a special group. combine or manipulate different regular languages
5. Useful in many things: Regular languages to create new languages. It's like playing with
are helpful in many areas. They can be building blocks to make new structures.
used to find specific words in a Here are some key operations on regular
document, validate email addresses, or languages explained in simpler terms :-
search for patterns in a large amount of
1. Union (U): Imagine you have two sets of
text.
toys. The union operation allows you to
In summary, a regular language is like a club combine the toys from both sets into one
with rules or patterns that words must follow to big set. In terms of regular languages, the
be part of it. Special machines, called finite state union operation combines two languages
machines, help us check if a word matches the to create a new language that contains all
pattern and belongs to the language. Regular the words from both languages. For
 example, if one language is {cat, dog} For example, let's say the original language is
and another language is {bird, fish}, the {cat, dog}. The complement of this language
union operation would give you a new would give you all the words that are not in the
language {cat, dog, bird, fish}. original language, such as {bird, fish}. It's like
saying, "Here are all the words that are not 'cat'
2. Concatenation(o) :- Concatenation is like
or 'dog'."
putting two puzzle pieces together. If you
have two strings or words, concatenation 6) Reverse of a Language (LR = {wR: w ϵ
allows you to join them to create a longer L}) : The reverse of a language is like
word. In regular languages, concatenation reading words backward. It's like
combines two languages to create a new reversing the order of the letters in each
language that consists of all possible word.
combinations of words from the original
For instance, consider the original language
languages. For example, if one language
{hello, world}. The reverse of this language
is {hello} and another language is
would give you {olleh, dlrow}. Each word in the
{world}, concatenation would give you
original language is reversed. It's like saying,
the language {helloworld}.
"Let's read the words backward."
3. Kleene Star (*): The Kleene star operation
Complement and reverse are additional operations
is like a magic wand that can make
that can be applied to regular languages to create
things repeat. If you have a language, the
new languages with different properties.
Kleene star operation allows you to create
Complement gives you the words not in the
a new language that includes all possible
original language, while reverse flips the order of
combinations of words from the original
letters in each word. These operations expand the
language, including the empty word. It's
possibilities of manipulating and exploring regular
like repeating and stacking the words
languages.
together. For example, if the original
language is {a}, the Kleene star operation These operations on regular languages are like
would give you the language {ε (empty tools that help you create new languages by
word), a, aa, aaa, ...}. combining or manipulating existing ones. Just
like how you can build different structures by
L* = L0 U L1 U L2…
playing with building blocks, operations on
regular languages allow you to create new
languages with different properties and patterns.
4) Positive Closure of L

L+ = L1 U L2 U L3…
NON-DETERMINISTIC FINITE AUTOMATA

A non-deterministic finite automaton (NFA) is a

type of finite state machine where multiple
transition rules can be applicable for a given
current state and input symbol. In simpler terms,
an NFA is like a machine that has more
flexibility and can make choices when deciding
• 5) Complement of a Language (L' = Σ* - its next state.
L) :- The complement of a language is
like flipping a switch. It gives you all the Here's a breakdown of NFA in simpler terms:
words that are not in the original 1. Multiple possible transitions: Unlike a
language. It's like looking at the "other deterministic finite automaton (DFA), an
side" of the language.
 NFA can have multiple transition rules for Formal Definition of Non-deterministic Finite
a given state and input symbol. It's like Automaton (NFA) :-
having several paths to choose from when
A non-deterministic finite automaton (NFA) is a
moving to the next state.
mathematical model represented by a 5-tuple (Q,
2. Guessing or branching: An NFA can make Σ, δ, q₀, F), where:
"guesses" or "branch" when deciding its
• Q is a finite set of states.
next state. It can explore different
• Σ is the input alphabet, a finite set of
possibilities simultaneously. It's like
symbols.
having multiple doors to open and
• δ is the transition function that maps a
deciding which one to choose.
state and an input symbol to a set of
3. Non-deterministic behavior: Because of the states or ε-transitions (empty string
multiple transition rules, the behavior of transitions).
an NFA is non-deterministic. It means • q₀ is the initial state.
that when an input symbol is received, • F is a set of accepting states.
the NFA may have several valid options
Simpler Explanation:
for its next state, and it can choose any
of them. An NFA is like a magical machine with different
rooms (states) that can move around based on the
4. Set of possible states: In an NFA, the next
symbols it receives. Here's a simpler breakdown:
state is not always unique. It can be a set
of states instead of a single state. This set 1. States: Imagine the NFA as a house with
represents all the possible states that the different rooms. Each room represents a
NFA can be in after processing the input specific state that the machine can be in.
symbol. For example, there can be rooms labeled
"A" "B," and "C."
5. Acceptance condition: Similar to a DFA,
an NFA can have accepting states. If any 2. Input symbols: The NFA understands
of the possible states in the set of next certain symbols or buttons. These symbols
states is an accepting state, then the input are part of the input alphabet. It can be
is accepted by the NFA. things like buttons labeled "0" and "1" or
different colors.
Non-deterministic finite automata provide a more
flexible and expressive way of defining languages. 3. Transition function: Each room has doors
They can represent more complex patterns and with labels on them. When the NFA
handle certain types of problems more efficiently receives a symbol, it checks the label on
than DFAs. However, their non-deterministic the door of the current room. The label
nature requires additional mechanisms, such as tells the machine which rooms it can
backtracking or exploring all possible paths, to move to. It can move to one or more
determine the acceptance of an input. rooms, or even stay in the same room.

It's important to note that non-deterministic finite 4. Initial state: The NFA starts its journey in
automata can be converted to equivalent one particular room. It's like the room it's
deterministic finite automata using specific initially placed in when you start playing
algorithms. with the machine.

5. Accepting states: Some rooms are special

and have a sign that says "Success!" on
the door. If the NFA ends up in one of
these rooms after following a series of
 symbols, it means it has successfully can read. These symbols can be letters,
recognized the input. These rooms are the digits, or any other valid input characters.
accepting states.
• 2^Q represents the power set of Q. The
Examples: power set of a set is the collection of all
possible subsets of that set. In this case,
Let's consider a simple NFA that recognizes
2^Q represents the set of all possible
strings with the pattern "ab" or "ac" in them.
subsets of states in Q. Each subset is a
• States (Q): {A, B, C} distinct combination of states.
• Input alphabet (Σ): {a, b, c}
Now, let's understand the transition function:
• Transition function (δ):
• δ(A, a) = {B, C} (if the NFA is in • The transition function δ is a function
state A and receives 'a', it can that takes two inputs: a state q from Q
move to states B or C) and an input symbol σ from Σ.
• δ(B, b) = {B} (if the NFA is in
• The output of the transition function is a
state B and receives 'b', it stays in
set of states, denoted by 2^Q. This set
state B)
contains the possible states that the NFA
• δ(C, c) = {C} (if the NFA is in
can transition to when it is in state q and
state C and receives 'c', it stays in
reads input symbol σ.
state C)
• Initial state (q₀): A (the NFA starts in To illustrate this, let's consider an example.
state A) Suppose we have an NFA with the following
• Accepting states (F): {B, C} (states B and components:
C are the accepting states)
• Q = {q0, q1, q2}: Set of states {q0, q1,
Using this NFA, if we input the string "abc", the q2}.
NFA will transition from A to B, then from B to • Σ = {0, 1}: Input alphabet {0, 1}.
C, successfully recognizing the pattern "ab" in the
Now, let's say we want to determine the
input.
transition for state q0 when the input symbol is
This is a simplified explanation of NFAs, but it 1. The transition function δ(q0, 1) will return a
captures the essential concepts of states, symbols, set of states that q0 can transition to when it
transitions, and accepting states in a more reads 1.
accessible manner.
For example, if δ(q0, 1) = {q1, q2}, it means
that from state q0, upon reading input symbol 1,
the NFA can transition to either state q1 or q2
TRANSITION FUNCTION OF NON-DETERMINISTIC
(or both).
AUTOMATA
This non-determinism in the transition function
The transition function of an NFA is defined as
allows the NFA to have multiple possible
follows:
transitions for a given state and input symbol
δ: Q × Σ → 2^Q combination, providing greater expressive power
than a deterministic finite automaton (DFA)
which has a single defined transition for each
• Q represents the set of states in the NFA.
state and input symbol pair.
Each state in Q can be represented by a
unique symbol or identifier.

• Σ is the input alphabet, which consists of

all possible input symbols that the NFA
CONSTRUCTION OF DFA indicate that the DFA recognizes a valid
string or pattern. For example, in the
Let's define the construction of a DFA binary string DFA, the "even" state might
(deterministic finite automaton) in detail: be the accepting state.

1. Understand the problem: The first step in 7. Handle remaining combinations: If there
constructing a DFA is to understand the are any remaining combinations of input
problem or language you want the DFA to symbols that have not been accounted for
recognize. Identify the patterns or rules by the existing transitions, direct them to
that define the valid strings in the a separate state called the "error state" or
language. For example, if you want to "dead state." This ensures that any
construct a DFA that recognizes binary unrecognized or invalid inputs are
strings with an even number of 1s, you handled appropriately.
need to understand the pattern of valid
8. Test and refine: Test the DFA with
binary strings.
various input strings to ensure it behaves
2. Determine the states: Based on the as expected and recognizes the desired
problem, determine the number of states patterns. If any issues or errors are
required in the DFA. The number of encountered, refine the DFA by adjusting
states can be determined by considering the transition function or state
the structure of the language or problem. designations.
For example, if you're constructing a DFA
By following these steps, you can construct a
for the language of even-length binary
DFA that recognizes the desired language or
strings, you might need two states to
pattern. The DFA acts as a machine with states,
represent even and odd lengths.
transitions, and accepting states, allowing it to
3. Define the alphabet: Identify the symbols process input strings and determine their validity
or inputs that the DFA can accept. This within the defined language.
set of symbols is known as the alphabet.
STEPS FOR CONSTRUCTION OF DFA
For a binary string DFA, the alphabet
would consist of the symbols {0, 1}. Following steps are followed to construct a DFA

4. Designate the initial state: Determine

Step – 01 :-
which state will be the starting or initial
state of the DFA. This is the state where ➔ Determine the minimum number of states
the machine begins processing input. required in the DFA

5. Define the transition function: Define the ➔ Calculate the length of sub string
transitions from one state to another
based on the input symbols. Create a ➔ All strings starting with “n” length sub
transition table or diagram that specifies string will always require minimum (n+2)
the next state for each combination of states in the DFA
current state and input symbol. For
Step – 02 :-
example, if the current state is "even"
and the input symbol is "0," the ➔ Decide the strings for which DFA will be
transition might lead to the "even" state constructed
again.
Step – 03 :-
6. Designate accepting states: Determine
which states in the DFA will be the • Construct a DFA for the strings decided in
accepting or final states. These states step 02
Step – 04 :- Example 2 :- Draw a DFA for the language
accepting strings starting with “a” over input
➔ Send all the left possible combinations to alphabets ∑ = {a, b}
the dead state

➔ Do not send the left possible combinations Solution-

over the starting state
Regular expression for the given language = a(a +
b)*
Example :- Draw a DFA for the language
accepting strings starting with ‘ab’ over input Step-01:
alphabets ∑ = {a, b}
•All strings of the language starts with
substring “a”.
Solution-
•So, length of substring = 1.
Regular expression for the given language =ab(a
+ b)*

Step-01: Thus, Minimum number of states required in the

DFA = 1 + 2 = 3
•All strings of the language starts with
substring “ab”. It suggests that minimized DFA will have 3
•So, length of substring = 2. states.

Thus, Minimum number of states required in the Step-02:

DFA = 2 + 2 = 4.
We will construct DFA for the following strings-
It suggests that minimized DFA will have 4
•a
states.
•aa
Step-02:
Step-03:
We will construct DFA for the following strings-
The required DFA is-
•ab

•aba

•abab

Step-03:

The required DFA is-

Example 3 :- Draw a DFA for the language

accepting strings starting with ‘101’ over input
alphabets ∑ = {0, 1}
Solution- Regular expression for the given REGULAR EXPRESSION
language =101(0 + 1)*
Regular expression is a powerful tool used for
Step-01: pattern matching and manipulating text. In
simpler terms, it's like a special code or language
•All strings of the language starts with
that helps you search for specific patterns or
substring “101”.
words within a larger text.
•So, length of substring = 3.
Here's a breakdown of regular expressions in
Thus, Minimum number of states required in the simpler terms :-
DFA = 3 + 2 = 5.
1. Pattern matching: Regular expressions
It suggests that minimized DFA will have 5 allow you to describe patterns in text by
states. using special characters and symbols. It's
like a secret code that tells you how to
Step-02: find certain words or patterns in a
We will construct DFA for the following strings- haystack of text.

2. Flexible search: With regular expressions,

•101
you can search for more than just literal
•1011 words. You can specify patterns like

•10110 "words that start with 'A' and end with

'B'," or "numbers that are three digits
•101101 long." It gives you the flexibility to find
Step-03: various types of patterns in text.

The required DFA is- 3. Special characters: Regular expressions use

special characters to represent different
elements in a pattern. For example, the
dot (.) represents any character, the
asterisk (*) means "zero or more
occurrences," and the plus sign (+) means
"one or more occurrences." These
characters form the building blocks of
regular expressions.

4. Examples :- Here are a few examples to

demonstrate the power of regular
expressions :-
CONSTRUCTION OF NFA
• Searching for all email addresses
in a document: Using a regular
CONVERSION FROM NFA TO DFA expression pattern like "[a-zA-Z0-
9]+@[a-zA-Z0-9]+.[a-zA-Z0-9]+"
can help find email addresses.
MINIMIZATION OF DFA

• Extracting phone numbers from a

webpage :- A regular expression
pattern like "\d{3}-\d{3}-\d{4}"
can match phone numbers in the
format "###-###-####."
Why do we use regular expressions?

Regular expressions are widely used in various

applications and programming languages because
they provide a concise and powerful way to
perform pattern matching and text manipulation
tasks. Here are a few reasons why we use regular
expressions :-
CONSTRUCTION OF FA FROM RE
1. Pattern searching: Regular expressions
help us efficiently search for specific
patterns or words within a large amount
of text. They save time and effort
compared to manually scanning through
the text.

2. Data validation :- Regular expressions are

useful for validating input data. We can
define patterns that the input must match,
such as a specific format for dates or
email addresses, and use regular
expressions to check if the input adheres
to the defined pattern.

3. Text manipulation: Regular expressions

enable us to manipulate text by replacing
or transforming specific patterns or
substrings. For example, we can replace
all occurrences of a word, remove
unwanted characters, or extract specific
information from a text document.

Other ways of pattern matching :-

Before regular expressions, pattern matching was

often done using custom algorithms or string
manipulation functions provided by programming
languages. While these methods could work, they
often required more code and were less flexible
compared to regular expressions.

Regular expressions provide a standardized and

widely adopted approach to pattern matching,
making it easier to write and understand code.
They offer a concise and powerful way to work
with patterns, resulting in more efficient and
maintainable code.

Example :-
FORMAL LANGUAGE , FORMAL GRAMMER AND • Formal grammars are used to
AUTOMATA generate or recognize strings in a
formal language, depending on the
type of grammar (e.g., regular
The relationship among formal languages, formal grammar, context-free grammar).
grammars, and automata lies at the core of • Context-free grammars are
theoretical computer science and computational particularly important as they are
linguistics. These concepts are interconnected and widely used in the description of
provide a foundation for understanding the programming languages and in
processing and generation of languages by parsing natural languages.
computational devices. Let's define each of these
in detail and explore their relationships :- 3. Automata :-
1. Formal Languages :- • An automaton is a computational
• A formal language is a set of model that reads an input string
strings composed of symbols from and transitions between states
a given alphabet. based on the input symbols it
• The alphabet is a finite set of reads.
symbols or characters that form • Automata can be categorized into
the building blocks of the different types based on their
language. capabilities, such as finite
• Formal languages are abstract automata, pushdown automata,
representations used to describe and Turing machines.
various types of languages, • Finite automata are simple
including natural languages like machines with a fixed set of states
English, programming languages that can recognize regular
like C++ or Python, and languages, which are described by
mathematical languages like regular grammars.
regular expressions or context-free • Pushdown automata can handle
grammars. context-free languages, which are
• Formal languages are essential for described by context-free
defining patterns, rules, and grammars, and are more powerful
structures within languages, and than finite automata.
they find applications in various • Turing machines are the most
areas of computer science and powerful computational model,
linguistics. capable of recognizing recursively
enumerable languages, which can
2. Formal Grammars :- be described by context-sensitive
grammars or recursively
• A formal grammar is a set of rules enumerable grammars.
that define the structure and
syntax of a formal language. Relationships :-
• It consists of a set of production • Formal languages are generated or
rules that specify how symbols described by formal grammars. Each type
from the alphabet can be of formal grammar corresponds to a
combined to form valid strings in specific class of formal languages. For
the language. example, regular grammars generate
regular languages, context-free grammars
 generate context-free languages, and so CHAPTER THREE
on.
• Automata can recognize or decide whether REGULAR GRAMMER
a given string belongs to a particular
formal language. Each type of automaton Formal grammars are mathematical models used
corresponds to a specific class of formal to describe the syntax or structure of formal
languages. For example, finite automata languages. They provide a set of rules for
recognize regular languages, pushdown generating valid strings in a language or for
automata recognize context-free languages, parsing and analyzing the structure of strings.
and Turing machines recognize recursively
Formal grammars consist of the following
enumerable languages.
components :-
• There is a strong connection between
formal grammars and automata through 1. Terminal Symbols: These are the basic
the Chomsky hierarchy, which classifies elements or atomic units of the language.
grammars and languages into four types: They represent the actual symbols that
Type 3 (regular), Type 2 (context-free), appear in valid strings. For example, in a
Type 1 (context-sensitive), and Type 0 programming language, terminal symbols
(recursively enumerable). Each type of could be identifiers, keywords, operators,
grammar corresponds to a specific class of or punctuation marks.
automaton with equivalent computational
2. Non-terminal Symbols: These symbols are
power.
placeholders that represent sets of strings
In summary, formal languages, formal grammars, in the language. They are used to define
and automata are interconnected concepts that rules for generating or transforming
provide a formal and mathematical foundation for strings. Non-terminal symbols are typically
understanding the structure and processing of represented by uppercase letters.
languages. Formal grammars generate or describe
3. Production Rules: These rules specify how
formal languages, while automata recognize or
the symbols can be combined or replaced
decide whether strings belong to specific formal
to generate valid strings in the language.
languages. The Chomsky hierarchy establishes a
A production rule consists of a non-
connection between grammars and automata,
terminal symbol on the left-hand side and
organizing them into classes with increasing
a sequence of terminal and/or non-
computational power. These fundamental concepts
terminal symbols on the right-hand side.
play a central role in various areas of computer
It represents a transformation or
science, linguistics, and theoretical research.
expansion of a non-terminal symbol into a
sequence of symbols.

4. Start Symbol: This symbol represents the

initial non-terminal symbol from which
the generation or parsing of strings
begins. It is often denoted by an
uppercase letter, such as "S."

By applying the production rules starting from

the start symbol, a formal grammar generates or
derives valid strings in the language. The process
of applying the production rules to generate
strings is called derivation.
Formal grammars are used in various areas, and α is a string of terminals and
including programming language design, syntax non-terminals.
analysis, natural language processing, and
computational linguistics. Different types of • Context-free languages can be
formal grammars, such as regular grammars, recognized by pushdown
context-free grammars, and context-sensitive automata, such as the non-
grammars, have different expressive power and deterministic pushdown automaton
are suitable for describing different types of (PDA).
languages with varying levels of complexity.
• Context-free grammars are widely
THE CHOMSKY HIERARCHY
used in programming languages,
parsing algorithms, and syntax
Noam Chomsky, a renowned linguist, proposed a
analysis.
classification of formal grammars into four types
known as the Chomsky hierarchy. These four
3. Type 1: Context-Sensitive Grammar
types, in order of increasing generative power,
(Context-Sensitive Languages) :-
are :-
• Production rules have the form
1. Type 3: Regular Grammar (Regular
αAβ → αγβ, where A is a non-
Languages)
terminal, α and β are strings of
2. Type 2: Context-Free Grammar (Context-
terminals and non-terminals, and
Free Languages)
γ is a non-empty string.
3. Type 1: Context-Sensitive Grammar
• Context-sensitive languages are
(Context-Sensitive Languages)
more expressive than context-free
4. Type 0: Unrestricted Grammar
languages.
(Recursively Enumerable Languages)
• Context-sensitive grammars are
Here's a breakdown of each type :- used in natural language
processing, compiler design, and
1. Type 3: Regular Grammar (Regular
linguistics.
Languages) :-

• Production rules have the form A 4. Type 0: Unrestricted Grammar

→ aB or A → a, where A and B (Recursively Enumerable Languages) :-
are non-terminals, a is a terminal,
• No restrictions are imposed on
and ε represents the empty string.
production rules.
• Regular languages can be
• Unrestricted grammars can
recognized by finite automata,
generate languages that are the
such as deterministic or non-
most general in terms of
deterministic finite automata (DFA
generative power.
or NFA), and regular expressions.
• Unrestricted grammars are closely
• Regular grammars are the simplest
related to Turing machines and
and most restricted type of
can generate any computable
grammars.
language.

2. Type 2: Context-Free Grammar (Context-

Free Languages) :-

• Production rules have the form A

→ α, where A is a non-terminal
 languages, automata theory, and pattern matching
algorithms.

Formal Definition Of A Grammer

A formal definition of a grammar consists of four

components :-

1. Terminal Alphabet (Σ): It is a finite set of

symbols called terminal symbols or
What are Regular Grammers terminals. These symbols represent the
basic units or elements that appear in the
In automata theory and complexity theory, a
strings of the language.
regular grammar refers to a type of formal
grammar that generates regular languages. 2. Non-terminal Alphabet (V): It is a finite
set of symbols called non-terminal
A formal grammar is a set of production rules
symbols or non-terminals. These symbols
that define the structure and syntax of a
represent variables or placeholders that
language. It consists of a set of non-terminal
can be replaced or expanded into
symbols, terminal symbols, a start symbol, and a
sequences of terminals and non-terminals.
set of production rules. The production rules
specify how symbols can be replaced or combined 3. Start Symbol (S): It is a special non-
to form valid strings in the language. terminal symbol that represents the initial
symbol from which the generation or
A regular grammar is a type of formal grammar
parsing of strings begins. The start symbol
where all production rules have one of the
is a member of the non-terminal alphabet
following forms:
V.
1. A → aB
4. Production Rules (P): It is a set of rules
2. A → a
that define how non-terminal symbols can
3. A → ε
be replaced or expanded into sequences of
In these rules, "A" and "B" are non-terminal terminals and non-terminals. Each
symbols, "a" is a terminal symbol, and ε production rule has the form A → β,
represents the empty string. where A is a non-terminal symbol and β
is a string of terminals and non-terminals.
Regular grammars are closely related to regular
The set of all production rules defines the
expressions and finite automata. In fact, any
transformation rules of the grammar.
regular grammar can be converted into an
equivalent finite automaton, and vice versa. Together, these components formally define a
Regular languages generated by regular grammars grammar as a 4-tuple (V, Σ, P, S), where:
are the simplest type of languages and can be
• V is the non-terminal alphabet,
recognized by finite automata.
• Σ is the terminal alphabet,
Regular grammars are of particular interest in • P is the set of production rules, and
automata theory and complexity theory because • S is the start symbol.
regular languages have simple and efficient
Using these components, a grammar specifies the
recognition algorithms. They can be recognized
syntax or structure of a formal language. By
and processed in linear time, making them
applying the production rules starting from the
computationally tractable. Regular grammars play
start symbol, valid strings in the language can be
a fundamental role in the study of formal
generated or recognized.
It's important to note that different types of • In left-linear grammars, all the
grammars, such as regular grammars, context-free derivations or transformations
grammars, and context-sensitive grammars, have occur from right to left.
specific restrictions and forms for their production • In the production rules, a terminal
rules, resulting in different expressive powers and symbol is always preceded by
language classes. either a non-terminal symbol or
the empty string ε.

RIGHT LINEAR AND LEFT LINEAR In summary, the main difference between right-
GRAMMERS linear and left-linear grammars lies in the
direction of the derivations or transformations. In
Before defining right-linear and left-linear right-linear grammars, the derivations proceed
grammars, let's start with a brief introduction to from left to right, while in left-linear grammars,
regular grammars. the derivations proceed from right to left. Both
types of grammars are examples of regular
Regular grammars belong to the class of formal
grammars and generate regular languages, but
grammars that generate regular languages. These
their production rules differ in the ordering of
grammars have production rules of the form A →
symbols.
aB, A → a, or A → ε, where A and B are non-
terminal symbols, a is a terminal symbol, and ε
represents the empty string. DERIVATION FROM A GRAMMER

Now, let's dive into the definitions of right-linear In the context of formal grammars, a derivation
and left-linear grammars :- is a sequence of production rule applications that

1. Right-Linear Grammar: transform an initial string, typically consisting of

non-terminals and terminals, into a final string
• A right-linear grammar is a type composed solely of terminals. The derivation
of regular grammar where all process follows the rules specified by the
production rules have the form A grammar's production rules.
→ aB or A → a, where A and B
are non-terminal symbols and a is Let's walk through an example to understand how
a terminal symbol. a derivation works :-
• In right-linear grammars, all the Consider the following grammar:
derivations or transformations
• Terminal Alphabet (Σ): {a, b}
occur from left to right.
• Non-terminal Alphabet (V): {S}
• In the production rules, a terminal
• Start Symbol (S): S
symbol is always followed by
• Production Rules (P):
either a non-terminal symbol or
• S → aSb
the empty string ε.
• S → ε

2. Left-Linear Grammar: Now, let's derive the string "aabbb" from the
start symbol S using the given grammar:
• A left-linear grammar is a type of
regular grammar where all 1. Start with the initial string: S
production rules have the form A 2. Apply the production rule S → aSb:
→ Ba or A → a, where A and B • S → aSb
are non-terminal symbols and a is • aSb
a terminal symbol. 3. Apply the production rule S → aSb again:
• S → aSb → aaSbb
• aaSbb
 4. Apply the production rule S → aSb one Here is a brief description of context-free
more time: grammars :-
• S → aSb → aaSbb → aaaSbbb
1. Non-terminal and Terminal Symbols :-
• aaaSbbb
5. No more non-terminals remain. The final • A context-free grammar consists of
string is composed solely of terminals: a set of non-terminal symbols
"aabbb". (also called variables) and a set of
terminal symbols.
The derivation process demonstrates how the
production rules are successively applied to
• Non-terminals represent syntactic
transform the non-terminals in each step until
categories or elements that can be
only terminals remain. Each step represents a rule
further expanded or replaced.
application, and the resulting string is obtained
by replacing a non-terminal symbol according to
• Terminals represent the basic units
the production rule.
or symbols of the language.
It's important to note that there can be multiple
derivations for the same string in a grammar, 2. Production Rules:
depending on which production rules are chosen
• The grammar includes a set of
at each step. Additionally, some grammars may
production rules that define how
have ambiguous derivations where multiple
non-terminals can be expanded or
sequences of rule applications lead to the same
replaced by a sequence of
final string.
terminals and non-terminals.
• Each production rule has the form
A → α, where A is a non-terminal
and α is a string of terminals
and/or non-terminals.
• The substitution of a non-terminal
with its right-hand side is
independent of the context in
which it appears, hence the term
"context-free."
CONTEXT FREE GRAMMER
3. Start Symbol:
Context-free grammars (CFGs) are a type of • The grammar designates a specific
formal grammar widely used in linguistics, non-terminal symbol as the start
computer science, and other fields. They are symbol, from which the derivation
named "context-free" because the left-hand side or parsing of strings begins.
of each production rule in the grammar is a
single non-terminal symbol, and the substitution 4. Language Generation:
or expansion of that non-terminal can occur
• A context-free grammar defines a
regardless of the context in which it appears. In
formal language by specifying the
other words, the replacement of a non-terminal
set of strings that can be
with its corresponding right-hand side can happen
generated by the grammar.
without considering the surrounding symbols.
• Starting from the start symbol,
valid strings in the language can
 be derived by successively In this grammar, the non-terminals are S, A, and
applying the production rules. B, and the terminals are a and b.

Now, let's look at the production rule S → AB.

5. Applications :-
This rule states that the non-terminal S can be
• Context-free grammars have broad replaced by the non-terminal A followed by the
applications in natural language non-terminal B. The important point here is that
processing, programming the replacement of S by AB can happen anywhere
languages, syntax analysis, in the derivation process, regardless of the
compiler design, parsing context or the symbols surrounding S.
algorithms, and more.
For instance, starting with the start symbol S, we
• They provide a foundation for
can derive the string "ab" using the following
understanding and modeling the
steps: S (Apply S → AB) AB (Apply A → a) aB
syntactic structure of languages.
(Apply B → b) ab
Context-free grammars are a powerful tool for
As you can see, the non-terminal S was replaced
describing the syntax of various formal languages.
by AB in the first step, without considering the
They capture many important features of natural
context (the presence of other symbols) in which
and programming languages, making them widely
S appeared.
used in both theoretical and practical aspects of
language analysis and processing. In summary, the term "context-free" in the
context of context-free grammars means that the
substitution or expansion of a non-terminal can
The Name Context Free
occur without considering the context or the
symbols surrounding it. The replacement can
In a context-free grammar, the term "context-
happen anywhere in the derivation process, solely
free" refers to the property that the left-hand side
based on the non-terminal being expanded and
of each production rule consists of a single non-
the production rule being applied.
terminal symbol. The left-hand side is the part
before the arrow (→) in a production rule.
The Difference Between Regular Grammers and
For example, consider the production rule: A → α Context Free Grammers
Here, A is the left-hand side, and α is the right-
hand side. The main differences between context-free
grammars (CFGs) and regular grammars lie in
The term "context-free" means that the
their expressive power and the types of languages
substitution or expansion of a non-terminal (such
they can generate. Here are the key distinctions:
as A in this case) can occur regardless of the
context in which it appears. This means that 1. Production Rules:
wherever the non-terminal A appears in the
• Regular Grammars: In regular
derivation process, it can be replaced by α,
grammars, the production rules
without considering the surrounding symbols.
have the form A → aB or A → a,
To further illustrate this, let's consider an where A and B are non-terminal
example CFG with the following production rules: symbols, a is a terminal symbol,
and ε represents the empty string.
S → AB
The right-hand side of the
A → a production rule can only have a
single terminal symbol followed by
B → b
an optional non-terminal symbol.
 • Context-Free Grammars: In CFGs, languages is generally higher than
the production rules have the form that of regular languages.
A → α, where A is a non-terminal
symbol, and α is a string of 4. Language Structures:
terminals and/or non-terminals.
• Regular Grammars: Regular
The right-hand side of the
grammars are suitable for
production rule can be any
describing regular structures, such
sequence of terminals and non-
as strings with simple patterns,
terminals.
finite sequences, regular
expressions, regular sets, and
2. Expressive Power:
regular automata.
• Regular Grammars: Regular
grammars are less expressive and • Context-Free Grammars: CFGs are
can generate regular languages. more suitable for describing
Regular languages are a subset of hierarchical and nested language
context-free languages and have structures, such as programming
certain limitations. They can languages, natural languages,
describe patterns such as finite syntax trees, recursive definitions,
automata, regular expressions, and nested parentheses, and more
simple language structures with complex patterns.
linear constraints.
In summary, context-free grammars are more
• Context-Free Grammars: CFGs are
expressive and can handle more complex
more expressive and can generate
language structures than regular grammars. They
context-free languages. Context-
can describe recursive patterns, nested constructs,
free languages are a broader class
and hierarchical relationships. Regular grammars,
of languages that includes regular
on the other hand, are more limited and suitable
languages. They can describe more
for describing simple, regular language patterns.
complex language structures,
hierarchical patterns, nested Formal Definition of a Context Free Grammer
constructs, and recursive
A formal definition of context-free grammars
definitions.
(CFGs) consists of the following components:

3. Parsing Capabilities: 1. Terminal Alphabet (Σ):

• Regular Grammars: Regular • Σ represents the set of terminal

grammars can be parsed efficiently symbols or alphabet of the
using deterministic finite automata language.
(DFAs) or non-deterministic finite 2. Non-terminal Alphabet (V):
automata (NFAs). Regular
• V represents the set of non-
languages have linear-time parsing
terminal symbols or variables used
algorithms.
to generate the language.
• Context-Free Grammars: CFGs
3. Start Symbol (S):
require more powerful parsing
techniques, such as pushdown • S is a distinguished symbol from V
automata or parsing algorithms that serves as the start symbol or
like CYK or LL(k)/LR(k), to initial non-terminal of the
recognize or generate strings. The grammar.
parsing complexity for context-free 4. Production Rules (P):
 • P is a set of production rules that 1. Start with the Start Symbol:
define how non-terminals can be
• Begin with the start symbol of the
replaced or expanded.
grammar as the initial non-
• Each production rule has the form
terminal.
A → α, where A is a non-terminal
symbol from V, and α is a string
2. Choose the Closest Production Rule:
of terminals and/or non-terminals
from (V ∪ Σ)*. • Look at the current non-terminal
• This rule specifies that A can be being expanded and compare it
replaced by α. with the next symbol in the given
string.
5. Language Generated: • Choose the production rule that
has the current non-terminal as its
• The language generated by the
left-hand side and a right-hand
CFG is the set of all strings of
side that matches the next symbol
terminals that can be derived from
in the given string.
the start symbol S using the
• If multiple production rules are
production rules.
available, select one based on a
In summary, a context-free grammar (CFG) is predefined priority or preference.
formally defined by the tuple (V, Σ, P, S), where
V is the set of non-terminals, Σ is the set of 3. Replace the Non-Terminal:
terminals, P is the set of production rules, and S
• Replace the chosen non-terminal
is the start symbol. The production rules specify
with the right-hand side of the
the ways in which non-terminals can be
selected production rule.
expanded, ultimately generating the language
• This substitution may include both
defined by the grammar.
terminals and non-terminals.
It's worth noting that CFGs are a fundamental
tool for language analysis and processing. They 4. Repeat the Process:
are widely used in areas such as programming
• Continue the process recursively
languages, natural language processing, syntax
by examining the next symbol in
analysis, compiler design, and parsing algorithms.
the string and the non-terminal in
the derived string.
Method to Find a string belongs to a • Choose the appropriate production
Grammer or Not rule based on the current context
until the entire string is generated
This basically can be achieved using top-down or no further production rules are
parsing or recursive descent parsing. It is a applicable.
method to determine whether a given string
belongs to a grammar by attempting to generate 5. Acceptance or Rejection:
the string starting from the start symbol and
• If the entire given string is
recursively applying the appropriate production
successfully generated, i.e., the
rules. Let's define the steps of this parsing
parsing process ends with all
method:
terminals matched and replaced,
then the string belongs to the
grammar.
 • If there are no further production • The internal nodes represent non-terminal
rules applicable and the entire symbols, which are expanded or replaced
string is not generated, then the by applying production rules.
string does not belong to the • The leaf nodes represent terminal
grammar. symbols, which are the actual symbols in
the generated string.
This top-down parsing approach relies on the
selection of production rules based on the current A left derivation tree and a right derivation tree
non-terminal and the next symbol in the string. It differ in the order in which the production rules
recursively expands non-terminals, replacing them are applied during the derivation process :-
with the corresponding right-hand sides of the
1. Left Derivation Tree:
chosen production rules until either the entire
string is generated or no further applicable rules • In a left derivation, the leftmost
are available. non-terminal is expanded first.
• At each step, the leftmost non-
It's important to note that the success of top-
terminal in the current derivation
down parsing depends on the grammar's
is replaced by the right-hand side
structure. Some grammars may require
of a production rule.
backtracking or additional techniques to handle
• The leftmost derivation tree
ambiguity or left recursion. Different parsing
illustrates the leftmost derivations
algorithms, such as LL(k) or recursive descent
of a string.
parsers, implement variations of this top-down
approach to efficiently determine the membership
2. Right Derivation Tree:
of a string in a grammar.
• In a right derivation, the
Example :-
rightmost non-terminal is
expanded first.
• At each step, the rightmost non-
terminal in the current derivation
is replaced by the right-hand side
of a production rule.
• The rightmost derivation tree
illustrates the rightmost
derivations of a string.

Both left and right derivations are valid and can

generate the same string from a given grammar.
LEFT AND RIGHT DERIVATION TREE The choice between left and right derivation
depends on the specific parsing algorithm or
A derivation tree, also known as a parse tree or grammar analysis technique used.
syntax tree, is a graphical representation of how
a string is derived or generated from a grammar.
It illustrates the hierarchical structure of a string
according to the production rules of the
grammar.

In a derivation tree:

• The root of the tree represents the start

symbol of the grammar.
Derivation trees are useful for understanding the CHAPTER FOUR
structure and derivation process of strings in a
grammar. They provide a visual representation of PUSH DOWN AUTOMATA
how a string is formed step by step, with non-
terminal symbols being replaced until the final A pushdown automaton (PDA) is a theoretical
string is generated. Derivation trees are model of computation that extends the
commonly used in compiler design, syntax capabilities of a finite automaton (FA) by
analysis, parsing algorithms, and understanding incorporating an additional stack or memory. It is
the behavior of context-free grammars. a type of automaton used in the field of formal
languages and automata theory. PDAs are often
Ambiguous Grammer employed to describe and analyze the behavior of
context-free languages.
An ambiguous grammar is a type of context-free
grammar (CFG) where a specific string in the
language can have more than one valid parse tree
or derivation. In other words, there can be
multiple ways to derive the same string using
different production rules and parse trees. This
ambiguity arises when the grammar allows
multiple interpretations or meanings for a
particular string.

Ambiguity in grammars can lead to parsing

difficulties and can make it challenging to
determine the correct interpretation of a given Here are the key characteristics and components
input. It can also result in different syntax trees of a push down automaton :-
or parse trees for the same input string, which 1. States :-
can affect the semantic analysis and
• Similar to finite automata, a PDA
understanding of the language.
has a finite set of states.
Example :-
2. Input Alphabet :-

• A PDA reads symbols from an

input alphabet as it processes
input strings. The input alphabet
consists of the symbols that the
PDA can read.

3. Stack :-

It's important to note that not all grammars are • A PDA has an additional
inherently ambiguous. A language can have component called the stack, which
unambiguous grammars that produce a unique provides extra memory.
parse tree for every valid input string. However,
some languages inherently have ambiguous
• The stack is a last-in-first-out
constructs, and it is necessary to carefully design
the grammar or use disambiguation techniques to (LIFO) data structure, meaning
avoid ambiguity. that the most recently added
symbol can be accessed and
removed first.
 FORMAL DEFINITION OF PUSH DOWN
• The stack allows the PDA to AUTOMATA
remember information from
previous input symbols and formal definition of a pushdown automaton (PDA)
perform context-sensitive consists of the following components :-
operations.
1. Input Alphabet (Σ):

4. Transitions :- • Σ represents the set of input

symbols or the input alphabet.
• The PDA transitions from one
state to another based on the
2. Stack Alphabet (Γ):
current input symbol, the symbol
at the top of the stack, and the • Γ represents the set of stack
next input symbol to be processed. symbols or the stack alphabet.
3. States (Q):
• The transition function specifies
• Q is a finite set of states that the
the next state, the symbol to be
PDA can be in.
replaced at the top of the stack (if
any), and the movement of the
4. Start State (q0):
stack pointer.
• q0 is the initial or start state from
5. Acceptance: which the PDA begins processing.

• A PDA can have different

5. Accept States (F):
acceptance criteria, such as
accepting by empty stack or final • F is a set of accepting states or
state. final states. If the PDA enters any
• If the PDA reaches an accepting of these states, it accepts the
state or empties its stack during input.
the input processing, it accepts the
input string as a valid member of 6. Transition Function (δ):
the language. Otherwise, it rejects
• δ is a function that maps the
the string.
current state, the current input
Push down automata are capable of recognizing symbol, and the top symbol of the
context-free languages, which include languages stack to the next state, a stack
generated by context-free grammars. They are operation, and the symbol to be
more powerful than finite automata due to the pushed or popped from the stack.
additional stack component, allowing them to • The transition function is defined
perform context-sensitive operations and handle as: δ: Q × (Σ ∪ {ε}) × Γ → 2^(Q
nested structures. PDAs serve as a theoretical × Γ*)
foundation for parsing algorithms used in
compiler design, natural language processing, and 7. Initial Stack Symbol (Z):
other areas where context-free languages play a
• Z is the initial symbol that is
significant role.
pushed onto the stack when the
PDA starts processing the input.

In summary, a pushdown automaton (PDA) is

formally defined by the tuple (Q, Σ, Γ, δ, q0, Z,
F), where Q represents the set of states, Σ Context – Sensitive Grammer
represents the input alphabet, Γ represents the
Context-sensitive grammars are a type of formal
stack alphabet, δ represents the transition
grammar used to describe context-sensitive
function, q0 is the initial state, Z is the initial
languages. A context-sensitive grammar allows the
stack symbol, and F represents the set of
rules to have more flexibility compared to
accepting states.
context-free grammars by considering the context
The PDA processes the input string by reading or surrounding symbols during the derivation
symbols from the input alphabet, using the process.
current state, the current input symbol, and the
Formally, a context-sensitive grammar consists of
top symbol of the stack to determine the next
the following components:
state, the stack operation, and the symbol to be
pushed or popped from the stack. The PDA can 1. Non-terminal symbols: These are symbols
accept the input if it enters an accepting state or that can be replaced or expanded during
rejects it if there is no valid transition or the the derivation process.
stack becomes empty before reaching an
2. Terminal symbols: These are symbols that
accepting state.
cannot be further expanded and represent
Pushdown automata are a key concept in the the basic units of the language.
study of formal languages and automata theory,
3. Production rules: These rules define how
particularly for recognizing and generating
the non-terminal symbols can be replaced
context-free languages. They provide a theoretical
by a sequence of symbols.
foundation for parsing algorithms and have
applications in areas such as compiler design, 4. Start symbol: It represents the initial non-
natural language processing, and the analysis of terminal symbol from which the
nested structures. derivation process begins.

The key characteristic of context-sensitive

grammars is that their production rules have a
specific form. Each production rule is of the form
α -> β, where α and β are strings of symbols,
and the length of β is greater than or equal to
the length of α. This restriction ensures that the
grammar can modify the context or surrounding
symbols during the derivation process.

A context-sensitive language is a language that

The output of push down Automata can be generated by a context-sensitive grammar.
It is a more general class of languages compared
to context-free languages. Context-sensitive
languages allow for more complex patterns and
dependencies among symbols, as the production
rules can adapt to the context or surrounding
symbols.

Context-sensitive languages find applications in

various fields, such as natural language
processing, programming language design, and
compiler construction. They can capture complex
syntactic and semantic structures in languages
that require context information for analysis and derivation process begins. The start
interpretation. symbol is typically denoted by S.

It's worth noting that the term "context-sensitive" The production rules of a context-free grammar
can also refer to other concepts in different define the transformations or expansions that can
contexts, such as context-sensitive rewriting rules be applied to the non-terminal symbols. During
in formal languages or context-sensitive rewriting the derivation process, a sequence of productions
systems in computational models. However, in is applied to the start symbol, replacing non-
the context of grammars and languages, context- terminal symbols with the corresponding RHS
sensitive grammars and context-sensitive symbols according to the production rules. This
languages refer to the concepts described above. process continues until only terminal symbols
remain, resulting in a valid string in the language
Formal Definition of Context sensitive Grammer
defined by the grammar.

A context-free grammar (CFG) is a formal Context-free grammars and languages are widely
grammar that describes a context-free language. It used in various areas, such as programming
is a widely used grammar type in formal language syntax, natural language processing, and
language theory, programming languages, and the design and implementation of parsing
compiler design. A context-free grammar consists algorithms. They provide a flexible and expressive
of a set of production rules that define how way to describe the syntactic structure of
symbols can be replaced or expanded. languages, allowing for concise and powerful
language definitions.
Formally, a context-free grammar consists of the
following components :- COMPARING CONTEXT FREE GRAMMER
1. Non-terminal symbols: These are symbols AND CONTEXT FREE LANGUAGES
that can be replaced or expanded during
the derivation process. Non-terminal Context-Free Grammar (CFG): A context-free
symbols are typically represented by grammar is a formal grammar where each
uppercase letters. production rule has the form A -> α, where A is
a non-terminal symbol and α is a string of
2. Terminal symbols: These are symbols that symbols (both terminals and non-terminals). The
cannot be further expanded and represent key characteristic of CFGs is that the replacement
the basic units or tokens of the language. or expansion of non-terminal symbols occurs
Terminal symbols are typically represented regardless of the context in which they appear.
by lowercase letters, digits, or other CFGs have a specific set of rules and restrictions
characters. that define their structure.
3. Production rules: These rules specify how Context-Sensitive Grammar (CSG): A context-
the non-terminal symbols can be replaced sensitive grammar is a formal grammar where
by a sequence of symbols, which can each production rule has the form α -> β, where
include both non-terminal and terminal α and β are strings of symbols, and the length of
symbols. Each production rule consists of β is greater than or equal to the length of α. In
a non-terminal symbol as the left-hand context-sensitive grammars, the replacement or
side (LHS) and a sequence of symbols as expansion of non-terminal symbols is influenced
the right-hand side (RHS), separated by by the context or surrounding symbols. This
an arrow symbol (->). means that the rules can modify the context or
4. Start symbol: It represents the initial non- neighboring symbols during the derivation
terminal symbol from which the process.
A context-sensitive grammar is a formal grammar Differences between Context-Free Grammar and
where each production rule has the form α -> β, Context-Sensitive Grammar :-
where α and β are strings of symbols, and the
1. Rule Formulation:
length of β is greater than or equal to the length
of α. • CFG: The production rules have
the form A -> α, where A is a
To illustrate this definition with an example,
non-terminal symbol and α is a
consider the following context-sensitive
string of symbols.
grammar :-
• CSG: The production rules have
S -> aSb the form α -> β, where α and β
are strings of symbols, and the
S -> ab
length of β is greater than or
In this grammar, the nonterminal symbol S can equal to the length of α.
be expanded as either "aSb" or "ab".
2. Context Dependency:
Let's analyze the production rules in terms of the
lengths of α and β: • CFG: The expansion of non-
terminal symbols occurs regardless
1. S -> aSb : Here, α is the string "S" with
of the context or surrounding
length 1, and β is the string "aSb" with
symbols. The rules are applied
length 3. The length of β is greater than
based on the non-terminal symbols
the length of α.
alone.
2. S -> ab : Here, α is the string "S" with • CSG: The expansion of non-
length 1, and β is the string "ab" with terminal symbols depends on the
length 2. The length of β is equal to the context or neighboring symbols.
length of α. The rules can modify the context
Both production rules in this example satisfy the or surrounding symbols during the
condition of a context-sensitive grammar, where derivation process.
the length of β is greater than or equal to the
length of α. This means that the grammar is 3. Generative Power:
context-sensitive. • CFG: CFGs can generate context-
Note that context-sensitive grammars allow for free languages, which are a subset
more flexibility than context-free grammars by of the Chomsky hierarchy. They
allowing the right-hand side (β) of a production are less expressive than context-
rule to be longer or equal in length to the left- sensitive languages.
hand side (α). This flexibility allows context- • CSG: CSGs can generate context-
sensitive grammars to define more complex sensitive languages, which are a
languages that cannot be captured by context-free more general class of languages
grammars. than context-free languages. They
have a higher generative power
and can capture more complex
patterns and dependencies.

4. Formal Definition:

• CFG: CFGs are defined by a set of

non-terminal symbols, terminal
symbols, production rules, and a
 start symbol. The rules specify CHAPTER FIVE
how the non-terminal symbols can
be replaced by sequences of TURING MACHINE
symbols.
• CSG: CSGs are defined by a set of A Turing machine is a theoretical model of
non-terminal symbols, terminal computation introduced by Alan Turing in the
symbols, production rules, and a 1930s. It is a fundamental concept in automata
start symbol. The rules specify theory and complexity theory. A Turing machine
how sequences of symbols can be consists of an infinite tape divided into discrete
transformed, taking into account cells, where each cell can store a symbol from a
the context or neighboring finite alphabet. The machine has a read/write
symbols. head that can move along the tape, reading the
In summary, the main difference between symbol at the current cell and modifying it.
context-free grammars and context-sensitive The Turing machine has a set of states, and at
grammars lies in the form of their production any given time, it is in one of these states. It
rules and the consideration of context during the starts in an initial state and can transition from
derivation process. Context-sensitive grammars one state to another based on the current symbol
have a higher generative power and can handle being read and the current state. Each transition
more complex languages by incorporating context specifies an action to be taken, such as changing
information into the derivation rules. the symbol on the tape, moving the head left or
right, or changing the state.

The machine operates according to a set of rules

known as the transition function, which
determines its behavior. These rules are defined
for each combination of current state and current
symbol and specify the next state, the symbol to
be written, and the direction in which the head
should move. The machine continues executing
these transitions until it reaches a halting state,
at which point it stops.

Turing machines are capable of solving a wide

range of computational problems. They can
simulate any algorithm that can be described in a
step-by-step manner. This property, known as
Turing completeness, makes Turing machines a
powerful tool for studying the limits and
possibilities of computation.

In complexity theory, Turing machines are used

to analyze the computational complexity of
problems. The time complexity of a problem is
measured by the number of steps a Turing
machine takes to solve it, while the space
complexity is measured by the amount of tape
cells used. Turing machines help classify problems
into different complexity classes, such as P
(problems solvable in polynomial time), NP
(nondeterministic polynomial time), and many symbol on the tape, and moves the tape head left
others. or right. The process continues until the machine
enters the qaccept or qreject state, at which point
Overall, a Turing machine is a mathematical
it halts.
model that captures the essence of general-
purpose computation. It serves as a foundation The formal definition of a Turing machine serves
for understanding the theoretical aspects of as a precise mathematical foundation for studying
computation, automata theory, and complexity the theoretical aspects of computation and is
theory. essential for understanding computability and
complexity theory.

FORMAL DEFINITION OF A TURING MACHINE Let’s Simplify the Turing Machine

The formal definition of a Turing machine (TM) Let's imagine a Turing machine as a special
consists of a 7-tuple: computer that can do different things step by
step. It has a tape, kind of like a long strip of
M = (Q, Σ, Γ, δ, q0, qaccept, qreject)
paper, and it can read and write symbols on this
Where: tape.
1. Q is the set of states: Q = {q0, q1, The machine also has a little "head" that can
q2, ..., qn}. move along the tape and read the symbol at the
2. Σ is the input alphabet (a finite set of position it's currently on. It has some rules that
symbols): Σ = {a1, a2, ..., am}. tell it what to do based on the symbol it sees
3. Γ is the tape alphabet (a finite set of and the state it's in.
symbols that includes the input alphabet
The machine starts at a special place called the
and a blank symbol): Σ ⊆ Γ, and it also
starting state. It looks at the symbol on the tape
contains a special blank symbol, usually
where the head is and follows the rules to decide
denoted as '□'.
what to do next. It might change the symbol on
4. δ is the transition function: δ: Q × Γ → Q
the tape, move the head left or right, or change
× Γ × {L, R}, where Q × Γ represents the
to a different state.
current state and symbol, and δ(q, a) =
(p, b, L) means if the machine is in state This process keeps going until the machine
q and reads symbol 'a', it changes to state reaches a special state called the accepting state,
p, writes symbol 'b' on the tape, and which means it's done and it says "yes" to
moves the tape head one cell to the left whatever it was trying to figure out. Or it might
(L) or right (R). reach another special state called the rejecting
5. q0 is the initial state: q0 ∈ Q. state, which means it's done and it says "no"
6. qaccept is the accepting state: qaccept ∈
Turing machines are really powerful because they
Q. If the machine reaches this state, it
can do all kinds of things, just like regular
halts and accepts the input.
computers. They can solve math problems,
7. qreject is the rejecting state: qreject ∈ Q.
simulate other computers, and lots more. They
If the machine reaches this state, it halts
help us understand how computers work and
and rejects the input.
what they can do.
The Turing machine starts in state q0 with the
So, in simple words, a Turing machine is like a
input tape head positioned at the leftmost symbol
special computer that can read and write symbols
of the input string. It then reads the symbol at
on a tape, follow rules to decide what to do, and
the current position and looks up the appropriate
figure out answers to different problems.
transition in the transition function δ. Based on
the transition, it changes state, writes a new
SUMMARY 6. Include anchors: Use anchors such as "^"
(caret) and "$" (dollar sign) to indicate
DESIGNING A REGULAR EXPRESSION the start and end of a string, respectively.
FROM A LANGUAGE These help to ensure that the regex
matches the entire string and not just a
Designing a regular expression (regex) involves part of it.
constructing a pattern that matches strings
belonging to a specific language. Regular
expressions are a powerful tool used for pattern
matching and string manipulation.

To design a regular expression for a given

language, you need to analyze the patterns and
rules that define the language. Here are some
steps to consider :-

1. Identify the language: Understand the

characteristics and rules of the language
you want to design a regex for. CONVERSION OF REGULAR EXPRESSION
Determine the types of strings that should TO FINITE AUTOMATA
be matched by the regex.
Converting a regular expression to a finite
2. Determine the alphabet: Identify the set of
automaton (FA) involves constructing a
characters (or symbols) that make up the
deterministic or non-deterministic finite
language. This will help you define the
automaton that recognizes the language defined
characters that should appear in your
by the regular expression. Here's a step-by-step
regex pattern.
process for this conversion :-
3. Define the pattern: Use the regex syntax
Step 1: Convert the regular expression to an
to construct a pattern that matches the
equivalent non-deterministic finite automaton
desired strings. Regular expressions consist
(NFA) There are different algorithms you can use
of a combination of characters,
to convert a regular expression to an NFA, such
metacharacters, and special symbols that
as Thompson's construction algorithm or the
define patterns. Examples of
recursive algorithm. These algorithms build the
metacharacters include "*", "+", "?", ".",
NFA by recursively breaking down the regular
etc.
expression into smaller components and
4. Specify repetitions :- Determine if the constructing the corresponding NFA fragments.
language has any specific requirements
Step 2:- Convert the NFA to an equivalent
regarding the repetition of certain
deterministic finite automaton (DFA) An NFA can
characters or groups of characters. Use
be transformed into an equivalent DFA using
quantifiers such as "*", "+", "{n}", or
algorithms like the subset construction or
"{n,m}" to indicate the desired repetitions
powerset construction. These algorithms create a
in the regex.
DFA that simulates the behavior of the NFA,
5. Consider alternation: If the language effectively recognizing the same language.
allows for different variations or
Step 3: Optimize the DFA (optional) Once you
alternatives, use the "|" symbol to specify
have the DFA, you can perform optimizations if
different options in the regex pattern.
desired. Techniques like state minimization and
dead state elimination can reduce the number of
states and transitions in the DFA while preserving • ->: This arrow symbol is used to indicate
its language recognition capabilities. "produces" or "expands to."

By following these steps, you can convert a

• α :- It represents a string of terminals and
regular expression to a finite automaton. It's
nonterminals, including the empty string
worth noting that there can be multiple DFAs
ε. Terminals are symbols that appear in
that recognize the same language, and the
the final generated strings and cannot be
conversion process may result in different DFAs
further expanded.
depending on the chosen algorithms and
optimizations. In the production rule A -> α, it means that the
nonterminal symbol A can be replaced by the
string α. This rule allows you to rewrite or
expand the nonterminal symbol A to generate
new strings in the language defined by the CFG.

For example, consider the following production

rule in a CFG :-

S -> AB

This rule states that the nonterminal symbol S

can be expanded or rewritten as the
CONTEXT FREE GRAMMER SUMMARY concatenation of the nonterminal symbols A and
B.
A context-free grammar is a formal grammar
where each production rule has the form A -> α, Every regular language is a context free language
where A is a non-terminal symbol and α is a but every context free is not a regular language ?
string of symbols (both terminals and non-
terminals). The key characteristic of CFGs is that Every regular language is a context-free language,
the replacement or expansion of non-terminal but not every context-free language is a regular
symbols occurs regardless of the context in which language.
they appear. CFGs have a specific set of rules A regular language can be described by a regular
and restrictions that define their structure. grammar or recognized by a finite-state
PRODUCTION RULE OF CFG automaton (e.g., deterministic finite automaton or
non-deterministic finite automaton). Regular
In the context of context-free grammars (CFGs), a languages have certain limitations on their
production rule is a formal representation of a expressiveness, and they can be represented by
grammar rule that defines how symbols can be regular expressions.
expanded or replaced in a derivation. The On the other hand, context-free languages can be
production rule has the following form :- described by context-free grammars, which allow
for more complex rules and recursive structures.
A -> α , A ∈ N, α ∈ (N U T)*
Context-free grammars can generate languages
Here's a breakdown of the components :- that regular grammars cannot, such as languages
• A :- It represents a nonterminal symbol, with nested structures or languages that require
also known as a variable. Non terminal counting. Context-free languages are recognized
symbols do not appear in the final strings by pushdown automata.
generated by the grammar and can be
expanded into other symbols.
In summary :- The language RE consists of strings that consist of
'a's followed by an equal number of 'b's. In other
• Every regular language can be described
words, the number of 'a's and 'b's must be the
by a regular grammar and is therefore a
same, and there must be at least one 'a' and one
context-free language.
'b'.
• However, there are context-free languages
that cannot be described by a regular For example, some valid strings in RE are: "ab",
grammar, and therefore, they are not "aabb", "aaabbb".
regular languages.
To express this language using a CFG, we can
This distinction highlights the hierarchy of define the following production rules:
language classes, where the regular languages are
1. S -> aSb
a subset of the context-free languages.
2. S -> ab
n
The Language a is a regular language
These rules state that the nonterminal symbol S
The language L = {a^n}, where n ≥ 0, is a can be expanded as either "aSb" or "ab". The
regular language. rule (1) allows for the recursive production of 'a's
followed by 'b's, ensuring that the number of 'a's
The language L consists of strings containing only
and 'b's is the same. The rule (2) covers the base
the letter 'a' repeated a certain number of times.
case where there is exactly one 'a' followed by
It can be easily recognized and generated using a
one 'b'.
regular grammar, regular expressions, and finite-
state automata such as deterministic finite Now, let's consider why this language cannot be
automata (DFA) or non-deterministic finite expressed by a regular expression (RE). Regular
automata (NFA). expressions have limited expressive power and
cannot handle the requirement of matching equal
For example, a regular expression that represents
numbers of 'a's and 'b's. The language RE =
L is "a*". This regular expression matches any
{a^nb^n} violates the pumping lemma for
number of 'a's, including the empty string.
regular languages, which states that for any
Additionally, a DFA can be constructed to regular language L, there exists a pumping length
recognize the language L. The DFA would have a p such that any string s in L with |s| ≥ p can
single state that is both the initial and accepting be divided into five parts: s = uvwxy, satisfying
state. It would have a transition labeled 'a' certain conditions. However, for the language RE,
looping back to the same state. no such partitioning can be achieved due to the
requirement of matching numbers of 'a's and 'b's.
Therefore, the language L = {a^n}, where n ≥
0, is a regular language. Therefore, the language RE = {a^nb^n} for n ≥
1 is an example of a language that can be
expressed by a CFG but not by a regular
RE = {anbn | n>=1} , is an example
expression.
language that can be expressed by CFG
but not by RE
L = {an bn cm} is not CFL
The language RE = {a^nb^n} for n ≥ 1 is an
example of a language that can be expressed by a The language L = {a^n b^n c^m} is not a
context-free grammar (CFG) but not by a regular context-free language (CFL).
expression (RE).
The language consists of strings with a sequence
To define this language more clearly, let's break of 'a's, followed by an equal number of 'b's, and
it down :- then a sequence of 'c's. In order for a language to
be context-free, it must be possible to generate or
recognize it using a context-free grammar (CFG) conditions of the pumping lemma hold. However,
or a pushdown automaton (PDA). if we try to divide s into five parts (uvwxy) and
pump it according to the conditions, we will
To see why L = {a^n b^n c^m} is not a CFL,
encounter a contradiction.
we can apply the pumping lemma for context-free
languages. The pumping lemma states that for For example, consider the string s = a^p b^p
any CFL L, there exists a constant p (the c^p. If we divide it as s = uvwxy, where |vwx|
pumping length) such that any string s in L of ≤ p and |vx| > 0, then pumping up or down
length |s| ≥ p can be divided into five parts: will result in the number of 'a's, 'b's, and 'c's
uvwxy, satisfying certain conditions. becoming unbalanced, violating the condition of L
= {a^n b^n c^n}.
Let's assume L = {a^n b^n c^m} is a CFL.
According to the pumping lemma, we can choose Hence, by applying the pumping lemma, we can
a string s in L such that |s| ≥ p and the conclude that L = {a^n b^n c^n} is not a
conditions of the pumping lemma hold. However, context-free language (CFL).
if we try to divide s into five parts (uvwxy) and
pump it according to the conditions, we will
PUMMPING LEMA
encounter a contradiction.

For example, consider the string s = a^p b^p The pumping lemma is a fundamental tool in the
c^p. If we divide it as s = uvwxy, where |vwx| theory of formal languages used to analyze and
≤ p and |vx| > 0, then pumping up or down prove properties of regular and context-free
will result in the number of 'a's, 'b's, and 'c's languages. Specifically, it helps to show that
becoming unbalanced, violating the condition of L certain languages are not regular or context-free.
= {a^n b^n c^m}.
The pumping lemma is used as a technique to
Hence, by applying the pumping lemma, we can prove that certain languages are not regular or
conclude that L = {a^n b^n c^m} is not a not context-free by demonstrating a contradiction
context-free language (CFL). when the pumping conditions are violated.

It's important to note that while the pumping

n n n
L = {a b c } is not CFL lemma can be a useful tool for proving non-
regularity or non-context-freeness, it cannot be
The language L = {a^n b^n c^n} is not a used to prove that a language is regular or
context-free language (CFL). context-free.

The language consists of strings with a sequence

of 'a's, followed by an equal number of 'b's, and
then an equal number of 'c's. It requires the
number of 'a's, 'b's, and 'c's to be the same.

To see why L = {a^n b^n c^n} is not a CFL,

we can apply the pumping lemma for context-free
languages. The pumping lemma states that for
any CFL L, there exists a constant p (the
pumping length) such that any string s in L of
length |s| ≥ p can be divided into five parts:
uvwxy, satisfying certain conditions.

Let's assume L = {a^n b^n c^n} is a CFL.

According to the pumping lemma, we can choose
a string s in L such that |s| ≥ p and the
CHAPTER SIX COMPLEXITY CLASSES
COMPLEXITY ANALYSIS P (Polynomial Time): P is the class of decision
problems that can be solved by a deterministic
Complexity theory is a field of computer science Turing machine in polynomial time. This means
that deals with understanding the resources there exists an algorithm that can efficiently solve
required to solve computational problems. It these problems. Polynomial time complexity
focuses on studying the inherent complexity of implies that the running time of the algorithm is
problems and classifying them into different bounded by a polynomial function of the input
categories known as complexity classes. These size.
classes help us analyze and compare the
Features :-
computational difficulty of problems.
P Problems :-
In complexity theory, problems are classified
based on factors such as time complexity (how • Easy to Find Solution : P problems are
long it takes to solve a problem) and space characterized by having a solution that
complexity (how much memory or storage is can be efficiently found or computed by a
needed). The most well-known complexity classes deterministic Turing machine in
include P (problems solvable in polynomial time), polynomial time. This means that there
NP (problems for which solutions can be exists an algorithm that can solve these
efficiently verified), PSPACE (problems solvable problems within a reasonable amount of
using polynomial space), and EXP (problems time.
requiring exponential time).
• Tractable: P problems are considered
Complexity theory also explores the concepts of
tractable, meaning they can be solved
hardness and completeness. Hardness refers to the
both in theory and in practice. This
level of computational difficulty of a problem,
implies that efficient algorithms exist for
while completeness signifies that a problem is
these problems, allowing them to be
representative of an entire complexity class. For
solved within a reasonable time frame
example, NP-hard problems are considered at
even for large inputs.
least as difficult as the hardest problems in the
NP class, while NP-complete problems capture the
complete essence of the NP class.
NP (Nondeterministic Polynomial Time): NP is the
Additionally, reducibility is a key concept in class of decision problems for which a potential
complexity theory. It allows us to compare the solution can be verified in polynomial time by a
computational complexity of different problems. If nondeterministic Turing machine. In other words,
problem A can be reduced to problem B, it if there is a "yes" answer, there exists a short
means that an algorithm solving problem B can proof that can be checked in polynomial time.
be used to solve problem A as well. This concept While finding a solution to NP problems might be
helps establish relationships and hierarchy challenging, verifying a solution is relatively easy.
between complexity classes.
Features :-
Overall, complexity theory provides a framework
NP Class :-
for understanding and analyzing the complexity
of computational problems, enabling researchers • Hard to Find Solutions :- The problems in
to assess the efficiency and difficulty of the NP class are characterized by
algorithms and develop strategies for problem- solutions that are difficult to find
solving in various domains. efficiently. These problems are typically
solved by a non-deterministic Turing
 machine, which can explore multiple • Limited Memory Usage: Problems in the
possible solutions simultaneously. While PSPACE class can be solved by a
finding a solution may be challenging, the deterministic Turing machine using
NP class allows for the possibility of polynomial space. This means that the
efficiently verifying a given solution. amount of memory or storage required to
solve these problems is bounded by a
• Easy to Verify Solutions :- One key
polynomial function of the input size.
feature of the NP class is that solutions to
PSPACE problems can be efficiently solved
its problems can be verified by a Turing
using limited memory resources.
machine in polynomial time. This means
that given a potential solution, there • Generalization of P and NP: PSPACE is a
exists an algorithm that can check more general class than both P and NP. P
whether the solution is correct or not is a subset of PSPACE, as problems in P
within a reasonable amount of time. can be solved in polynomial time and
Verification does not involve finding the thus can also be solved using polynomial
solution from scratch but rather ensuring space. NP is also a subset of PSPACE, as
that the proposed solution satisfies certain problems in NP can be verified in
criteria. polynomial time and therefore can be
solved using polynomial space. PSPACE
• Polynomial-Time Verification :- The ability
encompasses a wider range of problems
to verify solutions in polynomial time
that require more memory resources than
implies that the verification algorithm's
problems in P or NP.
running time is bounded by a polynomial
function of the input size. While finding • Captures Complex Problems: PSPACE
the solution may require exponential time, includes problems that require a higher
once a solution is provided, it can be level of computational resources than
efficiently checked to determine its problems in P or NP. These problems
correctness. often involve complex computations and
interactions between different components.
• Non-Deterministic Computation :- It's
Examples of PSPACE problems include
worth noting that the non-deterministic
games with large state spaces, puzzles
Turing machine, which is used to define
with high branching factors, and problems
the NP class, can explore multiple paths
related to formal verification of systems.
simultaneously. It can guess a potential
solution and then verify it efficiently. • Hierarchy of Complexity: Within PSPACE,
However, non-deterministic machines are there exist complexity hierarchies that
theoretical constructs, and their practical categorize problems based on their
implementation is not yet possible with memory requirements. For example, there
current technology. are subclasses of PSPACE such as
PSPACE-complete problems, which
represent the hardest problems within
PSPACE (Polynomial Space): PSPACE is the class PSPACE. These problems capture the
of decision problems that can be solved by a complete essence of PSPACE and are as
deterministic Turing machine using polynomial difficult as any other problem in PSPACE.
space. These problems can be efficiently solved
• Relationship to Other Complexity Classes:
using limited memory resources. The amount of
PSPACE is known to be equal to PSPACE-
memory used by the algorithm is bounded by a
complete, which means that any problem
polynomial function of the input size.
in PSPACE can be reduced to a PSPACE-
Features :- complete problem in polynomial time.
 PSPACE-complete problems serve as makes solving EXP problems infeasible for
benchmarks for the difficulty level of large input instances.
problems within PSPACE.
• Beyond P and NP: EXP is a class that
In summary, PSPACE represents the class of goes beyond both P and NP. P represents
problems that can be solved by a deterministic problems that can be solved in
Turing machine using polynomial space. It polynomial time, whereas NP represents
encompasses a wide range of problems that problems that can be verified in
require memory resources beyond what is needed polynomial time. EXP contains problems
for problems in P or NP. PSPACE captures that require more time resources than
complex computational problems and has a those in P or NP. It encompasses
hierarchical structure that includes subclasses problems that cannot be efficiently solved
such as PSPACE-complete. using current algorithms or techniques.

EXP (Exponential Time): EXP is the class of • Relationship to Other Complexity Classes:
decision problems that can be solved by a EXP is known to be a superclass of both
deterministic Turing machine in exponential time. P and NP. This means that any problem
These problems require resources that grow in P or NP can be solved in exponential
exponentially with the input size. As a result, time, as P and NP are subsets of EXP.
solving EXP problems is generally considered Additionally, EXP-hard and EXP-complete
difficult and computationally expensive. problems represent the hardest problems
within the EXP class, and they serve as
Features :-
benchmarks for the difficulty level of
• Exponential Time Complexity: Problems in problems in EXP.
the EXP class require exponential time to
• Intractable Nature :- Due to the
solve. This means that the running time
exponential time complexity, solving EXP
of any algorithm that solves these
problems is generally considered
problems grows exponentially with the
intractable. It often involves exhaustive
size of the input. As the input size
search or brute-force techniques, which
increases, the resources required to solve
become infeasible for larger input sizes.
EXP problems increase dramatically.
As a result, finding optimal solutions for
• High Computational Complexity: EXP EXP problems is often impractical or even
represents a class of computationally impossible in practice.
difficult problems. The exponential time
• In summary, the EXP class represents
complexity indicates that solving these
problems that require exponential time
problems is generally considered
resources to solve. These problems have
challenging and computationally
high computational complexity and go
expensive. It often involves exploring a
beyond the efficiency boundaries of P and
vast search space or performing repeated
NP. EXP problems are generally
computations.
considered intractable and require
• Resources Grow Exponentially: EXP exponentially growing resources as the
problems require exponentially growing input size increases.
resources, such as time, memory, or
In summary, the complexity classes P, NP,
computational power, to find a solution.
PSPACE, and EXP categorize decision problems
As the input size increases, the amount of
based on their computational complexity and the
time and memory needed to solve EXP
resources required to solve them. P represents
problems increases exponentially. This
efficiently solvable problems, NP includes
problems with efficiently verifiable solutions,
PSPACE encompasses problems solvable within
Reductions
polynomial space, and EXP consists of problems
that require exponential time resources.
Reducibility is a fundamental concept in
complexity theory that enables us to compare the
HARDNESS AND COMPLETENESS computational complexity of different problems. It
provides a way to establish relationships between
Hardness: problems and determine their relative difficulty.
Let's delve into the details of reducibility:
• Hardness refers to the level of difficulty
or computational intractability of a 1. Reducibility between Problems :-
problem.
• Reducibility allows us to analyze
• A problem is considered hard if solving it
the computational complexity of
is at least as difficult as solving any other
one problem in terms of another
problem in that complexity class.
problem.
• For example, an NP-hard problem is as
• If problem A is reducible to
hard as the hardest problems in the NP
problem B, it means that an
class. This means that if an efficient
algorithm that solves problem B
algorithm exists for any NP-hard problem,
can be used to solve problem A.
it can be used to solve any problem in
• The reduction provides a mapping
NP.
or transformation from instances
of problem A to instances of
Completeness:
problem B.
• Completeness refers to a problem's • This mapping must be efficient,
property that makes it representative or typically done in polynomial time,
complete for a particular complexity class. meaning that the transformation
• A problem is considered complete for a can be performed in a reasonable
class if it is both in the class and amount of time.
captures the essential characteristics of
that class. 2. Implications of Reducibility :-
• For example, a problem is NP-complete if
• If problem A is reducible to
it is in the NP class and every problem in
problem B, it implies that problem
NP can be reduced to it in polynomial
B is at least as difficult as
time.
problem A in terms of
• In other words, an NP-complete problem
computational complexity.
represents the "hardest" problems in NP
• If an efficient algorithm exists for
and serves as a benchmark for the
solving problem B, it can be
difficulty level of all problems in NP.
utilized to solve problem A by
To summarize, hardness refers to the difficulty or applying the reduction.
intractability of a problem within a complexity • In other words, if problem B is
class, while completeness refers to a problem's hard, then problem A is at least
property of representing the essential as hard as problem B.
characteristics of a complexity class. Hardness
provides a measure of how challenging a problem 3. Comparing Complexity :-
is within its class, and completeness establishes a
• By establishing reducibility
problem as a representative for a specific
relationships, we can classify
complexity class.
 problems into different complexity HIERARCHY AND RELATIONSHIPS
classes based on their BETWEEN COMPLEXITY CLASSES
computational difficulty.
• For example, if problem A is Hierarchy and relationships between complexity
reducible to problem B and classes play a crucial role in understanding the
problem B is known to be hard, relative difficulty and computational properties of
then problem A is placed in the problems. Here's an explanation of hierarchy and
same or a higher complexity class relationships between complexity classes :-
as problem B.
Hierarchy :-
• This allows us to compare the
complexity of problems and • Complexity classes can be organized in a
understand their relative difficulty hierarchy based on the resources required
within a given complexity class or to solve the problems within each class.
hierarchy. • The hierarchy reflects the relationship
between classes in terms of their
4. Types of Reductions :- computational power and the amount of
resources they utilize.
• There are different types of
• In general, complexity classes higher in
reductions used in complexity
the hierarchy encompass a wider range of
theory, such as polynomial-time
problems that require more resources or
reductions, logarithmic-space
have higher computational complexity
reductions, and many-one
than classes lower in the hierarchy.
reductions.
• Polynomial-time reductions are
most commonly employed, as they
Relationships :-
provide efficient mappings
between problems that can be 1. Inclusion :-
computed in polynomial time.
• One fundamental relationship
• These reductions are crucial for
between complexity classes is
defining completeness and
inclusion, where one class is
hardness within complexity
contained within another.
classes, as well as establishing
• For example, P is contained within
relationships between classes.
PSPACE, which means that any
In summary, reducibility is a powerful concept problem that can be solved in
that allows us to compare the computational polynomial time (P) can also be
complexity of different problems. It enables us to solved using polynomial space
determine the relative difficulty of problems, (PSPACE).
classify them into complexity classes, and • Similarly, NP is contained within
establish relationships between classes. PSPACE, indicating that any
Reducibility provides a way to map one problem problem with a solution verifiable
to another, indicating that if the latter problem is in polynomial time (NP) can also
hard, the former problem is at least as hard. By be solved using polynomial space
using efficient reductions, we gain insights into (PSPACE).
the computational landscape of problems and
their complexities. 2. Reduction :-

• Reduction is another important

concept in complexity theory that
 establishes relationships between the computational power and difficulty levels of
problems or classes. problems. Inclusion establishes containment
• A problem A is reducible to relationships, reductions compare the
problem B if an algorithm that computational complexity of different problems,
solves problem B can be used to completeness identifies representative problems
solve problem A. within classes, and equivalence explores the
• Reductions are typically done in possibility of classes being the same in terms of
polynomial time, meaning that the computational power. These concepts aid in
transformation from problem A to classifying problems, analyzing their complexity,
problem B should be efficient. and understanding the boundaries of efficient
• If problem A is reducible to computation.
problem B and problem B is hard,
then problem A is at least as hard
as problem B.

3. Completeness :-

• Completeness establishes a
problem as representative or
complete for a specific complexity
class.
• For example, a problem is NP-
complete if it is in the NP class
and every problem in NP can be
reduced to it in polynomial time.
• NP-complete problems serve as
benchmarks for the difficulty level
of problems within NP, as any
NP-complete problem is as hard as
the hardest problems in NP. HERE IS A BRIEF EXPLANATION ABOUT
HARDNESS AND COMPLETENESS
4. Equivalence :-
Hardness :-
• Equivalence denotes that two
In computer science, hardness refers to the
complexity classes are essentially
difficulty of solving a particular problem. It is a
the same in terms of the problems
measure of how much time and computational
they contain and their
resources are required to find a solution for that
computational power.
problem. If a problem is hard, it means that
• For instance, if P = NP, it implies
there is no known efficient algorithm (a step-by-
that all problems in NP can be
step process) that can solve the problem quickly
solved in polynomial time, making
for all possible inputs.
P and NP equivalent.
• However, the question of whether It's important to understand that "hardness" in
P = NP or other complexity class this context refers to the difficulty of solving the
equivalences remain unsolved problem, not the difficulty of the problem itself.
problems in computer science.
Example :- The Traveling Salesman Problem (TSP)
In summary, the hierarchy and relationships
One classic example of a hard problem is the
between complexity classes provide insights into
Traveling Salesman Problem (TSP). Imagine a
salesperson who needs to visit multiple cities and The Hamiltonian Cycle Problem is an example of
return to their starting point while traveling the an NP-hard problem. It asks whether a given
shortest distance possible. The TSP asks, "What is graph contains a cycle that visits every vertex
the shortest possible route that visits each city exactly once. It is known to be NP-hard but not
exactly once and returns to the starting city?" known to be in NP. However, if we find an
This problem is notoriously difficult because the efficient algorithm for this problem, we can
number of possible routes grows exponentially efficiently solve any problem in NP, making it an
with the number of cities, making it hard to find NP-complete problem.
the best solution quickly as the number of cities
Summary :-
increases.
In summary, hardness describes how difficult it is
Completeness :-
to solve a specific problem efficiently, while
Completeness, on the other hand, relates to a completeness refers to the property of a problem
special property of some problems that allows that makes it capable of solving all other
them to serve as a benchmark for the entire class problems in its class efficiently. NP-complete
of problems they belong to. If a problem is problems are the hardest problems within the NP
complete, it means that every other problem in class, and they play a significant role in
that class can be efficiently transformed into an understanding the difficulty of various
instance of that complete problem. Solving the computational problems and their relationships.
complete problem would then provide a solution
for all the other problems in the class.

Example: Boolean Satisfiability (SAT)

One example of a complete problem is the

Boolean Satisfiability Problem (SAT). In SAT, we
are given a boolean expression consisting of
variables and logical operators (AND, OR, NOT).
The question is whether there exists an
assignment of truth values to the variables that
makes the whole expression true. SAT is complete
for the complexity class NP, which means that
any problem in NP can be transformed into an
instance of SAT efficiently. If we can find an
efficient algorithm to solve SAT, we can solve
any problem in NP efficiently.

NP-Hard and NP-Complete:

There is another term, NP-hard, that is related to

completeness. An NP-hard problem is one that is
at least as hard as the hardest problems in the
NP class. It might belong to a larger complexity
class than NP. However, an NP-complete problem
is both NP-hard and belongs to the NP class. In
other words, NP-complete problems are the
hardest problems within NP.

Example: The Hamiltonian Cycle Problem