Context Free Grammars

Grammar: Grammar is a recursive definition of Language

(Natural or Programming)
Formally: Grammar G = {V,T,P,S}
Terminals: Basically T =
Variables: Non terminal symbols that represent sets of
strings being defined recursively
Start Symbols S: S belongs to V and is a special symbol
that generates the desired language
Production rules P: Recursive definitions
Note: T, V and P are always finite sets.

Context Free Grammars
L
eq
Example: The grammar: G
eq
= (V,T,P,S)
T = {0,1}
V = {S, A, B}
P = { S |oA|1B
A 1S|0AA
B 0S|1BB}
Notations: For set of rules
A
1
, A
2
, A
3
Short cut:
A
1
|
2
|
3
Context Free Grammars
Context Free Grammars (CFG): Are only allowed to
have production rules for substitution of the form:
A
1
,
2
,
3
,
k
Where: LHS A belongs to V
RHS
i
belongs to V U T for all i
Non Context Free Grammars: Rules might specify
context in which rules substitution can be performed.
e.g. 0A1 0
1

2

3

k
1
Thus rules cannot be applied in other contexts.
Context Free Grammars
Language of CFG G = {V, T, P, S} is defined as

Context Free Language is any language which has a Context
Free Grammar G
Terminology:
Sentence: Any w T* such that S => w
Sentential form: any (V U T)* such that S =>
Terminals: a, b, c
Variables: A, B, C,
Terminal Strings: , u, v, w, x, y, z
V U T: , X, Y, T
Sentential Form: , , ,
} | * { ) ( w S T w G L e =

Context Free Grammars
Reading assignment: From Textbook, (2
nd
edition,)
Theorem 5.7 (which talks about language of palindromes)
Example: Arithmetic Expressions
G = {V, T, P, S}
V = {E, I}
S = E;
T = { x, y, z, +, *, (, )}
P = { E I|(E)|E+E|E*E
I x|y|z }
Apply: E => E+E => E+E*E => I+E*E => x+E*E => x+I*E
=> x+y*E => x+y*I => x+y*z
Context Free Grammars
Non-Determinism: Consider E => E+E*E
Possible derivation steps:
Expand any one of three Es
Use any one of four rules for E
Non-deterministically decide
Which variable?
Which rule?
Observe: Deciding which rule to apply is important non-
determinism which gives CFG its power and flexibility
But deciding which variable to expand only serves to
confuse us (and or the parser)
Context Free Grammars
Left-most derivation: Always substitute the leftmost
variable with a production rule in sentential form arising in
course of a derivation.
Notation:
Example: E => E+E => I+E => x+E.

Similarly: We can define right-most derivations
Example: E => E+E => E=E*E => E=E*I.
Now we can talk about canonical derivation sequence
and

Context Free Grammars
Derivation Trees: Also called as parse trees in compilers
Idea is to represent the derivation pictorially




This tree represents the derivation x+y*z
In General to represent A =>
Root labeled A
Leaves in left to right order gives
Internal nodes are labeled with variables and their
children specify the production rule applied to them
Context Free Grammars
A-Tree: Any tree (or subtree) rooted at variable A.
But simply call it a parse tree if rooted at S.
Yield|Frontier of a tree is the sequence of leaves labeled
from left to right order
Theorem: is the yield of an A tree which implies and is
implied by the fact that A =>
Proof: By induction on the height of tree (See textbook)
Observe: Preceding parse tree does not specify a unique
way to derive from A
In fact it removes the non-determinism of which rules to
apply but leaves the order of application unspecified.
Context Free Grammars
Leftmost derivation(lm) is obtained by traversing the
tree in depth-first order always going into left subtrees
before right ones



Similarly, Rightmost derivation(rm) comes from depth-
first traversal going into right subtrees first.




Context Free Grammars
Claim: Following are all equivalent statements.
For CFG G = (V, T, P, S) and string w T*
a) w L(G)
b) S =(LM)=>w
c) S =(RM)=>w
d) There exists an S-tree which yields w
Of course: we could always use leftmost derivations to
specify a canonical way to derive any w belonging to L(G)
or convert parse tree to unique derivations Thus
simplifying the task of parsers.
But, what if some w belonging to L(G) has two distinct
parse trees and hence two distinct leftmost derivations?
Context Free Grammars
Example: x+y*z






Note: This is not just a syntactic problem, as we get two
different semantic interpretations
Tree1: x+(y*z)
Tree2: (x+y)*z

Context Free Grammars
Definition: A CFG is ambiguous if for some w L(G),
there exists more than one distinct parse tree.

In compilers parse trees determine interpretation and we
cannot allow ambiguity.

Of course we can force use of parenthesis, but we
should really redesign the grammar to be unambiguous
by encoding precedence of operators. (See textbook for
redesigning of grammars)

While above grammar can be redesigned to be
unambiguous, it is not always possible to do that.

Context Free Grammars
Definition: A CFG is called Inherently Ambiguous if all
its grammars are ambiguous.
Example: L = {a
n
b
n
c
m
d
m
| n,m1} U {a
n
b
m
c
m
d
n
|n.m1}
Consider the strings of the form a
k
b
k
c
k
d
k
,
We can never tell whether this string came from first or
second type of strings in L and any CFG must allow both
of these possibilities.
Push Down Automata(PDA)
PDA is a class of machines corresponding to CFGs
(Accepting only the Context Free Languages) useful in
designing parsers based on CFG
As we have discussed before, we must give PDA,
unbounded memory to allow it to handle non-regular
languages
However, we will restrict its access to memory!

Push Down Automata(PDA)
Setup: A PDA on transition
1. Consumes an input symbol
2. Goes to a new state(or stays in the old)
3. Replaces top of the stack by any string (does nothing,
pops the stack or pushes a string onto the stack)





Push Down Automata (PDA) is essentially an -NFA with
stack
Push Down Automata(PDA)
Stack Notation

Content: (top)ABBAC(bottom)
Pop: returns A; new content BBAC
Push(XYZ): new content XYZBBAC
Transitions:
Determined by:
Input or -move
Current state
Stack top
Effect:
New state
Pop
Push new string
Push Down Automata(PDA)
Formally: Push Down Automata is a seven tuple
represented as
M = {Q, , , , q
0
,
0
, F}
Where:
Q is finite set of states
is finite set of input alphabet
is finite set of stack alphabet
is the transition function
q
0
, is the start state
is the start symbol for stack, and
is the set of accepting states
*
2 } { :
I
I E
Q
Q c o
I e Z
0
Q F e
Push Down Automata(PDA)
Transition function: takes as argument a triple given as


Suppose we have (q, a, X) then
1. q is a state in Q
2. a is either an input symbol in or a = , the empty string which
is not to be assumed an input symbol.
3. X is the stack symbol in

Output of is finite set of pairs (p
i
,
i
) where p
i
is the new state
and
i
is the string of stack symbol that replaces X.
(q,a,X) = {(p
1
,
1
), (p
2
,
2
), }
*
2 } { :
I
I E
Q
Q c o
Push Down Automata(PDA)
Action: First PDA pops stack top to determine X, reads input
to determine a (unless it is an -transition) then knowing q, a, X
it selects non-deterministically one of the possibilities of (p
i
,
i
)
Finally:
State: goes from q to p
i
Input: scans past a (unless a = )
Stack: Loses old top symbol X but gets i pushed onto
it.
Note: We thus need Z
0
on stack initially to allow the first
transition to pop the stack
Convention: String in
*
: x, y, z
String in
*
: , ,
Push Down Automata(PDA)
Example: L = {o
n
1
n
|n 1}
PDA M: = {0,1} = {X, Z
0
} Q = {q
0
, q
1
, q
2
} F = {q
2
}
Transitions:
(q
0
, 0, Z
0
) = {(q
0
, XZ
0
)} [On input 0 add X to stack]
(q
0
, 0, X) = {(q
0
, XX)} [On input 0 add X to stack]

(q
0
, 1, X) = {(q
1
, )} [On input 1 switch to q
1
and
consume X]
(q
1
, 1, X) = {(q
1
, )} [On input 1 keep consuming Xs]

(q
1
, , Z
0
) = {(q
2
, )} [When Z
0
is found, consume
it and move to final state q
2
]
Push Down Automata(PDA)
Transition Diagram :





Remarks:
1. Will reject inputs not of the format 0*1* by not having
any transition defined.
2. If too few 1s , will never go to q2
3. If too many 1s will get stuck in q2 without reaching end
of input.
Push Down Automata(PDA)
Instantaneous description(ID): Succinct notation for describing the
entire configuration of PDA mid-stream in an execution
ID = <q, x, >
Where, q: current state
x: unread input
: Stack content
Acceptance by a PDA: PDA accepts input w if there is at least one
trace of executions which leads to final state when end of input is
reached
Rejection by PDA:
When no transition is possible (Stuck)
If input not over but stack is empty
If input is over but in non-final state
Of course this must happen on every track to reject w.
Push Down Automata(PDA)
Language of a PDA: Let M = {Q, , , , q
0
,
0
, F} be a PDA,
the language accepted by M i.e.

This theorem just says <q
0
, w, Z
0
> can possibly lead to <p,,>
but that is only one of the many possibilities.
Remark: This is called Final State Language, as it is accepted
by reaching one of the final states when input ends.
Empty Stack Language: We ignore the final state completely
and consider the stack being empty as a sign of acceptance.

As usual we say, w is accepted if for at least one execution
trace, when w is consumed, the stack is empty.
} , , , , , | { ) (
0 0
*
F p p Z w q w M L e > < > < E e = o c
} , , , , | { ) (
0 0
*
> < > < E e = c c p Z w q w M L
Push Down Automata(PDA)
Rejection in such cases: Along every execution trace one of
the following happens
Before w is over, stack gets empty
When w is over, stack is not empty
Before w is over, PDA gets stuck
Example: L = {ww
r
|w{a,b}*}
PDA M: Q = {q
1
, q
2
}, F =
= {a,b} = {A,B,Z
0
}
Goal: Accept by empty stack
Push Down Automata(PDA)
Idea:
1. q
0
pushes w onto stack one by one
2. Guess mid point of w and move to q
1
3. In q
1
, match input with stack top, one by one
4. At end, Z
0
should be at top, so remove it to halt and
accept
Key: In step3, stack pops w in reverse order.





Push Down Automata(PDA)
Example(Contd.) Input = aabbaa
Execution trace <q
0
, aabbaa, Z
0
>
<q
0
, abbaa, AZ
0
>
<q
0
, bbaa, AAZ
0
>
<q
0
, baa, BAAZ
0
>
<q
1
, baa, BAAZ
0
>
<q
1
, aa, AAZ
0
>
<q
1
, a, AZ
0
>
<q
1
, , Z
0
>
<q
1
, , > Accept by empty stack
Remark: Since PDA is non-deterministic, other execution
traces are possible
Push Down Automata(PDA)
Equivalence of language acceptance:
Theorem: L = L(P
f
) for some PDA P
f
is equivalent to some
PDA
L = N(P
n
) for some PDA P
n
Proof: Given P
f
= {Q, , , , q
0
,
0
, F} construct M
2
such that
N(M
2
) = L(M
1
)
M
2
= {Q
2
, ,
2
,
2
, p
0
, X
0
, }
with: Q
2
= Q U {p
0
, p}

2
= U {X
0
}
N in N(M) stands for null stack or empty stack
Push Down Automata(PDA)

2
: Idea
Start in p
0
with X
0
on stack
Move to q
0
with Z
0
X
0
on stack
Simulate P
f
From any final states of P
f
add transition to p which will
just empty the stack
X
0
: It prevents accidental acceptance by P
n
when P
f
empties
its stack and rejects
Push Down Automata(PDA)
The reverse: Given P
n
= {Q
2
, ,
2
,
2
, p
0
, X
0
, }
construct P
f
such that L(P
f
) = N(P
n
)
Idea: P
f
= {Q
1
, ,
1
,
1
, p
0
, X
0
, F
1
}
where, Q
1
= Q U {p
0
, p
f
}

1
= U {X
0
}
F
1
= {p
f
}
Idea:
Start with p
0
with X
0
in stack
Move to q
0
with Z
0
X
0
on stack
Simulate P
n
When P
n
empties its stack, it exposes X
0
From all states add an -move to p
f
whenever X
0
in on top of
the stack
Push Down Automata(PDA)
Push Down Automata(PDA)
Equivalence of CFGs and PDA
Claim: Every CFL is accepted by some PDA and every PDA
accepts some CFG
Theorem 1: If L is CFL L = N(M) for some PDA M
Proof: Suppose G is a CFG for L
Our goal: Construct a PDA M for G such that L(M) = N(M)
Idea: PDA M simulates LM derivations in G for input w such
that at any step the sentential form is represented by
a) A sequence of symbols consumed from input w by M
b) Followed by contents of Ms stack

Push Down Automata(PDA)
Formally given CFG G = (V, T, P, S)
Construct PDA M = {Q, , , , q
0
,
0
, }
with Q = {q}, q
0
= q, = T, = VUT, Z
0
= S
Defining : Two types
1. If terminal a is on stack top, then expect to see an a in
input and consume both note no change in sentential
form
2. If variable A is on stack top, then replace it by RHS of
any of its production rule in P note no change in input
consumed.
Thus:
(q, , A) = {(q, 1), (q, 2), , (q, k)}
Where A 1| 2|| k are in P
(q, a, a) = {(q, )}
V Ae
T ae
Push Down Automata(PDA)
Example: Consider G
S AS|
A 0A1|A1|01
PDA: M = {{q}, {0,1}, {0,1,A,S}, , q, S, }
: (q, , S) = {(q, AS), (q, )}
(q, , A) = {(q, 0A1), (q, A1), (q, 01)}
(q, 0, 0) = {(q, )}
(q, 1, 1) = {(q, )}
Push Down Automata(PDA)
Execution: Consider w = 011
In G: S AS A1S 011S 011
In M: <q, 011, S> | <q,011, AS>| <q,011, A1S>
| <q,011, 011S>| <q,11, 11S>
| <q,1, 1S>| <q, , S>
| <q, , >
Observe: one to one correspondence between LM derivation
and execution trace.
Of course there are many execution traces possible each
corresponding to a distinct derivations.
Beside that, observe if two distinct execution accepts w, there
exists two distinct LM derivations and thus the grammar is
ambiguous.
Push Down Automata(PDA)
In theorem, our construction heavily relied on the power of
non-determinism to allow the machine to guess the correct
derivation
But in real life (or in parsers/YACC), we dont have non-
deterministic power
So we need to convert PDAs to some form of deterministic
PDA
Definition: DPDA is a PDA with 2 restrictions:
a) (q, a, Z) has 1possibility
b) If (q, , Z) is defined then for all a , (q, a, Z) is
empty
Context Free Grammars
Simplification of Context Free Grammars: In CFG, we try to
eliminate those symbols and productions which are not useful
for derivation of sentences.
Useful Symbols: Any X belonging to VUT such that
S X w
With w T* and , (VUT)*
Thus useless symbols are those symbols which do not take
part in derivations and can safely eliminate without changing
language
Defn: X VUT is generating if X w for w *
Defn: X VUT is reachable if S X for , (VUT)*
Clearly: useful X is both generating and reachable
Context Free Grammars
Idea: Identify useless symbols by removing:
Step1: Non generating Xs
Step2: Non reachable Xs,
and all their productions
Observe: must do it in this order,
Example S AB|a
A b
Suppose we do Step2 first, all symbols are reachable so when we
do Step1 next we eliminate B as being non-generative
But if we do it in right order, we first eliminate B in Step1 and also
eliminate the production S AB
Now in Step2 we find that A is non-reachable so we eliminate A as
well.
In general we perform both steps recursively.
Context Free Grammars
Step1: Eliminating non-generative symbols
Basis: Label all terminals in T as generating
Induction: For all production: X X
1
X
2
X
k
, if each X
i
is
generating then X is generating.
Terminate when no new generating symbol could be found

Step2: Eliminate non-reachable symbols
Basis: S is reachable
Induction: For all production: X X
1
X
2
X
k
if X is reachable,
then label each X
1
, X
2
, X
3
X
i
as being reachable.
Context Free Grammars
Example: S AB|AC|CD
A BB
B AC|ab
C Ca|CC
D BC|b|d
Step1: Base: {a, b, d} is generating
{a, b, d, A, B, D} is generating
{a, b, d, A, B, D, S} is generating
As C is not found to be generating, remove C and all the
production that contain C either on LHS or RHS.
Context Free Grammars
New grammar:
G2: S AB
A BB
B ab
D b|d
Step2: Reachable?
Base: {S} is reachable
{S, A, B} is reachable
{S, A, B, a, b} is reachable
Remove D and all productions that contain D either in LHS or
RHS
Context Free Grammars
Finally: G3: S AB
A BB
B ab
Removing -moves: -moves slows down the parser
Definition: X belonging to V is nullable if X
Idea: Find nullable symbols recursively
Basis: If P contains A , then label A as nullabel
Induction: For all productions X X
1
X
2
X
3
X
k
, if X
i
is nullable,
label X as nullable
Terminate when no new symbol could be found
Context Free Grammars
Example: G1: S ABC|BCB
AaB|a
BCC|b
CS|
Finding nullable: Base: {C} is nullable
{B,C} is nullable
{S, B, C} is nullable
Suppose we eliminate C
Originally we could derive
S BCB, S CB, S BC, S BB, S C, S B
But now we cant. So we must add in all these effects via direct
productions.
Context Free Grammars
Overall algorithm:
a) Identify all nullable symbols
b) Replace any prod X X
1
X
2
X
3
X
k
by set of productions of
the form X
1

2

3

k
, where;
a)
i
=X
i
if X
i
is non-nullable
b)
i
=X
i
or if X
i
is nullable
c) Remove all -productions
So in previous example: new G2 becomes
S ABC|AB|AC|A|BCB|BC|CB|BB|B|C|
A aB|a
B CC|C|b|
C S|
Now remove all -productions.
Context Free Grammars
Glitch: Originally S was possible, but after final step we do lose
from L(G) This is unavoidable
Removing unit productions ( e.g. A B)
Algorithm: Step 1: Remove -productions
Step 2: For all X, Y belonging to V
if X Y and Y is not unit
then add X
Step 3: eliminate all unit productions
Finding X Y
Since no -production, X Y only if
X Y
1
Y
2
Y
3
Y
With all Y
i
being distinct. Thus k |V|. Can use reachability in
directed graphs

Context Free Grammars
Example: G: S A|B
A Sa|a
B S|b
Algorithm: S A, S B
B S, B A
Get S Sa|a|b|S|A|B
A Sa|a
B Sa|a|b|S|A|B
Removing unit productions
S Sa|a|b
A Sa|a
B Sa|a|b
Observe: A and B are now useless as not reachable
Context Free Grammars
Question to remove useless | -Prod|unit prod all together,
does order matter?
Observe:
a) Removing useless stuff cannot add -Prod|unit prod
b) Removing -Prod could add unit productions
c) Removing unit productions
a) Need to remove -Prod first
b) Could create useless symbols but not -Prod.
Thus use following order:
1. -Productions
2. Unit productions (no epsilons added)
3. Useless symbols (No productions added)
Context Free Grammars
Chomsky Normal Form:
CFG G is in Chomsky Normal Form(CNF) if all its productions
are of the form
A a
A XY,
Theorem: Given any CFG G
1
with not in language L(G
1
)
we can find CNF grammar G
2
such that
L(G
1
) = L(G
2
)
Construction: Three step process:
Step1: Eliminate unit productions and -productions
Now all productions are of the form
A a
A X
1
X
2
X
k
, with X
1
, X
2
, X
k
belongs to V U T
Context Free Grammars
Step 2: Remove mixed bodies
For each a belonging to T add new variable Va and
V
a
a
In each production A X
1
X
k
replace a by V
a
Now all productions are of the form
A a
A A
1
A
k
with A
i
belonging to V
Step 3: Factor long productions
For A A
1
A
2
A
k
, for k 3
Add new variables B
1
B
2
B
k-2

Context Free Grammars
Replace A A
1
A
k
By A A
1
B
1
B
1
A
2
B
2
B
2
A
3
B
3
B
k-2
A
k-1
A
k
Verify: Get CNF grammar and Language is preserved
Example: G1: S ABB|ab
A Ba|ba
B aAbB

Context Free Grammars
Step 2: V
a
a
V
b
b
S ABB|V
a
V
b
A BV
a|
V
b
V
a
B V
a
AV
b
B
Context Free Grammars
Step 3: V
a
a
V
b
b
S AX
1
|V
a
V
b
X
1
BB
A BV
a
|V
b
V
a
B V
a
Y
1
Y
1
AY
2
Y
2
V
b
B
Context Free Grammars
Greibach Normal Form (GNF)
Theorem: A CFG G is in Greibach Normal Form if every
production is of the form
A a, where belongs to V* and a belongs to .
Note: = (Allowed)
GNF is a natural generalization of regular grammar. In
regular grammar the productions are of the form A a,
where
} {c o e E e V and a
Context Free Grammars
Modifying productions(assume doesnt start with V)
Modification 1: Productions of type A B:
For any production of the form A B, where we have
other productions of the form B
1
|
2
||
k
, replace this
particular A-production with
A
1
|
2
||
k

Modification 2: Productions of the form A A:
For productions of the form A
A
1
|A
2
||A
k
|
1
|
2
||
m
, Let Z be a new variable.
Define new productions as follows:
a) A
1
|
2
||
m
, A
1
Z|
2
Z||
l
Z
b) Z
1
|
2
||
k
, Z
1
Z|
2
Z||
k
Z
Context Free Grammars
Steps 1: Eliminate all -productions and construct a grammar
G
1
in Chomsky Normal form
Rename all variables as A
1
, A
2
, A
3
, A
n
, where S = A
1
Step 2: Apply modification 1 on productions of type A
i
A
j
,
where j < i
Step 3: Apply modification 2 on productions of type A A
Step 4: Apply modification 1 on productions of type A
i
A
j
,
where j > i
Step 5: Modify Z productions to convert them to the form Z
a
____________________________________
Example: Convert G
1
= (V,T,P,S) defined as S AA|a, A
SS|b to a grammar G
2
in GNF.
Step 1: G
1
is already in CNF so rename variables as A
1
= S, A
2

= A
Such that, A
1
A
2
A
2
|a, A
2
A
1
A
1
|b
Context Free Grammars
Step 2: A
1
prod are in required form. A
2
b is in required
form
Modify A
2
A
1
A
1
. So resulting prod are
A
2
A
2
A
2
A
1
|aA
1
Production P now becomes:
A
1
A
2
A
2
|a
A
2
A
2
A
2
A
1
|aA
1
|b
Step 3: Apply modification 2 on A
2
prods. Let Z be new
variable.
A
2
aA
1
, A
2
b
A
2
aA
1
Z, A
2
bZ
Z A
2
A
1
, Z A
2
A
1
Z
Context Free Grammars
Step 4: Modify A
1
A
2
A
2
using modification 1. As A
2

productions are A
2
aA
1
|b|aA
1
Z|bZ,
the set of modified A
1
productions is:
A
1
a|aA
1
A
2
|bA
2
|aA
1
ZA
2
|bZA
2
Step 5: Modify Z productions. Z productions are
Z A
2
A
1
|A
2
A
1
Z
Applying modification 1, it becomes,
Z aA
1
A
1
|bA
1
|aA
1
ZA
1
|bZA
1
Z aA
1
A
1
Z|bA
1
Z|aA
1
ZA
1
Z|bZA
1
Z
Context Free Grammars
The resulting grammar thus has following productions
rules:
A
1
a|aA
1
A
2
|bA
2
|aA
1
ZA
1
|bZA
2
A
2
aA
1
|b|aA
1
Z|bZ
Z aA
1
A
1
|bA
1
|aA
1
ZA
1
|bZA
1
Z aA
1
A
1
Z|bA
1
Z|aA
1
ZA
1
Z|bZA
1
Z