Models for Ambiguity Resolution in Parsing

• Ambiguity resolution in parsing is essential for natural language processing

(NLP) and compiler design, as sentences or expressions often have multiple
possible interpretations.

• Various models help in resolving ambiguity, including probabilistic, syntactic,

semantic, and contextual approaches.
Types of Ambiguity Resolution

There are Three Types of Ambiguity Resolution in Parsing

1. Probabilistic Context-Free Grammar (PCFG)

2. Generative Model for Parsing

3. Discriminative Model for Parsing

 Probabilistic Context-Free
Grammar (PCFG)
• An extension of Context-Free Grammar (CFG) where each production
rule is assigned a probability.
• Helps in selecting the most probable parse tree when multiple structures
are possible.
Example:s
• Sentence: "I saw the man with the telescope."
• PCFG assigns different probabilities to two possible parse trees and selects the most
likely one.
 Introduction to PCFG

• Probabilistic Context-Free Grammar (PCFG) is an

extension of Context-Free Grammar (CFG) where each
production rule is assigned a probability. These
probabilities help in choosing the most likely parse tree
when multiple structures are possible for a sentence.
• PCFG is widely used in Natural Language Processing
(NLP), speech recognition, and syntactic parsing to
resolve ambiguities.
PCFG Algorithm – Parsing with Probabilities

Algorithm Steps:
• Input: A sentence to be parsed.
• Generate Parse Trees: Using CFG rules, generate all possible parse trees for
the sentence.
• Compute Probabilities:
• Multiply the probabilities of each rule used in a parse tree.
• The probability of a parse tree T is:

• where P(Ri) is the probability of each production rule applied in the tree.
• Select the Most Probable Parse Tree: The tree with the highest probability is
chosen as the correct parse.
 Real-Time Example of PCFG Parsing
Step1: Sentence:

"The dog chased the cat."

Step 1: Define the PCFG Rules

java
Aspect "chased" "slept"
S → NP VP [1.0]
NP → Det N [0.6] Intransitive
Transitive
NP → N [0.4] Action verb (no
verb (needs
Type object
VP → V NP [0.7] an object)
needed)
VP → V [0.3] 60% (More 40% (Less
Probability
Det → "the“ [1.0] frequent) frequent)
N → "dog" [0.5] Requires a
N → "cat" [0.5] subject and an Only needs a
Sentence
V → "chased" [0.6] object subject (The dog
Structure
V → "slept" [0.4]
(The dog chased slept)
the cat)
Generate Possible Parse Trees

• Step 2: Generate Possible Parse

Trees
• Two possible parse trees:
• Tree 1:
• S
• / \
• NP VP
• / \ / \
• Det N V NP
• | | | / \
• "the" "dog" "chased" Det N
• | |
• "the" "cat"
Algorithm for Parsing a Sentence
PCFG
Step 1: Start from S
Since S → NP VP with 100% probability (1.0), we must expand S into NP VP.
Step 2: Expand NP
Two choices:
•NP → Det N (0.6)
•NP → N (0.4)
Let's choose NP → Det N (which happens 60% of the time).
Step 3: Expand Det and N
•Det → "the" (1.0)
•N → "dog" (0.5) or "cat" (0.5)
Let's choose "dog" (50% probability).
Now, NP = "the dog".
Algorithm for Parsing a Sentence
Step 4: Expand VP
Two choices:
•VP → V NP (0.7)
•VP → V (0.3)
Let's choose VP → V NP (which happens 70% of the time).
Step 5: Expand V
•V → "chased" (0.6) or "slept" (0.4)
Let's choose "chased" (60% probability).
Algorithm for Parsing a Sentence
Step 6: Expand the Second NP
Two choices:
•NP → Det N (0.6)
•NP → N (0.4)
Let's choose NP → N (40% probability).
Step 7: Expand N
•N → "dog" (0.5) or "cat" (0.5)
Let's choose "cat" (50% probability).
Final Sentence & Probability Calculation

Generated Sentence:
✅ "The dog chased cat"
Now, let's compute the probability of generating this sentence:
1.S → NP VP = 1.0
2.NP → Det N = 0.6
3.Det → "the" = 1.0
4.N → "dog" = 0.5
5.VP → V NP = 0.7
6.V → "chased" = 0.6
7.NP → N = 0.4
8.N → "cat" = 0.5
Total probability =1.0 × 0.6 × 1.0 × 0.5 × 0.7 × 0.6 × 0.4 × 0.5 = 0.0252 (or
Total probability = 2.52%)
 Final Sentence & Probability
Calculation (Example 2)
Let's generate a different sentence following another probabilistic path.
Step 1: Start with S → NP VP
This is always 1.0.
Step 2: Expand NP
We choose NP → N (40% probability).
•N → "cat" (50% probability).
So, NP = "cat".
Step 3: Expand VP
We choose VP → V (30% probability).
V → "slept" (40% probability).
So, VP = "slept".
 Final Sentence

✅ "Cat slept"
Probability Calculation
1.S → NP VP = 1.0
2.NP → N = 0.4
3.N → "cat" = 0.5
4.VP → V = 0.3
5.V → "slept" = 0.4
Total probability =1.0 × 0.4 × 0.5 × 0.3 × 0.4 = 0.024 (or 2.4%)
 Comparison of Sentence Probabilities

Comparison of Sentence
Probabilities
Sentence Probability

The dog chased cat 2.52%

Cat slept 2.4%

 Generative Model for Parsing

• Parsing is the process of analyzing a sequence of words to determine its

grammatical structure. Generative models predict how likely a given parse tree is
for an input sentence by modeling the probability distribution of syntactic
structures.

Applications: Natural Language Processing (NLP), speech recognition, code

analysis.
Key Models:
• PCFG (Probabilistic Context-Free Grammar): Uses rules with probabilities to generate
valid sentences.
• Neural Parsers: Use deep learning (RNNs, Transformers) to predict parse structures.
Algorithm for Parsing (General Steps)

• Algorithm (PCFG-based Parsing)

• Define the grammar: Create rules with probabilities (e.g., S → NP VP
[0.9]).
• Tokenize input: Break the input sentence into words/tokens.
• Apply parsing algorithm: Use dynamic programming (like CKY
Algorithm) or neural models to find the best parse.
• Compute probabilities: Evaluate different parse trees and select the
most probable one.
• Output the parse tree: Return the structured representation.
Key Generative Parsing Models:

• Probabilistic Context-Free Grammar (PCFG) – Uses

probability rules to construct parse trees.
• Hidden Markov Models (HMMs) – Used for sequence
generation and syntactic parsing.
• Neural Generative Parsing (RNN, Transformer-based
models) – Learns probability distributions over tree
structures.
 Example 1: PCFG Parsing (Probabilistic
Context-Free Grammar)
Grammar Rules with Probabilities:
S → NP VP [0.9]
NP → Det N [0.6] | Pronoun [0.4]
VP → V NP [0.7] | V [0.3]
Det → "The" [1.0]
N → "dog" [0.8] | "cat" [0.2]
V → "barks" [1.0]
 Sentence: "The cat sleeps."
Possible Parse Tree:
S
/\
NP VP
/ \ |
Det N V
| | |
The cat sleeps
 Sentence: "The dog barks."

S [0.9]
Sentence: "The dog barks." / \
Possible Parse Trees and Their Probabilities: NP VP [0.7]
/ \ |
Tree 1 (Correct Structure) Det N V
| | |
The dog barks
 Probability Calculation:
• P(S → NP VP) = 0.9
• P(NP → Det N) = 0.6
• P(Det → "The") = 1.0
• P(N → "dog") = 0.8
• P(VP → V) = 0.3
• P(V → "barks") = 1.0
Total Probability = 0.9 × 0.6 × 1.0 × 0.8 × 0.3 × 1.0 = **0.1296**
 Example 2: HMM-Based Parsing (Sequence Prediction)
Sentence: "I saw a man with a telescope."
Goal: Predicting part-of-speech (POS) tags.
HMM States (POS tags):
•"I" → Pronoun
•"saw" → Verb
•"a" → Determiner
•"man" → Noun
•"with" → Preposition
•"a" → Determiner
•"telescope" → Noun
HMM predicts the most likely sequence of tags and disambiguates sentence meaning based on
transition probabilities.
Example 3: Neural Generative Parsing (Transformer-based)

A neural sequence-to-sequence model (like GPT) can generate syntactic parse trees.
Input Sentence: "She enjoys playing chess."
Generated Parse Tree (Dependency Format):
enjoys
/ \
She playing
|
chess