Scribd
Upload
7 views5 pages

DL 4

Uploaded by

953622243068
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
7 views5 pages

DL 4

Uploaded by

953622243068
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 5

EXP.

NO:04
LANGUAGE MODELING USING RNN
Aim:
To write a program for language modeling using RNN.
Algorithm:
STEP1: Start.
STEP2: Import Libraries:Import necessary libraries such as NumPy for
handling arrays and TensorFlow/Keras for building and training the neural
network.
STEP3: Create Dataset:Define a small dataset of sentences (e.g., "I love pizza").
This dataset will be used to train the model to predict the next word.
STEP4: Tokenize Sentences:Use the Tokenizer to convert the words in each
sentence into numerical values (e.g., "pizza" becomes 3). The tokenizer builds a
vocabulary and assigns a number to each unique word.
STEP5: Convert Sentences to Sequences:Transform each sentence into a
sequence of numbers using the tokenizer. For example, "I love pizza" becomes
[1, 2, 3] where each number corresponds to a word.
STEP6: Create Input (X) and Output (y):For each sentence, split it into smaller
parts:
X: A sequence of the first words (input).
y: The next word in the sequence (output).
STEP7: Pad Sequences:Ensure all sequences in X have the same length by
padding shorter sequences with zeros. This makes them the same size, which is
required for training the model.
STEP8: Build CNN Model:Create a simple Convolutional Neural Network
(CNN) model. The model includes:
Embedding layer: Converts numbers into word vectors.
Conv1D layer: Applies convolution to capture word patterns.
Max Pooling: Reduces the size of the feature maps.
Dense layers: Fully connected layers for word prediction.
STEP9: Compile the Model,Compile the model with:
Optimizer: Adam, for faster learning.
Loss Function: Sparse categorical cross-entropy, to measure prediction errors.
Train the Model:Train the model using the input X and output y for several
epochs (e.g., 500). The model learns to predict the next word in the sequence.
STEP10: Predict Next Word:Write a function to predict the next word given a
phrase (e.g., "I love"). The function uses the model to find the word with the
highest probability and returns the prediction.
STEP11: Stop.
PROGRAM:
import numpy as np
import tensorflow as tf
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, Conv1D,
GlobalMaxPooling1D, Dense

sentences = [
'I love pizza',
'I love dogs',
'I play football',
'Dogs are loyal',
'Pizza is delicious'
]

tokenizer = Tokenizer(num_words=1000)
tokenizer.fit_on_texts(sentences)
word_index = tokenizer.word_index
print("Word Index:", word_index)
sequences = tokenizer.texts_to_sequences(sentences)
print("Sequences:", sequences)

X = []
y = []
for seq in sequences:
for i in range(1, len(seq)):
X.append(seq[:i])
y.append(seq[i])

X = pad_sequences(X, maxlen=3)
y = np.array(y)
print("Padded X:", X)
print("y:", y)

model = Sequential([
Embedding(1000, 8, input_length=3),
Conv1D(64, 3, activation='relu'),
GlobalMaxPooling1D(),
Dense(32, activation='relu'),
Dense(1000, activation='softmax') ])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
model.fit(X, y, epochs=500, verbose=2)
def predict_next_word(model, tokenizer, text, max_len=3):
sequence = tokenizer.texts_to_sequences([text])[0]
sequence = pad_sequences([sequence], maxlen=max_len)
prediction = model.predict(sequence, verbose=0)
predicted_word_index = np.argmax(prediction)
for word, index in tokenizer.word_index.items():
if index == predicted_word_index:
return word
return ""
test_sentence = "I love"
predicted_word = predict_next_word(model, tokenizer, test_sentence)
print(f"Next word prediction for '{test_sentence}': {predicted_word}")
Output:

Result:
Thus the above program was executed successfully.

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy