Scribd
Upload
6 views7 pages

Neuralnetworks Research Assignment

Uploaded by

tpmsomrolc.bharathsl11c
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
6 views7 pages

Neuralnetworks Research Assignment

Uploaded by

tpmsomrolc.bharathsl11c
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 7

Ex.

No: 5
Sequencial Modeling-RNN

Aim: Using python program implement SimpleRNN using short and long
sequences dataset.

Data Description:

1. IMDb Dataset (Short Sequences)


 Description: The IMDb dataset consists of 50,000 highly
polarized movie reviews labeled as either positive or
negative. It is often used for binary sentiment classification
tasks.
 Data Split:
 25,000 reviews for training
 25,000 reviews for testing
 Input Format: The reviews are preprocessed into sequences of
integers, where each integer represents a word in the review.
Typically, the length of sequences is padded to a fixed size
(e.g., 100 or 200 words) to standardize input length.
 Task: Sentiment classification (positive or negative).
 Sequence Length: Considered short sequences, typically ranging
from 100 to 200 words (depending on the padding).

2. AG News Dataset (Long Sequences)


 Description: The AG News dataset is a collection of news
articles categorized into four classes: World, Sports,
Business, and Science/Technology. It consists of over 120,000
training samples and 7,600 testing samples.
 Data Split:
 120,000 samples for training
 7,600 samples for testing
 Input Format: The news articles are converted into sequences
of integers, similar to the IMDb dataset. However, news
articles are longer than movie reviews, so they are often
considered long sequences.
 Task: Text classification into one of four categories (World,
Sports, Business, Science/Technology).
 Sequence Length: Considered long sequences, with articles
often exceeding 200 words, typically truncated or padded for
sequence classification tasks.
Code:

import tensorflow as tf

from tensorflow.keras.datasets import imdb

from tensorflow.keras.preprocessing import sequence

max_features = 10000

maxlen_short = 250

(x_train, y_train), (x_test, y_test) =


imdb.load_data(num_words=max_features)

x_train = sequence.pad_sequences(x_train, maxlen=maxlen_short)

x_test = sequence.pad_sequences(x_test, maxlen=maxlen_short)

print(f"IMDb dataset - x_train shape: {x_train.shape}, y_train


shape: {y_train.shape}")

from datasets import load_dataset

from tensorflow.keras.preprocessing.text import Tokenizer

from tensorflow.keras.preprocessing.sequence import pad_sequences

# Load AG News dataset using Huggingface datasets

ag_news = load_dataset('ag_news')
# Separate into training data

texts = ag_news['train']['text']

labels = ag_news['train']['label']

# Tokenize and convert to sequences

tokenizer = Tokenizer(num_words=10000) # Use the same `max_fea-


tures` as before

tokenizer.fit_on_texts(texts)

sequences = tokenizer.texts_to_sequences(texts)

# Pad sequences for long sequences

maxlen_long = 500 # As before, 500 for long sequences

data = pad_sequences(sequences, maxlen=maxlen_long)

print(f"AG News dataset - shape: {data.shape}, labels shape:


{len(labels)}")

import time

import pandas as pd

def build_rnn_model(input_length, vocab_size, num_layers,


num_units, dropout_rate, optimizer):

model = Sequential()

model.add(Embedding(vocab_size, 128,
input_length=input_length))

for i in range(num_layers):

return_sequences = (i < num_layers - 1)


model.add(SimpleRNN(num_units, return_sequences=return_se-
quences))

model.add(Dropout(dropout_rate))

model.add(Dense(1, activation='sigmoid'))

model.compile(optimizer=optimizer, loss='binary_crossentropy',
metrics=['accuracy'])

return model

def evaluate_rnn_model(input_length, vocab_size, x_train, y_train,


x_test, y_test, config):

start_time = time.time()

model = build_rnn_model(input_length=input_length,
vocab_size=vocab_size,num_layers=config['num_layers'],num_units=co
nfig['num_units'],dropout_rate=config['dropout_rate'],optimizer=co
nfig['optimizer'])

print("Started Training")

history = model.fit(x_train, y_train, epochs=5, batch_size=64,


validation_data=(x_test, y_test), verbose=0)

print("Finished Taining")

end_time = time.time()

elapsed_time = end_time - start_time

final_acc = history.history['val_accuracy'][-1]

final_loss = history.history['val_loss'][-1]

return {'accuracy': final_acc, 'loss': final_loss, 'time':


elapsed_time}
rnn_configs = [

{'num_layers': 1, 'num_units': 32, 'dropout_rate': 0.5, 'opti-


mizer': 'adam'},

{'num_layers': 2, 'num_units': 64, 'dropout_rate': 0.3, 'opti-


mizer': 'adam'},

{'num_layers': 3, 'num_units': 128, 'dropout_rate': 0.4, 'op-


timizer': 'sgd'}

results_rnn_short = []

for config in rnn_configs:

result = evaluate_rnn_model(maxlen_short, max_features,


x_train, y_train, x_test, y_test, config)

result.update(config)

results_rnn_short.append(result)

rnn_short_df = pd.DataFrame(results_rnn_short)

print("RNN Short Sequences Results:")

print(rnn_short_df)

import numpy as np

data = np.array(data)
labels = np.array(labels)

results_rnn_long = []

for config in rnn_configs:

result = evaluate_rnn_model(maxlen_long, max_features, data,


labels, data[:500], labels[:500], config)

result.update(config)

results_rnn_long.append(result)

rnn_long_df = pd.DataFrame(results_rnn_long)

print("RNN Long Sequences Results:")

print(rnn_long_df)

Result:

RNN Short Sequences Results:

RNN Long Sequences Results:

Result Analysis:
The results show that for short sequences (IMDb), increasing the
number of RNN layers and units generally improves accuracy, with a
2-layer model achieving the best performance (76.18% accuracy),
while the 3-layer model with SGD struggles. For long sequences (AG
News), the models perform poorly, with accuracy consistently
around 8-10%, and a significant decrease in loss (suggesting in-
stability in training). This indicates that RNNs might not be
well-suited for handling long sequences, potentially due to van-
ishing gradient issues or the need for more sophisticated archi-
tectures like LSTMs.

Conclusion:

Thus, RNN using short and long sequences datasets were implemented
successfully, executed and verified.

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy