Classifying Movie Reviews: A Binary Classification Example

The IMDB dataset is commonly used for sentiment analysis as a binary classification
task. It contains 50,000 movie reviews, evenly split into training and testing sets, with an equal
number of positive and negative reviews. The reviews have been preprocessed into sequences of
integers, where each integer represents a specific word in a predefined dictionary. This task
involves building a machine learning model to classify reviews as expressing positive or
negative sentiment, using natural language processing (NLP) techniques.

Preparing the data: The data preparation process involves:

• Limiting the dataset to the 10,000 most frequent words for manageable vector data size.
• Converting lists of integers (word indices) into tensors through one-hot encoding, creating
vectors of 0s and 1s to represent words
• Vectorising labels, transforming them into numerical representations suitable for the neural
network.
Building the network: A simple stack of fully-connected (Dense) layers with relu activations is
used. The architecture includes:
• Two intermediate layers with 16 hidden units each, employing the relu (rectified linear unit)
activation function.
• A final layer with a sigmoid activation function to output a probability score between 0 and 1,
representing the likelihood of a review being positive8....
Validation and prediction:
The model is compiled using the RMSPROP optimizer and the binary_crossentropy loss
function, a standard choice for binary classification problems involving probabilities. Training is
conducted for a limited number of epochs to prevent over fitting, which occurs when the model
performs well on training data but poorly on unseen data. Once trained, the model can predict the
sentiment of new reviews using the predict method.

Implementation:

Step 1: Import Necessary Libraries

Use Python libraries like TensorFlow or PyTorch for building the neural network. For example:

import tensorflow as tf
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Embedding, LSTM, Dense
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.datasets import imdb
Step 2: Load and Preprocess the Data
1. Tokenization: Convert text into tokens (words or sub words).
tokenizer = Tokenizer(num_words=10000) # Keep only top 10,000 words
tokenizer.fit_on_texts(reviews) # Fit tokenizer on the dataset
sequences = tokenizer.texts_to_sequences(reviews) # Convert to sequences

2. Padding: Ensure all sequences are of the same length by padding or truncating them.
padded_sequences = pad_sequences(sequences, maxlen=100# Fixed length of 100
3. Split Data: Divide into training and test sets (e.g., 80%-20% split).

Step 3: Build the Neural Network Model: Use a simple feed forward neural network or a more
advanced model like LSTMs or GRUs for sequential data.

model = Sequential ([Embedding (input_dim=10000, output_dim=128,

input_length=100), # Embedding layer

LSTM (64, return_sequences=False), # LSTM layer for sequential processing
Dense (1, activation='sigmoid') # Output layer for binary classification ])

Step 4: Compile the Model: Define the loss function, optimizer, and evaluation metric.
model.compile (optimizer='adam', loss='binary_crossentropy',metrics=['accuracy'])

Step 5: Train the Model: Fit the model to the training data.
model.fit (padded_sequences_train, labels_train, epochs=10, batch_size=32,
validation_data = (padded_sequences_test, labels_test))

Step 6: Evaluate the Model: Test the model's performance on unseen data.
loss, accuracy = model. evaluate (padded_sequences_test, labels_test)
print (f "Test Accuracy: {accuracy}")

Example:

 Input: "This movie was fantastic!" → Model processes text.

 Prediction: Outputs a probability (e.g., 0.92 for positive).
 Label: Assigns positive if the probability > 0.5, otherwise negative.

Applications

 Sentiment Analysis for understanding audience feedback.

 Recommender Systems: Use sentiment classification to enhance recommendations.
 Marketing Insights: Analyze trends in reviews for promotional strategies.
Classifying News Wires: A Multiclass Classification
Approach
In today's digital era, an enormous amount of news is published daily, covering topics like
politics, sports, technology, health, and more. Organizing this vast information is challenging.
Multi-class classification helps by assigning each news article to one specific category or class,
making it easier to process and retrieve information. This technique is widely used in
applications like news aggregation websites, search engines, and recommendation systems.

The Reuters Dataset: A Benchmark for Text Classification

The Reuters dataset is a popular resource for testing text classification models. It includes short
newswires labeled with their topics, making it a valuable tool for researchers. The dataset
contains 8,982 training examples and 2,246 test examples, offering a challenging yet informative
way to refine multiclass classification techniques. The example used is classifying Reuters
newswires into 46 different topics

Data Preparation: The Foundation for Success

Before training a machine learning model, preparing the data carefully is essential. Here are the
main steps:

Feature Engineering:

 Feature Selection: To simplify the data and reduce computation, the dataset is limited to
the most frequent words, usually the top 10,000.
 Tokenization: Text is broken into smaller units (tokens), such as words or subwords, for
processing.

Vectorization:

 One-Hot Encoding: Words are represented as binary vectors, with '1' marking the index
of the word in the vocabulary and '0' elsewhere.
 Bag-of-Words (BoW): This method counts how often each word appears in a document,
ignoring the order of words.
 TF-IDF (Term Frequency-Inverse Document Frequency): Words are weighted based
on how important they are within a document and the entire dataset.
 Word Embeddings (Word2Vec, GloVe, FastText): Words are represented as dense
vectors that capture their meanings and relationships in a continuous space.
Label Encoding:

 One-Hot Encoding: Each category is represented as a binary vector with a '1' for the
corresponding category.
 Integer Encoding: Each category is assigned a unique number (e.g., Politics: 0, Sports:
1, Technology: 2).

Data Normalization:

Features are scaled to a common range (e.g., 0 to 1) to ensure they all contribute equally during
model training.

Train-Test Split:

The dataset is divided into training and testing sets to evaluate the model’s performance on
unseen data.

Building the Network: A Deep Learning Approach

To classify newswires, a deep learning model can be built with the following architecture:

 Input Layer: Takes the vectorized news article as input.

 Embedding Layer (Optional): Maps input vectors to dense vectors, capturing
relationships between words.
 Hidden Layers: Several densely connected layers with activation functions like ReLU
(Rectified Linear Unit) learn complex patterns in the data. Wider layers (e.g., 64 units)
can prevent information loss when handling many output classes.
 Output Layer: A dense layer with a softmax activation function outputs probabilities for
each of the 46 possible categories. Softmax ensures the probabilities add up to 1,
indicating the likelihood of the article belonging to each category.

Training and Evaluation

Compilation:

 The model is compiled using an optimizer (e.g., RMSprop) and a loss function.
 Loss Function: Categorical cross entropy is commonly used for multiclass classification
with one-hot encoded labels. It measures the difference between predicted and true label
distributions.

Training:

 The model is trained on the training data over multiple epochs (iterations).
 A validation set monitors training to prevent overfitting, where the model performs well
on training data but poorly on new data.
Evaluation:

 After training, the model is tested on the test set using metrics like accuracy, precision,
recall, and F1-score to measure its performance.

. Implementation in Keras

Here’s a sample implementation:

from keras. datasets import reuters

from keras. preprocessing. Text import Tokenizer
from keras. utils import to_categorical
from keras. models import Sequential
from keras. layers import Dense, Dropout, Embedding, Flatten

# Load Reuters dataset

(x_train, y_train), (x_test, y_test) = reuters. load_data(num_words=10000)

# Preprocess data
tokenizer = Tokenizer(num_words=10000)
x_train = tokenizer. sequences_to_matrix (x_train, mode='binary')
x_test = tokenizer. sequences_to_matrix (x_test, mode='binary')

# Convert labels to one-hot encoding

y_train = to_categorical (y_train, num_classes=46)
y_test = to_categorical (y_test, num_classes=46)

# Model architecture
model = Sequential ([Dense (512, activation='relu', input_shape= (10000,)),
Dropout (0.5),
Dense (46, activation='softmax’) ])

# Compile model
model. compile (optimizer='adam', loss='categorical_crossentropy',
metrics=['accuracy'])

# Train model
model.fit (x_train, y_train, epochs=10, batch_size=32, validation_split=0.2)

# Evaluate model
test_loss, test_acc = model. evaluate (x_test, y_test)

print (f "Test Accuracy: {test_acc}")