Problem Statement

In the digital age, the massive volume of news content makes it difficult for
readers to find relevant information. News classification is essential for organizing
articles into categories, improving access to specific topics and enhancing user
experience. Given the broad range of topics and the limitations of manual
categorization, automating the process with Natural Language Processing (NLP)
and machine learning is more efficient. This approach also enables personalized
recommendations, sentiment analysis, and trend detection. The project aims to
develop a robust classification model to streamline news categorization and
improve information access for users.
Objectives
1. Categorize News Reports: Accurately classify news articles into predefined
categories (e.g., politics, sports, entertainment).
2. Analyze Categorization Accuracy: Compare the performance of the Bag of
Words (BOW) model and TF-IDF in terms of classification accuracy.
3. Evaluate Misclassification Errors: Identify and analyze patterns in
misclassification errors to understand their causes.
4. Optimize Hyperparameters: Tune hyperparameters for the BOW model to
enhance classification performance and reduce errors.
5. Provide Insights for Improvement: Summarize findings and recommend
strategies to improve automated news categorization systems.
Implementation
Implementation
1. Data Collection: News articles are loaded from a CSV file and stored in a Pandas DataFrame
for analysis.
2. Data Preprocessing: Critical steps include handling missing and duplicated data, formatting
date columns, cleaning text (removing HTML, emojis, URLs, punctuation, stopwords), and
tokenizing the text for further processing.
3. Feature Extraction: A Bag of Words model is used to convert text into a numerical format for
machine learning.
4. Model Training: The system uses a Multinomial Naive Bayes classifier, with the dataset split
into 80% training and 20% testing.
5. Evaluation: Model performance is assessed using accuracy, confusion matrix, and classification
reports. Cross-validation ensures generalization.
6. Hyperparameter Tuning: Randomized Search CV is used to optimize parameters for feature
extraction and classification.
7. Final Model Evaluation: The model's results are visualized using bar charts, highlighting correct
and incorrect predictions for performance insights.
Dataset
Dataset
Result Analysis
• Dataset Shape and Quality: The initial exploration revealed the dataset's dimensions,
providing insights into the number of entries and features. The first few rows were
printed to visualize the structure and content of the data, ensuring its relevance and
integrity.
• Missing Values and Duplicates: An assessment for missing values was performed,
revealing any gaps that could affect model training. Similarly, checks for duplicate
entries were essential to maintain the quality of the dataset. The absence of significant
missing values or duplicates suggested that the dataset was largely clean and ready for
further processing.
Result Analysis
Cross-Validation:
◦ A five-fold cross-validation approach was applied, yielding mean accuracy
scores. This method involved partitioning the data into three subsets and training
the model on two subsets while validating it on the third. This iterative approach
enhances confidence in the model's predictive capability.

Cross-Validation Results
The cross-validation process using Stratified K-Folds revealed consistent performance across
different data splits. The accuracy scores for each of the three folds were as follows:
• Fold 1: 0.8806
• Fold 2: 0.8800
• Fold 3: 0.8806
This led to a mean accuracy of 0.88, indicating that the model was stable and performed well
regardless of the specific data split. The cross-validation helped ensure that the model's
performance was not overly dependent on any single train-test split.
Result Analysis
Fine-tuning
To further optimize the model, a RandomizedSearchCV was employed for hyperparameter tuning.
This method explored different configurations of the CountVectorizer and Multinomial Naive Bayes
hyperparameters, aiming to identify the combination that would yield the best performance. The following
parameters were considered:

• max_features: [5000, 10000, None]

• ngram_range: [(1, 1), (1, 2)]
• alpha: A range of values between 0.1 and 2.0 (for smoothing in Naive Bayes)
After conducting 25 fits, the best hyperparameters identified were:
• max_features: None
• ngram_range: (1, 2)
• alpha: 1.92
This optimized model was then re-trained on the training data, and when tested, it achieved a
significant improvement in accuracy, reaching 0.933 on the test set.
Result Analysis
Error Analysis
To further analyze the model's performance, the number of correct and incorrect predictions
was examined:

• Correct Predictions: 37,246

• Wrong Predictions: 2,696
This shows that the model correctly categorized a substantial portion of the test data,
although
there were still areas for improvement, particularly in reducing the number of
misclassifications.
A visualization of the prediction outcomes highlighted the proportion of correct versus wrong
predictions, with correct predictions (green) vastly outnumbering incorrect ones (red).
Result Analysis
Bag of Words:

The Bag of Words (BoW) model is a simple yet effective technique in Natural Language Processing that
converts text into numerical data by representing each document as a collection of word counts. It creates
a vocabulary of all unique words in a dataset and then transforms each document into a vector, where
each element represents the frequency of a specific word in that document. The model disregards word
order and grammar, focusing solely on word occurrence. BoW is commonly used for tasks like text
classification, where numerical representations of text are required for machine learning algorithms to
process.
Result Analysis
Multinomial Naive Bayes with Bag of Words
from sklearn.naive_bayes import MultinomialNB
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.pipeline import make_pipeline
from sklearn.metrics import accuracy_score,classification_report

model = make_pipeline(CountVectorizer(), MultinomialNB())

model.fit(X_train, y_train)
y_pred = model.predict(X_test)

accuracy = accuracy_score(y_test, y_pred)

print(f"MultinomialNB with Bag of Words accuracy: {accuracy:.3f}")
# Print classification report
print("Classification Report:\n", classification_report(y_test, y_pred))

In this code, the Bag of Words (BoW) model, implemented through CountVectorizer(), converts the text data from the news
articles into a numerical matrix by counting the occurrences of each word in the dataset's vocabulary. This transformed data is then fed
into the Multinomial Naive Bayes classifier (MultinomialNB()), which is effective for text classification tasks. The model is trained on
the X_train data, predicts labels for the X_test data, and evaluates its performance using metrics like accuracy and a classification
report. This pipeline efficiently processes text data for classification using BoW and Naive Bayes.
Implementation
○ qwerty
Implementation
Implementation
Implementation
Implementation
Implementation
Implementation
Conclusion
Visualization and Interpretation of Results
To provide a clearer understanding of the model's performance, several visualizations were
created:
• Correct vs. Wrong Predictions:
◦ A bar chart comparing correct and incorrect predictions illustrated the model's
effectiveness visually. The chart showed a substantial number of correct
predictions (green) relative to incorrect ones (red), reinforcing the model's
reliability.
• Final DataFrame of Predictions:
◦ A summary DataFrame was generated, showcasing the content of the articles
alongside predicted and actual labels. This presentation allows for
straightforward comparisons and highlights specific cases of misclassification,
which can be critical for further analysis and model refinement.
Conclusion
1. Bag of Words vs. TF-IDF: The model using Bag of Words (BoW) performed better than when
using TF-IDF, achieving higher accuracy. This suggests that BoW was more effective in capturing
the relevant features for this specific news classification task.
2. Optimal Hyperparameters: The best performance was obtained using the hyperparameters:
◦ max_features: None
◦ ngram_range: (1, 2)
◦ alpha: 1.92
3. Model Accuracy: After fine-tuning, the optimized model achieved an accuracy of 0.933,
with 37,246 correct predictions and 2,696 wrong predictions, reflecting a highly effective
classification system.
4. Potential for Improvement: Exploring other machine learning models (e.g., Support
Vector Machines or neural networks) might further enhance accuracy and reduce
misclassification.
References
1. Scikit-learn Documentation: Text Classification
https://scikit-learn.org/stable/tutorial/text_analytics/working_with_text_data.html
2. Natural Language Processing with Python
https://www.nltk.org/book/
3. Introduction to Natural Language Processing (NLP)
https://towardsdatascience.com/natural-language-processing-nlp-in-python-a-beginners-guide-5c93f0a7b4a6
4. Understanding Multinomial Naive Bayes for Text Classification
https://towardsdatascience.com/multinomial-naive-bayes-for-text-classification-5c30e1e473c7
5. Cross-Validation in Machine Learning
https://scikit-learn.org/stable/modules/cross_validation.html
THANK YOU