Scribd
Upload
3 views10 pages

Artificial Intelligence Lab 7

Uploaded by

rehanmunir276
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
3 views10 pages

Artificial Intelligence Lab 7

Uploaded by

rehanmunir276
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 10

Artificial Intelligence Lab

Lab No: 07

Name: Rehan Munir Awan


Enrollment No: 01-132222-036
Date: 30-May-2025
Submitted To: Sir Syed Muhammad Usman

DEPARTMENT OF COMPUTER ENGINEERING


BAHRIA UNIVERSITY ISLAMABAD
CAMPUS
Implementation of KNN & Naïve Bayes Classification
Model
Objective:
The objective of this lab is to implement and understand the working
principles of two fundamental classification algorithms: K-Nearest
Neighbors (KNN) and Naïve Bayes. Through this practical exercise,
students will gain hands-on experience in training, testing, and
evaluating these models on real-world datasets. The lab aims to
compare their performance, accuracy, and decision-making
approaches in different scenarios. Additionally, it will enhance
students’ ability to preprocess data, choose suitable parameters, and
interpret model outcomes, thereby strengthening their foundational
knowledge in machine learning and pattern recognition.
Software Used:
• Visual studio code
Introduction:
In the field of machine learning, classification is a fundamental task
that involves assigning labels to input data based on learned patterns.
Among the most widely used classification algorithms are K-Nearest
Neighbors (KNN) and Naïve Bayes. KNN is a non-parametric,
instance-based learning algorithm that classifies data points based on
the majority label of their nearest neighbors in the feature space. In
contrast, Naïve Bayes is a probabilistic classifier based on Bayes’
Theorem, assuming independence between features. This lab explores
the implementation and comparison of these two models, highlighting
their strengths, limitations, and suitable applications. By applying
these algorithms to real-world data, the lab provides practical insights
into how machine learning can be used for effective decision-making
and predictive analytics.
LAB TASK
Task No: 01
Write a program to implement KNN classifier and classify given
vector. (for k = 3)
Code:
from sklearn.neighbors import KNeighborsClassifier
x = [[25, 40000], [35, 60000], [45, 80000], [20, 20000], [35, 120000],
[52, 18000], [23, 95000], [40, 62000], [60, 100000], [48, 220000]]
y = [0, 0, 0, 0, 0, 0, 1, 1, 1, 1]
test = [[48, 142000]]
model = KNeighborsClassifier(n_neighbors=3)
model.fit(x, y)
output = model.predict(test)
print("Predicted class:", output[0])
Output:

Task No: 02
Modify Task 1 and perform it for all odd values of k from 1 to 10.
• Find accuracy for each value of k and display.
• Find the best k which gives highest value of accuracy and worst
k which gives worstaccuracy
• Also compute confusion matrix for both best and worst value of
k.
Code:
from sklearn.metrics import confusion_matrix, accuracy_score
data = [[25, 40000], [35, 60000], [45, 80000], [20, 20000], [35,
120000],
[52, 18000], [23, 95000], [40, 62000], [60, 100000], [48,
220000]]
labels = [0, 0, 0, 0, 0, 0, 1, 1, 1, 1]
test_data = [[33, 150000], [48, 142000]]
test_labels = [1, 1]
best_k = 1
worst_k = 1
high_acc = 0
low_acc = 1

for k in [1, 3, 5, 7, 9]:


model = KNeighborsClassifier(n_neighbors=k)
model.fit(data, labels)
preds = model.predict(test_data)
acc = accuracy_score(test_labels, preds)
print("K =", k, "Accuracy =", acc)
if acc > high_acc:
high_acc = acc
best_k = k
if acc < low_acc:
low_acc = acc
worst_k = k
print("Best k is", best_k)
print("Worst k is", worst_k)
model = KNeighborsClassifier(n_neighbors=best_k)
model.fit(data, labels)
p1 = model.predict(test_data)
print("Confusion Matrix for best k:")
print(confusion_matrix(test_labels, p1))
model = KNeighborsClassifier(n_neighbors=worst_k)
model.fit(data, labels)
p2 = model.predict(test_data)
print("Confusion Matrix for worst k:")
print(confusion_matrix(test_labels, p2))
Output:
Task No: 03
Implement KNN algorithm yourself in python for Iris Dataset without
using built-in KNNclassifier library.
• Load dataset
• Split dataset into test and train sets
• Perform KNN algorithm to make predictions for k=5
• Compute accuracy and confusion matrix
Code:
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from collections import Counter
import numpy as np
from sklearn.metrics import accuracy_score, confusion_matrix
data = load_iris()
x = data.data
y = data.target
train_x, test_x, train_y, test_y = train_test_split(x, y, test_size=0.2)
def calc_dist(a, b):
return np.sqrt(np.sum((a - b) ** 2))
def my_knn(train_x, train_y, test_x, k):
final = []
for test_point in test_x:
dists = []
for j in range(len(train_x)):
d = calc_dist(test_point, train_x[j])
dists.append((d, train_y[j]))
dists.sort()
votes = [label for _, label in dists[:k]]
pred = Counter(votes).most_common(1)[0][0]
final.append(pred)
return final
preds = my_knn(train_x, train_y, test_x, 5)
print("Accuracy is:", accuracy_score(test_y, preds))
print("Confusion matrix is:")
print(confusion_matrix(test_y, preds))
Output:

Task No: 04
Develop a python program to implement Bayesian classification
model for the following dataset and classify the given test vector:
Code:
from sklearn.naive_bayes import GaussianNB

data = [[25, 40000], [35, 60000], [45, 80000], [20, 20000], [35,
120000],
[52, 18000], [23, 95000], [40, 62000], [60, 100000], [48,
220000], [33, 150000]]

labels = [0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1]

model = GaussianNB()
model.fit(data, labels)
result = model.predict([[48, 142000]])
print(result)
Output:

Task No: 05
Use the given cancer dataset and classify it using Bayesian
classification model:
• First create a python script and load ‘breast_cancer’ file.
• Identify features and classes from the loaded dataset.
• Perform 2-fold cross validation on the dataset by splitting it into
testing and training parts.
• Implement a Bayesian classifier using the above algorithm and
use training dataset to classify each of the sample within testing
dataset.
• Compute the accuracy from the predicted test samples.
Code:
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import KFold
from sklearn.naive_bayes import GaussianNB
from sklearn.metrics import accuracy_score, confusion_matrix
data = load_breast_cancer()
x = data.data
y = data.target
kf = KFold(n_splits=2)
fold = 1
for train_index, test_index in kf.split(x):
print(f"\n--- Fold {fold} ---")
x_train = x[train_index]
x_test = x[test_index]
y_train = y[train_index]
y_test = y[test_index]
model = GaussianNB()
model.fit(x_train, y_train)
pred = model.predict(x_test)
print("Accuracy:", accuracy_score(y_test, pred))
print("Confusion Matrix:")
print(confusion_matrix(y_test, pred))

fold += 1
Output:

Conclusion:
In conclusion, the implementation of K-Nearest Neighbors and Naïve
Bayes classification models provided valuable insights into two
distinct approaches to solving classification problems. KNN
demonstrated its effectiveness through simplicity and flexibility,
especially when dealing with well-structured data, while Naïve Bayes
showed strong performance in cases where the assumption of feature
independence holds true. Through this lab, we observed the
importance of selecting appropriate algorithms based on dataset
characteristics and problem requirements. Overall, the practical
application of these models helped reinforce theoretical concepts and
improved our understanding of how different machine learning
techniques can be applied to real-world data classification tasks.

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy