TABLE OF CONTENTS

Page MARKS
EX.NO DATE NAME OF THE EXPERIMENT No AWARDED REMARKS

1 a Implementation of Uninformed search

algorithms - BFS
b Implementation of Uninformed search
algorithms - DFS
2 a Implementation of Informed search algorithms -
A*
b Implementation of Informed search algorithms -
memory-bounded A*
3
Implementation of naïve Bayes models
4 Implementation of Bayesian Networks
5
Build Regression models
6
a Build decision trees

b Build random forests

7
Build SVM models
8
Implement ensembling techniques
9
Implement clustering algorithms
10
Implement EM for Bayesian networks
11
Build simple NN models

12 Build deep learning NN models

13 Implement of applications of back propagation
algorithm.
Ex No. 1a Implementation of Uninformed Search Algorithms – BFS

Aim :

To implement the BFS algorithm using python.

Algorithm:

1. To begin, place any one vertices of our graph at the lower extreme of the queue.
2. Add the very first element in the created queue to the list of objects that have already
been checked out.
3. Create a list of all the nodes that seem to be near that vertex. Individual nodes
which are not in the visited list should be moved to the rear of the queue.
4. Repeat the above two steps, i.e., steps 2 and 3, till our queue is reduced to 0

Program:

graph = { '1':
['2','5','3'],
'2':['6', '4'],
'5':['4'],
'3':['4','7'],
'6':[],
'4':[],
'7':[]
}
visited=[]
queue=[]
def bfs(visited,graph,node):
visited.append(node)
queue.append(node)
while queue:
m = queue.pop(0)
print(m)
for neighbour in graph[m]:
if neighbour not in visited:
visited.append(neighbour)
queue.append(neighbour)
print("Following is the Breadth-First Search")
bfs(visited,graph,'1')
Output:

Result:
Thus, the program for the BFS algorithm using python was implemented, executed and
verified successfully.
Ex No. 1b Implementation of Uninformed Search Algorithms – DFS

Aim :

To implement the DFS algorithm using python.

Algorithm:

1. Start by putting any one of the graph’s vertices on top of a stack.

2. Take the top item of the stack and add it to the visited list.
3. Create a list of that vertex’s adjacent nodes. Add the ones which aren’t in the visited
list to the top of the stack.
4. Keep repeating steps 2 and 3 until the stack is empty.
Program:

graph = { '1':
['2','5','3'],
'2':['6', '4'],
'5':['4'],
'3':['4','7'],
'6':[],
'4':[],
'7':[]
}
visited = set()
def dfs(visited, graph, node):
if node not in visited:
print(node)
visited.add(node)
for neighbour in graph[node]:
dfs(visited, graph, neighbour)
print("Following is the Depth-First Search")
dfs(visited,graph,'1')
Output:

Result:
Thus, the program for the DFS algorithm using python was implemented, executed and
verified successfully.
Ex No. 2a Implementation of Informed Search Algorithm – A*

Aim :

To implement A* algorithm in python using search strategy.

Algorithm:

1. Define a Graph class that takes an adjacency list as input.

2. Implement a method called get_neighbors that returns the neighbors of a given node.
3. Implement a heuristic function h that returns the estimated cost from a given node
to the goal node.
4. Implement the A* algorithm by taking the start node and the stop node as input.
5. Once all neighbors of the current node have been examined, move the current node
from the open list to the closed list.
6. If the stop node is not found in the open list, then there is no path between the start and
the stop node, so return none.
Program:

from collections import deque

class Graph:
def init (self, adjacency_list):
self.adjacency_list = adjacency_list
def get_neighbors(self, v):
return self.adjacency_list[v]
def h(self, n):
H={
'A': 1,
'B': 1,
'C': 1,
'D': 1
}
return H[n]
def a_star_algorithm(self, start_node, stop_node):
open_list = set([start_node])
closed_list = set([])
g = {}
g[start_node] = 0
parents = {}
parents[start_node] = start_node
while len(open_list) > 0:
n = None
for v in open_list:
if n == None or g[v] + self.h(v) < g[n] +
self.h(n): n = v;
 if n == None:
print('Path does not exist!')
return None
if n == stop_node:
reconst_path = [] while parents[n] != n:
reconst_
path.app
end(n) n
=
parents[
n]
reconst_path.app
end(start_node)
reconst_path.rev
erse()
print('Path found:
{}'.format(reconst_path))
return reconst_path
for (m, weight) in self.get_neighbors(n):
if m not in open_list and m
not in closed_list:
open_list.add(m)
print('Path does not exist!')
return None
adjacency_list = {
'A': [('B', 1), ('C', 3), ('D', 7)],
'B': [('D', 5)],
'C': [('D', 12)]
}
graph1 = Graph(adjacency_list)
graph1.a_star_algorithm('A', 'D')

Output:

Result:
Thus, the program for the A* algorithm using python was implemented, executed and
verified successfully.
Ex No. 2b Implementation of Informed Search Algorithm
- Memory Bounded A*

Aim :

To implement memory bounded A* algorithm in python using search strategy.

Algorithm:

1. Set the initial threshold to ne the heuristic estimate of the distance from the start node to
the goal node.
2. While the threshold has not reached infinity :
2.1. Call the iterative_deepening_a_star_rec with the current threshold value.
2.2. If the return value is negative, print solution was found.
2.3. Else update the threshold to be the return value.
3. If the loop finishes without finding a solution, return -1.
4. Define the function iterative_deepening_a_star_rec.
5. Return the minimum distance.

Program:

V={}
E={}
V=({'A':7,'B':9,'C':6,'D':5,'E':6,'F':4.5,'H':4,'I':2,'J':3,'K':3.5,'G':0})
E=({('B','D'):2,('A','B'):4,('A','C'):4,('A','D'):7,('D','E'):6,('E','F'):5,('D','F'):8,('D','H'):5,('H','I'):3,
('I','J'):3,('J','K'):3,('K','H'):3,('F','G'):5})
INFINITY=10000000
cameFrom={}
def h(node):
return V[node]
def cost(node, succ):
return E[node,succ]
def successors(node):
neighbours=[]
for item in E:
if node==item[0][0]:
neighbours.append(item[1][0])
return neighbours
def reconstruct_path(cameFrom, current):
total_path = [current]
while current in cameFrom:
current = cameFrom[current]
total_path.append(current)
return total_path
 def ida_star(root,goal):
global cameFrom
def search(node, g, bound):
min_node=None global cameFrom f = g + h(node)
if f > bound:
return f
if node==goal:
return "FOUND"
minn = INFINITY
for succ in successors(node):
t = search(succ, g + cost(node, succ), bound) if t == "FOUND":return
"FOUND"
if t < minn: minn = t
min_node=succ cameFrom[min_node]=node return minn
bound= h(root) count =1
while 1:
print ("itertion"+str(count)) count+=1
t = search(root, 0, bound) if t == "FOUND":
print (reconstruct_path(cameFrom, goal)) return bound
if t == INFINITY:return "NOT_FOUND" bound = t
print (ida_star('A','G'))

Output:

Result:
Thus, the program for the Memory Bounded A* algorithm using python was
implemented, executed and verified successfully.
Ex No. 3 Implementation of Naïve Bayes Model

Aim :

To build a naïve bayes model using gaussian naïve bayes formula in python.

Algorithm:

1. Import the required math functions from the math module.

2. Define a function named separate_by_class that takes dataset as input.
3. Define a function named mean that takes in a list of numbers as input.
4. Calculate the sum of the numbers in the list using the sum function and get average.
5. Return the average as mean.
6. Define a function named stdev that takes a list of numbers to find mean.
7. Sum the squared difference calculated using list comprehension and divide by the length
of the list minus 1 to get the sample variance.
8. Return the standard deviation as the stdev.
9. Define a function summarize_dataset that takes in a dataset as its input.
10. Calculate the mean, standard deviation and count of each feature using the functions.
11. Return the list of summary tuples.
12. Calculate the prior probability for the class by dividing the count of rows for that class by
the total number of rows in the dataset.

Program:

from math import sqrt

from math import pi
from math import exp
def separate_by_class(dataset):
separated = dict()
for i in range(len(dataset)):
vector = dataset[i]
class_value = vector[-1]
if(class_value not in separated):
separated[class_value] = list()
separated[class_value].append(vector)
return separated
def mean(numbers):
return sum(numbers)/float(len(numbers))
def stdev(numbers):
avg = mean(numbers)
variance = sum([(x-avg)**2 for x in numbers])/float(len(numbers)-1)
return sqrt(variance)
def summarize_dataset(dataset):
summaries = [(mean(column),stdev(column),len(column)) for column in zip(*dataset)]
del(summaries[-1])
return summaries
def
summarize_by_class(d
ataset): separated =
separate_by_class(data
set) summaries = dict()
for class_value, rows in
separated.items():
summaries[class_value] =
summarize_dataset(rows)
return summaries
def calculate_probability(x,
mean, stdev): exponent =
exp(-((x-
mean)**2/(2*stdev**2)))
return
(1/(sqrt(2*pi)*stdev))*expone
nt
def calculate_class_probabilities(summaries,row):
total_rows = sum([summaries[label][0][2] for label
in summaries]) probabilities = dict()
for class_value, class_summaries in summaries.items():
probabilities[class_value] =
summaries[class_value][0][2]/float(total_rows) for i in
range(len(class_summaries)):
mean,stdev,_ = class_summaries[i]
probabilities[class_value]*=calculate_probability(row[i], mean, stdev)
return probabilities
dataset = [[3.393533211,2.331273381,0],
[3.110073483,1.781539638,0],
[1.343808831,3.368360954,0],
[3.582294042,4.67917911, 0],
[2.280362439,2.866990263,0],
[7.423436942,4.696522875,1],
[5.745051997,3.533989803,1],
[9.172168622,2.511101045,1],
[7.792783481,3.424088941,1],
[7.939820817,0.791637231,1]]
summaries = summarize_by_class(dataset)
probabilities =
calculate_class_probabilities(summaries,
dataset[0]) print(probabilities)
Output:

Result:
Thus, the program for the Naive Bayes algorithm using python was implemented,
executed and verified successfully.
Ex No. 4 Implementation of Bayesian Network

Aim :

To implement python code for Bayes theorem network.

Algorithm:

1. Import necessary libraries.

2. Create an empty BayesianModel object.
3. Add nodes to the BayesianModel object.
4. Add edges between nodes.
5. Define the conditional probability distributions for each node.
6. Add the defined TabularCPDs to the BayesianModel object.
7. If the model is valid, print Model is correct.

Program:

from pgmpy.models import BayesianModel

from pgmpy.factors.discrete import TabularCPD

from pgmpy.inference import VariableElimination
import numpy as np

bayesNet = BayesianModel()
bayesNet.add_node("M")
bayesNet.add_node("U")
bayesNet.add_node("R")
bayesNet.add_node("B")
bayesNet.add_node("S")
bayesNet.add_edge("M","R")
bayesNet.add_edge("U","R")
bayesNet.add_edge("B","R")
bayesNet.add_edge("B","S")
bayesNet.add_edge("R","S")

cpd_A = TabularCPD('M', 2, values=[[.95],[.05]])

cpd_U = TabularCPD('U', 2, values=[[.85],[.15]])
cpd_H = TabularCPD('B', 2, values=[[.90],[.10]])
cpd_S = TabularCPD('S', 2, values=[[.98,.88,.95,.6],[.02,.12,.05,.40]],
evidence = ['R','B'], evidence_card=[2,2])
cpd_R = TabularCPD('R', 2, values=[[.96,.86,.94,.82,.24,.15,.10,.05],[.04,.14,.06,.18,.76,.85,.9
 0,.95]],
evidence = ['M','B','U'], evidence_card=[2,2,2])
bayesNet.add_cpds(cpd_A,cpd_U,cpd_H,cpd_S,cpd_R)
bayesNet.check_model()
print("Model is correct")

Output:

Result:
Thus, the program for the Bayes Network using python was implemented, executed and
verified successfully.
Ex No. 5 BUILD REGRESSION MODELS

Aim :

To write a Python program to build Regression model..

Algorithm:

1. Initialize the parameters.

2. Predict the value of a dependent variable by giving an independent variable.
3. Calculate the error in prediction for all data points.
4. Calculate partial derivative w.r.t a0 and a1.
5. Calculate the cost for each number and add them.
6. Update the values of a0 and a1.

Program:

x=[14,16,27,42,39,50,83]
y=[2,5,7,9,10,13,20]
xy=0 x2=0
xy_all=0 x2_all=0 x_all=0
y_all=0
for i in range(7):
xy_all=xy_all+x[i]*y[i]
x2_all=x2_all+x[i]*x[i]
x_all=x_all+x[i]
y_all=y_all+y[i]
x_bar=x_all/7
y_bar=y_all/7
b=(xy_all-
7*x_bar*y_bar)/(x2_all-
7*x_bar*x_bar) a=(y_bar-
b*x_bar)
print(b)
print(a)
print("Y=",b,"(X)+",a)
Output:

Result:
Thus, the program for the Regression Model using python was implemented, executed
and verified successfully.
Ex No. 6a BUILD DECISION TREES

Aim :

To write a Python program to Build Decision trees.

Algorithm:

1. Initialize the parameters.

2. Predict the value of a dependent variable by giving an independent variable.
3. Calculate the error in prediction for all data points.
4. Calculate partial derivative w.r.t a0 and a1.
5. Calculate the cost for each number and add them.
6. Update the values of a0 and a1.

Program:

import math length={3:[2,0],4:[1,3],5:

[2,2]}
gills={'yes':[0,4],'no':[5,1]}
beak={'yes':[5,3],'no':[0,2]}
teeth={'many':[3,4],'few':[2,1]}
features=["length","gills","beak","teeth"]
entrophy=[]
te=[]
n=10;
val=0
for i in length.keys(): p=length[i][0]/(length[i]
[0]+length[i][1]) if(p== 0 or (1-p)== 0):
e=0;
else:
e = -p*math.log(p,2)-((1-p)* math.log((1-p),2))
entrophy=entrophy+[e]
j=0
for i in length.keys():
t=length[i][0]+length[i][1]
val+=(t/n)*entrophy[j]
j=j+1
print("Total Entrophy for Length",val) te=te+
[val]
val=0
entrophy=[]
for i in gills.keys():
p=gills[i][0]/(gills[i][0]+gills[i][1])
if(p== 0 or (1-p)== 0):
e=0;
else:
e = -p*math.log(p,2)-((1-p)* math.log((1-p),2))
entrophy=entrophy+[e]
j=0
for i in gills.keys():
t=gills[i][0]+gills[i][1]
val+=(t/n)*entrophy[j]
j=j+1
print("Total Entrophy for Gills",val)
te=te+[val]
val=0
entrophy=[]
for i in beak.keys():
p=beak[i][0]/(beak[i][0]+beak[i][1])
if(p== 0 or (1-p)== 0):
e=0;
else:
e = -p*math.log(p,2)-((1-p)* math.log((1-p),2))
entrophy=entrophy+[e]
j=0
for i in beak.keys():
t=beak[i][0]+beak[i][1]
val+=(t/n)*entrophy[j]
j=j+1
print("Total Entrophy for Beak",val)
te=te+[val]
val=0
entrophy=[]
for i in teeth.keys():
p=teeth[i][0]/(teeth[i][0]+teeth[i][1])
if(p== 0 or (1-p)== 0):
e=0;
else:
e = -p*math.log(p,2)-((1-p)* math.log((1-p),2))
entrophy=entrophy+[e]
j=0
for i in teeth.keys():
t=teeth[i][0]+teeth[i][1]
val+=(t/n)*entrophy[j]
j=j+1
print("Total Entrophy for Teeth",val) te=te+
[val]
j=0;
minval=te[0];
for i in range(1,4):
if(te[i]<minval):
minval=te[i]
print(minval)
for i in
range(4):
if(minval==te[i]):
break
print("The feature to put at the top is ",features[i])
Output:

Result:
Thus, the program for the Decision Tree using python was implemented, executed and
verified successfully.
Ex No. 6b BUILD RANDOM FOREST

Aim :

To write a Python program to Build Random Forest.

Algorithm:

1. In the Random forest model, a subset of data points and a subset of features is
selected for constructing each decision tree.
2. Individual decision trees are constructed for each sample.
3. Each decision tree will generate an output.
4. Final output is considered based on Majority Voting or Averaging for
5. Classification and regression, respectively.

Program:

from sklearn.ensemble import RandomForestClassifier

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix, accuracy_score
import matplotlib.pyplot as plt
iris = load_iris()
X = iris.data
y = iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
rf_clf = RandomForestClassifier(n_estimators=100, random_state=42)
rf_clf.fit(X_train, y_train)
y_pred = rf_clf.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)
print("Accuracy: {:.2f}%".format(accuracy*100))
conf_matrix = confusion_matrix(y_test, y_pred)
print("Confusion Matrix:\n", conf_matrix)
plt.plot(y_test,y_pred)
plt.show()
Output:

Result:
Thus, the program for the Random Forest using python was implemented, executed and
verified successfully.
Ex No. 7 BUILD SVM MODEL

Aim :

To write a Python program to Build SVM Model.

Algorithm:

1. Import the libraries needed.

2. Load the dataset.
3. Split the data set into X and Y.
4. Split the X and Y dataset into the training set and test set.
5. Perform feature scaling.
6. Fit SVM to the training set.
7. Predict the test set results.
8. Make the confusion matrix.
9. Visualize the test results

Program:

import matplotlib.pyplot as plt

from matplotlib import style
import numpy as np
style.use('ggplot')
class Support_Vector_Machine:
def init (self, visualization=True):
self.visualization = visualization
self.colors = {1:'r',-1:'b'}
if self.visualization:
self.fig = plt.figure()
self.ax = self.fig.add_subplot(1,1,1)
def fit(self, data):
self.data = data
opt_dict = {}
transforms = [[1,1], [-1,1], [-1,-1],[1,-1]]
all_data = []
for yi in self.data:
for featureset in self.data[yi]:
for feature in featureset:
all_data.append(feature)
self.max_feature_value = max(all_data)
self.min_feature_value = min(all_data)
all_data = None
 step_sizes = [self.max_feature_value * 0.1,
self.max_feature_value * 0.01,
self.max_feature_value * 0.001,
b_range_multiple = 2
b_multiple = 5
latest_optimum = self.max_feature_value*10
for step in step_sizes:
w = np.array([latest_optimum,latest_optimum])
optimized = False
while not optimized:
for b in np.arange(-1*(self.max_feature_value*b_range_multiple),
self.max_feature_value*b_range_multiple,
step*b_multiple):
for transformation in transforms:
w_t = w*transformation
found_option = True
for i in self.data:
for xi in self.data[i]:
yi=i
if not yi*(np.dot(w_t,xi)+b) >= 1:
found_option = False
if found_option:
opt_dict[np.linalg.norm(w_t)] = [w_t,b]
if w[0] < 0:
optimized = True
print('Optimized a step.')
else:
w = w - step
norms = sorted([n for n in opt_dict])
opt_choice = opt_dict[norms[0]]
self.w = opt_choice[0]
self.b = opt_choice[1]
latest_optimum = opt_choice[0][0]+step*
for i in self.data:
for xi in self.data[i]:
yi=i
print(xi,':',yi*(np.dot(self.w,xi)+self.b))
def predict(self,features):
classification = np.sign(np.dot(np.array(features),self.w)+self.b)
if classification !=0 and self.visualization:
self.ax.scatter(features[0], features[1], s=200, marker='*', c=self.colors[classification])
return classification
def visualize(self):
[[self.ax.scatter(x[0],x[1],s=100,color=self.colors[i]) for x in data_dict[i]] for i in data_dict]
def hyperplane(x,w,b,v):
return (-w[0]*x-b+v) / w[1]
 datarange = (self.min_feature_value*0.9,self.max_feature_value*1.1)
hyp_x_min = datarange[0]
hyp_x_max = datarange[1]
psv1 = hyperplane(hyp_x_min, self.w, self.b, 1)
psv2 = hyperplane(hyp_x_max, self.w, self.b, 1)
self.ax.plot([hyp_x_min,hyp_x_max],[psv1,psv2],
'k') nsv1 = hyperplane(hyp_x_min, self.w, self.b, -1)
nsv2 = hyperplane(hyp_x_max, self.w, self.b, -1)
self.ax.plot([hyp_x_min,hyp_x_max],[nsv1,nsv2],
'k') db1 = hyperplane(hyp_x_min, self.w, self.b, 0)
db2 = hyperplane(hyp_x_max, self.w, self.b, 0)
self.ax.plot([hyp_x_min,hyp_x_max],[db1,db2], 'y--')
plt.show()
data_dict = {-1:np.array([[1,7], [2,8], [3,8],]),
1:np.arr ay([[5,1], [6,-1], [7,3],])}
svm = Support_Vector_Machine()
svm.fit(data=data_dict)
predict_us = [[0,10], [1,3], [3,4], [3,5], [5,5],[5,6],[6,-5], [5,8]]
for p in predict_us:
svm.predict(p)
svm.visualize()
Output:

Result:
Thus, the program for the SVM Model using python was implemented, executed and
verified successfully.
Ex No. 8 IMPLEMENT ENSEMBLING TECHNIQUES

Aim :

To implement the ensembling techniques like Bagging, Boosting and Stacking using
Python.

BAGGING:
Algorithm:

1. Create multiple datasets from the train dataset by selecting observations with replacements.
2. Run a base model on each of the created data sets independently.
3. Combine the predictions of all the base models to each the final output.
4. Bagging normally uses only one base model.

Program:

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
import xgboost as xgb
from sklearn.ensemble import BaggingRegressor
df = pd.read_csv("train_data.csv")
target = df["target"]
train = df.drop("target")
X_train, X_test, y_train, y_test = train_test_split( train, target, test_size=0.20)
model = BaggingRegressor(base_estimator=xgb.XGBRegressor())
model.fit(X_train, y_train)
pred = model.predict(X_test)
print(mean_squared_error(y_test, pred_final))
Output:
4666

BOOSTING:
Algorithm:
1. Take a subset of the train dataset.
2. Train a base model on that dataset.
3. Use a third model to make predictions on the whole dataset.
4. Calculate errors using the predicted values and actual values.
5. Initialize all data points with the same weight.
 6. Assign higher weight to incorrectly predicted data points.
7. Make another model, make predictions using the new model in such a way that
errors made by the previous model are mitigated/corrected.
8. Similarly, create multiple models–each successive model correcting the errors of the
previous model.
9. The final model (strong learner) is the weighted mean of all the previous models
(weak learners).

Program:
import pandas as pd
from sklearn.model_selection import
train_test_split from sklearn.metrics import
mean_squared_error
from sklearn.ensemble import
GradientBoostingRegressor df =
pd.read_csv("train_data.csv")
target = df["target"]
train =
df.drop("target")
X_train, X_test, y_train, y_test = train_test_split( train, target,
test_size=0.20) model = GradientBoostingRegressor()
model.fit(X_train, y_train)
pred_final =
model.predict(X_test)
print(mean_squared_error(y_test, pred_final))

Output:
4789

STACKING:
Algorithm:

1. Split the train dataset into n parts

2. A base model (say linear regression) is fitted on n-1 parts and predictions are made for
the nth part. This is done for each one of the n part of the train set.
3. The base model is then fitted on the whole train dataset.
4. This model is used to predict the test dataset.
5. The Steps 2 to 4 are repeated for another base model which results in another set
of predictions for the train and test dataset.
6. The predictions on the train data set are used as a feature to build the
new model.
7. This final model is used to make the predictions on test dataset

Program:

mport pandas as pd
from sklearn.model_selection import train_test_split from
sklearn.metrics import mean_squared_error
 from sklearn.ensemble import RandomForestRegressor
import xgboost as xgb
from sklearn.linear_model import LinearRegression
from vecstack import stacking
df = pd.read_csv("train_data.csv")
target = df["target"]
train = df.drop("target")
X_train, X_test, y_train, y_test =
train_test_split( train, target, test_size=0.20)
model_1 = LinearRegression()
model_2 = xgb.XGBRegressor()
model_3 = RandomForestRegressor()
all_models = [model_1, model_2, model_3]
s_train, s_test = stacking(all_models, X_train, X_test,
y_train, regression=True, n_folds=4)
final_model = model_1
del = final_model.fit(s_train, y_train) pred_final = final_model.predict(X_test)
print(mean_squared_error(y_test, pred_final))

Output:
4510

Result:
Thus the above Python program for implementing the ensembling techniques like
Bagging, Boosting and Stacking was successfully executed and verified successfully.
Ex No. 9 IMPLEMENT CLUSTERING ALGORITHMS

Aim :

To implement the clustering algorithms like K-Means and Partitioning Around Medoids (PAM)
using python for the set of data points

k MEANS:
Algorithm:
1. Initialize k means with random values
2. For a given number of iterations:
3. Iterate through items:
4. Find the mean closest to the item
5. Assign item to mean
6. Update mean

Program:
import math
x = [1, 1, 2, 2, 3, 5]
y = [1.5, 4.5, 1.5, 3.5, 2.5, 6]
c1 = [1, 1.5]
c2 = [2, 1.5]
dist_c1 = []
dist_c2 = []
clust_1 = []
clust_2 = []
k=0
while k<2:
clust_1 = clust_2 = []
dist_c1 = dist_c2 = []
cx = cy = 0
for i in range(6):
dist_c1 = dist_c1 + [math.sqrt(((x[i] - c1[0])**2)+((y[i] - c1[1])**2))]
dist_c2 = dist_c2 + [math.sqrt(((x[i] - c2[0])**2)+((y[i] - c2[1])**2))]
if(dist_c1[i] < dist_c2[i]):
clust_1 = clust_1 + [i]
else:
clust_2 = clust_2 + [i]
print(" Cluster_1: ",end = "")
 for i in range(len(clust_1)):
print(clust_1[i],end="")
cx = cx + x[clust_1[i]]
cy = cy + y[clust_1[i]]
c1 = [cx/len(clust_1), cy/len(clust_1)]
print()
cx = cy = 0
print(" Cluster_2: ",end="")
for i in range(len(clust_2)):
print(clust_2[i],end="")
cx = cx + x[clust_2[i]]
cy = cy + y[clust_2[i]]
c2 = [cx/len(clust_2), cy/len(clust_2)]
print()
print("Centroid C1= (",round(c1[0],3),",
",round(c1[1],3),")") print("Centroid C2=
(",round(c2[0],3),", ",round(c2[1],3),")") k = k + 1

Output:

PAM:(PARTITION AROUND MEDOIDS)

Algorithm:

1. Initialize: select k of the n data points as the medoids

2. Associate each data point to the closest medoid.
3. While the cost of the configuration decreases:
4. For each medoid m, for each non-medoid data point o:
5. Swap m and o, associate each data point to the closest medoid, recompute the cost (sum
of distances of points to their medoid)
6. If the total cost of the configuration increased in the previous step, undo the swap

Program:

x = [2, 3, 3, 4, 6, 6, 7, 7, 8, 7]
y = [6, 4, 8, 7, 2, 4, 3, 4, 5, 6]
 print("Enter the representative objects for 1st
iteration:") i = int(input())
j = int(input())
dist_c1 = []
dist_c2 = []
clust_c1 = []
clust_c2 = []
for k in range(10):
dist_c1 = dist_c1 + [abs(x[k]-x[i]) + abs(y[k]-
y[i])] dist_c2 = dist_c2 + [abs(x[k]-x[j]) +
abs(y[k]-y[j])] if(dist_c1[k] < dist_c2[k]):
clust_c1 = clust_c1 + [k]
else:
clust_c2 = clust_c2 + [k]
q=0
print("Cluster 1: ",end="")
for i in range(len(clust_c1)):
print(clust_c1[i],"
",end="") q = q +
dist_c1[clust_c1[i]] print()
print("Cluster 2: ",end="")
for i in range(len(clust_c2)):
print(clust_c2[i],"
",end="") q = q +
dist_c2[clust_c2[i]] print()
print("Q = ",q) print("Enter the second representative points for second iteration:")
i = int(input())
j = int(input())
dist_c1 = []
dist_c2 = []
clust_c1 = []
clust_c2 = []
for k in range(10):
dist_c1 = dist_c1 + [abs(x[k]-x[i]) + abs(y[k]-y[i])]
dist_c2 = dist_c2 + [abs(x[k]-x[j]) + abs(y[k]-y[j])]
if(dist_c1[k] < dist_c2[k]):
clust_c1 = clust_c1 + [k]
else:
clust_c2 = clust_c2 + [k]
q=0
print("Cluster 1: ",end="")
 for i in range(len(clust_c1)):
print(clust_c1[i],"
",end="") q = q +
dist_c1[clust_c1[i]] print()
print("Cluster 2: ",end="")
for i in range(len(clust_c2)):
print(clust_c2[i],"
",end="") q = q +
dist_c2[clust_c2[i]] print()
print("Q = ",q)
Output:

Result:
Thus the clustering algorithms like K-Means and Partitioning Around Medoids(PAM) was
implemented and verified using python successfully.
Ex No. 10 IMPLEMENT EM FOR BAYESIAN NETWORKS

Aim :

To implement Expectation_Maximization(EM) algorithm for Bayesian networks using

python.

Algorithm:

1. Initialize the Bayesian network model with some initial structure and parameters.
The structure defines the conditional dependencies between variables, and the
parameters represent the conditional probability distributions for each variable given
its parents.
2. Initialize the missing values in the dataset (if any) using some heuristic or
random imputation method.
3. Repeat the following steps until convergence or a maximum number of iterations is
reached:
4. Return the final model with updated structure and parameters.

Program:

!pip install pgmpy

!pip install pandas
!pip install numpy
import pandas as pd
import numpy as np
class BayesianNetwork:
def _init_(self, structure, values):
self.structure = structure
self.values = values
def infer(self, evidence):
pass # TODO: implement inference
def learn_parameters(self, data):
for node in range(len(self.structure)):
parents = self.structure[node]
if parents == []:
# If the node has no parents, use MLE to estimate its parameters
self.values[node] = np.mean(data[:, node])
else:
# If the node has parents, use EM to estimate its parameters
n_parents = len(parents)
n_values = len(self.values[node])
n_data = data.shape[0]
theta = np.zeros((n_values, n_parents + 1))
theta[:, 0] = np.sum(data[:, node]) / n_data
for i in range(n_parents):
 parent = parents[i]
for j in range(n_values):
mask = (data[:, parent] == j)
theta[j, i+1] = np.sum(data[mask, node]) / np.sum(mask)
while True:
ss = np.zeros((n_values, n_parents + 1))
for i in range(n_data):
x = data[i, node]
parents_x = tuple(data[i, parents])
p_x_given_parents = np.prod(theta[x, 1:] ** parents_x * (1 - theta[x, 1:]) ** (1 -
parents_x))
p_parents = np.prod(theta[:, 1:] ** parents_x * (1 - theta[:, 1:]) ** (1 - parents_x))
ss[x, 0] += p_x_given_parents / p_parents
for j in range(n_parents):
parent_j = parents[j]
ss[x, j+1] += parents_x[j] * (p_x_given_parents / p_parents)
# M-step: update parameters
old_theta = np.copy(theta)
for j in range(n_values):
theta[j, 0] = ss[j, 0] / n_data
for k in range(n_parents):
theta[j, k+1] = ss[j, k+1] / ss[j, 0]
if np.allclose(theta, old_theta):
break
self.values[node] = theta[:, 0]
Output:

Result:
Thus the above Python program for implementing EM for Bayesian Network was
successfully executed and verified successfully.
Ex No. 11 BUILD SIMPLE NN MODELS

Aim :

To build and implement a Simple NN Model using python.

Algorithm:

1. Take the inputs from the training dataset, performed some adjustments based on
their weights, and siphoned them via a method that computed the output of the ANN.
Step
2. Compute the back-propagated error rate. In this case, it is the difference between
neuron’s predicted output and the expected output of the training dataset.
3. Based on the extent of the error got, we performed some minor weight adjustments using
the Error Weighted Derivative formula.
4. We iterated this process an arbitrary number of 15,000 times. In every iteration, the
whole training set is processed simultaneously.

Program:

import numpy as np
class NeuralNetwork():
def init (self):
np.random.seed(1)
self.synaptic_weights = 2 * np.random.random((3, 1)) - 1
def sigmoid(self, x):
return 1 / (1 + np.exp(-x))
def sigmoid_derivative(self, x):
#computing derivative to the Sigmoid function
return x * (1 - x)
def train(self, training_inputs, training_outputs, training_iterations):
for iteration in range(training_iterations):
output = self.think(training_inputs)
error = training_outputs - output
adjustments = np.dot(training_inputs.T, error * self.sigmoid_derivative(output))
self.synaptic_weights += adjustments
def think(self, inputs):
inputs = inputs.astype(float)
output = self.sigmoid(np.dot(inputs, self.synaptic_weights))
return output
if name == " main ":
neural_network =
NeuralNetwork()
print("Beginning Randomly Generated Weights: ")
print(neural_network.synaptic_weights)
training_inputs = np.array([[0,0,1],
[1,1,1],
 [1,0,1],
[0,1,1]])
training_outputs = np.array([[0,1,1,0]]).T
neural_network.train(training_inputs, training_outputs, 15000)
print("Ending Weights After Training: ")
print(neural_network.synaptic_weights)
user_input_one = str(input("User Input One: "))
user_input_two = str(input("User Input Two: "))
user_input_three = str(input("User Input Three: "))
print("Considering New Situation: ", user_input_one, user_input_two,
user_input_three) print("New Output data: ")
print(neural_network.think(np.array([user_input_one, user_input_two,
user_input_three]))) print("Success")
Output:

Result:
Thus we have successfully built and implemented a Simple NN Model using python.
Ex No. 12 BUILD DEEP LEARNING SIMPLE NN MODELS

Aim :

To build a Deep Learning NN Model using Keras model(tensorflow framework).

Algorithm:

1. Load data.
2. define keras model.
3. compile the keras model.
4. start training(fit the model).
5. evaluate the model.
6. making predictions.

Program:

import pandas as pd
data = pd.read_csv('diabetes.csv')
x = data.drop("Outcome", axis=1)
y = data["Outcome"]
from keras.models import Sequential
from keras.layers import Dense
model = Sequential()
model.add(Dense(12, input_dim=8, activation="relu"))
model.add(Dense(12, activation="relu"))
model.add(Dense(1, activation="sigmoid"))
model.compile(loss="binary_crossentropy", optimizer="adam",
metrics=["accuracy"]) model.fit(x,y, epochs=5, batch_size=10)
_, accuracy = model.evaluate(x, y)
print("Model accuracy: %.2f"%
(accuracy*100)) predictions = model.predict(x)
print([round(x[0]) for x in predictions])
model = Sequential() #define model
model.add(Dense(12, input_dim=8, activation="relu"))
model.add(Dense(8, activation="relu"))
model.add(Dense(1, activation="sigmoid"))
model.compile(loss="binary_crossentropy", optimizer="adam",
metrics=["accuracy"]) #compile model
model.fit(x,y, epochs=5, batch_size=10) #training
_, accuracy = model.evaluate(x,y) #testing
print("Model accuracy: %.2f"% (accuracy*100))
predictions = model.predict(x) #make predictions
#round the prediction
 rounded = [round(x[0]) for x in predictions]

Output:

Result:
Thus we have successfully built a Deep Learning NN Model using Keras model
(tensorflow framework).
Ex No. 13 IMPLEMENT OF APPLICATIONS OF BACK PROPAGATION
ALGORITHM

Aim :

To implement an applications of back propagation algorithm.

Algorithm:
1. Initialize data and parameters
Define XOR inputs/outputs and initialize weights/biases randomly.
2. Set activation functions
Use sigmoid and its derivative for forward and backward passes.
3. Forward propagation
Compute outputs of hidden and final layers using sigmoid.
4. Compute error
Calculate difference between predicted and actual output.
5. Backpropagation
Update weights and biases using gradients from error.
6. Repeat and display result
Train for many epochs, print loss periodically, and show final output.

Program:
import numpy as np
# Sigmoid activation and its derivative
def sigmoid(x):
return 1 / (1 + np.exp(-x))
def sigmoid_derivative(x):
return x * (1 - x)
# Input data for XOR
X = np.array([[0, 0],
[0, 1],
[1, 0],
[1, 1]])
# Output labels
y = np.array([[0],
[1],
[1],
[0]])
# Seed for reproducibility
np.random.seed(42)
# Initialize weights randomly
input_neurons = 2
hidden_neurons = 2
output_neurons = 1
# Weights and biases
weights_input_hidden = np.random.uniform(size=(input_neurons, hidden_neurons))
weights_hidden_output = np.random.uniform(size=(hidden_neurons, output_neurons))

bias_hidden = np.random.uniform(size=(1, hidden_neurons))

bias_output = np.random.uniform(size=(1, output_neurons))

# Training parameters
epochs = 10000
learning_rate = 0.1
# Training loop
for epoch in range(epochs):
# Forward Propagation
hidden_input = np.dot(X, weights_input_hidden) + bias_hidden
hidden_output = sigmoid(hidden_input)
final_input = np.dot(hidden_output, weights_hidden_output) + bias_output
final_output = sigmoid(final_input)
# Backpropagation
error = y - final_output
d_output = error * sigmoid_derivative(final_output)
error_hidden = d_output.dot(weights_hidden_output.T)
d_hidden = error_hidden * sigmoid_derivative(hidden_output)
# Updating weights and biases
weights_hidden_output += hidden_output.T.dot(d_output) * learning_rate
weights_input_hidden += X.T.dot(d_hidden) * learning_rate
bias_output += np.sum(d_output, axis=0, keepdims=True) * learning_rate
bias_hidden += np.sum(d_hidden, axis=0, keepdims=True) * learning_rate
# Optional: Print error every 1000 epochs
if epoch % 1000 == 0:
loss = np.mean(np.square(error))
print(f"Epoch {epoch}, Loss: {loss:.4f}")
# Final output after training
print("\nFinal Output After Training:")
print(final_output)
Output
Epoch 0, Loss: 0.2880
Epoch 1000, Loss: 0.2494
Epoch 2000, Loss: 0.2457
Epoch 3000, Loss: 0.2200
Epoch 4000, Loss: 0.1622
Epoch 5000, Loss: 0.0527
Epoch 6000, Loss: 0.0169
Epoch 7000, Loss: 0.0089
Epoch 8000, Loss: 0.0058
Epoch 9000, Loss: 0.0043

Final Output After Training:

[[0.06029012]
[0.94447222]
[0.944367 ]
[0.05997169]]

Result:
Thus we have successfully implemented an application of back propagation algorithm.