CCS364 Soft Computing

ALOK KUMAR
Assistant Professor/CSE
Government College of Engineering, Dharmapuri
 SYLLABUS

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

 SYLLABUS

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

 SYLLABUS

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

 TEXT BOOKS

● https://drive.google.com/file/d/1qk52Av-_0bO8P6XeZ8wW5pH3p
OCk5DR5/view?usp=drive_link

● https://drive.google.com/file/d/1wE0bT4Erz06PRwfKBVwP39IJ7y
LSvuzR/view?usp=sharing

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

 UNIT - 2
Neural Networks

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
 (Perceptron) (Single Layer NN)

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
● Biases helps the models to shift the transfer function towards the positive or
negative side.

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

 ( )

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Activation Functions

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
(Signum function)

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

 (tanH) (Hyperbolic Tangent)

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
ReLU (Rectified Linear Unit)

Sigmoid and tanh functions suffer

from the vanishing gradient
problem, where gradients become
very small during backpropagation
in deep neural networks.

ReLU avoids this problem and

allows the network to learn faster.

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

Multi-Layer Perceptron

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Example 2:

Output after 2 epochs

Bias Input is 1

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

 Kohonen’s Self Organizing Map
● Neural Networks use processing, inspired by the human brain, as a basis
to develop algorithms that can be used to model and understand complex
patterns and prediction problems.

● There are several types of neural networks and each has its own unique
use.

● The Self Organizing Map (SOM) is one such variant of the neural
network, also known as Kohonen’s Map.

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

● It is an unsupervised neural network that is trained using unsupervised
learning techniques to produce a low dimensional, discretized
representation from the input space of the training samples, known as a
map and is, therefore, a method to reduce data dimensions.

● Self-Organizing Maps are very different from other artificial neural

networks as they apply competitive learning techniques unlike others
using error-correction learning methods such as backpropagation with
gradient descent.

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
● Self-Organizing Maps consist of two important layers, the first one is
the input layer, and the second one is the output layer, which is also
known as a feature map.

● We compute distance between each neuron (neuron from the output

layer) and the input data, and the neuron with the lowest distance will
be the winner of the competition.

● The Euclidean metric is commonly used to measure distance.

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)
Example 2:

After 1 epoch
updated weights for
winning unit

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)

 Thanks !

Government College of Engineering, Dharmapuri (ALOK KUMAR, A.P/CSE)