1. Explain Perceptron model with Bias.

The Perceptron is the simplest type of artificial neural network used for binary classification. It
takes multiple inputs, applies weights to them, sums them up, adds a bias term, and passes the
result through an activation function (usually a step function) to produce an output.

Role of Bias

• Bias helps the model shift the decision boundary.

• Without bias, the model is forced to always pass through the origin, which is often
limiting.

• Bias works like an intercept term in linear equations.

Components of a Perceptron

1. Inputs (x1, x2, ..., xn)

These are the features of the input data. For example, in a spam filter:

o x1 might represent presence of the word "free"

o x2 could represent presence of a link

o x3 might be the length of the message, etc.

2. Weights (w1, w2, ..., wn)

Each input is multiplied by a weight. These weights are adjusted during training and
determine the importance of each feature.

3. Bias (b)
The bias allows the activation function to shift its decision boundary. It helps the
perceptron model even when all input features are zero.

Aditi Deorukhakar
 o Analogy: In a linear equation y=mx + c, the c is the bias — it shifts the line up or
down.

Why is Bias Important?

• Imagine drawing a line (decision boundary) to separate two classes of data.

• Without bias, the line is forced to go through the origin (0,0).

• Bias shifts this line, allowing better separation.

• This flexibility improves accuracy and learning capability.

Aditi Deorukhakar
 2. Draw a block diagram of the Error Back Propagation Algorithm and explain with the
flow chart the Error Back Propagation Concept.

1. Start

• The process begins when a neural network receives input data to perform a forward
pass.

2. Forward Pass

• The input data is passed through the neural network layers, and the output is computed
using the current weights and biases.

o Input Layer receives the data.

o The data is passed through Hidden Layers.

o The final output is generated at the Output Layer.

3. Calculate Output Error

• The difference between the predicted output (from the forward pass) and the actual
output (ground truth) is calculated using a loss function (e.g., Mean Squared Error,
Cross-Entropy).

o The formula for error might be:

Error = Actual Output − Predicted Output

4. Calculate Gradients (Backward Pass)

Aditi Deorukhakar
 • The error is propagated backward through the network to update the weights and
biases. This is done using the chain rule of calculus.

o Output Layer:

▪ Calculate the gradient of the error with respect to the output layer’s
weights and biases.

o Hidden Layers:

▪ For each hidden layer, calculate the gradients of the error with respect to
the weights and biases. This is done by using the error at the next layer,
multiplied by the derivative of the activation function of the current layer.

▪ The chain rule is applied to propagate the error from the output back to
the input.

5. Update Weights and Biases

• The weights and biases are updated using an optimization algorithm like Gradient
Descent.

o Learning Rate is applied to adjust the weights and biases in the direction that
minimizes the error.

o The weight update rule might be:

W new = W old – η ⋅ ∇W

where:

▪ W new is the updated weight,

▪ η is the learning rate,

▪ ∇W is the gradient of the loss with respect to the weight.

6. Check Convergence

• After updating the weights, check if the error is below a certain threshold or if the
maximum number of iterations (epochs) has been reached.

o If the error is below the threshold or if the training is complete, stop the
process.

o If not, go back to the forward pass for the next iteration.

Aditi Deorukhakar
Aditi Deorukhakar
 3. What are activation functions? Explain Binary, Bipolar, Continuous, and Ramp
activation functions.

Aditi Deorukhakar
Aditi Deorukhakar
 4. Explain the Perceptron Neural Network

In perceptron network, the key points are

i) perceptron-network consist of three units namely, sensory unit (input unit), associate
unit (hidden unit), response unit (output unit).

Aditi Deorukhakar
 5. Write a short note on LMS-Widrow Hoff

LMS (Least Mean Squares) - Widrow-Hoff Algorithm

The LMS (Least Mean Squares) algorithm, also known as the Widrow-Hoff rule, is a popular
adaptive filtering algorithm used in various signal processing and machine learning applications.
It is primarily used for adjusting the weights of a linear system or a neural network to minimize
the error between the predicted output and the actual desired output.

Overview:
• The Widrow-Hoff rule was introduced by Bernard Widrow and Ted Hoff in the 1960s.
• It is a type of stochastic gradient descent (SGD) algorithm designed for optimization,
where the goal is to minimize the mean squared error (MSE) between the predicted
output of a model and the target (desired) output.
Key Concepts:
1. Input Data:
The LMS algorithm takes input data (denoted as xx) and computes a weighted sum
based on the current weights of the system.
2. Output:
The system computes the output (denoted as y^\hat{y}) based on the weighted sum of
the input and current weights.
3. Error:
The error (denoted as ee) is the difference between the desired output (denoted as yy)
and the predicted output y^\hat{y}. Mathematically, this is expressed as:

Aditi Deorukhakar
Advantages:

• Simplicity: The LMS algorithm is easy to implement and computationally efficient.

• Convergence: The algorithm converges over time to minimize the error, provided the
learning rate is properly chosen.

• Adaptivity: The LMS algorithm adapts in real-time, making it useful for dynamic systems
where inputs change over time.

Applications:

• Signal Processing: LMS is commonly used in noise cancellation, echo cancellation, and
adaptive filtering.

• Neural Networks: It is used in training single-layer perceptrons and other models where
linear relationships need to be modeled.

• Control Systems: Adaptive control systems often use LMS for real-time adjustment of
parameters.

In summary, the LMS-Widrow Hoff algorithm is an effective and simple adaptive method for
minimizing error in linear systems, making it widely used in various fields, including signal
processing, machine learning, and control systems.

Aditi Deorukhakar
 6. Draw and explain biological neural networks and compare them with artificial neural
networks.

Aditi Deorukhakar
 Parameters ANN BNN

input dendrites
weight synapse
Structure
output axon
hidden layer cell body

very precise structures and

Learning they can tolerate ambiguity
formatted data

complex simple
Processor high speed low speed
one or a few large number

separate from a processor integrated into processor

Memory localized distributed
non-content addressable content-addressable

centralized distributed
Computing sequential parallel
stored programs self-learning

Reliability very vulnerable robust

numerical and symbolic perceptual

Expertise
manipulations problems

Operating well-defined poorly defined

Environment well-constrained un-constrained

performance degraded even on

Fault Tolerance the potential of fault tolerance
partial damage

Aditi Deorukhakar
 7. Explain in detail the MP neuron model.

Aditi Deorukhakar
 8. Draw a block diagram of the Error Back Propagation Algorithm and explain with the
flow chart the Error Back Propagation Concept.
9. What are Activation functions? Explain the Binary, Bipolar, Continuous, and Ramp
activation functions.
10. Draw Delta Learning Rule (LMS-Widrow Hoff) model and explain it with a training
process flowchart.

Aditi Deorukhakar
 11. Artificial Neural Network

Artificial Neural Networks contain artificial

neurons, which are called units. These units
are arranged in a series of layers that together
constitute the whole Artificial Neural
Network in a system. A layer can have only a
dozen units or millions of units, as this depends
on how the complex neural networks will be
required to learn the hidden patterns in the
dataset. Commonly, an Artificial Neural
Network has an input layer, an output layer, as
well as hidden layers. The input layer receives
data from the outside world, which the neural
network needs to analyze or learn about. Then,
this data passes through one or multiple
hidden layers that transform the input into
data that is valuable for the output layer.
Finally, the output layer provides an output in
the form of a response of the Artificial Neural
Networks to the input data provided.

In the majority of neural networks, units are

interconnected from one layer to another. Each
of these connections has weights that
determine the influence of one unit on another
unit. As the data transfers from one unit to
another, the neural network learns more and
more about the data, which eventually results
in an output from the output layer.

12. Perceptron Neural Network

Aditi Deorukhakar
 13. Draw and explain biological neuron

An artificial neural network is an efficient information processing system which resembles in

characteristics with a biological neural network BNN.

Aditi Deorukhakar
 14. Explain in detail MP neuron model

15. Draw a block diagram of the Error Back Propagation Algorithm and explain with the
flow chart the Error Back Propagation Concept.

16. What are Activation functions? Explain the Binary, Bipolar, Continuous, and Ramp
activation functions.

17. Draw Delta Learning Rule (LMS-Widrow Hoff) model and explain it with a training
process flowchart.

18. Artificial Neural Network

19. Perceptron Neural Network

20. Implement XOR function using McCulloch Pitts Model

Aditi Deorukhakar
Aditi Deorukhakar
Aditi Deorukhakar
 21. Discuss different activation functions used in Neural Networks. (Formula, Graph and
Range)
22. Implement the ANDNOT logic functions using McCulloch Pitts Model.

Aditi Deorukhakar
 23. Implement OR function (logic gate) using single layer perceptron. Assume initial values
of weights and learning rate as follows w1=0.6, w2=1.1 learning rate = 0.5, Threshold
=1

Aditi Deorukhakar
Aditi Deorukhakar
 24. Explain Multilayer perceptron with a neat diagram and its working with flowchart or
algorithm.

Multi-Layer Perceptron (MLP) is an artificial neural network widely used for solving
classification and regression tasks.

MLP consists of fully connected dense layers that transform input data from one dimension to
another. It is called “multi-layer” because it contains an input layer, one or more hidden layers,
and an output layer. The purpose of an MLP is to model complex relationships between inputs
and outputs, making it a powerful tool for various machine learning tasks.

Key Components of Multi-Layer Perceptron (MLP)

• Input Layer: Each neuron (or node) in this layer corresponds to an input feature. For
instance, if you have three input features, the input layer will have three neurons.

• Hidden Layers: An MLP can have any number of hidden layers, with each layer
containing any number of nodes. These layers process the information received from the
input layer.

• Output Layer: The output layer generates the final prediction or result. If there are
multiple outputs, the output layer will have a corresponding number of neurons.

Every connection in the diagram is a representation of the fully connected nature of an MLP.
This means that every node in one layer connects to every node in the next layer. As the data
moves through the network, each layer transforms it until the final output is generated in the
output layer.

working of a Multi-Layer Perceptron (MLP)

Step 1: Forward Propagation
• The input data moves forward through the network.

Aditi Deorukhakar
 • It starts from the input layer, goes through one or more hidden layers, and finally
reaches the output layer.
• Each neuron in the hidden layers:
1. Receives input data and multiplies it by some weights (which represent the
neuron’s importance).
2. Adds a small constant value (called bias).
3. Passes this result through an activation function (like ReLU, Sigmoid, or Tanh) to
decide whether it should fire (activate) or not.
• This process helps the network capture patterns in the data.
Step 2: Loss Function
• Once the MLP gives an output (a prediction), we compare it with the actual answer.
• This comparison tells us how wrong the prediction was — this is called the loss.
• If the prediction is far from the actual value, the loss will be high; if it’s close, the loss will
be low.
• For classification tasks, special loss formulas are used to measure how different the
predicted class is from the actual one.
• For number-based predictions (regression), we look at the average of the squared
differences between predicted and actual values.
Step 3: Backpropagation
• Now the model wants to learn from its mistake (loss).
• Backpropagation is the process where the error is sent backward through the network,
starting from the output layer and going to the input.
• During this process:
o The model calculates how much each weight and bias contributed to the error.
o Based on this, it updates those values — trying to make better predictions next
time
Step 4: Optimization
• To make the learning efficient, optimization algorithms are used.
• These algorithms decide how much to change the weights and biases based on the
backpropagation feedback.

25. Explain Back Propagation Neural Network with flowchart.

Aditi Deorukhakar