Scribd
Upload
50 views41 pages

Chapter 2

Uploaded by

varaprasad93
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
50 views41 pages

Chapter 2

Uploaded by

varaprasad93
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 41

PROBLEM SOLVING TECHNIQUES

HARD COMPUTING SOFT COMPUTING

Precise Models Approximate Models

Traditional Functional
Symbolic Numerical Approximate Approximation
Logic Modeling and Reasoning and Randomized
Reasoning Search Search
OVERVIEW OF TECHNIQUES IN SOFT COMPUTING

 Neural Networks

 Fuzzy Logic

 Genetic Algorithm

 Hybrid Systems
ARTIFICIAL NEURAL
NETWORKS: AN
INTRODUCTION
DEFINITION OF NEURAL NETWORKS
According to the DARPA Neural Network Study (1988, AFCEA
International Press, p. 60):

• ... a neural network is a system composed of many simple processing


elements operating in parallel whose function is determined by network
structure, connection strengths, and the processing performed at
computing elements or nodes.

According to Haykin (1994), p. 2:

A neural network is a massively parallel distributed processor that has


a natural propensity for storing experiential knowledge and making it
available for use. It resembles the brain in two respects:
•Knowledge is acquired by the network through a learning process.
•Interneuron connection strengths known as synaptic weights are
used to store the knowledge.
BRAIN COMPUTATION
The human brain contains about 10 billion nerve cells, or
neurons. On average, each neuron is connected to other
neurons through approximately 10,000 synapses.
Types of neurons
Brain
Sensory
Neuron
Lots of
interneurons

Motor
Neuron
What they look like
...or schematically
In fact, things are a bit more crowded
Neurons
communicate with
each other, we will
see later how this
works. This will be
the "neural network"
Thus, neurons need to be able to conduct
information in 2 ways:
1.From one end of a neuron to the other
end.This is accomplished electrically via
action potentials
2.Across the minute space separating one
neuron from another. This is accomplished
chemically via neurotransmitters.
Neural Networks
INTERCONNECTIONS IN BRAIN
BIOLOGICAL (MOTOR) NEURON
ARTIFICIAL NEURAL NET

 Information-processing system.

 Neurons process the information.

 The signals are transmitted by means of connection links.

 The links possess an associated weight.

 The output signal is obtained by applying activations to the net


input.
MOTIVATION FOR NEURAL NET

 Scientists are challenged to use machines more effectively for


tasks currently solved by humans.

 Symbolic rules don't reflect processes actually used by


humans.

 Traditional computing excels in many areas, but not in others.


The major areas being:

 Massive parallelism

 Distributed representation and computation

 Learning ability

 Generalization ability

 Adaptivity

 Inherent contextual information processing

 Fault tolerance

 Low energy consumption.


ARTIFICIAL NEURAL NET

W1
X1 Y

W2
X2

The figure shows a simple artificial neural net with two input neurons
(X1, X2) and one output neuron (Y). The inter connected weights are
given by W1 and W2.
ASSOCIATION OF BIOLOGICAL NET
WITH ARTIFICIAL NET
PROCESSING OF AN ARTIFICIAL NET
The neuron is the basic information processing unit of a NN. It consists
of:

1. A set of links, describing the neuron inputs, with weights W 1, W2,


…, Wm.

2. An adder function (linear combiner) for computing the weighted


sum of the inputs (real numbers):
m
u   W jX j
j 1

3. Activation function for limiting the amplitude of the neuron output.


y   (u  b)
BIAS OF AN ARTIFICIAL NEURON

The bias value is added to the weighted sum

∑wixi so that we can transform it from the origin.

Yin = ∑wixi + b, where b is the bias


x1-x2= -1
x2 x1-x2=0

x1-x2= 1

x1
MULTI LAYER ARTIFICIAL NEURAL NET
INPUT: records without class attribute with normalized attributes
values.

INPUT VECTOR: X = { x1, x2, …, xn} where n is the number of


(non-class) attributes.

INPUT LAYER: there are as many nodes as non-class attributes, i.e.


as the length of the input vector.

HIDDEN LAYER: the number of nodes in the hidden layer and the
number of hidden layers depends on implementation.

“Principles of Soft Computing, 2nd Edition”


by S.N. Sivanandam & SN Deepa
Copyright  2011 Wiley India Pvt. Ltd. All rights reserved.
OPERATION OF A NEURAL NET

- Bias
x0 w0j
x1 w1j
 f
Output y
xn wnj

Input Weight Weighted Activation


vector x vector w sum function
WEIGHT AND BIAS UPDATION
Per Sample Updating

•updating weights and biases after the presentation of each sample.

Per Training Set Updating (Epoch or Iteration)

•weight and bias increments could be accumulated in variables and


the weights and biases updated after all the samples of the training
set have been presented.
STOPPING CONDITION

 All change in weights (wij) in the previous epoch are below some
threshold, or

 The percentage of samples misclassified in the previous epoch is


below some threshold, or

 A pre-specified number of epochs has expired.

 In practice, several hundreds of thousands of epochs may be


required before the weights will converge.
BUILDING BLOCKS OF ARTIFICIAL NEURAL NET
 Network Architecture (Connection between Neurons)

 Setting the Weights (Training)

 Activation Function
LAYER PROPERTIES
 Input Layer: Each input unit may be designated by an attribute
value possessed by the instance.

 Hidden Layer: Not directly observable, provides nonlinearities for


the network.

 Output Layer: Encodes possible values.


TRAINING PROCESS
 Supervised Training - Providing the network with a series of
sample inputs and comparing the output with the expected
responses.

 Unsupervised Training - Most similar input vector is assigned to


the same output unit.

 Reinforcement Training - Right answer is not provided but


indication of whether ‘right’ or ‘wrong’ is provided.
ACTIVATION FUNCTION
 ACTIVATION LEVEL – DISCRETE OR CONTINUOUS

 HARD LIMIT FUCNTION (DISCRETE)


• Binary Activation function
• Bipolar activation function
• Identity function

 SIGMOIDAL ACTIVATION FUNCTION (CONTINUOUS)


• Binary Sigmoidal activation function
• Bipolar Sigmoidal activation function
ACTIVATION FUNCTION

Activation functions:

(A) Identity

(B) Binary step

(C) Bipolar step

(D) Binary sigmoidal

(E) Bipolar sigmoidal

(F) Ramp
CONSTRUCTING ANN
 Determine the network properties:
• Network topology
• Types of connectivity
• Order of connections
• Weight range

 Determine the node properties:


• Activation range

 Determine the system dynamics


• Weight initialization scheme
• Activation – calculating formula
• Learning rule
PROBLEM SOLVING
 Select a suitable NN model based on the nature of the problem.

 Construct a NN according to the characteristics of the application


domain.

 Train the neural network with the learning procedure of the


selected model.

 Use the trained network for making inference or solving problems.


NEURAL NETWORKS
 Neural Network learns by adjusting the weights so as to be able
to correctly classify the training data and hence, after testing
phase, to classify unknown data.

 Neural Network needs long time for training.

 Neural Network has a high tolerance to noisy and incomplete


data.
SALIENT FEATURES OF ANN

 Adaptive learning
 Self-organization
 Real-time operation
 Fault tolerance via redundant information coding
 Massive parallelism
 Learning and generalizing ability
 Distributed representation
McCULLOCH–PITTS NEURON
 Neurons are sparsely and randomly connected

 Firing state is binary (1 = firing, 0 = not firing)

 All but one neuron are excitatory (tend to increase voltage of other
cells)

• One inhibitory neuron connects to all other neurons


• It functions to regulate network activity (prevent too many
firings)
LINEAR SEPARABILITY

 Linear separability is the concept wherein the separation of the


input space into regions is based on whether the network response
is positive or negative.

 Consider a network having


positive response in the first
quadrant and negative response
in all other quadrants (AND
function) with either binary or
bipolar data, then the decision
line is drawn separating the
positive response region from
the negative response region.
HEBB NETWORK
Donald Hebb stated in 1949 that in the brain, the learning is performed
by the change in the synaptic gap. Hebb explained it:

“When an axon of cell A is near enough to excite cell B, and repeatedly


or permanently takes place in firing it, some growth process or
metabolic change takes place in one or both the cells such that A’s
efficiency, as one of the cells firing B, is increased.”
HEBB LEARNING
 The weights between neurons whose activities are positively
correlated are increased:
dw ij
~ correlation ( x i , x j )
dt

 Associative memory is produced automatically

 The Hebb rule can be used for pattern association, pattern


categorization, pattern classification and over a range of other
areas.
FEW APPLICATIONS OF NEURAL NETWORKS

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy