Artificial Neural Network

CS1545

Neural Network Learning

What is meant with learning?

The ability of the neural network (NN) to learn form its environment and to improve
its performance through learning.

 The neural network is simulated by an environment.

 The neural network undergoes changes in its free parameters.
 The neural network responds in a new way to the environment.

Some historical notes:

Pavlov’s conditioning experiments:a conditioned responses, salivation in response to
auditory stimulus.

Hebb: The organization of behavior, 1949→ long term potential , LPT, (1973 Bliss,
Lomo), AMPA receptor, Long Term Depression, LTD, NMDA receptor.

The nearest neighbor rule Fix and Hodges 1951.

The idea of competitive learning : Von de Malsburg 1973, the self- organization of
orientation -sensitive nerve cells in the striate cortex.

Lateral inhibition → Mach bands, Ernest mach 1865.

Statistical thermodynamics:
Statistical thermodynamics in the study of computing machinery, John Von Neumann,
theory and organization of complicated Automata, 1949.

Reinforcement learning:Minsky 1961, Thorndike 1911.

 The problem of designing an optimum linear filter ;Kolmogorov 1942,Wiener

1949, Zadeh 1953,Gabor 1954.

Definition of learning:Learnng is a process by which the free parameters of a neural

network are adapted through a process of stimulation by the environment in which the
network is embedded. The type of the learning is determined by the manner in which
the parameter changes take place.(Mendel and Mcdaren 1970)

Five Basic Learning Rule:

1) Error-correction learning←optimum filterning
2) Memory-based learning←Memorizing the training data explicitly.
3) Hebbian Learning←Neurobiological
4) Competitive Learning←Neurobiological
5) Boltzman Learning←Statistical mechanics
1. Error-correction learning:In error correction learning
a) Error signal =desire response-output signal
ek(n)= dk(n)-yk(n)
b) ek(n) actuates a control mechanism to make the output signal yk(n) come closer to
the desired response dk(n) in step by step manner.
c) A cost function ε(n)=1/2e2k(n) is the instantaneous value of the error energy → a
steady state.
d) Adelta rule or widrow -hoff rule
∆wkj(n)=ɳek(n) xj(n)
e) ɳ is the learning rate parameter.
f) The adjustment made to a synaptic weight of a neuron is proportional to the product
of the error signal and the input signal of the synapse in question.
Wkj(n+1)=Wkj(n)+∆Wkj(n)

2. Memory Based learning: In memory based learning all of the past experiences
are explicitly stored in a large memory of correctly classified input-output examples.
=> {( xidi)}n i=1
=> Criterian used for defining the local neighbourhood of the test vector Xtest.
=> Learning rule applied to the training examples in the local neighborhood of
xtest.
=>Nearest neighbor rule; the vector X’N ε{X1,X2,----XN}is the nearest neighbor
of Xtest if minid (xi,xtest)=d(XN,Xtest)
=>If the classified examples d(xi,di) are independently and identically distributed
according to the joint probability distribution of the example(x,d).
=> If the sample size N is infinitely large.
=> The classification error incurred by the nearest neighbor rule is bounded
above twice the Bayes probability of error.
=>K- nearest neighbor classifier.
=> Identify the K classified patterns that lie nearest to the test vector Xtest for
some integer K.
=> Assign Xtest to the class that is most frequently represented in the K nearest
neighbors to Xtest.

3. Hebbian Learning:
a) If two neurons on either side of synapse(condition) are activated simultaneously,
then the strength of that synapse is selectively increased.
b) If two neurons on either side of a synapse are activated asynchronously then that
synapse is selectively weakened or eliminated.
i) Time dependent mechanism.
ii) Local mechanism( spatiotemporal cogtiguity)
iii) Interactive mechanism
iv) Conjunctional or correlational mechanism.

→ A hebbian synapse increases its strength with positively correlated presynaptic

and postsynaptic signals, and decreases its strength when signals are either
uncorelated or negatively correlated.

The Hebbian learning in mathematical terms:

→∆Wkj(n)=F(yk(n),xj(n))
→ The simplest form:
∆Wkj(n)=ɳyk(n)xj(n))
→ Co-variance hypothesis :
∆Wkj=ɳ(xj-x’)(yj-y’)

Note That:a) Synaptic weight Wkj is enhanced if the conditions xj> x’ and yk>y’are
both are satisfied.
b)Synaptic weight wkj is depressed if there is xj>x’ and yk<y’ or
Yk>y’ and xj<x’.

4. Competitive Learning:
→ The output neurons of a neural network compete among themselves to become
active.
→ A set of neurons that are all the same ( excepts for synaptic weights).
→ A limit imposed on the strength of each neuron.
→ A mechanism that permits the neurons to compete→ a winner -takes -all
→the standard competitive learning rule
∆Wkj=ɳ( xj-wkj) if neuron k wins the competition.
=0 if the neuron k loses the competition.

5. Boltzmann Learning:
→ The neurons constitute a recurrent structure and they operate in a binary manner.
The machine is characterized by an energy function E

j
k

→E=-½ Wkjxkxj, j≠k

=>Machine operates by choosing a neuron at random then flipping the state of neuron
k from the state xk to state -xk at some temperature T with probability

→P(Xk →-Xk )=1/(1+exp(-∆Ek/T))

Clamped Conditions:
The visible neurons are al lclamped onto specific states determined by the
environment.
Free -Running Conditions:
→ All the neurons (visible and hidden) are allowed to operate freely.
→The Boltzmann learning rule:
∆Wkj= ɳ(Pkj+ -Pkj-),

J≠K

Credit Assignment:The credit assignment problem is the problem of assigning credit

or blame for overall outcomes to each of the internal decessions made by the learning
machine and which contributed to those outcomes.

Temporal credit assignment problem:The temporal credit assignment problem is

that it involves the instants of time when the actions that deserve credit where actually
taken.
Structural credit assignment problem: The structural credit assignment problem in
that it involves assigning credit to the internal structures of actions generated by the
system.

Learning with a teacher(Supervised Learning):

Environment Teacher

Desired
Response

Actual Response +

Learning System 
-

Error Signal

Fig.: Block diagram of learning with a teacher.

→ The teacher has knowledge of the environment

→ Error-performance surface.

Learning without a teacher:No labeled examples available of the function to be

learned.
a) Reinforcement learning
b) Unsupervised learning

a) Reinforcement learning: Primary reinforcement.

State(Input) Vector

Environment Critic

Actions Heuristic reinforcement

Learning System

Fig: Block diagram of reinforcement learning

The learning of input -output mapping is performed through continued inteaction with
the environment in order to minimize a scalar index of performance.

Delayed reinforcement :
→ Which means that the system observes a temporal sequence of stimuli.
→Difficult to performfor two reasons:
→There is no teacher to provide a desire response at each step of the learning process.
→The delay incurred in the generation of the primary reinforcement signal implies
that the machine must solve a temporal credit assignment problem.
→Reinforcement learning is closely related to dynamic programming.

Unsupervised learning:

Vector describing state of environment

Environment Learning System

Fig:Block diagram of unsupervised learning

→There is no external teacher or critic to oversee the learning process.

→The provision is made for a task independent measure of the quality of the
representation that the network is required to learn.

The issues of learning Tasks:

Associative Memory:An associative memory is a brain like distributed memory that

learns by association.

Auto association:A neural network is required to store a set of patterns by repeatedly

presenting then to the network .The network is presented a partial description of an
original pattern stored in it ,and the task is to retrieve that particular pattern.

Heteroassociation:
It differs from autoassociation in that an arbitary set of input patterns is paired with
another arbitary set of output patterns.

Input Vector Output vector

X Y
Pattern associator

Fig: Input -output relation of pattern associator.

→ Let Xk denote a key pattern and Yk denote a memorized pattern.The pattern

association is described by

→Xk →Yk K→ 1,2, ----------q.

→In an auto associative memory Xk =Yk.
→ In a heteroassociative memory Xk ≠Yk..
→ Storage phase
→ Recall phase
→q is a direct measure of the storage capacity.