Artificial Neural Network

Introduction:
Artificial neural networks models have been extensively studied with the
aim of achieving human-like performance, especially in the field of
pattern recognition.

These networks are composed of a numbers of nonlinear computational

elements which operate in parallel and are arranged in a manner
reminiscent of biological neural interconnections.
The area of neural networks is nowadays considered from main two
perspectives:
- The first perspective is cognitiv science (studying of the mind).
- The second perspective is connectionism, which is a theory of
information processing.
What is a Neural networks
- Work on artificial neural network (ANN) commonly referred to as
“neural networks” (NN).
- The human brain computes in a way which is differ than the
conventional computer. The brain is: a highly complex, nonlinear,
and parallel computer (information processing system). It has the
capability to organize its structure constituents known as neurons.
-The human vision represents an example on the information
processing in the brain. In which the brain routinely accomplishes
perceptual recognition tasks in approximately 100-200 ms, whereas
tasks of much lesser complexity take a great deal longer on a
powerful computer.

-Another example is the sonar of the bat. Sonar is an active

echolocation system (echo is the repetition of a sound by the
reflection of sound waves, or the reflected radio or radar beam). In

1
 addition to providing information about how far away a target is
(e.g. a flying insect), bat sonar conveys information about the
relative velocity of the target, the size of various features of the
target, and the azimuth and elevation of the target. The complex
neural computations needed to extract all this information from the
target echo occur within a brain of small size. Indeed, an
echolocating bat can capture its target with a facility and success
rate that would be the envy of a radar or sonar engineer.
- At birth, a brain already has considerable structure and the ability
to build up its own rules of behavior through what we usually refer
to as “experience”. Indeed experience is build up over time.
- In general, a neural network is a machine that is designed to model
the way in which the brain performs a particular task or function of
interest. The network is usually implemented by using electronic
components or is simulated in software on a digital computer. To
achieve good performance, neural networks employ a massive
interconnection of simple computing cells referred to as “neurons”
or “processing units”.
Definition of a neural network
 A neural network is a massively parallel distributed processor
made up of simple processing units that has a natural propensity for
storing experiential knowledge and making it available for use. It
resembles the brain in two respects:
1- Knowledge is acquired by the network from its environment
through a learning process.
2- Interneuron connection strengths known as “synaptic weights” are
used to store the acquired knowledge.

2
The procedure used to perform the learning process is called a learning
algorithm, the function of which is to modify the synaptic weights of the
network in an orderly fashion to attain a desired design objective.
- Artificial neural networks have been developed as generalizations
of mathematical models of human cognition or neural biology,
based on the assumptions that:
1. Information processing occurs at many simple elements called
neurons.
2. Signals are passed between neurons over connection links.
3. Each connection link has an associated weight, which, in a typical
neural net, multiplies the signal transmitted.
4. Each neuron applies an activation function (usually nonlinear) to
its net input (sum of weighted input signals) to determine its output
signal.

A neural network is characterized by:

(1) its pattern of connections between the neurons (called its
architecture).
(2) its method of determining the weights on the connections (called its
training, or learning, algorithm).
(3) its activation function.
Benefits of neural networks
1- Nonlinearity: An artificial neuron can be linear on nonlinear. A
neural network made up of an interconnection of nonlinear neurons, is
itself nonlinear. Nonlinearity is a highly important property,
particularly if the underlying physical mechanism responsible for
generation of the input signal is inherently nonlinear (e.g., speech
signal).

3
2- Input-Output mapping: The learning process involves modification
of the synaptic weights of N.N by applying a set of labeled training
examples. Each example consists of a unique input signal and a
corresponding desired (target) response. The training of the
networks is repeated for many examples in the set, until the network
reaches a steady state. The previously applied training examples may
be applied during the training session , but in a different order. Thus
the network learns from the examples by constructing an input-
output mapping for the problem at hand.
3- Adaptivity: N.Ns have a built-in capability to adapt their synaptic
weights to change in surrounding environment. And may be designed
to change their synaptic weights in real time. The adaptive capability
of the network makes it a useful tool in adaptive pattern classification,
adaptive signal processing, and adaptive control. As a general rule, it
may be said that the more adaptive we make the system, all the time
ensuring that the system remains stable, the more robust its
performance will likely be when the system is required to operate in
a nonstationary environment.
4- Evidential response : In the context of pattern classification, a neural
network can be designed to provide information not only about which
particular pattern to select, but also about the confidence in the
decision made. This latter information may be used to reject
ambiguous patterns, should they arise, and thereby improve the
classification performance of the network.
Neural Networks Applications
A wide variety of problems in many fields can be solved using
neural networks. Some of these applications:

4
- Aerospace
High performance aircraft autopilots, flight path simulations, aircraft
control systems, autopilot enhancements, aircraft component simulations,
aircraft component fault detectors
- Automotive
Automobile automatic guidance systems, warranty activity analyzers
- Banking
Check and other document readers, credit application evaluators
- Defense
Weapon steering, target tracking, object discrimination, facial
recognition, new kinds of sensors, sonar, radar and image signal
processing including data compression, signal/image identification
Electronics
Code sequence prediction, integrated circuit chip layout, process control,
chip failure analysis, voice synthesis , nonlinear modeling
Entertainment
Animation, special effects, market forecasting
Financial
Real estate appraisal, corporate financial analysis, currency price
prediction
Insurance
Policy application evaluation, product optimization
Manufacturing
Manufacturing process control, product design and analysis, process and
machine diagnosis, visual quality inspection systems, computer chip
quality analysis, analysis of grinding operations, chemical product design
analysis, machine maintenance analysis, planning and management,
dynamic modeling of chemical process systems
Medical

5
Breast cancer cell analysis, EEG and ECG analysis, hospital expense
reduction, hospital quality improvement
Oil and Gas
Exploration
Robotics
Trajectory control, forklift robot, manipulator controllers , vision systems
Speech
Speech recognition, speech compression, text to speech synthesis

Telecommunications
Image and data compression, automated information services, real-time
translation of spoken language.
Transportation
Truck brake diagnosis systems, vehicle scheduling, routing systems
The human brain
The human nervous system may be viewed as a three-stage system as
shown in fig. (1):
(Brain)
Stimulus Receptors Neural Net Effectors
(Nerves)
Response

Forward transmision of information

Feedback
Fig. (1) Block diagram representation of nervous system
1- Respectors: convert stimula from the human body or the external
environment into electrical impulses that convey information to the
neural net (brain).
2- Brain: continually receives information, perceive it, and make
appropraite decisions.

6
3- Effectors: convert electrical impulses generated by the neural net
into discernible response as system output.
- Neurons are slower than silicon logic gates. Events in a silicon
chip happen in the nanosecond range, while neural events
happen in the millisecond. But the brain have staggering number
of neurons with massive iterconnections between them. The net
result that the brain is an enormously efficient structure (there are
approximately 10 billion neurons and 60 trillion synapses or
connections).
- The energetic efficiency of the brain is approximately 10-6 Joules
(J) per operation per second, whereas the corresponding value for
the best computers is orders of magnitude larger.
- Synapses or nerve endings are elementary structural and functional
units that mediate the interaction between neurons.
- The most common kind of synapse is a chemical synapes, which
operates as follows:
o Synapse converts a presynptic electrical signal into
chemical signal and then back into a postsynaptic electrical
signal, as illustrate in fig. (2).
Presynaptic Chemical Postsynaptic
electrical signal signal electrical signal

Fig. (2) the operation of chemical synapse

- Neurons come in a wide variety of shapes and sizes in different
parts of the brain. Fig. (3) illustrate the shape of pyramidal cell,
which is one of the most common types.
- The pyramidal cell can receive 10,000 or more synaptic contacts,
and it can project onto thousands of target cells.
- The neurons encode their outputs as a series of brief voltage
pulses (known as action potential) originate at or close to the cell

7
body of the neurons and then propogate across the individual
neurons at constant velocity and amplitude. The axon of a neuron
is very long and thin and is characterized by high electrical
resistance and very large capacitance. The axon may therefore be
modeled as (RC) transmision line.

Fig. (3) The pyramidal Cell