HYBRID

INTELLIGENT
SYSTEMS
Neural expert systems and neuro-fuzzy systems
 A hybrid intelligent system is one
that combines at least two intelligent
technologies. For example, combining
a neural network with a fuzzy system
results in a hybrid neuro-fuzzy system.

INTRODU
CTION The combination of probabilistic
reasoning, fuzzy logic, neural networks
and evolutionary computation forms
the core of soft computing, an
emerging approach to building hybrid
intelligent systems capable of
reasoning and learning in an uncertain
and imprecise environment.
 Although words are less precise than
numbers, precision carries a high cost.
We use words when there is a
tolerance for imprecision. Soft
computing exploits the tolerance for
uncertainty and imprecision to
achieve greater tractability and
robustness, and lower the cost of
solutions.
 We also use words when the available
data is not precise enough to use
numbers. This is often the case with
complex problems, and while “hard”
computing fails to produce any
COMPARISON OF EXPERT
SYSTEMS, FUZZY SYSTEMS, ANN
AND GA
 Fuzzy logic and neural networks are
natural complementary tools in building
intelligent systems. While neural networks
are low-level computational structures that
NEURO- perform well when dealing with raw data,
fuzzy logic deals with reasoning on a
FUZZY higher level, using linguistic information
acquired from domain experts. However,
SYSTEMS fuzzy systems lack the ability to learn and
cannot adjust themselves to a new
environment. On the other hand, although
neural networks can learn, they are
opaque to the user.
NEURO-FUZZY
SYSTEMS
 Integrated neuro-fuzzy systems can combine the
parallel computation and learning abilities of neural
networks with the human-like knowledge representation
and explanation abilities of fuzzy systems. As a result,
neural networks become more transparent, while fuzzy
systems become capable of learning.
 A neuro-fuzzy system is a neural network which is
functionally equivalent to a fuzzy inference model. It
can be trained to develop IF-THEN fuzzy rules and
determine membership functions for input and output
variables of the system. Expert knowledge can be
incorporated into the structure of the neuro-fuzzy
system.
 The structure of a neuro-fuzzy system is similar to a
multi-layer neural network. In general, a neuro-fuzzy
system has input and output layers, and three hidden
layers that represent membership functions and fuzzy
rules.
NEURO-FUZZY SYSTEM
INPUT LAYER FUZZIFICATI FUZZY OUTPUT DEFUZZIFICA
ON RULE MEMBERSHI TION LAYER
LAYER LAYER P LAYER
Each layer in the neuro-fuzzy system is
associated
with a particular step in the fuzzy inference
process.
Layer 1 is the input layer. Each neuron in
this layer transmits external crisp signals
directly to the next layer. That is,

Layer 2 is the fuzzification layer.

Neurons in this layer represent fuzzy sets
used in the antecedents of fuzzy rules. A
fuzzification neuron receives a crisp input
and determines the degree to which this
The activation function of a membership
neuron is set to the function that specifies
the neuron’s fuzzy set. We use triangular
sets, and therefore, the activation functions
for the neurons in Layer 2 are set to the
triangular membership functions. A
triangular membership function can be
specified by two parameters {a, b} as
follows:
Layer 3 is the fuzzy rule layer. Each
neuron in this layer corresponds to a single
fuzzy rule. A fuzzy rule neuron receives
inputs from the fuzzification neurons that
represent fuzzy sets in the rule
antecedents. For instance, neuron R1,
which corresponds to Rule 1, receives
inputs from neurons A1 and B1.
In a neuro-fuzzy system, intersection can
be implemented by the product operator.
Thus, the output of neuron i in Layer 3 is
obtained
(3) (3) as:(3) (3) y (3)   
yi  x1i x2i  xki R1 A1 B1 R1
Layer 4 is the output membership
layer. Neurons in this layer represent
fuzzy sets used in the consequent of
fuzzy rules.
An output membership neuron combines
all its inputs by using the fuzzy
operation union.
This
( 4)
operation
( 4) ( 4)
can be
( 4)
implemented
( 4) by
 x1i  x2i   OR.
yi probabilistic
the xli ThatyC1  R 3   R 6  C1
is,

The value of C1 represents the

integrated firing strength of fuzzy rule
Layer 5 is the defuzzification layer.
Each neuron in this layer represents a
single output of the neuro-fuzzy
system. It takes the output fuzzy sets
clipped by the respective integrated
firing strengths and combines them into
a single fuzzy set.
Neuro-fuzzy systems can apply
standard defuzzification methods,
including the centroid technique.
We will use the sum-product
composition method.
The sum-product composition calculates
the crisp output as the weighted average
of the centroids of all output membership
functions. For example, the weighted
average of the centroids of the clipped
fuzzy sets C1 and C2 is calculated as,
C1 aC1 bC1  C 2 aC 2 bC 2
y
C1 bC1  C 2 bC 2
HOW DOES A NEURO-
FUZZY SYSTEM
LEARN?
A neuro-fuzzy system is essentially a multi-layer
neural network, and thus it can apply standard
learning algorithms developed for neural networks,
including the back-propagation algorithm.
When a training input-output example is
presented to the system, the back-propagation
algorithm computes the system output and
compares it with the desired output of the training
example. The error is propagated backwards
through the network from the output layer to the
input layer. The neuron activation functions are
modified as the error is propagated. To determine
the necessary modifications, the back-propagation
algorithm differentiates the activation functions of
the neurons.
FIVE-RULE NEURO-FUZZY
SYSTEM
 Suppose that fuzzy IF-THEN rules
incorporated into the system structure
are supplied by a domain expert. Prior or
existing knowledge can dramatically
expedite the system training.
 Besides, if the quality of training data is
poor, the expert knowledge may be the
only way to come to a solution at all.
However, experts do occasionally make
mistakes, and thus some rules used in a
neuro-fuzzy system may be false or
redundant. Therefore, a neuro-fuzzy
system should also be capable of
identifying bad rules.
 Given input and output linguistic values, a
neuro-fuzzy system can automatically
generate a complete set of fuzzy IF-THEN
rules.
 Let us create the system for the XOR
example. This system consists of 22  2 =
8 rules. Because expert knowledge is not
embodied in the system this time, we set
all initial weights between Layer 3 and
Layer 4 to 0.5.
 After training we can eliminate all rules
whose certainty factors are less than some
sufficiently small number, say 0.1. As a
result, we obtain the same set of four
EIGHT-RULE NEURO-FUZZY
SYSTEM
The combination of fuzzy logic and neural
networks constitutes a powerful means for
designing intelligent systems.

NEURO-
Domain knowledge can be put into a
neuro-fuzzy system by human experts in FUZZY
SYSTEMS:
the form of linguistic variables and fuzzy
rules.

When a representative set of examples is

SUMMARY
available, a neuro-fuzzy system can
automatically transform it into a robust set
of fuzzy IF-THEN rules, and thereby reduce
our dependence on expert knowledge
when building intelligent systems.
ANFIS:
ADAPTIVE NEURO-FUZZY
INFERENCE SYSTEM
The Sugeno fuzzy model was proposed for generating fuzzy rules from a
given input-output data set. A typical Sugeno fuzzy rule is expressed in
the following form:

IF x1 is A1
AND x2 is A2
. . . . .

AND xm is Am
THEN y = f (x1, x2, . . . , xm)

where x1, x2, . . . , xm are input variables; A1, A2, . . . , Am are fuzzy sets.
 When y is a constant, we obtain a
zero-order Sugeno fuzzy model in
which the consequent of a rule is
specified by a singleton.
 When y is a first-order polynomial, i.e.
y = k 0 + k 1 x 1 + k2 x 2 + . . . + k m x m
we obtain a first-order Sugeno
fuzzy model.
ADAPTIVE NEURO-FUZZY
INFERENCE SYSTEM
NORMALIZATIO
N LAYER
Layer 1 is the input layer. Neurons in
this layer simply pass external crisp
signals to Layer 2.

Layer 2 is the fuzzification layer.

Neurons in this layer perform
fuzzification. In Jang’s model,
fuzzification neurons have a bell
activation function.
Layer 3 is the rule layer. Each neuron in
this layer corresponds to a single Sugeno-
type fuzzy rule. A rule neuron receives
inputs from the respective fuzzification
neurons and calculates the firing strength
of the rule it represents. In an ANFIS, the
conjunction of the rule antecedents is
evaluated by the operator product. Thus,
the output of neuron i in Layer 3 is
obtained as,

where the value of 1 represents the firing

strength, or the truth value, of Rule 1.
Layer 4 is the normalisation layer. Each
neuron in this layer receives inputs from all
neurons in the rule layer, and calculates the
normalised firing strength of a given
rule.
The normalised firing strength is the ratio of
the firing strength of a given rule to the
sum of firing strengths of all rules. It
represents the contribution of a given rule
to the final result. Thus, the output of
neuron i in Layer 4 is determined as,
Layer 5 is the defuzzification layer.
Each neuron in this layer is connected
to the respective normalisation neuron,
and also receives initial inputs, x1 and
x2. A defuzzification neuron calculates
the weighted consequent value of a
given rule as,

where is the input and is the output of

defuzzification neuron i in Layer 5, and
ki0, ki1 and ki2 is a set of consequent
parameters of rule i.
Layer 6 is represented by a single
summation neuron. This neuron
calculates the sum of outputs of all
defuzzification neurons and produces
the overall ANFIS output, y,
Can an ANFIS deal with problems
where we do not have any prior
knowledge of the rule consequent
parameters?
It is not necessary to have any prior
knowledge of rule consequent
parameters. An ANFIS learns these
parameters and tunes membership
functions.
 Learning in the ANFIS
model
 An ANFIS uses a hybrid learning
algorithm that combines the least-
squares estimator and the gradient
descent method.
 In the ANFIS training algorithm, each
epoch is composed from a forward pass
and a backward pass. In the forward
pass, a training set of input patterns (an
input vector) is presented to the ANFIS,
neuron outputs are calculated on the
layer-by-layer basis, and rule consequent
 An ANFIS model with four
rules
Layer 1 Layer 2 Layer 3 Layer 4 Layer 5 Layer 6
x1 x2
A1 1 N1 1
x1

A2 2 N2 2
y

B1 3 N3 3
x2

B2 4 N4 4
 The ANFIS training data includes 101
training samples. They are represented
by a 101  3 matrix [x1 x2 yd], where
x1 and x2 are input vectors, and yd is a
desired output vector.
 The first input vector, x1, starts at 0,
increments by 0.1 and ends at 10.
 The second input vector, x2, is created
by taking sin from each element of
vector x1, with elements of the desired
output vector, yd, determined by the
function equation.
 A1

x1 A2 1 N1 1

A3

N2 2  y
2

B1
N3 3
3
x2

B2