Soft Computing Techniques

CH1: Introduction to Soft Computing

Techniques
Lecture Outline
• Introduction to Soft Computing
• Fuzzy Logic
• Neural Networks
• Genetic Algorithms
• Hybrid Systems
– Neuro Fuzzy Hybrid System
– Neuro Genetic Hybrid System
– Fuzzy Genetic Hybrid System
Introduction to Soft Computing
• Soft computing is foundation of conceptual intelligence in
machines.
• Soft computing differs from conventional (hard) computing;
where
– Hard computing is based on the concept of precise modeling and
analyzing to yield accurate results.
• It works well for simple problems.
– Soft Computing, unlike hard computing, is tolerant of
imprecision, uncertainty, partial truth, and approximation.
• Soft computing is well suited for real world problems where ideal
models are not available.
– In effect, the role model for soft computing is the human mind.
• The guiding principle of soft computing is to:
– Exploit the tolerance for imprecision, uncertainty, partial truth,
and approximation to achieve tractability, robustness and low
solution cost.
Introduction to Soft Computing
• The basic differences between hard computing
and soft computing is as in below:
Hard computing: Soft computing:
 Conventional computing that  Tolerant of imprecision,
requires a precisely stated uncertainty, approximation, …
analytical model.
 Often requires a lot of  Can solve some real world
computation time. problems in reasonably less time.
 Not suited for real world  Suitable for real world problems.
problems for which ideal model is
not present.
 It requires full truth.  Can work with partial truth.
 It is precise and accurate.  Imprecise.
 High cost for solution.  Low cost for solution.
Introduction to Soft Computing
• Soft computing may be viewed as a foundation component for the
emerging field of conceptual intelligence.
• Soft computing is an approach for constructing systems which are:
– computationally intelligent,
– possess human like expertise in particular domain,
– can adapt to the changing environment and can learn to do better, and
– can explain their decisions.
• The principal constituents/components of Soft Computing (SC) are:
– Fuzzy Logic (FL),
– Neural Computing (NC), based on neural networks and artificial neural
networks,
– Evolutionary Computation (EC), which is based on genetic algorithm,
– Machine Learning (ML) and
– Probabilistic Reasoning (PR).
• These are the principal constituent methodologies in soft computing
which are complementary rather than competitive.
Introduction to Soft Computing
• The complementarities of Fuzzy Logic, Neural
Computing, Genetic Computing, and Probability
Reasoning have an important consequence:
– In many cases a problem can be solved most effectively
by using FL, NC, GC and PR in combination rather than
exclusively.
– A striking example of a particularly effective
combination is what has come to be known as "neuro-
fuzzy systems”.
– Examples of such systems include products like photo
copy machines, washing machines, and to some extent
industrial applications.
Introduction to Soft Computing
• Fuzzy logic is mainly associated to imprecision,
approximate reasoning and computing with words,
• Neuro computing is to learning and curve fitting (also to
classification), and
• Probabilistic reasoning to uncertainty and belief
propagation (belief networks).
– These methods have in common that they
1. are nonlinear,
2. have ability to deal with non-linearities,
3. follow more human-like reasoning paths than classical methods,
4. utilize self-learning,
5. utilize yet-to-be-proven theorems,
6. are robust in the presence of noise or errors.
Introduction to Soft Computing
• Soft computing methods have been applied to
many real-world problems.
• Applications can be found in
– Signal processing,
– Pattern recognition,
– Quality assurance and industrial inspection,
– Business forecasting,
– Speech processing,
– Credit rating,
– Adaptive process control,
– Robotics control,
– Natural language understanding, etc.
Introduction to Soft Computing
• Soft computing can also be seen as a
foundation for the growing field of
computational intelligence (CI).
• The difference between traditional artificial
intelligence (AI) and computational intelligence
is that AI is based on hard computing whereas
CI is based on soft computing.
Fuzzy Logic - Introduction
• Fuzzy set theory was developed by Lotfi A. Zadeh, professor for computer
science at the University of California in Berkeley.

• Fussy set theory was developed to provide a mathematical tool for dealing
with the concepts used in natural language (linguistic variables).
• Fuzzy Logic is basically a multi-valued logic that allows intermediate values
to be defined between conventional evaluations(True and False).
– Proposed making the membership function (or the values F and T) operate over
the range of real numbers [0, 1].

• Fuzzy logic provides an inference morphology that enables approximate

human reasoning capabilities to be applied to knowledge-based systems.

• The theory of fuzzy logic provides a mathematical strength to capture the

uncertainties associated with human cognitive processes, such as thinking
and reasoning.
Fuzzy Logic - Introduction
• The developement of fuzzy logic was motivated in
large measure by the need for a conceptual
framework which can address the issue of
uncertainty and lexical imprecision.
• Some of the essential characteristics of fuzzy logic
relate to the following:
– In fuzzy logic, exact reasoning is viewed as a limiting
case of approximate reasoning.
– In fuzzy logic, everything is a matter of degree.
– In fuzzy logic, knowledge is interpreted a collection of
elastic or, equivalently, fuzzy constraint on a collection
of variables.
Fuzzy Logic - Introduction
• Inference is viewed as a process of propagation of
elastic constraints.
• Any logical system can be fuzzified.

• There are two main characteristics of fuzzy systems that

give them better performance for specific applications.

1. Fuzzy systems are suitable for uncertain or approximate

reasoning, especially for the system with a mathematical
model that is difficult to derive.
2. Fuzzy logic allows decision making with estimated values
under incomplete or uncertain information.
Neural Network – Introduction
• Neural network makes an attempt to simulate human
brain.
– The simulating is based on the present knowledge of brain
function, and this knowledge is even at its best primitive.
• The operation of brain is believed to be based on simple
basic elements called neurons which are connected to
each other with transmission lines called axons and
receptive lines called dendrites.
• The learning may be based on two mechanisms:
– the creation of new connections, and
– the modification of connections.
• Each neuron has an activation level which, in contrast
to Boolean logic, ranges between some minimum and
maximum value.
Neural Network – Introduction
• The following is a simple illustration of
biological and artificial neuron (perceptron).
Neural Network – Introduction
• In artificial neural networks the inputs of the neuron are
combined in a linear way with different weights.
– The result of this combination is then fed into a non-linear
activation unit (activation function), which can in its
simplest form be a threshold unit.
• Neural networks are often used to enhance and
optimize fuzzy logic based systems, e.g., by giving them
a learning ability.
– This learning ability is achieved by presenting a training set
of different examples to the network and using learning
algorithm which changes the weights (or the parameters of
activation functions) in such a way that the network will
reproduce a correct output with the correct input values.
Neural Network – Introduction
• The difficulty is how to guarantee generalization and to
determine when the network is sufficiently trained.
• Neural networks offer
– nonlinearity,
– input-output mapping,
– adaptivity and
– fault tolerance.
• Nonlinearity is a desired property if the generator of
input signal is inherently nonlinear.
• The high connectivity of the network ensures that the
influence of errors in a few terms will be minor, which
ideally gives a high fault tolerance.
Genetic algorithms – Introduction
• Probabilistic reasoning includes genetic algorithms, belief networks,
chaotic systems and parts of learning theory.
• Genetic algorithms optimize a given function by means of a random
search.
• They are best suited for optimization and tuning problems in the
cases where no prior information is available.
– As an optimization method genetic algorithms are much more effective
than a random search.

• They create a child generation from parent generation according to a

set of rules that mimic the genetic reproduction in biology.
• Randomness plays an important role, since
– the parents are selected randomly, but the best parents have greater
probability of being selected than the others,
– the number of ‘genes’ to be muted is selected randomly, and
– all bits in new child string can be flipped with a small probability.
Hybrid Systems
• Hybrid systems employ more than one
technology to solve a problem.
• Hybrid systems can be classified as:
– Sequential hybrid systems: the technologies are
used in a pipelining fashion;
– Auxiliary hybrid systems: the one technology calls
the other technology as subroutine;
– Embedded hybrid systems: the technologies
participating appear to be fused totally.
Hybrid Systems
• Hybridization of fuzzy logic, neural networks,
genetic algorithms has led to the creation of a
perspective scientific trend known as soft
computing.
– Neural networks mimic our ability to adapt to
circumstances and learn from past experience,
– Fuzzy logic addresses the imprecision or vagueness
in input and output,
– Genetic algorithms are inspired by biological
evolution, can systemize random search and reach
to optimum characteristics.
Hybrid Systems
Neuro-Fuzzy Hybrid Systems
• Neural Networks and Fuzzy Logic represents two
distinct methodologies to deal with uncertainty.
• Each of these has its own merits and demerits:
• Neural Networks:
– Merits: can model complex nonlinear relationships and
are appropriately suited for classification phenomenon
into predetermined classes.
– Demerits: Neural Network’s output,
• Precision is often limited to least squares errors;
• The training time required is quite large;
• The training data has to be chosen over entire range where
the variables are expected to change.
Hybrid Systems
Neuro-Fuzzy Hybrid Systems
• Fuzzy Logic:
– Merits: Fuzzy logic system, addresses the imprecision of
inputs and outputs defined by fuzzy sets and allow greater
flexibility in formulating detail system description.
• Integration of NN and FL, called Neuro-Fuzzy systems,
have the potential to extend the capabilities of systems
either of these two technologies applied individually.

• The integrated systems have turned out to be useful in:

– Accomplishing mathematical relationships among many
variables in a dynamic process,
– Performing mapping with some degree of imprecision, and
– Controlling nonlinear systems to an extent not possible with
conventional linear control systems.
Hybrid Systems
Neuro-Fuzzy Hybrid Systems
• There are two ways to do hybridization:
– One, is to provide NNs with fuzzy capabilities, there
by increasing the network’s expressiveness and
flexibility to adapt to uncertain environments.

– Second, is to apply neuronal learning capabilities to

fuzzy systems so that the fuzzy systems become
more adaptive to changing environments.
• This method is called NN driven fuzzy reasoning.
Hybrid Systems
Neuro-Genetic Hybrid Systems
• Neural networks
– can learn various tasks from examples, classy
phenomena and model nonlinear relationships.
• Genetic algorithms
– have offered themselves as potential candidates for the
optimization of parameters on NN.
• Integration of GAs and NNs has turned out to be
useful:
– Genetically evolved nets have reported comparable
results against their conventional counterparts.
– The gradient descent learning algorithms have reported
difficulties in learning the topology of networks whose
weights they optimize.
Hybrid Systems
Neuro-Genetic Hybrid Systems
– GA based algorithms have provided encouraging results
especially with regard to face recognition, animal
control, and others.
– Genetic algorithms encode the parameters of NNs as a
string of properties of the network, i.e. chromosomes.
• A large population of chromosomes representing many
possible parameters sets, for given NN, is generated.
– GA-NN have the ability to locate the neighborhood of
the optimal solution quicker than other conventional
search strategies.
• Drawback: large amount of memory required to handle and
manipulate chromosomes for a given network.
Hybrid Systems
Fuzzy-Genetic Hybrid Systems
• Here fuzzy systems have been integrated with GAs.
• The fuzzy systems like NNs (feed forward) are
universal approximator in the sense that they
exhibit the capability to approximate general
nonlinear functions to any desired degree of
accuracy.
• GAs are used to tackle the adjustments of system
parameters called for in the process, so that the
system output matches the training data.
• Several parameters which a fuzzy system involved
have been optimized using GAs.