Limitations of the traditional optimization

approaches

Limitations:

Computationally expensive.
For a discontinuous objective function, methods may fail.
Method may not be suitable for parallel computing.
Discrete (integer) variables are difficult to handle.
Methods may not necessarily adaptive.

Evolutionary algorithms have been evolved to address the above mentioned

limitations of solving optimization problems with traditional approaches.

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 1 / 26

Evolutionary Algorithms

The algorithms, which follow some biological and physical behaviors:

Biologic behaviors:
Genetics and Evolution – > Genetic Algorithms (GA) Behavior
of ant colony – > Ant Colony Optimization (ACO) Human
nervous system – > Artificial Neural Network (ANN)

In addition to that there are some algorithms inspired by some physical

behaviors:
Physical behaviors:
Annealing process – > Simulated Annealing (SA)
Swarming of particle – > Particle Swarming Optimization (PSO)
Learning – > Fuzzy Logic (FL)

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 2 / 26

Genetic Algorithm

It is a subset of evolutionary algorithm:

Ant Colony optimization

Swarm Particle Optimization

Models biological processes:

Genetics
Evolution
To optimize
highly
complex
objective
functions:
Very
difficult
to model
Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 3 / 26
Background of Genetic Algorithm

Firs time itriduced by Ptrof. John Holland (of Michigan University, USA,
1965).
But, the first article on GA was published in 1975.

Principles of GA based on two fundamental biological processes:

Genetics: Gregor Johan Mendel (1865)
Evolution: Charles Darwin (1875)

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 4 / 26

A brief account on genetics
The basic building blocks in living bodies are cells. Each cell carries the
basic unit of heredity, called gene
Nucleus

Chromosome

Other cell bodies

For a particular specie, number of chromosomes is fixed.

Examples
Mosquito: 6
Frogs: 26
Human: 46
Goldfish: 94
etc.
Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 5 / 26
A brief account on genetics
Genetic code

Spiral helix of protein substance is called DNA.

For a specie, DNA code is unique, that is, vary uniquely from one to
other.
DNA code (inherits some characteristics from one generation to next
generation) is used as biometric trait.
Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 6 / 26
A brief account on genetics

Reproduction

+ = Organism’s cell :
Cell division
x y
gamete diploid

(Reproductive cell has Each chromosome from both

haploid half the number of diploid haploids are combined to have
chromosomes) full numbers

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 7 / 26

A brief account on genetics
Crossing over

Kinetochore

Information from
two different Combined into so that diversity in
organism’s body information is possible
cells
Random crossover points
makes infinite diversities

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 8 / 26

A brief account on evolution
Evolution : Natural Selection

Four primary premises:

1 Information propagation: An offspring has many of its

characteristics of its parents (i.e. information passes from parent to its
offspring). [Heredity]
2 Population diversity: Variation in characteristics in the next
generation. [Diversity]
3 Survival for exitence: Only a small percentage of the offspring
produced survive to adulthood. [Selection]
4 Survival of the best: Offspring survived depends on their
inherited characteristics. [Ranking]

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 9 / 26

A brief account on evolution

Mutation:
To make the process forcefully dynamic when variations in population
going to stable.

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 10 / 26

Biological process : A quick overview

Genetics

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 11 / 26

Working of Genetic Algorithm

Definition of GA:
Genetic algorithm is a population-based probabilistic search and optimization
techniques, which works based on the mechanisms of natural genetics and
natural evaluation.

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 13 / 26

Framework of GA

Start

Note:
An individual in the
population is
corresponding to a
Initial Population possible solution

No
Converge ? Selection

Yes

Reproduction
Stop

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 14 / 26

Working of Genetic Algorithm

Note:

1 GA is an iterative process. It
2 is a searching technique.
3 Working cycle with / without
4 convergence.
Solution is not necessarily guranteed. Usually, terminated with a local
optima.

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 15 / 26

Framework of GA: A detail view

Start

Define parameters

Parameter representation

Create population
Initialize population
Apply cost
function to each of
the population

No
Converge ? Evaluate the fitness

Selectio
Yes

n
Select Mate
Stop

Crossover

Reproductio
Mutation

Inversion

n
Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 16 / 26
Optimization problem solving with GA

For the optimization problem, identify the following:

Objective function(s)

Constraint(s)

Input parameters

Fitness evaluation (it may be algorithm or mathematical formula)

Encoding

Decoding

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 17 / 26

GA Operators

In fact, a GA implementation involved with the realization of the

following operations.

1 Encoding: How to represent a solution to fit with GA framework.

2 Convergence: How to decide the termination criterion.
3 Mating pool: How to generate next solutions.
4 Fitness Evaluation: How to evaluate a solution.
5 Crossover: How to make the diverse set of next solutions.
6 Mutation: To explore other solution(s).
7 Inversion: To move from one optima to other.

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 18 / 26

Different GA Strategies

Simple Genetic Algorithm (SGA)

Steady State Genetic Algorithm (SSGA)

Messy Genetic Algorithm (MGA)

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 19 / 26

Simple GA

Start

Create Initial population

of size N

Evaluate each individuals

Convergence
Yes Return the individual(s) with
Criteria meet ?
best fitness value

No

Select Np individuals
(with repetition) Stop

Create mating pool (randomly) (Pair of

parent for generating new offspring)

Perform crossover and

create new offsprings Reproduction

Mutate the offspring

Perform inversion on
the offspring

Replace all individuals in the last generation

with new offsprings created

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 20 / 26

Important parameters involved in Simple GA

SGA Parameters

Initial population size : N

Size of mating pool, Np : Np = p%ofN

Convergence threshold δ

Mutation µ

Inversion η

Crossover ρ

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 21 / 26

Salient features in SGA

Simple GA features:

Have overlapping generation (Only fraction of individuals are

replaced).

Computationally expensive.

Good when initial population size is large. In

general, gives better results.

Selection is biased toward more highly fit individuals; Hence, the

average fitness (of overall population) is expected to increase in
succession.
The best individual may appear in any iteration.

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 22 / 26

Steady State Genetic Algorithm (SSGA)
Start

Generate Initial population of size N

Evaluate each individuals

Select two individual without

repetition

Crossover

Mutation

Inversion

Yes Reject the

offspring if
duplicated

No

Evaluate the offspring

If the offspring are better than the

worst individuals then replace the
worst individuals with the offspring

Convergence
meet ?

Return the solutions

Stop

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 23 / 26

Salient features in Steady-state GA

SGA Features:

Generation gap is small.

Only two offspring are produced in one generation.

It is applicable when

Population size is small Chromosomes

are of longer length
Evaluation operation is less computationally expensive (compare to
duplicate checking)

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 24 / 26

Salient features in Steady-state GA

Limitations in SSGA:

There is a chance of stuck at local optima, if

crossover/mutation/inversion is not strong enough to diversify the
population).

Premature convergence may result.

It is susceptible to stagnation. Inferiors are neglected or removed and

keeps making more trials for very long period of time without any
gain (i.e. long period of localized search).

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 25 / 26

**
*

Any Questions??

Debasis Samanta (IIT Kharagpur) Soft Computing Applications 26.02.2016 26 / 26