Unit – 5

Genetic Algorithms - Introduction

Genetic Algorithm (GA) is a search-based optimization technique based on the principles
of Genetics and Natural Selection. It is frequently used to find optimal or near-optimal
solutions to difficult problems which otherwise would take a lifetime to solve. It is frequently
used to solve optimization problems, in research, and in machine learning.

Introduction to Optimization
Optimization is the process of making something better.
Optimization refers to finding the values of inputs in such a way that we get the “best” output
values. The definition of “best” varies from problem to problem, but in mathematical terms, it
refers to maximizing or minimizing one or more objective functions, by varying the input
parameters.
The set of all possible solutions or values which the inputs can take make up the search space.
In this search space, lies a point or a set of points which gives the optimal solution. The aim of
optimization is to find that point or set of points in the search space.

What are Genetic Algorithms?

Nature has always been a great source of inspiration to all mankind. Genetic Algorithms (GAs)
are search based algorithms based on the concepts of natural selection and genetics. GAs are a
subset of a much larger branch of computation known as Evolutionary Computation.
GAs were developed by John Holland and his students and colleagues at the University of
Michigan, most notably David E. Goldberg and has since been tried on various optimization
problems with a high degree of success.
In GAs, we have a pool or a population of possible solutions to the given problem. These
solutions then undergo recombination and mutation (like in natural genetics), producing new
children, and the process is repeated over various generations. Each individual (or candidate
solution) is assigned a fitness value (based on its objective function value) and the fitter
individuals are given a higher chance to mate and yield more “fitter” individuals. This is in line
with the Darwinian Theory of “Survival of the Fittest”.
In this way we keep “evolving” better individuals or solutions over generations, till we reach a
stopping criterion.
Genetic Algorithms are sufficiently randomized in nature, but they perform much better than
random local search (in which we just try various random solutions, keeping track of the best so
far), as they exploit historical information as well.

Advantages of GAs
GAs have various advantages which have made them immensely popular. These include −
 • Does not require any derivative information (which may not be available for many real-
world problems).
• Is faster and more efficient as compared to the traditional methods.
• Has very good parallel capabilities.
• Optimizes both continuous and discrete functions and also multi-objective problems.
• Provides a list of “good” solutions and not just a single solution.
• Always gets an answer to the problem, which gets better over the time.
• Useful when the search space is very large and there are a large number of parameters
involved.

Limitations of GAs
Like any technique, GAs also suffer from a few limitations. These include −
• GAs are not suited for all problems, especially problems which are simple and for which
derivative information is available.
• Fitness value is calculated repeatedly which might be computationally expensive for
some problems.
• Being stochastic, there are no guarantees on the optimality or the quality of the solution.
• If not implemented properly, the GA may not converge to the optimal solution.

Basic Terminology
• Population − It is a subset of all the possible (encoded) solutions to the given problem.
The population for a GA is analogous to the population for human beings except that
instead of human beings, we have Candidate Solutions representing human beings.
• Chromosomes − A chromosome is one such solution to the given problem.
• Gene − A gene is one element position of a chromosome.
• Allele − It is the value a gene takes for a particular chromosome.
• Genotype − Genotype is the population in the computation space. In the computation
space, the solutions are represented in a way which can be easily understood and
manipulated using a computing system.
• Phenotype − Phenotype is the population in the actual real world solution space in
which solutions are represented in a way they are represented in real world situations.
 • Decoding and Encoding − For simple problems, the phenotype and genotype spaces
are the same. However, in most of the cases, the phenotype and genotype spaces are
different. Decoding is a process of transforming a solution from the genotype to the
phenotype space, while encoding is a process of transforming from the phenotype to
genotype space. Decoding should be fast as it is carried out repeatedly in a GA during
the fitness value calculation.
For example, consider the 0/1 Knapsack Problem. The Phenotype space consists of
solutions which just contain the item numbers of the items to be picked.
However, in the genotype space it can be represented as a binary string of length n
(where n is the number of items). A 0 at position x represents that xth item is picked
while a 1 represents the reverse. This is a case where genotype and phenotype spaces are
different.
• Fitness Function − A fitness function simply defined is a function which takes the
solution as input and produces the suitability of the solution as the output. In some cases,
the fitness function and the objective function may be the same, while in others it might
be different based on the problem.
• Genetic Operators − These alter the genetic composition of the offspring. These include
crossover, mutation, selection, etc.

Basic Structure
The basic structure of a GA is as follows −
We start with an initial population (which may be generated at random or seeded by other
heuristics), select parents from this population for mating. Apply crossover and mutation
operators on the parents to generate new off-springs. And finally these off-springs replace the
existing individuals in the population and the process repeats. In this way genetic algorithms
actually try to mimic the human evolution to some extent.
A generalized pseudo-code for a GA is explained in the following program –

GA()
initialize population
find fitness of population

while (termination criteria is reached) do

parent selection
crossover with probability pc
mutation with probability pm
decode and fitness calculation
survivor selection
find best
return best
Genotype Representation:
Binary Representation
This is one of the simplest and most widely used representation in GAs. In this type of
representation the genotype consists of bit strings.
For some problems when the solution space consists of Boolean decision variables – yes or no,
the binary representation is natural. Take for example the 0/1 Knapsack Problem. If there are n
items, we can represent a solution by a binary string of n elements, where the xth element tells
whether the item x is picked (1) or not (0).

0 0 1 1 1 0 1 0 0 1

Real Valued Representation

For problems where we want to define the genes using continuous rather than discrete variables,
the real valued representation is the most natural. The precision of these real valued or floating
point numbers is however limited to the computer.

0.5 0.2 0.4 0.1 0.9 0.4 0.2 0.7 0.1 0.8

Integer Representation
For discrete valued genes, we cannot always limit the solution space to binary ‘yes’ or ‘no’. For
example, if we want to encode the four distances – North, South, East and West, we can encode
them as {1,2,3,4}. In such cases, integer representation is desirable.

4 2 3 1 3 2 1 4 1 2

Permutation Representation
In many problems, the solution is represented by an order of elements. In such cases
permutation representation is the most suited.
A classic example of this representation is the travelling salesman problem (TSP). In this the
salesman has to take a tour of all the cities, visiting each city exactly once and come back to the
starting city. The total distance of the tour has to be minimized. The solution to this TSP is
naturally an ordering or permutation of all the cities and therefore using a permutation
representation makes sense for this problem.

1 9 7 2 4 3 8 6 5 0
Population Initialization
There are two primary methods to initialize a population in a GA. They are −
• Random Initialization − Populate the initial population with completely random
solutions.
• Heuristic initialization − Populate the initial population using a known heuristic for the
problem.

Population Models:
There are two population models widely in use −
Steady State
In steady state GA, we generate one or two off-springs in each iteration and they replace one or
two individuals from the population. A steady state GA is also known as Incremental GA.
Generational
In a generational model, we generate ‘n’ off-springs, where n is the population size, and the
entire population is replaced by the new one at the end of the iteration.

Genetic Algorithms - Fitness Function:

The fitness function simply defined is a function which takes a candidate solution to the
problem as input and produces as output how “fit” our how “good” the solution is with
respect to the problem in consideration.

Calculation of fitness value is done repeatedly in a GA and therefore it should be sufficiently

fast. A slow computation of the fitness value can adversely affect a GA and make it
exceptionally slow.

fitness function should possess the following characteristics −

• The fitness function should be sufficiently fast to compute.
• It must quantitatively measure how fit a given solution is or how fit individuals can be
produced from the given solution.

Genetic Algorithms - Parent Selection:

Roulette Wheel Selection
In a roulette wheel selection, the circular wheel is divided as described before. A fixed point is
chosen on the wheel circumference as shown and the wheel is rotated. The region of the wheel
which comes in front of the fixed point is chosen as the parent. For the second parent, the same
process is repeated.
It is clear that a fitter individual has a greater pie on the wheel and therefore a greater chance of
landing in front of the fixed point when the wheel is rotated. Therefore, the probability of
choosing an individual depends directly on its fitness.
Implementation wise, we use the following steps −
• Calculate S = the sum of a finesses.
• Generate a random number between 0 and S.
• Starting from the top of the population, keep adding the finesses to the partial sum P, till
P<S.
• The individual for which P exceeds S is the chosen individual.
Tournament Selection
In K-Way tournament selection, we select K individuals from the population at random and
select the best out of these to become a parent. The same process is repeated for selecting the
next parent. Tournament Selection is also popular as it can even work with negative fitness
values.
Rank Selection
Rank Selection also works with negative fitness values and is mostly used when the individuals
in the population have very close fitness values (this happens usually at the end of the run). This
leads to each individual having an almost equal share of the pie (like in case of fitness
proportionate selection) as shown in the following image and hence each individual no matter
how fit relative to each other has an approximately same probability of getting selected as a
parent. This in turn leads to a loss in the selection pressure towards fitter individuals, making
the GA to make poor parent selections in such situations.
In this, we remove the concept of a fitness value while selecting a parent. However, every
individual in the population is ranked according to their fitness. The selection of the parents
depends on the rank of each individual and not the fitness. The higher ranked individuals are
preferred more than the lower ranked ones.

Random Selection
In this strategy we randomly select parents from the existing population. There is no selection
pressure towards fitter individuals and therefore this strategy is usually avoided.

Genetic Algorithms – Crossover:

In this more than one parent is selected and one or more off-springs are produced using the genetic
material of the parents. Crossover is usually applied in a GA with a high probability – pc

One Point Crossover

In this one-point crossover, a random crossover point is selected and the tails of its two parents
are swapped to get new off-springs.
Multi Point Crossover
Multi point crossover is a generalization of the one-point crossover wherein alternating
segments are swapped to get new off-springs.

Uniform Crossover
In a uniform crossover, we don’t divide the chromosome into segments, rather we treat each
gene separately.

Arithmetic Crossover
This is commonly used for integer representations and works by taking the weighted average of
the two parents by using the following formulae −

• Child1 = α.x + (1-α).y

• Child2 = α.y + (1-α).x
Obviously, if α = 0.5, then both the children will be identical as shown in the following imag
Genetic Algorithms – Mutation:
In simple terms, mutation may be defined as a small random improvement in the chromosome, to get a
new solution. It is used to maintain and introduce diversity in the genetic population and is usually
applied with a low probability – pm.