Demonstration1 G As

GENETIC ALGORITHMS - A GENTLE GENETIC ALGORITHMS - A GENTLE INTRODUCTION INTRODUCTION

A Demonstration By:A Demonstration By: SAFI UR REHMANSAFI UR REHMAN

PhD Scholar at Department of Mining EngineeringPhD Scholar at Department of Mining Engineering

)(25.1)tan()sin( yxyxz

OverviewOverview• Definition of Genetic AlgorithmsDefinition of Genetic Algorithms• Basic IdeaBasic Idea

– AlgorithmsAlgorithms– OptimizationOptimization

• Background of Genetic Algorithms (GAs)Background of Genetic Algorithms (GAs)– Natural worldNatural world– Theory of Natural SelectionTheory of Natural Selection

• Introduction to GAsIntroduction to GAs– Steps / Flowchart of GAsSteps / Flowchart of GAs

• ExamplesExamples• Applications of GAsApplications of GAs

04/12/2304/12/23 00:1300:13 22Genetic Algorithms – A gentle IntroductionGenetic Algorithms – A gentle Introduction

04/12/2304/12/23 00:1300:13 Genetic Algorithms – A gentle IntroductionGenetic Algorithms – A gentle Introduction 33

““Genetic algorithms are stochastic search and optimization Genetic algorithms are stochastic search and optimization algorithms based on the mechanics of natural selection and algorithms based on the mechanics of natural selection and natural genetics subject to survival of the fittest. The algorithm natural genetics subject to survival of the fittest. The algorithm iteratively transforms a set (population) of mathematical iteratively transforms a set (population) of mathematical objects (string structures), each with an associated fitness objects (string structures), each with an associated fitness value, into a new population of offspring objects value, into a new population of offspring objects using using crossover and mutationcrossover and mutation””

• Stochastic Search AlgorithmStochastic Search Algorithm• Natural Selection and Genetics OperatorsNatural Selection and Genetics Operators• Darwin’s Theory (Survival of the Fittest)Darwin’s Theory (Survival of the Fittest)• Evolutionary process Evolutionary process

» PopulationPopulation» Crossover Crossover » Mutation Mutation

Definition of Genetic AlgorithmsDefinition of Genetic Algorithms

• Suppose you have a problemSuppose you have a problem• You don’t know how to solve itYou don’t know how to solve it• What can you do?What can you do?• Can you use a computer to somehow find a solution for you?Can you use a computer to somehow find a solution for you?• This would be nice! Can it be done?This would be nice! Can it be done?

A “BLIND GENERATE AND TEST” Algorithm:A “BLIND GENERATE AND TEST” Algorithm:

RepeatRepeat Generate a random possible solution.Generate a random possible solution.

Test the solution Test the solution and see how good it isand see how good it is

Until Until solution is good enoughsolution is good enough


Basic IdeaBasic Idea

)(25.1)tan()sin( yxyxz x x andand y y range from 1 – 10range from 1 – 10

Can we used this dumb idea?Can we used this dumb idea?

• Sometimes - yes:Sometimes - yes:– if there are only a few possible solutionsif there are only a few possible solutions– and you have enough timeand you have enough time– then such a method then such a method couldcould be used be used

• For most problems - no:For most problems - no:– many possible solutionsmany possible solutions– with no time to try them allwith no time to try them all– so this method so this method can notcan not be used be used


Basic IdeaBasic Idea

if x if x andand y y range from 1 – 1000range from 1 – 1000

Basic IdeaBasic IdeaA “less-dumb” idea (GA)A “less-dumb” idea (GA)

GenerateGenerate a a setset of random solutions of random solutions

RepeatRepeatTest each solution in the Test each solution in the

set set (rank them) Remove some bad (rank them) Remove some bad solutions from set solutions from set

Duplicate Duplicate some good some good solutions make small solutions make small changes changes to some of themto some of them

Until Until best solution is good enoughbest solution is good enough


Search for OptimizationSearch for Optimization

locallocal

globalglobal

I am not at the top.I am not at the top.My high is better!My high is better!

I am at the I am at the toptopMy Height is ..My Height is ..

I will continueI will continue



OptimizationOptimization


““Genetic algorithms are Genetic algorithms are stochastic search and optimization stochastic search and optimization

algorithmsalgorithms based on the mechanics of natural selection and based on the mechanics of natural selection and

natural genetics subject to survival of the fittest. natural genetics subject to survival of the fittest. The algorithmThe algorithm

iteratively transforms a set (population) of iteratively transforms a set (population) of mathematical mathematical

objectsobjects (string structures), each with an (string structures), each with an associated fitness associated fitness

value,value, into a new population of offspring objects into a new population of offspring objects using using

crossover and mutationcrossover and mutation””


Background of GAsBackground of GAs• Genetic AlgorithmsGenetic Algorithms– Developed by John Holland in 1975Developed by John Holland in 1975– Developed the theoretical basis of GAs through Developed the theoretical basis of GAs through

Schema theoremSchema theorem– John Koza, David Goldberg, De JongJohn Koza, David Goldberg, De Jong

• Natural World Vs Function OptimizationNatural World Vs Function Optimization– Natural WorldNatural WorldDiversityDiversity Complexity Complexity Useful featuresUseful features• Why it is so ?Why it is so ?• How they come into being?How they come into being?

Can we imagine natural world as a result of many iterations in Can we imagine natural world as a result of many iterations in a grand optimization algorithm ?a grand optimization algorithm ?



Natural WorldNatural World

• GenesGenes are the basic “instructions” for building an organism are the basic “instructions” for building an organism

• A A chromosomechromosome is a sequence of genes is a sequence of genes

• Biologists distinguish between an organism’sBiologists distinguish between an organism’s

genotypegenotype (the genes and chromosomes) and its (the genes and chromosomes) and its

phenotypephenotype (what the organism actually is like) (what the organism actually is like)

• Similarly, “genes” may Similarly, “genes” may describedescribe a possible solution to a a possible solution to a

problem, without actually problem, without actually beingbeing the solution the solution


Natural WorldNatural World• Human body is made up of trillions of cells. Each cell has a Human body is made up of trillions of cells. Each cell has a

core structure (nucleus) that contains chromosomes. core structure (nucleus) that contains chromosomes.

• Each chromosome is made up of tightly coiled strands of Each chromosome is made up of tightly coiled strands of deoxyribonucleic acid (DNA). Genes are segments of DNA that deoxyribonucleic acid (DNA). Genes are segments of DNA that determine specific traits, such as eye or hair color. A human determine specific traits, such as eye or hair color. A human have more than 20,000 genes. have more than 20,000 genes.


Theory of Natural SelectionTheory of Natural Selection• In his book In his book The Origin of SpeciesThe Origin of Species Charles Darwin outlined the Charles Darwin outlined the

principle of natural selection.principle of natural selection.

• IF there are organisms that reproduce, and IF there are organisms that reproduce, and • IF off springs inherit traits from their progenitors, and IF off springs inherit traits from their progenitors, and • IF there is variability of traits, and IF there is variability of traits, and • IF the environment cannot support all members of a growing IF the environment cannot support all members of a growing

population, population, • THEN those members of the population with less-adaptive THEN those members of the population with less-adaptive

traits (determined by the environment) will die out, and traits (determined by the environment) will die out, and • THEN those members with more-adaptive traits (determined THEN those members with more-adaptive traits (determined

by the environment) will thrive by the environment) will thrive

The result is the evolution of The result is the evolution of species.. ““Select The Best, Discard The Rest” - Survival of the FittestSelect The Best, Discard The Rest” - Survival of the Fittest

http://en.wikipedia.org/wiki/Species


• Inspired by Inspired by natural evolutionnatural evolution• PopulationPopulation of individuals of individuals

– Individual is feasible solution to problemIndividual is feasible solution to problem

• Each individual is characterized by a Each individual is characterized by a Fitness functionFitness function– Higher fitness is better solutionHigher fitness is better solution

• Based on their fitness, parents are selected to reproduce Based on their fitness, parents are selected to reproduce offspringoffspring for a new for a new generationgeneration– Fitter individuals have more chance to reproduceFitter individuals have more chance to reproduce– New generation has same size as old generation; old generation diesNew generation has same size as old generation; old generation dies

• Offspring has Offspring has combinationcombination of properties of two parents of properties of two parents• If well designed, population will If well designed, population will convergeconverge to optimal solution to optimal solution

Introduction to Genetic AlgorithmsIntroduction to Genetic Algorithms


““Genetic algorithms are stochastic search and optimization Genetic algorithms are stochastic search and optimization

algorithms based on the mechanics of natural selection and algorithms based on the mechanics of natural selection and

natural genetics subject to survival of the fittest. The algorithm natural genetics subject to survival of the fittest. The algorithm

iteratively transforms a set (population) of mathematical iteratively transforms a set (population) of mathematical

objects objects (string structures), (string structures), each with an associated fitness each with an associated fitness

value,value, into a new population of offspring objects into a new population of offspring objects using using



Randomly generate a population of potential solutionsRandomly generate a population of potential solutionsRandomly generate a population of potential solutionsRandomly generate a population of potential solutions

Evaluate fitness of population membersEvaluate fitness of population membersEvaluate fitness of population membersEvaluate fitness of population members

Select two parents from Select two parents from population based on fitnesspopulation based on fitness

Select two parents from Select two parents from population based on fitnesspopulation based on fitness

Produce two childrenProduce two childrenProduce two childrenProduce two children

Evaluate childrenEvaluate childrenEvaluate childrenEvaluate children

Crossover and Crossover and mutationmutation

Crossover and Crossover and mutationmutation

Is solution "Good“?Is solution "Good“?Is solution "Good“?Is solution "Good“?OutputOutput

best solution foundbest solution found

OutputOutput

best solution foundbest solution found

Mul

tiple

Rep

eats

M

ultip

le R

epea

ts

in o

ne it

erati

onin

one

iter

ation

Mul

tiple

Rep

eats

M

ultip

le R

epea

ts

in o

ne it

erati

onin

one

iter

ation

NoNo YesYes

Genetic Algorithms FlowchartGenetic Algorithms Flowchart


Initial PopulationInitial Population

Create an initial population Create an initial population of random individualsof random individuals

Phenotype Phenotype xx

Genotype Genotype gg

Gen. Phen. MappingGen. Phen. Mapping

PopulationPopulation

Objective Function Objective Function ffii

Population PopPopulation Pop

Cross overCross over

MutationMutation

Genotype Genotype gg


A Simple Example of Genetic AlgorithmsA Simple Example of Genetic Algorithms• Simple problem: max xSimple problem: max x22 over {0,1,…,31} over {0,1,…,31}• GA approach:GA approach:– Representation: binary code, e.g. 01101 Representation: binary code, e.g. 01101 13 13

– Population size: 4Population size: 4– 1-point Crossover, bitwise mutation 1-point Crossover, bitwise mutation – Roulette wheel selectionRoulette wheel selection– Random initializationRandom initialization

• One generational cycle will be shown One generational cycle will be shown

1616 88 44 22 11

1313 00 11 11 00 11

2424 11 11 00 00 00

88 00 11 00 00 00


A Simple Example of Genetic AlgorithmsA Simple Example of Genetic Algorithms


Roulette Wheel SelectionRoulette Wheel Selection

2211nn

33

Area is Proportional Area is Proportional to fitness valueto fitness valueIndividual Individual ii will have a will have a

probability to be chosen probability to be chosen i

ifif

)()(

44



1001110011101101

11101111010000001001110011000000

Parent A Parent A

Child of A and B Child of A and B Parent B Parent B

Crossover OperatorCrossover OperatorCrossover OperatorCrossover Operator


““Genetic algorithms are stochastic search and optimization Genetic algorithms are stochastic search and optimization

algorithms based on the mechanics of natural selection and algorithms based on the mechanics of natural selection and

natural genetics subject to survival of the fittest. The algorithm natural genetics subject to survival of the fittest. The algorithm

iteratively transforms a set (population) of mathematical iteratively transforms a set (population) of mathematical

objects (string structures), each with an associated fitness objects (string structures), each with an associated fitness

value, into a new population of offspring objects value, into a new population of offspring objects using using



04/12/23 00:13 Genetic Algorithms – A gentle Introduction 24

• Function OptimizationFunction Optimization• Multi-Objective OptimizationMulti-Objective Optimization• Combinatorial OptimizationCombinatorial Optimization• Economics and Finance Economics and Finance • Resource MinimizationResource Minimization• Scheduling Scheduling • RoboticsRobotics• Image ProcessingImage Processing• Chemistry, Chemical Engineering Chemistry, Chemical Engineering • Networking and Communication Networking and Communication • Constraint Satisfaction Problems (CSP)Constraint Satisfaction Problems (CSP)• Electrical Engineering and Circuit DesignElectrical Engineering and Circuit Design• Engineering, Structural Optimization, and DesignEngineering, Structural Optimization, and Design

Applications of Genetic AlgorithmsApplications of Genetic Algorithms

T H A N K ST H A N K S

Demonstration1 G As

Technology

Transcript of Demonstration1 G As