Slide 1

Institute of Intelligent Power Electronics IPE Page1 Introduction to Basics of Genetic Algorithms Docent Xiao-Zhi Gao Department of Electrical Engineering and Automation xiao-zhi.gao@aalto.fi

Slide 2

Institute of Intelligent Power Electronics IPE Page2 Background of Genetic Algorithms Genetic Algorithms (GA) are optimization methods based on ideas of natural selection and evolutionary processes [Holland 75] GAs unique characteristics Derivative free Stochastic/Random and Global optimization Parallel search GA are applied in optimization of complex systems with little information

Slide 3

Institute of Intelligent Power Electronics IPE Page3 Charles Darwin (1809-1882)

Slide 4

Institute of Intelligent Power Electronics IPE Page4 The Origin of Species by Darwin [1859]

Slide 5

Institute of Intelligent Power Electronics IPE Page5 Species Evolution and Natural Selection

Slide 6

Institute of Intelligent Power Electronics IPE Page6 Evolution of Human Beings?

Slide 7

Institute of Intelligent Power Electronics IPE Page7 Evolution of Human Beings?

Slide 8

Institute of Intelligent Power Electronics IPE Page8 Darwinian Paradigm Reproduction Competition SelectionSurvive

Slide 9

Institute of Intelligent Power Electronics IPE Page9 John Henry Holland: Father of Genetic Algorithms John Henry Holland (born on 2 February 1929) is a Professor of Electrical Engineering and Computer Science at the University of Michigan, Ann Arbor Book Adaptation in Natural and Artificial Systems (1975) http://en.wikipedia.org/wiki/Jo hn_Henry_Holland

Slide 10

Institute of Intelligent Power Electronics IPE Page10 Conceptual Genetic Algorithms

Slide 11

Institute of Intelligent Power Electronics IPE Page11 Procedure of Genetic Algorithms 1. Encode some possible/random solutions into bit strings (chromosomes), e.g, 1010011, and construct a population (group of chromosomes) Float (real-valued) encoding is also possible 2. Evaluate these chromosomes, and calculate their fitnesses (i.e., how good they are) Based on criteria function (fitness function ) 3. Create new chromosomes by using mutation and crossover operators on the existing chromosomes Mutation and crossover probabilities

Slide 12

Institute of Intelligent Power Electronics IPE Page12 Chromosome, Genes and Genomes

Slide 13

Institute of Intelligent Power Electronics IPE Page13 Historical Terms in GA Binary strings: Genetic Algorithms Real-valued vectors : Evolution Strategies Finite state machines: Evolutionary Programming LISP trees: Genetic Programming etc.

Slide 14

Institute of Intelligent Power Electronics IPE Page14 Mutation Operator Mutation Probability

Slide 15

Institute of Intelligent Power Electronics IPE Page15 Crossover Operator Crossover Probability Two-Point Crossover One-Point Crossover

Slide 16

Institute of Intelligent Power Electronics IPE Page16 Procedure of Genetic Algorithms 4. Evaluate new chromosomes, and calculate their fitnesses 5. Select the mostly fitted chromosomes using roulette selection scheme Retain size of population fixed Better fitted chromosomes must have higher possibilities of survival Chromosome with the best fitness is always kept 6. Repeat above steps until a pre-set criteria is met

Slide 17

Institute of Intelligent Power Electronics IPE Page17 Roulette Selection Scheme

Slide 18

Institute of Intelligent Power Electronics IPE Page18 Flow Chart of Genetic Algorithms

Slide 19

Institute of Intelligent Power Electronics IPE Page19 Structure of Genetic Algorithms Best Chromosomes

Slide 20

Institute of Intelligent Power Electronics IPE Page20 Typical Behavior of GA

Slide 21

Institute of Intelligent Power Electronics IPE Page21 Typical Run: Progression of Fitness

Slide 22

Institute of Intelligent Power Electronics IPE Page22 GA as Problem Solvers: Goldbergs 1989 View

Slide 23

Institute of Intelligent Power Electronics IPE Page23 GA as Problem Solvers: Michalewicz 1996 View

Slide 24

Institute of Intelligent Power Electronics IPE Page24 An Example: Traveling Salesman Problem (TSP) The Traveling Salesman Problem: Find a tour of a given set of cities so that each city is visited only once the total distance traveled is minimized

Slide 25

Institute of Intelligent Power Electronics IPE Page25 Representation

Slide 26

Institute of Intelligent Power Electronics IPE Page26 Crossover

Slide 27

Institute of Intelligent Power Electronics IPE Page27 Mutation

Slide 28

Institute of Intelligent Power Electronics IPE Page28 TSP Example: 30 Cities

Slide 29

Institute of Intelligent Power Electronics IPE Page29 Solution i (Distance = 941)

Slide 30

Institute of Intelligent Power Electronics IPE Page30 Solution j (Distance = 800)

Slide 31

Institute of Intelligent Power Electronics IPE Page31 Solution k (Distance = 652)

Slide 32

Institute of Intelligent Power Electronics IPE Page32 Best Solution (Distance = 420)

Slide 33

Institute of Intelligent Power Electronics IPE Page33 Overview of Performance

Slide 34

Institute of Intelligent Power Electronics IPE Page34 An Example: Maximization of Peak Function Using GA The peak function is defined as: It has two inputs: x and y The peak function is a highly nonlinear function, and has multiple local minima Maximization of peak function is hard

Slide 35

Institute of Intelligent Power Electronics IPE Page35 Peak Function

Slide 36

Institute of Intelligent Power Electronics IPE Page36 Initial Populations

Slide 37

Institute of Intelligent Power Electronics IPE Page37 Populations After Five Generations

Slide 38

Institute of Intelligent Power Electronics IPE Page38 Populations After 10 Generations

Slide 39

Institute of Intelligent Power Electronics IPE Page39 Performance of GA Optimization

Slide 40

Institute of Intelligent Power Electronics IPE Page40 Some GA Applications

Slide 41

Institute of Intelligent Power Electronics IPE Page41 Reference David Fogel (born in 1964) is a pioneer in evolutionary computation (Ph.D. from University of California, San Diego in 1992). He is currently the CEO of the Natural Selection, Inc.

Slide 42

Institute of Intelligent Power Electronics IPE Page42 Reference Evolutionary Computation: Toward a New Philosophy of Machine Intelligence by David Fogel, 2005

Slide 43

Institute of Intelligent Power Electronics IPE Page43 Fusion of GA with Neural Networks GA can be employed to optimize the parameters and structures of neural networks Parameter optimization Structure optimization (how to encode?) GA-based optimization is a universal solution Independent of problems to be solved

Slide 44

Institute of Intelligent Power Electronics IPE Page44 An Example: GA-based Optimization of Elman Neural Network [Gao 00] Initial outputs of context nodes in Elman neural network play an important role in network prediction accuracy Hybrid training of Elman neural network consists of two parts Gradient descent algorithm for weights Genetic algorithms for initial context nodes outputs

Slide 45

Institute of Intelligent Power Electronics IPE Page45 Elman Neural Network

Slide 46

Institute of Intelligent Power Electronics IPE Page46 Training Procedure of Elman Neural Network with BP Learning

Slide 47

Institute of Intelligent Power Electronics IPE Page47 GA-involved Optimization Process of Initial Context Nodes Outputs PE: Prediction Error

Slide 48

Institute of Intelligent Power Electronics IPE Page48 Motor Fault Diagnosis Using Elman Neural Network with GA-aided Training

Slide 49

Institute of Intelligent Power Electronics IPE Page49 Conclusions Basic knowledge and typical applications of GA are introduced Numerous variants of GA exist [Fogel 06] GA can be used to handle data mining GA-based optimization is a quite time consuming procedure Suitable for simulations on high-speed parallel processing computers Fast GA are being investigated