Post on 15-Jun-2022
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Computational IntelligenceComputational Intelligence
Assoc Prof. Sansanee Auephanwiriyakul, Ph.D.Assoc Prof. Sansanee Auephanwiriyakul, Ph.D.
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
TopicsTopics
• Introduction to Computational Intelligence• Introduction to artificial neural networks
– Neural networks foundation– Multi-layer perceptrons– Self organizing feature map
• Introduction to fuzzy system– Fuzzy set foundation– Fuzzy inference system– Fuzzy measure & fuzzy integral
• Introduction to Computational Intelligence• Introduction to artificial neural networks
– Neural networks foundation– Multi-layer perceptrons– Self organizing feature map
• Introduction to fuzzy system– Fuzzy set foundation– Fuzzy inference system– Fuzzy measure & fuzzy integral
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
TopicsTopics
• Introduction to evolutionary computing– Introduction to genetic algorithm– Introduction to genetic programming– Introduction to evolutionary computation– Co-evolution
• Introduction to swarm intelligence
• Introduction to evolutionary computing– Introduction to genetic algorithm– Introduction to genetic programming– Introduction to evolutionary computation– Co-evolution
• Introduction to swarm intelligence
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
ReferencesReferences
• [Bezdek94] J. C. Bezdek, “What is computational intelligence?” in Computational Intelligence: Imitating Life, J. M. Zurada, R. J. Marks, and C. J. Robinson (Eds.) Piscataway, NJ: IEEE Press, 1994, pp 1 – 11.
• [Bonabeau99] E. Bonabeau, M. Dorigo, and G. Theraulez, Swarm Intelligence:From Natural to Artificial Systems, Oxford University Press, Inc., New York, USA., 1999.
• [Clerc02] M. Clerc and J. Kennedy, “The particle swarm: explosion, stability, and convergence in a multi-dimensional complex space”, IEEE Transactions on Evolutionary Computation, vol 6. 2002, pp. 58 – 73.
• [Dorigo96] M. Dorigo, V. Maniezzo, and A. Colorni, “The ant system: optimization by a colony of cooperating agents”, IEEE Trans. Syst. Man Cybern. B, vol 26, 1996, pp. 29 – 41.
• [Bezdek94] J. C. Bezdek, “What is computational intelligence?” in Computational Intelligence: Imitating Life, J. M. Zurada, R. J. Marks, and C. J. Robinson (Eds.) Piscataway, NJ: IEEE Press, 1994, pp 1 – 11.
• [Bonabeau99] E. Bonabeau, M. Dorigo, and G. Theraulez, Swarm Intelligence:From Natural to Artificial Systems, Oxford University Press, Inc., New York, USA., 1999.
• [Clerc02] M. Clerc and J. Kennedy, “The particle swarm: explosion, stability, and convergence in a multi-dimensional complex space”, IEEE Transactions on Evolutionary Computation, vol 6. 2002, pp. 58 – 73.
• [Dorigo96] M. Dorigo, V. Maniezzo, and A. Colorni, “The ant system: optimization by a colony of cooperating agents”, IEEE Trans. Syst. Man Cybern. B, vol 26, 1996, pp. 29 – 41.
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
ReferencesReferences
• [Dorigo99] M. Dorigo, Artificial Life: The swarm intelligence approach, Tutorial TD1, Congress on Evolutionary Computing, Washington, DC., 1999.
• [Dubois82] Dubois, D. and Prade, H., “A Class of Fuzzy Measures based on Triangular norms”, Internat. J. General Systems, 8, 1982, pp. 43-61.
• [Dumitrescu00] D. Dumitrescu, B. Lazzerini, L.C. Jain, and A. Dumitrescu, Evolutionary Computation, CRC Press LLC, Florida, USA., 2000.
• [Eberhart07] R. Eberhart and Y. Shi, Computational Intelligence: Concepts to Implementations, Morgan Kaufmann Publishers, USA., 2007.
• [Eng05] G. K. Eng, and A. M. Ahmad, “Malay Speech Recognition using Slef-Organizing Map and Multilayer Perceptron”, Proceedings of the Postgraduate Annual Research Seminar, 2005.
• [Dorigo99] M. Dorigo, Artificial Life: The swarm intelligence approach, Tutorial TD1, Congress on Evolutionary Computing, Washington, DC., 1999.
• [Dubois82] Dubois, D. and Prade, H., “A Class of Fuzzy Measures based on Triangular norms”, Internat. J. General Systems, 8, 1982, pp. 43-61.
• [Dumitrescu00] D. Dumitrescu, B. Lazzerini, L.C. Jain, and A. Dumitrescu, Evolutionary Computation, CRC Press LLC, Florida, USA., 2000.
• [Eberhart07] R. Eberhart and Y. Shi, Computational Intelligence: Concepts to Implementations, Morgan Kaufmann Publishers, USA., 2007.
• [Eng05] G. K. Eng, and A. M. Ahmad, “Malay Speech Recognition using Slef-Organizing Map and Multilayer Perceptron”, Proceedings of the Postgraduate Annual Research Seminar, 2005.
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
ReferencesReferences
• [Engelbrecht02] A. P. Engelbrecht, Computational Intelligence: An Introduction, John Wiley & Sons, Ltd., West Sussex, England, 2002.
• [Engelbrecht07] A. P. Engelbrecht, Computational Intelligence: An Introduction, John Wiley & Sons, Ltd., West Sussex, England, 2007.
• [Fogel95] D. B. Fogel, “Review of Computational Intelligence: Imitating Life (book Review)”, Proceedings of the IEEE, Vol. 83, Issue 11, 1995, pp.1588 – 1592.
• [Fogel00a] D. B. Fogel, Evolutionary Computation: Principles and Practice for Signal Processing, SPIE-The international Society for Optical Engineering, Washington, USA., 2000.
• [Fogel00b] D.B. Fogel, Evolutionary Computation: Toward a New Philosophy of Machine Intelligence, IEEE Press, USA., 2000.
• [Engelbrecht02] A. P. Engelbrecht, Computational Intelligence: An Introduction, John Wiley & Sons, Ltd., West Sussex, England, 2002.
• [Engelbrecht07] A. P. Engelbrecht, Computational Intelligence: An Introduction, John Wiley & Sons, Ltd., West Sussex, England, 2007.
• [Fogel95] D. B. Fogel, “Review of Computational Intelligence: Imitating Life (book Review)”, Proceedings of the IEEE, Vol. 83, Issue 11, 1995, pp.1588 – 1592.
• [Fogel00a] D. B. Fogel, Evolutionary Computation: Principles and Practice for Signal Processing, SPIE-The international Society for Optical Engineering, Washington, USA., 2000.
• [Fogel00b] D.B. Fogel, Evolutionary Computation: Toward a New Philosophy of Machine Intelligence, IEEE Press, USA., 2000.
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
ReferencesReferences
• [Goldberg05] D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison Wesley Longman, Inc., USA., 2005.
• [Grabisch94] Grabisch, M. and Nicolas, J-M., “Classification by fuzzy integral: Performance and tests”, Fuzzy Set and Systems, 65, 1994, pp. 255-271.
• [Haykin94] S. Haykin, Neural Networks: A comprehensive Foundation, Macmillan Publishing Company, Inc. New Jersey, USA., 1994.
• [Haykin10] S. Haykin, Neural Networks and Learning Machines, Pearson Education, Inc., USA., 2010.
• [Karaboga05] D. Karaboga, “An Idea based on Honey Bee Swarm for Numerical Optimization”, Techincal Report –TR06, Erciyes University, Engineering Faculty, Computer Engineering Department, 2005.
• [Goldberg05] D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison Wesley Longman, Inc., USA., 2005.
• [Grabisch94] Grabisch, M. and Nicolas, J-M., “Classification by fuzzy integral: Performance and tests”, Fuzzy Set and Systems, 65, 1994, pp. 255-271.
• [Haykin94] S. Haykin, Neural Networks: A comprehensive Foundation, Macmillan Publishing Company, Inc. New Jersey, USA., 1994.
• [Haykin10] S. Haykin, Neural Networks and Learning Machines, Pearson Education, Inc., USA., 2010.
• [Karaboga05] D. Karaboga, “An Idea based on Honey Bee Swarm for Numerical Optimization”, Techincal Report –TR06, Erciyes University, Engineering Faculty, Computer Engineering Department, 2005.
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
ReferencesReferences
• [Keller94] Keller, J. M., Gader P., Tahani, H., Chiang, J-H. and Mohaned, M., “Advances in fuzzy integration for pattern recognition”, Fuzzy Sets and Systems, 65, 1994, pp.273-283.
• [Keller16] Keller, J.M., Liu, D. and Fogel, D.B., Fundamentals of Computational Intelligence:Neural Networks, Fuzzy Systems, and Evolutionary Computation, IEEE Press, Wiley, New Jersey, USA., 2016.
• [Kennedy98] J. Kennedy “The behavior of particles” in VW Porto, N. Saravanan, D. Waagen(eds), Proceedings of the 7th International Conference on Evolutionary Programming, 1998. pp. 581 – 589.
• [Klir95] G. J. Klir and B. Yuan, Fuzzy Stes and Fuzzy Logic: Theory and Applications, Prentice Hall Inc., New Jersey, USA., 1995.
• [Klir97] G. J. Klir, U. H. St. Clair, and B. Yuan, Fuzzy Set Theory: Foundations and Applications, Prentice Hall Inc., New Jersey, USA., 1997.
• [Keller94] Keller, J. M., Gader P., Tahani, H., Chiang, J-H. and Mohaned, M., “Advances in fuzzy integration for pattern recognition”, Fuzzy Sets and Systems, 65, 1994, pp.273-283.
• [Keller16] Keller, J.M., Liu, D. and Fogel, D.B., Fundamentals of Computational Intelligence:Neural Networks, Fuzzy Systems, and Evolutionary Computation, IEEE Press, Wiley, New Jersey, USA., 2016.
• [Kennedy98] J. Kennedy “The behavior of particles” in VW Porto, N. Saravanan, D. Waagen(eds), Proceedings of the 7th International Conference on Evolutionary Programming, 1998. pp. 581 – 589.
• [Klir95] G. J. Klir and B. Yuan, Fuzzy Stes and Fuzzy Logic: Theory and Applications, Prentice Hall Inc., New Jersey, USA., 1995.
• [Klir97] G. J. Klir, U. H. St. Clair, and B. Yuan, Fuzzy Set Theory: Foundations and Applications, Prentice Hall Inc., New Jersey, USA., 1997.
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
ReferencesReferences
• [Kohavi95] R. Kohavi, “A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection”, International Joint Conference on Artificial Intelligence (IJCAI), 1995
• [Kosko87] B. Kosko, “Fuzzy associative memories”, in Fuzzy Expert Systems, A. Kandel, Ed. Reading, MA: Addison-Wesley, 1987.
• [Kosko88] B. Kosko, “Bidirectional Associative Memories”, IEEE Transactions on Systems, Man, and Cybernetics, vol 18(1), 1988, pp. 49 –60.
• [Koza92] J. R. Koza Genetic Programming: On the Programming of Computers by Means of Natural Selection, The MIT Press, England, 1992.
• [Kreyszig05] E. Kreyszig, Advanced Angineering Mathematics, John Wiley & Sons, Inc., New Yorj USA., 2005.
• [Kruse95] R. Kruse, J. Gebhardt, and F. Klawonn, Foundations of Fuzzy Systems, John Wiley & Sons Ltd., West Sussex, England, 1995.
.
• [Kohavi95] R. Kohavi, “A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection”, International Joint Conference on Artificial Intelligence (IJCAI), 1995
• [Kosko87] B. Kosko, “Fuzzy associative memories”, in Fuzzy Expert Systems, A. Kandel, Ed. Reading, MA: Addison-Wesley, 1987.
• [Kosko88] B. Kosko, “Bidirectional Associative Memories”, IEEE Transactions on Systems, Man, and Cybernetics, vol 18(1), 1988, pp. 49 –60.
• [Koza92] J. R. Koza Genetic Programming: On the Programming of Computers by Means of Natural Selection, The MIT Press, England, 1992.
• [Kreyszig05] E. Kreyszig, Advanced Angineering Mathematics, John Wiley & Sons, Inc., New Yorj USA., 2005.
• [Kruse95] R. Kruse, J. Gebhardt, and F. Klawonn, Foundations of Fuzzy Systems, John Wiley & Sons Ltd., West Sussex, England, 1995.
.
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
ReferencesReferences• [Mitchell98] M. Mitchell, An Introducton to Genetic Algorithms, MIT
Press, Massachusetts Institute Technology, USA., 1998• [Ohnishi90] N. Ohnishi, A. Okamoto, and N. Sugiem, “Selective
Presentation of Learning Samples for Efficient Learning in Multi-Layer Perceptron”, Proceedings of the IEEE International Joint Conference on Neural Networks, vol 1, 1990, pp. 688 – 691.
• [Rosenblatt58] F. Rosenblatt, “The Perceptron: A propabilistic model for information storage and organization in the brain”, Psychological Review, vol 65, 1958, pp. 386 – 408.
• [Ross04] T. J. Ross, Fuzzy Logic with Engineering Applications, John Wiley & Sons Ltd., West Sussex, England, 2004.
• [Suganthan99] P. N. Suganthan, “Particle swarm optimizer with neighborhood operators”, Proceedings of the IEEE Congress on Evolutionary Computation, 1999, pp. 1958 – 1961.
• [Mitchell98] M. Mitchell, An Introducton to Genetic Algorithms, MIT Press, Massachusetts Institute Technology, USA., 1998
• [Ohnishi90] N. Ohnishi, A. Okamoto, and N. Sugiem, “Selective Presentation of Learning Samples for Efficient Learning in Multi-Layer Perceptron”, Proceedings of the IEEE International Joint Conference on Neural Networks, vol 1, 1990, pp. 688 – 691.
• [Rosenblatt58] F. Rosenblatt, “The Perceptron: A propabilistic model for information storage and organization in the brain”, Psychological Review, vol 65, 1958, pp. 386 – 408.
• [Ross04] T. J. Ross, Fuzzy Logic with Engineering Applications, John Wiley & Sons Ltd., West Sussex, England, 2004.
• [Suganthan99] P. N. Suganthan, “Particle swarm optimizer with neighborhood operators”, Proceedings of the IEEE Congress on Evolutionary Computation, 1999, pp. 1958 – 1961.
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
ReferencesReferences
• [Sugeno77] Sugeno, M., “ Fuzzy Measures and Fussy Integrals: A survey”, Fuzzy Automata and Decision Processes, Amsterdam, North Holland, 1977, pp. 89-102.
• [Tahani90] Tahani, H. and Keller, J. M., “Information Fusion in Computer Vision using the Fuzzy Integral”, IEEE Trans. on Systems, Man, and Cybernetics, 20(3), 1990, pp.773-741.
• [Theodoridis09] S. Theodoridis and K, Koutroumbas, Pattern Recognition, Academin Press, Elsevier Inc., London, UK., 2009.
• [Thodberg91] H. H. Thodberg, “Improving Generalization of Neural Networks through Prunning”, International Journal of Neural Systems, 1(4), 1991, pp. 317 – 326.
• [Turing50] A. M. Turing, Computing Machinery and Intelligence, Mind, vol 59, 1950, pp. 433 – 460.
• [Sugeno77] Sugeno, M., “ Fuzzy Measures and Fussy Integrals: A survey”, Fuzzy Automata and Decision Processes, Amsterdam, North Holland, 1977, pp. 89-102.
• [Tahani90] Tahani, H. and Keller, J. M., “Information Fusion in Computer Vision using the Fuzzy Integral”, IEEE Trans. on Systems, Man, and Cybernetics, 20(3), 1990, pp.773-741.
• [Theodoridis09] S. Theodoridis and K, Koutroumbas, Pattern Recognition, Academin Press, Elsevier Inc., London, UK., 2009.
• [Thodberg91] H. H. Thodberg, “Improving Generalization of Neural Networks through Prunning”, International Journal of Neural Systems, 1(4), 1991, pp. 317 – 326.
• [Turing50] A. M. Turing, Computing Machinery and Intelligence, Mind, vol 59, 1950, pp. 433 – 460.
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
ReferencesReferences
• [Wang92] Wang, Z. and Klir, G., “ Fuzzy Measure Theory”, New York, Plenum Press, 1992.
• [Zadeh95] L. A. Zadeh, “Fuzzy Sets”, Information and Control, 8(3), 1965,pp 338 -353.
• [Wang92] Wang, Z. and Klir, G., “ Fuzzy Measure Theory”, New York, Plenum Press, 1992.
• [Zadeh95] L. A. Zadeh, “Fuzzy Sets”, Information and Control, 8(3), 1965,pp 338 -353.
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Introduction to Computational IntelligenceIntroduction to Computational Intelligence
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
HistoryHistory
• Aristotle (384-322 BC) explain and codify styles of deductive reasoningsyllogisms
• Ramon Llull (1235-1316) Ars Magna machine consisting of a set of wheels supposed to be able to answer all questions.
• Gottfried Leibniz (1646 – 1716) calculus philosophicus a universal algebra that can be used to represent all knowledge in a deductive system
• George Boole (1854) developed the foundations of propositional logic part of AI tool
• Gottlieb Ferge (1879) developed the foundations of predicate calculus part of AI tool
• Alan Turing(1950) definition of AIstudy how machinary could be used to mimic processes of the human brain Book called “The chemical Basis of Morphogenesis artificial life
• Aristotle (384-322 BC) explain and codify styles of deductive reasoningsyllogisms
• Ramon Llull (1235-1316) Ars Magna machine consisting of a set of wheels supposed to be able to answer all questions.
• Gottfried Leibniz (1646 – 1716) calculus philosophicus a universal algebra that can be used to represent all knowledge in a deductive system
• George Boole (1854) developed the foundations of propositional logic part of AI tool
• Gottlieb Ferge (1879) developed the foundations of predicate calculus part of AI tool
• Alan Turing(1950) definition of AIstudy how machinary could be used to mimic processes of the human brain Book called “The chemical Basis of Morphogenesis artificial life
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
HistoryHistory
• Artificial intelligence first coined at the Dartmoutn conference organized by John Maccarthy (1956 )father of AI
• 1956 – 1969– Biological neurons
• Rosenblatt perceptrons• Widrow and Hoff adaline• Minsky and Papert (1969) cause a major set back to artificial neural networks conclude that the extension of simple perceptrons to multilayer perceptrons “is sterile”
• NN hibernation until the mid 1980s• Grossberg, Carpenter, Amari, Kohonen and Fukushimacontinue researching in
NN • Hopfield, Hinton and Rumelhart and McLelland (early and mid 1980s)
resurrection of NN research
• Artificial intelligence first coined at the Dartmoutn conference organized by John Maccarthy (1956 )father of AI
• 1956 – 1969– Biological neurons
• Rosenblatt perceptrons• Widrow and Hoff adaline• Minsky and Papert (1969) cause a major set back to artificial neural networks conclude that the extension of simple perceptrons to multilayer perceptrons “is sterile”
• NN hibernation until the mid 1980s• Grossberg, Carpenter, Amari, Kohonen and Fukushimacontinue researching in
NN • Hopfield, Hinton and Rumelhart and McLelland (early and mid 1980s)
resurrection of NN research
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
HistoryHistory
• Fraser, Bremermann and Reed (1950s) Genetic algorithm• John Holland father of Evolutionary Computation (EC)genetic
algorithms• Rechenberg (1960s) evolutionary strategies (ES)• Lawrence Fogel evolutionary programmingevolved behavior models• De Jong, Schaffer, Goldberge, Koza, Schwefel, Storn, and Price , important contricutors in EC field
• Gautama Buddha (563 BC) described things in shade of gray starting point of fuzzy logic
• Aristotle2-valued logic the birth of fuzzy logic• Lukasiewicz (1920)3-valued logic• Max Blackquasi-fuzzy sets
• Fraser, Bremermann and Reed (1950s) Genetic algorithm• John Holland father of Evolutionary Computation (EC)genetic
algorithms• Rechenberg (1960s) evolutionary strategies (ES)• Lawrence Fogel evolutionary programmingevolved behavior models• De Jong, Schaffer, Goldberge, Koza, Schwefel, Storn, and Price , important contricutors in EC field
• Gautama Buddha (563 BC) described things in shade of gray starting point of fuzzy logic
• Aristotle2-valued logic the birth of fuzzy logic• Lukasiewicz (1920)3-valued logic• Max Blackquasi-fuzzy sets
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
HistoryHistory
• Lotfi Zadehcontribute most in the field of fuzzy logicfather of Fuzzy Set
• Mamdani, Sugeno, Takagi and Bezdek famous in the fuzzy field• 1980 dark age for fuzzy research revive again in the late 1980• Pawlak (1991)rough set theory• Eugene N Marais (1871 – 1936) poetintroduce swarm intelligence
Books called “The Soul of the White Ant” and “The Soul of the Ape”• Eberhart and Kennedy (1995) particle swarm optimization• Marco Dorigo (1992)ant colonies
• ETC.• A lot of people are working in the CI field
• Lotfi Zadehcontribute most in the field of fuzzy logicfather of Fuzzy Set
• Mamdani, Sugeno, Takagi and Bezdek famous in the fuzzy field• 1980 dark age for fuzzy research revive again in the late 1980• Pawlak (1991)rough set theory• Eugene N Marais (1871 – 1936) poetintroduce swarm intelligence
Books called “The Soul of the White Ant” and “The Soul of the Ape”• Eberhart and Kennedy (1995) particle swarm optimization• Marco Dorigo (1992)ant colonies
• ETC.• A lot of people are working in the CI field
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Connection in CIConnection in CI
CI Study of adaptive mechanisms to enable or facilitateintelligent behavior in complex and changing environments
Artificial immune systems
Neural Networks
Evolutionary Computation
Fuzzy Systems
Swarm Intelligence
Probabilistic Techniques
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Neural Networks (NN)Neural Networks (NN)Nucleus
Dendrite
Cell Body
Axon
synapse
Biological Neural
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
NNNN
f(net)
output
input1
input2
inputp
weight1
weight2
weightp
Artificial neuron
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
NNNN
input layer
hidden layer
output layer
xp
x2
x1
p-m-2 :p input nodes, m hidden nodes, 2 output nodesNumber of hidden nodes or hidden layers will be shown here p-m1-m2-2 means that there are
2 hidden layers: one with m1 nodes and the second one is with m2 nodesNumber of output nodes normally corresponds to the number of classes
p-4-2
input layer
1st hiddenlayer
output layer
xp
x2
x1
2nd hiddenlayer
p-4-3-2
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Fuzzy SystemsFuzzy Systems
• Not only 0 or 1, there is a gray area• Unsharp boundary • Cope with uncertainty• Approximate reasoning
• Not only 0 or 1, there is a gray area• Unsharp boundary • Cope with uncertainty• Approximate reasoning
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Evolutionary ComputationEvolutionary Computation• Individualchromosomeinherit characteristic• Population of chromosome• Each characteristic --<gene– allel(gene value)• Each generationindividual compete to reproduce offspring• Best survival capability have the best chance to reproduce• Crossovercombine part of parents to generate offspring• Mutationalter some of allele of the chromosome• Survuval strength measured using fitness function objective function• Each generation individual culling or survive• Behaviorphenotype influence evolutionary process in genetic change and evolve
separately
• Individualchromosomeinherit characteristic• Population of chromosome• Each characteristic --<gene– allel(gene value)• Each generationindividual compete to reproduce offspring• Best survival capability have the best chance to reproduce• Crossovercombine part of parents to generate offspring• Mutationalter some of allele of the chromosome• Survuval strength measured using fitness function objective function• Each generation individual culling or survive• Behaviorphenotype influence evolutionary process in genetic change and evolve
separately
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Evolutionary ComputationEvolutionary Computation• EC algo
– Genrtic algorithmmodel genetic evolutiona– Genetic programmingsimilar to GA but individual are program– Evolutionary programmingderived from the simulation of adaptive behavior in
evolution (phenotype evolution)– Evolution strategiesmodel strategic parameters that control variation in
evolution– Differential evolutionsimilar to GA except reproduction mechanism used– Cultural evolution model the evolution of culture of a population and how the
culture influences the genetic and phenotypic evolution of individual– Coevolution “dump” individual evolve through cooperation or in competition
with one another, acquiring the necessary characteristics to survive
• EC algo– Genrtic algorithmmodel genetic evolutiona– Genetic programmingsimilar to GA but individual are program– Evolutionary programmingderived from the simulation of adaptive behavior in
evolution (phenotype evolution)– Evolution strategiesmodel strategic parameters that control variation in
evolution– Differential evolutionsimilar to GA except reproduction mechanism used– Cultural evolution model the evolution of culture of a population and how the
culture influences the genetic and phenotypic evolution of individual– Coevolution “dump” individual evolve through cooperation or in competition
with one another, acquiring the necessary characteristics to survive
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Swarm IntelligenceSwarm Intelligence• Study of colonies or swarm of social organisma• Study of social behavior of organisms (individuals) in swarm prompted the design f
very efficient optimization and clustering algorithm• Particle swarm optimization (PSO) global optimization approach modeled on social
behavior of bird flocks– A population based search procedure where the individuals (particles) are grouped
into a swarm– Particleindividualcandidate solution to the optimization problem
• Flown through the multidimensional search spaceadjusting its position in search space according to its own experience and that of neighboring particles
• Best position encounter by itself and that of its neighborposition itself to the global minimum• Performancemeasured according to a predefined fitness function which related to the problem
being solved
• Study of ant coloniesmodeling of pheromone depositing by ants in their search for the shortest paths to food sources resulted in the development of shortest path optimization algorithm
• Study of colonies or swarm of social organisma• Study of social behavior of organisms (individuals) in swarm prompted the design f
very efficient optimization and clustering algorithm• Particle swarm optimization (PSO) global optimization approach modeled on social
behavior of bird flocks– A population based search procedure where the individuals (particles) are grouped
into a swarm– Particleindividualcandidate solution to the optimization problem
• Flown through the multidimensional search spaceadjusting its position in search space according to its own experience and that of neighboring particles
• Best position encounter by itself and that of its neighborposition itself to the global minimum• Performancemeasured according to a predefined fitness function which related to the problem
being solved
• Study of ant coloniesmodeling of pheromone depositing by ants in their search for the shortest paths to food sources resulted in the development of shortest path optimization algorithm
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Introduction to Artificial Neural NetworksIntroduction to Artificial Neural Networks
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Neural Networks (NN)Neural Networks (NN)• Mapping function form I input space to K output space• Mapping function form I input space to K output space
: I KNNf
f(net)
o
x1
x2
xI
w1
w2
wI
orf(net)
o
x1
x2
xI
w1
w2
wI
1
I
i ii
net x w
or and called bias or threshold1
iI
wi
i
net x
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
NNNN
• Activation function or and
– linear functionwhere is a constant
• Activation function or and
– linear functionwhere is a constant
0f 1f 1f
f v v
v
f(v)
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
NNNN
• Step function or unit step function
normally 11 and 20 or 1
• Step function or unit step function
normally 11 and 20 or 1 1
2
if
if
vf v
v
v
f(v)
1
2
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
NNNN• Ramp function• Ramp function
if
if
if
v
f v v v
v
v
f(v)
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
NNNN
• Sigmoid function – Logistic function
– Hyperbolic tangent function
• Sigmoid function – Logistic function
– Hyperbolic tangent function
1
1 expf v
av
0
0
0.2
0.4
0.6
0.8
1
v
f(v)
1 exp 2tanh 1
2 1 exp 1 exp
v vf v
v v
0
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
v
f(v)
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
NNNN
• Gaussian function
where mean and standard deviation
• Gaussian function
where mean and standard deviation 22exp
vf v
00
v
f(v)
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Multi-layer perceptronsMulti-layer perceptrons• Example: AND Logic input vector x1,x2t with output y =0 are in the same class (1st class), one
with output 1 are in another class (2nd class) use unit step function with 11, 20 and 0as activation function
• Example: AND Logic input vector x1,x2t with output y =0 are in the same class (1st class), one with output 1 are in another class (2nd class) use unit step function with 11, 20 and 0as activation function
x1 x2 Y(desired output)
0 0 00 1 01 0 01 1 1
o
x1
x2
1
1
1
1.5
1 2 1.5v x x
x1
x2
1
1
0 1.5
1.5
x1x21.50
Decision boundary
Gray area 2nd class
White area 1st class
x1 x2 v o (program output)
0 0 1.5 00 1 0.5 01 0 0.5 01 1 0.5 1
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Multi-layer perceptronsMulti-layer perceptrons• Example : XOR Logic input vector x1,x2t with output y =0 are in the same class (1st class),
one with output 1 are in another class (2nd class) use unit step function with 11, 20 and 0 as activation function
• Example : XOR Logic input vector x1,x2t with output y =0 are in the same class (1st class), one with output 1 are in another class (2nd class) use unit step function with 11, 20 and 0 as activation function
x1 x2 Y(desired output)
0 0 00 1 11 0 11 1 0
x1 x2 v1o1 v2 o2 o (program
output)0 0 1.5 0 0.5 0 0.50 1 0.5 0 0.5 1 0.51 0 0.5 0 0.5 1 0.51 1 0.5 1 1.5 1 1.5
o1
1
o
x1
x2
1
1
1
1.5
1
1
1
0.5
0.5
2
1
1
2
3
o2
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
x1
x2
1
1
0 1.5
1.5
x1x21.50Output = 0
Output = 1
x1
x2
1
1
0 0.5
0.5
x1x20.50
Output = 1
Output = 0
x1
x2
1
1
0 0.5
0.5
Output = 0
1.5
1.5
Output = 1
Output = 0
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
x1
x2
1
2
3
4
5
x3
g1 g2
g3 g4
One hidden node create one decision boundary 4hidden nodes creates 4 decision boundary theseboundary is combined at the output node to createdecision for the decision making
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
w0w(t)
Ew
w(t1)
Ew
Ew
Gradient descent learning ruleDirection of gradient vector (Ew is in the increasingdirectionif we invert the direction we will have
New weight that is in the decreasing direction of the error space• if is small, the transient response of the algorithm will be
overdamped• If is large , the transient response of the algorithm will be
underdamped• If is larger than the critical value, the algorithm will be
unstable and may be diverged
( 1) ( )w t w t E E
1t t w w w E
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Backpropagation 2 passes;Forward pass and Backward pass
21
2j
j C
t e t
E
1
1 N
avn
nN
E E
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Multilayer PerceptronsMultilayer Perceptrons
• Initialization• Forward Computation
– Output from neuron j in layer l
wherewhere is the output signal of neuron i in the previous layer l1and is the synaptic weight of neuron j in layer l that is fed from neuron i in layer l1if l = 1, and if l = L,compute the error: ej(n) = dj(n) oj(n)
• Initialization• Forward Computation
– Output from neuron j in layer l
wherewhere is the output signal of neuron i in the previous layer l1and is the synaptic weight of neuron j in layer l that is fed from neuron i in layer l1if l = 1, and if l = L,compute the error: ej(n) = dj(n) oj(n)
( 1) ( )liy n
( ) ( )ljiw n
1( ) ( ) ( 1)
0( ) ( ) ( )
lml l lj ji i
iv n w n y n
( ) ( )l
j j jy n v n
(0) ( )j jy n x n (0) ( )j jy n o n
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Backward computation– Local gradients
where denotes differentiation with respect to the argument – Update weights
• Iteration: until the stopping criterion is met
• Backward computation– Local gradients
where denotes differentiation with respect to the argument – Update weights
• Iteration: until the stopping criterion is met
1 1
for neuron in output layer
for neuron in output layer
L Lj j j
lj l ll
j j k kjk
e n v n j Ln
v n n w n j l
j
11 1l l l l lji ji ji j jw n w n w n n y n
11 1l l l l l lji ji ji ji j jw n w n w n w n n y n
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• For
output layer
hidden layer
• For
output layer
hidden layer
( ) ( ) ( )( )
1
1 exp
l l lj j j l
j
y t v tv t
( )( )( ) ( )
( ) 2( )
exp
1 exp
lljj l l
j jl lj j
v ty tv t
v t v t
( ) ( )( ) ( )
1 11
1 exp 1 exp
l lj j l l
j j
v tv t v t
( ) ( ) ( ) ( )1l l l lj j j jv t y t y t
( ) ( ) 1L Lj j j jt e t o t o t
( ) ( ) ( ) ( 1) ( 1)1l l l l lj j j k kj
k
t y t y t t w t
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• For
output layer
hidden layer
• For
output layer
hidden layer
( )( ) ( ) ( )
( )
( ) 2tanh 1
2 1 exp
ljl l l
j j j lj
v ty t v t
v t
( )( )( ) ( )
( ) 2( )
2exp
1 exp
lljj l l
j jl lj j
v ty tv t
v t v t
( ) ( )( ) ( )
2 11
1 exp 1 exp
l lj j l l
j j
v tv t v t
( ) ( ) ( ) ( )2 1l l l lj j j jv t y t y t
( ) ( ) 2 1L Lj j j jt e t o t o t
( ) ( ) ( ) ( 1) ( 1)2 1l l l l lj j j k kj
k
t y t y t t w t
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Example: suppose at iteration t and and 0.2, 0.1. Find
• Example: suppose at iteration t and and 0.2, 0.1. Find
1
1t
x 0
1t
d (1)
32 1w t
x1
x2
1
2
3
1
2
-1.2
0.9
-0.6
-0.5
0.8
0.3-0.8
-2.1
-1.6
0.5-0.6
0.2-0.3
-0.20.5
-1.1
-0.7
l0 l1 l2
x1
x2
1
2
3
1
2
-1.0
-0.2
-0.2
-0.1
0.2
0.2-0.3
0.1
0.01
0.2-0.54
-1.10.2
-0.10.4
-0.5
1.2
l0 l1 l2
tt 1
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
2
(1) (1)1 1
0
1.2 (0.3)(1) ( 0.6)(1) 1.5i ii
v t w x
(1)1 (1)
1
1 1( ) 0.1824
1 exp(1.5)1 expy t
v t
2
(1) (1)2 2
0
0.9 ( 0.8)(1) (0.5)(1) 0.6i ii
v t w x
(1)2
1( ) 0.6457
1 exp( 0.6)y t
2
(1) (1)3 3
0
0.6 (0.5)(1) ( 1.6)(1) 1.7i ii
v t w x
(1)3
1( ) 0.1545
1 exp(1.7)y t
3
(2) (2) (1)1 1
0
0.5 (0.2)(0.1824) ( 0.2)(0.6457) ( 1.1)(0.1545) 0.7626i ii
v t w y
(2)1
1( ) 0.3181
1 exp(0.7626)y t
(2)1 ( ) 0 0.3181 0.3181e t
3
(2) (2) (1)2 1
0
0.8 ( 0.3)(0.1824) (0.5)(0.6457) ( 0.7)(0.1545) 0.96i ii
v t w y
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
(2)2
1( ) 0.7231
1 exp( 0.96)y t
(2)2 ( ) 1 0.7231 0.2769e t
(2) (2)1 1 1 11 ( 0.3181) 0.3181 1 0.3181 0.0690t e t o t o t
(2) (2)2 2 2 21 (0.2769) 0.7231(1 0.7231) 0.0554t e t o t o t
2
(1) (1) (1) (2) (2)3 3 3
1
1 k kjk
t y t y t t w t
(1)3 0.1545 1 0.1545 0.0690 1.1 0.0554 0.7 0.0048t
(2) (2) (2) (1)12 12 1 21w t w t t y t
(2)12 0.1 0.2 0.1 0.2 0.0690 0.6457 0.0189w t
(2) (2) (2)12 12 121 0.2 0.0189 0.2189w t w t w t
(1) (1) (1)32 32 3 21w t w t t x t
(1)32 0.1 1.6 0.01 0.2 0.0048 1 0.16w t
(1) (1) (1)32 32 321 1.6 0.16 1.76w t w t w t
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Generalization– Overtrainlook up tabledo not want– Properly fitwant– Factor
• Size of training data set• Architecture of NN• Physical complexity of the problem at hand
• Performance factor– Data preparation
• Missing valuediscard if have a lot of train data or replace the missing value with the average of that feature or with the most frequent
• Coding input value• Outlier discard that outlier if known and have a lot of train data if not,
adjust the objective function so that it can cope with the outlierrobustness
• Generalization– Overtrainlook up tabledo not want– Properly fitwant– Factor
• Size of training data set• Architecture of NN• Physical complexity of the problem at hand
• Performance factor– Data preparation
• Missing valuediscard if have a lot of train data or replace the missing value with the average of that feature or with the most frequent
• Coding input value• Outlier discard that outlier if known and have a lot of train data if not,
adjust the objective function so that it can cope with the outlierrobustness
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Scaling and normalization if some features dominate other featuresnormalize each feature with its own mean and standard deviation so that every features are in the same range
k feature of sample i where
and
– output value should be (0.1 or 0.9) or (-0.9 or 0.9) because if we use logistic function or hyperbolic tangent the computed valued will never goes to 0 or 1 (or -1 or 1) suppose there are 3 class: sample in class 1 will have the desire output be 0.9 0.1 0.1, that in class 2 will be 0.1 0.9 0.1 and that in class 3 will be 0.1 0.1 0.9 if use tanh, it should be 0.9 -0.9 0.9 (class 1), -0.9 0.9 -0.9 (class 2) and -0.9 -0.9 0.9 (class 3)
• Or – Noise injectionaround decision boundary– Training set manipulation selective presentationtypical pattern and
confusing pattern need priori information
• Scaling and normalization if some features dominate other featuresnormalize each feature with its own mean and standard deviation so that every features are in the same range
k feature of sample i where
and
– output value should be (0.1 or 0.9) or (-0.9 or 0.9) because if we use logistic function or hyperbolic tangent the computed valued will never goes to 0 or 1 (or -1 or 1) suppose there are 3 class: sample in class 1 will have the desire output be 0.9 0.1 0.1, that in class 2 will be 0.1 0.9 0.1 and that in class 3 will be 0.1 0.1 0.9 if use tanh, it should be 0.9 -0.9 0.9 (class 1), -0.9 0.9 -0.9 (class 2) and -0.9 -0.9 0.9 (class 3)
• Or – Noise injectionaround decision boundary– Training set manipulation selective presentationtypical pattern and
confusing pattern need priori information
ˆ ik kik
k
x xx
1
1 for 1,2,...,
N
k iki
x x k pN
22
1
1
1
N
k ik ki
x xN
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– Initialize weight with
– Learning rate and momentum rate
– Initialize weight with
– Learning rate and momentum rate
1 1,
fanin fanin
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Self organizing feature map (SOFM)Self organizing feature map (SOFM)
input (x)
winning neuron
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Competitive
Input neuron j withThere are l neurons
Winning neuron
• Cooperation
– hj,i(x) : symmetric about the maximum point defined by dji=0 (at winning neuron)– Amplitude of hj,i(x) decrease monotonically with increasing lateral distance dji , decaying
to 0 for dji=necessary condition for convergence
• Competitive
Input neuron j withThere are l neurons
Winning neuron
• Cooperation
– hj,i(x) : symmetric about the maximum point defined by dji=0 (at winning neuron)– Amplitude of hj,i(x) decrease monotonically with increasing lateral distance dji , decaying
to 0 for dji=necessary condition for convergence
1 2, , ,t
mx x xx 1 2, , ,t
j j j jmw w w w
argmin for jj
i x j x w A
1
hj,i(x)
i(x)
•hj,i(x
)
1-d lattice2-d lattice
22,j i j id r r
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Where between neuron j and winning neuron i in case of 1-d lattice and for 2-d lattice between ri and rj are
vector of neuron i and j
2,
, 2=exp for 2
j ij i
dh j
x A
00
1
dj,i
hj,i
2
01
exp for 0,1,2, ,n
n n
2,
, 2=exp for 0,1,2, ,2
j ij i
dh n n
n
x
,j id j i 22
,j i j id r r
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Synaptic adaptationsince update in 1 directionmight saturate, hence,include forgetting term (g(yj))
and
If neighborhood function is rectangle function
If neighborhood function is guassian
• Synaptic adaptationsince update in 1 directionmight saturate, hence,include forgetting term (g(yj))
and
If neighborhood function is rectangle function
If neighborhood function is guassian
for j j j jy g y j w x w A
This image cannot currently be displayed.
,j j iy h x
, for j j i jh j xw x w A
if is inside neighborhood function for
0 if is outside neighborhood function
jj
jj
j
x ww A
This image cannot currently be displayed.
if j is inside the neighborhood function1
if j is not inside the neighborhood function
j jj
j
t t t tt
t
w x ww
w
02
exp for 0,1,2, ,n
n n
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• 2 phase– Self-organizing or ordering phase
• might start from 0.1and decrease till the value is around 0.01 never goes to 0• Neighborhood function start from all neuron centered on the winning neuron
and shrink slowly with time– Convergence phase
• maintained at a small value on the order of 0.01 do not decrease to 0• Neighborhood function contain only the nearest neighbor of a winning neuron
eventually reduce to 1 or 0 neighbor
• 2 phase– Self-organizing or ordering phase
• might start from 0.1and decrease till the value is around 0.01 never goes to 0• Neighborhood function start from all neuron centered on the winning neuron
and shrink slowly with time– Convergence phase
• maintained at a small value on the order of 0.01 do not decrease to 0• Neighborhood function contain only the nearest neighbor of a winning neuron
eventually reduce to 1 or 0 neighbor
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Example
Input learning rate 0.1 use and hji(x) has the radius of 1
d(1) = 1.5033, d(2) =1.3038, d(3) = 0.1414, d(4) = 2.0616, d(5) = 3.4713Winning neuron is neuron 3 (i(x) = 3) , neighbor with radius of 1 will be node 2 and 4 only, hence, update weight at nodes 2, 3,and 4
• Example
Input learning rate 0.1 use and hji(x) has the radius of 1
d(1) = 1.5033, d(2) =1.3038, d(3) = 0.1414, d(4) = 2.0616, d(5) = 3.4713Winning neuron is neuron 3 (i(x) = 3) , neighbor with radius of 1 will be node 2 and 4 only, hence, update weight at nodes 2, 3,and 4
1 2 3 4 5
x
1
0.5
0.9
w 2
0.3
0.1
w 3
0.9
0.9
w4
0.3
0.6
w 5
1.3
1.6
w
1
1
x 2 21 1 2 2d x w x w
2
0.3 1 0.3 0.370.1
0.1 1 0.1 0.01
w 3
0.9 1 0.9 0.910.1
0.9 1 0.9 0.91
w
4
0.3 1 0.3 0.170.1
0.6 1 0.6 0.44
w 1
0.5
0.9
w5
1.3
1.6
w
1 2 3 4 5
x
hj,i(x)winning neuron
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• SOFM properties– Approximation of the input space– Topological ordering– Density matching– Feature selection
• SOFM properties– Approximation of the input space– Topological ordering– Density matching– Feature selection
x (input vector from linear predict code)
hidden layer
output layer
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Introduction to Fuzzy SystemsIntroduction to Fuzzy Systems
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Fuzzy systemsFuzzy systems
• Uncertainty and complexity– Driving in the unfamiliar condition
• Natural language– Cold: people in the north and in the south know what cold is but when it
happens people in these two area will say differently people in the north might say that 10 degree celsius is cold but people in the south might say that 20 degree celsius is cold
• Heap paradox
• Uncertainty and complexity– Driving in the unfamiliar condition
• Natural language– Cold: people in the north and in the south know what cold is but when it
happens people in these two area will say differently people in the north might say that 10 degree celsius is cold but people in the south might say that 20 degree celsius is cold
• Heap paradox
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Fuzzy Set FoundationFuzzy Set Foundation
• Characteristic functioncrisp set
• Membership function fuzzy setA: X 0,1 or A: X 0,1
• Characteristic functioncrisp set
• Membership function fuzzy setA: X 0,1 or A: X 0,1
ana1 a2
1
a-1 a0 an+10
A(x)
X
X
x1x2
xn
0 1
A
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
0 10 20 30 40 50 60 70 80 90 1000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
age
mem
bers
hip
valu
eMembership function of“old”
height
weight
0.3
0.5
0.9
Membership function of“big”: Countour diagram
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
No. Education level0 No education
1 Elementary school2 High school
3 2-year college4 Bachelor’s degree
5 Master’s degree6 Doctoral degree
0 1 2 3 4 5 60
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
education level
mem
bers
hip
valu
e
little education highly education
very highly education
Universal set 3 fuzzy sets (“little”, “high” and “very high” education)
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
name (symbol) Membership value in fuzzy set A
Carry (x1) 0.8Bill (x2) 0.3J-H (x3) 0.5
Wabei (x4) 0.9
0.8 0.3 0.5 0.9
Carry Bill J-H Wabei A
( )x
x
AA
( )
x
x
x A
A
How to write membership function as an equation
discrete continuous
How to write membership function
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Geometric representation
x1
x2
(1,0)
(1,1)(0,1)
(0,0)
A
0.5
0.3
universal set
1 2
0.5 0.3
x xA
Every point in this area are fuzzy setincluding 4corner pointsFuzzy empty set
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Analytic representation : function
A
5 5 6
( ) 7 6 7
0 else
x x
x x x
0 1 2 3 4 5 6 7 8 9 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
numberm
embe
rshi
p va
lue
example
Analytic representation
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
a
b
s s
A1
( )
0 else
x ab a s x a s
x s
a b dc
e
A( )
0 else
a x ea x b
a be b x c
xd x e
c x dd c
b
c
a
A
2
( )
x a
bx ce
Triangular shape
Trapezoidal shape
Bell-shape
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
a
0.5
b c
1
2
2
0 0
1
2
11
2
x a
x ax a x b
b a
x cb x c
c b
S
a
0.5
b c
1
d e f
-function is from S-function + reflected of S-function
S-function
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Interval-valued fuzzy set
A : ([0,1])X
0,1 denote the family of all closed interval of real number in [0,1] (0,1 P0,1)
1
a
2
A(x)
A(a) = [1, 2]
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Type-2 fuzzy set
denote the set of all ordinary fuzzy sets that can be defined within the universal [0,1]
: ([0,1])X A P
([0,1])P
1a
2
A(x)
3
4
1Ia(y)
y
b1
2
3
4
y
Ib(y)1
A(a) = fuzzy set (1, 2, 3, 4) A(b) = fuzzy set (1, 2, 3, 4)
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Level 2 fuzzy set
denote fuzzy power set of X( )XP : ([0,1])X A P
‘todd
ler’
‘young’‘c
hild
’
‘teen
age
’
‘child’
0 5 10
15
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Standard fuzzy complement
1 x x xA A X
Ax express degree to which x belong to fuzzy set A
express degree to which x does not belong to fuzzy set A xA
0 50 100 15010 20 30 40 60 70 80 90 110 120 130 1401400
0.2
0.4
0.6
0.8
1
0.1
0.3
0.5
0.7
0.90.9
credit hours
mem
bers
hip
valu
e
Inexperience
Experience
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Standard fuzzy union
ABx maxAx, Bx
patient high blood pressure(A)
High fever (B) High blood pressure or high
fever (AB)1 1 1 12 0.5 0.6 0.63 1 0.1 1
n 0.1 0.7 0.7
Do not satisfy the law of excluded middle
will not equal to universal setxA A
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Standard fuzzy intersection
ABx minAx, Bx
river Long river (A) Nevigable (B) Long and nevigable (AB)
Amazon 1 0.8 0.8Nile 0.9 0.7 0.7
Yang-Tsi 0.8 0.8 0.8Danube 0.5 0.6 0.5Rhine 0.4 0.3 0.3
Do not satisfy the law of contradiction
will not equal to fuzzy empty setxA A
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Inclusion: AB if AxBxxX
Equality: AB if Ax BxxX
Unary operation: Aax Axaif a>1more specific, if a <1less specific
0 10 20 30 40 50 60 70 80 90 1000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
age
mem
bers
hip
valu
e
Old
Very Old
Somewhat Old
Very old are from old2
somewhat old are from old0.5
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Support of fuzzy set A (supp(A) ):
supp(A) xX Ax>0
Height of fuzzy set A (h(A)) :
Largest value of membership value obtained by any element in that set that is >0
If hA1 normal fuzzy setIf hA<1 supnormal fuzzy set
If hA>1 supernormal fuzzy set
supx
h xA A
Core of fuzzy set A (core(A)) :
coreAxX AxhA since height is h(A) then we can havecoreAxX Ax = hA
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
-cut of fuzzy set A (A ): A xX Ax
Strong -cut of fuzzy set A (+A ): +A xX Ax>
0 10 20 30 40 50 60 70 80 90 1000
0.2
0.4
0.6
0.8
1
mem
bers
hip
valu
e
0E 0,100 or 0.2E 30,100 or 1E 70,100 (core(E))
0+E 20,100 (supp(E)) or 0.2+E 30,100 or 1+E
If 1<2 then 1A2Aand 1A2A2A , 1A2A1A
If 1<2 then 1+A2+Aand 1+A2+A2+A , 1+A2+A1+A
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Level set of fuzzy set A (LA or A ): LA A Ax=; xX
Special fuzzy set from -cut (A)
Ax Ax xX
ExampleA 0.2/x1 + 0.4/x2 + 0.6/x3 + 0.8/x4 + 1/x5
We have LA 0.2, 0.4, 0.6, 0.8, 1
-cut of A will be0.2A x1, x2, x3, x4, x5 1/ x1 + 1/ x2 + 1/ x3 +1/ x4 +1/ x50.4A 0/ x1 + 1/ x2 + 1/ x3 +1/ x4 +1/ x50.6A 0/ x1 + 0/ x2 + 1/ x3 +1/ x4 +1/ x50.8A 0/ x1 + 0/ x2 + 0/ x3 +1/ x4 +1/ x51A 0/ x1 + 0/ x2 + 0/ x3 +0/ x4 +1/ x5
Special fuzzy set will be0.2A 0.2/ x1 + 0.2/ x2 + 0.2/ x3 +0.2/ x4 +0.2/ x5
0.4A 0/ x1 + 0.4/ x2 + 0.4/ x3 +0.4/ x4 +0.4/ x5
0.6A 0/ x1 + 0/ x2 + 0.6/ x3 +0.6/ x4 +0.6/ x5
0.8A 0/ x1 + 0/ x2 + 0/ x3 +0.8/ x4 +0.8/ x5
1A 0/ x1 + 0/ x2 + 0/ x3 +0/ x4 +1/ x5
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Decomposition theorem
From special fuzzy set in the previous example0.2A 0.2/ x1 + 0.2/ x2 + 0.2/ x3 +0.2/ x4 +0.2/ x5
0.4A 0/ x1 + 0.4/ x2 + 0.4/ x3 +0.4/ x4 +0.4/ x5
0.6A 0/ x1 + 0/ x2 + 0.6/ x3 +0.6/ x4 +0.6/ x5
0.8A 0/ x1 + 0/ x2 + 0/ x3 +0.8/ x4 +0.8/ x5
1A 0/ x1 + 0/ x2 + 0/ x3 +0/ x4 +1/ x5
We can see that (0.2A0.4A0.6A0.8A1A) A
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
First theorem:
For and A ,
where Ax Ax , xX and is a standard fuzzy union
xP
0,1
A A
A
A
A
The standard fuzzy union of 3 special fuzzy sets A
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Second theorem:
For and A ,
where +Ax +Ax , xX and is a standard fuzzyunion
xP
0,1
A A
Third theorem:
For and A ,
where Ax Ax , xX and is a standard fuzzy union
xP
A
A A
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
A 0, 0.3, 0.6, 1 and 0A
therefore A is a union of 0.3A0.6A and 1A
This example shows that fordiscrete fuzzy set only 1st and 3rd
theorem will work fine
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Convex fuzzy set
A is a convex fuzzy set if
Meaning that membership values of all the points on the lineconnecting and is bigger than or equal to the minimummembership values of and
Or
If A is a convex fuzzy set then (1) -cut for all are notdisconnected, and (2)each -cut is convex in crisp sense
1 min , where 0 1r s r s A A A
r
rs
s
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
height
weight
0.90.5
0.3
r
s
All the points of the dashedline have a membershipvalues bigger than or equal tothe minimum membershipvalue of vector r and vector s
r
s
height
weight
Not convex fuzzy set
Convex fuzzy set
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Fuzzy Inference SystemFuzzy Inference System
• Hedge– Dilation: HAx = Ax1/2more or less of A– Concentration: HAx = Ax2 very A– Plus: HAx = Ax1.25
– Intensification:
slightly
– Vague
seldom
• Hedge– Dilation: HAx = Ax1/2more or less of A– Concentration: HAx = Ax2 very A– Plus: HAx = Ax1.25
– Intensification:
slightly
– Vague
seldom
2
2
12 if 0,
2
1 2 1 otherwise
x xx
x
A AHA
A
5.0)( if 2
)(1 1
5.0)( if 2
)(
)(xx
xx
xAA
AA
HA
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Generalized modus ponenimplication: If is A, then Y is Bpremise: is Aconclusion: Y is B
• Generalized modus ponenimplication: If is A, then Y is Bpremise: is Aconclusion: Y is B
x X
y
Y yf(x)
AX
B
Y yf(x)
x we get Yyfx is in A we get Y in
B yY yfx, xA
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
u X
B
Y
R
AX
B
Y
R
supmin , ,B A Rx X
y x x y
X X X
Relation R on XY if u and R we get YB where ByY<x,y>R
Relation R on XY if A we get YBwhereB yY <x,y>R, xA
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
X
B
YR
11 A
If R is a fuzzy relation on XY where Aand A are fuzzy sets on X, B and B arefuzzy sets on Y
If R (relation between A and B) and A aregiven, we can find B using
' supmin , ,x X
y x x y
B A R
Or BAR operator is a composition operatorHence, this is called compositional rule of inference
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Exampleimplication: if x and y are approximately equalpremise: x is littleconclusion: ?
Suppose little A 1,1, 2,0.6, 3,0.2, 4,0Suppose
if x and y are approximately equal
From B AR
• Exampleimplication: if x and y are approximately equalpremise: x is littleconclusion: ?
Suppose little A 1,1, 2,0.6, 3,0.2, 4,0Suppose
if x and y are approximately equal
From B AR
1.0 0.5 0.0 0.0
0.5 1.0 0.5 0.0
0.0 0.5 1.0 0.5
0.0 0.0 0.5 1.0
R
1.0 0.5 0.0 0.0
0.5 1.0 0.5 0.01.0 0.6 0.2 0.0
0.0 0.5 1.0 0.5
0.0 0.0 0.5 1.0
B B 1.0 0.6 0.5 0.2Max-min composition
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Several ways to create relation from implication:
– If is A, then Y is B for xX and yY , relation will beRx,y IAx, By
Where I is fuzzy implication• Lukasiewicz: IAx,By = min1, 1AxBy• Zadeh: IAx,By max1Ax, minAx,By• Kleen-Dienes: IAx,By max1Ax,By• Mamdani (correlation-min): IAx,By minAx,By• Correlation-product: IAx,By AxBy• Goedel:
• Several ways to create relation from implication:
– If is A, then Y is B for xX and yY , relation will beRx,y IAx, By
Where I is fuzzy implication• Lukasiewicz: IAx,By = min1, 1AxBy• Zadeh: IAx,By max1Ax, minAx,By• Kleen-Dienes: IAx,By max1Ax,By• Mamdani (correlation-min): IAx,By minAx,By• Correlation-product: IAx,By AxBy• Goedel:
1 if,
if
x yI x y
y x y
A BA B
B A B
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– If is A, then Y is BIf is A, then Y is B else V is UNKNOWN
where , Y and V are variables in X, Y and V and A, B and UNKNOWN
are fuzzy sets on X, Y and V
R AB UNKNOWN minimum, maximum
– If is A, then Y is B else Z is C
R AB C
– If is A, then Y is BIf is A, then Y is B else V is UNKNOWN
where , Y and V are variables in X, Y and V and A, B and UNKNOWN
are fuzzy sets on X, Y and V
R AB UNKNOWN minimum, maximum
– If is A, then Y is B else Z is C
R AB C
A
A
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• ExampleA 1/1 0.4/2 , B 0.4/2 1/3 and C 1/1 0.6/2
If is A, then Y is B else Z is C
• ExampleA 1/1 0.4/2 , B 0.4/2 1/3 and C 1/1 0.6/2
If is A, then Y is B else Z is C
1.0 0.0
0.4 0.0 0.4 1.0 0.6 1.0 0.6 0.0
0.0 1.0
R
0.0 0.4 1.0 0.0 0.0 0.0
0.0 0.4 0.4 0.6 0.6 0.0
0.0 0.0 0.0 1.0 0.6 0.0
R
0.0 0.4 1.0
0.6 0.6 0.4
1.0 0.6 0.0
R
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
If is A, then Y is B If is A, then Y is B
1.0 0.0
0.4 0.0 0.4 1.0 0.6 1.0 1.0 1.0
0.0 1.0
R
0.0 0.4 1.0 0.0 0.0 0.0
0.0 0.4 0.4 0.6 0.6 0.6
0.0 0.0 0.0 1.0 1.0 1.0
R
0.0 0.4 1.0
0.6 0.6 0.6
1.0 1.0 1.0
R
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Fuzzy Control: Simple Diagram
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Example: Inverted pendulum
input: angle and angular velocity output: current vt
membership functions same name for , and vt
negative large (NL), negative medium (NM), negative small (NS), zero (ZE), positive small (PS), positive medium (PM), and positive large (PL)
rule j: if is Aj and is Bj then vt is Cj
e.g. if is NL and is NL then vt is PL (NL,NL;PL)or if is ZE and is ZE then vt is ZE (ZE,ZE;ZE)
• Example: Inverted pendulum
input: angle and angular velocity output: current vt
membership functions same name for , and vt
negative large (NL), negative medium (NM), negative small (NS), zero (ZE), positive small (PS), positive medium (PM), and positive large (PL)
rule j: if is Aj and is Bj then vt is Cj
e.g. if is NL and is NL then vt is PL (NL,NL;PL)or if is ZE and is ZE then vt is ZE (ZE,ZE;ZE)
motor
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
NL NM NS ZE PS PM PL
NL PL
NM PM
NS PS
ZE PL PM PS ZE NS NM NL
PS NS
PM NM
PL NL
Total number of rules 7 7 7 =343 but some of rules do not make sense.Hence, the number of usable rules will be less than that
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Fuzzy associative memory (FAM)rule 1: If is A1, then Y is B1 (A1,B1)R1
rule 2: If is A2, then Y is B2 (A2,B2)R2
rule n: If is An, then Y is Bn (An,Bn)Rn
premise: is Aconclusion: Y is B
B w1B1 w2B2 ... wnBn where Bi ARi
• Fuzzy associative memory (FAM)rule 1: If is A1, then Y is B1 (A1,B1)R1
rule 2: If is A2, then Y is B2 (A2,B2)R2
rule n: If is An, then Y is Bn (An,Bn)Rn
premise: is Aconclusion: Y is B
B w1B1 w2B2 ... wnBn where Bi ARi
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Supposed: Ai a1/x1 a2/x2 ... am/xm Ai a1, a2,..., am
Correlation-min:
OR
Supposed: Ai a1/x1 a2/x2 ... am/xm Ai a1, a2,..., am
Correlation-min:
OR
Ti i i R A B
1
2
i
ii
m i
a
a
a
B
BR
B
T T T1 2i i i p ib b b R A A A
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Correlation-product :
OR
Correlation-product :
OR
Ti i i R A B
1
2
i
ii
m i
a
a
a
B
BR
B
T T T1 2i i i p ib b b R A A A
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Conclude B AR A
A
Conclude B AR A
A
n
jj R
supn
jj
R
supn
jj
A R
Tsupn
j jj
A A B
Tsupn
j jj
A A B
T T T T1 1 2 2sup , ,..., ,...,i i n n
B A A B A A B A A B A A B
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Example: A 0/x1 0/x2 ... 1/xi 0/xi+1 ... 0/xm
Correlation-min AAjT Bj minAixi, Bj
Correlation-product AAjT Bj AixiBj
Example: A 0/x1 0/x2 ... 1/xi 0/xi+1 ... 0/xm
Correlation-min AAjT Bj minAixi, Bj
Correlation-product AAjT Bj AixiBj
YX
Mem
bers
hip
Mem
bers
hip
A1B1
xi
1
ai1
min(ai1,B1)
1
Mem
bers
hip
Aj
xi
ai
Mem
bers
hip
Bj
aiBj
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Example: A 0/x1 0/x2 ... 1/xi 0/xi+1 ... 0/xm
Supposed that there are 2 rules (A1;B1) and (A2; B2)
B sup AA1T B1, AA2
T B2
where AA1T ai1 and AA2
T ai2
• Example: A 0/x1 0/x2 ... 1/xi 0/xi+1 ... 0/xm
Supposed that there are 2 rules (A1;B1) and (A2; B2)
B sup AA1T B1, AA2
T B2
where AA1T ai1 and AA2
T ai2
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Example
A is a fuzzy set not crisp set
Supposed that there are 2 rules(A1;B1) and (A2; B2)
B sup AA1T B1, AA2
T B2
where
• Example
A is a fuzzy set not crisp set
Supposed that there are 2 rules(A1;B1) and (A2; B2)
B sup AA1T B1, AA2
T B2
where supmin ,Tj j j
x Xr x x
A A A A A
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
YX
Mem
bers
hip
Mem
bers
hip
A1B1
xi
1
r1(A)
min(r1(A),B1))
YX
Mem
bers
hip
Mem
bers
hip
1
xi
A2B2
min(r2(A),B2)r2(A)
Y
Mem
bers
hip
B
A
A
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• If there are more than 1 input A,B;CA a1/x1 a2/x2 ... am/xm and B b1/y1 b2/y2 ... bm/ym
split into A;C and B;Cand
fact: A Ixi 0/x1 0/x2 ... 1/xi 0/xi+1 ... 0/xm and
B Iyj 0/y1 0/y2 ... 1/yj 0/ji+1 ... 0/ym
compute: FA, B C AMACBMBC where AMAC Ix
iMAC
aiC
• If there are more than 1 input A,B;CA a1/x1 a2/x2 ... am/xm and B b1/y1 b2/y2 ... bm/ym
split into A;C and B;Cand
fact: A Ixi 0/x1 0/x2 ... 1/xi 0/xi+1 ... 0/xm and
B Iyj 0/y1 0/y2 ... 1/yj 0/ji+1 ... 0/ym
compute: FA, B C AMACBMBC where AMAC Ix
iMAC
aiC
TACM A C TBCM B C
1
2Iix
m
a
a
a
C
C
C
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
and BMBC IyjMBC
bjC
Hence, the conclusion will be C aiC bjC minai,bjC
If use correlation-product C aiCbjC minai,bjC
and BMBC IyjMBC
bjC
Hence, the conclusion will be C aiC bjC minai,bjC
If use correlation-product C aiCbjC minai,bjC
1
2=Ijy
m
b
b
b
C
C
C
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Mamdani modelRule: If 1 is A(1), and 2 is A(2) and ... and n is A(n) then is B
where A(1), A(2 and ... and A(n) are linguistic terms in Txi for 1in and
B is linguistic term in Ty andsuppose there are more than 1 ruleLet Tx1A1
(1), A2(1),... ,AN1
(1)
Tx2 A1(2), A2
(2),... ,AN2(2)
Txn A1(n), A2
(n),... ,ANn(n)
and Ty B1, B2,... ,BN0
rule j: If 1 is Ai1,j(1), and 2 is Ai2,j
(2) and ... and n is Ain,j(n) then is Bi,j
where i11,2,...,N1, i21,2,...,N2,..., in1,2,...,Nn and
i1,2,...,N0
• Mamdani modelRule: If 1 is A(1), and 2 is A(2) and ... and n is A(n) then is B
where A(1), A(2 and ... and A(n) are linguistic terms in Txi for 1in and
B is linguistic term in Ty andsuppose there are more than 1 ruleLet Tx1A1
(1), A2(1),... ,AN1
(1)
Tx2 A1(2), A2
(2),... ,AN2(2)
Txn A1(n), A2
(n),... ,ANn(n)
and Ty B1, B2,... ,BN0
rule j: If 1 is Ai1,j(1), and 2 is Ai2,j
(2) and ... and n is Ain,j(n) then is Bi,j
where i11,2,...,N1, i21,2,...,N2,..., in1,2,...,Nn and
i1,2,...,N0
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Firing strength of rule j will be j minAi1,j
(1)(x1),Ai2,j(2)(x2),...,Ain,j
(n)(xn)Output of rule j will be
Overall output
Firing strength of rule j will be j minAi1,j
(1)(x1),Ai2,j(2)(x2),...,Ain,j
(n)(xn)Output of rule j will be
Overall output
1 2
1 21, 1 2, 2 , ,, ,..., min , ,..., ,
n
j ni j i j in j n i jx x x y x x x y OUT A A A B
1 2
1 2, ,..., 1, 1 2, 2 , ,
1,2,...,max min , ,..., ,
n
nx x x i j i j in j n i j
j ky x x x y
OUT A A A B
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Defuzzification
– Max membership select de_y where Bde_yBde_y for all
de_yYif Myy1, y2 ByhB where hB is a height of B
– Center of area (centroid)
• Defuzzification
– Max membership select de_y where Bde_yBde_y for all
de_yYif Myy1, y2 ByhB where hB is a height of B
– Center of area (centroid)
_ k
ky M
y
de yM
_
k kk
kk
y y
de yy
B
B
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– Simplified centroid
where cj is firing strength ( aij or rjA ) and y0Bj is the mode
of output of rule j
– Simplified centroid
where cj is firing strength ( aij or rjA ) and y0Bj is the mode
of output of rule j
N
N
0
_j
n
n
jj
jj
c y
de yc
B
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Examplerule 1: If x1 is L1 และ x2 is H2, then y is Lrule 2: If x1 is M1 และ x2 is M2, then y is H
input (-4,3)
1 minL1(-4),H2(3) min (0.67,0.5) 0.52 minM1(-4),M2(3) min (0.33,0.5) 0.5
• Examplerule 1: If x1 is L1 และ x2 is H2, then y is Lrule 2: If x1 is M1 และ x2 is M2, then y is H
input (-4,3)
1 minL1(-4),H2(3) min (0.67,0.5) 0.52 minM1(-4),M2(3) min (0.33,0.5) 0.5
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
H
6 42 0 2 4 6X1
0.2
0.40.6
0.8
1.0L1 M1 H1
6 42 0 2 4 6X2
0.2
0.40.6
0.8
1.0L2 M2 H2
6 42 0 2 4 6Y
0.2
0.40.6
0.8
1.0L M
6 42 0 2 4 6X1
0.2
0.40.6
0.8
1.0L1 M1 H1
6 42 0 2 4 6X2
0.2
0.40.6
0.8
1.0L2 M2 H2
6 42 0 2 4 6Y
0.2
0.40.6
0.8
1.0L M H
6 42 0 2 4 6Y
0.2
0.40.6
0.8
1.0
Defuzzify the output fuzzy setto get the final output
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Takagi-Sugeno modelRule: If 1 is A(1), and 2 is A(2) and ... and n is A(n) then is B
where A(1), A(2 and ... and A(n) are linguistic terms in Txi for 1in and B is linguistic term in Ty and suppose there are more than 1 ruleLet Tx1A1
(1), A2(1),... ,AN1
(1)
Tx2 A1(2), A2
(2),... ,AN2(2)
Txn A1(n), A2
(n),... ,ANn(n)
rule j: If 1 is Ai1,j(1), and 2 is Ai2,j
(2) and ... and n is Ain,j(n) then
j fj1, 2,..., nwhere i11,2,...,N1, i21,2,...,N2,..., in1,2,...,Nn and
fjx1, x2,..., xn a1jx1 a2
jx2 ... anjxn a0
j
• Takagi-Sugeno modelRule: If 1 is A(1), and 2 is A(2) and ... and n is A(n) then is B
where A(1), A(2 and ... and A(n) are linguistic terms in Txi for 1in and B is linguistic term in Ty and suppose there are more than 1 ruleLet Tx1A1
(1), A2(1),... ,AN1
(1)
Tx2 A1(2), A2
(2),... ,AN2(2)
Txn A1(n), A2
(n),... ,ANn(n)
rule j: If 1 is Ai1,j(1), and 2 is Ai2,j
(2) and ... and n is Ain,j(n) then
j fj1, 2,..., nwhere i11,2,...,N1, i21,2,...,N2,..., in1,2,...,Nn and
fjx1, x2,..., xn a1jx1 a2
jx2 ... anjxn a0
j
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
overall output
where j is the firing degree of rule j
No need for defuzzification
overall output
where j is the firing degree of rule j
No need for defuzzification
1 21
1
, ,...k
j j nj
k
jj
f x x x
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Example turning steering wheel control systeminput: 1, 2, 3 and 4
universal set: X10, 150cm, X20, 150 cm, X390, 90 degree and X40, 150 cm
output: rotation speed () of the steering wheel
Rule j: If 1 is Ai1,j(1), และ 2 is Ai2,j
(2) และ 3 is Aj3,j(3) และ 4 is Aj4,j then p0 p11
p22 p33 p44
• Example turning steering wheel control systeminput: 1, 2, 3 and 4
universal set: X10, 150cm, X20, 150 cm, X390, 90 degree and X40, 150 cm
output: rotation speed () of the steering wheel
Rule j: If 1 is Ai1,j(1), และ 2 is Ai2,j
(2) และ 3 is Aj3,j(3) และ 4 is Aj4,j then p0 p11
p22 p33 p44
1
24
3
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
rule 1 2 3 4 p0 p1 p2 p3 p41 outwar
dssmall
3.000
0.000
0.000
-0.045
-0.004
2 forward
small
3.000
0.000
0.000
-0.030
-0.090
3 small small outwards
3.000
-0.041
0.004
0.000
0.000
4 small small forward
0.303
-0.026
0.061
-0.050
0.000
5 small small inwards
0.000
-0.025
0.070
-0.075
0.000
6 small big outwards
3.000
-0.066
0.000
-0.034
0.000
7 small big forward
2.990
-0.017
0.000
-0.021
0.000
8 small big inwards
1.500
0.025
0.000
-0.050
0.000
9 medium
small outwards
3.000
-0.017
0.005
-0.036
0.000
10 medium
small forward
0.053
-0.038
0.080
-0.034
0.000
rule
1 2 3 4 p0 p1 p2 p3 p4
11 medium
small
inwards
-1.220
-0.016
0.047
-0.018
0.000
12 medium
big outwards
3.000
-0.027
0.000
-0.044
0.000
13 medium
big forward
7.000
-0.049
0.000
-0.041
0.000
14 medium
big inwards
4.000
-0.025
0.000
-0.100
0.000
15 big small
outwards
0.370
0.000
0.000
-0.007
0.000
16 big small
forward
-0.900
0.000
0.034
-0.030
0.000
17 big small
inwards
-1.500
0.000
0.005
-0.100
0.000
18 big big outwards
1.000
0.000
0.000
-0.013
0.000
19 big big forward
0.000
0.000
0.000
-0.006
0.000
20 big big inwards
0.000
0.000
0.000
-0.010
0.000
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Input1 10 cm, 2 30 cm, 3 0 degree (straight forward) and 4 50 cm
Only rule 4 and 7 fire4 0.25 and 4 0.3030.026100.061300.05000.00050 1.873
7 0.167 and 7 2.9900.017100.000300.02100.00050 2.820
Output of the system will be
Input1 10 cm, 2 30 cm, 3 0 degree (straight forward) and 4 50 cm
Only rule 4 and 7 fire4 0.25 and 4 0.3030.026100.061300.05000.00050 1.873
7 0.167 and 7 2.9900.017100.000300.02100.00050 2.820
Output of the system will be
0.25 1.873 0.167 2.8202.252
0.25 0.167
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Fuzzy measure & fuzzy integralFuzzy measure & fuzzy integral
• Fuzzy measure Address the ambiguity axis of uncertainty how likely can the answer to a question be found in various subsets of the sources of informationX = x1, x2, … xn, g: 2X [0,1]Properties1. g() = 02. g(A) g(B) if AB
gi=gxi fuzzy density function importance of the single information source xi in determine the answer to a particular question
• Fuzzy measure Address the ambiguity axis of uncertainty how likely can the answer to a question be found in various subsets of the sources of informationX = x1, x2, … xn, g: 2X [0,1]Properties1. g() = 02. g(A) g(B) if AB
gi=gxi fuzzy density function importance of the single information source xi in determine the answer to a particular question
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Sugeno Lambda measuresatisfy 2 properties with additional one3. For all A,B X with AB =
From
• Sugeno Lambda measuresatisfy 2 properties with additional one3. For all A,B X with AB =
From 1 and 1
XgxXn
ii
n
i
ig1
11
nnn
j
n
k
n
i
ikjn
j
n
jk
kjn
j
j ggggggggg ......1 212
1
1
1 1
31
1 1
2
1
1 BgAgBgAgBAg
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• ExampleX=x1, x2, x3 fuzzy densities g1=0.2, g2=0.3, g3=0.1
find using
Hence =3.2g(x1, x2) = 0.2+0.3+3.2(0.2)(0.3) =
0.69g(x1, x3) = 0.2+0.1+3.2(0.2)(0.1) =
0.36g(x2, x3) = 0.3+0.1+3.2(0.3)(0.1) =
0.5g(x1, x2 ,x3) = 0.69+0.1+3.2(0.69)(0.1)
= 1.011
• ExampleX=x1, x2, x3 fuzzy densities g1=0.2, g2=0.3, g3=0.1
find using
Hence =3.2g(x1, x2) = 0.2+0.3+3.2(0.2)(0.3) =
0.69g(x1, x3) = 0.2+0.1+3.2(0.2)(0.1) =
0.36g(x2, x3) = 0.3+0.1+3.2(0.3)(0.1) =
0.5g(x1, x2 ,x3) = 0.69+0.1+3.2(0.69)(0.1)
= 1.011
Subset g 0 from
propertyx1 0.2
x2 0.3
x3 0.1
x1, x2 0.69
x1, x3 0.36
x2, x3 0.5
x1, x2, x3 1 from property
1.013.012.011 20.006 0.11 0.4 0
20.11 0.11 4(0.006)( 0.4)3.2, 21.44
2(0.006)
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Fuzzy integral– Sugeno fuzzy integralContiuous case:
Finite case: X=x1, x2,…, xn reorder
Hence, sugeno fuzzy integral
• Fuzzy integral– Sugeno fuzzy integralContiuous case:
Finite case: X=x1, x2,…, xn reorder
Hence, sugeno fuzzy integral
0,1
supmin , where h x g g A A xh x
1 2 1 2, ,... so that ...n nX x x x h x h x h x
1 21maxmin , where , ,...
n
g i i i iiS h h x g A A x x x
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Example
X=x1, x2, x3 h(x1) = 0.7, h(x2) = 0.9, h(x3) = 0.2
and assume that g are the same as previous example
Reorder h(x2) > h(x1) > h(x3)
Sg=(0.9g (x2 )) (0.7 g ( x1, x2 )) (0.2 g ( x1, x2, x3 )) =0.69
• Example
X=x1, x2, x3 h(x1) = 0.7, h(x2) = 0.9, h(x3) = 0.2
and assume that g are the same as previous example
Reorder h(x2) > h(x1) > h(x3)
Sg=(0.9g (x2 )) (0.7 g ( x1, x2 )) (0.2 g ( x1, x2, x3 )) =0.69
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– Choquet fuzzy integralContiuous case:
Finite case: X=x1, x2,…, xn reorder
Hence, choquet fuzzy integral
– Choquet fuzzy integralContiuous case:
Finite case: X=x1, x2,…, xn reorder
Hence, choquet fuzzy integral
1 2 1 2, ,... so that ...n nX x x x h x h x h x
1 1 2 11
where , ,... and 0n
g i i i i i ni
C h h x h x g A A x x x h x
1
0
where X
h x g g A d A x h x
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Let i(g) = g(Ai) - g(Ai-1)
Hence
Let i(g) = g(Ai) - g(Ai-1)
Hence
11
1 2 1 2 3 2 1
1 1 2 2 1 1 1
...
...
n
g i i ii
n n n
n n n n n
C h h x h x g A
h x h x g A h x h x g A h x h x g A
h x g A h x g A g A h x g A g A h x g A
11
where n
g i ii i ii
C h g h x g g A g A
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
We can have linear order statistic
where w1, w2,…, wn satisfy wi [0,1] and
• ExampleX=x1, x2, x3 h(x1) = 0.7, h(x2) = 0.9, h(x3) = 0.2 and assume that g are the same as previous example
Reorder h(x2) > h(x1) > h(x3)
Cg=(0.9-0.7)(0.3)+(0.7-0.2)(0.69+(0.2-0.1)(1) = 0.605
We can have linear order statistic
where w1, w2,…, wn satisfy wi [0,1] and
• ExampleX=x1, x2, x3 h(x1) = 0.7, h(x2) = 0.9, h(x3) = 0.2 and assume that g are the same as previous example
Reorder h(x2) > h(x1) > h(x3)
Cg=(0.9-0.7)(0.3)+(0.7-0.2)(0.69+(0.2-0.1)(1) = 0.605
11
n
kkw
k
k k
w k kx w
LOS h x w h x
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Optimization trainingLet training set T=(oj, j)|j=1,2,…,m where oj=jth object and j= desired output of jth object, and there are m training samplesSuppose there are n sensors/algorithmsOutput from choquet integral of jth object will be
Want to find fuzzy measure
• Optimization trainingLet training set T=(oj, j)|j=1,2,…,m where oj=jth object and j= desired output of jth object, and there are m training samplesSuppose there are n sensors/algorithmsOutput from choquet integral of jth object will be
Want to find fuzzy measure
;;)(1
1
n
iiijijjg AgxohxohohC No h(x1) h(x2) … h(xn)
12…m
Training set with msamples each has desiredoutput (). There are nsensors/algorithms
X
n
g
ggg
gg
g
...
...
13
12
2
1
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
such that Cg(h(oj)) is close to j
Let
We have
Then
such that Cg(h(oj)) is close to j
Let
We have
Then
else 0
if ;; 1,
iAAxohxohv iijij
Aj
0
0
;;
;;0
0
1
21
njnj
jj
j
xohxoh
xohxoh
v
;)( njT
jjg xohgvohC
m
jjjg ohCE
1
22
2 2 with only 1 possible nonzeronjv R n
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Objective function
Hence
Objective function
Hence
m
jjjg ohCE
1
22
21
m
jjnj
m
j
Tjjnj
m
j
Tjj
T
m
jjnj
Tjjnj
Tjj
T
m
jjnj
Tj
T
jnjT
j
xohgvxohgvvg
xohgvxohgvvg
xohgvxohgv
1
2
11
1
2
1
;21 ;
21
; ;221
; ;21
22
1 1
1; and ; 2
m mT
j j j j jn nj j
v h o x v h o x
2 2
1
1 where ,2
mT T T
j jj
E g g v g v v
D D
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Similar toSimilar to1minimize 2
subject to and 0 1
T T
gg g v g
g b g
D
A
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Fromg1 g120g1 g130…g1 g12…n0… andg123…n-1 g123…n0…g23…n g123…n0
Fromg1 g120g1 g130…g1 g12…n0… andg123…n-1 g123…n0…g23…n g123…n0
1 0 0 0 0 0 0
0 0 1 0 0 0 0
0 0 1 0 0 0 10 0 0 1 0 0 1
A
1
10
0
b
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Example
Let find such
that E2 is minimized
• Example
Let find such
that E2 is minimized
No h(x1) h(x2) h(x3)
1 0.68 0.53 0.81 0.743
2 0.74 0.99 0.86 0.926
3 0.45 0.07 0.08 0.301
23
13
12
3
2
1
gggggg
g
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
O1: Cg(O1) = (0.81 – 0.68) g3+ (0.68 – 0.53) g13+ (0.53) = 0.13g3+ 0.15 g13+ 0.53
O2: Cg(O2) = (0.99 – 0.86) g2+ (0.86 – 0.74) g23+ (0.74) = 0.13g2+ 0.12 g23+ 0.74
O3: Cg(O3) = (0.45 – 0.08) g1+ (0.08 – 0.07) g13+ (0.07) = 0.37g3+ 0.01 g13+ 0.07
O1: Cg(O1) = (0.81 – 0.68) g3+ (0.68 – 0.53) g13+ (0.53) = 0.13g3+ 0.15 g13+ 0.53
O2: Cg(O2) = (0.99 – 0.86) g2+ (0.86 – 0.74) g23+ (0.74) = 0.13g2+ 0.12 g23+ 0.74
O3: Cg(O3) = (0.45 – 0.08) g1+ (0.08 – 0.07) g13+ (0.07) = 0.37g3+ 0.01 g13+ 0.07
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Hence
And
Hence
And
001.0
000
37.0
,
12.0000
13.00
,
015.0
013.0
00
321 vvv
3
1
0.0860.0240.028
; 0
0.0340.022
Tj j jn
j
v h o x v
3
1
0.14 0 0 0 0.004 00 0.017 0 0 0 0.0160 0 0.017 0 0.02 00 0 0 0 0 0
0.004 0 0.02 0 0.023 00 0.016 0 0 0 0.014
Tj j
j
v v
D
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
100000010000001000100100
010100100010
001010010001001001
A
111000000
b
Now we can compute1minimize 2
subject to and 0 1
T T
gg g v g
g b g
D
A
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Introduction to Evolutionary ComputationIntroduction to Evolutionary Computation
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Evolutionary computingEvolutionary computing
Without mutationpopulation tends to converge to a homogeneous state where individuals vary only slightly from each other
Without mutationpopulation tends to converge to a homogeneous state where individuals vary only slightly from each other
Initialize the population
Create offspring through random
variation
Evaluate the
“fitness” of each candidate solution
Apply selection Terminate ?
Yes
No
Component1. Chromosome encoding2. Fitness value or survival strength of
individual3. Initial population4. Selection operator5. Reproduction operator
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Introduction to Genetic AlgorithmIntroduction to Genetic Algorithm
• Genetic AlgorithmAlgo
1. Set t=02. Initial population P(t)3. Calculate each individual fitness value4. While not converge do
1. Select individual to P1 (intermediate population)Mating Pool (MP)2. Select from MP to mate P2 mutate chromosome in P2 P3
3. Select chromosome in P3 and P(t) for replacement P(t+1)4. Set t= t+1
5. End while
If (P3)=N then P3 become P(t+1) else if (P3)<N select q missing chromosome from P(t) or P1
• Genetic AlgorithmAlgo
1. Set t=02. Initial population P(t)3. Calculate each individual fitness value4. While not converge do
1. Select individual to P1 (intermediate population)Mating Pool (MP)2. Select from MP to mate P2 mutate chromosome in P2 P3
3. Select chromosome in P3 and P(t) for replacement P(t+1)4. Set t= t+1
5. End while
If (P3)=N then P3 become P(t+1) else if (P3)<N select q missing chromosome from P(t) or P1
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• ExampleChromosome A, B, C, D each with 8 genesFitness function: number of 1 in the string
• ExampleChromosome A, B, C, D each with 8 genesFitness function: number of 1 in the string
chromosome label
chromosome string
fitness value
A 00000110 2B 11101110 6C 00100000 1D 00110100 3
Average fitness value 12/4
CrossoverB and Dat crossing site: 1get E and FAnd B and C arecopied
chromosome label
chromosome string
fitness value
B 11101110 6C 00100000 1E 10110100 4F 01101110 5
B is mutated at 1st bitE is mutated at 6th bit
chromosome label
chromosome string
fitness value
B 01101110 5
C 00100000 1E 10110000 3
F 01101110 5Average fitness value 14/4
P(t)
P(t+1)
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Initial population– random gene values of each gene in each chromosomeuniform
representation of the entire search space– Small population small part of search space, time complexity is low,
need more generations to converge EA force to explore a larger search space by increasing the rate of mutation
– Large population large area of search space, less generation to converge, time complexity is increased
• Selection operation– Selection techniques exist
• Explicit fitness remappingfitness values is mapped into a new range, e.g., normalization to [0,1]
• Implicit fitness remappinguse actual fitness values for selection several selection operators
• Initial population– random gene values of each gene in each chromosomeuniform
representation of the entire search space– Small population small part of search space, time complexity is low,
need more generations to converge EA force to explore a larger search space by increasing the rate of mutation
– Large population large area of search space, less generation to converge, time complexity is increased
• Selection operation– Selection techniques exist
• Explicit fitness remappingfitness values is mapped into a new range, e.g., normalization to [0,1]
• Implicit fitness remappinguse actual fitness values for selection several selection operators
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– Random selectionrandom no reference to fitness each individual has an equal
chance of being selected– Proportional selection
total fitness
where f(xi)fitness
value of chromosome xiselection probability of xi
Expected number of copied of xi
Or
– Random selectionrandom no reference to fitness each individual has an equal
chance of being selected– Proportional selection
total fitness
where f(xi)fitness
value of chromosome xiselection probability of xi
Expected number of copied of xi
Or
1 2, ,..., NP t x x x
1
Ni
i
F f x
for 1,2,...,
i
i
f xp i N
F
for 1,2,...,
i i
i
f x f xn N i N
F f
for 1,2,...,i in Np i N
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– AlgorithmFor i=1 to N
Calculate
End forFor i=1 to N
random 0,1uniform distributionIf 0q1, chromosome x1 is selectedIf qi1qi for i1,2,…N then chromosome xi is selected
End for
– AlgorithmFor i=1 to N
Calculate
End forFor i=1 to N
random 0,1uniform distributionIf 0q1, chromosome x1 is selectedIf qi1qi for i1,2,…N then chromosome xi is selected
End for
1
for 1,2,...,i
i kk
q p i N
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Example with x is varied between 0-31,
assume random 4 times with the value of 0.8, 0.5, 0.1 and 0.6
• Example with x is varied between 0-31,
assume random 4 times with the value of 0.8, 0.5, 0.1 and 0.6
No Chromosome
x (in real number)
f(x) pi qi Ni Number of
copies1 01101 13 169 0.14 0.14 0.58 12 11000 24 576 0.49 0.63 1.97 23 01000 8 64 0.06 0.69 0.22 04 10011 19 361 0.31 1.00 1.23 1
Total 1170 1.00
Average 293 0.25
maximum 576 0.49
2max maxx x
f x x
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Crossover x1and x2 at crossover site 4x3and x4 at crossover site 2
Mutation probability is 0.001 there are 20 bits expected number of bit undergoes mutation is 20(0.001) = 0.02 bits suppose no mutationHence, New population will be
Crossover x1and x2 at crossover site 4x3and x4 at crossover site 2
Mutation probability is 0.001 there are 20 bits expected number of bit undergoes mutation is 20(0.001) = 0.02 bits suppose no mutationHence, New population will be
No Chromosome (x)
mate Crossoversite
New population
X (in real number)
f(x)
1 01101 2 4 01100 12 1442 11000 1 4 11001 25 6253 11000 4 2 11011 27 7294 10011 3 2 10000 16 256 Total 1754 Average 439
maximum 729
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Disadvantage– Premature convergence– Slow convergence
• Disadvantage– Premature convergence– Slow convergence
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Introduction to Genetic AlgorithmIntroduction to Genetic Algorithm
• Selection Based on Scaling and Ranking Mechanisms– Static Scaling Mechanisms
• linear scaling
• Power law scaling
• Selection Based on Scaling and Ranking Mechanisms– Static Scaling Mechanisms
• linear scaling
• Power law scaling
S x ax b
tf x S f x
Evaluation function: eval(x) tf(x) S(f(x)) af(x) b
ueval x tf x S f x f x
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Logarithmic scaling
T is controlling the selective pressure >decrease during search process
Selection pressuredegree to which highly fit individuals are allowed to produce offspring (copies) in the next generationLow values of selection pressure provide each individual in the population with a reasonable selection prob.High values of selection pressurestrongly favor the best individuals in the population population diversity decrease quickly
• Logarithmic scaling
T is controlling the selective pressure >decrease during search process
Selection pressuredegree to which highly fit individuals are allowed to produce offspring (copies) in the next generationLow values of selection pressure provide each individual in the population with a reasonable selection prob.High values of selection pressurestrongly favor the best individuals in the population population diversity decrease quickly
f x
Ttf x S f x e
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Dynamic Scaling mechanism– Sigma truncation
OR
• Dynamic Scaling mechanism– Sigma truncation
OR
if 0
else
x xT x
o
t tS x T x m c
t teval x tf x S f x T f x m c
if
0 if
t t t t
t t
f x m c f x m ceval x
f x m c
1+ if 0
0 if 0
tt
t
t
f x m
eval x
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– Window scaling
OR
where W is the set of all individuals contained in the last ggeneration, g1
– Window scaling
OR
where W is the set of all individuals contained in the last ggeneration, g1
miny P t
eval x f x f y
miny W
eval x f x f y
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Fitness remapping for minimization problem
where M is the upperbound of the objective function
where x(t)P(t) and Mt is max value of function f found so far or max value of f found within generation P(t)
• Fitness remapping for minimization problem
where M is the upperbound of the objective function
where x(t)P(t) and Mt is max value of function f found so far or max value of f found within generation P(t)
tf x M f x
teval x t tf x t M f x t
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
OR
where K is positive constant and fmin(t) is the minimum observed value of function f up to time t
OR
OR
where K is positive constant and fmin(t) is the minimum observed value of function f up to time t
OR
min
1eval x t
K f x t f t
1eval x
K f x
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Rank-based Selectionlet number of individuals in each generation N (P(t)N)– Linear Ranking selection
selection probability of xi
where ri rank of individual xi (ranked in increasing order)expected value of offspring of xi
And Then
• Rank-based Selectionlet number of individuals in each generation N (P(t)N)– Linear Ranking selection
selection probability of xi
where ri rank of individual xi (ranked in increasing order)expected value of offspring of xi
And Then
i in Np
1 1
1i
ir
p Min Max MinN N
1 1
N N
i ii i
n Np N
1
N
ii
Np N
1 1 1
1 1 1
1 1
N N Ni i
i i i
r rN Min Max Min Min Max Min N
N N N
1
11
1
N
ii
NMin Max Min r NN
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
But, since
Hence,
Min 2 Max OR Max Min 2
Since Min 0 and Min Max Max 2 and Max 1 1 Max 2
Hence and
Alternatively,
But, since
Hence,
Min 2 Max OR Max Min 2
Since Min 0 and Min Max Max 2 and Max 1 1 Max 2
Hence and
Alternatively,
1
1 1 0
11 1
2
N N N
ii i i
N Nr i i
NMax
pN
1
2 Max Minp
N N
1
12
Min Max Min
1 1
2 1 11
ii
rp Max Max
N N
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• nonlinear ranking selectionSelection probability will be
• SUS Algorithm
guarantee that xi will reproduce at least but not more than
• nonlinear ranking selectionSelection probability will be
• SUS Algorithm
guarantee that xi will reproduce at least but not more than
111 ir
ip ec
1. ptr Rand(); /*random within [0,1] (uniformdistribution)*/
2. For (sum=i=0; i<N;i++)1. For (sum +=ni; sum>ptr; ptr++)2. Select(i);3. End For
3. End For
in in
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Example
Suppose ptr is 0.8 the selection will be x1, x3 and x3 , respectively
• Example
Suppose ptr is 0.8 the selection will be x1, x3 and x3 , respectively
Chomosome f(x)x2 r1 pi ni
x110 100 1 3(0.27)0.81
x215 225 2 3(0.33)0.99
x320 400 3 0.4 3(0.4)=1.2
1 2 1
0.8 1.2 0.8 0.333 3 1
2 1.20.27
3
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Tournament selectionSimilar to rank selection in terms of selection pressure but more efficient – Binary tournament (deterministic)
• 2 individuals directly compete for selection• With or without reinsertion of the competing individual into the original
population• based on direct pairwise comparison of individuals
– 2 chromosomes are chosen at random from the population– The fitness of the chosen chromosomes is calculated– The most successful chromosome (the winner) is selected to intermediate
population
– Probability tournament– Step 3 random R 0,1 if R<p then the fitter of the two is selected;
otherwise the less fit is selectedwhere p is preselected or p varied with time
or where p00,1 and c is positive constant
• Tournament selectionSimilar to rank selection in terms of selection pressure but more efficient – Binary tournament (deterministic)
• 2 individuals directly compete for selection• With or without reinsertion of the competing individual into the original
population• based on direct pairwise comparison of individuals
– 2 chromosomes are chosen at random from the population– The fitness of the chosen chromosomes is calculated– The most successful chromosome (the winner) is selected to intermediate
population
– Probability tournament– Step 3 random R 0,1 if R<p then the fitter of the two is selected;
otherwise the less fit is selectedwhere p is preselected or p varied with time
or where p00,1 and c is positive constant
0ctp t p e
01 ln
cp t p
t
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– Boltzmann tournamentx current solution, y alternative solution binary tournament
between x and y with
Where T is the temperature reduced in a predefined manner in successive iterationThe winner of the trial will be the current solution in the next trial
• Elitism selection of a set of individual from the current generation to survive to the next generation
The fraction of new individual at each generation has been called generation gap
– Boltzmann tournamentx current solution, y alternative solution binary tournament
between x and y with
Where T is the temperature reduced in a predefined manner in successive iterationThe winner of the trial will be the current solution in the next trial
• Elitism selection of a set of individual from the current generation to survive to the next generation
The fraction of new individual at each generation has been called generation gap
1
1
f x f y
T
p x
e
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Recombination operatorcross over: C: XX XX Cx,yx,y
– One-point crossoverx x1 x2 x3… xL and y y1 y2 y3… yL after cross over x x1 x2 x3… xk1 xk yk1… yL and y y1 y2 y3… yk1 yk xk1… xL
• Recombination operatorcross over: C: XX XX Cx,yx,y
– One-point crossoverx x1 x2 x3… xL and y y1 y2 y3… yL after cross over x x1 x2 x3… xk1 xk yk1… yL and y y1 y2 y3… yk1 yk xk1… xL
Crossover site
1st parent
2nd parent
Crossover site
1st child
2nd child
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– Two-point cross over
– N-point cross over
– Two-point cross over
– N-point cross over
1st Crossover
site
2nd Crossover site
chromosome x
chromosome
y
1st Crossover
site
2nd Crossover site
chromosome
x
chromosome
y
2nd Crossover site
chromosome
x
chromosome
y
3rdCrossover site
1st Crossover site
1st
Crossover site
2nd
Crossover site
Chromosome x
chromosome
y
3rd Crossover site
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– N-point cross over algorithmP1: for each chromosome from intermediate population (P1)
P1.1: random q 0,1 uniform distributionP1.2: if q < pc (where pc is mating probability) then respective
chromosome is considered mating go to mating pool else not matingP2: let m*number of chromosome selected at P1
if m* is even then select pair chromosome (random)if m* is odd then discard or select chromosome from P1
P3: Cross over is applied on pairs in P2P3.1: for each pair, generate k (kL)
and random crossing site k1, k2,…,kN for (N2) P3.2: descendants obtained at P3.1 become potential members of P(t+1) candidate for mutation
P3.3: parents of newly generated chromosome are discarded from P1
P3.4: chromosomes left in P1 are added to P(t+1)
– N-point cross over algorithmP1: for each chromosome from intermediate population (P1)
P1.1: random q 0,1 uniform distributionP1.2: if q < pc (where pc is mating probability) then respective
chromosome is considered mating go to mating pool else not matingP2: let m*number of chromosome selected at P1
if m* is even then select pair chromosome (random)if m* is odd then discard or select chromosome from P1
P3: Cross over is applied on pairs in P2P3.1: for each pair, generate k (kL)
and random crossing site k1, k2,…,kN for (N2) P3.2: descendants obtained at P3.1 become potential members of P(t+1) candidate for mutation
P3.3: parents of newly generated chromosome are discarded from P1
P3.4: chromosomes left in P1 are added to P(t+1)
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– Punctuated crossover• Record the crossover points in the chromosome• If a certain crossover point has generated a very poor offspring this
crossover point will not be considered further• If the fitness of the descendant is good, the crossover point is maintained as
active• Cross-over point self adjust according to the previous dynamics of the
search processC x1 x1… xLk1 k1... kL when ki 1 active crossover point, ki
0 not a crossover point
During initialization probability of generating ki 1 is smallSet of crossover point used in the recombination of 2 chromosomes is the union of the parents
– Punctuated crossover• Record the crossover points in the chromosome• If a certain crossover point has generated a very poor offspring this
crossover point will not be considered further• If the fitness of the descendant is good, the crossover point is maintained as
active• Cross-over point self adjust according to the previous dynamics of the
search processC x1 x1… xLk1 k1... kL when ki 1 active crossover point, ki
0 not a crossover point
During initialization probability of generating ki 1 is smallSet of crossover point used in the recombination of 2 chromosomes is the union of the parents
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– Uniform crossover– Uniform crossover
1. Set mi 0 for i 1,2,…,L2. For each i
1. random U0,12. If pat_i then mi 1 where pat_i
is the crossover probability at ith
position x
y
1 0 1 0 0 1 0 0 0 1
m
x
y
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• MutationAverage number of position that will undergo a mutation B NLpmwhere pmmutation probability (pm 0.001,0.01 ), N number of chromosomes, Llength of chromosome
Algorithm
Weak mutationstep 2 changed to if q <pm then 1 of 0 or 1 is chosen at random
• MutationAverage number of position that will undergo a mutation B NLpmwhere pmmutation probability (pm 0.001,0.01 ), N number of chromosomes, Llength of chromosome
Algorithm
Weak mutationstep 2 changed to if q <pm then 1 of 0 or 1 is chosen at random
For each chromosome และ For each position of thechromosome
1. Generate a random number q from U(0,1)2. If q <pm invert that position
If qpm do not mutate
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– Non-uniform mutation
where 1
OR
where ,,Real-valued positive parameter
– Non-uniform mutation
where 1
OR
where ,,Real-valued positive parameter
tm mp t p e
12
21
2
t
me
p t
L N
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
OR
where 0tT pm(0) 0.5 and
OR
OR
where 0tT pm(0) 0.5 and
OR
12
2mp t
Lt
T 1
mp TL
1
2 1mp x
f x L
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Inversion operatorActs on a single chromosome. It inverts the values between 2 position of a chromosome. The 2 positions are chosen at random
• Inversion operatorActs on a single chromosome. It inverts the values between 2 position of a chromosome. The 2 positions are chosen at random
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Simple genetic algorithm• Simple genetic algorithm
1. Set t 0Initialize (P(0)) random a selection mechanism is chosen and if necessary a scaling mechanism
2. The chromosome of P(t) are evaluated using the fitness function. The most successfulindividual from P(t)is kept.3. Selection operator is applied N times
Selected chromosome make up an intermediate population P1 (having N members as well)representing candidate from the mating pool (MP). Some chromosomes from P(t) mayhave more copies in P1 while some others may have none
4. Crossover operator is applied to the chromosomes from MPNewly generated chromosomes forท P2
Parents of the chromosomes obtained through crossover are discarded from P1
Chromosomes that remained in P1 become member of P2
5. Mutation operator is applied to P2 outcome is the new generation(P(t1))6. set t t1
if t < T (T is the maximum number of generation) then go to step 2 otherwise stop
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Solution best member of the last population or best individual in all generation
• Can involve inversion or other types of operators
• Solution best member of the last population or best individual in all generation
• Can involve inversion or other types of operators
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Examplewhere
• Examplewhere
1 2 2 21 2
1,
1f x x
x x
chromosome
x1 x2 Fitnessvalue
C0 1 2 0.167C1 2 3 0.007C2 1.5 0.0 0.31C3 0.5 1.0 0.44
C3, C3, C2 and C0 areselected to P1
C3, C2 and C3, C0
chromosome
x1 x2 Fitnessvalue
C0 0.6 0 0.8C1 1.6 1.0 0.24C2 0.6 2.0 0.19C3 1.0 1.0 0.33
1 2
1 2,
max ,x x
f x x
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Introduction to Genetic ProgrammingIntroduction to Genetic Programming
• Each individual/chromosome 1 computer program represented using a tree structure
• Terminal set specifies all variables and constant elements of the terminal set form the leaf modes
• Function set all functions that can be applied to the elements of the terminal set may be mathematical arithmetic and/or Boolean function or if-then-elseform the non-leaf nodes.
• Each individual/chromosome 1 computer program represented using a tree structure
• Terminal set specifies all variables and constant elements of the terminal set form the leaf modes
• Function set all functions that can be applied to the elements of the terminal set may be mathematical arithmetic and/or Boolean function or if-then-elseform the non-leaf nodes.
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Examplefunction AND, OR, NOT and terminal set x1, x2 where
x1,x20,1
• Examplefunction AND, OR, NOT and terminal set x1, x2 where
x1,x20,1
x1
OR
AND AND
NOT
x2
NOT
x1
x2
XOR x1 AND NOT x2 OR NOT x1 AND x2
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• ExampleFunction ,,, and terminal set a,x,z,3.4 where a,x,z • ExampleFunction ,,, and terminal set a,x,z,3.4 where a,x,z
x
+
ln
a
3.4
y
sin exp
z
x
ln( ) sin( ) / exp( ) 3.4y x a z x
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
exp
a
3.4 x
sin
3.4 a
exp
a
3.4 x
z sin
3.4
+
Crossover 2 parents generating 1 child
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Crossover 2 parents generating 2 children
exp
a
3.4 x
sin
3.4 a
+
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Mutationchoose a random point in a tree and replacing the subtree beneath that point by a randomly generated subtree
• Calculate the fitness of each program by running it on a set of “fitness case”
• Mutationchoose a random point in a tree and replacing the subtree beneath that point by a randomly generated subtree
• Calculate the fitness of each program by running it on a set of “fitness case”
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Evolutionary programmingEvolutionary programming• Algorithm• Algorithm
1. Let current generation be t0
2. Initialize all parameters3. Initialize population C(0) with ns individual4. For each individual xi(t) C(t)
calculate fitness value f(xi(t))End
5. While no convergence• For each individual xi(t) C(t)
mutate xi(t) to produce offspring xi(t)
calculate fitness value (f(xi(t)))
add xi(t) in offspring set C(t)
End• Select new population C(t1) from C(t)C(t) using selection operator• t t1End
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
CoevolutionCoevolution
• Predator-pray competitive coevolution– Insects that eat plant
• To survive plant needs to evolve mechanism to defend itself from the insects, and• The insects need the plants as food source to survive.
– Inverse fitness interaction between 2 specieswin for 1 species failure for the other
• Symbiosis– Different species cooperate instead of competesuccess in 1 species improve
the survival strength of the other speciespositive feedback among the species that take part in this cooperating process
• Predator-pray competitive coevolution– Insects that eat plant
• To survive plant needs to evolve mechanism to defend itself from the insects, and• The insects need the plants as food source to survive.
– Inverse fitness interaction between 2 specieswin for 1 species failure for the other
• Symbiosis– Different species cooperate instead of competesuccess in 1 species improve
the survival strength of the other speciespositive feedback among the species that take part in this cooperating process
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Coevolutonary algorithm (CoEA)– Competitive coevolution
• Competition• Amensalism
– Cooperative coevolution• Mutualism• Commensalism• Parasitism
• Competitive coevolution (CCE)– Evolve 2 population simultaneously
• individual in 1 population solution (evolve to solve as many test cases as possible) fitness the number of test cases solved by the solution
• individual in another population test case (evolve to present an incrementally increasing level of difficult to the solution individuals)fitness is inversely proportional to the number of strategies that solve it
• Coevolutonary algorithm (CoEA)– Competitive coevolution
• Competition• Amensalism
– Cooperative coevolution• Mutualism• Commensalism• Parasitism
• Competitive coevolution (CCE)– Evolve 2 population simultaneously
• individual in 1 population solution (evolve to solve as many test cases as possible) fitness the number of test cases solved by the solution
• individual in another population test case (evolve to present an incrementally increasing level of difficult to the solution individuals)fitness is inversely proportional to the number of strategies that solve it
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– Relative fitness functionexpress the performance of individual in one population is comparison with individuals in another population
• Which individual from the competing population is used• Exactly how these competing individuals are used to compute the relative
fitness– Sampling scheme
• All versub all sampling• Random sampling• Tournament sampling• All versus best sampling• Shared sampling
– Relative fitness function f(C1.xi)• Simple function2 population C1 and C2aim to calculate the relative fitness of each individual C1.xi of C1
Sample of individuals is taken from C2 S
sum C1.xi = count the number of individuals in S that C1.xi winf(C1.xi) = sum C1.xi
– Relative fitness functionexpress the performance of individual in one population is comparison with individuals in another population
• Which individual from the competing population is used• Exactly how these competing individuals are used to compute the relative
fitness– Sampling scheme
• All versub all sampling• Random sampling• Tournament sampling• All versus best sampling• Shared sampling
– Relative fitness function f(C1.xi)• Simple function2 population C1 and C2aim to calculate the relative fitness of each individual C1.xi of C1
Sample of individuals is taken from C2 S
sum C1.xi = count the number of individuals in S that C1.xi winf(C1.xi) = sum C1.xi
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Fitness sharingsim C1.xi = similarities of C1.xi with all the other individuals in that population e.g., the number of individuals that also beats individual from C2 samples
• Competitive fitness sharing
Where C2.x1, C2.x2,…, C2.xC2.ns, form C2 and C1.nl = total number of individuals in C1 that defeat individual C2.xl
• Tournament fitness– Binary tournament determine a relative fitness ranking– Tournament fitness results in a tournament tree with root element as the best
individual• Each level 2 opponent are randomly selected from that level and the best of the 2
advances to next level– If odd number a single individual from the level moves to the next level– After tournament use any selection operator
• Fitness sharingsim C1.xi = similarities of C1.xi with all the other individuals in that population e.g., the number of individuals that also beats individual from C2 samples
• Competitive fitness sharing
Where C2.x1, C2.x2,…, C2.xC2.ns, form C2 and C1.nl = total number of individuals in C1 that defeat individual C2.xl
• Tournament fitness– Binary tournament determine a relative fitness ranking– Tournament fitness results in a tournament tree with root element as the best
individual• Each level 2 opponent are randomly selected from that level and the best of the 2
advances to next level– If odd number a single individual from the level moves to the next level– After tournament use any selection operator
11
1
...
ii
i
sumC xf C xsimC x
2 .
11 1
1..
sC n
il l
f C xC n
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Hall of frame– At each generation the best individual of a population is stored in that
population’s hall of frames– May have limited size– New individual inserted in hall of frame will replace the worst or the
oldest individual– Individual from 1 population complete against a sample of the
current opponent population and its hall of frame– Prevents overspecialization
• Hall of frame– At each generation the best individual of a population is stored in that
population’s hall of frames– May have limited size– New individual inserted in hall of frame will replace the worst or the
oldest individual– Individual from 1 population complete against a sample of the
current opponent population and its hall of frame– Prevents overspecialization
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Competitive coevolution with 2 population Algo– Initialize C1 (solution), C2 (test) population– While stopping condition(s) not true do
• For each C1.xi for i=1,…, C1.ns do– Select a sample of opponents from C2
– Evaluate the relative fitness of C1.xi w.r.t. this sample• End• For each C2.xi for i=1,…, C2.ns do
– Select a sample of opponents from C1
– Evaluate the relative fitness of C2.xi w.r.t. this sample• End• Evolve population C1 for 1 generation• Evolve population C2 for 1 generation
– end– Select the best individual from C1 as solution
• Competitive coevolution with 2 population Algo– Initialize C1 (solution), C2 (test) population– While stopping condition(s) not true do
• For each C1.xi for i=1,…, C1.ns do– Select a sample of opponents from C2
– Evaluate the relative fitness of C1.xi w.r.t. this sample• End• For each C2.xi for i=1,…, C2.ns do
– Select a sample of opponents from C1
– Evaluate the relative fitness of C2.xi w.r.t. this sample• End• Evolve population C1 for 1 generation• Evolve population C2 for 1 generation
– end– Select the best individual from C1 as solution
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Competitive coevolution with 1 population Algo– Initialize 1 population C– While stopping condition(s) not true do
• For each C.xi for i=1,…, C.ns do– Select a sample of opponents from C (exclude C.xi )– Evaluate the relative fitness of C.xi w.r.t. the sample
• End• Evolve population C for 1 generation
– end– Select the best individual from C as solution
• Competitive coevolution with 1 population Algo– Initialize 1 population C– While stopping condition(s) not true do
• For each C.xi for i=1,…, C.ns do– Select a sample of opponents from C (exclude C.xi )– Evaluate the relative fitness of C.xi w.r.t. the sample
• End• Evolve population C for 1 generation
– end– Select the best individual from C as solution
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Cooperative coevolution– Similar to divide and conquer– Only mutualism here– Individual from different species or subpopulation have to cooperate in
some way to solve a global task– Fitness depends on that individual’s ability to collaborate with
individual from other species– Ex. For an nx dimensional problemnx subpopulation ( 1 for each)
• each subpopulation responsible for optimizing one of the parameters of problem and no subpopulation can form a complete solution by itself
• Merge representative from each subpopulation• Consider j subpopulation (Cj) each Cj.xi perform a single collaboration
with the best individual from each other subpopulation complete solution
• Credit assign to Cj.xi fitness of the complete solution• Evolve independently from one another separate population is used to
evolve each subcomponent using some EA
• Cooperative coevolution– Similar to divide and conquer– Only mutualism here– Individual from different species or subpopulation have to cooperate in
some way to solve a global task– Fitness depends on that individual’s ability to collaborate with
individual from other species– Ex. For an nx dimensional problemnx subpopulation ( 1 for each)
• each subpopulation responsible for optimizing one of the parameters of problem and no subpopulation can form a complete solution by itself
• Merge representative from each subpopulation• Consider j subpopulation (Cj) each Cj.xi perform a single collaboration
with the best individual from each other subpopulation complete solution
• Credit assign to Cj.xi fitness of the complete solution• Evolve independently from one another separate population is used to
evolve each subcomponent using some EA
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Introduction to Swarm IntelligenceIntroduction to Swarm Intelligence
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Swarm Intelligence (SI)Swarm Intelligence (SI)
• Structured collection of interacting organisms (or agents)• Individual organism
– Simple in structure but their collective behavior can become complex• Tight coupling between individual behavior and the behabior of the entire
swarm• Social interactiondirect (through visual audio, chemical context), or
indirect (occur when one changes the environment and the other individuals respond to the new environment)
• 2 concepts– Self-organization
• Positive feedback
• Structured collection of interacting organisms (or agents)• Individual organism
– Simple in structure but their collective behavior can become complex• Tight coupling between individual behavior and the behabior of the entire
swarm• Social interactiondirect (through visual audio, chemical context), or
indirect (occur when one changes the environment and the other individuals respond to the new environment)
• 2 concepts– Self-organization
• Positive feedback
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
Swarm IntelligenceSwarm Intelligence
• 2 concepts– Self-organization
• Positive feedback• Negative feedback• Fluctuation• Multiple interaction
– Division of labor• SIno one is in charge, no one is giving orders, which the swarm
individuals should obey or follow• Social networkstructureof swarmform an integral part of the existence
of that swarmprovide the communication channel, able to self-organize to form the optimal nest structures
• 2 concepts– Self-organization
• Positive feedback• Negative feedback• Fluctuation• Multiple interaction
– Division of labor• SIno one is in charge, no one is giving orders, which the swarm
individuals should obey or follow• Social networkstructureof swarmform an integral part of the existence
of that swarmprovide the communication channel, able to self-organize to form the optimal nest structures
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Partical swarm optimization (PSO)– Simulation of social behavior of birds within flock– Particles are “flown” through hyperdimensional search space– Neighborhood topology
• Star topologyfirst version PSO gbest
• Partical swarm optimization (PSO)– Simulation of social behavior of birds within flock– Particles are “flown” through hyperdimensional search space– Neighborhood topology
• Star topologyfirst version PSO gbest
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Ring topology
n=2
• Wheel topology
• Ring topology
n=2
• Wheel topologyFocal particle
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Individual best (pbest)– Initialize the swarm P(t) at t0 of particles such that the position (xi(t))
of ith particle (PiP(t)) is random within the hyperspace– Evaluate the performance F of each particle using xi(t)– Compare the performance of each individual to its best performance
• If F(xi(t)) < pbestipbesti F(xi(t))
• End– Change the velocity vector of each particle
where is positive random number
– Move each particle to a new position
– Set tt1– Repeat until converge (go to step 2)
• Individual best (pbest)– Initialize the swarm P(t) at t0 of particles such that the position (xi(t))
of ith particle (PiP(t)) is random within the hyperspace– Evaluate the performance F of each particle using xi(t)– Compare the performance of each individual to its best performance
• If F(xi(t)) < pbestipbesti F(xi(t))
• End– Change the velocity vector of each particle
where is positive random number
– Move each particle to a new position
– Set tt1– Repeat until converge (go to step 2)
ipbest it tx x
1ii i pbest it t tv v x x
1i i it t tx x v
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Global best (gbest)– Initialize the swarm P(t) at t0 of particles such that the position (xi(t)) of ith
particle (PiP(t)) is random within the hyperspace– Evaluate the performance F of each particle using xi(t)– Compare the performance of each individual to its best performance
• If F(xi(t)) < pbestipbesti F(xi(t))
• End– Compare the performance of each individual to its best performance
• If F(xi(t)) < gbestgbest F(xi(t))
• Global best (gbest)– Initialize the swarm P(t) at t0 of particles such that the position (xi(t)) of ith
particle (PiP(t)) is random within the hyperspace– Evaluate the performance F of each particle using xi(t)– Compare the performance of each individual to its best performance
• If F(xi(t)) < pbestipbesti F(xi(t))
• End– Compare the performance of each individual to its best performance
• If F(xi(t)) < gbestgbest F(xi(t))
ipbest it tx x
Where 1 and 2 are random number 1 = r1 c1 and 2 = r2 c2 with r1
and r2 U0,1, c1 c2 4
gbest it tx x
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– Change the velocity vector of each particle
– Move each particle to a new position
– Set tt1– Repeat until converge (go to step 2)
– Change the velocity vector of each particle
– Move each particle to a new position
– Set tt1– Repeat until converge (go to step 2)
1i i it t tx x v
1 21ii i pbest i gbest it t t tv v x x x x
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Local best (lbest)– Initialize the swarm P(t) at t0 of particles such that the position (xi(t)) of ith
particle (PiP(t)) is random within the hyperspace– Evaluate the performance F of each particle using xi(t)– Compare the performance of each individual to its best performance
• If F(xi(t)) < pbestipbesti F(xi(t))
• End– Compare the performance of each individual to its best performance
• If F(xi(t)) < lbestlbest F(xi(t))
• Local best (lbest)– Initialize the swarm P(t) at t0 of particles such that the position (xi(t)) of ith
particle (PiP(t)) is random within the hyperspace– Evaluate the performance F of each particle using xi(t)– Compare the performance of each individual to its best performance
• If F(xi(t)) < pbestipbesti F(xi(t))
• End– Compare the performance of each individual to its best performance
• If F(xi(t)) < lbestlbest F(xi(t))
ipbest it tx x
lbest it tx x
Where 1 and 2 are random number 1 = r1 c1 and 2 = r2 c2 with r1
and r2 U0,1, c1 c2 4
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– Change the velocity vector of each particle
– Move each particle to a new position
– Set tt1– Repeat until converge (go to step 2)
– Change the velocity vector of each particle
– Move each particle to a new position
– Set tt1– Repeat until converge (go to step 2)
1i i it t tx x v
1 21ii i pbest i lbest it t t tv v x x x x
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– Fitness calculationcan be objective function same idea as in GA– Convergenceexecuted for a fixed number of iteration or velocity
changes are close to 0 for all particles– PSO system parameter
• Dimension of problem• Number of individual• Upper limit of • Upper limit on the maximum velocity (vmax )
If vij(t) vmax then vij(t) vmax and If vij(t) vmax then vij(t) vmax
where vij(t) is a velocity of ith particle at jth dimension at time tOR if do not want to use vmax
Where
and 12 4.0
– Fitness calculationcan be objective function same idea as in GA– Convergenceexecuted for a fixed number of iteration or velocity
changes are close to 0 for all particles– PSO system parameter
• Dimension of problem• Number of individual• Upper limit of • Upper limit on the maximum velocity (vmax )
If vij(t) vmax then vij(t) vmax and If vij(t) vmax then vij(t) vmax
where vij(t) is a velocity of ith particle at jth dimension at time tOR if do not want to use vmax
Where
and 12 4.0
1 21ii i pbest i gbest it t t tv v x x x x
2 411
2
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Neighborhood size– gbestsimply lbest with the entire swarm as the neighborhood– gbestmore susceptible to local minima
• Inertia weight
Where is inertia weight (1) and
– Modified PSO• Using selection perform selection before velocity updateAlgo
– for each particle in the swarm, score the performance of that particle w.r.t a random selected group of k particles
– Rank the particles according to these performance scores– Select the top half of the particle and copy their current position onto that of the
bottom half of the swarm without changing the personal best values of the bottom half of the swarm
• Neighborhood size– gbestsimply lbest with the entire swarm as the neighborhood– gbestmore susceptible to local minima
• Inertia weight
Where is inertia weight (1) and
– Modified PSO• Using selection perform selection before velocity updateAlgo
– for each particle in the swarm, score the performance of that particle w.r.t a random selected group of k particles
– Rank the particles according to these performance scores– Select the top half of the particle and copy their current position onto that of the
bottom half of the swarm without changing the personal best values of the bottom half of the swarm
1 21ii i pbest i gbest it t t tv v x x x x
1 21
12
c c
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Breeding PSOAlgorithm
– Calculate the particle velocities and new positions– For each particle assign a breeding probability pb
– Select 2 particles assume Pa and Pb and produce 2 offsprings using
Where r1, r2U0,1– Set personal best position of each particle involved in the breeding process to its
current position
• Breeding PSOAlgorithm
– Calculate the particle velocities and new positions– For each particle assign a breeding probability pb
– Select 2 particles assume Pa and Pb and produce 2 offsprings using
Where r1, r2U0,1– Set personal best position of each particle involved in the breeding process to its
current position
1 11 1a a bt r t r tx x x
2 21 1b b at r t r tx x x
1 a b
a aa b
t tt t
t t
v vv v
v v
1 a b
b ba b
t tt t
t t
v vv v
v v
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Neighborhood topology– Use of indices– Use spatial neighbor
Pb is said to be in the neighborhood of Pa if
where dmax is the largest distance between any 2 particles and
• Neighborhood topology– Use of indices– Use spatial neighbor
Pb is said to be in the neighborhood of Pa if
where dmax is the largest distance between any 2 particles and
max
a b
d
x x
max
max
3 0.6t t
t
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
• Ant Colony Optimization– tasks
• Reproduction• Defense• Foor collection• Brood care• Nest brooming• Nest building
– Natural stigmergy• Lack of central coordination• Communication and coordination among individuals in a colony are based
on local modifications of the environment• Positive feedback
• Ant Colony Optimization– tasks
• Reproduction• Defense• Foor collection• Brood care• Nest brooming• Nest building
– Natural stigmergy• Lack of central coordination• Communication and coordination among individuals in a colony are based
on local modifications of the environment• Positive feedback
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– Shortest path problem traveling salesman problem (TSP)– Virtual pheromone– Negative feedbackavoid premature feedback– Time scalenot too large and not too short– Ant System (AS)
• Find a closed tour of minimal length connected n given cities
distance between city i and city j
• Tabu list and set of all cities that ant k is has to visit when it is on city i
– Shortest path problem traveling salesman problem (TSP)– Virtual pheromone– Negative feedbackavoid premature feedback– Time scalenot too large and not too short– Ant System (AS)
• Find a closed tour of minimal length connected n given cities
distance between city i and city j
• Tabu list and set of all cities that ant k is has to visit when it is on city i
2 2
ij i j i jd x x y y
1
2
3
4Graph N,E With 4 cities and they are all connected
kiJ
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– AS algorithm– AS algorithm
For every edge (i,j)ij(0) 0 #initialize pheromone
End ForFor k 1 to m #for each ant k
Place ant k on a randomly chosen cityEnd ForSet T and LFor t 1 to tmax
For k 1to mbuild tour Tkt by applying n1 times the following steps: choose the next city j when the current city
is city i using the transition rule
Where and are adjustable and visibility
End For
if
0 else
ki
ij ij ki
kil ilij
l J
tj J
tp t 1
ijijd
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
For k 1 to mcompute length Lkt of tour Tkt produced by ant k
End ForIf an improved tour is found then
update T and L
End If
For every edge (i,j)
where
End ForEnd For
1 1 eij ij ij ijt t t e t
if ,
0 if ,
eij
Qi j T
Lt
i j T
1
mk
ij ijk
t t
if ,
0 if ,
kkk
ijk
Qi j T t
L tt
i j T t
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
– Ant colony system (ACS)• Different transition rule• Different pheromone trail update rule• Use of local updates of pheromone trail to favor exploration• Use of candidate list cl closest cities ordered by increasing distance ant
first restricts the choice of the next city to those in the list, and consider other cities if all in the list have been visited
– Ant colony system (ACS)• Different transition rule• Different pheromone trail update rule• Use of local updates of pheromone trail to favor exploration• Use of candidate list cl closest cities ordered by increasing distance ant
first restricts the choice of the next city to those in the list, and consider other cities if all in the list have been visited
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
For every edge (i,j)
ij(0) 0 #initialize pheromoneEnd For
For k 1 to m #for each ant kPlace ant k on a randomly chosen city
End ForSet T and L
For t 1 to tmax
For k 1to mbuild tour Tkt by applying n1 times the following steps:if exits at least one city jcandidate list then choose the next cityt j among cl citiy in the candicate
list as
where q U0,1 , 0 q0 1
where J is chosen according to
else choose the closest city j
end ifafter each transition ant k applies the local update rule where
End For
kiJ
0
0
argmax if
if
ki
iu iuu J
t q qj
J q q
kiJ
ki
iJ iJkiJ
il ill J
tp t
t
kiJ
01ij ijt t 10 nnnL
Computer Engineering, Faculty of Engineering, Biomedical Engineering Institute, Chiang Mai University
Sansanee Auephanwiriyakul, Senior Member IEEE
Sansanee Auephanwiriyakul
For k 1 to mcompute length Lkt of tour Tkt produced by ant k
End ForIf an improved tour is found then
update T and L
End IfFor every edge (i,j)
where
End ForEnd For
1ij ij ijt t t
1
ij tL