Pareto Analysis of Evolutionary and Learning Systems

Yaochu Jin, Robin Gruna, and Bernhard Sendhoff

Abstract— This paper attempts to argue that most adaptivesystems, such as evolutionary or learning systems, have in-herently multiple objectives to deal with. Very often, there isno single solution that can optimize all the objectives. In thiscase, the concept of Pareto optimality is key to analyzing thesesystems.

To support this argument, we first present an examplethat considers the robustness and evolvability trade-off in aredundant genetic representation for simulated evolution. It iswell known that robustness is critical for biological evolution,since without a sufficient degree of mutational robustness, it isimpossible for evolution to create new functionalities. On theother hand, the genetic representation should also provide thechance to find new phenotypes, i.e., the ability to innovate.This example shows quantitatively that a trade-off betweenrobustness and innovation does exist in the studied redundantrepresentation.

Interesting results will also be given to show that new insightsinto learning problems can be gained when the concept ofPareto optimality is applied to machine learning. In the firstexample, a Pareto-based multi-objective approach is employedto alleviate catastrophic forgetting in neural network learn-ing. We show that learning new information and memorizinglearned knowledge are two conflicting objectives, and a majorpart of both information can be memorized when the multi-objective learning approach is adopted. In the second example,we demonstrate that a Pareto-based approach can addressneural network regularization more elegantly. By analyzing thePareto-optimal solutions, it is possible to identifying interestingsolutions on the Pareto front.

I. I NTRODUCTION

In nature, species are evolving and living in constantlychanging environments. It seems that evolution has found dif-ferent mechanisms of adaptation [1], among which evolutionand learning are two main ones. Learning, or more generally,individual level adaptation, usually includes all forms ofindividual phenotypic changes during an individuals lifetime.In contrast, evolution, also referred to as population leveladaptation, represents the evolutionary cycle of selection andgenetic variations. In nature, evolution is the main adaptationmechanism for some species such as bacteria, whereas othersrely more on the individual level of adaptation.

Both evolution and learning in nature must achieve distinctgoals for a better adaptation to changing environments, whichis additionally constrained by limited resource of energy.These different goals may be conflicting with each otherand cannot be achieved simultaneously. To achieve thesegoals, nature seems to have evolved systems consisting ofa large number of functionally distinct yet systematicallyintegrated subsystems, which is believed to represent a

Yaochu Jin and Bernhard Sendhoff are with the Honda ResearchInstituteEurope, 63073 Offenbach, Germany. Robin Gruna is with the Institutfur Technische Informatik, Universitat Karlsruhe (TH), 76131 Karlsruhe,Germany. Email: yaochu.jin@honda-ri.de.

major characteristics of brain complexity [2]. In [3], multi-objectivity has been used to characterize the complexity ofan evolved creature.

This paper demonstrates, with examples, how the Pareto-based approach can be employed to analyze computationalevolutionary and learning systems. Section II introduces themathematical definition of multi-objective optimization andPareto dominance, and discusses some general properties ofPareto fronts. In Section III, the trade-off between robustnessand evolvability of a redundant Boolean representation isinvestigated by analyzing the Pareto front achieved witha multi-objective evolutionary algorithm. The advantage ofPareto-based approach to addressing catastrophic forgettingand neural network regularization is illustrated in Section IV.Section V briefly summarizes this paper and suggests somepotentially interesting topics.

II. PARETO-BASED MULTI -OBJECTIVEOPTIMIZATION

AND ANALYSIS

A. Multi-objective Optimization and Pareto Optimality

Consider the following multi-objective minimization prob-lem:

minimize fm(x) m = 1, 2, · · · ,M ; (1)

subject to gj(x) ≥ 0, j = 1, 2, · · · , J ; (2)

hk(x) = 0, k = 1, 2, · · · ,K; (3)

xLi ≤ xi ≤ xU

i , i = 1, 2, · · · , n, (4)

where fm(x) are the M different objective functions tobe minimized,x = (x1, x2, · · · , xn)T is the n-dimensionaldecision space,gj(x) are theJ inequality constraints,hk(x)are theK equality constraints, andxL

i andxUi are the lower

and upper bounds of thei-th decision parameter, respectively.For the multi-objective minimization problem defined

above, solutionx(1) is said to dominate solutionx(2), if x(1)

is no worse thanx(2) in all objectives, i.e.,

∀m = 1, 2, · · · ,M, fm(x(1)) ≤ fm(x(2)), (5)

and ifx(1) is strictly better thanx(2) in at least one objective:

∃m′ ∈ 1, 2, · · · ,M, such thatfm′(x(1)) < fm′(x(2)).(6)

If a solution x∗ is not dominated by any other feasible

solutions, solutionx∗ is called Pareto-optimal. For mostmulti-objective optimization problems, there are a finite orinfinite number of Pareto-optimal solutions, which are knownas the Pareto set in the decision space, and the Pareto frontin the objective space.

B. Analysis of Pareto Optimal Solutions

When there is no preference over a particular objective,the Pareto-optimal solutions are non-comparable, i.e., theyare equally good. To pick out one solution for the final use,a user needs to provide some preference that helps the userto decide which Pareto-optimal solution best meets user’spreference. Due to this reason, it has been argued that itis not necessarily to approximate the whole Pareto front,which might require more computational resources. Whilethis kind of argument is true in some cases, it is our beliefthat achieving the whole Pareto front is of great value dueto the following reasons:

• The shape of the Pareto front, in particular, the convex-ness or concaveness of the front, as well as the locationof knee points and extreme points on the Pareto front,can reveal much additional domain knowledge, refer toFig. 1. A knee point is a solution on the Pareto frontthat requires a large sacrifice in the other objectives toimprove in one objective. Such domain knowledge maygreatly help the user in decision-making for choosingthe final solution.

• When aspects other than the performance of the so-lution in terms of the function value are consideredimportant, for example the robustness of the Pareto-optimal solutions, it is then very helpful to approximatethe whole Pareto front. By analyzing the robustness ofthe solutions, the user may modify their preference andchoose the best solution for the problem at hand.

f

f 1

f

f 1

f f

f 1

2 2

22

f 1

(a) (b)

(c) (d)

Fig. 1. Convexness and interesting points of Pareto fronts.A filled squaremeans an extreme solution, and a filled circle denotes a knee point. (a)Convex Pareto front, (b) Concave Pareto front, (c) Convex Pareto front withtwo sections of piecewise linear curves, and (d) Convex Pareto front withtwo separate linear curves.

While it is of essential importance to analyze the Pareto-optimal solutions for multi-objective optimization problems,it is sometimes very helpful to multi-objectivize problems

that originally have only one objective. For example diversityhas been used as the second objective to facilitate solvingmulti-modal problems in reducing the number of local opti-mums [4], [5], [6], or solving dynamic problems [7].

One important research area in which Pareto-based multi-objective analysis has found to be of particular interest ismachine learning [8]. Many typical learning problems, suchas neural network regularization, ensemble generation anddata clustering, can be addressed by summarizing the dif-ferent objectives into a scalar objective. These problems canbe solved more elegantly when the Pareto-based approachis used. A few examples will be provided in Section IV toelaborate on this point.

III. PARETO ANALYSIS OF REDUNDANT GENETIC

REPRESENTATIONS OFEVOLUTIONARY SYSTEMS

A. Trade-offs in Biological Systems

Natural evolution can never be single-objective. Trade-offsbetween different targets have accompanied the history ofevolution mainly due to the limited amount of energy andtime. A few examples in biological systems are:

• A trade-off between energy efficiency and functionalityin evolution of brain size and organization of nervoussystems. It is believed that the intelligence level oforganisms is roughly proportional to their relative brainsize, e.g., the ratio of brain weight to body weight.However, a big brain is evolved at costs [9]. First,a big brain often means more energy consumption,and therefore more food consumption, which is notalways available. Second, animals with a bigger brainusually have a longer life span, which means that theyare likely to experience more extreme environmentalchanges and thus harder survival environments. In theevolution of nervous systems, it has been shown thatenergy efficiency has also been a main constraint forevolving their functionality. This kind of trade-offs hasalso been shown to lead to the emergence of biologicallyplausible structure in the artificial evolution of a neuralsystem [10].

• There is a trade-off between reproduction and survival(longevity) [11]. Research has been shown that malefruit flies supplied with virgin females have lowerlongevity than those kept without access to females [12].This trade-off has also been shown to be important inevolving an optimal lifetime in an artificial evolutionarysystem [13].

• Evidence has also been found that supports a trade-offbetween the number of offspring and their size [14].

• Trade-offs have been found in many research areas ofbioinformatics and computational biology. For example,in protein sequence alignment, maximizing the numberof matching bases and minimizing the number of gapsmay be two conflicting objectives. Conflicting objec-tives must also be taken into account in constructingphylogenetic trees and gene regulatory networks [15].

B. Pareto Analysis of Robustness-Evolvability Trade-off

It is a challenging and extremely important task to under-stand how natural evolution has managed to bring about thehuge biological diversity and complexity from simple par-ticles and molecules. In addition to environmental changes,it is believed that two important principles, i.e., robustnessand evolvability, may have played a central role in shapingbiological complexity.

Biological robustness means organisms’ ability to rela-tively maintain their functionality under a certain degreeof internal and external perturbations. An important is-sue directly related to robustness is evolvability, which isorganisms’ ability to evolve inheritable novel phenotypicfunctionalities that help the organism survive and reproduce.In the recently years, research on robustness and evolvabilityhas become one of the main research topics in systemsbiology [16], [17].

Research on robustness and evolvability is still in itsinfancy [18], [19]. Not only a sophisticated quantitativedefinition for biological robustness and evolvability is stillmissing, but the biological origin, that is, how evolution hasshaped the various biological mechanisms for robustness andevolvability remains to be understood.

In a broader sense, robustness contributes to evolvabil-ity in that without robustness, evolutionary tinkering willmost likely lead to the lethal consequences, thus preventingevolution from creating new functionalities. For a clearerunderstanding of the mechanisms underlying evolvabilityand robustness, we investigate here evolvability in a narrowsense, that is, systems’ ability to generate new phenotypes,termed innovation hereafter. Although it has been recognizedqualitatively that there is a trade-off between robustnessandinnovation, no quantitative results have been reported.

Biological robustness can be achieved with a varietyof mechanisms, such as feedback, genotypic redundancy,functional modularity, among others [16]. In the following,we investigate a redundant genotype-phenotype mapping toinvestigate quantitatively the robustness-evolvabilitytrade-off.

1) A Boolean Model for Genotype-Phenotype Mapping:Consider the following genotype-phenotype mapping fromn-dimensional genotype spaceG ∈ 0, 1n to m-dimensionalphenotype spaceP ∈ 0, 1m:

pi = fki

i (ci1(g1), ci2(g2), · · · , cin(gn)), (7)

wherepi, i = 1, 2, . . . ,m is thei-th phenotype trait,gj , j =1, 2, · · · , n is the j-th genotype, andC = (cij)m×m, where

cij =1 : ⇐⇒ phenotype traitpi is affected by genegj

cij =0 : ⇐⇒ phenotype traitpi is independent of genegj .

fki

i (.) is a Boolean function withki inputs, where

ki =

n∑

j=1

cij , (8)

is also known as the arity of the Boolean function.

If ki > 1, that is, if phenotypepi is influenced by morethan one genotype, it is called polygeny. In contrast, thenumber of phenotype traits affected by genegj is given bythe sum of the elements in the column:

lj =m∑

i=1

cij . (9)

If lj > 1, then genotypegj is said to be pleiotropy. Withoutloss of generality, it is assumed thatC is chosen such thatlj ≥ 1 for all i = 1, . . . , n. This means that each gene affectsat least one phenotype trait.

The mapping can be defined by the dependencies betweengenes and phenotype traits, and a set of Boolean functionsthat determine the values of the phenotype traits. Since theconnection between genotype and phenotype determines thenumber of inputs for a certain phenotype trait and thus for thecorresponding Boolean function, the dependencies betweengenes and phenotype traits are determined at first. Once theconnection matrixC has been fixed, the Boolean functionsfki

i : 0, 1ki → 0, 1, i = 1 . . . ,m can then be defined.An example of genotype-phenotype mapping with eight

genes and four phenotype traits is given in Fig. 2, wherethe arity of the Boolean function for the four phenotypes isthree, six, six, and two, respectively. The connection matrixin this example is as follows:

C =

1 0 0 1 0 0 0 11 1 1 1 1 0 1 00 1 1 0 1 1 0 00 0 1 0 0 0 1 0

. (10)

g1

g2

g3

g4

g5

g6

g7

g8

f (g , g , g )8411

f (g , g , g , g , g , g )2 1 2 3 4 5 7 2 3 53

f (g , g , g , g )6 4 3 7

f (g , g )

Fig. 2. An example of genotype-phenotype mapping, where the numberof genes is eight, and the number of phenotype traits is four.

In the following, we are going to evolve the connectionmatrix C as well as the Boolean functions to maximizethe robustness and evolvability using the NSGA-II [20],which is one of the most popular evolutionary multi-objectiveoptimization algorithms.

2) Encoding of the Model:The first stage of the multi-objective evolutionary approach is to determine the schemefor representing the genotype-phenotype mapping model.In this work, an encoding with a fixed coding length andrelatively compact has been adopted.

The encoding of the Boolean model consists of two parts:the encoding of the connection matrixC and the encodingof the Boolean functionsfi, i = 1, . . . ,m. The encoding ofC is trivial, since each entrycij can be written in a binaryvector, leading to a fixed encoding lengthmn, independentof the actual value ofC.

Finding an encoding forfi is more difficult. The con-nection matrixC determines the polygenyki =

∑n

j=1 cij

of each phenotype traitpi, i.e., the number of inputs ofthe Boolean functionfki

i . Sincefki

i is a Boolean functionfki

i : 0, 1ki → 0, 1, it is completely defined when thecorresponding outputs for each2ki inputs are determined. Itcan be seen that such a canonical encoding is dependenton the actual value of the connection matrixC. Duringevolutionary search the size of such an encoding is changingand a set ofad hocsearch operators have to be defined toguarantee that newly generated solutions are feasible.

A simpler option is to define an encoding forfki

i ()independent of its arityki. To this end, it is assumed thatki = n due to the fact that a phenotype trait cannot dependon more thann genes. Unfortunately, this approach wouldlead to an encoding with a length ofnm + 2nm. Even for amedium size of genotype and phenotype spaces, e.g.,m = 8and n = 16, the length of the chromosome will becomeintractably large. To address this problem, we impose morerestrictions on the Boolean model so that a reasonably smallencoding length can be achieved. Since the crucial part ofthe encoding is the encoding of the Boolean functionsfki

i ()and its dependency onki, we are going to define a class ofBoolean functions that are independent of the polygenyki.

Let fke ∈ Fke denote an arbitrary Boolean func-tion with arity ke. Then the k-ary extension of fke ,fk ↑fke : 0, 1k → 0, 1, is recursively defined by

fk ↑fke (g1, . . . , gk)

=

fke(g1, . . . , gke

), if k = ke;fke(fk−1 ↑fke (g1, . . . , gk−1), gk), if k > ke;

(11)for k ≥ ke. If k < ke, then,

fk ↑fke (g1, . . . , gk) = fke(1, . . . , 1︸ ︷︷ ︸

ke−k

, g1, . . . , gk), (12)

where the first(ke − k) positions are set to1. fke is termedthe elementary Boolean function andke its elementary arity.Fig. 3 shows an example of 4-ary extension of 2-ary Booleanfunction f2.

f 2

g2

g1

3g

f 2

f 2

g4

Fig. 3. An example of 4-ary extension of a 2-ary Boolean function:f4 ↑f2 .

With the definition of thek-ary extensionsfk ↑fke ofBoolean functions, the following three different approachesto encoding the Boolean functions are considered, each ofwhich results in a reasonably compact and fixed encodinglength.

• Multiple Boolean FunctionsAll fk ↑fke have the sameencoding size2ke , independent of their actual arityk,since only the elementary Boolean functionfke has tobe encoded. Consequently, a restricted version of theBoolean model can be defined as follows: determine afixed elementary arityke for the model and choose everyBoolean functionfi = fk ↑

fke

i

, i = 1, . . . m, whereki is the polygeny of the corresponding phenotypetrait pi and is determined by the connection matrixC. It is important to emphasize that each phenotypefunctionfi is thek-ary extension of different elementaryBoolean functionsfke

i . This model restriction leads toan encoding size ofnm+m2ke bit, which is reasonablysmall for ke = 2, 3, 4. The structure of the encoding isschematically depicted below:

connection matrix︷ ︸︸ ︷

c11 c12 · · · cmn

elementary Boolean functions︷ ︸︸ ︷

fke

1 fke

2 · · · fke

m (13)

• Single Boolean Function EncodingAn even simplerencoding can be achieved when the Boolean model isrestricted to a single elementary Boolean function. Indoing so, a fixed elementary arityke for the model isdetermined and the local Boolean functions areki-aryextensions of the same elementary Boolean function:fi := fk ↑fke , i = 1, . . . ,m. This restriction leads toan encoding size ofnm + 2ke bit. The structure of theencoding is schematically depicted as follows:

connection matrix︷ ︸︸ ︷

c11 c12 · · · cmn fke (14)

• Majority Rule EncodingIn a further simplification of theBoolean model, we dispense with an explicit definitionof the Boolean functionsfi. Instead, the phenotype traitsare determined by the majority rule. There are severalpossibilities to break ties. In our work, we set the outputof majority rule to its first input in the case of a tie. Thiskeeps the rule balanced and deterministic:

fi(gj1 , gj2 , . . . , gjki) =

1, ifki∑

l=1

gjl> ki/2;

0, ifki∑

l=1

gjl< ki/2;

gj1(i), otherwise.(15)

Hence, only the connection matrix has to be encoded:

c11 c12 · · · cmn (16)

These different encodings restrict the original model in astrong way and so one could say that each presents a differentmodel of its own.

3) Objective Setup I: Maximizing Local Neutral Degreeand Local Variability: There is no widely accepted quantita-tive definition for robustness and evolvability. In this setup,we use local neutral degree for estimating the robustness

and the maximum local innovation for approximating theevolvability of a genotype-phenotype mapping.

Given a genotype-phenotype mappingφ : G → P anda neighborhood relationN over the set of genotypesG,then thelocal neutral degreeνφ(g) of mappingφ is definedby [21]

νφ(g) :=|g′ ∈ G : φ(g) = φ(g′) ∧ N(g, g′)|

|g′ ∈ G : N(g, g′)|. (17)

Similar to the definition of local neutral degree, a definitionof the local variability can be defined as follows. Given agenotype-phenotype mappingφ : G → P and a neighborhoodrelation N over the set of genotypesG, then the localvariability δφ(g) of mappingφ at genotypeg is defined as

δφ(g) :=|g′ ∈ G : φ(g) 6= φ(g′) ∧ N(g, g′)|

|g′ ∈ G : N(g, g′)|. (18)

Note that the variabilityδ captures the fraction of uniquephenotypes in the non-neutral neighborhood, which is ac-complished by the set notation. Accordingly,∀g ∈ G, νφ(g)+δφ(g) ≤ 1 holds by definition.

(a) (b)

Fig. 4. Two illustrative examples on calculating local neutral degree andlocal variability. In the figure, filled circles, squares andpentagons denotedifferent phenotypes, while the lines represent local neighborhood in thegenotype. (a)νφ(g) = 0.5, δφ(g) = 0.25, and (b)νφ(g) = 0.5, δφ(g) =0.5.

Two illustrative examples on how to calculate local neutraldegree and local variability are provided in Fig. 4.

The definition of local neutral degree has been discussedin a number of studies. However, in most of these stud-ies, the local neutral degree measure is investigated withrespect to single genotypes or neutral networks mapped tothe same phenotype. In this work, we want to evaluatedifferent characteristics of neutrality with respect to entiregenotype space. We use the local neutral degree to calculatethe mean neutral degree of a genotype-phenotype mapping,which is accomplished by randomly sampling genotypes andaveraging their local neutral degree. A high mean neutraldegree indicates that there are many genotypes with a highlocal neutral degree, and therefore the mean neutral degreereflects how many neutral mutations or how much neutralityis provided by the mapping. In the genotype space wherethe neutral degree is high, mutations are most likely neutralrather than deleterious. As a consequence, populations onneutral networks tend to drift toward regions with a highneutral degree to evolve robustness. This gives rise to theconsideration that the mean neutral degree is related tomutational robustness.

In order to evaluate the robustness of a mapping, themean of its neutral degree distributionνφ is estimated.The local neutral degree of randomly sampled genotypes isdetermined and averaged over all samples. Consequently, themean neutral degree reflects the density of neutral spaces. Amapping with a high mean neutral degree has more regionsin the genotype space that are surrounded by neighborswhich map to the same phenotype. Therefore, phenotypesassociated with dense neutral spaces have stronger robustnessagainst mutations than those associated with less denseneutral spaces.

The mean variability of a genotype-phenotype mappingcan be calculated in a similar way. It is the sensitivity ofphenotypes to genotypic mutations and therefore can beconsidered as the opposite of robustness to genetic changes.A high mean variability can be viewed to contribute toevolvability, when a condition of evolvability is defined astoreduce the number of mutations needed to produce phenotyp-ically novel traits [22], [23]. In [24], Fontana states thatthecapacity to evolve in response to selective pressures dependson phenotypic variability. This suggests that variabilitycanbe considered as prerequisite for evolvability.

It is important to note that the neutral degree and variabil-ity are related according to∀g ∈ G : νφ(g) + δφ(g) ≤ 1 andthus the mean neutral degreeνφ and the mean variabilityδφ

are conflicting properties of a mapping. Therefore, the truePareto front is given byνφ + δφ = 1. In order to study thistrade-off relationship and how this is resolved by the Booleanmodel, we will evaluate different encodings of the model inthe proposed multi-objective optimization framework.

In the experiment, different encodings of the Booleanmodel were optimized with respect to the mean variabilityand mean neutral degree. The results for a population of 50individuals and 100 generations are summarized in Fig. 5.

It can be seen that the different encodings lead to quite dif-ferent results. The random initial population with the multipleBoolean function encoding is unevenly distributed across theobjective space. The mappings lie at some distance from thetrue Pareto frontνφ + δφ = 1, albeit mappings withν > 0.6are almost on the true Pareto front. After 100 generations, themappings lie evenly distributed on the theoretical trade-offsurface. This is different to the case in which single Booleanfunctions encoded. The randomly initialized population liesalmost entirely on the theoretical Pareto front, but only ona sectionν < 0.5. After multi-objective optimization wasperformed, the mappings are more evenly distributed alongthis section. Only a few mappings lie on the Pareto frontsection withν < 0.5. The solutions seem to have a uniformdistance from each other. For0.2 < ν < 0.5, however, noPareto optimal solutions have been found. In the case ofthe majority rule encoding, the randomly initialized mappingform a compact cluster and evolutionary search finds Paretooptimal mappings with0.5 < ν < 0.8. In summary, allgenotype-phenotype mappings found by different encodingsof the Boolean models and evolutionary search lie on the truePareto frontνφ + δφ = 1. However, the found Pareto optimal

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

ææ

æ

æ

æ

æ

æ

æ

æ

ææ

æ

æ

æ

æ

ææ

æ

æ

ææ

0.0 0.2 0.4 0.6 0.8 1.00.0

0.2

0.4

0.6

0.8

1.0

neutral degree

varia

bilit

y

a1æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

ææ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

0.0 0.2 0.4 0.6 0.8 1.00.0

0.2

0.4

0.6

0.8

1.0

neutral degree

varia

bilit

y

a2

æ

ææ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ æ

æ

æ

æ

ææ

æ

ææææ

æææ

ææ

æ

æææææ

æ

æææææææææææææææ

æ

ææ

æ

æ

æ

æ

æ

æ

æ

æ

æææ

æ

æ

æææ

æ

0.0 0.2 0.4 0.6 0.8 1.00.0

0.2

0.4

0.6

0.8

1.0

neutral degree

varia

bilit

y

b1æ

æ

æ

æ

æ

æææææææææææææææææææææææææææææ

æææææææææææ

ææææææææææææææææ

0.0 0.2 0.4 0.6 0.8 1.00.0

0.2

0.4

0.6

0.8

1.0

neutral degree

varia

bilit

y

b2

æ

æ

ææ

ææææææ

æ

æ

ææææææ

æ

æ

æ

æ

ææææææææææææææææ

ææ

æ

æææææææææ

æ

æææææææææææææ

æ

æ

æææææææææ

0.0 0.2 0.4 0.6 0.8 1.00.0

0.2

0.4

0.6

0.8

1.0

neutral degree

varia

bilit

y

c1

ææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææ

æ

0.0 0.2 0.4 0.6 0.8 1.00.0

0.2

0.4

0.6

0.8

1.0

neutral degree

varia

bilit

yc2

Fig. 5. Multi-objective optimization evaluation of the Boolean model (n = 16, m = 8) for maximal mean neutral degreeνφ and maximal mean variabilityδφ. The model is encoded by different techniques:a Multiple Boolean functions encoding with elementary arity 2. b Single Boolean function encodingwith elementary arity 2.c Majority rule encoding. Plots in the left panel show the results of 50 randomly initialized genotype-phenotype mappings withn = 16 andm = 8. Those in the right panel show the approximated Pareto optimalset of genotype-phenotype mappings after 100 generations. The dottedline indicates the theoretical Pareto front, determined by the definition of neutral degree and variability, i. e.νφ + δφ = 1. The different encodings resultin different approximations of a Pareto optimal set.

solutions are distributed along different regions. Only withthe multiple Boolean function encoding was the evolutionaryalgorithm able to find the complete Pareto optimal front.

These results indicate that the different encodings ofthe Boolean functions differ in their capability of realizingvarious genotype-phenotype mappings. Since the multipleBoolean function encoding restricts the original Booleanmodel at the least, it is able to approximate the widest rangeof the theoretical Pareto front. For example, a genotype-phenotype mappingφ : 0, 116 → 0, 18 with variabilityδφ = 1 and neutral degreeνφ = 0 can be implemented.Despite that this mapping has a redundancy of28, it pos-sesses no neutrality. This means that the genotype space isstructured in such a way that genotypes that map to thesame phenotype are never neighbored and thus every singlepoint mutations leads to a new phenotype. Therefore, such

a mapping can be thought of as having low robustness andhigh evolvability.

Alternatively, mappings with neutral degreeδφ ≥ 0.9 andvariability νφ ≤ 0.1 can be implemented by the multipleand single Boolean functions encoding. In this case, almostall single point mutations are neutral and lead to the samephenotype. Therefore, such a mapping can be considered ashighly robust and less evolvable.

In order to understand how the Boolean model is capableof implementing such different mappings, we analyze theunderlying parameters of the model. These are given by theconnection matrixC and the set of Boolean functionsfi, i =1, . . . ,m. To obtain a first impression of the parameters’structures, we visualized them. Fig. 6 shows three example ofencoding structures for different encoding schemes. Visually,no significant features or patterns in the encoding structure

can be identified. An in-depth analysis of the parameters isdifficult because of the high degree of freedom in the model.A statistical approach is presented in [25] to interpret theparameter structure of the majority rule encoding.

a b c

Fig. 6. Exemplary parameter visualization of the Boolean model(n = 16,m = 8) yielded by different encoding techniques. Each frame correspondsto an implementation of the connectivity matrixC and the set of phenotypefunctions (if necessary) of a genotype-phenotype mapping.a MultipleBoolean functions encoding with elementary arity 2. The leftarray depictsthe connection matrix, the right array illustrates the phenotype functionsfi, i = 1, . . . , m. Row i depicts the truth table of the elementary Booleanfunction f2

i . b Single Boolean functions encoding with elementary arity2. The left array depicts the connection matrix, the right array depicts thetruth table of the elementary Boolean functionf2. c Majority rule encoding.The array indicates the connection matrix. In all descriptions, black arrayentries corresponds to 1, whereas white array entries correspond to 0. Anentry cij = 1 in the connection matrix indicates, that phenotype traitpi

depends on genegj . Therefore, the pleiotropy of genegj is given by thesum of columnj, whereas the polygeny of phenotype traitpi is given bythe sum of rowi.

4) Objective Setup II: Maximizing Neutral Degree andUniformity Entropy: In order to gain a deeper insight into therelationship between neutral degree and entropy, we carriedout another multi-objective optimization with neutral degreeand entropy as two objectives. The entropy measures thediversity of the reachable phenotypes, which is therefore ableto reflect the evolvability. In calculating the entropy, geno-types are sampled randomly from the genotype space, andare mapped to their associated phenotypes. The occurrenceof each phenotype is counted and normalized to determiningthe probability of locating a certain phenotype by randomlychoosing a genotype. Then, a normalized entropy can becalculated. See e.g., [26] (pp.37). The experiment was per-formed with different encodings and the results are shown inFig. 7. The results vary slightly with the different encodings.They differ mainly in the way in which the random initializedpopulation is distributed in the objective space. After200generations, the results resemble each other. The mappingsencoded with single and multiple Boolean functions forma well-distributed Pareto front. Mappings encoded with themajority rule cover only a section of the Pareto front anddo not exceed a neutral degree of0.8. This has also beenobserved in the previous experiment and can be attributed tothe strong restriction that reduces the original Boolean modelto the majority rule.

The results suggest that a genotype-phenotype mappingimplemented by the Boolean model can provide high entropyup to a certain neutral degree threshold. It remains to beclarified if this observation is a consequence of the definitionof neutral degree and entropy, and so is true for arbitrarygenotype-phenotype mappings or this is a property of theBoolean model. Moreover, it would be interesting to investi-gate how the observed neutral degree threshold depends onother model parameters. We hypothesize that this thresholdis determined by the redundancy of the mapping, which is216 : 28 in our experiments, but we did not perform further

experiments to investigate this question.This threshold is important when considering neutral de-

gree as a measure for mutational robustness. Our resultssuggest that high entropy can only be guaranteed up toa certain mean neutrality degree, and with decreasing en-tropy more and more phenotypes become inaccessible tothe evolutionary system through the genotype-phenotypemapping. Therefore it must be discussed if the notion ofrobustness can be applied to mappings that are constant ornot surjective. Put differently, given two redundant genotype-phenotype mappingsφ1 andφ2, both defined over the samegenotype and phenotype spaces. Then the question mustbe raised whether the mappingφ1 with a smaller image,that is |φ1(g) | g ∈ G | < |φ2(g) | g ∈ G |, is inevitablythe mapping with higher mutational robustness. We thinkthat non-surjective genotype-phenotype mappings must beconsidered when defining robustness.

The above work on analyzing robustness and evolvabilitytrade-off of redundant genetic representations for simulatedevolution demonstrates that the Pareto-based approach isof great value in uncovering the relationship between re-dundancy and evolvability of genetic representations. Oneimportant implication of the above results is that in a fixedenvironment, different representations with a certain degreeof robustness and innovation are incomparable, i.e., no ofthe representations may be preferred. However, when theenvironment changes at a certain speed, a particular trade-offbetween robustness and evolvability may be optimal to thegiven environment, and therefore, a particular representationis better than others. For example, in a quickly changing en-vironment, a representation with a higher innovation degreeis more capable to adapt to the environment. In contrast, in aslowly changing environment, a representation with a higherdegree of robustness may have a larger probability to survive.As a result, if the environment changes at different speeds,a representation with a changing degree of robustness andevolvability will be preferred. This observation is consistentwith the findings on other redundant representations, e.g.,those reported in [27]. In the following, we will apply thePareto-based approach to studying machine learning systems.

IV. PARETO ANALYSIS OF LEARNING SYSTEMS

A. Trade-offs in Learning

Machine learning can be largely divided into supervisedlearning, unsupervised learning and reinforcement learning.Any machine learning method can be seen as an optimizationproblem, because the target of learning is either to minimizea cost function or maximize a reward or value function.

Machine learning has traditionally been treated as a singleobjective optimization problem. However, if we examinedifferent learning problems more closely, most of them haveto deal with more than one objective. Let us first take a lookat model selection strategies, which are the most importantissue in supervised or unsupervised learning.

A well-known criterion for model selection is the Akaike’s

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

ææ

æ

ææ

ææ

ææ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

ææ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

ææ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

ææ

æ

æ

æ

æ

ææ

æ

æ

æ

æ

æ

æ

æ

ææ

ææ

ææ

æ

æ

æ

ææ

æ

æ

æ

æ

ææ

0.0 0.2 0.4 0.6 0.8 1.00.0

0.2

0.4

0.6

0.8

1.0

neutral degree

entr

opy

a1æ æ

ææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææææ

ææææææææææææ

0.0 0.2 0.4 0.6 0.8 1.00.0

0.2

0.4

0.6

0.8

1.0

neutral degree

entr

opy

a2

ææ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

ææ

æ

æ

æ

æ

æ

æ

æææ

æææ

ææ

æ

æ

æ

æ

æææææææææææ

æææææææææææææ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æææ

ææ

0.0 0.2 0.4 0.6 0.8 1.00.0

0.2

0.4

0.6

0.8

1.0

neutral degree

entr

opy

b1æ æ æ

æ

æææææææææææææææææææææææææææææææææææææææææ

æææææææææææææææææææ

ææææææææææ

0.0 0.2 0.4 0.6 0.8 1.00.0

0.2

0.4

0.6

0.8

1.0

neutral degree

entr

opy

b2

æ

æ

æ

æææ

æ

æ

æ

æ

æ

æ

æ

æ

ææææ

æ

æ

æ

æ

ææ

ææææ

ææ

æ

æ

ææ

æ

ææ

æ

æ

æ

æ

æ

æ

æ

ææ

ææ

ææ

æ

æ

ææ

æ

æ

æ

ææ

æ

æææ

æ

æ

æææ

æ

æ

æ

æ

æ

ææ

0.0 0.2 0.4 0.6 0.8 1.00.0

0.2

0.4

0.6

0.8

1.0

neutral degree

entr

opy

c1æ ææææææææææ

æææææææææææææææææææææææææææææææææææææææææææææææ

ææææææææææææææææ

æ

0.0 0.2 0.4 0.6 0.8 1.00.0

0.2

0.4

0.6

0.8

1.0

neutral degree

entr

opy

c2

Fig. 7. Multi-objective optimization evaluation of the Boolean model with (n = 16, m = 8) for maximal mean neutral degree and uniformity entropy.The model is encoded by different techniques:a Multiple Boolean functions encoding with elementary arity 2. b Single Boolean function encoding withelementary arity 2.c Majority rule encoding. The figures1 in the first columns show 75 randomly initialized genotype-phenotype mappings withn = 16andm = 8. The figures2 in the second column the approximated Pareto optimal set of genotype-phenotype mappings after 100 generations. The differentencodings result in similar approximations of a Pareto optimalset.

Information Criterion (AIC):

AIC = −2log(L(θ|y, g) + 2K, (19)

whereg is the true function from which the training data areproduced,θ is the model,L(θ|y, g) is the maximized log-likelihood between the model and the true function given thetraining datay, andK is the effective number of parametersto be estimated in the modelθ. For a number of differentmodels trained from the same training data set, the onewith the minimal AIC will be chosen. From Equation 19,we can see that AIC consists of two terms. The first termis to maximize the accuracy of the model, whereas thesecond termK indicates the complexity of the model. Asthe model complexity increases, i.e., the number of freeparameters in the model increases, the first term in the AICtends to decrease, while the second becomes larger, whichin fact reflects the well-known bias-variance dilemma. From

the optimization point of view, these two terms cannot beminimized simultaneously and therefore, model selection,ormore generally, machine learning is actually a typical multi-objective optimization problem.

In supervised learning of neural networks and fuzzy sys-tems, several specific trade-off criteria have been consideredderived from the fundamental trade-off relationship definedby the AIC. The most commonly used criterion for modelfidelity is the error function, such as the mean square erroror the mean absolute error. Complexity of learning modelsis also very often used, including the Gaussian regularizerthat is the sum of the squared weights of the neural net-work, or the Laplacian regularizer, which is the sum of theabsolute weights. If a non-gradient based algorithm is usedfor learning, the number of hidden nodes or the number ofconnections can also be used to denote the complexity ofthe network. In generating fuzzy systems, complexity of the

fuzzy rules is directly associated with their interpretability,in other words, the easiness for human users to understandthe knowledge represented by the fuzzy rules [28], [29].Reducing the number of fuzzy subsets, the overlaps of thefuzzy subsets, the number of fuzzy rules and the rule length(in the rule premises) will improve the interpretability offuzzy rules. When ensembles of learning models are to begenerated, either functional or structural diversity of theensemble members can be used as the objective in ensemblegeneration [8].

Although the bias-variance trade-off in the AIC has notbeen explicitly taken into account in many learning algo-rithms, it is explicitly considered in the learning algorithms ofsupport vector machines (SVMs). Without loss of generality,we take SVMs for classification as an example. A typicalSVM for classification can be expressed as the followingoptimization problem:

minimize 12w

Tw + C

∑N

i=1 ξi, (20)

subject to yi(wT φ(xi) + b) ≥ 1 − ξi, (21)

ξi ≥ 0, (22)

(23)

where xi ∈ Rn, yi ∈ −1, 1, i = 1, 2, · · · , N are thetraining data set consisting ofN data pairs,φ(xi) is a kernelfunction, b is the bias,ξi is called interior deviation orslackness, andC is a constant to be determined by the user.Obviously, minimizingw

Tw and minimizing

∑N

i=1 ξi aretwo conflicting objectives, and solving the SVM is a multi-objective optimization problem [30].

One of the main principles in information processing in thebrain is that the fraction of neurons that are strongly activeat a time is below 1/2. This principle has been implementedin computational neuroscience, which results in a class ofcomputational models known as sparse coding [31], wherethe accuracy of the model trades off with the sparsenessof the active neurons in the model. So far, the coefficientdetermining the trade-off between the two terms has beenchosen heuristically, and no in-depth discussion about theinfluence of the co-efficient on the final performance hasbeen reported.

Another well-known issue in computational cognitive neu-roscience is the stability-plasticity dilemma [32], whichmeans that the learning system should be able to learn newinformation efficiently without completely forgetting whathas been learned previously. The stability versus plasticitydilemma is often known as catastrophic forgetting in neuralnetwork based machine learning [33]. Most existing tech-niques try to alleviate catastrophic forgetting with modelsusing distributed representation or a growing structure. Directrehearsal where the previous training data is assumed to beavailable, or pseudo-rehearsal using pseudo-data generatedfrom the trained model has also been suggested to addresscatastrophic forgetting [34].

checking

SimilarityRandom

Error on pseudo−patterns

Err

or o

n ne

w p

atte

rns

inputs

Network outputs

Pseudo−patterns

Newpatterns

Fig. 8. Pseudo-rehearsal through multi-objective optimization.

B. Illustrative Examples

In this section, we will provide two examples of Pareto-based multi-objective learning. The first example shows howcatastrophic forgetting can be approached using the Pareto-based approach. In the second example, neural networkregularization is tackled with by reformulating regularizationas a Pareto-based multi-objective optimization problem. Inaddition, we demonstrate that by analyzing the accuracy-complexity trade-off, it is able to identify Pareto optimalsolutions (learning models) with a complexity that mostlikely matches that of the problem in question.

1) Alleviating Catastrophic Forgetting:Catastrophic for-getting means that when a trained neural network learns newpatterns, the already learned information (the base patterns)will be destroyed (forgotten). Since learning the base patternsand learning the new patterns are very likely competitive,it is natural to deal with the conflicting objectives usingthe Pareto-based multi-objective learning [35]. In that work,pseudo-rehearsal is reformulated as a multi-objective opti-mization problem, as shown in Fig. 8. Before training theneural network with the new patterns, a number of pseudo-patterns are generated. The pseudo-patterns are created bygenerating random inputs to the trained neural network, andthen recording the corresponding outputs of the network.It is hoped that the pseudo-patterns will carry the same orsimilar knowledge as the base patterns. However, this is notalways true in practice. To address this problem, a similaritychecking can be performed so that the a randomly generatedinput is adopted only if it is sufficiently similar to one ofthe inputs of the base patterns. In this case, it is assumedthat the input information of the base patterns is availableduring learning the new patterns, though the outputs of thebase patterns are unavailable. Similarity checking in thiswork is based on the Hamming distance. Next, learning iscarried out as a bi-objective optimization problem, wherethe approximation error on the new patterns and that on thepseudo-patterns serve as the two objectives.

It has been found that it is non-trivial to properly performlife-time learning within the evolutionary cycles in multi-objective learning. This is particularly true when all theobjectives are concerned with approximation error. In the

0 5 10 15 20 250

5

10

15

20

25

Number of remembered base patterns

Num

ber

of r

emem

bere

d ne

w p

atte

rns

Fig. 9. Number of based and new patterns memorized by the Pareto-optimalsolutions.

case of alleviating catastrophic forgetting, three life-timelearning strategies have been investigated [35]. In strategy1, the union of pseudo-patterns and new patterns are usedin life-time learning. Strategy 2 randomly chooses either thepseudo-patterns or the new patterns for lifetime learning.Andin strategy 3, one of the three patterns, the pseudo-patterns,the new patterns, and the union of pseudo-patterns andnew patterns, is picked out randomly for life-time learning.It is found that using the union of pseudo-patterns andnew patterns in life-time only will dramatically reduce thepopulation diversity and only a few Pareto optimal solutionscan be obtained. The diversity of solutions will be improvedwhen strategy 2 or 3 is used. Fig. 9 shows the trade-offsolutions in terms of memorized based and new patterns.In this experiment, 25 pairs of base and new patterns aregenerated randomly, each pair of pattern containing a 10-dimensional input and a 10-dimensional output of valueeither 0 or 1. Then, neural networks with a maximum of10 hidden nodes are evolved to learn the base patterns. Afterthe training is completed, 24 of the 25 based patterns arecorrectly learned. Then, 25 pseudo-patterns are generatedbased on the trained neural network. From Fig. 9, we can seethat the Pareto-optimal solutions are able to memorize morethan 15 base patterns while learning over 20 new patterns.

2) Neural Network Regularization:To improve gener-alization of neural networks, regularization techniques areoften adopted by including an additional term in the errorfunction:

J = E + λΩ, (24)

where λ is a hyperparameter that controls the strength ofthe regularization,Ω is known as the regularizer, andE isusually the mean square error (MSE) on the training data:

f =1

N

N∑

i=1

(y(i) − yd(i))2, (25)

wherey(i) and yd(i) are the model output and the desiredoutput, respectively, andN is the number of data pairs inthe training data.

0 50 100 150 200 250 3000

10

20

30

40

50

60

Sum of squared weights

Num

ber

of c

onne

ctio

ns

Gaussian regularizer

Fig. 10. Relationship between the number of connections and the sum ofsquared weights.

The most popular regularization method is known asweight decay (also known as Gaussian regularizer):

Ω =1

2

∑

k

w2k, (26)

wherek is an index summing up all weights.One weakness of the weight decay method is that it is

not able to drive small irrelevant weights to zero, which mayresult in many small weights. The following regularizationterm has been proposed to address this problem (known asLaplacian regularizer):

Ω =∑

i

|wi|. (27)

This regularization was used for structure learning, becauseit is able to drive irrelevant weights to zero.

It is quite straightforward to see that the neural networkregularization in equation (24) can be reformulated as a bi-objective optimization problem:

min f1, f2 (28)

f1 = E, (29)

f2 = Ω, (30)

whereE is usually the mean square error, andΩ is one ofthe regularization terms defined in equation (26) or (27).

Since evolutionary algorithms are used to implement reg-ularized learning of neural networks, a new and more directindex for measuring complexity of neural networks can beemployed, which is the number of connections in the neuralnetwork:

Ω =∑

i

∑

j

cij , (31)

wherecij equals1 if there is connection from neuronj toneuroni, and0 if not.

When gradient-based learning algorithms are employed forregularization, the Laplace regularizer is usually believed tobe better than a Gaussian regularizer in that the Laplaceregularizer is able to drive irrelevant weights to zero. Inthis way, “structural learning” is realized with the help

0 20 40 60 80 100 1200

10

20

30

40

50

60

Num

ber

of c

onne

ctio

ns

Sum of absolute weights

Laplace regularizer

Fig. 11. Relationship between the number of connections and the sum ofabsolute weights.

of the Laplace regularizer. Using the evolutionary multi-objective approach, we show that there is no substantialdifference between the Gaussian and the Laplace regularizersin terms of their ability to realize structural learning, whenevolutionary algorithms are used as an optimizer.

To verify this assumption, we use the Breast Cancer Dataset available in the UCI Machine Learning Repository1. Theavailable data are split into a training data set and a testdata set, where 525 instances are used for training and 174instances for test.

From Figs. 10 and 11, we can see that a similar rela-tionship between the sum of squared or absolute weightsand the number of connections is observed. In other words,even when the Gaussian regularizer is used, the number ofconnections can also be reduced to the minimum when thesum of squared weights is minimized. It does not resultin many small weights as when gradient-based learningalgorithms are used.

3) Identifying Models of Suitable Complexity:In thissection, we show that the Pareto-approach to handling theaccuracy-complexity trade-off provides an empirical, yetinteresting alternative to selecting models that have goodgeneralization on unseen data. The basic argument is thatthe complexity of the model should match that of the data tobe learned and the ability of the learning algorithm. When thecomplexity of the model is overly large, learning becomessensitive to stochastic influences, and results on unseen datawill be unpredictable, i.e., overfitting can happen. To analyzethe relationship between the achieved accuracy-complexitytrade-off solutions, the normalized performance gain (NPG)is defined:

NPG =MSEj − MSEi

Ci − Cj

, (32)

whereMSEi,MSEj , andCi, Cj are the MSE on trainingdata, and the number of connections of thei-th and j-thPareto optimal solutions. When the solutions are ranked inthe order of increasing complexity, the following relation-

1http://archive.ics.uci.edu/ml/.

0 20 40 60 800

0.01

0.02

0.03

0.04

0.05

0.06

Number of Connections

MS

E

Models withappropriatecomplexity

Fig. 12. Accuracy versus complexity of the Pareto-optimal solutions fromtwo independent runs: Breast Cancer Data. Dots denote training data, andcircles test data.

0 10 20 30 40 50 60 70 800

0.005

0.01

0.015

0.02

0.025

Number of ConnectionsN

PG

0 10 20 30 40 50 60 70 800

0.005

0.01

0.015

0.02

0.025

Number of Connections

NP

G

(a) (b)

Fig. 13. Normalized performance gain from two independent runs for theBreast Cancer Data.

ships hold:

Ci+1 > Ci,

MSEi+1 ≤ MSEi.

We hypothesize that if the model complexity is lower thanthat of the data, an increase in complexity will result in sig-nificant increase in performance (NPG). As the complexitycontinues to increase, the NPG reduces gradually to zero.At this point, the complexity of the model matches that ofthe data. Further increase in complexity will probably bringabout further enhancement in performance on the trainingdata, but with a dramatically increasing risk of overfittingthe training data.

We are now going to verify empirically the suggestedmethod for model selection also on the Breast Cancer Dataset. The Pareto fronts generated from two independent runson the three benchmark problems are presented in Fig. 12.The dots denote the results on the training data set, whilethe circles the results on test data. The NPG from thetwo independent runs for the three problems are plotted inFig. 13. It can be seen from Fig. 12 that models with thenumber of connections larger than 10 to 15 start to overfit thedata, which roughly corresponds to the point in Fig. 13 wherethe NPG drops to zero after the first peak in performancegain. This empirical result is helpful for model selectionwhen the number of training data is too small to performmeaningful cross-validations and other statistical analysis.

Let us now summarize the main message of the aboveillustrative example. With the help of the Pareto-based multi-

objective approach, model selection can be performed byanalyzing the shape of the Pareto front, refer to Fig. 12,quantitatively realized through the analysis of the normalizedperformance gain of the obtained non-dominated solutions,see Fig. 13. This kind of analysis is based on the trainingdata only, and therefore, it is no longer necessary to split theavailable data into training and test data sets, neither is itnecessary to perform cross-validations. This may be of greatimportant when the number of available data is small.

V. CONCLUSIONS

This paper discusses the Pareto-based multi-objective anal-ysis of evolutionary and learning systems. With an evolu-tionary multi-objective optimization algorithm, a numberofPareto-optimal solutions (forming the Pareto front) can beobtained. By analyzing the Pareto front, we are able to gaina deeper insight into in the evolutionary and learning systems.We show in the first illustrative example how the Paretoapproach can be used to analyze the robustness-evolvabilitytrade-off in a class of redundant Boolean representations forsimulated evolution.

For learning systems, we show that the Pareto-basedapproach is able to find solutions that can learn new knowl-edge without a serious interference with the already learnedknowledge. In the multi-objective approach to neural networkregularization, we demonstrate that the Gaussian regularizerworks as efficient as the Laplacian regularizer in reducingthe complexity of neural networks, when an evolutionaryalgorithm is employed. In addition, we hypothesize thatPareto optimal solutions around the knee point are thosehaving the appropriate complexity for the given data, whichare most likely to generalize on unseen data.

Many interesting issues remain to be investigated. In thiswork, a simple and stationary Boolean representation is usedto study robustness and evolvability of genetic representa-tions. This should be extended to more complex, in particulardynamic genotype-phenotype mappings described by e.g.random Boolean networks or ordinary differential equations.In multi-objective learning, it is interesting to compare theconvergence speed of single and multi-objective learning.Multi-objective approaches to the analysis of sparse codingstill lacks. We believed that the Pareto-based multi-objectiveapproach will relieve much burden in tuning parameters andthus help to achieve a better understanding the problem athand.

ACKNOWLEDGMENT

The authors would like to thank Ingo Paenke for theinteresting discussions on the work on multi-objective op-timization of redundant representations.

REFERENCES

[1] L. Ancel and J. Bull, “Fighting change with change: Adaptive variationin an uncertain world,”Trends in Ecology and Evolution, vol. 17,no. 12, pp. 551–557, 2002.

[2] G. Tononi, O. Sporns, and G. Edelman, “A measure for brain com-plexity: Relating functional segregation and integrationin the nervoussystem,”PNAS, vol. 91, pp. 5033–5037, 1994.

[3] J. Teo and H. Abbass, “Multiobjectivity and complexity inembodiedcognition,” IEEE Transactions on Evolutionary Computation, vol. 9,no. 4, pp. 337–360, 2005.

[4] S. Louis and G. Rawlines, “Pareto optimality, GA-easiness and decep-tion,” in The Fifth International Conference on Genetic Algorithms.Morgan Kaufmann, 1993, pp. 118–123.

[5] J. Knowles, R. Watson, and D. Corne, “Reducing local optima insingle-objective problems by multi-objectivization,” inEMO 2001, ser.LNCS 1993. Springer, 2001, pp. 269–283.

[6] M. Jensen, “Helper-objectives: Using multi-objective evolutionaryalgorithms for single-objective optimization,”Journal of MathematicalModeling and Algorithms, vol. 3, no. 4, pp. 323–347, 2004.

[7] L. Bui, J. Branke, and H. Abbass, “Multi-objective optimization fordynamic environments,” inCongress on Evolutionary Computation.IEEE, 2005, pp. 2349–2356.

[8] Y. Jin and B. Sendhoff, “Pareto-based multi-objective machine learn-ing: An overview and case studies,”IEEE Transactions on Systems,Man, and Cybernetics, Part C: Applications and Reviews, vol. 38,no. 3, pp. 397–415, 2008.

[9] J. Allman, Evolving Brains. Scientific American Library, 1999.[10] B. Jones, Y. Jin, X. Yao, and B. Sendhoff, “Evolution of neural organi-

zation in the hydramat - a computational model of a radially-symmetricorganism,” ACM/IEEE Transactions on Computational Biology andBioinformations, 2008, in revision.

[11] S. Stearns, “Trade-offs in life-history evolution,”Functional Ecology,vol. 3, pp. 259–268, 1989.

[12] A. Mukhopadhyay and H. Tissenbaum, “Reproduction and longevity:secrets revealed by c. elegans,”Trends in Cell Biology, vol. 17, no. 2,pp. 65–71, 2007.

[13] I. Paenke, Y. Jin, and J. Branke, “Balancing populationand individuallevel adaptation in changing environments,”Adaptive Behavior, 2008,submitted.

[14] E. Charnov and S. Ernest, “The offspring-size / clutch-size trade-offin mammals,”The American Naturalist, vol. 167, no. 4, pp. 578–582,2006.

[15] J. Handl, D. Kell, and J. Knowles, “Multi-objective optimization incomputational biology and bioinformatics,”ACM/IEEE Transactionson Computational Biology and Bioinformations, vol. 4, pp. 279–292,2007.

[16] H. Kitano, “Biological robustness.”Nat Rev Genet, vol. 5, no. 11, pp.826–37, 2004.

[17] A. Wagner,Robustness and evolvability in living systems. PrincetonUniversity Press, 2007.

[18] R. Lenski, J. Barrick, and C. Ofria, “Balancing robustness and evolv-ability,” PLOS Biology, vol. 4, no. 2, p. e428, 2006.

[19] S. Cilibert, O. Martin, and A. Wagner, “Innovation and robustness incomplex gene networks,”PNAS, vol. 104, no. 34, pp. 13 591–13 596,2007.

[20] K. Deb, S. Agrawal, A. Pratab, and T. Meyarivan, “A FastElitist Non-Dominated Sorting Genetic Algorithm for Multi-Objective Optimization: NSGA-II,” inProceedings of the ParallelProblem Solving from Nature VI Conference, M. Schoenauer,K. Deb, G. Rudolph, X. Yao, E. Lutton, J. J. Merelo, andH.-P. Schwefel, Eds. Paris, France: Springer. Lecture Notes inComputer Science No. 1917, 2000, pp. 849–858. [Online]. Available:citeseer.ist.psu.edu/deb00fast.html

[21] P. K. Lehre and P. C. Haddow, “Phenotypic complex-ity and local variations in neutral degree,”Biosystems,vol. 87, pp. 233–242, Feb. 2007. [Online]. Avail-able: http://www.sciencedirect.com/science/article/B6T2K-4KVJCK6-8/2/099604f1791eecd1e23f7ea94729a5a9

[22] M. Kirschner and J. Gerhart, “Evolvability,”PNAS, vol. 95, no. 15,pp. 8420–8427, 1998.

[23] P. Fernandez and R. V. Sole, “Neutral fitness landscapes in signallingnetworks,”Journal of the Royal Society, Interface / the Royal Society,vol. 4, pp. 41–7, 2007.

[24] B. M. R. Stadler, P. F. Stadler, G. P. Wagner, and W. Fontana,“The topology of the possible: Formal spaces underlying patterns ofevolutionary change,”Journal of Theoretical Biology, vol. 213, no. 2,pp. 241–274(34), 2000.

[25] R. Gruna, “Analysis of redundant genotype-phenotype mappings -Investigation of the effect of neutrality on evolvability and robustness,”Master’s thesis, AIFB, Universitat Karlsruhe, 2007.

[26] A. Magurran,Ecological Diversity and its Measurement. PrincetonUniversity Press, 1988.

[27] T. Yu, “Program evolvability under environmental variations andneutrality,” in European Conference on Artificial Life (ECAL 2007),2007, pp. 835–844.

[28] Y. Jin, “Fuzzy modeling of high-dimensional systems: Ccomplexityreduction and interpretability improvement,”IEEE Transactions onFuzzy Systems, vol. 8, no. 2, pp. 212–221, 2000.

[29] ——, Advanced Fuzzy Systems Design and Applications. Heidelberg:Physica Verlag/Springer Verlag, 2003.

[30] C. Igel, “Multi-objective model selection for support vector machines,”in Evolutionary Multi-Criterion Optimization, ser. LNCS 3410, 2005,pp. 534–546.

[31] B. Olshausen, “Relations between the statistics of narutal image andthe response property of cortical cells,”Nature, vol. 381, pp. 607–609,1996.

[32] W. Abraham and A. Robins., “Memory retention - the synaptic stabilityversus plasticity dilemma,”Trends in Neuroscience, vol. 28, no. 2, pp.73–78, 2005.

[33] M. McCloskey and N. Cohen, “Catastrophic interferencein connec-tionist networks: The sequential learning problem,”The Psychology ofLearning and Motivation, vol. 24, pp. 109–165, 1989.

[34] R. French, “Catastrophic forgetting in connectionistnetworks,”Trendsin Cognitive Sciences, vol. 3, no. 4, pp. 128–135, 1999.

[35] Y. Jin and B. Sendhoff, “Alleviating catastrophic forgetting via multi-objective learning,” inInternational Joint Conference on Neural Net-works, 2006, pp. 3335–3342.