Linear Operators on CorrelatedLandscapesPeter F. Stadler

SFI WORKING PAPER: 1993-06-042

SFI Working Papers contain accounts of scientific work of the author(s) and do not necessarily represent theviews of the Santa Fe Institute. We accept papers intended for publication in peer-reviewed journals or proceedings volumes, but not papers that have already appeared in print. Except for papers by our externalfaculty, papers must be based on work done at SFI, inspired by an invited visit to or collaboration at SFI, orfunded by an SFI grant.©NOTICE: This working paper is included by permission of the contributing author(s) as a means to ensuretimely distribution of the scholarly and technical work on a non-commercial basis. Copyright and all rightstherein are maintained by the author(s). It is understood that all persons copying this information willadhere to the terms and constraints invoked by each author's copyright. These works may be reposted onlywith the explicit permission of the copyright holder.www.santafe.edu

SANTA FE INSTITUTE

Linear Operators on Correlated Landscapes

By

PETER F. STADLER

Santa Fe Institute*

1660 Old Pecos Trail, Santa Fe, NM 87501, USA

Institute for Theoretical Chemistry, University of Vienna

WiihringerstraBe 17, A-lOgO Vienna, Austria

Email: studla@itc.univie.ac.at or stadler@santafe.edu

Address for Correspondence. Phone *(505)984-8800, Fax *(505)982-0565

PACS numbers: 0250,0550

P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES

Abstract

In this contribution we consider the effect of a class of "averaging op­

erators" on isotropic fitness landscapes. Explicit expressions for the cor­

relation function of the averaged landscapes are derived. A new class of

tuneably rugged landscapes, obtained by iterated smoothing of the random

energy model, is established. The correlation structure of certain landscapes,

among them the Sherrington-Kirkpatrick model, remains unchanged under

the action of all averaging operators considered here.

1. Introduction

Evolutionary adaptation as well as combinatorial optimization take place

on "landscapes" that result from mapping (micro)configurations to scalar

quantities like fitness values or energies, or, more generally, to nonscalar

objects like structures. The assumption of a particular mechanism for muta-

tion, or the choice of a specific move-set for interconverting configurations,

introduces the notions of neighborhood and distance between configurations.

Viewing the set of all configurations, i.e., the configuration space, as a non­

directed graph r has turned out to be particularly useful. Each configura­

tion is a vertex of r, and neighboring configurations are connected by edges.

When configurations are sequences of equal length and the neighborhood

relations are defined by the number of differing positions, one obtains the

sequence space with the Hamming metric.

Among the most important properties of a landscape is its "rugged­

ness", which influences the speed of evolutionary adaptation as well as the

quality of solutions of heuristic optimization strategies. While ruggedness

has never been precisely defined, a number of empirical measures have been

proposed, such as the number of local optima or the average length of up­

or downhill walks (Kauffman and Levin 1987, Weinberger 1991a), and the

-1-

P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES

pair-correlation as a function of distance in configuration space (Eigen et al.

1989, Weinberger 1990).

The very notion of "uphill" or "optimum" implies a mapping to a scalar

quantity. The definition of a pair correlation, however, can be extended to

mappings to arbitrary objects, as long as there is a metric distance mea­

sure defined for them. Extensive studies on folding of RNA sequences into

secondary structures have been based on this fact (Fontana et al. 1993ab).

The landscapes of a number of well known combinatorial optimization

problems, such as the Travelling Salesman Problem (Stadler and Schnabl

1992), the Graph Bipartitioning Problem (Stadler and Happel 1992), and the

Graph Matching Problem (Stadler 1992) have been investigated. Detailed

information on the distribution of local optima and the statistical character­

istics of downhill walks have been obtained for the uncorrelated landscape

of the random energy model (Macken et al. 1991, Flyvbjerg and Lautrup

1993, Bak et al. 1993). Furthermore, two one-parameter families of tuneably

rugged landscapes have been studied in detail: the Nk model and its variants

(Kauffman and Levin 1987, Kauffman et al. 1988, Weinberger 1991a, Fontana

et al. 1993a), and the p-spin models (Amitrano et al. 1989, Weinberger and

Stadler 1993).

Here we will be concerned with the effect of averaging procedures related

to running averages on landscapes. After some formal definitions (sect. 2)

we discuss the Fourier decomposition of landscapes (sect. 3) which is used in

sect. 4 to derive the basic properties of linear operators applied to landscapes.

In sect. 5 we briefly address the relation of averages over ensembles of land­

scapes and averages over configurations in single instances of landscapes. In

sect. 6 particular operators, namely weighted averages over one-step neigh­

borhoods, are used to construct a novel tuneable family of landscapes. In

sect. 7 the behaviour of p-spin and Nk landscapes under the iterated action

of these operators is investigated.

-2-

P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES

2. Some Definitions

Let r be a graph with N ::; 00 vertices. A graph automorphism a is a

one-to-one mapping of r onto itself such that the vertices a(x) and a(y) are

connected by an edge if and only if x and yare connected by an edge. Two

vertices of r are said to be equivalent if there is an automorphism a such that

y = a(x). A graph is vertex tran8itive if all vertices are equivalent. A vertex

transitive graph is always regular, i.e., each vertex has the same number of

neighbors. The distance d(x, y) of two vertices x, y E r is defined as the

minimum number of edges that separate them. This distance is a metric. A

graph is di8tance tran8itive if for any two pairs of vertices (x,y) and (u,v)

with d(x,y) = d(u,v) there is an automorphism a such that a(x) = u and

a(y) = v. (See e.g. Buckley and Harrary 1990). A graph r is defined by its

adjacency matrix A, with A xy = 1 if the vertices x and yare connected by

an edge, and Axy = 0 otherwise.

In the following we will only consider graphs that are at least vertex

transitive. Following Bollobas (1979) we may then partition the vertex set

by the following procedure: Choose an arbitrary vertex as reference point,

which in the following will be labelled O. Then construct pairwise disjoint

sets of vertices V(O) = {O}, V(I), V(2), ... , V(M) such that

aJl":= L Axy

xEV(Jl)

is independent of y E V(v). (1)

This partitioning is induced by the symmetry of r: the vertices in each of

the sets V(t<) are permutated by automorphisms that leave 0 invariant. In

the worst case each set contains a single vertex. In general we will use greek

letters to index the sets of such a partition, while roman indices are reserved

for the vertices of r.

-3-

P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES

For highly symmetric graphs the collapsed adjacency matrix A = (&I'v)

can be much smaller than the adjacency matrix A. In case of distance tran­

sitive graphs, for instance, the distance classes (i.e., the sets of all vertices

with common distance from the reference vertex 0) form a partition fulfilling

equ.(I), and the collapsed adjacency matrix A coincides with the intersection

matrix as defined by Biggs (1974) (see also Cvetkovic et al. 1980). We will

use d instead of greek letters for indexing the distance classes.

The following property of collapsed adjacency matrices will be used in

the subsequent sections.

Lemma 1. L (Am)xy = (Am)I'K for all x E V(p), and all non-negativeyEV(K)

intergers m, and hence for each polynomial p holds 2:YEV(K)[P(A)]xy =

[P(A)JI'K' i.e., peA) = peA).

Proof. We proceed inductively: The assertion is true by definition for m = 1.

For m = 0 it holds trivially. Now suppose it is true for m - 1; then

L (Am-IA)xy = L L L (Am-l)xzAzy =yEV(K) v zEV(v) yEV(K)

= L L (Am-I)xz LAzy = L(Am-I)I'VAVK = (Am)I'Kv zEV(v) yEV(K) v

The generalization to polynomials is straight forward. •Definition 2. A landscape! : r --4 IR. is an instance of a random field :F on

the set of vertices of r defined by the distribution function

P(yl, Yz,···, Yn) = Prob {!(Xi) ::; Vi, 1::; i ::; N}

where N is the number of vertices of r.

(2)

Let £[.] denote the mathematical expectation. For example, the ex­

pected value of the product of the values !(Xk) and !(Xl) at the vertices Xk

and Xl is

£[!(Xk)!(Xl)] = JYkyldP(YI,yZ, ... ,YN).

-4-

(3)

P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES

Definition 3. A landscape is isotropic if

(i) E[f(x)] =! for all configurations x E rj(ii) for any two pairs of configurations (x, y) and (u, v) such that there

is a graph automorphism a with a(x) = u and a(y) = v holds

E[f(x)f(y)] = E[f(u)f(v)].

The second condition means that any two pairs of configurations that are

equivalent in configuration space have the same pair-correlation. The covari-

ance matrix of :F is defined by

Cxy = E[f(x)f(y)] - E[f(x)]£[f(y)]· (4)

By vertex transitivity there is an automorphism a such that a(x) = O.

Isotropy hence implies Cxy = CO,a(y) = c(p,), where yep,) is the partition

to which a(y) belongs. The variance is given by

Var[f] = Cxx = Coo = c(O)

The autocorrelation function is defined as pcp,) = c(p,)jc(O).

3. Fourier Series on Landscapes

(5)

Let C be the covariance matrix of a random field :F on some graph r.Denote by {<p s} a complete set of orthonormal eigenvectors (which exists by

symmetry of C). Although {<Ps} can be chosen real, we will admit complex

vectors as well. Then the landscape f on r can be represented in mean

square sense as

f(x) = La(y)<py(x) (6)yEr

This series representation is known as the Karhunen-Loeve expansion.

Remark. The coefficients a(y) are uncorrelated. For finite sets the Karhunen­

Loeve expansion coincides with the well known principal component analysis

introduced by Hotelling (1933) (see, e.g., Rao 1973).

-5-

P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES

-- -------------------::;;-" -----.... -

..,/

0.8

0.6

0.4

0.2

,.// >:./ .

..._...... n=25--_. n=50--- n=100-- n=200

1.0 ~~~....,,-~~--.-,--~----=:::::.. _;::=_..;:;;.. :;:::~...~.....3............................

0.8

0.6

0.4

0.2

........... n=25n_. n=50--- n=100-- n=200

0.0 '::--~~~--..,~~~~~":"":-~~~~~,.--~~~~0.0 0.5 1.0 1.5 2.0

r/n

Figure 3: Nearest neighbor correlation for ISLs on generalized hypercubes as a function

of the number of iterated smoothings. a) K=2, b) K=4.

-16 -

P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES

7. Averaging Operators on Spin Glass and Nk Landscapes

We first consider the p-spin models, i.e., the long range spin glasses with

Hamiltonian

7-f.p (<T) = L Aj.j, ... jp <Tj, <Tj, .•. <Tjp

j. <j,<...<jp

(27)

with <Ti E {-I, +1} and coupling constants Aij ... k drawn from some common

distribution. The ruggedness of the landscapes increases as the order p of

the spin-interactions increases. We have then

Theorem 14. Let Q be any polynomial of the adjacency matrix of the

Boolean Hypercube. Then the landscapes 7-f.p and Q7-f.p have the same auto­

correlation functions.

Proof. As shown in Weinberger and Stadler (1993) the autocorrelation

functions of the p-spin models are

(28)

These are suitably normalized Krawtchouk polynomials, which are known

to be the eigenvectors of the collapsed adjacency matrix of the Boolean Hy­

percube (see, e.g., Dunkl 1979, Stadler 1993). Application of Corollary 11

completes the proof. •In Kauffman's Nk-model the fitness of a binary string x is defined as the

average over "site fitnesses" Ii, which depend on Xi and k additional positions

i 1 through ik. The site fitnesses Ii = Ii(Xi,Xi" ... ,Xi.) are assumed to be

uncorrelated random variables drawn from a common distribution (Kauffman

and Levin 1987). The ruggedness of the landscape increases as the number k

of positions contributing to a single site fitness increases. Different variants

of these landscapes haven been investigated, depending on which positions

contribute to a given site fitness.

-17 -

P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES

In contrast to the p-spin models, the landscapes of Nk type are affected

by the smoothing operator iii.

Theorem 15. Let f be an arbitrary Nk landscape. Then the autocorrelation

function ofthe landscapes ili r f converges to p(d) = ~(1-2d/n)+(1-0(_1)d

with a < ~ :::; 1 as r tends towards infinity.

Proof. It is necessary and sufficient to show that the autocorrelation func­

tion p of f and PI are not orthogonal, i.e., that

(29)

Note that PI(d) = -PI(n - d). Equ.(29) can hence be rewritten as

(30)

If p(d) ~ p(n - d) for all d < n/2, and if the inequality holds strictly for

at least one d < n/2, we have ~ > O. It follows immediately from the very

definition of the Nk model (irrespective of how the neighboring sites are

choosen) that p is monotonically decreasing and non-constant. •Remark. The variants of the Nk model do not behave identically under

the action of iii. Whenever the autocorrelation function is a polynomial of

degree less than n, then we have ~ = 1 in theorem 15. For exact expressions of

the autocorrelation functions of various Nk models see (Weinberger 1991a)

and (Fontana et al. 1993). Recently Nk-models on sequences of arbitrary

alphabets have been considered (Weinberger 1991a). Theorem 15 holds with

~ = 1 for all of them.

As a final example consider an isotropic landscape on the circle of length

n > 4 (Millonas, pers. communication). The configuration space hence is the

Cayley graph of the cyclic group en = {,k 10 :::; k < n} with set of generators

<P = b, ,-I}. Its vertex degree is D = 2. The eigenvectors of the adjacency

-18 -

P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES

matrix are ek(X) = expe",::X), as an immediate consequence of lemma 5.

The eigenvectors of the collapsed adjacency matrix are Pk(d) = cos( 2",::d) for

o::; k ::; n/2, associated with the eigenvalues Ak = 2 cos( 2~k). The eigenvalue

of 4r2 corresponding to An /2 is much smaller that the one corresponding to

Al for n > 4, hence P converges to PI under the iterated action of q;r whenever

P is not orhogonal to Pl.

Suppose f has a monotone non-increasing autocorrelation function p.

As a particular example consider the random energy model on the circle.

By the same reasoning as in the proof of theorem 15, P is not orthogonal to

PI, and hence iterated smoothing eventually leads to a landscape with auto­

correlation function Pl. Note that the smoothed landscapes on the Boolean

Hypercube (and on related graphs) eventually have autocorrelation func­

tions of the form p(d) = 1 - ~d + O(d2 ) while for the circle graphs we find

p(d) = 1 - ad2 + O(d3). The landscape on the circle hence becomes ex­

ceptionally smooth (Weinberger and Stadler 1993) as a consequence of the

special geometry of the configuration space.

This example also shows that the degeneracy of q;r on the Boolean Hy­

percube is not a necessary consequence of the fact that the configuration

space is bipartite, i.e., that there is an eigenvalue -D of the adjacency ma­

trix, since the circle graphs are bipartite for even n, and q;r is not degenerate

there.

Discussion

A general theory of the action of linear (averaging) operators on isotropic

landscapes is developed that allows to predict the changes in the correlation

structure of the landscape. A new family of tuneably rugged landscapes is

established, which arises by iterated smoothing of the random energy model.

It is shown that there are particular landscapes, characterized by auto­

correlation functions that are eigenvectors of the collapsed adjacency matrix

-19 -

P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES

of the underlying configuration space, that are not affected by any such

"smoothing" operators. On Boolean Hypercubes the p-spin models, and in

particular the Sherrington-Kirkpatrick model are members of this class. An­

other example is the landscape of the graph-bipartitioning problem.

The structure of a landscape is closely linked to the geometry of the

underlying configuration space. Iterated smoothing of the random energy

model leads to a landscape with linearly decreasing autocorrelation function

on generalized hypercubes with alphabets of size I', > 2, while on Boolean

Hypercubes (I', = 2) one obtaines a linear combination of a linear function

and (_l)d. On a circle graph the same procedure leads to an autocorrelation

function of the form 1 - ad2 + ....The theory is not applicable for the TSP problem for the following rea­

sons: Firstly, natural configuration spaces for the TSP are Cayley graphs

of the Symmetric group with the set of all transposition, or the set of all

inversions (2opt-moves), as generators. It can be easily shown that these

graphs are not distance transitive, and of course the symmetric group is non­

commutative. Secondly, the landscape of the TSP is not isotropic, e.g., on

the graph obtained for transpositions. There the correlation of two tours dif­

fering by an exchange of two neighboring cities is larger than the correlation

of two tours differing by any other transposition. For details see Stadler and

Schnabl (1992).

It should be kept in mind that the results presented here depend on the

assumption of isotropic landscapes. While this requirement is often fulfilled

for simple statistical models, it is unlikely to hold for real biophysical or

physical systems. It has been shown, for instance, that the landscapes of

RNA free energies are not isotropic for natural sequences (Stadler and Gruner

1993).

- 20-

P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES

Acknowledgements

I thank the Max Planck Society, Germany, for supporting my stay at the

Santa Fe Institute. Stimulating discussions with W. Fontana, M. Millonas,

and S.A. Kauffman are gratefully acknowledged.

References

[1] Amitrano C., L. Peliti, and M. Saber. Population Dynamics in Spin

Glass Model of Chemical Evolution. J. Mol. Evol. 29, 513-525 (1989).

[2] Bak P., H. Flyvbjerg, and B. Lautrup. Coevolution in a rugged fitness

landscape. Phys. Rev. A[lS} 46, 6724-6730 (1992).

[3] Biggs N. Algebraic Graph Theory. Cambridge University Press, Cam­

bridge, 1974.

[4] Biggs N.L. and A.T. White Permutation Groups and Combinatorial

Structures. Cambridge Univ. Press, Cambridge, 1979.

[5J Binder K. and A.P. Young. Spin Glasses: Experimental Facts, Theo­

retical Concepts, and Open Questions. Rev. Mod. Phys. 58, 801-976

(1986).

[6] Bollobas B. Graph Theory - An Introductory Course. Springer-Verlag,

New York, 1979.

[7] Buckley F. and F. Harary. Distance in Graphs. Addison-Wesley,

Reading, MA, 1990.

[7J Cvetkovic D.M., M. Doob, and H. Sachs. Spectra of Graphs - Theory

and Applications, Academic Press, New York, 1980.

[9] Dunkl C.F. Orthogonal Functions on Some Permutation Groups. In:

Ray-Chaudhuri D.K. (ed.) Proceeding of Symposis in Pure Mathe­

matic 34. American Mathematical Society, New York 1979.

[10] Eigen M., J.S. McCaskill, and P. Schuster. The Molecular Quasis­

pecies. Adv. Phys. Chem. 75, 149-263 (1989).

- 21-

P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES

[11] Flyvbjerg H. and B. Lautrup. Evolution in a rugged fitness landscape.

Phys. Rev. AI15] 46,6714-6723 (1992).

[12J Fontana W, P.F. Stadler, E.G. Bomberg-Bauer, T. Griesmacher, 1.L.

Hofacker, M. Tacker, P. Tarazona, E.D. Weinberger, and P. Schuster.

RNA Folding and Combinatory Maps Phys. Rev. E 47, 2083-2099

(1993a).

[13J Fontana W., D.A.M. Konings, P.F. Stadler, P. Schuster. Statistics of

RNA Secondary Structures. Biopolymers in press 1993b.

[14] Hotelling H. Analysis of a complex of statistical variables into principal

components. J. Educ. Psych. 24, 417-441 and 498-520 (1933).

[15] Kauffman S.A. and S. Levin. Towards a General Theory of Adaptive

Walks on Rugged Landscapes. J. Theor. Bioi. 128, 11-45 (1987).

[16] Kauffman S.A~, E.D. Weinberger, and S.A. Perelson. Maturation of

the Immune Response Via Adaptive Walks on Affinity Landscapes. In:

Theoretical Immunology, Part I, ed.: Alan Perelson. Addison-Wesley,

Redwood City, 1983.

[17] Lovasz L. Spectra of graphs with transitive groups. Periodica Math.

Hung. 6, 191-195 (1975).

[18] Macken C.A., P.S. Hagan, and A.S. Perelson. Evolutionary Walks on

Rugged Landscapes. SIAM J. Appl. Math. 51, 799-827 (1991).

[19J Rao C.R. Linear Statistical Interference and Its Applications. 2nd ed.,

Wiley, New York (1973).

[20] Rumschitzky D. Spectral Properties of Eigen's evolution matrices. J.

Math. Bio!. 24, 667-680 (1987).

[21] Sherrington D., and S. Kirkpatrick. Solvable Model of a Spin Glass.

Phys. Rev. Lett. 35, 402-416 (1975).

[22] Stadler P.F. and W. Schnabl. The Landscape of the Travelling Sales­

man Problem. Phys. Lett. 1,161, 337-344 (1992).

- 22-

P.F. STADLER: LINEAR OPERATORS AND LANDSCAPES

[23J Stadler P.F. and R. Happel. Correlation Structure of the Graph Bi­

partitioning Problem. J. Phys. A: Math. Gen. 25, 3103-3110 (1992).

[24] Stadler P.F. Correlation in Landscapes of Combinatorial Optimization

Problems Europhys. Lett. 20479-482 (1992).

[25] Stadler P.F. and W. Gruner. Anisotropy in Fitness Landscapes. J.

Theor. Bioi. in press 1993.

[26J Stadler P.F. Random Walks and Orthogonal Functions Associated

with Highly Symmetric Graphs. SFI preprint 93-06-31 (1993).

[27] Weinberger E.D. Correlated and Uncorrelated Fitness Landscapes and

How to Tell the Difference. Bioi. Cybern. 63, 325-336 (1990).

[28] Weinberger E.D. Local Properties of the Nk-Model, a Tuneably Rugged

Landscape. Phys. Rev. A 44, 6399-6813 (1991a).

[29] Weinberger E.D. Fourier and Taylor Series on Landscapes Bioi. Cy­

bern. 65, 321-330 (1991b).

[30] Weinberger E.D. and P.F. Stadler. Why Some Fitness Landscapes are

Fractal. J. Theor. Bioi. in press 1993.

- 23-