Canalyzation in mathematical modeling

Boolean networksCanalyzing functions

Nested canalyzing functionsPartially NCFs

Canalyzation in mathematical modeling

Elena Dimitrova

Clemson University

KnowEscape 2014

Elena Dimitrova Canalyzation in mathematical modeling



Boolean networks

Canalyzing functionsDefinitionsExamples and applications

Nested canalyzing functionsDefinitionProperties of NCFsStructureNCFs and unate cascade functionsOther results

Partially NCFsDefinitionsDepth and stabilityDerrida plotsConclusions




Networks




What is a Boolean network?




What is a Boolean function?

I Boolean function (or switching function) is a function of theform f : Bn → B, where B = {0, 1}, n is nonnegative integer.

I Boolean functions are often written using the logical operatorsAND (∧), OR (∨), and NOT (¬).




Boolean function: Example

f (x , y) = x ∧ ¬y = x(y + 1)

x y f

0 0 00 1 01 0 11 1 0

Facts

I Every Boolean function can be represented as a polynomialover F2.

I Every function f : Fn2 → F2 can be represented as a Boolean

function.




What is a Boolean network?

I F = (f1, . . . , fn) : Bn → Bn is a Boolean network (BN) on nvariables if fi is a Boolean function in n variables for everyi = 1, . . . , n.

I BNs are used for modeling networks in which the nodeactivity/state can be described as a binary value: on-off,active-non active, 1-0, etc.




Boolean network: Example

f1(x1, x2) = x1 ∧ ¬x2

f2(x1, x2) = x1 ∨ x2




Applicatioins of Boolean networks

Boolean network models have been used to examine theconnections among diverse physical and engineered networks

I Signal transduction networks [Kauffman, Shmulevich et al.,Helikar et al., Kochi et al.]

I Biological networks [Klemm et al., Raeymaekers]

I Social networks [Flache et al., Green et al., Moreira et al.]

I Economic/prediction market networks [Jumadinova et al.]

I Neural networks [Huepe et al.]

I Complex networks in general [Wolfram]




Boolean networks: Biology

Segment polarity network in D. melanogasterR. Albert & H. Othmer: J. Theor. Bio. 223 (2003), 1–18.




Boolean model of segment polarity network

R. Albert & H. Othmer: J. Theor. Bio. 223 (2003), 1–18.




Boolean networks: Electric circuits




Boolean networks: Sociology

I Databases like Citebase harvest pre- and post-prints fromvarious archives for different fields. Nodes stand for publishedarticles and a directed edge represents a reference to apreviously published article.

I The International Movie Database is analyzed in manystudies. An actor collaboration network can be reconstructed,in which actors casted jointly are assumed to know each other.

I The phenomenon of rhythms of social interactions was studiedusing Boolean networks. A clear weekend pattern wasdiscovered while studying messaging within a massive onlinefriendship site providing insight to the social life of the users.(www.hpl.hp.com/research/idlpapers/facebook)




Variations on Boolean networks

I (Random/Probabilistic) Boolean networks (S. Kauffman)

I Polynomial dynamical systems (T. Thomas, D. Thieffry, R.Laubenbacher, B. Stigler, B. Pareigis, A. Jarrah)

I (Nested) canalyzing functions (S. Kauffman, I. Shmulevich)

I Cellular automata (von Neumann, J. Conway, S. Wolfram)

I Graphical models (Barret, Mortveit, Reidys)

I Logical models (R. Thomas)

I Petri nets (Google ‘Petri Nets World’)

I Biologically meaningful functions (L. Raeymaekers)

I ...




Question

Are all Boolean functions good for modeling??




DefinitionsExamples and applications

Canalyzing functions

Canalyzing function

A Boolean function f (x1, . . . , xn) is canalyzing if there exists anindex i and a, b ∈ {0, 1} such that

f (x1, . . . , xi−1, a, xi+1, . . . , xn) = b.

That is, when xi is given the input value a, f evaluates to bregardless of the input values of the other variables. Here a iscalled the canalyzing value and b is called the canalyzed value.





Examples: Canalyzing function

I The AND function x ∧ y is canalyzing in x . Since 0 ∧ y = 0for any input value of y , 0 is a canalyzing value for x withcanalyzed output value 0.

I XOR(x , y) := (x ∨ y) ∧ ¬(x ∧ y) is not canalyzing in eithervariable.





Canalyzing functions: role and applications

I Introduced by Kauffman [Kau93] as appropriate rules inBoolean network models of gene regulatory networks. Thedefinition is reminiscent of the concept of “canalisation”introduced by the geneticist C.H. Waddington [Wad42] torepresent the ability of a genotype to produce the samephenotype regardless of environmental variability.

I Play an important role in the study of random Booleannetworks [Kau93, Lyn95, Sta87, Ste99].

I Have been used extensively as models for dynamical systems,such as gene regulatory networks [Kau93], evolution [Ste99],and chaos [Lyn95].





But what if the input is NOT the canalyzing value??




DefinitionProperties of NCFsStructureNCFs and unate cascade functionsOther results

Nested Canalyzing Functions

Let f (x1, . . . , xn) be a Boolean function. For σ ∈ Sn, f is a nestedcanalyzing function (NCF) in the variable order xσ(1), . . . , xσ(n)with canalyzing values a1, . . . , an and canalyzed values b1, . . . , bn ifit can be expressed in the form

f =

b1 if xσ(1) = a1

b2 if xσ(1) 6= a1 and xσ(2) = a2

b3 if xσ(1) 6= a1 and xσ(2) 6= a2 and xσ(3) = a3

......

bn if xσ(1) 6= a1 and · · · and xσ(n−1) 6= an−1 and xσ(n) = an¬bn if xσ(1) 6= a1 and · · · and xσ(n) 6= an





Examples: NCFs

I f (x1, x2, x3) = x2 ∨ (¬x1 ∧ x3) is a NCF with b = [1, 0, 0],a = [1, 1, 0], and σ = [2, 1, 3]:

f (x1, x2, x3) =

1 if x2 = 10 if x2 = 0 and x1 = 10 if x2 = 0, x1 = 0, and x3 = 01 if x2 = 0, x1 = 0, and x3 = 1.

I f (x , y , z) = x(y − 1)z is a NCF.

I g(x , y , z ,w) = x(y + z) is not a NCF.





Properties of NCFs

I Introduced by Kauffman et al. in [KPST03].

I [KPST04] showed that networks made from NCFs have stabledynamic behavior and might be a good class of functions toexpress regulatory relationships in biochemical networks.

I Nested (hierarchically) canalyzing functions show orderedbehavior [NFW07].

I Systems of NCFs have a smaller average cycle length andaverage height (number of time steps it takes to converge toan attractor) of the state space graph compare to generalBoolean networks.





Representation of an NCF [JRL07]

Let yi = xσ(i) + ai + bi , 1 ≤ i ≤ n and let

g(x1, . . . , xn) = y1♦1(y2♦2(. . . (yn−1♦n−1yn) . . .),

where

♦i =

{∨ if bi = 1∧ if bi = 0

and ai , bi ∈ {0, 1} for 1 ≤ i ≤ n. Then g is an NCF in the variableorder xσ(1), . . . , xσ(n) with canalyzing values a1, . . . , an andcanalyzed values b1, . . . , bn. Further, any NCF can be representedin this form.





NCFs as a toric variety

I The ring of Boolean functions is isomorphic to the ringR = F2[x1, . . . , xn]/〈x2

i − xi : 1 ≤ i ≤ n〉.I Any polynomial in R can be represented as an point in F2n

2

corresponding to its coefficients.

I For σ ∈ Sn, the set of points corresponding to NCFs in thevariable order xσ(1), . . . xσ(n) form an algebraic variety VNCF

σ .

I VNCF = ∪σ∈SnVNCFσ is an algebraic variety as well, the

variety of nested canalyzing functions.

I VNCF is a toric variety [JL07].





Binary decision diagrams

I A binary decision diagram (BDD) is a directed, acyclic graphthat corresponds to the evaluation of a Boolean function.

I The average path length of a BDD is the length of the pathsfrom the source node to a terminal node averaged over allpossible inputs. Here, 2+2+3+3+3+3

8 = 2.





Unate cascade functions

I The unate cascade functions (UCFs) make up the unique classof Boolean functions with the smallest average path length ona BDD [Mat05].

I This smallest average path length is 2− 12n−1 where n is the

number of variables.

I Thus UCFs can be evaluated more quickly than any otherBoolean function.

I The classes of UCFs and NCFs are identical [JRL07].





BDD of a UCF/NCF

Recall f (x1, x2, x3) = x2 ∨ (¬x1 ∧ x3) is a NCF with b = [1, 0, 0],a = [1, 1, 0], and σ = [2, 1, 3]:

Average path length = 3+3+2+18 = 1.125.





Number of NCFs

I There is a recursive formula for the number of UCFs.

I Since UCFs are identical to NCFs, the formula can be used forcounting the number of NCFs in n variables, NCF (n):

NCF (n) = 2 · E (n), where E (1) = 1,E (2) = 4 and, for n ≥ 3,

E (n) =n−1∑r=2

(n

r − 1

)· 2r−1 · E (n − r + 1) + 2n.





Other results on NCFs

I NCFs were generalized to an arbitrary field by D. Murrugarra[Tom10] who also showed that some features of the dynamicsof these functions is similar to their Boolean counterparts.

I A.S. Jarrah and F. Hinkelmann developed an algorithm thatidentifies all systems of NCFs that fit a given data set [JH10].




DefinitionsDepth and stabilityDerrida plotsConclusions

I Recall NCFs:

f =

b1 if xσ(1) = a1

b2 if xσ(1) 6= a1 and xσ(2) = a2

b3 if xσ(1) 6= a1 and xσ(2) 6= a2 and xσ(3) = a3

......

bn if xσ(1) 6= a1 and · · · and xσ(n−1) 6= an−1 and xσ(n) = an¬bn if xσ(1) 6= a1 and · · · and xσ(n) 6= an

I Quite restrictive!





Nested canalyzing depth

The nested canalyzing depth of a Boolean function f is the largestindex σ(i) such that the variables xσ(1), . . . , xσ(i) exhibit the nestedcanalyzing structure in the definition of a NCF.Remarks:

I We make the convention that constant functions are NCFsand have depth n.

I If a function has i nested canalyzing variables and thefunction in the remaining n − i variables is constant, thisfunction also has depth n.

I Functions with no canalyzing variables have depth zero.





Example: Nested canalyzing depth

I x2 is canalyzing with canalyzing input 0 and output 0.I σ(1) = 2 and b1 = a1 = 0; σ(2) = 3, b2 = 1, and a2 = 0.I The remaining function in x1 and x4 is neither canalyzing nor

constant, so f has depth 2.





Activity

I Kauffman and Shmulevich explored the influence of thefunction’s variables on its output [SK04].

I The partial derivative of a function f (x1, . . . , xn) with respect

to the variable xj is defined as ∂f (x)∂xj

= f (xj ,0)⊕ f (xj ,1). Here,

xj ,i = (x1, . . . , xj−1, i , xj+1, . . . , xn) and ⊕ denotes the XORfunction.

I Activity of a variable: The average of the partial derivatives ofa variable over all possible inputs, αf

j = 12k

∑x∈{0,1}k

∂f (x)∂xj

.





Sensitivity

I The sensitivity of a function quantifies the sensitivity of theoutput to variations in the function inputs.

I It is given by s f (x) =∑k

i=1 χ [f (x⊕ ei ) 6= f (x)] , where eidenotes the ith unit vector and χ is an indicator function.

I Essentially computing the number of inputs with Hammingdistance one from an input x that gives a different functionoutput than x.

I The average sensitivity of a function, s f , is the expected valueof s f (x), s f = E

[s f (x)

]=∑k

i=1 αfi .





Sensitivity

I Shmulevich and Kauffman suggest that networks created usingfunctions that are less sensitive will be more “dynamicallyordered” (stable) than those with higher sensitivity.

I They show in [SK04] that a random Boolean function in nvariables has average sensitivity n

2 .

I Also, for a Boolean function with depth at least one, theexpected activities of the variables (x1, . . . , xn) are given by(

12 ,

14 ,

14 , . . . ,

14

), and hence the average sensitivity is n+1

4 .





Depth and sensitivity

I When reverse engineering gene networks, a nested canalyzingstructure may apply to some of the variables, while thebehavior of the remaining variables may be ambiguous orunknown.

I We say that a function has depth at least d if d of thevariables are known to be nested canalyzing, while theremaining n − d variables can take on any Boolean function,canalyzing or otherwise.

I We’ll show that the larger the value of d is, the less sensitivethe function will be on average.






Extending Kauffman and Shmuleviche’s result to functions of depth atleast d , we have the following result.

Result (Layne, D., Macauley [LDM12])

Let fd be a Boolean function in n variables with nested canalyzing depthat least d . Renumbering the variables if necessary, assume that x1, . . . , xdare the nested canalyzing variables with σ(i) = i . Then the expectedactivities of the variables (x1, . . . , xn) are given by

E[αfd]

=

(1

2,

1

4, . . . ,

1

2d,

1

2d+1, . . . ,

1

2d+1

).

Further, the average sensitivity of fd is

E [s fd ] =n − d

2d+1+

d∑i=1

1

2d=

n − d

2d+1+ 1− 1

2d.






Now, we see that

E [s fd ]− E [s fd+1 ] = 1− 1

2d+

n − d

2d+1−[

1− 1

2d+1+

n − d − 1

2d+2

]=

n − d − 1

2d+2,

which rapidly goes to zero, so each subsequent canalyzing variablehas a much smaller impact on the sensitivity.

I Thus, the difference in sensitivity between fully nestedcanalyzing functions and partially nested canalyzingfunctions of sufficient depth is very slight.





Moral

Using strictly NCFs for modeling when stability is a desirableproperty is an overkill. PNCFs of sufficient depth are good enough.





Derrida plots

x1(t) and x2(t): two states in random Boolean network.ρ(t) = 1

n ||x1(t)− x2(t)||1 (normalized Hamming distance), where

|| · ||1 is the standard `1 metric.

A Derrida curve is a plot of ρ(t + 1) vs. ρ(t) arranged uniformlyover different states and networks [DP86].





Phases

Consider the curve for small values of ρ(t).

I Frozen phase: Curve lies below y = x . The phase space ofsuch networks consists of many fixed points and smallattractor cycles.

I Chaotic phase: Curve lies above y = x . Small perturbationspropagate throughout the network.

I Critical phase: The boundary threshold between the frozenand chaotic phases.

There is evidence that many biological and social networks lie inthe critical phase, as they must be stable enough to endurechanges in the environment, yet flexible enough to adapt whennecessary [BABC+08, NPA+08, NPL+08, SKA05].





Derrida curves

Derrida curves for random Boolean networks with n = 100 nodes and k = 12 inputs per function.





Observations

I Networks of larger depth show more orderly dynamics thanthose of smaller depth.

I The curves move closer together as depth increases, i.e.,marginal benefit of stability as depth increases.

I Observations from Derrida plots match theoretical results.





Conclusions

I PNCFs are a reasonable generalization of NCFs.

I Stability increases as depth increases, however the marginalgain in stability drops off quickly.

I Just a few degrees of canalyzation are necessary to drop thenetwork into critical regime.





Thank you!





[BABC+08] E. Balleza, E. Alvarez-Buylla, A. Chaos, S.A. Kauffman, I. Shmulevich, and M. Aldana.Critical dynamics in genetic regulatory networks: Examples from four kingdoms.PLoS ONE, 3:e2456, 2008.

[DP86] B. Derrida and Y. Pomeau.Random networks of automata: a simple annealed approximation.Europhys. Lett., 1:45–49, 1986.

[JH10] Abdul Salam Jarrah and Franziska Hinkelmann.Inferring biologically relevant models: Nested canalyzing functions.In Algebraic and Numeric Biology 2010, 2010.

[JL07] Abdul Salam Jarrah and Reinhard C. Laubenbacher.Discrete models of biochemical networks: The toric variety of nested canalyzing functions.In AB, pages 15–22, 2007.

[JRL07] Abdul Salam Jarrah, Blessilda Raposa, and Reinhard Laubenbacher.Nested canalyzing, unate cascade, and polynomial functions.Physica D. Nonlinear phenomena, 233:167–174, 2007.

[Kau93] Stuart Kauffman.The Origins of Order: Self-Organization and Selection in Evolution.Oxford University Press, 1993.

[KPST03] Stuart Kauffman, Carsten Peterson, Bjorn Samuelsson, and Carl Troein.Random boolean network models and the yeast transcriptional network.PNAS, 100(25):14796–14799, 2003.

[KPST04] Stuart Kauffman, Carsten Peterson, Bjorn Samuelsson, and Carl Troein.Genetic networks with canalyzing boolean rules are always stable.Proceedings of the National Academy of Sciences of the United States of America,101(49):17102–17107, December 2004.





[LDM12] L. Layne, E.S. Dimitrova, and M. Macauley.Nested canalyzing depth and network stability.Bulletin of Mathematical Biology, 74:422–433, 2012.

[Lyn95] J. F. Lynch.On the threshold of chaos in random boolean cellular automata.Random Struct. Algorithms, 6(2–3):239–260, 1995.

[Mat05] Munehiro Matsuura.Average path length of binary decision diagrams.IEEE Trans. Comput., 54(9):1041–1053, 2005.Fellow-Butler, Jon T. and Fellow-Sasao, Tsutomu.

[NFW07] S. Nikolajewa, M. Friedel, and T. Wilhelm.Boolean networks with biologically relevant rules show ordered behavior.BioSystems, 90:40–47, 2007.

[NPA+08] M. Nykter, N.D. Price, M. Aldana, S.A. Ramsey, S.A. Kauffman, L.E. Hood, O. Yli-Harja, andI. Shmulevich.Gene expression dynamics in the macrophage exhibit criticality.Proc. Natl. Acad. Sci., 150:1897–1900, 2008.

[NPL+08] M. Nykter, N.D. Price, A. Larjo, T. Aho, S.A. Kauffman, O. Yli-Harja, and I. Shmulevich.Critical networks exhibit maximal information diversity in structure-dynamics relationships.Phys. Rev. Lett., 100:058702, 2008.

[SK04] I. Shmulevich and S.A. Kauffman.Activities and sensitivities in boolean network models.Phys Rev Lett, 93(4):048701, 2004.





[SKA05] I. Shmulevich, S.A. Kauffman, and M. Aldana.Eukaryotic cells are dynamically ordered or critical but not chaotic.Proc. Natl. Acad. Sci., 102:13439–13444, 2005.

[Sta87] D. Stauffer.On forcing functions in kauffmans random boolean networks.Journal of Statistical Physics, 46(3–4):789794, 1987.

[Ste99] M. D. Stern.Emergence of homeostasis and noise imprinting in an evolution model.PNAS, 96(19):10746–10751, 1999.

[Tom10] D. M. Tomairo.Multi-states nested canlyzing functions.Preprint, 2010.

[Wad42] C. H. Waddington.Canalisation of development and the inheritance of acquired characters.Nature, 150:563–564, 1942.


Canalyzation in mathematical modeling

Science

Transcript of Canalyzation in mathematical modeling