-
Learning, Generalization, and Functional Entropy in Random Automata Networks
Authors:
Alireza Goudarzi,
Christof Teuscher,
Natali Gulbahce,
Thimo Rohlf
Abstract:
It has been shown \citep{broeck90:physicalreview,patarnello87:europhys} that feedforward Boolean networks can learn to perform specific simple tasks and generalize well if only a subset of the learning examples is provided for learning. Here, we extend this body of work and show experimentally that random Boolean networks (RBNs), where both the interconnections and the Boolean transfer functions a…
▽ More
It has been shown \citep{broeck90:physicalreview,patarnello87:europhys} that feedforward Boolean networks can learn to perform specific simple tasks and generalize well if only a subset of the learning examples is provided for learning. Here, we extend this body of work and show experimentally that random Boolean networks (RBNs), where both the interconnections and the Boolean transfer functions are chosen at random initially, can be evolved by using a state-topology evolution to solve simple tasks. We measure the learning and generalization performance, investigate the influence of the average node connectivity $K$, the system size $N$, and introduce a new measure that allows to better describe the network's learning and generalization behavior. We show that the connectivity of the maximum entropy networks scales as a power-law of the system size $N$. Our results show that networks with higher average connectivity $K$ (supercritical) achieve higher memorization and partial generalization. However, near critical connectivity, the networks show a higher perfect generalization on the even-odd task.
△ Less
Submitted 25 June, 2013;
originally announced June 2013.
-
Emergent Criticality Through Adaptive Information Processing in Boolean Networks
Authors:
Alireza Goudarzi,
Christof Teuscher,
Natali Gulbahce,
Thimo Rohlf
Abstract:
We study information processing in populations of Boolean networks with evolving connectivity and systematically explore the interplay between the learning capability, robustness, the network topology, and the task complexity. We solve a long-standing open question and find computationally that, for large system sizes $N$, adaptive information processing drives the networks to a critical connectiv…
▽ More
We study information processing in populations of Boolean networks with evolving connectivity and systematically explore the interplay between the learning capability, robustness, the network topology, and the task complexity. We solve a long-standing open question and find computationally that, for large system sizes $N$, adaptive information processing drives the networks to a critical connectivity $K_{c}=2$. For finite size networks, the connectivity approaches the critical value with a power-law of the system size $N$. We show that network learning and generalization are optimized near criticality, given task complexity and the amount of information provided threshold values. Both random and evolved networks exhibit maximal topological diversity near $K_{c}$. We hypothesize that this supports efficient exploration and robustness of solutions. Also reflected in our observation is that the variance of the values is maximal in critical network populations. Finally, we discuss implications of our results for determining the optimal topology of adaptive dynamical networks that solve computational tasks.
△ Less
Submitted 26 April, 2012; v1 submitted 20 April, 2011;
originally announced April 2011.
-
Morphogenesis by coupled regulatory networks: Reliable control of positional information and proportion regulation
Authors:
Thimo Rohlf,
Stefan Bornholdt
Abstract:
Based on a non-equilibrium mechanism for spatial pattern formation we study how position information can be controlled by locally coupled discrete dynamical networks, similar to gene regulation networks of cells in a develo** multicellular organism. As an example we study the developmental problems of domain formation and proportion regulation in the presence of noise, as well as in the presen…
▽ More
Based on a non-equilibrium mechanism for spatial pattern formation we study how position information can be controlled by locally coupled discrete dynamical networks, similar to gene regulation networks of cells in a develo** multicellular organism. As an example we study the developmental problems of domain formation and proportion regulation in the presence of noise, as well as in the presence of cell flow. We find that networks that solve this task exhibit a hierarchical structure of information processing and are of similar complexity as developmental circuits of living cells. Proportion regulation is scalable with system size and leads to sharp, precisely localized boundaries of gene expression domains, even for large numbers of cells. A detailed analysis of noise-induced dynamics, using a mean-field approximation, shows that noise in gene expression states stabilizes (rather than disrupts) the spatial pattern in the presence of cell movements, both for stationary as well as growing systems. Finally, we discuss how this mechanism could be realized in the highly dynamic environment of growing tissues in multi-cellular organisms.
△ Less
Submitted 25 August, 2009;
originally announced August 2009.
-
Self-organized criticality and adaptation in discrete dynamical networks
Authors:
Thimo Rohlf,
Stefan Bornholdt
Abstract:
It has been proposed that adaptation in complex systems is optimized at the critical boundary between ordered and disordered dynamical regimes. Here, we review models of evolving dynamical networks that lead to self-organization of network topology based on a local coupling between a dynamical order parameter and rewiring of network connectivity, with convergence towards criticality in the limit…
▽ More
It has been proposed that adaptation in complex systems is optimized at the critical boundary between ordered and disordered dynamical regimes. Here, we review models of evolving dynamical networks that lead to self-organization of network topology based on a local coupling between a dynamical order parameter and rewiring of network connectivity, with convergence towards criticality in the limit of large network size $N$. In particular, two adaptive schemes are discussed and compared in the context of Boolean Networks and Threshold Networks: 1) Active nodes loose links, frozen nodes aquire new links, 2) Nodes with correlated activity connect, de-correlated nodes disconnect. These simple local adaptive rules lead to co-evolution of network topology and -dynamics. Adaptive networks are strikingly different from random networks: They evolve inhomogeneous topologies and broad plateaus of homeostatic regulation, dynamical activity exhibits $1/f$ noise and attractor periods obey a scale-free distribution. The proposed co-evolutionary mechanism of topological self-organization is robust against noise and does not depend on the details of dynamical transition rules. Using finite-size scaling, it is shown that networks converge to a self-organized critical state in the thermodynamic limit. Finally, we discuss open questions and directions for future research, and outline possible applications of these models to adaptive systems in diverse areas.
△ Less
Submitted 6 November, 2008;
originally announced November 2008.
-
Assessing Random Dynamical Network Architectures for Nanoelectronics
Authors:
Christof Teuscher,
Natali Gulbahce,
Thimo Rohlf
Abstract:
Independent of the technology, it is generally expected that future nanoscale devices will be built from vast numbers of densely arranged devices that exhibit high failure rates. Other than that, there is little consensus on what type of technology and computing architecture holds most promises to go far beyond today's top-down engineered silicon devices. Cellular automata (CA) have been propose…
▽ More
Independent of the technology, it is generally expected that future nanoscale devices will be built from vast numbers of densely arranged devices that exhibit high failure rates. Other than that, there is little consensus on what type of technology and computing architecture holds most promises to go far beyond today's top-down engineered silicon devices. Cellular automata (CA) have been proposed in the past as a possible class of architectures to the von Neumann computing architecture, which is not generally well suited for future parallel and fine-grained nanoscale electronics. While the top-down engineered semi-conducting technology favors regular and locally interconnected structures, future bottom-up self-assembled devices tend to have irregular structures because of the current lack precise control over these processes. In this paper, we will assess random dynamical networks, namely Random Boolean Networks (RBNs) and Random Threshold Networks (RTNs), as alternative computing architectures and models for future information processing devices. We will illustrate that--from a theoretical perspective--they offer superior properties over classical CA-based architectures, such as inherent robustness as the system scales up, more efficient information processing capabilities, and manufacturing benefits for bottom-up designed devices, which motivates this investigation. We will present recent results on the dynamic behavior and robustness of such random dynamical networks while also including manufacturing issues in the assessment.
△ Less
Submitted 17 May, 2008;
originally announced May 2008.
-
When correlations matter - response of dynamical networks to small perturbations
Authors:
Thimo Rohlf,
Natali Gulbahce,
Christof Teuscher
Abstract:
We systematically study and compare damage spreading for random Boolean and threshold networks under small external perturbations (damage), a problem which is relevant to many biological networks. We identify a new characteristic connectivity $K_s$, at which the average number of damaged nodes after a large number of dynamical updates is independent of the total number of nodes $N$. We estimate…
▽ More
We systematically study and compare damage spreading for random Boolean and threshold networks under small external perturbations (damage), a problem which is relevant to many biological networks. We identify a new characteristic connectivity $K_s$, at which the average number of damaged nodes after a large number of dynamical updates is independent of the total number of nodes $N$. We estimate the critical connectivity for finite $N$ and show that it systematically deviates from the annealed approximation. Extending the approach followed in a previous study, we present new results indicating that internal dynamical correlations tend to increase not only the probability for small, but also for very large damage events, leading to a broad, fat-tailed distribution of damage sizes. These findings indicate that the descriptive and predictive value of averaged order parameters for finite size networks - even for biologically highly relevant sizes up to several thousand nodes - is limited.
△ Less
Submitted 28 April, 2008;
originally announced April 2008.
-
Stochastic Network Model of Receptor Cross-Talk Predicts Anti-Angiogenic Effects
Authors:
Amy L. Bauer,
Trachette L. Jackson,
Yi Jiang,
Thimo Rohlf
Abstract:
Cancer invasion and metastasis depend on angiogenesis. The cellular processes (growth, migration, and apoptosis) that occur during angiogenesis are tightly regulated by signaling molecules. Thus, understanding how cells synthesize multiple biochemical signals initiated by key external stimuli can lead to the development of novel therapeutic strategies to combat cancer. In the face of large amoun…
▽ More
Cancer invasion and metastasis depend on angiogenesis. The cellular processes (growth, migration, and apoptosis) that occur during angiogenesis are tightly regulated by signaling molecules. Thus, understanding how cells synthesize multiple biochemical signals initiated by key external stimuli can lead to the development of novel therapeutic strategies to combat cancer. In the face of large amounts of disjoint experimental data generated from multitudes of laboratories using various assays, theoretical signal transduction models provide a framework to distill this vast amount of data. Such models offer an opportunity to formulate and test new hypotheses, and can be used to make experimentally verifiable predictions. This study is the first to propose a network model that highlights the cross-talk between the key receptors involved in angiogenesis, namely growth factor, integrin, and cadherin receptors. From available experimental data, we construct a stochastic Boolean network model of receptor cross-talk, and systematically analyze the dynamical stability of the network under continuous-time Boolean dynamics with a noisy production function. We find that the signal transduction network exhibits a robust and fast response to external signals, independent of the internal cell state. We derive an input-output table that maps external stimuli to cell phenotypes, which is extraordinarily stable against molecular noise with one important exception: an oscillatory feedback loop between the key signaling molecules RhoA and Rac1 is unstable under arbitrarily low noise, leading to erratic, dysfunctional cell motion. Finally, we show that the network exhibits an apoptotic response rate that increases with noise, suggesting that the probability of programmed cell death depends on cell health.
△ Less
Submitted 26 February, 2008;
originally announced February 2008.
-
Self-organization of heterogeneous topology and symmetry breaking in networks with adaptive thresholds and rewiring
Authors:
Thimo Rohlf
Abstract:
We study an evolutionary algorithm that locally adapts thresholds and wiring in Random Threshold Networks, based on measurements of a dynamical order parameter. A control parameter $p$ determines the probability of threshold adaptations vs. link rewiring. For any $p < 1$, we find spontaneous symmetry breaking into a new class of self-organized networks, characterized by a much higher average con…
▽ More
We study an evolutionary algorithm that locally adapts thresholds and wiring in Random Threshold Networks, based on measurements of a dynamical order parameter. A control parameter $p$ determines the probability of threshold adaptations vs. link rewiring. For any $p < 1$, we find spontaneous symmetry breaking into a new class of self-organized networks, characterized by a much higher average connectivity $\bar{K}_{evo}$ than networks without threshold adaptation ($p =1$). While $\bar{K}_{evo}$ and evolved out-degree distributions are independent from $p$ for $p <1$, in-degree distributions become broader when $p \to 1$, approaching a power-law. In this limit, time scale separation between threshold adaptions and rewiring also leads to strong correlations between thresholds and in-degree. Finally, evidence is presented that networks converge to self-organized criticality for large $N$.
△ Less
Submitted 12 August, 2007;
originally announced August 2007.
-
Critical Line in Random Threshold Networks with Inhomogeneous Thresholds
Authors:
Thimo Rohlf
Abstract:
We calculate analytically the critical connectivity $K_c$ of Random Threshold Networks (RTN) for homogeneous and inhomogeneous thresholds, and confirm the results by numerical simulations. We find a super-linear increase of $K_c$ with the (average) absolute threshold $|h|$, which approaches $K_c(|h|) \sim h^2/(2\ln{|h|})$ for large $|h|$, and show that this asymptotic scaling is universal for RT…
▽ More
We calculate analytically the critical connectivity $K_c$ of Random Threshold Networks (RTN) for homogeneous and inhomogeneous thresholds, and confirm the results by numerical simulations. We find a super-linear increase of $K_c$ with the (average) absolute threshold $|h|$, which approaches $K_c(|h|) \sim h^2/(2\ln{|h|})$ for large $|h|$, and show that this asymptotic scaling is universal for RTN with Poissonian distributed connectivity and threshold distributions with a variance that grows slower than $h^2$. Interestingly, we find that inhomogeneous distribution of thresholds leads to increased propagation of perturbations for sparsely connected networks, while for densely connected networks damage is reduced; the cross-over point yields a novel, characteristic connectivity $K_d$, that has no counterpart in Boolean networks. Last, local correlations between node thresholds and in-degree are introduced. Here, numerical simulations show that even weak (anti-)correlations can lead to a transition from ordered to chaotic dynamics, and vice versa. It is shown that the naive mean-field assumption typical for the annealed approximation leads to false predictions in this case, since correlations between thresholds and out-degree that emerge as a side-effect strongly modify damage propagation behavior.
△ Less
Submitted 21 February, 2008; v1 submitted 24 July, 2007;
originally announced July 2007.
-
Damage Spreading and Criticality in Finite Random Dynamical Networks
Authors:
Thimo Rohlf,
Natali Gulbahce,
Christof Teuscher
Abstract:
We systematically study and compare damage spreading at the sparse percolation (SP) limit for random boolean and threshold networks with perturbations that are independent of the network size $N$. This limit is relevant to information and damage propagation in many technological and natural networks. Using finite size scaling, we identify a new characteristic connectivity $K_s$, at which the ave…
▽ More
We systematically study and compare damage spreading at the sparse percolation (SP) limit for random boolean and threshold networks with perturbations that are independent of the network size $N$. This limit is relevant to information and damage propagation in many technological and natural networks. Using finite size scaling, we identify a new characteristic connectivity $K_s$, at which the average number of damaged nodes $\bar d$, after a large number of dynamical updates, is independent of $N$. Based on marginal damage spreading, we determine the critical connectivity $K_c^{sparse}(N)$ for finite $N$ at the SP limit and show that it systematically deviates from $K_c$, established by the annealed approximation, even for large system sizes. Our findings can potentially explain the results recently obtained for gene regulatory networks and have important implications for the evolution of dynamical networks that solve specific computational or functional tasks.
△ Less
Submitted 24 January, 2007;
originally announced January 2007.
-
Long-range memory elementary 1D cellular automata: Dynamics and nonextensivity
Authors:
Thimo Rohlf,
Constantino Tsallis
Abstract:
We numerically study the dynamics of elementary 1D cellular automata (CA), where the binary state $σ_i(t) \in \{0,1\}$ of a cell $i$ does not only depend on the states in its local neighborhood at time $t-1$, but also on the memory of its own past states $σ_i(t-2), σ_i(t-3),...,σ_i(t-τ),...$. We assume that the weight of this memory decays proportionally to $τ^{-α}$, with $α\ge 0$ (the limit…
▽ More
We numerically study the dynamics of elementary 1D cellular automata (CA), where the binary state $σ_i(t) \in \{0,1\}$ of a cell $i$ does not only depend on the states in its local neighborhood at time $t-1$, but also on the memory of its own past states $σ_i(t-2), σ_i(t-3),...,σ_i(t-τ),...$. We assume that the weight of this memory decays proportionally to $τ^{-α}$, with $α\ge 0$ (the limit $α\to \infty$ corresponds to the usual CA). Since the memory function is summable for $α>1$ and nonsummable for $0\le α\le 1$, we expect pronounced %qualitative and quantitative changes of the dynamical behavior near $α=1$. This is precisely what our simulations exhibit, particularly for the time evolution of the Hamming distance $H$ of initially close trajectories. We typically expect the asymptotic behavior $H(t) \propto t^{1/(1-q)}$, where $q$ is the entropic index associated with nonextensive statistical mechanics. In all cases, the function $q(α)$ exhibits a sensible change at $α\simeq 1$. We focus on the class II rules 61, 99 and 111. For rule 61, $q = 0$ for $0 \le α\le α_c \simeq 1.3$, and $q<0$ for $α> α_c$, whereas the opposite behavior is found for rule 111. For rule 99, the effect of the long-range memory on the spread of damage is quite dramatic. These facts point at a rich dynamics intimately linked to the interplay of local lookup rules and the range of the memory. Finite size scaling studies varying system size $N$ indicate that the range of the power-law regime for $H(t)$ typically diverges $\propto N^z$ with $0 \le z \le 1$. Similar studies have been carried out for other rules, e.g., the famous "universal computer" rule 110.
△ Less
Submitted 19 April, 2006;
originally announced April 2006.
-
Topological Evolution of Dynamical Networks: Global Criticality from Local Dynamics
Authors:
Stefan Bornholdt,
Thimo Rohlf
Abstract:
We evolve network topology of an asymmetrically connected threshold network by a simple local rewiring rule: quiet nodes grow links, active nodes lose links. This leads to convergence of the average connectivity of the network towards the critical value $K_c =2$ in the limit of large system size $N$. How this principle could generate self-organization in natural complex systems is discussed for…
▽ More
We evolve network topology of an asymmetrically connected threshold network by a simple local rewiring rule: quiet nodes grow links, active nodes lose links. This leads to convergence of the average connectivity of the network towards the critical value $K_c =2$ in the limit of large system size $N$. How this principle could generate self-organization in natural complex systems is discussed for two examples: neural networks and regulatory networks in the genome.
△ Less
Submitted 21 June, 2000; v1 submitted 13 March, 2000;
originally announced March 2000.
