Showing 1–27 of 27 results for author: Schwab, D

Noise driven phase transitions in eco-evolutionary systems

Authors: Jim Wu, David J. Schwab, Trevor GrandPre

Abstract: In complex ecosystems such as microbial communities, there is constant ecological and evolutionary feedback between the residing species and the environment occurring on concurrent timescales. Species respond and adapt to their surroundings by modifying their phenotypic traits, which in turn alters their environment and the resources available. To study this interplay between ecological and evolut… ▽ More In complex ecosystems such as microbial communities, there is constant ecological and evolutionary feedback between the residing species and the environment occurring on concurrent timescales. Species respond and adapt to their surroundings by modifying their phenotypic traits, which in turn alters their environment and the resources available. To study this interplay between ecological and evolutionary mechanisms, we develop a consumer-resource model that incorporates phenotypic mutations. In the absence of noise, we find that phase transitions require finely-tuned interaction kernels. Additionally, we quantify the effects of noise on frequency dependent selection by defining a time-integrated mutation current, which accounts for the rate at which mutations and speciation occurs. We find three distinct phases: homogeneous, patterned, and patterned traveling waves. The last phase represents one way in which co-evolution of species can happen in a fluctuating environment. Our results highlight the principal roles that noise and non-reciprocal interactions between resources and consumers play in phase transitions within eco-evolutionary systems. △ Less

Submitted 16 October, 2023; v1 submitted 12 October, 2023; originally announced October 2023.

Extrinsic vs Intrinsic Criticality in Systems with Many Components

Authors: Vudtiwat Ngampruetikorn, Ilya Nemenman, David J. Schwab

Abstract: Biological systems with many components often exhibit seemingly critical behaviors, characterized by atypically large correlated fluctuations. Yet the underlying causes remain unclear. Here we define and examine two types of criticality. Intrinsic criticality arises from interactions within the system which are fine-tuned to a critical point. Extrinsic criticality, in contrast, emerges without fin… ▽ More Biological systems with many components often exhibit seemingly critical behaviors, characterized by atypically large correlated fluctuations. Yet the underlying causes remain unclear. Here we define and examine two types of criticality. Intrinsic criticality arises from interactions within the system which are fine-tuned to a critical point. Extrinsic criticality, in contrast, emerges without fine tuning when observable degrees of freedom are coupled to unobserved fluctuating variables. We unify both types of criticality using the language of learning and information theory. We show that critical correlations, intrinsic or extrinsic, lead to diverging mutual information between two halves of the system, and are a feature of learning problems, in which the unobserved fluctuations are inferred from the observable degrees of freedom. We argue that extrinsic criticality is equivalent to standard inference, whereas intrinsic criticality describes fractional learning, in which the amount to be learned depends on the system size. We show further that both types of criticality are on the same continuum, connected by a smooth crossover. In addition, we investigate the observability of Zipf's law, a power-law rank-frequency distribution often used as an empirical signature of criticality. We find that Zipf's law is a robust feature of extrinsic criticality but can be nontrivial to observe for some intrinsically critical systems, including critical mean-field models. We further demonstrate that models with global dynamics, such as oscillatory models, can produce observable Zipf's law without relying on either external fluctuations or fine tuning. Our findings suggest that while possible in theory, fine tuning is not the only, nor the most likely, explanation for the apparent ubiquity of criticality in biological systems with many components. △ Less

Submitted 25 September, 2023; originally announced September 2023.

Comments: 13 pages, 9 figures

Generalized Information Bottleneck for Gaussian Variables

Authors: Vudtiwat Ngampruetikorn, David J. Schwab

Abstract: The information bottleneck (IB) method offers an attractive framework for understanding representation learning, however its applications are often limited by its computational intractability. Analytical characterization of the IB method is not only of practical interest, but it can also lead to new insights into learning phenomena. Here we consider a generalized IB problem, in which the mutual in… ▽ More The information bottleneck (IB) method offers an attractive framework for understanding representation learning, however its applications are often limited by its computational intractability. Analytical characterization of the IB method is not only of practical interest, but it can also lead to new insights into learning phenomena. Here we consider a generalized IB problem, in which the mutual information in the original IB method is replaced by correlation measures based on Renyi and Jeffreys divergences. We derive an exact analytical IB solution for the case of Gaussian correlated variables. Our analysis reveals a series of structural transitions, similar to those previously observed in the original IB case. We find further that although solving the original, Renyi and Jeffreys IB problems yields different representations in general, the structural transitions occur at the same critical tradeoff parameters, and the Renyi and Jeffreys IB solutions perform well under the original IB objective. Our results suggest that formulating the IB method with alternative correlation measures could offer a strategy for obtaining an approximate solution to the original IB problem. △ Less

Submitted 30 March, 2023; originally announced March 2023.

Comments: 7 pages, 3 figures

Emergence of local irreversibility in complex interacting systems

Authors: Christopher W. Lynn, Caroline M. Holmes, William Bialek, David J. Schwab

Abstract: Living systems are fundamentally irreversible, breaking detailed balance and establishing an arrow of time. But how does the evident arrow of time for a whole system arise from the interactions among its multiple elements? We show that the local evidence for the arrow of time, which is the entropy production for thermodynamic systems, can be decomposed. First, it can be split into two components:… ▽ More Living systems are fundamentally irreversible, breaking detailed balance and establishing an arrow of time. But how does the evident arrow of time for a whole system arise from the interactions among its multiple elements? We show that the local evidence for the arrow of time, which is the entropy production for thermodynamic systems, can be decomposed. First, it can be split into two components: an independent term reflecting the dynamics of individual elements and an interaction term driven by the dependencies among elements. Adapting tools from non--equilibrium physics, we further decompose the interaction term into contributions from pairs of elements, triplets, and higher--order terms. We illustrate our methods on models of cellular sensing and logical computations, as well as on patterns of neural activity in the retina as it responds to visual inputs. We find that neural activity can define the arrow of time even when the visual inputs do not, and that the dominant contribution to this breaking of detailed balance comes from interactions among pairs of neurons. △ Less

Submitted 3 June, 2022; v1 submitted 3 March, 2022; originally announced March 2022.

Comments: 16 pages, 10 figures

Decomposing the local arrow of time in interacting systems

Authors: Christopher W. Lynn, Caroline M. Holmes, William Bialek, David J. Schwab

Abstract: We show that the evidence for a local arrow of time, which is equivalent to the entropy production in thermodynamic systems, can be decomposed. In a system with many degrees of freedom, there is a term that arises from the irreversible dynamics of the individual variables, and then a series of non--negative terms contributed by correlations among pairs, triplets, and higher--order combinations of… ▽ More We show that the evidence for a local arrow of time, which is equivalent to the entropy production in thermodynamic systems, can be decomposed. In a system with many degrees of freedom, there is a term that arises from the irreversible dynamics of the individual variables, and then a series of non--negative terms contributed by correlations among pairs, triplets, and higher--order combinations of variables. We illustrate this decomposition on simple models of noisy logical computations, and then apply it to the analysis of patterns of neural activity in the retina as it responds to complex dynamic visual scenes. We find that neural activity breaks detailed balance even when the visual inputs do not, and that this irreversibility arises primarily from interactions between pairs of neurons. △ Less

Submitted 3 June, 2022; v1 submitted 29 December, 2021; originally announced December 2021.

Comments: 6 pages, 3 figures

Inferring couplings in networks across order-disorder phase transitions

Authors: Vudtiwat Ngampruetikorn, Vedant Sachdeva, Johanna Torrence, Jan Humplik, David J. Schwab, Stephanie E. Palmer

Abstract: Statistical inference is central to many scientific endeavors, yet how it works remains unresolved. Answering this requires a quantitative understanding of the intrinsic interplay between statistical models, inference methods and data structure. To this end, we characterize the efficacy of direct coupling analysis (DCA)--a highly successful method for analyzing amino acid sequence data--in inferri… ▽ More Statistical inference is central to many scientific endeavors, yet how it works remains unresolved. Answering this requires a quantitative understanding of the intrinsic interplay between statistical models, inference methods and data structure. To this end, we characterize the efficacy of direct coupling analysis (DCA)--a highly successful method for analyzing amino acid sequence data--in inferring pairwise interactions from samples of ferromagnetic Ising models on random graphs. Our approach allows for physically motivated exploration of qualitatively distinct data regimes separated by phase transitions. We show that inference quality depends strongly on the nature of generative models: optimal accuracy occurs at an intermediate temperature where the detrimental effects from macroscopic order and thermal noise are minimal. Importantly our results indicate that DCA does not always outperform its local-statistics-based predecessors; while DCA excels at low temperatures, it becomes inferior to simple correlation thresholding at virtually all temperatures when data are limited. Our findings offer new insights into the regime in which DCA operates so successfully and more broadly how inference interacts with data structure. △ Less

Submitted 25 August, 2021; v1 submitted 4 June, 2021; originally announced June 2021.

Comments: 10 pages, 7 figures

Journal ref: Phys. Rev. Research 4, 023240 (2022)

Understanding Species Abundance Distributions in Complex Ecosystems of Interacting Species

Authors: Jim Wu, Pankaj Mehta, David Schwab

Abstract: Niche and neutral theory are two prevailing, yet much debated, ideas in ecology proposed to explain the patterns of biodiversity. Whereas niche theory emphasizes selective differences between species and interspecific interactions in sha** the community, neutral theory supposes functional equivalence between species and points to stochasticity as the primary driver of ecological dynamics. In thi… ▽ More Niche and neutral theory are two prevailing, yet much debated, ideas in ecology proposed to explain the patterns of biodiversity. Whereas niche theory emphasizes selective differences between species and interspecific interactions in sha** the community, neutral theory supposes functional equivalence between species and points to stochasticity as the primary driver of ecological dynamics. In this work, we draw a bridge between these two opposing theories. Starting from a Lotka-Volterra (LV) model with demographic noise and random symmetric interactions, we analytically derive the stationary population statistics and species abundance distribution (SAD). Using these results, we demonstrate that the model can exhibit three classes of SADs commonly found in niche and neutral theories and found conditions that allow an ecosystem to transition between these various regimes. Thus, we reconcile how neutral-like statistics may arise from a diverse community with niche differentiation. △ Less

Submitted 4 March, 2021; v1 submitted 2 March, 2021; originally announced March 2021.

Superlinear Precision and Memory in Simple Population Codes

Authors: Jimmy H. J. Kim, Ila Fiete, David J. Schwab

Abstract: The brain constructs population codes to represent stimuli through widely distributed patterns of activity across neurons. An important figure of merit of population codes is how much information about the original stimulus can be decoded from them. Fisher information is widely used to quantify coding precision and specify optimal codes, because of its relationship to mean squared error (MSE) unde… ▽ More The brain constructs population codes to represent stimuli through widely distributed patterns of activity across neurons. An important figure of merit of population codes is how much information about the original stimulus can be decoded from them. Fisher information is widely used to quantify coding precision and specify optimal codes, because of its relationship to mean squared error (MSE) under certain assumptions. When neural firing is sparse, however, optimizing Fisher information can result in codes that are highly sub-optimal in terms of MSE. We find that this discrepancy arises from the non-local component of error not accounted for by the Fisher information. Using this insight, we construct optimal population codes by directly minimizing the MSE. We study the scaling properties of MSE with coding parameters, focusing on the tuning curve width. We find that the optimal tuning curve width for coding no longer scales as the inverse population size, and the quadratic scaling of precision with system size predicted by Fisher information alone no longer holds. However, superlinearity is still preserved with only a logarithmic slowdown. We derive analogous results for networks storing the memory of a stimulus through continuous attractor dynamics, and show that similar scaling properties optimize memory and representation. △ Less

Submitted 2 August, 2020; originally announced August 2020.

Comments: 5 pages, 4 figures

Theory of gating in recurrent neural networks

Authors: Kamesh Krishnamurthy, Tankut Can, David J. Schwab

Abstract: Recurrent neural networks (RNNs) are powerful dynamical models, widely used in machine learning (ML) and neuroscience. Prior theoretical work has focused on RNNs with additive interactions. However, gating - i.e. multiplicative - interactions are ubiquitous in real neurons and also the central feature of the best-performing RNNs in ML. Here, we show that gating offers flexible control of two salie… ▽ More Recurrent neural networks (RNNs) are powerful dynamical models, widely used in machine learning (ML) and neuroscience. Prior theoretical work has focused on RNNs with additive interactions. However, gating - i.e. multiplicative - interactions are ubiquitous in real neurons and also the central feature of the best-performing RNNs in ML. Here, we show that gating offers flexible control of two salient features of the collective dynamics: i) timescales and ii) dimensionality. The gate controlling timescales leads to a novel, marginally stable state, where the network functions as a flexible integrator. Unlike previous approaches, gating permits this important function without parameter fine-tuning or special symmetries. Gates also provide a flexible, context-dependent mechanism to reset the memory trace, thus complementing the memory function. The gate modulating the dimensionality can induce a novel, discontinuous chaotic transition, where inputs push a stable system to strong chaotic activity, in contrast to the typically stabilizing effect of inputs. At this transition, unlike additive RNNs, the proliferation of critical points (topological complexity) is decoupled from the appearance of chaotic dynamics (dynamical complexity). The rich dynamics are summarized in phase diagrams, thus providing a map for principled parameter initialization choices to ML practitioners. △ Less

Submitted 1 December, 2021; v1 submitted 29 July, 2020; originally announced July 2020.

Comments: 13 figures

Non-equilibrium statistical mechanics of continuous attractors

Authors: Weishun Zhong, Zhiyue Lu, David J Schwab, Arvind Murugan

Abstract: Continuous attractors have been used to understand recent neuroscience experiments where persistent activity patterns encode internal representations of external attributes like head direction or spatial location. However, the conditions under which the emergent bump of neural activity in such networks can be manipulated by space and time-dependent external sensory or motor signals are not underst… ▽ More Continuous attractors have been used to understand recent neuroscience experiments where persistent activity patterns encode internal representations of external attributes like head direction or spatial location. However, the conditions under which the emergent bump of neural activity in such networks can be manipulated by space and time-dependent external sensory or motor signals are not understood. Here, we find fundamental limits on how rapidly internal representations encoded along continuous attractors can be updated by an external signal. We apply these results to place cell networks to derive a velocity-dependent non-equilibrium memory capacity in neural networks. △ Less

Submitted 30 December, 2018; v1 submitted 28 September, 2018; originally announced September 2018.

Comments: 17 pages

Coordination of size-control, reproduction and generational memory in freshwater planarians

Authors: Xingbo Yang, Kelson J. Kaj, David J. Schwab, Eva-Maria S. Collins

Abstract: Uncovering the mechanisms that control size, growth, and division rates of systems reproducing through binary division means understanding basic principles of their life cycle. Recent work has focused on how division rates are regulated in bacteria and yeast, but this question has not yet been addressed in more complex, multicellular organisms. We have acquired a unique large-scale data set on the… ▽ More Uncovering the mechanisms that control size, growth, and division rates of systems reproducing through binary division means understanding basic principles of their life cycle. Recent work has focused on how division rates are regulated in bacteria and yeast, but this question has not yet been addressed in more complex, multicellular organisms. We have acquired a unique large-scale data set on the growth and asexual reproduction of two freshwater planarian species, Dugesia japonica and Dugesia tigrina, which reproduce by transverse fission and succeeding regeneration of head and tail pieces into new planarians. We show that generation-dependent memory effects in planarian reproduction need to be taken into account to accurately capture the experimental data. To achieve this, we developed a new additive model that mixes multiple size control strategies based on planarian size, growth, and time between divisions. Our model quantifies the proportions of each strategy in the mixed dynamics, revealing the ability of the two planarian species to utilize different strategies in a coordinated manner for size control. Additionally, we found that head and tail offspring of both species employ different mechanisms to monitor and trigger their reproduction cycles. Thus, we find a diversity of strategies not only between species but also between heads and tails within species. Our additive model provides two advantages over existing 2D models that fit a multivariable splitting rate function to the data for size control: Firstly, it can be fit to relatively small data sets and can thus be applied to systems where available data is limited. Secondly, it enables new biological insights because it explicitly shows the contributions of different size control strategies for each offspring type. △ Less

Submitted 14 March, 2017; originally announced March 2017.

Comments: 26 pages, 10 figures

Associative pattern recognition through macro-molecular self-assembly

Authors: Weishun Zhong, David J. Schwab, Arvind Murugan

Abstract: We show that macro-molecular self-assembly can recognize and classify high-dimensional patterns in the concentrations of $N$ distinct molecular species. Similar to associative neural networks, the recognition here leverages dynamical attractors to recognize and reconstruct partially corrupted patterns. Traditional parameters of pattern recognition theory, such as sparsity, fidelity, and capacity a… ▽ More We show that macro-molecular self-assembly can recognize and classify high-dimensional patterns in the concentrations of $N$ distinct molecular species. Similar to associative neural networks, the recognition here leverages dynamical attractors to recognize and reconstruct partially corrupted patterns. Traditional parameters of pattern recognition theory, such as sparsity, fidelity, and capacity are related to physical parameters, such as nucleation barriers, interaction range, and non-equilibrium assembly forces. Notably, we find that self-assembly bears greater similarity to continuous attractor neural networks, such as place cell networks that store spatial memories, rather than discrete memory networks. This relationship suggests that features and trade-offs seen here are not tied to details of self-assembly or neural network models but are instead intrinsic to associative pattern recognition carried out through short-ranged interactions. △ Less

Submitted 24 February, 2017; v1 submitted 6 January, 2017; originally announced January 2017.

Comments: 13 pages, 7 figures; reference added

The deterministic information bottleneck

Authors: DJ Strouse, David J Schwab

Abstract: Lossy compression and clustering fundamentally involve a decision about what features are relevant and which are not. The information bottleneck method (IB) by Tishby, Pereira, and Bialek formalized this notion as an information-theoretic optimization problem and proposed an optimal tradeoff between throwing away as many bits as possible, and selectively kee** those that are most important. In t… ▽ More Lossy compression and clustering fundamentally involve a decision about what features are relevant and which are not. The information bottleneck method (IB) by Tishby, Pereira, and Bialek formalized this notion as an information-theoretic optimization problem and proposed an optimal tradeoff between throwing away as many bits as possible, and selectively kee** those that are most important. In the IB, compression is measure my mutual information. Here, we introduce an alternative formulation that replaces mutual information with entropy, which we call the deterministic information bottleneck (DIB), that we argue better captures this notion of compression. As suggested by its name, the solution to the DIB problem turns out to be a deterministic encoder, or hard clustering, as opposed to the stochastic encoder, or soft clustering, that is optimal under the IB. We compare the IB and DIB on synthetic data, showing that the IB and DIB perform similarly in terms of the IB cost function, but that the DIB significantly outperforms the IB in terms of the DIB cost function. We also empirically find that the DIB offers a considerable gain in computational efficiency over the IB, over a range of convergence parameters. Our derivation of the DIB also suggests a method for continuously interpolating between the soft clustering of the IB and the hard clustering of the DIB. △ Less

Submitted 19 December, 2016; v1 submitted 1 April, 2016; originally announced April 2016.

Comments: 15 pages, 4 figures

Landauer in the age of synthetic biology: energy consumption and information processing in biochemical networks

Authors: Pankaj Mehta, Alex H. Lang, David J. Schwab

Abstract: A central goal of synthetic biology is to design sophisticated synthetic cellular circuits that can perform complex computations and information processing tasks in response to specific inputs. The tremendous advances in our ability to understand and manipulate cellular information processing networks raises several fundamental physics questions: How do the molecular components of cellular circuit… ▽ More A central goal of synthetic biology is to design sophisticated synthetic cellular circuits that can perform complex computations and information processing tasks in response to specific inputs. The tremendous advances in our ability to understand and manipulate cellular information processing networks raises several fundamental physics questions: How do the molecular components of cellular circuits exploit energy consumption to improve information processing? Can one utilize ideas from thermodynamics to improve the design of synthetic cellular circuits and modules? Here, we summarize recent theoretical work addressing these questions. Energy consumption in cellular circuits serves five basic purposes: (1) increasing specificity, (2) manipulating dynamics, (3) reducing variability, (4) amplifying signal, and (5) erasing memory. We demonstrate these ideas using several simple examples and discuss the implications of these theoretical ideas for the emerging field of synthetic biology. We conclude by discussing how it may be possible to overcome these limitations using "post-translational" synthetic biology that exploits reversible protein modification. △ Less

Submitted 10 May, 2015; originally announced May 2015.

Comments: 9 pages, 4 figures

Multiscale modeling of oscillations and spiral waves in Dictyostelium populations

Authors: Javad Noorbakhsh, David Schwab, Allyson Sgro, Thomas Gregor, Pankaj Mehta

Abstract: Unicellular organisms exhibit elaborate collective behaviors in response to environmental cues. These behaviors are controlled by complex biochemical networks within individual cells and coordinated through cell-to-cell communication. Describing these behaviors requires new mathematical models that can bridge scales -- from biochemical networks within individual cells to spatially structured cellu… ▽ More Unicellular organisms exhibit elaborate collective behaviors in response to environmental cues. These behaviors are controlled by complex biochemical networks within individual cells and coordinated through cell-to-cell communication. Describing these behaviors requires new mathematical models that can bridge scales -- from biochemical networks within individual cells to spatially structured cellular populations. Here, we present a family of multiscale models for the emergence of spiral waves in the social amoeba Dictyostelium discoideum. Our models exploit new experimental advances that allow for the direct measurement and manipulation of the small signaling molecule cAMP used by Dictyostelium cells to coordinate behavior in cellular populations. Inspired by recent experiments, we model the Dictyostelium signaling network as an excitable system coupled to various pre-processing modules. We use this family of models to study spatially unstructured populations by constructing phase diagrams that relate the properties of population-level oscillations to parameters in the underlying biochemical network. We then extend our models to include spatial structure and show how they naturally give rise to spiral waves. Our models exhibit a wide range of novel phenomena including a density dependent frequency change, bistability, and dynamic death due to slow cAMP dynamics. Our modeling approach provides a powerful tool for bridging scales in modeling of Dictyostelium populations. △ Less

Submitted 12 September, 2014; v1 submitted 30 July, 2014; originally announced July 2014.

Journal ref: Phys. Rev. E 91, 062711 (2015)

A binary Hopfield network with $1/\log(n)$ information rate and applications to grid cell decoding

Authors: Ila Fiete, David J. Schwab, Ngoc M. Tran

Abstract: A Hopfield network is an auto-associative, distributive model of neural memory storage and retrieval. A form of error-correcting code, the Hopfield network can learn a set of patterns as stable points of the network dynamic, and retrieve them from noisy inputs -- thus Hopfield networks are their own decoders. Unlike in coding theory, where the information rate of a good code (in the Shannon sense)… ▽ More A Hopfield network is an auto-associative, distributive model of neural memory storage and retrieval. A form of error-correcting code, the Hopfield network can learn a set of patterns as stable points of the network dynamic, and retrieve them from noisy inputs -- thus Hopfield networks are their own decoders. Unlike in coding theory, where the information rate of a good code (in the Shannon sense) is finite but the cost of decoding does not play a role in the rate, the information rate of Hopfield networks trained with state-of-the-art learning algorithms is of the order ${\log(n)}/{n}$, a quantity that tends to zero asymptotically with $n$, the number of neurons in the network. For specially constructed networks, the best information rate currently achieved is of order ${1}/{\sqrt{n}}$. In this work, we design simple binary Hopfield networks that have asymptotically vanishing error rates at an information rate of ${1}/{\log(n)}$. These networks can be added as the decoders of any neural code with noisy neurons. As an example, we apply our network to a binary neural decoder of the grid cell code to attain information rate ${1}/{\log(n)}$. △ Less

Submitted 22 July, 2014; originally announced July 2014.

Comments: extended abstract, 4 pages, 2 figures

From Intracellular Signaling to Population Oscillations: Bridging Scales in Collective Behavior

Authors: Allyson E. Sgro, David J. Schwab, Javad Noorbakhsh, Troy Mestler, Pankaj Mehta, Thomas Gregor

Abstract: Collective behavior in cellular populations is coordinated by biochemical signaling networks within individual cells. Connecting the dynamics of these intracellular networks to the population phenomena they control poses a considerable challenge because of network complexity and our limited knowledge of kinetic parameters. However, from physical systems we know that behavioral changes in the indiv… ▽ More Collective behavior in cellular populations is coordinated by biochemical signaling networks within individual cells. Connecting the dynamics of these intracellular networks to the population phenomena they control poses a considerable challenge because of network complexity and our limited knowledge of kinetic parameters. However, from physical systems we know that behavioral changes in the individual constituents of a collectively-behaving system occur in a limited number of well-defined classes, and these can be described using simple models. Here we apply such an approach to the emergence of collective oscillations in cellular populations of the social amoeba Dictyostelium discoideum. Through direct tests of our model with quantitative in vivo measurements of single-cell and population signaling dynamics, we show how a simple model can effectively describe a complex molecular signaling network and its effects at multiple size and temporal scales. The model predicts novel noise-driven single-cell and population-level signaling phenomena that we then experimentally observe. Our results suggest that like physical systems, collective behavior in biology may be universal and described using simple mathematical models. △ Less

Submitted 25 June, 2014; originally announced June 2014.

Zipf's law and criticality in multivariate data without fine-tuning

Authors: David J. Schwab, Ilya Nemenman, Pankaj Mehta

Abstract: The joint probability distribution of many degrees of freedom in biological systems, such as firing patterns in neural networks or antibody sequence composition in zebrafish, often follow Zipf's law, where a power law is observed on a rank-frequency plot. This behavior has recently been shown to imply that these systems reside near to a unique critical point where the extensive parts of the entrop… ▽ More The joint probability distribution of many degrees of freedom in biological systems, such as firing patterns in neural networks or antibody sequence composition in zebrafish, often follow Zipf's law, where a power law is observed on a rank-frequency plot. This behavior has recently been shown to imply that these systems reside near to a unique critical point where the extensive parts of the entropy and energy are exactly equal. Here we show analytically, and via numerical simulations, that Zipf-like probability distributions arise naturally if there is an unobserved variable (or variables) that affects the system, e. g. for neural networks an input stimulus that causes individual neurons in the network to fire at time-varying rates. In statistics and machine learning, these models are called latent-variable or mixture models. Our model shows that no fine-tuning is required, i.e. Zipf's law arises generically without tuning parameters to a point, and gives insight into the ubiquity of Zipf's law in a wide range of systems. △ Less

Submitted 18 June, 2014; v1 submitted 1 October, 2013; originally announced October 2013.

Comments: 5 pages, 3 figures

Journal ref: Phys. Rev. Lett. 113, 068102 (2014)

Quantifying the role of population subdivision in evolution on rugged fitness landscapes

Authors: Anne-Florence Bitbol, David J. Schwab

Abstract: Natural selection drives populations towards higher fitness, but crossing fitness valleys or plateaus may facilitate progress up a rugged fitness landscape involving epistasis. We investigate quantitatively the effect of subdividing an asexual population on the time it takes to cross a fitness valley or plateau. We focus on a generic and minimal model that includes only population subdivision into… ▽ More Natural selection drives populations towards higher fitness, but crossing fitness valleys or plateaus may facilitate progress up a rugged fitness landscape involving epistasis. We investigate quantitatively the effect of subdividing an asexual population on the time it takes to cross a fitness valley or plateau. We focus on a generic and minimal model that includes only population subdivision into equivalent demes connected by global migration, and does not require significant size changes of the demes, environmental heterogeneity or specific geographic structure. We determine the optimal speedup of valley or plateau crossing that can be gained by subdivision, if the process is driven by the deme that crosses fastest. We show that isolated demes have to be in the sequential fixation regime for subdivision to significantly accelerate crossing. Using Markov chain theory, we obtain analytical expressions for the conditions under which optimal speedup is achieved: valley or plateau crossing by the subdivided population is then as fast as that of its fastest deme. We verify our analytical predictions through stochastic simulations. We demonstrate that subdivision can substantially accelerate the crossing of fitness valleys and plateaus in a wide range of parameters extending beyond the optimal window. We study the effect of varying the degree of subdivision of a population, and investigate the trade-off between the magnitude of the optimal speedup and the width of the parameter range over which it occurs. Our results also hold for weakly beneficial intermediate mutations. We extend our work to the case of a population connected by migration to one or several smaller islands. Our results demonstrate that subdivision with migration alone can significantly accelerate the crossing of fitness valleys and plateaus, and shed light onto the quantitative conditions necessary for this to occur. △ Less

Submitted 14 August, 2014; v1 submitted 1 August, 2013; originally announced August 2013.

Comments: 27 pages, 4 figures, published version

Journal ref: PLoS Computational Biology 10(8): e1003778 (2014)

The Energetic Costs of Cellular Computation

Authors: Pankaj Mehta, David J. Schwab

Abstract: Cells often perform computations in response to environmental cues. A simple example is the classic problem, first considered by Berg and Purcell, of determining the concentration of a chemical ligand in the surrounding media. On general theoretical grounds (Landuer's Principle), it is expected that such computations require cells to consume energy. Here, we explicitly calculate the energetic cost… ▽ More Cells often perform computations in response to environmental cues. A simple example is the classic problem, first considered by Berg and Purcell, of determining the concentration of a chemical ligand in the surrounding media. On general theoretical grounds (Landuer's Principle), it is expected that such computations require cells to consume energy. Here, we explicitly calculate the energetic costs of computing ligand concentration for a simple two-component cellular network that implements a noisy version of the Berg-Purcell strategy. We show that learning about external concentrations necessitates the breaking of detailed balance and consumption of energy, with greater learning requiring more energy. Our calculations suggest that the energetic costs of cellular computation may be an important constraint on networks designed to function in resource poor environments such as the spore germination networks of bacteria. △ Less

Submitted 10 April, 2012; v1 submitted 24 March, 2012; originally announced March 2012.

Comments: 9 Pages (including Appendix); 4 Figures; v3 corrects even more typos

Dynamical quorum-sensing and synchronization of nonlinear oscillators coupled through an external medium

Authors: David J. Schwab, Ania Baetica, Pankaj Mehta

Abstract: Many biological and physical systems exhibit population-density dependent transitions to synchronized oscillations in a process often termed "dynamical quorum sensing". Synchronization frequently arises through chemical communication via signaling molecules distributed through an external media. We study a simple theoretical model for dynamical quorum sensing: a heterogenous population of limit-cy… ▽ More Many biological and physical systems exhibit population-density dependent transitions to synchronized oscillations in a process often termed "dynamical quorum sensing". Synchronization frequently arises through chemical communication via signaling molecules distributed through an external media. We study a simple theoretical model for dynamical quorum sensing: a heterogenous population of limit-cycle oscillators diffusively coupled through a common media. We show that this model exhibits a rich phase diagram with four qualitatively distinct mechanisms fueling population-dependent transitions to global oscillations, including a new type of transition we term "dynamic death". We derive a single pair of analytic equations that allows us to calculate all phase boundaries as a function of population density and show that the model reproduces many of the qualitative features of recent experiments of BZ catalytic particles as well as synthetically engineered bacteria. △ Less

Submitted 21 December, 2010; originally announced December 2010.

Comments: 17 pages including Supporting Information, 3 figures, submitted to Physical Review Letters

Statistical mechanics of transcription-factor binding site discovery using Hidden Markov Models

Authors: Pankaj Mehta, David Schwab, Anirvan M. Sengupta

Abstract: Hidden Markov Models (HMMs) are a commonly used tool for inference of transcription factor (TF) binding sites from DNA sequence data. We exploit the mathematical equivalence between HMMs for TF binding and the "inverse" statistical mechanics of hard rods in a one-dimensional disordered potential to investigate learning in HMMs. We derive analytic expressions for the Fisher information, a commonly… ▽ More Hidden Markov Models (HMMs) are a commonly used tool for inference of transcription factor (TF) binding sites from DNA sequence data. We exploit the mathematical equivalence between HMMs for TF binding and the "inverse" statistical mechanics of hard rods in a one-dimensional disordered potential to investigate learning in HMMs. We derive analytic expressions for the Fisher information, a commonly employed measure of confidence in learned parameters, in the biologically relevant limit where the density of binding sites is low. We then use techniques from statistical mechanics to derive a scaling principle relating the specificity (binding energy) of a TF to the minimum amount of training data necessary to learn it. △ Less

Submitted 27 October, 2010; v1 submitted 18 August, 2010; originally announced August 2010.

Comments: 25 pages, 2 figures, 1 table V2 - typos fixed and new references added

Statistical Mechanics of Integral Membrane Protein Assembly

Authors: Karim Wahba, David J. Schwab, Robijn Bruinsma

Abstract: During the synthesis of integral membrane proteins (IMPs), the hydrophobic amino acids of the polypeptide sequence are partitioned mostly into the membrane interior and hydrophilic amino acids mostly into the aqueous exterior. We analyze the minimum free energy state of polypeptide sequences partitioned into alpha-helical transmembrane (TM) segments and the role of thermal fluctuations using a m… ▽ More During the synthesis of integral membrane proteins (IMPs), the hydrophobic amino acids of the polypeptide sequence are partitioned mostly into the membrane interior and hydrophilic amino acids mostly into the aqueous exterior. We analyze the minimum free energy state of polypeptide sequences partitioned into alpha-helical transmembrane (TM) segments and the role of thermal fluctuations using a many-body statistical mechanics model. Results suggest that IMP TM segment partitioning shares important features with general theories of protein folding. For random polypeptide sequences, the minimum free energy state at room temperature is characterized by fluctuations in the number of TM segments with very long relaxation times. Simple assembly scenarios do not produce a unique number of TM segments and jamming phenomena interfere with segment placement. For sequences corresponding to IMPs, the minimum free energy structure with the wildtype number of segments is free of number fluctuations due to an anomalous gap in the energy spectrum, and simple assembly scenarios produce this structure. There is a threshold number of random point mutations beyond which the size of this gap is reduced so that the wildtype groundstate is destabilized and number fluctuations reappear. △ Less

Submitted 4 August, 2009; originally announced August 2009.

Rhythmogenic neuronal networks, pacemakers, and k-cores

Authors: David J. Schwab, Robijn F. Bruinsma, Alex J. Levine

Abstract: Neuronal networks are controlled by a combination of the dynamics of individual neurons and the connectivity of the network that links them together. We study a minimal model of the preBotzinger complex, a small neuronal network that controls the breathing rhythm of mammals through periodic firing bursts. We show that the properties of a such a randomly connected network of identical excitatory… ▽ More Neuronal networks are controlled by a combination of the dynamics of individual neurons and the connectivity of the network that links them together. We study a minimal model of the preBotzinger complex, a small neuronal network that controls the breathing rhythm of mammals through periodic firing bursts. We show that the properties of a such a randomly connected network of identical excitatory neurons are fundamentally different from those of uniformly connected neuronal networks as described by mean-field theory. We show that (i) the connectivity properties of the networks determines the location of emergent pacemakers that trigger the firing bursts and (ii) that the collective desensitization that terminates the firing bursts is determined again by the network connectivity, through k-core clusters of neurons. △ Less

Submitted 5 December, 2008; originally announced December 2008.

Comments: 4+ pages, 4 figures, submitted to Phys. Rev. Lett

How many species have mass M?

Authors: Aaron Clauset, David J. Schwab, Sidney Redner

Abstract: Within large taxonomic assemblages, the number of species with adult body mass M is characterized by a broad but asymmetric distribution, with the largest mass being orders of magnitude larger than the typical mass. This canonical shape can be explained by cladogenetic diffusion that is bounded below by a hard limit on viable species mass and above by extinction risks that increase weakly with m… ▽ More Within large taxonomic assemblages, the number of species with adult body mass M is characterized by a broad but asymmetric distribution, with the largest mass being orders of magnitude larger than the typical mass. This canonical shape can be explained by cladogenetic diffusion that is bounded below by a hard limit on viable species mass and above by extinction risks that increase weakly with mass. Here we introduce and analytically solve a simplified cladogenetic diffusion model. When appropriately parameterized, the diffusion-reaction equation predicts mass distributions that are in good agreement with data on 4002 terrestrial mammal from the late Quaternary and 8617 extant bird species. Under this model, we show that a specific tradeoff between the strength of within-lineage drift toward larger masses (Cope's rule) and the increased risk of extinction from increased mass is necessary to produce realistic mass distributions for both taxa. We then make several predictions about the evolution of avian species masses. △ Less

Submitted 25 August, 2008; originally announced August 2008.

Comments: 7 pages, 3 figures

Journal ref: American Naturalist 173, 256-263 (2009)

Nucleosome Switching

Authors: David J. Schwab, Robijn F. Bruinsma, Joseph Rudnick, Jonathan Widom

Abstract: We present a statistical-mechanical analysis of the positioning of nucleosomes along one of the chromosomes of yeast DNA as a function of the strength of the binding potential and of the chemical potential of the nucleosomes. We find a significant density of two-level nucleosome switching regions where, as a function of the chemical potential, the nucleosome distribution undergoes a "micro" firs… ▽ More We present a statistical-mechanical analysis of the positioning of nucleosomes along one of the chromosomes of yeast DNA as a function of the strength of the binding potential and of the chemical potential of the nucleosomes. We find a significant density of two-level nucleosome switching regions where, as a function of the chemical potential, the nucleosome distribution undergoes a "micro" first-order transition. The location of these nucleosome switches shows a strong correlation with the location of transcription-factor binding sites. △ Less

Submitted 6 December, 2007; originally announced December 2007.

Comments: 15 pages, 3 figures

Endogenous versus Exogenous Origins of Diseases

Authors: D. Sornette, V. I. Yukalov, E. P. Yukalova, J. -Y. Henry, D. Schwab, J. P. Cobb

Abstract: Many illnesses are associated with an alteration of the immune system homeostasis due to any combination of factors, including exogenous bacterial insult, endogenous breakdown (e.g., development of a disease that results in immuno suppression), or an exogenous hit like surgery that simultaneously alters immune responsiveness and provides access to bacteria, or genetic disorder. We conjecture tha… ▽ More Many illnesses are associated with an alteration of the immune system homeostasis due to any combination of factors, including exogenous bacterial insult, endogenous breakdown (e.g., development of a disease that results in immuno suppression), or an exogenous hit like surgery that simultaneously alters immune responsiveness and provides access to bacteria, or genetic disorder. We conjecture that, as a consequence of the co-evolution of the immune system of individuals with the ecology of pathogens, the homeostasis of the immune system requires the influx of pathogens. This allows the immune system to keep the ever present pathogens under control and to react and adjust fast to bursts of infections. We construct the simplest and most general system of rate equations which describes the dynamics of five compartments: healthy cells, altered cells, adaptive and innate immune cells, and pathogens. We study four regimes obtained with or without auto-immune disorder and with or without spontaneous proliferation of infected cells. Over all regimes, we find that seven different states are naturally described by the model: (i) strong healthy immune system, (ii) healthy organism with evanescent immune cells, (iii) chronic infections, (iv) strong infections, (v) cancer, (vi) critically ill state and (vii) death. The analysis of stability conditions demonstrates that these seven states depend on the balance between the robustness of the immune system and the influx of pathogens. △ Less

Submitted 8 June, 2009; v1 submitted 20 October, 2007; originally announced October 2007.

Comments: 33 pages + 11 figures: final published version

Journal ref: Journal of Biological Systems 17 (2), 225-267 (2009)