Showing 1–48 of 48 results for author: Girvan, M

Hybridizing Traditional and Next-Generation Reservoir Computing to Accurately and Efficiently Forecast Dynamical Systems

Authors: Ravi Chepuri, Dael Amzalag, Thomas Antonsen Jr., Michelle Girvan

Abstract: Reservoir computers (RCs) are powerful machine learning architectures for time series prediction. Recently, next generation reservoir computers (NGRCs) have been introduced, offering distinct advantages over RCs, such as reduced computational expense and lower training data requirements. However, NGRCs have their own practical difficulties, including sensitivity to sampling time and type of nonlin… ▽ More Reservoir computers (RCs) are powerful machine learning architectures for time series prediction. Recently, next generation reservoir computers (NGRCs) have been introduced, offering distinct advantages over RCs, such as reduced computational expense and lower training data requirements. However, NGRCs have their own practical difficulties, including sensitivity to sampling time and type of nonlinearities in the data. Here, we introduce a hybrid RC-NGRC approach for time series forecasting of dynamical systems. We show that our hybrid approach can produce accurate short term predictions and capture the long term statistics of chaotic dynamical systems in situations where the RC and NGRC components alone are insufficient, e.g., due to constraints from limited computational resources, sub-optimal hyperparameters, sparsely-sampled training data, etc. Under these conditions, we show for multiple model chaotic systems that the hybrid RC-NGRC method with a small reservoir can achieve prediction performance approaching that of a traditional RC with a much larger reservoir, illustrating that the hybrid approach can offer significant gains in computational efficiency over traditional RCs while simultaneously addressing some of the limitations of NGRCs. Our results suggest that hybrid RC-NGRC approach may be particularly beneficial in cases when computational efficiency is a high priority and an NGRC alone is not adequate. △ Less

Submitted 5 June, 2024; v1 submitted 4 March, 2024; originally announced March 2024.

Comments: 12 pages, 7 figures

Journal ref: Chaos 1 June 2024; 34 (6): 063114

Stabilizing Machine Learning Prediction of Dynamics: Noise and Noise-inspired Regularization

Authors: Alexander Wikner, Joseph Harvey, Michelle Girvan, Brian R. Hunt, Andrew Pomerance, Thomas Antonsen, Edward Ott

Abstract: Recent work has shown that machine learning (ML) models can be trained to accurately forecast the dynamics of unknown chaotic dynamical systems. Short-term predictions of the state evolution and long-term predictions of the statistical patterns of the dynamics (``climate'') can be produced by employing a feedback loop, whereby the model is trained to predict forward one time step, then the model o… ▽ More Recent work has shown that machine learning (ML) models can be trained to accurately forecast the dynamics of unknown chaotic dynamical systems. Short-term predictions of the state evolution and long-term predictions of the statistical patterns of the dynamics (``climate'') can be produced by employing a feedback loop, whereby the model is trained to predict forward one time step, then the model output is used as input for multiple time steps. In the absence of mitigating techniques, however, this technique can result in artificially rapid error growth. In this article, we systematically examine the technique of adding noise to the ML model input during training to promote stability and improve prediction accuracy. Furthermore, we introduce Linearized Multi-Noise Training (LMNT), a regularization technique that deterministically approximates the effect of many small, independent noise realizations added to the model input during training. Our case study uses reservoir computing, a machine-learning method using recurrent neural networks, to predict the spatiotemporal chaotic Kuramoto-Sivashinsky equation. We find that reservoir computers trained with noise or with LMNT produce climate predictions that appear to be indefinitely stable and have a climate very similar to the true system, while reservoir computers trained without regularization are unstable. Compared with other regularization techniques that yield stability in some cases, we find that both short-term and climate predictions from reservoir computers trained with noise or with LMNT are substantially more accurate. Finally, we show that the deterministic aspect of our LMNT regularization facilitates fast hyperparameter tuning when compared to training with noise. △ Less

Submitted 12 December, 2022; v1 submitted 9 November, 2022; originally announced November 2022.

Comments: 39 pages, 8 figures, 5 tables

A Meta-learning Approach to Reservoir Computing: Time Series Prediction with Limited Data

Authors: Daniel Canaday, Andrew Pomerance, Michelle Girvan

Abstract: Recent research has established the effectiveness of machine learning for data-driven prediction of the future evolution of unknown dynamical systems, including chaotic systems. However, these approaches require large amounts of measured time series data from the process to be predicted. When only limited data is available, forecasters are forced to impose significant model structure that may or m… ▽ More Recent research has established the effectiveness of machine learning for data-driven prediction of the future evolution of unknown dynamical systems, including chaotic systems. However, these approaches require large amounts of measured time series data from the process to be predicted. When only limited data is available, forecasters are forced to impose significant model structure that may or may not accurately represent the process of interest. In this work, we present a Meta-learning Approach to Reservoir Computing (MARC), a data-driven approach to automatically extract an appropriate model structure from experimentally observed "related" processes that can be used to vastly reduce the amount of data required to successfully train a predictive model. We demonstrate our approach on a simple benchmark problem, where it beats the state of the art meta-learning techniques, as well as a challenging chaotic problem. △ Less

Submitted 7 October, 2021; originally announced October 2021.

Parallel Machine Learning for Forecasting the Dynamics of Complex Networks

Authors: Keshav Srinivasan, Nolan Coble, Joy Hamlin, Thomas Antonsen, Edward Ott, Michelle Girvan

Abstract: Forecasting the dynamics of large complex networks from previous time-series data is important in a wide range of contexts. Here we present a machine learning scheme for this task using a parallel architecture that mimics the topology of the network of interest. We demonstrate the utility and scalability of our method implemented using reservoir computing on a chaotic network of oscillators. Two l… ▽ More Forecasting the dynamics of large complex networks from previous time-series data is important in a wide range of contexts. Here we present a machine learning scheme for this task using a parallel architecture that mimics the topology of the network of interest. We demonstrate the utility and scalability of our method implemented using reservoir computing on a chaotic network of oscillators. Two levels of prior knowledge are considered: (i) the network links are known; and (ii) the network links are unknown and inferred via a data-driven approach to approximately optimize prediction. △ Less

Submitted 27 August, 2021; originally announced August 2021.

Using Data Assimilation to Train a Hybrid Forecast System that Combines Machine-Learning and Knowledge-Based Components

Authors: Alexander Wikner, Jaideep Pathak, Brian R. Hunt, Istvan Szunyogh, Michelle Girvan, Edward Ott

Abstract: We consider the problem of data-assisted forecasting of chaotic dynamical systems when the available data is in the form of noisy partial measurements of the past and present state of the dynamical system. Recently there have been several promising data-driven approaches to forecasting of chaotic dynamical systems using machine learning. Particularly promising among these are hybrid approaches tha… ▽ More We consider the problem of data-assisted forecasting of chaotic dynamical systems when the available data is in the form of noisy partial measurements of the past and present state of the dynamical system. Recently there have been several promising data-driven approaches to forecasting of chaotic dynamical systems using machine learning. Particularly promising among these are hybrid approaches that combine machine learning with a knowledge-based model, where a machine-learning technique is used to correct the imperfections in the knowledge-based model. Such imperfections may be due to incomplete understanding and/or limited resolution of the physical processes in the underlying dynamical system, e.g., the atmosphere or the ocean. Previously proposed data-driven forecasting approaches tend to require, for training, measurements of all the variables that are intended to be forecast. We describe a way to relax this assumption by combining data assimilation with machine learning. We demonstrate this technique using the Ensemble Transform Kalman Filter (ETKF) to assimilate synthetic data for the 3-variable Lorenz system and for the Kuramoto-Sivashinsky system, simulating model error in each case by a misspecified parameter value. We show that by using partial measurements of the state of the dynamical system, we can train a machine learning model to improve predictions made by an imperfect knowledge-based model. △ Less

Submitted 15 February, 2021; originally announced February 2021.

Comments: 28 pages, 9 figures

Hybrid Backpropagation Parallel Reservoir Networks

Authors: Matthew Evanusa, Snehesh Shrestha, Michelle Girvan, Cornelia Fermüller, Yiannis Aloimonos

Abstract: In many real-world applications, fully-differentiable RNNs such as LSTMs and GRUs have been widely deployed to solve time series learning tasks. These networks train via Backpropagation Through Time, which can work well in practice but involves a biologically unrealistic unrolling of the network in time for gradient updates, are computationally expensive, and can be hard to tune. A second paradigm… ▽ More In many real-world applications, fully-differentiable RNNs such as LSTMs and GRUs have been widely deployed to solve time series learning tasks. These networks train via Backpropagation Through Time, which can work well in practice but involves a biologically unrealistic unrolling of the network in time for gradient updates, are computationally expensive, and can be hard to tune. A second paradigm, Reservoir Computing, keeps the recurrent weight matrix fixed and random. Here, we propose a novel hybrid network, which we call Hybrid Backpropagation Parallel Echo State Network (HBP-ESN) which combines the effectiveness of learning random temporal features of reservoirs with the readout power of a deep neural network with batch normalization. We demonstrate that our new network outperforms LSTMs and GRUs, including multi-layer "deep" versions of these networks, on two complex real-world multi-dimensional time series datasets: gesture recognition using skeleton keypoints from ChaLearn, and the DEAP dataset for emotion recognition from EEG measurements. We show also that the inclusion of a novel meta-ring structure, which we call HBP-ESN M-Ring, achieves similar performance to one large reservoir while decreasing the memory required by an order of magnitude. We thus offer this new hybrid reservoir deep learning paradigm as a new alternative direction for RNN learning of temporal or sequential data. △ Less

Submitted 27 October, 2020; originally announced October 2020.

Combining Machine Learning with Knowledge-Based Modeling for Scalable Forecasting and Subgrid-Scale Closure of Large, Complex, Spatiotemporal Systems

Authors: Alexander Wikner, Jaideep Pathak, Brian Hunt, Michelle Girvan, Troy Arcomano, Istvan Szunyogh, Andrew Pomerance, Edward Ott

Abstract: We consider the commonly encountered situation (e.g., in weather forecasting) where the goal is to predict the time evolution of a large, spatiotemporally chaotic dynamical system when we have access to both time series data of previous system states and an imperfect model of the full system dynamics. Specifically, we attempt to utilize machine learning as the essential tool for integrating the us… ▽ More We consider the commonly encountered situation (e.g., in weather forecasting) where the goal is to predict the time evolution of a large, spatiotemporally chaotic dynamical system when we have access to both time series data of previous system states and an imperfect model of the full system dynamics. Specifically, we attempt to utilize machine learning as the essential tool for integrating the use of past data into predictions. In order to facilitate scalability to the common scenario of interest where the spatiotemporally chaotic system is very large and complex, we propose combining two approaches:(i) a parallel machine learning prediction scheme; and (ii) a hybrid technique, for a composite prediction system composed of a knowledge-based component and a machine-learning-based component. We demonstrate that not only can this method combining (i) and (ii) be scaled to give excellent performance for very large systems, but also that the length of time series data needed to train our multiple, parallel machine learning components is dramatically less than that necessary without parallelization. Furthermore, considering cases where computational realization of the knowledge-based component does not resolve subgrid-scale processes, our scheme is able to use training data to incorporate the effect of the unresolved short-scale dynamics upon the resolved longer-scale dynamics ("subgrid-scale closure"). △ Less

Submitted 10 February, 2020; originally announced February 2020.

Comments: 45 pages, 15 figures

Separation of Chaotic Signals by Reservoir Computing

Authors: Sanjukta Krishnagopal, Michelle Girvan, Edward Ott, Brian Hunt

Abstract: We demonstrate the utility of machine learning in the separation of superimposed chaotic signals using a technique called Reservoir Computing. We assume no knowledge of the dynamical equations that produce the signals, and require only training data consisting of finite time samples of the component signals. We test our method on signals that are formed as linear combinations of signals from two L… ▽ More We demonstrate the utility of machine learning in the separation of superimposed chaotic signals using a technique called Reservoir Computing. We assume no knowledge of the dynamical equations that produce the signals, and require only training data consisting of finite time samples of the component signals. We test our method on signals that are formed as linear combinations of signals from two Lorenz systems with different parameters. Comparing our nonlinear method with the optimal linear solution to the separation problem, the Wiener filter, we find that our method significantly outperforms the Wiener filter in all the scenarios we study. Furthermore, this difference is particularly striking when the component signals have similar frequency spectra. Indeed, our method works well when the component frequency spectra are indistinguishable - a case where a Wiener filter performs essentially no separation. △ Less

Submitted 25 October, 2019; v1 submitted 17 October, 2019; originally announced October 2019.

Comments: 12 pages, 7 figures

Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics

Authors: Pantelis R. Vlachas, Jaideep Pathak, Brian R. Hunt, Themistoklis P. Sapsis, Michelle Girvan, Edward Ott, Petros Koumoutsakos

Abstract: We examine the efficiency of Recurrent Neural Networks in forecasting the spatiotemporal dynamics of high dimensional and reduced order complex systems using Reservoir Computing (RC) and Backpropagation through time (BPTT) for gated network architectures. We highlight advantages and limitations of each method and discuss their implementation for parallel computing architectures. We quantify the re… ▽ More We examine the efficiency of Recurrent Neural Networks in forecasting the spatiotemporal dynamics of high dimensional and reduced order complex systems using Reservoir Computing (RC) and Backpropagation through time (BPTT) for gated network architectures. We highlight advantages and limitations of each method and discuss their implementation for parallel computing architectures. We quantify the relative prediction accuracy of these algorithms for the longterm forecasting of chaotic systems using as benchmarks the Lorenz-96 and the Kuramoto-Sivashinsky (KS) equations. We find that, when the full state dynamics are available for training, RC outperforms BPTT approaches in terms of predictive performance and in capturing of the long-term statistics, while at the same time requiring much less training time. However, in the case of reduced order data, large scale RC models can be unstable and more likely than the BPTT algorithms to diverge. In contrast, RNNs trained via BPTT show superior forecasting abilities and capture well the dynamics of reduced order systems. Furthermore, the present study quantifies for the first time the Lyapunov Spectrum of the KS equation with BPTT, achieving similar accuracy as RC. This study establishes that RNNs are a potent computational framework for the learning and forecasting of complex spatiotemporal systems. △ Less

Submitted 17 February, 2020; v1 submitted 9 October, 2019; originally announced October 2019.

Comments: 41 pages, submitted to Elsevier Journal of Neural Networks (accepted)

Identifying and Predicting Parkinson's Disease Subtypes through Trajectory Clustering via Bipartite Networks

Authors: Sanjukta Krishnagopal, Rainer Von Coelln, Lisa M. Shulman, Michelle Girvan

Abstract: Parkinson's disease (PD) is a common neurodegenerative disease with a high degree of heterogeneity in its clinical features, rate of progression, and change of variables over time. In this work, we present a novel data-driven, network-based Trajectory Profile Clustering (TPC) algorithm for 1) identification of PD subtypes and 2) early prediction of disease progression in individual patients. Our s… ▽ More Parkinson's disease (PD) is a common neurodegenerative disease with a high degree of heterogeneity in its clinical features, rate of progression, and change of variables over time. In this work, we present a novel data-driven, network-based Trajectory Profile Clustering (TPC) algorithm for 1) identification of PD subtypes and 2) early prediction of disease progression in individual patients. Our subtype identification is based not only on PD variables, but also on their complex patterns of progression, providing a useful tool for the analysis of large heterogenous, longitudinal data. Specifically, we cluster patients based on the similarity of their trajectories through a time series of bipartite networks connecting patients to demographic, clinical, and genetic variables. We apply this approach to demographic and clinical data from the Parkinson's Progression Markers Initiative (PPMI) dataset and identify 3 patient clusters, consistent with 3 distinct PD subtypes, each with a characteristic variable progression profile. Additionally, TPC predicts an individual patient's subtype and future disease trajectory, based on baseline assessments. Application of our approach resulted in 74% accurate subtype prediction in year 5 in a test/validation cohort. Furthermore, we show that genetic variability can be integrated seamlessly in our TPC approach. In summary, using PD as a model for chronic progressive diseases, we show that TPC leverages high-dimensional longitudinal datasets for subtype identification and early prediction of individual disease subtype. We anticipate this approach will be broadly applicable to multidimensional longitudinal datasets in diverse chronic diseases. △ Less

Submitted 7 July, 2019; v1 submitted 12 June, 2019; originally announced June 2019.

Comments: 14 pages, 4 figures

Critical Network Cascades with Re-excitable nodes: Why tree-like approximations usually work, when they breakdown, and how to correct them

Authors: Sarthak Chandra, Edward Ott, Michelle Girvan

Abstract: Network science is a rapidly expanding field, with a large and growing body of work on network-based dynamical processes. Most theoretical results in this area rely on the so-called \emph{locally tree-like approximation}. This is, however, usually an `uncontrolled' approximation, in the sense that the magnitudes of the error are typically unknown, although numerical results show that this error is… ▽ More Network science is a rapidly expanding field, with a large and growing body of work on network-based dynamical processes. Most theoretical results in this area rely on the so-called \emph{locally tree-like approximation}. This is, however, usually an `uncontrolled' approximation, in the sense that the magnitudes of the error are typically unknown, although numerical results show that this error is often surprisingly small. In this paper, we place this approximation on more rigorous footing by calculating the magnitude of deviations away from tree-based theories in the context of discrete-time critical network cascades with re-excitable nodes. We discuss the conditions under which tree-like approximations give good results for calculating network criticality, and also explain the reasons for deviation from this approximation, in terms of the density of certain kinds of network motifs. Using this understanding, we derive results for network criticality that apply to general networks that explicitly do not satisfy the locally tree-like approximation. In particular, we focus on the bi-parallel motif, the smallest motif relevant to the failure of a tree-based theory in this context, and we derive the corrections due to such motifs on the conditions for criticality. We verify our claims on computer-generated networks, and we confirm that our theory accurately predicts the observed deviations from criticality. Using our theory, we explain why numerical simulations often show that deviations from a tree-based theory are surprisingly small. More specifically, we show that these deviations are negligible for networks whose average degree is even modestly large compared to one, justifying why tree-based theories appear to work well for most real-world networks. △ Less

Submitted 8 June, 2020; v1 submitted 17 May, 2019; originally announced May 2019.

Comments: 22 pages, 8 figures

Journal ref: Phys. Rev. E 101, 062304 (2020)

Computational landscape of user behavior on social media

Authors: David Darmon, William Rand, Michelle Girvan

Abstract: With the increasing abundance of 'digital footprints' left by human interactions in online environments, e.g., social media and app use, the ability to model complex human behavior has become increasingly possible. Many approaches have been proposed, however, most previous model frameworks are fairly restrictive. We introduce a new social modeling approach that enables the creation of models direc… ▽ More With the increasing abundance of 'digital footprints' left by human interactions in online environments, e.g., social media and app use, the ability to model complex human behavior has become increasingly possible. Many approaches have been proposed, however, most previous model frameworks are fairly restrictive. We introduce a new social modeling approach that enables the creation of models directly from data with minimal a priori restrictions on the model class. In particular, we infer the minimally complex, maximally predictive representation of an individual's behavior when viewed in isolation and as driven by a social input. We then apply this framework to a heterogeneous catalog of human behavior collected from fifteen thousand users on the microblogging platform Twitter. The models allow us to describe how a user processes their past behavior and their social inputs. Despite the diversity of observed user behavior, most models inferred fall into a small subclass of all possible finite-state processes. Thus, our work demonstrates that user behavior, while quite complex, belies simple underlying computational structures. △ Less

Submitted 25 January, 2019; originally announced January 2019.

Journal ref: Physical Review E 98.6 (2018): 062306

Reversibility of granular rotations and translations

Authors: Anton Peshkov, Michelle Girvan, Derek C. Richardson, Wolfgang Losert

Abstract: We analyze reversibility of both displacements and rotations of spherical grains in three-dimensional compression experiments. Using transparent acrylic beads with cylindrical holes and index matching techniques, we are not only capable of tracking displacements but also, for the first time, analyze reversibility of rotations. We observe that for moderate compression amplitudes, up to one bead dia… ▽ More We analyze reversibility of both displacements and rotations of spherical grains in three-dimensional compression experiments. Using transparent acrylic beads with cylindrical holes and index matching techniques, we are not only capable of tracking displacements but also, for the first time, analyze reversibility of rotations. We observe that for moderate compression amplitudes, up to one bead diameter, the translational displacements of the beads after each cycle become mostly reversible after an initial transient. By contrast, granular rotations are largely irreversible. We find a weak correlation between translational and rotational displacements, indicating that rotational reversibility depends on more subtle changes in contact distributions and contact forces between grains compared with displacement reversibility. △ Less

Submitted 11 October, 2019; v1 submitted 30 October, 2018; originally announced October 2018.

Journal ref: Phys. Rev. E 100, 042905 (2019)

Complexity Reduction Ansatz for Systems of Interacting Orientable Agents: Beyond The Kuramoto Model

Authors: Sarthak Chandra, Michelle Girvan, Edward Ott

Abstract: Previous results have shown that a large class of complex systems consisting of many interacting heterogeneous phase oscillators exhibit an attracting invariant manifold. This result has enabled reduced analytic system descriptions from which all the long term dynamics of these systems can be calculated. Although very useful, these previous results are limited by the restriction that the individua… ▽ More Previous results have shown that a large class of complex systems consisting of many interacting heterogeneous phase oscillators exhibit an attracting invariant manifold. This result has enabled reduced analytic system descriptions from which all the long term dynamics of these systems can be calculated. Although very useful, these previous results are limited by the restriction that the individual interacting system components have one-dimensional dynamics, with states described by a single, scalar, angle-like variable (e.g., the Kuramoto model). In this paper we consider a generalization to an appropriate class of coupled agents with higher-dimensional dynamics. For this generalized class of model systems we demonstrate that the dynamics again contain an invariant manifold, hence enabling previously inaccessible analysis and improved numerical study, allowing a similar simplified description of these systems. We also discuss examples illustrating the potential utility of our results for a wide range of interesting situations. △ Less

Submitted 17 May, 2019; v1 submitted 3 September, 2018; originally announced September 2018.

Comments: 8 pages (incl. 1 appendix), 2 figures

Continuous versus Discontinuous Transitions in the $D$-Dimensional Generalized Kuramoto Model: Odd $D$ is Different

Authors: Sarthak Chandra, Michelle Girvan, Edward Ott

Abstract: The Kuramoto model, originally proposed to model the dynamics of many interacting oscillators, has been used and generalized for a wide range of applications involving the collective behavior of large heterogeneous groups of dynamical units whose states are characterized by a scalar angle variable. One such application in which we are interested is the alignment of orientation vectors among member… ▽ More The Kuramoto model, originally proposed to model the dynamics of many interacting oscillators, has been used and generalized for a wide range of applications involving the collective behavior of large heterogeneous groups of dynamical units whose states are characterized by a scalar angle variable. One such application in which we are interested is the alignment of orientation vectors among members of a swarm. Despite being commonly used for this purpose, the Kuramoto model can only describe swarms in 2 dimensions, and hence the results obtained do not apply to the often relevant situation of swarms in 3 dimensions. Partly based on this motivation, as well as on relevance to the classical, mean-field, zero-temperature Heisenberg model with quenched site disorder, in this paper we study the Kuramoto model generalized to $D$ dimensions. We show that in the important case of 3 dimensions, as well as for any odd number of dimensions, the $D$-dimensional generalized Kuramoto model for heterogeneous units has dynamics that are remarkably different from the dynamics in 2 dimensions. In particular, for odd $D$ the transition to coherence occurs discontinuously as the inter-unit coupling constant $K$ is increased through zero, as opposed to the $D=2$ case (and, as we show, also the case of even $D$) for which the transition to coherence occurs continuously as $K$ increases through a positive critical value $K_c$. We also demonstrate the qualitative applicability of our results to related models constructed specifically to capture swarming and flocking dynamics in three dimensions. △ Less

Submitted 29 November, 2018; v1 submitted 4 June, 2018; originally announced June 2018.

Comments: 19 pages, 10 figures; Minor edits + re-ordering of sections

Journal ref: Phys. Rev. X 9, 011002 (2019)

Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model

Authors: Jaideep Pathak, Alexander Wikner, Rebeckah Fussell, Sarthak Chandra, Brian Hunt, Michelle Girvan, Edward Ott

Abstract: A model-based approach to forecasting chaotic dynamical systems utilizes knowledge of the physical processes governing the dynamics to build an approximate mathematical model of the system. In contrast, machine learning techniques have demonstrated promising results for forecasting chaotic systems purely from past time series measurements of system state variables (training data), without prior kn… ▽ More A model-based approach to forecasting chaotic dynamical systems utilizes knowledge of the physical processes governing the dynamics to build an approximate mathematical model of the system. In contrast, machine learning techniques have demonstrated promising results for forecasting chaotic systems purely from past time series measurements of system state variables (training data), without prior knowledge of the system dynamics. The motivation for this paper is the potential of machine learning for filling in the gaps in our underlying mechanistic knowledge that cause widely-used knowledge-based models to be inaccurate. Thus we here propose a general method that leverages the advantages of these two approaches by combining a knowledge-based model and a machine learning technique to build a hybrid forecasting scheme. Potential applications for such an approach are numerous (e.g., improving weather forecasting). We demonstrate and test the utility of this approach using a particular illustrative version of a machine learning known as reservoir computing, and we apply the resulting hybrid forecaster to a low-dimensional chaotic system, as well as to a high-dimensional spatiotemporal chaotic system. These tests yield extremely promising results in that our hybrid technique is able to accurately predict for a much longer period of time than either its machine-learning component or its model-based component alone. △ Less

Submitted 9 March, 2018; originally announced March 2018.

Evidence of Herding and Stubbornness in Jury Deliberations

Authors: Keith Burghardt, William Rand, Michelle Girvan

Abstract: We explore how the mechanics of collective decision-making, especially of jury deliberation, can be inferred from macroscopic statistics. We first hypothesize that the dynamics of competing opinions can leave a "fingerprint" in the joint distribution of final votes and time to reach a decision. We probe this hypothesis by modeling jury datasets from different states collected in different years an… ▽ More We explore how the mechanics of collective decision-making, especially of jury deliberation, can be inferred from macroscopic statistics. We first hypothesize that the dynamics of competing opinions can leave a "fingerprint" in the joint distribution of final votes and time to reach a decision. We probe this hypothesis by modeling jury datasets from different states collected in different years and identifying which of the models best explains opinion dynamics in juries. In our best-fit model, individual jurors have a "herding" tendency to adopt the majority opinion of the jury, but as the amount of time they have held their current opinion increases, so too does their resistance to changing their opinion (what we call "increasing stubbornness"). By contrast, other models without increasing stubbornness, or without herding, create poorer fits to data. Our findings suggest that both stubbornness and herding play an important role in collective decision-making. △ Less

Submitted 1 November, 2017; v1 submitted 30 October, 2017; originally announced October 2017.

Comments: 9 pages, 13 figures (5 figures in the main text, and 8 figures in the SI). Watermarks removed in version 2 and two very minor typos fixed

Journal ref: PLoS ONE 14(7): e0218312 (2019)

Using Machine Learning to Replicate Chaotic Attractors and Calculate Lyapunov Exponents from Data

Authors: Jaideep Pathak, Zhixin Lu, Brian R. Hunt, Michelle Girvan, Edward Ott

Abstract: We use recent advances in the machine learning area known as 'reservoir computing' to formulate a method for model-free estimation from data of the Lyapunov exponents of a chaotic process. The technique uses a limited time series of measurements as input to a high-dimensional dynamical system called a 'reservoir'. After the reservoir's response to the data is recorded, linear regression is used to… ▽ More We use recent advances in the machine learning area known as 'reservoir computing' to formulate a method for model-free estimation from data of the Lyapunov exponents of a chaotic process. The technique uses a limited time series of measurements as input to a high-dimensional dynamical system called a 'reservoir'. After the reservoir's response to the data is recorded, linear regression is used to learn a large set of parameters, called the 'output weights'. The learned output weights are then used to form a modified autonomous reservoir designed to be capable of producing arbitrarily long time series whose ergodic properties approximate those of the input signal. When successful, we say that the autonomous reservoir reproduces the attractor's 'climate'. Since the reservoir equations and output weights are known, we can compute derivatives needed to determine the Lyapunov exponents of the autonomous reservoir, which we then use as estimates of the Lyapunov exponents for the original input generating system. We illustrate the effectiveness of our technique with two examples, the Lorenz system, and the Kuramoto-Sivashinsky (KS) equation. In particular, we use the Lorenz system to show that achieving climate reproduction may require tuning of the reservoir parameters. For the case of the KS equation, we note that as the system's spatial size is increased, the number of Lyapunov exponents increases, thus yielding a challenging test of our method, which we find the method successfully passes. △ Less

Submitted 19 October, 2017; originally announced October 2017.

Modeling the Network Dynamics of Pulse-Coupled Neurons

Authors: Sarthak Chandra, David Hathcock, Kimberly Crain, Thomas M. Antonsen, Michelle Girvan, Edward Ott

Abstract: We derive a mean-field approximation for the macroscopic dynamics of large networks of pulse-coupled theta neurons in order to study the effects of different network degree distributions, as well as degree correlations (assortativity). Using the ansatz of Ott and Antonsen (Chaos, 19 (2008) 037113), we obtain a reduced system of ordinary differential equations describing the mean-field dynamics, wi… ▽ More We derive a mean-field approximation for the macroscopic dynamics of large networks of pulse-coupled theta neurons in order to study the effects of different network degree distributions, as well as degree correlations (assortativity). Using the ansatz of Ott and Antonsen (Chaos, 19 (2008) 037113), we obtain a reduced system of ordinary differential equations describing the mean-field dynamics, with significantly lower dimensionality compared with the complete set of dynamical equations for the system. We find that, for sufficiently large networks and degrees, the dynamical behavior of the reduced system agrees well with that of the full network. This dimensional reduction allows for an efficient characterization of system phase transitions and attractors. For networks with tightly peaked degree distributions, the macroscopic behavior closely resembles that of fully connected networks previously studied by others. In contrast, networks with scale-free degree distributions exhibit different macroscopic dynamics due to the emergence of degree dependent behavior of different oscillators. For nonassortative networks (i.e., networks without degree correlations) we observe the presence of a synchronously firing phase that can be suppressed by the presence of either assortativity or disassortativity in the network. We show that the results derived here can be used to analyze the effects of network topology on macroscopic behavior in neuronal networks in a computationally efficient fashion. △ Less

Submitted 1 January, 2017; originally announced January 2017.

Comments: 10 pages, 7 figures

The Myopia of Crowds: A Study of Collective Evaluation on Stack Exchange

Authors: Keith Burghardt, Emanuel F. Alsina, Michelle Girvan, William Rand, Kristina Lerman

Abstract: Crowds can often make better decisions than individuals or small groups of experts by leveraging their ability to aggregate diverse information. Question answering sites, such as Stack Exchange, rely on the "wisdom of crowds" effect to identify the best answers to questions asked by users. We analyze data from 250 communities on the Stack Exchange network to pinpoint factors affecting which answer… ▽ More Crowds can often make better decisions than individuals or small groups of experts by leveraging their ability to aggregate diverse information. Question answering sites, such as Stack Exchange, rely on the "wisdom of crowds" effect to identify the best answers to questions asked by users. We analyze data from 250 communities on the Stack Exchange network to pinpoint factors affecting which answers are chosen as the best answers. Our results suggest that, rather than evaluate all available answers to a question, users rely on simple cognitive heuristics to choose an answer to vote for or accept. These cognitive heuristics are linked to an answer's salience, such as the order in which it is listed and how much screen space it occupies. While askers appear to depend more on heuristics, compared to voting users, when choosing an answer to accept as the most helpful one, voters use acceptance itself as a heuristic: they are more likely to choose the answer after it is accepted than before that very same answer was accepted. These heuristics become more important in explaining and predicting behavior as the number of available answers increases. Our findings suggest that crowd judgments may become less reliable as the number of answers grow. △ Less

Submitted 23 February, 2016; originally announced February 2016.

Comments: 10 pages, 9 figures

Journal ref: PLoS ONE 12(3): e0173610

Dynamical Transitions in Large Systems of Mean Field-Coupled Landau-Stuart Oscillators: Extensive Chaos and Clumped States

Authors: Wai Lim Ku, Michelle Girvan, Edward Ott

Abstract: In this paper, we study dynamical systems in which a large number $N$ of identical Landau-Stuart oscillators are globally coupled via a mean-field. Previously, it has been observed that this type of system can exhibit a variety of different dynamical behaviors including clumped states in which each oscillator is in one of a small number of groups for which all oscillators in each group have the sa… ▽ More In this paper, we study dynamical systems in which a large number $N$ of identical Landau-Stuart oscillators are globally coupled via a mean-field. Previously, it has been observed that this type of system can exhibit a variety of different dynamical behaviors including clumped states in which each oscillator is in one of a small number of groups for which all oscillators in each group have the same state which is different from group to group, as well as situations in which all oscillators have different states and the macroscopic dynamics of the mean field is chaotic. We argue that this second type of behavior is $^{\backprime}$extensive$^{\prime}$ in the sense that the chaotic attractor in the full phase space of the system has a fractal dimension that scales linearly with $N$ and that the number of positive Lyapunov exponents of the attractor also scales with linearly $N$. An important focus of this paper is the transition between clumped states and extensive chaos as the system is subjected to slow adiabatic parameter change. We observe explosive (i.e., discontinuous) transitions between the clumped states (which correspond to low dimensional dynamics) and the extensively chaotic states. Furthermore, examining the clumped state, as the system approaches the explosive transition to extensive chaos, we find that the oscillator population distribution between the clumps continually evolves so that the clumped state is always marginally stable. This behavior is used to reveal the mechanism of the explosive transition. We also apply the Kaplan-Yorke formula to study the fractal structure of the extensively chaotic attractors. △ Less

Submitted 11 December, 2014; originally announced December 2014.

MSC Class: Complex dynamical systems

The Impact of Imperfect Information on Network Attack

Authors: Andrew Melchionna, Jesus Caloca, Shane Squires, Thomas M. Antonsen, Edward Ott, Michelle Girvan

Abstract: This paper explores the effectiveness of network attack when the attacker has imperfect information about the network. For Erdős-Rényi networks, we observe that dynamical importance and betweenness centrality-based attacks are surprisingly robust to the presence of a moderate amount of imperfect information and are more effective compared with simpler degree-based attacks even at moderate levels o… ▽ More This paper explores the effectiveness of network attack when the attacker has imperfect information about the network. For Erdős-Rényi networks, we observe that dynamical importance and betweenness centrality-based attacks are surprisingly robust to the presence of a moderate amount of imperfect information and are more effective compared with simpler degree-based attacks even at moderate levels of network information error. In contrast, for scale-free networks the effectiveness of attack is much less degraded by a moderate level of information error. Furthermore, in the Erdőos-Rényi case the effectiveness of network attack is much more degraded by missing links as compared with the same number of false links. △ Less

Submitted 10 December, 2014; originally announced December 2014.

Competing opinions and stubbornness: connecting models to data

Authors: Keith A. Burghardt, William Rand, Michelle Girvan

Abstract: We introduce a general contagion-like model for competing opinions that includes dynamic resistance to alternative opinions. We show that this model can describe candidate vote distributions, spatial vote correlations, and a slow approach to opinion consensus with sensible parameter values. These empirical properties of large group dynamics, previously understood using distinct models, may be diff… ▽ More We introduce a general contagion-like model for competing opinions that includes dynamic resistance to alternative opinions. We show that this model can describe candidate vote distributions, spatial vote correlations, and a slow approach to opinion consensus with sensible parameter values. These empirical properties of large group dynamics, previously understood using distinct models, may be different aspects of human behavior that can be captured by a more unified model, such as the one introduced in this paper. △ Less

Submitted 7 March, 2016; v1 submitted 26 November, 2014; originally announced November 2014.

Comments: 13 pages, 10 figures, 2 column revetx4 format

Journal ref: Phys. Rev. E 93, 032305 (2016)

Spatially embedded growing small-world networks

Authors: Ari Zitin, Alex Gorowora, Shane Squires, Mark Herrera, Thomas M. Antonsen, Michelle Girvan, Edward Ott

Abstract: Networks in nature are often formed within a spatial domain in a dynamical manner, gaining links and nodes as they develop over time. We propose a class of spatially-based growing network models and investigate the relationship between the resulting statistical network properties and the dimension and topology of the space in which the networks are embedded. In particular, we consider models in wh… ▽ More Networks in nature are often formed within a spatial domain in a dynamical manner, gaining links and nodes as they develop over time. We propose a class of spatially-based growing network models and investigate the relationship between the resulting statistical network properties and the dimension and topology of the space in which the networks are embedded. In particular, we consider models in which nodes are placed one by one in random locations in space, with each such placement followed by configuration relaxation toward uniform node density, and connection of the new node with spatially nearby nodes. We find that such growth processes naturally result in networks with small-world features, including a short characteristic path length and nonzero clustering. These properties do not appear to depend strongly on the topology of the embedding space, but do depend strongly on its dimension; higher-dimensional spaces result in shorter path lengths but less clustering. △ Less

Submitted 26 December, 2013; originally announced December 2013.

Stability of Boolean networks: The joint effects of topology and update rules

Authors: Shane Squires, Andrew Pomerance, Michelle Girvan, Edward Ott

Abstract: We study the stability of orbits in large Boolean networks with given complex topology. We impose no restrictions on the form of the update rules, which may be correlated with local topological properties of the network. While recent past work has addressed the separate effects of nontrivial network topology and certain special classes of update rules on stability, only crude results exist about h… ▽ More We study the stability of orbits in large Boolean networks with given complex topology. We impose no restrictions on the form of the update rules, which may be correlated with local topological properties of the network. While recent past work has addressed the separate effects of nontrivial network topology and certain special classes of update rules on stability, only crude results exist about how these effects interact. We present a widely applicable solution to this problem. Numerical experiments confirm our theory and show that local correlations between topology and update rules can have profound effects on the qualitative behavior of these systems. △ Less

Submitted 4 October, 2013; originally announced October 2013.

Comments: Includes supplemental information

Journal ref: Phys. Rev. E 90, 022814 (2014)

Robustness of Network Measures to Link Errors

Authors: John Platig, Ed Ott, Michelle Girvan

Abstract: In various applications involving complex networks, network measures are employed to assess the relative importance of network nodes. However, the robustness of such measures in the presence of link inaccuracies has not been well characterized. Here we present two simple stochastic models of false and missing links and study the effect of link errors on three commonly used node centrality measures… ▽ More In various applications involving complex networks, network measures are employed to assess the relative importance of network nodes. However, the robustness of such measures in the presence of link inaccuracies has not been well characterized. Here we present two simple stochastic models of false and missing links and study the effect of link errors on three commonly used node centrality measures: degree centrality, betweenness centrality, and dynamical importance. We perform numerical simulations to assess robustness of these three centrality measures. We also develop an analytical theory, which we compare with our simulations, obtaining very good agreement. △ Less

Submitted 18 September, 2013; originally announced September 2013.

Comments: 9 pages, 9 figures

Journal ref: Phys. Rev. E 88, 062812 (2013)

A hierarchical network heuristic for solving the orientation problem in genome assembly

Authors: Karl R. B. Schmitt, Aleksey V. Zimin, Guillaume Marcaçs, James A. Yorke, Michelle Girvan

Abstract: In the past several years, the problem of genome assembly has received considerable attention from both biologists and computer scientists. An important component of current assembly methods is the scaffolding process. This process involves building ordered and oriented linear collections of contigs (continuous overlap** sequence reads) called scaffolds and relies on the use of mate pair data. A… ▽ More In the past several years, the problem of genome assembly has received considerable attention from both biologists and computer scientists. An important component of current assembly methods is the scaffolding process. This process involves building ordered and oriented linear collections of contigs (continuous overlap** sequence reads) called scaffolds and relies on the use of mate pair data. A mate pair is a set of two reads that are sequenced from the ends of a single fragment of DNA, and therefore have opposite mutual orientations. When two reads of a mate-pair are placed into two different contigs, one can infer the mutual orientation of these contigs. While several orientation algorithms exist as part of assembly programs, all encounter challenges while solving the orientation problem due to errors from mis-assemblies in contigs or errors in read placements. In this paper we present an algorithm based on hierarchical clustering that independently solves the orientation problem and is robust to errors. We show that our algorithm can correctly solve the orientation problem for both faux (generated) assembly data and real assembly data for {\em R. sphaeroides bacteria}. We demonstrate that our algorithm is stable to both changes in the initial orientations as well as noise in the data, making it advantageous compared to traditional approaches. △ Less

Submitted 1 July, 2013; originally announced July 2013.

Comments: 18 pages, 11 figures

Understanding the Predictive Power of Computational Mechanics and Echo State Networks in Social Media

Authors: David Darmon, Jared Sylvester, Michelle Girvan, William Rand

Abstract: There is a large amount of interest in understanding users of social media in order to predict their behavior in this space. Despite this interest, user predictability in social media is not well-understood. To examine this question, we consider a network of fifteen thousand users on Twitter over a seven week period. We apply two contrasting modeling paradigms: computational mechanics and echo sta… ▽ More There is a large amount of interest in understanding users of social media in order to predict their behavior in this space. Despite this interest, user predictability in social media is not well-understood. To examine this question, we consider a network of fifteen thousand users on Twitter over a seven week period. We apply two contrasting modeling paradigms: computational mechanics and echo state networks. Both methods attempt to model the behavior of users on the basis of their past behavior. We demonstrate that the behavior of users on Twitter can be well-modeled as processes with self-feedback. We find that the two modeling approaches perform very similarly for most users, but that they differ in performance on a small subset of the users. By exploring the properties of these performance-differentiated users, we highlight the challenges faced in applying predictive models to dynamic social data. △ Less

Submitted 23 August, 2013; v1 submitted 25 June, 2013; originally announced June 2013.

Modeling the dynamics of bivalent histone modifications

Authors: Wai Lim Ku, Michelle Girvan, Guo-Cheng Yuan, Francesco Sorrentino, Edward Ott

Abstract: Epigenetic modifications to histones may promote either activation or repression of the transcription of nearby genes. Recent experimental studies show that the promoters of many lineage-control genes in stem cells have "bivalent domains" in which the nucleosomes contain both active (H3K4me3) and repressive (H3K27me3) marks. It is generally agreed that bivalent domains play an important role in st… ▽ More Epigenetic modifications to histones may promote either activation or repression of the transcription of nearby genes. Recent experimental studies show that the promoters of many lineage-control genes in stem cells have "bivalent domains" in which the nucleosomes contain both active (H3K4me3) and repressive (H3K27me3) marks. It is generally agreed that bivalent domains play an important role in stem cell differentiation, but the underlying mechanisms remain unclear. Here we formulate a mathematical model to investigate the dynamic properties of histone modification patterns. We then illustrate that our modeling framework can be used to capture key features of experimentally observed combinatorial chromatin states. △ Less

Submitted 7 June, 2013; originally announced June 2013.

Comments: 23 pages, 10 figures

Weakly Explosive Percolation in Directed Networks

Authors: Shane Squires, Katherine Sytwu, Diego Alcala, Thomas Antonsen, Edward Ott, Michelle Girvan

Abstract: Percolation, the formation of a macroscopic connected component, is a key feature in the description of complex networks. The dynamical properties of a variety of systems can be understood in terms of percolation, including the robustness of power grids and information networks, the spreading of epidemics and forest fires, and the stability of gene regulatory networks. Recent studies have shown th… ▽ More Percolation, the formation of a macroscopic connected component, is a key feature in the description of complex networks. The dynamical properties of a variety of systems can be understood in terms of percolation, including the robustness of power grids and information networks, the spreading of epidemics and forest fires, and the stability of gene regulatory networks. Recent studies have shown that if network edges are added "competitively" in undirected networks, the onset of percolation is abrupt or "explosive." The unusual qualitative features of this phase transition have been the subject of much recent attention. Here we generalize this previously studied network growth process from undirected networks to directed networks and use finite-size scaling theory to find several scaling exponents. We find that this process is also characterized by a very rapid growth in the giant component, but that this growth is not as sudden as in undirected networks. △ Less

Submitted 20 June, 2013; v1 submitted 9 March, 2013; originally announced March 2013.

Journal ref: Phys. Rev. E 87 (5), 052127 (2013)

Finding New Order in Biological Functions from the Network Structure of Gene Annotations

Authors: Kimberly Glass, Michelle Girvan

Abstract: The Gene Ontology (GO) provides biologists with a controlled terminology that describes how genes are associated with functions and how functional terms are related to each other. These term-term relationships encode how scientists conceive the organization of biological functions, and they take the form of a directed acyclic graph (DAG). Here, we propose that the network structure of gene-term an… ▽ More The Gene Ontology (GO) provides biologists with a controlled terminology that describes how genes are associated with functions and how functional terms are related to each other. These term-term relationships encode how scientists conceive the organization of biological functions, and they take the form of a directed acyclic graph (DAG). Here, we propose that the network structure of gene-term annotations made using GO can be employed to establish an alternate natural way to group the functional terms which is different from the hierarchical structure established in the GO DAG. Instead of relying on an externally defined organization for biological functions, our method connects biological functions together if they are performed by the same genes, as indicated in a compendium of gene annotation data from numerous different experiments. We show that grou** terms by this alternate scheme is distinct from term relationships defined in the ontological structure and provides a new framework with which to describe and predict the functions of experimentally identified sets of genes. △ Less

Submitted 3 May, 2013; v1 submitted 28 September, 2012; originally announced October 2012.

Annotation Enrichment Analysis: An Alternative Method for Evaluating the Functional Properties of Gene Sets

Authors: Kimberly Glass, Michelle Girvan

Abstract: Gene annotation databases (compendiums maintained by the scientific community that describe the biological functions performed by individual genes) are commonly used to evaluate the functional properties of experimentally derived gene sets. Overlap statistics, such as Fisher's Exact Test (FET), are often employed to assess these associations, but don't account for non-uniformity in the number of g… ▽ More Gene annotation databases (compendiums maintained by the scientific community that describe the biological functions performed by individual genes) are commonly used to evaluate the functional properties of experimentally derived gene sets. Overlap statistics, such as Fisher's Exact Test (FET), are often employed to assess these associations, but don't account for non-uniformity in the number of genes annotated to individual functions or the number of functions associated with individual genes. We find FET is strongly biased toward over-estimating overlap significance if a gene set has an unusually high number of annotations. To correct for these biases, we develop Annotation Enrichment Analysis (AEA), which properly accounts for the non-uniformity of annotations. We show that AEA is able to identify biologically meaningful functional enrichments that are obscured by numerous false-positive enrichment scores in FET, and we therefore suggest it be used to more accurately assess the biological properties of gene sets. △ Less

Submitted 3 May, 2013; v1 submitted 20 August, 2012; originally announced August 2012.

Consequences of Anomalous Diffusion in Disordered Systems Under Cyclic Forcing

Authors: Mitch Mailman, Michelle Girvan, Wolfgang Losert

Abstract: We use numerical simulations to study the behavior of 2D frictionless disk systems under cyclic shear as a function of reversal amplitude γ_r. Our studies focus on mean bulk and disk dynamics. These measurements suggest a crossover from a subdiffusive, γ_r dependent regime to a regime where the grain motions are diffusive, with properties dependent only on total shear strain. We discuss model stoc… ▽ More We use numerical simulations to study the behavior of 2D frictionless disk systems under cyclic shear as a function of reversal amplitude γ_r. Our studies focus on mean bulk and disk dynamics. These measurements suggest a crossover from a subdiffusive, γ_r dependent regime to a regime where the grain motions are diffusive, with properties dependent only on total shear strain. We discuss model stochastic processes that are consistent with these observations. Finally, we introduce a modified Mean-Squared Displacement (mMSD) which takes into account the motion of the neighborhood of nearby grains and yields new insights into local displacement fluctuations. We find that scaling properties of the displacement distributions are consistent with well studied stochastic models of anomalous diffusion and suggest scale-invariant cage dynamics. △ Less

Submitted 29 June, 2012; originally announced July 2012.

Journal ref: Phys. Rev. Lett. 112, 228001 (2014)

Dynamical Instability in Boolean Networks as a Percolation Problem

Authors: Shane Squires, Edward Ott, Michelle Girvan

Abstract: Boolean networks, widely used to model gene regulation, exhibit a phase transition between regimes in which small perturbations either die out or grow exponentially. We show and numerically verify that this phase transition in the dynamics can be mapped onto a static percolation problem which predicts the long-time average Hamming distance between perturbed and unperturbed orbits. Boolean networks, widely used to model gene regulation, exhibit a phase transition between regimes in which small perturbations either die out or grow exponentially. We show and numerically verify that this phase transition in the dynamics can be mapped onto a static percolation problem which predicts the long-time average Hamming distance between perturbed and unperturbed orbits. △ Less

Submitted 25 July, 2012; v1 submitted 7 January, 2012; originally announced January 2012.

Journal ref: Phys. Rev. Lett. 109, 085701 (2012)

Multiscale Dynamics in Communities of Phase Oscillators

Authors: Dustin Anderson, Ari Tenzer, Gilad Barlev, Michelle Girvan, Thomas M. Antonsen, Edward Ott

Abstract: We investigate the dynamics of systems of many coupled phase oscillators with het- erogeneous frequencies. We suppose that the oscillators occur in M groups. Each oscillator is connected to other oscillators in its group with "attractive" coupling, such that the coupling promotes synchronization within the group. The coupling between oscillators in different groups is "repulsive"; i.e., their osci… ▽ More We investigate the dynamics of systems of many coupled phase oscillators with het- erogeneous frequencies. We suppose that the oscillators occur in M groups. Each oscillator is connected to other oscillators in its group with "attractive" coupling, such that the coupling promotes synchronization within the group. The coupling between oscillators in different groups is "repulsive"; i.e., their oscillation phases repel. To address this problem, we reduce the governing equations to a lower-dimensional form via the ansatz of Ott and Antonsen . We first consider the symmetric case where all group parameters are the same, and the attractive and repulsive coupling are also the same for each of the M groups. We find a manifold L of neutrally stable equilibria, and we show that all other equilibria are unstable. For M \geq 3, L has dimension M - 2, and for M = 2 it has dimension 1. To address the general asymmetric case, we then introduce small deviations from symmetry in the group and coupling param- eters. Doing a slow/fast timescale analysis, we obtain slow time evolution equations for the motion of the M groups on the manifold L. We use these equations to study the dynamics of the groups and compare the results with numerical simulations. △ Less

Submitted 5 December, 2011; v1 submitted 30 November, 2011; originally announced December 2011.

Comments: 29 pages, 6 figures

Onset of Irreversibility in Cyclic Shear of Granular Packings

Authors: Steven Slotterback, Mitch Mailman, Krisztian Ronaszegi, Martin van Hecke, Michelle Girvan, Wolfgang Losert

Abstract: We investigate the onset of irreversibility in a dense granular medium subjected to cyclic shear in a split-bottom geometry. To probe the micro and mesoscale we image bead trajectories in 3D throughout a series of shear strain oscillations. Though beads lose and regain contact with neighbors during a cycle, the topology of the contact network exhibits reversible properties for small oscillation am… ▽ More We investigate the onset of irreversibility in a dense granular medium subjected to cyclic shear in a split-bottom geometry. To probe the micro and mesoscale we image bead trajectories in 3D throughout a series of shear strain oscillations. Though beads lose and regain contact with neighbors during a cycle, the topology of the contact network exhibits reversible properties for small oscillation amplitudes. With increasing reversal amplitude a transition to an irreversible diffusive regime occurs. △ Less

Submitted 24 January, 2012; v1 submitted 23 September, 2011; originally announced September 2011.

The path to fracture in granular flows: dynamics of contact networks

Authors: Mark Herrera, Shane McCarthy, Steven Slotterback, Emmanuel Cephas, Wolfgang Losert, Michelle Girvan

Abstract: Capturing the dynamics of granular flows at intermediate length scales can often be difficult. We propose studying the dynamics of contact networks as a new tool to study fracture at intermediate scales. Using experimental three-dimensional flow fields with particle-scale resolution, we calculate the time evolving broken-links network and find that a giant component of this network is formed as sh… ▽ More Capturing the dynamics of granular flows at intermediate length scales can often be difficult. We propose studying the dynamics of contact networks as a new tool to study fracture at intermediate scales. Using experimental three-dimensional flow fields with particle-scale resolution, we calculate the time evolving broken-links network and find that a giant component of this network is formed as shear is applied to this system. We implement a model of link breakages where the probability of a link breaking is proportional to the average rate of longitudinal strain (elongation) in the direction of the edge and find that the model demonstrates qualitative agreement with the data when studying the onset of the giant component. We note, however, that the broken-links network formed in the model is less clustered than our experimental observations, indicating that the model reflects less localized breakage events and does not fully capture the dynamics of the granular flow. △ Less

Submitted 27 June, 2011; v1 submitted 1 February, 2011; originally announced February 2011.

Comments: 15 pages, 6 figures, accepted for publication in Phys. Rev. E

Journal ref: Phys. Rev. E 83, 061303 (2011)

Detecting Functional Communities in Complex Networks

Authors: Sanjeev Chauhan, Michelle Girvan, Edward Ott

Abstract: We consider an alternate definition of community structure that is functionally motivated. We define network community structure-based on the function the network system is intended to perform. In particular, as a specific example of this approach, we consider communities whose function is enhanced by the ability to synchronize and/or by resilience to node failures. Previous work has shown that, i… ▽ More We consider an alternate definition of community structure that is functionally motivated. We define network community structure-based on the function the network system is intended to perform. In particular, as a specific example of this approach, we consider communities whose function is enhanced by the ability to synchronize and/or by resilience to node failures. Previous work has shown that, in many cases, the largest eigenvalue of the network's adjacency matrix controls the onset of both synchronization and percolation processes. Thus, for networks whose functional performance is dependent on these processes, we propose a method that divides a given network into communities based on maximizing a function of the largest eigenvalues of the adjacency matrices of the resulting communities. We also explore the differences between the partitions obtained by our method and the modularity approach (which is based solely on consideration of network structure). We do this for several different classes of networks. We find that, in many cases, modularity-based partitions do almost as well as our function-based method in finding functional communities, even though modularity does not specifically incorporate consideration of function. △ Less

Submitted 2 March, 2012; v1 submitted 13 November, 2009; originally announced November 2009.

Comments: Changed the title and made minor modifications to the manuscript

The Stability of Boolean Networks with Generalized Canalizing Rules

Authors: Andrew Pomerance, Michelle Girvan, Ed Ott

Abstract: Boolean networks are discrete dynamical systems in which the state (zero or one) of each node is updated at each time t to a state determined by the states at time t-1 of those nodes that have links to it. When these systems are used to model genetic control, the case of 'canalizing' update rules is of particular interest. A canalizing rule is one for which a node state at time $t$ is determined b… ▽ More Boolean networks are discrete dynamical systems in which the state (zero or one) of each node is updated at each time t to a state determined by the states at time t-1 of those nodes that have links to it. When these systems are used to model genetic control, the case of 'canalizing' update rules is of particular interest. A canalizing rule is one for which a node state at time $t$ is determined by the state at time t-1 of a single one of its inputs when that inputting node is in its canalizing state. Previous work on the order/disorder transition in Boolean networks considered complex, non-random network topology. In the current paper we extend this previous work to account for canalizing behavior. △ Less

Submitted 24 February, 2012; v1 submitted 2 October, 2009; originally announced October 2009.

Comments: 20 pages, 3 figures

Universality under conditions of self-tuning

Authors: Ole Peters, Michelle Girvan

Abstract: We study systems with a continuous phase transition that tune their parameters to maximize a quantity that diverges solely at a unique critical point. Varying the size of these systems with dynamically adjusting parameters, the same finite-size scaling is observed as in systems where all relevant parameters are fixed at their critical values. This scheme is studied using a self-tuning variant of t… ▽ More We study systems with a continuous phase transition that tune their parameters to maximize a quantity that diverges solely at a unique critical point. Varying the size of these systems with dynamically adjusting parameters, the same finite-size scaling is observed as in systems where all relevant parameters are fixed at their critical values. This scheme is studied using a self-tuning variant of the Ising model. It is contrasted with a scheme where systems approach criticality through a target value for the order parameter that vanishes with increasing system size. In the former scheme, the universal exponents are observed in naive finite-size scaling studies, whereas in the latter they are not. △ Less

Submitted 9 August, 2010; v1 submitted 11 February, 2009; originally announced February 2009.

Comments: 5 pages, 3 figures

Journal ref: J. Stat. Phys. 141, 1, 53--59, 2010

The effect of network topology on the stability of discrete state models of genetic control

Authors: Andrew Pomerance, Edward Ott, Michelle Girvan, Wolfgang Losert

Abstract: Boolean networks have been proposed as potentially useful models for genetic control. An important aspect of these networks is the stability of their dynamics in response to small perturbations. Previous approaches to stability have assumed uncorrelated random network structure. Real gene networks typically have nontrivial topology significantly different from the random network paradigm. In ord… ▽ More Boolean networks have been proposed as potentially useful models for genetic control. An important aspect of these networks is the stability of their dynamics in response to small perturbations. Previous approaches to stability have assumed uncorrelated random network structure. Real gene networks typically have nontrivial topology significantly different from the random network paradigm. In order to address such situations, we present a general method for determining the stability of large Boolean networks of any specified network topology and predicting their steady-state behavior in response to small perturbations. Additionally, we generalize to the case where individual genes have a distribution of `expression biases,' and we consider non-synchronous update, as well as extension of our method to non-Boolean models in which there are more than two possible gene states. We find that stability is governed by the maximum eigenvalue of a modified adjacency matrix, and we test this result by comparison with numerical simulations. We also discuss the possible application of our work to experimentally inferred gene networks. △ Less

Submitted 26 February, 2009; v1 submitted 27 January, 2009; originally announced January 2009.

Comments: 25 pages, 4 figures; added supplementary information, fixed typos and figure, reformatted

Echo Phenomena in Large Systems of Coupled Oscillators

Authors: Edward Ott, John H. Platig, Thomas M. Antonsen, Michelle Girvan

Abstract: As exemplified by the Kuramoto model, large systems of coupled oscillators may undergo a transition to phase coherence with increasing coupling strength. It is shown that below the critical coupling strength for this transition such systems may be expected to exhibit `echo' phenomena: a stimulation by two successive pulses separated by a time interval tau leads to the spontaneous formation of re… ▽ More As exemplified by the Kuramoto model, large systems of coupled oscillators may undergo a transition to phase coherence with increasing coupling strength. It is shown that below the critical coupling strength for this transition such systems may be expected to exhibit `echo' phenomena: a stimulation by two successive pulses separated by a time interval tau leads to the spontaneous formation of response pulses at a time tau, 2 tau, 3 tau ..., after the second stimulus pulse. Analysis of this phenomenon, as well as illustrative numerical experiments, are presented. The theoretical significance and potential uses of echoes in such systems are discussed. △ Less

Submitted 28 July, 2008; originally announced July 2008.

Comments: 23 pages, 7 figures

External Periodic Driving of Large Systems of Globally Coupled Phase Oscillators

Authors: T. M. Antonsen Jr., R. T. Faghih, M. Girvan, E. Ott, J. Platig

Abstract: Large systems of coupled oscillators subjected to a periodic external drive occur in many situations in physics and biology. Here the simple, paradigmatic case of equal-strength, all-to-all sine-coupling of phase oscillators subject to a sinusoidal external drive is considered. The stationary states and their stability are determined. Using the stability information and numerical experiments, pa… ▽ More Large systems of coupled oscillators subjected to a periodic external drive occur in many situations in physics and biology. Here the simple, paradigmatic case of equal-strength, all-to-all sine-coupling of phase oscillators subject to a sinusoidal external drive is considered. The stationary states and their stability are determined. Using the stability information and numerical experiments, parameter space phase diagrams showing when different types of system behavior apply are constructed, and the bifurcations marking transitions between different types of behavior are delineated. The analysis is supported by results of direct numerical simulation of an ensemble of oscillators. △ Less

Submitted 26 November, 2007; originally announced November 2007.

Finding and evaluating community structure in networks

Authors: M. E. J. Newman, M. Girvan

Abstract: We propose and study a set of algorithms for discovering community structure in networks -- natural divisions of network nodes into densely connected subgroups. Our algorithms all share two definitive features: first, they involve iterative removal of edges from the network to split it into communities, the edges removed being identified using one of a number of possible "betweenness" measures,… ▽ More We propose and study a set of algorithms for discovering community structure in networks -- natural divisions of network nodes into densely connected subgroups. Our algorithms all share two definitive features: first, they involve iterative removal of edges from the network to split it into communities, the edges removed being identified using one of a number of possible "betweenness" measures, and second, these measures are, crucially, recalculated after each removal. We also propose a measure for the strength of the community structure found by our algorithms, which gives us an objective metric for choosing the number of communities into which a network should be divided. We demonstrate that our algorithms are highly effective at discovering community structure in both computer-generated and real-world network data, and show how they can be used to shed light on the sometimes dauntingly complex structure of networked systems. △ Less

Submitted 11 August, 2003; originally announced August 2003.

Comments: 16 pages, 13 figures

Journal ref: Phys. Rev. E 69, 026113 (2004)

Mixing patterns and community structure in networks

Authors: M. E. J. Newman, M. Girvan

Abstract: Common experience suggests that many networks might possess community structure - division of vertices into groups, with a higher density of edges within groups than between them. Here we describe a new computer algorithm that detects structure of this kind. We apply the algorithm to a number of real-world networks and show that they do indeed possess non-trivial community structure. We suggest… ▽ More Common experience suggests that many networks might possess community structure - division of vertices into groups, with a higher density of edges within groups than between them. Here we describe a new computer algorithm that detects structure of this kind. We apply the algorithm to a number of real-world networks and show that they do indeed possess non-trivial community structure. We suggest a possible explanation for this structure in the mechanism of assortative mixing, which is the preferential association of network vertices with others that are like them in some way. We show by simulation that this mechanism can indeed account for community structure. We also look in detail at one particular example of assortative mixing, namely mixing by vertex degree, in which vertices with similar degree prefer to be connected to one another. We propose a measure for mixing of this type which we apply to a variety of networks, and also discuss the implications for network structure and the formation of a giant component in assortatively mixed networks. △ Less

Submitted 7 October, 2002; originally announced October 2002.

Comments: 21 pages, 9 postscript figures, 2 tables

Journal ref: in Statistical Mechanics of Complex Networks, R. Pastor-Satorras, J. Rubi and A. Diaz-Guilera (eds.), pp. 66-87, Springer, Berlin (2003)

Optimal design, robustness, and risk aversion

Authors: M. E. J. Newman, Michelle Girvan, J. Doyne Farmer

Abstract: Highly optimized tolerance is a model of optimization in engineered systems, which gives rise to power-law distributions of failure events in such systems. The archetypal example is the highly optimized forest fire model. Here we give an analytic solution for this model which explains the origin of the power laws. We also generalize the model to incorporate risk aversion, which results in trunca… ▽ More Highly optimized tolerance is a model of optimization in engineered systems, which gives rise to power-law distributions of failure events in such systems. The archetypal example is the highly optimized forest fire model. Here we give an analytic solution for this model which explains the origin of the power laws. We also generalize the model to incorporate risk aversion, which results in truncation of the tails of the power law so that the probability of disastrously large events is dramatically lowered, giving the system more robustness. △ Less

Submitted 19 February, 2002; originally announced February 2002.

Comments: 11 pages, 2 figures

Journal ref: Phys. Rev. Lett. 89, 028301 (2002)

Community structure in social and biological networks

Authors: Michelle Girvan, M. E. J. Newman

Abstract: A number of recent studies have focused on the statistical properties of networked systems such as social networks and the World-Wide Web. Researchers have concentrated particularly on a few properties which seem to be common to many networks: the small-world property, power-law degree distributions, and network transitivity. In this paper, we highlight another property which is found in many ne… ▽ More A number of recent studies have focused on the statistical properties of networked systems such as social networks and the World-Wide Web. Researchers have concentrated particularly on a few properties which seem to be common to many networks: the small-world property, power-law degree distributions, and network transitivity. In this paper, we highlight another property which is found in many networks, the property of community structure, in which network nodes are joined together in tightly-knit groups between which there are only looser connections. We propose a new method for detecting such communities, built around the idea of using centrality indices to find community boundaries. We test our method on computer generated and real-world graphs whose community structure is already known, and find that it detects this known structure with high sensitivity and reliability. We also apply the method to two networks whose community structure is not well-known - a collaboration network and a food web - and find that it detects significant and informative community divisions in both cases. △ Less

Submitted 7 December, 2001; originally announced December 2001.

Comments: 8 pages, 7 figures

Journal ref: Proc. Natl. Acad. Sci. USA 99, 7821-7826 (2002)

A Simple Model of Epidemics with Pathogen Mutation

Authors: Michelle Girvan, Duncan S. Callaway, M. E. J. Newman, Steven H. Strogatz

Abstract: We study how the interplay between the memory immune response and pathogen mutation affects epidemic dynamics in two related models. The first explicitly models pathogen mutation and individual memory immune responses, with contacted individuals becoming infected only if they are exposed to strains that are significantly different from other strains in their memory repertoire. The second model i… ▽ More We study how the interplay between the memory immune response and pathogen mutation affects epidemic dynamics in two related models. The first explicitly models pathogen mutation and individual memory immune responses, with contacted individuals becoming infected only if they are exposed to strains that are significantly different from other strains in their memory repertoire. The second model is a reduction of the first to a system of difference equations. In this case, individuals spend a fixed amount of time in a generalized immune class. In both models, we observe four fundamentally different types of behavior, depending on parameters: (1) pathogen extinction due to lack of contact between individuals, (2) endemic infection (3) periodic epidemic outbreaks, and (4) one or more outbreaks followed by extinction of the epidemic due to extremely low minima in the oscillations. We analyze both models to determine the location of each transition. Our main result is that pathogens in highly connected populations must mutate rapidly in order to remain viable. △ Less

Submitted 18 May, 2001; originally announced May 2001.

Comments: 9 pages, 11 figures