-
Learning Continuous Chaotic Attractors with a Reservoir Computer
Authors:
Lindsay M. Smith,
Jason Z. Kim,
Zhixin Lu,
Dani S. Bassett
Abstract:
Neural systems are well known for their ability to learn and store information as memories. Even more impressive is their ability to abstract these memories to create complex internal representations, enabling advanced functions such as the spatial manipulation of mental representations. While recurrent neural networks (RNNs) are capable of representing complex information, the exact mechanisms of…
▽ More
Neural systems are well known for their ability to learn and store information as memories. Even more impressive is their ability to abstract these memories to create complex internal representations, enabling advanced functions such as the spatial manipulation of mental representations. While recurrent neural networks (RNNs) are capable of representing complex information, the exact mechanisms of how dynamical neural systems perform abstraction are still not well-understood, thereby hindering the development of more advanced functions. Here, we train a 1000-neuron RNN -- a reservoir computer (RC) -- to abstract a continuous dynamical attractor memory from isolated examples of dynamical attractor memories. Further, we explain the abstraction mechanism with new theory. By training the RC on isolated and shifted examples of either stable limit cycles or chaotic Lorenz attractors, the RC learns a continuum of attractors, as quantified by an extra Lyapunov exponent equal to zero. We propose a theoretical mechanism of this abstraction by combining ideas from differentiable generalized synchronization and feedback dynamics. Our results quantify abstraction in simple neural systems, enabling us to design artificial RNNs for abstraction, and leading us towards a neural basis of abstraction.
△ Less
Submitted 16 October, 2021;
originally announced October 2021.
-
Partitioning Networks with Node Attributes by Compressing Information Flow
Authors:
Laura M. Smith,
Linhong Zhu,
Kristina Lerman,
Allon G. Percus
Abstract:
Real-world networks are often organized as modules or communities of similar nodes that serve as functional units. These networks are also rich in content, with nodes having distinguishing features or attributes. In order to discover a network's modular structure, it is necessary to take into account not only its links but also node attributes. We describe an information-theoretic method that iden…
▽ More
Real-world networks are often organized as modules or communities of similar nodes that serve as functional units. These networks are also rich in content, with nodes having distinguishing features or attributes. In order to discover a network's modular structure, it is necessary to take into account not only its links but also node attributes. We describe an information-theoretic method that identifies modules by compressing descriptions of information flow on a network. Our formulation introduces node content into the description of information flow, which we then minimize to discover groups of nodes with similar attributes that also tend to trap the flow of information. The method has several advantages: it is conceptually simple and does not require ad-hoc parameters to specify the number of modules or to control the relative contribution of links and node attributes to network structure. We apply the proposed method to partition real-world networks with known community structure. We demonstrate that adding node attributes helps recover the underlying community structure in content-rich networks more effectively than using links alone. In addition, we show that our method is faster and more accurate than alternative state-of-the-art algorithms.
△ Less
Submitted 16 May, 2014;
originally announced May 2014.
-
Stochastic Models Predict User Behavior in Social Media
Authors:
Tad Hogg,
Kristina Lerman,
Laura M. Smith
Abstract:
User response to contributed content in online social media depends on many factors. These include how the site lays out new content, how frequently the user visits the site, how many friends the user follows, how active these friends are, as well as how interesting or useful the content is to the user. We present a stochastic modeling framework that relates a user's behavior to details of the sit…
▽ More
User response to contributed content in online social media depends on many factors. These include how the site lays out new content, how frequently the user visits the site, how many friends the user follows, how active these friends are, as well as how interesting or useful the content is to the user. We present a stochastic modeling framework that relates a user's behavior to details of the site's user interface and user activity and describe a procedure for estimating model parameters from available data. We apply the model to study discussions of controversial topics on Twitter, specifically, to predict how followers of an advocate for a topic respond to the advocate's posts. We show that a model of user behavior that explicitly accounts for a user transitioning through a series of states before responding to an advocate's post better predicts response than models that fail to take these states into account. We demonstrate other benefits of stochastic models, such as their ability to identify users who are highly interested in advocate's posts.
△ Less
Submitted 12 August, 2013;
originally announced August 2013.
-
Spectral Clustering with Epidemic Diffusion
Authors:
Laura M. Smith,
Kristina Lerman,
Cristina Garcia-Cardona,
Allon G. Percus,
Rumi Ghosh
Abstract:
Spectral clustering is widely used to partition graphs into distinct modules or communities. Existing methods for spectral clustering use the eigenvalues and eigenvectors of the graph Laplacian, an operator that is closely associated with random walks on graphs. We propose a new spectral partitioning method that exploits the properties of epidemic diffusion. An epidemic is a dynamic process that,…
▽ More
Spectral clustering is widely used to partition graphs into distinct modules or communities. Existing methods for spectral clustering use the eigenvalues and eigenvectors of the graph Laplacian, an operator that is closely associated with random walks on graphs. We propose a new spectral partitioning method that exploits the properties of epidemic diffusion. An epidemic is a dynamic process that, unlike the random walk, simultaneously transitions to all the neighbors of a given node. We show that the replicator, an operator describing epidemic diffusion, is equivalent to the symmetric normalized Laplacian of a reweighted graph with edges reweighted by the eigenvector centralities of their incident nodes. Thus, more weight is given to edges connecting more central nodes. We describe a method that partitions the nodes based on the componentwise ratio of the replicator's second eigenvector to the first, and compare its performance to traditional spectral clustering techniques on synthetic graphs with known community structure. We demonstrate that the replicator gives preference to dense, clique-like structures, enabling it to more effectively discover communities that may be obscured by dense intercommunity linking.
△ Less
Submitted 4 October, 2013; v1 submitted 11 March, 2013;
originally announced March 2013.
