-
Tuning the activation function to optimize the forecast horizon of a reservoir computer
Authors:
Lauren A. Hurley,
Juan G. Restrepo,
Sean E. Shaheen
Abstract:
Reservoir computing is a machine learning framework where the readouts from a nonlinear system (the reservoir) are trained so that the output from the reservoir, when forced with an input signal, reproduces a desired output signal. A common implementation of reservoir computers is to use a recurrent neural network as the reservoir. The design of this network can have significant effects on the per…
▽ More
Reservoir computing is a machine learning framework where the readouts from a nonlinear system (the reservoir) are trained so that the output from the reservoir, when forced with an input signal, reproduces a desired output signal. A common implementation of reservoir computers is to use a recurrent neural network as the reservoir. The design of this network can have significant effects on the performance of the reservoir computer. In this paper we study the effect of the node activation function on the ability of reservoir computers to learn and predict chaotic time series. We find that the Forecast Horizon (FH), the time during which the reservoir's predictions remain accurate, can vary by an order of magnitude across a set of 16 activation functions used in machine learning. By using different functions from this set, and by modifying their parameters, we explore whether the entropy of node activation levels or the curvature of the activation functions determine the predictive ability of the reservoirs. We find that the FH is low when the activation function is used in a region where it has low curvature, and a positive correlation between curvature and FH. For the activation functions studied we find that the largest FH generally occurs at intermediate levels of the entropy of node activation levels. Our results show that the performance of reservoir computers is very sensitive to the activation function shape. Therefore, modifying this shape in hyperparameter optimization algorithms can lead to improvements in reservoir computer performance.
△ Less
Submitted 20 December, 2023;
originally announced December 2023.
-
Chemically inspired Erdős-Rényi oriented hypergraphs
Authors:
Angel Garcia-Chung,
Marisol Bermúdez-Montaña,
Peter F. Stadler,
Jürgen Jost,
Guillermo Restrepo
Abstract:
High-order structures have been recognised as suitable models for systems going beyond the binary relationships for which graph models are appropriate. Despite their importance and surge in research on these structures, their random cases have been only recently become subjects of interest. One of these high-order structures is the oriented hypergraph, which relates couples of subsets of an arbitr…
▽ More
High-order structures have been recognised as suitable models for systems going beyond the binary relationships for which graph models are appropriate. Despite their importance and surge in research on these structures, their random cases have been only recently become subjects of interest. One of these high-order structures is the oriented hypergraph, which relates couples of subsets of an arbitrary number of vertices. Here we develop the Erdős-Rényi model for oriented hypergraphs, which corresponds to the random realisation of oriented hyperedges of the complete oriented hypergraph. A particular feature of random oriented hypergraphs is that the ratio between their expected number of oriented hyperedges and their expected degree or size is 3/2 for large number of vertices. We highlight the suitability of oriented hypergraphs for modelling large collections of chemical reactions and the importance of random oriented hypergraphs to analyse the unfolding of chemistry.
△ Less
Submitted 12 September, 2023;
originally announced September 2023.
-
Detecting disturbances in network-coupled dynamical systems with machine learning
Authors:
Per Sebastian Skardal,
Juan G. Restrepo
Abstract:
Identifying disturbances in network-coupled dynamical systems without knowledge of the disturbances or underlying dynamics is a problem with a wide range of applications. For example, one might want to know which nodes in the network are being disturbed and identify the type of disturbance. Here we present a model-free method based on machine learning to identify such unknown disturbances based on…
▽ More
Identifying disturbances in network-coupled dynamical systems without knowledge of the disturbances or underlying dynamics is a problem with a wide range of applications. For example, one might want to know which nodes in the network are being disturbed and identify the type of disturbance. Here we present a model-free method based on machine learning to identify such unknown disturbances based only on prior observations of the system when forced by a known training function. We find that this method is able to identify the locations and properties of many different types of unknown disturbances using a variety of known forcing functions. We illustrate our results both with linear and nonlinear disturbances using food web and neuronal activity models. Finally, we discuss how to scale our method to large networks.
△ Less
Submitted 24 July, 2023;
originally announced July 2023.
-
Suppressing unknown disturbances to dynamical systems using machine learning
Authors:
Juan G. Restrepo,
Clayton P. Byers,
Per Sebastian Skardal
Abstract:
Identifying and suppressing unknown disturbances to dynamical systems is a problem with applications in many different fields. Here we present a model-free method to identify and suppress an unknown disturbance to an unknown system based only on previous observations of the system under the influence of a known forcing function. We find that, under very mild restrictions on the training function,…
▽ More
Identifying and suppressing unknown disturbances to dynamical systems is a problem with applications in many different fields. Here we present a model-free method to identify and suppress an unknown disturbance to an unknown system based only on previous observations of the system under the influence of a known forcing function. We find that, under very mild restrictions on the training function, our method is able to robustly identify and suppress a large class of unknown disturbances. We illustrate our scheme with the identification of both deterministic and stochastic unknown disturbances to an analog electric chaotic circuit and with a numerical example where a chaotic disturbance to the Lorenz system is identified and suppressed.
△ Less
Submitted 12 June, 2024; v1 submitted 19 June, 2023;
originally announced July 2023.
-
Opinion disparity in hypergraphs with community structure
Authors:
Nicholas W. Landry,
Juan G. Restrepo
Abstract:
The division of a social group into subgroups with opposing opinions, which we refer to as opinion disparity, is a prevalent phenomenon in society. This phenomenon has been modeled by including mechanisms such as opinion homophily, bounded confidence interactions, and social reinforcement mechanisms. In this paper we study a complementary mechanism for the formation of opinion disparity based on h…
▽ More
The division of a social group into subgroups with opposing opinions, which we refer to as opinion disparity, is a prevalent phenomenon in society. This phenomenon has been modeled by including mechanisms such as opinion homophily, bounded confidence interactions, and social reinforcement mechanisms. In this paper we study a complementary mechanism for the formation of opinion disparity based on higher-order interactions, i.e., simultaneous interactions between multiple agents. We present an extension of the planted partition model for uniform hypergraphs as a simple model of community structure and consider the hypergraph SIS model on a hypergraph with two communities where the binary ideology can spread via links (pairwise interactions) and triangles (three-way interactions). We approximate this contagion process with a mean-field model and find that for strong enough community structure, the two communities can hold very different average opinions. We determine the regimes of structural and infectious parameters for which this opinion disparity can exist and find that the existence of these disparities is much more sensitive to the triangle community structure than to the link community structure. We show that the existence and type of opinion disparities are extremely sensitive to differences in the sizes of the two communities.
△ Less
Submitted 24 August, 2023; v1 submitted 27 February, 2023;
originally announced February 2023.
-
On Clustering Categories of Categorical Predictors in Generalized Linear Models
Authors:
Emilio Carrizosa,
Marcela Galvis Restrepo,
Dolores Romero Morales
Abstract:
We propose a method to reduce the complexity of Generalized Linear Models in the presence of categorical predictors. The traditional one-hot encoding, where each category is represented by a dummy variable, can be wasteful, difficult to interpret, and prone to overfitting, especially when dealing with high-cardinality categorical predictors. This paper addresses these challenges by finding a reduc…
▽ More
We propose a method to reduce the complexity of Generalized Linear Models in the presence of categorical predictors. The traditional one-hot encoding, where each category is represented by a dummy variable, can be wasteful, difficult to interpret, and prone to overfitting, especially when dealing with high-cardinality categorical predictors. This paper addresses these challenges by finding a reduced representation of the categorical predictors by clustering their categories. This is done through a numerical method which aims to preserve (or even, improve) accuracy, while reducing the number of coefficients to be estimated for the categorical predictors. Thanks to its design, we are able to derive a proximity measure between categories of a categorical predictor that can be easily visualized. We illustrate the performance of our approach in real-world classification and count-data datasets where we see that clustering the categorical predictors reduces complexity substantially without harming accuracy.
△ Less
Submitted 19 October, 2021;
originally announced October 2021.
-
Dirac synchronization is rhythmic and explosive
Authors:
Lucille Calmon,
Juan G. Restrepo,
Joaquín J. Torres,
Ginestra Bianconi
Abstract:
Topological signals defined on nodes, links and higher dimensional simplices define the dynamical state of a network or of a simplicial complex. As such, topological signals are attracting increasing attention in network theory, dynamical systems, signal processing and machine learning. Topological signals defined on the nodes are typically studied in network dynamics, while topological signals de…
▽ More
Topological signals defined on nodes, links and higher dimensional simplices define the dynamical state of a network or of a simplicial complex. As such, topological signals are attracting increasing attention in network theory, dynamical systems, signal processing and machine learning. Topological signals defined on the nodes are typically studied in network dynamics, while topological signals defined on links are much less explored. Here we investigate Dirac synchronization, describing locally coupled topological signals defined on the nodes and on the links of a network, and treated using the topological Dirac operator. The dynamics of signals defined on the nodes is affected by a phase lag depending on the dynamical state of nearby links and vice versa. We show that Dirac synchronization on a fully connected network is explosive with a hysteresis loop characterized by a discontinuous forward transition and a continuous backward transition. The analytical investigation of the phase diagram provides a theoretical understanding of this topological explosive synchronization. The model also displays an exotic coherent synchronized phase, also called rhythmic phase, characterized by non-stationary order parameters which can shed light on topological mechanisms for the emergence of brain rhythms.
△ Less
Submitted 3 September, 2022; v1 submitted 11 July, 2021;
originally announced July 2021.
-
Using Machine Learning to Assess Short Term Causal Dependence and Infer Network Links
Authors:
Amitava Banerjee,
Jaideep Pathak,
Rajarshi Roy,
Juan G. Restrepo,
Edward Ott
Abstract:
We introduce and test a general machine-learning-based technique for the inference of short term causal dependence between state variables of an unknown dynamical system from time series measurements of its state variables. Our technique leverages the results of a machine learning process for short time prediction to achieve our goal. The basic idea is to use the machine learning to estimate the e…
▽ More
We introduce and test a general machine-learning-based technique for the inference of short term causal dependence between state variables of an unknown dynamical system from time series measurements of its state variables. Our technique leverages the results of a machine learning process for short time prediction to achieve our goal. The basic idea is to use the machine learning to estimate the elements of the Jacobian matrix of the dynamical flow along an orbit. The type of machine learning that we employ is reservoir computing. We present numerical tests on link inference of a network of interacting dynamical nodes. It is seen that dynamical noise can greatly enhance the effectiveness of our technique, while observational noise degrades the effectiveness. We believe that the competition between these two opposing types of noise will be the key factor determining the success of causal inference in many of the most important application situations.
△ Less
Submitted 5 December, 2019;
originally announced December 2019.
-
Formal structure of periodic system of elements
Authors:
Wilmer Leal,
Guillermo Restrepo
Abstract:
For more than 150 years the structure of the periodic system of the chemical elements has intensively motivated research in different areas of chemistry and physics. However, there is still no unified picture of what a periodic system is. Herein, based on the relations of order and similarity, we report a formal mathematical structure for the periodic system, which corresponds to an ordered hyperg…
▽ More
For more than 150 years the structure of the periodic system of the chemical elements has intensively motivated research in different areas of chemistry and physics. However, there is still no unified picture of what a periodic system is. Herein, based on the relations of order and similarity, we report a formal mathematical structure for the periodic system, which corresponds to an ordered hypergraph. It is shown that the current periodic system of chemical elements is an instance of the general structure. The definition is used to devise a tailored periodic system of polarizability of single covalent bonds, where order relationships are quantified within subsets of similar bonds and among these classes. The generalised periodic system allows envisioning periodic systems in other disciplines of science and humanities.
△ Less
Submitted 27 February, 2019;
originally announced February 2019.
-
Forman-Ricci Curvature for Hypergraphs
Authors:
Wilmer Leal,
Guillermo Restrepo,
Peter F. Stadler,
Jürgen Jost
Abstract:
In contrast to graph-based models for complex networks, hypergraphs are more general structures going beyond binary relations of graphs. For graphs, statistics gauging different aspects of their structures have been devised and there is undergoing research for devising them for hypergraphs. Forman-Ricci curvature is a statistics for graphs, which is based on Riemannian geometry, and that stresses…
▽ More
In contrast to graph-based models for complex networks, hypergraphs are more general structures going beyond binary relations of graphs. For graphs, statistics gauging different aspects of their structures have been devised and there is undergoing research for devising them for hypergraphs. Forman-Ricci curvature is a statistics for graphs, which is based on Riemannian geometry, and that stresses the relational character of vertices in a network through the analysis of edges rather than vertices. In spite of the different applications of this curvature, it has not yet been formulated for hypergraphs. Here we devise the Forman-Ricci curvature for directed and undirected hypergraphs, where the curvature for graphs is a particular case. We report its upper and lower bounds and the respective bounds for the graph case. The curvature quantifies the trade-off between hyperedge(arc) size and the degree of participation of hyperedge(arc) vertices in other hyperedges(arcs). We calculated the curvature for two large networks: Wikipedia vote network and \emph{Escherichia coli} metabolic network. In the first case the curvature is ruled by hyperedge size, while in the second by hyperedge degree. We found that the number of users involved in Wikipedia elections goes hand-in-hand with the participation of experienced users. The curvature values of the metabolic network allowed detecting redundant and bottle neck reactions. It is found that ADP phosphorilation is the metabolic bottle neck reaction but that the reverse reaction is not that central for the metabolism.
△ Less
Submitted 19 November, 2018;
originally announced November 2018.
-
Downlink Analysis for a Heterogeneous Cellular Network
Authors:
Prasanna Madhusudhanan,
Juan G. Restrepo,
Youjian,
Liu,
Timothy X Brown
Abstract:
In this paper, a comprehensive study of the the downlink performance in a heterogeneous cellular network (or hetnet) is conducted. A general hetnet model is considered consisting of an arbitrary number of open-access and closed-access tier of base stations (BSs) arranged according to independent homogeneous Poisson point processes. The BSs of each tier have a constant transmission power, random fa…
▽ More
In this paper, a comprehensive study of the the downlink performance in a heterogeneous cellular network (or hetnet) is conducted. A general hetnet model is considered consisting of an arbitrary number of open-access and closed-access tier of base stations (BSs) arranged according to independent homogeneous Poisson point processes. The BSs of each tier have a constant transmission power, random fading coefficient with an arbitrary distribution and arbitrary path-loss exponent of the power-law path-loss model. For such a system, analytical characterizations for the coverage probability and average rate at an arbitrary mobile-station (MS), and average per-tier load are derived for both the max-SINR connectivity and nearest-BS connectivity models. Using stochastic ordering, interesting properties and simplifications for the hetnet downlink performance are derived by relating these two connectivity models to the maximum instantaneous received power (MIRP) connectivity model and the maximum biased received power (MBRP) connectivity models, respectively, providing good insights about the hetnets and the downlink performance in these complex networks. Furthermore, the results also demonstrate the effectiveness and analytical tractability of the stochastic geometric approach to study the hetnet performance.
△ Less
Submitted 29 March, 2014;
originally announced March 2014.
-
Downlink Coverage Analysis in a Heterogeneous Cellular Network
Authors:
Prasanna Madhusudhanan,
Juan G. Restrepo,
Youjian,
Liu,
Timothy X. Brown
Abstract:
In this paper, we consider the downlink signal-to-interference-plus-noise ratio (SINR) analysis in a heterogeneous cellular network with K tiers. Each tier is characterized by a base-station (BS) arrangement according to a homogeneous Poisson point process with certain BS density, transmission power, random shadow fading factors with arbitrary distribution, arbitrary path-loss exponent and a certa…
▽ More
In this paper, we consider the downlink signal-to-interference-plus-noise ratio (SINR) analysis in a heterogeneous cellular network with K tiers. Each tier is characterized by a base-station (BS) arrangement according to a homogeneous Poisson point process with certain BS density, transmission power, random shadow fading factors with arbitrary distribution, arbitrary path-loss exponent and a certain bias towards admitting the mobile-station (MS). The MS associates with the BS that has the maximum SINR under the open access cell association scheme. For such a general setting, we provide an analytical characterization of the coverage probability at the MS.
△ Less
Submitted 20 June, 2012;
originally announced June 2012.
-
Stochastic Ordering based Carrier-to-Interference Ratio Analysis for the Shotgun Cellular Systems
Authors:
Prasanna Madhusudhanan,
Juan G. Restrepo,
Youjian,
Liu,
Timothy X Brown,
Kenneth R. Baker
Abstract:
A simple analytical tool based on stochastic ordering is developed to compare the distributions of carrier-to-interference ratio at the mobile station of two cellular systems where the base stations are distributed randomly according to certain non-homogeneous Poisson point processes. The comparison is conveniently done by studying only the base station densities without having to solve for the di…
▽ More
A simple analytical tool based on stochastic ordering is developed to compare the distributions of carrier-to-interference ratio at the mobile station of two cellular systems where the base stations are distributed randomly according to certain non-homogeneous Poisson point processes. The comparison is conveniently done by studying only the base station densities without having to solve for the distributions of the carrier-to-interference ratio, that are often hard to obtain.
△ Less
Submitted 14 October, 2011;
originally announced October 2011.
-
Multi-tier Network Performance Analysis using a Shotgun Cellular System
Authors:
Prasanna Madhusudhanan,
Juan G. Restrepo,
Youjian,
Liu,
Timothy X Brown,
Kenneth R. Baker
Abstract:
This paper studies the carrier-to-interference ratio (CIR) and carrier-to-interference-plus-noise ratio (CINR) performance at the mobile station (MS) within a multi-tier network composed of M tiers of wireless networks, with each tier modeled as the homogeneous n-dimensional (n-D, n=1,2, and 3) shotgun cellular system, where the base station (BS) distribution is given by the homogeneous Poisson po…
▽ More
This paper studies the carrier-to-interference ratio (CIR) and carrier-to-interference-plus-noise ratio (CINR) performance at the mobile station (MS) within a multi-tier network composed of M tiers of wireless networks, with each tier modeled as the homogeneous n-dimensional (n-D, n=1,2, and 3) shotgun cellular system, where the base station (BS) distribution is given by the homogeneous Poisson point process in n-D. The CIR and CINR at the MS in a single tier network are thoroughly analyzed to simplify the analysis of the multi-tier network. For the multi-tier network with given system parameters, the following are the main results of this paper: (1) semi-analytical expressions for the tail probabilities of CIR and CINR; (2) a closed form expression for the tail probability of CIR in the range [1,Infinity); (3) a closed form expression for the tail probability of an approximation to CIR in the entire range [0,Infinity); (4) a lookup table based approach for obtaining the tail probability of CINR, and (5) the study of the effect of shadow fading and BSs with ideal sectorized antennas on the CIR and CINR. Based on these results, it is shown that, in a practical cellular system, the installation of additional wireless networks (microcells, picocells and femtocells) with low power BSs over the already existing macrocell network will always improve the CINR performance at the MS.
△ Less
Submitted 14 October, 2011;
originally announced October 2011.
-
Network connectivity during mergers and growth: optimizing the addition of a module
Authors:
Dane Taylor,
Juan G. Restrepo
Abstract:
The principal eigenvalue $λ$ of a network's adjacency matrix often determines dynamics on the network (e.g., in synchronization and spreading processes) and some of its structural properties (e.g., robustness against failure or attack) and is therefore a good indicator for how ``strongly'' a network is connected. We study how $λ$ is modified by the addition of a module, or community, which has bro…
▽ More
The principal eigenvalue $λ$ of a network's adjacency matrix often determines dynamics on the network (e.g., in synchronization and spreading processes) and some of its structural properties (e.g., robustness against failure or attack) and is therefore a good indicator for how ``strongly'' a network is connected. We study how $λ$ is modified by the addition of a module, or community, which has broad applications, ranging from those involving a single modification (e.g., introduction of a drug into a biological process) to those involving repeated additions (e.g., power-grid and transit development). We describe how to optimally connect the module to the network to either maximize or minimize the shift in $λ$, noting several applications of directing dynamics on networks.
△ Less
Submitted 6 June, 2011; v1 submitted 23 February, 2011;
originally announced February 2011.
-
Downlink Performance Analysis for a Generalized Shotgun Cellular System
Authors:
Prasanna Madhusudhanan,
Juan G. Restrepo,
Youjian Liu,
Timothy X Brown,
Kenneth R. Baker
Abstract:
In this paper, we analyze the signal-to-interference-plus-noise ratio (SINR) performance at a mobile station (MS) in a random cellular network. The cellular network is formed by base-stations (BSs) placed in a one, two or three dimensional space according to a possibly non-homogeneous Poisson point process, which is a generalization of the so-called shotgun cellular system. We develop a sequence o…
▽ More
In this paper, we analyze the signal-to-interference-plus-noise ratio (SINR) performance at a mobile station (MS) in a random cellular network. The cellular network is formed by base-stations (BSs) placed in a one, two or three dimensional space according to a possibly non-homogeneous Poisson point process, which is a generalization of the so-called shotgun cellular system. We develop a sequence of equivalence relations for the SCSs and use them to derive semi-analytical expressions for the coverage probability at the MS when the transmissions from each BS may be affected by random fading with arbitrary distributions as well as attenuation following arbitrary path-loss models. For homogeneous Poisson point processes in the interference-limited case with power-law path-loss model, we show that the SINR distribution is the same for all fading distributions and is not a function of the base station density. In addition, the influence of random transmission powers, power control, multiple channel reuse groups on the downlink performance are also discussed. The techniques developed for the analysis of SINR have applications beyond cellular networks and can be used in similar studies for cognitive radio networks, femtocell networks and other heterogeneous and multi-tier networks.
△ Less
Submitted 12 December, 2012; v1 submitted 20 February, 2010;
originally announced February 2010.
