Showing 1–14 of 14 results for author: Pomerance, A

Stabilizing Machine Learning Prediction of Dynamics: Noise and Noise-inspired Regularization

Authors: Alexander Wikner, Joseph Harvey, Michelle Girvan, Brian R. Hunt, Andrew Pomerance, Thomas Antonsen, Edward Ott

Abstract: Recent work has shown that machine learning (ML) models can be trained to accurately forecast the dynamics of unknown chaotic dynamical systems. Short-term predictions of the state evolution and long-term predictions of the statistical patterns of the dynamics (``climate'') can be produced by employing a feedback loop, whereby the model is trained to predict forward one time step, then the model o… ▽ More Recent work has shown that machine learning (ML) models can be trained to accurately forecast the dynamics of unknown chaotic dynamical systems. Short-term predictions of the state evolution and long-term predictions of the statistical patterns of the dynamics (``climate'') can be produced by employing a feedback loop, whereby the model is trained to predict forward one time step, then the model output is used as input for multiple time steps. In the absence of mitigating techniques, however, this technique can result in artificially rapid error growth. In this article, we systematically examine the technique of adding noise to the ML model input during training to promote stability and improve prediction accuracy. Furthermore, we introduce Linearized Multi-Noise Training (LMNT), a regularization technique that deterministically approximates the effect of many small, independent noise realizations added to the model input during training. Our case study uses reservoir computing, a machine-learning method using recurrent neural networks, to predict the spatiotemporal chaotic Kuramoto-Sivashinsky equation. We find that reservoir computers trained with noise or with LMNT produce climate predictions that appear to be indefinitely stable and have a climate very similar to the true system, while reservoir computers trained without regularization are unstable. Compared with other regularization techniques that yield stability in some cases, we find that both short-term and climate predictions from reservoir computers trained with noise or with LMNT are substantially more accurate. Finally, we show that the deterministic aspect of our LMNT regularization facilitates fast hyperparameter tuning when compared to training with noise. △ Less

Submitted 12 December, 2022; v1 submitted 9 November, 2022; originally announced November 2022.

Comments: 39 pages, 8 figures, 5 tables

On the Edge Derivative of the Normalized Laplacian with Applications to Kemeny's Constant

Authors: Connor Albright, Kimberly P. Hadaway, Ari Holcombe Pomerance, Joel Jeffries, Kate J. Lorenzen, Abigail K. Nix

Abstract: In a connected graph, Kemeny's constant gives the expected time of a random walk from an arbitrary vertex $x$ to reach a randomly-chosen vertex $y$. Because of this, Kemeny's constant can be interpreted as a measure of how well a graph is connected. It is generally unknown how the addition or removal of edges affects Kemeny's constant. Inspired by the directional derivative of the normalized Lapla… ▽ More In a connected graph, Kemeny's constant gives the expected time of a random walk from an arbitrary vertex $x$ to reach a randomly-chosen vertex $y$. Because of this, Kemeny's constant can be interpreted as a measure of how well a graph is connected. It is generally unknown how the addition or removal of edges affects Kemeny's constant. Inspired by the directional derivative of the normalized Laplacian, we derive the directional derivative of Kemeny's constant for several graph families. In addition, we find sharp bounds for the directional derivative of an eigenvalue of the normalized Laplacian and bounds for the directional derivative of Kemeny's constant. △ Less

Submitted 13 July, 2023; v1 submitted 2 November, 2022; originally announced November 2022.

Mathematical Model of Strong Physically Unclonable Functions Based on Hybrid Boolean Networks

Authors: Noeloikeau Charlot, Daniel J. Gauthier, Daniel Canaday, Andrew Pomerance

Abstract: We introduce a mathematical framework for simulating Hybrid Boolean Network (HBN) Physically Unclonable Functions (PUFs, HBN-PUFs). We verify that the model is able to reproduce the experimentally observed PUF statistics for uniqueness $μ_{inter}$ and reliability $μ_{intra}$ obtained from experiments of HBN-PUFs on Cyclone V FPGAs. Our results suggest that the HBN-PUF is a true `strong' PUF in the… ▽ More We introduce a mathematical framework for simulating Hybrid Boolean Network (HBN) Physically Unclonable Functions (PUFs, HBN-PUFs). We verify that the model is able to reproduce the experimentally observed PUF statistics for uniqueness $μ_{inter}$ and reliability $μ_{intra}$ obtained from experiments of HBN-PUFs on Cyclone V FPGAs. Our results suggest that the HBN-PUF is a true `strong' PUF in the sense that its security properties depend exponentially on both the manufacturing variation and the challenge-response space. Our Python simulation methods are open-source and available at https://github.com/Noeloikeau/networkm. △ Less

Submitted 19 July, 2022; originally announced July 2022.

Comments: Presented at HOST 2022 conference. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

A Meta-learning Approach to Reservoir Computing: Time Series Prediction with Limited Data

Authors: Daniel Canaday, Andrew Pomerance, Michelle Girvan

Abstract: Recent research has established the effectiveness of machine learning for data-driven prediction of the future evolution of unknown dynamical systems, including chaotic systems. However, these approaches require large amounts of measured time series data from the process to be predicted. When only limited data is available, forecasters are forced to impose significant model structure that may or m… ▽ More Recent research has established the effectiveness of machine learning for data-driven prediction of the future evolution of unknown dynamical systems, including chaotic systems. However, these approaches require large amounts of measured time series data from the process to be predicted. When only limited data is available, forecasters are forced to impose significant model structure that may or may not accurately represent the process of interest. In this work, we present a Meta-learning Approach to Reservoir Computing (MARC), a data-driven approach to automatically extract an appropriate model structure from experimentally observed "related" processes that can be used to vastly reduce the amount of data required to successfully train a predictive model. We demonstrate our approach on a simple benchmark problem, where it beats the state of the art meta-learning techniques, as well as a challenging chaotic problem. △ Less

Submitted 7 October, 2021; originally announced October 2021.

High-Resolution Waveform Capture Device on a Cyclone-V FPGA

Authors: Noeloikeau Charlot, Daniel J. Gauthier, Andrew Pomerance

Abstract: We introduce the waveform capture device (WCD), a flexible measurement system capable of recording complex digital signals on trillionth-of-a-second (ps) time scales. The WCD is implemented via modular code on an off-the-shelf field-programmable gate-array (FPGA, Intel/Altera Cyclone V), and incorporates both time-to-digital converter (TDC) and digital storage oscilloscope (DSO) functionality. The… ▽ More We introduce the waveform capture device (WCD), a flexible measurement system capable of recording complex digital signals on trillionth-of-a-second (ps) time scales. The WCD is implemented via modular code on an off-the-shelf field-programmable gate-array (FPGA, Intel/Altera Cyclone V), and incorporates both time-to-digital converter (TDC) and digital storage oscilloscope (DSO) functionality. The device captures a waveform by taking snapshots of a signal as it propagates down an ultra-fast transmission line known as a carry chain (CC). It is calibrated via a novel dynamic phase-shifting (DPS) method that requires substantially less data and resources than the state-of-the-art. Using DPS, we find the measurement resolution - or mean propagation delay from one CC element to the next - to be 4.91 +/- 0.04 ps (4.54 +/- 0.02 ps) for a pulse of logic high (low). Similarly, we find the single-shot precision - or mean error on the timing of the waveform - to be 29.52 ps (27.14 ps) for pulses of logic high (low). We verify these findings by reproducing commercial oscilloscope measurements of asynchronous ring-oscillators on FPGAs, finding the mean pulse width to be 0.240 +/- 0.002 ns per inverter gate. Finally, we present a careful analysis of design constraints, introduce a novel error correction algorithm, and sketch a simple extension to the analog domain. We also provide the Verilog code instantiating the our design on an FPGA in an Appendix, and make our methods available as an open-source Python library at https://github.com/Noeloikeau/fpyga. △ Less

Submitted 16 August, 2021; originally announced September 2021.

Model-Free Control of Dynamical Systems with Deep Reservoir Computing

Authors: Daniel Canaday, Andrew Pomerance, Daniel J Gauthier

Abstract: We propose and demonstrate a nonlinear control method that can be applied to unknown, complex systems where the controller is based on a type of artificial neural network known as a reservoir computer. In contrast to many modern neural-network-based control techniques, which are robust to system uncertainties but require a model nonetheless, our technique requires no prior knowledge of the system… ▽ More We propose and demonstrate a nonlinear control method that can be applied to unknown, complex systems where the controller is based on a type of artificial neural network known as a reservoir computer. In contrast to many modern neural-network-based control techniques, which are robust to system uncertainties but require a model nonetheless, our technique requires no prior knowledge of the system and is thus model-free. Further, our approach does not require an initial system identification step, resulting in a relatively simple and efficient learning process. Reservoir computers are well-suited to the control problem because they require small training data sets and remarkably low training times. By iteratively training and adding layers of reservoir computers to the controller, a precise and efficient control law is identified quickly. With examples on both numerical and high-speed experimental systems, we demonstrate that our approach is capable of controlling highly complex dynamical systems that display deterministic chaos to nontrivial target trajectories. △ Less

Submitted 5 October, 2020; originally announced October 2020.

Combining Machine Learning with Knowledge-Based Modeling for Scalable Forecasting and Subgrid-Scale Closure of Large, Complex, Spatiotemporal Systems

Authors: Alexander Wikner, Jaideep Pathak, Brian Hunt, Michelle Girvan, Troy Arcomano, Istvan Szunyogh, Andrew Pomerance, Edward Ott

Abstract: We consider the commonly encountered situation (e.g., in weather forecasting) where the goal is to predict the time evolution of a large, spatiotemporally chaotic dynamical system when we have access to both time series data of previous system states and an imperfect model of the full system dynamics. Specifically, we attempt to utilize machine learning as the essential tool for integrating the us… ▽ More We consider the commonly encountered situation (e.g., in weather forecasting) where the goal is to predict the time evolution of a large, spatiotemporally chaotic dynamical system when we have access to both time series data of previous system states and an imperfect model of the full system dynamics. Specifically, we attempt to utilize machine learning as the essential tool for integrating the use of past data into predictions. In order to facilitate scalability to the common scenario of interest where the spatiotemporally chaotic system is very large and complex, we propose combining two approaches:(i) a parallel machine learning prediction scheme; and (ii) a hybrid technique, for a composite prediction system composed of a knowledge-based component and a machine-learning-based component. We demonstrate that not only can this method combining (i) and (ii) be scaled to give excellent performance for very large systems, but also that the length of time series data needed to train our multiple, parallel machine learning components is dramatically less than that necessary without parallelization. Furthermore, considering cases where computational realization of the knowledge-based component does not resolve subgrid-scale processes, our scheme is able to use training data to incorporate the effect of the unresolved short-scale dynamics upon the resolved longer-scale dynamics ("subgrid-scale closure"). △ Less

Submitted 10 February, 2020; originally announced February 2020.

Comments: 45 pages, 15 figures

Forecasting Chaotic Systems with Very Low Connectivity Reservoir Computers

Authors: Aaron Griffith, Andrew Pomerance, Daniel J. Gauthier

Abstract: We explore the hyperparameter space of reservoir computers used for forecasting of the chaotic Lorenz '63 attractor with Bayesian optimization. We use a new measure of reservoir performance, designed to emphasize learning the global climate of the forecasted system rather than short-term prediction. We find that optimizing over this measure more quickly excludes reservoirs that fail to reproduce t… ▽ More We explore the hyperparameter space of reservoir computers used for forecasting of the chaotic Lorenz '63 attractor with Bayesian optimization. We use a new measure of reservoir performance, designed to emphasize learning the global climate of the forecasted system rather than short-term prediction. We find that optimizing over this measure more quickly excludes reservoirs that fail to reproduce the climate. The results of optimization are surprising: the optimized parameters often specify a reservoir network with very low connectivity. Inspired by this observation, we explore reservoir designs with even simpler structure, and find well-performing reservoirs that have zero spectral radius and no recurrence. These simple reservoirs provide counterexamples to widely used heuristics in the field, and may be useful for hardware implementations of reservoir computers. △ Less

Submitted 15 November, 2019; v1 submitted 1 October, 2019; originally announced October 2019.

Hybrid Boolean Networks as Physically Unclonable Functions

Authors: Noeloikeau Charlot, Daniel Canaday, Andrew Pomerance, Daniel J. Gauthier

Abstract: We introduce a Physically Unclonable Function (PUF) based on an ultra-fast chaotic network known as a Hybrid Boolean Network (HBN) implemented on a field programmable gate array. The network, consisting of $N$ coupled asynchronous logic gates displaying dynamics on the sub-nanosecond time scale, acts as a `digital fingerprint' by amplifying small manufacturing variations during a period of transie… ▽ More We introduce a Physically Unclonable Function (PUF) based on an ultra-fast chaotic network known as a Hybrid Boolean Network (HBN) implemented on a field programmable gate array. The network, consisting of $N$ coupled asynchronous logic gates displaying dynamics on the sub-nanosecond time scale, acts as a `digital fingerprint' by amplifying small manufacturing variations during a period of transient chaos. In contrast to other PUF designs, we use both $N$-bits per challenge and obtain $N$-bits per response by considering challenges to be initial states of the $N$-node network and responses to be states captured during the subsequent chaotic transient. We find that the presence of chaos amplifies the frozen-in randomness due to manufacturing differences and that the extractable entropy is approximately $50\%$ of the maximum of $N2^{N}$ bits. We obtain PUF uniqueness and reliability metrics $μ_{inter}$ = 0.40$\pm$0.01 and $μ_{intra}$ = 0.05$\pm$0.00, respectively, for an $N=256$ network. These metrics correspond to an expected Hamming distance of 102.4 bits per response. Moreover, a simple cherry-picking scheme that discards noisy bits yields $μ_{intra} < 0.01$ while still retaining $\sim200$ bits/response (corresponding to a Hamming distance of $\sim80$ bits/response). In addition to characterizing the uniqueness and reliability, we demonstrate super-exponential scaling in the entropy up to $N=512$ and demonstrate that PUFmeter, a recent PUF analysis tool, is unable to model our PUF. Finally, we characterize the temperature variation of the HBN-PUF and propose future improvements. △ Less

Submitted 6 April, 2021; v1 submitted 29 July, 2019; originally announced July 2019.

Journal ref: IEEE Access 9 (2021): 44855-44867

Stability of Boolean networks: The joint effects of topology and update rules

Authors: Shane Squires, Andrew Pomerance, Michelle Girvan, Edward Ott

Abstract: We study the stability of orbits in large Boolean networks with given complex topology. We impose no restrictions on the form of the update rules, which may be correlated with local topological properties of the network. While recent past work has addressed the separate effects of nontrivial network topology and certain special classes of update rules on stability, only crude results exist about h… ▽ More We study the stability of orbits in large Boolean networks with given complex topology. We impose no restrictions on the form of the update rules, which may be correlated with local topological properties of the network. While recent past work has addressed the separate effects of nontrivial network topology and certain special classes of update rules on stability, only crude results exist about how these effects interact. We present a widely applicable solution to this problem. Numerical experiments confirm our theory and show that local correlations between topology and update rules can have profound effects on the qualitative behavior of these systems. △ Less

Submitted 4 October, 2013; originally announced October 2013.

Comments: Includes supplemental information

Journal ref: Phys. Rev. E 90, 022814 (2014)

The Stability of Boolean Networks with Generalized Canalizing Rules

Authors: Andrew Pomerance, Michelle Girvan, Ed Ott

Abstract: Boolean networks are discrete dynamical systems in which the state (zero or one) of each node is updated at each time t to a state determined by the states at time t-1 of those nodes that have links to it. When these systems are used to model genetic control, the case of 'canalizing' update rules is of particular interest. A canalizing rule is one for which a node state at time $t$ is determined b… ▽ More Boolean networks are discrete dynamical systems in which the state (zero or one) of each node is updated at each time t to a state determined by the states at time t-1 of those nodes that have links to it. When these systems are used to model genetic control, the case of 'canalizing' update rules is of particular interest. A canalizing rule is one for which a node state at time $t$ is determined by the state at time t-1 of a single one of its inputs when that inputting node is in its canalizing state. Previous work on the order/disorder transition in Boolean networks considered complex, non-random network topology. In the current paper we extend this previous work to account for canalizing behavior. △ Less

Submitted 24 February, 2012; v1 submitted 2 October, 2009; originally announced October 2009.

Comments: 20 pages, 3 figures

Approximating the Largest Eigenvalue of the Modified Adjacency Matrix of Networks with Heterogeneous Node Biases

Authors: Edward Ott, Andrew Pomerance

Abstract: Motivated by its relevance to various types of dynamical behavior of network systems, the maximum eigenvalue $λ_Q$ of the adjacency matrix $A$ of a network has been considered, and mean-field-type approximations to $λ_Q$ have been developed for different kinds of networks. Here $A$ is defined by $A_{ij} = 1$ ($A_{ij} = 0$) if there is (is not) a directed network link to $i$ from $j$. However, in a… ▽ More Motivated by its relevance to various types of dynamical behavior of network systems, the maximum eigenvalue $λ_Q$ of the adjacency matrix $A$ of a network has been considered, and mean-field-type approximations to $λ_Q$ have been developed for different kinds of networks. Here $A$ is defined by $A_{ij} = 1$ ($A_{ij} = 0$) if there is (is not) a directed network link to $i$ from $j$. However, in at least two recent problems involving networks with heterogeneous node properties (percolation on a directed network and the stability of Boolean models of gene networks), an analogous but different eigenvalue problem arises, namely, that of finding the largest eigenvalue $λ_Q$ of the matrix $Q$, where $Q_{ij} = q_i A_{ij}$ and the `bias' $q_i$ may be different at each node $i$. (In the previously mentioned percolation and gene network contexts, $q_i$ is a probability and so lies in the range $0 \le q_i \le 1$.) The purposes of this paper are to extend the previous considerations of the maximum eigenvalue $λ_A$ of $A$ to $λ_Q$, to develop suitable analytic approximations to $λ_Q$, and to test these approximations with numerical experiments. In particular, three issues considered are (i) the effect of the correlation (or anticorrelation) between the value of $q_i$ and the number of links to and from node $i$; (ii) the effect of correlation between the properties of two nodes at either end of a network link (`assortativity'); and (iii) the effect of community structure allowing for a situation in which different $q$-values are associated with different communities. △ Less

Submitted 2 January, 2011; v1 submitted 9 February, 2009; originally announced February 2009.

Comments: 15 pages, 3 figures

The effect of network topology on the stability of discrete state models of genetic control

Authors: Andrew Pomerance, Edward Ott, Michelle Girvan, Wolfgang Losert

Abstract: Boolean networks have been proposed as potentially useful models for genetic control. An important aspect of these networks is the stability of their dynamics in response to small perturbations. Previous approaches to stability have assumed uncorrelated random network structure. Real gene networks typically have nontrivial topology significantly different from the random network paradigm. In ord… ▽ More Boolean networks have been proposed as potentially useful models for genetic control. An important aspect of these networks is the stability of their dynamics in response to small perturbations. Previous approaches to stability have assumed uncorrelated random network structure. Real gene networks typically have nontrivial topology significantly different from the random network paradigm. In order to address such situations, we present a general method for determining the stability of large Boolean networks of any specified network topology and predicting their steady-state behavior in response to small perturbations. Additionally, we generalize to the case where individual genes have a distribution of `expression biases,' and we consider non-synchronous update, as well as extension of our method to non-Boolean models in which there are more than two possible gene states. We find that stability is governed by the maximum eigenvalue of a modified adjacency matrix, and we test this result by comparison with numerical simulations. We also discuss the possible application of our work to experimentally inferred gene networks. △ Less

Submitted 26 February, 2009; v1 submitted 27 January, 2009; originally announced January 2009.

Comments: 25 pages, 4 figures; added supplementary information, fixed typos and figure, reformatted

Feedback control of unstable cellular solidification fronts

Authors: A. J. Pons, A. Karma, S. Akamatsu, M. Newey, A. Pomerance, H. Singer, W. Losert

Abstract: We present a numerical and experimental study of feedback control of unstable cellular patterns in directional solidification (DS). The sample, a dilute binary alloy, solidifies in a 2D geometry under a control scheme which applies local heating close to the cell tips which protrude ahead of the other. For the experiments, we use a real-time image processing algorithm to track cell tips, coupled… ▽ More We present a numerical and experimental study of feedback control of unstable cellular patterns in directional solidification (DS). The sample, a dilute binary alloy, solidifies in a 2D geometry under a control scheme which applies local heating close to the cell tips which protrude ahead of the other. For the experiments, we use a real-time image processing algorithm to track cell tips, coupled with a movable laser spot array device, to heat locally. We show, numerically and experimentally, that spacings well below the threshold for a period-doubling instability can be stabilized. As predicted by the numerical calculations, cellular arrays become stable, and the spacing becomes uniform through feedback control which is maintained with minimal heating. △ Less

Submitted 15 August, 2006; originally announced August 2006.

Comments: 4 pages, 4 figures, 1 table