-
Time-shift selection for reservoir computing using a rank-revealing QR algorithm
Authors:
Joseph D. Hart,
Francesco Sorrentino,
Thomas L. Carroll
Abstract:
Reservoir computing, a recurrent neural network paradigm in which only the output layer is trained, has demonstrated remarkable performance on tasks such as prediction and control of nonlinear systems. Recently, it was demonstrated that adding time-shifts to the signals generated by a reservoir can provide large improvements in performance accuracy. In this work, we present a technique to choose t…
▽ More
Reservoir computing, a recurrent neural network paradigm in which only the output layer is trained, has demonstrated remarkable performance on tasks such as prediction and control of nonlinear systems. Recently, it was demonstrated that adding time-shifts to the signals generated by a reservoir can provide large improvements in performance accuracy. In this work, we present a technique to choose the time-shifts by maximizing the rank of the reservoir matrix using a rank-revealing QR algorithm. This technique, which is not task dependent, does not require a model of the system, and therefore is directly applicable to analog hardware reservoir computers. We demonstrate our time-shift selection technique on two types of reservoir computer: one based on an opto-electronic oscillator and the traditional recurrent network with a $tanh$ activation function. We find that our technique provides improved accuracy over random time-shift selection in essentially all cases.
△ Less
Submitted 25 April, 2023; v1 submitted 29 November, 2022;
originally announced November 2022.
-
Creating New Chaotic Signals with Reservoir Computers
Authors:
Thomas L. Carroll
Abstract:
While there have been many publications on potential applications of chaos to fields such as communications, radar, sonar, random signal generation, channel equalization and others, designing continuous chaotic systems is still an unsolved problem. There are a number of well known chaotic systems used for applications, but if any application is to become widely used, some way of generating many di…
▽ More
While there have been many publications on potential applications of chaos to fields such as communications, radar, sonar, random signal generation, channel equalization and others, designing continuous chaotic systems is still an unsolved problem. There are a number of well known chaotic systems used for applications, but if any application is to become widely used, some way of generating many different chaotic signals is necessary. This work shows that one may use a reservoir computer to create a set of chaotic signals that are correlated but easily distinguishable from one chaotic signal with desirable properties. The ability to distinguish the new signals is demonstrated with a simple communications example.
△ Less
Submitted 9 September, 2022;
originally announced October 2022.
-
Time Shifts to Reduce the Size of Reservoir Computers
Authors:
Thomas L. Carroll,
Joseph D. Hart
Abstract:
A reservoir computer is a type of dynamical system arranged to do computation. Typically, a reservoir computer is constructed by connecting a large number of nonlinear nodes in a network that includes recurrent connections. In order to achieve accurate results, the reservoir usually contains hundreds to thousands of nodes. This high dimensionality makes it difficult to analyze the reservoir comput…
▽ More
A reservoir computer is a type of dynamical system arranged to do computation. Typically, a reservoir computer is constructed by connecting a large number of nonlinear nodes in a network that includes recurrent connections. In order to achieve accurate results, the reservoir usually contains hundreds to thousands of nodes. This high dimensionality makes it difficult to analyze the reservoir computer using tools from dynamical systems theory. Additionally, the need to create and connect large numbers of nonlinear nodes makes it difficult to design and build analog reservoir computers that can be faster and consume less power than digital reservoir computers. We demonstrate here that a reservoir computer may be divided into two parts; a small set of nonlinear nodes (the reservoir), and a separate set of time-shifted reservoir output signals. The time-shifted output signals serve to increase the rank and memory of the reservoir computer, and the set of nonlinear nodes may create an embedding of the input dynamical system. We use this time-shifting technique to obtain excellent performance from an opto-electronic delay-based reservoir computer with only a small number of virtual nodes. Because only a few nonlinear nodes are required, construction of a reservoir computer becomes much easier, and delay-based reservoir computers can operate at much higher speeds.
△ Less
Submitted 3 May, 2022;
originally announced May 2022.
-
Light in the Larynx: a Miniaturized Robotic Optical Fiber for In-office Laser Surgery of the Vocal Folds
Authors:
Alex J. Chiluisa,
Nicholas E. Pacheco,
Hoang S. Do,
Ryan M. Tougas,
Emily V. Minch,
Rositsa Mihaleva,
Yao Shen,
Yuxiang Liu,
Thomas L. Carroll,
Loris Fichera
Abstract:
This letter reports the design, construction, and experimental validation of a novel hand-held robot for in-office laser surgery of the vocal folds. In-office endoscopic laser surgery is an emerging trend in Laryngology: It promises to deliver the same patient outcomes of traditional surgical treatment (i.e., in the operating room), at a fraction of the cost. Unfortunately, office procedures can b…
▽ More
This letter reports the design, construction, and experimental validation of a novel hand-held robot for in-office laser surgery of the vocal folds. In-office endoscopic laser surgery is an emerging trend in Laryngology: It promises to deliver the same patient outcomes of traditional surgical treatment (i.e., in the operating room), at a fraction of the cost. Unfortunately, office procedures can be challenging to perform; the optical fibers used for laser delivery can only emit light forward in a line-of-sight fashion, which severely limits anatomical access. The robot we present in this letter aims to overcome these challenges. The end effector of the robot is a steerable laser fiber, created through the combination of a thin optical fiber (0.225 mm) with a tendon-actuated Nickel-Titanium notched sheath that provides bending. This device can be seamlessly used with most commercially available endoscopes, as it is sufficiently small (1.1 mm) to pass through a working channel. To control the fiber, we propose a compact actuation unit that can be mounted on top of the endoscope handle, so that, during a procedure, the operating physician can operate both the endoscope and the steerable fiber with a single hand. We report simulation and phantom experiments demonstrating that the proposed device substantially enhances surgical access compared to current clinical fibers.
△ Less
Submitted 24 August, 2022; v1 submitted 27 April, 2022;
originally announced April 2022.
-
Optimizing Memory in Reservoir Computers
Authors:
Thomas L. Carroll
Abstract:
A reservoir computer is a way of using a high dimensional dynamical system for computation. One way to construct a reservoir computer is by connecting a set of nonlinear nodes into a network. Because the network creates feedback between nodes, the reservoir computer has memory. If the reservoir computer is to respond to an input signal in a consistent way (a necessary condition for computation), t…
▽ More
A reservoir computer is a way of using a high dimensional dynamical system for computation. One way to construct a reservoir computer is by connecting a set of nonlinear nodes into a network. Because the network creates feedback between nodes, the reservoir computer has memory. If the reservoir computer is to respond to an input signal in a consistent way (a necessary condition for computation), the memory must be fading; that is, the influence of the initial conditions fades over time. How long this memory lasts is important for determining how well the reservoir computer can solve a particular problem. In this paper I describe ways to vary the length of the fading memory in reservoir computers. Tuning the memory can be important to achieve optimal results in some problems; too much or too little memory degrades the accuracy of the computation.
△ Less
Submitted 5 January, 2022;
originally announced January 2022.
-
Do Reservoir Computers Work Best at the Edge of Chaos?
Authors:
Thomas L. Carroll
Abstract:
It has been demonstrated that cellular automata had the highest computational capacity at the edge of chaos, the parameter at which their behavior transitioned from ordered to chaotic. This same concept has been applied to reservoir computers; a number of researchers have stated that the highest computational capacity for a reservoir computer is at the edge of chaos, although others have suggested…
▽ More
It has been demonstrated that cellular automata had the highest computational capacity at the edge of chaos, the parameter at which their behavior transitioned from ordered to chaotic. This same concept has been applied to reservoir computers; a number of researchers have stated that the highest computational capacity for a reservoir computer is at the edge of chaos, although others have suggested that this rule is not universally true. Because many reservoir computers do not show chaotic behavior but merely become unstable, it is felt that a more accurate term for this instability transition is the "edge of stability"Here I find two examples where the computational capacity of a reservoir computer decreases as the edge of stability is approached; in one case, because generalized synchronization breaks down, and in the other case because the reservoir computer is a poor match to the problem being solved. The edge of stability as an optimal operating point for a reservoir computer is not in general true, although it may be true in some cases.
△ Less
Submitted 2 December, 2020;
originally announced December 2020.
-
Adding Filters to Improve Reservoir Computer Performance
Authors:
Thomas L. Carroll
Abstract:
Reservoir computers are a type of neuromorphic computer that may be built a an analog system, potentially creating powerful computers that are small, light and consume little power. Typically a reservoir computer is build by connecting together a set of nonlinear nodes into a network; connecting the nonlinear nodes may be difficult or expensive, however. This work shows how a reservoir computer ma…
▽ More
Reservoir computers are a type of neuromorphic computer that may be built a an analog system, potentially creating powerful computers that are small, light and consume little power. Typically a reservoir computer is build by connecting together a set of nonlinear nodes into a network; connecting the nonlinear nodes may be difficult or expensive, however. This work shows how a reservoir computer may be expanded by adding functions to its output. The particular functions described here are linear filters, but other functions are possible. The design and construction of linear filters is well known, and such filters may be easily implemented in hardware such as field programmable gate arrays (FPGA's). The effect of adding filters on the reservoir computer performance is simulated for a signal fitting problem, a prediction problem and a signal classification problem.
△ Less
Submitted 19 October, 2020; v1 submitted 24 August, 2020;
originally announced August 2020.
-
Dimension of Reservoir Computers
Authors:
Thomas L. Carroll
Abstract:
A reservoir computer is a complex dynamical system, often created by coupling nonlinear nodes in a network. The nodes are all driven by a common driving signal. In this work, three dimension estimation methods, false nearest neighbor, covariance and Kaplan-Yorke dimensions, are used to estimate the dimension of the reservoir dynamical system. It is shown that the signals in the reservoir system ex…
▽ More
A reservoir computer is a complex dynamical system, often created by coupling nonlinear nodes in a network. The nodes are all driven by a common driving signal. In this work, three dimension estimation methods, false nearest neighbor, covariance and Kaplan-Yorke dimensions, are used to estimate the dimension of the reservoir dynamical system. It is shown that the signals in the reservoir system exist on a relatively low dimensional surface. Changing the spectral radius of the reservoir network can increase the fractal dimension of the reservoir signals, leading to an increase in testing error.
△ Less
Submitted 10 December, 2019;
originally announced December 2019.
-
Mutual Information and the Edge of Chaos in Reservoir Computers
Authors:
Thomas L. Carroll
Abstract:
A reservoir computer is a dynamical system that may be used to perform computations. A reservoir computer usually consists of a set of nonlinear nodes coupled together in a network so that there are feedback paths. Training the reservoir computer consists of inputing a signal of interest and fitting the time series signals of the reservoir computer nodes to a training signal that is related to the…
▽ More
A reservoir computer is a dynamical system that may be used to perform computations. A reservoir computer usually consists of a set of nonlinear nodes coupled together in a network so that there are feedback paths. Training the reservoir computer consists of inputing a signal of interest and fitting the time series signals of the reservoir computer nodes to a training signal that is related to the input signal. It is believed that dynamical systems function most efficiently as computers at the "edge of chaos", the point at which the largest Lyapunov exponent of the dynamical system transitions from negative to positive. In this work I simulate several different reservoir computers and ask if the best performance really does come at this edge of chaos. I find that while it is possible to get optimum performance at the edge of chaos, there may also be parameter values where the edge of chaos regime produces poor performance. This ambiguous parameter dependance has implications for building reservoir computers from analog physical systems, where the parameter range is restricted.
△ Less
Submitted 19 July, 2019; v1 submitted 6 June, 2019;
originally announced June 2019.
-
Network Structure Effects in Reservoir Computers
Authors:
Thomas L. Carroll,
Louis M. Pecora
Abstract:
A reservoir computer is a complex nonlinear dynamical system that has been shown to be useful for solving certain problems, such as prediction of chaotic signals, speech recognition or control of robotic systems. Typically a reservoir computer is constructed by connecting a large number of nonlinear nodes in a network, driving the nodes with an input signal and using the node outputs to fit a trai…
▽ More
A reservoir computer is a complex nonlinear dynamical system that has been shown to be useful for solving certain problems, such as prediction of chaotic signals, speech recognition or control of robotic systems. Typically a reservoir computer is constructed by connecting a large number of nonlinear nodes in a network, driving the nodes with an input signal and using the node outputs to fit a training signal. In this work, we set up reservoirs where the edges (or connections) between all the network nodes are either +1 or 0, and proceed to alter the network structure by flip** some of these edges from +1 to -1. We use this simple network because it turns out to be easy to characterize; we may use the fraction of edges flipped as a measure of how much we have altered the network. In some cases, the network can be rearranged in a finite number of ways without changing its structure; these rearrangements are symmetries of the network, and the number of symmetries is also useful for characterizing the network. We find that changing the number of edges flipped in the network changes the rank of the covariance of a matrix consisting of the time series from the different nodes in the network, and speculate that this rank is important for understanding the reservoir computer performance.
△ Less
Submitted 7 August, 2019; v1 submitted 28 March, 2019;
originally announced March 2019.
-
Using reservoir computers to distinguish chaotic signals
Authors:
Thomas L. Carroll
Abstract:
Several recent papers have shown that reservoir computers are useful for analyzing and predicting dynamical systems. Reservoir computers have also been shown to be useful for various classification problems. In this work, a reservoir computer is used to identify one out of the 19 different Sprott systems. An advantage of reservoir computers for this problem is that no embedding is necessary. Some…
▽ More
Several recent papers have shown that reservoir computers are useful for analyzing and predicting dynamical systems. Reservoir computers have also been shown to be useful for various classification problems. In this work, a reservoir computer is used to identify one out of the 19 different Sprott systems. An advantage of reservoir computers for this problem is that no embedding is necessary. Some guidance on choosing the reservoir computer parameters is given. The dependance on number of points, number of reservoir nodes and noise in identifying the Sprott systems is explored.
△ Less
Submitted 21 September, 2018;
originally announced October 2018.
-
Testing Dynamical System Variables for Reconstruction
Authors:
Thomas L. Carroll
Abstract:
Analyzing data from dynamical systems often begins with creating a reconstruction of the trajectory based on one or more variables, but not all variables are suitable for reconstructing the trajectory. The concept of nonlinear observability has been investigated as a way to determine if a dynamical system can be reconstructed from one signal or a combination of signals, however nonlinear observabi…
▽ More
Analyzing data from dynamical systems often begins with creating a reconstruction of the trajectory based on one or more variables, but not all variables are suitable for reconstructing the trajectory. The concept of nonlinear observability has been investigated as a way to determine if a dynamical system can be reconstructed from one signal or a combination of signals, however nonlinear observability can be difficult to calculate for a high dimensional system. In this work I compare the results from nonlinear observability to a continuity statistic that indicates the likelihood that there is a continuous function between two sets of multidimensional points- in this case two different reconstructions of the same attractor from different signals simultaneously measured.
Without a metric against which to test the ability to reconstruct a system, the predictions of nonlinear observability and continuity are ambiguous. As a additional test how well different signals can predict the ability to reconstruct a dynamical system I use the fitting error from training a reservoir computer.
△ Less
Submitted 21 September, 2018; v1 submitted 27 August, 2018;
originally announced August 2018.
-
A Unified Approach to Attractor Reconstruction
Authors:
Louis M. Pecora,
Linda Moniz,
Jonathan Nichols,
Thomas L. Carroll
Abstract:
In the analysis of complex, nonlinear time series, scientists in a variety of disciplines have relied on a time delayed embedding of their data, i.e. attractor reconstruction. The process has focused primarily on heuristic and empirical arguments for selection of the key embedding parameters, delay and embedding dimension. This approach has left several long-standing, but common problems unresol…
▽ More
In the analysis of complex, nonlinear time series, scientists in a variety of disciplines have relied on a time delayed embedding of their data, i.e. attractor reconstruction. The process has focused primarily on heuristic and empirical arguments for selection of the key embedding parameters, delay and embedding dimension. This approach has left several long-standing, but common problems unresolved in which the standard approaches produce inferior results or give no guidance at all. We view the current reconstruction process as unnecessarily broken into separate problems. We propose an alternative approach that views the problem of choosing all embedding parameters as being one and the same problem addressable using a single statistical test formulated directly from the reconstruction theorems. This allows for varying time delays appropriate to the data and simultaneously helps decide on embedding dimension. A second new statistic, undersampling, acts as a check against overly long time delays and overly large embedding dimension. Our approach is more flexible than those currently used, but is more directly connected with the mathematical requirements of embedding. In addition, the statistics developed guide the user by allowing optimization and warning when embedding parameters are chosen beyond what the data can support. We demonstrate our approach on uni- and multivariate data, data possessing multiple time scales, and chaotic data. This unified approach resolves all the main issues in attractor reconstruction.
△ Less
Submitted 30 June, 2006; v1 submitted 23 February, 2006;
originally announced February 2006.
