-
Efficient Optimisation of Physical Reservoir Computers using only a Delayed Input
Authors:
Enrico Picco,
Lina Jaurigue,
Kathy Lüdge,
Serge Massar
Abstract:
We present an experimental validation of a recently proposed optimization technique for reservoir computing, using an optoelectronic setup. Reservoir computing is a robust framework for signal processing applications, and the development of efficient optimization approaches remains a key challenge. The technique we address leverages solely a delayed version of the input signal to identify the opti…
▽ More
We present an experimental validation of a recently proposed optimization technique for reservoir computing, using an optoelectronic setup. Reservoir computing is a robust framework for signal processing applications, and the development of efficient optimization approaches remains a key challenge. The technique we address leverages solely a delayed version of the input signal to identify the optimal operational region of the reservoir, simplifying the traditionally time-consuming task of hyperparameter tuning. We verify the effectiveness of this approach on different benchmark tasks and reservoir operating conditions.
△ Less
Submitted 25 January, 2024;
originally announced January 2024.
-
Reducing hyperparameter dependence by external timescale tailoring
Authors:
Lina C. Jaurigue,
Kathy Lüdge
Abstract:
Task specific hyperparameter tuning in reservoir computing is an open issue, and is of particular relevance for hardware implemented reservoirs. We investigate the influence of directly including externally controllable task specific timescales on the performance and hyperparameter sensitivity of reservoir computing approaches. We show that the need for hyperparameter optimisation can be reduced i…
▽ More
Task specific hyperparameter tuning in reservoir computing is an open issue, and is of particular relevance for hardware implemented reservoirs. We investigate the influence of directly including externally controllable task specific timescales on the performance and hyperparameter sensitivity of reservoir computing approaches. We show that the need for hyperparameter optimisation can be reduced if timescales of the reservoir are tailored to the specific task. Our results are mainly relevant for temporal tasks requiring memory of past inputs, for example chaotic timeseries prediciton. We consider various methods of including task specific timescales in the reservoir computing approach and demonstrate the universality of our message by looking at both time-multiplexed and spatially multiplexed reservoir computing.
△ Less
Submitted 24 October, 2023; v1 submitted 17 July, 2023;
originally announced July 2023.
-
The role of delay-times in delay-based Photonic Reservoir Computing
Authors:
Tobias Hülser,
Felix Köster,
Lina Jaurigue,
Kathy Lüdge
Abstract:
Delay-based reservoir computing has gained a lot of attention due to the relative simplicity with which this concept can be implemented in hardware. However,there is still an misconception about the relationship between the delay-time and the input clock-cycle which has noticeable consequences for the performance. We review the existing literature on this subject and introduce the concept of delay…
▽ More
Delay-based reservoir computing has gained a lot of attention due to the relative simplicity with which this concept can be implemented in hardware. However,there is still an misconception about the relationship between the delay-time and the input clock-cycle which has noticeable consequences for the performance. We review the existing literature on this subject and introduce the concept of delay-based reservoir computing in a manner that demonstrates that there is no predefined relationship between these two times-scales. Further, we discuss ways to improve the computing performance of a reservoir formed by delay-coupled oscillators and show the crucial impact of delay-time tuning in those multi-delay systems.
△ Less
Submitted 22 December, 2021;
originally announced December 2021.
-
Master memory function for delay-based reservoir computers with single-variable dynamics
Authors:
Felix Köster,
Serhiy Yanchuk,
Kathy Lüdge
Abstract:
We show that many delay-based reservoir computers considered in the literature can be characterized by a universal master memory function (MMF).
Once computed for two independent parameters, this function provides linear memory capacity for any delay-based single-variable reservoir with small inputs. Moreover, we propose an analytical description of the MMF that enables its efficient and fast co…
▽ More
We show that many delay-based reservoir computers considered in the literature can be characterized by a universal master memory function (MMF).
Once computed for two independent parameters, this function provides linear memory capacity for any delay-based single-variable reservoir with small inputs. Moreover, we propose an analytical description of the MMF that enables its efficient and fast computation.
Our approach can be applied not only to reservoirs governed by known dynamical rules such as Mackey-Glass or Ikeda-like systems but also to reservoirs whose dynamical model is not available. We also present results comparing the performance of the reservoir computer and the memory capacity given by the MMF.
△ Less
Submitted 28 August, 2021;
originally announced August 2021.
-
Limitations of the recall capabilities in delay based reservoir computing systems
Authors:
Felix Köster,
Dominik Ehlert,
Kathy Lüdge
Abstract:
We analyze the memory capacity of a delay based reservoir computer with a Hopf normal form as nonlinearity and numerically compute the linear as well as the higher order recall capabilities. A possible physical realisation could be a laser with external cavity, for which the information is fed via electrical injection. A task independent quantification of the computational capability of the reserv…
▽ More
We analyze the memory capacity of a delay based reservoir computer with a Hopf normal form as nonlinearity and numerically compute the linear as well as the higher order recall capabilities. A possible physical realisation could be a laser with external cavity, for which the information is fed via electrical injection. A task independent quantification of the computational capability of the reservoir system is done via a complete orthonormal set of basis functions. Our results suggest that even for constant readout dimension the total memory capacity is dependent on the ratio between the information input period, also called the clock cycle, and the time delay in the system. Optimal performance is found for a time delay about 1.6 times the clock cycle
△ Less
Submitted 16 September, 2020;
originally announced October 2020.
-
Insight into Delay Based Reservoir Computing via Eigenvalue Analysis
Authors:
Felix Köster,
Serhiy Yanchuk,
Kathy Lüdge
Abstract:
In this paper we give a profound insight into the computation capability of delay-based reservoir computing via an eigenvalue analysis. We concentrate on the task-independent memory capacity to quantify the reservoir performance and compare these with the eigenvalue spectrum of the dynamical system. We show that these two quantities are deeply connected, and thus the reservoir computing performanc…
▽ More
In this paper we give a profound insight into the computation capability of delay-based reservoir computing via an eigenvalue analysis. We concentrate on the task-independent memory capacity to quantify the reservoir performance and compare these with the eigenvalue spectrum of the dynamical system. We show that these two quantities are deeply connected, and thus the reservoir computing performance is predictable by analyzing the small signal response of the reservoir. Our results suggest that any dynamical system used as a reservoir can be analyzed in this way. We apply our method exemplarily to a photonic laser system with feedback and compare the numerically computed recall capabilities with the eigenvalue spectrum. Optimal performance is found for a system with the eigenvalues having real parts close to zero and off-resonant imaginary parts.
△ Less
Submitted 22 March, 2021; v1 submitted 16 September, 2020;
originally announced September 2020.
-
Deep Time-Delay Reservoir Computing: Dynamics and Memory Capacity
Authors:
Mirko Goldmann,
Felix Köster,
Kathy Lüdge,
Serhiy Yanchuk
Abstract:
The Deep Time-Delay Reservoir Computing concept utilizes unidirectionally connected systems with time-delays for supervised learning. We present how the dynamical properties of a deep Ikeda-based reservoir are related to its memory capacity (MC) and how that can be used for optimization. In particular, we analyze bifurcations of the corresponding autonomous system and compute conditional Lyapunov…
▽ More
The Deep Time-Delay Reservoir Computing concept utilizes unidirectionally connected systems with time-delays for supervised learning. We present how the dynamical properties of a deep Ikeda-based reservoir are related to its memory capacity (MC) and how that can be used for optimization. In particular, we analyze bifurcations of the corresponding autonomous system and compute conditional Lyapunov exponents, which measure the generalized synchronization between the input and the layer dynamics. We show how the MC is related to the systems distance to bifurcations or magnitude of the conditional Lyapunov exponent. The interplay of different dynamical regimes leads to a adjustable distribution between linear and nonlinear MC. Furthermore, numerical simulations show resonances between clock cycle and delays of the layers in all degrees of the MC. Contrary to MC losses in a single-layer reservoirs, these resonances can boost separate degrees of the MC and can be used, e.g., to design a system with maximum linear MC. Accordingly, we present two configurations that empower either high nonlinear MC or long time linear MC.
△ Less
Submitted 25 August, 2020; v1 submitted 11 June, 2020;
originally announced June 2020.
-
Performance boost of time-delay reservoir computing by non-resonant clock cycle
Authors:
Florian Stelzer,
André Röhm,
Kathy Lüdge,
Serhiy Yanchuk
Abstract:
The time-delay-based reservoir computing setup has seen tremendous success in both experiment and simulation. It allows for the construction of large neuromorphic computing systems with only few components. However, until now the interplay of the different timescales has not been investigated thoroughly. In this manuscript, we investigate the effects of a mismatch between the time-delay and the cl…
▽ More
The time-delay-based reservoir computing setup has seen tremendous success in both experiment and simulation. It allows for the construction of large neuromorphic computing systems with only few components. However, until now the interplay of the different timescales has not been investigated thoroughly. In this manuscript, we investigate the effects of a mismatch between the time-delay and the clock cycle for a general model. Typically, these two time scales are considered to be equal. Here we show that the case of equal or resonant time-delay and clock cycle could be actively detrimental and leads to an increase of the approximation error of the reservoir. In particular, we can show that non-resonant ratios of these time scales have maximal memory capacities. We achieve this by translating the periodically driven delay-dynamical system into an equivalent network. Networks that originate from a system with resonant delay-times and clock cycles fail to utilize all of their degrees of freedom, which causes the degradation of their performance.
△ Less
Submitted 23 January, 2020; v1 submitted 7 May, 2019;
originally announced May 2019.
-
Reservoir computing with simple oscillators: Virtual and real networks
Authors:
André Röhm,
Kathy Lüdge
Abstract:
The reservoir computing scheme is a machine learning mechanism which utilizes the naturally occuring computational capabilities of dynamical systems. One important subset of systems that has proven powerful both in experiments and theory are delay-systems. In this work, we investigate the reservoir computing performance of hybrid network-delay systems systematically by evaluating the NARMA10 and t…
▽ More
The reservoir computing scheme is a machine learning mechanism which utilizes the naturally occuring computational capabilities of dynamical systems. One important subset of systems that has proven powerful both in experiments and theory are delay-systems. In this work, we investigate the reservoir computing performance of hybrid network-delay systems systematically by evaluating the NARMA10 and the Sante Fe task.. We construct 'multiplexed networks' that can be seen as intermediate steps on the scale from classical networks to the 'virtual networks' of delay systems. We find that the delay approach can be extended to the network case without loss of computational power, enabling the construction of faster reservoir computing substrates.
△ Less
Submitted 23 February, 2018;
originally announced February 2018.