-
Asymptotic Stability in Reservoir Computing
Authors:
Jonathan Dong,
Erik Börve,
Mushegh Rafayelyan,
Michael Unser
Abstract:
Reservoir Computing is a class of Recurrent Neural Networks with internal weights fixed at random. Stability relates to the sensitivity of the network state to perturbations. It is an important property in Reservoir Computing as it directly impacts performance. In practice, it is desirable to stay in a stable regime, where the effect of perturbations does not explode exponentially, but also close…
▽ More
Reservoir Computing is a class of Recurrent Neural Networks with internal weights fixed at random. Stability relates to the sensitivity of the network state to perturbations. It is an important property in Reservoir Computing as it directly impacts performance. In practice, it is desirable to stay in a stable regime, where the effect of perturbations does not explode exponentially, but also close to the chaotic frontier where reservoir dynamics are rich. Open questions remain today regarding input regularization and discontinuous activation functions. In this work, we use the recurrent kernel limit to draw new insights on stability in reservoir computing. This limit corresponds to large reservoir sizes, and it already becomes relevant for reservoirs with a few hundred neurons. We obtain a quantitative characterization of the frontier between stability and chaos, which can greatly benefit hyperparameter tuning. In a broader sense, our results contribute to understanding the complex dynamics of Recurrent Neural Networks.
△ Less
Submitted 7 June, 2022;
originally announced June 2022.
-
Tuneable spin-glass optical simulator based on multiple light scattering
Authors:
Gianni Jacucci,
Louis Delloye,
Davide Pierangeli,
Mushegh Rafayelyan,
Claudio Conti,
Sylvain Gigan
Abstract:
The race to heuristically solve non-deterministic polynomial-time (NP) problems through efficient methods is ongoing. Recently, optics was demonstrated as a promising tool to find the ground state of a spin-glass Ising Hamiltonian, which represents an archetypal NP problem. However, achieving completely programmable spin couplings in these large-scale optical Ising simulators remains an open chall…
▽ More
The race to heuristically solve non-deterministic polynomial-time (NP) problems through efficient methods is ongoing. Recently, optics was demonstrated as a promising tool to find the ground state of a spin-glass Ising Hamiltonian, which represents an archetypal NP problem. However, achieving completely programmable spin couplings in these large-scale optical Ising simulators remains an open challenge. Here, by exploiting the knowledge of the transmission matrix of a random medium, we experimentally demonstrate the possibility of controlling the couplings of a fully connected Ising spin system. By further tailoring the input wavefront we showcase the possibility of modifying the Ising Hamiltonian both by accounting for an external magnetic field and by controlling the number of degenerate ground states and their properties and probabilities. Our results represent a relevant step toward the realisation of fully-programmable Ising machines on thin optical platforms, capable of solving complex spin-glass Hamiltonians on a large scale.
△ Less
Submitted 22 February, 2022; v1 submitted 12 November, 2021;
originally announced November 2021.
-
Reservoir Computing meets Recurrent Kernels and Structured Transforms
Authors:
Jonathan Dong,
Ruben Ohana,
Mushegh Rafayelyan,
Florent Krzakala
Abstract:
Reservoir Computing is a class of simple yet efficient Recurrent Neural Networks where internal weights are fixed at random and only a linear output layer is trained. In the large size limit, such random neural networks have a deep connection with kernel methods. Our contributions are threefold: a) We rigorously establish the recurrent kernel limit of Reservoir Computing and prove its convergence.…
▽ More
Reservoir Computing is a class of simple yet efficient Recurrent Neural Networks where internal weights are fixed at random and only a linear output layer is trained. In the large size limit, such random neural networks have a deep connection with kernel methods. Our contributions are threefold: a) We rigorously establish the recurrent kernel limit of Reservoir Computing and prove its convergence. b) We test our models on chaotic time series prediction, a classic but challenging benchmark in Reservoir Computing, and show how the Recurrent Kernel is competitive and computationally efficient when the number of data points remains moderate. c) When the number of samples is too large, we leverage the success of structured Random Features for kernel approximation by introducing Structured Reservoir Computing. The two proposed methods, Recurrent Kernel and Structured Reservoir Computing, turn out to be much faster and more memory-efficient than conventional Reservoir Computing.
△ Less
Submitted 21 October, 2020; v1 submitted 12 June, 2020;
originally announced June 2020.
-
Scalable spin-glass optical simulator
Authors:
Davide Pierangeli,
Mushegh Rafayelyan,
Claudio Conti,
Sylvain Gigan
Abstract:
Many developments in science and engineering depend on tackling complex optimizations on large scales. The challenge motivates intense search for specific computing hardware that takes advantage from quantum features, nonlinear dynamics, or photonics. A paradigmatic optimization problem is finding low-energy states in classical spin systems with fully-random interactions. To date no alternative co…
▽ More
Many developments in science and engineering depend on tackling complex optimizations on large scales. The challenge motivates intense search for specific computing hardware that takes advantage from quantum features, nonlinear dynamics, or photonics. A paradigmatic optimization problem is finding low-energy states in classical spin systems with fully-random interactions. To date no alternative computing platform can address such spin-glass problems on a large scale. Here we propose and realize an optical scalable spin-glass simulator based on spatial light modulation and multiple light scattering. By tailoring optical transmission through a disordered medium, we optically accelerate the computation of the ground state of large spin networks with all-to-all random couplings. Scaling of the operation time with the problem size demonstrates optical advantage over conventional computing. Our results point out optical vector-matrix multiplication as a tool for spin-glass problems and provide a general route towards large-scale computing that exploits speed, parallelism and coherence of light.
△ Less
Submitted 30 March, 2021; v1 submitted 1 June, 2020;
originally announced June 2020.
-
Large-Scale Optical Reservoir Computing for Spatiotemporal Chaotic Systems Prediction
Authors:
Mushegh Rafayelyan,
Jonathan Dong,
Yongqi Tan,
Florent Krzakala,
Sylvain Gigan
Abstract:
Reservoir computing is a relatively recent computational paradigm that originates from a recurrent neural network and is known for its wide range of implementations using different physical technologies. Large reservoirs are very hard to obtain in conventional computers, as both the computation complexity and memory usage grow quadratically. We propose an optical scheme performing reservoir comput…
▽ More
Reservoir computing is a relatively recent computational paradigm that originates from a recurrent neural network and is known for its wide range of implementations using different physical technologies. Large reservoirs are very hard to obtain in conventional computers, as both the computation complexity and memory usage grow quadratically. We propose an optical scheme performing reservoir computing over very large networks potentially being able to host several millions of fully connected photonic nodes thanks to its intrinsic properties of parallelism and scalability. Our experimental studies confirm that, in contrast to conventional computers, the computation time of our optical scheme is only linearly dependent on the number of photonic nodes of the network, which is due to electronic overheads, while the optical part of computation remains fully parallel and independent of the reservoir size. To demonstrate the scalability of our optical scheme, we perform for the first time predictions on large spatiotemporal chaotic datasets obtained from the Kuramoto-Sivashinsky equation using optical reservoirs with up to 50 000 optical nodes. Our results are extremely challenging for conventional von Neumann machines, and they significantly advance the state of the art of unconventional reservoir computing approaches, in general.
△ Less
Submitted 31 December, 2020; v1 submitted 24 January, 2020;
originally announced January 2020.
-
Optical Reservoir Computing using multiple light scattering for chaotic systems prediction
Authors:
Jonathan Dong,
Mushegh Rafayelyan,
Florent Krzakala,
Sylvain Gigan
Abstract:
Reservoir Computing is a relatively recent computational framework based on a large Recurrent Neural Network with fixed weights. Many physical implementations of Reservoir Computing have been proposed to improve speed and energy efficiency. In this study, we report new advances in Optical Reservoir Computing using multiple light scattering to accelerate the recursive computation of the reservoir s…
▽ More
Reservoir Computing is a relatively recent computational framework based on a large Recurrent Neural Network with fixed weights. Many physical implementations of Reservoir Computing have been proposed to improve speed and energy efficiency. In this study, we report new advances in Optical Reservoir Computing using multiple light scattering to accelerate the recursive computation of the reservoir states. Two different spatial light modulation technologies, namely, phase or binary amplitude modulations, are compared. Phase modulation is a promising direction already employed in other photonic implementations of Reservoir Computing. Additionally, we report a Digital-Micromirror-based Reservoir Computing at up to 640 Hz, more than double the previously reported frequency using a remotely controlled optical device developed by LightOn, and present new binarization strategies to improve the performance of binarized Reservoir Computing.
△ Less
Submitted 27 August, 2019; v1 submitted 1 July, 2019;
originally announced July 2019.
-
Ultra-broadband gradient-pitch Bragg-Berry mirrors
Authors:
Mushegh Rafayelyan,
Gonzague Agez,
Etienne Brasselet
Abstract:
The realization of geometric phase optical device operating over a broad spectral range is usually confronted with intrinsic limitations depending of the physical process at play. Here we propose to use chiral nematic liquid crystal slabs with helical ordering that varies in three dimensions. Namely, gradient-pitch cholesterics endowed with in-plane space-variant angular positioning of the supramo…
▽ More
The realization of geometric phase optical device operating over a broad spectral range is usually confronted with intrinsic limitations depending of the physical process at play. Here we propose to use chiral nematic liquid crystal slabs with helical ordering that varies in three dimensions. Namely, gradient-pitch cholesterics endowed with in-plane space-variant angular positioning of the supramolecular helix. By doing so, we show that the recently introduced Bragg-Berry mirrors [Opt. Lett. {\bf 41}, 3972-3975 (2016)] can be endowed with ultra-broadband spectral range. Experimental demonstration is made in the case of ultra-broadband optical vortex generation in the visible domain. These results offer practical solution to the polychromatic management of the orbital angular momentum of light combining the circular Bragg reflection of chiral media with the Berry phase.
△ Less
Submitted 16 September, 2017;
originally announced September 2017.
-
Polarization Properties of the Cholesteric Liquid Crystals
Authors:
A. H. Gevorgyan,
M. Z. Harutyunyan,
K. B. Oganesyan,
E. A. Ayryan,
Michal Hnatic,
M. Rafayelyan,
Yuri V. Rostovtsev,
Gershon Kurizki
Abstract:
In the present paper, we investigate the polarization properties of the cholesteric liquid crystals (CLCs) with an isotropic/anisotropic defect inside them. Possibilities of amplification of the polarization plane rotation and stabilization of the light polarization azimuth by these systems are investigated in details.
In the present paper, we investigate the polarization properties of the cholesteric liquid crystals (CLCs) with an isotropic/anisotropic defect inside them. Possibilities of amplification of the polarization plane rotation and stabilization of the light polarization azimuth by these systems are investigated in details.
△ Less
Submitted 11 April, 2017;
originally announced April 2017.
-
Bragg-Berry mirrors: reflective broadband q-plates
Authors:
Mushegh Rafayelyan,
Etienne Brasselet
Abstract:
We report on the experimental realization of flat mirrors enabling the broadband generation of optical vortices upon reflection. The effect is based on the geometric Berry phase associated with the circular Bragg reflection phenomenon from chiral uniaxial media. We show the reflective optical vortex generation from both diffractive and nondiffractive paraxial light beams using spatially patterned…
▽ More
We report on the experimental realization of flat mirrors enabling the broadband generation of optical vortices upon reflection. The effect is based on the geometric Berry phase associated with the circular Bragg reflection phenomenon from chiral uniaxial media. We show the reflective optical vortex generation from both diffractive and nondiffractive paraxial light beams using spatially patterned chiral liquid crystal films. The intrinsic spectrally broadband character of spin-orbit generation of optical phase singularities is demonstrated over the full visible domain. Our results do not rely on any birefringent retardation requirement and consequently foster the development of a novel generation of robust optical elements for spin-orbit photonic technologies.
△ Less
Submitted 30 August, 2016; v1 submitted 8 June, 2016;
originally announced June 2016.
-
An Algorithm for the Simulations of the Magnetized Neutron Star Cooling
Authors:
H. Grigorian,
A. Ayriyan,
E. Chubarian,
A. Piloyan,
M. Rafayelyan
Abstract:
The model and algorithm for the cooling of the magnetized neutron stars are presented. The cooling evolution described by system of parabolic partial differential equations with non-linear coefficients is solved using Alternating Direction Implicit method. The difference scheme and the preliminary results of simulations are presented.
The model and algorithm for the cooling of the magnetized neutron stars are presented. The cooling evolution described by system of parabolic partial differential equations with non-linear coefficients is solved using Alternating Direction Implicit method. The difference scheme and the preliminary results of simulations are presented.
△ Less
Submitted 17 November, 2015;
originally announced November 2015.
