-
Dynamically writing coupled memories using a reinforcement learning agent, meeting physical bounds
Authors:
Théo Jules,
Laura Michel,
Adèle Douin,
Frédéric Lechenault
Abstract:
Traditional memory writing operations proceed one bit at a time, where e.g. an individual magnetic domain is force-flipped by a localized external field. One way to increase material storage capacity would be to write several bits at a time in the bulk of the material. However, the manipulation of bits is commonly done through quasi-static operations. While simple to model, this method is known to…
▽ More
Traditional memory writing operations proceed one bit at a time, where e.g. an individual magnetic domain is force-flipped by a localized external field. One way to increase material storage capacity would be to write several bits at a time in the bulk of the material. However, the manipulation of bits is commonly done through quasi-static operations. While simple to model, this method is known to reduce memory capacity. In this paper, we demonstrate how a reinforcement learning agent can exploit the dynamical response of a simple multi-bit mechanical system to restore its memory to full capacity. To do so, we introduce a model framework consisting of a chain of bi-stable springs, which is manipulated on one end by the external action of the agent. We show that the agent manages to learn how to reach all available states for three springs, even though some states are not reachable through adiabatic manipulation, and that both the training speed and convergence within physical parameter space are improved using transfer learning techniques. Interestingly, the agent also points to an optimal design of the system in terms of writing time. In fact, it appears to learn how to take advantage of the underlying physics: the control time exhibits a non-monotonic dependence on the internal dissipation, reaching a minimum at a cross-over shown to verify a mechanically motivated scaling relation.
△ Less
Submitted 6 May, 2022;
originally announced May 2022.
-
KnitCity: a machine learning-based, game-theoretical framework for prediction assessment and seismic risk policy design
Authors:
Adèle Douin,
J. P. Bruneton,
Frédéric Lechenault
Abstract:
Knitted fabric exhibits avalanche-like events when deformed: by analogy with eathquakes, we are interested in predicting these "knitquakes". However, as in most analogous seismic models, the peculiar statistics of the corresponding time-series severely jeopardize this endeavour, due to the time intermittence and scale-invariance of these events. But more importantly, such predictions are hard to {…
▽ More
Knitted fabric exhibits avalanche-like events when deformed: by analogy with eathquakes, we are interested in predicting these "knitquakes". However, as in most analogous seismic models, the peculiar statistics of the corresponding time-series severely jeopardize this endeavour, due to the time intermittence and scale-invariance of these events. But more importantly, such predictions are hard to {\it assess}: depending on the choice of what to predict, the results can be very different and not easily compared. Furthermore, forecasting models may be trained with various generic metrics which ignore some important specificities of the problem at hand, in our case seismic risk. Finally, these models often do not provide a clear strategy regarding the best way to use these predictions in practice. Here we introduce a framework that allows to design, evaluate and compare not only predictors but also decision-making policies: a model seismically active {\it city} subjected to the crackling dynamics observed in the mechanical response of knitted fabric. We thus proceed to study the population of KnitCity, introducing a policy through which the mayor of the town can decide to either keep people in, which in case of large events cause human loss, or evacuate the city, which costs a daily fee. The policy only relies on past seismic observations. We construct efficient policies using a reinforcement learning environment and various time-series predictors based on artificial neural networks. By inducing a physically motivated metric on the predictors, this mechanism allows quantitative assessment and comparison of their relevance in the decision-making process.
△ Less
Submitted 5 May, 2022;
originally announced May 2022.
-
Exploration and Exploitation in Symbolic Regression using Quality-Diversity and Evolutionary Strategies Algorithms
Authors:
J. -P. Bruneton,
L. Cazenille,
A. Douin,
V. Reverdy
Abstract:
By combining Genetic Programming, MAP-Elites and Covariance Matrix Adaptation Evolution Strategy, we demonstrate very high success rates in Symbolic Regression problems. MAP-Elites is used to improve exploration while preserving diversity and avoiding premature convergence and bloat. Then, a Covariance Matrix Adaptation-Evolution Strategy is used to evaluate free scalars through a non-gradient-bas…
▽ More
By combining Genetic Programming, MAP-Elites and Covariance Matrix Adaptation Evolution Strategy, we demonstrate very high success rates in Symbolic Regression problems. MAP-Elites is used to improve exploration while preserving diversity and avoiding premature convergence and bloat. Then, a Covariance Matrix Adaptation-Evolution Strategy is used to evaluate free scalars through a non-gradient-based black-box optimizer. Although this evaluation approach is not computationally scalable to high dimensional problems, our algorithm is able to find exactly most of the $31$ targets extracted from the literature on which we evaluate it.
△ Less
Submitted 10 June, 2019;
originally announced June 2019.
