-
Finite-time Lyapunov exponents of deep neural networks
Authors:
L. Storm,
H. Linander,
J. Bec,
K. Gustavsson,
B. Mehlig
Abstract:
We compute how small input perturbations affect the output of deep neural networks, exploring an analogy between deep networks and dynamical systems, where the growth or decay of local perturbations is characterised by finite-time Lyapunov exponents. We show that the maximal exponent forms geometrical structures in input space, akin to coherent structures in dynamical systems. Ridges of large posi…
▽ More
We compute how small input perturbations affect the output of deep neural networks, exploring an analogy between deep networks and dynamical systems, where the growth or decay of local perturbations is characterised by finite-time Lyapunov exponents. We show that the maximal exponent forms geometrical structures in input space, akin to coherent structures in dynamical systems. Ridges of large positive exponents divide input space into different regions that the network associates with different classes. These ridges visualise the geometry that deep networks construct in input space, shedding light on the fundamental mechanisms underlying their learning capabilities.
△ Less
Submitted 21 June, 2023;
originally announced June 2023.
-
Constraints on parameter choices for successful reservoir computing
Authors:
L. Storm,
K. Gustavsson,
B. Mehlig
Abstract:
Echo-state networks are simple models of discrete dynamical systems driven by a time series. By selecting network parameters such that the dynamics of the network is contractive, characterized by a negative maximal Lyapunov exponent, the network may synchronize with the driving signal. Exploiting this synchronization, the echo-state network may be trained to autonomously reproduce the input dynami…
▽ More
Echo-state networks are simple models of discrete dynamical systems driven by a time series. By selecting network parameters such that the dynamics of the network is contractive, characterized by a negative maximal Lyapunov exponent, the network may synchronize with the driving signal. Exploiting this synchronization, the echo-state network may be trained to autonomously reproduce the input dynamics, enabling time-series prediction. However, while synchronization is a necessary condition for prediction, it is not sufficient. Here, we study what other conditions are necessary for successful time-series prediction. We identify two key parameters for prediction performance, and conduct a parameter sweep to find regions where prediction is successful. These regions differ significantly depending on whether full or partial phase space information about the input is provided to the network during training. We explain how these regions emerge.
△ Less
Submitted 3 June, 2022;
originally announced June 2022.
-
Optimal control of point-to-point navigation in turbulent time-dependent flows using Reinforcement Learning
Authors:
Michele Buzzicotti,
Luca Biferale,
Fabio Bonaccorso,
Patricio Clark di Leoni,
Kristian Gustavsson
Abstract:
We present theoretical and numerical results concerning the problem to find the path that minimizes the time to navigate between two given points in a complex fluid under realistic navigation constraints. We contrast deterministic Optimal Navigation (ON) control with stochastic policies obtained by Reinforcement Learning (RL) algorithms. We show that Actor-Critic RL algorithms are able to find qua…
▽ More
We present theoretical and numerical results concerning the problem to find the path that minimizes the time to navigate between two given points in a complex fluid under realistic navigation constraints. We contrast deterministic Optimal Navigation (ON) control with stochastic policies obtained by Reinforcement Learning (RL) algorithms. We show that Actor-Critic RL algorithms are able to find quasi-optimal solutions in the presence of either time-independent or chaotically evolving flow configurations. For our application, ON solutions develop unstable behavior within the typical duration of the navigation process, and are therefore not useful in practice. We first explore navigation of turbulent flow using a constant propulsion speed. Based on a discretized phase-space, the propulsion direction is adjusted with the aim to minimize the time spent to reach the target. Further, we explore a case where additional control is obtained by allowing the engine to power off. Exploiting advection of the underlying flow, allows the target to be reached with less energy consumption. In this case, we optimize a linear combination between the total navigation time and the total time the engine is switched off. Our approach can be generalized to other setups, for example, navigation under imperfect environmental forecast or with different models for the moving vessel.
△ Less
Submitted 27 February, 2021;
originally announced March 2021.
-
Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning
Authors:
Luca Biferale,
Fabio Bonaccorso,
Michele Buzzicotti,
Patricio Clark Di Leoni,
Kristian Gustavsson
Abstract:
To find the path that minimizes the time to navigate between two given points in a fluid flow is known as Zermelo's problem. Here, we investigate it by using a Reinforcement Learning (RL) approach for the case of a vessel which has a slip velocity with fixed intensity, Vs , but variable direction and navigating in a 2D turbulent sea. We show that an Actor-Critic RL algorithm is able to find quasi-…
▽ More
To find the path that minimizes the time to navigate between two given points in a fluid flow is known as Zermelo's problem. Here, we investigate it by using a Reinforcement Learning (RL) approach for the case of a vessel which has a slip velocity with fixed intensity, Vs , but variable direction and navigating in a 2D turbulent sea. We show that an Actor-Critic RL algorithm is able to find quasi-optimal solutions for both time-independent and chaotically evolving flow configurations. For the frozen case, we also compared the results with strategies obtained analytically from continuous Optimal Navigation (ON) protocols. We show that for our application, ON solutions are unstable for the typical duration of the navigation process, and are therefore not useful in practice. On the other hand, RL solutions are much more robust with respect to small changes in the initial conditions and to external noise, even when V s is much smaller than the maximum flow velocity. Furthermore, we show how the RL approach is able to take advantage of the flow properties in order to reach the target, especially when the steering speed is small.
△ Less
Submitted 26 September, 2019; v1 submitted 17 July, 2019;
originally announced July 2019.
-
Finding Efficient Swimming Strategies in a Three Dimensional Chaotic Flow by Reinforcement Learning
Authors:
K. Gustavsson,
L. Biferale,
A. Celani,
S. Colabrese
Abstract:
We apply a reinforcement learning algorithm to show how smart particles can learn approximately optimal strategies to navigate in complex flows. In this paper we consider microswimmers in a paradigmatic three-dimensional case given by a stationary superposition of two Arnold-Beltrami-Childress flows with chaotic advection along streamlines. In such a flow, we study the evolution of point-like part…
▽ More
We apply a reinforcement learning algorithm to show how smart particles can learn approximately optimal strategies to navigate in complex flows. In this paper we consider microswimmers in a paradigmatic three-dimensional case given by a stationary superposition of two Arnold-Beltrami-Childress flows with chaotic advection along streamlines. In such a flow, we study the evolution of point-like particles which can decide in which direction to swim, while kee** the velocity amplitude constant. We show that it is sufficient to endow the swimmers with a very restricted set of actions (six fixed swimming directions in our case) to have enough freedom to find efficient strategies to move upward and escape local fluid traps. The key ingredient is the learning-from-experience structure of the algorithm, which assigns positive or negative rewards depending on whether the taken action is, or is not, profitable for the predetermined goal in the long term horizon. This is another example supporting the efficiency of the reinforcement learning approach to learn how to accomplish difficult tasks in complex fluid environments.
△ Less
Submitted 31 December, 2017; v1 submitted 15 November, 2017;
originally announced November 2017.
-
Flow Navigation by Smart Microswimmers via Reinforcement Learning
Authors:
Simona Colabrese,
Kristian Gustavsson,
Antonio Celani,
Luca Biferale
Abstract:
Smart active particles can acquire some limited knowledge of the fluid environment from simple mechanical cues and exert a control on their preferred steering direction. Their goal is to learn the best way to navigate by exploiting the underlying flow whenever possible. As an example, we focus our attention on smart gravitactic swimmers. These are active particles whose task is to reach the highes…
▽ More
Smart active particles can acquire some limited knowledge of the fluid environment from simple mechanical cues and exert a control on their preferred steering direction. Their goal is to learn the best way to navigate by exploiting the underlying flow whenever possible. As an example, we focus our attention on smart gravitactic swimmers. These are active particles whose task is to reach the highest altitude within some time horizon, given the constraints enforced by fluid mechanics. By means of numerical experiments, we show that swimmers indeed learn nearly optimal strategies just by experience. A reinforcement learning algorithm allows particles to learn effective strategies even in difficult situations when, in the absence of control, they would end up being trapped by flow structures. These strategies are highly nontrivial and cannot be easily guessed in advance. This Letter illustrates the potential of reinforcement learning algorithms to model adaptive behavior in complex flows and paves the way towards the engineering of smart microswimmers that solve difficult navigation problems.
△ Less
Submitted 26 July, 2017; v1 submitted 30 January, 2017;
originally announced January 2017.
-
Two Computational Models for Simulating the Tumbling Motion of Elongated Particles in Fluids
Authors:
Dominik Bartuschat,
Ellen Fischermeier,
Katarina Gustavsson,
Ulrich Rüde
Abstract:
Suspensions with fiber-like particles in the low Reynolds number regime are modeled by two different approaches that both use a Lagrangian representation of individual particles. The first method is the well-established formulation based on Stokes flow that is formulated as integral equations. It uses a slender body approximation for the fibers to represent the interaction between them directly wi…
▽ More
Suspensions with fiber-like particles in the low Reynolds number regime are modeled by two different approaches that both use a Lagrangian representation of individual particles. The first method is the well-established formulation based on Stokes flow that is formulated as integral equations. It uses a slender body approximation for the fibers to represent the interaction between them directly without explicitly computing the flow field. The second is a new technique using the 3D lattice Boltzmann method on parallel supercomputers. Here the flow computation is coupled to a computational model of the dynamics of rigid bodies using fluid-structure interaction techniques. Both methods can be applied to simulate fibers in fluid flow. They are carefully validated and compared against each other, exposing systematically their strengths and weaknesses regarding their accuracy, the computational cost, and possible model extensions.
△ Less
Submitted 23 March, 2015;
originally announced March 2015.
