-
SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning
Authors:
Nicholas Zolman,
Urban Fasel,
J. Nathan Kutz,
Steven L. Brunton
Abstract:
Deep reinforcement learning (DRL) has shown significant promise for uncovering sophisticated control policies that interact in environments with complicated dynamics, such as stabilizing the magnetohydrodynamics of a tokamak fusion reactor or minimizing the drag force exerted on an object in a fluid flow. However, these algorithms require an abundance of training examples and may become prohibitiv…
▽ More
Deep reinforcement learning (DRL) has shown significant promise for uncovering sophisticated control policies that interact in environments with complicated dynamics, such as stabilizing the magnetohydrodynamics of a tokamak fusion reactor or minimizing the drag force exerted on an object in a fluid flow. However, these algorithms require an abundance of training examples and may become prohibitively expensive for many applications. In addition, the reliance on deep neural networks often results in an uninterpretable, black-box policy that may be too computationally expensive to use with certain embedded systems. Recent advances in sparse dictionary learning, such as the sparse identification of nonlinear dynamics (SINDy), have shown promise for creating efficient and interpretable data-driven models in the low-data regime. In this work we introduce SINDy-RL, a unifying framework for combining SINDy and DRL to create efficient, interpretable, and trustworthy representations of the dynamics model, reward function, and control policy. We demonstrate the effectiveness of our approaches on benchmark control environments and challenging fluids problems. SINDy-RL achieves comparable performance to state-of-the-art DRL algorithms using significantly fewer interactions in the environment and results in an interpretable control policy orders of magnitude smaller than a deep neural network policy.
△ Less
Submitted 14 March, 2024;
originally announced March 2024.
-
A Unified Framework to Enforce, Discover, and Promote Symmetry in Machine Learning
Authors:
Samuel E. Otto,
Nicholas Zolman,
J. Nathan Kutz,
Steven L. Brunton
Abstract:
Symmetry is present throughout nature and continues to play an increasingly central role in physics and machine learning. Fundamental symmetries, such as Poincaré invariance, allow physical laws discovered in laboratories on Earth to be extrapolated to the farthest reaches of the universe. Symmetry is essential to achieving this extrapolatory power in machine learning applications. For example, tr…
▽ More
Symmetry is present throughout nature and continues to play an increasingly central role in physics and machine learning. Fundamental symmetries, such as Poincaré invariance, allow physical laws discovered in laboratories on Earth to be extrapolated to the farthest reaches of the universe. Symmetry is essential to achieving this extrapolatory power in machine learning applications. For example, translation invariance in image classification allows models with fewer parameters, such as convolutional neural networks, to be trained on smaller data sets and achieve state-of-the-art performance. In this paper, we provide a unifying theoretical and methodological framework for incorporating symmetry into machine learning models in three ways: 1. enforcing known symmetry when training a model; 2. discovering unknown symmetries of a given model or data set; and 3. promoting symmetry during training by learning a model that breaks symmetries within a user-specified group of candidates when there is sufficient evidence in the data. We show that these tasks can be cast within a common mathematical framework whose central object is the Lie derivative associated with fiber-linear Lie group actions on vector bundles. We extend and unify several existing results by showing that enforcing and discovering symmetry are linear-algebraic tasks that are dual with respect to the bilinear structure of the Lie derivative. We also propose a novel way to promote symmetry by introducing a class of convex regularization functions based on the Lie derivative and nuclear norm relaxation to penalize symmetry breaking during training of machine learning models. We explain how these ideas can be applied to a wide range of machine learning models including basis function regression, dynamical systems discovery, multilayer perceptrons, and neural networks acting on spatial fields such as images.
△ Less
Submitted 31 October, 2023;
originally announced November 2023.
-
Learning to Predict 3D Rotational Dynamics from Images of a Rigid Body with Unknown Mass Distribution
Authors:
Justice Mason,
Christine Allen-Blanchette,
Nicholas Zolman,
Elizabeth Davison,
Naomi Ehrich Leonard
Abstract:
In many real-world settings, image observations of freely rotating 3D rigid bodies may be available when low-dimensional measurements are not. However, the high-dimensionality of image data precludes the use of classical estimation techniques to learn the dynamics. The usefulness of standard deep learning methods is also limited, because an image of a rigid body reveals nothing about the distribut…
▽ More
In many real-world settings, image observations of freely rotating 3D rigid bodies may be available when low-dimensional measurements are not. However, the high-dimensionality of image data precludes the use of classical estimation techniques to learn the dynamics. The usefulness of standard deep learning methods is also limited, because an image of a rigid body reveals nothing about the distribution of mass inside the body, which, together with initial angular velocity, is what determines how the body will rotate. We present a physics-based neural network model to estimate and predict 3D rotational dynamics from image sequences. We achieve this using a multi-stage prediction pipeline that maps individual images to a latent representation homeomorphic to $\mathbf{SO}(3)$, computes angular velocities from latent pairs, and predicts future latent states using the Hamiltonian equations of motion. We demonstrate the efficacy of our approach on new rotating rigid-body datasets of sequences of synthetic images of rotating objects, including cubes, prisms and satellites, with unknown uniform and non-uniform mass distributions. Our model outperforms competing baselines on our datasets, producing better qualitative predictions and reducing the error observed for the state-of-the-art Hamiltonian Generative Network by a factor of 2.
△ Less
Submitted 10 April, 2024; v1 submitted 24 August, 2023;
originally announced August 2023.
-
Learning Interpretable Dynamics from Images of a Freely Rotating 3D Rigid Body
Authors:
Justice Mason,
Christine Allen-Blanchette,
Nicholas Zolman,
Elizabeth Davison,
Naomi Leonard
Abstract:
In many real-world settings, image observations of freely rotating 3D rigid bodies, such as satellites, may be available when low-dimensional measurements are not. However, the high-dimensionality of image data precludes the use of classical estimation techniques to learn the dynamics and a lack of interpretability reduces the usefulness of standard deep learning methods. In this work, we present…
▽ More
In many real-world settings, image observations of freely rotating 3D rigid bodies, such as satellites, may be available when low-dimensional measurements are not. However, the high-dimensionality of image data precludes the use of classical estimation techniques to learn the dynamics and a lack of interpretability reduces the usefulness of standard deep learning methods. In this work, we present a physics-informed neural network model to estimate and predict 3D rotational dynamics from image sequences. We achieve this using a multi-stage prediction pipeline that maps individual images to a latent representation homeomorphic to $\mathbf{SO}(3)$, computes angular velocities from latent pairs, and predicts future latent states using the Hamiltonian equations of motion with a learned representation of the Hamiltonian. We demonstrate the efficacy of our approach on a new rotating rigid-body dataset with sequences of rotating cubes and rectangular prisms with uniform and non-uniform density.
△ Less
Submitted 23 August, 2023; v1 submitted 22 September, 2022;
originally announced September 2022.
-
Adinkras, Dessins, Origami, and Supersymmetry Spectral Triples
Authors:
Matilde Marcolli,
Nick Zolman
Abstract:
We investigate the spectral geometry and spectral action functionals associated to 1D Supersymmetry Algebras, using the classification of these superalgebras in terms of Adinkra graphs and the construction of associated dessin d'enfant and origami curves. The resulting spectral action functionals are computed in terms of the Selberg (super) trace formula.
We investigate the spectral geometry and spectral action functionals associated to 1D Supersymmetry Algebras, using the classification of these superalgebras in terms of Adinkra graphs and the construction of associated dessin d'enfant and origami curves. The resulting spectral action functionals are computed in terms of the Selberg (super) trace formula.
△ Less
Submitted 17 July, 2016; v1 submitted 14 June, 2016;
originally announced June 2016.
-
The Mattig Expression for a d-dimensional Gauss Bonnet FRWL Cosmology
Authors:
Keith Andrew,
Eric Steinfelds,
Nick Zolman
Abstract:
Here we study the form of the Mattig equation applied in a cosmological setting for spacetime metric gravity models described by the Gauss-Bonnet action. We start with expressing the Mattig relation for cosmological magnitudes in terms of standard metric functions and redshift values. Then we present the Gauss-Bonnet field equations and the associated limits for special solutions in an arbitrary n…
▽ More
Here we study the form of the Mattig equation applied in a cosmological setting for spacetime metric gravity models described by the Gauss-Bonnet action. We start with expressing the Mattig relation for cosmological magnitudes in terms of standard metric functions and redshift values. Then we present the Gauss-Bonnet field equations and the associated limits for special solutions in an arbitrary number of dimensions. These solutions are used to rewrite the Mattig relation with correction terms from the Gauss-Bonnet contributions for the case where the Gauss-Bonnet scale factor can be directly used to find the distance modulus and for the case where the Gauss-Bonnet field equations can be expressed as a small high z perturbation on the standard Einstein field equations. As a result we can express the perturbative distance modulus, which includes the apparent magnitude, as an additive correction to the standard distance modulus. This results in a small shift in the apparent magnitude of a high z object which gives a contribution depending on the Gauss-Bonnet coupling, spacetime dimension and the cosmological constant.
△ Less
Submitted 22 January, 2016;
originally announced January 2016.
-
The Origin and Evolution of the Galaxy Mass-Metallicity Relation
Authors:
Xiangcheng Ma,
Philip F. Hopkins,
Claude-Andre Faucher-Giguere,
Nick Zolman,
Alexander L. Muratov,
Dusan Keres,
Eliot Quataert
Abstract:
We use high-resolution cosmological zoom-in simulations from the Feedback in Realistic Environment (FIRE) project to study the galaxy mass-metallicity relations (MZR) from z=0-6. These simulations include explicit models of the multi-phase ISM, star formation, and stellar feedback. The simulations cover halo masses Mhalo=10^9-10^13 Msun and stellar mass Mstar=10^4-10^11 Msun at z=0 and have been s…
▽ More
We use high-resolution cosmological zoom-in simulations from the Feedback in Realistic Environment (FIRE) project to study the galaxy mass-metallicity relations (MZR) from z=0-6. These simulations include explicit models of the multi-phase ISM, star formation, and stellar feedback. The simulations cover halo masses Mhalo=10^9-10^13 Msun and stellar mass Mstar=10^4-10^11 Msun at z=0 and have been shown to produce many observed galaxy properties from z=0-6. For the first time, our simulations agree reasonably well with the observed mass-metallicity relations at z=0-3 for a broad range of galaxy masses. We predict the evolution of the MZR from z=0-6 as log(Zgas/Zsun)=12+log(O/H)-9.0=0.35[log(Mstar/Msun)-10]+0.93 exp(-0.43 z)-1.05 and log(Zstar/Zsun)=[Fe/H]-0.2=0.40[log(Mstar/Msun)-10]+0.67 exp(-0.50 z)-1.04, for gas-phase and stellar metallicity, respectively. Our simulations suggest that the evolution of MZR is associated with the evolution of stellar/gas mass fractions at different redshifts, indicating the existence of a universal metallicity relation between stellar mass, gas mass, and metallicities. In our simulations, galaxies above Mstar=10^6 Msun are able to retain a large fraction of their metals inside the halo, because metal-rich winds fail to escape completely and are recycled into the galaxy. This resolves a long-standing discrepancy between "sub-grid" wind models (and semi-analytic models) and observations, where common sub-grid models cannot simultaneously reproduce the MZR and the stellar mass functions.
△ Less
Submitted 17 October, 2015; v1 submitted 8 April, 2015;
originally announced April 2015.
