-
Pseudo-Hamiltonian neural networks for learning partial differential equations
Authors:
Sølve Eidnes,
Kjetil Olsen Lye
Abstract:
Pseudo-Hamiltonian neural networks (PHNN) were recently introduced for learning dynamical systems that can be modelled by ordinary differential equations. In this paper, we extend the method to partial differential equations. The resulting model is comprised of up to three neural networks, modelling terms representing conservation, dissipation and external forces, and discrete convolution operator…
▽ More
Pseudo-Hamiltonian neural networks (PHNN) were recently introduced for learning dynamical systems that can be modelled by ordinary differential equations. In this paper, we extend the method to partial differential equations. The resulting model is comprised of up to three neural networks, modelling terms representing conservation, dissipation and external forces, and discrete convolution operators that can either be learned or be given as input. We demonstrate numerically the superior performance of PHNN compared to a baseline model that models the full dynamics by a single neural network. Moreover, since the PHNN model consists of three parts with different physical interpretations, these can be studied separately to gain insight into the system, and the learned model is applicable also if external forces are removed or changed.
△ Less
Submitted 2 January, 2024; v1 submitted 27 April, 2023;
originally announced April 2023.
-
Constrained mixers for the quantum approximate optimization algorithm
Authors:
Franz G. Fuchs,
Kjetil Olsen Lye,
Halvor Møll Nilsen,
Alexander J. Stasik,
Giorgio Sartor
Abstract:
The quantum approximate optimization algorithm/quantum alternating operator ansatz (QAOA) is a heuristic to find approximate solutions of combinatorial optimization problems. Most literature is limited to quadratic problems without constraints. However, many practically relevant optimization problems do have (hard) constraints that need to be fulfilled. In this article, we present a framework for…
▽ More
The quantum approximate optimization algorithm/quantum alternating operator ansatz (QAOA) is a heuristic to find approximate solutions of combinatorial optimization problems. Most literature is limited to quadratic problems without constraints. However, many practically relevant optimization problems do have (hard) constraints that need to be fulfilled. In this article, we present a framework for constructing mixing operators that restrict the evolution to a subspace of the full Hilbert space given by these constraints; We generalize the "XY"-mixer designed to preserve the subspace of "one-hot" states to the general case of subspaces given by a number of computational basis states. We expose the underlying mathematical structure which reveals more of how mixers work and how one can minimize their cost in terms of number of CX gates, particularly when Trotterization is taken into account. Our analysis also leads to valid Trotterizations for "XY"-mixer with fewer CX gates than is known to date. In view of practical implementations, we also describe algorithms for efficient decomposition into basis gates. Several examples of more general cases are presented and analyzed.
△ Less
Submitted 22 June, 2022; v1 submitted 11 March, 2022;
originally announced March 2022.
-
Convergence rates of monotone schemes for conservation laws for data with unbounded total variation
Authors:
Ulrik Skre Fjordholm,
Kjetil Olsen Lye
Abstract:
We prove convergence rates of monotone schemes for conservation laws for Hölder continuous initial data with unbounded total variation, provided that the Hölder exponent of the initial data is greater than $1/2$. For strictly $\mathrm{Lip}^+$ stable monotone schemes, we prove convergence for any positive Hölder exponent. Numerical experiments are presented which verify the theory.
We prove convergence rates of monotone schemes for conservation laws for Hölder continuous initial data with unbounded total variation, provided that the Hölder exponent of the initial data is greater than $1/2$. For strictly $\mathrm{Lip}^+$ stable monotone schemes, we prove convergence for any positive Hölder exponent. Numerical experiments are presented which verify the theory.
△ Less
Submitted 15 October, 2020;
originally announced October 2020.
-
Iterative Surrogate Model Optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks
Authors:
Kjetil O. Lye,
Siddhartha Mishra,
Deep Ray,
Praveen Chandrasekhar
Abstract:
We present a novel active learning algorithm, termed as iterative surrogate model optimization (ISMO), for robust and efficient numerical approximation of PDE constrained optimization problems. This algorithm is based on deep neural networks and its key feature is the iterative selection of training data through a feedback loop between deep neural networks and any underlying standard optimization…
▽ More
We present a novel active learning algorithm, termed as iterative surrogate model optimization (ISMO), for robust and efficient numerical approximation of PDE constrained optimization problems. This algorithm is based on deep neural networks and its key feature is the iterative selection of training data through a feedback loop between deep neural networks and any underlying standard optimization algorithm. Under suitable hypotheses, we show that the resulting optimizers converge exponentially fast (and with exponentially decaying variance), with respect to increasing number of training samples. Numerical examples for optimal control, parameter identification and shape optimization problems for PDEs are provided to validate the proposed theory and to illustrate that ISMO significantly outperforms a standard deep neural network based surrogate optimization algorithm.
△ Less
Submitted 13 August, 2020;
originally announced August 2020.
-
A Multi-level procedure for enhancing accuracy of machine learning algorithms
Authors:
Kjetil O. Lye,
Siddhartha Mishra,
Roberto Molinaro
Abstract:
We propose a multi-level method to increase the accuracy of machine learning algorithms for approximating observables in scientific computing, particularly those that arise in systems modeled by differential equations. The algorithm relies on judiciously combining a large number of computationally cheap training data on coarse resolutions with a few expensive training samples on fine grid resoluti…
▽ More
We propose a multi-level method to increase the accuracy of machine learning algorithms for approximating observables in scientific computing, particularly those that arise in systems modeled by differential equations. The algorithm relies on judiciously combining a large number of computationally cheap training data on coarse resolutions with a few expensive training samples on fine grid resolutions. Theoretical arguments for lowering the generalization error, based on reducing the variance of the underlying maps, are provided and numerical evidence, indicating significant gains over underlying single-level machine learning algorithms, are presented. Moreover, we also apply the multi-level algorithm in the context of forward uncertainty quantification and observe a considerable speed-up over competing algorithms.
△ Less
Submitted 3 July, 2020; v1 submitted 20 September, 2019;
originally announced September 2019.
-
Deep learning observables in computational fluid dynamics
Authors:
Kjetil O. Lye,
Siddhartha Mishra,
Deep Ray
Abstract:
Many large scale problems in computational fluid dynamics such as uncertainty quantification, Bayesian inversion, data assimilation and PDE constrained optimization are considered very challenging computationally as they require a large number of expensive (forward) numerical solutions of the corresponding PDEs. We propose a machine learning algorithm, based on deep artificial neural networks, tha…
▽ More
Many large scale problems in computational fluid dynamics such as uncertainty quantification, Bayesian inversion, data assimilation and PDE constrained optimization are considered very challenging computationally as they require a large number of expensive (forward) numerical solutions of the corresponding PDEs. We propose a machine learning algorithm, based on deep artificial neural networks, that predicts the underlying \emph{input parameters to observable} map from a few training samples (computed realizations of this map). By a judicious combination of theoretical arguments and empirical observations, we find suitable network architectures and training hyperparameters that result in robust and efficient neural network approximations of the parameters to observable map. Numerical experiments are presented to demonstrate low prediction errors for the trained network networks, even when the network has been trained with a few samples, at a computational cost which is several orders of magnitude lower than the underlying PDE solver.
Moreover, we combine the proposed deep learning algorithm with Monte Carlo (MC) and Quasi-Monte Carlo (QMC) methods to efficiently compute uncertainty propagation for nonlinear PDEs. Under the assumption that the underlying neural networks generalize well, we prove that the deep learning MC and QMC algorithms are guaranteed to be faster than the baseline (quasi-) Monte Carlo methods. Numerical experiments demonstrating one to two orders of magnitude speed up over baseline QMC and MC algorithms, for the intricate problem of computing probability distributions of the observable, are also presented.
△ Less
Submitted 16 December, 2019; v1 submitted 7 March, 2019;
originally announced March 2019.
-
Multilevel Monte-Carlo for measure valued solutions
Authors:
Kjetil Olsen Lye
Abstract:
We propose a Multilevel Monte-Carlo (MLMC) method for computing entropy measure valued solutions of hyperbolic conservation laws. Sharp bounds for the narrow convergence of MLMC for the entropy measure valued solutions are proposed. An optimal work-vs-error bound for the MLMC method is derived assuming only an abstract decay criterion on the variance. Finally, we display numerical experiments of c…
▽ More
We propose a Multilevel Monte-Carlo (MLMC) method for computing entropy measure valued solutions of hyperbolic conservation laws. Sharp bounds for the narrow convergence of MLMC for the entropy measure valued solutions are proposed. An optimal work-vs-error bound for the MLMC method is derived assuming only an abstract decay criterion on the variance. Finally, we display numerical experiments of cases where MLMC is, and is not, efficient when compared to Monte-Carlo.
△ Less
Submitted 23 November, 2016;
originally announced November 2016.
