-
Accelerating the convergence of Newton's method for nonlinear elliptic PDEs using Fourier neural operators
Authors:
Joubine Aghili,
Emmanuel Franck,
Romain Hild,
Victor Michel-Dansac,
Vincent Vigon
Abstract:
It is well known that Newton's method, especially when applied to large problems such as the discretization of nonlinear partial differential equations (PDEs), can have trouble converging if the initial guess is too far from the solution. This work focuses on accelerating this convergence, in the context of the discretization of nonlinear elliptic PDEs. We first provide a quick review of existing…
▽ More
It is well known that Newton's method, especially when applied to large problems such as the discretization of nonlinear partial differential equations (PDEs), can have trouble converging if the initial guess is too far from the solution. This work focuses on accelerating this convergence, in the context of the discretization of nonlinear elliptic PDEs. We first provide a quick review of existing methods, and justify our choice of learning an initial guess with a Fourier neural operator (FNO). This choice was motivated by the mesh-independence of such operators, whose training and evaluation can be performed on grids with different resolutions. The FNO is trained using a loss minimization over generated data, loss functions based on the PDE discretization. Numerical results, in one and two dimensions, show that the proposed initial guess accelerates the convergence of Newton's method by a large margin compared to a naive initial guess, especially for highly nonlinear or anisotropic problems.
△ Less
Submitted 5 March, 2024;
originally announced March 2024.
-
A Dynamical Neural Galerkin Scheme for Filtering Problems
Authors:
Joubine Aghili,
Joy Zialesi Atokple,
Marie Billaud-Friess,
Guillaume Garnier,
Olga Mula,
Norbert Tognon
Abstract:
This paper considers the filtering problem which consists in reconstructing the state of a dynamical system with partial observations coming from sensor measurements, and the knowledge that the dynamics are governed by a physical PDE model with unknown parameters. We present a filtering algorithm where the reconstruction of the dynamics is done with neural network approximations whose weights are…
▽ More
This paper considers the filtering problem which consists in reconstructing the state of a dynamical system with partial observations coming from sensor measurements, and the knowledge that the dynamics are governed by a physical PDE model with unknown parameters. We present a filtering algorithm where the reconstruction of the dynamics is done with neural network approximations whose weights are dynamically updated using observational data. In addition to the estimate of the state, we also obtain time-dependent parameter estimations of the PDE parameters governing the observed evolution. We illustrate the behavior of the method in a one-dimensional KdV equation involving the transport of solutions with local support. Our numerical investigation reveals the importance of the location and number of the observations. In particular, it suggests to consider dynamical sensor placement.
△ Less
Submitted 31 January, 2024;
originally announced January 2024.
-
Depth-Adaptive Neural Networks from the Optimal Control viewpoint
Authors:
Joubine Aghili,
Olga Mula
Abstract:
In recent years, deep learning has been connected with optimal control as a way to define a notion of a continuous underlying learning problem. In this view, neural networks can be interpreted as a discretization of a parametric Ordinary Differential Equation which, in the limit, defines a continuous-depth neural network. The learning task then consists in finding the best ODE parameters for the p…
▽ More
In recent years, deep learning has been connected with optimal control as a way to define a notion of a continuous underlying learning problem. In this view, neural networks can be interpreted as a discretization of a parametric Ordinary Differential Equation which, in the limit, defines a continuous-depth neural network. The learning task then consists in finding the best ODE parameters for the problem under consideration, and their number increases with the accuracy of the time discretization. Although important steps have been taken to realize the advantages of such continuous formulations, most current learning techniques fix a discretization (i.e. the number of layers is fixed). In this work, we propose an iterative adaptive algorithm where we progressively refine the time discretization (i.e. we increase the number of layers). Provided that certain tolerances are met across the iterations, we prove that the strategy converges to the underlying continuous problem. One salient advantage of such a shallow-to-deep approach is that it helps to benefit in practice from the higher approximation properties of deep networks by mitigating over-parametrization issues. The performance of the approach is illustrated in several numerical examples.
△ Less
Submitted 5 July, 2020;
originally announced July 2020.
-
An advection-robust Hybrid High-Order method for the Oseen problem
Authors:
Joubine Aghili,
Daniele A. Di Pietro
Abstract:
In this work, we study advection-robust Hybrid High-Order discretizations of the Oseen equations. For a given integer $k\ge 0$, the discrete velocity unknowns are vector-valued polynomials of total degree $\le k$ on mesh elements and faces, while the pressure unknowns are discontinuous polynomials of total degree $\le k$ on the mesh. From the discrete unknowns, three relevant quantities are recons…
▽ More
In this work, we study advection-robust Hybrid High-Order discretizations of the Oseen equations. For a given integer $k\ge 0$, the discrete velocity unknowns are vector-valued polynomials of total degree $\le k$ on mesh elements and faces, while the pressure unknowns are discontinuous polynomials of total degree $\le k$ on the mesh. From the discrete unknowns, three relevant quantities are reconstructed inside each element: a velocity of total degree $\le(k+1)$, a discrete advective derivative, and a discrete divergence. These reconstructions are used to formulate the discretizations of the viscous, advective, and velocity-pressure coupling terms, respectively. Well-posedness is ensured through appropriate high-order stabilization terms. We prove energy error estimates that are advection-robust for the velocity, and show that each mesh element $T$ of diameter $h_T$ contributes to the discretization error with an $\mathcal{O}(h_T^{k+1})$-term in the diffusion-dominated regime, an $\mathcal{O}(h_T^{k+\frac12})$-term in the advection-dominated regime, and scales with intermediate powers of $h_T$ in between. Numerical results complete the exposition.
△ Less
Submitted 18 February, 2018; v1 submitted 7 December, 2017;
originally announced December 2017.
