-
Neuro-symbolic partial differential equation solver
Authors:
Pouria Mistani,
Samira Pakravan,
Rajesh Ilango,
Sanjay Choudhry,
Frederic Gibou
Abstract:
We present a highly scalable strategy for develo** mesh-free neuro-symbolic partial differential equation solvers from existing numerical discretizations found in scientific computing. This strategy is unique in that it can be used to efficiently train neural network surrogate models for the solution functions and the differential operators, while retaining the accuracy and convergence propertie…
▽ More
We present a highly scalable strategy for develo** mesh-free neuro-symbolic partial differential equation solvers from existing numerical discretizations found in scientific computing. This strategy is unique in that it can be used to efficiently train neural network surrogate models for the solution functions and the differential operators, while retaining the accuracy and convergence properties of state-of-the-art numerical solvers. This neural bootstrap** method is based on minimizing residuals of discretized differential systems on a set of random collocation points with respect to the trainable parameters of the neural network, achieving unprecedented resolution and optimal scaling for solving physical and biological systems.
△ Less
Submitted 25 October, 2022;
originally announced October 2022.
-
JAX-DIPS: Neural bootstrap** of finite discretization methods and application to elliptic problems with discontinuities
Authors:
Pouria Mistani,
Samira Pakravan,
Rajesh Ilango,
Frederic Gibou
Abstract:
We present a scalable strategy for development of mesh-free hybrid neuro-symbolic partial differential equation solvers based on existing mesh-based numerical discretization methods. Particularly, this strategy can be used to efficiently train neural network surrogate models of partial differential equations by (i) leveraging the accuracy and convergence properties of advanced numerical methods, s…
▽ More
We present a scalable strategy for development of mesh-free hybrid neuro-symbolic partial differential equation solvers based on existing mesh-based numerical discretization methods. Particularly, this strategy can be used to efficiently train neural network surrogate models of partial differential equations by (i) leveraging the accuracy and convergence properties of advanced numerical methods, solvers, and preconditioners, as well as (ii) better scalability to higher order PDEs by strictly limiting optimization to first order automatic differentiation. The presented neural bootstrap** method (hereby dubbed NBM) is based on evaluation of the finite discretization residuals of the PDE system obtained on implicit Cartesian cells centered on a set of random collocation points with respect to trainable parameters of the neural network. Importantly, the conservation laws and symmetries present in the bootstrapped finite discretization equations inform the neural network about solution regularities within local neighborhoods of training points. We apply NBM to the important class of elliptic problems with jump conditions across irregular interfaces in three spatial dimensions. We show the method is convergent such that model accuracy improves by increasing number of collocation points in the domain and predonditioning the residuals. We show NBM is competitive in terms of memory and training speed with other PINN-type frameworks. The algorithms presented here are implemented using \texttt{JAX} in a software package named \texttt{JAX-DIPS} (https://github.com/JAX-DIPS/JAX-DIPS), standing for differentiable interfacial PDE solver. We open sourced \texttt{JAX-DIPS} to facilitate research into use of differentiable algorithms for develo** hybrid PDE solvers.
△ Less
Submitted 4 September, 2023; v1 submitted 25 October, 2022;
originally announced October 2022.
-
Machine learning algorithms for three-dimensional mean-curvature computation in the level-set method
Authors:
Luis Ángel Larios-Cárdenas,
Frédéric Gibou
Abstract:
We propose a data-driven mean-curvature solver for the level-set method. This work is the natural extension to $\mathbb{R}^3$ of our two-dimensional strategy in [DOI: 10.1007/s10915-022-01952-2][1] and the hybrid inference system of [DOI: 10.1016/j.jcp.2022.111291][2]. However, in contrast to [1,2], which built resolution-dependent neural-network dictionaries, here we develop a pair of models in…
▽ More
We propose a data-driven mean-curvature solver for the level-set method. This work is the natural extension to $\mathbb{R}^3$ of our two-dimensional strategy in [DOI: 10.1007/s10915-022-01952-2][1] and the hybrid inference system of [DOI: 10.1016/j.jcp.2022.111291][2]. However, in contrast to [1,2], which built resolution-dependent neural-network dictionaries, here we develop a pair of models in $\mathbb{R}^3$, regardless of the mesh size. Our feedforward networks ingest transformed level-set, gradient, and curvature data to fix numerical mean-curvature approximations selectively for interface nodes. To reduce the problem's complexity, we have used the Gaussian curvature to classify stencils and fit our models separately to non-saddle and saddle patterns. Non-saddle stencils are easier to handle because they exhibit a curvature error distribution characterized by monotonicity and symmetry. While the latter has allowed us to train only on half the mean-curvature spectrum, the former has helped us blend the data-driven and the baseline estimations seamlessly near flat regions. On the other hand, the saddle-pattern error structure is less clear; thus, we have exploited no latent information beyond what is known. In this regard, we have trained our models on not only spherical but also sinusoidal and hyperbolic paraboloidal patches. Our approach to building their data sets is systematic but gleans samples randomly while ensuring well-balancedness. We have also resorted to standardization and dimensionality reduction and integrated regularization to minimize outliers. In addition, we leverage curvature rotation/reflection invariance to improve precision at inference time. Several experiments confirm that our proposed system can yield more accurate mean-curvature estimations than modern particle-based interface reconstruction and level-set schemes around under-resolved regions.
△ Less
Submitted 9 December, 2022; v1 submitted 18 August, 2022;
originally announced August 2022.
-
Error-Correcting Neural Networks for Two-Dimensional Curvature Computation in the Level-Set Method
Authors:
Luis Ángel Larios-Cárdenas,
Frédéric Gibou
Abstract:
We present an error-neural-modeling-based strategy for approximating two-dimensional curvature in the level-set method. Our main contribution is a redesigned hybrid solver [Larios-Cárdenas and Gibou, J. Comput. Phys. (May 2022), 10.1016/j.jcp.2022.111291] that relies on numerical schemes to enable machine-learning operations on demand. In particular, our routine features double predicting to harne…
▽ More
We present an error-neural-modeling-based strategy for approximating two-dimensional curvature in the level-set method. Our main contribution is a redesigned hybrid solver [Larios-Cárdenas and Gibou, J. Comput. Phys. (May 2022), 10.1016/j.jcp.2022.111291] that relies on numerical schemes to enable machine-learning operations on demand. In particular, our routine features double predicting to harness curvature symmetry invariance in favor of precision and stability. The core of this solver is a multilayer perceptron trained on circular- and sinusoidal-interface samples. Its role is to quantify the error in numerical curvature approximations and emit corrected estimates for select grid vertices along the free boundary. These corrections arise in response to preprocessed context level-set, curvature, and gradient data. To promote neural capacity, we have adopted sample negative-curvature normalization, reorientation, and reflection-based augmentation. In the same manner, our system incorporates dimensionality reduction, well-balancedness, and regularization to minimize outlying effects. Our training approach is likewise scalable across mesh sizes. For this purpose, we have introduced dimensionless parametrization and probabilistic subsampling during data production. Together, all these elements have improved the accuracy and efficiency of curvature calculations around under-resolved regions. In most experiments, our strategy has outperformed the numerical baseline at twice the number of redistancing steps while requiring only a fraction of the cost.
△ Less
Submitted 28 September, 2022; v1 submitted 22 January, 2022;
originally announced January 2022.
-
Error-correcting neural networks for semi-Lagrangian advection in the level-set method
Authors:
Luis Ángel Larios-Cárdenas,
Frédéric Gibou
Abstract:
We present a machine learning framework that blends image super-resolution technologies with passive, scalar transport in the level-set method. Here, we investigate whether we can compute on-the-fly, data-driven corrections to minimize numerical viscosity in the coarse-mesh evolution of an interface. The proposed system's starting point is the semi-Lagrangian formulation. And, to reduce numerical…
▽ More
We present a machine learning framework that blends image super-resolution technologies with passive, scalar transport in the level-set method. Here, we investigate whether we can compute on-the-fly, data-driven corrections to minimize numerical viscosity in the coarse-mesh evolution of an interface. The proposed system's starting point is the semi-Lagrangian formulation. And, to reduce numerical dissipation, we introduce an error-quantifying multilayer perceptron. The role of this neural network is to improve the numerically estimated surface trajectory. To do so, it processes localized level-set, velocity, and positional data in a single time frame for select vertices near the moving front. Our main contribution is thus a novel machine-learning-augmented transport algorithm that operates alongside selective redistancing and alternates with conventional advection to keep the adjusted interface trajectory smooth. Consequently, our procedure is more efficient than full-scan convolutional-based applications because it concentrates computational effort only around the free boundary. Also, we show through various tests that our strategy is effective at counteracting both numerical diffusion and mass loss. In simple advection problems, for example, our method can achieve the same precision as the baseline scheme at twice the resolution but at a fraction of the cost. Similarly, our hybrid technique can produce feasible solidification fronts for crystallization processes. On the other hand, tangential shear flows and highly deforming simulations can precipitate bias artifacts and inference deterioration. Likewise, stringent design velocity constraints can limit our solver's application to problems involving rapid interface changes. In the latter cases, we have identified several opportunities to enhance robustness without forgoing our approach's basic concept.
△ Less
Submitted 28 September, 2022; v1 submitted 22 October, 2021;
originally announced October 2021.
-
A hybrid inference system for improved curvature estimation in the level-set method using machine learning
Authors:
Luis Ángel Larios-Cárdenas,
Frédéric Gibou
Abstract:
We present a novel hybrid strategy based on machine learning to improve curvature estimation in the level-set method. The proposed inference system couples enhanced neural networks with standard numerical schemes to compute curvature more accurately. The core of our hybrid framework is a switching mechanism that relies on well established numerical techniques to gauge curvature. If the curvature m…
▽ More
We present a novel hybrid strategy based on machine learning to improve curvature estimation in the level-set method. The proposed inference system couples enhanced neural networks with standard numerical schemes to compute curvature more accurately. The core of our hybrid framework is a switching mechanism that relies on well established numerical techniques to gauge curvature. If the curvature magnitude is larger than a resolution-dependent threshold, it uses a neural network to yield a better approximation. Our networks are multilayer perceptrons fitted to synthetic data sets composed of sinusoidal- and circular-interface samples at various configurations. To reduce data set size and training complexity, we leverage the problem's characteristic symmetry and build our models on just half of the curvature spectrum. These savings lead to a powerful inference system able to outperform any of its numerical or neural component alone. Experiments with stationary, smooth interfaces show that our hybrid solver is notably superior to conventional numerical methods in coarse grids and along steep interface regions. Compared to prior research, we have observed outstanding gains in precision after training the regression model with data pairs from more than a single interface type and transforming data with specialized input preprocessing. In particular, our findings confirm that machine learning is a promising venue for reducing or removing mass loss in the level-set method.
△ Less
Submitted 28 September, 2022; v1 submitted 7 April, 2021;
originally announced April 2021.
-
A fractional stochastic theory for interfacial polarization of cell aggregates
Authors:
Pouria A. Mistani,
Samira Pakravan,
Frederic G. Gibou
Abstract:
We present a theoretical framework to model the electric response of cell aggregates. We establish a coarse representation for each cell as a combination of membrane and cytoplasm dipole moments. Then we compute the effective conductivity of the resulting system, and thereafter derive a Fokker-Planck partial differential equation that captures the time-dependent evolution of the distribution of in…
▽ More
We present a theoretical framework to model the electric response of cell aggregates. We establish a coarse representation for each cell as a combination of membrane and cytoplasm dipole moments. Then we compute the effective conductivity of the resulting system, and thereafter derive a Fokker-Planck partial differential equation that captures the time-dependent evolution of the distribution of induced cellular polarizations in an ensemble of cells. Our model predicts that the polarization density parallel to an applied pulse follows a skewed t-distribution, while the transverse polarization density follows a symmetric t-distribution, which are in accordance with our direct numerical simulations. Furthermore, we report a reduced order model described by a coupled pair of ordinary differential equations that reproduces the average and the variance of induced dipole moments in the aggregate. We extend our proposed formulation by considering fractional order time derivatives that we find necessary to explain anomalous relaxation phenomena observed in experiments as well as our direct numerical simulations. Owing to its time-domain formulation, our framework can be easily used to consider nonlinear membrane effects or intercellular couplings that arise in several scientific, medical and technological applications.
△ Less
Submitted 25 August, 2020;
originally announced August 2020.
-
A deep learning approach for the computation of curvature in the level-set method
Authors:
Luis Ángel Larios-Cárdenas,
Frederic Gibou
Abstract:
We propose a deep learning strategy to estimate the mean curvature of two-dimensional implicit interfaces in the level-set method. Our approach is based on fitting feed-forward neural networks to synthetic data sets constructed from circular interfaces immersed in uniform grids of various resolutions. These multilayer perceptrons process the level-set values from mesh points next to the free bound…
▽ More
We propose a deep learning strategy to estimate the mean curvature of two-dimensional implicit interfaces in the level-set method. Our approach is based on fitting feed-forward neural networks to synthetic data sets constructed from circular interfaces immersed in uniform grids of various resolutions. These multilayer perceptrons process the level-set values from mesh points next to the free boundary and output the dimensionless curvature at their closest locations on the interface. Accuracy analyses involving irregular interfaces, in both uniform and adaptive grids, show that our models are competitive with traditional numerical schemes in the $L^1$ and $L^2$ norms. In particular, our neural networks approximate curvature with comparable precision in coarse resolutions, when the interface features steep curvature regions, and when the number of iterations to reinitialize the level-set function is small. Although the conventional numerical approach is more robust than our framework, our results have unveiled the potential of machine learning for dealing with computational tasks where the level-set method is known to experience difficulties. We also establish that an application-dependent map of local resolutions to neural models can be devised to estimate mean curvature more effectively than a universal neural network.
△ Less
Submitted 28 September, 2022; v1 submitted 3 February, 2020;
originally announced February 2020.
-
Solving inverse-PDE problems with physics-aware neural networks
Authors:
Samira Pakravan,
Pouria A. Mistani,
Miguel Angel Aragon-Calvo,
Frederic Gibou
Abstract:
We propose a novel composite framework to find unknown fields in the context of inverse problems for partial differential equations (PDEs). We blend the high expressibility of deep neural networks as universal function estimators with the accuracy and reliability of existing numerical algorithms for partial differential equations as custom layers in semantic autoencoders. Our design brings togethe…
▽ More
We propose a novel composite framework to find unknown fields in the context of inverse problems for partial differential equations (PDEs). We blend the high expressibility of deep neural networks as universal function estimators with the accuracy and reliability of existing numerical algorithms for partial differential equations as custom layers in semantic autoencoders. Our design brings together techniques of computational mathematics, machine learning and pattern recognition under one umbrella to incorporate domain-specific knowledge and physical constraints to discover the underlying hidden fields. The network is explicitly aware of the governing physics through a hard-coded PDE solver layer in contrast to most existing methods that incorporate the governing equations in the loss function or rely on trainable convolutional layers to discover proper discretizations from data. This subsequently focuses the computational load to only the discovery of the hidden fields and therefore is more data efficient. We call this architecture Blended inverse-PDE networks (hereby dubbed BiPDE networks) and demonstrate its applicability for recovering the variable diffusion coefficient in Poisson problems in one and two spatial dimensions, as well as the diffusion coefficient in the time-dependent and nonlinear Burgers' equation in one dimension. We also show that this approach is robust to noise.
△ Less
Submitted 18 November, 2020; v1 submitted 10 January, 2020;
originally announced January 2020.
