Differentiable modeling to unify machine learning and physical models and advance Geosciences
Authors:
Chaopeng Shen,
Alison P. Appling,
Pierre Gentine,
Toshiyuki Bandai,
Hoshin Gupta,
Alexandre Tartakovsky,
Marco Baity-Jesi,
Fabrizio Fenicia,
Daniel Kifer,
Li Li,
Xiaofeng Liu,
Wei Ren,
Yi Zheng,
Ciaran J. Harman,
Martyn Clark,
Matthew Farthing,
Dapeng Feng,
Praveen Kumar,
Doaa Aboelyazeed,
Farshid Rahmani,
Hylke E. Beck,
Tadd Bindas,
Dipankar Dwivedi,
Kuai Fang,
Marvin Höge
, et al. (5 additional authors not shown)
Abstract:
Process-Based Modeling (PBM) and Machine Learning (ML) are often perceived as distinct paradigms in the geosciences. Here we present differentiable geoscientific modeling as a powerful pathway toward dissolving the perceived barrier between them and ushering in a paradigm shift. For decades, PBM offered benefits in interpretability and physical consistency but struggled to efficiently leverage lar…
▽ More
Process-Based Modeling (PBM) and Machine Learning (ML) are often perceived as distinct paradigms in the geosciences. Here we present differentiable geoscientific modeling as a powerful pathway toward dissolving the perceived barrier between them and ushering in a paradigm shift. For decades, PBM offered benefits in interpretability and physical consistency but struggled to efficiently leverage large datasets. ML methods, especially deep networks, presented strong predictive skills yet lacked the ability to answer specific scientific questions. While various methods have been proposed for ML-physics integration, an important underlying theme -- differentiable modeling -- is not sufficiently recognized. Here we outline the concepts, applicability, and significance of differentiable geoscientific modeling (DG). "Differentiable" refers to accurately and efficiently calculating gradients with respect to model variables, critically enabling the learning of high-dimensional unknown relationships. DG refers to a range of methods connecting varying amounts of prior knowledge to neural networks and training them together, capturing a different scope than physics-guided machine learning and emphasizing first principles. Preliminary evidence suggests DG offers better interpretability and causality than ML, improved generalizability and extrapolation capability, and strong potential for knowledge discovery, while approaching the performance of purely data-driven ML. DG models require less training data while scaling favorably in performance and efficiency with increasing amounts of data. With DG, geoscientists may be better able to frame and investigate questions, test hypotheses, and discover unrecognized linkages.
△ Less
Submitted 26 December, 2023; v1 submitted 10 January, 2023;
originally announced January 2023.
Forward and inverse modeling of water flow in unsaturated soils with discontinuous hydraulic conductivities using physics-informed neural networks with domain decomposition
Authors:
Toshiyuki Bandai,
Teamrat A. Ghezzehei
Abstract:
Modeling water flow in unsaturated soils is vital for describing various hydrological and ecological phenomena. Soil water dynamics is described by well-established physical laws (Richardson-Richards equation (RRE)). Solving the RRE is difficult due to the inherent non-linearity of the processes, and various numerical methods have been proposed to solve the issue. However, applying the methods to…
▽ More
Modeling water flow in unsaturated soils is vital for describing various hydrological and ecological phenomena. Soil water dynamics is described by well-established physical laws (Richardson-Richards equation (RRE)). Solving the RRE is difficult due to the inherent non-linearity of the processes, and various numerical methods have been proposed to solve the issue. However, applying the methods to practical situations is challenging because they require well-defined initial and boundary conditions. Here, we present a physics-informed neural networks (PINNs) method that approximates the solution to the RRE using neural networks while concurrently matching available soil moisture data. Although the ability of PINNs to solve partial differential equations, including the RRE, has been demonstrated previously, its potential applications and limitations are not fully known. This study conducted a comprehensive analysis of PINNs and carefully tested the accuracy of the solutions by comparing them with analytical solutions and accepted traditional numerical solutions. We demonstrated that the solutions by PINNs with adaptive activation functions are comparable with those by traditional methods. We showed that soil moisture dynamics in layered soils with discontinuous hydraulic conductivities are correctly simulated by PINNs with domain decomposition. We demonstrated that the unspecified upper boundary condition can be estimated from sparse soil moisture measurements. Nevertheless, there remain challenges that require further development. Chiefly, PINNs are sensitive to the initialization of NNs and are significantly slower than traditional numerical methods.
△ Less
Submitted 22 February, 2022;
originally announced February 2022.
