Differentiable modeling to unify machine learning and physical models and advance Geosciences
Authors:
Chaopeng Shen,
Alison P. Appling,
Pierre Gentine,
Toshiyuki Bandai,
Hoshin Gupta,
Alexandre Tartakovsky,
Marco Baity-Jesi,
Fabrizio Fenicia,
Daniel Kifer,
Li Li,
Xiaofeng Liu,
Wei Ren,
Yi Zheng,
Ciaran J. Harman,
Martyn Clark,
Matthew Farthing,
Dapeng Feng,
Praveen Kumar,
Doaa Aboelyazeed,
Farshid Rahmani,
Hylke E. Beck,
Tadd Bindas,
Dipankar Dwivedi,
Kuai Fang,
Marvin Höge
, et al. (5 additional authors not shown)
Abstract:
Process-Based Modeling (PBM) and Machine Learning (ML) are often perceived as distinct paradigms in the geosciences. Here we present differentiable geoscientific modeling as a powerful pathway toward dissolving the perceived barrier between them and ushering in a paradigm shift. For decades, PBM offered benefits in interpretability and physical consistency but struggled to efficiently leverage lar…
▽ More
Process-Based Modeling (PBM) and Machine Learning (ML) are often perceived as distinct paradigms in the geosciences. Here we present differentiable geoscientific modeling as a powerful pathway toward dissolving the perceived barrier between them and ushering in a paradigm shift. For decades, PBM offered benefits in interpretability and physical consistency but struggled to efficiently leverage large datasets. ML methods, especially deep networks, presented strong predictive skills yet lacked the ability to answer specific scientific questions. While various methods have been proposed for ML-physics integration, an important underlying theme -- differentiable modeling -- is not sufficiently recognized. Here we outline the concepts, applicability, and significance of differentiable geoscientific modeling (DG). "Differentiable" refers to accurately and efficiently calculating gradients with respect to model variables, critically enabling the learning of high-dimensional unknown relationships. DG refers to a range of methods connecting varying amounts of prior knowledge to neural networks and training them together, capturing a different scope than physics-guided machine learning and emphasizing first principles. Preliminary evidence suggests DG offers better interpretability and causality than ML, improved generalizability and extrapolation capability, and strong potential for knowledge discovery, while approaching the performance of purely data-driven ML. DG models require less training data while scaling favorably in performance and efficiency with increasing amounts of data. With DG, geoscientists may be better able to frame and investigate questions, test hypotheses, and discover unrecognized linkages.
△ Less
Submitted 26 December, 2023; v1 submitted 10 January, 2023;
originally announced January 2023.
