-
Surrogate-data-enriched Physics-Aware Neural Networks
Authors:
Raphael Leiteritz,
Patrick Buchfink,
Bernard Haasdonk,
Dirk Pflüger
Abstract:
Neural networks can be used as surrogates for PDE models. They can be made physics-aware by penalizing underlying equations or the conservation of physical properties in the loss function during training. Current approaches allow to additionally respect data from numerical simulations or experiments in the training process. However, this data is frequently expensive to obtain and thus only scarcel…
▽ More
Neural networks can be used as surrogates for PDE models. They can be made physics-aware by penalizing underlying equations or the conservation of physical properties in the loss function during training. Current approaches allow to additionally respect data from numerical simulations or experiments in the training process. However, this data is frequently expensive to obtain and thus only scarcely available for complex models. In this work, we investigate how physics-aware models can be enriched with computationally cheaper, but inexact, data from other surrogate models like Reduced-Order Models (ROMs). In order to avoid trusting too-low-fidelity surrogate solutions, we develop an approach that is sensitive to the error in inexact data. As a proof of concept, we consider the one-dimensional wave equation and show that the training accuracy is increased by two orders of magnitude when inexact data from ROMs is incorporated.
△ Less
Submitted 15 December, 2021; v1 submitted 10 December, 2021;
originally announced December 2021.
-
Universality and Optimality of Structured Deep Kernel Networks
Authors:
Tizian Wenzel,
Gabriele Santin,
Bernard Haasdonk
Abstract:
Kernel based methods yield approximation models that are flexible, efficient and powerful. In particular, they utilize fixed feature maps of the data, being often associated to strong analytical results that prove their accuracy. On the other hand, the recent success of machine learning methods has been driven by deep neural networks (NNs). They achieve a significant accuracy on very high-dimensio…
▽ More
Kernel based methods yield approximation models that are flexible, efficient and powerful. In particular, they utilize fixed feature maps of the data, being often associated to strong analytical results that prove their accuracy. On the other hand, the recent success of machine learning methods has been driven by deep neural networks (NNs). They achieve a significant accuracy on very high-dimensional data, in that they are able to learn also efficient data representations or data-based feature maps. In this paper, we leverage a recent deep kernel representer theorem to connect the two approaches and understand their interplay. In particular, we show that the use of special types of kernels yield models reminiscent of neural networks that are founded in the same theoretical framework of classical kernel methods, while enjoying many computational properties of deep neural networks. Especially the introduced Structured Deep Kernel Networks (SDKNs) can be viewed as neural networks with optimizable activation functions obeying a representer theorem. Analytic properties show their universal approximation properties in different asymptotic regimes of unbounded number of centers, width and depth. Especially in the case of unbounded depth, the constructions is asymptotically better than corresponding constructions for ReLU neural networks, which is made possible by the flexibility of kernel approximation
△ Less
Submitted 15 May, 2021;
originally announced May 2021.
-
Structured Deep Kernel Networks for Data-Driven Closure Terms of Turbulent Flows
Authors:
Tizian Wenzel,
Marius Kurz,
Andrea Beck,
Gabriele Santin,
Bernard Haasdonk
Abstract:
Standard kernel methods for machine learning usually struggle when dealing with large datasets. We review a recently introduced Structured Deep Kernel Network (SDKN) approach that is capable of dealing with high-dimensional and huge datasets - and enjoys typical standard machine learning approximation properties. We extend the SDKN to combine it with standard machine learning modules and compare i…
▽ More
Standard kernel methods for machine learning usually struggle when dealing with large datasets. We review a recently introduced Structured Deep Kernel Network (SDKN) approach that is capable of dealing with high-dimensional and huge datasets - and enjoys typical standard machine learning approximation properties. We extend the SDKN to combine it with standard machine learning modules and compare it with Neural Networks on the scientific challenge of data-driven prediction of closure terms of turbulent flows. We show experimentally that the SDKNs are capable of dealing with large datasets and achieve near-perfect accuracy on the given application.
△ Less
Submitted 25 March, 2021;
originally announced March 2021.
-
Kernel methods for center manifold approximation and a data-based version of the Center Manifold Theorem
Authors:
Bernard Haasdonk,
Boumediene Hamzi,
Gabriele Santin,
Dominik Wittwar
Abstract:
For dynamical systems with a non hyperbolic equilibrium, it is possible to significantly simplify the study of stability by means of the center manifold theory. This theory allows to isolate the complicated asymptotic behavior of the system close to the equilibrium point and to obtain meaningful predictions of its behavior by analyzing a reduced order system on the so-called center manifold.
Sin…
▽ More
For dynamical systems with a non hyperbolic equilibrium, it is possible to significantly simplify the study of stability by means of the center manifold theory. This theory allows to isolate the complicated asymptotic behavior of the system close to the equilibrium point and to obtain meaningful predictions of its behavior by analyzing a reduced order system on the so-called center manifold.
Since the center manifold is usually not known, good approximation methods are important as the center manifold theorem states that the stability properties of the origin of the reduced order system are the same as those of the origin of the full order system.
In this work, we establish a data-based version of the center manifold theorem that works by considering an approximation in place of an exact manifold. Also the error between the approximated and the original reduced dynamics are quantified.
We then use an apposite data-based kernel method to construct a suitable approximation of the manifold close to the equilibrium, which is compatible with our general error theory. The data are collected by repeated numerical simulation of the full system by means of a high-accuracy solver, which generates sets of discrete trajectories that are then used as a training set. The method is tested on different examples which show promising performance and good accuracy.
△ Less
Submitted 1 December, 2020;
originally announced December 2020.
-
Biomechanical surrogate modelling using stabilized vectorial greedy kernel methods
Authors:
Bernard Haasdonk,
Tizian Wenzel,
Gabriele Santin,
Syn Schmitt
Abstract:
Greedy kernel approximation algorithms are successful techniques for sparse and accurate data-based modelling and function approximation. Based on a recent idea of stabilization of such algorithms in the scalar output case, we here consider the vectorial extension built on VKOGA. We introduce the so called $γ$-restricted VKOGA, comment on analytical properties and present numerical evaluation on d…
▽ More
Greedy kernel approximation algorithms are successful techniques for sparse and accurate data-based modelling and function approximation. Based on a recent idea of stabilization of such algorithms in the scalar output case, we here consider the vectorial extension built on VKOGA. We introduce the so called $γ$-restricted VKOGA, comment on analytical properties and present numerical evaluation on data from a clinically relevant application, the modelling of the human spine. The experiments show that the new stabilized algorithms result in improved accuracy and stability over the non-stabilized algorithms.
△ Less
Submitted 28 April, 2020; v1 submitted 27 April, 2020;
originally announced April 2020.
-
Deep recurrent Gaussian process with variational Sparse Spectrum approximation
Authors:
Roman Föll,
Bernard Haasdonk,
Markus Hanselmann,
Holger Ulmer
Abstract:
Modeling sequential data has become more and more important in practice. Some applications are autonomous driving, virtual sensors and weather forecasting. To model such systems, so called recurrent models are frequently used. In this paper we introduce several new Deep recurrent Gaussian process (DRGP) models based on the Sparse Spectrum Gaussian process (SSGP) and the improved version, called va…
▽ More
Modeling sequential data has become more and more important in practice. Some applications are autonomous driving, virtual sensors and weather forecasting. To model such systems, so called recurrent models are frequently used. In this paper we introduce several new Deep recurrent Gaussian process (DRGP) models based on the Sparse Spectrum Gaussian process (SSGP) and the improved version, called variational Sparse Spectrum Gaussian process (VSSGP). We follow the recurrent structure given by an existing DRGP based on a specific variational sparse Nyström approximation, the recurrent Gaussian process (RGP). Similar to previous work, we also variationally integrate out the input-space and hence can propagate uncertainty through the Gaussian process (GP) layers. Our approach can deal with a larger class of covariance functions than the RGP, because its spectral nature allows variational integration in all stationary cases. Furthermore, we combine the (variational) Sparse Spectrum ((V)SS) approximations with a well known inducing-input regularization framework. We improve over current state of the art methods in prediction accuracy for experimental data-sets used for their evaluation and introduce a new data-set for engine control, named Emission.
△ Less
Submitted 27 September, 2019;
originally announced September 2019.
-
Enabling Interactive Mobile Simulations Through Distributed Reduced Models
Authors:
Christoph Dibak,
Bernard Haasdonk,
Andreas Schmidt,
Frank Dürr,
Kurt Rothermel
Abstract:
Currently, various hardware and software companies are develo** augmented reality devices, most prominently Microsoft with its Hololens. Besides gaming, such devices can be used for serious pervasive applications, like interactive mobile simulations to support engineers in the field. Interactive simulations have high demands on resources, which the mobile device alone is unable to satisfy. There…
▽ More
Currently, various hardware and software companies are develo** augmented reality devices, most prominently Microsoft with its Hololens. Besides gaming, such devices can be used for serious pervasive applications, like interactive mobile simulations to support engineers in the field. Interactive simulations have high demands on resources, which the mobile device alone is unable to satisfy. Therefore, we propose a framework to support mobile simulations by distributing the computation between the mobile device and a remote server based on the reduced basis method. Evaluations show that we can speed-up the numerical computation by over 131 times while using 73 times less energy.
△ Less
Submitted 14 February, 2018;
originally announced February 2018.
-
Comparison of data-driven uncertainty quantification methods for a carbon dioxide storage benchmark scenario
Authors:
Markus Köppel,
Fabian Franzelin,
Ilja Kröker,
Sergey Oladyshkin,
Gabriele Santin,
Dominik Wittwar,
Andrea Barth,
Bernard Haasdonk,
Wolfgang Nowak,
Dirk Pflüger,
Christian Rohde
Abstract:
A variety of methods is available to quantify uncertainties arising with\-in the modeling of flow and transport in carbon dioxide storage, but there is a lack of thorough comparisons. Usually, raw data from such storage sites can hardly be described by theoretical statistical distributions since only very limited data is available. Hence, exact information on distribution shapes for all uncertain…
▽ More
A variety of methods is available to quantify uncertainties arising with\-in the modeling of flow and transport in carbon dioxide storage, but there is a lack of thorough comparisons. Usually, raw data from such storage sites can hardly be described by theoretical statistical distributions since only very limited data is available. Hence, exact information on distribution shapes for all uncertain parameters is very rare in realistic applications. We discuss and compare four different methods tested for data-driven uncertainty quantification based on a benchmark scenario of carbon dioxide storage. In the benchmark, for which we provide data and code, carbon dioxide is injected into a saline aquifer modeled by the nonlinear capillarity-free fractional flow formulation for two incompressible fluid phases, namely carbon dioxide and brine. To cover different aspects of uncertainty quantification, we incorporate various sources of uncertainty such as uncertainty of boundary conditions, of conceptual model definitions and of material properties. We consider recent versions of the following non-intrusive and intrusive uncertainty quantification methods: arbitary polynomial chaos, spatially adaptive sparse grids, kernel-based greedy interpolation and hybrid stochastic Galerkin. The performance of each approach is demonstrated assessing expectation value and standard deviation of the carbon dioxide saturation against a reference statistic based on Monte Carlo sampling. We compare the convergence of all methods reporting on accuracy with respect to the number of model runs and resolution. Finally we offer suggestions about the methods' advantages and disadvantages that can guide the modeler for uncertainty quantification in carbon dioxide storage and beyond.
△ Less
Submitted 8 February, 2018;
originally announced February 2018.
-
An algorithmic comparison of the Hyper-Reduction and the Discrete Empirical Interpolation Method for a nonlinear thermal problem
Authors:
Felix Fritzen,
Bernhard Haasdonk,
David Ryckelynck,
Sebastian Schöps
Abstract:
A novel algorithmic discussion of the methodological and numerical differences of competing parametric model reduction techniques for nonlinear problems are presented. First, the Galerkin reduced basis (RB) formulation is presented which fails at providing significant gains with respect to the computational efficiency for nonlinear problems. Renown methods for the reduction of the computing time o…
▽ More
A novel algorithmic discussion of the methodological and numerical differences of competing parametric model reduction techniques for nonlinear problems are presented. First, the Galerkin reduced basis (RB) formulation is presented which fails at providing significant gains with respect to the computational efficiency for nonlinear problems. Renown methods for the reduction of the computing time of nonlinear reduced order models are the Hyper-Reduction and the (Discrete) Empirical Interpolation Method (EIM, DEIM). An algorithmic description and a methodological comparison of both methods are provided. The accuracy of the predictions of the hyper-reduced model and the (D)EIM in comparison to the Galerkin RB is investigated. All three approaches are applied to a simple uncertainty quantification of a planar nonlinear thermal conduction problem. The results are compared to computationally intense finite element simulations.
△ Less
Submitted 19 December, 2017; v1 submitted 17 October, 2016;
originally announced October 2016.
