-
Conditional Korhunen-Loéve regression model with Basis Adaptation for high-dimensional problems: uncertainty quantification and inverse modeling
Authors:
Yu-Hong Yeung,
Ramakrishna Tipireddy,
David A. Barajas-Solano,
Alexandre M. Tartakovsky
Abstract:
We propose a methodology for improving the accuracy of surrogate models of the observable response of physical systems as a function of the systems' spatially heterogeneous parameter fields with applications to uncertainty quantification and parameter estimation in high-dimensional problems. Practitioners often formulate finite-dimensional representations of spatially heterogeneous parameter field…
▽ More
We propose a methodology for improving the accuracy of surrogate models of the observable response of physical systems as a function of the systems' spatially heterogeneous parameter fields with applications to uncertainty quantification and parameter estimation in high-dimensional problems. Practitioners often formulate finite-dimensional representations of spatially heterogeneous parameter fields using truncated unconditional Karhunen-Loéve expansions (KLEs) for a certain choice of unconditional covariance kernel and construct surrogate models of the observable response with respect to the random variables in the KLE. When direct measurements of the parameter fields are available, we propose improving the accuracy of these surrogate models by representing the parameter fields via conditional Karhunen-Loéve expansions (CKLEs). CKLEs are constructed by conditioning the covariance kernel of the unconditional expansion on the direct measurements via Gaussian process regression and then truncating the corresponding KLE. We apply the proposed methodology to constructing surrogate models via the Basis Adaptation (BA) method of the stationary hydraulic head response, measured at spatially discrete observation locations, of a groundwater flow model of the Hanford Site, as a function of the 1,000-dimensional representation of the model's log-transmissivity field. We find that BA surrogate models of the hydraulic head based on CKLEs are more accurate than BA surrogate models based on unconditional expansions for forward uncertainty quantification tasks. Furthermore, we find that inverse estimates of the hydraulic transmissivity field computed using CKLE-based BA surrogate models are more accurate than those computed using unconditional BA surrogate models.
△ Less
Submitted 5 July, 2023;
originally announced July 2023.
-
Gaussian process regression and conditional Karhunen-Loéve models for data assimilation in inverse problems
Authors:
Yu-Hong Yeung,
David A. Barajas-Solano,
Alexandre M. Tartakovsky
Abstract:
We present a model inversion algorithm, CKLEMAP, for data assimilation and parameter estimation in partial differential equation models of physical systems with spatially heterogeneous parameter fields. These fields are approximated using low-dimensional conditional Karhunen-Loéve expansions, which are constructed using Gaussian process regression models of these fields trained on the parameters'…
▽ More
We present a model inversion algorithm, CKLEMAP, for data assimilation and parameter estimation in partial differential equation models of physical systems with spatially heterogeneous parameter fields. These fields are approximated using low-dimensional conditional Karhunen-Loéve expansions, which are constructed using Gaussian process regression models of these fields trained on the parameters' measurements. We then assimilate measurements of the state of the system and compute the maximum a posteriori estimate of the CKLE coefficients by solving a nonlinear least-squares problem. When solving this optimization problem, we efficiently compute the Jacobian of the vector objective by exploiting the sparsity structure of the linear system of equations associated with the forward solution of the physics problem. The CKLEMAP method provides better scalability compared to the standard MAP method. In the MAP method, the number of unknowns to be estimated is equal to the number of elements in the numerical forward model. On the other hand, in CKLEMAP, the number of unknowns (CKLE coefficients) is controlled by the smoothness of the parameter field and the number of measurements, and is in general much smaller than the number of discretization nodes, which leads to a significant reduction of computational cost with respect to the standard MAP method. To show its advantage in scalability, we apply CKLEMAP to estimate the transmissivity field in a two-dimensional steady-state subsurface flow model of the Hanford Site by assimilating synthetic measurements of transmissivity and hydraulic head. We find that the execution time of CKLEMAP scales nearly linearly as $N^{1.33}$, where $N$ is the number of discretization nodes, while the execution time of standard MAP scales as $N^{2.91}$. The CKLEMAP method improved execution time without sacrificing accuracy when compared to the standard MAP.
△ Less
Submitted 26 January, 2023;
originally announced January 2023.
-
Physics-Informed Neural Network Method for Parabolic Differential Equations with Sharply Perturbed Initial Conditions
Authors:
Yifei Zong,
QiZhi He,
Alexandre M. Tartakovsky
Abstract:
In this paper, we develop a physics-informed neural network (PINN) model for parabolic problems with a sharply perturbed initial condition. As an example of a parabolic problem, we consider the advection-dispersion equation (ADE) with a point (Gaussian) source initial condition. In the $d$-dimensional ADE, perturbations in the initial condition decay with time $t$ as $t^{-d/2}$, which can cause a…
▽ More
In this paper, we develop a physics-informed neural network (PINN) model for parabolic problems with a sharply perturbed initial condition. As an example of a parabolic problem, we consider the advection-dispersion equation (ADE) with a point (Gaussian) source initial condition. In the $d$-dimensional ADE, perturbations in the initial condition decay with time $t$ as $t^{-d/2}$, which can cause a large approximation error in the PINN solution. Localized large gradients in the ADE solution make the (common in PINN) Latin hypercube sampling of the equation's residual highly inefficient. Finally, the PINN solution of parabolic equations is sensitive to the choice of weights in the loss function. We propose a normalized form of ADE where the initial perturbation of the solution does not decrease in amplitude and demonstrate that this normalization significantly reduces the PINN approximation error. We propose criteria for weights in the loss function that produce a more accurate PINN solution than those obtained with the weights selected via other methods. Finally, we proposed an adaptive sampling scheme that significantly reduces the PINN solution error for the same number of the sampling (residual) points. We demonstrate the accuracy of the proposed PINN model for forward, inverse, and backward ADEs.
△ Less
Submitted 18 August, 2022;
originally announced August 2022.
-
Physics-Informed Machine Learning Method for Large-Scale Data Assimilation Problems
Authors:
Yu-Hong Yeung,
David A. Barajas-Solano,
Alexandre M. Tartakovsky
Abstract:
We develop a physics-informed machine learning approach for large-scale data assimilation and parameter estimation and apply it for estimating transmissivity and hydraulic head in the two-dimensional steady-state subsurface flow model of the Hanford Site given synthetic measurements of said variables. In our approach, we extend the physics-informed conditional Karhunen-Loéve expansion (PICKLE) met…
▽ More
We develop a physics-informed machine learning approach for large-scale data assimilation and parameter estimation and apply it for estimating transmissivity and hydraulic head in the two-dimensional steady-state subsurface flow model of the Hanford Site given synthetic measurements of said variables. In our approach, we extend the physics-informed conditional Karhunen-Loéve expansion (PICKLE) method for modeling subsurface flow with unknown flux (Neumann) and varying head (Dirichlet) boundary conditions. We demonstrate that the PICKLE method is comparable in accuracy with the standard maximum a posteriori (MAP) method, but is significantly faster than MAP for large-scale problems. Both methods use a mesh to discretize the computational domain. In MAP, the parameters and states are discretized on the mesh; therefore, the size of the MAP parameter estimation problem directly depends on the mesh size. In PICKLE, the mesh is used to evaluate the residuals of the governing equation, while the parameters and states are approximated by the truncated conditional Karhunen-Loéve expansions with the number of parameters controlled by the smoothness of the parameter and state fields, and not by the mesh size. For a considered example, we demonstrate that the computational cost of PICKLE increases near linearly (as $N_{FV}^{1.15}$) with the number of grid points $N_{FV}$, while that of MAP increases much faster as $N_{FV}^{3.28}$. We demonstrated that once trained for one set of Dirichlet boundary conditions (i.e., one river stage), the PICKLE method provides accurate estimates of the hydraulic head for any value of the Dirichlet boundary conditions (i.e., for any river stage).
△ Less
Submitted 30 July, 2021;
originally announced August 2021.
-
Time-dependent stochastic basis adaptation for uncertainty quantification
Authors:
Ramakrishna Tipireddy,
Panos Stinis,
Alexandre M. Tartakovsky
Abstract:
We extend stochastic basis adaptation and spatial domain decomposition methods to solve time varying stochastic partial differential equations (SPDEs) with a large number of input random parameters. Stochastic basis adaptation allows the determination of a low dimensional stochastic basis representation of a quantity of interest (QoI). Extending basis adaptation to time-dependent problems is chall…
▽ More
We extend stochastic basis adaptation and spatial domain decomposition methods to solve time varying stochastic partial differential equations (SPDEs) with a large number of input random parameters. Stochastic basis adaptation allows the determination of a low dimensional stochastic basis representation of a quantity of interest (QoI). Extending basis adaptation to time-dependent problems is challenging because small errors introduced in the previous time steps of the low dimensional approximate solution accumulate over time and cause divergence from the true solution. To address this issue we have introduced an approach where the basis adaptation varies at every time step so that the low dimensional basis is adapted to the QoI at that time step. We have coupled the time-dependent basis adaptation with domain decomposition to further increase the accuracy in the representation of the QoI. To illustrate the construction, we present numerical results for one-dimensional time varying linear and nonlinear diffusion equations with random space-dependent diffusion coefficients. Stochastic dimension reduction techniques proposed in the literature have mainly focused on quantifying the uncertainty in time independent and scalar QoI. To the best of our knowledge, this is the first attempt to extend dimensional reduction techniques to time varying and spatially dependent quantities such as the solution of SPDEs.
△ Less
Submitted 4 March, 2021;
originally announced March 2021.
-
Inverse Modeling of Viscoelasticity Materials using Physics Constrained Learning
Authors:
Kailai Xu,
Alexandre M. Tartakovsky,
Jeff Burghardt,
Eric Darve
Abstract:
We propose a novel approach to model viscoelasticity materials using neural networks, which capture rate-dependent and nonlinear constitutive relations. However, inputs and outputs of the neural networks are not directly observable, and therefore common training techniques with input-output pairs for the neural networks are inapplicable. To that end, we develop a novel computational approach to bo…
▽ More
We propose a novel approach to model viscoelasticity materials using neural networks, which capture rate-dependent and nonlinear constitutive relations. However, inputs and outputs of the neural networks are not directly observable, and therefore common training techniques with input-output pairs for the neural networks are inapplicable. To that end, we develop a novel computational approach to both calibrate parametric and learn neural-network-based constitutive relations of viscoelasticity materials from indirect displacement data in the context of multi-physics interactions. We show that limited displacement data hold sufficient information to quantify the viscoelasticity behavior. We formulate the inverse computation---modeling viscoelasticity properties from observed displacement data---as a PDE-constrained optimization problem and minimize the error functional using a gradient-based optimization method. The gradients are computed by a combination of automatic differentiation and physics constrained learning. The effectiveness of our method is demonstrated through numerous benchmark problems in geomechanics and porous media transport.
△ Less
Submitted 9 May, 2020;
originally announced May 2020.
-
Physics-Informed Machine Learning with Conditional Karhunen-Loève Expansions
Authors:
Alexandre M. Tartakovsky,
David A. Barajas-Solano,
Qizhi He
Abstract:
We present a new physics-informed machine learning approach for the inversion of PDE models with heterogeneous parameters. In our approach, the space-dependent partially-observed parameters and states are approximated via Karhunen-Loève expansions (KLEs). Each of these KLEs is then conditioned on their corresponding measurements, resulting in low-dimensional models of the parameters and states tha…
▽ More
We present a new physics-informed machine learning approach for the inversion of PDE models with heterogeneous parameters. In our approach, the space-dependent partially-observed parameters and states are approximated via Karhunen-Loève expansions (KLEs). Each of these KLEs is then conditioned on their corresponding measurements, resulting in low-dimensional models of the parameters and states that resolve observed data. Finally, the coefficients of the KLEs are estimated by minimizing the norm of the residual of the PDE model evaluated at a finite set of points in the computational domain, ensuring that the reconstructed parameters and states are consistent with both the observations and the PDE model to an arbitrary level of accuracy.
In our approach, KLEs are constructed using the eigendecomposition of covariance models of spatial variability. For the model parameters, we employ a parameterized covariance model calibrated on parameter observations; for the model states, the covariance is estimated from a number of forward simulations of the PDE model corresponding to realizations of the parameters drawn from their KLE. We apply the proposed approach to identifying heterogeneous log-diffusion coefficients in diffusion equations from spatially sparse measurements of the log-diffusion coefficient and the solution of the diffusion equation. We find that the proposed approach compares favorably against state-of-the-art point estimates such as maximum a posteriori estimation and physics-informed neural networks.
△ Less
Submitted 4 December, 2019;
originally announced December 2019.
-
Gaussian Process Regression and Conditional Polynomial Chaos for Parameter Estimation
Authors:
**g Li,
Alexandre M Tartakovsky
Abstract:
We present a new approach for constructing a data-driven surrogate model and using it for Bayesian parameter estimation in partial differential equation (PDE) models. We first use parameter observations and Gaussian Process regression to condition the Karhunen-Loéve (KL) expansion of the unknown space-dependent parameters and then build the conditional generalized Polynomial Chaos (gPC) surrogate…
▽ More
We present a new approach for constructing a data-driven surrogate model and using it for Bayesian parameter estimation in partial differential equation (PDE) models. We first use parameter observations and Gaussian Process regression to condition the Karhunen-Loéve (KL) expansion of the unknown space-dependent parameters and then build the conditional generalized Polynomial Chaos (gPC) surrogate model of the PDE states. Next, we estimate the unknown parameters by computing coefficients in the KL expansion minimizing the square difference between the gPC predictions and measurements of the states using the Markov Chain Monte Carlo method. Our approach addresses two major challenges in the Bayesian parameter estimation. First, it reduces dimensionality of the parameter space and replaces expensive direct solutions of PDEs with the conditional gPC surrogates. Second, the estimated parameter field exactly matches the parameter measurements. In addition, we show that the conditional gPC surrogate can be used to estimate the states variance, which, in turn, can be used to guide data acquisition. We demonstrate that our approach improves its accuracy with application to one- and two-dimensional Darcy equation with (unknown) space-dependent hydraulic conductivity. We also discuss the effect of hydraulic conductivity and head locations on the accuracy of the hydraulic conductivity estimations.
△ Less
Submitted 31 July, 2019;
originally announced August 2019.
-
Conditional Karhunen-Loève expansion for uncertainty quantification and active learning in partial differential equation models
Authors:
Ramakrishna Tipireddy,
David A Barajas-Solano,
Alexandre M. Tartakovsky
Abstract:
We use a conditional Karhunen-Loève (KL) model to quantify and reduce uncertainty in a stochastic partial differential equation (SPDE) problem with partially-known space-dependent coefficient, $Y(x)$.
We assume that a small number of $Y(x)$ measurements are available and model $Y(x)$ with a KL expansion.
We achieve reduction in uncertainty by conditioning the KL expansion coefficients on measu…
▽ More
We use a conditional Karhunen-Loève (KL) model to quantify and reduce uncertainty in a stochastic partial differential equation (SPDE) problem with partially-known space-dependent coefficient, $Y(x)$.
We assume that a small number of $Y(x)$ measurements are available and model $Y(x)$ with a KL expansion.
We achieve reduction in uncertainty by conditioning the KL expansion coefficients on measurements.
We consider two approaches for conditioning the KL expansion: In Approach 1, we condition the KL model first and then truncate it.
In Approach 2, we first truncate the KL expansion and then condition it.
We employ the conditional KL expansion together with Monte Carlo and sparse grid collocation methods to compute the moments of the solution of the SPDE problem.
Uncertainty of the problem is further reduced by adaptively selecting additional observation locations using two active learning methods.
Method 1 minimizes the variance of the PDE coefficient, while Method 2 minimizes the variance of the solution of the PDE.
We demonstrate that conditioning leads to dimension reduction of the KL representation of $Y(x)$.
For a linear diffusion SPDE with uncertain log-normal coefficient, we show that Approach 1 provides a more accurate approximation of the conditional log-normal coefficient and solution of the SPDE than Approach 2 for the same number of random dimensions in a conditional KL expansion.
Furthermore, Approach 2 provides a good estimate for the number of terms of the truncated KL expansion of the conditional field of Approach 1.
Finally, we demonstrate that active learning based on Method 2 is more efficient for uncertainty reduction in the SPDE's states (i.e., it leads to a larger reduction of the variance) than active learning using Method 2.
△ Less
Submitted 16 April, 2019;
originally announced April 2019.
-
Learning Parameters and Constitutive Relationships with Physics Informed Deep Neural Networks
Authors:
Alexandre M. Tartakovsky,
Carlos Ortiz Marrero,
Paris Perdikaris,
Guzel D. Tartakovsky,
David Barajas-Solano
Abstract:
We present a physics informed deep neural network (DNN) method for estimating parameters and unknown physics (constitutive relationships) in partial differential equation (PDE) models. We use PDEs in addition to measurements to train DNNs to approximate unknown parameters and constitutive relationships as well as states. The proposed approach increases the accuracy of DNN approximations of partial…
▽ More
We present a physics informed deep neural network (DNN) method for estimating parameters and unknown physics (constitutive relationships) in partial differential equation (PDE) models. We use PDEs in addition to measurements to train DNNs to approximate unknown parameters and constitutive relationships as well as states. The proposed approach increases the accuracy of DNN approximations of partially known functions when a limited number of measurements is available and allows for training DNNs when no direct measurements of the functions of interest are available. We employ physics informed DNNs to estimate the unknown space-dependent diffusion coefficient in a linear diffusion equation and an unknown constitutive relationship in a non-linear diffusion equation. For the parameter estimation problem, we assume that partial measurements of the coefficient and states are available and demonstrate that under these conditions, the proposed method is more accurate than state-of-the-art methods. For the non-linear diffusion PDE model with a fully unknown constitutive relationship (i.e., no measurements of constitutive relationship are available), the physics informed DNN method can accurately estimate the non-linear constitutive relationship based on state measurements only. Finally, we demonstrate that the proposed method remains accurate in the presence of measurement noise.
△ Less
Submitted 17 August, 2018; v1 submitted 9 August, 2018;
originally announced August 2018.
-
Solving differential equations with unknown constitutive relations as recurrent neural networks
Authors:
Tobias Hagge,
Panos Stinis,
Enoch Yeung,
Alexandre M. Tartakovsky
Abstract:
We solve a system of ordinary differential equations with an unknown functional form of a sink (reaction rate) term. We assume that the measurements (time series) of state variables are partially available, and we use recurrent neural network to "learn" the reaction rate from this data. This is achieved by including a discretized ordinary differential equations as part of a recurrent neural networ…
▽ More
We solve a system of ordinary differential equations with an unknown functional form of a sink (reaction rate) term. We assume that the measurements (time series) of state variables are partially available, and we use recurrent neural network to "learn" the reaction rate from this data. This is achieved by including a discretized ordinary differential equations as part of a recurrent neural network training problem. We extend TensorFlow's recurrent neural network architecture to create a simple but scalable and effective solver for the unknown functions, and apply it to a fedbatch bioreactor simulation problem. Use of techniques from recent deep learning literature enables training of functions with behavior manifesting over thousands of time steps. Our networks are structurally similar to recurrent neural networks, but differences in design and function require modifications to the conventional wisdom about training such networks.
△ Less
Submitted 5 October, 2017;
originally announced October 2017.