-
UQpy v4.1: Uncertainty Quantification with Python
Authors:
Dimitrios Tsapetis,
Michael D. Shields,
Dimitris G. Giovanis,
Audrey Olivier,
Lukas Novak,
Promit Chakroborty,
Himanshu Sharma,
Mohit Chauhan,
Katiana Kontolati,
Lohit Vandanapu,
Dimitrios Loukrezis,
Michael Gardner
Abstract:
This paper presents the latest improvements introduced in Version 4 of the UQpy, Uncertainty Quantification with Python, library. In the latest version, the code was restructured to conform with the latest Python coding conventions, refactored to simplify previous tightly coupled features, and improve its extensibility and modularity. To improve the robustness of UQpy, software engineering best pr…
▽ More
This paper presents the latest improvements introduced in Version 4 of the UQpy, Uncertainty Quantification with Python, library. In the latest version, the code was restructured to conform with the latest Python coding conventions, refactored to simplify previous tightly coupled features, and improve its extensibility and modularity. To improve the robustness of UQpy, software engineering best practices were adopted. A new software development workflow significantly improved collaboration between team members, and continous integration and automated testing ensured the robustness and reliability of software performance. Continuous deployment of UQpy allowed its automated packaging and distribution in system agnostic format via multiple channels, while a Docker image enables the use of the toolbox regardless of operating system limitations.
△ Less
Submitted 16 May, 2023;
originally announced May 2023.
-
Learning in latent spaces improves the predictive accuracy of deep neural operators
Authors:
Katiana Kontolati,
Somdatta Goswami,
George Em Karniadakis,
Michael D. Shields
Abstract:
Operator regression provides a powerful means of constructing discretization-invariant emulators for partial-differential equations (PDEs) describing physical systems. Neural operators specifically employ deep neural networks to approximate map**s between infinite-dimensional Banach spaces. As data-driven models, neural operators require the generation of labeled observations, which in cases of…
▽ More
Operator regression provides a powerful means of constructing discretization-invariant emulators for partial-differential equations (PDEs) describing physical systems. Neural operators specifically employ deep neural networks to approximate map**s between infinite-dimensional Banach spaces. As data-driven models, neural operators require the generation of labeled observations, which in cases of complex high-fidelity models result in high-dimensional datasets containing redundant and noisy features, which can hinder gradient-based optimization. Map** these high-dimensional datasets to a low-dimensional latent space of salient features can make it easier to work with the data and also enhance learning. In this work, we investigate the latent deep operator network (L-DeepONet), an extension of standard DeepONet, which leverages latent representations of high-dimensional PDE input and output functions identified with suitable autoencoders. We illustrate that L-DeepONet outperforms the standard approach in terms of both accuracy and computational efficiency across diverse time-dependent PDEs, e.g., modeling the growth of fracture in brittle materials, convective fluid flows, and large-scale atmospheric flows exhibiting multiscale dynamical features.
△ Less
Submitted 15 April, 2023;
originally announced April 2023.
-
Deep transfer operator learning for partial differential equations under conditional shift
Authors:
Somdatta Goswami,
Katiana Kontolati,
Michael D. Shields,
George Em Karniadakis
Abstract:
Transfer learning (TL) enables the transfer of knowledge gained in learning to perform one task (source) to a related but different task (target), hence addressing the expense of data acquisition and labeling, potential computational power limitations, and dataset distribution mismatches. We propose a new TL framework for task-specific learning (functional regression in partial differential equati…
▽ More
Transfer learning (TL) enables the transfer of knowledge gained in learning to perform one task (source) to a related but different task (target), hence addressing the expense of data acquisition and labeling, potential computational power limitations, and dataset distribution mismatches. We propose a new TL framework for task-specific learning (functional regression in partial differential equations (PDEs)) under conditional shift based on the deep operator network (DeepONet). Task-specific operator learning is accomplished by fine-tuning task-specific layers of the target DeepONet using a hybrid loss function that allows for the matching of individual target samples while also preserving the global properties of the conditional distribution of target data. Inspired by the conditional embedding operator theory, we minimize the statistical distance between labeled target data and the surrogate prediction on unlabeled target data by embedding conditional distributions onto a reproducing kernel Hilbert space. We demonstrate the advantages of our approach for various TL scenarios involving nonlinear PDEs under diverse conditions due to shift in the geometric domain and model dynamics. Our TL framework enables fast and efficient learning of heterogeneous tasks despite significant differences between the source and target domains.
△ Less
Submitted 20 November, 2022; v1 submitted 20 April, 2022;
originally announced April 2022.
-
On the influence of over-parameterization in manifold based surrogates and deep neural operators
Authors:
Katiana Kontolati,
Somdatta Goswami,
Michael D. Shields,
George Em Karniadakis
Abstract:
Constructing accurate and generalizable approximators for complex physico-chemical processes exhibiting highly non-smooth dynamics is challenging. In this work, we propose new developments and perform comparisons for two promising approaches: manifold-based polynomial chaos expansion (m-PCE) and the deep neural operator (DeepONet), and we examine the effect of over-parameterization on generalizati…
▽ More
Constructing accurate and generalizable approximators for complex physico-chemical processes exhibiting highly non-smooth dynamics is challenging. In this work, we propose new developments and perform comparisons for two promising approaches: manifold-based polynomial chaos expansion (m-PCE) and the deep neural operator (DeepONet), and we examine the effect of over-parameterization on generalization. We demonstrate the performance of these methods in terms of generalization accuracy by solving the 2D time-dependent Brusselator reaction-diffusion system with uncertainty sources, modeling an autocatalytic chemical reaction between two species. We first propose an extension of the m-PCE by constructing a map** between latent spaces formed by two separate embeddings of input functions and output QoIs. To enhance the accuracy of the DeepONet, we introduce weight self-adaptivity in the loss function. We demonstrate that the performance of m-PCE and DeepONet is comparable for cases of relatively smooth input-output map**s. However, when highly non-smooth dynamics is considered, DeepONet shows higher accuracy. We also find that for m-PCE, modest over-parameterization leads to better generalization, both within and outside of distribution, whereas aggressive over-parameterization leads to over-fitting. In contrast, an even highly over-parameterized DeepONet leads to better generalization for both smooth and non-smooth dynamics. Furthermore, we compare the performance of the above models with another operator learning model, the Fourier Neural Operator, and show that its over-parameterization also leads to better generalization. Our studies show that m-PCE can provide very good accuracy at very low training cost, whereas a highly over-parameterized DeepONet can provide better accuracy and robustness to noise but at higher training cost. In both methods, the inference cost is negligible.
△ Less
Submitted 9 March, 2022;
originally announced March 2022.
-
A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems
Authors:
Katiana Kontolati,
Dimitrios Loukrezis,
Dimitris G. Giovanis,
Lohit Vandanapu,
Michael D. Shields
Abstract:
Constructing surrogate models for uncertainty quantification (UQ) on complex partial differential equations (PDEs) having inherently high-dimensional $\mathcal{O}(10^{\ge 2})$ stochastic inputs (e.g., forcing terms, boundary conditions, initial conditions) poses tremendous challenges. The curse of dimensionality can be addressed with suitable unsupervised learning techniques used as a pre-processi…
▽ More
Constructing surrogate models for uncertainty quantification (UQ) on complex partial differential equations (PDEs) having inherently high-dimensional $\mathcal{O}(10^{\ge 2})$ stochastic inputs (e.g., forcing terms, boundary conditions, initial conditions) poses tremendous challenges. The curse of dimensionality can be addressed with suitable unsupervised learning techniques used as a pre-processing tool to encode inputs onto lower-dimensional subspaces while retaining its structural information and meaningful properties. In this work, we review and investigate thirteen dimension reduction methods including linear and nonlinear, spectral, blind source separation, convex and non-convex methods and utilize the resulting embeddings to construct a map** to quantities of interest via polynomial chaos expansions (PCE). We refer to the general proposed approach as manifold PCE (m-PCE), where manifold corresponds to the latent space resulting from any of the studied dimension reduction methods. To investigate the capabilities and limitations of these methods we conduct numerical tests for three physics-based systems (treated as black-boxes) having high-dimensional stochastic inputs of varying complexity modeled as both Gaussian and non-Gaussian random fields to investigate the effect of the intrinsic dimensionality of input data. We demonstrate both the advantages and limitations of the unsupervised learning methods and we conclude that a suitable m-PCE model provides a cost-effective approach compared to alternative algorithms proposed in the literature, including recently proposed expensive deep neural network-based surrogates and can be readily applied for high-dimensional UQ in stochastic PDEs.
△ Less
Submitted 16 May, 2022; v1 submitted 9 February, 2022;
originally announced February 2022.
-
Grassmannian diffusion maps based surrogate modeling via geometric harmonics
Authors:
Ketson R. M. dos Santos,
Dimitrios G. Giovanis,
Katiana Kontolati,
Dimitrios Loukrezis,
Michael D. Shields
Abstract:
In this paper, a novel surrogate model based on the Grassmannian diffusion maps (GDMaps) and utilizing geometric harmonics is developed for predicting the response of engineering systems and complex physical phenomena. The method utilizes the GDMaps to obtain a low-dimensional representation of the underlying behavior of physical/mathematical systems with respect to uncertainties in the input para…
▽ More
In this paper, a novel surrogate model based on the Grassmannian diffusion maps (GDMaps) and utilizing geometric harmonics is developed for predicting the response of engineering systems and complex physical phenomena. The method utilizes the GDMaps to obtain a low-dimensional representation of the underlying behavior of physical/mathematical systems with respect to uncertainties in the input parameters. Using this representation, geometric harmonics, an out-of-sample function extension technique, is employed to create a global map from the space of input parameters to a Grassmannian diffusion manifold. Geometric harmonics is also employed to locally map points on the diffusion manifold onto the tangent space of a Grassmann manifold. The exponential map is then used to project the points in the tangent space onto the Grassmann manifold, where reconstruction of the full solution is performed. The performance of the proposed surrogate modeling is verified with three examples. The first problem is a toy example used to illustrate the development of the technique. In the second example, errors associated with the various map**s employed in the technique are assessed by studying response predictions of the electric potential of a dielectric cylinder in a homogeneous electric field. The last example applies the method for uncertainty prediction in the strain field evolution in a model amorphous material using the shear transformation zone (STZ) theory of plasticity. In all examples, accurate predictions are obtained, showing that the present technique is a strong candidate for the application of uncertainty quantification in large-scale models.
△ Less
Submitted 28 September, 2021;
originally announced September 2021.
-
Neural density estimation and uncertainty quantification for laser induced breakdown spectroscopy spectra
Authors:
Katiana Kontolati,
Natalie Klein,
Nishant Panda,
Diane Oyen
Abstract:
Constructing probability densities for inference in high-dimensional spectral data is often intractable. In this work, we use normalizing flows on structured spectral latent spaces to estimate such densities, enabling downstream inference tasks. In addition, we evaluate a method for uncertainty quantification when predicting unobserved state vectors associated with each spectrum. We demonstrate th…
▽ More
Constructing probability densities for inference in high-dimensional spectral data is often intractable. In this work, we use normalizing flows on structured spectral latent spaces to estimate such densities, enabling downstream inference tasks. In addition, we evaluate a method for uncertainty quantification when predicting unobserved state vectors associated with each spectrum. We demonstrate the capability of this approach on laser-induced breakdown spectroscopy data collected by the ChemCam instrument on the Mars rover Curiosity. Using our approach, we are able to generate realistic spectral samples and to accurately predict state vectors with associated well-calibrated uncertainties. We anticipate that this methodology will enable efficient probabilistic modeling of spectral data, leading to potential advances in several areas, including out-of-distribution detection and sensitivity analysis.
△ Less
Submitted 16 August, 2021;
originally announced August 2021.
-
Manifold learning-based polynomial chaos expansions for high-dimensional surrogate models
Authors:
Katiana Kontolati,
Dimitrios Loukrezis,
Ketson R. M. dos Santos,
Dimitrios G. Giovanis,
Michael D. Shields
Abstract:
In this work we introduce a manifold learning-based method for uncertainty quantification (UQ) in systems describing complex spatiotemporal processes. Our first objective is to identify the embedding of a set of high-dimensional data representing quantities of interest of the computational or analytical model. For this purpose, we employ Grassmannian diffusion maps, a two-step nonlinear dimension…
▽ More
In this work we introduce a manifold learning-based method for uncertainty quantification (UQ) in systems describing complex spatiotemporal processes. Our first objective is to identify the embedding of a set of high-dimensional data representing quantities of interest of the computational or analytical model. For this purpose, we employ Grassmannian diffusion maps, a two-step nonlinear dimension reduction technique which allows us to reduce the dimensionality of the data and identify meaningful geometric descriptions in a parsimonious and inexpensive manner. Polynomial chaos expansion is then used to construct a map** between the stochastic input parameters and the diffusion coordinates of the reduced space. An adaptive clustering technique is proposed to identify an optimal number of clusters of points in the latent space. The similarity of points allows us to construct a number of geometric harmonic emulators which are finally utilized as a set of inexpensive pre-trained models to perform an inverse map of realizations of latent features to the ambient space and thus perform accurate out-of-sample predictions. Thus, the proposed method acts as an encoder-decoder system which is able to automatically handle very high-dimensional data while simultaneously operating successfully in the small-data regime. The method is demonstrated on two benchmark problems and on a system of advection-diffusion-reaction equations which model a first-order chemical reaction between two species. In all test cases, the proposed method is able to achieve highly accurate approximations which ultimately lead to the significant acceleration of UQ tasks.
△ Less
Submitted 20 July, 2021;
originally announced July 2021.
-
Manifold learning for coarse-graining atomistic simulations: Application to amorphous solids
Authors:
Katiana Kontolati,
Darius Alix-Williams,
Nicholas M. Boffi,
Michael L. Falk,
Chris H. Rycroft,
Michael D. Shields
Abstract:
We introduce a generalized machine learning framework to probabilistically parameterize upper-scale models in the form of nonlinear PDEs consistent with a continuum theory, based on coarse-grained atomistic simulation data of mechanical deformation and flow processes. The proposed framework utilizes a hypothesized coarse-graining methodology with manifold learning and surrogate-based optimization…
▽ More
We introduce a generalized machine learning framework to probabilistically parameterize upper-scale models in the form of nonlinear PDEs consistent with a continuum theory, based on coarse-grained atomistic simulation data of mechanical deformation and flow processes. The proposed framework utilizes a hypothesized coarse-graining methodology with manifold learning and surrogate-based optimization techniques. Coarse-grained high-dimensional data describing quantities of interest of the multiscale models are projected onto a nonlinear manifold whose geometric and topological structure is exploited for measuring behavioral discrepancies in the form of manifold distances. A surrogate model is constructed using Gaussian process regression to identify a map** between stochastic parameters and distances. Derivative-free optimization is employed to adaptively identify a unique set of parameters of the upper-scale model capable of rapidly reproducing the system's behavior while maintaining consistency with coarse-grained atomic-level simulations. The proposed method is applied to learn the parameters of the shear transformation zone (STZ) theory of plasticity that describes plastic deformation in amorphous solids as well as coarse-graining parameters needed to translate between atomistic and continuum representations. We show that the methodology is able to successfully link coarse-grained microscale simulations to macroscale observables and achieve a high-level of parity between the models across scales.
△ Less
Submitted 23 July, 2021; v1 submitted 1 March, 2021;
originally announced March 2021.
-
Numerical analysis of a mechanotransduction dynamical model reveals homoclinic bifurcations of extracellular matrix mediated oscillations of the mesenchymal stem cell fate
Authors:
Katiana Kontolati,
Constantinos Siettos
Abstract:
We perform one and two-parameter numerical bifurcation analysis of a mechanotransduction model approximating the dynamics of mesenchymal stem cell differentiation into neurons, adipocytes, myocytes and osteoblasts. For our analysis, we use as bifurcation parameters the stiffness of the extracellular matrix and parameters linked with the positive feedback mechanisms that up-regulate the production…
▽ More
We perform one and two-parameter numerical bifurcation analysis of a mechanotransduction model approximating the dynamics of mesenchymal stem cell differentiation into neurons, adipocytes, myocytes and osteoblasts. For our analysis, we use as bifurcation parameters the stiffness of the extracellular matrix and parameters linked with the positive feedback mechanisms that up-regulate the production of the YAP/TAZ transcriptional regulators (TRs) and the cell adhesion area. Our analysis reveals a rich nonlinear behaviour of the cell differentiation including regimes of hysteresis and multistability, stable oscillations of the effective adhesion area, the YAP/TAZ TRs and the PPAR$γ$ receptors associated with the adipogenic fate, as well as homoclinic bifurcations that interrupt relatively high-amplitude oscillations abruptly. The two-parameter bifurcation analysis of the Andronov-Hopf points that give birth to the oscillating patterns predicts their existence for soft extracellular substrates ($<1kPa$), a regime that favours the neurogenic and the adipogenic cell fate. Furthermore, in these regimes, the analysis reveals the presence of homoclinic bifurcations that result in the sudden loss of the stable oscillations of the cell-substrate adhesion towards weaker adhesion and high expression levels of the gene encoding Tubulin beta-3 chain, thus favouring the phase transition from the adipogenic to the neurogenic fate.
△ Less
Submitted 4 February, 2019;
originally announced February 2019.
