Showing 1–13 of 13 results for author: Giovanis, D G

Discovering deposition process regimes: leveraging unsupervised learning for process insights, surrogate modeling, and sensitivity analysis

Authors: Geremy Loachamín Suntaxi, Paris Papavasileiou, Eleni D. Koronaki, Dimitrios G. Giovanis, Georgios Gakis, Ioannis G. Aviziotis, Martin Kathrein, Gabriele Pozzetti, Christoph Czettl, Stéphane P. A. Bordas, Andreas G. Boudouvis

Abstract: This work introduces a comprehensive approach utilizing data-driven methods to elucidate the deposition process regimes in Chemical Vapor Deposition (CVD) reactors and the interplay of physical mechanism that dominate in each one of them. Through this work, we address three key objectives. Firstly, our methodology relies on process outcomes, derived by a detailed CFD model, to identify clusters of… ▽ More This work introduces a comprehensive approach utilizing data-driven methods to elucidate the deposition process regimes in Chemical Vapor Deposition (CVD) reactors and the interplay of physical mechanism that dominate in each one of them. Through this work, we address three key objectives. Firstly, our methodology relies on process outcomes, derived by a detailed CFD model, to identify clusters of "outcomes" corresponding to distinct process regimes, wherein the relative influence of input variables undergoes notable shifts. This phenomenon is experimentally validated through Arrhenius plot analysis, affirming the efficacy of our approach. Secondly, we demonstrate the development of an efficient surrogate model, based on Polynomial Chaos Expansion (PCE), that maintains accuracy, facilitating streamlined computational analyses. Finally, as a result of PCE, sensitivity analysis is made possible by means of Sobol' indices, that quantify the impact of process inputs across identified regimes. The insights gained from our analysis contribute to the formulation of hypotheses regarding phenomena occurring beyond the transition regime. Notably, the significance of temperature even in the diffusion-limited regime, as evidenced by the Arrhenius plot, suggests activation of gas phase reactions at elevated temperatures. Importantly, our proposed methods yield insights that align with experimental observations and theoretical principles, aiding decision-making in process design and optimization. By circumventing the need for costly and time-consuming experiments, our approach offers a pragmatic pathway towards enhanced process efficiency. Moreover, this study underscores the potential of data-driven computational methods for innovating reactor design paradigms. △ Less

Submitted 24 May, 2024; originally announced May 2024.

Integrating supervised and unsupervised learning approaches to unveil critical process inputs

Authors: Paris Papavasileiou, Dimitrios G. Giovanis, Gabriele Pozzetti, Martin Kathrein, Christoph Czettl, Ioannis G. Kevrekidis, Andreas G. Boudouvis, Stéphane P. A. Bordas, Eleni D. Koronaki

Abstract: This study introduces a machine learning framework tailored to large-scale industrial processes characterized by a plethora of numerical and categorical inputs. The framework aims to (i) discern critical parameters influencing the output and (ii) generate accurate out-of-sample qualitative and quantitative predictions of production outcomes. Specifically, we address the pivotal question of the sig… ▽ More This study introduces a machine learning framework tailored to large-scale industrial processes characterized by a plethora of numerical and categorical inputs. The framework aims to (i) discern critical parameters influencing the output and (ii) generate accurate out-of-sample qualitative and quantitative predictions of production outcomes. Specifically, we address the pivotal question of the significance of each input in sha** the process outcome, using an industrial Chemical Vapor Deposition (CVD) process as an example. The initial objective involves merging subject matter expertise and clustering techniques exclusively on the process output, here, coating thickness measurements at various positions in the reactor. This approach identifies groups of production runs that share similar qualitative characteristics, such as film mean thickness and standard deviation. In particular, the differences of the outcomes represented by the different clusters can be attributed to differences in specific inputs, indicating that these inputs are critical for the production outcome. Leveraging this insight, we subsequently implement supervised classification and regression methods using the identified critical process inputs. The proposed methodology proves to be valuable in scenarios with a multitude of inputs and insufficient data for the direct application of deep learning techniques, providing meaningful insights into the underlying processes. △ Less

Submitted 13 May, 2024; originally announced May 2024.

Polynomial Chaos Expansions on Principal Geodesic Grassmannian Submanifolds for Surrogate Modeling and Uncertainty Quantification

Authors: Dimitris G. Giovanis, Dimitrios Loukrezis, Ioannis G. Kevrekidis, Michael D. Shields

Abstract: In this work we introduce a manifold learning-based surrogate modeling framework for uncertainty quantification in high-dimensional stochastic systems. Our first goal is to perform data mining on the available simulation data to identify a set of low-dimensional (latent) descriptors that efficiently parameterize the response of the high-dimensional computational model. To this end, we employ Princ… ▽ More In this work we introduce a manifold learning-based surrogate modeling framework for uncertainty quantification in high-dimensional stochastic systems. Our first goal is to perform data mining on the available simulation data to identify a set of low-dimensional (latent) descriptors that efficiently parameterize the response of the high-dimensional computational model. To this end, we employ Principal Geodesic Analysis on the Grassmann manifold of the response to identify a set of disjoint principal geodesic submanifolds, of possibly different dimension, that captures the variation in the data. Since operations on the Grassmann require the data to be concentrated, we propose an adaptive algorithm based on Riemanniann K-means and the minimization of the sample Frechet variance on the Grassmann manifold to identify "local" principal geodesic submanifolds that represent different system behavior across the parameter space. Polynomial chaos expansion is then used to construct a map** between the random input parameters and the projection of the response on these local principal geodesic submanifolds. The method is demonstrated on four test cases, a toy-example that involves points on a hypersphere, a Lotka-Volterra dynamical system, a continuous-flow stirred-tank chemical reactor system, and a two-dimensional Rayleigh-Benard convection problem △ Less

Submitted 29 January, 2024; originally announced January 2024.

Comments: 50 pages, 17 figures

Machine Learning for the identification of phase-transitions in interacting agent-based systems

Authors: Nikolaos Evangelou, Dimitrios G. Giovanis, George A. Kevrekidis, Grigorios A. Pavliotis, Ioannis G. Kevrekidis

Abstract: Deriving closed-form, analytical expressions for reduced-order models, and judiciously choosing the closures leading to them, has long been the strategy of choice for studying phase- and noise-induced transitions for agent-based models (ABMs). In this paper, we propose a data-driven framework that pinpoints phase transitions for an ABM in its mean-field limit, using a smaller number of variables t… ▽ More Deriving closed-form, analytical expressions for reduced-order models, and judiciously choosing the closures leading to them, has long been the strategy of choice for studying phase- and noise-induced transitions for agent-based models (ABMs). In this paper, we propose a data-driven framework that pinpoints phase transitions for an ABM in its mean-field limit, using a smaller number of variables than traditional closed-form models. To this end, we use the manifold learning algorithm Diffusion Maps to identify a parsimonious set of data-driven latent variables, and show that they are in one-to-one correspondence with the expected theoretical order parameter of the ABM. We then utilize a deep learning framework to obtain a conformal reparametrization of the data-driven coordinates that facilitates, in our example, the identification of a single parameter-dependent ODE in these coordinates. We identify this ODE through a residual neural network inspired by a numerical integration scheme (forward Euler). We then use the identified ODE -- enabled through an odd symmetry transformation -- to construct the bifurcation diagram exhibiting the phase transition. △ Less

Submitted 29 October, 2023; originally announced October 2023.

Comments: 14 pages, 9 Figures

UQpy v4.1: Uncertainty Quantification with Python

Authors: Dimitrios Tsapetis, Michael D. Shields, Dimitris G. Giovanis, Audrey Olivier, Lukas Novak, Promit Chakroborty, Himanshu Sharma, Mohit Chauhan, Katiana Kontolati, Lohit Vandanapu, Dimitrios Loukrezis, Michael Gardner

Abstract: This paper presents the latest improvements introduced in Version 4 of the UQpy, Uncertainty Quantification with Python, library. In the latest version, the code was restructured to conform with the latest Python coding conventions, refactored to simplify previous tightly coupled features, and improve its extensibility and modularity. To improve the robustness of UQpy, software engineering best pr… ▽ More This paper presents the latest improvements introduced in Version 4 of the UQpy, Uncertainty Quantification with Python, library. In the latest version, the code was restructured to conform with the latest Python coding conventions, refactored to simplify previous tightly coupled features, and improve its extensibility and modularity. To improve the robustness of UQpy, software engineering best practices were adopted. A new software development workflow significantly improved collaboration between team members, and continous integration and automated testing ensured the robustness and reliability of software performance. Continuous deployment of UQpy allowed its automated packaging and distribution in system agnostic format via multiple channels, while a Docker image enables the use of the toolbox regardless of operating system limitations. △ Less

Submitted 16 May, 2023; originally announced May 2023.

A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems

Authors: Katiana Kontolati, Dimitrios Loukrezis, Dimitris G. Giovanis, Lohit Vandanapu, Michael D. Shields

Abstract: Constructing surrogate models for uncertainty quantification (UQ) on complex partial differential equations (PDEs) having inherently high-dimensional $\mathcal{O}(10^{\ge 2})$ stochastic inputs (e.g., forcing terms, boundary conditions, initial conditions) poses tremendous challenges. The curse of dimensionality can be addressed with suitable unsupervised learning techniques used as a pre-processi… ▽ More Constructing surrogate models for uncertainty quantification (UQ) on complex partial differential equations (PDEs) having inherently high-dimensional $\mathcal{O}(10^{\ge 2})$ stochastic inputs (e.g., forcing terms, boundary conditions, initial conditions) poses tremendous challenges. The curse of dimensionality can be addressed with suitable unsupervised learning techniques used as a pre-processing tool to encode inputs onto lower-dimensional subspaces while retaining its structural information and meaningful properties. In this work, we review and investigate thirteen dimension reduction methods including linear and nonlinear, spectral, blind source separation, convex and non-convex methods and utilize the resulting embeddings to construct a map** to quantities of interest via polynomial chaos expansions (PCE). We refer to the general proposed approach as manifold PCE (m-PCE), where manifold corresponds to the latent space resulting from any of the studied dimension reduction methods. To investigate the capabilities and limitations of these methods we conduct numerical tests for three physics-based systems (treated as black-boxes) having high-dimensional stochastic inputs of varying complexity modeled as both Gaussian and non-Gaussian random fields to investigate the effect of the intrinsic dimensionality of input data. We demonstrate both the advantages and limitations of the unsupervised learning methods and we conclude that a suitable m-PCE model provides a cost-effective approach compared to alternative algorithms proposed in the literature, including recently proposed expensive deep neural network-based surrogates and can be readily applied for high-dimensional UQ in stochastic PDEs. △ Less

Submitted 16 May, 2022; v1 submitted 9 February, 2022; originally announced February 2022.

Comments: 45 pages, 14 figures

Data-driven Uncertainty Quantification in Computational Human Head Models

Authors: Kshitiz Upadhyay, Dimitris G. Giovanis, Ahmed Alshareef, Andrew K. Knutsen, Curtis L. Johnson, Aaron Carass, Philip V. Bayly, Michael D. Shields, K. T. Ramesh

Abstract: Computational models of the human head are promising tools for estimating the impact-induced response of brain, and thus play an important role in the prediction of traumatic brain injury. Modern biofidelic head model simulations are associated with very high computational cost, and high-dimensional inputs and outputs, which limits the applicability of traditional uncertainty quantification (UQ) m… ▽ More Computational models of the human head are promising tools for estimating the impact-induced response of brain, and thus play an important role in the prediction of traumatic brain injury. Modern biofidelic head model simulations are associated with very high computational cost, and high-dimensional inputs and outputs, which limits the applicability of traditional uncertainty quantification (UQ) methods on these systems. In this study, a two-stage, data-driven manifold learning-based framework is proposed for UQ of computational head models. This framework is demonstrated on a 2D subject-specific head model, where the goal is to quantify uncertainty in the simulated strain fields (i.e., output), given variability in the material properties of different brain substructures (i.e., input). In the first stage, a data-driven method based on multi-dimensional Gaussian kernel-density estimation and diffusion maps is used to generate realizations of the input random vector directly from the available data. Computational simulations of a small number of realizations provide input-output pairs for training data-driven surrogate models in the second stage. The surrogate models employ nonlinear dimensionality reduction using Grassmannian diffusion maps, Gaussian process regression to create a low-cost map** between the input random vector and the reduced solution space, and geometric harmonics models for map** between the reduced space and the Grassmann manifold. It is demonstrated that the surrogate models provide highly accurate approximations of the computational model while significantly reducing the computational cost. Monte Carlo simulations of the surrogate models are used for uncertainty propagation. UQ of strain fields highlight significant spatial variation in model uncertainty, and reveal key differences in uncertainty among commonly used strain-based brain injury predictor variables. △ Less

Submitted 2 February, 2022; v1 submitted 29 October, 2021; originally announced October 2021.

Journal ref: Comput. Methods Appl. Mech. Eng., 398 (2022), 115108

Imprecise Subset Simulation

Authors: Dimitris G. Giovanis, Michael Shields

Abstract: The objective of this work is to quantify the uncertainty in probability of failure estimates resulting from incomplete knowledge of the probability distributions for the input random variables. We propose a framework that couples the widely used Subset simulation (SuS) with Bayesian/information theoretic multi-model inference. The process starts with data used to infer probability distributions f… ▽ More The objective of this work is to quantify the uncertainty in probability of failure estimates resulting from incomplete knowledge of the probability distributions for the input random variables. We propose a framework that couples the widely used Subset simulation (SuS) with Bayesian/information theoretic multi-model inference. The process starts with data used to infer probability distributions for the model inputs. Often such data sets are small. Multi-model inference is used to assess uncertainty associated with the model-form and parameters of these random variables in the form of model probabilities and the associated joint parameter probability densities. A sampling procedure is used to construct a set of equally probable candidate probability distributions and an optimal importance sampling distribution is determined analytically from this set. Subset simulation is then performed using this optimal sampling density and the resulting conditional probabilities are re-weighted using importance sampling. The result of this process are empirical probability distributions of failure probabilities that provide direct estimates of the uncertainty in failure probability estimates that result from inference on small data sets. The method is demonstrated to be both computationally efficient -- requiring only a single subset simulation and nominal cost of sample re-weighting -- and to provide reasonable estimates of the uncertainty in failure probabilities. △ Less

Submitted 21 October, 2021; originally announced October 2021.

Grassmannian diffusion maps based surrogate modeling via geometric harmonics

Authors: Ketson R. M. dos Santos, Dimitrios G. Giovanis, Katiana Kontolati, Dimitrios Loukrezis, Michael D. Shields

Abstract: In this paper, a novel surrogate model based on the Grassmannian diffusion maps (GDMaps) and utilizing geometric harmonics is developed for predicting the response of engineering systems and complex physical phenomena. The method utilizes the GDMaps to obtain a low-dimensional representation of the underlying behavior of physical/mathematical systems with respect to uncertainties in the input para… ▽ More In this paper, a novel surrogate model based on the Grassmannian diffusion maps (GDMaps) and utilizing geometric harmonics is developed for predicting the response of engineering systems and complex physical phenomena. The method utilizes the GDMaps to obtain a low-dimensional representation of the underlying behavior of physical/mathematical systems with respect to uncertainties in the input parameters. Using this representation, geometric harmonics, an out-of-sample function extension technique, is employed to create a global map from the space of input parameters to a Grassmannian diffusion manifold. Geometric harmonics is also employed to locally map points on the diffusion manifold onto the tangent space of a Grassmann manifold. The exponential map is then used to project the points in the tangent space onto the Grassmann manifold, where reconstruction of the full solution is performed. The performance of the proposed surrogate modeling is verified with three examples. The first problem is a toy example used to illustrate the development of the technique. In the second example, errors associated with the various map**s employed in the technique are assessed by studying response predictions of the electric potential of a dielectric cylinder in a homogeneous electric field. The last example applies the method for uncertainty prediction in the strain field evolution in a model amorphous material using the shear transformation zone (STZ) theory of plasticity. In all examples, accurate predictions are obtained, showing that the present technique is a strong candidate for the application of uncertainty quantification in large-scale models. △ Less

Submitted 28 September, 2021; originally announced September 2021.

Manifold learning-based polynomial chaos expansions for high-dimensional surrogate models

Authors: Katiana Kontolati, Dimitrios Loukrezis, Ketson R. M. dos Santos, Dimitrios G. Giovanis, Michael D. Shields

Abstract: In this work we introduce a manifold learning-based method for uncertainty quantification (UQ) in systems describing complex spatiotemporal processes. Our first objective is to identify the embedding of a set of high-dimensional data representing quantities of interest of the computational or analytical model. For this purpose, we employ Grassmannian diffusion maps, a two-step nonlinear dimension… ▽ More In this work we introduce a manifold learning-based method for uncertainty quantification (UQ) in systems describing complex spatiotemporal processes. Our first objective is to identify the embedding of a set of high-dimensional data representing quantities of interest of the computational or analytical model. For this purpose, we employ Grassmannian diffusion maps, a two-step nonlinear dimension reduction technique which allows us to reduce the dimensionality of the data and identify meaningful geometric descriptions in a parsimonious and inexpensive manner. Polynomial chaos expansion is then used to construct a map** between the stochastic input parameters and the diffusion coordinates of the reduced space. An adaptive clustering technique is proposed to identify an optimal number of clusters of points in the latent space. The similarity of points allows us to construct a number of geometric harmonic emulators which are finally utilized as a set of inexpensive pre-trained models to perform an inverse map of realizations of latent features to the ambient space and thus perform accurate out-of-sample predictions. Thus, the proposed method acts as an encoder-decoder system which is able to automatically handle very high-dimensional data while simultaneously operating successfully in the small-data regime. The method is demonstrated on two benchmark problems and on a system of advection-diffusion-reaction equations which model a first-order chemical reaction between two species. In all test cases, the proposed method is able to achieve highly accurate approximations which ultimately lead to the significant acceleration of UQ tasks. △ Less

Submitted 20 July, 2021; originally announced July 2021.

Comments: 29 pages, 14 figures

Grassmannian diffusion maps based dimension reduction and classification for high-dimensional data

Authors: K. R. M. dos Santos, D. G. Giovanis, M. D. Shields

Abstract: This work introduces the Grassmannian Diffusion Maps, a novel nonlinear dimensionality reduction technique that defines the affinity between points through their representation as low-dimensional subspaces corresponding to points on the Grassmann manifold. The method is designed for applications, such as image recognition and data-based classification of high-dimensional data that can be compactly… ▽ More This work introduces the Grassmannian Diffusion Maps, a novel nonlinear dimensionality reduction technique that defines the affinity between points through their representation as low-dimensional subspaces corresponding to points on the Grassmann manifold. The method is designed for applications, such as image recognition and data-based classification of high-dimensional data that can be compactly represented in a lower dimensional subspace. The GDMaps is composed of two stages. The first is a pointwise linear dimensionality reduction wherein each high-dimensional object is mapped onto the Grassmann. The second stage is a multi-point nonlinear kernel-based dimension reduction using Diffusion maps to identify the subspace structure of the points on the Grassmann manifold. To this aim, an appropriate Grassmannian kernel is used to construct the transition matrix of a random walk on a graph connecting points on the Grassmann manifold. Spectral analysis of the transition matrix yields low-dimensional Grassmannian diffusion coordinates embedding the data into a low-dimensional reproducing kernel Hilbert space. Further, a novel data classification/recognition technique is developed based on the construction of an overcomplete dictionary of reduced dimension whose atoms are given by the Grassmannian diffusion coordinates. Three examples are considered. First, a "toy" example shows that the GDMaps can identify an appropriate parametrization of structured points on the unit sphere. The second example demonstrates the ability of the GDMaps to reveal the intrinsic subspace structure of high-dimensional random field data. In the last example, a face recognition problem is solved considering face images subject to varying illumination conditions, changes in face expressions, and occurrence of occlusions. △ Less

Submitted 31 May, 2021; v1 submitted 16 September, 2020; originally announced September 2020.

MSC Class: 53Z50; 14M15; 60J20

Data-driven surrogates for high dimensional models using Gaussian process regression on the Grassmann manifold

Authors: Dimitris G. Giovanis, Michael D. Shields

Abstract: This paper introduces a surrogate modeling scheme based on Grassmannian manifold learning to be used for cost-efficient predictions of high-dimensional stochastic systems. The method exploits subspace-structured features of each solution by projecting it onto a Grassmann manifold. The method utilizes a solution clustering approach in order to identify regions of the parameter space over which solu… ▽ More This paper introduces a surrogate modeling scheme based on Grassmannian manifold learning to be used for cost-efficient predictions of high-dimensional stochastic systems. The method exploits subspace-structured features of each solution by projecting it onto a Grassmann manifold. The method utilizes a solution clustering approach in order to identify regions of the parameter space over which solutions are sufficiently similarly such that they can be interpolated on the Grassmannian. In this clustering, the reduced-order solutions are partitioned into disjoint clusters on the Grassmann manifold using the eigen-structure of properly defined Grassmannian kernels and, the Karcher mean of each cluster is estimated. Then, the points in each cluster are projected onto the tangent space with origin at the corresponding Karcher mean using the exponential map**. For each cluster, a Gaussian process regression model is trained that maps the input parameters of the system to the reduced solution points of the corresponding cluster projected onto the tangent space. Using this Gaussian process model, the full-field solution can be efficiently predicted at any new point in the parameter space. In certain cases, the solution clusters will span disjoint regions of the parameter space. In such cases, for each of the solution clusters we utilize a second, density-based spatial clustering to group their corresponding input parameter points in the Euclidean space. The proposed method is applied to two numerical examples. The first is a nonlinear stochastic ordinary differential equation with uncertain initial conditions. The second involves modeling of plastic deformation in a model amorphous solid using the Shear Transformation Zone theory of plasticity. △ Less

Submitted 9 April, 2020; v1 submitted 23 March, 2020; originally announced March 2020.

Comments: 31 pages, 22 Figures

Uncertainty quantification for complex systems with very high dimensional response using Grassmann manifold variations

Authors: D. G Giovanis, M. D. Shields

Abstract: This paper addresses uncertainty quantification (UQ) for problems where scalar (or low-dimensional vector) response quantities are insufficient and, instead, full-field (very high-dimensional) responses are of interest. To do so, an adaptive stochastic simulation-based methodology is introduced that refines the probability space based on Grassmann manifold variations. The proposed method has a mul… ▽ More This paper addresses uncertainty quantification (UQ) for problems where scalar (or low-dimensional vector) response quantities are insufficient and, instead, full-field (very high-dimensional) responses are of interest. To do so, an adaptive stochastic simulation-based methodology is introduced that refines the probability space based on Grassmann manifold variations. The proposed method has a multi-element character discretizing the probability space into simplex elements using a Delaunay triangulation. For every simplex, the high-dimensional solutions corresponding to its vertices (sample points) are projected onto the Grassmann manifold. The pairwise distances between these points are calculated using appropriately defined metrics and the elements with large total distance are sub-sampled and refined. As a result, regions of the probability space that produce significant changes in the full-field solution are accurately resolved. An added benefit is that an approximation of the solution within each element can be obtained by interpolation on the Grassmann manifold without the need to develop a mathematical surrogate model. The method is applied to study the probability of shear band formation in a bulk metallic glass using the shear transformation zone theory. △ Less

Submitted 27 November, 2017; originally announced November 2017.