-
Constrained or Unconstrained? Neural-Network-Based Equation Discovery from Data
Authors:
Grant Norman,
Jacqueline Wentz,
Hemanth Kolla,
Kurt Maute,
Alireza Doostan
Abstract:
Throughout many fields, practitioners often rely on differential equations to model systems. Yet, for many applications, the theoretical derivation of such equations and/or accurate resolution of their solutions may be intractable. Instead, recently developed methods, including those based on parameter estimation, operator subset selection, and neural networks, allow for the data-driven discovery…
▽ More
Throughout many fields, practitioners often rely on differential equations to model systems. Yet, for many applications, the theoretical derivation of such equations and/or accurate resolution of their solutions may be intractable. Instead, recently developed methods, including those based on parameter estimation, operator subset selection, and neural networks, allow for the data-driven discovery of both ordinary and partial differential equations (PDEs), on a spectrum of interpretability. The success of these strategies is often contingent upon the correct identification of representative equations from noisy observations of state variables and, as importantly and intertwined with that, the mathematical strategies utilized to enforce those equations. Specifically, the latter has been commonly addressed via unconstrained optimization strategies. Representing the PDE as a neural network, we propose to discover the PDE by solving a constrained optimization problem and using an intermediate state representation similar to a Physics-Informed Neural Network (PINN). The objective function of this constrained optimization problem promotes matching the data, while the constraints require that the PDE is satisfied at several spatial collocation points. We present a penalty method and a widely used trust-region barrier method to solve this constrained optimization problem, and we compare these methods on numerical examples. Our results on the Burgers' and the Korteweg-De Vreis equations demonstrate that the latter constrained method outperforms the penalty method, particularly for higher noise levels or fewer collocation points. For both methods, we solve these discovered neural network PDEs with classical methods, such as finite difference methods, as opposed to PINNs-type methods relying on automatic differentiation. We briefly highlight other small, yet crucial, implementation details.
△ Less
Submitted 29 May, 2024;
originally announced June 2024.
-
Online randomized interpolative decomposition with a posteriori error estimator for temporal PDE data reduction
Authors:
Angran Li,
Stephen Becker,
Alireza Doostan
Abstract:
Traditional low-rank approximation is a powerful tool to compress the huge data matrices that arise in simulations of partial differential equations (PDE), but suffers from high computational cost and requires several passes over the PDE data. The compressed data may also lack interpretability thus making it difficult to identify feature patterns from the original data. To address this issue, we p…
▽ More
Traditional low-rank approximation is a powerful tool to compress the huge data matrices that arise in simulations of partial differential equations (PDE), but suffers from high computational cost and requires several passes over the PDE data. The compressed data may also lack interpretability thus making it difficult to identify feature patterns from the original data. To address this issue, we present an online randomized algorithm to compute the interpolative decomposition (ID) of large-scale data matrices in situ. Compared to previous randomized IDs that used the QR decomposition to determine the column basis, we adopt a streaming ridge leverage score-based column subset selection algorithm that dynamically selects proper basis columns from the data and thus avoids an extra pass over the data to compute the coefficient matrix of the ID. In particular, we adopt a single-pass error estimator based on the non-adaptive Hutch++ algorithm to provide real-time error approximation for determining the best coefficients. As a result, our approach only needs a single pass over the original data and thus is suitable for large and high-dimensional matrices stored outside of core memory or generated in PDE simulations. We also provide numerical experiments on turbulent channel flow and ignition simulations, and on the NSTX Gas Puff Image dataset, comparing our algorithm with the offline ID algorithm to demonstrate its utility in real-world applications.
△ Less
Submitted 25 May, 2024;
originally announced May 2024.
-
Bi-fidelity Variational Auto-encoder for Uncertainty Quantification
Authors:
Nuo** Cheng,
Osman Asif Malik,
Subhayan De,
Stephen Becker,
Alireza Doostan
Abstract:
Quantifying the uncertainty of quantities of interest (QoIs) from physical systems is a primary objective in model validation. However, achieving this goal entails balancing the need for computational efficiency with the requirement for numerical accuracy. To address this trade-off, we propose a novel bi-fidelity formulation of variational auto-encoders (BF-VAE) designed to estimate the uncertaint…
▽ More
Quantifying the uncertainty of quantities of interest (QoIs) from physical systems is a primary objective in model validation. However, achieving this goal entails balancing the need for computational efficiency with the requirement for numerical accuracy. To address this trade-off, we propose a novel bi-fidelity formulation of variational auto-encoders (BF-VAE) designed to estimate the uncertainty associated with a QoI from low-fidelity (LF) and high-fidelity (HF) samples of the QoI. This model allows for the approximation of the statistics of the HF QoI by leveraging information derived from its LF counterpart. Specifically, we design a bi-fidelity auto-regressive model in the latent space that is integrated within the VAE's probabilistic encoder-decoder structure. An effective algorithm is proposed to maximize the variational lower bound of the HF log-likelihood in the presence of limited HF data, resulting in the synthesis of HF realizations with a reduced computational cost. Additionally, we introduce the concept of the bi-fidelity information bottleneck (BF-IB) to provide an information-theoretic interpretation of the proposed BF-VAE model. Our numerical results demonstrate that BF-VAE leads to considerably improved accuracy, as compared to a VAE trained using only HF data, when limited HF data is available.
△ Less
Submitted 17 October, 2023; v1 submitted 25 May, 2023;
originally announced May 2023.
-
Derivative-based SINDy (DSINDy): Addressing the challenge of discovering governing equations from noisy data
Authors:
Jacqueline Wentz,
Alireza Doostan
Abstract:
Recent advances in the field of data-driven dynamics allow for the discovery of ODE systems using state measurements. One approach, known as Sparse Identification of Nonlinear Dynamics (SINDy), assumes the dynamics are sparse within a predetermined basis in the states and finds the expansion coefficients through linear regression with sparsity constraints. This approach requires an accurate estima…
▽ More
Recent advances in the field of data-driven dynamics allow for the discovery of ODE systems using state measurements. One approach, known as Sparse Identification of Nonlinear Dynamics (SINDy), assumes the dynamics are sparse within a predetermined basis in the states and finds the expansion coefficients through linear regression with sparsity constraints. This approach requires an accurate estimation of the state time derivatives, which is not necessarily possible in the high-noise regime without additional constraints. We present an approach called Derivative-based SINDy (DSINDy) that combines two novel methods to improve ODE recovery at high noise levels. First, we denoise the state variables by applying a projection operator that leverages the assumed basis for the system dynamics. Second, we use a second order cone program (SOCP) to find the derivative and governing equations simultaneously. We derive theoretical results for the projection-based denoising step, which allow us to estimate the values of hyperparameters used in the SOCP formulation. This underlying theory helps limit the number of required user-specified parameters. We present results demonstrating that our approach leads to improved system recovery for the Van der Pol oscillator, the Duffing oscillator, and the Rössler attractor.
△ Less
Submitted 13 February, 2023; v1 submitted 10 November, 2022;
originally announced November 2022.
-
QuadConv: Quadrature-Based Convolutions with Applications to Non-Uniform PDE Data Compression
Authors:
Kevin Doherty,
Cooper Simpson,
Stephen Becker,
Alireza Doostan
Abstract:
We present a new convolution layer for deep learning architectures which we call QuadConv -- an approximation to continuous convolution via quadrature. Our operator is developed explicitly for use on non-uniform, mesh-based data, and accomplishes this by learning a continuous kernel that can be sampled at arbitrary locations. Moreover, the construction of our operator admits an efficient implement…
▽ More
We present a new convolution layer for deep learning architectures which we call QuadConv -- an approximation to continuous convolution via quadrature. Our operator is developed explicitly for use on non-uniform, mesh-based data, and accomplishes this by learning a continuous kernel that can be sampled at arbitrary locations. Moreover, the construction of our operator admits an efficient implementation which we detail and construct. As an experimental validation of our operator, we consider the task of compressing partial differential equation (PDE) simulation data from fixed meshes. We show that QuadConv can match the performance of standard discrete convolutions on uniform grid data by comparing a QuadConv autoencoder (QCAE) to a standard convolutional autoencoder (CAE). Further, we show that the QCAE can maintain this accuracy even on non-uniform data. In both cases, QuadConv also outperforms alternative unstructured convolution methods such as graph convolution.
△ Less
Submitted 28 August, 2023; v1 submitted 9 November, 2022;
originally announced November 2022.
-
Quadrature Sampling of Parametric Models with Bi-fidelity Boosting
Authors:
Nuo** Cheng,
Osman Asif Malik,
Yiming Xu,
Stephen Becker,
Alireza Doostan,
Akil Narayan
Abstract:
Least squares regression is a ubiquitous tool for building emulators (a.k.a. surrogate models) of problems across science and engineering for purposes such as design space exploration and uncertainty quantification. When the regression data are generated using an experimental design process (e.g., a quadrature grid) involving computationally expensive models, or when the data size is large, sketch…
▽ More
Least squares regression is a ubiquitous tool for building emulators (a.k.a. surrogate models) of problems across science and engineering for purposes such as design space exploration and uncertainty quantification. When the regression data are generated using an experimental design process (e.g., a quadrature grid) involving computationally expensive models, or when the data size is large, sketching techniques have shown promise to reduce the cost of the construction of the regression model while ensuring accuracy comparable to that of the full data. However, random sketching strategies, such as those based on leverage scores, lead to regression errors that are random and may exhibit large variability. To mitigate this issue, we present a novel boosting approach that leverages cheaper, lower-fidelity data of the problem at hand to identify the best sketch among a set of candidate sketches. This in turn specifies the sketch of the intended high-fidelity model and the associated data. We provide theoretical analyses of this bi-fidelity boosting (BFB) approach and discuss the conditions the low- and high-fidelity data must satisfy for a successful boosting. In doing so, we derive a bound on the residual norm of the BFB sketched solution relating it to its ideal, but computationally expensive, high-fidelity boosted counterpart. Empirical results on both manufactured and PDE data corroborate the theoretical analyses and illustrate the efficacy of the BFB solution in reducing the regression error, as compared to the non-boosted solution.
△ Less
Submitted 12 September, 2022;
originally announced September 2022.
-
Fast Algorithms for Monotone Lower Subsets of Kronecker Least Squares Problems
Authors:
Osman Asif Malik,
Yiming Xu,
Nuo** Cheng,
Stephen Becker,
Alireza Doostan,
Akil Narayan
Abstract:
Approximate solutions to large least squares problems can be computed efficiently using leverage score-based row-sketches, but directly computing the leverage scores, or sampling according to them with naive methods, still requires an expensive manipulation and processing of the design matrix. In this paper we develop efficient leverage score-based sampling methods for matrices with certain Kronec…
▽ More
Approximate solutions to large least squares problems can be computed efficiently using leverage score-based row-sketches, but directly computing the leverage scores, or sampling according to them with naive methods, still requires an expensive manipulation and processing of the design matrix. In this paper we develop efficient leverage score-based sampling methods for matrices with certain Kronecker product-type structure; in particular we consider matrices that are monotone lower column subsets of Kronecker product matrices. Our discussion is general, encompassing least squares problems on infinite domains, in which case matrices formally have infinitely many rows. We briefly survey leverage score-based sampling guarantees from the numerical linear algebra and approximation theory communities, and follow this with efficient algorithms for sampling when the design matrix has Kronecker-type structure. Our numerical examples confirm that sketches based on exact leverage score sampling for our class of structured matrices achieve superior residual compared to approximate leverage score sampling methods.
△ Less
Submitted 12 September, 2022;
originally announced September 2022.
-
Simultaneous Identification and Denoising of Dynamical Systems
Authors:
Jeffrey M. Hokanson,
Gianluca Iaccarino,
Alireza Doostan
Abstract:
In recent years there has been a push to discover the governing equations dynamical systems directly from measurements of the state, often motivated by systems that are too complex to directly model. Although there has been substantial work put into such a discovery, doing so in the case of large noise has proved challenging. Here we develop an algorithm for Simultaneous Identification and Denoisi…
▽ More
In recent years there has been a push to discover the governing equations dynamical systems directly from measurements of the state, often motivated by systems that are too complex to directly model. Although there has been substantial work put into such a discovery, doing so in the case of large noise has proved challenging. Here we develop an algorithm for Simultaneous Identification and Denoising of a Dynamical System (SIDDS). We infer the noise in the state measurements by requiring that the denoised data satisfies the dynamical system with an equality constraint. This is unlike existing work where the mismatch in the dynamical system is treated as a penalty in the objective. We assume the dynamics is represented in a pre-defined basis and develop a sequential quadratic programming approach to solve the SIDDS problem featuring a direct solution of KKT system with a specialized preconditioner. In addition, we show how we can include sparsity promoting regularization using an iteratively reweighted least squares approach. The resulting algorithm leads to estimates of the dynamical system that approximately achieve the Cramér-Rao lower bound and, with sparsity promotion, can correctly identify the sparsity structure for higher levels of noise than existing techniques. Moreover, because SIDDS decouples the data from the evolution of the dynamical system, we show how to modify the problem to accurately identify systems from low sample rate measurements. The inverse problem approach and solution framework used by SIDDS has the potential to be expanded to related problems identifying governing equations from noisy data.
△ Less
Submitted 9 January, 2023; v1 submitted 25 March, 2022;
originally announced March 2022.
-
Topology Optimization under Microscale Uncertainty using Stochastic Gradients
Authors:
Subhayan De,
Kurt Maute,
Alireza Doostan
Abstract:
This paper considers the design of structures made of engineered materials, accounting for uncertainty in material properties. We present a topology optimization approach that optimizes the structural shape and topology at the macroscale assuming design-independent uncertain microstructures. The structural geometry at the macroscale is described by an explicit level set approach, and the macroscop…
▽ More
This paper considers the design of structures made of engineered materials, accounting for uncertainty in material properties. We present a topology optimization approach that optimizes the structural shape and topology at the macroscale assuming design-independent uncertain microstructures. The structural geometry at the macroscale is described by an explicit level set approach, and the macroscopic structural response is predicted by the eXtended Finite Element Method (XFEM). We describe the microscopic layout by either an analytic geometric model with uncertain parameters or a level cut from a Gaussian random field. The macroscale properties of the microstructured material are predicted by homogenization. Considering the large number of possible microscale configurations, one of the main challenges of solving such topology optimization problems is the computational cost of estimating the statistical moments of the cost and constraint functions and their gradients with respect to the design variables. Methods for predicting these moments, such as Monte Carlo sampling, and Taylor series and polynomial chaos expansions often require many random samples resulting in an impractical computation. To reduce this cost, we propose an approach wherein, at every design iteration, we only use a small number of microstructure configurations to generate an independent, stochastic approximation of the gradients. These gradients are then used either with a gradient descent algorithm, namely Adam, or the globally convergent method of moving asymptotes (GCMMA). Three numerical examples from structural mechanics are used to show that the proposed approach provides a computationally efficient way for macroscale topology optimization in the presence of microstructural uncertainty and enables the designers to consider a new class of problems that are out of reach today with conventional tools.
△ Less
Submitted 25 October, 2021;
originally announced October 2021.
-
Deterministic matrix sketches for low-rank compression of high-dimensional simulation data
Authors:
Alec Michael Dunton,
Alireza Doostan
Abstract:
Matrices arising in scientific applications frequently admit linear low-rank approximations due to smoothness in the physical and/or temporal domain of the problem. In large-scale problems, computing an optimal low-rank approximation can be prohibitively expensive. Matrix sketching addresses this by reducing the input matrix to a smaller, but representative matrix via a low-dimensional linear embe…
▽ More
Matrices arising in scientific applications frequently admit linear low-rank approximations due to smoothness in the physical and/or temporal domain of the problem. In large-scale problems, computing an optimal low-rank approximation can be prohibitively expensive. Matrix sketching addresses this by reducing the input matrix to a smaller, but representative matrix via a low-dimensional linear embedding. If the sketch matrix produced by the embedding captures sufficient geometric properties of the original matrix, then a near-optimal approximation may be obtained. Much of the work done in matrix sketching has centered on random projection. Alternatively, in this work, deterministic matrix sketches which generate coarse representations, compatible with the corresponding PDE solve, are considered in the computation of the singular value decomposition and matrix interpolative decomposition. The deterministic sketching approaches in this work have many advantages over randomized sketches. Broadly, randomized sketches are data-agnostic, whereas the proposed sketching methods exploit structures within data generated in complex PDE systems. These deterministic sketches are often faster, require access to a small fraction of the input matrix, and do not need to be explicitly constructed. A novel single-pass, i.e., requiring one read over the input, power iteration algorithm is also presented. The power iteration method is particularly effective in improving low-rank approximations when the singular value decay of data is slow. Finally, theoretical error bounds and estimates, as well as numerical results across three application problems, are provided.
△ Less
Submitted 3 May, 2021;
originally announced May 2021.
-
Rapid Aerodynamic Shape Optimization Under Parametric and Turbulence Model Uncertainty: A Stochastic Gradient Approach
Authors:
Lluís Jofre,
Alireza Doostan
Abstract:
Aerodynamic optimization is ubiquitous in the design of most engineering systems interacting with fluids. A common approach is to optimize a performance function defined by a choice of an aerodynamic model, e.g., turbulence RANS model, and at nominal operating conditions. Practical experience indicates that such a deterministic approach may result in considerably sub-optimal designs when the adopt…
▽ More
Aerodynamic optimization is ubiquitous in the design of most engineering systems interacting with fluids. A common approach is to optimize a performance function defined by a choice of an aerodynamic model, e.g., turbulence RANS model, and at nominal operating conditions. Practical experience indicates that such a deterministic approach may result in considerably sub-optimal designs when the adopted aerodynamic model does not lead to accurate flow predictions or when the actual operating conditions differ from those considered in the design. One approach to address this shortcoming is to consider an average or robust design, wherein the statistical moments of the performance function, given the uncertainty in the operating conditions and the aerodynamic model, is optimized. However, when the number of uncertain inputs is large or the performance function exhibits significant variability, an accurate evaluation of these moments may require a large number of forward and/or adjoint solves, at each iteration of a gradient-based scheme. This, in turn, renders the design computationally expensive, if not infeasible. To tackle this difficulty, we consider a variant of the stochastic gradient descent method where, in each optimization iteration, a stochastic approximation of the objective, constraints, and their gradients are generated. This is done via a small number of forward/adjoint solves corresponding to random selections of the uncertain parameters and aerodynamic model. The methodology is applied to the robust optimization of the standard NACA-0012 subject to parametric and turbulence model uncertainty. With a cost that is a small factor larger than that of the deterministic approach, the stochastic gradient approach significantly improves the performance (mean and variance) of the aerodynamic design for a wide range of operating conditions and turbulence models.
△ Less
Submitted 3 May, 2021;
originally announced May 2021.
-
Bi-fidelity Reduced Polynomial Chaos Expansion for Uncertainty Quantification
Authors:
Felix Newberry,
Jerrad Hampton,
Kenneth Jansen,
Alireza Doostan
Abstract:
A ubiquitous challenge in design space exploration or uncertainty quantification of complex engineering problems is the minimization of computational cost. A useful tool to ease the burden of solving such systems is model reduction. This work considers a stochastic model reduction method (SMR), in the context of polynomial chaos (PC) expansions, where low-fidelity (LF) samples are leveraged to for…
▽ More
A ubiquitous challenge in design space exploration or uncertainty quantification of complex engineering problems is the minimization of computational cost. A useful tool to ease the burden of solving such systems is model reduction. This work considers a stochastic model reduction method (SMR), in the context of polynomial chaos (PC) expansions, where low-fidelity (LF) samples are leveraged to form a stochastic reduced basis. The reduced basis enables the construction of a bi-fidelity (BF) estimate of a quantity of interest from a small number of high-fidelity (HF) samples. A successful BF estimate approximates the quantity of interest with accuracy comparable to the HF model and computational expense close to the LF model. We develop new error bounds for the SMR approach and present a procedure to practically utilize these bounds in order to assess the appropriateness of a given pair of LF and HF models for BF estimation. The effectiveness of the SMR approach, and the utility of the error bound are presented in three numerical examples.
△ Less
Submitted 15 April, 2021;
originally announced April 2021.
-
Reliability-based Topology Optimization using Stochastic Gradients
Authors:
Subhayan De,
Kurt Maute,
Alireza Doostan
Abstract:
This paper addresses the computational challenges in reliability-based topology optimization (RBTO) of structures associated with the estimation of statistics of the objective and constraints using standard sampling methods, and overcomes the accuracy issues of traditional methods that rely on approximating the limit state function. Herein, we present a stochastic gradient-based approach, where we…
▽ More
This paper addresses the computational challenges in reliability-based topology optimization (RBTO) of structures associated with the estimation of statistics of the objective and constraints using standard sampling methods, and overcomes the accuracy issues of traditional methods that rely on approximating the limit state function. Herein, we present a stochastic gradient-based approach, where we estimate the probability of failure at every few iterations using an efficient sampling strategy. To estimate the gradients of the failure probability with respect to the design parameters, we apply Bayes' rule wherein we assume a parametric exponential model for the probability density function of the design parameters conditioned on the failure. The design parameters as well as the parameters of this probability density function are updated using a stochastic gradient descent approach requiring only a small, e.g., O(1), number of random samples per iteration, thus leading to a considerable reduction of the computational cost as compared to standard RBTO techniques. We illustrate the proposed approach with a benchmark example that has an analytical solution as well as two widely used problems in structural topology optimization. These examples illustrate the efficacy of the approach in producing reliable designs.
△ Less
Submitted 25 July, 2021; v1 submitted 15 March, 2021;
originally announced March 2021.
-
A stochastic subspace approach to gradient-free optimization in high dimensions
Authors:
David Kozak,
Stephen Becker,
Alireza Doostan,
Luis Tenorio
Abstract:
We present a stochastic descent algorithm for unconstrained optimization that is particularly efficient when the objective function is slow to evaluate and gradients are not easily obtained, as in some PDE-constrained optimization and machine learning problems. The algorithm maps the gradient onto a low-dimensional random subspace of dimension $\ell$ at each iteration, similar to coordinate descen…
▽ More
We present a stochastic descent algorithm for unconstrained optimization that is particularly efficient when the objective function is slow to evaluate and gradients are not easily obtained, as in some PDE-constrained optimization and machine learning problems. The algorithm maps the gradient onto a low-dimensional random subspace of dimension $\ell$ at each iteration, similar to coordinate descent but without restricting directional derivatives to be along the axes. Without requiring a full gradient, this map** can be performed by computing $\ell$ directional derivatives (e.g., via forward-mode automatic differentiation). We give proofs for convergence in expectation under various convexity assumptions as well as probabilistic convergence results under strong-convexity. Our method extends the well-known Gaussian smoothing technique to descent in subspaces of dimension greater than one, opening the doors to new analysis of Gaussian smoothing when more than one directional derivative is used at each iteration. We also provide a finite-dimensional variant of a special case of the Johnson-Lindenstrauss lemma. Experimentally, we show that our method compares favorably to coordinate descent, Gaussian smoothing, gradient descent and BFGS (when gradients are calculated via forward-mode automatic differentiation) on problems from the machine learning and shape optimization literature.
△ Less
Submitted 6 May, 2021; v1 submitted 4 March, 2020;
originally announced March 2020.
-
Bi-fidelity Stochastic Gradient Descent for Structural Optimization under Uncertainty
Authors:
Subhayan De,
Kurt Maute,
Alireza Doostan
Abstract:
The presence of uncertainty in material properties and geometry of a structure is ubiquitous. The design of robust engineering structures, therefore, needs to incorporate uncertainty in the optimization process. Stochastic gradient descent (SGD) method can alleviate the cost of optimization under uncertainty, which includes statistical moments of quantities of interest in the objective and constra…
▽ More
The presence of uncertainty in material properties and geometry of a structure is ubiquitous. The design of robust engineering structures, therefore, needs to incorporate uncertainty in the optimization process. Stochastic gradient descent (SGD) method can alleviate the cost of optimization under uncertainty, which includes statistical moments of quantities of interest in the objective and constraints. However, the design may change considerably during the initial iterations of the optimization process which impedes the convergence of the traditional SGD method and its variants. In this paper, we present two SGD based algorithms, where the computational cost is reduced by employing a low-fidelity model in the optimization process. In the first algorithm, most of the stochastic gradient calculations are performed on the low-fidelity model and only a handful of gradients from the high-fidelity model are used per iteration, resulting in an improved convergence. In the second algorithm, we use gradients from low-fidelity models to be used as control variate, a variance reduction technique, to reduce the variance in the search direction. These two bi-fidelity algorithms are illustrated first with a conceptual example. Then, the convergence of the proposed bi-fidelity algorithms is studied with two numerical examples of shape and topology optimization and compared to popular variants of the SGD method that do not use low-fidelity models. The results show that the proposed use of a bi-fidelity approach for the SGD method can improve the convergence. Two analytical proofs are also provided that show the linear convergence of these two algorithms under appropriate assumptions.
△ Less
Submitted 23 November, 2019;
originally announced November 2019.
-
Pass-efficient methods for compression of high-dimensional turbulent flow data
Authors:
Alec M. Dunton,
Lluís Jofre,
Gianluca Iaccarino,
Alireza Doostan
Abstract:
The future of high-performance computing, specifically on future Exascale computers, will presumably see memory capacity and bandwidth fail to keep pace with data generated, for instance, from massively parallel partial differential equation (PDE) systems. Current strategies proposed to address this bottleneck entail the omission of large fractions of data, as well as the incorporation of…
▽ More
The future of high-performance computing, specifically on future Exascale computers, will presumably see memory capacity and bandwidth fail to keep pace with data generated, for instance, from massively parallel partial differential equation (PDE) systems. Current strategies proposed to address this bottleneck entail the omission of large fractions of data, as well as the incorporation of $\textit{in situ}$ compression algorithms to avoid overuse of memory. To ensure that post-processing operations are successful, this must be done in a way that a sufficiently accurate representation of the solution is stored. Moreover, in situations where the input/output system becomes a bottleneck in analysis, visualization, etc., or the execution of the PDE solver is expensive, the the number of passes made over the data must be minimized. In the interest of addressing this problem, this work focuses on the utility of pass-efficient, parallelizable, low-rank, matrix decomposition methods in compressing high-dimensional simulation data from turbulent flows. A particular emphasis is placed on using coarse representation of the data -- compatible with the PDE discretization grid -- to accelerate the construction of the low-rank factorization. This includes the presentation of a novel single-pass matrix decomposition algorithm for computing the so-called interpolative decomposition. The methods are described extensively and numerical experiments on two turbulent channel flow data are performed. In the first (unladen) channel flow case, compression factors exceeding $400$ are achieved while maintaining accuracy with respect to first- and second-order flow statistics. In the particle-laden case, compression factors of 100 are achieved and the compressed data is used to recover particle velocities.
△ Less
Submitted 28 January, 2020; v1 submitted 30 May, 2019;
originally announced May 2019.
-
Physics-Informed Tolerance Allocation: A Surrogate-Based Framework for the Control of Geometric Variation on System Performance
Authors:
Joseph Benzaken,
Alireza Doostan,
John A. Evans
Abstract:
In this paper, we present a novel tolerance allocation algorithm for the assessment and control of geometric variation on system performance that is applicable to any system of partial differential equations. In particular, we parameterize the geometric domain of the system in terms of design parameters and subsequently measure the effect of design parameter variation on system performance. A surr…
▽ More
In this paper, we present a novel tolerance allocation algorithm for the assessment and control of geometric variation on system performance that is applicable to any system of partial differential equations. In particular, we parameterize the geometric domain of the system in terms of design parameters and subsequently measure the effect of design parameter variation on system performance. A surrogate model via a tensor representation is constructed to map the design parameter variation to the system performance. A set of optimization problems over this surrogate model restricted to nested hyperrectangles represents the effect of prescribing design tolerances, where the maximizer of this restricted function depicts the worst-case member, i.e. the worst-case design. Moreover, the loci of these tolerance hyperrectangles with maximizers attaining, but not surpassing, the performance constraint represents the boundary to the feasible region of allocatable tolerances. Every tolerance in this domain is measured through a user-specified, weighted norm which is informed by design considerations such as cost and manufacturability. The boundary of the feasible set is elucidated as an immersed manifold of codimension one, over which a suite of optimization routines exist and are employed to efficiently determine an optimal feasible tolerance with respect to the specified measure. Examples of this algorithm are presented with applications to a plate with a hole described by two design parameters, a plate with a hole described by six design parameters, and an L-Bracket described by seventeen design parameters.
△ Less
Submitted 13 April, 2019;
originally announced April 2019.
-
Stochastic Subspace Descent
Authors:
David Kozak,
Stephen Becker,
Alireza Doostan,
Luis Tenorio
Abstract:
We present two stochastic descent algorithms that apply to unconstrained optimization and are particularly efficient when the objective function is slow to evaluate and gradients are not easily obtained, as in some PDE-constrained optimization and machine learning problems. The basic algorithm projects the gradient onto a random subspace at each iteration, similar to coordinate descent but without…
▽ More
We present two stochastic descent algorithms that apply to unconstrained optimization and are particularly efficient when the objective function is slow to evaluate and gradients are not easily obtained, as in some PDE-constrained optimization and machine learning problems. The basic algorithm projects the gradient onto a random subspace at each iteration, similar to coordinate descent but without restricting directional derivatives to be along the axes. This algorithm is previously known but we provide new analysis. We also extend the popular SVRG method to this framework but without requiring that the objective function be written as a finite sum. We provide proofs of convergence for our methods under various convexity assumptions and show favorable results when compared to gradient descent and BFGS on non-convex problems from the machine learning and shape optimization literature. We also note that our analysis gives a proof that the iterates of SVRG and several other popular first-order stochastic methods, in their original formulation, converge almost surely to the optimum; to our knowledge, prior to this work the iterates of SVRG had only been known to converge in expectation.
△ Less
Submitted 29 April, 2019; v1 submitted 1 April, 2019;
originally announced April 2019.
-
Topology Optimization under Uncertainty using a Stochastic Gradient-based Approach
Authors:
Subhayan De,
Jerrad Hampton,
Kurt Maute,
Alireza Doostan
Abstract:
Topology optimization under uncertainty (TOuU) often defines objectives and constraints by statistical moments of geometric and physical quantities of interest. Most traditional TOuU methods use gradient-based optimization algorithms and rely on accurate estimates of the statistical moments and their gradients, e.g., via adjoint calculations. When the number of uncertain inputs is large or the qua…
▽ More
Topology optimization under uncertainty (TOuU) often defines objectives and constraints by statistical moments of geometric and physical quantities of interest. Most traditional TOuU methods use gradient-based optimization algorithms and rely on accurate estimates of the statistical moments and their gradients, e.g., via adjoint calculations. When the number of uncertain inputs is large or the quantities of interest exhibit large variability, a large number of adjoint (and/or forward) solves may be required to ensure the accuracy of these gradients. The optimization procedure itself often requires a large number of iterations, which may render TOuU computationally expensive, if not infeasible. To tackle this difficulty, we here propose an optimization approach that generates a stochastic approximation of the objective, constraints, and their gradients via a small number of adjoint (and/or forward) solves, per iteration. A statistically independent (stochastic) approximation of these quantities is generated at each optimization iteration. The total cost of this approach is only a small factor larger than that of the corresponding deterministic TO problem. We incorporate the stochastic approximation of objective, constraints and their design sensitivities into two classes of optimization algorithms. First, we investigate the stochastic gradient descent (SGD) method and a number of its variants, which have been successfully applied to large-scale optimization problems for machine learning. Second, we study the use of the proposed stochastic approximation approach within conventional nonlinear programming methods, focusing on the Globally Convergent Method of Moving Asymptotes (GCMMA). The performance of these algorithms is investigated with structural design optimization problems utilizing a Solid Isotropic Material with Penalization (SIMP), as well as an explicit level set method.
△ Less
Submitted 1 November, 2019; v1 submitted 11 February, 2019;
originally announced February 2019.
-
Level Set Methods for Stochastic Discontinuity Detection in Nonlinear Problems
Authors:
Per Pettersson,
Alireza Doostan,
Jan Nordström
Abstract:
Stochastic physical problems governed by nonlinear conservation laws are challenging due to solution discontinuities in stochastic and physical space. In this paper, we present a level set method to track discontinuities in stochastic space by solving a Hamilton-Jacobi equation. By introducing a speed function that vanishes at discontinuities, the iso-zero of the level set problem coincide with th…
▽ More
Stochastic physical problems governed by nonlinear conservation laws are challenging due to solution discontinuities in stochastic and physical space. In this paper, we present a level set method to track discontinuities in stochastic space by solving a Hamilton-Jacobi equation. By introducing a speed function that vanishes at discontinuities, the iso-zero of the level set problem coincide with the discontinuities of the conservation law. The level set problem is solved on a sequence of successively finer grids in stochastic space. The method is adaptive in the sense that costly evaluations of the conservation law of interest are only performed in the vicinity of the discontinuities during the refinement stage. In regions of stochastic space where the solution is smooth, a surrogate method replaces expensive evaluations of the conservation law. The proposed method is tested in conjunction with different sets of localized orthogonal basis functions on simplex elements, as well as frames based on piecewise polynomials conforming to the level set function. The performance of the proposed method is compared to existing adaptive multi-element generalized polynomial chaos methods.
△ Less
Submitted 8 January, 2019; v1 submitted 16 October, 2018;
originally announced October 2018.
-
Bi-fidelity approximation for uncertainty quantification and sensitivity analysis of irradiated particle-laden turbulence
Authors:
Hillary R. Fairbanks,
Lluis Jofre,
Gianluca Geraci,
Gianluca Iaccarino,
Alireza Doostan
Abstract:
Efficiently performing predictive studies of irradiated particle-laden turbulent flows has the potential of providing significant contributions towards better understanding and optimizing, for example, concentrated solar power systems. As there are many uncertainties inherent in such flows, uncertainty quantification is fundamental to improve the predictive capabilities of the numerical simulation…
▽ More
Efficiently performing predictive studies of irradiated particle-laden turbulent flows has the potential of providing significant contributions towards better understanding and optimizing, for example, concentrated solar power systems. As there are many uncertainties inherent in such flows, uncertainty quantification is fundamental to improve the predictive capabilities of the numerical simulations. For large-scale, multi-physics problems exhibiting high-dimensional uncertainty, characterizing the stochastic solution presents a significant computational challenge as many methods require a large number of high-fidelity solves. This requirement results in the need for a possibly infeasible number of simulations when a typical converged high-fidelity simulation requires intensive computational resources. To reduce the cost of quantifying high-dimensional uncertainties, we investigate the application of a non-intrusive, bi-fidelity approximation to estimate statistics of quantities of interest associated with an irradiated particle-laden turbulent flow. This method relies on exploiting the low-rank structure of the solution to accelerate the stochastic sampling and approximation processes by means of cheaper-to-run, lower fidelity representations. The application of this bi-fidelity approximation results in accurate estimates of the QoI statistics while requiring a small number of high-fidelity model evaluations.
△ Less
Submitted 16 August, 2018;
originally announced August 2018.
-
Sparse Polynomial Chaos Expansions via Compressed Sensing and D-optimal Design
Authors:
Paul Diaz,
Alireza Doostan,
Jerrad Hampton
Abstract:
In the field of uncertainty quantification, sparse polynomial chaos (PC) expansions are commonly used by researchers for a variety of purposes, such as surrogate modeling. Ideas from compressed sensing may be employed to exploit this sparsity in order to reduce computational costs. A class of greedy compressed sensing algorithms use least squares minimization to approximate PC coefficients. This l…
▽ More
In the field of uncertainty quantification, sparse polynomial chaos (PC) expansions are commonly used by researchers for a variety of purposes, such as surrogate modeling. Ideas from compressed sensing may be employed to exploit this sparsity in order to reduce computational costs. A class of greedy compressed sensing algorithms use least squares minimization to approximate PC coefficients. This least squares problem lends itself to the theory of optimal design of experiments (ODE). Our work focuses on selecting an experimental design that improves the accuracy of sparse PC approximations for a fixed computational budget. We propose DSP, a novel sequential design, greedy algorithm for sparse PC approximation. The algorithm sequentially augments an experimental design according to a set of the basis polynomials deemed important by the magnitude of their coefficients, at each iteration. Our algorithm incorporates topics from ODE to estimate the PC coefficients. A variety of numerical simulations are performed on three physical models and manufactured sparse PC expansions to provide a comparative study between our proposed algorithm and other non-adaptive methods. Further, we examine the importance of sampling by comparing different strategies in terms of their ability to generate a candidate pool from which an optimal experimental design is chosen. It is demonstrated that the most accurate PC coefficient approximations, with the least variability, are produced with our design-adaptive greedy algorithm and the use of a studied importance sampling strategy. We provide theoretical and numerical results which show that using an optimal sampling strategy for the candidate pool is key, both in terms of accuracy in the approximation, but also in terms of constructing an optimal design.
△ Less
Submitted 29 December, 2017;
originally announced December 2017.
-
Parametric/Stochastic Model Reduction: Low-Rank Representation, Non-Intrusive Bi-Fidelity Approximation, and Convergence Analysis
Authors:
Jerrad Hampton,
Hillary Fairbanks,
Akil Narayan,
Alireza Doostan
Abstract:
For practical model-based demands, such as design space exploration and uncertainty quantification (UQ), a high-fidelity model that produces accurate outputs often has high computational cost, while a low-fidelity model with less accurate outputs has low computational cost. It is often possible to construct a bi-fidelity model having accuracy comparable with the high-fidelity model and computation…
▽ More
For practical model-based demands, such as design space exploration and uncertainty quantification (UQ), a high-fidelity model that produces accurate outputs often has high computational cost, while a low-fidelity model with less accurate outputs has low computational cost. It is often possible to construct a bi-fidelity model having accuracy comparable with the high-fidelity model and computational cost comparable with the low-fidelity model. This work presents the construction and analysis of a non-intrusive (i.e., sample-based) bi-fidelity model that relies on the low-rank structure of the map between model parameters/uncertain inputs and the solution of interest, if exists. Specifically, we derive a novel, pragmatic estimate for the error committed by this bi-fidelity model. We show that this error bound can be used to determine if a given pair of low- and high-fidelity models will lead to an accurate bi-fidelity approximation. The cost of this error bound is relatively small and depends on the solution rank. The value of this error estimate is demonstrated using two example problems in the context of UQ, involving linear and non-linear partial differential equations.
△ Less
Submitted 11 September, 2017;
originally announced September 2017.
-
Least Squares Polynomial Chaos Expansion: A Review of Sampling Strategies
Authors:
Mohammad Hadigol,
Alireza Doostan
Abstract:
As non-institutive polynomial chaos expansion (PCE) techniques have gained growing popularity among researchers, we here provide a comprehensive review of major sampling strategies for the least squares based PCE. Traditional sampling methods, such as Monte Carlo, Latin hypercube, quasi-Monte Carlo, optimal design of experiments (ODE), Gaussian quadratures, as well as more recent techniques, such…
▽ More
As non-institutive polynomial chaos expansion (PCE) techniques have gained growing popularity among researchers, we here provide a comprehensive review of major sampling strategies for the least squares based PCE. Traditional sampling methods, such as Monte Carlo, Latin hypercube, quasi-Monte Carlo, optimal design of experiments (ODE), Gaussian quadratures, as well as more recent techniques, such as coherence-optimal and randomized quadratures are discussed. We also propose a hybrid sampling method, dubbed alphabetic-coherence-optimal, that employs the so-called alphabetic optimality criteria used in the context of ODE in conjunction with coherence-optimal samples. A comparison between the empirical performance of the selected sampling methods applied to three numerical examples, including high-order PCE's, high-dimensional problems, and low oversampling ratios, is presented to provide a road map for practitioners seeking the most suitable sampling technique for a problem at hand. We observed that the alphabetic-coherence-optimal technique outperforms other sampling methods, specially when high-order ODE are employed and/or the oversampling ratio is low.
△ Less
Submitted 23 June, 2017;
originally announced June 2017.
-
Basis Adaptive Sample Efficient Polynomial Chaos (BASE-PC)
Authors:
Jerrad Hampton,
Alireza Doostan
Abstract:
For a large class of orthogonal basis functions, there has been a recent identification of expansion methods for computing accurate, stable approximations of a quantity of interest. This paper presents, within the context of uncertainty quantification, a practical implementation using basis adaptation, and coherence motivated sampling, which under assumptions has satisfying guarantees. This implem…
▽ More
For a large class of orthogonal basis functions, there has been a recent identification of expansion methods for computing accurate, stable approximations of a quantity of interest. This paper presents, within the context of uncertainty quantification, a practical implementation using basis adaptation, and coherence motivated sampling, which under assumptions has satisfying guarantees. This implementation is referred to as Basis Adaptive Sample Efficient Polynomial Chaos (BASE-PC). A key component of this is the use of anisotropic polynomial order which admits evolving global bases for approximation in an efficient manner, leading to consistently stable approximation for a practical class of smooth functionals. This fully adaptive, non-intrusive method, requires no a priori information of the solution, and has satisfying theoretical guarantees of recovery. A key contribution to stability is the use of a presented correction sampling for coherence-optimal sampling in order to improve stability and accuracy within the adaptive basis scheme. Theoretically, the method may dramatically reduce the impact of dimensionality in function approximation, and numerically the method is demonstrated to perform well on problems with dimension up to 1000.
△ Less
Submitted 3 February, 2017;
originally announced February 2017.
-
Time-dependent global sensitivity analysis with active subspaces for a lithium ion battery model
Authors:
Paul G. Constantine,
Alireza Doostan
Abstract:
Renewable energy researchers use computer simulation to aid the design of lithium ion storage devices. The underlying models contain several physical input parameters that affect model predictions. Effective design and analysis must understand the sensitivity of model predictions to changes in model parameters, but global sensitivity analyses become increasingly challenging as the number of input…
▽ More
Renewable energy researchers use computer simulation to aid the design of lithium ion storage devices. The underlying models contain several physical input parameters that affect model predictions. Effective design and analysis must understand the sensitivity of model predictions to changes in model parameters, but global sensitivity analyses become increasingly challenging as the number of input parameters increases. Active subspaces are part of an emerging set of tools for discovering and exploiting low-dimensional structures in the map from high-dimensional inputs to model outputs. We extend linear and quadratic model-based heuristic for active sub- space discovery to time-dependent processes and apply the resulting technique to a lithium ion battery model. The results reveal low-dimensional structure and sensitivity metrics that a designer may exploit to study the relationship between parameters and predictions.
△ Less
Submitted 1 March, 2017; v1 submitted 27 June, 2016;
originally announced June 2016.
-
Optimization via Separated Representations and the Canonical Tensor Decomposition
Authors:
Matthew J Reynolds,
Gregory Beylkin,
Alireza Doostan
Abstract:
We introduce a new, quadratically convergent algorithm for finding maximum absolute value entries of tensors represented in the canonical format. The computational complexity of the algorithm is linear in the dimension of the tensor. We show how to use this algorithm to find global maxima of non-convex multivariate functions in separated form. We demonstrate the performance of the new algorithms o…
▽ More
We introduce a new, quadratically convergent algorithm for finding maximum absolute value entries of tensors represented in the canonical format. The computational complexity of the algorithm is linear in the dimension of the tensor. We show how to use this algorithm to find global maxima of non-convex multivariate functions in separated form. We demonstrate the performance of the new algorithms on several examples.
△ Less
Submitted 18 May, 2016;
originally announced May 2016.
-
Randomized Alternating Least Squares for Canonical Tensor Decompositions: Application to a PDE with Random Data
Authors:
Matthew Reynolds,
Alireza Doostan,
Gregory Beylkin
Abstract:
This paper introduces a randomized variation of the alternating least squares (ALS) algorithm for rank reduction of canonical tensor formats. The aim is to address the potential numerical ill-conditioning of least squares matrices at each ALS iteration. The proposed algorithm, dubbed randomized ALS, mitigates large condition numbers via projections onto random tensors, a technique inspired by well…
▽ More
This paper introduces a randomized variation of the alternating least squares (ALS) algorithm for rank reduction of canonical tensor formats. The aim is to address the potential numerical ill-conditioning of least squares matrices at each ALS iteration. The proposed algorithm, dubbed randomized ALS, mitigates large condition numbers via projections onto random tensors, a technique inspired by well-established randomized projection methods for solving overdetermined least squares problems in a matrix setting. A probabilistic bound on the condition numbers of the randomized ALS matrices is provided, demonstrating reductions relative to their standard counterparts. Additionally, results are provided that guarantee comparable accuracy of the randomized ALS solution at each iteration. The performance of the randomized algorithm is studied with three examples, including manufactured tensors and an elliptic PDE with random inputs. In particular, for the latter, tests illustrate not only improvements in condition numbers, but also improved accuracy of the iterative solver for the PDE solution represented in a canonical tensor format.
△ Less
Submitted 5 October, 2015;
originally announced October 2015.
-
On Polynomial Chaos Expansion via Gradient-enhanced $\ell_1$-minimization
Authors:
Ji Peng,
Jerrad Hampton,
Alireza Doostan
Abstract:
Gradient-enhanced Uncertainty Quantification (UQ) has received recent attention, in which the derivatives of a Quantity of Interest (QoI) with respect to the uncertain parameters are utilized to improve the surrogate approximation. Polynomial chaos expansions (PCEs) are often employed in UQ, and when the QoI can be represented by a sparse PCE, $\ell_1$-minimization can identify the PCE coefficient…
▽ More
Gradient-enhanced Uncertainty Quantification (UQ) has received recent attention, in which the derivatives of a Quantity of Interest (QoI) with respect to the uncertain parameters are utilized to improve the surrogate approximation. Polynomial chaos expansions (PCEs) are often employed in UQ, and when the QoI can be represented by a sparse PCE, $\ell_1$-minimization can identify the PCE coefficients with a relatively small number of samples. In this work, we investigate a gradient-enhanced $\ell_1$-minimization, where derivative information is computed to accelerate the identification of the PCE coefficients. For this approach, stability and convergence analysis are lacking, and thus we address these here with a probabilistic result. In particular, with an appropriate normalization, we show the inclusion of derivative information will almost-surely lead to improved conditions, e.g. related to the null-space and coherence of the measurement matrix, for a successful solution recovery. Further, we demonstrate our analysis empirically via three numerical examples: a manufactured PCE, an elliptic partial differential equation with random inputs, and a plane Poiseuille flow with random boundaries. These examples all suggest that including derivative information admits solution recovery at reduced computational cost.
△ Less
Submitted 1 June, 2015;
originally announced June 2015.
-
On Uncertainty Quantification of Lithium-ion Batteries: Application to an LiC$_6$/LiCoO$_2$ cell
Authors:
Mohammad Hadigol,
Kurt Maute,
Alireza Doostan
Abstract:
In this work, a stochastic, physics-based model for Lithium-ion batteries (LIBs) is presented in order to study the effects of parametric model uncertainties on the cell capacity, voltage, and concentrations. To this end, the proposed uncertainty quantification (UQ) approach, based on sparse polynomial chaos expansions, relies on a small number of battery simulations. Within this UQ framework, the…
▽ More
In this work, a stochastic, physics-based model for Lithium-ion batteries (LIBs) is presented in order to study the effects of parametric model uncertainties on the cell capacity, voltage, and concentrations. To this end, the proposed uncertainty quantification (UQ) approach, based on sparse polynomial chaos expansions, relies on a small number of battery simulations. Within this UQ framework, the identification of most important uncertainty sources is achieved by performing a global sensitivity analysis via computing the so-called Sobol' indices. Such information aids in designing more efficient and targeted quality control procedures, which consequently may result in reducing the LIB production cost. An LiC$_6$/LiCoO$_2$ cell with 19 uncertain parameters discharged at 0.25C, 1C and 4C rates is considered to study the performance and accuracy of the proposed UQ approach. The results suggest that, for the considered cell, the battery discharge rate is a key factor affecting not only the performance variability of the cell, but also the determination of most important random inputs.
△ Less
Submitted 15 September, 2015; v1 submitted 28 May, 2015;
originally announced May 2015.
-
Coherence Motivated Sampling and Convergence Analysis of Least-Squares Polynomial Chaos Regression
Authors:
Jerrad Hampton,
Alireza Doostan
Abstract:
Independent sampling of orthogonal polynomial bases via Monte Carlo is of interest for uncertainty quantification of models, using Polynomial Chaos (PC) expansions. It is known that bounding the spectral radius of a random matrix consisting of PC samples, yields a bound on the number of samples necessary to identify coefficients in the PC expansion via solution to a least-squares regression proble…
▽ More
Independent sampling of orthogonal polynomial bases via Monte Carlo is of interest for uncertainty quantification of models, using Polynomial Chaos (PC) expansions. It is known that bounding the spectral radius of a random matrix consisting of PC samples, yields a bound on the number of samples necessary to identify coefficients in the PC expansion via solution to a least-squares regression problem. We present a related analysis which guarantees a mean square convergence using a coherence parameter of the sampled PC basis that may be both analytically bounded and computationally estimated. Utilizing asymptotic results for orthogonal polynomials, we bound the coherence parameter for polynomials of Hermite and Legendre type under each respective natural sampling distribution. In both polynomial bases we identify an importance sampling distribution which yields a bound with weaker dependence on the order of the PC basis. For more general orthonormal bases, we propose the coherence-optimal sampling: a Markov Chain Monte Carlo sampling, which directly uses the basis functions under consideration to achieve a statistical optimality among all such sampling schemes with identical support, and which guarantees recovery with a number of samples that is, up to logarithmic factors, linear in the number of basis functions considered. We demonstrate these different sampling strategies numerically in both high-order and high-dimensional manufactured PC expansions. In addition, the quality of each sampling method is compared in the identification of solutions to two differential equations, one with a high-dimensional random input and the other with a high-order PC expansion. In all observed cases the coherence-optimal sampling leads to similar or considerably improved accuracy over the other considered sampling distributions.
△ Less
Submitted 7 October, 2014;
originally announced October 2014.
-
Compressive Sampling of Polynomial Chaos Expansions: Convergence Analysis and Sampling Strategies
Authors:
Jerrad Hampton,
Alireza Doostan
Abstract:
Sampling orthogonal polynomial bases via Monte Carlo is of interest for uncertainty quantification of models with high-dimensional random inputs, using Polynomial Chaos (PC) expansions. It is known that bounding a probabilistic parameter, referred to as {\it coherence}, yields a bound on the number of samples necessary to identify coefficients in a sparse PC expansion via solution to an $\ell_1$-m…
▽ More
Sampling orthogonal polynomial bases via Monte Carlo is of interest for uncertainty quantification of models with high-dimensional random inputs, using Polynomial Chaos (PC) expansions. It is known that bounding a probabilistic parameter, referred to as {\it coherence}, yields a bound on the number of samples necessary to identify coefficients in a sparse PC expansion via solution to an $\ell_1$-minimization problem. Utilizing asymptotic results for orthogonal polynomials, we bound the coherence parameter for polynomials of Hermite and Legendre type under the respective natural sampling distribution. In both polynomial bases we identify an importance sampling distribution which yields a bound with weaker dependence on the order of the approximation. For more general orthonormal bases, we propose the {\it coherence-optimal} sampling: a Markov Chain Monte Carlo sampling, which directly uses the basis functions under consideration to achieve a statistical optimality among all sampling schemes with identical support. We demonstrate these different sampling strategies numerically in both high-order and high-dimensional, manufactured PC expansions. In addition, the quality of each sampling method is compared in the identification of solutions to two differential equations, one with a high-dimensional random input and the other with a high-order PC expansion. In both cases the coherence-optimal sampling scheme leads to similar or considerably improved accuracy.
△ Less
Submitted 24 September, 2014; v1 submitted 18 August, 2014;
originally announced August 2014.
-
A Simple and Efficient Preconditioning Scheme for Heaviside Enriched XFEM
Authors:
Christapher Lang,
David Makhija,
Alireza Doostan,
Kurt Maute
Abstract:
The eXtended Finite Element Method (XFEM) is an approach for solving problems with non-smooth solutions. In the XFEM, the approximate solution is locally enriched to capture discontinuities without requiring a mesh which conforms to the geometric features. One drawback of the XFEM is that an ill-conditioned system of equations results when the ratio of volumes on either side of the interface in an…
▽ More
The eXtended Finite Element Method (XFEM) is an approach for solving problems with non-smooth solutions. In the XFEM, the approximate solution is locally enriched to capture discontinuities without requiring a mesh which conforms to the geometric features. One drawback of the XFEM is that an ill-conditioned system of equations results when the ratio of volumes on either side of the interface in an element is small. In this paper, to avoid this ill-conditioning, a simple and efficient scheme based on a geometric preconditioner and constraining degrees of freedom to zero for small intersections is proposed. This geometric preconditioner is computed from the nodal basis functions, and therefore may be constructed prior to building the system of equations. This feature and the low-cost of constructing the preconditioning matrix makes it well suited for nonlinear problems with fixed and moving interfaces.
△ Less
Submitted 20 December, 2013;
originally announced December 2013.
-
Variational multiscale analysis: the fine-scale Green's function for stochastic partial differential equations
Authors:
Jayanth Jagalur-Mohan,
Onkar Sahni,
Alireza Doostan,
Assad Oberai
Abstract:
We present the variational multiscale (VMS) method for partial differential equations (PDEs) with stochastic coefficients and source terms. We use it as a method for generating accurate coarse-scale solutions while accounting for the effect of the unresolved fine scales through a model term that contains a fine-scale stochastic Green's function. For a natural choice of an "optimal" coarse-scale so…
▽ More
We present the variational multiscale (VMS) method for partial differential equations (PDEs) with stochastic coefficients and source terms. We use it as a method for generating accurate coarse-scale solutions while accounting for the effect of the unresolved fine scales through a model term that contains a fine-scale stochastic Green's function. For a natural choice of an "optimal" coarse-scale solution and L^2-orthogonal stochastic basis functions, we demonstrate that the fine-scale stochastic Green's function is intimately linked to its deterministic counterpart. In particular, (i) we demonstrate that whenever the deterministic fine-scale function vanishes, the stochastic fine-scale function satisfies a weaker, and discrete notion of vanishing stochastic coefficients, and (ii) derive an explicit formula for the fine-scale stochastic Green's function that only involves quantities needed to evaluate the fine-scale deterministic Green's function. We present numerical results that support our claims about the physical support of the stochastic fine-scale function, and demonstrate the benefit of using the VMS method when the fine-scale Green's function is approximated by an easier to implement, element Green's function.
△ Less
Submitted 7 October, 2013;
originally announced October 2013.
-
A weighted L1-minimization approach for sparse polynomial chaos expansions
Authors:
Ji Peng,
Jerrad Hampton,
Alireza Doostan
Abstract:
This work proposes a method for sparse polynomial chaos (PC) approximation of high-dimensional stochastic functions based on non-adapted random sampling. We modify the standard l1 -minimization algorithm, originally proposed in the context of compressive sampling, using a priori information about the decay of the PC coefficients and refer to the resulting algorithm as weighted l1 -minimization. We…
▽ More
This work proposes a method for sparse polynomial chaos (PC) approximation of high-dimensional stochastic functions based on non-adapted random sampling. We modify the standard l1 -minimization algorithm, originally proposed in the context of compressive sampling, using a priori information about the decay of the PC coefficients and refer to the resulting algorithm as weighted l1 -minimization. We provide conditions under which we may guarantee recovery using this weighted scheme. Numerical tests are used to compare the weighted and non-weighted methods for the recovery of solutions to two differential equations with high-dimensional random inputs: a boundary value problem with a random elliptic operator and a 2-D thermally driven cavity flow with random boundary condition.
△ Less
Submitted 2 August, 2013;
originally announced August 2013.
-
Partitioned treatment of uncertainty in coupled domain problems: A separated representation approach
Authors:
Mohammad Hadigol,
Alireza Doostan,
Hermann G. Matthies,
Rainer Niekamp
Abstract:
This work is concerned with the propagation of uncertainty across coupled domain problems with high-dimensional random inputs. A stochastic model reduction approach based on low-rank separated representations is proposed for the partitioned treatment of the uncertainty space. The construction of the coupled domain solution is achieved though a sequence of approximations with respect to the dimensi…
▽ More
This work is concerned with the propagation of uncertainty across coupled domain problems with high-dimensional random inputs. A stochastic model reduction approach based on low-rank separated representations is proposed for the partitioned treatment of the uncertainty space. The construction of the coupled domain solution is achieved though a sequence of approximations with respect to the dimensionality of the random inputs associated with each individual sub-domain and not the combined dimensionality, hence drastically reducing the overall computational cost. The coupling between the sub-domain solutions is done via the classical Finite Element Tearing and Interconnecting (FETI) method, thus providing a well suited framework for parallel computing. Two high-dimensional stochastic problems, a 2D elliptic PDE with random diffusion coefficient and a stochastic linear elasticity problem, have been considered to study the performance and accuracy of the proposed stochastic coupling approach.
△ Less
Submitted 29 May, 2013;
originally announced May 2013.
-
A non-adapted sparse approximation of PDEs with stochastic inputs
Authors:
Alireza Doostan,
Houman Owhadi
Abstract:
We propose a method for the approximation of solutions of PDEs with stochastic coefficients based on the direct, i.e., non-adapted, sampling of solutions. This sampling can be done by using any legacy code for the deterministic problem as a black box. The method converges in probability (with probabilistic error bounds) as a consequence of sparsity and a concentration of measure phenomenon on the…
▽ More
We propose a method for the approximation of solutions of PDEs with stochastic coefficients based on the direct, i.e., non-adapted, sampling of solutions. This sampling can be done by using any legacy code for the deterministic problem as a black box. The method converges in probability (with probabilistic error bounds) as a consequence of sparsity and a concentration of measure phenomenon on the empirical correlation between samples. We show that the method is well suited for truly high-dimensional problems (with slow decay in the spectrum).
△ Less
Submitted 10 June, 2010;
originally announced June 2010.
