Showing 1–20 of 20 results for author: Renganathan, A

Multifidelity Cross-validation

Authors: S. Ashwin Renganathan, Kade Carlson

Abstract: Emulating the map** between quantities of interest and their control parameters using surrogate models finds widespread application in engineering design, including in numerical optimization and uncertainty quantification. Gaussian process models can serve as a probabilistic surrogate model of unknown functions, thereby making them highly suitable for engineering design and decision-making in th… ▽ More Emulating the map** between quantities of interest and their control parameters using surrogate models finds widespread application in engineering design, including in numerical optimization and uncertainty quantification. Gaussian process models can serve as a probabilistic surrogate model of unknown functions, thereby making them highly suitable for engineering design and decision-making in the presence of uncertainty. In this work, we are interested in emulating quantities of interest observed from models of a system at multiple fidelities, which trade accuracy for computational efficiency. Using multifidelity Gaussian process models, to efficiently fuse models at multiple fidelities, we propose a novel method to actively learn the surrogate model via leave-one-out cross-validation (LOO-CV). Our proposed multifidelity cross-validation (\texttt{MFCV}) approach develops an adaptive approach to reduce the LOO-CV error at the target (highest) fidelity, by learning the correlations between the LOO-CV at all fidelities. \texttt{MFCV} develops a two-step lookahead policy to select optimal input-fidelity pairs, both in sequence and in batches, both for continuous and discrete fidelity spaces. We demonstrate the utility of our method on several synthetic test problems as well as on the thermal stress analysis of a gas turbine blade. △ Less

Submitted 1 July, 2024; originally announced July 2024.

Comments: arXiv admin note: text overlap with arXiv:2203.01436

qPOTS: Efficient batch multiobjective Bayesian optimization via Pareto optimal Thompson sampling

Authors: S. Ashwin Renganathan

Abstract: Classical evolutionary approaches for multiobjective optimization are quite effective but incur a lot of queries to the objectives; this can be prohibitive when objectives are expensive oracles. A sample-efficient approach to solving multiobjective optimization is via Gaussian process (GP) surrogates and Bayesian optimization (BO). Multiobjective Bayesian optimization (MOBO) involves the construct… ▽ More Classical evolutionary approaches for multiobjective optimization are quite effective but incur a lot of queries to the objectives; this can be prohibitive when objectives are expensive oracles. A sample-efficient approach to solving multiobjective optimization is via Gaussian process (GP) surrogates and Bayesian optimization (BO). Multiobjective Bayesian optimization (MOBO) involves the construction of an acquisition function which is optimized to acquire new observation candidates. This ``inner'' optimization can be hard due to various reasons: acquisition functions being nonconvex, nondifferentiable and/or unavailable in analytical form; the success of MOBO heavily relies on this inner optimization. We do away with this hard acquisition function optimization step and propose a simple, but effective, Thompson sampling based approach ($q\texttt{POTS}$) where new candidate(s) are chosen from the Pareto frontier of random GP posterior sample paths obtained by solving a much cheaper multiobjective optimization problem. To further improve computational tractability in higher dimensions we propose an automated active set of candidates selection combined with a Nyström approximation. Our approach applies to arbitrary GP prior assumptions and demonstrates strong empirical performance over the state of the art, both in terms of accuracy and computational efficiency, on synthetic as well as real-world experiments. △ Less

Submitted 24 October, 2023; originally announced October 2023.

Comments: 12 pages, 4 figures

Task Aware Modulation using Representation Learning: An Approach for Few Shot Learning in Heterogeneous Systems

Authors: Arvind Renganathan, Rahul Ghosh, Ankush Khandelwal, Vipin Kumar

Abstract: We present a Task-aware modulation using Representation Learning (TAM-RL) framework that enhances personalized predictions in few-shot settings for heterogeneous systems when individual task characteristics are not known. TAM-RL extracts embeddings representing the actual inherent characteristics of these entities and uses these characteristics to personalize the predictions for each entity/task.… ▽ More We present a Task-aware modulation using Representation Learning (TAM-RL) framework that enhances personalized predictions in few-shot settings for heterogeneous systems when individual task characteristics are not known. TAM-RL extracts embeddings representing the actual inherent characteristics of these entities and uses these characteristics to personalize the predictions for each entity/task. Using real-world hydrological and flux tower benchmark data sets, we show that TAM-RL can significantly outperform existing baseline approaches such as MAML and multi-modal MAML (MMAML) while being much faster and simpler to train due to less complexity. Specifically, TAM-RL eliminates the need for sensitive hyper-parameters like inner loop steps and inner loop learning rate, which are crucial for model convergence in MAML, MMAML. We further present an empirical evaluation via synthetic data to explore the impact of heterogeneity amongst the entities on the relative performance of MAML, MMAML, and TAM-RL. We show that TAM-RL significantly improves predictive performance for cases where it is possible to learn distinct representations for different tasks. △ Less

Submitted 7 October, 2023; originally announced October 2023.

Uncertainty Quantification in Inverse Models in Hydrology

Authors: Somya Sharma Chatterjee, Rahul Ghosh, Arvind Renganathan, Xiang Li, Snigdhansu Chatterjee, John Nieber, Christopher Duffy, Vipin Kumar

Abstract: In hydrology, modeling streamflow remains a challenging task due to the limited availability of basin characteristics information such as soil geology and geomorphology. These characteristics may be noisy due to measurement errors or may be missing altogether. To overcome this challenge, we propose a knowledge-guided, probabilistic inverse modeling method for recovering physical characteristics fr… ▽ More In hydrology, modeling streamflow remains a challenging task due to the limited availability of basin characteristics information such as soil geology and geomorphology. These characteristics may be noisy due to measurement errors or may be missing altogether. To overcome this challenge, we propose a knowledge-guided, probabilistic inverse modeling method for recovering physical characteristics from streamflow and weather data, which are more readily available. We compare our framework with state-of-the-art inverse models for estimating river basin characteristics. We also show that these estimates offer improvement in streamflow modeling as opposed to using the original basin characteristic values. Our inverse model offers 3\% improvement in R$^2$ for the inverse model (basin characteristic estimation) and 6\% for the forward model (streamflow prediction). Our framework also offers improved explainability since it can quantify uncertainty in both the inverse and the forward model. Uncertainty quantification plays a pivotal role in improving the explainability of machine learning models by providing additional insights into the reliability and limitations of model predictions. In our analysis, we assess the quality of the uncertainty estimates. Compared to baseline uncertainty quantification methods, our framework offers 10\% improvement in the dispersion of epistemic uncertainty and 13\% improvement in coverage rate. This information can help stakeholders understand the level of uncertainty associated with the predictions and provide a more comprehensive view of the potential outcomes. △ Less

Submitted 3 October, 2023; originally announced October 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2210.06213

Message Propagation Through Time: An Algorithm for Sequence Dependency Retention in Time Series Modeling

Authors: Shaoming Xu, Ankush Khandelwal, Arvind Renganathan, Vipin Kumar

Abstract: Time series modeling, a crucial area in science, often encounters challenges when training Machine Learning (ML) models like Recurrent Neural Networks (RNNs) using the conventional mini-batch training strategy that assumes independent and identically distributed (IID) samples and initializes RNNs with zero hidden states. The IID assumption ignores temporal dependencies among samples, resulting in… ▽ More Time series modeling, a crucial area in science, often encounters challenges when training Machine Learning (ML) models like Recurrent Neural Networks (RNNs) using the conventional mini-batch training strategy that assumes independent and identically distributed (IID) samples and initializes RNNs with zero hidden states. The IID assumption ignores temporal dependencies among samples, resulting in poor performance. This paper proposes the Message Propagation Through Time (MPTT) algorithm to effectively incorporate long temporal dependencies while preserving faster training times relative to the stateful solutions. MPTT utilizes two memory modules to asynchronously manage initial hidden states for RNNs, fostering seamless information exchange between samples and allowing diverse mini-batches throughout epochs. MPTT further implements three policies to filter outdated and preserve essential information in the hidden states to generate informative initial hidden states for RNNs, facilitating robust training. Experimental results demonstrate that MPTT outperforms seven strategies on four climate datasets with varying levels of temporal dependencies. △ Less

Submitted 28 September, 2023; originally announced September 2023.

Koopman Invertible Autoencoder: Leveraging Forward and Backward Dynamics for Temporal Modeling

Authors: Kshitij Tayal, Arvind Renganathan, Rahul Ghosh, Xiaowei Jia, Vipin Kumar

Abstract: Accurate long-term predictions are the foundations for many machine learning applications and decision-making processes. However, building accurate long-term prediction models remains challenging due to the limitations of existing temporal models like recurrent neural networks (RNNs), as they capture only the statistical connections in the training data and may fail to learn the underlying dynamic… ▽ More Accurate long-term predictions are the foundations for many machine learning applications and decision-making processes. However, building accurate long-term prediction models remains challenging due to the limitations of existing temporal models like recurrent neural networks (RNNs), as they capture only the statistical connections in the training data and may fail to learn the underlying dynamics of the target system. To tackle this challenge, we propose a novel machine learning model based on Koopman operator theory, which we call Koopman Invertible Autoencoders (KIA), that captures the inherent characteristic of the system by modeling both forward and backward dynamics in the infinite-dimensional Hilbert space. This enables us to efficiently learn low-dimensional representations, resulting in more accurate predictions of long-term system behavior. Moreover, our method's invertibility design guarantees reversibility and consistency in both forward and inverse operations. We illustrate the utility of KIA on pendulum and climate datasets, demonstrating 300% improvements in long-term prediction capability for pendulum while maintaining robustness against noise. Additionally, our method excels in long-term climate prediction, further validating our method's effectiveness. △ Less

Submitted 18 September, 2023; originally announced September 2023.

Comments: Accepted at IEEE International Conference on Data Mining (ICDM) 2023

Contour Location for Reliability in Airfoil Simulation Experiments using Deep Gaussian Processes

Authors: Annie S. Booth, S. Ashwin Renganathan, Robert B. Gramacy

Abstract: Bayesian deep Gaussian processes (DGPs) outperform ordinary GPs as surrogate models of complex computer experiments when response surface dynamics are non-stationary, which is especially prevalent in aerospace simulations. Yet DGP surrogates have not been deployed for the canonical downstream task in that setting: reliability analysis through contour location (CL). In that context, we are motivate… ▽ More Bayesian deep Gaussian processes (DGPs) outperform ordinary GPs as surrogate models of complex computer experiments when response surface dynamics are non-stationary, which is especially prevalent in aerospace simulations. Yet DGP surrogates have not been deployed for the canonical downstream task in that setting: reliability analysis through contour location (CL). In that context, we are motivated by a simulation of an RAE-2822 transonic airfoil which demarcates efficient and inefficient flight conditions. Level sets separating passable versus failable operating conditions are best learned through strategic sequential design. There are two limitations to modern CL methodology which hinder DGP integration in this setting. First, derivative-based optimization underlying acquisition functions is thwarted by sampling-based Bayesian (i.e., MCMC) inference, which is essential for DGP posterior integration. Second, canonical acquisition criteria, such as entropy, are famously myopic to the extent that optimization may even be undesirable. Here we tackle both of these limitations at once, proposing a hybrid criterion that explores along the Pareto front of entropy and (predictive) uncertainty, requiring evaluation only at strategically located "triangulation" candidates. We showcase DGP CL performance in several synthetic benchmark exercises and on the RAE-2822 airfoil. △ Less

Submitted 24 April, 2024; v1 submitted 8 August, 2023; originally announced August 2023.

Comments: 19 pages, 11 figures

Entity Aware Modelling: A Survey

Authors: Rahul Ghosh, Haoyu Yang, Ankush Khandelwal, Erhu He, Arvind Renganathan, Somya Sharma, Xiaowei Jia, Vipin Kumar

Abstract: Personalized prediction of responses for individual entities caused by external drivers is vital across many disciplines. Recent machine learning (ML) advances have led to new state-of-the-art response prediction models. Models built at a population level often lead to sub-optimal performance in many personalized prediction settings due to heterogeneity in data across entities (tasks). In personal… ▽ More Personalized prediction of responses for individual entities caused by external drivers is vital across many disciplines. Recent machine learning (ML) advances have led to new state-of-the-art response prediction models. Models built at a population level often lead to sub-optimal performance in many personalized prediction settings due to heterogeneity in data across entities (tasks). In personalized prediction, the goal is to incorporate inherent characteristics of different entities to improve prediction performance. In this survey, we focus on the recent developments in the ML community for such entity-aware modeling approaches. ML algorithms often modulate the network using these entity characteristics when they are readily available. However, these entity characteristics are not readily available in many real-world scenarios, and different ML methods have been proposed to infer these characteristics from the data. In this survey, we have organized the current literature on entity-aware modeling based on the availability of these characteristics as well as the amount of training data. We highlight how recent innovations in other disciplines, such as uncertainty quantification, fairness, and knowledge-guided machine learning, can improve entity-aware modeling. △ Less

Submitted 16 February, 2023; originally announced February 2023.

Comments: Submitted to IJCAI, Survey Track

Probabilistic Inverse Modeling: An Application in Hydrology

Authors: Somya Sharma, Rahul Ghosh, Arvind Renganathan, Xiang Li, Snigdhansu Chatterjee, John Nieber, Christopher Duffy, Vipin Kumar

Abstract: The astounding success of these methods has made it imperative to obtain more explainable and trustworthy estimates from these models. In hydrology, basin characteristics can be noisy or missing, impacting streamflow prediction. For solving inverse problems in such applications, ensuring explainability is pivotal for tackling issues relating to data bias and large search space. We propose a probab… ▽ More The astounding success of these methods has made it imperative to obtain more explainable and trustworthy estimates from these models. In hydrology, basin characteristics can be noisy or missing, impacting streamflow prediction. For solving inverse problems in such applications, ensuring explainability is pivotal for tackling issues relating to data bias and large search space. We propose a probabilistic inverse model framework that can reconstruct robust hydrology basin characteristics from dynamic input weather driver and streamflow response data. We address two aspects of building more explainable inverse models, uncertainty estimation and robustness. This can help improve the trust of water managers, handling of noisy data and reduce costs. We propose uncertainty based learning method that offers 6\% improvement in $R^2$ for streamflow prediction (forward modeling) from inverse model inferred basin characteristic estimates, 17\% reduction in uncertainty (40\% in presence of noise) and 4\% higher coverage rate for basin characteristics. △ Less

Submitted 12 October, 2022; originally announced October 2022.

CAMERA: A Method for Cost-aware, Adaptive, Multifidelity, Efficient Reliability Analysis

Authors: S. Ashwin Renganathan, Vishwas Rao, Ionel M. Navon

Abstract: Estimating probability of failure in aerospace systems is a critical requirement for flight certification and qualification. Failure probability estimation involves resolving tails of probability distribution, and Monte Carlo sampling methods are intractable when expensive high-fidelity simulations have to be queried. We propose a method to use models of multiple fidelities that trade accuracy for… ▽ More Estimating probability of failure in aerospace systems is a critical requirement for flight certification and qualification. Failure probability estimation involves resolving tails of probability distribution, and Monte Carlo sampling methods are intractable when expensive high-fidelity simulations have to be queried. We propose a method to use models of multiple fidelities that trade accuracy for computational efficiency. Specifically, we propose the use of multifidelity Gaussian process models to efficiently fuse models at multiple fidelity, thereby offering a cheap surrogate model that emulates the original model at all fidelities. Furthermore, we propose a novel sequential \emph{acquisition function}-based experiment design framework that can automatically select samples from appropriate fidelity models to make predictions about quantities of interest in the highest fidelity. We use our proposed approach in an importance sampling setting and demonstrate our method on the failure level set estimation and probability estimation on synthetic test functions as well as two real-world applications, namely, the reliability analysis of a gas turbine engine blade using a finite element method and a transonic aerodynamic wing test case using Reynolds-averaged Navier--Stokes equations. We demonstrate that our method predicts the failure boundary and probability more accurately and computationally efficiently while using varying fidelity models compared with using just a single expensive high-fidelity model. △ Less

Submitted 20 September, 2022; v1 submitted 2 March, 2022; originally announced March 2022.

Comments: 35 page, 16 figures

Robust Inverse Framework using Knowledge-guided Self-Supervised Learning: An application to Hydrology

Authors: Rahul Ghosh, Arvind Renganathan, Kshitij Tayal, Xiang Li, Ankush Khandelwal, Xiaowei Jia, Chris Duffy, John Neiber, Vipin Kumar

Abstract: Machine Learning is beginning to provide state-of-the-art performance in a range of environmental applications such as streamflow prediction in a hydrologic basin. However, building accurate broad-scale models for streamflow remains challenging in practice due to the variability in the dominant hydrologic processes, which are best captured by sets of process-related basin characteristics. Existing… ▽ More Machine Learning is beginning to provide state-of-the-art performance in a range of environmental applications such as streamflow prediction in a hydrologic basin. However, building accurate broad-scale models for streamflow remains challenging in practice due to the variability in the dominant hydrologic processes, which are best captured by sets of process-related basin characteristics. Existing basin characteristics suffer from noise and uncertainty, among many other things, which adversely impact model performance. To tackle the above challenges, in this paper, we propose a novel Knowledge-guided Self-Supervised Learning (KGSSL) inverse framework to extract system characteristics from driver and response data. This first-of-its-kind framework achieves robust performance even when characteristics are corrupted. We show that KGSSL achieves state-of-the-art results for streamflow modeling for CAMELS (Catchment Attributes and MEteorology for Large-sample Studies) which is a widely used hydrology benchmark dataset. Specifically, KGSSL outperforms other methods by up to 16 \% in reconstructing characteristics. Furthermore, we show that KGSSL is relatively more robust to distortion than baseline methods, and outperforms the baseline model by 35\% when plugging in KGSSL inferred characteristics. △ Less

Submitted 8 June, 2022; v1 submitted 14 September, 2021; originally announced September 2021.

Comments: Accepted at KDD 2022

Data-Driven Wind Turbine Wake Modeling via Probabilistic Machine Learning

Authors: S. Ashwin Renganathan, Romit Maulik, Stefano Letizia, Giacomo Valerio Iungo

Abstract: Wind farm design primarily depends on the variability of the wind turbine wake flows to the atmospheric wind conditions, and the interaction between wakes. Physics-based models that capture the wake flow-field with high-fidelity are computationally very expensive to perform layout optimization of wind farms, and, thus, data-driven reduced order models can represent an efficient alternative for sim… ▽ More Wind farm design primarily depends on the variability of the wind turbine wake flows to the atmospheric wind conditions, and the interaction between wakes. Physics-based models that capture the wake flow-field with high-fidelity are computationally very expensive to perform layout optimization of wind farms, and, thus, data-driven reduced order models can represent an efficient alternative for simulating wind farms. In this work, we use real-world light detection and ranging (LiDAR) measurements of wind-turbine wakes to construct predictive surrogate models using machine learning. Specifically, we first demonstrate the use of deep autoencoders to find a low-dimensional \emph{latent} space that gives a computationally tractable approximation of the wake LiDAR measurements. Then, we learn the map** between the parameter space and the (latent space) wake flow-fields using a deep neural network. Additionally, we also demonstrate the use of a probabilistic machine learning technique, namely, Gaussian process modeling, to learn the parameter-space-latent-space map** in addition to the epistemic and aleatoric uncertainty in the data. Finally, to cope with training large datasets, we demonstrate the use of variational Gaussian process models that provide a tractable alternative to the conventional Gaussian process models for large datasets. Furthermore, we introduce the use of active learning to adaptively build and improve a conventional Gaussian process model predictive capability. Overall, we find that our approach provides accurate approximations of the wind-turbine wake flow field that can be queried at an orders-of-magnitude cheaper cost than those generated with high-fidelity physics-based simulations. △ Less

Submitted 16 November, 2021; v1 submitted 6 September, 2021; originally announced September 2021.

Comments: 18 pages, 10 figures

Machine-learning identification of the variability of mean velocity and turbulence intensity for wakes generated by onshore wind turbines: Cluster analysis of wind LiDAR measurements

Authors: G. Valerio Iungo, Romit Maulik, S. Ashwin Renganathan, Stefano Letizia

Abstract: Wind turbine wakes are the result of the extraction of kinetic energy from the incoming atmospheric wind exerted from a wind turbine rotor. Therefore, the reduced mean velocity and enhanced turbulence intensity within the wake are affected by the characteristics of the incoming wind, turbine blade aerodynamics, and the turbine control settings. In this work, LiDAR measurements of isolated wakes ge… ▽ More Wind turbine wakes are the result of the extraction of kinetic energy from the incoming atmospheric wind exerted from a wind turbine rotor. Therefore, the reduced mean velocity and enhanced turbulence intensity within the wake are affected by the characteristics of the incoming wind, turbine blade aerodynamics, and the turbine control settings. In this work, LiDAR measurements of isolated wakes generated by wind turbines installed at an onshore wind farm are leveraged to characterize the variability of the wake mean velocity and turbulence intensity during typical operations encompassing a breadth of atmospheric stability regimes, levels of power capture, and, in turn, rotor thrust coefficients. For the statistical analysis of the wake velocity fields, the LiDAR measurements are clustered through a k-means algorithm, which enables to identify of the most representative realizations of the wind turbine wakes while avoiding the imposition of thresholds for the various wind and turbine parameters, which can be biased by preconceived, and potentially incorrect, notions. Considering the large number of LiDAR samples collected to probe the wake velocity field over the horizontal plane at hub height, the dimensionality of the experimental dataset is reduced by projecting the LiDAR data on an intelligently-truncated basis obtained with the proper orthogonal decomposition (POD). The coefficients of only five physics-informed POD modes, which are considered sufficient to reproduce the observed wake variability, are then injected in the k-means algorithm for clustering the LiDAR dataset. The analysis of the clustered LiDAR data, and the associated SCADA and meteorological data, enables the study of the variability of the wake velocity deficit, wake extent, and wake-added turbulence intensity for different thrust coefficients of the turbine rotor and regimes of atmospheric stability. △ Less

Submitted 3 September, 2021; originally announced September 2021.

Lookahead Acquisition Functions for Finite-Horizon Time-Dependent Bayesian Optimization and Application to Quantum Optimal Control

Authors: S. Ashwin Renganathan, Jeffrey Larson, Stefan M. Wild

Abstract: We propose a novel Bayesian method to solve the maximization of a time-dependent expensive-to-evaluate stochastic oracle. We are interested in the decision that maximizes the oracle at a finite time horizon, given a limited budget of noisy evaluations of the oracle that can be performed before the horizon. Our recursive two-step lookahead acquisition function for Bayesian optimization makes nonmyo… ▽ More We propose a novel Bayesian method to solve the maximization of a time-dependent expensive-to-evaluate stochastic oracle. We are interested in the decision that maximizes the oracle at a finite time horizon, given a limited budget of noisy evaluations of the oracle that can be performed before the horizon. Our recursive two-step lookahead acquisition function for Bayesian optimization makes nonmyopic decisions at every stage by maximizing the expected utility at the specified time horizon. Specifically, we propose a generalized two-step lookahead framework with a customizable \emph{value} function that allows users to define the utility. We illustrate how lookahead versions of classic acquisition functions such as the expected improvement, probability of improvement, and upper confidence bound can be obtained with this framework. We demonstrate the utility of our proposed approach on several carefully constructed synthetic cases and a real-world quantum optimal control problem. △ Less

Submitted 20 May, 2021; originally announced May 2021.

Comments: 22 pages, 11 figures

Enhanced data efficiency using deep neural networks and Gaussian processes for aerodynamic design optimization

Authors: S. Ashwin Renganathan, Romit Maulik and, Jai Ahuja

Abstract: Adjoint-based optimization methods are attractive for aerodynamic shape design primarily due to their computational costs being independent of the dimensionality of the input space and their ability to generate high-fidelity gradients that can then be used in a gradient-based optimizer. This makes them very well suited for high-fidelity simulation based aerodynamic shape optimization of highly par… ▽ More Adjoint-based optimization methods are attractive for aerodynamic shape design primarily due to their computational costs being independent of the dimensionality of the input space and their ability to generate high-fidelity gradients that can then be used in a gradient-based optimizer. This makes them very well suited for high-fidelity simulation based aerodynamic shape optimization of highly parametrized geometries such as aircraft wings. However, the development of adjoint-based solvers involve careful mathematical treatment and their implementation require detailed software development. Furthermore, they can become prohibitively expensive when multiple optimization problems are being solved, each requiring multiple restarts to circumvent local optima. In this work, we propose a machine learning enabled, surrogate-based framework that replaces the expensive adjoint solver, without compromising on predicting predictive accuracy. Specifically, we first train a deep neural network (DNN) from training data generated from evaluating the high-fidelity simulation model on a model-agnostic, design of experiments on the geometry shape parameters. The optimum shape may then be computed by using a gradient-based optimizer coupled with the trained DNN. Subsequently, we also perform a gradient-free Bayesian optimization, where the trained DNN is used as the prior mean. We observe that the latter framework (DNN-BO) improves upon the DNN-only based optimization strategy for the same computational cost. Overall, this framework predicts the true optimum with very high accuracy, while requiring far fewer high-fidelity function calls compared to the adjoint-based method. Furthermore, we show that multiple optimization problems can be solved with the same machine learning model with high accuracy, to amortize the offline costs associated with constructing our models. △ Less

Submitted 15 August, 2020; originally announced August 2020.

Comments: 25 pages, 13 figures

Recursive Two-Step Lookahead Expected Payoff for Time-Dependent Bayesian Optimization

Authors: S. Ashwin Renganathan, Jeffrey Larson, Stefan Wild

Abstract: We propose a novel Bayesian method to solve the maximization of a time-dependent expensive-to-evaluate oracle. We are interested in the decision that maximizes the oracle at a finite time horizon, when relatively few noisy evaluations can be performed before the horizon. Our recursive, two-step lookahead expected payoff ($\texttt{r2LEY}$) acquisition function makes nonmyopic decisions at every sta… ▽ More We propose a novel Bayesian method to solve the maximization of a time-dependent expensive-to-evaluate oracle. We are interested in the decision that maximizes the oracle at a finite time horizon, when relatively few noisy evaluations can be performed before the horizon. Our recursive, two-step lookahead expected payoff ($\texttt{r2LEY}$) acquisition function makes nonmyopic decisions at every stage by maximizing the estimated expected value of the oracle at the horizon. $\texttt{r2LEY}$ circumvents the evaluation of the expensive multistep (more than two steps) lookahead acquisition function by recursively optimizing a two-step lookahead acquisition function at every stage; unbiased estimators of this latter function and its gradient are utilized for efficient optimization. $\texttt{r2LEY}$ is shown to exhibit natural exploration properties far from the time horizon, enabling accurate emulation of the oracle, which is exploited in the final decision made at the horizon. To demonstrate the utility of $\texttt{r2LEY}$, we compare it with time-dependent extensions of popular myopic acquisition functions via both synthetic and real-world datasets. △ Less

Submitted 8 December, 2020; v1 submitted 14 June, 2020; originally announced June 2020.

Comments: 24pages, 5 figures

Machine-Learning for Nonintrusive Model Order Reduction of the Parametric Inviscid Transonic Flow past an airfoil

Authors: S. Ashwin Renganathan, Romit Maulik, Vishwas Rao

Abstract: Fluid flow in the transonic regime finds relevance in aerospace engineering, particularly in the design of commercial air transportation vehicles. Computational fluid dynamics models of transonic flow for aerospace applications are computationally expensive to solve because of the high degrees of freedom as well as the coupled nature of the conservation laws. While these issues pose a bottleneck f… ▽ More Fluid flow in the transonic regime finds relevance in aerospace engineering, particularly in the design of commercial air transportation vehicles. Computational fluid dynamics models of transonic flow for aerospace applications are computationally expensive to solve because of the high degrees of freedom as well as the coupled nature of the conservation laws. While these issues pose a bottleneck for the use of such models in aerospace design, computational costs can be significantly minimized by constructing special, structure-preserving surrogate models called reduced-order models. Such models are known to incur huge off-line costs, however, which can sometimes outweigh their potential benefits. Furthermore, their prediction accuracy is known to be poor under transonic flow conditions. In this work, we propose a machine learning method to construct reduced-order models via deep neural networks, and we demonstrate its ability to preserve accuracy with significantly lower offline and online costs. In addition, our machine learning methodology is physics-informed and constrained through the utilization of an interpretable encoding by way of proper orthogonal decomposition. Application to the inviscid transonic flow past the RAE2822 airfoil under varying freestream Mach numbers and angles of attack, as well as airfoil shape parameters with a deforming mesh, shows that the proposed approach adapts to high-dimensional parameter variation well. Notably, the proposed framework precludes knowledge of numerical operators utilized in the data generation phase, thereby demonstrating its potential utility in fast exploration of design space for diverse engineering applications. △ Less

Submitted 11 January, 2020; v1 submitted 18 November, 2019; originally announced November 2019.

Aerodynamic Data Fusion Towards the Digital Twin Paradigm

Authors: S. Ashwin Renganathan, Kohei Harada, Dimitri N. Mavris

Abstract: We consider the fusion of two aerodynamic data sets originating from differing fidelity physical or computer experiments. We specifically address the fusion of: 1) noisy and in-complete fields from wind tunnel measurements and 2) deterministic but biased fields from numerical simulations. These two data sources are fused in order to estimate the \emph{true} field that best matches measured quantit… ▽ More We consider the fusion of two aerodynamic data sets originating from differing fidelity physical or computer experiments. We specifically address the fusion of: 1) noisy and in-complete fields from wind tunnel measurements and 2) deterministic but biased fields from numerical simulations. These two data sources are fused in order to estimate the \emph{true} field that best matches measured quantities that serves as the ground truth. For example, two sources of pressure fields about an aircraft are fused based on measured forces and moments from a wind-tunnel experiment. A fundamental challenge in this problem is that the true field is unknown and can not be estimated with 100\% certainty. We employ a Bayesian framework to infer the true fields conditioned on measured quantities of interest; essentially we perform a \emph{statistical correction} to the data. The fused data may then be used to construct more accurate surrogate models suitable for early stages of aerospace design. We also introduce an extension of the Proper Orthogonal Decomposition with constraints to solve the same problem. Both methods are demonstrated on fusing the pressure distributions for flow past the RAE2822 airfoil and the Common Research Model wing at transonic conditions. Comparison of both methods reveal that the Bayesian method is more robust when data is scarce while capable of also accounting for uncertainties in the data. Furthermore, given adequate data, the POD based and Bayesian approaches lead to \emph{similar} results. △ Less

Submitted 1 November, 2019; originally announced November 2019.

Comments: 33 pages, 19 figures

Journal ref: AIAA Journal 2020

Koopman-Based Approach to Non-intrusive Projection-Based Reduced-Order Modeling with Black-Box High-Fidelity Models. Part II: Application

Authors: S. Ashwin Renganathan

Abstract: A methodology for non-intrusive, projection-based non-linear model reduction originally presented by Renganathan et. al. (2018)~\cite{renganathan2018koopman} is further extended towards parametric systems with focus on application to aerospace design. Specifically, we extend the method for static systems with parametric geometry (that deforms the mesh), in addition to parametric boundary condition… ▽ More A methodology for non-intrusive, projection-based non-linear model reduction originally presented by Renganathan et. al. (2018)~\cite{renganathan2018koopman} is further extended towards parametric systems with focus on application to aerospace design. Specifically, we extend the method for static systems with parametric geometry (that deforms the mesh), in addition to parametric boundary conditions. The main idea is to first perform a transformation on the governing equations such that it is lifted to a higher dimensional but linear under-determined system. This enables one to extract the system matrices easily compared to that of the original non-linear system. The under-determined system is closed with a set of model-dependent non-linear constraints upon which the model reduction is finally performed. The methodology is validated on the subsonic and transonic inviscid flow past the NACA0012 and the RAE2822 airfoils. We further demonstrate the utility of the approach by applying it to two common problems in aerospace design namely, derivative-free global optimization and parametric uncertainty quantification with Monte Carlo sampling. Overall, the methodology is shown to achieve accuracy upto 5\% and computational speed-up of 2-3 orders of magnitude as that of the full-order model. Comparison against another non-intrusive model reduction method revealed that the proposed approach is more robust, accurate and retains the consistency between the state variables. △ Less

Submitted 17 June, 2019; v1 submitted 10 November, 2018; originally announced November 2018.

Comments: 32 pages, 12 figures

Journal ref: AIAA Journal 2020

A Methodology for Projection-Based Model Reduction with Black-Box High-Fidelity Models

Authors: S. Ashwin Renganathan, Yingjie Liu, Dimitri N. Mavris

Abstract: This paper presents a methodology that enables projection-based model reduction for black-box high-fidelity models such as commercial CFD codes. The methodology specifically addresses the situation where the high-fidelity model may be a black-box but there is complete knowledge of the governing equations. The main idea is that the linear operator matrix, resulting from the discretization of the li… ▽ More This paper presents a methodology that enables projection-based model reduction for black-box high-fidelity models such as commercial CFD codes. The methodology specifically addresses the situation where the high-fidelity model may be a black-box but there is complete knowledge of the governing equations. The main idea is that the linear operator matrix, resulting from the discretization of the linear differential terms is approximated directly using a suitable discretization method such as the Finite Volume Method and requires only the computational grid as input. In this regard, the governing equations are first cast in terms of a set of scalar observables of the state variables, leading to a linear set of equations. By applying the snapshots of the observables to the discrete linear operator, a right hand side vector is obtained, providing the necessary system matrices for the Galerkin projection step. This way an online database of ROMs are generated for various parameter snapshots which are then interpolated online to predict the state for new parameter instances. Finally, the reduced order model is posed as a non-linear constrained optimization problem that can be solved at a significantly cheaper cost compared to the full order model. The method is successfully demonstrated on on a canonical non-linear parametric PDE with exponential non-linearity, followed by the compressible inviscid ow past the NACA0012 airfoil. As a first step, this paper focuses only on establishing feasibility of the method. △ Less

Submitted 16 October, 2017; v1 submitted 25 September, 2017; originally announced September 2017.

Comments: 36 pages. Submitted to the AIAA Journal