-
Data-Driven Estimation of Failure Probabilities in Correlated Structure-Preserving Stochastic Power System Models
Authors:
Hongli Zhao,
Tyler E. Maltba,
D. Adrian Maldonado,
Emil Constantinescu,
Mihai Anitescu
Abstract:
We propose a data-driven approach for propagating uncertainty in stochastic power grid simulations and apply it to the estimation of transmission line failure probabilities. A reduced-order equation governing the evolution of the observed line energy probability density function is derived from the Fokker--Planck equation of the full-order continuous Markov process. Our method consists of estimate…
▽ More
We propose a data-driven approach for propagating uncertainty in stochastic power grid simulations and apply it to the estimation of transmission line failure probabilities. A reduced-order equation governing the evolution of the observed line energy probability density function is derived from the Fokker--Planck equation of the full-order continuous Markov process. Our method consists of estimates produced by numerically integrating this reduced equation. Numerical experiments for scalar- and vector-valued energy functions are conducted using the classical multimachine model under spatiotemporally correlated noise perturbation. The method demonstrates a more sample-efficient approach for computing probabilities of tail events when compared with kernel density estimation. Moreover, it produces vastly more accurate estimates of joint event occurrence when compared with independent models.
△ Less
Submitted 4 January, 2024;
originally announced January 2024.
-
Centralized calibration of power system dynamic models using variational data assimilation
Authors:
Ahmed Attia,
D. Adrian Maldonado,
Emil Constantinescu,
Mihai Anitescu
Abstract:
This paper presents a novel centralized, variational data assimilation approach for calibrating transient dynamic models in electrical power systems, focusing on load model parameters. With the increasing importance of inverter-based resources, assessing power systems' dynamic performance under disturbances has become challenging, necessitating robust model calibration methods. The proposed approa…
▽ More
This paper presents a novel centralized, variational data assimilation approach for calibrating transient dynamic models in electrical power systems, focusing on load model parameters. With the increasing importance of inverter-based resources, assessing power systems' dynamic performance under disturbances has become challenging, necessitating robust model calibration methods. The proposed approach expands on previous Bayesian frameworks by establishing a posterior distribution of parameters using an approximation around the maximum a posteriori value. We illustrate the efficacy of our method by generating events of varying intensity, highlighting its ability to capture the systems' evolution accurately and with associated uncertainty estimates. This research improves the precision of dynamic performance assessments in modern power systems, with potential applications in managing uncertainties and optimizing system operations.
△ Less
Submitted 13 November, 2023;
originally announced November 2023.
-
Enhancing Low-Order Discontinuous Galerkin Methods with Neural Ordinary Differential Equations for Compressible Navier--Stokes Equations
Authors:
Shinhoo Kang,
Emil M. Constantinescu
Abstract:
The growing computing power over the years has enabled simulations to become more complex and accurate. While immensely valuable for scientific discovery and problem-solving, however, high-fidelity simulations come with significant computational demands. As a result, it is common to run a low-fidelity model with a subgrid-scale model to reduce the computational cost, but selecting the appropriate…
▽ More
The growing computing power over the years has enabled simulations to become more complex and accurate. While immensely valuable for scientific discovery and problem-solving, however, high-fidelity simulations come with significant computational demands. As a result, it is common to run a low-fidelity model with a subgrid-scale model to reduce the computational cost, but selecting the appropriate subgrid-scale models and tuning them are challenging. We propose a novel method for learning the subgrid-scale model effects when simulating partial differential equations augmented by neural ordinary differential operators in the context of discontinuous Galerkin (DG) spatial discretization. Our approach learns the missing scales of the low-order DG solver at a continuous level and hence improves the accuracy of the low-order DG approximations as well as accelerates the filtered high-order DG simulations with a certain degree of precision. We demonstrate the performance of our approach through multidimensional Taylor-Green vortex examples at different Reynolds numbers and times, which cover laminar, transitional, and turbulent regimes. The proposed method not only reconstructs the subgrid-scale from the low-order (1st-order) approximation but also speeds up the filtered high-order DG (6th-order) simulation by two orders of magnitude.
△ Less
Submitted 30 January, 2024; v1 submitted 29 October, 2023;
originally announced October 2023.
-
Smooth Number Message Authentication Code in the IoT Landscape
Authors:
Eduard-Matei Constantinescu,
Mohammed Elhajj,
Luca Mariot
Abstract:
This paper presents the Smooth Number Message Authentication Code (SNMAC) for the context of lightweight IoT devices. The proposal is based on the use of smooth numbers in the field of cryptography, and investigates how one can use them to improve the security and performance of various algorithms or security constructs. The literature findings suggest that current IoT solutions are viable and pro…
▽ More
This paper presents the Smooth Number Message Authentication Code (SNMAC) for the context of lightweight IoT devices. The proposal is based on the use of smooth numbers in the field of cryptography, and investigates how one can use them to improve the security and performance of various algorithms or security constructs. The literature findings suggest that current IoT solutions are viable and promising, yet they should explore the potential usage of smooth numbers. The methodology involves several processes, including the design, implementation, and results evaluation. After introducing the algorithm, provides a detailed account of the experimental performance analysis of the SNMAC solution, showcasing its efficiency in real-world scenarios. Furthermore, the paper also explores the security aspects of the proposed SNMAC algorithm, offering valuable insights into its robustness and applicability for ensuring secure communication within IoT environments.
△ Less
Submitted 21 October, 2023;
originally announced October 2023.
-
Scalable Implicit Solvers with Dynamic Mesh Adaptation for a Relativistic Drift-Kinetic Fokker-Planck-Boltzmann Model
Authors:
Johann Rudi,
Max Heldman,
Emil M. Constantinescu,
Qi Tang,
Xian-Zhu Tang
Abstract:
In this work we consider a relativistic drift-kinetic model for runaway electrons along with a Fokker-Planck operator for small-angle Coulomb collisions, a radiation dam** operator, and a secondary knock-on (Boltzmann) collision source. We develop a new scalable fully implicit solver utilizing finite volume and conservative finite difference schemes and dynamic mesh adaptivity. A new data manage…
▽ More
In this work we consider a relativistic drift-kinetic model for runaway electrons along with a Fokker-Planck operator for small-angle Coulomb collisions, a radiation dam** operator, and a secondary knock-on (Boltzmann) collision source. We develop a new scalable fully implicit solver utilizing finite volume and conservative finite difference schemes and dynamic mesh adaptivity. A new data management framework in the PETSc library based on the p4est library is developed to enable simulations with dynamic adaptive mesh refinement (AMR), distributed memory parallelization, and dynamic load balancing of computational work. This framework and the runaway electron solver building on the framework are able to dynamically capture both bulk Maxwellian at the low-energy region and a runaway tail at the high-energy region. To effectively capture features via the AMR algorithm, a new AMR indicator prediction strategy is proposed that is performed alongside the implicit time evolution of the solution. This strategy is complemented by the introduction of computationally cheap feature-based AMR indicators that are analyzed theoretically. Numerical results quantify the advantages of the prediction strategy in better capturing features compared with nonpredictive strategies; and we demonstrate trade-offs regarding computational costs. The robustness with respect to model parameters, algorithmic scalability, and parallel scalability are demonstrated through several benchmark problems including manufactured solutions and solutions of different physics models. We focus on demonstrating the advantages of using implicit time step** and AMR for runaway electron simulations.
△ Less
Submitted 20 March, 2024; v1 submitted 29 March, 2023;
originally announced March 2023.
-
Efficient Computation of Power System Maximum Transient Linear Growth
Authors:
Daniel Adrian Maldonado,
Emil Constantinescu,
Junbo Zhao,
Mihai Anitescu
Abstract:
Existing methods to determine the stability of a power system to small perturbations are based on eigenvalue analysis and focus on the asymptotic (long-term) behavior of the power grid. During the preasymptotic (short-term) transient, however, the system can exhibit large growth that is not explained by eigenvalues alone. In this paper we propose a new framework to determine the maximum (optimal)…
▽ More
Existing methods to determine the stability of a power system to small perturbations are based on eigenvalue analysis and focus on the asymptotic (long-term) behavior of the power grid. During the preasymptotic (short-term) transient, however, the system can exhibit large growth that is not explained by eigenvalues alone. In this paper we propose a new framework to determine the maximum (optimal) preasymptotic growth using the singular value decomposition. The approach is tailored to the analysis of quantities of interest in power system dynamics, such as the set of rotor speed deviations. Matrix-free techniques are developed to avoid the explicit formation of dense matrices and enable the analysis of large-scale systems without reaching memory bounds. Extensive results carried out from small to very large-scale systems (e.g., 70k-bus system) verify the theoretical aspects of the technique.
△ Less
Submitted 20 February, 2023;
originally announced February 2023.
-
Learning Subgrid-scale Models with Neural Ordinary Differential Equations
Authors:
Shinhoo Kang,
Emil M. Constantinescu
Abstract:
We propose a new approach to learning the subgrid-scale model when simulating partial differential equations (PDEs) solved by the method of lines and their representation in chaotic ordinary differential equations, based on neural ordinary differential equations (NODEs). Solving systems with fine temporal and spatial grid scales is an ongoing computational challenge, and closure models are general…
▽ More
We propose a new approach to learning the subgrid-scale model when simulating partial differential equations (PDEs) solved by the method of lines and their representation in chaotic ordinary differential equations, based on neural ordinary differential equations (NODEs). Solving systems with fine temporal and spatial grid scales is an ongoing computational challenge, and closure models are generally difficult to tune. Machine learning approaches have increased the accuracy and efficiency of computational fluid dynamics solvers. In this approach neural networks are used to learn the coarse- to fine-grid map, which can be viewed as subgrid-scale parameterization. We propose a strategy that uses the NODE and partial knowledge to learn the source dynamics at a continuous level. Our method inherits the advantages of NODEs and can be used to parameterize subgrid scales, approximate coupling operators, and improve the efficiency of low-order solvers. Numerical results with the two-scale Lorenz 96 ODE, the convection-diffusion PDE, and the viscous Burgers' PDE are used to illustrate this approach.
△ Less
Submitted 12 April, 2023; v1 submitted 19 December, 2022;
originally announced December 2022.
-
GPU-Accelerated DNS of Compressible Turbulent Flows
Authors:
Youngdae Kim,
Debojyoti Ghosh,
Emil M. Constantinescu,
Ramesh Balakrishnan
Abstract:
This paper explores strategies to transform an existing CPU-based high-performance computational fluid dynamics solver, HyPar, for compressible flow simulations on emerging exascale heterogeneous (CPU+GPU) computing platforms. The scientific motivation for develo** a GPU-enhanced version of HyPar is to simulate canonical turbulent flows at the highest resolution possible on such platforms. We sh…
▽ More
This paper explores strategies to transform an existing CPU-based high-performance computational fluid dynamics solver, HyPar, for compressible flow simulations on emerging exascale heterogeneous (CPU+GPU) computing platforms. The scientific motivation for develo** a GPU-enhanced version of HyPar is to simulate canonical turbulent flows at the highest resolution possible on such platforms. We show that optimizing memory operations and thread blocks results in 200x speedup of computationally intensive kernels compared with a CPU core. Using multiple GPUs and CUDA-aware MPI communication, we demonstrate both strong and weak scaling of our GPU-based HyPar implementation on the NVIDIA Volta V100 GPUs. We simulate the decay of homogeneous isotropic turbulence in a triply periodic box on grids with up to $1024^3$ points (5.3 billion degrees of freedom) and on up to 1,024 GPUs. We compare the wall times for CPU-only and CPU+GPU simulations. The results presented in the paper are obtained on the Summit and Lassen supercomputers at Oak Ridge and Lawrence Livermore National Laboratories, respectively.
△ Less
Submitted 5 December, 2022; v1 submitted 29 November, 2022;
originally announced November 2022.
-
A scalable space-time domain decomposition approach for solving large-scale nonlinear regularized inverse ill-posed problems in 4D variational data assimilation
Authors:
Luisa D'Amore. Emil Constantinescu,
Luisa Carracciuolo
Abstract:
We develop innovative algorithms for solving the strong-constraint formulation of four-dimensional variational data assimilation in large-scale applications. We present a space-time decomposition approach that employs domain decomposition along both the spatial and temporal directions in the overlap** case and involves partitioning of both the solution and the operators. Starting from the global…
▽ More
We develop innovative algorithms for solving the strong-constraint formulation of four-dimensional variational data assimilation in large-scale applications. We present a space-time decomposition approach that employs domain decomposition along both the spatial and temporal directions in the overlap** case and involves partitioning of both the solution and the operators. Starting from the global functional defined on the entire domain, we obtain a type of regularized local functionals on the set of subdomains providing the order reduction of both the predictive and the data assimilation models. We analyze the algorithm convergence and its performance in terms of reduction of time complexity and algorithmic scalability. The numerical experiments are carried out on the shallow water equation on the sphere according to the setup available at the Ocean Synthesis/Reanalysis Directory provided by Hamburg University.
△ Less
Submitted 11 April, 2022;
originally announced May 2022.
-
Electron dynamics in extended systems within real-time time-dependent density functional theory
Authors:
Alina Kononov,
Cheng-Wei Lee,
Tatiane Pereira dos Santos,
Brian Robinson,
Yifan Yao,
Yi Yao,
Xavier Andrade,
Andrew David Baczewski,
Emil Constantinescu,
Alfredo A. Correa,
Yosuke Kanai,
Normand Modine,
Andre Schleife
Abstract:
Due to a beneficial balance of computational cost and accuracy, real-time time-dependent density functional theory has emerged as a promising first-principles framework to describe electron real-time dynamics. Here we discuss recent implementations around this approach, in particular in the context of complex, extended systems. Results include an analysis of the computational cost associated with…
▽ More
Due to a beneficial balance of computational cost and accuracy, real-time time-dependent density functional theory has emerged as a promising first-principles framework to describe electron real-time dynamics. Here we discuss recent implementations around this approach, in particular in the context of complex, extended systems. Results include an analysis of the computational cost associated with numerical propagation and when using absorbing boundary conditions. We extensively explore the shortcomings for describing electron-electron scattering in real time and compare to many-body perturbation theory. Modern improvements of the description of exchange and correlation are reviewed. In this work, we specifically focus on the Qb@ll code, which we have mainly used for these types of simulations over the last years, and we conclude by pointing to further progress needed going forward.
△ Less
Submitted 9 May, 2022;
originally announced May 2022.
-
Multirate Partitioned Runge-Kutta Methods for Coupled Navier-Stokes Equations
Authors:
Shinhoo Kang,
Alp Dener,
Aidan Hamilton,
Hong Zhang,
Emil M. Constantinescu,
Robert L. Jacob
Abstract:
Earth system models are complex integrated models of atmosphere, ocean, sea ice, and land surface. Coupling the components can be a significant challenge due to the difference in physics, temporal, and spatial scales. This study explores new coupling strategies for the fluid-fluid interaction problem based on multirate partitioned Runge-Kutta methods. We consider compressible Navier-Stokes equatio…
▽ More
Earth system models are complex integrated models of atmosphere, ocean, sea ice, and land surface. Coupling the components can be a significant challenge due to the difference in physics, temporal, and spatial scales. This study explores new coupling strategies for the fluid-fluid interaction problem based on multirate partitioned Runge-Kutta methods. We consider compressible Navier-Stokes equations with gravity coupled through a rigid-lid interface. Our large-scale numerical experiments reveal that multirate partitioned Runge-Kutta coupling schemes (1) can conserve total mass; (2) have second-order accuracy in time; and (3) provide favorable strong- and weak-scaling performance on modern computing architectures. We also show that the speedup factors of multirate partitioned Runge-Kutta methods match theoretical expectations over their base (single-rate) method.
△ Less
Submitted 10 April, 2023; v1 submitted 23 February, 2022;
originally announced February 2022.
-
Implicit Extensions of an Explicit Multirate Runge-Kutta Scheme
Authors:
Emil M. Constantinescu
Abstract:
We propose a new method that extends conservative explicit multirate methods to implicit explicit-multirate methods. We develop extensions of order one and two with different stability properties on the implicit side. The method is suitable for time-step** adaptive mesh refinement PDE discretizations with different degrees of stiffness. A numerical example with an advection-diffusion problem ill…
▽ More
We propose a new method that extends conservative explicit multirate methods to implicit explicit-multirate methods. We develop extensions of order one and two with different stability properties on the implicit side. The method is suitable for time-step** adaptive mesh refinement PDE discretizations with different degrees of stiffness. A numerical example with an advection-diffusion problem illustrates the new method's properties.
△ Less
Submitted 20 December, 2021;
originally announced December 2021.
-
Efficient high-dimensional variational data assimilation with machine-learned reduced-order models
Authors:
Romit Maulik,
Vishwas Rao,
Jiali Wang,
Gianmarco Mengaldo,
Emil Constantinescu,
Bethany Lusch,
Prasanna Balaprakash,
Ian Foster,
Rao Kotamarthi
Abstract:
Data assimilation (DA) in the geophysical sciences remains the cornerstone of robust forecasts from numerical models. Indeed, DA plays a crucial role in the quality of numerical weather prediction, and is a crucial building block that has allowed dramatic improvements in weather forecasting over the past few decades. DA is commonly framed in a variational setting, where one solves an optimization…
▽ More
Data assimilation (DA) in the geophysical sciences remains the cornerstone of robust forecasts from numerical models. Indeed, DA plays a crucial role in the quality of numerical weather prediction, and is a crucial building block that has allowed dramatic improvements in weather forecasting over the past few decades. DA is commonly framed in a variational setting, where one solves an optimization problem within a Bayesian formulation using raw model forecasts as a prior, and observations as likelihood. This leads to a DA objective function that needs to be minimized, where the decision variables are the initial conditions specified to the model. In traditional DA, the forward model is numerically and computationally expensive. Here we replace the forward model with a low-dimensional, data-driven, and differentiable emulator. Consequently, gradients of our DA objective function with respect to the decision variables are obtained rapidly via automatic differentiation. We demonstrate our approach by performing an emulator-assisted DA forecast of geopotential height. Our results indicate that emulator-assisted DA is faster than traditional equation-based DA forecasts by four orders of magnitude, allowing computations to be performed on a workstation rather than a dedicated high-performance computer. In addition, we describe accuracy benefits of emulator-assisted DA when compared to simply using the emulator for forecasting (i.e., without DA).
△ Less
Submitted 14 December, 2021;
originally announced December 2021.
-
Entropy-Preserving and Entropy-Stable Relaxation IMEX and Multirate Time-Step** Methods
Authors:
Shinhoo Kang,
Emil M. Constantinescu
Abstract:
We propose entropy-preserving and entropy-stable partitioned Runge--Kutta (RK) methods. In particular, we extend the explicit relaxation Runge--Kutta methods to IMEX--RK methods and a class of explicit second-order multirate methods for stiff problems arising from scale-separable or grid-induced stiffness in a system. The proposed approaches not only mitigate system stiffness but also fully suppor…
▽ More
We propose entropy-preserving and entropy-stable partitioned Runge--Kutta (RK) methods. In particular, we extend the explicit relaxation Runge--Kutta methods to IMEX--RK methods and a class of explicit second-order multirate methods for stiff problems arising from scale-separable or grid-induced stiffness in a system. The proposed approaches not only mitigate system stiffness but also fully support entropy-preserving and entropy-stability properties at a discrete level. The key idea of the relaxation approach is to adjust the step completion with a relaxation parameter so that the time-adjusted solution satisfies the entropy condition at a discrete level. The relaxation parameter is computed by solving a scalar nonlinear equation at each timestep in general; however, as for a quadratic entropy function, we theoretically derive the explicit form of the relaxation parameter and numerically confirm that the relaxation parameter works the Burgers equation. Several numerical results for ordinary differential equations and the Burgers equation are presented to demonstrate the entropy-conserving/stable behavior of these methods. We also compare the relaxation approach and the incremental direction technique for the Burgers equation with and without a limiter in the presence of shocks.
△ Less
Submitted 19 July, 2022; v1 submitted 19 August, 2021;
originally announced August 2021.
-
Trust-region approximation of extreme trajectories in power system dynamics
Authors:
Daniel Adrian Maldonado,
Emil Constantinescu,
Hong Zhang,
Vishwas Rao,
Mihai Anitescu
Abstract:
In this work we present a novel technique, based on a trust-region optimization algorithm and second-order trajectory sensitivities, to compute the extreme trajectories of power system dynamic simulations given a bounded set that represents parametric uncertainty. We show how this method, while remaining computationally efficient compared with sampling-based techniques, overcomes the limitations o…
▽ More
In this work we present a novel technique, based on a trust-region optimization algorithm and second-order trajectory sensitivities, to compute the extreme trajectories of power system dynamic simulations given a bounded set that represents parametric uncertainty. We show how this method, while remaining computationally efficient compared with sampling-based techniques, overcomes the limitations of previous sensitivity-based techniques to approximate the bounds of the trajectories, when the local approximation loses validity because of the nonlinearity. In addition, we show how this method can be adapted to account for those cases in which the initial conditions depend on the uncertain parameter. To conclude, we present several numerical experiments that showcase the accuracy and scalability of the technique, including a demonstration on the IEEE New England test system.
△ Less
Submitted 8 February, 2022; v1 submitted 30 June, 2021;
originally announced June 2021.
-
Optimal Checkpointing for Adjoint Multistage Time-Step** Schemes
Authors:
Hong Zhang,
Emil Constantinescu
Abstract:
We consider checkpointing strategies that minimize the number of recomputations needed when performing discrete adjoint computations using multistage time-step** schemes, which requires computing several substeps within one complete time step. In this case we propose two algorithms that can generate optimal checkpointing schedules under weak assumptions. The first is an extension of the seminal…
▽ More
We consider checkpointing strategies that minimize the number of recomputations needed when performing discrete adjoint computations using multistage time-step** schemes, which requires computing several substeps within one complete time step. In this case we propose two algorithms that can generate optimal checkpointing schedules under weak assumptions. The first is an extension of the seminal Revolve algorithm adapted to multistage schemes. The second algorithm, named CAMS, is developed based on dynamic programming, and it requires the least number of recomputations when compared with other algorithms. The CAMS algorithm is made publicly available in a library with bindings to C and Python. Numerical results illustrate that the proposed algorithms can deliver up to two times the speedup compared with that of classical Revolve. Moreover, we discuss a tailored implementation of an adjoint computation that is arguably better suited for mature scientific computing libraries by avoiding the central control assumed by the original checkpointing strategy. The proposed algorithms have been adopted by the PETSc TSAdjoint library. Their performance has been demonstrated with a large-scale PDE-constrained optimization problem on a leadership-class supercomputer.
△ Less
Submitted 28 April, 2022; v1 submitted 25 June, 2021;
originally announced June 2021.
-
Mass-Conserving Implicit-Explicit Methods for Coupled Compressible Navier-Stokes Equations
Authors:
Shinhoo Kang,
Emil M. Constantinescu,
Hong Zhang,
Robert L. Jacob
Abstract:
Earth system models are composed of coupled components that separately model systems such as the global atmosphere, ocean, and land surface. While these components are well developed, coupling them in a single system can be a significant challenge. Computational efficiency, accuracy, and stability are principal concerns. In this study, we focus on these issues. In particular, implicit-explicit (IM…
▽ More
Earth system models are composed of coupled components that separately model systems such as the global atmosphere, ocean, and land surface. While these components are well developed, coupling them in a single system can be a significant challenge. Computational efficiency, accuracy, and stability are principal concerns. In this study, we focus on these issues. In particular, implicit-explicit (IMEX) tight and loose coupling strategies are explored for handling different time scales. For a simplified model for the air-sea interaction problem, we consider coupled compressible Navier-Stokes equations with an interface condition. Under the rigid-lid assumption, horizontal momentum and heat flux are exchanged through the interface. Several numerical experiments are presented to demonstrate the stability of the coupling schemes. We show both numerically and theoretically that our IMEX coupling methods are mass conservative for a coupled compressible Navier-Stokes system with the rigid-lid condition.
△ Less
Submitted 19 May, 2021; v1 submitted 22 January, 2021;
originally announced January 2021.
-
Optimal Experimental Design for Inverse Problems in the Presence of Observation Correlations
Authors:
Ahmed Attia,
Emil Constantinescu
Abstract:
Optimal experimental design (OED) is the general formalism of sensor placement and decisions about the data collection strategy for engineered or natural experiments. This approach is prevalent in many critical fields such as battery design, numerical weather prediction, geosciences, and environmental and urban studies. State-of-the-art computational methods for experimental design, however, do no…
▽ More
Optimal experimental design (OED) is the general formalism of sensor placement and decisions about the data collection strategy for engineered or natural experiments. This approach is prevalent in many critical fields such as battery design, numerical weather prediction, geosciences, and environmental and urban studies. State-of-the-art computational methods for experimental design, however, do not accommodate correlation structure in observational errors produced by many expensive-to-operate devices such as X-ray machines or radar and satellite retrievals. Discarding evident data correlations leads to biased results, poor data collection decisions, and waste of valuable resources. We present a general formulation of the OED formalism for model-constrained large-scale Bayesian linear inverse problems, where measurement errors are generally correlated. The proposed approach utilizes the Hadamard product of matrices to formulate the weighted likelihood and is valid for both finite- and infinite- dimensional Bayesian inverse problems. We also discuss widely used approaches for relaxation of the binary OED problem, in light of the proposed pointwise weighting approach, and present a clear interpretation of the relaxed design and its effect on the observational error covariance. Extensive numerical experiments are carried out for empirical verification of the proposed approach by using an advection-diffusion model, where the objective is to optimally place a small set of sensors, under a limited budget, to predict the concentration of a contaminant in a bounded domain.
△ Less
Submitted 25 June, 2022; v1 submitted 28 July, 2020;
originally announced July 2020.
-
A Machine-Learning-Based Importance Sampling Method to Compute Rare Event Probabilities
Authors:
Vishwas Rao,
Romit Maulik,
Emil Constantinescu,
Mihai Anitescu
Abstract:
We develop a novel computational method for evaluating the extreme excursion probabilities arising from random initialization of nonlinear dynamical systems. The method uses excursion probability theory to formulate a sequence of Bayesian inverse problems that, when solved, yields the biasing distribution. Solving multiple Bayesian inverse problems can be expensive; more so in higher dimensions. T…
▽ More
We develop a novel computational method for evaluating the extreme excursion probabilities arising from random initialization of nonlinear dynamical systems. The method uses excursion probability theory to formulate a sequence of Bayesian inverse problems that, when solved, yields the biasing distribution. Solving multiple Bayesian inverse problems can be expensive; more so in higher dimensions. To alleviate the computational cost, we build machine-learning-based surrogates to solve the Bayesian inverse problems that give rise to the biasing distribution. This biasing distribution can then be used in an importance sampling procedure to estimate the extreme excursion probabilities.
△ Less
Submitted 4 June, 2020;
originally announced June 2020.
-
Probabilistic analysis of masked loads with aggregated photovoltaic production
Authors:
Shaohui Liu,
Daniel Adrian Maldonado,
Emil M. Constantinescu
Abstract:
In this paper we present a probabilistic analysis framework to estimate behind-the-meter photovoltaic generation in real time. We develop a forward model consisting of a spatiotemporal stochastic process that represents the photovoltaic generation and a stochastic differential equation with jumps that represents the demand. We employ this model to disaggregate the behind-the-meter photovoltaic gen…
▽ More
In this paper we present a probabilistic analysis framework to estimate behind-the-meter photovoltaic generation in real time. We develop a forward model consisting of a spatiotemporal stochastic process that represents the photovoltaic generation and a stochastic differential equation with jumps that represents the demand. We employ this model to disaggregate the behind-the-meter photovoltaic generation using net load and irradiance measurements.
△ Less
Submitted 21 April, 2020;
originally announced April 2020.
-
PETSc TSAdjoint: a discrete adjoint ODE solver for first-order and second-order sensitivity analysis
Authors:
Hong Zhang,
Emil M. Constantinescu,
Barry F. Smith
Abstract:
We present a new software system PETSc TSAdjoint for first-order and second-order adjoint sensitivity analysis of time-dependent nonlinear differential equations. The derivative calculation in PETSc TSAdjoint is essentially a high-level algorithmic differentiation process. The adjoint models are derived by differentiating the timestep** algorithms and implemented based on the parallel infrastruc…
▽ More
We present a new software system PETSc TSAdjoint for first-order and second-order adjoint sensitivity analysis of time-dependent nonlinear differential equations. The derivative calculation in PETSc TSAdjoint is essentially a high-level algorithmic differentiation process. The adjoint models are derived by differentiating the timestep** algorithms and implemented based on the parallel infrastructure in PETSc. Full differentiation of the library code including MPI routines thus is avoided, and users do not need to derive their own adjoint models for their specific applications. PETSc TSAdjoint can compute the first-order derivative, that is, the gradient of a scalar functional, and the Hessian-vector product that carries second-order derivative information, while requiring minimal input (a few callbacks) from the users. Optimal checkpointing schemes are employed by the adjoint model in a manner that is transparent to users. Usability, efficiency, and scalability are demonstrated through examples from a variety of applications.
△ Less
Submitted 26 October, 2021; v1 submitted 16 December, 2019;
originally announced December 2019.
-
Stability Analysis of Interface Conditions for Ocean-Atmosphere Coupling
Authors:
Hong Zhang,
Zhengyu Liu,
Emil Constantinescu,
Robert Jacob
Abstract:
In this paper we analyze the stability of different coupling strategies for multidomain PDEs that arise in general circulation models used in climate simulations. We focus on fully coupled ocean-atmosphere models that are needed to represent and understand the complicated interactions of these two systems, becoming increasingly important in climate change assessment in recent years. Numerical stab…
▽ More
In this paper we analyze the stability of different coupling strategies for multidomain PDEs that arise in general circulation models used in climate simulations. We focus on fully coupled ocean-atmosphere models that are needed to represent and understand the complicated interactions of these two systems, becoming increasingly important in climate change assessment in recent years. Numerical stability issues typically arise because of different time-step** strategies applied to the coupled PDE system. In particular, the contributing factors include using large time steps, lack of accurate interface flux, and singe-iteration coupling. We investigate the stability of the coupled ocean-atmosphere models for various interface conditions such as the Dirichlet-Neumann condition and the bulk interface condition, which is unique to climate modeling. By analyzing a simplified model, we demonstrate how the parameterization of the bulk condition and other numerical and physical parameters affect the coupling stability.
△ Less
Submitted 16 September, 2019; v1 submitted 2 September, 2019;
originally announced September 2019.
-
Statistical Treatment of Inverse Problems Constrained by Differential Equations-Based Models with Stochastic Terms
Authors:
Emil M. Constantinescu,
Noemi Petra,
Julie Bessac,
Cosmin G. Petra
Abstract:
This paper introduces a statistical treatment of inverse problems constrained by models with stochastic terms. The solution of the forward problem is given by a distribution represented numerically by an ensemble of simulations. The goal is to formulate the inverse problem, in particular the objective function, to find the closest forward distribution (i.e., the output of the stochastic forward pr…
▽ More
This paper introduces a statistical treatment of inverse problems constrained by models with stochastic terms. The solution of the forward problem is given by a distribution represented numerically by an ensemble of simulations. The goal is to formulate the inverse problem, in particular the objective function, to find the closest forward distribution (i.e., the output of the stochastic forward problem) that best explains the distribution of the observations in a certain metric. We use proper scoring rules, a concept employed in statistical forecast verification, namely energy, variogram, and hybrid (i.e., combination of the two) scores. We study the performance of the proposed formulation in the context of two applications: a coefficient field inversion for subsurface flow governed by an elliptic partial differential equation (PDE) with a stochastic source and a parameter inversion for power grid governed by differential-algebraic equations (DAEs). In both cases we show that the variogram and the hybrid scores show better parameter inversion results than does the energy score, whereas the energy score leads to better probabilistic predictions.
△ Less
Submitted 15 April, 2019; v1 submitted 15 October, 2018;
originally announced October 2018.
-
Validation of a PETSc based software implementing a 4DVAR Data Assimilation algorithm: a case study related with an Oceanic Model based on Shallow Water equation
Authors:
Luisa Carracciuolo,
Emil M. Constantinescu,
Luisa D'Amore
Abstract:
In this work are presented and discussed some results related to the validation process of a software module based on PETSc which implements a Data Assimilation algorithm.
In this work are presented and discussed some results related to the validation process of a software module based on PETSc which implements a Data Assimilation algorithm.
△ Less
Submitted 3 October, 2018; v1 submitted 2 October, 2018;
originally announced October 2018.
-
An Optimal Experimental Design Framework for Adaptive Inflation and Covariance Localization for Ensemble Filters
Authors:
Ahmed Attia,
Emil Constantinescu
Abstract:
We develop an optimal experimental design framework for adapting the covariance inflation and localization in data assimilation problems. Covariance inflation and localization are ubiquitously employed to alleviate the effect of using ensembles of finite sizes in all practical data assimilation systems. The choice of both the inflation factor and the localization radius can have a significant impa…
▽ More
We develop an optimal experimental design framework for adapting the covariance inflation and localization in data assimilation problems. Covariance inflation and localization are ubiquitously employed to alleviate the effect of using ensembles of finite sizes in all practical data assimilation systems. The choice of both the inflation factor and the localization radius can have a significant impact on the performance of the assimilation scheme. These parameters are generally tuned by trial and error, rendering them expensive to optimize in practice. Spatially and temporally varying inflation parameter and localization radii have been recently proposed and have been empirically proven to enhance the performance of the employed assimilation filter. In this study, we present a variational framework for adaptive tuning of the inflation and localization parameters. Each of these parameters is optimized independently, with an objective to minimize the uncertainty in the posterior state. The proposed framework does not assume uncorrelated observations or prior errors and can in principle be applied without expert knowledge about the model and the observations. Thus, it is adequate for handling dense as well as sparse observational networks. We present the mathematical formulation, algorithmic description of the approach, and numerical experiments using the two-layer Lorenz-96 model.
△ Less
Submitted 24 March, 2019; v1 submitted 27 June, 2018;
originally announced June 2018.
-
PETSc/TS: A Modern Scalable ODE/DAE Solver Library
Authors:
Shrirang Abhyankar,
Jed Brown,
Emil M. Constantinescu,
Debojyoti Ghosh,
Barry F. Smith,
Hong Zhang
Abstract:
High-quality ordinary differential equation (ODE) solver libraries have a long history, going back to the 1970s. Over the past several years we have implemented, on top of the PETSc linear and nonlinear solver package, a new general-purpose, extensive, extensible library for solving ODEs and differential algebraic equations (DAEs). Package includes support for both forward and adjoint sensitivitie…
▽ More
High-quality ordinary differential equation (ODE) solver libraries have a long history, going back to the 1970s. Over the past several years we have implemented, on top of the PETSc linear and nonlinear solver package, a new general-purpose, extensive, extensible library for solving ODEs and differential algebraic equations (DAEs). Package includes support for both forward and adjoint sensitivities that can be easily utilized by the TAO optimization package, which is also part of PETSc. The ODE/DAE integrator library strives to be highly scalable but also to deliver high efficiency for modest-sized problems. The library includes explicit solvers, implicit solvers, and a collection of implicit-explicit solvers, all with a common user interface and runtime selection of solver types, adaptive error control, and monitoring of solution progress. The library also offers enormous flexibility in selection of nonlinear and linear solvers, including the entire suite of PETSc iterative solvers, as well as several parallel direct solvers.
△ Less
Submitted 4 June, 2018;
originally announced June 2018.
-
A scalable matrix-free spectral element approach for unsteady PDE constrained optimization using PETSc/TAO
Authors:
Oana Marin,
Emil Constantinescu,
Barry Smith
Abstract:
We provide a new approach for the efficient matrix-free application of the transpose of the Jacobian for the spectral element method for the adjoint based solution of partial differential equation (PDE) constrained optimization. This results in optimizations of nonlinear PDEs using explicit integrators where the integration of the adjoint problem is not more expensive than the forward simulation.…
▽ More
We provide a new approach for the efficient matrix-free application of the transpose of the Jacobian for the spectral element method for the adjoint based solution of partial differential equation (PDE) constrained optimization. This results in optimizations of nonlinear PDEs using explicit integrators where the integration of the adjoint problem is not more expensive than the forward simulation. Solving PDE constrained optimization problems entails combining expertise from multiple areas, including simulation, computation of derivatives, and optimization. The Portable, Extensible Toolkit for Scientific computation (PETSc) together with its companion package, the Toolkit for Advanced Optimization (TAO), is an integrated numerical software library that contains an algorithmic/software stack for solving linear systems, nonlinear systems, ordinary differential equations, differential algebraic equations, and large-scale optimization problems and, as such, is an ideal tool for performing PDE-constrained optimization. This paper describes an efficient approach in which the software stack provided by PETSc/TAO can be used for large-scale nonlinear time-dependent problems. While time integration can involve a range of high-order methods, both implicit and explicit. The PDE-constrained optimization algorithm used is gradient-based and seamlessly integrated with the simulation of the physical problem.
△ Less
Submitted 26 May, 2020; v1 submitted 4 June, 2018;
originally announced June 2018.
-
Global sensitivity analysis for statistical model parameters
Authors:
Joseph Hart,
Julie Bessac,
Emil Constantinescu
Abstract:
Global sensitivity analysis (GSA) is frequently used to analyze the influence of uncertain parameters in mathematical models and simulations. In principle, tools from GSA may be extended to analyze the influence of parameters in statistical models. Such analyses may enable reduced or parsimonious modeling and greater predictive capability. However, difficulties such as parameter correlation, model…
▽ More
Global sensitivity analysis (GSA) is frequently used to analyze the influence of uncertain parameters in mathematical models and simulations. In principle, tools from GSA may be extended to analyze the influence of parameters in statistical models. Such analyses may enable reduced or parsimonious modeling and greater predictive capability. However, difficulties such as parameter correlation, model stochasticity, multivariate model output, and unknown parameter distributions prohibit a direct application of GSA tools to statistical models. By leveraging a loss function associated with the statistical model, we introduce a novel framework to address these difficulties and enable efficient GSA for statistical model parameters. Theoretical and computational properties are considered and illustrated on a synthetic example. The framework is applied to a Gaussian process model from the literature, which depends on 95 parameters. Non-influential parameters are discovered through GSA and a reduced model with equal or stronger predictive capability is constructed by using only 79 parameters.
△ Less
Submitted 28 June, 2018; v1 submitted 24 August, 2017;
originally announced August 2017.
-
Acceleration of the Implicit-Explicit Non-hydrostatic Unified Model of the Atmosphere (NUMA) on Manycore Processors
Authors:
Daniel S. Abdi,
Francis X. Giraldo,
Emil M. Constantinescu,
Lester E. Carr III,
Lucas C. Wilcox,
Timothy C. Warburton
Abstract:
We present the acceleration of an IMplicit-EXplicit (IMEX) non-hydrostatic atmospheric model on manycore processors such as GPUs and Intel's MIC architecture. IMEX time integration methods sidestep the constraint imposed by the Courant-Friedrichs-Lewy condition on explicit methods through corrective implicit solves within each time step. In this work, we implement and evaluate the performance of I…
▽ More
We present the acceleration of an IMplicit-EXplicit (IMEX) non-hydrostatic atmospheric model on manycore processors such as GPUs and Intel's MIC architecture. IMEX time integration methods sidestep the constraint imposed by the Courant-Friedrichs-Lewy condition on explicit methods through corrective implicit solves within each time step. In this work, we implement and evaluate the performance of IMEX on manycore processors relative to explicit methods. Using 3D-IMEX at Courant number C=15 , we obtained a speedup of about 4X relative to an explicit time step** method run with the maximum allowable C=1. In addition, we demonstrate a much larger speedup of 100X at C=150 using 1D-IMEX due to the unconditional stability of the method in the vertical direction. Several improvements on the IMEX procedure were necessary in order to outperform our results with explicit methods: a) reducing the number of degrees of freedom of the IMEX formulation by forming the Schur complement; b) formulating a horizontally-explicit vertically-implicit (HEVI) 1D-IMEX scheme that has a lower workload and potentially better scalability than 3D-IMEX; c) using high-order polynomial preconditioners to reduce the condition number of the resulting system; d) using a direct solver for the 1D-IMEX method by performing and storing LU factorizations once to obtain a constant cost for any Courant number. Without all of these improvements, explicit time integration methods turned out to be difficult to beat. We discuss in detail the IMEX infrastructure required for formulating and implementing efficient methods on manycore processors. Finally, we validate our results with standard benchmark problems in NWP and evaluate the performance and scalability of the IMEX method using up to 4192 GPUs and 16 Knights Landing processors.
△ Less
Submitted 13 February, 2017;
originally announced February 2017.
-
A continuous/discontinuous Galerkin solution of the shallow water equations with dynamic viscosity, high-order wetting and drying, and implicit time integration
Authors:
Simone Marras,
Michal A. Kopera,
Emil M. Constantinescu,
Jenny Suckale,
Francis X. Giraldo
Abstract:
The high-order numerical solution of the non-linear shallow water equations (and of hyperbolic systems in general) is susceptible to unphysical Gibbs oscillations that form in the proximity of strong gradients. The solution to this problem is still an active field of research as no general cure has been found yet. In this paper, we tackle this issue by presenting a dynamically adaptive viscosity b…
▽ More
The high-order numerical solution of the non-linear shallow water equations (and of hyperbolic systems in general) is susceptible to unphysical Gibbs oscillations that form in the proximity of strong gradients. The solution to this problem is still an active field of research as no general cure has been found yet. In this paper, we tackle this issue by presenting a dynamically adaptive viscosity based on a residual-based sub-grid scale model that has the following properties: $(i)$ it removes the spurious oscillations in the proximity of strong wave fronts while preserving the overall accuracy and sharpness of the solution. This is possible because of the residual-based definition of the dynamic diffusion coefficient. $(ii)$ For coarse grids, it prevents energy from building up at small wave-numbers. $(iii)$ The model has no tunable parameter. Our interest in the shallow water equations is tied to the simulation of coastal inundation, where a careful handling of the transition between dry and wet surfaces is particularly challenging for high-order Galerkin approximations. In this paper, we extend to a unified continuous/discontinuous Galerkin (CG/DG) framework a very simple, yet effective wetting and drying algorithm originally designed for DG [Xing, Zhang, Shu (2010)]. We show its effectiveness for problems in one and two dimensions on domains of increasing characteristic lengths varying from centimeters to kilometers. Finally, to overcome the time-step restriction incurred by the high-order Galerkin approximation, we advance the equations forward in time via a three stage, second order explicit-first-stage, singly diagonally implicit Runge-Kutta (ESDIRK) time integration scheme. Via ESDIRK, we are able to preserve numerical stability for an advective CFL number 10 times larger than its explicit counterpart.
△ Less
Submitted 15 July, 2016;
originally announced July 2016.
-
A Bayesian Approach for Parameter Estimation with Uncertainty for Dynamic Power Systems
Authors:
Noemi Petra,
Cosmin G. Petra,
Zheng Zhang,
Emil M. Constantinescu,
Mihai Anitescu
Abstract:
We address the problem of estimating the uncertainty in the solution of power grid inverse problems within the framework of Bayesian inference. We investigate two approaches, an adjoint-based method and a stochastic spectral method. These methods are used to estimate the maximum a posteriori point of the parameters and their variance, which quantifies their uncertainty. Within this framework we es…
▽ More
We address the problem of estimating the uncertainty in the solution of power grid inverse problems within the framework of Bayesian inference. We investigate two approaches, an adjoint-based method and a stochastic spectral method. These methods are used to estimate the maximum a posteriori point of the parameters and their variance, which quantifies their uncertainty. Within this framework we estimate several parameters of the dynamic power system, such as generator inertias, which are not quantifiable in steady-state models. We illustrate the performance of these approaches on a 9-bus power grid example and analyze the dependence on measurement frequency, estimation horizon, perturbation size, and measurement noise. We assess the computational efficiency, and discuss the expected performance when these methods are applied to large systems.
△ Less
Submitted 11 February, 2016; v1 submitted 27 January, 2016;
originally announced January 2016.
-
Stochastic simulation of predictive space-time scenarios of wind speed using observations and physical models
Authors:
Julie Bessac,
Emil Mihai Constantinescu,
Mihai Anitescu
Abstract:
We propose a statistical space-time model for predicting atmospheric wind speed based on deterministic numerical weather predictions and historical measurements. We consider a Gaussian multivariate space-time framework that combines multiple sources of past physical model outputs and measurements along with model predictions in order to produce a probabilistic wind speed forecast within the predic…
▽ More
We propose a statistical space-time model for predicting atmospheric wind speed based on deterministic numerical weather predictions and historical measurements. We consider a Gaussian multivariate space-time framework that combines multiple sources of past physical model outputs and measurements along with model predictions in order to produce a probabilistic wind speed forecast within the prediction window. We illustrate this strategy on a ground wind speed forecast for several months in 2012 for a region near the Great Lakes in the United States. The results show that the prediction is improved in the mean-squared sense relative to the numerical forecasts as well as in probabilistic scores. Moreover, the samples are shown to produce realistic wind scenarios based on the sample spectrum.
△ Less
Submitted 20 October, 2016; v1 submitted 30 November, 2015;
originally announced November 2015.
-
Semi-Implicit Time Integration of Atmospheric Flows with Characteristic-Based Flux Partitioning
Authors:
Debojyoti Ghosh,
Emil M. Constantinescu
Abstract:
This paper presents a characteristic-based flux partitioning for the semi-implicit time integration of atmospheric flows. Nonhydrostatic models require the solution of the compressible Euler equations. The acoustic time-scale is significantly faster than the advective scale, yet it is typically not relevant to atmospheric and weather phenomena. The acoustic and advective components of the hyperbol…
▽ More
This paper presents a characteristic-based flux partitioning for the semi-implicit time integration of atmospheric flows. Nonhydrostatic models require the solution of the compressible Euler equations. The acoustic time-scale is significantly faster than the advective scale, yet it is typically not relevant to atmospheric and weather phenomena. The acoustic and advective components of the hyperbolic flux are separated in the characteristic space. High-order, conservative additive Runge-Kutta methods are applied to the partitioned equations so that the acoustic component is integrated in time implicitly with an unconditionally stable method, while the advective component is integrated explicitly. The time step of the overall algorithm is thus determined by the advective scale. Benchmark flow problems are used to demonstrate the accuracy, stability, and convergence of the proposed algorithm. The computational cost of the partitioned semi-implicit approach is compared with that of explicit time integration.
△ Less
Submitted 14 April, 2016; v1 submitted 20 October, 2015;
originally announced October 2015.
-
Estimating Global Errors in Time Step**
Authors:
Emil Constantinescu
Abstract:
This study introduces new time-step** strategies with built-in global error estimators. The new methods propagate the defect along with the numerical solution much like solving for the correction or Zadunaisky's procedure; however, the proposed approach allows for overlapped internal computations and, therefore, represents a generalization of the classical numerical schemes for solving different…
▽ More
This study introduces new time-step** strategies with built-in global error estimators. The new methods propagate the defect along with the numerical solution much like solving for the correction or Zadunaisky's procedure; however, the proposed approach allows for overlapped internal computations and, therefore, represents a generalization of the classical numerical schemes for solving differential equations with global error estimation. The resulting algorithms can be effectively represented as general linear methods. We present a few explicit self-starting schemes akin to Runge-Kutta methods with global error estimation and illustrate the theoretical considerations on several examples.
△ Less
Submitted 9 May, 2017; v1 submitted 17 March, 2015;
originally announced March 2015.
