-
A Note on Dimensionality Reduction in Deep Neural Networks using Empirical Interpolation Method
Authors:
Harbir Antil,
Madhu Gupta,
Randy Price
Abstract:
Empirical interpolation method (EIM) is a well-known technique to efficiently approximate parameterized functions. This paper proposes to use EIM algorithm to efficiently reduce the dimension of the training data within supervised machine learning. This is termed as DNN-EIM. Applications in data science (e.g., MNIST) and parameterized (and time-dependent) partial differential equations (PDEs) are…
▽ More
Empirical interpolation method (EIM) is a well-known technique to efficiently approximate parameterized functions. This paper proposes to use EIM algorithm to efficiently reduce the dimension of the training data within supervised machine learning. This is termed as DNN-EIM. Applications in data science (e.g., MNIST) and parameterized (and time-dependent) partial differential equations (PDEs) are considered. The proposed DNNs in case of classification are trained in parallel for each class. This approach is sequential, i.e., new classes can be added without having to retrain the network. In case of PDEs, a DNN is designed corresponding to each EIM point. Again, these networks can be trained in parallel, for each EIM point. In all cases, the parallel networks require fewer than ten times the number of training weights. Significant gains are observed in terms of training times, without sacrificing accuracy.
△ Less
Submitted 16 May, 2023;
originally announced May 2023.
-
A stochastic Galerkin method with adaptive time-step** for the Navier-Stokes equations
Authors:
Bedřich Sousedík,
Randy Price
Abstract:
We study the time-dependent Navier-Stokes equations in the context of stochastic finite element discretizations. Specifically, we assume that the viscosity is a random field given in the form of a generalized polynomial chaos expansion, and we use the stochastic Galerkin method to extend the methodology from [D. A. Kay et al., \textit{SIAM J. Sci. Comput.} 32(1), pp. 111--128, 2010] into this fram…
▽ More
We study the time-dependent Navier-Stokes equations in the context of stochastic finite element discretizations. Specifically, we assume that the viscosity is a random field given in the form of a generalized polynomial chaos expansion, and we use the stochastic Galerkin method to extend the methodology from [D. A. Kay et al., \textit{SIAM J. Sci. Comput.} 32(1), pp. 111--128, 2010] into this framework. For the resulting stochastic problem, we explore the properties of the resulting stochastic solutions, and we also compare the results with that of Monte Carlo and stochastic collocation. Since the time-step** scheme is fully implicit, we also propose strategies for efficient solution of the stochastic Galerkin linear systems using a preconditioned Krylov subspace method. The effectiveness of the stochastic Galerkin method is illustrated by numerical experiments.
△ Less
Submitted 10 July, 2022;
originally announced July 2022.
-
NINNs: Nudging Induced Neural Networks
Authors:
Harbir Antil,
Rainald Löhner,
Randy Price
Abstract:
New algorithms called nudging induced neural networks (NINNs), to control and improve the accuracy of deep neural networks (DNNs), are introduced. The NINNs framework can be applied to almost all pre-existing DNNs, with forward propagation, with costs comparable to existing DNNs. NINNs work by adding a feedback control term to the forward propagation of the network. The feedback term nudges the ne…
▽ More
New algorithms called nudging induced neural networks (NINNs), to control and improve the accuracy of deep neural networks (DNNs), are introduced. The NINNs framework can be applied to almost all pre-existing DNNs, with forward propagation, with costs comparable to existing DNNs. NINNs work by adding a feedback control term to the forward propagation of the network. The feedback term nudges the neural network towards a desired quantity of interest. NINNs offer multiple advantages, for instance, they lead to higher accuracy when compared with existing data assimilation algorithms such as nudging. Rigorous convergence analysis is established for NINNs. The algorithmic and theoretical findings are illustrated on examples from data assimilation and chemically reacting flows.
△ Less
Submitted 15 March, 2022;
originally announced March 2022.
-
Data Assimilation with Deep Neural Nets Informed by Nudging
Authors:
Harbir Antil,
Rainald Löhner,
Randy Price
Abstract:
The nudging data assimilation algorithm is a powerful tool used to forecast phenomena of interest given incomplete and noisy observations. Machine learning is becoming increasingly popular in data assimilation given its ease of computation and forecasting ability. This work proposes a new approach to data assimilation via machine learning where Deep Neural Networks (DNNs) are being taught the nudg…
▽ More
The nudging data assimilation algorithm is a powerful tool used to forecast phenomena of interest given incomplete and noisy observations. Machine learning is becoming increasingly popular in data assimilation given its ease of computation and forecasting ability. This work proposes a new approach to data assimilation via machine learning where Deep Neural Networks (DNNs) are being taught the nudging algorithm. The accuracy of the proposed DNN based algorithm is comparable to the nudging algorithm and it is confirmed by the Lorenz 63 and Lorenz 96 numerical examples. The key advantage of the proposed approach is the fact that, once trained, DNNs are cheap to evaluate in comparison to nudging where typically differential equations are needed to be solved. Standard exponential type approximation results are established for the Lorenz 63 model for both the continuous and discrete in time models. These results can be directly coupled with estimates for DNNs (whenever available), to derive the overall approximation error estimates of the proposed algorithm.
△ Less
Submitted 22 November, 2021;
originally announced November 2021.
-
On Surrogate Learning for Linear Stability Assessment of Navier-Stokes Equations with Stochastic Viscosity
Authors:
Bedřich Sousedík,
Howard C. Elman,
Kook** Lee,
Randy Price
Abstract:
We study linear stability of solutions to the Navier\textendash Stokes equations with stochastic viscosity. Specifically, we assume that the viscosity is given in the form of a~stochastic expansion. Stability analysis requires a solution of the steady-state Navier-Stokes equation and then leads to a generalized eigenvalue problem, from which we wish to characterize the real part of the rightmost e…
▽ More
We study linear stability of solutions to the Navier\textendash Stokes equations with stochastic viscosity. Specifically, we assume that the viscosity is given in the form of a~stochastic expansion. Stability analysis requires a solution of the steady-state Navier-Stokes equation and then leads to a generalized eigenvalue problem, from which we wish to characterize the real part of the rightmost eigenvalue. While this can be achieved by Monte Carlo simulation, due to its computational cost we study three surrogates based on generalized polynomial chaos, Gaussian process regression and a shallow neural network. The results of linear stability analysis assessment obtained by the surrogates are compared to that of Monte Carlo simulation using a set of numerical experiments.
△ Less
Submitted 3 March, 2022; v1 submitted 28 February, 2021;
originally announced March 2021.
-
Continuous Data Assimilation for the Three Dimensional Navier-Stokes Equations
Authors:
Animikh Biswas,
Randy Price
Abstract:
In this paper, we provide conditions, \emph{based solely on the observed data}, for the global well-posedness, regularity and convergence of the Azouni-Olson-Titi data assimilation algorithm (AOT algorithm) for a Leray-Hopf weak solutions of the three dimensional Navier-Stokes equations (3D NSE). The aforementioned conditions on the observations, which in this case comprise either of \emph{modal}…
▽ More
In this paper, we provide conditions, \emph{based solely on the observed data}, for the global well-posedness, regularity and convergence of the Azouni-Olson-Titi data assimilation algorithm (AOT algorithm) for a Leray-Hopf weak solutions of the three dimensional Navier-Stokes equations (3D NSE). The aforementioned conditions on the observations, which in this case comprise either of \emph{modal} or \emph{volume element observations}, are automatically satisfied for solutions that are globally regular and are uniformly bounded in the $H^1$-norm. However, neither regularity nor uniqueness is necessary for the efficacy of the AOT algorithm. To the best of our knowledge, this is the first such rigorous analysis of the AOT data assimilation algorithm for the 3D NSE.
△ Less
Submitted 2 March, 2020;
originally announced March 2020.
-
Word Length Perturbations in Certain Symmetric Presentations of Dihedral Groups
Authors:
Michael P. Allocca,
Jason M. Graham,
Candice R. Price,
Shannon N. Talbott,
Jennifer F. Vasquez
Abstract:
Given a finite group with a generating subset there is a well-established notion of length for a group element given in terms of its minimal length expression as a product of elements from the generating set. Recently, certain quantities called $λ_{1}$ and $λ_{2}$ have been defined that allow for a precise measure of how stable a group is under certain types of small perturbations in the generatin…
▽ More
Given a finite group with a generating subset there is a well-established notion of length for a group element given in terms of its minimal length expression as a product of elements from the generating set. Recently, certain quantities called $λ_{1}$ and $λ_{2}$ have been defined that allow for a precise measure of how stable a group is under certain types of small perturbations in the generating expressions for the elements of the group. These quantities provide a means to measure differences among all possible paths in a Cayley graph for a group, establish a group theoretic analog for the notion of stability in nonlinear dynamical systems, and play an important role in the application of groups to computational genomics. In this paper, we further expose the fundamental properties of $λ_{1}$ and $λ_{2}$ by establishing their bounds when the underlying group is a dihedral group. An essential step in our approach is to completely characterize so-called symmetric presentations of the dihedral groups, providing insight into the manner in which $λ_{1}$ and $λ_{2}$ interact with finite group presentations. This is of interest independent of the study of the quantities $λ_{1},\; λ_{2}$. Finally, we discuss several conjectures and open questions for future consideration.
△ Less
Submitted 30 December, 2016; v1 submitted 9 December, 2015;
originally announced December 2015.
