-
Probabilistic neural networks for predicting energy dissipation rates in geophysical turbulent flows
Authors:
Sam F. Lewin,
Stephen M. de Bruyn Kops,
Gavin D. Portwood,
Colm-cille P. Caulfield
Abstract:
Motivated by oceanographic observational datasets, we propose a probabilistic neural network (PNN) model for calculating turbulent energy dissipation rates from vertical columns of velocity and density gradients in density stratified turbulent flows. We train and test the model on high-resolution simulations of decaying turbulence designed to emulate geophysical conditions similar to those found i…
▽ More
Motivated by oceanographic observational datasets, we propose a probabilistic neural network (PNN) model for calculating turbulent energy dissipation rates from vertical columns of velocity and density gradients in density stratified turbulent flows. We train and test the model on high-resolution simulations of decaying turbulence designed to emulate geophysical conditions similar to those found in the ocean. The PNN model outperforms a baseline theoretical model widely used to compute dissipation rates from oceanographic observations of vertical shear, being more robust in capturing the tails of the output distributions at multiple different time points during turbulent decay. A differential sensitivity analysis indicates that this improvement may be attributed to the ability of the network to capture additional underlying physics introduced by density gradients in the flow.
△ Less
Submitted 2 December, 2021;
originally announced December 2021.
-
Validation and parameterization of a novel physics-constrained neural dynamics model applied to turbulent fluid flow
Authors:
Varun Shankar,
Gavin D. Portwood,
Arvind T. Mohan,
Peetak P. Mitra,
Dilip Krishnamurthy,
Christopher Rackauckas,
Lucas A. Wilson,
David P. Schmidt,
Venkatasubramanian Viswanathan
Abstract:
In fluid physics, data-driven models to enhance or accelerate solution methods are becoming increasingly popular for many application domains, such as alternatives to turbulence closures, system surrogates, or for new physics discovery. In the context of reduced order models of high-dimensional time-dependent fluid systems, machine learning methods grant the benefit of automated learning from data…
▽ More
In fluid physics, data-driven models to enhance or accelerate solution methods are becoming increasingly popular for many application domains, such as alternatives to turbulence closures, system surrogates, or for new physics discovery. In the context of reduced order models of high-dimensional time-dependent fluid systems, machine learning methods grant the benefit of automated learning from data, but the burden of a model lies on its reduced-order representation of both the fluid state and physical dynamics. In this work, we build a physics-constrained, data-driven reduced order model for the Navier-Stokes equations to approximate spatio-temporal turbulent fluid dynamics. The model design choices mimic numerical and physical constraints by, for example, implicitly enforcing the incompressibility constraint and utilizing continuous Neural Ordinary Differential Equations for tracking the evolution of the differential equation. We demonstrate this technique on three-dimensional, moderate Reynolds number turbulent fluid flow. In assessing the statistical quality and characteristics of the machine-learned model through rigorous diagnostic tests, we find that our model is capable of reconstructing the dynamics of the flow over large integral timescales, favoring accuracy at the larger length scales. More significantly, comprehensive diagnostics suggest that physically-interpretable model parameters, corresponding to the representations of the fluid state and dynamics, have attributable and quantifiable impact on the quality of the model predictions and computational complexity.
△ Less
Submitted 21 October, 2021;
originally announced October 2021.
-
Idealised Turbulent Wake With Steady, Non-Uniform Ambient Density Stratification
Authors:
G. D. Portwood,
S. M. de Bruyn Kops
Abstract:
Density stratification in geophysical environments can be non-uniform, particularly in thermohaline staircases and atmospheric layer transitions. Non-uniform stratification, however, is often approximated by the average ambient density change with height. This approximation is frequently made in numerical simulations because it greatly simplifies the calculations. In this paper, direct numerical s…
▽ More
Density stratification in geophysical environments can be non-uniform, particularly in thermohaline staircases and atmospheric layer transitions. Non-uniform stratification, however, is often approximated by the average ambient density change with height. This approximation is frequently made in numerical simulations because it greatly simplifies the calculations. In this paper, direct numerical simulations using $4096 \times 2048 \times 2048$ grid points to resolve an idealised turbulent wake in a non-uniformly stratified fluid are analysed to understand the consequences of assuming linear stratification. For flows with the same average change in density with height, but varied local stratification $d\bar ρ(z)/dz$, flow dynamics are dependent on the ratio $ξ=δ_{\cal U}/δ_ρ$, where $δ_{\cal U}$ and $δ_ρ$ are characteristic velocity and density vertical scale heights of the mean flow. Results suggest that a stably stratified flow will demonstrate characteristics similar to nonstratified flow when $ξ> 2$, even though the average stratification is quite strong. In particular, the results show that mixing is enhanced when $ξ> 2$.
△ Less
Submitted 9 November, 2020;
originally announced November 2020.
-
Implications of inertial subrange scaling for stably stratified mixing
Authors:
G. D. Portwood,
S. M. de Bruyn Kops,
C. P. Caulfield
Abstract:
The effects of turbulent dynamic range on scalar mixing in stably stratified turbulence are investigated by an adaptation of the theoretical passive scalar modelling arguments of Beguier et al. (1978) and demonstrated statistically using direct numerical simulations of statistically stationary homogeneous stratified and sheared turbulence (SHSST). By analysis of inertial and inertial-convective su…
▽ More
The effects of turbulent dynamic range on scalar mixing in stably stratified turbulence are investigated by an adaptation of the theoretical passive scalar modelling arguments of Beguier et al. (1978) and demonstrated statistically using direct numerical simulations of statistically stationary homogeneous stratified and sheared turbulence (SHSST). By analysis of inertial and inertial-convective subrange scaling, we show that the relationship between active scalar and turbulence time scales is predicted by the ratio of the Kolmogorov and Oboukhov-Corrsin constants provided there is sufficient scale separation for inertial and inertial-convective subrange scalings to be valid. With this analysis, we show that the turbulent mixing coefficient, $Γ\equiv χ/ε$, that is, within this context defined to be the ratio of available potential energy ($E_p$) and turbulent kinetic energy ($E_k$) dissipation rates, can be estimated by $E_p,E_k$ and a universal constant provided a Reynolds number is sufficiently high, observed here at $Re_b \equiv ε/ νN^2 \gtrapprox 300$ where $ν$ is the kinematic viscosity and $N$ is the characteristic buoyancy frequency. We propose a model for diapycnal mixing with robust theoretical parametrisation and asymptotic behaviour in this high-$Re_b$ limit.
△ Less
Submitted 1 April, 2022; v1 submitted 5 November, 2020;
originally announced November 2020.
-
Accelerating Training in Artificial Neural Networks with Dynamic Mode Decomposition
Authors:
Mauricio E. Tano,
Gavin D. Portwood,
Jean C. Ragusa
Abstract:
Training of deep neural networks (DNNs) frequently involves optimizing several millions or even billions of parameters. Even with modern computing architectures, the computational expense of DNN training can inhibit, for instance, network architecture design optimization, hyper-parameter studies, and integration into scientific research cycles. The key factor limiting performance is that both the…
▽ More
Training of deep neural networks (DNNs) frequently involves optimizing several millions or even billions of parameters. Even with modern computing architectures, the computational expense of DNN training can inhibit, for instance, network architecture design optimization, hyper-parameter studies, and integration into scientific research cycles. The key factor limiting performance is that both the feed-forward evaluation and the back-propagation rule are needed for each weight during optimization in the update rule. In this work, we propose a method to decouple the evaluation of the update rule at each weight. At first, Proper Orthogonal Decomposition (POD) is used to identify a current estimate of the principal directions of evolution of weights per layer during training based on the evolution observed with a few backpropagation steps. Then, Dynamic Mode Decomposition (DMD) is used to learn the dynamics of the evolution of the weights in each layer according to these principal directions. The DMD model is used to evaluate an approximate converged state when training the ANN. Afterward, some number of backpropagation steps are performed, starting from the DMD estimates, leading to an update to the principal directions and DMD model. This iterative process is repeated until convergence. By fine-tuning the number of backpropagation steps used for each DMD model estimation, a significant reduction in the number of operations required to train the neural networks can be achieved. In this paper, the DMD acceleration method will be explained in detail, along with the theoretical justification for the acceleration provided by DMD. This method is illustrated using a regression problem of key interest for the scientific machine learning community: the prediction of a pollutant concentration field in a diffusion, advection, reaction problem.
△ Less
Submitted 18 June, 2020;
originally announced June 2020.
-
Interpreting neural network models of residual scalar flux
Authors:
Gavin D. Portwood,
Balasubramanya T. Nadiga,
Juan A. Saenz,
Daniel Livescu
Abstract:
We show that in addition to providing effective and competitive closures, when analysed in terms of dynamics and physically-relevant diagnostics, artificial neural networks (ANNs) can be both interpretable and provide useful insights in the on-going task of develo** and improving turbulence closures. In the context of large-eddy simulations (LES) of a passive scalar in homogeneous isotropic turb…
▽ More
We show that in addition to providing effective and competitive closures, when analysed in terms of dynamics and physically-relevant diagnostics, artificial neural networks (ANNs) can be both interpretable and provide useful insights in the on-going task of develo** and improving turbulence closures. In the context of large-eddy simulations (LES) of a passive scalar in homogeneous isotropic turbulence, exact subfilter fluxes obtained by filtering direct numerical simulations (DNS) are used both to train deep ANN models as a function of filtered variables, and to optimise the coefficients of a turbulent Prandtl number LES closure. \textit{A-priori} analysis of the subfilter scalar variance transfer rate demonstrates that learnt ANN models out-perform optimised turbulent Prandtl number closures and Clark-type gradient models. Next, \textit{a-posteriori} solutions are obtained with each model over several integral timescales. These experiments reveal, with single- and multi-point diagnostics, that ANN models temporally track exact resolved scalar variance with greater accuracy compared to other subfilter flux models for a given filter length scale. Finally, we interpret the artificial neural networks statistically with differential sensitivity analysis to show that the ANN models feature dynamics reminiscent of so-called "mixed models", where mixed models are understood as comprising both a structural and functional component. Besides enabling enhanced-accuracy LES of passive scalars henceforth, we anticipate this work to contribute to utilising neural network models as a tool in interpretability, robustness and model discovery.
△ Less
Submitted 5 October, 2020; v1 submitted 15 April, 2020;
originally announced April 2020.
-
Turbulence forecasting via Neural ODE
Authors:
Gavin D. Portwood,
Peetak P. Mitra,
Mateus Dias Ribeiro,
Tan Minh Nguyen,
Balasubramanya T. Nadiga,
Juan A. Saenz,
Michael Chertkov,
Animesh Garg,
Anima Anandkumar,
Andreas Dengel,
Richard Baraniuk,
David P. Schmidt
Abstract:
Fluid turbulence is characterized by strong coupling across a broad range of scales. Furthermore, besides the usual local cascades, such coupling may extend to interactions that are non-local in scale-space. As such the computational demands associated with explicitly resolving the full set of scales and their interactions, as in the Direct Numerical Simulation (DNS) of the Navier-Stokes equations…
▽ More
Fluid turbulence is characterized by strong coupling across a broad range of scales. Furthermore, besides the usual local cascades, such coupling may extend to interactions that are non-local in scale-space. As such the computational demands associated with explicitly resolving the full set of scales and their interactions, as in the Direct Numerical Simulation (DNS) of the Navier-Stokes equations, in most problems of practical interest are so high that reduced modeling of scales and interactions is required before further progress can be made. While popular reduced models are typically based on phenomenological modeling of relevant turbulent processes, recent advances in machine learning techniques have energized efforts to further improve the accuracy of such reduced models. In contrast to such efforts that seek to improve an existing turbulence model, we propose a machine learning(ML) methodology that captures, de novo, underlying turbulence phenomenology without a pre-specified model form. To illustrate the approach, we consider transient modeling of the dissipation of turbulent kinetic energy, a fundamental turbulent process that is central to a wide range of turbulence models using a Neural ODE approach. After presenting details of the methodology, we show that this approach outperforms state-of-the-art approaches.
△ Less
Submitted 12 November, 2019;
originally announced November 2019.
-
Unsupervised Machine Learning to Teach Fluid Dynamicists to Think in 15 Dimensions
Authors:
S. M. de Bruyn Kops,
D. J. Saunders,
E. A. Rietman,
G. D. Portwood
Abstract:
An autoencoder is used to compress and then reconstruct three-dimensional stratified turbulence data in order to better understand fluid dynamics by studying the errors in the reconstruction. The original single data set is resolved on approximately $6.9\times10^{10}$ grid points, and 15 fluid variables in three spatial dimensions are used, for a total of about $10^{12}$ input quantities in thre…
▽ More
An autoencoder is used to compress and then reconstruct three-dimensional stratified turbulence data in order to better understand fluid dynamics by studying the errors in the reconstruction. The original single data set is resolved on approximately $6.9\times10^{10}$ grid points, and 15 fluid variables in three spatial dimensions are used, for a total of about $10^{12}$ input quantities in three dimensions. The objective is to understand which of the input variables contains the most relevant information about the local turbulence regimes in stably stratified turbulence (SST). This is accomplished by observing flow features that appear in one input variable but then `bleed over' to multiple output variables. The bleed over is shown to be robust with respect to the number of layers in the autoencoder. In this proof of concept, the errors in the reconstruction include information about the spatial variation of vertical velocity in most of the components of the reconstructed rate-of-strain tensor and density gradient, which suggests that vertical velocity is an important marker for turbulence features of interest in SST. This result is consistent with what fluid dynamicists already understand about SST and, therefore, suggests an approach to understanding turbulence based on more detailed analyses of the reconstruction on errors in an autoencoding algorithm.
△ Less
Submitted 23 July, 2019;
originally announced July 2019.
-
Asymptotic dynamics of high dynamic range stratified turbulence
Authors:
G. D. Portwood,
S. M. de Bruyn Kops,
C. P. Caulfield
Abstract:
Direct numerical simulations of homogeneous sheared and stably stratified turbulence are considered to probe the asymptotic high-dynamic range regime suggested by Gargett et al. 1984 and Shih et al. 2005. We consider statistically stationary configurations of the flow that span three decades in dynamic range defined by the separation between the Ozmidov length scale, $L_O=\sqrt{ε/N^3}$, and the Ko…
▽ More
Direct numerical simulations of homogeneous sheared and stably stratified turbulence are considered to probe the asymptotic high-dynamic range regime suggested by Gargett et al. 1984 and Shih et al. 2005. We consider statistically stationary configurations of the flow that span three decades in dynamic range defined by the separation between the Ozmidov length scale, $L_O=\sqrt{ε/N^3}$, and the Kolmogorov length scale, $L_K=(ν^3/ε)^{1/4}$, up to $\mathrm{Re_b}\equiv (L_O/L_K)^{4/3}=ε/(νN^2) \sim O(1000)$, where $ε$ is the mean turbulent kinetic energy dissipation rate, $ν$ is the kinematic viscosity, and $N$ is the buoyancy frequency. We isolate the effects of $\mathrm{Re_b}$, particularly on irreversible mixing, from the effects of other flow parameters of stratified and sheared turbulence. Specifically, we evaluate the influence of dynamic range independent of initial conditions. We present evidence that the flow approaches an asymptotic state for $\mathrm{Re_b}\gtrapprox 300$, characterized both by an asymptotic partitioning between the potential and kinetic energies and by the approach of components of the dissipation rate to their expected values under the assumption of isotropy. As $\mathrm{Re_b}$ increases above 100, there is a slight decrease in the turbulent flux coefficient $Γ=χ/ε$, where $χ$ is the dissipation rate of buoyancy variance, but, for this flow, there is no evidence of the commonly suggested $Γ\propto \mathrm{Re_b}^{-1/2}$ dependence when $100 \leq \mathrm{Re_b} \leq 1000$.
△ Less
Submitted 10 April, 2019;
originally announced April 2019.
