-
Validation and parameterization of a novel physics-constrained neural dynamics model applied to turbulent fluid flow
Authors:
Varun Shankar,
Gavin D. Portwood,
Arvind T. Mohan,
Peetak P. Mitra,
Dilip Krishnamurthy,
Christopher Rackauckas,
Lucas A. Wilson,
David P. Schmidt,
Venkatasubramanian Viswanathan
Abstract:
In fluid physics, data-driven models to enhance or accelerate solution methods are becoming increasingly popular for many application domains, such as alternatives to turbulence closures, system surrogates, or for new physics discovery. In the context of reduced order models of high-dimensional time-dependent fluid systems, machine learning methods grant the benefit of automated learning from data…
▽ More
In fluid physics, data-driven models to enhance or accelerate solution methods are becoming increasingly popular for many application domains, such as alternatives to turbulence closures, system surrogates, or for new physics discovery. In the context of reduced order models of high-dimensional time-dependent fluid systems, machine learning methods grant the benefit of automated learning from data, but the burden of a model lies on its reduced-order representation of both the fluid state and physical dynamics. In this work, we build a physics-constrained, data-driven reduced order model for the Navier-Stokes equations to approximate spatio-temporal turbulent fluid dynamics. The model design choices mimic numerical and physical constraints by, for example, implicitly enforcing the incompressibility constraint and utilizing continuous Neural Ordinary Differential Equations for tracking the evolution of the differential equation. We demonstrate this technique on three-dimensional, moderate Reynolds number turbulent fluid flow. In assessing the statistical quality and characteristics of the machine-learned model through rigorous diagnostic tests, we find that our model is capable of reconstructing the dynamics of the flow over large integral timescales, favoring accuracy at the larger length scales. More significantly, comprehensive diagnostics suggest that physically-interpretable model parameters, corresponding to the representations of the fluid state and dynamics, have attributable and quantifiable impact on the quality of the model predictions and computational complexity.
△ Less
Submitted 21 October, 2021;
originally announced October 2021.
-
Parallelized Training of Restricted Boltzmann Machines using Markov-Chain Monte Carlo Methods
Authors:
Pei Yang,
Srinivas Varadharajan,
Lucas A. Wilson,
Don D. Smith II,
John A Lockman III,
Vineet Gundecha,
Quy Ta
Abstract:
Restricted Boltzmann Machine (RBM) is a generative stochastic neural network that can be applied to collaborative filtering technique used by recommendation systems. Prediction accuracy of the RBM model is usually better than that of other models for recommendation systems. However, training the RBM model involves Markov-Chain Monte Carlo (MCMC) method, which is computationally expensive. In this…
▽ More
Restricted Boltzmann Machine (RBM) is a generative stochastic neural network that can be applied to collaborative filtering technique used by recommendation systems. Prediction accuracy of the RBM model is usually better than that of other models for recommendation systems. However, training the RBM model involves Markov-Chain Monte Carlo (MCMC) method, which is computationally expensive. In this paper, we have successfully applied distributed parallel training using Horovod framework to improve the training time of the RBM model. Our tests show that the distributed training approach of the RBM model has a good scaling efficiency. We also show that this approach effectively reduces the training time to little over 12 minutes on 64 CPU nodes compared to 5 hours on a single CPU node. This will make RBM models more practically applicable in recommendation systems.
△ Less
Submitted 13 October, 2019;
originally announced October 2019.
-
Densifying Assumed-sparse Tensors: Improving Memory Efficiency and MPI Collective Performance during Tensor Accumulation for Parallelized Training of Neural Machine Translation Models
Authors:
Derya Cavdar,
Valeriu Codreanu,
Can Karakus,
John A. Lockman III,
Damian Podareanu,
Vikram Saletore,
Alexander Sergeev,
Don D. Smith II,
Victor Suthichai,
Quy Ta,
Srinivas Varadharajan,
Lucas A. Wilson,
Rengan Xu,
Pei Yang
Abstract:
Neural machine translation - using neural networks to translate human language - is an area of active research exploring new neuron types and network topologies with the goal of dramatically improving machine translation performance. Current state-of-the-art approaches, such as the multi-head attention-based transformer, require very large translation corpuses and many epochs to produce models of…
▽ More
Neural machine translation - using neural networks to translate human language - is an area of active research exploring new neuron types and network topologies with the goal of dramatically improving machine translation performance. Current state-of-the-art approaches, such as the multi-head attention-based transformer, require very large translation corpuses and many epochs to produce models of reasonable quality. Recent attempts to parallelize the official TensorFlow "Transformer" model across multiple nodes have hit roadblocks due to excessive memory use and resulting out of memory errors when performing MPI collectives. This paper describes modifications made to the Horovod MPI-based distributed training framework to reduce memory usage for transformer models by converting assumed-sparse tensors to dense tensors, and subsequently replacing sparse gradient gather with dense gradient reduction. The result is a dramatic increase in scale-out capability, with CPU-only scaling tests achieving 91% weak scaling efficiency up to 1200 MPI processes (300 nodes), and up to 65% strong scaling efficiency up to 400 MPI processes (200 nodes) using the Stampede2 supercomputer.
△ Less
Submitted 10 May, 2019;
originally announced May 2019.
-
Empirical Study of Phased Model of Software Change
Authors:
Leon A. Wilson,
Yoann Senin,
Yibin Wang,
Václav Rajlich
Abstract:
Software change is the basic task of software evolution and maintenance. Phased Model for Software Change (PMSC) is a process model for software changes that localize in the code. It consists of several phases that cover both program comprehension and code modifications. This paper presents an empirical study of an enactment of PMSC, enhanced by the use of tool JRipples. The subjects are graduate…
▽ More
Software change is the basic task of software evolution and maintenance. Phased Model for Software Change (PMSC) is a process model for software changes that localize in the code. It consists of several phases that cover both program comprehension and code modifications. This paper presents an empirical study of an enactment of PMSC, enhanced by the use of tool JRipples. The subjects are graduate students with varying degree of programming experience. The empirical findings demonstrate that programmers with knowledge of PMSC and supported by JRipples perform perfective software changes in unfamiliar software in significantly less time (about half time) than unaided programmers. Substantial time improvements were witnessed in both code comprehension and implementation efforts.
△ Less
Submitted 11 April, 2019;
originally announced April 2019.
-
The Kappa Andromedae System: New Constraints on the Companion Mass, System Age & Further Multiplicity
Authors:
Sasha Hinkley,
Laurent Pueyo,
Jacqueline K. Faherty,
Ben R. Oppenheimer,
Eric E. Mamajek,
Adam L. Kraus,
Emily L. Rice,
Michael J. Ireland,
Trevor David,
Lynne A. Hillenbrand,
Gautam Vasisht,
Eric Cady,
Douglas Brenner,
Aaron Veicht,
Ricky Nilsson,
Neil Zimmerman,
Ian R. Parry,
Charles Beichman,
Richard Dekany,
Jennifer E. Roberts,
Lewis C Roberts Jr.,
Christoph Baranec,
Justin R. Crepp,
Rick Burruss,
J. Kent Wallace
, et al. (7 additional authors not shown)
Abstract:
Kappa Andromedae is a B9IVn star at 52 pc for which a faint substellar companion separated by 55 AU was recently announced. In this work, we present the first spectrum of the companion, "kappa And B," using the Project 1640 high-contrast imaging platform. Comparison of our low-resolution YJH-band spectra to empirical brown dwarf spectra suggests an early-L spectral type. Fitting synthetic spectra…
▽ More
Kappa Andromedae is a B9IVn star at 52 pc for which a faint substellar companion separated by 55 AU was recently announced. In this work, we present the first spectrum of the companion, "kappa And B," using the Project 1640 high-contrast imaging platform. Comparison of our low-resolution YJH-band spectra to empirical brown dwarf spectra suggests an early-L spectral type. Fitting synthetic spectra from PHOENIX model atmospheres to our observed spectrum allows us to constrain the effective temperature to ~2000K, as well as place constraints on the companion surface gravity. Further, we use previously reported log(g) and effective temperature measurements of the host star to argue that the kappa And system has an isochronal age of 220 +/- 100 Myr, older than the 30 Myr age reported previously. This interpretation of an older age is corroborated by the photometric properties of kappa And B, which appear to be marginally inconsistent with other 10-100 Myr low-gravity L-dwarfs for the spectral type range we derive. In addition, we use Keck aperture masking interferometry combined with published radial velocity measurements to rule out the existence of any tight stellar companions to kappa And A that might be responsible for the system's overluminosity. Further, we show that luminosity enhancements due to a nearly "pole-on" viewing angle coupled with extremely rapid rotation is unlikely. Kappa And A is thus consistent with its slightly evolved luminosity class (IV) and we propose here that kappa And, with a revised age of 220 +/- 100 Myr, is an interloper to the 30 Myr Columba association with which it was previously associated. The photometric and spectroscopic evidence for kappa And B combined with our re-assesment of the system age implies a substellar companion mass of 50^{+16}_{-13} Jupiter Masses, consistent with a brown dwarf rather than a planetary mass companion.
△ Less
Submitted 13 September, 2013;
originally announced September 2013.
