-
Geometric Priors for Scientific Generative Models in Inertial Confinement Fusion
Authors:
Ankita Shukla,
Rushil Anirudh,
Eugene Kur,
Jayaraman J. Thiagarajan,
Peer-Timo Bremer,
Brian K. Spears,
Tammy Ma,
Pavan Turaga
Abstract:
In this paper, we develop a Wasserstein autoencoder (WAE) with a hyperspherical prior for multimodal data in the application of inertial confinement fusion. Unlike a typical hyperspherical generative model that requires computationally inefficient sampling from distributions like the von Mis Fisher, we sample from a normal distribution followed by a projection layer before the generator. Finally,…
▽ More
In this paper, we develop a Wasserstein autoencoder (WAE) with a hyperspherical prior for multimodal data in the application of inertial confinement fusion. Unlike a typical hyperspherical generative model that requires computationally inefficient sampling from distributions like the von Mis Fisher, we sample from a normal distribution followed by a projection layer before the generator. Finally, to determine the validity of the generated samples, we exploit a known relationship between the modalities in the dataset as a scientific constraint, and study different properties of the proposed model.
△ Less
Submitted 24 November, 2021;
originally announced November 2021.
-
Suppressing simulation bias using multi-modal data
Authors:
Bogdan Kustowski,
Jim A. Gaffney,
Brian K. Spears,
Gemma J. Anderson,
Rushil Anirudh,
Peer-Timo Bremer,
Jayaraman J. Thiagarajan,
Michael K. G. Kruse,
Ryan C. Nora
Abstract:
Many problems in science and engineering require making predictions based on few observations. To build a robust predictive model, these sparse data may need to be augmented with simulated data, especially when the design space is multi-dimensional. Simulations, however, often suffer from an inherent bias. Estimation of this bias may be poorly constrained not only because of data sparsity, but als…
▽ More
Many problems in science and engineering require making predictions based on few observations. To build a robust predictive model, these sparse data may need to be augmented with simulated data, especially when the design space is multi-dimensional. Simulations, however, often suffer from an inherent bias. Estimation of this bias may be poorly constrained not only because of data sparsity, but also because traditional predictive models fit only one type of observed outputs, such as scalars or images, instead of all available output data modalities, which might have been acquired and simulated at great cost. To break this limitation and open up the path for multi-modal calibration, we propose to combine a novel, transfer learning technique for suppressing the bias with recent developments in deep learning, which allow building predictive models with multi-modal outputs. First, we train an initial neural network model on simulated data to learn important correlations between different output modalities and between simulation inputs and outputs. Then, the model is partially retrained, or transfer learned, to fit the experiments; a method that has never been implemented in this type of architecture. Using fewer than 10 inertial confinement fusion experiments for training, transfer learning systematically improves the simulation predictions while a simple output calibration, which we design as a baseline, makes the predictions worse. We also offer extensive cross-validation with real and carefully designed synthetic data. The method described in this paper can be applied to a wide range of problems that require transferring knowledge from simulations to the domain of experiments.
△ Less
Submitted 15 March, 2022; v1 submitted 19 April, 2021;
originally announced April 2021.
-
Cognitive simulation models for inertial confinement fusion: Combining simulation and experimental data
Authors:
K. D. Humbird,
J. L. Peterson,
J. Salmonson,
B. K. Spears
Abstract:
The design space for inertial confinement fusion (ICF) experiments is vast and experiments are extremely expensive. Researchers rely heavily on computer simulations to explore the design space in search of high-performing implosions. However, ICF multiphysics codes must make simplifying assumptions, and thus deviate from experimental measurements for complex implosions. For more effective design a…
▽ More
The design space for inertial confinement fusion (ICF) experiments is vast and experiments are extremely expensive. Researchers rely heavily on computer simulations to explore the design space in search of high-performing implosions. However, ICF multiphysics codes must make simplifying assumptions, and thus deviate from experimental measurements for complex implosions. For more effective design and investigation, simulations require input from past experimental data to better predict future performance. In this work, we describe a cognitive simulation method for combining simulation and experimental data into a common, predictive model. This method leverages a machine learning technique called transfer learning, the process of taking a model trained to solve one task, and partially retraining it on a sparse dataset to solve a different, but related task. In the context of ICF design, neural network models trained on large simulation databases and partially retrained on experimental data, producing models that are far more accurate than simulations alone. We demonstrate improved model performance for a range of ICF experiments at the National Ignition Facility, and predict the outcome of recent experiments with less than ten percent error for several key observables. We discuss how the methods might be used to carry out a data-driven experimental campaign to optimize performance, illustrating the key product -- models that become increasingly accurate as data is acquired.
△ Less
Submitted 18 March, 2021;
originally announced March 2021.
-
Meaningful uncertainties from deep neural network surrogates of large-scale numerical simulations
Authors:
Gemma J. Anderson,
Jim A. Gaffney,
Brian K. Spears,
Peer-Timo Bremer,
Rushil Anirudh,
Jayaraman J. Thiagarajan
Abstract:
Large-scale numerical simulations are used across many scientific disciplines to facilitate experimental development and provide insights into underlying physical processes, but they come with a significant computational cost. Deep neural networks (DNNs) can serve as highly-accurate surrogate models, with the capacity to handle diverse datatypes, offering tremendous speed-ups for prediction and ma…
▽ More
Large-scale numerical simulations are used across many scientific disciplines to facilitate experimental development and provide insights into underlying physical processes, but they come with a significant computational cost. Deep neural networks (DNNs) can serve as highly-accurate surrogate models, with the capacity to handle diverse datatypes, offering tremendous speed-ups for prediction and many other downstream tasks. An important use-case for these surrogates is the comparison between simulations and experiments; prediction uncertainty estimates are crucial for making such comparisons meaningful, yet standard DNNs do not provide them. In this work we define the fundamental requirements for a DNN to be useful for scientific applications, and demonstrate a general variational inference approach to equip predictions of scalar and image data from a DNN surrogate model trained on inertial confinement fusion simulations with calibrated Bayesian uncertainties. Critically, these uncertainties are interpretable, meaningful and preserve physics-correlations in the predicted quantities.
△ Less
Submitted 26 October, 2020;
originally announced October 2020.
-
Improved Surrogates in Inertial Confinement Fusion with Manifold and Cycle Consistencies
Authors:
Rushil Anirudh,
Jayaraman J. Thiagarajan,
Peer-Timo Bremer,
Brian K. Spears
Abstract:
Neural networks have become very popular in surrogate modeling because of their ability to characterize arbitrary, high dimensional functions in a data driven fashion. This paper advocates for the training of surrogates that are consistent with the physical manifold -- i.e., predictions are always physically meaningful, and are cyclically consistent -- i.e., when the predictions of the surrogate,…
▽ More
Neural networks have become very popular in surrogate modeling because of their ability to characterize arbitrary, high dimensional functions in a data driven fashion. This paper advocates for the training of surrogates that are consistent with the physical manifold -- i.e., predictions are always physically meaningful, and are cyclically consistent -- i.e., when the predictions of the surrogate, when passed through an independently trained inverse model give back the original input parameters. We find that these two consistencies lead to surrogates that are superior in terms of predictive performance, more resilient to sampling artifacts, and tend to be more data efficient. Using Inertial Confinement Fusion (ICF) as a test bed problem, we model a 1D semi-analytic numerical simulator and demonstrate the effectiveness of our approach. Code and data are available at https://github.com/rushilanirudh/macc/
△ Less
Submitted 17 December, 2019;
originally announced December 2019.
-
Exploring Generative Physics Models with Scientific Priors in Inertial Confinement Fusion
Authors:
Rushil Anirudh,
Jayaraman J. Thiagarajan,
Shusen Liu,
Peer-Timo Bremer,
Brian K. Spears
Abstract:
There is significant interest in using modern neural networks for scientific applications due to their effectiveness in modeling highly complex, non-linear problems in a data-driven fashion. However, a common challenge is to verify the scientific plausibility or validity of outputs predicted by a neural network. This work advocates the use of known scientific constraints as a lens into evaluating,…
▽ More
There is significant interest in using modern neural networks for scientific applications due to their effectiveness in modeling highly complex, non-linear problems in a data-driven fashion. However, a common challenge is to verify the scientific plausibility or validity of outputs predicted by a neural network. This work advocates the use of known scientific constraints as a lens into evaluating, exploring, and understanding such predictions for the problem of inertial confinement fusion.
△ Less
Submitted 3 October, 2019;
originally announced October 2019.
-
Scalable Topological Data Analysis and Visualization for Evaluating Data-Driven Models in Scientific Applications
Authors:
Shusen Liu,
Di Wang,
Dan Maljovec,
Rushil Anirudh,
Jayaraman J. Thiagarajan,
Sam Ade Jacobs,
Brian C. Van Essen,
David Hysom,
Jae-Seung Yeom,
Jim Gaffney,
Luc Peterson,
Peter B. Robinson,
Harsh Bhatia,
Valerio Pascucci,
Brian K. Spears,
Peer-Timo Bremer
Abstract:
With the rapid adoption of machine learning techniques for large-scale applications in science and engineering comes the convergence of two grand challenges in visualization. First, the utilization of black box models (e.g., deep neural networks) calls for advanced techniques in exploring and interpreting model behaviors. Second, the rapid growth in computing has produced enormous datasets that re…
▽ More
With the rapid adoption of machine learning techniques for large-scale applications in science and engineering comes the convergence of two grand challenges in visualization. First, the utilization of black box models (e.g., deep neural networks) calls for advanced techniques in exploring and interpreting model behaviors. Second, the rapid growth in computing has produced enormous datasets that require techniques that can handle millions or more samples. Although some solutions to these interpretability challenges have been proposed, they typically do not scale beyond thousands of samples, nor do they provide the high-level intuition scientists are looking for. Here, we present the first scalable solution to explore and analyze high-dimensional functions often encountered in the scientific data analysis pipeline. By combining a new streaming neighborhood graph construction, the corresponding topology computation, and a novel data aggregation scheme, namely topology aware datacubes, we enable interactive exploration of both the topological and the geometric aspect of high-dimensional data. Following two use cases from high-energy-density (HED) physics and computational biology, we demonstrate how these capabilities have led to crucial new insights in both applications.
△ Less
Submitted 18 July, 2019;
originally announced July 2019.
-
Contemporary machine learning: a guide for practitioners in the physical sciences
Authors:
Brian K. Spears
Abstract:
Machine learning is finding increasingly broad application in the physical sciences. This most often involves building a model relationship between a dependent, measurable output and an associated set of controllable, but complicated, independent inputs. We present a tutorial on current techniques in machine learning -- a jum**-off point for interested researchers to advance their work. We focus…
▽ More
Machine learning is finding increasingly broad application in the physical sciences. This most often involves building a model relationship between a dependent, measurable output and an associated set of controllable, but complicated, independent inputs. We present a tutorial on current techniques in machine learning -- a jum**-off point for interested researchers to advance their work. We focus on deep neural networks with an emphasis on demystifying deep learning. We begin with background ideas in machine learning and some example applications from current research in plasma physics. We discuss supervised learning techniques for modeling complicated functions, beginning with familiar regression schemes, then advancing to more sophisticated deep learning methods. We also address unsupervised learning and techniques for reducing the dimensionality of input spaces. Along the way, we describe methods for practitioners to help ensure that their models generalize from their training data to as-yet-unseen test data. We describe classes of tasks -- predicting scalars, handling images, fitting time-series -- and prepare the reader to choose an appropriate technique. We finally point out some limitations to modern machine learning and speculate on some ways that practitioners from the physical sciences may be particularly suited to help.
△ Less
Submitted 20 December, 2017;
originally announced December 2017.
