-
A Convex Optimization Framework for Computing Robustness Margins of Kalman Filters
Authors:
Himanshu Prabhat,
Raktim Bhattacharya
Abstract:
This paper proposes a novel convex optimization framework for designing robust Kalman filters that guarantee a user-specified steady-state error while maximizing process and sensor noise. The proposed framework simultaneously determines the Kalman gain and the robustness margin in terms of the process and sensor noise. This is the first paper to present such a joint formulation for Kalman filterin…
▽ More
This paper proposes a novel convex optimization framework for designing robust Kalman filters that guarantee a user-specified steady-state error while maximizing process and sensor noise. The proposed framework simultaneously determines the Kalman gain and the robustness margin in terms of the process and sensor noise. This is the first paper to present such a joint formulation for Kalman filtering. The proposed methodology is validated through two distinct examples: the Clohessy-Wiltshire-Hill equations for a chaser spacecraft in an elliptical orbit and the longitudinal motion model of an F-16 aircraft.
△ Less
Submitted 5 March, 2024;
originally announced March 2024.
-
Distributed Reinforcement Learning of Targeted Gras** with Active Vision for Mobile Manipulators
Authors:
Yasuhiro Fujita,
Kota Uenishi,
Avinash Ummadisingu,
Prabhat Nagarajan,
Shimpei Masuda,
Mario Ynocente Castro
Abstract:
Develo** personal robots that can perform a diverse range of manipulation tasks in unstructured environments necessitates solving several challenges for robotic gras** systems. We take a step towards this broader goal by presenting the first RL-based system, to our knowledge, for a mobile manipulator that can (a) achieve targeted gras** generalizing to unseen target objects, (b) learn comple…
▽ More
Develo** personal robots that can perform a diverse range of manipulation tasks in unstructured environments necessitates solving several challenges for robotic gras** systems. We take a step towards this broader goal by presenting the first RL-based system, to our knowledge, for a mobile manipulator that can (a) achieve targeted gras** generalizing to unseen target objects, (b) learn complex gras** strategies for cluttered scenes with occluded objects, and (c) perform active vision through its movable wrist camera to better locate objects. The system is informed of the desired target object in the form of a single, arbitrary-pose RGB image of that object, enabling the system to generalize to unseen objects without retraining. To achieve such a system, we combine several advances in deep reinforcement learning and present a large-scale distributed training system using synchronous SGD that seamlessly scales to multi-node, multi-GPU infrastructure to make rapid prototy** easier. We train and evaluate our system in a simulated environment, identify key components for improving performance, analyze its behaviors, and transfer to a real-world setup.
△ Less
Submitted 14 October, 2020; v1 submitted 15 July, 2020;
originally announced July 2020.
-
Approaches and Applications of Early Classification of Time Series: A Review
Authors:
Ashish Gupta,
Hari Prabhat Gupta,
Bhaskar Biswas,
Tanima Dutta
Abstract:
Early classification of time series has been extensively studied for minimizing class prediction delay in time-sensitive applications such as healthcare and finance. A primary task of an early classification approach is to classify an incomplete time series as soon as possible with some desired level of accuracy. Recent years have witnessed several approaches for early classification of time serie…
▽ More
Early classification of time series has been extensively studied for minimizing class prediction delay in time-sensitive applications such as healthcare and finance. A primary task of an early classification approach is to classify an incomplete time series as soon as possible with some desired level of accuracy. Recent years have witnessed several approaches for early classification of time series. As most of the approaches have solved the early classification problem with different aspects, it becomes very important to make a thorough review of the existing solutions to know the current status of the area. These solutions have demonstrated reasonable performance in a wide range of applications including human activity recognition, gene expression based health diagnostic, industrial monitoring, and so on. In this paper, we present a systematic review of current literature on early classification approaches for both univariate and multivariate time series. We divide various existing approaches into four exclusive categories based on their proposed solution strategies. The four categories include prefix based, shapelet based, model based, and miscellaneous approaches. The authors also discuss the applications of early classification in many areas including industrial monitoring, intelligent transportation, and medical. Finally, we provide a quick summary of the current literature with future research directions.
△ Less
Submitted 15 October, 2020; v1 submitted 6 May, 2020;
originally announced May 2020.
-
MeshfreeFlowNet: A Physics-Constrained Deep Continuous Space-Time Super-Resolution Framework
Authors:
Chiyu Max Jiang,
Soheil Esmaeilzadeh,
Kamyar Azizzadenesheli,
Karthik Kashinath,
Mustafa Mustafa,
Hamdi A. Tchelepi,
Philip Marcus,
Prabhat,
Anima Anandkumar
Abstract:
We propose MeshfreeFlowNet, a novel deep learning-based super-resolution framework to generate continuous (grid-free) spatio-temporal solutions from the low-resolution inputs. While being computationally efficient, MeshfreeFlowNet accurately recovers the fine-scale quantities of interest. MeshfreeFlowNet allows for: (i) the output to be sampled at all spatio-temporal resolutions, (ii) a set of Par…
▽ More
We propose MeshfreeFlowNet, a novel deep learning-based super-resolution framework to generate continuous (grid-free) spatio-temporal solutions from the low-resolution inputs. While being computationally efficient, MeshfreeFlowNet accurately recovers the fine-scale quantities of interest. MeshfreeFlowNet allows for: (i) the output to be sampled at all spatio-temporal resolutions, (ii) a set of Partial Differential Equation (PDE) constraints to be imposed, and (iii) training on fixed-size inputs on arbitrarily sized spatio-temporal domains owing to its fully convolutional encoder. We empirically study the performance of MeshfreeFlowNet on the task of super-resolution of turbulent flows in the Rayleigh-Benard convection problem. Across a diverse set of evaluation metrics, we show that MeshfreeFlowNet significantly outperforms existing baselines. Furthermore, we provide a large scale implementation of MeshfreeFlowNet and show that it efficiently scales across large clusters, achieving 96.80% scaling efficiency on up to 128 GPUs and a training time of less than 4 minutes.
△ Less
Submitted 21 August, 2020; v1 submitted 1 May, 2020;
originally announced May 2020.
-
Coronavirus (COVID-19): ARIMA based time-series analysis to forecast near future
Authors:
Hiteshi Tandon,
Prabhat Ranjan,
Tanmoy Chakraborty,
Vandana Suhag
Abstract:
COVID-19, a novel coronavirus, is currently a major worldwide threat. It has infected more than a million people globally leading to hundred-thousands of deaths. In such grave circumstances, it is very important to predict the future infected cases to support prevention of the disease and aid in the healthcare service preparation. Following that notion, we have developed a model and then employed…
▽ More
COVID-19, a novel coronavirus, is currently a major worldwide threat. It has infected more than a million people globally leading to hundred-thousands of deaths. In such grave circumstances, it is very important to predict the future infected cases to support prevention of the disease and aid in the healthcare service preparation. Following that notion, we have developed a model and then employed it for forecasting future COVID-19 cases in India. The study indicates an ascending trend for the cases in the coming days. A time series analysis also presents an exponential increase in the number of cases. It is supposed that the present prediction models will assist the government and medical personnel to be prepared for the upcoming conditions and have more readiness in healthcare systems.
△ Less
Submitted 16 April, 2020;
originally announced April 2020.
-
Learning Occupational Task-Shares Dynamics for the Future of Work
Authors:
Subhro Das,
Sebastian Steffen,
Wyatt Clarke,
Prabhat Reddy,
Erik Brynjolfsson,
Martin Fleming
Abstract:
The recent wave of AI and automation has been argued to differ from previous General Purpose Technologies (GPTs), in that it may lead to rapid change in occupations' underlying task requirements and persistent technological unemployment. In this paper, we apply a novel methodology of dynamic task shares to a large dataset of online job postings to explore how exactly occupational task demands have…
▽ More
The recent wave of AI and automation has been argued to differ from previous General Purpose Technologies (GPTs), in that it may lead to rapid change in occupations' underlying task requirements and persistent technological unemployment. In this paper, we apply a novel methodology of dynamic task shares to a large dataset of online job postings to explore how exactly occupational task demands have changed over the past decade of AI innovation, especially across high, mid and low wage occupations. Notably, big data and AI have risen significantly among high wage occupations since 2012 and 2016, respectively. We built an ARIMA model to predict future occupational task demands and showcase several relevant examples in Healthcare, Administration, and IT. Such task demands predictions across occupations will play a pivotal role in retraining the workforce of the future.
△ Less
Submitted 28 January, 2020;
originally announced February 2020.
-
Learning Latent State Spaces for Planning through Reward Prediction
Authors:
Aaron Havens,
Yi Ouyang,
Prabhat Nagarajan,
Yasuhiro Fujita
Abstract:
Model-based reinforcement learning methods typically learn models for high-dimensional state spaces by aiming to reconstruct and predict the original observations. However, drawing inspiration from model-free reinforcement learning, we propose learning a latent dynamics model directly from rewards. In this work, we introduce a model-based planning framework which learns a latent reward prediction…
▽ More
Model-based reinforcement learning methods typically learn models for high-dimensional state spaces by aiming to reconstruct and predict the original observations. However, drawing inspiration from model-free reinforcement learning, we propose learning a latent dynamics model directly from rewards. In this work, we introduce a model-based planning framework which learns a latent reward prediction model and then plans in the latent state-space. The latent representation is learned exclusively from multi-step reward prediction which we show to be the only necessary information for successful planning. With this framework, we are able to benefit from the concise model-free representation, while still enjoying the data-efficiency of model-based algorithms. We demonstrate our framework in multi-pendulum and multi-cheetah environments where several pendulums or cheetahs are shown to the agent but only one of which produces rewards. In these environments, it is important for the agent to construct a concise latent representation to filter out irrelevant observations. We find that our method can successfully learn an accurate latent reward prediction model in the presence of the irrelevant information while existing model-based methods fail. Planning in the learned latent state-space shows strong performance and high sample efficiency over model-free and model-based baselines.
△ Less
Submitted 9 December, 2019;
originally announced December 2019.
-
ChainerRL: A Deep Reinforcement Learning Library
Authors:
Yasuhiro Fujita,
Prabhat Nagarajan,
Toshiki Kataoka,
Takahiro Ishikawa
Abstract:
In this paper, we introduce ChainerRL, an open-source deep reinforcement learning (DRL) library built using Python and the Chainer deep learning framework. ChainerRL implements a comprehensive set of DRL algorithms and techniques drawn from state-of-the-art research in the field. To foster reproducible research, and for instructional purposes, ChainerRL provides scripts that closely replicate the…
▽ More
In this paper, we introduce ChainerRL, an open-source deep reinforcement learning (DRL) library built using Python and the Chainer deep learning framework. ChainerRL implements a comprehensive set of DRL algorithms and techniques drawn from state-of-the-art research in the field. To foster reproducible research, and for instructional purposes, ChainerRL provides scripts that closely replicate the original papers' experimental settings and reproduce published benchmark results for several algorithms. Lastly, ChainerRL offers a visualization tool that enables the qualitative inspection of trained agents. The ChainerRL source code can be found on GitHub: https://github.com/chainer/chainerrl.
△ Less
Submitted 11 April, 2021; v1 submitted 9 December, 2019;
originally announced December 2019.
-
Highly-scalable, physics-informed GANs for learning solutions of stochastic PDEs
Authors:
Liu Yang,
Sean Treichler,
Thorsten Kurth,
Keno Fischer,
David Barajas-Solano,
Josh Romero,
Valentin Churavy,
Alexandre Tartakovsky,
Michael Houston,
Prabhat,
George Karniadakis
Abstract:
Uncertainty quantification for forward and inverse problems is a central challenge across physical and biomedical disciplines. We address this challenge for the problem of modeling subsurface flow at the Hanford Site by combining stochastic computational models with observational data using physics-informed GAN models. The geographic extent, spatial heterogeneity, and multiple correlation length s…
▽ More
Uncertainty quantification for forward and inverse problems is a central challenge across physical and biomedical disciplines. We address this challenge for the problem of modeling subsurface flow at the Hanford Site by combining stochastic computational models with observational data using physics-informed GAN models. The geographic extent, spatial heterogeneity, and multiple correlation length scales of the Hanford Site require training a computationally intensive GAN model to thousands of dimensions. We develop a hierarchical scheme for exploiting domain parallelism, map discriminators and generators to multiple GPUs, and employ efficient communication schemes to ensure training stability and convergence. We developed a highly optimized implementation of this scheme that scales to 27,500 NVIDIA Volta GPUs and 4584 nodes on the Summit supercomputer with a 93.1% scaling efficiency, achieving peak and sustained half-precision rates of 1228 PF/s and 1207 PF/s.
△ Less
Submitted 28 October, 2019;
originally announced October 2019.
-
Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale
Authors:
Atılım Güneş Baydin,
Lei Shao,
Wahid Bhimji,
Lukas Heinrich,
Lawrence Meadows,
Jialin Liu,
Andreas Munk,
Saeid Naderiparizi,
Bradley Gram-Hansen,
Gilles Louppe,
Mingfei Ma,
Xiaohui Zhao,
Philip Torr,
Victor Lee,
Kyle Cranmer,
Prabhat,
Frank Wood
Abstract:
Probabilistic programming languages (PPLs) are receiving widespread attention for performing Bayesian inference in complex generative models. However, applications to science remain limited because of the impracticability of rewriting complex scientific simulators in a PPL, the computational cost of inference, and the lack of scalable implementations. To address these, we present a novel PPL frame…
▽ More
Probabilistic programming languages (PPLs) are receiving widespread attention for performing Bayesian inference in complex generative models. However, applications to science remain limited because of the impracticability of rewriting complex scientific simulators in a PPL, the computational cost of inference, and the lack of scalable implementations. To address these, we present a novel PPL framework that couples directly to existing scientific simulators through a cross-platform probabilistic execution protocol and provides Markov chain Monte Carlo (MCMC) and deep-learning-based inference compilation (IC) engines for tractable inference. To guide IC inference, we perform distributed training of a dynamic 3DCNN--LSTM architecture with a PyTorch-MPI-based framework on 1,024 32-core CPU nodes of the Cori supercomputer with a global minibatch size of 128k: achieving a performance of 450 Tflop/s through enhancements to PyTorch. We demonstrate a Large Hadron Collider (LHC) use-case with the C++ Sherpa simulator and achieve the largest-scale posterior inference in a Turing-complete PPL.
△ Less
Submitted 27 August, 2019; v1 submitted 7 July, 2019;
originally announced July 2019.
-
Enforcing Statistical Constraints in Generative Adversarial Networks for Modeling Chaotic Dynamical Systems
Authors:
**-Long Wu,
Karthik Kashinath,
Adrian Albert,
Dragos Chirila,
Prabhat,
Heng Xiao
Abstract:
Simulating complex physical systems often involves solving partial differential equations (PDEs) with some closures due to the presence of multi-scale physics that cannot be fully resolved. Therefore, reliable and accurate closure models for unresolved physics remains an important requirement for many computational physics problems, e.g., turbulence simulation. Recently, several researchers have a…
▽ More
Simulating complex physical systems often involves solving partial differential equations (PDEs) with some closures due to the presence of multi-scale physics that cannot be fully resolved. Therefore, reliable and accurate closure models for unresolved physics remains an important requirement for many computational physics problems, e.g., turbulence simulation. Recently, several researchers have adopted generative adversarial networks (GANs), a novel paradigm of training machine learning models, to generate solutions of PDEs-governed complex systems without having to numerically solve these PDEs. However, GANs are known to be difficult in training and likely to converge to local minima, where the generated samples do not capture the true statistics of the training data. In this work, we present a statistical constrained generative adversarial network by enforcing constraints of covariance from the training data, which results in an improved machine-learning-based emulator to capture the statistics of the training data generated by solving fully resolved PDEs. We show that such a statistical regularization leads to better performance compared to standard GANs, measured by (1) the constrained model's ability to more faithfully emulate certain physical properties of the system and (2) the significantly reduced (by up to 80%) training time to reach the solution. We exemplify this approach on the Rayleigh-Benard convection, a turbulent flow system that is an idealized model of the Earth's atmosphere. With the growth of high-fidelity simulation databases of physical systems, this work suggests great potential for being an alternative to the explicit modeling of closures or parameterizations for unresolved physics, which are known to be a major source of uncertainty in simulating multi-scale physical systems, e.g., turbulence or Earth's climate.
△ Less
Submitted 13 May, 2019;
originally announced May 2019.
-
Machine Learning at Microsoft with ML .NET
Authors:
Zeeshan Ahmed,
Saeed Amizadeh,
Mikhail Bilenko,
Rogan Carr,
Wei-Sheng Chin,
Yael Dekel,
Xavier Dupre,
Vadim Eksarevskiy,
Eric Erhardt,
Costin Eseanu,
Senja Filipi,
Tom Finley,
Abhishek Goswami,
Monte Hoover,
Scott Inglis,
Matteo Interlandi,
Shon Katzenberger,
Najeeb Kazmi,
Gleb Krivosheev,
Pete Luferenko,
Ivan Matantsev,
Sergiy Matusevych,
Shahab Moradi,
Gani Nazirov,
Justin Ormont
, et al. (9 additional authors not shown)
Abstract:
Machine Learning is transitioning from an art and science into a technology available to every developer. In the near future, every application on every platform will incorporate trained models to encode data-based decisions that would be impossible for developers to author. This presents a significant engineering challenge, since currently data science and modeling are largely decoupled from stan…
▽ More
Machine Learning is transitioning from an art and science into a technology available to every developer. In the near future, every application on every platform will incorporate trained models to encode data-based decisions that would be impossible for developers to author. This presents a significant engineering challenge, since currently data science and modeling are largely decoupled from standard software development processes. This separation makes incorporating machine learning capabilities inside applications unnecessarily costly and difficult, and furthermore discourage developers from embracing ML in first place. In this paper we present ML .NET, a framework developed at Microsoft over the last decade in response to the challenge of making it easy to ship machine learning models in large software applications. We present its architecture, and illuminate the application demands that shaped it. Specifically, we introduce DataView, the core data abstraction of ML .NET which allows it to capture full predictive pipelines efficiently and consistently across training and inference lifecycles. We close the paper with a surprisingly favorable performance study of ML .NET compared to more recent entrants, and a discussion of some lessons learned.
△ Less
Submitted 15 May, 2019; v1 submitted 14 May, 2019;
originally announced May 2019.
-
Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations
Authors:
Daniel S. Brown,
Wonjoon Goo,
Prabhat Nagarajan,
Scott Niekum
Abstract:
A critical flaw of existing inverse reinforcement learning (IRL) methods is their inability to significantly outperform the demonstrator. This is because IRL typically seeks a reward function that makes the demonstrator appear near-optimal, rather than inferring the underlying intentions of the demonstrator that may have been poorly executed in practice. In this paper, we introduce a novel reward-…
▽ More
A critical flaw of existing inverse reinforcement learning (IRL) methods is their inability to significantly outperform the demonstrator. This is because IRL typically seeks a reward function that makes the demonstrator appear near-optimal, rather than inferring the underlying intentions of the demonstrator that may have been poorly executed in practice. In this paper, we introduce a novel reward-learning-from-observation algorithm, Trajectory-ranked Reward EXtrapolation (T-REX), that extrapolates beyond a set of (approximately) ranked demonstrations in order to infer high-quality reward functions from a set of potentially poor demonstrations. When combined with deep reinforcement learning, T-REX outperforms state-of-the-art imitation learning and IRL methods on multiple Atari and MuJoCo benchmark tasks and achieves performance that is often more than twice the performance of the best demonstration. We also demonstrate that T-REX is robust to ranking noise and can accurately extrapolate intention by simply watching a learner noisily improve at a task over time.
△ Less
Submitted 8 July, 2019; v1 submitted 12 April, 2019;
originally announced April 2019.
-
Graph Neural Networks for IceCube Signal Classification
Authors:
Nicholas Choma,
Federico Monti,
Lisa Gerhardt,
Tomasz Palczewski,
Zahra Ronaghi,
Prabhat,
Wahid Bhimji,
Michael M. Bronstein,
Spencer R. Klein,
Joan Bruna
Abstract:
Tasks involving the analysis of geometric (graph- and manifold-structured) data have recently gained prominence in the machine learning community, giving birth to a rapidly develo** field of geometric deep learning. In this work, we leverage graph neural networks to improve signal detection in the IceCube neutrino observatory. The IceCube detector array is modeled as a graph, where vertices are…
▽ More
Tasks involving the analysis of geometric (graph- and manifold-structured) data have recently gained prominence in the machine learning community, giving birth to a rapidly develo** field of geometric deep learning. In this work, we leverage graph neural networks to improve signal detection in the IceCube neutrino observatory. The IceCube detector array is modeled as a graph, where vertices are sensors and edges are a learned function of the sensors' spatial coordinates. As only a subset of IceCube's sensors is active during a given observation, we note the adaptive nature of our GNN, wherein computation is restricted to the input signal support. We demonstrate the effectiveness of our GNN architecture on a task classifying IceCube events, where it outperforms both a traditional physics-based method as well as classical 3D convolution neural networks.
△ Less
Submitted 17 September, 2018;
originally announced September 2018.
-
Optimizing the Union of Intersections LASSO ($UoI_{LASSO}$) and Vector Autoregressive ($UoI_{VAR}$) Algorithms for Improved Statistical Estimation at Scale
Authors:
Mahesh Balasubramanian,
Trevor Ruiz,
Brandon Cook,
Sharmodeep Bhattacharyya,
Prabhat,
Aviral Shrivastava,
Kristofer Bouchard
Abstract:
The analysis of scientific data of increasing size and complexity requires statistical machine learning methods that are both interpretable and predictive. Union of Intersections (UoI), a recently developed framework, is a two-step approach that separates model selection and model estimation. A linear regression algorithm based on UoI, $UoI_{LASSO}$, simultaneously achieves low false positives and…
▽ More
The analysis of scientific data of increasing size and complexity requires statistical machine learning methods that are both interpretable and predictive. Union of Intersections (UoI), a recently developed framework, is a two-step approach that separates model selection and model estimation. A linear regression algorithm based on UoI, $UoI_{LASSO}$, simultaneously achieves low false positives and low false negative feature selection as well as low bias and low variance estimates. Together, these qualities make the results both predictive and interpretable. In this paper, we optimize the $UoI_{LASSO}$ algorithm for single-node execution on NERSC's Cori Knights Landing, a Xeon Phi based supercomputer. We then scale $UoI_{LASSO}$ to execute on cores ranging from 68-278,528 cores on a range of dataset sizes demonstrating the weak and strong scaling of the implementation. We also implement a variant of $UoI_{LASSO}$, $UoI_{VAR}$ for vector autoregressive models, to analyze high dimensional time-series data. We perform single node optimization and multi-node scaling experiments for $UoI_{VAR}$ to demonstrate the effectiveness of the algorithm for weak and strong scaling. Our implementations enable to use estimate the largest VAR model (1000 nodes) we are aware of, and apply it to large neurophysiology data 192 nodes).
△ Less
Submitted 21 August, 2018;
originally announced August 2018.
-
Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model
Authors:
Atılım Güneş Baydin,
Lukas Heinrich,
Wahid Bhimji,
Lei Shao,
Saeid Naderiparizi,
Andreas Munk,
Jialin Liu,
Bradley Gram-Hansen,
Gilles Louppe,
Lawrence Meadows,
Philip Torr,
Victor Lee,
Prabhat,
Kyle Cranmer,
Frank Wood
Abstract:
We present a novel probabilistic programming framework that couples directly to existing large-scale simulators through a cross-platform probabilistic execution protocol, which allows general-purpose inference engines to record and control random number draws within simulators in a language-agnostic way. The execution of existing simulators as probabilistic programs enables highly interpretable po…
▽ More
We present a novel probabilistic programming framework that couples directly to existing large-scale simulators through a cross-platform probabilistic execution protocol, which allows general-purpose inference engines to record and control random number draws within simulators in a language-agnostic way. The execution of existing simulators as probabilistic programs enables highly interpretable posterior inference in the structured model defined by the simulator code base. We demonstrate the technique in particle physics, on a scientifically accurate simulation of the tau lepton decay, which is a key ingredient in establishing the properties of the Higgs boson. Inference efficiency is achieved via inference compilation where a deep recurrent neural network is trained to parameterize proposal distributions and control the stochastic simulator in a sequential importance sampling scheme, at a fraction of the computational cost of a Markov chain Monte Carlo baseline.
△ Less
Submitted 17 February, 2020; v1 submitted 20 July, 2018;
originally announced July 2018.
-
Alchemist: An Apache Spark <=> MPI Interface
Authors:
Alex Gittens,
Kai Rothauge,
Shusen Wang,
Michael W. Mahoney,
Jey Kottalam,
Lisa Gerhardt,
Prabhat,
Michael Ringenburg,
Kristyn Maschhoff
Abstract:
The Apache Spark framework for distributed computation is popular in the data analytics community due to its ease of use, but its MapReduce-style programming model can incur significant overheads when performing computations that do not map directly onto this model. One way to mitigate these costs is to off-load computations onto MPI codes. In recent work, we introduced Alchemist, a system for the…
▽ More
The Apache Spark framework for distributed computation is popular in the data analytics community due to its ease of use, but its MapReduce-style programming model can incur significant overheads when performing computations that do not map directly onto this model. One way to mitigate these costs is to off-load computations onto MPI codes. In recent work, we introduced Alchemist, a system for the analysis of large-scale data sets. Alchemist calls MPI-based libraries from within Spark applications, and it has minimal coding, communication, and memory overheads. In particular, Alchemist allows users to retain the productivity benefits of working within the Spark software ecosystem without sacrificing performance efficiency in linear algebra, machine learning, and other related computations.
In this paper, we discuss the motivation behind the development of Alchemist, and we provide a detailed overview its design and usage. We also demonstrate the efficiency of our approach on medium-to-large data sets, using some standard linear algebra operations, namely matrix multiplication and the truncated singular value decomposition of a dense matrix, and we compare the performance of Spark with that of Spark+Alchemist. These computations are run on the NERSC supercomputer Cori Phase 1, a Cray XC40.
△ Less
Submitted 3 June, 2018;
originally announced June 2018.
-
Foresee: Attentive Future Projections of Chaotic Road Environments with Online Training
Authors:
Anil Sharma,
Prabhat Kumar
Abstract:
In this paper, we train a recurrent neural network to learn dynamics of a chaotic road environment and to project the future of the environment on an image. Future projection can be used to anticipate an unseen environment for example, in autonomous driving. Road environment is highly dynamic and complex due to the interaction among traffic participants such as vehicles and pedestrians. Even in th…
▽ More
In this paper, we train a recurrent neural network to learn dynamics of a chaotic road environment and to project the future of the environment on an image. Future projection can be used to anticipate an unseen environment for example, in autonomous driving. Road environment is highly dynamic and complex due to the interaction among traffic participants such as vehicles and pedestrians. Even in this complex environment, a human driver is efficacious to safely drive on chaotic roads irrespective of the number of traffic participants. The proliferation of deep learning research has shown the efficacy of neural networks in learning this human behavior. In the same direction, we investigate recurrent neural networks to understand the chaotic road environment which is shared by pedestrians, vehicles (cars, trucks, bicycles etc.), and sometimes animals as well. We propose \emph{Foresee}, a unidirectional gated recurrent units (GRUs) network with attention to project future of the environment in the form of images. We have collected several videos on Delhi roads consisting of various traffic participants, background and infrastructure differences (like 3D pedestrian crossing) at various times on various days. We train \emph{Foresee} in an unsupervised way and we use online training to project frames up to $0.5$ seconds in advance. We show that our proposed model performs better than state of the art methods (prednet and Enc. Dec. LSTM) and finally, we show that our trained model generalizes to a public dataset for future projections.
△ Less
Submitted 30 May, 2018;
originally announced May 2018.
-
Accelerating Large-Scale Data Analysis by Offloading to High-Performance Computing Libraries using Alchemist
Authors:
Alex Gittens,
Kai Rothauge,
Shusen Wang,
Michael W. Mahoney,
Lisa Gerhardt,
Prabhat,
Jey Kottalam,
Michael Ringenburg,
Kristyn Maschhoff
Abstract:
Apache Spark is a popular system aimed at the analysis of large data sets, but recent studies have shown that certain computations---in particular, many linear algebra computations that are the basis for solving common machine learning problems---are significantly slower in Spark than when done using libraries written in a high-performance computing framework such as the Message-Passing Interface…
▽ More
Apache Spark is a popular system aimed at the analysis of large data sets, but recent studies have shown that certain computations---in particular, many linear algebra computations that are the basis for solving common machine learning problems---are significantly slower in Spark than when done using libraries written in a high-performance computing framework such as the Message-Passing Interface (MPI).
To remedy this, we introduce Alchemist, a system designed to call MPI-based libraries from Apache Spark. Using Alchemist with Spark helps accelerate linear algebra, machine learning, and related computations, while still retaining the benefits of working within the Spark environment. We discuss the motivation behind the development of Alchemist, and we provide a brief overview of its design and implementation.
We also compare the performances of pure Spark implementations with those of Spark implementations that leverage MPI-based codes via Alchemist. To do so, we use data science case studies: a large-scale application of the conjugate gradient method to solve very large linear systems arising in a speech classification problem, where we see an improvement of an order of magnitude; and the truncated singular value decomposition (SVD) of a 400GB three-dimensional ocean temperature data set, where we see a speedup of up to 7.9x. We also illustrate that the truncated SVD computation is easily scalable to terabyte-sized data by applying it to data sets of sizes up to 17.6TB.
△ Less
Submitted 30 May, 2018;
originally announced May 2018.
-
Approximate Inference for Constructing Astronomical Catalogs from Images
Authors:
Jeffrey Regier,
Andrew C. Miller,
David Schlegel,
Ryan P. Adams,
Jon D. McAuliffe,
Prabhat
Abstract:
We present a new, fully generative model for constructing astronomical catalogs from optical telescope image sets. Each pixel intensity is treated as a random variable with parameters that depend on the latent properties of stars and galaxies. These latent properties are themselves modeled as random. We compare two procedures for posterior inference. One procedure is based on Markov chain Monte Ca…
▽ More
We present a new, fully generative model for constructing astronomical catalogs from optical telescope image sets. Each pixel intensity is treated as a random variable with parameters that depend on the latent properties of stars and galaxies. These latent properties are themselves modeled as random. We compare two procedures for posterior inference. One procedure is based on Markov chain Monte Carlo (MCMC) while the other is based on variational inference (VI). The MCMC procedure excels at quantifying uncertainty, while the VI procedure is 1000 times faster. On a supercomputer, the VI procedure efficiently uses 665,000 CPU cores to construct an astronomical catalog from 50 terabytes of images in 14.6 minutes, demonstrating the scaling characteristics necessary to construct catalogs for upcoming astronomical surveys.
△ Less
Submitted 9 April, 2019; v1 submitted 28 February, 2018;
originally announced March 2018.
-
Union of Intersections (UoI) for Interpretable Data Driven Discovery and Prediction
Authors:
Kristofer E. Bouchard,
Alejandro F. Bujan,
Farbod Roosta-Khorasani,
Shashanka Ubaru,
Prabhat,
Antoine M. Snijders,
Jian-Hua Mao,
Edward F. Chang,
Michael W. Mahoney,
Sharmodeep Bhattacharyya
Abstract:
The increasing size and complexity of scientific data could dramatically enhance discovery and prediction for basic scientific applications. Realizing this potential, however, requires novel statistical analysis methods that are both interpretable and predictive. We introduce Union of Intersections (UoI), a flexible, modular, and scalable framework for enhanced model selection and estimation. Meth…
▽ More
The increasing size and complexity of scientific data could dramatically enhance discovery and prediction for basic scientific applications. Realizing this potential, however, requires novel statistical analysis methods that are both interpretable and predictive. We introduce Union of Intersections (UoI), a flexible, modular, and scalable framework for enhanced model selection and estimation. Methods based on UoI perform model selection and model estimation through intersection and union operations, respectively. We show that UoI-based methods achieve low-variance and nearly unbiased estimation of a small number of interpretable features, while maintaining high-quality prediction accuracy. We perform extensive numerical investigation to evaluate a UoI algorithm ($UoI_{Lasso}$) on synthetic and real data. In doing so, we demonstrate the extraction of interpretable functional networks from human electrophysiology recordings as well as accurate prediction of phenotypes from genotype-phenotype data with reduced features. We also show (with the $UoI_{L1Logistic}$ and $UoI_{CUR}$ variants of the basic framework) improved prediction parsimony for classification and matrix factorization on several benchmark biomedical data sets. These results suggest that methods based on the UoI framework could improve interpretation and prediction in data-driven discovery across scientific fields.
△ Less
Submitted 2 November, 2017; v1 submitted 22 May, 2017;
originally announced May 2017.
-
3D Reconstruction of the Magnetic Vector Potential using Model Based Iterative Reconstruction
Authors:
Prabhat KC,
K. Aditya Mohan,
Charudatta Phatak,
Charles Bouman,
Marc De Graef
Abstract:
Lorentz Transmission Electron Microscopy (TEM) observations of magnetic nanoparticles contain information on the magnetic and electrostatic potentials. Vector Field Electron Tomography (VFET) can be used to reconstruct electromagnetic potentials of the nanoparticles from their corresponding LTEM images. The VFET approach is based on the conventional filtered back projection approach to tomographic…
▽ More
Lorentz Transmission Electron Microscopy (TEM) observations of magnetic nanoparticles contain information on the magnetic and electrostatic potentials. Vector Field Electron Tomography (VFET) can be used to reconstruct electromagnetic potentials of the nanoparticles from their corresponding LTEM images. The VFET approach is based on the conventional filtered back projection approach to tomographic reconstructions and the availability of an incomplete set of measurements due to experimental limitations means that the reconstructed vector fields exhibit significant artifacts. In this paper, we outline a model-based iterative reconstruction (MBIR) algorithm to reconstruct the magnetic vector potential of magnetic nanoparticles. We combine a forward model for image formation in TEM experiments with a prior model to formulate the tomographic problem as a maximum a-posteriori probability estimation problem (MAP). The MAP cost function is minimized iteratively to determine the vector potential. A comparative reconstruction study of simulated as well as experimental data sets show that the MBIR approach yields quantifiably better reconstructions than the VFET approach.
△ Less
Submitted 23 April, 2017;
originally announced April 2017.
-
ExtremeWeather: A large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events
Authors:
Evan Racah,
Christopher Beckham,
Tegan Maharaj,
Samira Ebrahimi Kahou,
Prabhat,
Christopher Pal
Abstract:
Then detection and identification of extreme weather events in large-scale climate simulations is an important problem for risk management, informing governmental policy decisions and advancing our basic understanding of the climate system. Recent work has shown that fully supervised convolutional neural networks (CNNs) can yield acceptable accuracy for classifying well-known types of extreme weat…
▽ More
Then detection and identification of extreme weather events in large-scale climate simulations is an important problem for risk management, informing governmental policy decisions and advancing our basic understanding of the climate system. Recent work has shown that fully supervised convolutional neural networks (CNNs) can yield acceptable accuracy for classifying well-known types of extreme weather events when large amounts of labeled data are available. However, many different types of spatially localized climate patterns are of interest including hurricanes, extra-tropical cyclones, weather fronts, and blocking events among others. Existing labeled data for these patterns can be incomplete in various ways, such as covering only certain years or geographic areas and having false negatives. This type of climate data therefore poses a number of interesting machine learning challenges. We present a multichannel spatiotemporal CNN architecture for semi-supervised bounding box prediction and exploratory data analysis. We demonstrate that our approach is able to leverage temporal information and unlabeled data to improve the localization of extreme weather events. Further, we explore the representations learned by our model in order to better understand this important data. We present a dataset, ExtremeWeather, to encourage machine learning research in this area and to help facilitate further work in understanding and mitigating the effects of climate change. The dataset is available at extremeweatherdataset.github.io and the code is available at https://github.com/eracah/hur-detect.
△ Less
Submitted 25 November, 2017; v1 submitted 6 December, 2016;
originally announced December 2016.
-
Learning an Astronomical Catalog of the Visible Universe through Scalable Bayesian Inference
Authors:
Jeffrey Regier,
Kiran Pamnany,
Ryan Giordano,
Rollin Thomas,
David Schlegel,
Jon McAuliffe,
Prabhat
Abstract:
Celeste is a procedure for inferring astronomical catalogs that attains state-of-the-art scientific results. To date, Celeste has been scaled to at most hundreds of megabytes of astronomical images: Bayesian posterior inference is notoriously demanding computationally. In this paper, we report on a scalable, parallel version of Celeste, suitable for learning catalogs from modern large-scale astron…
▽ More
Celeste is a procedure for inferring astronomical catalogs that attains state-of-the-art scientific results. To date, Celeste has been scaled to at most hundreds of megabytes of astronomical images: Bayesian posterior inference is notoriously demanding computationally. In this paper, we report on a scalable, parallel version of Celeste, suitable for learning catalogs from modern large-scale astronomical datasets. Our algorithmic innovations include a fast numerical optimization routine for Bayesian posterior inference and a statistically efficient scheme for decomposing astronomical optimization problems into subproblems.
Our scalable implementation is written entirely in Julia, a new high-level dynamic programming language designed for scientific and numerical computing. We use Julia's high-level constructs for shared and distributed memory parallelism, and demonstrate effective load balancing and efficient scaling on up to 8192 Xeon cores on the NERSC Cori supercomputer.
△ Less
Submitted 10 November, 2016;
originally announced November 2016.
-
Revealing Fundamental Physics from the Daya Bay Neutrino Experiment using Deep Neural Networks
Authors:
Evan Racah,
Seyoon Ko,
Peter Sadowski,
Wahid Bhimji,
Craig Tull,
Sang-Yun Oh,
Pierre Baldi,
Prabhat
Abstract:
Experiments in particle physics produce enormous quantities of data that must be analyzed and interpreted by teams of physicists. This analysis is often exploratory, where scientists are unable to enumerate the possible types of signal prior to performing the experiment. Thus, tools for summarizing, clustering, visualizing and classifying high-dimensional data are essential. In this work, we show…
▽ More
Experiments in particle physics produce enormous quantities of data that must be analyzed and interpreted by teams of physicists. This analysis is often exploratory, where scientists are unable to enumerate the possible types of signal prior to performing the experiment. Thus, tools for summarizing, clustering, visualizing and classifying high-dimensional data are essential. In this work, we show that meaningful physical content can be revealed by transforming the raw data into a learned high-level representation using deep neural networks, with measurements taken at the Daya Bay Neutrino Experiment as a case study. We further show how convolutional deep neural networks can provide an effective classification filter with greater than 97% accuracy across different classes of physics events, significantly better than other machine learning approaches.
△ Less
Submitted 6 December, 2016; v1 submitted 27 January, 2016;
originally announced January 2016.
-
Celeste: Variational inference for a generative model of astronomical images
Authors:
Jeffrey Regier,
Andrew Miller,
Jon McAuliffe,
Ryan Adams,
Matt Hoffman,
Dustin Lang,
David Schlegel,
Prabhat
Abstract:
We present a new, fully generative model of optical telescope image sets, along with a variational procedure for inference. Each pixel intensity is treated as a Poisson random variable, with a rate parameter dependent on latent properties of stars and galaxies. Key latent properties are themselves random, with scientific prior distributions constructed from large ancillary data sets. We check our…
▽ More
We present a new, fully generative model of optical telescope image sets, along with a variational procedure for inference. Each pixel intensity is treated as a Poisson random variable, with a rate parameter dependent on latent properties of stars and galaxies. Key latent properties are themselves random, with scientific prior distributions constructed from large ancillary data sets. We check our approach on synthetic images. We also run it on images from a major sky survey, where it exceeds the performance of the current state-of-the-art method for locating celestial bodies and measuring their colors.
△ Less
Submitted 3 June, 2015;
originally announced June 2015.
-
Scalable Bayesian Optimization Using Deep Neural Networks
Authors:
Jasper Snoek,
Oren Rippel,
Kevin Swersky,
Ryan Kiros,
Nadathur Satish,
Narayanan Sundaram,
Md. Mostofa Ali Patwary,
Prabhat,
Ryan P. Adams
Abstract:
Bayesian optimization is an effective methodology for the global optimization of functions with expensive evaluations. It relies on querying a distribution over functions defined by a relatively cheap surrogate model. An accurate model for this distribution over functions is critical to the effectiveness of the approach, and is typically fit using Gaussian processes (GPs). However, since GPs scale…
▽ More
Bayesian optimization is an effective methodology for the global optimization of functions with expensive evaluations. It relies on querying a distribution over functions defined by a relatively cheap surrogate model. An accurate model for this distribution over functions is critical to the effectiveness of the approach, and is typically fit using Gaussian processes (GPs). However, since GPs scale cubically with the number of observations, it has been challenging to handle objectives whose optimization requires many evaluations, and as such, massively parallelizing the optimization.
In this work, we explore the use of neural networks as an alternative to GPs to model distributions over functions. We show that performing adaptive basis function regression with a neural network as the parametric form performs competitively with state-of-the-art GP-based approaches, but scales linearly with the number of data rather than cubically. This allows us to achieve a previously intractable degree of parallelism, which we apply to large scale hyperparameter optimization, rapidly finding competitive models on benchmark object recognition tasks using convolutional networks, and image caption generation using neural language models.
△ Less
Submitted 13 July, 2015; v1 submitted 19 February, 2015;
originally announced February 2015.
-
Parallelizing Gaussian Process Calculations in R
Authors:
Christopher J. Paciorek,
Benjamin Lipshitz,
Wei Zhuo,
Prabhat,
Cari G. Kaufman,
Rollin C. Thomas
Abstract:
We consider parallel computation for Gaussian process calculations to overcome computational and memory constraints on the size of datasets that can be analyzed. Using a hybrid parallelization approach that uses both threading (shared memory) and message-passing (distributed memory), we implement the core linear algebra operations used in spatial statistics and Gaussian process regression in an R…
▽ More
We consider parallel computation for Gaussian process calculations to overcome computational and memory constraints on the size of datasets that can be analyzed. Using a hybrid parallelization approach that uses both threading (shared memory) and message-passing (distributed memory), we implement the core linear algebra operations used in spatial statistics and Gaussian process regression in an R package called bigGP that relies on C and MPI. The approach divides the matrix into blocks such that the computational load is balanced across processes while communication between processes is limited. The package provides an API enabling R programmers to implement Gaussian process-based methods by using the distributed linear algebra operations without any C or MPI coding. We illustrate the approach and software by analyzing an astrophysics dataset with n=67,275 observations.
△ Less
Submitted 21 May, 2013;
originally announced May 2013.
