-
Gaussian Ensemble Belief Propagation for Efficient Inference in High-Dimensional Systems
Authors:
Dan MacKinlay,
Russell Tsuchida,
Dan Pagendam,
Petra Kuhnert
Abstract:
Efficient inference in high-dimensional models remains a central challenge in machine learning. This paper introduces the Gaussian Ensemble Belief Propagation (GEnBP) algorithm, a fusion of the Ensemble Kalman filter and Gaussian Belief Propagation (GaBP) methods. GEnBP updates ensembles by passing low-rank local messages over a graphical model. This combination inherits favourable qualities from…
▽ More
Efficient inference in high-dimensional models remains a central challenge in machine learning. This paper introduces the Gaussian Ensemble Belief Propagation (GEnBP) algorithm, a fusion of the Ensemble Kalman filter and Gaussian Belief Propagation (GaBP) methods. GEnBP updates ensembles by passing low-rank local messages over a graphical model. This combination inherits favourable qualities from each method. Ensemble techniques allow GEnBP to handle high-dimensional states, parameters and intricate, noisy, black-box generation processes. The use of local messages in a graphical model structure ensures that the approach can efficiently handle complex dependence structures. GEnBP is advantageous when the ensemble size may be considerably smaller than the inference dimension. This scenario often arises in fields such as spatiotemporal modelling, image processing and physical model inversion. GEnBP can be applied to general problem structures, including data assimilation, system identification and hierarchical models.
Supporting code is available at https://github.com/danmackinlay/GEnBP
△ Less
Submitted 22 May, 2024; v1 submitted 12 February, 2024;
originally announced February 2024.
-
Bayesian Physics Informed Neural Networks for Data Assimilation and Spatio-Temporal Modelling of Wildfires
Authors:
Joel Janek Dabrowski,
Daniel Edward Pagendam,
James Hilton,
Conrad Sanderson,
Daniel MacKinlay,
Carolyn Huston,
Andrew Bolt,
Petra Kuhnert
Abstract:
We apply the Physics Informed Neural Network (PINN) to the problem of wildfire fire-front modelling. We use the PINN to solve the level-set equation, which is a partial differential equation that models a fire-front through the zero-level-set of a level-set function. The result is a PINN that simulates a fire-front as it propagates through the spatio-temporal domain. We show that popular optimisat…
▽ More
We apply the Physics Informed Neural Network (PINN) to the problem of wildfire fire-front modelling. We use the PINN to solve the level-set equation, which is a partial differential equation that models a fire-front through the zero-level-set of a level-set function. The result is a PINN that simulates a fire-front as it propagates through the spatio-temporal domain. We show that popular optimisation cost functions used in the literature can result in PINNs that fail to maintain temporal continuity in modelled fire-fronts when there are extreme changes in exogenous forcing variables such as wind direction. We thus propose novel additions to the optimisation cost function that improves temporal continuity under these extreme changes. Furthermore, we develop an approach to perform data assimilation within the PINN such that the PINN predictions are drawn towards observations of the fire-front. Finally, we incorporate our novel approaches into a Bayesian PINN (B-PINN) to provide uncertainty quantification in the fire-front predictions. This is significant as the standard solver, the level-set method, does not naturally offer the capability for data assimilation and uncertainty quantification. Our results show that, with our novel approaches, the B-PINN can produce accurate predictions with high quality uncertainty quantification on real-world data.
△ Less
Submitted 26 April, 2023; v1 submitted 2 December, 2022;
originally announced December 2022.
-
PDEBENCH: An Extensive Benchmark for Scientific Machine Learning
Authors:
Makoto Takamoto,
Timothy Praditia,
Raphael Leiteritz,
Dan MacKinlay,
Francesco Alesiani,
Dirk Pflüger,
Mathias Niepert
Abstract:
Machine learning-based modeling of physical systems has experienced increased interest in recent years. Despite some impressive progress, there is still a lack of benchmarks for Scientific ML that are easy to use but still challenging and representative of a wide range of problems. We introduce PDEBench, a benchmark suite of time-dependent simulation tasks based on Partial Differential Equations (…
▽ More
Machine learning-based modeling of physical systems has experienced increased interest in recent years. Despite some impressive progress, there is still a lack of benchmarks for Scientific ML that are easy to use but still challenging and representative of a wide range of problems. We introduce PDEBench, a benchmark suite of time-dependent simulation tasks based on Partial Differential Equations (PDEs). PDEBench comprises both code and data to benchmark the performance of novel machine learning models against both classical numerical simulations and machine learning baselines. Our proposed set of benchmark problems contribute the following unique features: (1) A much wider range of PDEs compared to existing benchmarks, ranging from relatively common examples to more realistic and difficult problems; (2) much larger ready-to-use datasets compared to prior work, comprising multiple simulation runs across a larger number of initial and boundary conditions and PDE parameters; (3) more extensible source codes with user-friendly APIs for data generation and baseline results with popular machine learning models (FNO, U-Net, PINN, Gradient-Based Inverse Method). PDEBench allows researchers to extend the benchmark freely for their own purposes using a standardized API and to compare the performance of new models to existing baseline methods. We also propose new evaluation metrics with the aim to provide a more holistic understanding of learning methods in the context of Scientific ML. With those metrics we identify tasks which are challenging for recent ML methods and propose these tasks as future challenges for the community. The code is available at https://github.com/pdebench/PDEBench.
△ Less
Submitted 13 March, 2023; v1 submitted 13 October, 2022;
originally announced October 2022.
-
A Universal Splitting Estimator for the Performance Evaluation of Wireless Communications Systems
Authors:
Nadhir Ben Rached,
Daniel MacKinlay,
Zdravko Botev,
Raul Tempone,
Mohamed-Slim Alouini
Abstract:
We propose a unified rare-event estimator for the performance evaluation of wireless communication systems. The estimator is derived from the well-known multilevel splitting algorithm. In its original form, the splitting algorithm cannot be applied to the simulation and estimation of time-independent problems, because splitting requires an underlying continuous-time Markov process whose trajectori…
▽ More
We propose a unified rare-event estimator for the performance evaluation of wireless communication systems. The estimator is derived from the well-known multilevel splitting algorithm. In its original form, the splitting algorithm cannot be applied to the simulation and estimation of time-independent problems, because splitting requires an underlying continuous-time Markov process whose trajectories can be split. We tackle this problem by embedding the static problem of interest within a continuous-time Markov process, so that the target time-independent distribution becomes the distribution of the Markov process at a given time instant. The main feature of the proposed multilevel splitting algorithm is its large scope of applicability. For illustration, we show how the same algorithm can be applied to the problem of estimating the cumulative distribution function (CDF) of sums of random variables (RVs), the CDF of partial sums of ordered RVs, the CDF of ratios of RVs, and the CDF of weighted sums of Poisson RVs. We investigate the computational efficiency of the proposed estimator via a number of simulation studies and find that it compares favorably with existing estimators.
△ Less
Submitted 28 August, 2019;
originally announced August 2019.
