Showing 1–16 of 16 results for author: Takao, S

Scalable Data Assimilation with Message Passing

Authors: Oscar Key, So Takao, Daniel Giles, Marc Peter Deisenroth

Abstract: Data assimilation is a core component of numerical weather prediction systems. The large quantity of data processed during assimilation requires the computation to be distributed across increasingly many compute nodes, yet existing approaches suffer from synchronisation overhead in this setting. In this paper, we exploit the formulation of data assimilation as a Bayesian inference problem and appl… ▽ More Data assimilation is a core component of numerical weather prediction systems. The large quantity of data processed during assimilation requires the computation to be distributed across increasingly many compute nodes, yet existing approaches suffer from synchronisation overhead in this setting. In this paper, we exploit the formulation of data assimilation as a Bayesian inference problem and apply a message-passing algorithm to solve the spatial inference problem. Since message passing is inherently based on local computations, this approach lends itself to parallel and distributed computation. In combination with a GPU-accelerated implementation, we can scale the algorithm to very large grid sizes while retaining good accuracy and compute and memory requirements. △ Less

Submitted 19 April, 2024; originally announced April 2024.

Iterated INLA for State and Parameter Estimation in Nonlinear Dynamical Systems

Authors: Rafael Anderka, Marc Peter Deisenroth, So Takao

Abstract: Data assimilation (DA) methods use priors arising from differential equations to robustly interpolate and extrapolate data. Popular techniques such as ensemble methods that handle high-dimensional, nonlinear PDE priors focus mostly on state estimation, however can have difficulty learning the parameters accurately. On the other hand, machine learning based approaches can naturally learn the state… ▽ More Data assimilation (DA) methods use priors arising from differential equations to robustly interpolate and extrapolate data. Popular techniques such as ensemble methods that handle high-dimensional, nonlinear PDE priors focus mostly on state estimation, however can have difficulty learning the parameters accurately. On the other hand, machine learning based approaches can naturally learn the state and parameters, but their applicability can be limited, or produce uncertainties that are hard to interpret. Inspired by the Integrated Nested Laplace Approximation (INLA) method in spatial statistics, we propose an alternative approach to DA based on iteratively linearising the dynamical model. This produces a Gaussian Markov random field at each iteration, enabling one to use INLA to infer the state and parameters. Our approach can be used for arbitrary nonlinear systems, while retaining interpretability, and is furthermore demonstrated to outperform existing methods on the DA task. By providing a more nuanced approach to handling nonlinear PDE priors, our methodology offers improved accuracy and robustness in predictions, especially where data sparsity is prevalent. △ Less

Submitted 3 June, 2024; v1 submitted 26 February, 2024; originally announced February 2024.

Semimartingale driven mechanics and reduction by symmetry for stochastic and dissipative dynamical systems

Authors: Oliver D. Street, So Takao

Abstract: The recent interest in structure preserving stochastic Lagrangian and Hamiltonian systems raises questions regarding how such models are to be understood and the principles through which they are to be derived. By considering a mathematically sound extension of the Hamilton-Pontryagin principle, we derive a stochastic analogue of the Euler-Lagrange equations, driven by independent semimartingales.… ▽ More The recent interest in structure preserving stochastic Lagrangian and Hamiltonian systems raises questions regarding how such models are to be understood and the principles through which they are to be derived. By considering a mathematically sound extension of the Hamilton-Pontryagin principle, we derive a stochastic analogue of the Euler-Lagrange equations, driven by independent semimartingales. Using this as a starting point, we can apply symmetry reduction carefully to derive non-canonical stochastic Lagrangian / Hamiltonian systems, including the stochastic Euler-Poincaré / Lie-Poisson equations, studied extensively in the literature. Furthermore, we develop a framework to include dissipation that balances the structure-preserving noise in such a way that the overall stochastic dynamics preserves the Gibbs measure on the symplectic manifold, where the dynamics effectively take place. In particular, this leads to a new derivation of double-bracket dissipation by considering Lie group invariant stochastic dissipative dynamics, taking place on the cotangent bundle of the group. △ Less

Submitted 19 December, 2023; v1 submitted 15 December, 2023; originally announced December 2023.

A Geometric Extension of the Itô-Wentzell and Kunita's Formulas

Authors: Aythami Bethencourt de León, So Takao

Abstract: We extend the Itô-Wentzell formula for the evolution along a continuous semimartingale of a time-dependent stochastic field driven by a continuous semimartingale to tensor field-valued stochastic processes on manifolds. More concretely, we investigate how the pull-back (respectively, the push-forward) by a stochastic flow of diffeomorphisms of a time-dependent stochastic tensor field driven by a c… ▽ More We extend the Itô-Wentzell formula for the evolution along a continuous semimartingale of a time-dependent stochastic field driven by a continuous semimartingale to tensor field-valued stochastic processes on manifolds. More concretely, we investigate how the pull-back (respectively, the push-forward) by a stochastic flow of diffeomorphisms of a time-dependent stochastic tensor field driven by a continuous semimartingale evolves with time, deriving it under suitable regularity conditions. We call this result the Kunita-Itô-Wentzell (KIW) formula for the advection of tensor-valued stochastic processes. Equations of this nature bear significance in stochastic fluid dynamics and well-posedness by noise problems, facilitating the development of certain geometric extensions within existing theories. △ Less

Submitted 7 November, 2023; originally announced November 2023.

Gaussian Processes on Cellular Complexes

Authors: Mathieu Alain, So Takao, Brooks Paige, Marc Peter Deisenroth

Abstract: In recent years, there has been considerable interest in develo** machine learning models on graphs in order to account for topological inductive biases. In particular, recent attention was given to Gaussian processes on such structures since they can additionally account for uncertainty. However, graphs are limited to modelling relations between two vertices. In this paper, we go beyond this dy… ▽ More In recent years, there has been considerable interest in develo** machine learning models on graphs in order to account for topological inductive biases. In particular, recent attention was given to Gaussian processes on such structures since they can additionally account for uncertainty. However, graphs are limited to modelling relations between two vertices. In this paper, we go beyond this dyadic setting and consider polyadic relations that include interactions between vertices, edges and one of their generalisations, known as cells. Specifically, we propose Gaussian processes on cellular complexes, a generalisation of graphs that captures interactions between these higher-order cells. One of our key contributions is the derivation of two novel kernels, one that generalises the graph Matérn kernel and one that additionally mixes information of different cell types. △ Less

Submitted 2 November, 2023; originally announced November 2023.

Actually Sparse Variational Gaussian Processes

Authors: Harry Jake Cunningham, Daniel Augusto de Souza, So Takao, Mark van der Wilk, Marc Peter Deisenroth

Abstract: Gaussian processes (GPs) are typically criticised for their unfavourable scaling in both computational and memory requirements. For large datasets, sparse GPs reduce these demands by conditioning on a small set of inducing variables designed to summarise the data. In practice however, for large datasets requiring many inducing variables, such as low-lengthscale spatial data, even sparse GPs can be… ▽ More Gaussian processes (GPs) are typically criticised for their unfavourable scaling in both computational and memory requirements. For large datasets, sparse GPs reduce these demands by conditioning on a small set of inducing variables designed to summarise the data. In practice however, for large datasets requiring many inducing variables, such as low-lengthscale spatial data, even sparse GPs can become computationally expensive, limited by the number of inducing variables one can use. In this work, we propose a new class of inter-domain variational GP, constructed by projecting a GP onto a set of compactly supported B-spline basis functions. The key benefit of our approach is that the compact support of the B-spline basis functions admits the use of sparse linear algebra to significantly speed up matrix operations and drastically reduce the memory footprint. This allows us to very efficiently model fast-varying spatial phenomena with tens of thousands of inducing variables, where previous approaches failed. △ Less

Submitted 11 April, 2023; originally announced April 2023.

Comments: 14 pages, 5 figures, published in AISTATS 2023

Short-term Prediction and Filtering of Solar Power Using State-Space Gaussian Processes

Authors: Sean Nassimiha, Peter Dudfield, Jack Kelly, Marc Peter Deisenroth, So Takao

Abstract: Short-term forecasting of solar photovoltaic energy (PV) production is important for powerplant management. Ideally these forecasts are equipped with error bars, so that downstream decisions can account for uncertainty. To produce predictions with error bars in this setting, we consider Gaussian processes (GPs) for modelling and predicting solar photovoltaic energy production in the UK. A standard… ▽ More Short-term forecasting of solar photovoltaic energy (PV) production is important for powerplant management. Ideally these forecasts are equipped with error bars, so that downstream decisions can account for uncertainty. To produce predictions with error bars in this setting, we consider Gaussian processes (GPs) for modelling and predicting solar photovoltaic energy production in the UK. A standard application of GP regression on the PV timeseries data is infeasible due to the large data size and non-Gaussianity of PV readings. However, this is made possible by leveraging recent advances in scalable GP inference, in particular, by using the state-space form of GPs, combined with modern variational inference techniques. The resulting model is not only scalable to large datasets but can also handle continuous data streams via Kalman filtering. △ Less

Submitted 30 March, 2023; v1 submitted 1 February, 2023; originally announced February 2023.

Comments: Workshop paper submitted to "Tackling Climate Change with Machine Learning: workshop at NeurIPS 2022"

Transport noise restores uniqueness and prevents blow-up in geometric transport equations

Authors: Aythami Bethencourt-de-León, So Takao

Abstract: In this work, we demonstrate well-posedness and regularisation by noise results for a class of geometric transport equations that contains, among others, the linear transport and continuity equations. This class is known as linear advection of $k$-forms. In particular, we prove global existence and uniqueness of $L^p$-solutions to the stochastic equation, driven by a spatially $α$-Hölder drift… ▽ More In this work, we demonstrate well-posedness and regularisation by noise results for a class of geometric transport equations that contains, among others, the linear transport and continuity equations. This class is known as linear advection of $k$-forms. In particular, we prove global existence and uniqueness of $L^p$-solutions to the stochastic equation, driven by a spatially $α$-Hölder drift $b$, uniformly bounded in time, with an integrability condition on the distributional derivative of $b$, and sufficiently regular diffusion vector fields. Furthermore, we prove that all our solutions are continuous if the initial datum is continuous. Finally, we show that our class of equations without noise admits infinitely many $L^p$-solutions and is hence ill-posed. Moreover, the deterministic solutions can be discontinuous in both time and space independently of the regularity of the initial datum. We also demonstrate that for certain initial data of class $C^\infty_{0},$ the deterministic $L^p$-solutions blow up instantaneously in the space $L^{\infty}_{loc}$. In order to establish our results, we employ characteristics-based techniques that exploit the geometric structure of our equations. △ Less

Submitted 26 November, 2022; originally announced November 2022.

MSC Class: 60H50; 60H15; 35R25; 53Z05

Vector-valued Gaussian Processes on Riemannian Manifolds via Gauge Independent Projected Kernels

Authors: Michael Hutchinson, Alexander Terenin, Viacheslav Borovitskiy, So Takao, Yee Whye Teh, Marc Peter Deisenroth

Abstract: Gaussian processes are machine learning models capable of learning unknown functions in a way that represents uncertainty, thereby facilitating construction of optimal decision-making systems. Motivated by a desire to deploy Gaussian processes in novel areas of science, a rapidly-growing line of research has focused on constructively extending these models to handle non-Euclidean domains, includin… ▽ More Gaussian processes are machine learning models capable of learning unknown functions in a way that represents uncertainty, thereby facilitating construction of optimal decision-making systems. Motivated by a desire to deploy Gaussian processes in novel areas of science, a rapidly-growing line of research has focused on constructively extending these models to handle non-Euclidean domains, including Riemannian manifolds, such as spheres and tori. We propose techniques that generalize this class to model vector fields on Riemannian manifolds, which are important in a number of application areas in the physical sciences. To do so, we present a general recipe for constructing gauge independent kernels, which induce Gaussian vector fields, i.e. vector-valued Gaussian processes coherent with geometry, from scalar-valued Riemannian kernels. We extend standard Gaussian process training methods, such as variational inference, to this setting. This enables vector-valued Gaussian processes on Riemannian manifolds to be trained using standard methods and makes them accessible to machine learning practitioners. △ Less

Submitted 25 November, 2021; v1 submitted 27 October, 2021; originally announced October 2021.

Journal ref: Advances in Neural Information Processing Systems, 2021

A Unifying and Canonical Description of Measure-Preserving Diffusions

Authors: Alessandro Barp, So Takao, Michael Betancourt, Alexis Arnaudon, Mark Girolami

Abstract: A complete recipe of measure-preserving diffusions in Euclidean space was recently derived unifying several MCMC algorithms into a single framework. In this paper, we develop a geometric theory that improves and generalises this construction to any manifold. We thereby demonstrate that the completeness result is a direct consequence of the topology of the underlying manifold and the geometry induc… ▽ More A complete recipe of measure-preserving diffusions in Euclidean space was recently derived unifying several MCMC algorithms into a single framework. In this paper, we develop a geometric theory that improves and generalises this construction to any manifold. We thereby demonstrate that the completeness result is a direct consequence of the topology of the underlying manifold and the geometry induced by the target measure $P$; there is no need to introduce other structures such as a Riemannian metric, local coordinates, or a reference measure. Instead, our framework relies on the intrinsic geometry of $P$ and in particular its canonical derivative, the deRham rotationnel, which allows us to parametrise the Fokker--Planck currents of measure-preserving diffusions using potentials. The geometric formalism can easily incorporate constraints and symmetries, and deliver new important insights, for example, a new complete recipe of Langevin-like diffusions that are suited to the construction of samplers. We also analyse the reversibility and dissipative properties of the diffusions, the associated deterministic flow on the space of measures, and the geometry of Langevin processes. Our article connects ideas from various literature and frames the theory of measure-preserving diffusions in its appropriate mathematical context. △ Less

Submitted 6 May, 2021; originally announced May 2021.

Modelling the climate and weather of a 2D Lagrangian-averaged Euler-Boussinesq equation with transport noise

Authors: Diego Alonso-Oran, Aythami Bethencourt de Leon, Darryl Holm, So Takao

Abstract: The prediction of climate change and its impact on extreme weather events is one of the great societal and intellectual challenges of our time. The first part of the problem is to make the distinction between weather and climate. The second part is to understand the dynamics of the fluctuations of the physical variables. The third part is to predict how the variances of the fluctuations are affect… ▽ More The prediction of climate change and its impact on extreme weather events is one of the great societal and intellectual challenges of our time. The first part of the problem is to make the distinction between weather and climate. The second part is to understand the dynamics of the fluctuations of the physical variables. The third part is to predict how the variances of the fluctuations are affected by statistical correlations in their fluctuating dynamics. This paper investigates a framework called LA SALT which can meet all three parts of the challenge for the problem of climate change. As a tractable example of this framework, we consider the Euler--Boussinesq (EB) equations for an incompressible stratified fluid flowing under gravity in a vertical plane with no other external forcing. All three parts of the problem are solved for this case. In fact, for this problem, the framework also delivers global well-posedness of the dynamics of the physical variables and closed dynamical equations for the moments of their fluctuations. Thus, in a well-posed mathematical setting, the framework developed in this paper shows that the mean field dynamics combines with an intricate array of correlations in the fluctuation dynamics to drive the evolution of the mean statistics. The results of the framework for 2D EB model analysis define its climate, as well as climate change, weather dynamics, and change of weather statistics, all in the context of a model system of SPDEs with unique global strong solutions. △ Less

Submitted 1 September, 2019; originally announced September 2019.

Well-posedness by noise for linear advection of $k$-forms

Authors: Aythami Bethencourt de Leon, So Takao

Abstract: In this work, we extend existing well-posedness by noise results for the stochastic transport and continuity equations by treating them as special cases of the linear advection equation of $k$-forms, which arises naturally in geometric fluid dynamics. In particular, we prove the existence and uniqueness of weak $L^p$-solutions to the stochastic linear advection equation of $k$-forms that is driven… ▽ More In this work, we extend existing well-posedness by noise results for the stochastic transport and continuity equations by treating them as special cases of the linear advection equation of $k$-forms, which arises naturally in geometric fluid dynamics. In particular, we prove the existence and uniqueness of weak $L^p$-solutions to the stochastic linear advection equation of $k$-forms that is driven by a Hölder continuous, $W^{1,1}_{loc}$ drift and smooth diffusion vector fields, such that the equation without noise admits infinitely many solutions. △ Less

Submitted 26 November, 2022; v1 submitted 30 April, 2019; originally announced April 2019.

Comments: Fundamental inaccuracies. These were addressed in a different project, which is NOT an update of this paper. Do not cite

Irreversible Langevin MCMC on Lie Groups

Authors: Alexis Arnaudon, Alessandro Barp, So Takao

Abstract: It is well-known that irreversible MCMC algorithms converge faster to their stationary distributions than reversible ones. Using the special geometric structure of Lie groups $\mathcal G$ and dissipation fields compatible with the symplectic structure, we construct an irreversible HMC-like MCMC algorithm on $\mathcal G$, where we first update the momentum by solving an OU process on the correspond… ▽ More It is well-known that irreversible MCMC algorithms converge faster to their stationary distributions than reversible ones. Using the special geometric structure of Lie groups $\mathcal G$ and dissipation fields compatible with the symplectic structure, we construct an irreversible HMC-like MCMC algorithm on $\mathcal G$, where we first update the momentum by solving an OU process on the corresponding Lie algebra $\mathfrak g$, and then approximate the Hamiltonian system on $\mathcal G \times \mathfrak g$ with a reversible symplectic integrator followed by a Metropolis-Hastings correction step. In particular, when the OU process is simulated over sufficiently long times, we recover HMC as a special case. We illustrate this algorithm numerically using the example $\mathcal G = SO(3)$. △ Less

Submitted 21 March, 2019; originally announced March 2019.

Implications of Kunita-Itô-Wentzell formula for $k$-forms in stochastic fluid dynamics

Authors: Aythami Bethencourt de Léon, Darryl Holm, Erwin Luesink, So Takao

Abstract: We extend the Itô-Wentzell formula for the evolution of a time-dependent stochastic field along a semimartingale to $k$-form-valued stochastic processes. The result is the Kunita-Itô-Wentzell (KIW) formula for $k$-forms. We also establish a correspondence between the KIW formula for $k$-forms derived here and a certain class of stochastic fluid dynamics models which preserve the geometric structur… ▽ More We extend the Itô-Wentzell formula for the evolution of a time-dependent stochastic field along a semimartingale to $k$-form-valued stochastic processes. The result is the Kunita-Itô-Wentzell (KIW) formula for $k$-forms. We also establish a correspondence between the KIW formula for $k$-forms derived here and a certain class of stochastic fluid dynamics models which preserve the geometric structure of deterministic ideal fluid dynamics. This geometric structure includes Eulerian and Lagrangian variational principles, Lie--Poisson Hamiltonian formulations and natural analogues of the Kelvin circulation theorem, all derived in the stochastic setting. △ Less

Submitted 17 March, 2019; originally announced March 2019.

The Burgers' equation with stochastic transport: shock formation, local and global existence of smooth solutions

Authors: Diego Alonso-Orán, Aythami Bethencourt de León, So Takao

Abstract: In this work, we examine the solution properties of the Burgers' equation with stochastic transport. First, we prove results on the formation of shocks in the stochastic equation and then obtain a stochastic Rankine-Hugoniot condition that the shocks satisfy. Next, we establish the local existence and uniqueness of smooth solutions in the inviscid case and construct a blow-up criterion. Finally, i… ▽ More In this work, we examine the solution properties of the Burgers' equation with stochastic transport. First, we prove results on the formation of shocks in the stochastic equation and then obtain a stochastic Rankine-Hugoniot condition that the shocks satisfy. Next, we establish the local existence and uniqueness of smooth solutions in the inviscid case and construct a blow-up criterion. Finally, in the viscous case, we prove global existence and uniqueness of smooth solutions. △ Less

Submitted 8 November, 2022; v1 submitted 23 August, 2018; originally announced August 2018.

Networks of Coadjoint Orbits: from Geometric to Statistical Mechanics

Authors: Alexis Arnaudon, So Takao

Abstract: A class of network models with symmetry group $G$ that evolve as a Lie-Poisson system is derived from the framework of geometric mechanics, which generalises the classical Heisenberg model studied in statistical mechanics. We considered two ways of coupling the spins: one via the momentum and the other via the position and studied in details the equilibrium solutions and their corresponding nonlin… ▽ More A class of network models with symmetry group $G$ that evolve as a Lie-Poisson system is derived from the framework of geometric mechanics, which generalises the classical Heisenberg model studied in statistical mechanics. We considered two ways of coupling the spins: one via the momentum and the other via the position and studied in details the equilibrium solutions and their corresponding nonlinear stability properties using the energy-Casimir method. We then took the example $G=SO(3)$ and saw that the momentum-coupled system reduces to the classical Heisenberg model with massive spins and the position-coupled case reduces to a new system that has a broken symmetry group $SO(3)/SO(2)$ similar to the heavy top. In the latter system, we numerically observed an interesting synchronisation-like phenomenon for a certain class of initial conditions. Adding a type of noise and dissipation that preserves the coadjoint orbit of the network model, we found that the invariant measure is given by the Gibbs measure, from which the notion of temperature is defined. We then observed a surprising `triple-humped' phase transition in the heavy top-like lattice model, where the spins switched from one equilibrium position to another before losing magnetisation as we increased the temperature. This work is only a first step towards connecting geometric mechanics with statistical mechanics and several interesting problems are open for further investigation. △ Less

Submitted 30 April, 2018; originally announced April 2018.