-
Variational Gaussian approximation of the Kushner optimal filter
Authors:
Marc Lambert,
Silvère Bonnabel,
Francis Bach
Abstract:
In estimation theory, the Kushner equation provides the evolution of the probability density of the state of a dynamical system given continuous-time observations. Building upon our recent work, we propose a new way to approximate the solution of the Kushner equation through tractable variational Gaussian approximations of two proximal losses associated with the propagation and Bayesian update of…
▽ More
In estimation theory, the Kushner equation provides the evolution of the probability density of the state of a dynamical system given continuous-time observations. Building upon our recent work, we propose a new way to approximate the solution of the Kushner equation through tractable variational Gaussian approximations of two proximal losses associated with the propagation and Bayesian update of the probability density. The first is a proximal loss based on the Wasserstein metric and the second is a proximal loss based on the Fisher metric. The solution to this last proximal loss is given by implicit updates on the mean and covariance that we proposed earlier. These two variational updates can be fused and shown to satisfy a set of stochastic differential equations on the Gaussian's mean and covariance matrix. This Gaussian flow is consistent with the Kalman-Bucy and Riccati flows in the linear case and generalize them in the nonlinear one.
△ Less
Submitted 3 October, 2023;
originally announced October 2023.
-
Variational inference via Wasserstein gradient flows
Authors:
Marc Lambert,
Sinho Chewi,
Francis Bach,
Silvère Bonnabel,
Philippe Rigollet
Abstract:
Along with Markov chain Monte Carlo (MCMC) methods, variational inference (VI) has emerged as a central computational approach to large-scale Bayesian inference. Rather than sampling from the true posterior $π$, VI aims at producing a simple but effective approximation $\hat π$ to $π$ for which summary statistics are easy to compute. However, unlike the well-studied MCMC methodology, algorithmic g…
▽ More
Along with Markov chain Monte Carlo (MCMC) methods, variational inference (VI) has emerged as a central computational approach to large-scale Bayesian inference. Rather than sampling from the true posterior $π$, VI aims at producing a simple but effective approximation $\hat π$ to $π$ for which summary statistics are easy to compute. However, unlike the well-studied MCMC methodology, algorithmic guarantees for VI are still relatively less well-understood. In this work, we propose principled methods for VI, in which $\hat π$ is taken to be a Gaussian or a mixture of Gaussians, which rest upon the theory of gradient flows on the Bures--Wasserstein space of Gaussian measures. Akin to MCMC, it comes with strong theoretical guarantees when $π$ is log-concave.
△ Less
Submitted 21 April, 2023; v1 submitted 31 May, 2022;
originally announced May 2022.
-
Denoising IMU Gyroscopes with Deep Learning for Open-Loop Attitude Estimation
Authors:
Martin Brossard,
Silvere Bonnabel,
Axel Barrau
Abstract:
This paper proposes a learning method for denoising gyroscopes of Inertial Measurement Units (IMUs) using ground truth data, and estimating in real time the orientation (attitude) of a robot in dead reckoning. The obtained algorithm outperforms the state-of-the-art on the (unseen) test sequences. The obtained performances are achieved thanks to a well-chosen model, a proper loss function for orien…
▽ More
This paper proposes a learning method for denoising gyroscopes of Inertial Measurement Units (IMUs) using ground truth data, and estimating in real time the orientation (attitude) of a robot in dead reckoning. The obtained algorithm outperforms the state-of-the-art on the (unseen) test sequences. The obtained performances are achieved thanks to a well-chosen model, a proper loss function for orientation increments, and through the identification of key points when training with high-frequency inertial data. Our approach builds upon a neural network based on dilated convolutions, without requiring any recurrent neural network. We demonstrate how efficient our strategy is for 3D attitude estimation on the EuRoC and TUM-VI datasets. Interestingly, we observe our dead reckoning algorithm manages to beat top-ranked visual-inertial odometry systems in terms of attitude estimation although it does not use vision sensors. We believe this paper offers new perspectives for visual-inertial localization and constitutes a step toward more efficient learning methods involving IMUs. Our open-source implementation is available at https://github.com/mbrossar/denoise-imu-gyro.
△ Less
Submitted 26 June, 2020; v1 submitted 25 February, 2020;
originally announced February 2020.
-
AI-IMU Dead-Reckoning
Authors:
Martin Brossard,
Axel Barrau,
Silvère Bonnabel
Abstract:
In this paper we propose a novel accurate method for dead-reckoning of wheeled vehicles based only on an Inertial Measurement Unit (IMU). In the context of intelligent vehicles, robust and accurate dead-reckoning based on the IMU may prove useful to correlate feeds from imaging sensors, to safely navigate through obstructions, or for safe emergency stops in the extreme case of exteroceptive sensor…
▽ More
In this paper we propose a novel accurate method for dead-reckoning of wheeled vehicles based only on an Inertial Measurement Unit (IMU). In the context of intelligent vehicles, robust and accurate dead-reckoning based on the IMU may prove useful to correlate feeds from imaging sensors, to safely navigate through obstructions, or for safe emergency stops in the extreme case of exteroceptive sensors failure. The key components of the method are the Kalman filter and the use of deep neural networks to dynamically adapt the noise parameters of the filter. The method is tested on the KITTI odometry dataset, and our dead-reckoning inertial method based only on the IMU accurately estimates 3D position, velocity, orientation of the vehicle and self-calibrates the IMU biases. We achieve on average a 1.10% translational error and the algorithm competes with top-ranked methods which, by contrast, use LiDAR or stereo vision. We make our implementation open-source at: https://github.com/mbrossar/ai-imu-dr
△ Less
Submitted 12 April, 2019;
originally announced April 2019.
-
On stability of a class of filters for non-linear stochastic systems
Authors:
Toni Karvonen,
Silvère Bonnabel,
Eric Moulines,
Simo Särkkä
Abstract:
This article develops a comprehensive framework for stability analysis of a broad class of commonly used continuous and discrete time-filters for stochastic dynamic systems with non-linear state dynamics and linear measurements under certain strong assumptions. The class of filters encompasses the extended and unscented Kalman filters and most other Gaussian assumed density filters and their numer…
▽ More
This article develops a comprehensive framework for stability analysis of a broad class of commonly used continuous and discrete time-filters for stochastic dynamic systems with non-linear state dynamics and linear measurements under certain strong assumptions. The class of filters encompasses the extended and unscented Kalman filters and most other Gaussian assumed density filters and their numerical integration approximations. The stability results are in the form of time-uniform mean square bounds and exponential concentration inequalities for the filtering error. In contrast to existing results, it is not always necessary for the model to be exponentially stable or fully observed. We review three classes of models that can be rigorously shown to satisfy the stringent assumptions of the stability theorems. Numerical experiments using synthetic data validate the derived error bounds.
△ Less
Submitted 10 June, 2020; v1 submitted 15 September, 2018;
originally announced September 2018.
-
An intrinsic Cramér-Rao bound on SO(3) for (dynamic) attitude filtering
Authors:
Silvère Bonnabel,
Axel Barrau
Abstract:
In this note an intrinsic version of the Cramér-Rao bound on estimation accuracy is established on the Special Orthogonal group $SO(3)$. It is intrinsic in the sense that it does not rely on a specific choice of coordinates on $SO(3)$: the result is derived using rotation matrices, but remains valid when using other parameterizations, such as quaternions. For any estimator $\hat R$ of…
▽ More
In this note an intrinsic version of the Cramér-Rao bound on estimation accuracy is established on the Special Orthogonal group $SO(3)$. It is intrinsic in the sense that it does not rely on a specific choice of coordinates on $SO(3)$: the result is derived using rotation matrices, but remains valid when using other parameterizations, such as quaternions. For any estimator $\hat R$ of $R\in SO(3)$ we give indeed a lower bound on the quantity $E(\log(R\hat R^T))$, that is, the estimation error expressed in terms of group multiplication, whereas the usual estimation error $E(\hat R-R)$ is meaningless on $SO(3)$. The result is first applied to Whaba's problem. Then, we consider the problem of a continuous-time nonlinear deterministic system on $SO(3)$ with discrete measurements subject to additive isotropic Gaussian noise, and we derive a lower bound to the estimation error covariance matrix. We prove the intrinsic Cramér-Rao bound coincides with the covariance matrix returned by the Invariant EKF, and thus can be computed online. This is in sharp contrast with the general case, where the bound can only be computed if the true trajectory of the system is known.
△ Less
Submitted 12 October, 2015; v1 submitted 16 March, 2015;
originally announced March 2015.
-
Stochastic gradient descent on Riemannian manifolds
Authors:
Silvere Bonnabel
Abstract:
Stochastic gradient descent is a simple approach to find the local minima of a cost function whose evaluations are corrupted by noise. In this paper, we develop a procedure extending stochastic gradient descent algorithms to the case where the function is defined on a Riemannian manifold. We prove that, as in the Euclidian case, the gradient descent algorithm converges to a critical point of the c…
▽ More
Stochastic gradient descent is a simple approach to find the local minima of a cost function whose evaluations are corrupted by noise. In this paper, we develop a procedure extending stochastic gradient descent algorithms to the case where the function is defined on a Riemannian manifold. We prove that, as in the Euclidian case, the gradient descent algorithm converges to a critical point of the cost function. The algorithm has numerous potential applications, and is illustrated here by four examples. In particular a novel gossip algorithm on the set of covariance matrices is derived and tested numerically.
△ Less
Submitted 19 November, 2013; v1 submitted 22 November, 2011;
originally announced November 2011.
