Showing 1–15 of 15 results for author: Septier, F

Spherical Sliced-Wasserstein

Authors: Clément Bonet, Paul Berg, Nicolas Courty, François Septier, Lucas Drumetz, Minh-Tan Pham

Abstract: Many variants of the Wasserstein distance have been introduced to reduce its original computational burden. In particular the Sliced-Wasserstein distance (SW), which leverages one-dimensional projections for which a closed-form solution of the Wasserstein distance is available, has received a lot of interest. Yet, it is restricted to data living in Euclidean spaces, while the Wasserstein distance… ▽ More Many variants of the Wasserstein distance have been introduced to reduce its original computational burden. In particular the Sliced-Wasserstein distance (SW), which leverages one-dimensional projections for which a closed-form solution of the Wasserstein distance is available, has received a lot of interest. Yet, it is restricted to data living in Euclidean spaces, while the Wasserstein distance has been studied and used recently on manifolds. We focus more specifically on the sphere, for which we define a novel SW discrepancy, which we call spherical Sliced-Wasserstein, making a first step towards defining SW discrepancies on manifolds. Our construction is notably based on closed-form solutions of the Wasserstein distance on the circle, together with a new spherical Radon transform. Along with efficient algorithms and the corresponding implementations, we illustrate its properties in several machine learning use cases where spherical representations of data are at stake: sampling on the sphere, density estimation on real earth data or hyperspherical auto-encoders. △ Less

Submitted 30 January, 2023; v1 submitted 17 June, 2022; originally announced June 2022.

Comments: Published as a conference paper at ICLR 2023

State space partitioning based on constrained spectral clustering for block particle filtering

Authors: Rui Min, Christelle Garnier, François Septier, John Klein

Abstract: The particle filter (PF) is a powerful inference tool widely used to estimate the filtering distribution in non-linear and/or non-Gaussian problems. To overcome the curse of dimensionality of PF, the block PF (BPF) inserts a blocking step to partition the state space into several subspaces or blocks of smaller dimension so that the correction and resampling steps can be performed independently on… ▽ More The particle filter (PF) is a powerful inference tool widely used to estimate the filtering distribution in non-linear and/or non-Gaussian problems. To overcome the curse of dimensionality of PF, the block PF (BPF) inserts a blocking step to partition the state space into several subspaces or blocks of smaller dimension so that the correction and resampling steps can be performed independently on each subspace. Using blocks of small size reduces the variance of the filtering distribution estimate, but in turn the correlation between blocks is broken and a bias is introduced. When the dependence relationships between state variables are unknown, it is not obvious to decide how to split the state space into blocks and a significant error overhead may arise from a poor choice of partitioning. In this paper, we formulate the partitioning problem in the BPF as a clustering problem and we propose a state space partitioning method based on spectral clustering (SC). We design a generalized BPF algorithm that contains two new steps: (i) estimation of the state vector correlation matrix from predicted particles, (ii) SC using this estimate as the similarity matrix to determine an appropriate partition. In addition, a constraint is imposed on the maximal cluster size to prevent SC from providing too large blocks. We show that the proposed method can bring together in the same blocks the most correlated state variables while successfully esca** the curse of dimensionality. △ Less

Submitted 7 March, 2022; originally announced March 2022.

Sequential Detection of a Temporary Change in Multivariate Time Series

Authors: V. Watson, F. Septier, P. Armand, C. Duchenne

Abstract: In this work, we aim to provide a new and efficient recursive detection method for temporarily monitored signals. Motivated by the case of the propagation of an event over a field of sensors, we assumed that the change in the statistical properties in the monitored signals can only be temporary. Unfortunately, to our best knowledge, existing recursive and simple detection techniques such as the on… ▽ More In this work, we aim to provide a new and efficient recursive detection method for temporarily monitored signals. Motivated by the case of the propagation of an event over a field of sensors, we assumed that the change in the statistical properties in the monitored signals can only be temporary. Unfortunately, to our best knowledge, existing recursive and simple detection techniques such as the ones based on the cumulative sum (CUSUM) do not consider the temporary aspect of the change in a multivariate time series. In this paper, we propose a novel simple and efficient sequential detection algorithm, named Temporary-Event-CUSUM (TE-CUSUM). By combining with a new adaptive way to aggregate local CUSUM variables from each data stream, we empirically show that the TE-CUSUM has a very good detection rate in the case of an event passing through a field of sensors in a very noisy environment. △ Less

Submitted 15 March, 2022; v1 submitted 29 October, 2021; originally announced October 2021.

Efficient Gradient Flows in Sliced-Wasserstein Space

Authors: Clément Bonet, Nicolas Courty, François Septier, Lucas Drumetz

Abstract: Minimizing functionals in the space of probability distributions can be done with Wasserstein gradient flows. To solve them numerically, a possible approach is to rely on the Jordan-Kinderlehrer-Otto (JKO) scheme which is analogous to the proximal scheme in Euclidean spaces. However, it requires solving a nested optimization problem at each iteration, and is known for its computational challenges,… ▽ More Minimizing functionals in the space of probability distributions can be done with Wasserstein gradient flows. To solve them numerically, a possible approach is to rely on the Jordan-Kinderlehrer-Otto (JKO) scheme which is analogous to the proximal scheme in Euclidean spaces. However, it requires solving a nested optimization problem at each iteration, and is known for its computational challenges, especially in high dimension. To alleviate it, very recent works propose to approximate the JKO scheme leveraging Brenier's theorem, and using gradients of Input Convex Neural Networks to parameterize the density (JKO-ICNN). However, this method comes with a high computational cost and stability issues. Instead, this work proposes to use gradient flows in the space of probability measures endowed with the sliced-Wasserstein (SW) distance. We argue that this method is more flexible than JKO-ICNN, since SW enjoys a closed-form differentiable approximation. Thus, the density at each step can be parameterized by any generative model which alleviates the computational burden and makes it tractable in higher dimensions. △ Less

Submitted 15 November, 2022; v1 submitted 21 October, 2021; originally announced October 2021.

Comments: Published in Transactions on Machine Learning Research (November 2022)

Subspace Detours Meet Gromov-Wasserstein

Authors: Clément Bonet, Nicolas Courty, François Septier, Lucas Drumetz

Abstract: In the context of optimal transport methods, the subspace detour approach was recently presented by Muzellec and Cuturi (2019). It consists in building a nearly optimal transport plan in the measures space from an optimal transport plan in a wisely chosen subspace, onto which the original measures are projected. The contribution of this paper is to extend this category of methods to the Gromov-Was… ▽ More In the context of optimal transport methods, the subspace detour approach was recently presented by Muzellec and Cuturi (2019). It consists in building a nearly optimal transport plan in the measures space from an optimal transport plan in a wisely chosen subspace, onto which the original measures are projected. The contribution of this paper is to extend this category of methods to the Gromov-Wasserstein problem, which is a particular type of transport distance involving the inner geometry of the compared distributions. After deriving the associated formalism and properties, we also discuss a specific cost for which we can show connections with the Knothe-Rosenblatt rearrangement. We finally give an experimental illustration on a shape matching problem. △ Less

Submitted 21 October, 2021; originally announced October 2021.

Semi-independent resampling for particle filtering

Authors: Roland Lamberti, Yohan Petetin, François Desbouvries, François Septier

Abstract: Among Sequential Monte Carlo (SMC) methods,Sampling Importance Resampling (SIR) algorithms are based on Importance Sampling (IS) and on some resampling-based)rejuvenation algorithm which aims at fighting against weight degeneracy. However %whichever the resampling technique used this mechanism tends to be insufficient when applied to informative or high-dimensional models. In this paper we revisit… ▽ More Among Sequential Monte Carlo (SMC) methods,Sampling Importance Resampling (SIR) algorithms are based on Importance Sampling (IS) and on some resampling-based)rejuvenation algorithm which aims at fighting against weight degeneracy. However %whichever the resampling technique used this mechanism tends to be insufficient when applied to informative or high-dimensional models. In this paper we revisit the rejuvenation mechanism and propose a class of parameterized SIR-based solutions which enable to adjust the tradeoff between computational cost and statistical performances. △ Less

Submitted 15 October, 2017; originally announced October 2017.

Independent Resampling Sequential Monte Carlo Algorithms

Authors: Roland Lamberti, Yohan Petetin, François Desbouvries, François Septier

Abstract: Sequential Monte Carlo algorithms, or Particle Filters, are Bayesian filtering algorithms which propagate in time a discrete and random approximation of the a posteriori distribution of interest. Such algorithms are based on Importance Sampling with a bootstrap resampling step which aims at struggling against weights degeneracy. However, in some situations (informative measurements, high dimension… ▽ More Sequential Monte Carlo algorithms, or Particle Filters, are Bayesian filtering algorithms which propagate in time a discrete and random approximation of the a posteriori distribution of interest. Such algorithms are based on Importance Sampling with a bootstrap resampling step which aims at struggling against weights degeneracy. However, in some situations (informative measurements, high dimensional model), the resampling step can prove inefficient. In this paper, we revisit the fundamental resampling mechanism which leads us back to Rubin's static resampling mechanism. We propose an alternative rejuvenation scheme in which the resampled particles share the same marginal distribution as in the classical setup, but are now independent. This set of independent particles provides a new alternative to compute a moment of the target distribution and the resulting estimate is analyzed through a CLT. We next adapt our results to the dynamic case and propose a particle filtering algorithm based on independent resampling. This algorithm can be seen as a particular auxiliary particle filter algorithm with a relevant choice of the first-stage weights and instrumental distributions. Finally we validate our results via simulations which carefully take into account the computational budget. △ Less

Submitted 19 July, 2016; originally announced July 2016.

Sequential Markov Chain Monte Carlo for Bayesian Filtering with Massive Data

Authors: Allan De Freitas, François Septier, Lyudmila Mihaylova

Abstract: Advances in digital sensors, digital data storage and communications have resulted in systems being capable of accumulating large collections of data. In the light of dealing with the challenges that massive data present, this work proposes solutions to inference and filtering problems within the Bayesian framework. Two novel Bayesian inference algorithms are developed for non-linear and non-Gauss… ▽ More Advances in digital sensors, digital data storage and communications have resulted in systems being capable of accumulating large collections of data. In the light of dealing with the challenges that massive data present, this work proposes solutions to inference and filtering problems within the Bayesian framework. Two novel Bayesian inference algorithms are developed for non-linear and non-Gaussian state space models, able to deal with large volumes of data (or observations). These are sequential Markov chain Monte Carlo (MCMC) approaches relying on two key ideas: 1) subsample the massive data and utilise a smaller subset for filtering and inference, and 2) a divide and conquer type approach computing local filtering distributions each using a subset of the measurements. Simulation results highlight the accuracy and the large computational savings, that can reach 90% by the proposed algorithms when compared with standard techniques. △ Less

Submitted 8 December, 2015; originally announced December 2015.

A Bayesian Compressed Sensing Kalman Filter for Direction of Arrival Estimation

Authors: Matthew Hawes, Lyudmila Mihaylova, Francois Septier, Simon Godsill

Abstract: In this paper, we look to address the problem of estimating the dynamic direction of arrival (DOA) of a narrowband signal im**ing on a sensor array from the far field. The initial estimate is made using a Bayesian compressive sensing (BCS) framework and then tracked using a Bayesian compressed sensing Kalman filter (BCSKF). The BCS framework splits the angular region into N potential DOAs and en… ▽ More In this paper, we look to address the problem of estimating the dynamic direction of arrival (DOA) of a narrowband signal im**ing on a sensor array from the far field. The initial estimate is made using a Bayesian compressive sensing (BCS) framework and then tracked using a Bayesian compressed sensing Kalman filter (BCSKF). The BCS framework splits the angular region into N potential DOAs and enforces a belief that only a few of the DOAs will have a non-zero valued signal present. A BCSKF can then be used to track the change in the DOA using the same framework. There can be an issue when the DOA approaches the endfire of the array. In this angular region current methods can struggle to accurately estimate and track changes in the DOAs. To tackle this problem, we propose changing the traditional sparse belief associated with BCS to a belief that the estimated signals will match the predicted signals given a known DOA change. This is done by modelling the difference between the expected sparse received signals and the estimated sparse received signals as a Gaussian distribution. Example test scenarios are provided and comparisons made with the traditional BCS based estimation method. They show that an improvement in estimation accuracy is possible without a significant increase in computational complexity. △ Less

Submitted 21 September, 2015; originally announced September 2015.

Comments: Fusion 2015 paper

How Can Subsampling Reduce Complexity in Sequential MCMC Methods and Deal with Big Data in Target Tracking?

Authors: Allan De Freitas, François Septier, Lyudmila Mihaylova, Simon Godsill

Abstract: Target tracking faces the challenge in co** with large volumes of data which requires efficient methods for real time applications. The complexity considered in this paper is when there is a large number of measurements which are required to be processed at each time step. Sequential Markov chain Monte Carlo (MCMC) has been shown to be a promising approach to target tracking in complex environme… ▽ More Target tracking faces the challenge in co** with large volumes of data which requires efficient methods for real time applications. The complexity considered in this paper is when there is a large number of measurements which are required to be processed at each time step. Sequential Markov chain Monte Carlo (MCMC) has been shown to be a promising approach to target tracking in complex environments, especially when dealing with clutter. However, a large number of measurements usually results in large processing requirements. This paper goes beyond the current state-of-the-art and presents a novel Sequential MCMC approach that can overcome this challenge through adaptively subsampling the set of measurements. Instead of using the whole large volume of available data, the proposed algorithm performs a trade off between the number of measurements to be used and the desired accuracy of the estimates to be obtained in the presence of clutter. We show results with large improvements in processing time, more than 40% with a negligible loss in tracking performance, compared with the solution without subsampling. △ Less

Submitted 30 July, 2015; originally announced July 2015.

Comments: International Conference on Information Fusion, 2015

New Perspectives on Multiple Source Localization in Wireless Sensor Networks

Authors: Thi Le Thu Nguyen, Francois Septier, Harizo Rajaona, Gareth W. Peters, Ido Nevat, Yves Delignon

Abstract: In this paper we address the challenging problem of multiple source localization in Wireless Sensor Networks (WSN). We develop an efficient statistical algorithm, based on the novel application of Sequential Monte Carlo (SMC) sampler methodology, that is able to deal with an unknown number of sources given quantized data obtained at the fusion center from different sensors with imperfect wireless… ▽ More In this paper we address the challenging problem of multiple source localization in Wireless Sensor Networks (WSN). We develop an efficient statistical algorithm, based on the novel application of Sequential Monte Carlo (SMC) sampler methodology, that is able to deal with an unknown number of sources given quantized data obtained at the fusion center from different sensors with imperfect wireless channels. We also derive the Posterior Cramér-Rao Bound (PCRB) of the source location estimate. The PCRB is used to analyze the accuracy of the proposed SMC sampler algorithm and the impact that quantization has on the accuracy of location estimates of the sources. Extensive experiments show that the benefits of the proposed scheme in terms of the accuracy of the estimation method that are required for model selection (i.e., the number of sources) and the estimation of the source characteristics compared to the classical importance sampling method. △ Less

Submitted 22 April, 2015; originally announced April 2015.

SMC-ABC methods for the estimation of stochastic simulation models of the limit order book

Authors: Gareth W. Peters, Efstathios Panayi, Francois Septier

Abstract: In this paper we consider classes of models that have been recently developed for quantitative finance that involve modelling a highly complex multivariate, multi-attribute stochastic process known as the Limit Order Book (LOB). The LOB is the primary data structure recorded each day intra-daily for all assets on every electronic exchange in the world in which trading takes place. As such, it repr… ▽ More In this paper we consider classes of models that have been recently developed for quantitative finance that involve modelling a highly complex multivariate, multi-attribute stochastic process known as the Limit Order Book (LOB). The LOB is the primary data structure recorded each day intra-daily for all assets on every electronic exchange in the world in which trading takes place. As such, it represents one of the most important fundamental structures to study from a stochastic process perspective if one wishes to characterize features of stochastic dynamics for price, volume, liquidity and other important attributes for a traded asset. In this paper we aim to adopt the model structure which develops a stochastic model framework for the LOB of a given asset and to explain how to perform calibration of this stochastic model to real observed LOB data for a range of different assets. △ Less

Submitted 22 April, 2015; originally announced April 2015.

Efficient Sequential Monte-Carlo Samplers for Bayesian Inference

Authors: Thi Le Thu Nguyen, Francois Septier, Gareth W. Peters, Yves Delignon

Abstract: In many problems, complex non-Gaussian and/or nonlinear models are required to accurately describe a physical system of interest. In such cases, Monte Carlo algorithms are remarkably flexible and extremely powerful approaches to solve such inference problems. However, in the presence of a high-dimensional and/or multimodal posterior distribution, it is widely documented that standard Monte-Carlo t… ▽ More In many problems, complex non-Gaussian and/or nonlinear models are required to accurately describe a physical system of interest. In such cases, Monte Carlo algorithms are remarkably flexible and extremely powerful approaches to solve such inference problems. However, in the presence of a high-dimensional and/or multimodal posterior distribution, it is widely documented that standard Monte-Carlo techniques could lead to poor performance. In this paper, the study is focused on a Sequential Monte-Carlo (SMC) sampler framework, a more robust and efficient Monte Carlo algorithm. Although this approach presents many advantages over traditional Monte-Carlo methods, the potential of this emergent technique is however largely underexploited in signal processing. In this work, we aim at proposing some novel strategies that will improve the efficiency and facilitate practical implementation of the SMC sampler specifically for signal processing applications. Firstly, we propose an automatic and adaptive strategy that selects the sequence of distributions within the SMC sampler that minimizes the asymptotic variance of the estimator of the posterior normalization constant. This is critical for performing model selection in modelling applications in Bayesian signal processing. The second original contribution we present improves the global efficiency of the SMC sampler by introducing a novel correction mechanism that allows the use of the particles generated through all the iterations of the algorithm (instead of only particles from the last iteration). This is a significant contribution as it removes the need to discard a large portion of the samples obtained, as is standard in standard SMC methods. This will improve estimation performance in practical settings where computational budget is important to consider. △ Less

Submitted 22 April, 2015; originally announced April 2015.

Comments: arXiv admin note: text overlap with arXiv:1303.3123 by other authors

Langevin and Hamiltonian based Sequential MCMC for Efficient Bayesian Filtering in High-dimensional Spaces

Authors: Francois Septier, Gareth W. Peters

Abstract: Nonlinear non-Gaussian state-space models arise in numerous applications in statistics and signal processing. In this context, one of the most successful and popular approximation techniques is the Sequential Monte Carlo (SMC) algorithm, also known as particle filtering. Nevertheless, this method tends to be inefficient when applied to high dimensional problems. In this paper, we focus on another… ▽ More Nonlinear non-Gaussian state-space models arise in numerous applications in statistics and signal processing. In this context, one of the most successful and popular approximation techniques is the Sequential Monte Carlo (SMC) algorithm, also known as particle filtering. Nevertheless, this method tends to be inefficient when applied to high dimensional problems. In this paper, we focus on another class of sequential inference methods, namely the Sequential Markov Chain Monte Carlo (SMCMC) techniques, which represent a promising alternative to SMC methods. After providing a unifying framework for the class of SMCMC approaches, we propose novel efficient strategies based on the principle of Langevin diffusion and Hamiltonian dynamics in order to cope with the increasing number of high-dimensional applications. Simulation results show that the proposed algorithms achieve significantly better performance compared to existing algorithms. △ Less

Submitted 29 October, 2015; v1 submitted 22 April, 2015; originally announced April 2015.

Generalized Interference Models in Doubly Stochastic Poisson Random Fields for Wideband Communications: the PNSC(alpha) model

Authors: Gareth W. Peters, Ido Nevat, Francois Septier, Laurent Clavier

Abstract: A general stochastic model is developed for the total interference in wideband systems, denoted as the PNSC(alpha) Interference Model. It allows one to obtain, analytic representations in situations where (a) interferers are distributed according to either a homogeneous or an inhomogeneous in time or space Cox point process and (b) when the frequency bands occupied by each of the unknown number of… ▽ More A general stochastic model is developed for the total interference in wideband systems, denoted as the PNSC(alpha) Interference Model. It allows one to obtain, analytic representations in situations where (a) interferers are distributed according to either a homogeneous or an inhomogeneous in time or space Cox point process and (b) when the frequency bands occupied by each of the unknown number of interferers is also a random variable in the allowable bandwidth. The analytic representations obtained are generalizations of Cox processes to the family of sub-exponential models characterized by distributions from the alpha-stable family. We develop general parametric density representations for the interference models via doubly stochastic Poisson mixture representations of Scaled Mixture of Normal's via the Normal-Stable variance mixture. To illustrate members of this class of interference model we also develop two special cases for a moderately impulsive interference (alpha=3/2) and a highly impulsive interference (alpha=2/3) where closed form representations can be obtained either by the SMiN representation or via function expansions based on the Holtsmark distribution or Whittaker functions. To illustrate the paper we propose expressions for the Capacity of a BPSK system under a PNSC(alpha) interference, via analytic expressions for the Likelihood Ratio Test statistic. △ Less

Submitted 6 July, 2012; originally announced July 2012.

Comments: 40 pages, 7 figures