-
Steered Response Power for Sound Source Localization: A Tutorial Review
Authors:
Eric Grinstein,
Elisa Tengan,
Bilgesu Çakmak,
Thomas Dietzen,
Leonardo Nunes,
Toon van Waterschoot,
Mike Brookes,
Patrick A. Naylor
Abstract:
In the last three decades, the Steered Response Power (SRP) method has been widely used for the task of Sound Source Localization (SSL), due to its satisfactory localization performance on moderately reverberant and noisy scenarios. Many works have analyzed and extended the original SRP method to reduce its computational cost, to allow it to locate multiple sources, or to improve its performance i…
▽ More
In the last three decades, the Steered Response Power (SRP) method has been widely used for the task of Sound Source Localization (SSL), due to its satisfactory localization performance on moderately reverberant and noisy scenarios. Many works have analyzed and extended the original SRP method to reduce its computational cost, to allow it to locate multiple sources, or to improve its performance in adverse environments. In this work, we review over 200 papers on the SRP method and its variants, with emphasis on the SRP-PHAT method. We also present eXtensible-SRP, or X-SRP, a generalized and modularized version of the SRP algorithm which allows the reviewed extensions to be implemented. We provide a Python implementation of the algorithm which includes selected extensions from the literature.
△ Less
Submitted 9 May, 2024; v1 submitted 5 May, 2024;
originally announced May 2024.
-
A Convergence Analysis of Approximate Message Passing with Non-Separable Functions and Applications to Multi-Class Classification
Authors:
Burak Çakmak,
Yue M. Lu,
Manfred Opper
Abstract:
Motivated by the recent application of approximate message passing (AMP) to the analysis of convex optimizations in multi-class classifications [Loureiro, et. al., 2021], we present a convergence analysis of AMP dynamics with non-separable multivariate nonlinearities. As an application, we present a complete (and independent) analysis of the motivated convex optimization problem.
Motivated by the recent application of approximate message passing (AMP) to the analysis of convex optimizations in multi-class classifications [Loureiro, et. al., 2021], we present a convergence analysis of AMP dynamics with non-separable multivariate nonlinearities. As an application, we present a complete (and independent) analysis of the motivated convex optimization problem.
△ Less
Submitted 13 February, 2024;
originally announced February 2024.
-
Head Orientation Estimation with Distributed Microphones Using Speech Radiation Patterns
Authors:
Kaspar Müller,
Bilgesu Çakmak,
Paul Didier,
Simon Doclo,
Jan Østergaard,
Tobias Wolff
Abstract:
Determining the head orientation of a talker is not only beneficial for various speech signal processing applications, such as source localization or speech enhancement, but also facilitates intuitive voice control and interaction with smart environments or modern car assistants. Most approaches for head orientation estimation are based on visual cues. However, this requires camera systems which o…
▽ More
Determining the head orientation of a talker is not only beneficial for various speech signal processing applications, such as source localization or speech enhancement, but also facilitates intuitive voice control and interaction with smart environments or modern car assistants. Most approaches for head orientation estimation are based on visual cues. However, this requires camera systems which often are not available. We present an approach which purely uses audio signals captured with only a few distributed microphones around the talker. Specifically, we propose a novel method that directly incorporates measured or modeled speech radiation patterns to infer the talker's orientation during active speech periods based on a cosine similarity measure. Moreover, an automatic gain adjustment technique is proposed for uncalibrated, irregular microphone setups, such as ad-hoc sensor networks. In experiments with signals recorded in both anechoic and reverberant environments, the proposed method outperforms state-of-the-art approaches, using either measured or modeled speech radiation patterns.
△ Less
Submitted 4 December, 2023;
originally announced December 2023.
-
Joint Message Detection and Channel Estimation for Unsourced Random Access in Cell-Free User-Centric Wireless Networks
Authors:
Burak Çakmak,
Eleni Gkiouzepi,
Manfred Opper,
Giuseppe Caire
Abstract:
We consider unsourced random access (uRA) in a cell-free (CF) user-centric wireless network, where a large number of potential users compete for a random access slot, while only a finite subset is active. The random access users transmit codewords of length $L$ symbols from a shared codebook, which are received by $B$ geographically distributed radio units (RUs) equipped with $M$ antennas each. Ou…
▽ More
We consider unsourced random access (uRA) in a cell-free (CF) user-centric wireless network, where a large number of potential users compete for a random access slot, while only a finite subset is active. The random access users transmit codewords of length $L$ symbols from a shared codebook, which are received by $B$ geographically distributed radio units (RUs) equipped with $M$ antennas each. Our goal is to devise and analyze a \emph{centralized} decoder to detect the transmitted messages (without prior knowledge of the active users) and estimate the corresponding channel state information. A specific challenge lies in the fact that, due to the geographically distributed nature of the CF network, there is no fixed correspondence between codewords and large-scale fading coefficients (LSFCs). To overcome this problem, we propose a scheme where the access codebook is partitioned in "location-based" subcodes, such that users in a particular location make use of the corresponding subcode. The joint message detection and channel estimation is obtained via a novel {\em Approximated Message Passing} (AMP) algorithm to estimate the linear superposition of matrix-valued "sources" corrupted by Gaussian noise. The matrices to be estimated exhibit zero rows for inactive messages and Gaussian-distributed rows corresponding to the active messages. The asymmetry in the LSFCs and message activity probabilities leads to \emph{different statistics} for the matrix sources, which distinguishes the AMP formulation from previous cases. In the regime where the codebook size scales linearly with $L$, while $B$ and $M$ are fixed, we present a rigorous high-dimensional analysis of the proposed AMP algorithm. Then, exploiting the fundamental decoupling principle of AMP, we provide a comprehensive analysis of Neyman-Pearson message detection, along with the subsequent channel estimation.
△ Less
Submitted 5 February, 2024; v1 submitted 24 April, 2023;
originally announced April 2023.
-
Analysis of Random Sequential Message Passing Algorithms for Approximate Inference
Authors:
Burak Çakmak,
Yue M. Lu,
Manfred Opper
Abstract:
We analyze the dynamics of a random sequential message passing algorithm for approximate inference with large Gaussian latent variable models in a student-teacher scenario. To model nontrivial dependencies between the latent variables, we assume random covariance matrices drawn from rotation invariant ensembles. Moreover, we consider a model mismatching setting, where the teacher model and the one…
▽ More
We analyze the dynamics of a random sequential message passing algorithm for approximate inference with large Gaussian latent variable models in a student-teacher scenario. To model nontrivial dependencies between the latent variables, we assume random covariance matrices drawn from rotation invariant ensembles. Moreover, we consider a model mismatching setting, where the teacher model and the one used by the student may be different. By means of dynamical functional approach, we obtain exact dynamical mean-field equations characterizing the dynamics of the inference algorithm. We also derive a range of model parameters for which the sequential algorithm does not converge. The boundary of this parameter range coincides with the de Almeida Thouless (AT) stability condition of the replica symmetric ansatz for the static probabilistic model.
△ Less
Submitted 16 February, 2022;
originally announced February 2022.
-
Exact solution to the random sequential dynamics of a message passing algorithm
Authors:
Burak Çakmak,
Manfred Opper
Abstract:
We analyze the random sequential dynamics of a message passing algorithm for Ising models with random interactions in the large system limit. We derive exact results for the two-time correlation functions and the speed of convergence. The {\em de Almedia-Thouless} stability criterion of the static problem is found to be necessary and sufficient for the global convergence of the random sequential d…
▽ More
We analyze the random sequential dynamics of a message passing algorithm for Ising models with random interactions in the large system limit. We derive exact results for the two-time correlation functions and the speed of convergence. The {\em de Almedia-Thouless} stability criterion of the static problem is found to be necessary and sufficient for the global convergence of the random sequential dynamics.
△ Less
Submitted 2 March, 2021; v1 submitted 5 January, 2021;
originally announced January 2021.
-
A Dynamical Mean-Field Theory for Learning in Restricted Boltzmann Machines
Authors:
Burak Çakmak,
Manfred Opper
Abstract:
We define a message-passing algorithm for computing magnetizations in Restricted Boltzmann machines, which are Ising models on bipartite graphs introduced as neural network models for probability distributions over spin configurations. To model nontrivial statistical dependencies between the spins' couplings, we assume that the rectangular coupling matrix is drawn from an arbitrary bi-rotation inv…
▽ More
We define a message-passing algorithm for computing magnetizations in Restricted Boltzmann machines, which are Ising models on bipartite graphs introduced as neural network models for probability distributions over spin configurations. To model nontrivial statistical dependencies between the spins' couplings, we assume that the rectangular coupling matrix is drawn from an arbitrary bi-rotation invariant random matrix ensemble. Using the dynamical functional method of statistical mechanics we exactly analyze the dynamics of the algorithm in the large system limit. We prove the global convergence of the algorithm under a stability criterion and compute asymptotic convergence rates showing excellent agreement with numerical simulations.
△ Less
Submitted 4 May, 2020;
originally announced May 2020.
-
Understanding the dynamics of message passing algorithms: a free probability heuristics
Authors:
Manfred Opper,
Burak Çakmak
Abstract:
We use freeness assumptions of random matrix theory to analyze the dynamical behavior of inference algorithms for probabilistic models with dense coupling matrices in the limit of large systems. For a toy Ising model, we are able to recover previous results such as the property of vanishing effective memories and the analytical convergence rate of the algorithm.
We use freeness assumptions of random matrix theory to analyze the dynamical behavior of inference algorithms for probabilistic models with dense coupling matrices in the limit of large systems. For a toy Ising model, we are able to recover previous results such as the property of vanishing effective memories and the analytical convergence rate of the algorithm.
△ Less
Submitted 3 February, 2020;
originally announced February 2020.
-
Analysis of Bayesian Inference Algorithms by the Dynamical Functional Approach
Authors:
Burak Çakmak,
Manfred Opper
Abstract:
We analyze the dynamics of an algorithm for approximate inference with large Gaussian latent variable models in a student-teacher scenario. To model nontrivial dependencies between the latent variables, we assume random covariance matrices drawn from rotation invariant ensembles. For the case of perfect data-model matching, the knowledge of static order parameters derived from the replica method a…
▽ More
We analyze the dynamics of an algorithm for approximate inference with large Gaussian latent variable models in a student-teacher scenario. To model nontrivial dependencies between the latent variables, we assume random covariance matrices drawn from rotation invariant ensembles. For the case of perfect data-model matching, the knowledge of static order parameters derived from the replica method allows us to obtain efficient algorithmic updates in terms of matrix-vector multiplications with a fixed matrix. Using the dynamical functional approach, we obtain an exact effective stochastic process in the thermodynamic limit for a single node. From this, we obtain closed-form expressions for the rate of the convergence. Analytical results are excellent agreement with simulations of single instances of large models.
△ Less
Submitted 14 January, 2020;
originally announced January 2020.
-
Memory-free dynamics for the TAP equations of Ising models with arbitrary rotation invariant ensembles of random coupling matrices
Authors:
Burak Çakmak,
Manfred Opper
Abstract:
We propose an iterative algorithm for solving the Thouless-Anderson-Palmer (TAP) equations of Ising models with arbitrary rotation invariant (random) coupling matrices. In the thermodynamic limit, we prove by means of the dynamical functional method that the proposed algorithm converges when the so-called de Almeida Thouless (AT) criterion is fulfilled. Moreover, we give exact analytical expressio…
▽ More
We propose an iterative algorithm for solving the Thouless-Anderson-Palmer (TAP) equations of Ising models with arbitrary rotation invariant (random) coupling matrices. In the thermodynamic limit, we prove by means of the dynamical functional method that the proposed algorithm converges when the so-called de Almeida Thouless (AT) criterion is fulfilled. Moreover, we give exact analytical expressions for the rate of the convergence.
△ Less
Submitted 7 March, 2019; v1 submitted 24 January, 2019;
originally announced January 2019.
-
Expectation Propagation for Approximate Inference: Free Probability Framework
Authors:
Burak Çakmak,
Manfred Opper
Abstract:
We study asymptotic properties of expectation propagation (EP) -- a method for approximate inference originally developed in the field of machine learning. Applied to generalized linear models, EP iteratively computes a multivariate Gaussian approximation to the exact posterior distribution. The computational complexity of the repeated update of covariance matrices severely limits the application…
▽ More
We study asymptotic properties of expectation propagation (EP) -- a method for approximate inference originally developed in the field of machine learning. Applied to generalized linear models, EP iteratively computes a multivariate Gaussian approximation to the exact posterior distribution. The computational complexity of the repeated update of covariance matrices severely limits the application of EP to large problem sizes. In this study, we present a rigorous analysis by means of free probability theory that allows us to overcome this computational bottleneck if specific data matrices in the problem fulfill certain properties of asymptotic freeness. We demonstrate the relevance of our approach on the gene selection problem of a microarray dataset.
△ Less
Submitted 9 May, 2018; v1 submitted 16 January, 2018;
originally announced January 2018.
-
Dynamical Functional Theory for Compressed Sensing
Authors:
Burak Çakmak,
Manfred Opper,
Ole Winther,
Bernard H. Fleury
Abstract:
We introduce a theoretical approach for designing generalizations of the approximate message passing (AMP) algorithm for compressed sensing which are valid for large observation matrices that are drawn from an invariant random matrix ensemble. By design, the fixed points of the algorithm obey the Thouless-Anderson-Palmer (TAP) equations corresponding to the ensemble. Using a dynamical functional a…
▽ More
We introduce a theoretical approach for designing generalizations of the approximate message passing (AMP) algorithm for compressed sensing which are valid for large observation matrices that are drawn from an invariant random matrix ensemble. By design, the fixed points of the algorithm obey the Thouless-Anderson-Palmer (TAP) equations corresponding to the ensemble. Using a dynamical functional approach we are able to derive an effective stochastic process for the marginal statistics of a single component of the dynamics. This allows us to design memory terms in the algorithm in such a way that the resulting fields become Gaussian random variables allowing for an explicit analysis. The asymptotic statistics of these fields are consistent with the replica ansatz of the compressed sensing problem.
△ Less
Submitted 11 May, 2017;
originally announced May 2017.
-
Self-Averaging Expectation Propagation
Authors:
Burak Çakmak,
Manfred Opper,
Bernard H. Fleury,
Ole Winther
Abstract:
We investigate the problem of approximate Bayesian inference for a general class of observation models by means of the expectation propagation (EP) framework for large systems under some statistical assumptions. Our approach tries to overcome the numerical bottleneck of EP caused by the inversion of large matrices. Assuming that the measurement matrices are realizations of specific types of ensemb…
▽ More
We investigate the problem of approximate Bayesian inference for a general class of observation models by means of the expectation propagation (EP) framework for large systems under some statistical assumptions. Our approach tries to overcome the numerical bottleneck of EP caused by the inversion of large matrices. Assuming that the measurement matrices are realizations of specific types of ensembles we use the concept of freeness from random matrix theory to show that the EP cavity variances exhibit an asymptotic self-averaging property. They can be pre-computed using specific generating functions, i.e. the R- and/or S-transforms in free probability, which do not require matrix inversions. Our approach extends the framework of (generalized) approximate message passing -- assumes zero-mean iid entries of the measurement matrix -- to a general class of random matrix ensembles. The generalization is via a simple formulation of the R- and/or S-transforms of the limiting eigenvalue distribution of the Gramian of the measurement matrix. We demonstrate the performance of our approach on a signal recovery problem of nonlinear compressed sensing and compare it with that of EP.
△ Less
Submitted 23 August, 2016;
originally announced August 2016.
-
Cooperative Localization for Mobile Networks: A Distributed Belief Propagation - Mean Field Message Passing Algorithm
Authors:
Burak Çakmak,
Daniel N. Urup,
Florian Meyer,
Troels Pedersen,
Bernard H. Fleury,
Franz Hlawatsch
Abstract:
We propose a hybrid message passing method for distributed cooperative localization and tracking of mobile agents. Belief propagation and mean field message passing are employed for, respectively, the motion-related and measurement-related part of the factor graph. Using a Gaussian belief approximation, only three real values per message passing iteration have to be broadcast to neighboring agents…
▽ More
We propose a hybrid message passing method for distributed cooperative localization and tracking of mobile agents. Belief propagation and mean field message passing are employed for, respectively, the motion-related and measurement-related part of the factor graph. Using a Gaussian belief approximation, only three real values per message passing iteration have to be broadcast to neighboring agents. Despite these very low communication requirements, the estimation accuracy can be comparable to that of particle-based belief propagation.
△ Less
Submitted 3 April, 2016; v1 submitted 24 December, 2015;
originally announced December 2015.
-
A Theory of Solving TAP Equations for Ising Models with General Invariant Random Matrices
Authors:
Manfred Opper,
Burak Çakmak,
Ole Winther
Abstract:
We consider the problem of solving TAP mean field equations by iteration for Ising model with coupling matrices that are drawn at random from general invariant ensembles. We develop an analysis of iterative algorithms using a dynamical functional approach that in the thermodynamic limit yields an effective dynamics of a single variable trajectory. Our main novel contribution is the expression for…
▽ More
We consider the problem of solving TAP mean field equations by iteration for Ising model with coupling matrices that are drawn at random from general invariant ensembles. We develop an analysis of iterative algorithms using a dynamical functional approach that in the thermodynamic limit yields an effective dynamics of a single variable trajectory. Our main novel contribution is the expression for the implicit memory term of the dynamics for general invariant ensembles. By subtracting these terms, that depend on magnetizations at previous time steps, the implicit memory terms cancel making the iteration dependent on a Gaussian distributed field only. The TAP magnetizations are stable fixed points if an AT stability criterion is fulfilled. We illustrate our method explicitly for coupling matrices drawn from the random orthogonal ensemble.
△ Less
Submitted 28 March, 2016; v1 submitted 3 September, 2015;
originally announced September 2015.
-
S-AMP for Non-linear Observation Models
Authors:
Burak Çakmak,
Ole Winther,
Bernard H. Fleury
Abstract:
Recently we extended Approximate message passing (AMP) algorithm to be able to handle general invariant matrix ensembles. In this contribution we extend our S-AMP approach to non-linear observation models. We obtain generalized AMP (GAMP) algorithm as the special case when the measurement matrix has zero-mean iid Gaussian entries. Our derivation is based upon 1) deriving expectation propagation (E…
▽ More
Recently we extended Approximate message passing (AMP) algorithm to be able to handle general invariant matrix ensembles. In this contribution we extend our S-AMP approach to non-linear observation models. We obtain generalized AMP (GAMP) algorithm as the special case when the measurement matrix has zero-mean iid Gaussian entries. Our derivation is based upon 1) deriving expectation propagation (EP) like algorithms from the stationary-points equations of the Gibbs free energy under first- and second-moment constraints and 2) applying additive free convolution in free probability theory to get low-complexity updates for the second moment quantities.
△ Less
Submitted 25 January, 2015;
originally announced January 2015.
-
S-AMP: Approximate Message Passing for General Matrix Ensembles
Authors:
Burak Çakmak,
Ole Winther,
Bernard H. Fleury
Abstract:
In this work we propose a novel iterative estimation algorithm for linear observation systems called S-AMP whose fixed points are the stationary points of the exact Gibbs free energy under a set of (first- and second-) moment consistency constraints in the large system limit. S-AMP extends the approximate message-passing (AMP) algorithm to general matrix ensembles. The generalization is based on t…
▽ More
In this work we propose a novel iterative estimation algorithm for linear observation systems called S-AMP whose fixed points are the stationary points of the exact Gibbs free energy under a set of (first- and second-) moment consistency constraints in the large system limit. S-AMP extends the approximate message-passing (AMP) algorithm to general matrix ensembles. The generalization is based on the S-transform (in free probability) of the spectrum of the measurement matrix. Furthermore, we show that the optimality of S-AMP follows directly from its design rather than from solving a separate optimization problem as done for AMP.
△ Less
Submitted 12 May, 2014;
originally announced May 2014.
-
Capacity Scaling in MIMO Systems with General Unitarily Invariant Random Matrices
Authors:
Burak Çakmak,
Ralf R. Müller,
Bernard H. Fleury
Abstract:
We investigate the capacity scaling of MIMO systems with the system dimensions. To that end, we quantify how the mutual information varies when the number of antennas (at either the receiver or transmitter side) is altered. For a system comprising $R$ receive and $T$ transmit antennas with $R>T$, we find the following: By removing as many receive antennas as needed to obtain a square system (provi…
▽ More
We investigate the capacity scaling of MIMO systems with the system dimensions. To that end, we quantify how the mutual information varies when the number of antennas (at either the receiver or transmitter side) is altered. For a system comprising $R$ receive and $T$ transmit antennas with $R>T$, we find the following: By removing as many receive antennas as needed to obtain a square system (provided the channel matrices before and after the removal have full rank) the maximum resulting loss of mutual information over all signal-to-noise ratios (SNRs) depends only on $R$, $T$ and the matrix of left-singular vectors of the initial channel matrix, but not on its singular values. In particular, if the latter matrix is Haar distributed the ergodic rate loss is given by $\sum_{t=1}^{T}\sum_{r=T+1}^{R}\frac{1}{r-t}$ nats. Under the same assumption, if $T,R\to \infty$ with the ratio $φ\triangleq T/R$ fixed, the rate loss normalized by $R$ converges almost surely to $H(φ)$ bits with $H(\cdot)$ denoting the binary entropy function. We also quantify and study how the mutual information as a function of the system dimensions deviates from the traditionally assumed linear growth in the minimum of the system dimensions at high SNR.
△ Less
Submitted 5 March, 2018; v1 submitted 11 June, 2013;
originally announced June 2013.
