-
On Optimal Quantization in Sequential Detection
Authors:
Michael Fauß,
Manuel S. Stein,
H. Vincent Poor
Abstract:
The problem of designing optimal quantization rules for sequential detectors is investigated. First, it is shown that this task can be solved within the general framework of active sequential detection. Using this approach, the optimal sequential detector and the corresponding quantizer are characterized and their properties are briefly discussed. In particular, it is shown that designing optimal…
▽ More
The problem of designing optimal quantization rules for sequential detectors is investigated. First, it is shown that this task can be solved within the general framework of active sequential detection. Using this approach, the optimal sequential detector and the corresponding quantizer are characterized and their properties are briefly discussed. In particular, it is shown that designing optimal quantization rules requires solving a nonconvex optimization problem, which can lead to issues in terms of computational complexity and numerical stability. Motivated by these difficulties, two performance bounds are proposed that are easier to evaluate than the true performance measures and are potentially tighter than the bounds currently available in the literature. The usefulness of the bounds and the properties of the optimal quantization rules are illustrated with two numerical examples.
△ Less
Submitted 28 July, 2021;
originally announced July 2021.
-
Minimax Robust Detection: Classic Results and Recent Advances
Authors:
Michael Fauß,
Abdelhak M. Zoubir,
H. Vincent Poor
Abstract:
This paper provides an overview of results and concepts in minimax robust hypothesis testing for two and multiple hypotheses. It starts with an introduction to the subject, highlighting its connection to other areas of robust statistics and giving a brief recount of the most prominent developments. Subsequently, the minimax principle is introduced and its strengths and limitations are discussed. T…
▽ More
This paper provides an overview of results and concepts in minimax robust hypothesis testing for two and multiple hypotheses. It starts with an introduction to the subject, highlighting its connection to other areas of robust statistics and giving a brief recount of the most prominent developments. Subsequently, the minimax principle is introduced and its strengths and limitations are discussed. The first part of the paper focuses on the two-hypothesis case. After briefly reviewing the basics of statistical hypothesis testing, uncertainty sets are introduced as a generic way of modeling distributional uncertainty. The design of minimax detectors is then shown to reduce to the problem of determining a pair of least favorable distributions, and different criteria for their characterization are discussed. Explicit expressions are given for least favorable distributions under three types of uncertainty: $\varepsilon$-contamination, probability density bands, and $f$-divergence balls. Using examples, it is shown how the properties of these least favorable distributions translate to properties of the corresponding minimax detectors. The second part of the paper deals with the problem of robustly testing multiple hypotheses, starting with a discussion of why this is fundamentally different from the binary problem. Sequential detection is then introduced as a technique that enables the design of strictly minimax optimal tests in the multi-hypothesis case. Finally, the usefulness of robust detectors in practice is showcased using the example of ground penetrating radar. The paper concludes with an outlook on robust detection beyond the minimax principle and a brief summary of the presented material.
△ Less
Submitted 20 May, 2021;
originally announced May 2021.
-
Asymptotically Optimal Procedures for Sequential Joint Detection and Estimation
Authors:
Dominik Reinhard,
Michael Fauß,
Abdelhak M. Zoubir
Abstract:
We investigate the problem of jointly testing multiple hypotheses and estimating a random parameter of the underlying distribution in a sequential setup. The aim is to jointly infer the true hypothesis and the true parameter while using on average as few samples as possible and kee** the detection and estimation errors below predefined levels. Based on mild assumptions on the underlying model, w…
▽ More
We investigate the problem of jointly testing multiple hypotheses and estimating a random parameter of the underlying distribution in a sequential setup. The aim is to jointly infer the true hypothesis and the true parameter while using on average as few samples as possible and kee** the detection and estimation errors below predefined levels. Based on mild assumptions on the underlying model, we propose an asymptotically optimal procedure, i.e., a procedure that becomes optimal when the tolerated detection and estimation error levels tend to zero. The implementation of the resulting asymptotically optimal stop** rule is computationally cheap and, hence, applicable for high-dimensional data. We further propose a projected quasi-Newton method to optimally choose the coefficients that parameterize the instantaneous cost function such that the constraints are fulfilled with equality. The proposed theory is validated by numerical examples.
△ Less
Submitted 31 January, 2024; v1 submitted 11 May, 2021;
originally announced May 2021.
-
MMSE Bounds Under Kullback-Leibler Divergence Constraints on the Joint Input-Output Distribution
Authors:
Michael Fauß,
Alex Dysto,
H. Vincent Poor
Abstract:
This paper proposes a new family of lower and upper bounds on the minimum mean squared error (MMSE). The key idea is to minimize/maximize the MMSE subject to the constraint that the joint distribution of the input-output statistics lies in a Kullback-Leibler divergence ball centered at some Gaussian reference distribution. Both bounds are tight and are attained by Gaussian distributions whose mean…
▽ More
This paper proposes a new family of lower and upper bounds on the minimum mean squared error (MMSE). The key idea is to minimize/maximize the MMSE subject to the constraint that the joint distribution of the input-output statistics lies in a Kullback-Leibler divergence ball centered at some Gaussian reference distribution. Both bounds are tight and are attained by Gaussian distributions whose mean is identical to that of the reference distribution and whose covariance matrix is determined by a scalar parameter that can be obtained by finding the root of a monotonic function. The upper bound corresponds to a minimax optimal estimator and provides performance guarantees under distributional uncertainty. The lower bound provides an alternative to well-known inequalities in estimation theory, such as the Cramér-Rao bound, that is potentially tighter and defined for a larger class of distributions. Examples of applications in signal processing and information theory illustrate the usefulness of the proposed bounds in practice.
△ Less
Submitted 5 June, 2020;
originally announced June 2020.
-
Nonparametric Estimation of the Fisher Information and Its Applications
Authors:
Wei Cao,
Alex Dytso,
Michael Fauß,
H. Vincent Poor,
Gang Feng
Abstract:
This paper considers the problem of estimation of the Fisher information for location from a random sample of size $n$. First, an estimator proposed by Bhattacharya is revisited and improved convergence rates are derived. Second, a new estimator, termed a clipped estimator, is proposed. Superior upper bounds on the rates of convergence can be shown for the new estimator compared to the Bhattachary…
▽ More
This paper considers the problem of estimation of the Fisher information for location from a random sample of size $n$. First, an estimator proposed by Bhattacharya is revisited and improved convergence rates are derived. Second, a new estimator, termed a clipped estimator, is proposed. Superior upper bounds on the rates of convergence can be shown for the new estimator compared to the Bhattacharya estimator, albeit with different regularity conditions. Third, both of the estimators are evaluated for the practically relevant case of a random variable contaminated by Gaussian noise. Moreover, using Brown's identity, which relates the Fisher information and the minimum mean squared error (MMSE) in Gaussian noise, two corresponding consistent estimators for the MMSE are proposed. Simulation examples for the Bhattacharya estimator and the clipped estimator as well as the MMSE estimators are presented. The examples demonstrate that the clipped estimator can significantly reduce the required sample size to guarantee a specific confidence interval compared to the Bhattacharya estimator.
△ Less
Submitted 7 May, 2020;
originally announced May 2020.
-
Bayesian Sequential Joint Detection and Estimation under Multiple Hypotheses
Authors:
Dominik Reinhard,
Michael Fauß,
Abdelhak M. Zoubir
Abstract:
We consider the problem of jointly testing multiple hypotheses and estimating a random parameter of the underlying distribution. This problem is investigated in a sequential setup under mild assumptions on the underlying random process. The optimal method minimizes the expected number of samples while ensuring that the average detection/estimation errors do not exceed a certain level. After conver…
▽ More
We consider the problem of jointly testing multiple hypotheses and estimating a random parameter of the underlying distribution. This problem is investigated in a sequential setup under mild assumptions on the underlying random process. The optimal method minimizes the expected number of samples while ensuring that the average detection/estimation errors do not exceed a certain level. After converting the constrained problem to an unconstrained one, we characterize the general solution by a non-linear Bellman equation, which is parametrized by a set of cost coefficients. A strong connection between the derivatives of the cost function with respect to the coefficients and the detection/estimation errors of the sequential procedure is derived. Based on this fundamental property, we further show that for suitably chosen cost coefficients the solutions of the constrained and the unconstrained problem coincide. We present two approaches to finding the optimal coefficients. For the first approach, the final optimization problem is converted into a linear program, whereas the second approach solves it with a projected gradient ascent. To illustrate the theoretical results, we consider two problems for which the optimal schemes are designed numerically. Using Monte Carlo simulations, it is validated that the numerical results agree with the theory.
△ Less
Submitted 6 May, 2021; v1 submitted 27 March, 2020;
originally announced March 2020.
-
The Vector Poisson Channel: On the Linearity of the Conditional Mean Estimator
Authors:
Alex Dytso,
Michael Fauss,
H. Vincent Poor
Abstract:
This work studies properties of the conditional mean estimator in vector Poisson noise. The main emphasis is to study conditions on prior distributions that induce linearity of the conditional mean estimator. The paper consists of two main results. The first result shows that the only distribution that induces the linearity of the conditional mean estimator is a product gamma distribution. Moreove…
▽ More
This work studies properties of the conditional mean estimator in vector Poisson noise. The main emphasis is to study conditions on prior distributions that induce linearity of the conditional mean estimator. The paper consists of two main results. The first result shows that the only distribution that induces the linearity of the conditional mean estimator is a product gamma distribution. Moreover, it is shown that the conditional mean estimator cannot be linear when the dark current parameter of the Poisson noise is non-zero. The second result produces a quantitative refinement of the first result. Specifically, it is shown that if the conditional mean estimator is close to linear in a mean squared error sense, then the prior distribution must be close to a product gamma distribution in terms of their characteristic functions. Finally, the results are compared to their Gaussian counterparts.
△ Less
Submitted 19 March, 2020;
originally announced March 2020.
-
Distributed Joint Detection and Estimation: A Sequential Approach
Authors:
Dominik Reinhard,
Michael Fauß,
Abdelhak M. Zoubir
Abstract:
We investigate the problem of jointly testing two hypotheses and estimating a random parameter based on data that is observed sequentially by sensors in a distributed network. In particular, we assume the data to be drawn from a Gaussian distribution, whose random mean is to be estimated. Forgoing the need for a fusion center, the processing is performed locally and the sensors interact with their…
▽ More
We investigate the problem of jointly testing two hypotheses and estimating a random parameter based on data that is observed sequentially by sensors in a distributed network. In particular, we assume the data to be drawn from a Gaussian distribution, whose random mean is to be estimated. Forgoing the need for a fusion center, the processing is performed locally and the sensors interact with their neighbors following the consensus+innovations approach. We design the test at the individual sensors such that the performance measures, namely, error probabilities and mean-squared error, do not exceed pre-defined levels while the average sample number is minimized. After converting the constrained problem to an unconstrained problem and the subsequent reduction to an optimal stop** problem, we solve the latter utilizing dynamic programming. The solution is shown to be characterized by a set of non-linear Bellman equations, parametrized by cost coefficients, which are then determined by linear programming as to fulfill the performance specifications. A numerical example validates the proposed theory.
△ Less
Submitted 3 March, 2020; v1 submitted 1 March, 2020;
originally announced March 2020.
-
A Cramér-Rao Type Bound for Bayesian Risk with Bregman Loss
Authors:
Alex Dytso,
Michael Fauß,
H. Vincent Poor
Abstract:
A general class of Bayesian lower bounds when the underlying loss function is a Bregman divergence is demonstrated. This class can be considered as an extension of the Weinstein--Weiss family of bounds for the mean squared error and relies on finding a variational characterization of Bayesian risk. The approach allows for the derivation of a version of the Cramér--Rao bound that is specific to a g…
▽ More
A general class of Bayesian lower bounds when the underlying loss function is a Bregman divergence is demonstrated. This class can be considered as an extension of the Weinstein--Weiss family of bounds for the mean squared error and relies on finding a variational characterization of Bayesian risk. The approach allows for the derivation of a version of the Cramér--Rao bound that is specific to a given Bregman divergence. The new generalization of the Cramér--Rao bound reduces to the classical one when the loss function is taken to be the Euclidean norm. The effectiveness of the new bound is evaluated in the Poisson noise setting and the Binomial noise setting.
△ Less
Submitted 15 June, 2020; v1 submitted 29 January, 2020;
originally announced January 2020.
-
Latency Analysis for Sequential Detection in Low-Complexity Binary Radio Systems
Authors:
Manuel S. Stein,
Michael Fauß
Abstract:
We consider the problem of making a quick decision in favor of one of two possible physical signal models while the numerical measurements are acquired by sensing devices featuring minimal digitization complexity. Therefore, the digital data streams available for statistical processing are binary and exhibit temporal and spatial dependencies. To handle the intractable multivariate binary data mode…
▽ More
We consider the problem of making a quick decision in favor of one of two possible physical signal models while the numerical measurements are acquired by sensing devices featuring minimal digitization complexity. Therefore, the digital data streams available for statistical processing are binary and exhibit temporal and spatial dependencies. To handle the intractable multivariate binary data model, we first consider sequential tests for exponential family distributions. Within this generic probabilistic framework, we identify adaptive approximations for the log-likelihood ratio and the Kullback-Leibler divergence. The results allow designing sequential detectors for binary radio systems and analyzing their average run-time along classical arguments of Wald. In particular, the derived tests exploit the spatio-temporal correlation structure of the analog sensor signals engraved into the binary measurements. As an application, we consider the specification of binary sensing architectures for cognitive radio and GNSS spectrum monitoring where our results characterize the sequential detection latency as a function of the temporal oversampling and the number of antennas. Finally, we evaluate the efficiency of the proposed algorithms and illustrate the accuracy of our analysis via Monte-Carlo simulations.
△ Less
Submitted 27 October, 2019; v1 submitted 21 May, 2019;
originally announced May 2019.
-
Tight Bounds on the Weighted Sum of MMSEs with Applications in Distributed Estimation
Authors:
Michael Fauß,
Abdelhak M. Zoubir,
Alex Dytso,
H. Vincent Poor,
K. G. Nagananda
Abstract:
In this paper, tight upper and lower bounds are derived on the weighted sum of minimum mean-squared errors for additive Gaussian noise channels. The bounds are obtained by constraining the input distribution to be close to a Gaussian reference distribution in terms of the Kullback--Leibler divergence. The distributions that attain these bounds are shown to be Gaussian whose covariance matrices are…
▽ More
In this paper, tight upper and lower bounds are derived on the weighted sum of minimum mean-squared errors for additive Gaussian noise channels. The bounds are obtained by constraining the input distribution to be close to a Gaussian reference distribution in terms of the Kullback--Leibler divergence. The distributions that attain these bounds are shown to be Gaussian whose covariance matrices are defined implicitly via systems of matrix equations. Furthermore, the estimators that attain the upper bound are shown to be minimax robust against deviations from the assumed input distribution. The lower bound provides a potentially tighter alternative to well-known inequalities such as the Cramér--Rao lower bound. Numerical examples are provided to verify the theoretical findings of the paper. The results derived in this paper can be used to obtain performance bounds, robustness guarantees, and engineering guidelines for the design of local estimators for distributed estimation problems which commonly arise in wireless communication systems and sensor networks.
△ Less
Submitted 22 January, 2020; v1 submitted 20 February, 2019;
originally announced February 2019.
-
Bayesian Sequential Joint Detection and Estimation
Authors:
Dominik Reinhard,
Michael Fauss,
Abdelhak M. Zoubir
Abstract:
Joint detection and estimation refers to deciding between two or more hypotheses and, depending on the test outcome, simultaneously estimating the unknown parameters of the underlying distribution. This problem is investigated in a sequential framework under mild assumptions on the underlying random process. We formulate an unconstrained sequential decision problem, whose cost function is the weig…
▽ More
Joint detection and estimation refers to deciding between two or more hypotheses and, depending on the test outcome, simultaneously estimating the unknown parameters of the underlying distribution. This problem is investigated in a sequential framework under mild assumptions on the underlying random process. We formulate an unconstrained sequential decision problem, whose cost function is the weighted sum of the expected run-length and the detection/estimation errors. Then, a strong connection between the derivatives of the cost function with respect to the weights, which can be interpreted as Lagrange multipliers, and the detection/estimation errors of the underlying scheme is shown. This property is used to characterize the solution of a closely related sequential decision problem, whose objective function is the expected run-length under constraints on the average detection/estimation errors. We show that the solution of the constrained problem coincides with the solution of the unconstrained problem with suitably chosen weights. These weights are characterized as the solution of a linear program, which can be solved using efficient off-the-shelf solvers. The theoretical results are illustrated with two example problems, for which optimal sequential schemes are designed numerically and whose performance is validated via Monte Carlo simulations.
△ Less
Submitted 18 April, 2019; v1 submitted 9 July, 2018;
originally announced July 2018.
-
In a One-Bit Rush: Low-Latency Wireless Spectrum Monitoring with Binary Sensor Arrays
Authors:
Manuel S. Stein,
Michael Fauß
Abstract:
Detecting the presence of a random wireless source with minimum latency utilizing an array of radio sensors is considered. The problem is studied under the constraint that the analog-to-digital conversion at each sensor is restricted to reading the sign of the analog received signal. We formulate the resulting digital signal processing task as a sequential hypothesis test in simple form. To circum…
▽ More
Detecting the presence of a random wireless source with minimum latency utilizing an array of radio sensors is considered. The problem is studied under the constraint that the analog-to-digital conversion at each sensor is restricted to reading the sign of the analog received signal. We formulate the resulting digital signal processing task as a sequential hypothesis test in simple form. To circumvent the intractable probabilistic model of the multivariate binary array data, a reduced model representation within the exponential family in conjunction with a log-likelihood ratio approximation is employed. This approach allows us to design a likelihood-based sequential test and to analyze its analytic performance along Wald's classical arguments. In the context of wireless spectrum monitoring for satellite-based navigation and synchronization systems, we study the achievable processing latency, characterized by the average sample number, as a function of the binary sensors in use. The practical feasibility and potential of the discussed low-complexity sensing and decision-making technology is demonstrated via simulations.
△ Less
Submitted 30 April, 2018; v1 submitted 9 February, 2018;
originally announced February 2018.
-
On the Minimization of Convex Functionals of Probability Distributions Under Band Constraints
Authors:
Michael Fauss,
Abdelhak M. Zoubir
Abstract:
The problem of minimizing convex functionals of probability distributions is solved under the assumption that the density of every distribution is bounded from above and below. A system of sufficient and necessary first-order optimality conditions as well as a bound on the optimality gap of feasible candidate solutions are derived. Based on these results, two numerical algorithms are proposed that…
▽ More
The problem of minimizing convex functionals of probability distributions is solved under the assumption that the density of every distribution is bounded from above and below. A system of sufficient and necessary first-order optimality conditions as well as a bound on the optimality gap of feasible candidate solutions are derived. Based on these results, two numerical algorithms are proposed that iteratively solve the system of optimality conditions on a grid of discrete points. Both algorithms use a block coordinate descent strategy and terminate once the optimality gap falls below the desired tolerance. While the first algorithm is conceptually simpler and more efficient, it is not guaranteed to converge for objective functions that are not strictly convex. This shortcoming is overcome in the second algorithm, which uses an additional outer proximal iteration, and, which is proven to converge under mild assumptions. Two examples are given to demonstrate the theoretical usefulness of the optimality conditions as well as the high efficiency and accuracy of the proposed numerical algorithms.
△ Less
Submitted 4 December, 2018; v1 submitted 17 March, 2017;
originally announced March 2017.
-
Sequential joint signal detection and signal-to-noise ratio estimation
Authors:
M. Fauß,
K. G. Nagananda,
A. M. Zoubir,
H. V. Poor
Abstract:
The sequential analysis of the problem of joint signal detection and signal-to-noise ratio (SNR) estimation for a linear Gaussian observation model is considered. The problem is posed as an optimization setup where the goal is to minimize the number of samples required to achieve the desired (i) type I and type II error probabilities and (ii) mean squared error performance. This optimization probl…
▽ More
The sequential analysis of the problem of joint signal detection and signal-to-noise ratio (SNR) estimation for a linear Gaussian observation model is considered. The problem is posed as an optimization setup where the goal is to minimize the number of samples required to achieve the desired (i) type I and type II error probabilities and (ii) mean squared error performance. This optimization problem is reduced to a more tractable formulation by transforming the observed signal and noise sequences to a single sequence of Bernoulli random variables; joint detection and estimation is then performed on the Bernoulli sequence. This transformation renders the problem easily solvable, and results in a computationally simpler sufficient statistic compared to the one based on the (untransformed) observation sequences. Experimental results demonstrate the advantages of the proposed method, making it feasible for applications having strict constraints on data storage and computation.
△ Less
Submitted 18 January, 2017;
originally announced January 2017.
-
Old Bands, New Tracks---Revisiting the Band Model for Robust Hypothesis Testing
Authors:
Michael Fauß,
Abdelhak M. Zoubir
Abstract:
The density band model proposed by Kassam for robust hypothesis testing is revisited in this paper. First, a novel criterion for the general characterization of least favorable distributions is proposed, which unifies existing results. This criterion is then used to derive an implicit definition of the least favorable distributions under band uncertainties. In contrast to the existing solution, it…
▽ More
The density band model proposed by Kassam for robust hypothesis testing is revisited in this paper. First, a novel criterion for the general characterization of least favorable distributions is proposed, which unifies existing results. This criterion is then used to derive an implicit definition of the least favorable distributions under band uncertainties. In contrast to the existing solution, it only requires two scalar values to be determined and eliminates the need for case-by-case statements. Based on this definition, a generic fixed-point algorithm is proposed that iteratively calculates the least favorable distributions for arbitrary band specifications. Finally, three different types of robust tests that emerge from band models are discussed and a numerical example is presented to illustrate their potential use in practice.
△ Less
Submitted 2 March, 2018; v1 submitted 15 October, 2015;
originally announced October 2015.
