Showing 1–2 of 2 results for author: Antonello, N

A t-distribution based operator for enhancing out of distribution robustness of neural network classifiers

Authors: Niccolò Antonello, Philip N. Garner

Abstract: Neural Network (NN) classifiers can assign extreme probabilities to samples that have not appeared during training (out-of-distribution samples) resulting in erroneous and unreliable predictions. One of the causes for this unwanted behaviour lies in the use of the standard softmax operator which pushes the posterior probabilities to be either zero or unity hence failing to model uncertainty. The s… ▽ More Neural Network (NN) classifiers can assign extreme probabilities to samples that have not appeared during training (out-of-distribution samples) resulting in erroneous and unreliable predictions. One of the causes for this unwanted behaviour lies in the use of the standard softmax operator which pushes the posterior probabilities to be either zero or unity hence failing to model uncertainty. The statistical derivation of the softmax operator relies on the assumption that the distributions of the latent variables for a given class are Gaussian with known variance. However, it is possible to use different assumptions in the same derivation and attain from other families of distributions as well. This allows derivation of novel operators with more favourable properties. Here, a novel operator is proposed that is derived using $t$-distributions which are capable of providing a better description of uncertainty. It is shown that classifiers that adopt this novel operator can be more robust to out of distribution samples, often outperforming NNs that use the standard softmax operator. These enhancements can be reached with minimal changes to the NN architecture. △ Less

Submitted 9 October, 2020; v1 submitted 9 June, 2020; originally announced June 2020.

Comments: 5 pages, 5 figures, to be published in IEEE Signal Processing Letters, reproducible code https://github.com/idiap/tsoftmax

Proximal Gradient Algorithms: Applications in Signal Processing

Authors: Niccolò Antonello, Lorenzo Stella, Panagiotis Patrinos, Toon van Waterschoot

Abstract: Advances in numerical optimization have supported breakthroughs in several areas of signal processing. This paper focuses on the recent enhanced variants of the proximal gradient numerical optimization algorithm, which combine quasi-Newton methods with forward-adjoint oracles to tackle large-scale problems and reduce the computational burden of many applications. These proximal gradient algorithms… ▽ More Advances in numerical optimization have supported breakthroughs in several areas of signal processing. This paper focuses on the recent enhanced variants of the proximal gradient numerical optimization algorithm, which combine quasi-Newton methods with forward-adjoint oracles to tackle large-scale problems and reduce the computational burden of many applications. These proximal gradient algorithms are here described in an easy-to-understand way, illustrating how they are able to address a wide variety of problems arising in signal processing. A new high-level modeling language is presented which is used to demonstrate the versatility of the presented algorithms in a series of signal processing application examples such as sparse deconvolution, total variation denoising, audio de-clip** and others. △ Less

Submitted 27 January, 2020; v1 submitted 5 March, 2018; originally announced March 2018.

Comments: 30 pages, 14 figures

ACM Class: G.1.6; I.2; D.2