Showing 1–6 of 6 results for author: Brando, A

Shedding Light on Large Generative Networks: Estimating Epistemic Uncertainty in Diffusion Models

Authors: Lucas Berry, Axel Brando, David Meger

Abstract: Generative diffusion models, notable for their large parameter count (exceeding 100 million) and operation within high-dimensional image spaces, pose significant challenges for traditional uncertainty estimation methods due to computational demands. In this work, we introduce an innovative framework, Diffusion Ensembles for Capturing Uncertainty (DECU), designed for estimating epistemic uncertaint… ▽ More Generative diffusion models, notable for their large parameter count (exceeding 100 million) and operation within high-dimensional image spaces, pose significant challenges for traditional uncertainty estimation methods due to computational demands. In this work, we introduce an innovative framework, Diffusion Ensembles for Capturing Uncertainty (DECU), designed for estimating epistemic uncertainty for diffusion models. The DECU framework introduces a novel method that efficiently trains ensembles of conditional diffusion models by incorporating a static set of pre-trained parameters, drastically reducing the computational burden and the number of parameters that require training. Additionally, DECU employs Pairwise-Distance Estimators (PaiDEs) to accurately measure epistemic uncertainty by evaluating the mutual information between model outputs and weights in high-dimensional spaces. The effectiveness of this framework is demonstrated through experiments on the ImageNet dataset, highlighting its capability to capture epistemic uncertainty, specifically in under-sampled image classes. △ Less

Submitted 5 June, 2024; originally announced June 2024.

NEUROPULS: NEUROmorphic energy-efficient secure accelerators based on Phase change materials aUgmented siLicon photonicS

Authors: Fabio Pavanello, Cedric Marchand, Ian O'Connor, Regis Orobtchouk, Fabien Mandorlo, Xavier Letartre, Sebastien Cueff, Elena Ioana Vatajelu, Giorgio Di Natale, Benoit Cluzel, Aurelien Coillet, Benoit Charbonnier, Pierre Noe, Frantisek Kavan, Martin Zoldak, Michal Szaj, Peter Bienstman, Thomas Van Vaerenbergh, Ulrich Ruhrmair, Paulo Flores, Luis Guerra e Silva, Ricardo Chaves, Luis-Miguel Silveira, Mariano Ceccato, Dimitris Gizopoulos , et al. (12 additional authors not shown)

Abstract: This special session paper introduces the Horizon Europe NEUROPULS project, which targets the development of secure and energy-efficient RISC-V interfaced neuromorphic accelerators using augmented silicon photonics technology. Our approach aims to develop an augmented silicon photonics platform, an FPGA-powered RISC-V-connected computing platform, and a complete simulation platform to demonstrate… ▽ More This special session paper introduces the Horizon Europe NEUROPULS project, which targets the development of secure and energy-efficient RISC-V interfaced neuromorphic accelerators using augmented silicon photonics technology. Our approach aims to develop an augmented silicon photonics platform, an FPGA-powered RISC-V-connected computing platform, and a complete simulation platform to demonstrate the neuromorphic accelerator capabilities. In particular, their main advantages and limitations will be addressed concerning the underpinning technology for each platform. Then, we will discuss three targeted use cases for edge-computing applications: Global National Satellite System (GNSS) anti-jamming, autonomous driving, and anomaly detection in edge devices. Finally, we will address the reliability and security aspects of the stand-alone accelerator implementation and the project use cases. △ Less

Submitted 4 May, 2023; originally announced May 2023.

Comments: 10 pages, 2 figures, conference

Journal ref: IEEE European Test Symposium 2023

Retrospective Uncertainties for Deep Models using Vine Copulas

Authors: Nataša Tagasovska, Firat Ozdemir, Axel Brando

Abstract: Despite the major progress of deep models as learning machines, uncertainty estimation remains a major challenge. Existing solutions rely on modified loss functions or architectural changes. We propose to compensate for the lack of built-in uncertainty estimates by supplementing any network, retrospectively, with a subsequent vine copula model, in an overall compound we call Vine-Copula Neural Net… ▽ More Despite the major progress of deep models as learning machines, uncertainty estimation remains a major challenge. Existing solutions rely on modified loss functions or architectural changes. We propose to compensate for the lack of built-in uncertainty estimates by supplementing any network, retrospectively, with a subsequent vine copula model, in an overall compound we call Vine-Copula Neural Network (VCNN). Through synthetic and real-data experiments, we show that VCNNs could be task (regression/classification) and architecture (recurrent, fully connected) agnostic while providing reliable and better-calibrated uncertainty estimates, comparable to state-of-the-art built-in uncertainty solutions. △ Less

Submitted 24 February, 2023; originally announced February 2023.

Comments: Accepted at AISTATS 2023

Deep Non-Crossing Quantiles through the Partial Derivative

Authors: Axel Brando, Joan Gimeno, Jose A. Rodríguez-Serrano, Jordi Vitrià

Abstract: Quantile Regression (QR) provides a way to approximate a single conditional quantile. To have a more informative description of the conditional distribution, QR can be merged with deep learning techniques to simultaneously estimate multiple quantiles. However, the minimisation of the QR-loss function does not guarantee non-crossing quantiles, which affects the validity of such predictions and intr… ▽ More Quantile Regression (QR) provides a way to approximate a single conditional quantile. To have a more informative description of the conditional distribution, QR can be merged with deep learning techniques to simultaneously estimate multiple quantiles. However, the minimisation of the QR-loss function does not guarantee non-crossing quantiles, which affects the validity of such predictions and introduces a critical issue in certain scenarios. In this article, we propose a generic deep learning algorithm for predicting an arbitrary number of quantiles that ensures the quantile monotonicity constraint up to the machine precision and maintains its modelling performance with respect to alternative models. The presented method is evaluated over several real-world datasets obtaining state-of-the-art results as well as showing that it scales to large-size data sets. △ Less

Submitted 30 January, 2022; originally announced January 2022.

Comments: In the Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS)

Modelling heterogeneous distributions with an Uncountable Mixture of Asymmetric Laplacians

Authors: Axel Brando, Jose A. Rodríguez-Serrano, Jordi Vitrià, Alberto Rubio

Abstract: In regression tasks, aleatoric uncertainty is commonly addressed by considering a parametric distribution of the output variable, which is based on strong assumptions such as symmetry, unimodality or by supposing a restricted shape. These assumptions are too limited in scenarios where complex shapes, strong skews or multiple modes are present. In this paper, we propose a generic deep learning fram… ▽ More In regression tasks, aleatoric uncertainty is commonly addressed by considering a parametric distribution of the output variable, which is based on strong assumptions such as symmetry, unimodality or by supposing a restricted shape. These assumptions are too limited in scenarios where complex shapes, strong skews or multiple modes are present. In this paper, we propose a generic deep learning framework that learns an Uncountable Mixture of Asymmetric Laplacians (UMAL), which will allow us to estimate heterogeneous distributions of the output variable and shows its connections to quantile regression. Despite having a fixed number of parameters, the model can be interpreted as an infinite mixture of components, which yields a flexible approximation for heterogeneous distributions. Apart from synthetic cases, we apply this model to room price forecasting and to predict financial operations in personal bank accounts. We demonstrate that UMAL produces proper distributions, which allows us to extract richer insights and to sharpen decision-making. △ Less

Submitted 29 October, 2019; v1 submitted 27 October, 2019; originally announced October 2019.

Comments: 12 pages, 4 figures, Paper accepted as poster at the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada

Uncertainty Modelling in Deep Networks: Forecasting Short and Noisy Series

Authors: Axel Brando, Jose A. Rodríguez-Serrano, Mauricio Ciprian, Roberto Maestre, Jordi Vitrià

Abstract: Deep Learning is a consolidated, state-of-the-art Machine Learning tool to fit a function when provided with large data sets of examples. However, in regression tasks, the straightforward application of Deep Learning models provides a point estimate of the target. In addition, the model does not take into account the uncertainty of a prediction. This represents a great limitation for tasks where c… ▽ More Deep Learning is a consolidated, state-of-the-art Machine Learning tool to fit a function when provided with large data sets of examples. However, in regression tasks, the straightforward application of Deep Learning models provides a point estimate of the target. In addition, the model does not take into account the uncertainty of a prediction. This represents a great limitation for tasks where communicating an erroneous prediction carries a risk. In this paper we tackle a real-world problem of forecasting impending financial expenses and incomings of customers, while displaying predictable monetary amounts on a mobile app. In this context, we investigate if we would obtain an advantage by applying Deep Learning models with a Heteroscedastic model of the variance of a network's output. Experimentally, we achieve a higher accuracy than non-trivial baselines. More importantly, we introduce a mechanism to discard low-confidence predictions, which means that they will not be visible to users. This should help enhance the user experience of our product. △ Less

Submitted 24 July, 2018; originally announced July 2018.

Comments: 17 pages, 5 figures, Applied Data Science Track of The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD 2018)