-
Thin and Deep Gaussian Processes
Authors:
Daniel Augusto de Souza,
Alexander Nikitin,
ST John,
Magnus Ross,
Mauricio A. Álvarez,
Marc Peter Deisenroth,
João P. P. Gomes,
Diego Mesquita,
César Lincoln C. Mattos
Abstract:
Gaussian processes (GPs) can provide a principled approach to uncertainty quantification with easy-to-interpret kernel hyperparameters, such as the lengthscale, which controls the correlation distance of function values. However, selecting an appropriate kernel can be challenging. Deep GPs avoid manual kernel engineering by successively parameterizing kernels with GP layers, allowing them to learn…
▽ More
Gaussian processes (GPs) can provide a principled approach to uncertainty quantification with easy-to-interpret kernel hyperparameters, such as the lengthscale, which controls the correlation distance of function values. However, selecting an appropriate kernel can be challenging. Deep GPs avoid manual kernel engineering by successively parameterizing kernels with GP layers, allowing them to learn low-dimensional embeddings of the inputs that explain the output data. Following the architecture of deep neural networks, the most common deep GPs warp the input space layer-by-layer but lose all the interpretability of shallow GPs. An alternative construction is to successively parameterize the lengthscale of a kernel, improving the interpretability but ultimately giving away the notion of learning lower-dimensional embeddings. Unfortunately, both methods are susceptible to particular pathologies which may hinder fitting and limit their interpretability. This work proposes a novel synthesis of both previous approaches: Thin and Deep GP (TDGP). Each TDGP layer defines locally linear transformations of the original input data maintaining the concept of latent embeddings while also retaining the interpretation of lengthscales of a kernel. Moreover, unlike the prior solutions, TDGP induces non-pathological manifolds that admit learning lower-dimensional representations. We show with theoretical and experimental results that i) TDGP is, unlike previous models, tailored to specifically discover lower-dimensional manifolds in the input data, ii) TDGP behaves well when increasing the number of layers, and iii) TDGP performs well in standard benchmark datasets.
△ Less
Submitted 17 October, 2023;
originally announced October 2023.
-
Minimal Learning Machine for Multi-Label Learning
Authors:
Joonas Hämäläinen,
Amauri Souza,
César L. C. Mattos,
João P. P. Gomes,
Tommi Kärkkäinen
Abstract:
Distance-based supervised method, the minimal learning machine, constructs a predictive model from data by learning a map** between input and output distance matrices. In this paper, we propose methods and evaluate how this technique and its core component, the distance map**, can be adapted to multi-label learning. The proposed approach is based on combining the distance map** with an inver…
▽ More
Distance-based supervised method, the minimal learning machine, constructs a predictive model from data by learning a map** between input and output distance matrices. In this paper, we propose methods and evaluate how this technique and its core component, the distance map**, can be adapted to multi-label learning. The proposed approach is based on combining the distance map** with an inverse distance weighting. Although the proposal is one of the simplest methods in the multi-label learning literature, it achieves state-of-the-art performance for small to moderate-sized multi-label learning problems. Besides its simplicity, the proposed method is fully deterministic and its hyper-parameter can be selected via ranking loss-based statistic which has a closed form, thus avoiding conventional cross-validation-based hyper-parameter tuning. In addition, due to its simple linear distance map**-based construction, we demonstrate that the proposed method can assess predictions' uncertainty for multi-label classification, which is a valuable capability for data-centric machine learning pipelines.
△ Less
Submitted 9 May, 2023;
originally announced May 2023.
-
Reduction of Surgical Risk Through the Evaluation of Medical Imaging Diagnostics
Authors:
Marco A. V. M. Grinet,
Nuno M. Garcia,
Ana I. R. Gouveia,
Jose A. F. Moutinho,
Abel J. P. Gomes
Abstract:
Computer aided diagnosis (CAD) of Breast Cancer (BRCA) images has been an active area of research in recent years. The main goals of this research is to develop reliable automatic methods for detecting and diagnosing different types of BRCA from diagnostic images. In this paper, we present a review of the state of the art CAD methods applied to magnetic resonance (MRI) and mammography images of BR…
▽ More
Computer aided diagnosis (CAD) of Breast Cancer (BRCA) images has been an active area of research in recent years. The main goals of this research is to develop reliable automatic methods for detecting and diagnosing different types of BRCA from diagnostic images. In this paper, we present a review of the state of the art CAD methods applied to magnetic resonance (MRI) and mammography images of BRCA patients. The review aims to provide an extensive introduction to different features extracted from BRCA images through texture and statistical analysis and to categorize deep learning frameworks and data structures capable of using metadata to aggregate relevant information to assist oncologists and radiologists. We divide the existing literature according to the imaging modality and into radiomics, machine learning, or combination of both. We also emphasize the difference between each modality and methods strengths and weaknesses and analyze their performance in detecting BRCA through a quantitative comparison. We compare the results of various approaches for implementing CAD systems for the detection of BRCA. Each approachs standard workflow components are reviewed and summary tables provided. We present an extensive literature review of radiomics feature extraction techniques and machine learning methods applied in BRCA diagnosis and detection, focusing on data preparation, data structures, pre processing and post processing strategies available in the literature. There is a growing interest on radiomic feature extraction and machine learning methods for BRCA detection through histopathological images, MRI and mammography images. However, there isnt a CAD method able to combine distinct data types to provide the best diagnostic results. Employing data fusion techniques to medical images and patient data could lead to improved detection and classification results.
△ Less
Submitted 8 March, 2020;
originally announced March 2020.
-
Pose Guided Attention for Multi-label Fashion Image Classification
Authors:
Beatriz Quintino Ferreira,
João P. Costeira,
Ricardo G. Sousa,
Liang-Yan Gui,
João P. Gomes
Abstract:
We propose a compact framework with guided attention for multi-label classification in the fashion domain. Our visual semantic attention model (VSAM) is supervised by automatic pose extraction creating a discriminative feature space. VSAM outperforms the state of the art for an in-house dataset and performs on par with previous works on the DeepFashion dataset, even without using any landmark anno…
▽ More
We propose a compact framework with guided attention for multi-label classification in the fashion domain. Our visual semantic attention model (VSAM) is supervised by automatic pose extraction creating a discriminative feature space. VSAM outperforms the state of the art for an in-house dataset and performs on par with previous works on the DeepFashion dataset, even without using any landmark annotations. Additionally, we show that our semantic attention module brings robustness to large quantities of wrong annotations and provides more interpretable results.
△ Less
Submitted 12 November, 2019;
originally announced November 2019.
-
Minimal Learning Machine: Theoretical Results and Clustering-Based Reference Point Selection
Authors:
Joonas Hämäläinen,
Alisson S. C. Alencar,
Tommi Kärkkäinen,
César L. C. Mattos,
Amauri H. Souza Júnior,
João P. P. Gomes
Abstract:
The Minimal Learning Machine (MLM) is a nonlinear supervised approach based on learning a linear map** between distance matrices computed in the input and output data spaces, where distances are calculated using a subset of points called reference points. Its simple formulation has attracted several recent works on extensions and applications. In this paper, we aim to address some open questions…
▽ More
The Minimal Learning Machine (MLM) is a nonlinear supervised approach based on learning a linear map** between distance matrices computed in the input and output data spaces, where distances are calculated using a subset of points called reference points. Its simple formulation has attracted several recent works on extensions and applications. In this paper, we aim to address some open questions related to the MLM. First, we detail theoretical aspects that assure the interpolation and universal approximation capabilities of the MLM, which were previously only empirically verified. Second, we identify the task of selecting reference points as having major importance for the MLM's generalization capability. Several clustering-based methods for reference point selection in regression scenarios are then proposed and analyzed. Based on an extensive empirical evaluation, we conclude that the evaluated methods are both scalable and useful. Specifically, for a small number of reference points, the clustering-based methods outperformed the standard random selection of the original MLM formulation.
△ Less
Submitted 6 October, 2020; v1 submitted 22 September, 2019;
originally announced September 2019.
-
No-PASt-BO: Normalized Portfolio Allocation Strategy for Bayesian Optimization
Authors:
Thiago de P. Vasconcelos,
Daniel A. R. M. A. de Souza,
César L. C. Mattos,
João P. P. Gomes
Abstract:
Bayesian Optimization (BO) is a framework for black-box optimization that is especially suitable for expensive cost functions. Among the main parts of a BO algorithm, the acquisition function is of fundamental importance, since it guides the optimization algorithm by translating the uncertainty of the regression model in a utility measure for each point to be evaluated. Considering such aspect, se…
▽ More
Bayesian Optimization (BO) is a framework for black-box optimization that is especially suitable for expensive cost functions. Among the main parts of a BO algorithm, the acquisition function is of fundamental importance, since it guides the optimization algorithm by translating the uncertainty of the regression model in a utility measure for each point to be evaluated. Considering such aspect, selection and design of acquisition functions are one of the most popular research topics in BO. Since no single acquisition function was proved to have better performance in all tasks, a well-established approach consists of selecting different acquisition functions along the iterations of a BO execution. In such an approach, the GP-Hedge algorithm is a widely used option given its simplicity and good performance. Despite its success in various applications, GP-Hedge shows an undesirable characteristic of accounting on all past performance measures of each acquisition function to select the next function to be used. In this case, good or bad values obtained in an initial iteration may impact the choice of the acquisition function for the rest of the algorithm. This fact may induce a dominant behavior of an acquisition function and impact the final performance of the method. Aiming to overcome such limitation, in this work we propose a variant of GP-Hedge, named No-PASt-BO, that reduce the influence of far past evaluations. Moreover, our method presents a built-in normalization that avoids the functions in the portfolio to have similar probabilities, thus improving the exploration. The obtained results on both synthetic and real-world optimization tasks indicate that No-PASt-BO presents competitive performance and always outperforms GP-Hedge.
△ Less
Submitted 1 August, 2019;
originally announced August 2019.
-
Learning GPLVM with arbitrary kernels using the unscented transformation
Authors:
Daniel Augusto R. M. A. de Souza,
Diego Mesquita,
César Lincoln C. Mattos,
João Paulo P. Gomes
Abstract:
Gaussian Process Latent Variable Model (GPLVM) is a flexible framework to handle uncertain inputs in Gaussian Processes (GPs) and incorporate GPs as components of larger graphical models. Nonetheless, the standard GPLVM variational inference approach is tractable only for a narrow family of kernel functions. The most popular implementations of GPLVM circumvent this limitation using quadrature meth…
▽ More
Gaussian Process Latent Variable Model (GPLVM) is a flexible framework to handle uncertain inputs in Gaussian Processes (GPs) and incorporate GPs as components of larger graphical models. Nonetheless, the standard GPLVM variational inference approach is tractable only for a narrow family of kernel functions. The most popular implementations of GPLVM circumvent this limitation using quadrature methods, which may become a computational bottleneck even for relatively low dimensions. For instance, the widely employed Gauss-Hermite quadrature has exponential complexity on the number of dimensions. In this work, we propose using the unscented transformation instead. Overall, this method presents comparable, if not better, performance than offthe-shelf solutions to GPLVM and its computational complexity scales only linearly on dimension. In contrast to Monte Carlo methods, our approach is deterministic and works well with quasi-Newton methods, such as the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm. We illustrate the applicability of our method with experiments on dimensionality reduction and multistep-ahead prediction with uncertainty propagation.
△ Less
Submitted 10 November, 2020; v1 submitted 3 July, 2019;
originally announced July 2019.
-
Pi-surfaces: products of implicit surfaces towards constructive composition of 3D objects
Authors:
Adriano N. Raposo,
Abel J. P. Gomes
Abstract:
Implicit functions provide a fundamental basis to model 3D objects, no matter they are rigid or deformable, in computer graphics and geometric modeling. This paper introduces a new constructive scheme of implicitly-defined 3D objects based on products of implicit functions. This scheme is in contrast with popular approaches like blobbies, meta balls and soft objects, which rely on the sum of speci…
▽ More
Implicit functions provide a fundamental basis to model 3D objects, no matter they are rigid or deformable, in computer graphics and geometric modeling. This paper introduces a new constructive scheme of implicitly-defined 3D objects based on products of implicit functions. This scheme is in contrast with popular approaches like blobbies, meta balls and soft objects, which rely on the sum of specific implicit functions to fit a 3D object to a set of spheres.
△ Less
Submitted 16 June, 2019;
originally announced June 2019.
-
LS-SVR as a Bayesian RBF network
Authors:
Diego P. P. Mesquita,
Luis A. Freitas,
João P. P. Gomes,
César L. C. Mattos
Abstract:
We show theoretical similarities between the Least Squares Support Vector Regression (LS-SVR) model with a Radial Basis Functions (RBF) kernel and maximum a posteriori (MAP) inference on Bayesian RBF networks with a specific Gaussian prior on the regression weights. Although previous works have pointed out similar expressions between those learning approaches, we explicit and formally state the ex…
▽ More
We show theoretical similarities between the Least Squares Support Vector Regression (LS-SVR) model with a Radial Basis Functions (RBF) kernel and maximum a posteriori (MAP) inference on Bayesian RBF networks with a specific Gaussian prior on the regression weights. Although previous works have pointed out similar expressions between those learning approaches, we explicit and formally state the existing correspondences. We empirically demonstrate our result by performing computational experiments with standard regression benchmarks. Our findings open a range of possibilities to improve LS-SVR by borrowing strength from well-established developments in Bayesian methodology.
△ Less
Submitted 2 August, 2019; v1 submitted 1 May, 2019;
originally announced May 2019.
