-
PARMESAN: Parameter-Free Memory Search and Transduction for Dense Prediction Tasks
Authors:
Philip Matthias Winter,
Maria Wimmer,
David Major,
Dimitrios Lenis,
Astrid Berg,
Theresa Neubauer,
Gaia Romana De Paolis,
Johannes Novotny,
Sophia Ulonska,
Katja Bühler
Abstract:
In this work we address flexibility in deep learning by means of transductive reasoning. For adaptation to new tasks or new data, existing methods typically involve tuning of learnable parameters or even complete re-training from scratch, rendering such approaches unflexible in practice. We argue that the notion of separating computation from memory by the means of transduction can act as a steppi…
▽ More
In this work we address flexibility in deep learning by means of transductive reasoning. For adaptation to new tasks or new data, existing methods typically involve tuning of learnable parameters or even complete re-training from scratch, rendering such approaches unflexible in practice. We argue that the notion of separating computation from memory by the means of transduction can act as a step** stone for solving these issues. We therefore propose PARMESAN (parameter-free memory search and transduction), a scalable transduction method which leverages a memory module for solving dense prediction tasks. At inference, hidden representations in memory are being searched to find corresponding examples. In contrast to other methods, PARMESAN learns without the requirement for any continuous training or fine-tuning of learnable parameters simply by modifying the memory content. Our method is compatible with commonly used neural architectures and canonically transfers to 1D, 2D, and 3D grid-based data. We demonstrate the capabilities of our approach at complex tasks such as continual and few-shot learning. PARMESAN learns up to 370 times faster than common baselines while being on par in terms of predictive performance, knowledge retention, and data-efficiency.
△ Less
Submitted 18 March, 2024;
originally announced March 2024.
-
Exact-corrected confidence interval for risk difference in noninferiority binomial trials
Authors:
Nour Hawila,
Arthur Berg
Abstract:
A novel confidence interval estimator is proposed for the risk difference in noninferiority binomial trials. The confidence interval is consistent with an exact unconditional test that preserves the type-I error, and has improved power, particularly for smaller sample sizes, compared to the confidence interval by Chan & Zhang (1999). The improved performance of the proposed confidence interval is…
▽ More
A novel confidence interval estimator is proposed for the risk difference in noninferiority binomial trials. The confidence interval is consistent with an exact unconditional test that preserves the type-I error, and has improved power, particularly for smaller sample sizes, compared to the confidence interval by Chan & Zhang (1999). The improved performance of the proposed confidence interval is theoretically justified and demonstrated with simulations and examples. An R package is also distributed that implements the proposed methods along with other confidence interval estimators.
△ Less
Submitted 14 October, 2021; v1 submitted 9 April, 2021;
originally announced April 2021.
-
Worldsheet: Wrap** the World in a 3D Sheet for View Synthesis from a Single Image
Authors:
Ronghang Hu,
Nikhila Ravi,
Alexander C. Berg,
Deepak Pathak
Abstract:
We present Worldsheet, a method for novel view synthesis using just a single RGB image as input. The main insight is that simply shrink-wrap** a planar mesh sheet onto the input image, consistent with the learned intermediate depth, captures underlying geometry sufficient to generate photorealistic unseen views with large viewpoint changes. To operationalize this, we propose a novel differentiab…
▽ More
We present Worldsheet, a method for novel view synthesis using just a single RGB image as input. The main insight is that simply shrink-wrap** a planar mesh sheet onto the input image, consistent with the learned intermediate depth, captures underlying geometry sufficient to generate photorealistic unseen views with large viewpoint changes. To operationalize this, we propose a novel differentiable texture sampler that allows our wrapped mesh sheet to be textured and rendered differentiably into an image from a target viewpoint. Our approach is category-agnostic, end-to-end trainable without using any 3D supervision, and requires a single image at test time. We also explore a simple extension by stacking multiple layers of Worldsheets to better handle occlusions. Worldsheet consistently outperforms prior state-of-the-art methods on single-image view synthesis across several datasets. Furthermore, this simple idea captures novel views surprisingly well on a wide range of high-resolution in-the-wild images, converting them into navigable 3D pop-ups. Video results and code are available at https://worldsheet.github.io.
△ Less
Submitted 18 August, 2021; v1 submitted 17 December, 2020;
originally announced December 2020.
-
Similarity Search for Efficient Active Learning and Search of Rare Concepts
Authors:
Cody Coleman,
Edward Chou,
Julian Katz-Samuels,
Sean Culatana,
Peter Bailis,
Alexander C. Berg,
Robert Nowak,
Roshan Sumbaly,
Matei Zaharia,
I. Zeki Yalniz
Abstract:
Many active learning and search approaches are intractable for large-scale industrial settings with billions of unlabeled examples. Existing approaches search globally for the optimal examples to label, scaling linearly or even quadratically with the unlabeled data. In this paper, we improve the computational efficiency of active learning and search methods by restricting the candidate pool for la…
▽ More
Many active learning and search approaches are intractable for large-scale industrial settings with billions of unlabeled examples. Existing approaches search globally for the optimal examples to label, scaling linearly or even quadratically with the unlabeled data. In this paper, we improve the computational efficiency of active learning and search methods by restricting the candidate pool for labeling to the nearest neighbors of the currently labeled set instead of scanning over all of the unlabeled data. We evaluate several selection strategies in this setting on three large-scale computer vision datasets: ImageNet, OpenImages, and a de-identified and aggregated dataset of 10 billion images provided by a large internet company. Our approach achieved similar mean average precision and recall as the traditional global approach while reducing the computational cost of selection by up to three orders of magnitude, thus enabling web-scale active learning.
△ Less
Submitted 22 July, 2021; v1 submitted 30 June, 2020;
originally announced July 2020.
-
Deep Ordinal Regression with Label Diversity
Authors:
Axel Berg,
Magnus Oskarsson,
Mark O'Connor
Abstract:
Regression via classification (RvC) is a common method used for regression problems in deep learning, where the target variable belongs to a set of continuous values. By discretizing the target into a set of non-overlap** classes, it has been shown that training a classifier can improve neural network accuracy compared to using a standard regression approach. However, it is not clear how the set…
▽ More
Regression via classification (RvC) is a common method used for regression problems in deep learning, where the target variable belongs to a set of continuous values. By discretizing the target into a set of non-overlap** classes, it has been shown that training a classifier can improve neural network accuracy compared to using a standard regression approach. However, it is not clear how the set of discrete classes should be chosen and how it affects the overall solution. In this work, we propose that using several discrete data representations simultaneously can improve neural network learning compared to a single representation. Our approach is end-to-end differentiable and can be added as a simple extension to conventional learning methods, such as deep neural networks. We test our method on three challenging tasks and show that our method reduces the prediction error compared to a baseline RvC approach while maintaining a similar model complexity.
△ Less
Submitted 29 June, 2020;
originally announced June 2020.
-
LU-Net: a multi-task network to improve the robustness of segmentation of left ventriclular structures by deep learning in 2D echocardiography
Authors:
Sarah Leclerc,
Erik Smistad,
Andreas Østvik,
Frederic Cervenansky,
Florian Espinosa,
Torvald Espeland,
Erik Andreas Rye Berg,
Thomas Grenier,
Carole Lartizien,
Pierre-Marc Jodoin,
Lasse Lovstakken,
Olivier Bernard
Abstract:
Segmentation of cardiac structures is one of the fundamental steps to estimate volumetric indices of the heart. This step is still performed semi-automatically in clinical routine, and is thus prone to inter- and intra-observer variability. Recent studies have shown that deep learning has the potential to perform fully automatic segmentation. However, the current best solutions still suffer from a…
▽ More
Segmentation of cardiac structures is one of the fundamental steps to estimate volumetric indices of the heart. This step is still performed semi-automatically in clinical routine, and is thus prone to inter- and intra-observer variability. Recent studies have shown that deep learning has the potential to perform fully automatic segmentation. However, the current best solutions still suffer from a lack of robustness. In this work, we introduce an end-to-end multi-task network designed to improve the overall accuracy of cardiac segmentation while enhancing the estimation of clinical indices and reducing the number of outliers. Results obtained on a large open access dataset show that our method outperforms the current best performing deep learning solution and achieved an overall segmentation accuracy lower than the intra-observer variability for the epicardial border (i.e. on average a mean absolute error of 1.5mm and a Hausdorff distance of 5.1mm) with 11% of outliers. Moreover, we demonstrate that our method can closely reproduce the expert analysis for the end-diastolic and end-systolic left ventricular volumes, with a mean correlation of 0.96 and a mean absolute error of 7.6ml. Concerning the ejection fraction of the left ventricle, results are more contrasted with a mean correlation coefficient of 0.83 and an absolute mean error of 5.0%, producing scores that are slightly below the intra-observer margin. Based on this observation, areas for improvement are suggested.
△ Less
Submitted 4 April, 2020;
originally announced April 2020.
-
Connecting Bayes factor and the Region of Practical Equivalence (ROPE) Procedure for testing interval null hypothesis
Authors:
J. G. Liao,
Vishal Midya,
Arthur Berg
Abstract:
There has been strong recent interest in testing interval null hypothesis for improved scientific inference. For example, Lakens et al (2018) and Lakens and Harms (2017) use this approach to study if there is a pre-specified meaningful treatment effect in gerontology and clinical trials, which is different from the more traditional point null hypothesis that tests for any treatment effect. Two pop…
▽ More
There has been strong recent interest in testing interval null hypothesis for improved scientific inference. For example, Lakens et al (2018) and Lakens and Harms (2017) use this approach to study if there is a pre-specified meaningful treatment effect in gerontology and clinical trials, which is different from the more traditional point null hypothesis that tests for any treatment effect. Two popular Bayesian approaches are available for interval null hypothesis testing. One is the standard Bayes factor and the other is the Region of Practical Equivalence (ROPE) procedure championed by Kruschke and others over many years. This paper establishes a formal connection between these two approaches with two benefits. First, it helps to better understand and improve the ROPE procedure. Second, it leads to a simple and effective algorithm for computing Bayes factor in a wide range of problems using draws from posterior distributions generated by standard Bayesian programs such as BUGS, JAGS and Stan. The tedious and error-prone task of coding custom-made software specific for Bayes factor is then avoided.
△ Less
Submitted 30 April, 2019; v1 submitted 7 March, 2019;
originally announced March 2019.
-
A robustified posterior for Bayesian inference on a large number of parallel effects
Authors:
J G Liao,
Arthur Berg,
Timothy L McMurry
Abstract:
Many modern experiments, such as microarray gene expression and genome-wide association studies, present the problem of estimating a large number of parallel effects. Bayesian inference is a popular approach for analyzing such data by modeling the large number of unknown parameters as random effects from a common prior distribution. However, misspecification of the prior distribution can lead to e…
▽ More
Many modern experiments, such as microarray gene expression and genome-wide association studies, present the problem of estimating a large number of parallel effects. Bayesian inference is a popular approach for analyzing such data by modeling the large number of unknown parameters as random effects from a common prior distribution. However, misspecification of the prior distribution can lead to erroneous estimates of the random effects, especially for the largest and most interesting effects. This paper has two aims. First, we propose a robustified posterior distribution for a parametric Bayesian hierarchical model that can substantially reduce the impact of a misspecified prior. Second, we conduct a systematic comparison of the standard parametric posterior, the proposed robustified parametric posterior, and a nonparametric Bayesian posterior which uses a Dirichlet process mixture prior. The proposed robustifed posterior when combined with a flexible parametric prior can be a superior alternative to nonparametric Bayesian methods.
△ Less
Submitted 25 October, 2018; v1 submitted 27 June, 2018;
originally announced June 2018.
-
CDF and Survival Function Estimation with Infinite-Order Kernels
Authors:
Arthur Berg,
Dimitris N. Politis
Abstract:
A reduced-bias nonparametric estimator of the cumulative distribution function (CDF) and the survival function is proposed using infinite-order kernels. Fourier transform theory on generalized functions is utilized to obtain the improved bias estimates. The new estimators are analyzed in terms of their relative deficiency to the empirical distribution function and Kaplan-Meier estimator, and eve…
▽ More
A reduced-bias nonparametric estimator of the cumulative distribution function (CDF) and the survival function is proposed using infinite-order kernels. Fourier transform theory on generalized functions is utilized to obtain the improved bias estimates. The new estimators are analyzed in terms of their relative deficiency to the empirical distribution function and Kaplan-Meier estimator, and even improvements in terms of asymptotic relative efficiency (ARE) are present under specified assumptions on the data. The deficiency analysis introduces a deficiency rate which provides a continuum between the classical deficiency analysis and an efficiency analysis. Additionally, an automatic bandwidth selection algorithm, specially tailored to the infinite-order kernels, is incorporated into the estimators. In small sample sizes these estimators can significantly improve the estimation of the CDF and survival function as is illustrated through the deficiency analysis and computer simulations.
△ Less
Submitted 17 March, 2009;
originally announced March 2009.
