-
Statistical Inference on Grayscale Images via the Euler-Radon Transform
Authors:
Kun Meng,
Mattie Ji,
**yu Wang,
Kexin Ding,
Henry Kirveslahti,
Ani Eloyan,
Lorin Crawford
Abstract:
Tools from topological data analysis have been widely used to represent binary images in many scientific applications. Methods that aim to represent grayscale images (i.e., where pixel intensities instead take on continuous values) have been relatively underdeveloped. In this paper, we introduce the Euler-Radon transform, which generalizes the Euler characteristic transform to grayscale images by…
▽ More
Tools from topological data analysis have been widely used to represent binary images in many scientific applications. Methods that aim to represent grayscale images (i.e., where pixel intensities instead take on continuous values) have been relatively underdeveloped. In this paper, we introduce the Euler-Radon transform, which generalizes the Euler characteristic transform to grayscale images by using o-minimal structures and Euler integration over definable functions. Coupling the Karhunen-Loeve expansion with our proposed topological representation, we offer hypothesis-testing algorithms based on the chi-squared distribution for detecting significant differences between two groups of grayscale images. We illustrate our framework via extensive numerical experiments and simulations.
△ Less
Submitted 27 August, 2023;
originally announced August 2023.
-
Should I Stop or Should I Go: Early Stop** with Heterogeneous Populations
Authors:
Hammaad Adam,
Fan Yin,
Huibin,
Hu,
Neil Tenenholtz,
Lorin Crawford,
Lester Mackey,
Allison Koenecke
Abstract:
Randomized experiments often need to be stopped prematurely due to the treatment having an unintended harmful effect. Existing methods that determine when to stop an experiment early are typically applied to the data in aggregate and do not account for treatment effect heterogeneity. In this paper, we study the early stop** of experiments for harm on heterogeneous populations. We first establish…
▽ More
Randomized experiments often need to be stopped prematurely due to the treatment having an unintended harmful effect. Existing methods that determine when to stop an experiment early are typically applied to the data in aggregate and do not account for treatment effect heterogeneity. In this paper, we study the early stop** of experiments for harm on heterogeneous populations. We first establish that current methods often fail to stop experiments when the treatment harms a minority group of participants. We then use causal machine learning to develop CLASH, the first broadly-applicable method for heterogeneous early stop**. We demonstrate CLASH's performance on simulated and real data and show that it yields effective early stop** for both clinical trials and A/B tests.
△ Less
Submitted 27 October, 2023; v1 submitted 20 June, 2023;
originally announced June 2023.
-
A Simple Approach for Local and Global Variable Importance in Nonlinear Regression Models
Authors:
Emily T. Winn-Nuñez,
Maryclare Griffin,
Lorin Crawford
Abstract:
The ability to interpret machine learning models has become increasingly important as their usage in data science continues to rise. Most current interpretability methods are optimized to work on either (\textit{i}) a global scale, where the goal is to rank features based on their contributions to overall variation in an observed population, or (\textit{ii}) the local level, which aims to detail o…
▽ More
The ability to interpret machine learning models has become increasingly important as their usage in data science continues to rise. Most current interpretability methods are optimized to work on either (\textit{i}) a global scale, where the goal is to rank features based on their contributions to overall variation in an observed population, or (\textit{ii}) the local level, which aims to detail on how important a feature is to a particular individual in the data set. In this work, a new operator is proposed called the "GlObal And Local Score" (GOALS): a simple \textit{post hoc} approach to simultaneously assess local and global feature variable importance in nonlinear models. Motivated by problems in biomedicine, the approach is demonstrated using Gaussian process regression where the task of understanding how genetic markers are associated with disease progression both within individuals and across populations is of high interest. Detailed simulations and real data analyses illustrate the flexible and efficient utility of GOALS over state-of-the-art variable importance strategies.
△ Less
Submitted 10 August, 2023; v1 submitted 3 February, 2023;
originally announced February 2023.
-
Randomness of Shapes and Statistical Inference on Shapes via the Smooth Euler Characteristic Transform
Authors:
Kun Meng,
**yu Wang,
Lorin Crawford,
Ani Eloyan
Abstract:
In this article, we establish the mathematical foundations for modeling the randomness of shapes and conducting statistical inference on shapes using the smooth Euler characteristic transform. Based on these foundations, we propose two chi-squared statistic-based algorithms for testing hypotheses on random shapes. Simulation studies are presented to validate our mathematical derivations and to com…
▽ More
In this article, we establish the mathematical foundations for modeling the randomness of shapes and conducting statistical inference on shapes using the smooth Euler characteristic transform. Based on these foundations, we propose two chi-squared statistic-based algorithms for testing hypotheses on random shapes. Simulation studies are presented to validate our mathematical derivations and to compare our algorithms with state-of-the-art methods to demonstrate the utility of our proposed framework. As real applications, we analyze a data set of mandibular molars from four genera of primates and show that our algorithms have the power to detect significant shape differences that recapitulate known morphological variation across suborders. Altogether, our discussions bridge the following fields: algebraic and computational topology, probability theory and stochastic processes, Sobolev spaces and functional analysis, analysis of variance for functional data, and geometric morphometrics.
△ Less
Submitted 23 May, 2024; v1 submitted 27 April, 2022;
originally announced April 2022.
-
Hollow-tree Super: a directional and scalable approach for feature importance in boosted tree models
Authors:
Stephane Doyen,
Hugh Taylor,
Peter Nicholas,
Lewis Crawford,
Isabella Young,
Michael Sughrue
Abstract:
Current limitations in boosted tree modelling prevent the effective scaling to datasets with a large feature number, particularly when investigating the magnitude and directionality of various features on classification. We present a novel methodology, Hollow-tree Super (HOTS), to resolve and visualize feature importance in boosted tree models involving a large number of features. Further, HOTS al…
▽ More
Current limitations in boosted tree modelling prevent the effective scaling to datasets with a large feature number, particularly when investigating the magnitude and directionality of various features on classification. We present a novel methodology, Hollow-tree Super (HOTS), to resolve and visualize feature importance in boosted tree models involving a large number of features. Further, HOTS allows for investigation of the directionality and magnitude various features have on classification. Using the Iris dataset, we first compare HOTS to Gini Importance, Partial Dependence Plots, and Permutation Importance, and demonstrate how HOTS resolves the weaknesses present in these methods. We then show how HOTS can be utilized in high dimensional neuroscientific data, by taking 60 Schizophrenic subjects and applying the method to determine which brain regions were most important for classification of schizophrenia as determined by the PANSS. HOTS effectively replicated and supported the findings of Gini importance, Partial Dependence Plots and Permutation importance within the Iris dataset. When applied to the schizophrenic brain dataset, HOTS was able to resolve the top 10 most important features for classification, as well as their directionality for classification and magnitude compared to other features. Cross-validation supported that these same 10 features were consistently used in the decision-making process across multiple trees, and these features were localised primarily to the occipital and parietal cortices, commonly disturbed brain regions in those with Schizophrenia. It is imperative that a methodology is developed that is able to handle the demands of working with large datasets that contain a large number of features. HOTS represents a unique way to investigate both the directionality and magnitude of feature importance when working at scale with boosted-tree modelling.
△ Less
Submitted 7 April, 2021;
originally announced April 2021.
-
Generalizing Variational Autoencoders with Hierarchical Empirical Bayes
Authors:
Wei Cheng,
Gregory Darnell,
Sohini Ramachandran,
Lorin Crawford
Abstract:
Variational Autoencoders (VAEs) have experienced recent success as data-generating models by using simple architectures that do not require significant fine-tuning of hyperparameters. However, VAEs are known to suffer from over-regularization which can lead to failure to escape local maxima. This phenomenon, known as posterior collapse, prevents learning a meaningful latent encoding of the data. R…
▽ More
Variational Autoencoders (VAEs) have experienced recent success as data-generating models by using simple architectures that do not require significant fine-tuning of hyperparameters. However, VAEs are known to suffer from over-regularization which can lead to failure to escape local maxima. This phenomenon, known as posterior collapse, prevents learning a meaningful latent encoding of the data. Recent methods have mitigated this issue by deterministically moment-matching an aggregated posterior distribution to an aggregate prior. However, abandoning a probabilistic framework (and thus relying on point estimates) can both lead to a discontinuous latent space and generate unrealistic samples. Here we present Hierarchical Empirical Bayes Autoencoder (HEBAE), a computationally stable framework for probabilistic generative models. Our key contributions are two-fold. First, we make gains by placing a hierarchical prior over the encoding distribution, enabling us to adaptively balance the trade-off between minimizing the reconstruction loss function and avoiding over-regularization. Second, we show that assuming a general dependency structure between variables in the latent space produces better convergence onto the mean-field assumption for improved posterior inference. Overall, HEBAE is more robust to a wide-range of hyperparameter initializations than an analogous VAE. Using data from MNIST and CelebA, we illustrate the ability of HEBAE to generate higher quality samples based on FID score than existing autoencoder-based approaches.
△ Less
Submitted 20 July, 2020;
originally announced July 2020.
-
Interpreting Deep Neural Networks Through Variable Importance
Authors:
Jonathan Ish-Horowicz,
Dana Udwin,
Seth Flaxman,
Sarah Filippi,
Lorin Crawford
Abstract:
While the success of deep neural networks (DNNs) is well-established across a variety of domains, our ability to explain and interpret these methods is limited. Unlike previously proposed local methods which try to explain particular classification decisions, we focus on global interpretability and ask a universally applicable question: given a trained model, which features are the most important?…
▽ More
While the success of deep neural networks (DNNs) is well-established across a variety of domains, our ability to explain and interpret these methods is limited. Unlike previously proposed local methods which try to explain particular classification decisions, we focus on global interpretability and ask a universally applicable question: given a trained model, which features are the most important? In the context of neural networks, a feature is rarely important on its own, so our strategy is specifically designed to leverage partial covariance structures and incorporate variable dependence into feature ranking. Our methodological contributions in this paper are two-fold. First, we propose an effect size analogue for DNNs that is appropriate for applications with highly collinear predictors (ubiquitous in computer vision). Second, we extend the recently proposed "RelATive cEntrality" (RATE) measure (Crawford et al., 2019) to the Bayesian deep learning setting. RATE applies an information theoretic criterion to the posterior distribution of effect sizes to assess feature significance. We apply our framework to three broad application areas: computer vision, natural language processing, and social science.
△ Less
Submitted 28 April, 2020; v1 submitted 28 January, 2019;
originally announced January 2019.
-
Variable Prioritization in Nonlinear Black Box Methods: A Genetic Association Case Study
Authors:
Lorin Crawford,
Seth R. Flaxman,
Daniel E. Runcie,
Mike West
Abstract:
The central aim in this paper is to address variable selection questions in nonlinear and nonparametric regression. Motivated by statistical genetics, where nonlinear interactions are of particular interest, we introduce a novel and interpretable way to summarize the relative importance of predictor variables. Methodologically, we develop the "RelATive cEntrality" (RATE) measure to prioritize cand…
▽ More
The central aim in this paper is to address variable selection questions in nonlinear and nonparametric regression. Motivated by statistical genetics, where nonlinear interactions are of particular interest, we introduce a novel and interpretable way to summarize the relative importance of predictor variables. Methodologically, we develop the "RelATive cEntrality" (RATE) measure to prioritize candidate genetic variants that are not just marginally important, but whose associations also stem from significant covarying relationships with other variants in the data. We illustrate RATE through Bayesian Gaussian process regression, but the methodological innovations apply to other "black box" methods. It is known that nonlinear models often exhibit greater predictive accuracy than linear models, particularly for phenotypes generated by complex genetic architectures. With detailed simulations and two real data association map** studies, we show that applying RATE enables an explanation for this improved performance.
△ Less
Submitted 26 August, 2018; v1 submitted 22 January, 2018;
originally announced January 2018.
-
Predicting Clinical Outcomes in Glioblastoma: An Application of Topological and Functional Data Analysis
Authors:
Lorin Crawford,
Anthea Monod,
Andrew X. Chen,
Sayan Mukherjee,
Raúl Rabadán
Abstract:
Glioblastoma multiforme (GBM) is an aggressive form of human brain cancer that is under active study in the field of cancer biology. Its rapid progression and the relative time cost of obtaining molecular data make other readily-available forms of data, such as images, an important resource for actionable measures in patients. Our goal is to utilize information given by medical images taken from G…
▽ More
Glioblastoma multiforme (GBM) is an aggressive form of human brain cancer that is under active study in the field of cancer biology. Its rapid progression and the relative time cost of obtaining molecular data make other readily-available forms of data, such as images, an important resource for actionable measures in patients. Our goal is to utilize information given by medical images taken from GBM patients in statistical settings. To do this, we design a novel statistic---the smooth Euler characteristic transform (SECT)---that quantifies magnetic resonance images (MRIs) of tumors. Due to its well-defined inner product structure, the SECT can be used in a wider range of functional and nonparametric modeling approaches than other previously proposed topological summary statistics. When applied to a cohort of GBM patients, we find that the SECT is a better predictor of clinical outcomes than both existing tumor shape quantifications and common molecular assays. Specifically, we demonstrate that SECT features alone explain more of the variance in GBM patient survival than gene expression, volumetric features, and morphometric features. The main takeaways from our findings are thus twofold. First, they suggest that images contain valuable information that can play an important role in clinical prognosis and other medical decisions. Second, they show that the SECT is a viable tool for the broader study of medical imaging informatics.
△ Less
Submitted 12 September, 2019; v1 submitted 21 November, 2016;
originally announced November 2016.
-
Bayesian Approximate Kernel Regression with Variable Selection
Authors:
Lorin Crawford,
Kris C. Wood,
Xiang Zhou,
Sayan Mukherjee
Abstract:
Nonlinear kernel regression models are often used in statistics and machine learning because they are more accurate than linear models. Variable selection for kernel regression models is a challenge partly because, unlike the linear regression setting, there is no clear concept of an effect size for regression coefficients. In this paper, we propose a novel framework that provides an effect size a…
▽ More
Nonlinear kernel regression models are often used in statistics and machine learning because they are more accurate than linear models. Variable selection for kernel regression models is a challenge partly because, unlike the linear regression setting, there is no clear concept of an effect size for regression coefficients. In this paper, we propose a novel framework that provides an effect size analog of each explanatory variable for Bayesian kernel regression models when the kernel is shift-invariant --- for example, the Gaussian kernel. We use function analytic properties of shift-invariant reproducing kernel Hilbert spaces (RKHS) to define a linear vector space that: (i) captures nonlinear structure, and (ii) can be projected onto the original explanatory variables. The projection onto the original explanatory variables serves as an analog of effect sizes. The specific function analytic property we use is that shift-invariant kernel functions can be approximated via random Fourier bases. Based on the random Fourier expansion we propose a computationally efficient class of Bayesian approximate kernel regression (BAKR) models for both nonlinear regression and binary classification for which one can compute an analog of effect sizes. We illustrate the utility of BAKR by examining two important problems in statistical genetics: genomic selection (i.e. phenotypic prediction) and association map** (i.e. inference of significant variants or loci). State-of-the-art methods for genomic selection and association map** are based on kernel regression and linear models, respectively. BAKR is the first method that is competitive in both settings.
△ Less
Submitted 9 June, 2017; v1 submitted 5 August, 2015;
originally announced August 2015.
