-
Supervised Radio Frequency Interference Detection with SNNs
Authors:
Nicholas J. Pritchard,
Andreas Wicenec,
Mohammed Bennamoun,
Richard Dodson
Abstract:
Radio Frequency Interference (RFI) poses a significant challenge in radio astronomy, arising from terrestrial and celestial sources, disrupting observations conducted by radio telescopes. Addressing RFI involves intricate heuristic algorithms, manual examination, and, increasingly, machine learning methods. Given the dynamic and temporal nature of radio astronomy observations, Spiking Neural Netwo…
▽ More
Radio Frequency Interference (RFI) poses a significant challenge in radio astronomy, arising from terrestrial and celestial sources, disrupting observations conducted by radio telescopes. Addressing RFI involves intricate heuristic algorithms, manual examination, and, increasingly, machine learning methods. Given the dynamic and temporal nature of radio astronomy observations, Spiking Neural Networks (SNNs) emerge as a promising approach. In this study, we cast RFI detection as a supervised multi-variate time-series segmentation problem. Notably, our investigation explores the encoding of radio astronomy visibility data for SNN inference, considering six encoding schemes: rate, latency, delta-modulation, and three variations of the step-forward algorithm. We train a small two-layer fully connected SNN on simulated data derived from the Hydrogen Epoch of Reionization Array (HERA) telescope and perform extensive hyper-parameter optimization. Results reveal that latency encoding exhibits superior performance, achieving a per-pixel accuracy of 98.8% and an f1-score of 0.761. Remarkably, these metrics approach those of contemporary RFI detection algorithms, notwithstanding the simplicity and compactness of our proposed network architecture. This study underscores the potential of RFI detection as a benchmark problem for SNN researchers, emphasizing the efficacy of SNNs in addressing complex time-series segmentation tasks in radio astronomy.
△ Less
Submitted 10 June, 2024;
originally announced June 2024.
-
Formal Definition and Implementation of Reproducibility Tenets for Computational Workflows
Authors:
Nicholas J. Pritchard,
Andreas Wicenec
Abstract:
Computational workflow management systems power contemporary data-intensive sciences. The slowly resolving reproducibility crisis presents both a sobering warning and an opportunity to iterate on what science and data processing entails. The Square Kilometre Array (SKA), the world's largest radio telescope, is among the most extensive scientific projects underway and presents grand scientific coll…
▽ More
Computational workflow management systems power contemporary data-intensive sciences. The slowly resolving reproducibility crisis presents both a sobering warning and an opportunity to iterate on what science and data processing entails. The Square Kilometre Array (SKA), the world's largest radio telescope, is among the most extensive scientific projects underway and presents grand scientific collaboration and data-processing challenges. This work presents a scale and system-agnostic computational workflow model and extends five well-known reproducibility tenets into seven defined for our workflow model. Subsequent implementation of these definitions, powered by blockchain primitives, into the Data Activated Flow Graph Engine (DALiuGE), a workflow management system for the SKA, demonstrates the possibility of facilitating automatic formal verification of scientific quality in amortized constant time. We validate our approach with a simple yet representative astronomical processing task; filtering a noisy signal with a lowpass filter with both CPU and GPU methods. Our framework illuminates otherwise obscure scientific discrepancies and similarities between principally identical workflow executions.
△ Less
Submitted 3 June, 2024;
originally announced June 2024.
-
Toward the Identifiability of Comparative Deep Generative Models
Authors:
Romain Lopez,
Jan-Christian Huetter,
Ehsan Hajiramezanali,
Jonathan Pritchard,
Aviv Regev
Abstract:
Deep Generative Models (DGMs) are versatile tools for learning data representations while adequately incorporating domain knowledge such as the specification of conditional probability distributions. Recently proposed DGMs tackle the important task of comparing data sets from different sources. One such example is the setting of contrastive analysis that focuses on describing patterns that are enr…
▽ More
Deep Generative Models (DGMs) are versatile tools for learning data representations while adequately incorporating domain knowledge such as the specification of conditional probability distributions. Recently proposed DGMs tackle the important task of comparing data sets from different sources. One such example is the setting of contrastive analysis that focuses on describing patterns that are enriched in a target data set compared to a background data set. The practical deployment of those models often assumes that DGMs naturally infer interpretable and modular latent representations, which is known to be an issue in practice. Consequently, existing methods often rely on ad-hoc regularization schemes, although without any theoretical grounding. Here, we propose a theory of identifiability for comparative DGMs by extending recent advances in the field of non-linear independent component analysis. We show that, while these models lack identifiability across a general class of mixing functions, they surprisingly become identifiable when the mixing function is piece-wise affine (e.g., parameterized by a ReLU neural network). We also investigate the impact of model misspecification, and empirically show that previously proposed regularization techniques for fitting comparative DGMs help with identifiability when the number of latent variables is not known in advance. Finally, we introduce a novel methodology for fitting comparative DGMs that improves the treatment of multiple data sources via multi-objective optimization and that helps adjust the hyperparameter for the regularization in an interpretable manner, using constrained optimization. We empirically validate our theory and new methodology using simulated data as well as a recent data set of genetic perturbations in cells profiled via single-cell RNA sequencing.
△ Less
Submitted 29 January, 2024;
originally announced January 2024.
-
RFI Detection with Spiking Neural Networks
Authors:
Nicholas J. Pritchard,
Andreas Wicenec,
Mohammed Bennamoun,
Richard Dodson
Abstract:
Detecting and mitigating Radio Frequency Interference (RFI) is critical for enabling and maximising the scientific output of radio telescopes. The emergence of machine learning methods has led to their application in radio astronomy, and in RFI detection. Spiking Neural Networks (SNNs), inspired by biological systems, are well-suited for processing spatio-temporal data. This study introduces the f…
▽ More
Detecting and mitigating Radio Frequency Interference (RFI) is critical for enabling and maximising the scientific output of radio telescopes. The emergence of machine learning methods has led to their application in radio astronomy, and in RFI detection. Spiking Neural Networks (SNNs), inspired by biological systems, are well-suited for processing spatio-temporal data. This study introduces the first exploratory application of SNNs to an astronomical data-processing task, specifically RFI detection. We adapt the nearest-latent-neighbours (NLN) algorithm and auto-encoder architecture proposed by previous authors to SNN execution by direct ANN2SNN conversion, enabling simplified downstream RFI detection by sampling the naturally varying latent space from the internal spiking neurons. Our subsequent evaluation aims to determine whether SNNs are viable for future RFI detection schemes. We evaluate detection performance with the simulated HERA telescope and hand-labelled LOFAR observation dataset the original authors provided. We additionally evaluate detection performance with a new MeerKAT-inspired simulation dataset that provides a technical challenge for machine-learnt RFI detection methods. This dataset focuses on satellite-based RFI, an increasingly important class of RFI and is an additional contribution. Our approach remains competitive with existing methods in AUROC, AUPRC and F1 scores for the HERA dataset but exhibits difficulty in the LOFAR and Tabascal datasets. Our method maintains this accuracy while completely removing the compute and memory-intense latent sampling step found in NLN. This work demonstrates the viability of SNNs as a promising avenue for machine-learning-based RFI detection in radio telescopes by establishing a minimal performance baseline on traditional and nascent satellite-based RFI sources and is the first work to our knowledge to apply SNNs in astronomy.
△ Less
Submitted 22 March, 2024; v1 submitted 24 November, 2023;
originally announced November 2023.
-
A Bibliometric Review of Neuromorphic Computing and Spiking Neural Networks
Authors:
Nicholas J. Pritchard,
Andreas Wicenec,
Mohammed Bennamoun,
Richard Dodson
Abstract:
Neuromorphic computing and spiking neural networks aim to leverage biological inspiration to achieve greater energy efficiency and computational power beyond traditional von Neumann architectured machines. In particular, spiking neural networks hold the potential to advance artificial intelligence as the basis of third-generation neural networks. Aided by developments in memristive and compute-in-…
▽ More
Neuromorphic computing and spiking neural networks aim to leverage biological inspiration to achieve greater energy efficiency and computational power beyond traditional von Neumann architectured machines. In particular, spiking neural networks hold the potential to advance artificial intelligence as the basis of third-generation neural networks. Aided by developments in memristive and compute-in-memory technologies, neuromorphic computing hardware is transitioning from laboratory prototype devices to commercial chipsets; ushering in an era of low-power computing. As a nexus of biological, computing, and material sciences, the literature surrounding these concepts is vast, varied, and somewhat distinct from artificial neural network sources. This article uses bibliometric analysis to survey the last 22 years of literature, seeking to establish trends in publication and citation volumes (III-A); analyze impactful authors, journals and institutions (III-B); generate an introductory reading list (III-C); survey collaborations between countries, institutes and authors (III-D), and to analyze changes in research topics over the years (III-E). We analyze literature data from the Clarivate Web of Science using standard bibliometric methods. By briefly introducing the most impactful literature in this field from the last two decades, we encourage AI practitioners and researchers to look beyond contemporary technologies toward a potentially spiking future of computing.
△ Less
Submitted 13 April, 2023;
originally announced April 2023.
-
Learning Causal Representations of Single Cells via Sparse Mechanism Shift Modeling
Authors:
Romain Lopez,
Nataša Tagasovska,
Stephen Ra,
Kyunghyn Cho,
Jonathan K. Pritchard,
Aviv Regev
Abstract:
Latent variable models such as the Variational Auto-Encoder (VAE) have become a go-to tool for analyzing biological data, especially in the field of single-cell genomics. One remaining challenge is the interpretability of latent variables as biological processes that define a cell's identity. Outside of biological applications, this problem is commonly referred to as learning disentangled represen…
▽ More
Latent variable models such as the Variational Auto-Encoder (VAE) have become a go-to tool for analyzing biological data, especially in the field of single-cell genomics. One remaining challenge is the interpretability of latent variables as biological processes that define a cell's identity. Outside of biological applications, this problem is commonly referred to as learning disentangled representations. Although several disentanglement-promoting variants of the VAE were introduced, and applied to single-cell genomics data, this task has been shown to be infeasible from independent and identically distributed measurements, without additional structure. Instead, recent methods propose to leverage non-stationary data, as well as the sparse mechanism shift assumption in order to learn disentangled representations with a causal semantic. Here, we extend the application of these methodological advances to the analysis of single-cell genomics data with genetic or chemical perturbations. More precisely, we propose a deep generative model of single-cell gene expression data for which each perturbation is treated as a stochastic intervention targeting an unknown, but sparse, subset of latent variables. We benchmark these methods on simulated single-cell data to evaluate their performance at latent units recovery, causal target identification and out-of-domain generalization. Finally, we apply those approaches to two real-world large-scale gene perturbation data sets and find that models that exploit the sparse mechanism shift hypothesis surpass contemporary methods on a transfer learning task. We implement our new model and benchmarks using the scvi-tools library, and release it as open-source software at https://github.com/Genentech/sVAE.
△ Less
Submitted 16 February, 2023; v1 submitted 7 November, 2022;
originally announced November 2022.
-
Large-Scale Differentiable Causal Discovery of Factor Graphs
Authors:
Romain Lopez,
Jan-Christian Hütter,
Jonathan K. Pritchard,
Aviv Regev
Abstract:
A common theme in causal inference is learning causal relationships between observed variables, also known as causal discovery. This is usually a daunting task, given the large number of candidate causal graphs and the combinatorial nature of the search space. Perhaps for this reason, most research has so far focused on relatively small causal graphs, with up to hundreds of nodes. However, recent…
▽ More
A common theme in causal inference is learning causal relationships between observed variables, also known as causal discovery. This is usually a daunting task, given the large number of candidate causal graphs and the combinatorial nature of the search space. Perhaps for this reason, most research has so far focused on relatively small causal graphs, with up to hundreds of nodes. However, recent advances in fields like biology enable generating experimental data sets with thousands of interventions followed by rich profiling of thousands of variables, raising the opportunity and urgent need for large causal graph models. Here, we introduce the notion of factor directed acyclic graphs (f-DAGs) as a way to restrict the search space to non-linear low-rank causal interaction models. Combining this novel structural assumption with recent advances that bridge the gap between causal discovery and continuous optimization, we achieve causal discovery on thousands of variables. Additionally, as a model for the impact of statistical noise on this estimation procedure, we study a model of edge perturbations of the f-DAG skeleton based on random graphs and quantify the effect of such perturbations on the f-DAG rank. This theoretical analysis suggests that the set of candidate f-DAGs is much smaller than the whole DAG space and thus may be more suitable as a search space in the high-dimensional regime where the underlying skeleton is hard to assess. We propose Differentiable Causal Discovery of Factor Graphs (DCD-FG), a scalable implementation of -DAG constrained causal discovery for high-dimensional interventional data. DCD-FG uses a Gaussian non-linear low-rank structural equation model and shows significant improvements compared to state-of-the-art methods in both simulations as well as a recent large-scale single-cell RNA sequencing data set with hundreds of genetic interventions.
△ Less
Submitted 7 October, 2022; v1 submitted 15 June, 2022;
originally announced June 2022.
-
Guided Weak Supervision for Action Recognition with Scarce Data to Assess Skills of Children with Autism
Authors:
Prashant Pandey,
Prathosh AP,
Manu Kohli,
Josh Pritchard
Abstract:
Diagnostic and intervention methodologies for skill assessment of autism typically requires a clinician repetitively initiating several stimuli and recording the child's response. In this paper, we propose to automate the response measurement through video recording of the scene following the use of Deep Neural models for human action recognition from videos. However, supervised learning of neural…
▽ More
Diagnostic and intervention methodologies for skill assessment of autism typically requires a clinician repetitively initiating several stimuli and recording the child's response. In this paper, we propose to automate the response measurement through video recording of the scene following the use of Deep Neural models for human action recognition from videos. However, supervised learning of neural networks demand large amounts of annotated data that are hard to come by. This issue is addressed by leveraging the `similarities' between the action categories in publicly available large-scale video action (source) datasets and the dataset of interest. A technique called guided weak supervision is proposed, where every class in the target data is matched to a class in the source data using the principle of posterior likelihood maximization. Subsequently, classifier on the target data is re-trained by augmenting samples from the matched source classes, along with a new loss encouraging inter-class separability. The proposed method is evaluated on two skill assessment autism datasets, SSBD and a real world Autism dataset comprising 37 children of different ages and ethnicity who are diagnosed with autism. Our proposed method is found to improve the performance of the state-of-the-art multi-class human action recognition models in-spite of supervision with scarce data.
△ Less
Submitted 28 January, 2020; v1 submitted 11 November, 2019;
originally announced November 2019.
