-
Baseline Drift Tolerant Signal Encoding for ECG Classification with Deep Learning
Authors:
Robert O Shea,
Prabodh Katti,
Bipin Rajendran
Abstract:
Common artefacts such as baseline drift, rescaling, and noise critically limit the performance of machine learningbased automated ECG analysis and interpretation. This study proposes Derived Peak (DP) encoding, a non-parametric method that generates signed spikes corresponding to zero crossings of the signals first and second-order time derivatives. Notably, DP encoding is invariant to shift and s…
▽ More
Common artefacts such as baseline drift, rescaling, and noise critically limit the performance of machine learningbased automated ECG analysis and interpretation. This study proposes Derived Peak (DP) encoding, a non-parametric method that generates signed spikes corresponding to zero crossings of the signals first and second-order time derivatives. Notably, DP encoding is invariant to shift and scaling artefacts, and its implementation is further simplified by the absence of userdefined parameters. DP encoding was used to encode the 12-lead ECG data from the PTB-XL dataset (n=18,869 participants) and was fed to 1D-ResNet-18 models trained to identify myocardial infarction, conductive deficits and ST-segment abnormalities. Robustness to artefacts was assessed by corrupting ECG data with sinusoidal baseline drift, shift, rescaling and noise, before encoding. The addition of these artefacts resulted in a significant drop in accuracy for seven other methods from prior art, while DP encoding maintained a baseline AUC of 0.88 under drift, shift and rescaling. DP achieved superior performance to unencoded inputs in the presence of shift (AUC under 1mV shift: 0.91 vs 0.62), and rescaling artefacts (AUC 0.91 vs 0.79). Thus, DP encoding is a simple method by which robustness to common ECG artefacts may be improved for automated ECG analysis and interpretation.
△ Less
Submitted 26 April, 2024;
originally announced May 2024.
-
Stochastic Spiking Attention: Accelerating Attention with Stochastic Computing in Spiking Networks
Authors:
Zihang Song,
Prabodh Katti,
Osvaldo Simeone,
Bipin Rajendran
Abstract:
Spiking Neural Networks (SNNs) have been recently integrated into Transformer architectures due to their potential to reduce computational demands and to improve power efficiency. Yet, the implementation of the attention mechanism using spiking signals on general-purpose computing platforms remains inefficient. In this paper, we propose a novel framework leveraging stochastic computing (SC) to eff…
▽ More
Spiking Neural Networks (SNNs) have been recently integrated into Transformer architectures due to their potential to reduce computational demands and to improve power efficiency. Yet, the implementation of the attention mechanism using spiking signals on general-purpose computing platforms remains inefficient. In this paper, we propose a novel framework leveraging stochastic computing (SC) to effectively execute the dot-product attention for SNN-based Transformers. We demonstrate that our approach can achieve high classification accuracy ($83.53\%$) on CIFAR-10 within 10 time steps, which is comparable to the performance of a baseline artificial neural network implementation ($83.66\%$). We estimate that the proposed SC approach can lead to over $6.3\times$ reduction in computing energy and $1.7\times$ reduction in memory access costs for a digital CMOS-based ASIC design. We experimentally validate our stochastic attention block design through an FPGA implementation, which is shown to achieve $48\times$ lower latency as compared to a GPU implementation, while consuming $15\times$ less power.
△ Less
Submitted 14 February, 2024;
originally announced February 2024.
-
Bayesian Inference Accelerator for Spiking Neural Networks
Authors:
Prabodh Katti,
Anagha Nimbekar,
Chen Li,
Amit Acharyya,
Bashir M. Al-Hashimi,
Bipin Rajendran
Abstract:
Bayesian neural networks offer better estimates of model uncertainty compared to frequentist networks. However, inference involving Bayesian models requires multiple instantiations or sampling of the network parameters, requiring significant computational resources. Compared to traditional deep learning networks, spiking neural networks (SNNs) have the potential to reduce computational area and po…
▽ More
Bayesian neural networks offer better estimates of model uncertainty compared to frequentist networks. However, inference involving Bayesian models requires multiple instantiations or sampling of the network parameters, requiring significant computational resources. Compared to traditional deep learning networks, spiking neural networks (SNNs) have the potential to reduce computational area and power, thanks to their event-driven and spike-based computational framework. Most works in literature either address frequentist SNN models or non-spiking Bayesian neural networks. In this work, we demonstrate an optimization framework for develo** and implementing efficient Bayesian SNNs in hardware by additionally restricting network weights to be binary-valued to further decrease power and area consumption. We demonstrate accuracies comparable to Bayesian binary networks with full-precision Bernoulli parameters, while requiring up to $25\times$ less spikes than equivalent binary SNN implementations. We show the feasibility of the design by map** it onto Zynq-7000, a lightweight SoC, and achieve a $6.5 \times$ improvement in GOPS/DSP while utilizing up to 30 times less power compared to the state-of-the-art.
△ Less
Submitted 27 January, 2024;
originally announced January 2024.
-
Bayesian Inference on Binary Spiking Networks Leveraging Nanoscale Device Stochasticity
Authors:
Prabodh Katti,
Nicolas Skatchkovsky,
Osvaldo Simeone,
Bipin Rajendran,
Bashir M. Al-Hashimi
Abstract:
Bayesian Neural Networks (BNNs) can overcome the problem of overconfidence that plagues traditional frequentist deep neural networks, and are hence considered to be a key enabler for reliable AI systems. However, conventional hardware realizations of BNNs are resource intensive, requiring the implementation of random number generators for synaptic sampling. Owing to their inherent stochasticity du…
▽ More
Bayesian Neural Networks (BNNs) can overcome the problem of overconfidence that plagues traditional frequentist deep neural networks, and are hence considered to be a key enabler for reliable AI systems. However, conventional hardware realizations of BNNs are resource intensive, requiring the implementation of random number generators for synaptic sampling. Owing to their inherent stochasticity during programming and read operations, nanoscale memristive devices can be directly leveraged for sampling, without the need for additional hardware resources. In this paper, we introduce a novel Phase Change Memory (PCM)-based hardware implementation for BNNs with binary synapses. The proposed architecture consists of separate weight and noise planes, in which PCM cells are configured and operated to represent the nominal values of weights and to generate the required noise for sampling, respectively. Using experimentally observed PCM noise characteristics, for the exemplary Breast Cancer Dataset classification problem, we obtain hardware accuracy and expected calibration error matching that of an 8-bit fixed-point (FxP8) implementation, with projected savings of over 9$\times$ in terms of core area transistor count.
△ Less
Submitted 2 February, 2023;
originally announced February 2023.
-
Signal Fluctuation Sensitivity: an improved metric for optimizing detection of resting-state fMRI networks
Authors:
D. J. DeDora,
S. Nedic,
P. Katti,
S. Arnab,
L. L. Wald,
A. Takahashi,
K. R. A. Van Dijk,
H. H. Strey,
L. R. Mujica-Parodi
Abstract:
Task-free connectivity analyses have emerged as a powerful tool in functional neuroimaging. Because the cross-correlations that underlie connectivity measures are sensitive to distortion of time-series, here we used a novel dynamic phantom to provide a ground truth for dynamic fidelity between blood oxygen level dependent (BOLD)-like inputs and fMRI outputs. We found that the de facto quality-metr…
▽ More
Task-free connectivity analyses have emerged as a powerful tool in functional neuroimaging. Because the cross-correlations that underlie connectivity measures are sensitive to distortion of time-series, here we used a novel dynamic phantom to provide a ground truth for dynamic fidelity between blood oxygen level dependent (BOLD)-like inputs and fMRI outputs. We found that the de facto quality-metric for task-free fMRI, temporal signal to noise ratio (tSNR), correlated inversely with dynamic fidelity; thus, studies optimized for tSNR actually produced time-series that showed the greatest distortion of signal dynamics. Instead, the phantom showed that dynamic fidelity is reasonably approximated by a measure that, unlike tSNR, dissociates signal dynamics from scanner artifact. We then tested this measure, signal fluctuation sensitivity (SFS), against human resting-state data. As predicted by the phantom, SFS--and not tSNR--is associated with enhanced sensitivity to both local and long-range connectivity within the brain's default mode network.
△ Less
Submitted 13 November, 2015;
originally announced November 2015.
-
Effect of node mobility on AOMDV protocol in MANET
Authors:
Indrani Das,
D. K Lobiyal,
C. P Katti
Abstract:
In this paper, we have analyzed the effect of node mobility on the performance of AOMDV multipath routing protocol. This routing protocol in ad hoc network has been analyzed with random way point mobility model only. This is not sufficient to evaluate the behavior of a routing protocol. Therefore, in this paper, we have considered Random waypoint, Random Direction and Probabilistic Random Walk mob…
▽ More
In this paper, we have analyzed the effect of node mobility on the performance of AOMDV multipath routing protocol. This routing protocol in ad hoc network has been analyzed with random way point mobility model only. This is not sufficient to evaluate the behavior of a routing protocol. Therefore, in this paper, we have considered Random waypoint, Random Direction and Probabilistic Random Walk mobility Model for performance analysis of AOMDV protocol. The result reveals that packet delivery ratio decreases with the increasing node mobility for all mobility models. Also, average end-to-end delay is also vary with varying node speed, initially upto 20 nodes in all mobility models delay is minimum.
△ Less
Submitted 29 June, 2014;
originally announced July 2014.
