-
Seizure Detection and Prediction by Parallel Memristive Convolutional Neural Networks
Authors:
Chenqi Li,
Corey Lammie,
Xuening Dong,
Amirali Amirsoleimani,
Mostafa Rahimi Azghadi,
Roman Genov
Abstract:
During the past two decades, epileptic seizure detection and prediction algorithms have evolved rapidly. However, despite significant performance improvements, their hardware implementation using conventional technologies, such as Complementary Metal-Oxide-Semiconductor (CMOS), in power and area-constrained settings remains a challenging task; especially when many recording channels are used. In t…
▽ More
During the past two decades, epileptic seizure detection and prediction algorithms have evolved rapidly. However, despite significant performance improvements, their hardware implementation using conventional technologies, such as Complementary Metal-Oxide-Semiconductor (CMOS), in power and area-constrained settings remains a challenging task; especially when many recording channels are used. In this paper, we propose a novel low-latency parallel Convolutional Neural Network (CNN) architecture that has between 2-2,800x fewer network parameters compared to SOTA CNN architectures and achieves 5-fold cross validation accuracy of 99.84% for epileptic seizure detection, and 99.01% and 97.54% for epileptic seizure prediction, when evaluated using the University of Bonn Electroencephalogram (EEG), CHB-MIT and SWEC-ETHZ seizure datasets, respectively. We subsequently implement our network onto analog crossbar arrays comprising Resistive Random-Access Memory (RRAM) devices, and provide a comprehensive benchmark by simulating, laying out, and determining hardware requirements of the CNN component of our system. To the best of our knowledge, we are the first to parallelize the execution of convolution layer kernels on separate analog crossbars to enable 2 orders of magnitude reduction in latency compared to SOTA hybrid Memristive-CMOS DL accelerators. Furthermore, we investigate the effects of non-idealities on our system and investigate Quantization Aware Training (QAT) to mitigate the performance degradation due to low ADC/DAC resolution. Finally, we propose a stuck weight offsetting methodology to mitigate performance degradation due to stuck RON/ROFF memristor weights, recovering up to 32% accuracy, without requiring retraining. The CNN component of our platform is estimated to consume approximately 2.791W of power while occupying an area of 31.255mm$^2$ in a 22nm FDSOI CMOS process.
△ Less
Submitted 20 June, 2022;
originally announced June 2022.
-
Toward A Formalized Approach for Spike Sorting Algorithms and Hardware Evaluation
Authors:
Tim Zhang,
Corey Lammie,
Mostafa Rahimi Azghadi,
Amirali Amirsoleimani,
Majid Ahmadi,
Roman Genov
Abstract:
Spike sorting algorithms are used to separate extracellular recordings of neuronal populations into single-unit spike activities. The development of customized hardware implementing spike sorting algorithms is burgeoning. However, there is a lack of a systematic approach and a set of standardized evaluation criteria to facilitate direct comparison of both software and hardware implementations. In…
▽ More
Spike sorting algorithms are used to separate extracellular recordings of neuronal populations into single-unit spike activities. The development of customized hardware implementing spike sorting algorithms is burgeoning. However, there is a lack of a systematic approach and a set of standardized evaluation criteria to facilitate direct comparison of both software and hardware implementations. In this paper, we formalize a set of standardized criteria and a publicly available synthetic dataset entitled Synthetic Simulations Of Extracellular Recordings (SSOER), which was constructed by aggregating existing synthetic datasets with varying Signal-To-Noise Ratios (SNRs). Furthermore, we present a benchmark for future comparison, and use our criteria to evaluate a simulated Resistive Random-Access Memory (RRAM) In-Memory Computing (IMC) system using the Discrete Wavelet Transform (DWT) for feature extraction. Our system consumes approximately (per channel) 10.72mW and occupies an area of 0.66mm$^2$ in a 22nm FDSOI Complementary Metal-Oxide-Semiconductor (CMOS) process.
△ Less
Submitted 13 May, 2022;
originally announced May 2022.
-
PRUNIX: Non-Ideality Aware Convolutional Neural Network Pruning for Memristive Accelerators
Authors:
Ali Alshaarawy,
Amirali Amirsoleimani,
Roman Genov
Abstract:
In this work, PRUNIX, a framework for training and pruning convolutional neural networks is proposed for deployment on memristor crossbar based accelerators. PRUNIX takes into account the numerous non-ideal effects of memristor crossbars including weight quantization, state-drift, aging and stuck-at-faults. PRUNIX utilises a novel Group Sawtooth Regularization intended to improve non-ideality tole…
▽ More
In this work, PRUNIX, a framework for training and pruning convolutional neural networks is proposed for deployment on memristor crossbar based accelerators. PRUNIX takes into account the numerous non-ideal effects of memristor crossbars including weight quantization, state-drift, aging and stuck-at-faults. PRUNIX utilises a novel Group Sawtooth Regularization intended to improve non-ideality tolerance as well as sparsity, and a novel Adaptive Pruning Algorithm (APA) intended to minimise accuracy loss by considering the sensitivity of different layers of a CNN to pruning. We compare our regularization and pruning methods with other standards on multiple CNN architectures, and observe an improvement of 13% test accuracy when quantization and other non-ideal effects are accounted for with an overall sparsity of 85%, which is similar to other methods
△ Less
Submitted 3 February, 2022;
originally announced February 2022.
-
SDEX: Monte Carlo Simulation of Stochastic Differential Equations on Memristor Crossbars
Authors:
Louis Primeau,
Amirali Amirsoleimani,
Roman Genov
Abstract:
Here we present stochastic differential equations (SDEs) on a memristor crossbar, where the source of gaussian noise is derived from the random conductance due to ion drift in the devices during programming. We examine the effects of line resistance on the generation of normal random vectors, showing the skew and kurtosis are within acceptable bounds. We then show the implementation of a stochasti…
▽ More
Here we present stochastic differential equations (SDEs) on a memristor crossbar, where the source of gaussian noise is derived from the random conductance due to ion drift in the devices during programming. We examine the effects of line resistance on the generation of normal random vectors, showing the skew and kurtosis are within acceptable bounds. We then show the implementation of a stochastic differential equation solver for the Black-Scholes SDE, and compare the distribution with the analytic solution. We determine that the random number generation works as intended, and calculate the energy cost of the simulation.
△ Less
Submitted 2 February, 2022;
originally announced February 2022.
-
HYPERLOCK: In-Memory Hyperdimensional Encryption in Memristor Crossbar Array
Authors:
Jack Cai,
Amirali Amirsoleimani,
Roman Genov
Abstract:
We present a novel cryptography architecture based on memristor crossbar array, binary hypervectors, and neural network. Utilizing the stochastic and unclonable nature of memristor crossbar and error tolerance of binary hypervectors and neural network, implementation of the algorithm on memristor crossbar simulation is made possible. We demonstrate that with an increasing dimension of the binary h…
▽ More
We present a novel cryptography architecture based on memristor crossbar array, binary hypervectors, and neural network. Utilizing the stochastic and unclonable nature of memristor crossbar and error tolerance of binary hypervectors and neural network, implementation of the algorithm on memristor crossbar simulation is made possible. We demonstrate that with an increasing dimension of the binary hypervectors, the non-idealities in the memristor circuit can be effectively controlled. At the fine level of controlled crossbar non-ideality, noise from memristor circuit can be used to encrypt data while being sufficiently interpretable by neural network for decryption. We applied our algorithm on image cryptography for proof of concept, and to text en/decryption with 100% decryption accuracy despite crossbar noises. Our work shows the potential and feasibility of using memristor crossbars as an unclonable stochastic encoder unit of cryptography on top of their existing functionality as a vector-matrix multiplication acceleration device.
△ Less
Submitted 27 January, 2022;
originally announced January 2022.
-
Design Space Exploration of Dense and Sparse Map** Schemes for RRAM Architectures
Authors:
Corey Lammie,
Jason K. Eshraghian,
Chenqi Li,
Amirali Amirsoleimani,
Roman Genov,
Wei D. Lu,
Mostafa Rahimi Azghadi
Abstract:
The impact of device and circuit-level effects in mixed-signal Resistive Random Access Memory (RRAM) accelerators typically manifest as performance degradation of Deep Learning (DL) algorithms, but the degree of impact varies based on algorithmic features. These include network architecture, capacity, weight distribution, and the type of inter-layer connections. Techniques are continuously emergin…
▽ More
The impact of device and circuit-level effects in mixed-signal Resistive Random Access Memory (RRAM) accelerators typically manifest as performance degradation of Deep Learning (DL) algorithms, but the degree of impact varies based on algorithmic features. These include network architecture, capacity, weight distribution, and the type of inter-layer connections. Techniques are continuously emerging to efficiently train sparse neural networks, which may have activation sparsity, quantization, and memristive noise. In this paper, we present an extended Design Space Exploration (DSE) methodology to quantify the benefits and limitations of dense and sparse map** schemes for a variety of network architectures. While sparsity of connectivity promotes less power consumption and is often optimized for extracting localized features, its performance on tiled RRAM arrays may be more susceptible to noise due to under-parameterization, when compared to dense map** schemes. Moreover, we present a case study quantifying and formalizing the trade-offs of typical non-idealities introduced into 1-Transistor-1-Resistor (1T1R) tiled memristive architectures and the size of modular crossbar tiles using the CIFAR-10 dataset.
△ Less
Submitted 24 January, 2022; v1 submitted 17 January, 2022;
originally announced January 2022.
-
Node-Centric Graph Learning from Data for Brain State Identification
Authors:
Nafiseh Ghoroghchian,
David M. Groppe,
Roman Genov,
Taufik A. Valiante,
Stark C. Draper
Abstract:
Data-driven graph learning models a network by determining the strength of connections between its nodes. The data refers to a graph signal which associates a value with each graph node. Existing graph learning methods either use simplified models for the graph signal, or they are prohibitively expensive in terms of computational and memory requirements. This is particularly true when the number o…
▽ More
Data-driven graph learning models a network by determining the strength of connections between its nodes. The data refers to a graph signal which associates a value with each graph node. Existing graph learning methods either use simplified models for the graph signal, or they are prohibitively expensive in terms of computational and memory requirements. This is particularly true when the number of nodes is high or there are temporal changes in the network. In order to consider richer models with a reasonable computational tractability, we introduce a graph learning method based on representation learning on graphs. Representation learning generates an embedding for each graph node, taking the information from neighbouring nodes into account. Our graph learning method further modifies the embeddings to compute the graph similarity matrix. In this work, graph learning is used to examine brain networks for brain state identification. We infer time-varying brain graphs from an extensive dataset of intracranial electroencephalographic (iEEG) signals from ten patients. We then apply the graphs as input to a classifier to distinguish seizure vs. non-seizure brain states. Using the binary classification metric of area under the receiver operating characteristic curve (AUC), this approach yields an average of 9.13 percent improvement when compared to two widely used brain network modeling methods.
△ Less
Submitted 4 November, 2020;
originally announced November 2020.
-
A Hierarchical Graph Signal Processing Approach to Inference from Spatiotemporal Signals
Authors:
Nafiseh Ghoroghchian,
Stark C. Draper,
Roman Genov
Abstract:
Motivated by the emerging area of graph signal processing (GSP), we introduce a novel method to draw inference from spatiotemporal signals. Data acquisition in different locations over time is common in sensor networks, for diverse applications ranging from object tracking in wireless networks to medical uses such as electroencephalography (EEG) signal processing. In this paper we leverage novel t…
▽ More
Motivated by the emerging area of graph signal processing (GSP), we introduce a novel method to draw inference from spatiotemporal signals. Data acquisition in different locations over time is common in sensor networks, for diverse applications ranging from object tracking in wireless networks to medical uses such as electroencephalography (EEG) signal processing. In this paper we leverage novel techniques of GSP to develop a hierarchical feature extraction approach by map** the data onto a series of spatiotemporal graphs. Such a model maps signals onto vertices of a graph and the time-space dependencies among signals are modeled by the edge weights. Signal components acquired from different locations and time often have complicated functional dependencies. Accordingly, their corresponding graph weights are learned from data and used in two ways. First, they are used as a part of the embedding related to the topology of graph, such as density. Second, they provide the connectivities of the base graph for extracting higher level GSP-based features. The latter include the energies of the signal's graph Fourier transform in different frequency bands. We test our approach on the intracranial EEG (iEEG) data set of the Kaggle epileptic seizure detection contest. In comparison to the winning code, the results show a slight net improvement and up to 6 percent improvement in per subject analysis, while the number of features are decreased by 75 percent on average.
△ Less
Submitted 25 October, 2020;
originally announced October 2020.
-
AIDX: Adaptive Inference Scheme to Mitigate State-Drift in Memristive VMM Accelerators
Authors:
Tony Liu,
Amirali Amirsoleimani,
Fabien Alibart,
Serge Ecoffey,
Dominique Drouin,
Roman Genov
Abstract:
An adaptive inference method for crossbar (AIDX) is presented based on an optimization scheme for adjusting the duration and amplitude of input voltage pulses. AIDX minimizes the long-term effects of memristance drift on artificial neural network accuracy. The sub-threshold behavior of memristor has been modeled and verified by comparing with fabricated device data. The proposed method has been ev…
▽ More
An adaptive inference method for crossbar (AIDX) is presented based on an optimization scheme for adjusting the duration and amplitude of input voltage pulses. AIDX minimizes the long-term effects of memristance drift on artificial neural network accuracy. The sub-threshold behavior of memristor has been modeled and verified by comparing with fabricated device data. The proposed method has been evaluated by testing on different network structures and applications, e.g., image reconstruction and classification tasks. The results showed an average of 60% improvement in convolutional neural network (CNN) performance on CIFAR10 dataset after 10000 inference operations as well as 78.6% error reduction in image reconstruction.
△ Less
Submitted 31 August, 2020;
originally announced September 2020.
