-
Seizure Detection and Prediction by Parallel Memristive Convolutional Neural Networks
Authors:
Chenqi Li,
Corey Lammie,
Xuening Dong,
Amirali Amirsoleimani,
Mostafa Rahimi Azghadi,
Roman Genov
Abstract:
During the past two decades, epileptic seizure detection and prediction algorithms have evolved rapidly. However, despite significant performance improvements, their hardware implementation using conventional technologies, such as Complementary Metal-Oxide-Semiconductor (CMOS), in power and area-constrained settings remains a challenging task; especially when many recording channels are used. In t…
▽ More
During the past two decades, epileptic seizure detection and prediction algorithms have evolved rapidly. However, despite significant performance improvements, their hardware implementation using conventional technologies, such as Complementary Metal-Oxide-Semiconductor (CMOS), in power and area-constrained settings remains a challenging task; especially when many recording channels are used. In this paper, we propose a novel low-latency parallel Convolutional Neural Network (CNN) architecture that has between 2-2,800x fewer network parameters compared to SOTA CNN architectures and achieves 5-fold cross validation accuracy of 99.84% for epileptic seizure detection, and 99.01% and 97.54% for epileptic seizure prediction, when evaluated using the University of Bonn Electroencephalogram (EEG), CHB-MIT and SWEC-ETHZ seizure datasets, respectively. We subsequently implement our network onto analog crossbar arrays comprising Resistive Random-Access Memory (RRAM) devices, and provide a comprehensive benchmark by simulating, laying out, and determining hardware requirements of the CNN component of our system. To the best of our knowledge, we are the first to parallelize the execution of convolution layer kernels on separate analog crossbars to enable 2 orders of magnitude reduction in latency compared to SOTA hybrid Memristive-CMOS DL accelerators. Furthermore, we investigate the effects of non-idealities on our system and investigate Quantization Aware Training (QAT) to mitigate the performance degradation due to low ADC/DAC resolution. Finally, we propose a stuck weight offsetting methodology to mitigate performance degradation due to stuck RON/ROFF memristor weights, recovering up to 32% accuracy, without requiring retraining. The CNN component of our platform is estimated to consume approximately 2.791W of power while occupying an area of 31.255mm$^2$ in a 22nm FDSOI CMOS process.
△ Less
Submitted 20 June, 2022;
originally announced June 2022.
-
Toward A Formalized Approach for Spike Sorting Algorithms and Hardware Evaluation
Authors:
Tim Zhang,
Corey Lammie,
Mostafa Rahimi Azghadi,
Amirali Amirsoleimani,
Majid Ahmadi,
Roman Genov
Abstract:
Spike sorting algorithms are used to separate extracellular recordings of neuronal populations into single-unit spike activities. The development of customized hardware implementing spike sorting algorithms is burgeoning. However, there is a lack of a systematic approach and a set of standardized evaluation criteria to facilitate direct comparison of both software and hardware implementations. In…
▽ More
Spike sorting algorithms are used to separate extracellular recordings of neuronal populations into single-unit spike activities. The development of customized hardware implementing spike sorting algorithms is burgeoning. However, there is a lack of a systematic approach and a set of standardized evaluation criteria to facilitate direct comparison of both software and hardware implementations. In this paper, we formalize a set of standardized criteria and a publicly available synthetic dataset entitled Synthetic Simulations Of Extracellular Recordings (SSOER), which was constructed by aggregating existing synthetic datasets with varying Signal-To-Noise Ratios (SNRs). Furthermore, we present a benchmark for future comparison, and use our criteria to evaluate a simulated Resistive Random-Access Memory (RRAM) In-Memory Computing (IMC) system using the Discrete Wavelet Transform (DWT) for feature extraction. Our system consumes approximately (per channel) 10.72mW and occupies an area of 0.66mm$^2$ in a 22nm FDSOI Complementary Metal-Oxide-Semiconductor (CMOS) process.
△ Less
Submitted 13 May, 2022;
originally announced May 2022.
-
PRUNIX: Non-Ideality Aware Convolutional Neural Network Pruning for Memristive Accelerators
Authors:
Ali Alshaarawy,
Amirali Amirsoleimani,
Roman Genov
Abstract:
In this work, PRUNIX, a framework for training and pruning convolutional neural networks is proposed for deployment on memristor crossbar based accelerators. PRUNIX takes into account the numerous non-ideal effects of memristor crossbars including weight quantization, state-drift, aging and stuck-at-faults. PRUNIX utilises a novel Group Sawtooth Regularization intended to improve non-ideality tole…
▽ More
In this work, PRUNIX, a framework for training and pruning convolutional neural networks is proposed for deployment on memristor crossbar based accelerators. PRUNIX takes into account the numerous non-ideal effects of memristor crossbars including weight quantization, state-drift, aging and stuck-at-faults. PRUNIX utilises a novel Group Sawtooth Regularization intended to improve non-ideality tolerance as well as sparsity, and a novel Adaptive Pruning Algorithm (APA) intended to minimise accuracy loss by considering the sensitivity of different layers of a CNN to pruning. We compare our regularization and pruning methods with other standards on multiple CNN architectures, and observe an improvement of 13% test accuracy when quantization and other non-ideal effects are accounted for with an overall sparsity of 85%, which is similar to other methods
△ Less
Submitted 3 February, 2022;
originally announced February 2022.
-
SDEX: Monte Carlo Simulation of Stochastic Differential Equations on Memristor Crossbars
Authors:
Louis Primeau,
Amirali Amirsoleimani,
Roman Genov
Abstract:
Here we present stochastic differential equations (SDEs) on a memristor crossbar, where the source of gaussian noise is derived from the random conductance due to ion drift in the devices during programming. We examine the effects of line resistance on the generation of normal random vectors, showing the skew and kurtosis are within acceptable bounds. We then show the implementation of a stochasti…
▽ More
Here we present stochastic differential equations (SDEs) on a memristor crossbar, where the source of gaussian noise is derived from the random conductance due to ion drift in the devices during programming. We examine the effects of line resistance on the generation of normal random vectors, showing the skew and kurtosis are within acceptable bounds. We then show the implementation of a stochastic differential equation solver for the Black-Scholes SDE, and compare the distribution with the analytic solution. We determine that the random number generation works as intended, and calculate the energy cost of the simulation.
△ Less
Submitted 2 February, 2022;
originally announced February 2022.
-
HYPERLOCK: In-Memory Hyperdimensional Encryption in Memristor Crossbar Array
Authors:
Jack Cai,
Amirali Amirsoleimani,
Roman Genov
Abstract:
We present a novel cryptography architecture based on memristor crossbar array, binary hypervectors, and neural network. Utilizing the stochastic and unclonable nature of memristor crossbar and error tolerance of binary hypervectors and neural network, implementation of the algorithm on memristor crossbar simulation is made possible. We demonstrate that with an increasing dimension of the binary h…
▽ More
We present a novel cryptography architecture based on memristor crossbar array, binary hypervectors, and neural network. Utilizing the stochastic and unclonable nature of memristor crossbar and error tolerance of binary hypervectors and neural network, implementation of the algorithm on memristor crossbar simulation is made possible. We demonstrate that with an increasing dimension of the binary hypervectors, the non-idealities in the memristor circuit can be effectively controlled. At the fine level of controlled crossbar non-ideality, noise from memristor circuit can be used to encrypt data while being sufficiently interpretable by neural network for decryption. We applied our algorithm on image cryptography for proof of concept, and to text en/decryption with 100% decryption accuracy despite crossbar noises. Our work shows the potential and feasibility of using memristor crossbars as an unclonable stochastic encoder unit of cryptography on top of their existing functionality as a vector-matrix multiplication acceleration device.
△ Less
Submitted 27 January, 2022;
originally announced January 2022.
-
Design Space Exploration of Dense and Sparse Map** Schemes for RRAM Architectures
Authors:
Corey Lammie,
Jason K. Eshraghian,
Chenqi Li,
Amirali Amirsoleimani,
Roman Genov,
Wei D. Lu,
Mostafa Rahimi Azghadi
Abstract:
The impact of device and circuit-level effects in mixed-signal Resistive Random Access Memory (RRAM) accelerators typically manifest as performance degradation of Deep Learning (DL) algorithms, but the degree of impact varies based on algorithmic features. These include network architecture, capacity, weight distribution, and the type of inter-layer connections. Techniques are continuously emergin…
▽ More
The impact of device and circuit-level effects in mixed-signal Resistive Random Access Memory (RRAM) accelerators typically manifest as performance degradation of Deep Learning (DL) algorithms, but the degree of impact varies based on algorithmic features. These include network architecture, capacity, weight distribution, and the type of inter-layer connections. Techniques are continuously emerging to efficiently train sparse neural networks, which may have activation sparsity, quantization, and memristive noise. In this paper, we present an extended Design Space Exploration (DSE) methodology to quantify the benefits and limitations of dense and sparse map** schemes for a variety of network architectures. While sparsity of connectivity promotes less power consumption and is often optimized for extracting localized features, its performance on tiled RRAM arrays may be more susceptible to noise due to under-parameterization, when compared to dense map** schemes. Moreover, we present a case study quantifying and formalizing the trade-offs of typical non-idealities introduced into 1-Transistor-1-Resistor (1T1R) tiled memristive architectures and the size of modular crossbar tiles using the CIFAR-10 dataset.
△ Less
Submitted 24 January, 2022; v1 submitted 17 January, 2022;
originally announced January 2022.
-
AIDX: Adaptive Inference Scheme to Mitigate State-Drift in Memristive VMM Accelerators
Authors:
Tony Liu,
Amirali Amirsoleimani,
Fabien Alibart,
Serge Ecoffey,
Dominique Drouin,
Roman Genov
Abstract:
An adaptive inference method for crossbar (AIDX) is presented based on an optimization scheme for adjusting the duration and amplitude of input voltage pulses. AIDX minimizes the long-term effects of memristance drift on artificial neural network accuracy. The sub-threshold behavior of memristor has been modeled and verified by comparing with fabricated device data. The proposed method has been ev…
▽ More
An adaptive inference method for crossbar (AIDX) is presented based on an optimization scheme for adjusting the duration and amplitude of input voltage pulses. AIDX minimizes the long-term effects of memristance drift on artificial neural network accuracy. The sub-threshold behavior of memristor has been modeled and verified by comparing with fabricated device data. The proposed method has been evaluated by testing on different network structures and applications, e.g., image reconstruction and classification tasks. The results showed an average of 60% improvement in convolutional neural network (CNN) performance on CIFAR10 dataset after 10000 inference operations as well as 78.6% error reduction in image reconstruction.
△ Less
Submitted 31 August, 2020;
originally announced September 2020.
-
Optimized Implementation of Memristor-Based Full Adder by Material Implication Logic
Authors:
Mehri Teimoory,
Amirali Amirsoleimani,
Jafar Shamsi,
Arash Ahmadi,
Shahpour Alirezaee,
Majid Ahmadi
Abstract:
Recently memristor-based applications and circuits are receiving an increased attention. Furthermore, memristors are also applied in logic circuit design. Material implication logic is one of the main areas with memristors. In this paper an optimized memristor-based full adder design by material implication logic is presented. This design needs 27 memristors and less area in comparison with typica…
▽ More
Recently memristor-based applications and circuits are receiving an increased attention. Furthermore, memristors are also applied in logic circuit design. Material implication logic is one of the main areas with memristors. In this paper an optimized memristor-based full adder design by material implication logic is presented. This design needs 27 memristors and less area in comparison with typical CMOS-based 8-bit full adders. Also the presented full adder needs only 184 computational steps which enhance former full adder design speed by 20 percent.
△ Less
Submitted 3 January, 2015;
originally announced January 2015.