-
Synaptogen: A cross-domain generative device model for large-scale neuromorphic circuit design
Authors:
Tyler Hennen,
Leon Brackmann,
Tobias Ziegler,
Sebastian Siegel,
Stephan Menzel,
Rainer Waser,
Dirk J. Wouters,
Daniel Bedau
Abstract:
We present a fast generative modeling approach for resistive memories that reproduces the complex statistical properties of real-world devices. To enable efficient modeling of analog circuits, the model is implemented in Verilog-A. By training on extensive measurement data of integrated 1T1R arrays (6,000 cycles of 512 devices), an autoregressive stochastic process accurately accounts for the cros…
▽ More
We present a fast generative modeling approach for resistive memories that reproduces the complex statistical properties of real-world devices. To enable efficient modeling of analog circuits, the model is implemented in Verilog-A. By training on extensive measurement data of integrated 1T1R arrays (6,000 cycles of 512 devices), an autoregressive stochastic process accurately accounts for the cross-correlations between the switching parameters, while non-linear transformations ensure agreement with both cycle-to-cycle (C2C) and device-to-device (D2D) variability. Benchmarks show that this statistically comprehensive model achieves read/write throughputs exceeding those of even highly simplified and deterministic compact models.
△ Less
Submitted 9 April, 2024;
originally announced April 2024.
-
A High Throughput Generative Vector Autoregression Model for Stochastic Synapses
Authors:
T. Hennen,
A. Elias,
J. F. Nodin,
G. Molas,
R. Waser,
D. J. Wouters,
D. Bedau
Abstract:
By imitating the synaptic connectivity and plasticity of the brain, emerging electronic nanodevices offer new opportunities as the building blocks of neuromorphic systems. One challenge for largescale simulations of computational architectures based on emerging devices is to accurately capture device response, hysteresis, noise, and the covariance structure in the temporal domain as well as betwee…
▽ More
By imitating the synaptic connectivity and plasticity of the brain, emerging electronic nanodevices offer new opportunities as the building blocks of neuromorphic systems. One challenge for largescale simulations of computational architectures based on emerging devices is to accurately capture device response, hysteresis, noise, and the covariance structure in the temporal domain as well as between the different device parameters. We address this challenge with a high throughput generative model for synaptic arrays that is based on a recently available type of electrical measurement data for resistive memory cells. We map this real world data onto a vector autoregressive stochastic process to accurately reproduce the device parameters and their cross-correlation structure. While closely matching the measured data, our model is still very fast; we provide parallelized implementations for both CPUs and GPUs and demonstrate array sizes above one billion cells and throughputs exceeding one hundred million weight updates per second, above the pixel rate of a 30 frames/s 4K video stream.
△ Less
Submitted 10 May, 2022;
originally announced May 2022.
-
Non-Volatile Memory Array Based Quantization- and Noise-Resilient LSTM Neural Networks
Authors:
Wen Ma,
Pi-Feng Chiu,
Won Ho Choi,
Minghai Qin,
Daniel Bedau,
Martin Lueker-Boden
Abstract:
In cloud and edge computing models, it is important that compute devices at the edge be as power efficient as possible. Long short-term memory (LSTM) neural networks have been widely used for natural language processing, time series prediction and many other sequential data tasks. Thus, for these applications there is increasing need for low-power accelerators for LSTM model inference at the edge.…
▽ More
In cloud and edge computing models, it is important that compute devices at the edge be as power efficient as possible. Long short-term memory (LSTM) neural networks have been widely used for natural language processing, time series prediction and many other sequential data tasks. Thus, for these applications there is increasing need for low-power accelerators for LSTM model inference at the edge. In order to reduce power dissipation due to data transfers within inference devices, there has been significant interest in accelerating vector-matrix multiplication (VMM) operations using non-volatile memory (NVM) weight arrays. In NVM array-based hardware, reduced bit-widths also significantly increases the power efficiency. In this paper, we focus on the application of quantization-aware training algorithm to LSTM models, and the benefits these models bring in terms of resilience against both quantization error and analog device noise. We have shown that only 4-bit NVM weights and 4-bit ADC/DACs are needed to produce equivalent LSTM network performance as floating-point baseline. Reasonable levels of ADC quantization noise and weight noise can be naturally tolerated within our NVMbased quantized LSTM network. Benchmark analysis of our proposed LSTM accelerator for inference has shown at least 2.4x better computing efficiency and 40x higher area efficiency than traditional digital approaches (GPU, FPGA, and ASIC). Some other novel approaches based on NVM promise to deliver higher computing efficiency (up to 4.7x) but require larger arrays with potential higher error rates.
△ Less
Submitted 24 February, 2020;
originally announced February 2020.
-
Two-dimensional Decoding Algorithms and Recording Techniques for Bit Patterned Media Feasibility Demonstrations
Authors:
Yuri Obukhov,
Pierre-Olivier Jubert,
Daniel Bedau,
Michael Grobis
Abstract:
Recording experiments and decoding algorithms are presented for evaluating the bit-error-rate of state-of-the-art magnetic bitpatterned media. The recording experiments are performed with a static tester and conventional hard-disk-drive heads. As the reader dimensions are larger than the bit dimensions in both the down-track and the cross-track directions, a two-dimensional bit decoding algorithm…
▽ More
Recording experiments and decoding algorithms are presented for evaluating the bit-error-rate of state-of-the-art magnetic bitpatterned media. The recording experiments are performed with a static tester and conventional hard-disk-drive heads. As the reader dimensions are larger than the bit dimensions in both the down-track and the cross-track directions, a two-dimensional bit decoding algorithm is required. Two such algorithms are presented in details together with the methodology implemented to accurately retrieve island positions during recording. Using these techniques, a 1.6 Td/in$^2$ magnetic bit pattern media is demonstrated to support 2D bit error rates below 1e-2 under shingled magnetic recording conditions.
△ Less
Submitted 1 June, 2015;
originally announced June 2015.
