-
Opportunities for Analog Coding in Emerging Memory Systems
Authors:
Jesse H. Engel,
S. Burc Eryilmaz,
SangBum Kim,
Matthew BrightSky,
Chung Lam,
Hsiang-Lan Lung,
Bruno A. Olshausen,
H. -S. Philip Wong
Abstract:
The exponential growth in data generation and large-scale data analysis creates an unprecedented need for inexpensive, low-latency, and high-density information storage. This need has motivated significant research into multi-level memory systems that can store multiple bits of information per device. Although both the memory state of these devices and much of the data they store are intrinsically…
▽ More
The exponential growth in data generation and large-scale data analysis creates an unprecedented need for inexpensive, low-latency, and high-density information storage. This need has motivated significant research into multi-level memory systems that can store multiple bits of information per device. Although both the memory state of these devices and much of the data they store are intrinsically analog-valued, both are quantized for use with digital systems and discrete error correcting codes. Using phase change memory as a prototypical multi-level storage technology, we herein demonstrate that analog-valued devices can achieve higher capacities when paired with analog codes. Further, we find that storing analog signals directly through joint-coding can achieve low distortion with reduced coding complexity. By jointly optimizing for signal statistics, device statistics, and a distortion metric, finite-length analog encodings can perform comparable to digital systems with asymptotically infinite large encodings. These results show that end-to-end analog memory systems have not only the potential to reach higher storage capacities than discrete systems, but also to significantly lower coding complexity, leading to faster and more energy efficient storage.
△ Less
Submitted 21 January, 2017;
originally announced January 2017.
-
Training a Probabilistic Graphical Model with Resistive Switching Electronic Synapses
Authors:
S. Burc Eryilmaz,
Emre Neftci,
Siddharth Joshi,
SangBum Kim,
Matthew BrightSky,
Hsiang-Lan Lung,
Chung Lam,
Gert Cauwenberghs,
H. -S. Philip Wong
Abstract:
Current large scale implementations of deep learning and data mining require thousands of processors, massive amounts of off-chip memory, and consume gigajoules of energy. Emerging memory technologies such as nanoscale two-terminal resistive switching memory devices offer a compact, scalable and low power alternative that permits on-chip co-located processing and memory in fine-grain distributed p…
▽ More
Current large scale implementations of deep learning and data mining require thousands of processors, massive amounts of off-chip memory, and consume gigajoules of energy. Emerging memory technologies such as nanoscale two-terminal resistive switching memory devices offer a compact, scalable and low power alternative that permits on-chip co-located processing and memory in fine-grain distributed parallel architecture. Here we report first use of resistive switching memory devices for implementing and training a Restricted Boltzmann Machine (RBM), a generative probabilistic graphical model as a key component for unsupervised learning in deep networks. We experimentally demonstrate a 45-synapse RBM realized with 90 resistive switching phase change memory (PCM) elements trained with a bio-inspired variant of the Contrastive Divergence (CD) algorithm, implementing Hebbian and anti-Hebbian weight updates. The resistive PCM devices show a two-fold to ten-fold reduction in error rate in a missing pixel pattern completion task trained over 30 epochs, compared to untrained case. Measured programming energy consumption is 6.1 nJ per epoch with the resistive switching PCM devices, a factor of ~150 times lower than conventional processor-memory systems. We analyze and discuss the dependence of learning performance on cycle-to-cycle variations as well as number of gradual levels in the PCM analog memory devices.
△ Less
Submitted 9 October, 2016; v1 submitted 27 September, 2016;
originally announced September 2016.
-
Brain-like associative learning using a nanoscale non-volatile phase change synaptic device array
Authors:
Sukru Burc Eryilmaz,
Duygu Kuzum,
Rakesh Jeyasingh,
SangBum Kim,
Matthew BrightSky,
Chung Lam,
H. -S. Philip Wong
Abstract:
Recent advances in neuroscience together with nanoscale electronic device technology have resulted in huge interests in realizing brain-like computing hardwares using emerging nanoscale memory devices as synaptic elements. Although there has been experimental work that demonstrated the operation of nanoscale synaptic element at the single device level, network level studies have been limited to si…
▽ More
Recent advances in neuroscience together with nanoscale electronic device technology have resulted in huge interests in realizing brain-like computing hardwares using emerging nanoscale memory devices as synaptic elements. Although there has been experimental work that demonstrated the operation of nanoscale synaptic element at the single device level, network level studies have been limited to simulations. In this work, we demonstrate, using experiments, array level associative learning using phase change synaptic devices connected in a grid like configuration similar to the organization of the biological brain. Implementing Hebbian learning with phase change memory cells, the synaptic grid was able to store presented patterns and recall missing patterns in an associative brain-like fashion. We found that the system is robust to device variations, and large variations in cell resistance states can be accommodated by increasing the number of training epochs. We illustrated the tradeoff between variation tolerance of the network and the overall energy consumption, and found that energy consumption is decreased significantly for lower variation tolerance.
△ Less
Submitted 13 July, 2014; v1 submitted 19 June, 2014;
originally announced June 2014.
-
Experimental Demonstration of Array-level Learning with Phase Change Synaptic Devices
Authors:
S. Burc Eryilmaz,
Duygu Kuzum,
Rakesh G. D. Jeyasingh,
SangBum Kim,
Matthew BrightSky,
Chung Lam,
H. -S. Philip Wong
Abstract:
The computational performance of the biological brain has long attracted significant interest and has led to inspirations in operating principles, algorithms, and architectures for computing and signal processing. In this work, we focus on hardware implementation of brain-like learning in a brain-inspired architecture. We demonstrate, in hardware, that 2-D crossbar arrays of phase change synaptic…
▽ More
The computational performance of the biological brain has long attracted significant interest and has led to inspirations in operating principles, algorithms, and architectures for computing and signal processing. In this work, we focus on hardware implementation of brain-like learning in a brain-inspired architecture. We demonstrate, in hardware, that 2-D crossbar arrays of phase change synaptic devices can achieve associative learning and perform pattern recognition. Device and array-level studies using an experimental 10x10 array of phase change synaptic devices have shown that pattern recognition is robust against synaptic resistance variations and large variations can be tolerated by increasing the number of training iterations. Our measurements show that increase in initial variation from 9 % to 60 % causes required training iterations to increase from 1 to 11.
△ Less
Submitted 3 June, 2014; v1 submitted 29 May, 2014;
originally announced May 2014.
