-
A memristive deep belief neural network based on silicon synapses
Authors:
Wei Wang,
Loai Danial,
Yang Li,
Eric Herbelin,
Evgeny Pikhay,
Yakov Roizin,
Barak Hoffer,
Zhongrui Wang,
Shahar Kvatinsky
Abstract:
Memristor-based neuromorphic computing could overcome the limitations of traditional von Neumann computing architectures -- in which data are shuffled between separate memory and processing units -- and improve the performance of deep neural networks. However, this will require accurate synaptic-like device performance, and memristors typically suffer from poor yield and a limited number of reliab…
▽ More
Memristor-based neuromorphic computing could overcome the limitations of traditional von Neumann computing architectures -- in which data are shuffled between separate memory and processing units -- and improve the performance of deep neural networks. However, this will require accurate synaptic-like device performance, and memristors typically suffer from poor yield and a limited number of reliable conductance states. Here we report floating gate memristive synaptic devices that are fabricated in a commercial complementary metal-oxide-semiconductor (CMOS) process. These silicon synapses offer analogue tunability, high endurance, long retention times, predictable cycling degradation, moderate device-to-device variations, and high yield. They also provide two orders of magnitude higher energy efficiency for multiply-accumulate operations than graphics processing units. We use two 12-by-8 arrays of the memristive devices for in-situ training of a 19-by-8 memristive restricted Boltzmann machine for pattern recognition via a gradient descent algorithm based on contrastive divergence. We then create a memristive deep belief neural network consisting of three memristive restricted Boltzmann machines. We test this on the modified National Institute of Standards and Technology (MNIST) dataset, demonstrating recognition accuracy up to 97.05%.
△ Less
Submitted 20 July, 2023; v1 submitted 16 March, 2022;
originally announced March 2022.
-
Efficient Training of the Memristive Deep Belief Net Immune to Non-Idealities of the Synaptic Devices
Authors:
Wei Wang,
Barak Hoffer,
Tzofnat Greenberg-Toledo,
Yang Li,
Minhui Zou,
Eric Herbelin,
Ronny Ronen,
Xiaoxin Xu,
Yulin Zhao,
Jianguo Yang,
Shahar Kvatinsky
Abstract:
The tunability of conductance states of various emerging non-volatile memristive devices emulates the plasticity of biological synapses, making it promising in the hardware realization of large-scale neuromorphic systems. The inference of the neural network can be greatly accelerated by the vector-matrix multiplication (VMM) performed within a crossbar array of memristive devices in one step. Neve…
▽ More
The tunability of conductance states of various emerging non-volatile memristive devices emulates the plasticity of biological synapses, making it promising in the hardware realization of large-scale neuromorphic systems. The inference of the neural network can be greatly accelerated by the vector-matrix multiplication (VMM) performed within a crossbar array of memristive devices in one step. Nevertheless, the implementation of the VMM needs complex peripheral circuits and the complexity further increases since non-idealities of memristive devices prevent precise conductance tuning (especially for the online training) and largely degrade the performance of the deep neural networks (DNNs). Here, we present an efficient online training method of the memristive deep belief net (DBN). The proposed memristive DBN uses stochastically binarized activations, reducing the complexity of peripheral circuits, and uses the contrastive divergence (CD) based gradient descent learning algorithm. The analog VMM and digital CD are performed separately in a mixed-signal hardware arrangement, making the memristive DBN high immune to non-idealities of synaptic devices. The number of write operations on memristive devices is reduced by two orders of magnitude. The recognition accuracy of 95%~97% can be achieved for the MNIST dataset using pulsed synaptic behaviors of various memristive synaptic devices.
△ Less
Submitted 15 March, 2022;
originally announced March 2022.
-
Physical based compact model of Y-Flash memristor for neuromorphic computation
Authors:
Wei Wang,
Loai Danial,
Eric Herbelin,
Barak Hoffer,
Batel Oved,
Tzofnat Greenberg-Toledo,
Evgeny Pikhay,
Yakov Roizin,
Shahar Kvatinsky
Abstract:
Y-Flash memristors utilize the mature technology of single polysilicon floating gate non-volatile memories (NVM). It can be operated in a two-terminal configuration similar to the other emerging memristive devices, i.e., resistive random-access memory (RRAM), phase-change memory (PCM), etc. Fabricated in production complementary metal-oxide-semiconductor (CMOS) technology, Y-Flash memristors allow…
▽ More
Y-Flash memristors utilize the mature technology of single polysilicon floating gate non-volatile memories (NVM). It can be operated in a two-terminal configuration similar to the other emerging memristive devices, i.e., resistive random-access memory (RRAM), phase-change memory (PCM), etc. Fabricated in production complementary metal-oxide-semiconductor (CMOS) technology, Y-Flash memristors allow excellent repro-ducibility reflected in high neuromorphic products yields. Working in the subthreshold region, the device can be programmed to a large number of fine-tuned intermediate states in an analog fashion and allows low readout currents (1 nA ~ 5 $μ$ A). However, currently, there are no accurate models to describe the dynamic switching in this type of memristive device and account for multiple operational configurations. In this paper, we provide a physical-based compact model that describes Y-Flash memristor performance both in DC and AC regimes, and consistently describes the dynamic program and erase operations. The model is integrated into the commercial circuit design tools and is ready to be used in applications related to neuromorphic computation.
△ Less
Submitted 16 February, 2022;
originally announced February 2022.
