-
Implementing Binarized Neural Networks with Magnetoresistive RAM without Error Correction
Authors:
Tifenn Hirtzlin,
Bogdan Penkovsky,
Jacques-Olivier Klein,
Nicolas Locatelli,
Adrien F. Vincent,
Marc Bocquet,
Jean-Michel Portal,
Damien Querlioz
Abstract:
One of the most exciting applications of Spin Torque Magnetoresistive Random Access Memory (ST-MRAM) is the in-memory implementation of deep neural networks, which could allow improving the energy efficiency of Artificial Intelligence by orders of magnitude with regards to its implementation on computers and graphics cards. In particular, ST-MRAM could be ideal for implementing Binarized Neural Ne…
▽ More
One of the most exciting applications of Spin Torque Magnetoresistive Random Access Memory (ST-MRAM) is the in-memory implementation of deep neural networks, which could allow improving the energy efficiency of Artificial Intelligence by orders of magnitude with regards to its implementation on computers and graphics cards. In particular, ST-MRAM could be ideal for implementing Binarized Neural Networks (BNNs), a type of deep neural networks discovered in 2016, which can achieve state-of-the-art performance with a highly reduced memory footprint with regards to conventional artificial intelligence approaches. The challenge of ST-MRAM, however, is that it is prone to write errors and usually requires the use of error correction. In this work, we show that these bit errors can be tolerated by BNNs to an outstanding level, based on examples of image recognition tasks (MNIST, CIFAR-10 and ImageNet): bit error rates of ST-MRAM up to 0.1% have little impact on recognition accuracy. The requirements for ST-MRAM are therefore considerably relaxed for BNNs with regards to traditional applications. By consequence, we show that for BNNs, ST-MRAMs can be programmed with weak (low-energy) programming conditions, without error correcting codes. We show that this result can allow the use of low energy and low area ST-MRAM cells, and show that the energy savings at the system level can reach a factor two.
△ Less
Submitted 12 August, 2019;
originally announced August 2019.
-
Microwave neural processing and broadcasting with spintronic nano-oscillators
Authors:
P. Talatchian,
M. Romera,
S. Tsunegi,
F. Abreu Araujo,
V. Cros,
P. Bortolotti,
J. Trastoy,
K. Yakushiji,
A. Fukushima,
H. Kubota,
S. Yuasa,
M. Ernoult,
D. Vodenicarevic,
T. Hirtzlin,
N. Locatelli,
D. Querlioz,
J. Grollier
Abstract:
Can we build small neuromorphic chips capable of training deep networks with billions of parameters? This challenge requires hardware neurons and synapses with nanometric dimensions, which can be individually tuned, and densely connected. While nanosynaptic devices have been pursued actively in recent years, much less has been done on nanoscale artificial neurons. In this paper, we show that spint…
▽ More
Can we build small neuromorphic chips capable of training deep networks with billions of parameters? This challenge requires hardware neurons and synapses with nanometric dimensions, which can be individually tuned, and densely connected. While nanosynaptic devices have been pursued actively in recent years, much less has been done on nanoscale artificial neurons. In this paper, we show that spintronic nano-oscillators are promising to implement analog hardware neurons that can be densely interconnected through electromagnetic signals. We show how spintronic oscillators maps the requirements of artificial neurons. We then show experimentally how an ensemble of four coupled oscillators can learn to classify all twelve American vowels, realizing the most complicated tasks performed by nanoscale neurons.
△ Less
Submitted 25 April, 2019;
originally announced April 2019.
-
Use of Magnetoresistive Random-Access Memory as Approximate Memory for Training Neural Networks
Authors:
Nicolas Locatelli,
Adrien F. Vincent,
Damien Querlioz
Abstract:
Hardware neural networks that implement synaptic weights with embedded non-volatile memory, such as spin torque memory (ST-MRAM), are a major lead for low energy artificial intelligence. In this work, we propose an approximate storage approach for their memory. We show that this strategy grants effective control of the bit error rate by modulating the programming pulse amplitude or duration. Accou…
▽ More
Hardware neural networks that implement synaptic weights with embedded non-volatile memory, such as spin torque memory (ST-MRAM), are a major lead for low energy artificial intelligence. In this work, we propose an approximate storage approach for their memory. We show that this strategy grants effective control of the bit error rate by modulating the programming pulse amplitude or duration. Accounting for the devices variability issue, we evaluate energy savings, and show how they translate when training a hardware neural network. On an image recognition example, 74% of programming energy can be saved by losing only 1% on the recognition performance.
△ Less
Submitted 25 October, 2018;
originally announced October 2018.
-
A Neural Network Based on Synchronized Pairs of Nano-Oscillators
Authors:
Damir Vodenicarevic,
Nicolas Locatelli,
Damien Querlioz
Abstract:
Artificial neural networks are intensively used to perform cognitive tasks such as image classification on traditional computers. With the end of CMOS scaling and increasing demand for efficient neural networks, alternative architectures implementing neural functions efficiently are being studied. This study leverages the demonstrated frequency tuning capabilities of compact nano-oscillators and t…
▽ More
Artificial neural networks are intensively used to perform cognitive tasks such as image classification on traditional computers. With the end of CMOS scaling and increasing demand for efficient neural networks, alternative architectures implementing neural functions efficiently are being studied. This study leverages the demonstrated frequency tuning capabilities of compact nano-oscillators and their synchronization dynamics to implement a neuron using a pair of synchronized oscillators, and which features an unconventional response curve. We show that this compact neuron can naturally implement generic logic gates, including XOR. A simulated oscillator-based neural network is then shown to achieve results equivalent to standard approaches on two reference classification tasks. Finally, the performance of the system is evaluated in the presence of oscillator phase noise, an important issue of oscillating nanodevices. These results open the way for the design of alternative architectures adapted to efficient neural network execution.
△ Less
Submitted 7 September, 2017;
originally announced September 2017.
-
Synchronization Detection in Networks of Coupled Oscillators for Pattern Recognition
Authors:
Damir Vodenicarevic,
Nicolas Locatelli,
Julie Grollier,
Damien Querlioz
Abstract:
Coupled oscillator-based networks are an attractive approach for implementing hardware neural networks based on emerging nanotechnologies. However, the readout of the state of a coupled oscillator network is a difficult challenge in hardware implementations, as it necessitates complex signal processing to evaluate the degree of synchronization between oscillators, possibly more complicated than th…
▽ More
Coupled oscillator-based networks are an attractive approach for implementing hardware neural networks based on emerging nanotechnologies. However, the readout of the state of a coupled oscillator network is a difficult challenge in hardware implementations, as it necessitates complex signal processing to evaluate the degree of synchronization between oscillators, possibly more complicated than the coupled oscillator network itself. In this work, we focus on a coupled oscillator network particularly adapted to emerging technologies, and evaluate two schemes for reading synchronization patterns that can be readily implemented with basic CMOS circuits. Through simulation of a simple generic coupled oscillator network, we compare the operation of these readout techniques with a previously proposed full statistics evaluation scheme. Our approaches provide results nearly identical to the mathematical method, but also show better resilience to moderate noise, which is a major concern for hardware implementations. These results open the door to widespread realization of hardware coupled oscillator-based neural systems.
△ Less
Submitted 7 July, 2016;
originally announced July 2016.
