-
Roadmap to Neuromorphic Computing with Emerging Technologies
Authors:
Adnan Mehonic,
Daniele Ielmini,
Kaushik Roy,
Onur Mutlu,
Shahar Kvatinsky,
Teresa Serrano-Gotarredona,
Bernabe Linares-Barranco,
Sabina Spiga,
Sergey Savelev,
Alexander G Balanov,
Nitin Chawla,
Giuseppe Desoli,
Gerardo Malavena,
Christian Monzio Compagnoni,
Zhongrui Wang,
J Joshua Yang,
Ghazi Sarwat Syed,
Abu Sebastian,
Thomas Mikolajick,
Beatriz Noheda,
Stefan Slesazeck,
Bernard Dieny,
Tuo-Hung,
Hou,
Akhil Varri
, et al. (28 additional authors not shown)
Abstract:
The roadmap is organized into several thematic sections, outlining current computing challenges, discussing the neuromorphic computing approach, analyzing mature and currently utilized technologies, providing an overview of emerging technologies, addressing material challenges, exploring novel computing concepts, and finally examining the maturity level of emerging technologies while determining t…
▽ More
The roadmap is organized into several thematic sections, outlining current computing challenges, discussing the neuromorphic computing approach, analyzing mature and currently utilized technologies, providing an overview of emerging technologies, addressing material challenges, exploring novel computing concepts, and finally examining the maturity level of emerging technologies while determining the next essential steps for their advancement.
△ Less
Submitted 2 July, 2024;
originally announced July 2024.
-
Emerging Nonvolatile Memories for Machine Learning
Authors:
Adnan Mehonic,
Dovydas Joksas
Abstract:
Digital computers have been getting exponentially faster for decades, but huge challenges exist today. Transistor scaling, described by Moore's law, has been slowing down over the last few years, ending the era of fully predictable performance improvements. Furthermore, the data-centric computing demands fueled by machine learning applications are rapidly growing, and current computing systems --…
▽ More
Digital computers have been getting exponentially faster for decades, but huge challenges exist today. Transistor scaling, described by Moore's law, has been slowing down over the last few years, ending the era of fully predictable performance improvements. Furthermore, the data-centric computing demands fueled by machine learning applications are rapidly growing, and current computing systems -- even with the historical rate of improvements driven by Moore's law -- cannot keep up with these enormous computational demands. Some are turning to analogue in-memory computing as a solution, where specialised systems operating on physical principles accelerate specific tasks. We explore how emerging nonvolatile memories can be used to implement such systems tailored for machine learning. In particular, we discuss how memristive crossbar arrays can accelerate key linear algebra operations used in neural networks, what technological challenges remain, and how they can be overcome.
△ Less
Submitted 7 August, 2023;
originally announced August 2023.
-
Vision for Bosnia and Herzegovina in Artificial Intelligence Age: Global Trends, Potential Opportunities, Selected Use-cases and Realistic Goals
Authors:
Zlatan Ajanović,
Emina Aličković,
Aida Branković,
Sead Delalić,
Eldar Kurtić,
Salem Malikić,
Adnan Mehonić,
Hamza Merzić,
Kenan Šehić,
Bahrudin Trbalić
Abstract:
Artificial Intelligence (AI) is one of the most promising technologies of the 21. century, with an already noticeable impact on society and the economy. With this work, we provide a short overview of global trends, applications in industry and selected use-cases from our international experience and work in industry and academia. The goal is to present global and regional positive practices and pr…
▽ More
Artificial Intelligence (AI) is one of the most promising technologies of the 21. century, with an already noticeable impact on society and the economy. With this work, we provide a short overview of global trends, applications in industry and selected use-cases from our international experience and work in industry and academia. The goal is to present global and regional positive practices and provide an informed opinion on the realistic goals and opportunities for positioning B&H on the global AI scene.
△ Less
Submitted 8 September, 2022;
originally announced September 2022.
-
Memristive, Spintronic, and 2D-Materials-Based Devices to Improve and Complement Computing Hardware
Authors:
Dovydas Joksas,
AbdulAziz AlMutairi,
Oscar Lee,
Murat Cubukcu,
Antonio Lombardo,
Hidekazu Kurebayashi,
Anthony J. Kenyon,
Adnan Mehonic
Abstract:
In a data-driven economy, virtually all industries benefit from advances in information technology -- powerful computing systems are critically important for rapid technological progress. However, this progress might be at risk of slowing down if we do not address the discrepancy between our current computing power demands and what the existing technologies can offer. Key limitations to improving…
▽ More
In a data-driven economy, virtually all industries benefit from advances in information technology -- powerful computing systems are critically important for rapid technological progress. However, this progress might be at risk of slowing down if we do not address the discrepancy between our current computing power demands and what the existing technologies can offer. Key limitations to improving energy efficiency are the excessive growth of data transfer costs associated with the von Neumann architecture and the fundamental limits of complementary metal-oxide-semiconductor (CMOS) technologies, such as transistors. In this perspective article, we discuss three technologies that will likely play an essential role in future computing systems: memristive electronics, spintronics, and electronics based on 2D materials. We present how these may transform conventional digital computers and contribute to the adoption of new paradigms, like neuromorphic computing.
△ Less
Submitted 1 July, 2022; v1 submitted 11 March, 2022;
originally announced March 2022.
-
Nonideality-Aware Training for Accurate and Robust Low-Power Memristive Neural Networks
Authors:
Dovydas Joksas,
Erwei Wang,
Nikolaos Barmpatsalos,
Wing H. Ng,
Anthony J. Kenyon,
George A. Constantinides,
Adnan Mehonic
Abstract:
Recent years have seen a rapid rise of artificial neural networks being employed in a number of cognitive tasks. The ever-increasing computing requirements of these structures have contributed to a desire for novel technologies and paradigms, including memristor-based hardware accelerators. Solutions based on memristive crossbars and analog data processing promise to improve the overall energy eff…
▽ More
Recent years have seen a rapid rise of artificial neural networks being employed in a number of cognitive tasks. The ever-increasing computing requirements of these structures have contributed to a desire for novel technologies and paradigms, including memristor-based hardware accelerators. Solutions based on memristive crossbars and analog data processing promise to improve the overall energy efficiency. However, memristor nonidealities can lead to the degradation of neural network accuracy, while the attempts to mitigate these negative effects often introduce design trade-offs, such as those between power and reliability. In this work, we design nonideality-aware training of memristor-based neural networks capable of dealing with the most common device nonidealities. We demonstrate the feasibility of using high-resistance devices that exhibit high $I$-$V$ nonlinearity -- by analyzing experimental data and employing nonideality-aware training, we estimate that the energy efficiency of memristive vector-matrix multipliers is improved by three orders of magnitude ($0.715\ \mathrm{TOPs}^{-1}\mathrm{W}^{-1}$ to $381\ \mathrm{TOPs}^{-1}\mathrm{W}^{-1}$) while maintaining similar accuracy. We show that associating the parameters of neural networks with individual memristors allows to bias these devices towards less conductive states through regularization of the corresponding optimization problem, while modifying the validation procedure leads to more reliable estimates of performance. We demonstrate the universality and robustness of our approach when dealing with a wide range of nonidealities.
△ Less
Submitted 5 May, 2022; v1 submitted 13 December, 2021;
originally announced December 2021.
-
Brain-inspired computing: We need a master plan
Authors:
Adnan Mehonic,
Anthony J Kenyon
Abstract:
New computing technologies inspired by the brain promise fundamentally different ways to process information with extreme energy efficiency and the ability to handle the avalanche of unstructured and noisy data that we are generating at an ever-increasing rate. To realise this promise requires a brave and coordinated plan to bring together disparate research communities and to provide them with th…
▽ More
New computing technologies inspired by the brain promise fundamentally different ways to process information with extreme energy efficiency and the ability to handle the avalanche of unstructured and noisy data that we are generating at an ever-increasing rate. To realise this promise requires a brave and coordinated plan to bring together disparate research communities and to provide them with the funding, focus and support needed. We have done this in the past with digital technologies; we are in the process of doing it with quantum technologies; can we now do it for brain-inspired computing?
△ Less
Submitted 29 April, 2021;
originally announced April 2021.
-
Nanoscale plasticity and neuromorphic dynamics in silicon suboxide RRAM
Authors:
Mark Buckwell,
Wing Hung Ng,
Daniel John Mannion,
Stephen Hudziak,
Adnan Mehonic,
Anthony Joseph Kenyon
Abstract:
Resistive random-access memories, also known as memristors, whose resistance can be modulated by the electrically driven formation and disruption of conductive filaments within an insulator, are promising candidates for neuromorphic applications due to their scalability, low-power operation and diverse functional behaviours. However, understanding the dynamics of individual filaments, and the surr…
▽ More
Resistive random-access memories, also known as memristors, whose resistance can be modulated by the electrically driven formation and disruption of conductive filaments within an insulator, are promising candidates for neuromorphic applications due to their scalability, low-power operation and diverse functional behaviours. However, understanding the dynamics of individual filaments, and the surrounding material, is challenging, owing to the typically very large cross-sectional areas of test devices relative to the nanometre scale of individual filaments. In the present work, conductive atomic force microscopy is used to study the evolution of conductivity at the nanoscale in a fully CMOS-compatible silicon suboxide thin film. Distinct filamentary plasticity and background conductivity enhancement are reported, suggesting that device behaviour might be best described by composite core (filament) and shell (background conductivity) dynamics. Furthermore, constant current measurements demonstrate an interplay between filament formation and rupture, resulting in current-controlled voltage spiking in nanoscale regions, with an estimated optimal energy consumption of 25 attojoules per spike. This is very promising for extremely low-power neuromorphic computation and suggests that the dynamic behaviour observed in larger devices should persist and improve as dimensions are scaled down.
△ Less
Submitted 28 October, 2020;
originally announced October 2020.
-
Memristors -- from In-memory computing, Deep Learning Acceleration, Spiking Neural Networks, to the Future of Neuromorphic and Bio-inspired Computing
Authors:
Adnan Mehonic,
Abu Sebastian,
Bipin Rajendran,
Osvaldo Simeone,
Eleni Vasilaki,
Anthony J. Kenyon
Abstract:
Machine learning, particularly in the form of deep learning, has driven most of the recent fundamental developments in artificial intelligence. Deep learning is based on computational models that are, to a certain extent, bio-inspired, as they rely on networks of connected simple computing units operating in parallel. Deep learning has been successfully applied in areas such as object/pattern reco…
▽ More
Machine learning, particularly in the form of deep learning, has driven most of the recent fundamental developments in artificial intelligence. Deep learning is based on computational models that are, to a certain extent, bio-inspired, as they rely on networks of connected simple computing units operating in parallel. Deep learning has been successfully applied in areas such as object/pattern recognition, speech and natural language processing, self-driving vehicles, intelligent self-diagnostics tools, autonomous robots, knowledgeable personal assistants, and monitoring. These successes have been mostly supported by three factors: availability of vast amounts of data, continuous growth in computing power, and algorithmic innovations. The approaching demise of Moore's law, and the consequent expected modest improvements in computing power that can be achieved by scaling, raise the question of whether the described progress will be slowed or halted due to hardware limitations. This paper reviews the case for a novel beyond CMOS hardware technology, memristors, as a potential solution for the implementation of power-efficient in-memory computing, deep learning accelerators, and spiking neural networks. Central themes are the reliance on non-von-Neumann computing architectures and the need for develo** tailored learning and inference algorithms. To argue that lessons from biology can be useful in providing directions for further progress in artificial intelligence, we briefly discuss an example based reservoir computing. We conclude the review by speculating on the big picture view of future neuromorphic and brain-inspired computing systems.
△ Less
Submitted 30 April, 2020;
originally announced April 2020.
-
Committee machines -- a universal method to deal with non-idealities in memristor-based neural networks
Authors:
D. Joksas,
P. Freitas,
Z. Chai,
W. H. Ng,
M. Buckwell,
C. Li,
W. D. Zhang,
Q. Xia,
A. J. Kenyon,
A. Mehonic
Abstract:
Artificial neural networks are notoriously power- and time-consuming when implemented on conventional von Neumann computing systems. Consequently, recent years have seen an emergence of research in machine learning hardware that strives to bring memory and computing closer together. A popular approach is to realise artificial neural networks in hardware by implementing their synaptic weights using…
▽ More
Artificial neural networks are notoriously power- and time-consuming when implemented on conventional von Neumann computing systems. Consequently, recent years have seen an emergence of research in machine learning hardware that strives to bring memory and computing closer together. A popular approach is to realise artificial neural networks in hardware by implementing their synaptic weights using memristive devices. However, various device- and system-level non-idealities usually prevent these physical implementations from achieving high inference accuracy. We suggest applying a well-known concept in computer science -- committee machines -- in the context of memristor-based neural networks. Using simulations and experimental data from three different types of memristive devices, we show that committee machines employing ensemble averaging can successfully increase inference accuracy in physically implemented neural networks that suffer from faulty devices, device-to-device variability, random telegraph noise and line resistance. Importantly, we demonstrate that the accuracy can be improved even without increasing the total number of memristors.
△ Less
Submitted 27 August, 2020; v1 submitted 14 September, 2019;
originally announced September 2019.
