Showing 1–35 of 35 results for author: Kvatinsky, S

Roadmap to Neuromorphic Computing with Emerging Technologies

Authors: Adnan Mehonic, Daniele Ielmini, Kaushik Roy, Onur Mutlu, Shahar Kvatinsky, Teresa Serrano-Gotarredona, Bernabe Linares-Barranco, Sabina Spiga, Sergey Savelev, Alexander G Balanov, Nitin Chawla, Giuseppe Desoli, Gerardo Malavena, Christian Monzio Compagnoni, Zhongrui Wang, J Joshua Yang, Ghazi Sarwat Syed, Abu Sebastian, Thomas Mikolajick, Beatriz Noheda, Stefan Slesazeck, Bernard Dieny, Tuo-Hung, Hou, Akhil Varri , et al. (28 additional authors not shown)

Abstract: The roadmap is organized into several thematic sections, outlining current computing challenges, discussing the neuromorphic computing approach, analyzing mature and currently utilized technologies, providing an overview of emerging technologies, addressing material challenges, exploring novel computing concepts, and finally examining the maturity level of emerging technologies while determining t… ▽ More The roadmap is organized into several thematic sections, outlining current computing challenges, discussing the neuromorphic computing approach, analyzing mature and currently utilized technologies, providing an overview of emerging technologies, addressing material challenges, exploring novel computing concepts, and finally examining the maturity level of emerging technologies while determining the next essential steps for their advancement. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Comments: 90 pages, 22 figures, roadmap

A Pipelined Memristive Neural Network Analog-to-Digital Converter

Authors: Loai Danial, Kanishka Sharma, Shahar Kvatinsky

Abstract: With the advent of high-speed, high-precision, and low-power mixed-signal systems, there is an ever-growing demand for accurate, fast, and energy-efficient analog-to-digital (ADCs) and digital-to-analog converters (DACs). Unfortunately, with the downscaling of CMOS technology, modern ADCs trade off speed, power and accuracy. Recently, memristive neuromorphic architectures of four-bit ADC/DAC have… ▽ More With the advent of high-speed, high-precision, and low-power mixed-signal systems, there is an ever-growing demand for accurate, fast, and energy-efficient analog-to-digital (ADCs) and digital-to-analog converters (DACs). Unfortunately, with the downscaling of CMOS technology, modern ADCs trade off speed, power and accuracy. Recently, memristive neuromorphic architectures of four-bit ADC/DAC have been proposed. Such converters can be trained in real-time using machine learning algorithms, to break through the speedpower-accuracy trade-off while optimizing the conversion performance for different applications. However, scaling such architectures above four bits is challenging. This paper proposes a scalable and modular neural network ADC architecture based on a pipeline of four-bit converters, preserving their inherent advantages in application reconfiguration, mismatch selfcalibration, noise tolerance, and power optimization, while approaching higher resolution and throughput in penalty of latency. SPICE evaluation shows that an 8-bit pipelined ADC achieves 0.18 LSB INL, 0.20 LSB DNL, 7.6 ENOB, and 0.97 fJ/conv FOM. This work presents a significant step towards the realization of large-scale neuromorphic data converters. △ Less

Submitted 4 June, 2024; originally announced June 2024.

Transimpedance Amplifier with Automatic Gain Control Based on Memristors for Optical Signal Acquisition

Authors: Sariel Hodisan, Shahar Kvatinsky

Abstract: Transimpedance amplifiers (TIA) play a crucial role in various electronic systems, especially in optical signal acquisition. However, their performance is often hampered by saturation issues due to high input currents, leading to prolonged recovery times. This paper addresses this challenge by introducing a novel approach utilizing a memristive automatic gain control (AGC) to adjust the TIA's gain… ▽ More Transimpedance amplifiers (TIA) play a crucial role in various electronic systems, especially in optical signal acquisition. However, their performance is often hampered by saturation issues due to high input currents, leading to prolonged recovery times. This paper addresses this challenge by introducing a novel approach utilizing a memristive automatic gain control (AGC) to adjust the TIA's gain and enhance its dynamic range. We replace the typical feedback resistor of a TIA with a valence-change mechanism (VCM) memristor. This substitution enables the TIA to adapt to a broader range of input signals, leveraging the substantial OFF/ON resistance ratio of the memristor. This paper also presents the reading and resetting sub-circuits essential for monitoring and controling the memristor's state. The proposed circuit is evaluated through SPICE simulations. Furthermore, we extend our evaluation to practical testing using a printed circuit board (PCB) integrating the TIA and memristor. We show a remarkable 40 dB increase in the dynamic range of our TIA memristor circuit compared to traditional resistor-based TIAs. △ Less

Submitted 3 May, 2024; originally announced May 2024.

Experimental Demonstration of Non-Stateful In-Memory Logic with 1T1R OxRAM Valence Change Mechanism Memristors

Authors: Henriette Padberg, Amir Regev, Giuseppe Piccolboni, Alessandro Bricalli, Gabriel Molas, Jean Francois Nodin, Shahar Kvatinsky

Abstract: Processing-in-memory (PIM) is attractive to overcome the limitations of modern computing systems. Numerous PIM systems exist, varying by the technologies and logic techniques used. Successful operation of specific logic functions is crucial for effective processing-in-memory. Memristive non-stateful logic techniques are compatible with CMOS logic and can be integrated into a 1T1R memory array, sim… ▽ More Processing-in-memory (PIM) is attractive to overcome the limitations of modern computing systems. Numerous PIM systems exist, varying by the technologies and logic techniques used. Successful operation of specific logic functions is crucial for effective processing-in-memory. Memristive non-stateful logic techniques are compatible with CMOS logic and can be integrated into a 1T1R memory array, similar to commercial RRAM products. This paper analyzes and demonstrates two non-stateful logic techniques: 1T1R logic and scouting logic. As a first step, the used 1T1R SiO\textsubscript{x} valence change mechanism memristors are characterized in reference to their feasibility to perform logic functions. Various logical functions of the two logic techniques are experimentally demonstrated, showing correct functionality in all cases. Following the results, the challenges and limitations of the RRAM characteristics and 1T1R configuration for the application in logical functions are discussed. △ Less

Submitted 6 October, 2023; originally announced October 2023.

Comments: 5 pages, 6 figures

Journal ref: IEEE Transactions on Circuits and Systems II: Express Briefs (2023), pages 1-1

TDPP: Two-Dimensional Permutation-Based Protection of Memristive Deep Neural Networks

Authors: Minhui Zou, Zhenhua Zhu, Tzofnat Greenberg-Toledo, Orian Leitersdorf, Jiang Li, Junlong Zhou, Yu Wang, Nan Du, Shahar Kvatinsky

Abstract: The execution of deep neural network (DNN) algorithms suffers from significant bottlenecks due to the separation of the processing and memory units in traditional computer systems. Emerging memristive computing systems introduce an in situ approach that overcomes this bottleneck. The non-volatility of memristive devices, however, may expose the DNN weights stored in memristive crossbars to potenti… ▽ More The execution of deep neural network (DNN) algorithms suffers from significant bottlenecks due to the separation of the processing and memory units in traditional computer systems. Emerging memristive computing systems introduce an in situ approach that overcomes this bottleneck. The non-volatility of memristive devices, however, may expose the DNN weights stored in memristive crossbars to potential theft attacks. Therefore, this paper proposes a two-dimensional permutation-based protection (TDPP) method that thwarts such attacks. We first introduce the underlying concept that motivates the TDPP method: permuting both the rows and columns of the DNN weight matrices. This contrasts with previous methods, which focused solely on permuting a single dimension of the weight matrices, either the rows or columns. While it's possible for an adversary to access the matrix values, the original arrangement of rows and columns in the matrices remains concealed. As a result, the extracted DNN model from the accessed matrix values would fail to operate correctly. We consider two different memristive computing systems (designed for layer-by-layer and layer-parallel processing, respectively) and demonstrate the design of the TDPP method that could be embedded into the two systems. Finally, we present a security analysis. Our experiments demonstrate that TDPP can achieve comparable effectiveness to prior approaches, with a high level of security when appropriately parameterized. In addition, TDPP is more scalable than previous methods and results in reduced area and power overheads. The area and power are reduced by, respectively, 1218$\times$ and 2815$\times$ for the layer-by-layer system and by 178$\times$ and 203$\times$ for the layer-parallel system compared to prior works. △ Less

Submitted 10 October, 2023; originally announced October 2023.

Comments: 14 pages, 11 figures

CUDA-PIM: End-to-End Integration of Digital Processing-in-Memory from High-Level C++ to Microarchitectural Design

Authors: Orian Leitersdorf, Ronny Ronen, Shahar Kvatinsky

Abstract: Digital processing-in-memory (PIM) architectures mitigate the memory wall problem by facilitating parallel bitwise operations directly within memory. Recent works have demonstrated their algorithmic potential for accelerating data-intensive applications; however, there remains a significant gap in the programming model and microarchitectural design. This is further exacerbated by the emerging mode… ▽ More Digital processing-in-memory (PIM) architectures mitigate the memory wall problem by facilitating parallel bitwise operations directly within memory. Recent works have demonstrated their algorithmic potential for accelerating data-intensive applications; however, there remains a significant gap in the programming model and microarchitectural design. This is further exacerbated by the emerging model of partitions, which significantly complicates control and periphery. Therefore, inspired by NVIDIA CUDA, this paper provides an end-to-end architectural integration of digital memristive PIM from an abstract high-level C++ programming interface for vector operations to the low-level microarchitecture. We begin by proposing an efficient microarchitecture and instruction set architecture (ISA) that bridge the gap between the low-level control periphery and an abstraction of PIM parallelism into warps and threads. We subsequently propose a PIM compilation library that converts high-level C++ to ISA instructions, and a PIM driver that translates ISA instructions into PIM micro-operations. This drastically simplifies the development of PIM applications and enables PIM integration within larger existing C++ CPU/GPU programs for heterogeneous computing with significant ease. Lastly, we present an efficient GPU-accelerated simulator for the proposed PIM microarchitecture. Although slower than a theoretical PIM chip, this simulator provides an accessible platform for developers to start executing and debugging PIM algorithms. To validate our approach, we implement state-of-the-art matrix operations and FFT PIM-based algorithms as case studies. These examples demonstrate drastically simplified development without compromising performance, showing the potential and significance of CUDA-PIM. △ Less

Submitted 27 August, 2023; originally announced August 2023.

An Asynchronous and Low-Power True Random Number Generator using STT-MTJ

Authors: Ben Perach, Shahar Kvatinsky

Abstract: The emerging Spin Transfer Torque Magnetic Tunnel Junction (STT-MTJ) technology exhibits interesting stochastic behavior combined with small area and low operation energy. It is, therefore, a promising technology for security applications, specifically the generation of random numbers. In this paper, STT-MTJ is used to construct an asynchronous true random number generator (TRNG) with low power an… ▽ More The emerging Spin Transfer Torque Magnetic Tunnel Junction (STT-MTJ) technology exhibits interesting stochastic behavior combined with small area and low operation energy. It is, therefore, a promising technology for security applications, specifically the generation of random numbers. In this paper, STT-MTJ is used to construct an asynchronous true random number generator (TRNG) with low power and a high entropy rate. The asynchronous design enables decoupling of the random number generation from the system clock, allowing it to be embedded in low-power devices. The proposed TRNG is evaluated by a numerical simulation, using the Landau-Lifshitz-Gilbert (LLG) equation as the model of the STT-MTJ devices. Design considerations, attack analysis, and process variation are discussed and evaluated. We show that our design is robust to process variation, achieving an entropy generating rate between 99.7Mbps and 127.8Mbps with 6-7.7 pJ per bit for 90% of the instances. △ Less

Submitted 26 July, 2023; originally announced July 2023.

Comments: Published in: IEEE Transactions on Very Large Scale Integration (VLSI) Systems ( Volume: 27, Issue: 11, November 2019)

Accelerating Relational Database Analytical Processing with Bulk-Bitwise Processing-in-Memory

Authors: Ben Perach, Ronny Ronen, Shahar Kvatinsky

Abstract: Online Analytical Processing (OLAP) for relational databases is a business decision support application. The application receives queries about the business database, usually requesting to summarize many database records, and produces few results. Existing OLAP requires transferring a large amount of data between the memory and the CPU, having a few operations per datum, and producing a small outp… ▽ More Online Analytical Processing (OLAP) for relational databases is a business decision support application. The application receives queries about the business database, usually requesting to summarize many database records, and produces few results. Existing OLAP requires transferring a large amount of data between the memory and the CPU, having a few operations per datum, and producing a small output. Hence, OLAP is a good candidate for processing-in-memory (PIM), where computation is performed where the data is stored, thus accelerating applications by reducing data movement between the memory and CPU. In particular, bulk-bitwise PIM, where the memory array is a bit-vector processing unit, seems a good match for OLAP. With the extensive inherent parallelism and minimal data movement of bulk-bitwise PIM, OLAP applications can process the entire database in parallel in memory, transferring only the results to the CPU. This paper shows a full stack adaptation of a bulk-bitwise PIM, from compiling SQL to hardware implementation, for supporting OLAP applications. Evaluating the Star Schema Benchmark (SSB), bulk-bitwise PIM achieves a 4.65X speedup over Monet-DB, a standard database system. △ Less

Submitted 2 July, 2023; originally announced July 2023.

Comments: Presented at the 21st IEEE Interregional NEWCAS conference in Edinburgh, Scotland, on June 2023

ConvPIM: Evaluating Digital Processing-in-Memory through Convolutional Neural Network Acceleration

Authors: Orian Leitersdorf, Ronny Ronen, Shahar Kvatinsky

Abstract: Processing-in-memory (PIM) architectures are emerging to reduce data movement in data-intensive applications. These architectures seek to exploit the same physical devices for both information storage and logic, thereby dwarfing the required data transfer and utilizing the full internal memory bandwidth. Whereas analog PIM utilizes the inherent connectivity of crossbar arrays for approximate matri… ▽ More Processing-in-memory (PIM) architectures are emerging to reduce data movement in data-intensive applications. These architectures seek to exploit the same physical devices for both information storage and logic, thereby dwarfing the required data transfer and utilizing the full internal memory bandwidth. Whereas analog PIM utilizes the inherent connectivity of crossbar arrays for approximate matrix-vector multiplication in the analog domain, digital PIM architectures enable bitwise logic operations with massive parallelism across columns of data within memory arrays. Several recent works have extended the computational capabilities of digital PIM architectures towards the full-precision (single-precision floating-point) acceleration of convolutional neural networks (CNNs); yet, they lack a comprehensive comparison to GPUs. In this paper, we examine the potential of digital PIM for CNN acceleration through an updated quantitative comparison with GPUs, supplemented with an analysis of the overall limitations of digital PIM. We begin by investigating the different PIM architectures from a theoretical perspective to understand the underlying performance limitations and improvements compared to state-of-the-art hardware. We then uncover the tradeoffs between the different strategies through a series of benchmarks ranging from memory-bound vectored arithmetic to CNN acceleration. We conclude with insights into the general performance of digital PIM architectures for different data-intensive applications. △ Less

Submitted 6 May, 2023; originally announced May 2023.

FourierPIM: High-Throughput In-Memory Fast Fourier Transform and Polynomial Multiplication

Authors: Orian Leitersdorf, Yahav Boneh, Gonen Gazit, Ronny Ronen, Shahar Kvatinsky

Abstract: The Discrete Fourier Transform (DFT) is essential for various applications ranging from signal processing to convolution and polynomial multiplication. The groundbreaking Fast Fourier Transform (FFT) algorithm reduces DFT time complexity from the naive O(n^2) to O(n log n), and recent works have sought further acceleration through parallel architectures such as GPUs. Unfortunately, accelerators su… ▽ More The Discrete Fourier Transform (DFT) is essential for various applications ranging from signal processing to convolution and polynomial multiplication. The groundbreaking Fast Fourier Transform (FFT) algorithm reduces DFT time complexity from the naive O(n^2) to O(n log n), and recent works have sought further acceleration through parallel architectures such as GPUs. Unfortunately, accelerators such as GPUs cannot exploit their full computing capabilities as memory access becomes the bottleneck. Therefore, this paper accelerates the FFT algorithm using digital Processing-in-Memory (PIM) architectures that shift computation into the memory by exploiting physical devices capable of storage and logic (e.g., memristors). We propose an O(log n) in-memory FFT algorithm that can also be performed in parallel across multiple arrays for high-throughput batched execution, supporting both fixed-point and floating-point numbers. Through the convolution theorem, we extend this algorithm to O(log n) polynomial multiplication - a fundamental task for applications such as cryptography. We evaluate FourierPIM on a publicly-available cycle-accurate simulator that verifies both correctness and performance, and demonstrate 5-15x throughput and 4-13x energy improvement over the NVIDIA cuFFT library on state-of-the-art GPUs for FFT and polynomial multiplication. △ Less

Submitted 5 April, 2023; originally announced April 2023.

Journal ref: Memories - Materials, Devices, Circuits and Systems, Volume 4, 2023

ClaPIM: Scalable Sequence CLAssification using Processing-In-Memory

Authors: Marcel Khalifa, Barak Hoffer, Orian Leitersdorf, Robert Hanhan, Ben Perach, Leonid Yavits, Shahar Kvatinsky

Abstract: DNA sequence classification is a fundamental task in computational biology with vast implications for applications such as disease prevention and drug design. Therefore, fast high-quality sequence classifiers are significantly important. This paper introduces ClaPIM, a scalable DNA sequence classification architecture based on the emerging concept of hybrid in-crossbar and near-crossbar memristive… ▽ More DNA sequence classification is a fundamental task in computational biology with vast implications for applications such as disease prevention and drug design. Therefore, fast high-quality sequence classifiers are significantly important. This paper introduces ClaPIM, a scalable DNA sequence classification architecture based on the emerging concept of hybrid in-crossbar and near-crossbar memristive processing-in-memory (PIM). We enable efficient and high-quality classification by uniting the filter and search stages within a single algorithm. Specifically, we propose a custom filtering technique that drastically narrows the search space and a search approach that facilitates approximate string matching through a distance function. ClaPIM is the first PIM architecture for scalable approximate string matching that benefits from the high density of memristive crossbar arrays and the massive computational parallelism of PIM. Compared with Kraken2, a state-of-the-art software classifier, ClaPIM provides significantly higher classification quality (up to 20x improvement in F1 score) and also demonstrates a 1.8x throughput improvement. Compared with EDAM, a recently-proposed SRAM-based accelerator that is restricted to small datasets, we observe both a 30.4x improvement in normalized throughput per area and a 7% increase in classification precision. △ Less

Submitted 5 November, 2023; v1 submitted 16 February, 2023; originally announced February 2023.

Enabling Relational Database Analytical Processing in Bulk-Bitwise Processing-In-Memory

Authors: Ben Perach, Ronny Ronen, Shahar Kvatinsky

Abstract: Bulk-bitwise processing-in-memory (PIM), an emerging computational paradigm utilizing memory arrays as computational units, has been shown to benefit database applications. This paper demonstrates how GROUP-BY and JOIN, database operations not supported by previous works, can be performed efficiently in bulk-bitwise PIM for relational database analytical processing. We extend the gem5 simulator an… ▽ More Bulk-bitwise processing-in-memory (PIM), an emerging computational paradigm utilizing memory arrays as computational units, has been shown to benefit database applications. This paper demonstrates how GROUP-BY and JOIN, database operations not supported by previous works, can be performed efficiently in bulk-bitwise PIM for relational database analytical processing. We extend the gem5 simulator and evaluated our hardware modifications on the Star Schema Benchmark. We show that compared to previous works, our modifications improve (on average) execution time by 1.83X, energy by 4.31X, and the system's lifetime by 3.21X. We also achieved a speedup of 4.65X over MonetDB, a modern state-of-the-art in-memory database. △ Less

Submitted 2 November, 2023; v1 submitted 3 February, 2023; originally announced February 2023.

Comments: Included in conference proceedings of the 2023 IEEE 36th International System-on-Chip Conference (SOCC)

Stateful Logic using Phase Change Memory

Authors: Barak Hoffer, Nicolás Wainstein, Christopher M. Neumann, Eric Pop, Eilam Yalon, Shahar Kvatinsky

Abstract: Stateful logic is a digital processing-in-memory technique that could address von Neumann memory bottleneck challenges while maintaining backward compatibility with standard von Neumann architectures. In stateful logic, memory cells are used to perform the logic operations without reading or moving any data outside the memory array. Stateful logic has been previously demonstrated using several res… ▽ More Stateful logic is a digital processing-in-memory technique that could address von Neumann memory bottleneck challenges while maintaining backward compatibility with standard von Neumann architectures. In stateful logic, memory cells are used to perform the logic operations without reading or moving any data outside the memory array. Stateful logic has been previously demonstrated using several resistive memory types, mostly by resistive RAM (RRAM). Here we present a new method to design stateful logic using a different resistive memory - phase change memory (PCM). We propose and experimentally demonstrate four logic gate types (NOR, IMPLY, OR, NIMP) using commonly used PCM materials. Our stateful logic circuits are different than previously proposed circuits due to the different switching mechanism and functionality of PCM compared to RRAM. Since the proposed stateful logic form a functionally complete set, these gates enable sequential execution of any logic function within the memory, paving the way to PCM-based digital processing-in-memory systems. △ Less

Submitted 29 December, 2022; originally announced December 2022.

Journal ref: IEEE Journal on Exploratory Solid-State Computational Devices and Circuits (Volume: 8, Issue: 2, December 2022)

Review of security techniques for memristor computing systems

Authors: Minhui Zou, Nan Du, Shahar Kvatinsky

Abstract: Neural network (NN) algorithms have become the dominant tool in visual object recognition, natural language processing, and robotics. To enhance the computational efficiency of these algorithms, in comparison to the traditional von Neuman computing architectures, researchers have been focusing on memristor computing systems. A major drawback when using memristor computing systems today is that, in… ▽ More Neural network (NN) algorithms have become the dominant tool in visual object recognition, natural language processing, and robotics. To enhance the computational efficiency of these algorithms, in comparison to the traditional von Neuman computing architectures, researchers have been focusing on memristor computing systems. A major drawback when using memristor computing systems today is that, in the artificial intelligence (AI) era, well-trained NN models are intellectual property and, when loaded in the memristor computing systems, face theft threats, especially when running in edge devices. An adversary may steal the well-trained NN models through advanced attacks such as learning attacks and side-channel analysis. In this paper, we review different security techniques for protecting memristor computing systems. Two threat models are described based on their assumptions regarding the adversary's capabilities: a black-box (BB) model and a white-box (WB) model. We categorize the existing security techniques into five classes in the context of these threat models: thwarting learning attacks (BB), thwarting side-channel attacks (BB), NN model encryption (WB), NN weight transformation (WB), and fingerprint embedding (WB). We also present a cross-comparison of the limitations of the security techniques. This paper could serve as an aid when designing secure memristor computing systems. △ Less

Submitted 19 December, 2022; originally announced December 2022.

Comments: 15 pages, 5 figures

Journal ref: Front. Electron. Mater, 19 December 2022, Sec. Semiconducting Materials and Devices Sec. Semiconducting Materials and Devices

On Consistency for Bulk-Bitwise Processing-in-Memory

Authors: Ben Perach, Ronny Ronnen, Shahar Kvatinsky

Abstract: Processing-in-memory (PIM) architectures allow software to explicitly initiate computation in the memory. This effectively makes PIM operations a new class of memory operations, alongside standard memory operations (e.g., load, store). For software correctness, it is crucial to have ordering rules for a PIM operation with other PIM operations and other memory operations, i.e., a consistency model… ▽ More Processing-in-memory (PIM) architectures allow software to explicitly initiate computation in the memory. This effectively makes PIM operations a new class of memory operations, alongside standard memory operations (e.g., load, store). For software correctness, it is crucial to have ordering rules for a PIM operation with other PIM operations and other memory operations, i.e., a consistency model that takes into account PIM operations is vital. To the best of our knowledge, little attention to PIM operation consistency has been given in existing works. In this paper, we focus on a specific PIM approach, named bulk-bitwise PIM. In bulk-bitwise PIM, large bitwise operations are performed directly and stored in the memory array. We show that previous solutions for the related topic of maintaining coherency of bulk-bitwise PIM have broken the host native consistency model and prevent any guaranteed correctness. As a solution, we propose and evaluate four consistency models for bulk-bitwise PIM, from strict to relaxed. Our designs also preserve coherency between PIM and the host processor. Evaluating the proposed designs' performance with a gem5 simulation, using the YCSB short-range scan benchmark and TPC-H queries, shows that the run time overhead of guaranteeing correctness is at most $6\%$, and in many cases the run time is even improved. The hardware overhead of our design is less than $0.22\%$. △ Less

Submitted 7 December, 2022; v1 submitted 14 November, 2022; originally announced November 2022.

Comments: To be published in the 29th IEEE International Symposium on High-Performance Computer Architecture (HPCA-29)

abstractPIM: A Technology Backward-Compatible Compilation Flow for Processing-In-Memory

Authors: Adi Eliahu, Rotem Ben-Hur, Ronny Ronen, Shahar Kvatinsky

Abstract: The von Neumann architecture, in which the memory and the computation units are separated, demands massive data traffic between the memory and the CPU. To reduce data movement, new technologies and computer architectures have been explored. The use of memristors, which are devices with both memory and computation capabilities, has been considered for different processing-in-memory (PIM) solutions,… ▽ More The von Neumann architecture, in which the memory and the computation units are separated, demands massive data traffic between the memory and the CPU. To reduce data movement, new technologies and computer architectures have been explored. The use of memristors, which are devices with both memory and computation capabilities, has been considered for different processing-in-memory (PIM) solutions, including using memristive stateful logic for a programmable digital PIM system. Nevertheless, all previous work has focused on a specific stateful logic family, and on optimizing the execution for a certain target machine. These solutions require new compiler and compilation when changing the target machine, and provide no backward compatibility with other target machines. In this chapter, we present abstractPIM, a new compilation concept and flow which enables executing any function within the memory, using different stateful logic families and different instruction set architectures (ISAs). By separating the code generation into two independent components, intermediate representation of the code using target independent ISA and then microcode generation for a specific target machine, we provide a flexible flow with backward compatibility and lay foundations for a PIM compiler. Using abstractPIM, we explore various logic technologies and ISAs and how they impact each other, and discuss the challenges associated with it, such as the increase in execution time. △ Less

Submitted 30 August, 2022; originally announced August 2022.

Comments: Link to the book chapter in Springer: https://link.springer.com/chapter/10.1007/978-3-030-81641-4_16

Performing Stateful Logic Using Spin-Orbit Torque (SOT) MRAM

Authors: Barak Hoffer, Shahar Kvatinsky

Abstract: Stateful logic is a promising processing-in-memory (PIM) paradigm to perform logic operations using emerging nonvolatile memory cells. While most stateful logic circuits to date focused on technologies such as resistive RAM, we propose two approaches to designing stateful logic using spin orbit torque (SOT) MRAM. The first approach utilizes the separation of read and write paths in SOT devices to… ▽ More Stateful logic is a promising processing-in-memory (PIM) paradigm to perform logic operations using emerging nonvolatile memory cells. While most stateful logic circuits to date focused on technologies such as resistive RAM, we propose two approaches to designing stateful logic using spin orbit torque (SOT) MRAM. The first approach utilizes the separation of read and write paths in SOT devices to perform logic operations. In contrast to previous work, our method utilizes a standard memory structure, and each row can be used as input or output. The second approach uses voltage-gated SOT switching to allow stateful logic in denser memory arrays. We present array structures to support the two approaches and evaluate their functionality using SPICE simulations in the presence of process variation and device mismatch. △ Less

Submitted 1 August, 2022; originally announced August 2022.

Comments: Published in 2022 22nd IEEE International Conference on Nanotechnology (NANO)

MatPIM: Accelerating Matrix Operations with Memristive Stateful Logic

Authors: Orian Leitersdorf, Ronny Ronen, Shahar Kvatinsky

Abstract: The emerging memristive Memory Processing Unit (mMPU) overcomes the memory wall through memristive devices that unite storage and logic for real processing-in-memory (PIM) systems. At the core of the mMPU is stateful logic, which is accelerated with memristive partitions to enable logic with massive inherent parallelism within crossbar arrays. This paper vastly accelerates the fundamental operatio… ▽ More The emerging memristive Memory Processing Unit (mMPU) overcomes the memory wall through memristive devices that unite storage and logic for real processing-in-memory (PIM) systems. At the core of the mMPU is stateful logic, which is accelerated with memristive partitions to enable logic with massive inherent parallelism within crossbar arrays. This paper vastly accelerates the fundamental operations of matrix-vector multiplication and convolution in the mMPU, with either full-precision or binary elements. These proposed algorithms establish an efficient foundation for large-scale mMPU applications such as neural-networks, image processing, and numerical methods. We overcome the inherent asymmetry limitation in the previous in-memory full-precision matrix-vector multiplication solutions by utilizing techniques from block matrix multiplication and reduction. We present the first fast in-memory binary matrix-vector multiplication algorithm by utilizing memristive partitions with a tree-based popcount reduction (39x faster than previous work). For convolution, we present a novel in-memory input-parallel concept which we utilize for a full-precision algorithm that overcomes the asymmetry limitation in convolution, while also improving latency (2x faster than previous work), and the first fast binary algorithm (12x faster than previous work). △ Less

Submitted 30 June, 2022; originally announced June 2022.

Enhancing Security of Memristor Computing System Through Secure Weight Map**

Authors: Minhui Zou, Junlong Zhou, Xiaotong Cui, Wei Wang, Shahar Kvatinsky

Abstract: Emerging memristor computing systems have demonstrated great promise in improving the energy efficiency of neural network (NN) algorithms. The NN weights stored in memristor crossbars, however, may face potential theft attacks due to the nonvolatility of the memristor devices. In this paper, we propose to protect the NN weights by map** selected columns of them in the form of 1's complements and… ▽ More Emerging memristor computing systems have demonstrated great promise in improving the energy efficiency of neural network (NN) algorithms. The NN weights stored in memristor crossbars, however, may face potential theft attacks due to the nonvolatility of the memristor devices. In this paper, we propose to protect the NN weights by map** selected columns of them in the form of 1's complements and leaving the other columns in their original form, preventing the adversary from knowing the exact representation of each weight. The results show that compared with prior work, our method achieves effectiveness comparable to the best of them and reduces the hardware overhead by more than 18X. △ Less

Submitted 29 June, 2022; originally announced June 2022.

Comments: 6 pages, 4 figures, accepted by IEEE ISVLSI 2022

AritPIM: High-Throughput In-Memory Arithmetic

Authors: Orian Leitersdorf, Dean Leitersdorf, Jonathan Gal, Mor Dahan, Ronny Ronen, Shahar Kvatinsky

Abstract: Digital processing-in-memory (PIM) architectures are rapidly emerging to overcome the memory-wall bottleneck by integrating logic within memory elements. Such architectures provide vast computational power within the memory itself in the form of parallel bitwise logic operations. We develop novel algorithmic techniques for PIM that, combined with new perspectives on computer arithmetic, extend thi… ▽ More Digital processing-in-memory (PIM) architectures are rapidly emerging to overcome the memory-wall bottleneck by integrating logic within memory elements. Such architectures provide vast computational power within the memory itself in the form of parallel bitwise logic operations. We develop novel algorithmic techniques for PIM that, combined with new perspectives on computer arithmetic, extend this bitwise parallelism to the four fundamental arithmetic operations (addition, subtraction, multiplication, and division), for both fixed-point and floating-point numbers, and using both bit-serial and bit-parallel approaches. We propose a state-of-the-art suite of arithmetic algorithms, demonstrating the first algorithm in the literature of digital PIM for a majority of cases - including cases previously considered impossible for digital PIM, such as floating-point addition. Through a case study on memristive PIM, we compare the proposed algorithms to an NVIDIA RTX 3070 GPU and demonstrate significant throughput and energy improvements. △ Less

Submitted 15 April, 2023; v1 submitted 8 June, 2022; originally announced June 2022.

Comments: Accepted to IEEE Transactions on Emerging Topics in Computing (TETC)

PartitionPIM: Practical Memristive Partitions for Fast Processing-in-Memory

Authors: Orian Leitersdorf, Ronny Ronen, Shahar Kvatinsky

Abstract: Digital memristive processing-in-memory overcomes the memory wall through a fundamental storage device capable of stateful logic within crossbar arrays. Dynamically dividing the crossbar arrays by adding memristive partitions further increases parallelism, thereby overcoming an inherent trade-off in memristive processing-in-memory. The algorithmic topology of partitions is highly unique, and was r… ▽ More Digital memristive processing-in-memory overcomes the memory wall through a fundamental storage device capable of stateful logic within crossbar arrays. Dynamically dividing the crossbar arrays by adding memristive partitions further increases parallelism, thereby overcoming an inherent trade-off in memristive processing-in-memory. The algorithmic topology of partitions is highly unique, and was recently exploited to accelerate multiplication (11x with 32 partitions) and sorting (14x with 16 partitions). Yet, the physical implementation of memristive partitions, such as the peripheral decoders and the control message, has never been considered and may lead to vast impracticality. This paper overcomes that challenge with several novel techniques, presenting efficient practical designs of memristive partitions. We begin by formalizing the algorithmic properties of memristive partitions into serial, parallel, and semi-parallel operations. Peripheral overhead is addressed via a novel technique of half-gates that enables efficient decoding with negligible overhead. Control overhead is addressed by carefully reducing the operation set of memristive partitions, while resulting in negligible performance impact, by utilizing techniques such as shared indices and pattern generators. Ultimately, these efficient practical solutions, combined with the vast algorithmic potential, may revolutionize digital memristive processing-in-memory. △ Less

Submitted 8 June, 2022; originally announced June 2022.

Making Real Memristive Processing-in-Memory Faster and Reliable

Authors: Shahar Kvatinsky

Abstract: Memristive technologies are attractive candidates to replace conventional memory technologies, and can also be used to perform logic and arithmetic operations using a technique called 'stateful logic.' Combining data storage and computation in the memory array enables a novel non-von Neumann architecture, where both the operations are performed within a memristive Memory Processing Unit (mMPU). Th… ▽ More Memristive technologies are attractive candidates to replace conventional memory technologies, and can also be used to perform logic and arithmetic operations using a technique called 'stateful logic.' Combining data storage and computation in the memory array enables a novel non-von Neumann architecture, where both the operations are performed within a memristive Memory Processing Unit (mMPU). The mMPU relies on adding computing capabilities to the memristive memory cells without changing the basic memory array structure. The use of an mMPU alleviates the primary restriction on performance and energy in a von Neumann machine, which is the data transfer between CPU and memory. Here, the various aspects of mMPU are discussed, including its architecture and implications on the computing system and software, as well as examining the microarchitectural aspects. We show how mMPU can be improved to accelerate different applications and how the poor reliability of memristors can be improved as part of the mMPU operation. △ Less

Submitted 29 May, 2022; originally announced May 2022.

HashPIM: High-Throughput SHA-3 via Memristive Digital Processing-in-Memory

Authors: Batel Oved, Orian Leitersdorf, Ronny Ronen, Shahar Kvatinsky

Abstract: Recent research has sought to accelerate cryptographic hash functions as they are at the core of modern cryptography. Traditional designs, however, suffer from the von Neumann bottleneck that originates from the separation of processing and memory units. An emerging solution to overcome this bottleneck is processing-in-memory (PIM): performing logic within the same devices responsible for memory t… ▽ More Recent research has sought to accelerate cryptographic hash functions as they are at the core of modern cryptography. Traditional designs, however, suffer from the von Neumann bottleneck that originates from the separation of processing and memory units. An emerging solution to overcome this bottleneck is processing-in-memory (PIM): performing logic within the same devices responsible for memory to eliminate data-transfer and simultaneously provide massive computational parallelism. In this paper, we seek to vastly accelerate the state-of-the-art SHA-3 cryptographic function using the memristive memory processing unit (mMPU), a general-purpose memristive PIM architecture. To that end, we propose a novel in-memory algorithm for variable rotation, and utilize an efficient map** of the SHA-3 state vector for memristive crossbar arrays to efficiently exploit PIM parallelism. We demonstrate a massive energy efficiency of 1,422 Gbps/W, improving a state-of-the-art memristive SHA-3 accelerator (SHINE-2) by 4.6x. △ Less

Submitted 1 June, 2022; v1 submitted 26 May, 2022; originally announced May 2022.

Comments: Accepted to International Conference on Modern Circuits and Systems Technologies (MOCAST) 2022

Understanding Bulk-Bitwise Processing In-Memory Through Database Analytics

Authors: Ben Perach, Ronny Ronen, Benny Kimelfeld, Shahar Kvatinsky

Abstract: Bulk-bitwise processing-in-memory (PIM), where large bitwise operations are performed in parallel by the memory array itself, is an emerging form of computation with the potential to mitigate the memory wall problem. This paper examines the capabilities of bulk-bitwise PIM by constructing PIMDB, a fully-digital system based on memristive stateful logic, utilizing and focusing on in-memory bulk-bit… ▽ More Bulk-bitwise processing-in-memory (PIM), where large bitwise operations are performed in parallel by the memory array itself, is an emerging form of computation with the potential to mitigate the memory wall problem. This paper examines the capabilities of bulk-bitwise PIM by constructing PIMDB, a fully-digital system based on memristive stateful logic, utilizing and focusing on in-memory bulk-bitwise operations, designed to accelerate a real-life workload: analytical processing of relational databases. We introduce a host processor programming model to support bulk-bitwise PIM in virtual memory, develop techniques to efficiently perform in-memory filtering and aggregation operations, and adapt the application data set into the memory. To understand bulk-bitwise PIM, we compare it to an equivalent in-memory database on the same host system. We show that bulk-bitwise PIM substantially lowers the number of required memory read operations, thus accelerating TPC-H filter operations by 1.6$\times$--18$\times$ and full queries by 56$\times$--608$\times$, while reducing the energy consumption by 1.7$\times$--18.6$\times$ and 0.81$\times$--12$\times$ for these benchmarks, respectively. Our extensive evaluation uses the gem5 full-system simulation environment. The simulations also evaluate cell endurance, showing that the required endurance is within the range of existing endurance of RRAM devices. △ Less

Submitted 26 September, 2023; v1 submitted 20 March, 2022; originally announced March 2022.

Comments: published in IEEE Transactions on Emerging Topics in Computing

A memristive deep belief neural network based on silicon synapses

Authors: Wei Wang, Loai Danial, Yang Li, Eric Herbelin, Evgeny Pikhay, Yakov Roizin, Barak Hoffer, Zhongrui Wang, Shahar Kvatinsky

Abstract: Memristor-based neuromorphic computing could overcome the limitations of traditional von Neumann computing architectures -- in which data are shuffled between separate memory and processing units -- and improve the performance of deep neural networks. However, this will require accurate synaptic-like device performance, and memristors typically suffer from poor yield and a limited number of reliab… ▽ More Memristor-based neuromorphic computing could overcome the limitations of traditional von Neumann computing architectures -- in which data are shuffled between separate memory and processing units -- and improve the performance of deep neural networks. However, this will require accurate synaptic-like device performance, and memristors typically suffer from poor yield and a limited number of reliable conductance states. Here we report floating gate memristive synaptic devices that are fabricated in a commercial complementary metal-oxide-semiconductor (CMOS) process. These silicon synapses offer analogue tunability, high endurance, long retention times, predictable cycling degradation, moderate device-to-device variations, and high yield. They also provide two orders of magnitude higher energy efficiency for multiply-accumulate operations than graphics processing units. We use two 12-by-8 arrays of the memristive devices for in-situ training of a 19-by-8 memristive restricted Boltzmann machine for pattern recognition via a gradient descent algorithm based on contrastive divergence. We then create a memristive deep belief neural network consisting of three memristive restricted Boltzmann machines. We test this on the modified National Institute of Standards and Technology (MNIST) dataset, demonstrating recognition accuracy up to 97.05%. △ Less

Submitted 20 July, 2023; v1 submitted 16 March, 2022; originally announced March 2022.

Journal ref: Nature Electronics, 5, 870 (2022)

Physical based compact model of Y-Flash memristor for neuromorphic computation

Authors: Wei Wang, Loai Danial, Eric Herbelin, Barak Hoffer, Batel Oved, Tzofnat Greenberg-Toledo, Evgeny Pikhay, Yakov Roizin, Shahar Kvatinsky

Abstract: Y-Flash memristors utilize the mature technology of single polysilicon floating gate non-volatile memories (NVM). It can be operated in a two-terminal configuration similar to the other emerging memristive devices, i.e., resistive random-access memory (RRAM), phase-change memory (PCM), etc. Fabricated in production complementary metal-oxide-semiconductor (CMOS) technology, Y-Flash memristors allow… ▽ More Y-Flash memristors utilize the mature technology of single polysilicon floating gate non-volatile memories (NVM). It can be operated in a two-terminal configuration similar to the other emerging memristive devices, i.e., resistive random-access memory (RRAM), phase-change memory (PCM), etc. Fabricated in production complementary metal-oxide-semiconductor (CMOS) technology, Y-Flash memristors allow excellent repro-ducibility reflected in high neuromorphic products yields. Working in the subthreshold region, the device can be programmed to a large number of fine-tuned intermediate states in an analog fashion and allows low readout currents (1 nA ~ 5 $μ$ A). However, currently, there are no accurate models to describe the dynamic switching in this type of memristive device and account for multiple operational configurations. In this paper, we provide a physical-based compact model that describes Y-Flash memristor performance both in DC and AC regimes, and consistently describes the dynamic program and erase operations. The model is integrated into the commercial circuit design tools and is ready to be used in applications related to neuromorphic computation. △ Less

Submitted 16 February, 2022; originally announced February 2022.

Making Memristive Processing-in-Memory Reliable

Authors: Orian Leitersdorf, Ronny Ronen, Shahar Kvatinsky

Abstract: Processing-in-memory (PIM) solutions vastly accelerate systems by reducing data transfer between computation and memory. Memristors possess a unique property that enables storage and logic within the same device, which is exploited in the memristive Memory Processing Unit (mMPU). The mMPU expands fundamental stateful logic techniques, such as IMPLY, MAGIC and FELIX, to high-throughput parallel log… ▽ More Processing-in-memory (PIM) solutions vastly accelerate systems by reducing data transfer between computation and memory. Memristors possess a unique property that enables storage and logic within the same device, which is exploited in the memristive Memory Processing Unit (mMPU). The mMPU expands fundamental stateful logic techniques, such as IMPLY, MAGIC and FELIX, to high-throughput parallel logic and arithmetic operations within the memory. Unfortunately, memristive processing-in-memory is highly vulnerable to soft errors and this massive parallelism is not compatible with traditional reliability techniques, such as error-correcting-code (ECC). In this paper, we discuss reliability techniques that efficiently support the mMPU by utilizing the same principles as the mMPU computation. We detail ECC techniques that are based on the unique properties of the mMPU to efficiently utilize the massive parallelism. Furthermore, we present novel solutions for efficiently implementing triple modular redundancy (TMR). The short-term and long-term reliability of large-scale applications, such as neural-network acceleration, are evaluated. The analysis clearly demonstrates the importance of high-throughput reliability mechanisms for memristive processing-in-memory. △ Less

Submitted 20 September, 2021; originally announced September 2021.

Comments: Accepted to 28th International Conference on Electronics Circuits and Systems (ICECS) 2021

MultPIM: Fast Stateful Multiplication for Processing-in-Memory

Authors: Orian Leitersdorf, Ronny Ronen, Shahar Kvatinsky

Abstract: Processing-in-memory (PIM) seeks to eliminate computation/memory data transfer using devices that support both storage and logic. Stateful logic techniques such as IMPLY, MAGIC and FELIX can perform logic gates within memristive crossbar arrays with massive parallelism. Multiplication via stateful logic is an active field of research due to the wide implications. Recently, RIME has become the stat… ▽ More Processing-in-memory (PIM) seeks to eliminate computation/memory data transfer using devices that support both storage and logic. Stateful logic techniques such as IMPLY, MAGIC and FELIX can perform logic gates within memristive crossbar arrays with massive parallelism. Multiplication via stateful logic is an active field of research due to the wide implications. Recently, RIME has become the state-of-the-art algorithm for stateful single-row multiplication by using memristive partitions, reducing the latency of the previous state-of-the-art by 5.1x. In this paper, we begin by proposing novel partition-based computation techniques for broadcasting and shifting data. Then, we design an in-memory multiplication algorithm based on the carry-save add-shift (CSAS) technique. Finally, we develop a novel stateful full-adder that significantly improves the state-of-the-art (FELIX) design. These contributions constitute MultPIM, a multiplier that reduces state-of-the-art time complexity from quadratic to linear-log. For 32-bit numbers, MultPIM improves latency by an additional 4.2x over RIME, while even slightly reducing area overhead. Furthermore, we optimize MultPIM for full-precision matrix-vector multiplication and improve latency by 25.5x over FloatPIM matrix-vector multiplication. △ Less

Submitted 20 September, 2021; v1 submitted 30 August, 2021; originally announced August 2021.

Comments: Accepted to IEEE Transactions On Circuits And Systems-II (TCAS-II)

The Bitlet Model: A Parameterized Analytical Model to Compare PIM and CPU Systems

Authors: Ronny Ronen, Adi Eliahu, Orian Leitersdorf, Natan Peled, Kunal Korgaonkar, Anupam Chattopadhyay, Ben Perach, Shahar Kvatinsky

Abstract: Nowadays, data-intensive applications are gaining popularity and, together with this trend, processing-in-memory (PIM)-based systems are being given more attention and have become more relevant. This paper describes an analytical modeling tool called Bitlet that can be used, in a parameterized fashion, to estimate the performance and the power/energy of a PIM-based system and thereby assess the af… ▽ More Nowadays, data-intensive applications are gaining popularity and, together with this trend, processing-in-memory (PIM)-based systems are being given more attention and have become more relevant. This paper describes an analytical modeling tool called Bitlet that can be used, in a parameterized fashion, to estimate the performance and the power/energy of a PIM-based system and thereby assess the affinity of workloads for PIM as opposed to traditional computing. The tool uncovers interesting tradeoffs between, mainly, the PIM computation complexity (cycles required to perform a computation through PIM), the amount of memory used for PIM, the system memory bandwidth, and the data transfer size. Despite its simplicity, the model reveals new insights when applied to real-life examples. The model is demonstrated for several synthetic examples and then applied to explore the influence of different parameters on two systems - IMAGING and FloatPIM. Based on the demonstrations, insights about PIM and its combination with CPU are concluded. △ Less

Submitted 21 July, 2021; originally announced July 2021.

Comments: Accepted to ACM JETC

Efficient Error-Correcting-Code Mechanism for High-Throughput Memristive Processing-in-Memory

Authors: Orian Leitersdorf, Ben Perach, Ronny Ronen, Shahar Kvatinsky

Abstract: Inefficient data transfer between computation and memory inspired emerging processing-in-memory (PIM) technologies. Many PIM solutions enable storage and processing using memristors in a crossbar-array structure, with techniques such as memristor-aided logic (MAGIC) used for computation. This approach provides highly-paralleled logic computation with minimal data movement. However, memristors are… ▽ More Inefficient data transfer between computation and memory inspired emerging processing-in-memory (PIM) technologies. Many PIM solutions enable storage and processing using memristors in a crossbar-array structure, with techniques such as memristor-aided logic (MAGIC) used for computation. This approach provides highly-paralleled logic computation with minimal data movement. However, memristors are vulnerable to soft errors and standard error-correcting-code (ECC) techniques are difficult to implement without moving data outside the memory. We propose a novel technique for efficient ECC implementation along diagonals to support reliable computation inside the memory without explicitly reading the data. Our evaluation demonstrates an improvement of over eight orders of magnitude in reliability (mean time to failure) for an increase of about 26% in computation latency. △ Less

Submitted 10 May, 2021; originally announced May 2021.

Comments: Accepted to 58th Design Automation Conference (DAC) 2021

CONTRA: Area-Constrained Technology Map** Framework For Memristive Memory Processing Unit

Authors: Debjyoti Bhattacharjee, Anupam Chattopadhyay, Srijit Dutta, Ronny Ronen, Shahar Kvatinsky

Abstract: Data-intensive applications are poised to benefit directly from processing-in-memory platforms, such as memristive Memory Processing Units, which allow leveraging data locality and performing stateful logic operations. Develo** design automation flows for such platforms is a challenging and highly relevant research problem. In this work, we investigate the problem of minimizing delay under arbit… ▽ More Data-intensive applications are poised to benefit directly from processing-in-memory platforms, such as memristive Memory Processing Units, which allow leveraging data locality and performing stateful logic operations. Develo** design automation flows for such platforms is a challenging and highly relevant research problem. In this work, we investigate the problem of minimizing delay under arbitrary area constraint for MAGIC-based in-memory computing platforms. We propose an end-to-end area constrained technology map** framework, CONTRA. CONTRA uses Look-Up Table(LUT) based map** of the input function on the crossbar array to maximize parallel operations and uses a novel search technique to move data optimally inside the array. CONTRA supports benchmarks in a variety of formats, along with crossbar dimensions as input to generate MAGIC instructions. CONTRA scales for large benchmarks, as demonstrated by our experiments. CONTRA allows map** benchmarks to smaller crossbar dimensions than achieved by any other technique before, while allowing a wide variety of area-delay trade-offs. CONTRA improves the composite metric of area-delay product by 2.1x to 13.1x compared to seven existing technology map** approaches. △ Less

Submitted 2 September, 2020; originally announced September 2020.

Comments: 9 pages

Training of Quantized Deep Neural Networks using a Magnetic Tunnel Junction-Based Synapse

Authors: Tzofnat Greenberg Toledo, Ben Perach, Itay Hubara, Daniel Soudry, Shahar Kvatinsky

Abstract: Quantized neural networks (QNNs) are being actively researched as a solution for the computational complexity and memory intensity of deep neural networks. This has sparked efforts to develop algorithms that support both inference and training with quantized weight and activation values, without sacrificing accuracy. A recent example is the GXNOR framework for stochastic training of ternary (TNN)… ▽ More Quantized neural networks (QNNs) are being actively researched as a solution for the computational complexity and memory intensity of deep neural networks. This has sparked efforts to develop algorithms that support both inference and training with quantized weight and activation values, without sacrificing accuracy. A recent example is the GXNOR framework for stochastic training of ternary (TNN) and binary (BNN) neural networks. In this paper, we show how magnetic tunnel junction (MTJ) devices can be used to support QNN training. We introduce a novel hardware synapse circuit that uses the MTJ stochastic behavior to support the quantize update. The proposed circuit enables processing near memory (PNM) of QNN training, which subsequently reduces data movement. We simulated MTJ-based stochastic training of a TNN over the MNIST, SVHN, and CIFAR10 datasets and achieved an accuracy of 98.61%, 93.99% and 82.71%, respectively (less than 1% degradation compared to the GXNOR algorithm). We evaluated the synapse array performance potential and showed that the proposed synapse circuit can train ternary networks in situ, with 18.3TOPs/W for feedforward and 3TOPs/W for weight update. △ Less

Submitted 29 May, 2022; v1 submitted 29 December, 2019; originally announced December 2019.

Comments: Published in Semiconductor Science and Technology, Vol 36

Journal ref: Semicond. Sci. Technol. 36 114003 (2021)

The Bitlet Model: Defining a Litmus Test for the Bitwise Processing-in-Memory Paradigm

Authors: Kunal Korgaonkar, Ronny Ronen, Anupam Chattopadhyay, Shahar Kvatinsky

Abstract: This paper describes an analytical modeling tool called Bitlet that can be used, in a parameterized fashion, to understand the affinity of workloads to processing-in-memory (PIM) as opposed to traditional computing. The tool uncovers interesting trade-offs between operation complexity (cycles required to perform an operation through PIM) and other key parameters, such as system memory bandwidth, d… ▽ More This paper describes an analytical modeling tool called Bitlet that can be used, in a parameterized fashion, to understand the affinity of workloads to processing-in-memory (PIM) as opposed to traditional computing. The tool uncovers interesting trade-offs between operation complexity (cycles required to perform an operation through PIM) and other key parameters, such as system memory bandwidth, data transfer size, the extent of data alignment, and effective memory capacity involved in PIM computations. Despite its simplicity, the model has already proven useful. In the future, we intend to extend and refine Bitlet to further increase its utility. △ Less

Submitted 22 October, 2019; originally announced October 2019.

A Systematic Approach to Blocking Convolutional Neural Networks

Authors: Xuan Yang, **g Pu, Blaine Burton Rister, Nikhil Bhagdikar, Stephen Richardson, Shahar Kvatinsky, Jonathan Ragan-Kelley, Ardavan Pedram, Mark Horowitz

Abstract: Convolutional Neural Networks (CNNs) are the state of the art solution for many computer vision problems, and many researchers have explored optimized implementations. Most implementations heuristically block the computation to deal with the large data sizes and high data reuse of CNNs. This paper explores how to block CNN computations for memory locality by creating an analytical model for CNN-li… ▽ More Convolutional Neural Networks (CNNs) are the state of the art solution for many computer vision problems, and many researchers have explored optimized implementations. Most implementations heuristically block the computation to deal with the large data sizes and high data reuse of CNNs. This paper explores how to block CNN computations for memory locality by creating an analytical model for CNN-like loop nests. Using this model we automatically derive optimized blockings for common networks that improve the energy efficiency of custom hardware implementations by up to an order of magnitude. Compared to traditional CNN CPU implementations based on highly-tuned, hand-optimized BLAS libraries,our x86 programs implementing the optimal blocking reduce the number of memory accesses by up to 90%. △ Less

Submitted 14 June, 2016; originally announced June 2016.

Dark Memory and Accelerator-Rich System Optimization in the Dark Silicon Era

Authors: Ardavan Pedram, Stephen Richardson, Sameh Galal, Shahar Kvatinsky, Mark A. Horowitz

Abstract: The key challenge to improving performance in the age of Dark Silicon is how to leverage transistors when they cannot all be used at the same time. In modern SOCs, these transistors are often used to create specialized accelerators which improve energy efficiency for some applications by 10-1000X. While this might seem like the magic bullet we need, for most CPU applications more energy is dissipa… ▽ More The key challenge to improving performance in the age of Dark Silicon is how to leverage transistors when they cannot all be used at the same time. In modern SOCs, these transistors are often used to create specialized accelerators which improve energy efficiency for some applications by 10-1000X. While this might seem like the magic bullet we need, for most CPU applications more energy is dissipated in the memory system than in the processor: these large gains in efficiency are only possible if the DRAM and memory hierarchy are mostly idle. We refer to this desirable state as Dark Memory, and it only occurs for applications with an extreme form of locality. To show our findings, we introduce Pareto curves in the energy/op and mm$^2$/(ops/s) metric space for compute units, accelerators, and on-chip memory/interconnect. These Pareto curves allow us to solve the power, performance, area constrained optimization problem to determine which accelerators should be used, and how to set their design parameters to optimize the system. This analysis shows that memory accesses create a floor to the achievable energy-per-op. Thus high performance requires Dark Memory, which in turn requires co-design of the algorithm for parallelism and locality, with the hardware. △ Less

Submitted 26 April, 2016; v1 submitted 12 February, 2016; originally announced February 2016.

Comments: 8 pages, To appear in IEEE Design and Test Journal

Journal ref: IEEE Design & Test ( Volume: 34, Issue: 2, April 2017 )