Showing 1–10 of 10 results for author: Carpegna, A

Compressed Latent Replays for Lightweight Continual Learning on Spiking Neural Networks

Authors: Alberto Dequino, Alessio Carpegna, Davide Nadalini, Alessandro Savino, Luca Benini, Stefano Di Carlo, Francesco Conti

Abstract: Rehearsal-based Continual Learning (CL) has been intensely investigated in Deep Neural Networks (DNNs). However, its application in Spiking Neural Networks (SNNs) has not been explored in depth. In this paper we introduce the first memory-efficient implementation of Latent Replay (LR)-based CL for SNNs, designed to seamlessly integrate with resource-constrained devices. LRs combine new samples wit… ▽ More Rehearsal-based Continual Learning (CL) has been intensely investigated in Deep Neural Networks (DNNs). However, its application in Spiking Neural Networks (SNNs) has not been explored in depth. In this paper we introduce the first memory-efficient implementation of Latent Replay (LR)-based CL for SNNs, designed to seamlessly integrate with resource-constrained devices. LRs combine new samples with latent representations of previously learned data, to mitigate forgetting. Experiments on the Heidelberg SHD dataset with Sample and Class-Incremental tasks reach a Top-1 accuracy of 92.5% and 92%, respectively, without forgetting the previously learned information. Furthermore, we minimize the LRs' requirements by applying a time-domain compression, reducing by two orders of magnitude their memory requirement, with respect to a naive rehearsal setup, with a maximum accuracy drop of 4%. On a Multi-Class-Incremental task, our SNN learns 10 new classes from an initial set of 10, reaching a Top-1 accuracy of 78.4% on the full test set. △ Less

Submitted 8 May, 2024; originally announced July 2024.

SpikeExplorer: hardware-oriented Design Space Exploration for Spiking Neural Networks on FPGA

Authors: Dario Padovano, Alessio Carpegna, Alessandro Savino, Stefano Di Carlo

Abstract: One of today's main concerns is to bring Artificial Intelligence power to embedded systems for edge applications. The hardware resources and power consumption required by state-of-the-art models are incompatible with the constrained environments observed in edge systems, such as IoT nodes and wearable devices. Spiking Neural Networks (SNNs) can represent a solution in this sense: inspired by neuro… ▽ More One of today's main concerns is to bring Artificial Intelligence power to embedded systems for edge applications. The hardware resources and power consumption required by state-of-the-art models are incompatible with the constrained environments observed in edge systems, such as IoT nodes and wearable devices. Spiking Neural Networks (SNNs) can represent a solution in this sense: inspired by neuroscience, they reach unparalleled power and resource efficiency when run on dedicated hardware accelerators. However, when designing such accelerators, the amount of choices that can be taken is huge. This paper presents SpikExplorer, a modular and flexible Python tool for hardware-oriented Automatic Design Space Exploration to automate the configuration of FPGA accelerators for SNNs. Using Bayesian optimizations, SpikerExplorer enables hardware-centric multi-objective optimization, supporting factors such as accuracy, area, latency, power, and various combinations during the exploration process. The tool searches the optimal network architecture, neuron model, and internal and training parameters, trying to reach the desired constraints imposed by the user. It allows for a straightforward network configuration, providing the full set of explored points for the user to pick the trade-off that best fits the needs. The potential of SpikExplorer is showcased using three benchmark datasets. It reaches 95.8% accuracy on the MNIST dataset, with a power consumption of 180mW/image and a latency of 0.12 ms/image, making it a powerful tool for automatically optimizing SNNs. △ Less

Submitted 4 April, 2024; originally announced April 2024.

SpikingJET: Enhancing Fault Injection for Fully and Convolutional Spiking Neural Networks

Authors: Anil Bayram Gogebakan, Enrico Magliano, Alessio Carpegna, Annachiara Ruospo, Alessandro Savino, Stefano Di Carlo

Abstract: As artificial neural networks become increasingly integrated into safety-critical systems such as autonomous vehicles, devices for medical diagnosis, and industrial automation, ensuring their reliability in the face of random hardware faults becomes paramount. This paper introduces SpikingJET, a novel fault injector designed specifically for fully connected and convolutional Spiking Neural Network… ▽ More As artificial neural networks become increasingly integrated into safety-critical systems such as autonomous vehicles, devices for medical diagnosis, and industrial automation, ensuring their reliability in the face of random hardware faults becomes paramount. This paper introduces SpikingJET, a novel fault injector designed specifically for fully connected and convolutional Spiking Neural Networks (SNNs). Our work underscores the critical need to evaluate the resilience of SNNs to hardware faults, considering their growing prominence in real-world applications. SpikingJET provides a comprehensive platform for assessing the resilience of SNNs by inducing errors and injecting faults into critical components such as synaptic weights, neuron model parameters, internal states, and activation functions. This paper demonstrates the effectiveness of Spiking-JET through extensive software-level experiments on various SNN architectures, revealing insights into their vulnerability and resilience to hardware faults. Moreover, highlighting the importance of fault resilience in SNNs contributes to the ongoing effort to enhance the reliability and safety of Neural Network (NN)-powered systems in diverse domains. △ Less

Submitted 30 March, 2024; originally announced April 2024.

ACM Class: I.2

A Micro Architectural Events Aware Real-Time Embedded System Fault Injector

Authors: Enrico Magliano, Alessio Carpegna, Alessadro Savino, Stefano Di Carlo

Abstract: In contemporary times, the increasing complexity of the system poses significant challenges to the reliability, trustworthiness, and security of the SACRES. Key issues include the susceptibility to phenomena such as instantaneous voltage spikes, electromagnetic interference, neutron strikes, and out-of-range temperatures. These factors can induce switch state changes in transistors, resulting in b… ▽ More In contemporary times, the increasing complexity of the system poses significant challenges to the reliability, trustworthiness, and security of the SACRES. Key issues include the susceptibility to phenomena such as instantaneous voltage spikes, electromagnetic interference, neutron strikes, and out-of-range temperatures. These factors can induce switch state changes in transistors, resulting in bit-flip**, soft errors, and transient corruption of stored data in memory. The occurrence of soft errors, in turn, may lead to system faults that can propel the system into a hazardous state. Particularly in critical sectors like automotive, avionics, or aerospace, such malfunctions can have real-world implications, potentially causing harm to individuals. This paper introduces a novel fault injector designed to facilitate the monitoring, aggregation, and examination of micro-architectural events. This is achieved by harnessing the microprocessor's PMU and the debugging interface, specifically focusing on ensuring the repeatability of fault injections. The fault injection methodology targets bit-flip** within the memory system, affecting CPU registers and RAM. The outcomes of these fault injections enable a thorough analysis of the impact of soft errors and establish a robust correlation between the identified faults and the essential timing predictability demanded by SACRES. △ Less

Submitted 11 June, 2024; v1 submitted 16 January, 2024; originally announced January 2024.

Journal ref: 2024 IEEE 25th Latin American Test Symposium (LATS)

Spiker+: a framework for the generation of efficient Spiking Neural Networks FPGA accelerators for inference at the edge

Authors: Alessio Carpegna, Alessandro Savino, Stefano Di Carlo

Abstract: Including Artificial Neural Networks in embedded systems at the edge allows applications to exploit Artificial Intelligence capabilities directly within devices operating at the network periphery. This paper introduces Spiker+, a comprehensive framework for generating efficient, low-power, and low-area customized Spiking Neural Networks (SNN) accelerators on FPGA for inference at the edge. Spiker+… ▽ More Including Artificial Neural Networks in embedded systems at the edge allows applications to exploit Artificial Intelligence capabilities directly within devices operating at the network periphery. This paper introduces Spiker+, a comprehensive framework for generating efficient, low-power, and low-area customized Spiking Neural Networks (SNN) accelerators on FPGA for inference at the edge. Spiker+ presents a configurable multi-layer hardware SNN, a library of highly efficient neuron architectures, and a design framework, enabling the development of complex neural network accelerators with few lines of Python code. Spiker+ is tested on two benchmark datasets, the MNIST and the Spiking Heidelberg Digits (SHD). On the MNIST, it demonstrates competitive performance compared to state-of-the-art SNN accelerators. It outperforms them in terms of resource allocation, with a requirement of 7,612 logic cells and 18 Block RAMs (BRAMs), which makes it fit in very small FPGA, and power consumption, draining only 180mW for a complete inference on an input image. The latency is comparable to the ones observed in the state-of-the-art, with 780us/img. To the authors' knowledge, Spiker+ is the first SNN accelerator tested on the SHD. In this case, the accelerator requires 18,268 logic cells and 51 BRAM, with an overall power consumption of 430mW and a latency of 54 us for a complete inference on input data. This underscores the significance of Spiker+ in the hardware-accelerated SNN landscape, making it an excellent solution to deploy configurable and tunable SNN architectures in resource and power-constrained edge applications. △ Less

Submitted 2 January, 2024; originally announced January 2024.

NEUROPULS: NEUROmorphic energy-efficient secure accelerators based on Phase change materials aUgmented siLicon photonicS

Authors: Fabio Pavanello, Cedric Marchand, Ian O'Connor, Regis Orobtchouk, Fabien Mandorlo, Xavier Letartre, Sebastien Cueff, Elena Ioana Vatajelu, Giorgio Di Natale, Benoit Cluzel, Aurelien Coillet, Benoit Charbonnier, Pierre Noe, Frantisek Kavan, Martin Zoldak, Michal Szaj, Peter Bienstman, Thomas Van Vaerenbergh, Ulrich Ruhrmair, Paulo Flores, Luis Guerra e Silva, Ricardo Chaves, Luis-Miguel Silveira, Mariano Ceccato, Dimitris Gizopoulos , et al. (12 additional authors not shown)

Abstract: This special session paper introduces the Horizon Europe NEUROPULS project, which targets the development of secure and energy-efficient RISC-V interfaced neuromorphic accelerators using augmented silicon photonics technology. Our approach aims to develop an augmented silicon photonics platform, an FPGA-powered RISC-V-connected computing platform, and a complete simulation platform to demonstrate… ▽ More This special session paper introduces the Horizon Europe NEUROPULS project, which targets the development of secure and energy-efficient RISC-V interfaced neuromorphic accelerators using augmented silicon photonics technology. Our approach aims to develop an augmented silicon photonics platform, an FPGA-powered RISC-V-connected computing platform, and a complete simulation platform to demonstrate the neuromorphic accelerator capabilities. In particular, their main advantages and limitations will be addressed concerning the underpinning technology for each platform. Then, we will discuss three targeted use cases for edge-computing applications: Global National Satellite System (GNSS) anti-jamming, autonomous driving, and anomaly detection in edge devices. Finally, we will address the reliability and security aspects of the stand-alone accelerator implementation and the project use cases. △ Less

Submitted 4 May, 2023; originally announced May 2023.

Comments: 10 pages, 2 figures, conference

Journal ref: IEEE European Test Symposium 2023

Special Session: Neuromorphic hardware design and reliability from traditional CMOS to emerging technologies

Authors: Fabio Pavanello, Elena Ioana Vatajelu, Alberto Bosio, Thomas Van Vaerenbergh, Peter Bienstman, Benoit Charbonnier, Alessio Carpegna, Stefano Di Carlo, Alessandro Savino

Abstract: The field of neuromorphic computing has been rapidly evolving in recent years, with an increasing focus on hardware design and reliability. This special session paper provides an overview of the recent developments in neuromorphic computing, focusing on hardware design and reliability. We first review the traditional CMOS-based approaches to neuromorphic hardware design and identify the challenges… ▽ More The field of neuromorphic computing has been rapidly evolving in recent years, with an increasing focus on hardware design and reliability. This special session paper provides an overview of the recent developments in neuromorphic computing, focusing on hardware design and reliability. We first review the traditional CMOS-based approaches to neuromorphic hardware design and identify the challenges related to scalability, latency, and power consumption. We then investigate alternative approaches based on emerging technologies, specifically integrated photonics approaches within the NEUROPULS project. Finally, we examine the impact of device variability and aging on the reliability of neuromorphic hardware and present techniques for mitigating these effects. This review is intended to serve as a valuable resource for researchers and practitioners in neuromorphic computing. △ Less

Submitted 2 May, 2023; originally announced May 2023.

Comments: 10 pages, 4 figures, 4 tables

Journal ref: 2023 IEEE 41st VLSI Test Symposium (VTS)

Fast Exploration of the Impact of Precision Reduction on Spiking Neural Networks

Authors: Sepide Saeedi, Alessio Carpegna, Alessandro Savino, Stefano Di Carlo

Abstract: Approximate Computing (AxC) techniques trade off the computation accuracy for performance, energy, and area reduction gains. The trade-off is particularly convenient when the applications are intrinsically tolerant to some accuracy loss, as in the Spiking Neural Networks (SNNs) case. SNNs are a practical choice when the target hardware reaches the edge of computing, but this requires some area min… ▽ More Approximate Computing (AxC) techniques trade off the computation accuracy for performance, energy, and area reduction gains. The trade-off is particularly convenient when the applications are intrinsically tolerant to some accuracy loss, as in the Spiking Neural Networks (SNNs) case. SNNs are a practical choice when the target hardware reaches the edge of computing, but this requires some area minimization strategies. In this work, we employ an Interval Arithmetic (IA) model to develop an exploration methodology that takes advantage of the capability of such a model to propagate the approximation error to detect when the approximation exceeds tolerable limits by the application. Experimental results confirm the capability of reducing the exploration time significantly, providing the chance to reduce the network parameters' size further and with more fine-grained results. △ Less

Submitted 22 November, 2022; originally announced December 2022.

Micro-Architectural features as soft-error induced fault executions markers in embedded safety-critical systems: a preliminary study

Authors: Deniz Kasap, Alessio Carpegna, Alessandro Savino, Stefano Di Carlo

Abstract: Radiation-induced soft errors are one of the most challenging issues in Safety Critical Real-Time Embedded System (SACRES) reliability, usually handled using different flavors of Double Modular Redundancy (DMR) techniques. This solution is becoming unaffordable due to the complexity of modern micro-processors in all domains. This paper addresses the promising field of using Artificial Intelligence… ▽ More Radiation-induced soft errors are one of the most challenging issues in Safety Critical Real-Time Embedded System (SACRES) reliability, usually handled using different flavors of Double Modular Redundancy (DMR) techniques. This solution is becoming unaffordable due to the complexity of modern micro-processors in all domains. This paper addresses the promising field of using Artificial Intelligence (AI) based hardware detectors for soft errors. To create such cores and make them general enough to work with different software applications, microarchitectural attributes are a fascinating option as candidate fault detection features. Several processors already track these features through dedicated Performance Monitoring Unit (PMU). However, there is an open question to understand to what extent they are enough to detect faulty executions. Exploiting the capability of gem5 to simulate real computing systems, perform fault injection experiments and profile microarchitectural attributes (i.e., gem5 Stats), this paper presents the results of a comprehensive analysis regarding the potential attributes to detect soft error and the associated models that can be trained with these features. △ Less

Submitted 16 February, 2023; v1 submitted 23 November, 2022; originally announced November 2022.

Journal ref: IEEE European Test Symposium 2023

Spiker: an FPGA-optimized Hardware acceleration for Spiking Neural Networks

Authors: Alessio Carpegna, Alessandro Savino, Stefano Di Carlo

Abstract: Spiking Neural Networks (SNN) are an emerging type of biologically plausible and efficient Artificial Neural Network (ANN). This work presents the development of a hardware accelerator for a SNN for high-performance inference, targeting a Xilinx Artix-7 Field Programmable Gate Array (FPGA). The model used inside the neuron is the Leaky Integrate and Fire (LIF). The execution is clock-driven, meani… ▽ More Spiking Neural Networks (SNN) are an emerging type of biologically plausible and efficient Artificial Neural Network (ANN). This work presents the development of a hardware accelerator for a SNN for high-performance inference, targeting a Xilinx Artix-7 Field Programmable Gate Array (FPGA). The model used inside the neuron is the Leaky Integrate and Fire (LIF). The execution is clock-driven, meaning that the internal state of the neuron is updated at every clock cycle, even in absence of spikes. The inference capabilities of the accelerator are evaluated using the MINST dataset. The training is performed offline on a full precision model. The results show a good improvement in performance if compared with the state-of-the-art accelerators, requiring 215μs per image. The energy consumption is slightly higher than the most optimized design, with an average value of 13mJ per image. The test design consists of a single layer of four-hundred neurons and uses around 40% of the available resources on the FPGA. This makes it suitable for a time-constrained application at the edge, leaving space for other acceleration tasks on the FPGA. △ Less

Submitted 26 May, 2022; v1 submitted 18 January, 2022; originally announced January 2022.

Comments: 6 pages, 3 figures, 4 tables