-
Reliable edge machine learning hardware for scientific applications
Authors:
Tommaso Baldi,
Javier Campos,
Ben Hawks,
Jennifer Ngadiuba,
Nhan Tran,
Daniel Diaz,
Javier Duarte,
Ryan Kastner,
Andres Meza,
Melissa Quinnan,
Olivia Weng,
Caleb Geniesse,
Amir Gholami,
Michael W. Mahoney,
Vladimir Loncar,
Philip Harris,
Joshua Agar,
Shuyu Qin
Abstract:
Extreme data rate scientific experiments create massive amounts of data that require efficient ML edge processing. This leads to unique validation challenges for VLSI implementations of ML algorithms: enabling bit-accurate functional simulations for performance validation in experimental software frameworks, verifying those ML models are robust under extreme quantization and pruning, and enabling…
▽ More
Extreme data rate scientific experiments create massive amounts of data that require efficient ML edge processing. This leads to unique validation challenges for VLSI implementations of ML algorithms: enabling bit-accurate functional simulations for performance validation in experimental software frameworks, verifying those ML models are robust under extreme quantization and pruning, and enabling ultra-fine-grained model inspection for efficient fault tolerance. We discuss approaches to develo** and validating reliable algorithms at the scientific edge under such strict latency, resource, power, and area requirements in extreme experimental environments. We study metrics for develo** robust algorithms, present preliminary results and mitigation strategies, and conclude with an outlook of these and future directions of research towards the longer-term goal of develo** autonomous scientific experimentation methods for accelerated scientific discovery.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
-
Gradient-based Automatic Per-Weight Mixed Precision Quantization for Neural Networks On-Chip
Authors:
Chang Sun,
Thea K. Årrestad,
Vladimir Loncar,
Jennifer Ngadiuba,
Maria Spiropulu
Abstract:
Model size and inference speed at deployment time, are major challenges in many deep learning applications. A promising strategy to overcome these challenges is quantization. However, a straightforward uniform quantization to very low precision can result in significant accuracy loss. Mixed-precision quantization, based on the idea that certain parts of the network can accommodate lower precision…
▽ More
Model size and inference speed at deployment time, are major challenges in many deep learning applications. A promising strategy to overcome these challenges is quantization. However, a straightforward uniform quantization to very low precision can result in significant accuracy loss. Mixed-precision quantization, based on the idea that certain parts of the network can accommodate lower precision without compromising performance compared to other parts, offers a potential solution. In this work, we present High Granularity Quantization (HGQ), an innovative quantization-aware training method designed to fine-tune the per-weight and per-activation precision in an automatic way for ultra-low latency and low power neural networks which are to be deployed on FPGAs. We demonstrate that HGQ can outperform existing methods by a substantial margin, achieving resource reduction by up to a factor of 20 and latency improvement by a factor of 5 while preserving accuracy.
△ Less
Submitted 1 May, 2024;
originally announced May 2024.
-
Sets are all you need: Ultrafast jet classification on FPGAs for HL-LHC
Authors:
Patrick Odagiu,
Zhiqiang Que,
Javier Duarte,
Johannes Haller,
Gregor Kasieczka,
Artur Lobanov,
Vladimir Loncar,
Wayne Luk,
Jennifer Ngadiuba,
Maurizio Pierini,
Philipp Rincke,
Arpita Seksaria,
Sioni Summers,
Andre Sznajder,
Alexander Tapper,
Thea K. Aarrestad
Abstract:
We study various machine learning based algorithms for performing accurate jet flavor classification on field-programmable gate arrays and demonstrate how latency and resource consumption scale with the input size and choice of algorithm. These architectures provide an initial design for models that could be used for tagging at the CERN LHC during its high-luminosity phase. The high-luminosity upg…
▽ More
We study various machine learning based algorithms for performing accurate jet flavor classification on field-programmable gate arrays and demonstrate how latency and resource consumption scale with the input size and choice of algorithm. These architectures provide an initial design for models that could be used for tagging at the CERN LHC during its high-luminosity phase. The high-luminosity upgrade will lead to a five-fold increase in its instantaneous luminosity for proton-proton collisions and, in turn, higher data volume and complexity, such as the availability of jet constituents. Through quantization-aware training and efficient hardware implementations, we show that O(100) ns inference of complex architectures such as deep sets and interaction networks is feasible at a low computational resource cost.
△ Less
Submitted 2 February, 2024;
originally announced February 2024.
-
Ultra Fast Transformers on FPGAs for Particle Physics Experiments
Authors:
Zhixing Jiang,
Dennis Yin,
Elham E Khoda,
Vladimir Loncar,
Ekaterina Govorkova,
Eric Moreno,
Philip Harris,
Scott Hauck,
Shih-Chieh Hsu
Abstract:
This work introduces a highly efficient implementation of the transformer architecture on a Field-Programmable Gate Array (FPGA) by using the \texttt{hls4ml} tool. Given the demonstrated effectiveness of transformer models in addressing a wide range of problems, their application in experimental triggers within particle physics becomes a subject of significant interest. In this work, we have imple…
▽ More
This work introduces a highly efficient implementation of the transformer architecture on a Field-Programmable Gate Array (FPGA) by using the \texttt{hls4ml} tool. Given the demonstrated effectiveness of transformer models in addressing a wide range of problems, their application in experimental triggers within particle physics becomes a subject of significant interest. In this work, we have implemented critical components of a transformer model, such as multi-head attention and softmax layers. To evaluate the effectiveness of our implementation, we have focused on a particle physics jet flavor tagging problem, employing a public dataset. We recorded latency under 2 $μ$s on the Xilinx UltraScale+ FPGA, which is compatible with hardware trigger requirements at the CERN Large Hadron Collider experiments.
△ Less
Submitted 1 February, 2024;
originally announced February 2024.
-
SymbolNet: Neural Symbolic Regression with Adaptive Dynamic Pruning
Authors:
Ho Fung Tsoi,
Vladimir Loncar,
Sridhara Dasu,
Philip Harris
Abstract:
Contrary to the use of genetic programming, the neural network approach to symbolic regression can scale well with high input dimension and leverage gradient methods for faster equation searching. Common ways of constraining expression complexity have relied on multistage pruning methods with fine-tuning, but these often lead to significant performance loss. In this work, we propose SymbolNet, a n…
▽ More
Contrary to the use of genetic programming, the neural network approach to symbolic regression can scale well with high input dimension and leverage gradient methods for faster equation searching. Common ways of constraining expression complexity have relied on multistage pruning methods with fine-tuning, but these often lead to significant performance loss. In this work, we propose SymbolNet, a neural network approach to symbolic regression in a novel framework that enables dynamic pruning of model weights, input features, and mathematical operators in a single training, where both training loss and expression complexity are optimized simultaneously. We introduce a sparsity regularization term per pruning type, which can adaptively adjust its own strength and lead to convergence to a target sparsity level. In contrast to most existing symbolic regression methods that cannot efficiently handle datasets with more than $O$(10) inputs, we demonstrate the effectiveness of our model on the LHC jet tagging task (16 inputs), MNIST (784 inputs), and SVHN (3072 inputs).
△ Less
Submitted 18 January, 2024;
originally announced January 2024.
-
FPGA Resource-aware Structured Pruning for Real-Time Neural Networks
Authors:
Benjamin Ramhorst,
Vladimir Loncar,
George A. Constantinides
Abstract:
Neural networks achieve state-of-the-art performance in image classification, speech recognition, scientific analysis and many more application areas. Due to the high computational complexity and memory footprint of neural networks, various compression techniques, such as pruning and quantization, have been proposed in literature. Pruning sparsifies a neural network, reducing the number of multipl…
▽ More
Neural networks achieve state-of-the-art performance in image classification, speech recognition, scientific analysis and many more application areas. Due to the high computational complexity and memory footprint of neural networks, various compression techniques, such as pruning and quantization, have been proposed in literature. Pruning sparsifies a neural network, reducing the number of multiplications and memory. However, pruning often fails to capture properties of the underlying hardware, causing unstructured sparsity and load-balance inefficiency, thus bottlenecking resource improvements. We propose a hardware-centric formulation of pruning, by formulating it as a knapsack problem with resource-aware tensor structures. Evaluated on a range of tasks, including sub-microsecond particle classification at CERN's Large Hadron Collider and fast image classification, the proposed method achieves reductions ranging between 55% and 92% in the DSP utilization and up to 81% in BRAM utilization.
△ Less
Submitted 12 December, 2023; v1 submitted 9 August, 2023;
originally announced August 2023.
-
Symbolic Regression on FPGAs for Fast Machine Learning Inference
Authors:
Ho Fung Tsoi,
Adrian Alan Pol,
Vladimir Loncar,
Ekaterina Govorkova,
Miles Cranmer,
Sridhara Dasu,
Peter Elmer,
Philip Harris,
Isobel Ojalvo,
Maurizio Pierini
Abstract:
The high-energy physics community is investigating the potential of deploying machine-learning-based solutions on Field-Programmable Gate Arrays (FPGAs) to enhance physics sensitivity while still meeting data processing time constraints. In this contribution, we introduce a novel end-to-end procedure that utilizes a machine learning technique called symbolic regression (SR). It searches the equati…
▽ More
The high-energy physics community is investigating the potential of deploying machine-learning-based solutions on Field-Programmable Gate Arrays (FPGAs) to enhance physics sensitivity while still meeting data processing time constraints. In this contribution, we introduce a novel end-to-end procedure that utilizes a machine learning technique called symbolic regression (SR). It searches the equation space to discover algebraic relations approximating a dataset. We use PySR (a software to uncover these expressions based on an evolutionary algorithm) and extend the functionality of hls4ml (a package for machine learning inference in FPGAs) to support PySR-generated expressions for resource-constrained production environments. Deep learning models often optimize the top metric by pinning the network size because the vast hyperparameter space prevents an extensive search for neural architecture. Conversely, SR selects a set of models on the Pareto front, which allows for optimizing the performance-resource trade-off directly. By embedding symbolic forms, our implementation can dramatically reduce the computational resources needed to perform critical tasks. We validate our method on a physics benchmark: the multiclass classification of jets produced in simulated proton-proton collisions at the CERN Large Hadron Collider. We show that our approach can approximate a 3-layer neural network using an inference model that achieves up to a 13-fold decrease in execution time, down to 5 ns, while still preserving more than 90% approximation accuracy.
△ Less
Submitted 17 January, 2024; v1 submitted 6 May, 2023;
originally announced May 2023.
-
Tailor: Altering Skip Connections for Resource-Efficient Inference
Authors:
Olivia Weng,
Gabriel Marcano,
Vladimir Loncar,
Alireza Khodamoradi,
Nojan Sheybani,
Andres Meza,
Farinaz Koushanfar,
Kristof Denolf,
Javier Mauricio Duarte,
Ryan Kastner
Abstract:
Deep neural networks use skip connections to improve training convergence. However, these skip connections are costly in hardware, requiring extra buffers and increasing on- and off-chip memory utilization and bandwidth requirements. In this paper, we show that skip connections can be optimized for hardware when tackled with a hardware-software codesign approach. We argue that while a network's sk…
▽ More
Deep neural networks use skip connections to improve training convergence. However, these skip connections are costly in hardware, requiring extra buffers and increasing on- and off-chip memory utilization and bandwidth requirements. In this paper, we show that skip connections can be optimized for hardware when tackled with a hardware-software codesign approach. We argue that while a network's skip connections are needed for the network to learn, they can later be removed or shortened to provide a more hardware efficient implementation with minimal to no accuracy loss. We introduce Tailor, a codesign tool whose hardware-aware training algorithm gradually removes or shortens a fully trained network's skip connections to lower their hardware cost. Tailor improves resource utilization by up to 34% for BRAMs, 13% for FFs, and 16% for LUTs for on-chip, dataflow-style architectures. Tailor increases performance by 30% and reduces memory bandwidth by 45% for a 2D processing element array architecture.
△ Less
Submitted 15 September, 2023; v1 submitted 17 January, 2023;
originally announced January 2023.
-
Ultra-low latency recurrent neural network inference on FPGAs for physics applications with hls4ml
Authors:
Elham E Khoda,
Dylan Rankin,
Rafael Teixeira de Lima,
Philip Harris,
Scott Hauck,
Shih-Chieh Hsu,
Michael Kagan,
Vladimir Loncar,
Chaitanya Paikara,
Richa Rao,
Sioni Summers,
Caterina Vernieri,
Aaron Wang
Abstract:
Recurrent neural networks have been shown to be effective architectures for many tasks in high energy physics, and thus have been widely adopted. Their use in low-latency environments has, however, been limited as a result of the difficulties of implementing recurrent architectures on field-programmable gate arrays (FPGAs). In this paper we present an implementation of two types of recurrent neura…
▽ More
Recurrent neural networks have been shown to be effective architectures for many tasks in high energy physics, and thus have been widely adopted. Their use in low-latency environments has, however, been limited as a result of the difficulties of implementing recurrent architectures on field-programmable gate arrays (FPGAs). In this paper we present an implementation of two types of recurrent neural network layers -- long short-term memory and gated recurrent unit -- within the hls4ml framework. We demonstrate that our implementation is capable of producing effective designs for both small and large models, and can be customized to meet specific design requirements for inference latencies and FPGA resources. We show the performance and synthesized designs for multiple neural networks, many of which are trained specifically for jet identification tasks at the CERN Large Hadron Collider.
△ Less
Submitted 1 July, 2022;
originally announced July 2022.
-
QONNX: Representing Arbitrary-Precision Quantized Neural Networks
Authors:
Alessandro Pappalardo,
Yaman Umuroglu,
Michaela Blott,
Jovan Mitrevski,
Ben Hawks,
Nhan Tran,
Vladimir Loncar,
Sioni Summers,
Hendrik Borras,
Jules Muhizi,
Matthew Trahms,
Shih-Chieh Hsu,
Scott Hauck,
Javier Duarte
Abstract:
We present extensions to the Open Neural Network Exchange (ONNX) intermediate representation format to represent arbitrary-precision quantized neural networks. We first introduce support for low precision quantization in existing ONNX-based quantization formats by leveraging integer clip**, resulting in two new backward-compatible variants: the quantized operator format with clip** and quantiz…
▽ More
We present extensions to the Open Neural Network Exchange (ONNX) intermediate representation format to represent arbitrary-precision quantized neural networks. We first introduce support for low precision quantization in existing ONNX-based quantization formats by leveraging integer clip**, resulting in two new backward-compatible variants: the quantized operator format with clip** and quantize-clip-dequantize (QCDQ) format. We then introduce a novel higher-level ONNX format called quantized ONNX (QONNX) that introduces three new operators -- Quant, BipolarQuant, and Trunc -- in order to represent uniform quantization. By kee** the QONNX IR high-level and flexible, we enable targeting a wider variety of platforms. We also present utilities for working with QONNX, as well as examples of its usage in the FINN and hls4ml toolchains. Finally, we introduce the QONNX model zoo to share low-precision quantized neural networks.
△ Less
Submitted 24 June, 2022; v1 submitted 15 June, 2022;
originally announced June 2022.
-
Real-time semantic segmentation on FPGAs for autonomous vehicles with hls4ml
Authors:
Nicolò Ghielmetti,
Vladimir Loncar,
Maurizio Pierini,
Marcel Roed,
Sioni Summers,
Thea Aarrestad,
Christoffer Petersson,
Hampus Linander,
Jennifer Ngadiuba,
Kelvin Lin,
Philip Harris
Abstract:
In this paper, we investigate how field programmable gate arrays can serve as hardware accelerators for real-time semantic segmentation tasks relevant for autonomous driving. Considering compressed versions of the ENet convolutional neural network architecture, we demonstrate a fully-on-chip deployment with a latency of 4.9 ms per image, using less than 30% of the available resources on a Xilinx Z…
▽ More
In this paper, we investigate how field programmable gate arrays can serve as hardware accelerators for real-time semantic segmentation tasks relevant for autonomous driving. Considering compressed versions of the ENet convolutional neural network architecture, we demonstrate a fully-on-chip deployment with a latency of 4.9 ms per image, using less than 30% of the available resources on a Xilinx ZCU102 evaluation board. The latency is reduced to 3 ms per image when increasing the batch size to ten, corresponding to the use case where the autonomous vehicle receives inputs from multiple cameras simultaneously. We show, through aggressive filter reduction and heterogeneous quantization-aware training, and an optimized implementation of convolutional layers, that the power consumption and resource utilization can be significantly reduced while maintaining accuracy on the Cityscapes dataset.
△ Less
Submitted 16 May, 2022;
originally announced May 2022.
-
Lightweight Jet Reconstruction and Identification as an Object Detection Task
Authors:
Adrian Alan Pol,
Thea Aarrestad,
Ekaterina Govorkova,
Roi Halily,
Anat Klempner,
Tal Kopetz,
Vladimir Loncar,
Jennifer Ngadiuba,
Maurizio Pierini,
Olya Sirkin,
Sioni Summers
Abstract:
We apply object detection techniques based on deep convolutional blocks to end-to-end jet identification and reconstruction tasks encountered at the CERN Large Hadron Collider (LHC). Collision events produced at the LHC and represented as an image composed of calorimeter and tracker cells are given as an input to a Single Shot Detection network. The algorithm, named PFJet-SSD performs simultaneous…
▽ More
We apply object detection techniques based on deep convolutional blocks to end-to-end jet identification and reconstruction tasks encountered at the CERN Large Hadron Collider (LHC). Collision events produced at the LHC and represented as an image composed of calorimeter and tracker cells are given as an input to a Single Shot Detection network. The algorithm, named PFJet-SSD performs simultaneous localization, classification and regression tasks to cluster jets and reconstruct their features. This all-in-one single feed-forward pass gives advantages in terms of execution time and an improved accuracy w.r.t. traditional rule-based methods. A further gain is obtained from network slimming, homogeneous quantization, and optimized runtime for meeting memory and latency constraints of a typical real-time processing environment. We experiment with 8-bit and ternary quantization, benchmarking their accuracy and inference latency against a single-precision floating-point. We show that the ternary network closely matches the performance of its full-precision equivalent and outperforms the state-of-the-art rule-based algorithm. Finally, we report the inference latency on different hardware platforms and discuss future applications.
△ Less
Submitted 9 February, 2022;
originally announced February 2022.
-
Accelerating Recurrent Neural Networks for Gravitational Wave Experiments
Authors:
Zhiqiang Que,
Erwei Wang,
Umar Marikar,
Eric Moreno,
Jennifer Ngadiuba,
Hamza Javed,
Bartłomiej Borzyszkowski,
Thea Aarrestad,
Vladimir Loncar,
Sioni Summers,
Maurizio Pierini,
Peter Y Cheung,
Wayne Luk
Abstract:
This paper presents novel reconfigurable architectures for reducing the latency of recurrent neural networks (RNNs) that are used for detecting gravitational waves. Gravitational interferometers such as the LIGO detectors capture cosmic events such as black hole mergers which happen at unknown times and of varying durations, producing time-series data. We have developed a new architecture capable…
▽ More
This paper presents novel reconfigurable architectures for reducing the latency of recurrent neural networks (RNNs) that are used for detecting gravitational waves. Gravitational interferometers such as the LIGO detectors capture cosmic events such as black hole mergers which happen at unknown times and of varying durations, producing time-series data. We have developed a new architecture capable of accelerating RNN inference for analyzing time-series data from LIGO detectors. This architecture is based on optimizing the initiation intervals (II) in a multi-layer LSTM (Long Short-Term Memory) network, by identifying appropriate reuse factors for each layer. A customizable template for this architecture has been designed, which enables the generation of low-latency FPGA designs with efficient resource utilization using high-level synthesis tools. The proposed approach has been evaluated based on two LSTM models, targeting a ZYNQ 7045 FPGA and a U250 FPGA. Experimental results show that with balanced II, the number of DSPs can be reduced up to 42% while achieving the same IIs. When compared to other FPGA-based LSTM designs, our design can achieve about 4.92 to 12.4 times lower latency.
△ Less
Submitted 26 June, 2021;
originally announced June 2021.
-
A reconfigurable neural network ASIC for detector front-end data compression at the HL-LHC
Authors:
Giuseppe Di Guglielmo,
Farah Fahim,
Christian Herwig,
Manuel Blanco Valentin,
Javier Duarte,
Cristian Gingu,
Philip Harris,
James Hirschauer,
Martin Kwok,
Vladimir Loncar,
Yingyi Luo,
Llovizna Miranda,
Jennifer Ngadiuba,
Daniel Noonan,
Seda Ogrenci-Memik,
Maurizio Pierini,
Sioni Summers,
Nhan Tran
Abstract:
Despite advances in the programmable logic capabilities of modern trigger systems, a significant bottleneck remains in the amount of data to be transported from the detector to off-detector logic where trigger decisions are made. We demonstrate that a neural network autoencoder model can be implemented in a radiation tolerant ASIC to perform lossy data compression alleviating the data transmission…
▽ More
Despite advances in the programmable logic capabilities of modern trigger systems, a significant bottleneck remains in the amount of data to be transported from the detector to off-detector logic where trigger decisions are made. We demonstrate that a neural network autoencoder model can be implemented in a radiation tolerant ASIC to perform lossy data compression alleviating the data transmission problem while preserving critical information of the detector energy profile. For our application, we consider the high-granularity calorimeter from the CMS experiment at the CERN Large Hadron Collider. The advantage of the machine learning approach is in the flexibility and configurability of the algorithm. By changing the neural network weights, a unique data compression algorithm can be deployed for each sensor in different detector regions, and changing detector or collider conditions. To meet area, performance, and power constraints, we perform a quantization-aware training to create an optimized neural network hardware implementation. The design is achieved through the use of high-level synthesis tools and the hls4ml framework, and was processed through synthesis and physical layout flows based on a LP CMOS 65 nm technology node. The flow anticipates 200 Mrad of ionizing radiation to select gates, and reports a total area of 3.6 mm^2 and consumes 95 mW of power. The simulated energy consumption per inference is 2.4 nJ. This is the first radiation tolerant on-detector ASIC implementation of a neural network that has been designed for particle physics applications.
△ Less
Submitted 4 May, 2021;
originally announced May 2021.
-
hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devices
Authors:
Farah Fahim,
Benjamin Hawks,
Christian Herwig,
James Hirschauer,
Sergo **dariani,
Nhan Tran,
Luca P. Carloni,
Giuseppe Di Guglielmo,
Philip Harris,
Jeffrey Krupa,
Dylan Rankin,
Manuel Blanco Valentin,
Josiah Hester,
Yingyi Luo,
John Mamish,
Seda Orgrenci-Memik,
Thea Aarrestad,
Hamza Javed,
Vladimir Loncar,
Maurizio Pierini,
Adrian Alan Pol,
Sioni Summers,
Javier Duarte,
Scott Hauck,
Shih-Chieh Hsu
, et al. (5 additional authors not shown)
Abstract:
Accessible machine learning algorithms, software, and diagnostic tools for energy-efficient devices and systems are extremely valuable across a broad range of application domains. In scientific domains, real-time near-sensor processing can drastically improve experimental design and accelerate scientific discoveries. To support domain scientists, we have developed hls4ml, an open-source software-h…
▽ More
Accessible machine learning algorithms, software, and diagnostic tools for energy-efficient devices and systems are extremely valuable across a broad range of application domains. In scientific domains, real-time near-sensor processing can drastically improve experimental design and accelerate scientific discoveries. To support domain scientists, we have developed hls4ml, an open-source software-hardware codesign workflow to interpret and translate machine learning algorithms for implementation with both FPGA and ASIC technologies. We expand on previous hls4ml work by extending capabilities and techniques towards low-power implementations and increased usability: new Python APIs, quantization-aware pruning, end-to-end FPGA workflows, long pipeline kernels for low power, and new device backends include an ASIC workflow. Taken together, these and continued efforts in hls4ml will arm a new generation of domain scientists with accessible, efficient, and powerful tools for machine-learning-accelerated discovery.
△ Less
Submitted 23 March, 2021; v1 submitted 9 March, 2021;
originally announced March 2021.
-
Fast convolutional neural networks on FPGAs with hls4ml
Authors:
Thea Aarrestad,
Vladimir Loncar,
Nicolò Ghielmetti,
Maurizio Pierini,
Sioni Summers,
Jennifer Ngadiuba,
Christoffer Petersson,
Hampus Linander,
Yutaro Iiyama,
Giuseppe Di Guglielmo,
Javier Duarte,
Philip Harris,
Dylan Rankin,
Sergo **dariani,
Kevin Pedro,
Nhan Tran,
Mia Liu,
Edward Kreinar,
Zhenbin Wu,
Duc Hoang
Abstract:
We introduce an automated tool for deploying ultra low-latency, low-power deep neural networks with convolutional layers on FPGAs. By extending the hls4ml library, we demonstrate an inference latency of $5\,μ$s using convolutional architectures, targeting microsecond latency applications like those at the CERN Large Hadron Collider. Considering benchmark models trained on the Street View House Num…
▽ More
We introduce an automated tool for deploying ultra low-latency, low-power deep neural networks with convolutional layers on FPGAs. By extending the hls4ml library, we demonstrate an inference latency of $5\,μ$s using convolutional architectures, targeting microsecond latency applications like those at the CERN Large Hadron Collider. Considering benchmark models trained on the Street View House Numbers Dataset, we demonstrate various methods for model compression in order to fit the computational constraints of a typical FPGA device used in trigger and data acquisition systems of particle detectors. In particular, we discuss pruning and quantization-aware training, and demonstrate how resource utilization can be significantly reduced with little to no loss in model accuracy. We show that the FPGA critical resource consumption can be reduced by 97% with zero loss in model accuracy, and by 99% when tolerating a 6% accuracy degradation.
△ Less
Submitted 29 April, 2021; v1 submitted 13 January, 2021;
originally announced January 2021.
-
Accelerated Charged Particle Tracking with Graph Neural Networks on FPGAs
Authors:
Aneesh Heintz,
Vesal Razavimaleki,
Javier Duarte,
Gage DeZoort,
Isobel Ojalvo,
Savannah Thais,
Markus Atkinson,
Mark Neubauer,
Lindsey Gray,
Sergo **dariani,
Nhan Tran,
Philip Harris,
Dylan Rankin,
Thea Aarrestad,
Vladimir Loncar,
Maurizio Pierini,
Sioni Summers,
Jennifer Ngadiuba,
Mia Liu,
Edward Kreinar,
Zhenbin Wu
Abstract:
We develop and study FPGA implementations of algorithms for charged particle tracking based on graph neural networks. The two complementary FPGA designs are based on OpenCL, a framework for writing programs that execute across heterogeneous platforms, and hls4ml, a high-level-synthesis-based compiler for neural network to firmware conversion. We evaluate and compare the resource usage, latency, an…
▽ More
We develop and study FPGA implementations of algorithms for charged particle tracking based on graph neural networks. The two complementary FPGA designs are based on OpenCL, a framework for writing programs that execute across heterogeneous platforms, and hls4ml, a high-level-synthesis-based compiler for neural network to firmware conversion. We evaluate and compare the resource usage, latency, and tracking performance of our implementations based on a benchmark dataset. We find a considerable speedup over CPU-based execution is possible, potentially enabling such algorithms to be used effectively in future computing workflows and the FPGA-based Level-1 trigger at the CERN Large Hadron Collider.
△ Less
Submitted 30 November, 2020;
originally announced December 2020.
-
Distance-Weighted Graph Neural Networks on FPGAs for Real-Time Particle Reconstruction in High Energy Physics
Authors:
Yutaro Iiyama,
Gianluca Cerminara,
Abhijay Gupta,
Jan Kieseler,
Vladimir Loncar,
Maurizio Pierini,
Shah Rukh Qasim,
Marcel Rieger,
Sioni Summers,
Gerrit Van Onsem,
Kinga Wozniak,
Jennifer Ngadiuba,
Giuseppe Di Guglielmo,
Javier Duarte,
Philip Harris,
Dylan Rankin,
Sergo **dariani,
Mia Liu,
Kevin Pedro,
Nhan Tran,
Edward Kreinar,
Zhenbin Wu
Abstract:
Graph neural networks have been shown to achieve excellent performance for several crucial tasks in particle physics, such as charged particle tracking, jet tagging, and clustering. An important domain for the application of these networks is the FGPA-based first layer of real-time data filtering at the CERN Large Hadron Collider, which has strict latency and resource constraints. We discuss how t…
▽ More
Graph neural networks have been shown to achieve excellent performance for several crucial tasks in particle physics, such as charged particle tracking, jet tagging, and clustering. An important domain for the application of these networks is the FGPA-based first layer of real-time data filtering at the CERN Large Hadron Collider, which has strict latency and resource constraints. We discuss how to design distance-weighted graph networks that can be executed with a latency of less than 1$μ\mathrm{s}$ on an FPGA. To do so, we consider a representative task associated to particle reconstruction and identification in a next-generation calorimeter operating at a particle collider. We use a graph network architecture developed for such purposes, and apply additional simplifications to match the computing constraints of Level-1 trigger systems, including weight quantization. Using the $\mathtt{hls4ml}$ library, we convert the compressed models into firmware to be implemented on an FPGA. Performance of the synthesized models is presented both in terms of inference accuracy and resource usage.
△ Less
Submitted 3 February, 2021; v1 submitted 8 August, 2020;
originally announced August 2020.
-
Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors
Authors:
Claudionor N. Coelho Jr.,
Aki Kuusela,
Shan Li,
Hao Zhuang,
Thea Aarrestad,
Vladimir Loncar,
Jennifer Ngadiuba,
Maurizio Pierini,
Adrian Alan Pol,
Sioni Summers
Abstract:
Although the quest for more accurate solutions is pushing deep learning research towards larger and more complex algorithms, edge devices demand efficient inference and therefore reduction in model size, latency and energy consumption. One technique to limit model size is quantization, which implies using fewer bits to represent weights and biases. Such an approach usually results in a decline in…
▽ More
Although the quest for more accurate solutions is pushing deep learning research towards larger and more complex algorithms, edge devices demand efficient inference and therefore reduction in model size, latency and energy consumption. One technique to limit model size is quantization, which implies using fewer bits to represent weights and biases. Such an approach usually results in a decline in performance. Here, we introduce a method for designing optimally heterogeneously quantized versions of deep neural network models for minimum-energy, high-accuracy, nanosecond inference and fully automated deployment on chip. With a per-layer, per-parameter type automatic quantization procedure, sampling from a wide range of quantizers, model energy consumption and size are minimized while high accuracy is maintained. This is crucial for the event selection procedure in proton-proton collisions at the CERN Large Hadron Collider, where resources are strictly limited and a latency of ${\mathcal O}(1)~μ$s is required. Nanosecond inference and a resource consumption reduced by a factor of 50 when implemented on field-programmable gate array hardware are achieved.
△ Less
Submitted 21 June, 2021; v1 submitted 15 June, 2020;
originally announced June 2020.
-
Compressing deep neural networks on FPGAs to binary and ternary precision with HLS4ML
Authors:
Giuseppe Di Guglielmo,
Javier Duarte,
Philip Harris,
Duc Hoang,
Sergo **dariani,
Edward Kreinar,
Mia Liu,
Vladimir Loncar,
Jennifer Ngadiuba,
Kevin Pedro,
Maurizio Pierini,
Dylan Rankin,
Sheila Sagear,
Sioni Summers,
Nhan Tran,
Zhenbin Wu
Abstract:
We present the implementation of binary and ternary neural networks in the hls4ml library, designed to automatically convert deep neural network models to digital circuits with FPGA firmware. Starting from benchmark models trained with floating point precision, we investigate different strategies to reduce the network's resource consumption by reducing the numerical precision of the network parame…
▽ More
We present the implementation of binary and ternary neural networks in the hls4ml library, designed to automatically convert deep neural network models to digital circuits with FPGA firmware. Starting from benchmark models trained with floating point precision, we investigate different strategies to reduce the network's resource consumption by reducing the numerical precision of the network parameters to binary or ternary. We discuss the trade-off between model accuracy and resource consumption. In addition, we show how to balance between latency and accuracy by retaining full precision on a selected subset of network components. As an example, we consider two multiclass classification tasks: handwritten digit recognition with the MNIST data set and jet identification with simulated proton-proton collisions at the CERN Large Hadron Collider. The binary and ternary implementation has similar performance to the higher precision implementation while using drastically fewer FPGA resources.
△ Less
Submitted 29 June, 2020; v1 submitted 11 March, 2020;
originally announced March 2020.
-
Fast inference of Boosted Decision Trees in FPGAs for particle physics
Authors:
Sioni Summers,
Giuseppe Di Guglielmo,
Javier Duarte,
Philip Harris,
Duc Hoang,
Sergo **dariani,
Edward Kreinar,
Vladimir Loncar,
Jennifer Ngadiuba,
Maurizio Pierini,
Dylan Rankin,
Nhan Tran,
Zhenbin Wu
Abstract:
We describe the implementation of Boosted Decision Trees in the hls4ml library, which allows the translation of a trained model into FPGA firmware through an automated conversion process. Thanks to its fully on-chip implementation, hls4ml performs inference of Boosted Decision Tree models with extremely low latency. With a typical latency less than 100 ns, this solution is suitable for FPGA-based…
▽ More
We describe the implementation of Boosted Decision Trees in the hls4ml library, which allows the translation of a trained model into FPGA firmware through an automated conversion process. Thanks to its fully on-chip implementation, hls4ml performs inference of Boosted Decision Tree models with extremely low latency. With a typical latency less than 100 ns, this solution is suitable for FPGA-based real-time processing, such as in the Level-1 Trigger system of a collider experiment. These developments open up prospects for physicists to deploy BDTs in FPGAs for identifying the origin of jets, better reconstructing the energies of muons, and enabling better selection of rare signal processes.
△ Less
Submitted 19 February, 2020; v1 submitted 5 February, 2020;
originally announced February 2020.
-
OpenMP GNU and Intel Fortran programs for solving the time-dependent Gross-Pitaevskii equation
Authors:
Luis E. Young-S.,
Paulsamy Muruganandam,
Sadhan K. Adhikari,
Vladimir Loncar,
Dusan Vudragovic,
Antun Balaz
Abstract:
We present Open Multi-Processing (OpenMP) version of Fortran 90 programs for solving the Gross-Pitaevskii (GP) equation for a Bose-Einstein condensate in one, two, and three spatial dimensions, optimized for use with GNU and Intel compilers. We use the split-step Crank-Nicolson algorithm for imaginary- and real-time propagation, which enables efficient calculation of stationary and non-stationary…
▽ More
We present Open Multi-Processing (OpenMP) version of Fortran 90 programs for solving the Gross-Pitaevskii (GP) equation for a Bose-Einstein condensate in one, two, and three spatial dimensions, optimized for use with GNU and Intel compilers. We use the split-step Crank-Nicolson algorithm for imaginary- and real-time propagation, which enables efficient calculation of stationary and non-stationary solutions, respectively. The present OpenMP programs are designed for computers with multi-core processors and optimized for compiling with both commercially-licensed Intel Fortran and popular free open-source GNU Fortran compiler. The programs are easy to use and are elaborated with helpful comments for the users. All input parameters are listed at the beginning of each program. Different output files provide physical quantities such as energy, chemical potential, root-mean-square sizes, densities, etc. We also present speedup test results for new versions of the programs.
△ Less
Submitted 13 September, 2017;
originally announced September 2017.
-
OpenMP, OpenMP/MPI, and CUDA/MPI C programs for solving the time-dependent dipolar Gross-Pitaevskii equation
Authors:
Vladimir Loncar,
Luis E. Young-S.,
Srdjan Skrbic,
Paulsamy Muruganandam,
Sadhan K. Adhikari,
Antun Balaz
Abstract:
We present new versions of the previously published C and CUDA programs for solving the dipolar Gross-Pitaevskii equation in one, two, and three spatial dimensions, which calculate stationary and non-stationary solutions by propagation in imaginary or real time. Presented programs are improved and parallelized versions of previous programs, divided into three packages according to the type of para…
▽ More
We present new versions of the previously published C and CUDA programs for solving the dipolar Gross-Pitaevskii equation in one, two, and three spatial dimensions, which calculate stationary and non-stationary solutions by propagation in imaginary or real time. Presented programs are improved and parallelized versions of previous programs, divided into three packages according to the type of parallelization. First package contains improved and threaded version of sequential C programs using OpenMP. Second package additionally parallelizes three-dimensional variants of the OpenMP programs using MPI, allowing them to be run on distributed-memory systems. Finally, previous three-dimensional CUDA-parallelized programs are further parallelized using MPI, similarly as the OpenMP programs. We also present speedup test results obtained using new versions of programs in comparison with the previous sequential C and parallel CUDA programs. The improvements to the sequential version yield a speedup of 1.1 to 1.9, depending on the program. OpenMP parallelization yields further speedup of 2 to 12 on a 16-core workstation, while OpenMP/MPI version demonstrates a speedup of 11.5 to 16.5 on a computer cluster with 32 nodes used. CUDA/MPI version shows a speedup of 9 to 10 on a computer cluster with 32 nodes.
△ Less
Submitted 1 August, 2022; v1 submitted 17 October, 2016;
originally announced October 2016.
