-
High Pileup Particle Tracking with Object Condensation
Authors:
Kilian Lieret,
Gage DeZoort,
Devdoot Chatterjee,
Jian Park,
Siqi Miao,
Pan Li
Abstract:
Recent work has demonstrated that graph neural networks (GNNs) can match the performance of traditional algorithms for charged particle tracking while improving scalability to meet the computing challenges posed by the HL-LHC. Most GNN tracking algorithms are based on edge classification and identify tracks as connected components from an initial graph containing spurious connections. In this talk…
▽ More
Recent work has demonstrated that graph neural networks (GNNs) can match the performance of traditional algorithms for charged particle tracking while improving scalability to meet the computing challenges posed by the HL-LHC. Most GNN tracking algorithms are based on edge classification and identify tracks as connected components from an initial graph containing spurious connections. In this talk, we consider an alternative based on object condensation (OC), a multi-objective learning framework designed to cluster points (hits) belonging to an arbitrary number of objects (tracks) and regress the properties of each object. Building on our previous results, we present a streamlined model and show progress toward a one-shot OC tracking algorithm in a high-pileup environment.
△ Less
Submitted 6 December, 2023;
originally announced December 2023.
-
Principles for Initialization and Architecture Selection in Graph Neural Networks with ReLU Activations
Authors:
Gage DeZoort,
Boris Hanin
Abstract:
This article derives and validates three principles for initialization and architecture selection in finite width graph neural networks (GNNs) with ReLU activations. First, we theoretically derive what is essentially the unique generalization to ReLU GNNs of the well-known He-initialization. Our initialization scheme guarantees that the average scale of network outputs and gradients remains order…
▽ More
This article derives and validates three principles for initialization and architecture selection in finite width graph neural networks (GNNs) with ReLU activations. First, we theoretically derive what is essentially the unique generalization to ReLU GNNs of the well-known He-initialization. Our initialization scheme guarantees that the average scale of network outputs and gradients remains order one at initialization. Second, we prove in finite width vanilla ReLU GNNs that oversmoothing is unavoidable at large depth when using fixed aggregation operator, regardless of initialization. We then prove that using residual aggregation operators, obtained by interpolating a fixed aggregation operator with the identity, provably alleviates oversmoothing at initialization. Finally, we show that the common practice of using residual connections with a fixup-type initialization provably avoids correlation collapse in final layer features at initialization. Through ablation studies we find that using the correct initialization, residual aggregation operators, and residual connections in the forward pass significantly and reliably speeds up early training dynamics in deep ReLU GNNs on a variety of tasks.
△ Less
Submitted 20 June, 2023;
originally announced June 2023.
-
Graph Neural Networks in Particle Physics: Implementations, Innovations, and Challenges
Authors:
Savannah Thais,
Paolo Calafiura,
Grigorios Chachamis,
Gage DeZoort,
Javier Duarte,
Sanmay Ganguly,
Michael Kagan,
Daniel Murnane,
Mark S. Neubauer,
Kazuhiro Terao
Abstract:
Many physical systems can be best understood as sets of discrete data with associated relationships. Where previously these sets of data have been formulated as series or image data to match the available machine learning architectures, with the advent of graph neural networks (GNNs), these systems can be learned natively as graphs. This allows a wide variety of high- and low-level physical featur…
▽ More
Many physical systems can be best understood as sets of discrete data with associated relationships. Where previously these sets of data have been formulated as series or image data to match the available machine learning architectures, with the advent of graph neural networks (GNNs), these systems can be learned natively as graphs. This allows a wide variety of high- and low-level physical features to be attached to measurements and, by the same token, a wide variety of HEP tasks to be accomplished by the same GNN architectures. GNNs have found powerful use-cases in reconstruction, tagging, generation and end-to-end analysis. With the wide-spread adoption of GNNs in industry, the HEP community is well-placed to benefit from rapid improvements in GNN latency and memory usage. However, industry use-cases are not perfectly aligned with HEP and much work needs to be done to best match unique GNN capabilities to unique HEP obstacles. We present here a range of these capabilities, predictions of which are currently being well-adopted in HEP communities, and which are still immature. We hope to capture the landscape of graph techniques in machine learning as well as point out the most significant gaps that are inhibiting potentially large leaps in research.
△ Less
Submitted 25 March, 2022; v1 submitted 23 March, 2022;
originally announced March 2022.
-
Graph Neural Networks for Charged Particle Tracking on FPGAs
Authors:
Abdelrahman Elabd,
Vesal Razavimaleki,
Shi-Yu Huang,
Javier Duarte,
Markus Atkinson,
Gage DeZoort,
Peter Elmer,
Scott Hauck,
**-Xuan Hu,
Shih-Chieh Hsu,
Bo-Cheng Lai,
Mark Neubauer,
Isobel Ojalvo,
Savannah Thais,
Matthew Trahms
Abstract:
The determination of charged particle trajectories in collisions at the CERN Large Hadron Collider (LHC) is an important but challenging problem, especially in the high interaction density conditions expected during the future high-luminosity phase of the LHC (HL-LHC). Graph neural networks (GNNs) are a type of geometric deep learning algorithm that has successfully been applied to this task by em…
▽ More
The determination of charged particle trajectories in collisions at the CERN Large Hadron Collider (LHC) is an important but challenging problem, especially in the high interaction density conditions expected during the future high-luminosity phase of the LHC (HL-LHC). Graph neural networks (GNNs) are a type of geometric deep learning algorithm that has successfully been applied to this task by embedding tracker data as a graph -- nodes represent hits, while edges represent possible track segments -- and classifying the edges as true or fake track segments. However, their study in hardware- or software-based trigger applications has been limited due to their large computational cost. In this paper, we introduce an automated translation workflow, integrated into a broader tool called $\texttt{hls4ml}$, for converting GNNs into firmware for field-programmable gate arrays (FPGAs). We use this translation tool to implement GNNs for charged particle tracking, trained using the TrackML challenge dataset, on FPGAs with designs targeting different graph sizes, task complexites, and latency/throughput requirements. This work could enable the inclusion of charged particle tracking GNNs at the trigger level for HL-LHC experiments.
△ Less
Submitted 23 March, 2022; v1 submitted 3 December, 2021;
originally announced December 2021.
-
Charged particle tracking via edge-classifying interaction networks
Authors:
Gage DeZoort,
Savannah Thais,
Javier Duarte,
Vesal Razavimaleki,
Markus Atkinson,
Isobel Ojalvo,
Mark Neubauer,
Peter Elmer
Abstract:
Recent work has demonstrated that geometric deep learning methods such as graph neural networks (GNNs) are well suited to address a variety of reconstruction problems in high energy particle physics. In particular, particle tracking data is naturally represented as a graph by identifying silicon tracker hits as nodes and particle trajectories as edges; given a set of hypothesized edges, edge-class…
▽ More
Recent work has demonstrated that geometric deep learning methods such as graph neural networks (GNNs) are well suited to address a variety of reconstruction problems in high energy particle physics. In particular, particle tracking data is naturally represented as a graph by identifying silicon tracker hits as nodes and particle trajectories as edges; given a set of hypothesized edges, edge-classifying GNNs identify those corresponding to real particle trajectories. In this work, we adapt the physics-motivated interaction network (IN) GNN toward the problem of particle tracking in pileup conditions similar to those expected at the high-luminosity Large Hadron Collider. Assuming idealized hit filtering at various particle momenta thresholds, we demonstrate the IN's excellent edge-classification accuracy and tracking efficiency through a suite of measurements at each stage of GNN-based tracking: graph construction, edge classification, and track building. The proposed IN architecture is substantially smaller than previously studied GNN tracking architectures; this is particularly promising as a reduction in size is critical for enabling GNN-based tracking in constrained computing environments. Furthermore, the IN may be represented as either a set of explicit matrix operations or a message passing GNN. Efforts are underway to accelerate each representation via heterogeneous computing resources towards both high-level and low-latency triggering applications.
△ Less
Submitted 18 November, 2021; v1 submitted 30 March, 2021;
originally announced March 2021.
-
Performance of a Geometric Deep Learning Pipeline for HL-LHC Particle Tracking
Authors:
Xiangyang Ju,
Daniel Murnane,
Paolo Calafiura,
Nicholas Choma,
Sean Conlon,
Steve Farrell,
Yaoyuan Xu,
Maria Spiropulu,
Jean-Roch Vlimant,
Adam Aurisano,
V Hewes,
Giuseppe Cerati,
Lindsey Gray,
Thomas Klijnsma,
Jim Kowalkowski,
Markus Atkinson,
Mark Neubauer,
Gage DeZoort,
Savannah Thais,
Aditi Chauhan,
Alex Schuy,
Shih-Chieh Hsu,
Alex Ballow,
and Alina Lazar
Abstract:
The Exa.TrkX project has applied geometric learning concepts such as metric learning and graph neural networks to HEP particle tracking. Exa.TrkX's tracking pipeline groups detector measurements to form track candidates and filters them. The pipeline, originally developed using the TrackML dataset (a simulation of an LHC-inspired tracking detector), has been demonstrated on other detectors, includ…
▽ More
The Exa.TrkX project has applied geometric learning concepts such as metric learning and graph neural networks to HEP particle tracking. Exa.TrkX's tracking pipeline groups detector measurements to form track candidates and filters them. The pipeline, originally developed using the TrackML dataset (a simulation of an LHC-inspired tracking detector), has been demonstrated on other detectors, including DUNE Liquid Argon TPC and CMS High-Granularity Calorimeter. This paper documents new developments needed to study the physics and computing performance of the Exa.TrkX pipeline on the full TrackML dataset, a first step towards validating the pipeline using ATLAS and CMS data. The pipeline achieves tracking efficiency and purity similar to production tracking algorithms. Crucially for future HEP applications, the pipeline benefits significantly from GPU acceleration, and its computational requirements scale close to linearly with the number of particles in the event.
△ Less
Submitted 21 September, 2021; v1 submitted 11 March, 2021;
originally announced March 2021.
-
Instance Segmentation GNNs for One-Shot Conformal Tracking at the LHC
Authors:
Savannah Thais,
Gage DeZoort
Abstract:
3D instance segmentation remains a challenging problem in computer vision. Particle tracking at colliders like the LHC can be conceptualized as an instance segmentation task: beginning from a point cloud of hits in a particle detector, an algorithm must identify which hits belong to individual particle trajectories and extract track properties. Graph Neural Networks (GNNs) have shown promising per…
▽ More
3D instance segmentation remains a challenging problem in computer vision. Particle tracking at colliders like the LHC can be conceptualized as an instance segmentation task: beginning from a point cloud of hits in a particle detector, an algorithm must identify which hits belong to individual particle trajectories and extract track properties. Graph Neural Networks (GNNs) have shown promising performance on standard instance segmentation tasks. In this work we demonstrate the applicability of instance segmentation GNN architectures to particle tracking; moreover, we re-imagine the traditional Cartesian space approach to track-finding and instead work in a conformal geometry that allows the GNN to identify tracks and extract parameters in a single shot.
△ Less
Submitted 11 March, 2021;
originally announced March 2021.
-
Accelerated Charged Particle Tracking with Graph Neural Networks on FPGAs
Authors:
Aneesh Heintz,
Vesal Razavimaleki,
Javier Duarte,
Gage DeZoort,
Isobel Ojalvo,
Savannah Thais,
Markus Atkinson,
Mark Neubauer,
Lindsey Gray,
Sergo **dariani,
Nhan Tran,
Philip Harris,
Dylan Rankin,
Thea Aarrestad,
Vladimir Loncar,
Maurizio Pierini,
Sioni Summers,
Jennifer Ngadiuba,
Mia Liu,
Edward Kreinar,
Zhenbin Wu
Abstract:
We develop and study FPGA implementations of algorithms for charged particle tracking based on graph neural networks. The two complementary FPGA designs are based on OpenCL, a framework for writing programs that execute across heterogeneous platforms, and hls4ml, a high-level-synthesis-based compiler for neural network to firmware conversion. We evaluate and compare the resource usage, latency, an…
▽ More
We develop and study FPGA implementations of algorithms for charged particle tracking based on graph neural networks. The two complementary FPGA designs are based on OpenCL, a framework for writing programs that execute across heterogeneous platforms, and hls4ml, a high-level-synthesis-based compiler for neural network to firmware conversion. We evaluate and compare the resource usage, latency, and tracking performance of our implementations based on a benchmark dataset. We find a considerable speedup over CPU-based execution is possible, potentially enabling such algorithms to be used effectively in future computing workflows and the FPGA-based Level-1 trigger at the CERN Large Hadron Collider.
△ Less
Submitted 30 November, 2020;
originally announced December 2020.
