Showing 1–50 of 103 results for author: Duarte, J

HHH Whitepaper

Authors: Vuko Brigljevic, Dinko Ferencek, Greg Landsberg, Tania Robens, Marko Stamenkovic, Tatjana Susa, Hamza Abouabid, Abdesslam Arhrib, Hannah Arnold, Duarte Azevedo, Daniel Diaz, Javier Duarte, Tristan du Pree, Jaouad El Falaki, Pedro. M. Ferreira, Benjamin Fuks, Sanmay Ganguly, Marina Kolosova, Jacobo Konigsberg, Bingxuan Liu, Brian Moser, Margarete Muehlleitner, Andreas Papaefstathiou, Roman Pasechnik, Rui Santos , et al. (7 additional authors not shown)

Abstract: We here report on the progress of the HHH Workshop, that took place in Dubrovnik in July 2023. After the discovery of a particle that complies with the properties of the Higgs boson of the Standard Model, all SM parameters are in principle determined. However, in order to verify or falsify the model, the full form of the potential has to be determined. This includes the measurement of the triple a… ▽ More We here report on the progress of the HHH Workshop, that took place in Dubrovnik in July 2023. After the discovery of a particle that complies with the properties of the Higgs boson of the Standard Model, all SM parameters are in principle determined. However, in order to verify or falsify the model, the full form of the potential has to be determined. This includes the measurement of the triple and quartic scalar couplings. We here report on ongoing progress of measurements for multi scalar final states, with an emphasis on three SM-like scalar bosons at 125 GeV, but also mentioning other options. We discuss both experimental progress and challenges as well as theoretical studies and models that can enhance such rates with respect to the SM predictions. △ Less

Submitted 3 July, 2024; originally announced July 2024.

Comments: 117 pages, 56 figures; Whitepaper resulting from HHH Workshop in Dubrovnik 2023, https://indico.cern.ch/event/1232581/

Reliable edge machine learning hardware for scientific applications

Authors: Tommaso Baldi, Javier Campos, Ben Hawks, Jennifer Ngadiuba, Nhan Tran, Daniel Diaz, Javier Duarte, Ryan Kastner, Andres Meza, Melissa Quinnan, Olivia Weng, Caleb Geniesse, Amir Gholami, Michael W. Mahoney, Vladimir Loncar, Philip Harris, Joshua Agar, Shuyu Qin

Abstract: Extreme data rate scientific experiments create massive amounts of data that require efficient ML edge processing. This leads to unique validation challenges for VLSI implementations of ML algorithms: enabling bit-accurate functional simulations for performance validation in experimental software frameworks, verifying those ML models are robust under extreme quantization and pruning, and enabling… ▽ More Extreme data rate scientific experiments create massive amounts of data that require efficient ML edge processing. This leads to unique validation challenges for VLSI implementations of ML algorithms: enabling bit-accurate functional simulations for performance validation in experimental software frameworks, verifying those ML models are robust under extreme quantization and pruning, and enabling ultra-fine-grained model inspection for efficient fault tolerance. We discuss approaches to develo** and validating reliable algorithms at the scientific edge under such strict latency, resource, power, and area requirements in extreme experimental environments. We study metrics for develo** robust algorithms, present preliminary results and mitigation strategies, and conclude with an outlook of these and future directions of research towards the longer-term goal of develo** autonomous scientific experimentation methods for accelerated scientific discovery. △ Less

Submitted 27 June, 2024; originally announced June 2024.

Comments: IEEE VLSI Test Symposium 2024 (VTS)

Report number: FERMILAB-CONF-24-0116-CSAID

Modelling the impact of host galaxy dust on type Ia supernova distance measurements

Authors: B. Popovic, P. Wiseman, M. Sullivan, M. Smith, S. González-Gaitán, D. Scolnic, J. Duarte, P. Armstrong, J. Asorey, D. Brout, D. Carollo, L. Galbany, K. Glazebrook, L. Kelsey, R. Kessler, C. Lidman, J. Lee, G. F. Lewis, A. Möller, R. C. Nichol, B. O. Sánchez, M. Toy, B. E. Tucker, M. Vincenzi, T. M. C. Abbott , et al. (43 additional authors not shown)

Abstract: Type Ia Supernovae (SNe Ia) are a critical tool in measuring the accelerating expansion of the universe. Recent efforts to improve these standard candles have focused on incorporating the effects of dust on distance measurements with SNe Ia. In this paper, we use the state-of-the-art Dark Energy Survey 5 year sample to evaluate two different families of dust models: empirical extinction models der… ▽ More Type Ia Supernovae (SNe Ia) are a critical tool in measuring the accelerating expansion of the universe. Recent efforts to improve these standard candles have focused on incorporating the effects of dust on distance measurements with SNe Ia. In this paper, we use the state-of-the-art Dark Energy Survey 5 year sample to evaluate two different families of dust models: empirical extinction models derived from SNe Ia data, and physical attenuation models from the spectra of galaxies. Among the SNe Ia-derived models, we find that a logistic function of the total-to-selective extinction RV best recreates the correlations between supernova distance measurements and host galaxy properties, though an additional 0.02 magnitudes of grey scatter are needed to fully explain the scatter in SNIa brightness in all cases. These empirically-derived extinction distributions are highly incompatible with the physical attenuation models from galactic spectral measurements. From these results, we conclude that SNe Ia must either preferentially select extreme ends of galactic dust distributions, or that the characterisation of dust along the SNe Ia line-of-sight is incompatible with that of galactic dust distributions. △ Less

Submitted 7 June, 2024; originally announced June 2024.

The Dark Energy Survey Supernova Program: Light curves and 5-Year data release

Authors: B. O. Sánchez, D. Brout, M. Vincenzi, M. Sako, K. Herner, R. Kessler, T. M. Davis, D. Scolnic, M. Acevedo, J. Lee, A. Möller, H. Qu, L. Kelsey, P. Wiseman, P. Armstrong, B. Rose, R. Camilleri, R. Chen, L. Galbany, E. Kovacs, C. Lidman, B. Popovic, M. Smith, M. Sullivan, M. Toy , et al. (60 additional authors not shown)

Abstract: We present $griz$ photometric light curves for the full 5 years of the Dark Energy Survey Supernova program (DES-SN), obtained with both forced Point Spread Function (PSF) photometry on Difference Images (DIFFIMG) performed during survey operations, and Scene Modelling Photometry (SMP) on search images processed after the survey. This release contains $31,636$ DIFFIMG and $19,706$ high-quality SMP… ▽ More We present $griz$ photometric light curves for the full 5 years of the Dark Energy Survey Supernova program (DES-SN), obtained with both forced Point Spread Function (PSF) photometry on Difference Images (DIFFIMG) performed during survey operations, and Scene Modelling Photometry (SMP) on search images processed after the survey. This release contains $31,636$ DIFFIMG and $19,706$ high-quality SMP light curves, the latter of which contains $1635$ photometrically-classified supernovae that pass cosmology quality cuts. This sample spans the largest redshift ($z$) range ever covered by a single SN survey ($0.1<z<1.13$) and is the largest single sample from a single instrument of SNe ever used for cosmological constraints. We describe in detail the improvements made to obtain the final DES-SN photometry and provide a comparison to what was used in the DES-SN3YR spectroscopically-confirmed SN Ia sample. We also include a comparative analysis of the performance of the SMP photometry with respect to the real-time DIFFIMG forced photometry and find that SMP photometry is more precise, more accurate, and less sensitive to the host-galaxy surface brightness anomaly. The public release of the light curves and ancillary data can be found at https://github.com/des-science/DES-SN5YR. Finally, we discuss implications for future transient surveys, such as the forthcoming Vera Rubin Observatory Legacy Survey of Space and Time (LSST). △ Less

Submitted 7 June, 2024; originally announced June 2024.

Towards establishing best practice in the analysis of hydrogen and deuterium by atom probe tomography

Authors: Baptiste Gault, Aparna Saksena, Xavier Sauvage, Paul Bagot, Leonardo S. Aota, Jonas Arlt, Lisa T. Belkacemi, Torben Boll, Yi-Sheng Chen, Luke Daly, Milos B. Djukic, James O. Douglas, Maria J. Duarte, Peter J. Felfer, Richard G. Forbes, **g Fu, Hazel M. Gardner, Ryota Gemma, Stephan S. A. Gerstl, Yilun Gong, Guillaume Hachet, Severin Jakob, Benjamin M. Jenkins, Megan E. Jones, Heena Khanchandani , et al. (20 additional authors not shown)

Abstract: As hydrogen is touted as a key player in the decarbonization of modern society, it is critical to enable quantitative H analysis at high spatial resolution, if possible at the atomic scale. Indeed, H has a known deleterious impact on the mechanical properties (strength, ductility, toughness) of most materials that can hinder their use as part of the infrastructure of a hydrogen-based economy. Enab… ▽ More As hydrogen is touted as a key player in the decarbonization of modern society, it is critical to enable quantitative H analysis at high spatial resolution, if possible at the atomic scale. Indeed, H has a known deleterious impact on the mechanical properties (strength, ductility, toughness) of most materials that can hinder their use as part of the infrastructure of a hydrogen-based economy. Enabling H map**, including local hydrogen concentration analyses at specific microstructural features, is essential for understanding the multiple ways that H affect the properties of materials, including for instance embrittlement mechanisms and their synergies, but also spatial map** and quantification of hydrogen isotopes is essential to accurately predict tritium inventory of future fusion power plants, ensuring their safe and efficient operation for example. Atom probe tomography (APT) has the intrinsic capabilities for detecting hydrogen (H), and deuterium (D), and in principle the capacity for performing quantitative map** of H within a material's microstructure. Yet the accuracy and precision of H analysis by APT remain affected by the influence of residual hydrogen from the ultra-high vacuum chamber that can obscure the signal of H from within the material, along with a complex field evaporation behavior. The present article reports the essence of discussions at a focused workshop held at the Max-Planck Institute for Sustainable Materials in April 2024. The workshop was organized to pave the way to establishing best practices in reporting APT data for the analysis of H. We first summarize the key aspects of the intricacies of H analysis by APT and propose a path for better reporting of the relevant data to support interpretation of APT-based H analysis in materials. △ Less

Submitted 21 May, 2024; originally announced May 2024.

Accelerating Resonance Searches via Signature-Oriented Pre-training

Authors: Congqiao Li, Antonios Agapitos, Jovin Drews, Javier Duarte, Dawei Fu, Leyun Gao, Raghav Kansal, Gregor Kasieczka, Louis Moureaux, Huilin Qu, Cristina Mantilla Suarez, Qiang Li

Abstract: The search for heavy resonances beyond the Standard Model (BSM) is a key objective at the LHC. While the recent use of advanced deep neural networks for boosted-jet tagging significantly enhances the sensitivity of dedicated searches, it is limited to specific final states, leaving vast potential BSM phase space underexplored. We introduce a novel experimental method, Signature-Oriented Pre-traini… ▽ More The search for heavy resonances beyond the Standard Model (BSM) is a key objective at the LHC. While the recent use of advanced deep neural networks for boosted-jet tagging significantly enhances the sensitivity of dedicated searches, it is limited to specific final states, leaving vast potential BSM phase space underexplored. We introduce a novel experimental method, Signature-Oriented Pre-training for Heavy-resonance ObservatioN (Sophon), which leverages deep learning to cover an extensive number of boosted final states. Pre-trained on the comprehensive JetClass-II dataset, the Sophon model learns intricate jet signatures, ensuring the optimal constructions of various jet tagging discriminates and enabling high-performance transfer learning capabilities. We show that the method can not only push widespread model-specific searches to their sensitivity frontier, but also greatly improve model-agnostic approaches, accelerating LHC resonance searches in a broad sense. △ Less

Submitted 21 May, 2024; originally announced May 2024.

Comments: 14 pages, 5 figures

Architectural Implications of Neural Network Inference for High Data-Rate, Low-Latency Scientific Applications

Authors: Olivia Weng, Alexander Redding, Nhan Tran, Javier Mauricio Duarte, Ryan Kastner

Abstract: With more scientific fields relying on neural networks (NNs) to process data incoming at extreme throughputs and latencies, it is crucial to develop NNs with all their parameters stored on-chip. In many of these applications, there is not enough time to go off-chip and retrieve weights. Even more so, off-chip memory such as DRAM does not have the bandwidth required to process these NNs as fast as… ▽ More With more scientific fields relying on neural networks (NNs) to process data incoming at extreme throughputs and latencies, it is crucial to develop NNs with all their parameters stored on-chip. In many of these applications, there is not enough time to go off-chip and retrieve weights. Even more so, off-chip memory such as DRAM does not have the bandwidth required to process these NNs as fast as the data is being produced (e.g., every 25 ns). As such, these extreme latency and bandwidth requirements have architectural implications for the hardware intended to run these NNs: 1) all NN parameters must fit on-chip, and 2) codesigning custom/reconfigurable logic is often required to meet these latency and bandwidth constraints. In our work, we show that many scientific NN applications must run fully on chip, in the extreme case requiring a custom chip to meet such stringent constraints. △ Less

Submitted 13 March, 2024; originally announced March 2024.

Locality-Sensitive Hashing-Based Efficient Point Transformer with Applications in High-Energy Physics

Authors: Siqi Miao, Zhiyuan Lu, Mia Liu, Javier Duarte, Pan Li

Abstract: This study introduces a novel transformer model optimized for large-scale point cloud processing in scientific domains such as high-energy physics (HEP) and astrophysics. Addressing the limitations of graph neural networks and standard transformers, our model integrates local inductive bias and achieves near-linear complexity with hardware-friendly regular operations. One contribution of this work… ▽ More This study introduces a novel transformer model optimized for large-scale point cloud processing in scientific domains such as high-energy physics (HEP) and astrophysics. Addressing the limitations of graph neural networks and standard transformers, our model integrates local inductive bias and achieves near-linear complexity with hardware-friendly regular operations. One contribution of this work is the quantitative analysis of the error-complexity tradeoff of various sparsification techniques for building efficient transformers. Our findings highlight the superiority of using locality-sensitive hashing (LSH), especially OR & AND-construction LSH, in kernel approximation for large-scale point cloud data with local inductive bias. Based on this finding, we propose LSH-based Efficient Point Transformer (HEPT), which combines E$^2$LSH with OR & AND constructions and is built upon regular computations. HEPT demonstrates remarkable performance on two critical yet time-consuming HEP tasks, significantly outperforming existing GNNs and transformers in accuracy and computational speed, marking a significant advancement in geometric deep learning and large-scale scientific data processing. Our code is available at https://github.com/Graph-COM/HEPT. △ Less

Submitted 5 June, 2024; v1 submitted 19 February, 2024; originally announced February 2024.

Comments: Accepted to ICML 2024 (Oral)

Sets are all you need: Ultrafast jet classification on FPGAs for HL-LHC

Authors: Patrick Odagiu, Zhiqiang Que, Javier Duarte, Johannes Haller, Gregor Kasieczka, Artur Lobanov, Vladimir Loncar, Wayne Luk, Jennifer Ngadiuba, Maurizio Pierini, Philipp Rincke, Arpita Seksaria, Sioni Summers, Andre Sznajder, Alexander Tapper, Thea K. Aarrestad

Abstract: We study various machine learning based algorithms for performing accurate jet flavor classification on field-programmable gate arrays and demonstrate how latency and resource consumption scale with the input size and choice of algorithm. These architectures provide an initial design for models that could be used for tagging at the CERN LHC during its high-luminosity phase. The high-luminosity upg… ▽ More We study various machine learning based algorithms for performing accurate jet flavor classification on field-programmable gate arrays and demonstrate how latency and resource consumption scale with the input size and choice of algorithm. These architectures provide an initial design for models that could be used for tagging at the CERN LHC during its high-luminosity phase. The high-luminosity upgrade will lead to a five-fold increase in its instantaneous luminosity for proton-proton collisions and, in turn, higher data volume and complexity, such as the availability of jet constituents. Through quantization-aware training and efficient hardware implementations, we show that O(100) ns inference of complex architectures such as deep sets and interaction networks is feasible at a low computational resource cost. △ Less

Submitted 2 February, 2024; originally announced February 2024.

Comments: 13 pages, 3 figures, 3 tables

Report number: FERMILAB-PUB-24-0030-CMS-CSAID-PPD

Developments and applications of the OPTIMADE API for materials discovery, design, and data exchange

Authors: Matthew L. Evans, Johan Bergsma, Andrius Merkys, Casper W. Andersen, Oskar B. Andersson, Daniel Beltrán, Evgeny Blokhin, Tara M. Boland, Rubén Castañeda Balderas, Kamal Choudhary, Alberto Díaz Díaz, Rodrigo Domínguez García, Hagen Eckert, Kristjan Eimre, María Elena Fuentes Montero, Adam M. Krajewski, Jens Jørgen Mortensen, José Manuel Nápoles Duarte, Jacob Pietryga, Ji Qi, Felipe de Jesús Trejo Carrillo, Antanas Vaitkus, Jusong Yu, Adam Zettel, Pedro Baptista de Castro , et al. (34 additional authors not shown)

Abstract: The Open Databases Integration for Materials Design (OPTIMADE) application programming interface (API) empowers users with holistic access to a growing federation of databases, enhancing the accessibility and discoverability of materials and chemical data. Since the first release of the OPTIMADE specification (v1.0), the API has undergone significant development, leading to the upcoming v1.2 relea… ▽ More The Open Databases Integration for Materials Design (OPTIMADE) application programming interface (API) empowers users with holistic access to a growing federation of databases, enhancing the accessibility and discoverability of materials and chemical data. Since the first release of the OPTIMADE specification (v1.0), the API has undergone significant development, leading to the upcoming v1.2 release, and has underpinned multiple scientific studies. In this work, we highlight the latest features of the API format, accompanying software tools, and provide an update on the implementation of OPTIMADE in contributing materials databases. We end by providing several use cases that demonstrate the utility of the OPTIMADE API in materials research that continue to drive its ongoing development. △ Less

Submitted 5 April, 2024; v1 submitted 1 February, 2024; originally announced February 2024.

Neural Architecture Codesign for Fast Bragg Peak Analysis

Authors: Luke McDermott, Jason Weitz, Dmitri Demler, Daniel Cummings, Nhan Tran, Javier Duarte

Abstract: We develop an automated pipeline to streamline neural architecture codesign for fast, real-time Bragg peak analysis in high-energy diffraction microscopy. Traditional approaches, notably pseudo-Voigt fitting, demand significant computational resources, prompting interest in deep learning models for more efficient solutions. Our method employs neural architecture search and AutoML to enhance these… ▽ More We develop an automated pipeline to streamline neural architecture codesign for fast, real-time Bragg peak analysis in high-energy diffraction microscopy. Traditional approaches, notably pseudo-Voigt fitting, demand significant computational resources, prompting interest in deep learning models for more efficient solutions. Our method employs neural architecture search and AutoML to enhance these models, including hardware costs, leading to the discovery of more hardware-efficient neural architectures. Our results match the performance, while achieving a 13$\times$ reduction in bit operations compared to the previous state-of-the-art. We show further speedup through model compression techniques such as quantization-aware-training and neural network pruning. Additionally, our hierarchical search space provides greater flexibility in optimization, which can easily extend to other tasks and domains. △ Less

Submitted 11 December, 2023; v1 submitted 10 December, 2023; originally announced December 2023.

Comments: To appear in 3rd Annual AAAI Workshop on AI to Accelerate Science and Engineering (AI2ASE)

Report number: FERMILAB-CONF-23-0813-CSAID-PPD

Induced Generative Adversarial Particle Transformers

Authors: Anni Li, Venkat Krishnamohan, Raghav Kansal, Rounak Sen, Steven Tsan, Zhaoyu Zhang, Javier Duarte

Abstract: In high energy physics (HEP), machine learning methods have emerged as an effective way to accurately simulate particle collisions at the Large Hadron Collider (LHC). The message-passing generative adversarial network (MPGAN) was the first model to simulate collisions as point, or ``particle'', clouds, with state-of-the-art results, but suffered from quadratic time complexity. Recently, generative… ▽ More In high energy physics (HEP), machine learning methods have emerged as an effective way to accurately simulate particle collisions at the Large Hadron Collider (LHC). The message-passing generative adversarial network (MPGAN) was the first model to simulate collisions as point, or ``particle'', clouds, with state-of-the-art results, but suffered from quadratic time complexity. Recently, generative adversarial particle transformers (GAPTs) were introduced to address this drawback; however, results did not surpass MPGAN. We introduce induced GAPT (iGAPT) which, by integrating ``induced particle-attention blocks'' and conditioning on global jet attributes, not only offers linear time complexity but is also able to capture intricate jet substructure, surpassing MPGAN in many metrics. Our experiments demonstrate the potential of iGAPT to simulate complex HEP data accurately and efficiently. △ Less

Submitted 7 December, 2023; originally announced December 2023.

Comments: 5 pages, 3 figures, 2 tables, to appear in the workshop on Machine Learning and the Physical Sciences (NeurIPS 2023)

Report number: FERMILAB-CONF-23-751-CMS-PPD

Introducing Modelling, Analysis and Control of Three-Phase Electrical Systems Using Geometric Algebra

Authors: Manel Velasco, Isiah Zaplana, Arnau Dòria-Cerezo, Josué Duarte, Pau Martí

Abstract: State-of-the-art techniques for modeling, analysis and control of three-phase electrical systems belong to the real-valued multi-input/multi-output (MIMO) domain, or to the complex-valued nonlinear single-input/single-output (SISO) domain. In order to complement both domains while simplifying complexity and offering new analysis and design perspectives, this paper introduces the application of geo… ▽ More State-of-the-art techniques for modeling, analysis and control of three-phase electrical systems belong to the real-valued multi-input/multi-output (MIMO) domain, or to the complex-valued nonlinear single-input/single-output (SISO) domain. In order to complement both domains while simplifying complexity and offering new analysis and design perspectives, this paper introduces the application of geometric algebra (GA) principles to the modeling, analysis and control of three-phase electrical systems. The key contribution for the modeling part is the identification of the transformation that allows transferring real-valued linear MIMO systems into GA-valued linear SISO representations (with independence of having a balanced or unbalanced system). Closed-loop stability analysis in the new space is addressed by using intrinsic properties of GA. In addition, a recipe for designing stabilizing and decoupling GA-valued controllers is provided. Numerical examples illustrate key developments and experiments corroborate the main findings. △ Less

Submitted 3 December, 2023; originally announced December 2023.

LHC Hadronic Jet Generation Using Convolutional Variational Autoencoders with Normalizing Flows

Authors: Breno Orzari, Nadezda Chernyavskaya, Raphael Cobe, Javier Duarte, Jefferson Fialho, Dimitrios Gunopulos, Raghav Kansal, Maurizio Pierini, Thiago Tomei, Mary Touranakou

Abstract: In high energy physics, one of the most important processes for collider data analysis is the comparison of collected and simulated data. Nowadays the state-of-the-art for data generation is in the form of Monte Carlo (MC) generators. However, because of the upcoming high-luminosity upgrade of the LHC, there will not be enough computational power or time to match the amount of needed simulated dat… ▽ More In high energy physics, one of the most important processes for collider data analysis is the comparison of collected and simulated data. Nowadays the state-of-the-art for data generation is in the form of Monte Carlo (MC) generators. However, because of the upcoming high-luminosity upgrade of the LHC, there will not be enough computational power or time to match the amount of needed simulated data using MC methods. An alternative approach under study is the usage of machine learning generative methods to fulfill that task.Since the most common final-state objects of high-energy proton collisions are hadronic jets, which are collections of particles collimated in a given region of space, this work aims to develop a convolutional variational autoencoder (ConVAE) for the generation of particle-based LHC hadronic jets. Given the ConVAE's limitations, a normalizing flow (NF) network is coupled to it in a two-step training process, which shows improvements on the results for the generated jets. The ConVAE+NF network is capable of generating a jet in $18.30 \pm 0.04 \ μ$s, making it one of the fastest methods for this task up to now. △ Less

Submitted 8 November, 2023; v1 submitted 19 October, 2023; originally announced October 2023.

Comments: 17 pages, 4 figures and 8 tables

Journal ref: Mach. Learn.: Sci. Technol. 4 045023 (2023)

Improved particle-flow event reconstruction with scalable neural networks for current and future particle detectors

Authors: Joosep Pata, Eric Wulff, Farouk Mokhtar, David Southwick, Mengke Zhang, Maria Girone, Javier Duarte

Abstract: Efficient and accurate algorithms are necessary to reconstruct particles in the highly granular detectors anticipated at the High-Luminosity Large Hadron Collider and the Future Circular Collider. We study scalable machine learning models for event reconstruction in electron-positron collisions based on a full detector simulation. Particle-flow reconstruction can be formulated as a supervised lear… ▽ More Efficient and accurate algorithms are necessary to reconstruct particles in the highly granular detectors anticipated at the High-Luminosity Large Hadron Collider and the Future Circular Collider. We study scalable machine learning models for event reconstruction in electron-positron collisions based on a full detector simulation. Particle-flow reconstruction can be formulated as a supervised learning task using tracks and calorimeter clusters. We compare a graph neural network and kernel-based transformer and demonstrate that we can avoid quadratic operations while achieving realistic reconstruction. We show that hyperparameter tuning significantly improves the performance of the models. The best graph neural network model shows improvement in the jet transverse momentum resolution by up to 50% compared to the rule-based algorithm. The resulting model is portable across Nvidia, AMD and Habana hardware. Accurate and fast machine-learning based reconstruction can significantly improve future measurements at colliders. △ Less

Submitted 8 March, 2024; v1 submitted 13 September, 2023; originally announced September 2023.

Comments: 21 pages, 10 figures

On the gravitational energy problem and the energy of photons

Authors: J. B. Formiga, João Duarte

Abstract: The lack of a well-established solution for the gravitational energy problem might be one of the reasons why a clear road to quantum gravity does not exist. In this paper, the gravitational energy is studied in detail with the help of the teleparallel approach that is equivalent to general relativity. This approach is applied to the solutions of the Einstein-Maxwell equations known as $pp$-wave sp… ▽ More The lack of a well-established solution for the gravitational energy problem might be one of the reasons why a clear road to quantum gravity does not exist. In this paper, the gravitational energy is studied in detail with the help of the teleparallel approach that is equivalent to general relativity. This approach is applied to the solutions of the Einstein-Maxwell equations known as $pp$-wave spacetimes. The quantization of the electromagnetic energy is assumed and it is shown that the proper area measured by an observer must satisfy an equation for consistency. The meaning of this equation is discussed and it is argued that the spacetime geometry should become discrete once all matter fields are quantized, including the constituents of the frame; it is shown that for a harmonic oscillation with wavelength $λ_0$, the area and the volume take the form $A=4(N+1/2)l_p^2/n$ and $V=2(N+1/2)l_p^2λ_0$, where $N$ is the number of photons, $l_p$ the Planck length, and $n$ is a natural number associated with the length along the $z$-axis of a box with cross-sectional area $A$. The localization of the gravitational energy problem is also discussed. The stress-energy tensors for the gravitational and electromagnetic fields are decomposed into energy density, pressures and heat flow. The resultant expressions are consistent with the properties of the fields, thus indicating that one can have a well-defined energy density for the gravitational field regardless of the principle of equivalence. △ Less

Submitted 24 August, 2023; originally announced August 2023.

Comments: It has been published in Physical Review D under the title Gravitational energy problem and the energy of photons

Journal ref: Phys. Rev. D 108, 044043 (2023)

Low Latency Edge Classification GNN for Particle Trajectory Tracking on FPGAs

Authors: Shi-Yu Huang, Yun-Chen Yang, Yu-Ru Su, Bo-Cheng Lai, Javier Duarte, Scott Hauck, Shih-Chieh Hsu, **-Xuan Hu, Mark S. Neubauer

Abstract: In-time particle trajectory reconstruction in the Large Hadron Collider is challenging due to the high collision rate and numerous particle hits. Using GNN (Graph Neural Network) on FPGA has enabled superior accuracy with flexible trajectory classification. However, existing GNN architectures have inefficient resource usage and insufficient parallelism for edge classification. This paper introduce… ▽ More In-time particle trajectory reconstruction in the Large Hadron Collider is challenging due to the high collision rate and numerous particle hits. Using GNN (Graph Neural Network) on FPGA has enabled superior accuracy with flexible trajectory classification. However, existing GNN architectures have inefficient resource usage and insufficient parallelism for edge classification. This paper introduces a resource-efficient GNN architecture on FPGAs for low latency particle tracking. The modular architecture facilitates design scalability to support large graphs. Leveraging the geometric properties of hit detectors further reduces graph complexity and resource usage. Our results on Xilinx UltraScale+ VU9P demonstrate 1625x and 1574x performance improvement over CPU and GPU respectively. △ Less

Submitted 27 June, 2023; v1 submitted 20 June, 2023; originally announced June 2023.

Applications of Deep Learning to physics workflows

Authors: Manan Agarwal, Jay Alameda, Jeroen Audenaert, Will Benoit, Damon Beveridge, Meghna Bhattacharya, Chayan Chatterjee, Deep Chatterjee, Andy Chen, Muhammed Saleem Cholayil, Chia-Jui Chou, Sunil Choudhary, Michael Coughlin, Maximilian Dax, Aman Desai, Andrea Di Luca, Javier Mauricio Duarte, Steven Farrell, Yongbin Feng, Pooyan Goodarzi, Ekaterina Govorkova, Matthew Graham, Jonathan Guiang, Alec Gunny, Weichangfeng Guo , et al. (43 additional authors not shown)

Abstract: Modern large-scale physics experiments create datasets with sizes and streaming rates that can exceed those from industry leaders such as Google Cloud and Netflix. Fully processing these datasets requires both sufficient compute power and efficient workflows. Recent advances in Machine Learning (ML) and Artificial Intelligence (AI) can either improve or replace existing domain-specific algorithms… ▽ More Modern large-scale physics experiments create datasets with sizes and streaming rates that can exceed those from industry leaders such as Google Cloud and Netflix. Fully processing these datasets requires both sufficient compute power and efficient workflows. Recent advances in Machine Learning (ML) and Artificial Intelligence (AI) can either improve or replace existing domain-specific algorithms to increase workflow efficiency. Not only can these algorithms improve the physics performance of current algorithms, but they can often be executed more quickly, especially when run on coprocessors such as GPUs or FPGAs. In the winter of 2023, MIT hosted the Accelerating Physics with ML at MIT workshop, which brought together researchers from gravitational-wave physics, multi-messenger astrophysics, and particle physics to discuss and share current efforts to integrate ML tools into their workflows. The following white paper highlights examples of algorithms and computing frameworks discussed during this workshop and summarizes the expected computing needs for the immediate future of the involved fields. △ Less

Submitted 13 June, 2023; originally announced June 2023.

Comments: Whitepaper resulting from Accelerating Physics with ML@MIT workshop in Jan/Feb 2023

Differentiable Earth Mover's Distance for Data Compression at the High-Luminosity LHC

Authors: Rohan Shenoy, Javier Duarte, Christian Herwig, James Hirschauer, Daniel Noonan, Maurizio Pierini, Nhan Tran, Cristina Mantilla Suarez

Abstract: The Earth mover's distance (EMD) is a useful metric for image recognition and classification, but its usual implementations are not differentiable or too slow to be used as a loss function for training other algorithms via gradient descent. In this paper, we train a convolutional neural network (CNN) to learn a differentiable, fast approximation of the EMD and demonstrate that it can be used as a… ▽ More The Earth mover's distance (EMD) is a useful metric for image recognition and classification, but its usual implementations are not differentiable or too slow to be used as a loss function for training other algorithms via gradient descent. In this paper, we train a convolutional neural network (CNN) to learn a differentiable, fast approximation of the EMD and demonstrate that it can be used as a substitute for computing-intensive EMD implementations. We apply this differentiable approximation in the training of an autoencoder-inspired neural network (encoder NN) for data compression at the high-luminosity LHC at CERN. The goal of this encoder NN is to compress the data while preserving the information related to the distribution of energy deposits in particle detectors. We demonstrate that the performance of our encoder NN trained using the differentiable EMD CNN surpasses that of training with loss functions based on mean squared error. △ Less

Submitted 29 December, 2023; v1 submitted 7 June, 2023; originally announced June 2023.

Comments: 16 pages, 7 figures

Report number: FERMILAB-PUB-23-288-CMS-CSAID

Journal ref: Mach. Learn.: Sci. Technol. 4 (2023) 045058

End-to-end codesign of Hessian-aware quantized neural networks for FPGAs and ASICs

Authors: Javier Campos, Zhen Dong, Javier Duarte, Amir Gholami, Michael W. Mahoney, Jovan Mitrevski, Nhan Tran

Abstract: We develop an end-to-end workflow for the training and implementation of co-designed neural networks (NNs) for efficient field-programmable gate array (FPGA) and application-specific integrated circuit (ASIC) hardware. Our approach leverages Hessian-aware quantization (HAWQ) of NNs, the Quantized Open Neural Network Exchange (QONNX) intermediate representation, and the hls4ml tool flow for transpi… ▽ More We develop an end-to-end workflow for the training and implementation of co-designed neural networks (NNs) for efficient field-programmable gate array (FPGA) and application-specific integrated circuit (ASIC) hardware. Our approach leverages Hessian-aware quantization (HAWQ) of NNs, the Quantized Open Neural Network Exchange (QONNX) intermediate representation, and the hls4ml tool flow for transpiling NNs into FPGA and ASIC firmware. This makes efficient NN implementations in hardware accessible to nonexperts, in a single open-sourced workflow that can be deployed for real-time machine learning applications in a wide range of scientific and industrial settings. We demonstrate the workflow in a particle physics application involving trigger decisions that must operate at the 40 MHz collision rate of the CERN Large Hadron Collider (LHC). Given the high collision rate, all data processing must be implemented on custom ASIC and FPGA hardware within a strict area and latency. Based on these constraints, we implement an optimized mixed-precision NN classifier for high-momentum particle jets in simulated LHC proton-proton collisions. △ Less

Submitted 13 April, 2023; originally announced April 2023.

Comments: 19 pages, 6 figures, 2 tables

Report number: FERMILAB-PUB-23-150-CSAID-ETD

Progress towards an improved particle flow algorithm at CMS with machine learning

Authors: Farouk Mokhtar, Joosep Pata, Javier Duarte, Eric Wulff, Maurizio Pierini, Jean-Roch Vlimant

Abstract: The particle-flow (PF) algorithm, which infers particles based on tracks and calorimeter clusters, is of central importance to event reconstruction in the CMS experiment at the CERN LHC, and has been a focus of development in light of planned Phase-2 running conditions with an increased pileup and detector granularity. In recent years, the machine learned particle-flow (MLPF) algorithm, a graph ne… ▽ More The particle-flow (PF) algorithm, which infers particles based on tracks and calorimeter clusters, is of central importance to event reconstruction in the CMS experiment at the CERN LHC, and has been a focus of development in light of planned Phase-2 running conditions with an increased pileup and detector granularity. In recent years, the machine learned particle-flow (MLPF) algorithm, a graph neural network that performs PF reconstruction, has been explored in CMS, with the possible advantages of directly optimizing for the physical quantities of interest, being highly reconfigurable to new conditions, and being a natural fit for deployment to heterogeneous accelerators. We discuss progress in CMS towards an improved implementation of the MLPF reconstruction, now optimized using generator/simulation-level particle information as the target for the first time. This paves the way to potentially improving the detector response in terms of physical quantities of interest. We describe the simulation-based training target, progress and studies on event-based loss terms, details on the model hyperparameter tuning, as well as physics validation with respect to the current PF algorithm in terms of high-level physical quantities such as the jet and missing transverse momentum resolutions. We find that the MLPF algorithm, trained on a generator/simulator level particle information for the first time, results in broadly compatible particle and jet reconstruction performance with the baseline PF, setting the stage for improving the physics performance by additional training statistics and model tuning. △ Less

Submitted 30 March, 2023; originally announced March 2023.

Comments: 7 pages, 4 Figures, 1 Table

Journal ref: ACAT 2022: 21st International Workshop on Advanced Computing and Analysis Techniques in Physics Research

Microsecond Isomer at the N=20 Island of Shape Inversion Observed at FRIB

Authors: T. J. Gray, J. M. Allmond, Z. Xu, T. T. King, R. S. Lubna, H. L. Crawford, V. Tripathi, B. P. Crider, R. Grzywacz, S. N. Liddick, A. O. Macchiavelli, T. Miyagi, A. Poves, A. Andalib, E. Argo, C. Benetti, S. Bhattacharya, C. M. Campbell, M. P. Carpenter, J. Chan, A. Chester, J. Christie, B. R. Clark, I. Cox, A. A. Doetsch , et al. (41 additional authors not shown)

Abstract: Excited-state spectroscopy from the first Facility for Rare Isotope Beams (FRIB) experiment is reported. A 24(2)-$μ$s isomer was observed with the FRIB Decay Station initiator (FDSi) through a cascade of 224- and 401-keV $γ$ rays in coincidence with $^{32}\textrm{Na}$ nuclei. This is the only known microsecond isomer ($1{\text{ }μ\text{s}}\leq T_{1/2} < 1\text{ ms}$) in the region. This nucleus is… ▽ More Excited-state spectroscopy from the first Facility for Rare Isotope Beams (FRIB) experiment is reported. A 24(2)-$μ$s isomer was observed with the FRIB Decay Station initiator (FDSi) through a cascade of 224- and 401-keV $γ$ rays in coincidence with $^{32}\textrm{Na}$ nuclei. This is the only known microsecond isomer ($1{\text{ }μ\text{s}}\leq T_{1/2} < 1\text{ ms}$) in the region. This nucleus is at the heart of the $N=20$ island of shape inversion and is at the crossroads of spherical shell-model, deformed shell-model, and ab initio theories. It can be represented as the coupling of a proton hole and neutron particle to $^{32}\textrm{Mg}$, $^{32}\textrm{Mg}+π^{-1} + ν^{+1}$. This odd-odd coupling and isomer formation provides a sensitive measure of the underlying shape degrees of freedom of $^{32}\textrm{Mg}$, where the onset of spherical-to-deformed shape inversion begins with a low-lying deformed $2^+$ state at 885 keV and a low-lying shape-coexisting $0_2^+$ state at 1058 keV. We suggest two possible explanations for the 625-keV isomer in $^{32}$Na: a $6^-$ spherical shape isomer that decays by $E2$ or a $0^+$ deformed spin isomer that decays by $M2$. The present results and calculations are most consistent with the latter, indicating that the low-lying states are dominated by deformation. △ Less

Submitted 26 April, 2023; v1 submitted 22 February, 2023; originally announced February 2023.

Comments: 7 pages, 5 figures, accepted by Physical Review Letters

Tailor: Altering Skip Connections for Resource-Efficient Inference

Authors: Olivia Weng, Gabriel Marcano, Vladimir Loncar, Alireza Khodamoradi, Nojan Sheybani, Andres Meza, Farinaz Koushanfar, Kristof Denolf, Javier Mauricio Duarte, Ryan Kastner

Abstract: Deep neural networks use skip connections to improve training convergence. However, these skip connections are costly in hardware, requiring extra buffers and increasing on- and off-chip memory utilization and bandwidth requirements. In this paper, we show that skip connections can be optimized for hardware when tackled with a hardware-software codesign approach. We argue that while a network's sk… ▽ More Deep neural networks use skip connections to improve training convergence. However, these skip connections are costly in hardware, requiring extra buffers and increasing on- and off-chip memory utilization and bandwidth requirements. In this paper, we show that skip connections can be optimized for hardware when tackled with a hardware-software codesign approach. We argue that while a network's skip connections are needed for the network to learn, they can later be removed or shortened to provide a more hardware efficient implementation with minimal to no accuracy loss. We introduce Tailor, a codesign tool whose hardware-aware training algorithm gradually removes or shortens a fully trained network's skip connections to lower their hardware cost. Tailor improves resource utilization by up to 34% for BRAMs, 13% for FFs, and 16% for LUTs for on-chip, dataflow-style architectures. Tailor increases performance by 30% and reduces memory bandwidth by 45% for a 2D processing element array architecture. △ Less

Submitted 15 September, 2023; v1 submitted 17 January, 2023; originally announced January 2023.

Lorentz group equivariant autoencoders

Authors: Zichun Hao, Raghav Kansal, Javier Duarte, Nadezda Chernyavskaya

Abstract: There has been significant work recently in develo** machine learning (ML) models in high energy physics (HEP) for tasks such as classification, simulation, and anomaly detection. Often these models are adapted from those designed for datasets in computer vision or natural language processing, which lack inductive biases suited to HEP data, such as equivariance to its inherent symmetries. Such b… ▽ More There has been significant work recently in develo** machine learning (ML) models in high energy physics (HEP) for tasks such as classification, simulation, and anomaly detection. Often these models are adapted from those designed for datasets in computer vision or natural language processing, which lack inductive biases suited to HEP data, such as equivariance to its inherent symmetries. Such biases have been shown to make models more performant and interpretable, and reduce the amount of training data needed. To that end, we develop the Lorentz group autoencoder (LGAE), an autoencoder model equivariant with respect to the proper, orthochronous Lorentz group $\mathrm{SO}^+(3,1)$, with a latent space living in the representations of the group. We present our architecture and several experimental results on jets at the LHC and find it outperforms graph and convolutional neural network baseline models on several compression, reconstruction, and anomaly detection metrics. We also demonstrate the advantage of such an equivariant model in analyzing the latent space of the autoencoder, which can improve the explainability of potential anomalies discovered by such ML models. △ Less

Submitted 10 June, 2023; v1 submitted 14 December, 2022; originally announced December 2022.

Comments: 11 pages, 7 figures, 4 tables, and a 3 page appendix

Journal ref: Eur. Phys. J. C 83, 485 (2023)

FAIR AI Models in High Energy Physics

Authors: Javier Duarte, Haoyang Li, Avik Roy, Ruike Zhu, E. A. Huerta, Daniel Diaz, Philip Harris, Raghav Kansal, Daniel S. Katz, Ishaan H. Kavoori, Volodymyr V. Kindratenko, Farouk Mokhtar, Mark S. Neubauer, Sang Eon Park, Melissa Quinnan, Roger Rusack, Zhizhen Zhao

Abstract: The findable, accessible, interoperable, and reusable (FAIR) data principles provide a framework for examining, evaluating, and improving how data is shared to facilitate scientific discovery. Generalizing these principles to research software and other digital products is an active area of research. Machine learning (ML) models -- algorithms that have been trained on data without being explicitly… ▽ More The findable, accessible, interoperable, and reusable (FAIR) data principles provide a framework for examining, evaluating, and improving how data is shared to facilitate scientific discovery. Generalizing these principles to research software and other digital products is an active area of research. Machine learning (ML) models -- algorithms that have been trained on data without being explicitly programmed -- and more generally, artificial intelligence (AI) models, are an important target for this because of the ever-increasing pace with which AI is transforming scientific domains, such as experimental high energy physics (HEP). In this paper, we propose a practical definition of FAIR principles for AI models in HEP and describe a template for the application of these principles. We demonstrate the template's use with an example AI model applied to HEP, in which a graph neural network is used to identify Higgs bosons decaying to two bottom quarks. We report on the robustness of this FAIR AI model, its portability across hardware architectures and software frameworks, and its interpretability. △ Less

Submitted 29 December, 2023; v1 submitted 9 December, 2022; originally announced December 2022.

Comments: 34 pages, 9 figures, 10 tables

Journal ref: Mach. Learn.: Sci. Technol. 4 (2023) 045062

A Sample of Dust Attenuation Laws for DES Supernova Host Galaxies

Authors: J. Duarte, S. González-Gaitán, A. Mourao, A. Paulino-Afonso, P. Guilherme-Garcia, J. Aguas, L. Galbany, L. Kelsey, D. Scolnic, M. Sullivan, D. Brout, A. Palmese, P. Wiseman, A. Pieres, A. A. Plazas Malagón, A. Carnero Rosell, C. To, D. Gruen, D. Bacon, D. Brooks, D. L. Burke, D. W. Gerdes, D. J. James, D. L. Hollowood, D. Friedel , et al. (36 additional authors not shown)

Abstract: Type Ia supernovae (SNe Ia) are useful distance indicators in cosmology, provided their luminosity is standardized by applying empirical corrections based on light-curve properties. One factor behind these corrections is dust extinction, accounted for in the color-luminosity relation of the standardization. This relation is usually assumed to be universal, which could potentially introduce systema… ▽ More Type Ia supernovae (SNe Ia) are useful distance indicators in cosmology, provided their luminosity is standardized by applying empirical corrections based on light-curve properties. One factor behind these corrections is dust extinction, accounted for in the color-luminosity relation of the standardization. This relation is usually assumed to be universal, which could potentially introduce systematics into the standardization. The ``mass-step'' observed for SNe Ia Hubble residuals has been suggested as one such systematic. We seek to obtain a completer view of dust attenuation properties for a sample of 162 SN Ia host galaxies and to probe their link to the ``mass-step''. We infer attenuation laws towards hosts from both global and local (4 kpc) Dark Energy Survey photometry and Composite Stellar Population model fits. We recover a optical depth/attenuation slope relation, best explained by differing star/dust geometry for different galaxy orientations, which is significantly different from the optical depth/extinction slope relation observed directly for SNe. We obtain a large variation of attenuation slopes and confirm these change with host properties, like stellar mass and age, meaning a universal SN Ia correction should ideally not be assumed. Analyzing the cosmological standardization, we find evidence for a ``mass-step'' and a two dimensional ``dust-step'', both more pronounced for red SNe. Although comparable, the two steps are found no to be completely analogous. We conclude that host galaxy dust data cannot fully account for the ``mass-step'', using either an alternative SN standardization with extinction proxied by host attenuation or a ``dust-step'' approach. △ Less

Submitted 19 December, 2023; v1 submitted 25 November, 2022; originally announced November 2022.

Comments: 20 pages, 10 figues, 9 tables. Supplementary material included (10 pages). Accepted for publication on A&A

Report number: DES-2022-0694; FERMILAB-PUB-22-760-PPD

Journal ref: A&A 680, A56 (2023)

Evaluating generative models in high energy physics

Authors: Raghav Kansal, Anni Li, Javier Duarte, Nadezda Chernyavskaya, Maurizio Pierini, Breno Orzari, Thiago Tomei

Abstract: There has been a recent explosion in research into machine-learning-based generative modeling to tackle computational challenges for simulations in high energy physics (HEP). In order to use such alternative simulators in practice, we need well-defined metrics to compare different generative models and evaluate their discrepancy from the true distributions. We present the first systematic review a… ▽ More There has been a recent explosion in research into machine-learning-based generative modeling to tackle computational challenges for simulations in high energy physics (HEP). In order to use such alternative simulators in practice, we need well-defined metrics to compare different generative models and evaluate their discrepancy from the true distributions. We present the first systematic review and investigation into evaluation metrics and their sensitivity to failure modes of generative models, using the framework of two-sample goodness-of-fit testing, and their relevance and viability for HEP. Inspired by previous work in both physics and computer vision, we propose two new metrics, the Fréchet and kernel physics distances (FPD and KPD, respectively), and perform a variety of experiments measuring their performance on simple Gaussian-distributed, and simulated high energy jet datasets. We find FPD, in particular, to be the most sensitive metric to all alternative jet distributions tested and recommend its adoption, along with the KPD and Wasserstein distances between individual feature distributions, for evaluating generative models in HEP. We finally demonstrate the efficacy of these proposed metrics in evaluating and comparing a novel attention-based generative adversarial particle transformer to the state-of-the-art message-passing generative adversarial network jet simulation model. The code for our proposed metrics is provided in the open source JetNet Python library. △ Less

Submitted 21 April, 2023; v1 submitted 18 November, 2022; originally announced November 2022.

Comments: 11 pages, 5 figures, 3 tables, and a 5 page appendix

Report number: FERMILAB-PUB-22-872-CMS-PPD

Journal ref: Phys. Rev. D 107, 076017 (2023)

Do graph neural networks learn traditional jet substructure?

Authors: Farouk Mokhtar, Raghav Kansal, Javier Duarte

Abstract: At the CERN LHC, the task of jet tagging, whose goal is to infer the origin of a jet given a set of final-state particles, is dominated by machine learning methods. Graph neural networks have been used to address this task by treating jets as point clouds with underlying, learnable, edge connections between the particles inside. We explore the decision-making process for one such state-of-the-art… ▽ More At the CERN LHC, the task of jet tagging, whose goal is to infer the origin of a jet given a set of final-state particles, is dominated by machine learning methods. Graph neural networks have been used to address this task by treating jets as point clouds with underlying, learnable, edge connections between the particles inside. We explore the decision-making process for one such state-of-the-art network, ParticleNet, by looking for relevant edge connections identified using the layerwise-relevance propagation technique. As the model is trained, we observe changes in the distribution of relevant edges connecting different intermediate clusters of particles, known as subjets. The resulting distribution of subjet connections is different for signal jets originating from top quarks, whose subjets typically correspond to its three decay products, and background jets originating from lighter quarks and gluons. This behavior indicates that the model is using traditional jet substructure observables, such as the number of prongs -- energetic particle clusters -- within a jet, when identifying jets. △ Less

Submitted 17 November, 2022; originally announced November 2022.

Comments: 5 pages, 4 figures. Accepted to Machine Learning for Physical Sciences NeurIPS 2022 workshop

FAIR for AI: An interdisciplinary and international community building perspective

Authors: E. A. Huerta, Ben Blaiszik, L. Catherine Brinson, Kristofer E. Bouchard, Daniel Diaz, Caterina Doglioni, Javier M. Duarte, Murali Emani, Ian Foster, Geoffrey Fox, Philip Harris, Lukas Heinrich, Shantenu Jha, Daniel S. Katz, Volodymyr Kindratenko, Christine R. Kirkpatrick, Kati Lassila-Perini, Ravi K. Madduri, Mark S. Neubauer, Fotis E. Psomopoulos, Avik Roy, Oliver Rübel, Zhizhen Zhao, Ruike Zhu

Abstract: A foundational set of findable, accessible, interoperable, and reusable (FAIR) principles were proposed in 2016 as prerequisites for proper data management and stewardship, with the goal of enabling the reusability of scholarly data. The principles were also meant to apply to other digital assets, at a high level, and over time, the FAIR guiding principles have been re-interpreted or extended to i… ▽ More A foundational set of findable, accessible, interoperable, and reusable (FAIR) principles were proposed in 2016 as prerequisites for proper data management and stewardship, with the goal of enabling the reusability of scholarly data. The principles were also meant to apply to other digital assets, at a high level, and over time, the FAIR guiding principles have been re-interpreted or extended to include the software, tools, algorithms, and workflows that produce data. FAIR principles are now being adapted in the context of AI models and datasets. Here, we present the perspectives, vision, and experiences of researchers from different countries, disciplines, and backgrounds who are leading the definition and adoption of FAIR principles in their communities of practice, and discuss outcomes that may result from pursuing and incentivizing FAIR AI research. The material for this report builds on the FAIR for AI Workshop held at Argonne National Laboratory on June 7, 2022. △ Less

Submitted 1 August, 2023; v1 submitted 30 September, 2022; originally announced October 2022.

Comments: 10 pages, comments welcome!; v2: 12 pages, accepted to Scientific Data

ACM Class: I.2.0; E.0

Journal ref: Scientific Data 10, 487 (2023)

FastStamp: Accelerating Neural Steganography and Digital Watermarking of Images on FPGAs

Authors: Shehzeen Hussain, Nojan Sheybani, Paarth Neekhara, Xinqiao Zhang, Javier Duarte, Farinaz Koushanfar

Abstract: Steganography and digital watermarking are the tasks of hiding recoverable data in image pixels. Deep neural network (DNN) based image steganography and watermarking techniques are quickly replacing traditional hand-engineered pipelines. DNN based watermarking techniques have drastically improved the message capacity, imperceptibility and robustness of the embedded watermarks. However, this improv… ▽ More Steganography and digital watermarking are the tasks of hiding recoverable data in image pixels. Deep neural network (DNN) based image steganography and watermarking techniques are quickly replacing traditional hand-engineered pipelines. DNN based watermarking techniques have drastically improved the message capacity, imperceptibility and robustness of the embedded watermarks. However, this improvement comes at the cost of increased computational overhead of the watermark encoder neural network. In this work, we design the first accelerator platform FastStamp to perform DNN based steganography and digital watermarking of images on hardware. We first propose a parameter efficient DNN model for embedding recoverable bit-strings in image pixels. Our proposed model can match the success metrics of prior state-of-the-art DNN based watermarking methods while being significantly faster and lighter in terms of memory footprint. We then design an FPGA based accelerator framework to further improve the model throughput and power consumption by leveraging data parallelism and customized computation paths. FastStamp allows embedding hardware signatures into images to establish media authenticity and ownership of digital media. Our best design achieves 68 times faster inference as compared to GPU implementations of prior DNN based watermark encoder while consuming less power. △ Less

Submitted 25 September, 2022; originally announced September 2022.

Comments: Accepted at ICCAD 2022

Snowmass 2021 Computational Frontier CompF4 Topical Group Report: Storage and Processing Resource Access

Authors: W. Bhimji, D. Carder, E. Dart, J. Duarte, I. Fisk, R. Gardner, C. Guok, B. Jayatilaka, T. Lehman, M. Lin, C. Maltzahn, S. McKee, M. S. Neubauer, O. Rind, O. Shadura, N. V. Tran, P. van Gemmeren, G. Watts, B. A. Weaver, F. Würthwein

Abstract: Computing plays a significant role in all areas of high energy physics. The Snowmass 2021 CompF4 topical group's scope is facilities R&D, where we consider "facilities" as the computing hardware and software infrastructure inside the data centers plus the networking between data centers, irrespective of who owns them, and what policies are applied for using them. In other words, it includes commer… ▽ More Computing plays a significant role in all areas of high energy physics. The Snowmass 2021 CompF4 topical group's scope is facilities R&D, where we consider "facilities" as the computing hardware and software infrastructure inside the data centers plus the networking between data centers, irrespective of who owns them, and what policies are applied for using them. In other words, it includes commercial clouds, federally funded High Performance Computing (HPC) systems for all of science, and systems funded explicitly for a given experimental or theoretical program. This topical group report summarizes the findings and recommendations for the storage, processing, networking and associated software service infrastructures for future high energy physics research, based on the discussions organized through the Snowmass 2021 community study. △ Less

Submitted 29 September, 2022; v1 submitted 19 September, 2022; originally announced September 2022.

Comments: Snowmass 2021 Computational Frontier CompF4 topical group report. v2: Expanded introduction. Updated author list. 52 pages, 6 figures

Report of the Topical Group on Higgs Physics for Snowmass 2021: The Case for Precision Higgs Physics

Authors: Sally Dawson, Patrick Meade, Isobel Ojalvo, Caterina Vernieri, S. Adhikari, F. Abu-Ajamieh, A. Alberta, H. Bahl, R. Barman, M. Basso, A. Beniwal, I. Bozovi-Jelisav, S. Bright-Thonney, V. Cairo, F. Celiberto, S. Chang, M. Chen, C. Damerell, J. Davis, J. de Blas, W. Dekens, J. Duarte, D. Egana-Ugrinovic, U. Einhaus, Y. Gao , et al. (56 additional authors not shown)

Abstract: A future Higgs Factory will provide improved precision on measurements of Higgs couplings beyond those obtained by the LHC, and will enable a broad range of investigations across the fields of fundamental physics, including the mechanism of electroweak symmetry breaking, the origin of the masses and mixing of fundamental particles, the predominance of matter over antimatter, and the nature of dark… ▽ More A future Higgs Factory will provide improved precision on measurements of Higgs couplings beyond those obtained by the LHC, and will enable a broad range of investigations across the fields of fundamental physics, including the mechanism of electroweak symmetry breaking, the origin of the masses and mixing of fundamental particles, the predominance of matter over antimatter, and the nature of dark matter. Future colliders will measure Higgs couplings to a few per cent, giving a window to beyond the Standard Model (BSM) physics in the 1-10 TeV range. In addition, they will make precise measurements of the Higgs width, and characterize the Higgs self-coupling. This report details the work of the EF01 and EF02 working groups for the Snowmass 2021 study. △ Less

Submitted 20 December, 2022; v1 submitted 15 September, 2022; originally announced September 2022.

Comments: 44 pages, 40 figures, Report of the Topical Group on Higgs Physics for Snowmass 2021. The first four authors are the Conveners, with Contributions from the other authors

Muon Collider Forum Report

Authors: K. M. Black, S. **dariani, D. Li, F. Maltoni, P. Meade, D. Stratakis, D. Acosta, R. Agarwal, K. Agashe, C. Aime, D. Ally, A. Apresyan, A. Apyan, P. Asadi, D. Athanasakos, Y. Bao, E. Barzi, N. Bartosik, L. A. T. Bauerdick, J. Beacham, S. Belomestnykh, J. S. Berg, J. Berryhill, A. Bertolin, P. C. Bhat , et al. (160 additional authors not shown)

Abstract: A multi-TeV muon collider offers a spectacular opportunity in the direct exploration of the energy frontier. Offering a combination of unprecedented energy collisions in a comparatively clean leptonic environment, a high energy muon collider has the unique potential to provide both precision measurements and the highest energy reach in one machine that cannot be paralleled by any currently availab… ▽ More A multi-TeV muon collider offers a spectacular opportunity in the direct exploration of the energy frontier. Offering a combination of unprecedented energy collisions in a comparatively clean leptonic environment, a high energy muon collider has the unique potential to provide both precision measurements and the highest energy reach in one machine that cannot be paralleled by any currently available technology. The topic generated a lot of excitement in Snowmass meetings and continues to attract a large number of supporters, including many from the early career community. In light of this very strong interest within the US particle physics community, Snowmass Energy, Theory and Accelerator Frontiers created a cross-frontier Muon Collider Forum in November of 2020. The Forum has been meeting on a monthly basis and organized several topical workshops dedicated to physics, accelerator technology, and detector R&D. Findings of the Forum are summarized in this report. △ Less

Submitted 8 August, 2023; v1 submitted 2 September, 2022; originally announced September 2022.

End-to-end Graph-constrained Vectorized Floorplan Generation with Panoptic Refinement

Authors: Jiachen Liu, Yuan Xue, Jose Duarte, Krishnendra Shekhawat, Zihan Zhou, Xiaolei Huang

Abstract: The automatic generation of floorplans given user inputs has great potential in architectural design and has recently been explored in the computer vision community. However, the majority of existing methods synthesize floorplans in the format of rasterized images, which are difficult to edit or customize. In this paper, we aim to synthesize floorplans as sequences of 1-D vectors, which eases user… ▽ More The automatic generation of floorplans given user inputs has great potential in architectural design and has recently been explored in the computer vision community. However, the majority of existing methods synthesize floorplans in the format of rasterized images, which are difficult to edit or customize. In this paper, we aim to synthesize floorplans as sequences of 1-D vectors, which eases user interaction and design customization. To generate high fidelity vectorized floorplans, we propose a novel two-stage framework, including a draft stage and a multi-round refining stage. In the first stage, we encode the room connectivity graph input by users with a graph convolutional network (GCN), then apply an autoregressive transformer network to generate an initial floorplan sequence. To polish the initial design and generate more visually appealing floorplans, we further propose a novel panoptic refinement network(PRN) composed of a GCN and a transformer network. The PRN takes the initial generated sequence as input and refines the floorplan design while encouraging the correct room connectivity with our proposed geometric loss. We have conducted extensive experiments on a real-world floorplan dataset, and the results show that our method achieves state-of-the-art performance under different settings and evaluation metrics. △ Less

Submitted 26 July, 2022; originally announced July 2022.

Comments: ECCV 2022

Data Science and Machine Learning in Education

Authors: Gabriele Benelli, Thomas Y. Chen, Javier Duarte, Matthew Feickert, Matthew Graham, Lindsey Gray, Dan Hackett, Phil Harris, Shih-Chieh Hsu, Gregor Kasieczka, Elham E. Khoda, Matthias Komm, Mia Liu, Mark S. Neubauer, Scarlet Norberg, Alexx Perloff, Marcel Rieger, Claire Savard, Kazuhiro Terao, Savannah Thais, Avik Roy, Jean-Roch Vlimant, Grigorios Chachamis

Abstract: The growing role of data science (DS) and machine learning (ML) in high-energy physics (HEP) is well established and pertinent given the complex detectors, large data, sets and sophisticated analyses at the heart of HEP research. Moreover, exploiting symmetries inherent in physics data have inspired physics-informed ML as a vibrant sub-field of computer science research. HEP researchers benefit gr… ▽ More The growing role of data science (DS) and machine learning (ML) in high-energy physics (HEP) is well established and pertinent given the complex detectors, large data, sets and sophisticated analyses at the heart of HEP research. Moreover, exploiting symmetries inherent in physics data have inspired physics-informed ML as a vibrant sub-field of computer science research. HEP researchers benefit greatly from materials widely available materials for use in education, training and workforce development. They are also contributing to these materials and providing software to DS/ML-related fields. Increasingly, physics departments are offering courses at the intersection of DS, ML and physics, often using curricula developed by HEP researchers and involving open software and data used in HEP. In this white paper, we explore synergies between HEP research and DS/ML education, discuss opportunities and challenges at this intersection, and propose community activities that will be mutually beneficial. △ Less

Submitted 19 July, 2022; originally announced July 2022.

Comments: Contribution to Snowmass 2021

FastML Science Benchmarks: Accelerating Real-Time Scientific Edge Machine Learning

Authors: Javier Duarte, Nhan Tran, Ben Hawks, Christian Herwig, Jules Muhizi, Shvetank Prakash, Vijay Janapa Reddi

Abstract: Applications of machine learning (ML) are growing by the day for many unique and challenging scientific applications. However, a crucial challenge facing these applications is their need for ultra low-latency and on-detector ML capabilities. Given the slowdown in Moore's law and Dennard scaling, coupled with the rapid advances in scientific instrumentation that is resulting in growing data rates,… ▽ More Applications of machine learning (ML) are growing by the day for many unique and challenging scientific applications. However, a crucial challenge facing these applications is their need for ultra low-latency and on-detector ML capabilities. Given the slowdown in Moore's law and Dennard scaling, coupled with the rapid advances in scientific instrumentation that is resulting in growing data rates, there is a need for ultra-fast ML at the extreme edge. Fast ML at the edge is essential for reducing and filtering scientific data in real-time to accelerate science experimentation and enable more profound insights. To accelerate real-time scientific edge ML hardware and software solutions, we need well-constrained benchmark tasks with enough specifications to be generically applicable and accessible. These benchmarks can guide the design of future edge ML hardware for scientific applications capable of meeting the nanosecond and microsecond level latency requirements. To this end, we present an initial set of scientific ML benchmarks, covering a variety of ML and embedded system techniques. △ Less

Submitted 16 July, 2022; originally announced July 2022.

Comments: 9 pages, 4 figures, Contribution to 3rd Workshop on Benchmarking Machine Learning Workloads on Emerging Hardware (MLBench) at 5th Conference on Machine Learning and Systems (MLSys)

Report number: FERMILAB-CONF-22-534-PPD-SCD

Open-source FPGA-ML codesign for the MLPerf Tiny Benchmark

Authors: Hendrik Borras, Giuseppe Di Guglielmo, Javier Duarte, Nicolò Ghielmetti, Ben Hawks, Scott Hauck, Shih-Chieh Hsu, Ryan Kastner, Jason Liang, Andres Meza, Jules Muhizi, Tai Nguyen, Rushil Roy, Nhan Tran, Yaman Umuroglu, Olivia Weng, Aidan Yokuda, Michaela Blott

Abstract: We present our development experience and recent results for the MLPerf Tiny Inference Benchmark on field-programmable gate array (FPGA) platforms. We use the open-source hls4ml and FINN workflows, which aim to democratize AI-hardware codesign of optimized neural networks on FPGAs. We present the design and implementation process for the keyword spotting, anomaly detection, and image classificatio… ▽ More We present our development experience and recent results for the MLPerf Tiny Inference Benchmark on field-programmable gate array (FPGA) platforms. We use the open-source hls4ml and FINN workflows, which aim to democratize AI-hardware codesign of optimized neural networks on FPGAs. We present the design and implementation process for the keyword spotting, anomaly detection, and image classification benchmark tasks. The resulting hardware implementations are quantized, configurable, spatial dataflow architectures tailored for speed and efficiency and introduce new generic optimizations and common workflows developed as a part of this work. The full workflow is presented from quantization-aware training to FPGA implementation. The solutions are deployed on system-on-chip (Pynq-Z2) and pure FPGA (Arty A7-100T) platforms. The resulting submissions achieve latencies as low as 20 $μ$s and energy consumption as low as 30 $μ$J per inference. We demonstrate how emerging ML benchmarks on heterogeneous hardware platforms can catalyze collaboration and the development of new techniques and more accessible tools. △ Less

Submitted 23 June, 2022; originally announced June 2022.

Comments: 15 pages, 7 figures, Contribution to 3rd Workshop on Benchmarking Machine Learning Workloads on Emerging Hardware (MLBench) at 5th Conference on Machine Learning and Systems (MLSys)

Report number: FERMILAB-CONF-22-479-SCD

QONNX: Representing Arbitrary-Precision Quantized Neural Networks

Authors: Alessandro Pappalardo, Yaman Umuroglu, Michaela Blott, Jovan Mitrevski, Ben Hawks, Nhan Tran, Vladimir Loncar, Sioni Summers, Hendrik Borras, Jules Muhizi, Matthew Trahms, Shih-Chieh Hsu, Scott Hauck, Javier Duarte

Abstract: We present extensions to the Open Neural Network Exchange (ONNX) intermediate representation format to represent arbitrary-precision quantized neural networks. We first introduce support for low precision quantization in existing ONNX-based quantization formats by leveraging integer clip**, resulting in two new backward-compatible variants: the quantized operator format with clip** and quantiz… ▽ More We present extensions to the Open Neural Network Exchange (ONNX) intermediate representation format to represent arbitrary-precision quantized neural networks. We first introduce support for low precision quantization in existing ONNX-based quantization formats by leveraging integer clip**, resulting in two new backward-compatible variants: the quantized operator format with clip** and quantize-clip-dequantize (QCDQ) format. We then introduce a novel higher-level ONNX format called quantized ONNX (QONNX) that introduces three new operators -- Quant, BipolarQuant, and Trunc -- in order to represent uniform quantization. By kee** the QONNX IR high-level and flexible, we enable targeting a wider variety of platforms. We also present utilities for working with QONNX, as well as examples of its usage in the FINN and hls4ml toolchains. Finally, we introduce the QONNX model zoo to share low-precision quantized neural networks. △ Less

Submitted 24 June, 2022; v1 submitted 15 June, 2022; originally announced June 2022.

Comments: 9 pages, 5 figures, Contribution to 4th Workshop on Accelerated Machine Learning (AccML) at HiPEAC 2022 Conference

Report number: FERMILAB-CONF-22-471-SCD

Physics Community Needs, Tools, and Resources for Machine Learning

Authors: Philip Harris, Erik Katsavounidis, William Patrick McCormack, Dylan Rankin, Yongbin Feng, Abhijith Gandrakota, Christian Herwig, Burt Holzman, Kevin Pedro, Nhan Tran, Tingjun Yang, Jennifer Ngadiuba, Michael Coughlin, Scott Hauck, Shih-Chieh Hsu, Elham E Khoda, Deming Chen, Mark Neubauer, Javier Duarte, Georgia Karagiorgi, Mia Liu

Abstract: Machine learning (ML) is becoming an increasingly important component of cutting-edge physics research, but its computational requirements present significant challenges. In this white paper, we discuss the needs of the physics community regarding ML across latency and throughput regimes, the tools and resources that offer the possibility of addressing these needs, and how these can be best utiliz… ▽ More Machine learning (ML) is becoming an increasingly important component of cutting-edge physics research, but its computational requirements present significant challenges. In this white paper, we discuss the needs of the physics community regarding ML across latency and throughput regimes, the tools and resources that offer the possibility of addressing these needs, and how these can be best utilized and accessed in the coming years. △ Less

Submitted 30 March, 2022; originally announced March 2022.

Comments: Contribution to Snowmass 2021, 33 pages, 5 figures

Graph Neural Networks in Particle Physics: Implementations, Innovations, and Challenges

Authors: Savannah Thais, Paolo Calafiura, Grigorios Chachamis, Gage DeZoort, Javier Duarte, Sanmay Ganguly, Michael Kagan, Daniel Murnane, Mark S. Neubauer, Kazuhiro Terao

Abstract: Many physical systems can be best understood as sets of discrete data with associated relationships. Where previously these sets of data have been formulated as series or image data to match the available machine learning architectures, with the advent of graph neural networks (GNNs), these systems can be learned natively as graphs. This allows a wide variety of high- and low-level physical featur… ▽ More Many physical systems can be best understood as sets of discrete data with associated relationships. Where previously these sets of data have been formulated as series or image data to match the available machine learning architectures, with the advent of graph neural networks (GNNs), these systems can be learned natively as graphs. This allows a wide variety of high- and low-level physical features to be attached to measurements and, by the same token, a wide variety of HEP tasks to be accomplished by the same GNN architectures. GNNs have found powerful use-cases in reconstruction, tagging, generation and end-to-end analysis. With the wide-spread adoption of GNNs in industry, the HEP community is well-placed to benefit from rapid improvements in GNN latency and memory usage. However, industry use-cases are not perfectly aligned with HEP and much work needs to be done to best match unique GNN capabilities to unique HEP obstacles. We present here a range of these capabilities, predictions of which are currently being well-adopted in HEP communities, and which are still immature. We hope to capture the landscape of graph techniques in machine learning as well as point out the most significant gaps that are inhibiting potentially large leaps in research. △ Less

Submitted 25 March, 2022; v1 submitted 23 March, 2022; originally announced March 2022.

Comments: contribution to Snowmass 2021

Improving Di-Higgs Sensitivity at Future Colliders in Hadronic Final States with Machine Learning

Authors: Artur Apresyan, Daniel Diaz, Javier Duarte, Sanmay Ganguly, Raghav Kansal, Nan Lu, Cristina Mantilla Suarez, Samadrita Mukherjee, Cristían Peña, Brian Sheldon, Si Xie

Abstract: One of the central goals of the physics program at the future colliders is to elucidate the origin of electroweak symmetry breaking, including precision measurements of the Higgs sector. This includes a detailed study of Higgs boson (H) pair production, which can reveal the H self-coupling. Since the discovery of the Higgs boson, a large campaign of measurements of the properties of the Higgs boso… ▽ More One of the central goals of the physics program at the future colliders is to elucidate the origin of electroweak symmetry breaking, including precision measurements of the Higgs sector. This includes a detailed study of Higgs boson (H) pair production, which can reveal the H self-coupling. Since the discovery of the Higgs boson, a large campaign of measurements of the properties of the Higgs boson has begun and many new ideas have emerged during the completion of this program. One such idea is the use of highly boosted and merged hadronic decays of the Higgs boson ($\mathrm{H}\to\mathrm{b}\bar{\mathrm{b}}$, $\mathrm{H}\to\mathrm{W}\mathrm{W}\to\mathrm{q}\bar{\mathrm{q}}\mathrm{q}\bar{\mathrm{q}}$) with machine learning methods to improve the signal-to-background discrimination. In this white paper, we champion the use of these modes to boost the sensitivity of future collider physics programs to Higgs boson pair production, the Higgs self-coupling, and Higgs-vector boson couplings. We demonstrate the potential improvement possible at the Future Circular Collider in hadron mode, especially with the use of graph neural networks. △ Less

Submitted 4 April, 2022; v1 submitted 14 March, 2022; originally announced March 2022.

Comments: Contribution to Snowmass 2022 Summer Study

Report number: FERMILAB-CONF-22-316-PPD-QIS

Particle-based Fast Jet Simulation at the LHC with Variational Autoencoders

Authors: Mary Touranakou, Nadezda Chernyavskaya, Javier Duarte, Dimitrios Gunopulos, Raghav Kansal, Breno Orzari, Maurizio Pierini, Thiago Tomei, Jean-Roch Vlimant

Abstract: We study how to use Deep Variational Autoencoders for a fast simulation of jets of particles at the LHC. We represent jets as a list of constituents, characterized by their momenta. Starting from a simulation of the jet before detector effects, we train a Deep Variational Autoencoder to return the corresponding list of constituents after detection. Doing so, we bypass both the time-consuming detec… ▽ More We study how to use Deep Variational Autoencoders for a fast simulation of jets of particles at the LHC. We represent jets as a list of constituents, characterized by their momenta. Starting from a simulation of the jet before detector effects, we train a Deep Variational Autoencoder to return the corresponding list of constituents after detection. Doing so, we bypass both the time-consuming detector simulation and the collision reconstruction steps of a traditional processing chain, speeding up significantly the events generation workflow. Through model optimization and hyperparameter tuning, we achieve state-of-the-art precision on the jet four-momentum, while providing an accurate description of the constituents momenta, and an inference time comparable to that of a rule-based fast simulation. △ Less

Submitted 1 March, 2022; originally announced March 2022.

Comments: 11 pages, 8 figures

Journal ref: Mach. Learn.: Sci. Technol. 3, 035003 (2022)

Machine Learning for Particle Flow Reconstruction at CMS

Authors: Joosep Pata, Javier Duarte, Farouk Mokhtar, Eric Wulff, Jieun Yoo, Jean-Roch Vlimant, Maurizio Pierini, Maria Girone

Abstract: We provide details on the implementation of a machine-learning based particle flow algorithm for CMS. The standard particle flow algorithm reconstructs stable particles based on calorimeter clusters and tracks to provide a global event reconstruction that exploits the combined information of multiple detector subsystems, leading to strong improvements for quantities such as jets and missing transv… ▽ More We provide details on the implementation of a machine-learning based particle flow algorithm for CMS. The standard particle flow algorithm reconstructs stable particles based on calorimeter clusters and tracks to provide a global event reconstruction that exploits the combined information of multiple detector subsystems, leading to strong improvements for quantities such as jets and missing transverse energy. We have studied a possible evolution of particle flow towards heterogeneous computing platforms such as GPUs using a graph neural network. The machine-learned PF model reconstructs particle candidates based on the full list of tracks and calorimeter clusters in the event. For validation, we determine the physics performance directly in the CMS software framework when the proposed algorithm is interfaced with the offline reconstruction of jets and missing transverse energy. We also report the computational performance of the algorithm, which scales approximately linearly in runtime and memory usage with the input size. △ Less

Submitted 1 March, 2022; originally announced March 2022.

Comments: 12 pages, 6 figures. Presented at the ACAT 2021: 20th International Workshop on Advanced Computing and Analysis Techniques in Physics Research, Daejeon, Kr, 29 Nov - 3 Dec 2021

Journal ref: J. Phys.: Conf. Ser. 2438, 012100 (2023)

Graph Neural Networks for Charged Particle Tracking on FPGAs

Authors: Abdelrahman Elabd, Vesal Razavimaleki, Shi-Yu Huang, Javier Duarte, Markus Atkinson, Gage DeZoort, Peter Elmer, Scott Hauck, **-Xuan Hu, Shih-Chieh Hsu, Bo-Cheng Lai, Mark Neubauer, Isobel Ojalvo, Savannah Thais, Matthew Trahms

Abstract: The determination of charged particle trajectories in collisions at the CERN Large Hadron Collider (LHC) is an important but challenging problem, especially in the high interaction density conditions expected during the future high-luminosity phase of the LHC (HL-LHC). Graph neural networks (GNNs) are a type of geometric deep learning algorithm that has successfully been applied to this task by em… ▽ More The determination of charged particle trajectories in collisions at the CERN Large Hadron Collider (LHC) is an important but challenging problem, especially in the high interaction density conditions expected during the future high-luminosity phase of the LHC (HL-LHC). Graph neural networks (GNNs) are a type of geometric deep learning algorithm that has successfully been applied to this task by embedding tracker data as a graph -- nodes represent hits, while edges represent possible track segments -- and classifying the edges as true or fake track segments. However, their study in hardware- or software-based trigger applications has been limited due to their large computational cost. In this paper, we introduce an automated translation workflow, integrated into a broader tool called $\texttt{hls4ml}$, for converting GNNs into firmware for field-programmable gate arrays (FPGAs). We use this translation tool to implement GNNs for charged particle tracking, trained using the TrackML challenge dataset, on FPGAs with designs targeting different graph sizes, task complexites, and latency/throughput requirements. This work could enable the inclusion of charged particle tracking GNNs at the trigger level for HL-LHC experiments. △ Less

Submitted 23 March, 2022; v1 submitted 3 December, 2021; originally announced December 2021.

Comments: 28 pages, 17 figures, 1 table, published version

Journal ref: Front. Big Data 5 (2022) 828666

Particle Graph Autoencoders and Differentiable, Learned Energy Mover's Distance

Authors: Steven Tsan, Raghav Kansal, Anthony Aportela, Daniel Diaz, Javier Duarte, Sukanya Krishna, Farouk Mokhtar, Jean-Roch Vlimant, Maurizio Pierini

Abstract: Autoencoders have useful applications in high energy physics in anomaly detection, particularly for jets - collimated showers of particles produced in collisions such as those at the CERN Large Hadron Collider. We explore the use of graph-based autoencoders, which operate on jets in their "particle cloud" representations and can leverage the interdependencies among the particles within a jet, for… ▽ More Autoencoders have useful applications in high energy physics in anomaly detection, particularly for jets - collimated showers of particles produced in collisions such as those at the CERN Large Hadron Collider. We explore the use of graph-based autoencoders, which operate on jets in their "particle cloud" representations and can leverage the interdependencies among the particles within a jet, for such tasks. Additionally, we develop a differentiable approximation to the energy mover's distance via a graph neural network, which may subsequently be used as a reconstruction loss function for autoencoders. △ Less

Submitted 24 November, 2021; originally announced November 2021.

Comments: 5 pages, 2 figures. Accepted to the Machine Learning for the Physical Sciences workshop at NeurIPS 2021. arXiv admin note: text overlap with arXiv:2101.08320

Explaining machine-learned particle-flow reconstruction

Authors: Farouk Mokhtar, Raghav Kansal, Daniel Diaz, Javier Duarte, Joosep Pata, Maurizio Pierini, Jean-Roch Vlimant

Abstract: The particle-flow (PF) algorithm is used in general-purpose particle detectors to reconstruct a comprehensive particle-level view of the collision by combining information from different subdetectors. A graph neural network (GNN) model, known as the machine-learned particle-flow (MLPF) algorithm, has been developed to substitute the rule-based PF algorithm. However, understanding the model's decis… ▽ More The particle-flow (PF) algorithm is used in general-purpose particle detectors to reconstruct a comprehensive particle-level view of the collision by combining information from different subdetectors. A graph neural network (GNN) model, known as the machine-learned particle-flow (MLPF) algorithm, has been developed to substitute the rule-based PF algorithm. However, understanding the model's decision making is not straightforward, especially given the complexity of the set-to-set prediction task, dynamic graph building, and message-passing steps. In this paper, we adapt the layerwise-relevance propagation technique for GNNs and apply it to the MLPF algorithm to gauge the relevant nodes and features for its predictions. Through this process, we gain insight into the model's decision-making. △ Less

Submitted 24 November, 2021; originally announced November 2021.

Comments: 5 pages, 3 figures. Accepted to Machine Learning for Physical Sciences NeurIPS 2021 workshop

Applications and Techniques for Fast Machine Learning in Science

Authors: Allison McCarn Deiana, Nhan Tran, Joshua Agar, Michaela Blott, Giuseppe Di Guglielmo, Javier Duarte, Philip Harris, Scott Hauck, Mia Liu, Mark S. Neubauer, Jennifer Ngadiuba, Seda Ogrenci-Memik, Maurizio Pierini, Thea Aarrestad, Steffen Bahr, Jurgen Becker, Anne-Sophie Berthold, Richard J. Bonventre, Tomas E. Muller Bravo, Markus Diefenthaler, Zhen Dong, Nick Fritzsche, Amir Gholami, Ekaterina Govorkova, Kyle J Hazelwood , et al. (62 additional authors not shown)

Abstract: In this community review report, we discuss applications and techniques for fast machine learning (ML) in science -- the concept of integrating power ML methods into the real-time experimental data processing loop to accelerate scientific discovery. The material for the report builds on two workshops held by the Fast ML for Science community and covers three main areas: applications for fast ML ac… ▽ More In this community review report, we discuss applications and techniques for fast machine learning (ML) in science -- the concept of integrating power ML methods into the real-time experimental data processing loop to accelerate scientific discovery. The material for the report builds on two workshops held by the Fast ML for Science community and covers three main areas: applications for fast ML across a number of scientific domains; techniques for training and implementing performant and resource-efficient ML algorithms; and computing architectures, platforms, and technologies for deploying these algorithms. We also present overlap** challenges across the multiple scientific domains where common solutions can be found. This community report is intended to give plenty of examples and inspiration for scientific discovery through integrated and accelerated ML solutions. This is followed by a high-level overview and organization of technical advances, including an abundance of pointers to source material, which can enable these breakthroughs. △ Less

Submitted 25 October, 2021; originally announced October 2021.

Comments: 66 pages, 13 figures, 5 tables

Report number: FERMILAB-PUB-21-502-AD-E-SCD

Journal ref: Front. Big Data 5, 787421 (2022)

Improving Variational Autoencoders for New Physics Detection at the LHC with Normalizing Flows

Authors: Pratik Jawahar, Thea Aarrestad, Nadezda Chernyavskaya, Maurizio Pierini, Kinga A. Wozniak, Jennifer Ngadiuba, Javier Duarte, Steven Tsan

Abstract: We investigate how to improve new physics detection strategies exploiting variational autoencoders and normalizing flows for anomaly detection at the Large Hadron Collider. As a working example, we consider the DarkMachines challenge dataset. We show how different design choices (e.g., event representations, anomaly score definitions, network architectures) affect the result on specific benchmark… ▽ More We investigate how to improve new physics detection strategies exploiting variational autoencoders and normalizing flows for anomaly detection at the Large Hadron Collider. As a working example, we consider the DarkMachines challenge dataset. We show how different design choices (e.g., event representations, anomaly score definitions, network architectures) affect the result on specific benchmark new physics models. Once a baseline is established, we discuss how to improve the anomaly detection accuracy by exploiting normalizing flow layers in the latent space of the variational autoencoder. △ Less

Submitted 15 December, 2021; v1 submitted 16 October, 2021; originally announced October 2021.

Comments: 10 + 3 pages, 7 figures

Journal ref: Front. Big Data 5, 803685 (2022)

Building a Noisy Audio Dataset to Evaluate Machine Learning Approaches for Automatic Speech Recognition Systems

Authors: Julio Cesar Duarte, Sérgio Colcher

Abstract: Automatic speech recognition systems are part of people's daily lives, embedded in personal assistants and mobile phones, hel** as a facilitator for human-machine interaction while allowing access to information in a practically intuitive way. Such systems are usually implemented using machine learning techniques, especially with deep neural networks. Even with its high performance in the task o… ▽ More Automatic speech recognition systems are part of people's daily lives, embedded in personal assistants and mobile phones, hel** as a facilitator for human-machine interaction while allowing access to information in a practically intuitive way. Such systems are usually implemented using machine learning techniques, especially with deep neural networks. Even with its high performance in the task of transcribing text from speech, few works address the issue of its recognition in noisy environments and, usually, the datasets used do not contain noisy audio examples, while only mitigating this issue using data augmentation techniques. This work aims to present the process of building a dataset of noisy audios, in a specific case of degenerated audios due to interference, commonly present in radio transmissions. Additionally, we present initial results of a classifier that uses such data for evaluation, indicating the benefits of using this dataset in the recognizer's training process. Such recognizer achieves an average result of 0.4116 in terms of character error rate in the noisy set (SNR = 30). △ Less

Submitted 4 October, 2021; originally announced October 2021.

Comments: Tech report series Monografias em Ciência da Computação, september, 2021, Dep. Informática PUC-Rio, RJ, BRAZIL, ISSN 0103-9741

Report number: MCC no. 05/2021

Sparse Data Generation for Particle-Based Simulation of Hadronic Jets in the LHC

Authors: Breno Orzari, Thiago Tomei, Maurizio Pierini, Mary Touranakou, Javier Duarte, Raghav Kansal, Jean-Roch Vlimant, Dimitrios Gunopulos

Abstract: We develop a generative neural network for the generation of sparse data in particle physics using a permutation-invariant and physics-informed loss function. The input dataset used in this study consists of the particle constituents of hadronic jets due to its sparsity and the possibility of evaluating the network's ability to accurately describe the particles and jets properties. A variational a… ▽ More We develop a generative neural network for the generation of sparse data in particle physics using a permutation-invariant and physics-informed loss function. The input dataset used in this study consists of the particle constituents of hadronic jets due to its sparsity and the possibility of evaluating the network's ability to accurately describe the particles and jets properties. A variational autoencoder composed of convolutional layers in the encoder and decoder is used as the generator. The loss function consists of a reconstruction error term and the Kullback-Leibler divergence between the output of the encoder and the latent vector variables. The permutation-invariant loss on the particles' properties is combined with two mean-squared error terms that measure the difference between input and output jets mass and transverse momentum, which improves the network's generation capability as it imposes physics constraints, allowing the model to learn the kinematics of the jets. △ Less

Submitted 30 September, 2021; originally announced September 2021.

Comments: 4 pages, 2 figures, 1 table. Contribution to Proceedings of the LatinX in AI (LXAI) Research workshop at ICML 2021