-
Acceleration beyond lowest order event generation: An outlook on further parallelism within MadGraph5_aMC@NLO
Authors:
Zenny Wettersten,
Olivier Mattelaer,
Stefan Roiser,
Robert Schöfbeck,
Andrea Valassi
Abstract:
An important area of high energy physics studies at the Large Hadron Collider (LHC) currently concerns the need for more extensive and precise comparison data. Important tools in this realm are event reweighing and evaluation of more precise next-to-leading order (NLO) processes via Monte Carlo event generators, especially in the context of the upcoming High Luminosity LHC. Current event generator…
▽ More
An important area of high energy physics studies at the Large Hadron Collider (LHC) currently concerns the need for more extensive and precise comparison data. Important tools in this realm are event reweighing and evaluation of more precise next-to-leading order (NLO) processes via Monte Carlo event generators, especially in the context of the upcoming High Luminosity LHC. Current event generators need to improve throughputs for these studies. MadGraph5_aMC@NLO (MG5aMC) is an event generator being used by LHC experiments which has been accelerated considerably with a port to GPU and vector CPU architectures, but as of yet only for leading order processes. In this contribution a prototype for event reweighing using the accelerated MG5aMC software, as well as plans for an NLO implementation, are presented.
△ Less
Submitted 12 December, 2023;
originally announced December 2023.
-
Madgraph5_aMC@NLO on GPUs and vector CPUs Experience with the first alpha release
Authors:
Stephan Hageboeck,
Taylor Childers,
Walter Hopkins,
Olivier Mattelaer,
Nathan Nichols,
Stefan Roiser,
Jørgen Teig,
Andrea Valassi,
Carl Vuosalo,
Zenny Wettersten
Abstract:
Madgraph5_aMC@NLO is one of the most-frequently used Monte-Carlo event generators at the LHC, and an important consumer of compute resources. The software has been reengineered to maintain the overall look and feel of the user interface while speeding up event generation on CPUs and GPUs. The most computationally intensive part, the calculation of "matrix elements", is offloaded to new implementat…
▽ More
Madgraph5_aMC@NLO is one of the most-frequently used Monte-Carlo event generators at the LHC, and an important consumer of compute resources. The software has been reengineered to maintain the overall look and feel of the user interface while speeding up event generation on CPUs and GPUs. The most computationally intensive part, the calculation of "matrix elements", is offloaded to new implementations optimised for GPUs and for CPU vector instructions, using event-level data parallelism. We present the work to support accelerated leading-order QCD processes, and discuss how this work is going to be released to Madgraph5_aMC@NLO's users.
△ Less
Submitted 5 December, 2023;
originally announced December 2023.
-
Speeding up Madgraph5 aMC@NLO through CPU vectorization and GPU offloading: towards a first alpha release
Authors:
Andrea Valassi,
Taylor Childers,
Laurence Field,
Stephan Hageböck,
Walter Hopkins,
Olivier Mattelaer,
Nathan Nichols,
Stefan Roiser,
David Smith,
Jorgen Teig,
Carl Vuosalo,
Zenny Wettersten
Abstract:
The matrix element (ME) calculation in any Monte Carlo physics event generator is an ideal fit for implementing data parallelism with lockstep processing on GPUs and vector CPUs. For complex physics processes where the ME calculation is the computational bottleneck of event generation workflows, this can lead to large overall speedups by efficiently exploiting these hardware architectures, which a…
▽ More
The matrix element (ME) calculation in any Monte Carlo physics event generator is an ideal fit for implementing data parallelism with lockstep processing on GPUs and vector CPUs. For complex physics processes where the ME calculation is the computational bottleneck of event generation workflows, this can lead to large overall speedups by efficiently exploiting these hardware architectures, which are now largely underutilized in HEP. In this paper, we present the status of our work on the reengineering of the Madgraph5_aMC@NLO event generator at the time of the ACAT2022 conference. The progress achieved since our previous publication in the ICHEP2022 proceedings is discussed, for our implementations of the ME calculations in vectorized C++, in CUDA and in the SYCL framework, as well as in their integration into the existing MadEvent framework. The outlook towards a first alpha release of the software supporting QCD LO processes usable by the LHC experiments is also discussed.
△ Less
Submitted 9 December, 2023; v1 submitted 31 March, 2023;
originally announced March 2023.
-
Developments in Performance and Portability for MadGraph5_aMC@NLO
Authors:
Andrea Valassi,
Taylor Childers,
Laurence Field,
Stefan Hageböck,
Walter Hopkins,
Olivier Mattelaer,
Nathan Nichols,
Stefan Roiser,
David Smith
Abstract:
Event generators simulate particle interactions using Monte Carlo techniques, providing the primary connection between experiment and theory in experimental high energy physics. These software packages, which are the first step in the simulation worflow of collider experiments, represent approximately 5 to 20% of the annual WLCG usage for the ATLAS and CMS experiments. With computing architectures…
▽ More
Event generators simulate particle interactions using Monte Carlo techniques, providing the primary connection between experiment and theory in experimental high energy physics. These software packages, which are the first step in the simulation worflow of collider experiments, represent approximately 5 to 20% of the annual WLCG usage for the ATLAS and CMS experiments. With computing architectures becoming more heterogeneous, it is important to ensure that these key software frameworks can be run on future systems, large and small. In this contribution, recent progress on porting and speeding up the Madgraph5_aMC@NLO event generator on hybrid architectures, i.e. CPU with GPU accelerators, is discussed. The main focus of this work has been in the calculation of scattering amplitudes and "matrix elements", which is the computational bottleneck of an event generation application. For physics processes limited to QCD leading order, the code generation toolkit has been expanded to produce matrix element calculations using C++ vector instructions on CPUs and using CUDA for NVidia GPUs, as well as using Alpaka, Kokkos and SYCL for multiple CPU and GPU architectures. Performance is reported in terms of matrix element calculations per time on NVidia, Intel, and AMD devices. The status and outlook for the integration of this work into a production release usable by the LHC experiments, with the same functionalities and very similar user interfaces as the current Fortran version, is also described.
△ Less
Submitted 20 October, 2022;
originally announced October 2022.
-
HL-LHC Computing Review Stage-2, Common Software Projects: Event Generators
Authors:
The HSF Physics Event Generator WG,
:,
Efe Yazgan,
Josh McFayden,
Andrea Valassi,
Simone Amoroso,
Enrico Bothmann,
Andy Buckley,
John Campbell,
Gurpreet Singh Chahal,
Taylor Childers,
Gloria Corti,
Rikkert Frederix,
Stefano Frixione,
Francesco Giuli,
Alexander Grohsjean,
Stefan Hoeche,
Phil Ilten,
Frank Krauss,
Michal Kreps,
David Lange,
Leif Lonnblad,
Zach Marshall,
Olivier Mattelaer,
Stephen Mrenna
, et al. (14 additional authors not shown)
Abstract:
This paper has been prepared by the HEP Software Foundation (HSF) Physics Event Generator Working Group (WG), as an input to the second phase of the LHCC review of High-Luminosity LHC (HL-LHC) computing, which is due to take place in November 2021. It complements previous documents prepared by the WG in the context of the first phase of the LHCC review in 2020, including in particular the WG paper…
▽ More
This paper has been prepared by the HEP Software Foundation (HSF) Physics Event Generator Working Group (WG), as an input to the second phase of the LHCC review of High-Luminosity LHC (HL-LHC) computing, which is due to take place in November 2021. It complements previous documents prepared by the WG in the context of the first phase of the LHCC review in 2020, including in particular the WG paper on the specific challenges in Monte Carlo event generator software for HL-LHC, which has since been updated and published, and which we are also submitting to the November 2021 review as an integral part of our contribution.
△ Less
Submitted 30 September, 2021;
originally announced September 2021.
-
Design and engineering of a simplified workflow execution for the MG5aMC event generator on GPUs and vector CPUs
Authors:
Andrea Valassi,
Stefan Roiser,
Olivier Mattelaer,
Stephan Hageboeck
Abstract:
Physics event generators are essential components of the data analysis software chain of high energy physics experiments, and important consumers of their CPU resources. Improving the software performance of these packages on modern hardware architectures, such as those deployed at HPC centers, is essential in view of the upcoming HL-LHC physics programme. In this paper, we describe an ongoing act…
▽ More
Physics event generators are essential components of the data analysis software chain of high energy physics experiments, and important consumers of their CPU resources. Improving the software performance of these packages on modern hardware architectures, such as those deployed at HPC centers, is essential in view of the upcoming HL-LHC physics programme. In this paper, we describe an ongoing activity to reengineer the Madgraph5_aMC@NLO physics event generator, primarily to port it and allow its efficient execution on GPUs, but also to modernize it and optimize its performance on vector CPUs. We describe the motivation, engineering process and software architecture design of our developments, as well as the current challenges and future directions for this project. This paper is based on our submission to vCHEP2021 in March 2021,complemented with a few preliminary results that we presented during the conference. Further details and updated results will be given in later publications.
△ Less
Submitted 13 July, 2021; v1 submitted 23 June, 2021;
originally announced June 2021.
-
Software Training in HEP
Authors:
Sudhir Malik,
Samuel Meehan,
Kilian Lieret,
Meirin Oan Evans,
Michel H. Villanueva,
Daniel S. Katz,
Graeme A. Stewart,
Peter Elmer,
Sizar Aziz,
Matthew Bellis,
Riccardo Maria Bianchi,
Gianluca Bianco,
Johan Sebastian Bonilla,
Angela Burger,
Jackson Burzynski,
David Chamont,
Matthew Feickert,
Philipp Gadow,
Bernhard Manfred Gruber,
Daniel Guest,
Stephan Hageboeck,
Lukas Heinrich,
Maximilian M. Horzela,
Marc Huwiler,
Clemens Lange
, et al. (22 additional authors not shown)
Abstract:
Long term sustainability of the high energy physics (HEP) research software ecosystem is essential for the field. With upgrades and new facilities coming online throughout the 2020s this will only become increasingly relevant throughout this decade. Meeting this sustainability challenge requires a workforce with a combination of HEP domain knowledge and advanced software skills. The required softw…
▽ More
Long term sustainability of the high energy physics (HEP) research software ecosystem is essential for the field. With upgrades and new facilities coming online throughout the 2020s this will only become increasingly relevant throughout this decade. Meeting this sustainability challenge requires a workforce with a combination of HEP domain knowledge and advanced software skills. The required software skills fall into three broad groups. The first is fundamental and generic software engineering (e.g. Unix, version control,C++, continuous integration). The second is knowledge of domain specific HEP packages and practices (e.g., the ROOT data format and analysis framework). The third is more advanced knowledge involving more specialized techniques. These include parallel programming, machine learning and data science tools, and techniques to preserve software projects at all scales. This paper dis-cusses the collective software training program in HEP and its activities led by the HEP Software Foundation (HSF) and the Institute for Research and Innovation in Software in HEP (IRIS-HEP). The program equips participants with an array of software skills that serve as ingredients from which solutions to the computing challenges of HEP can be formed. Beyond serving the community by ensuring that members are able to pursue research goals, this program serves individuals by providing intellectual capital and transferable skills that are becoming increasingly important to careers in the realm of software and computing, whether inside or outside HEP
△ Less
Submitted 6 August, 2021; v1 submitted 28 February, 2021;
originally announced March 2021.
-
HL-LHC Computing Review: Common Tools and Community Software
Authors:
HEP Software Foundation,
:,
Thea Aarrestad,
Simone Amoroso,
Markus Julian Atkinson,
Joshua Bendavid,
Tommaso Boccali,
Andrea Bocci,
Andy Buckley,
Matteo Cacciari,
Paolo Calafiura,
Philippe Canal,
Federico Carminati,
Taylor Childers,
Vitaliano Ciulli,
Gloria Corti,
Davide Costanzo,
Justin Gage Dezoort,
Caterina Doglioni,
Javier Mauricio Duarte,
Agnieszka Dziurda,
Peter Elmer,
Markus Elsing,
V. Daniel Elvira,
Giulio Eulisse
, et al. (85 additional authors not shown)
Abstract:
Common and community software packages, such as ROOT, Geant4 and event generators have been a key part of the LHC's success so far and continued development and optimisation will be critical in the future. The challenges are driven by an ambitious physics programme, notably the LHC accelerator upgrade to high-luminosity, HL-LHC, and the corresponding detector upgrades of ATLAS and CMS. In this doc…
▽ More
Common and community software packages, such as ROOT, Geant4 and event generators have been a key part of the LHC's success so far and continued development and optimisation will be critical in the future. The challenges are driven by an ambitious physics programme, notably the LHC accelerator upgrade to high-luminosity, HL-LHC, and the corresponding detector upgrades of ATLAS and CMS. In this document we address the issues for software that is used in multiple experiments (usually even more widely than ATLAS and CMS) and maintained by teams of developers who are either not linked to a particular experiment or who contribute to common software within the context of their experiment activity. We also give space to general considerations for future software and projects that tackle upcoming challenges, no matter who writes it, which is an area where community convergence on best practice is extremely useful.
△ Less
Submitted 31 August, 2020;
originally announced August 2020.
-
Challenges in Monte Carlo event generator software for High-Luminosity LHC
Authors:
The HSF Physics Event Generator WG,
:,
Andrea Valassi,
Efe Yazgan,
Josh McFayden,
Simone Amoroso,
Joshua Bendavid,
Andy Buckley,
Matteo Cacciari,
Taylor Childers,
Vitaliano Ciulli,
Rikkert Frederix,
Stefano Frixione,
Francesco Giuli,
Alexander Grohsjean,
Christian Gütschow,
Stefan Höche,
Walter Hopkins,
Philip Ilten,
Dmitri Konstantinov,
Frank Krauss,
Qiang Li,
Leif Lönnblad,
Fabio Maltoni,
Michelangelo Mangano
, et al. (16 additional authors not shown)
Abstract:
We review the main software and computing challenges for the Monte Carlo physics event generators used by the LHC experiments, in view of the High-Luminosity LHC (HL-LHC) physics programme. This paper has been prepared by the HEP Software Foundation (HSF) Physics Event Generator Working Group as an input to the LHCC review of HL-LHC computing, which has started in May 2020.
We review the main software and computing challenges for the Monte Carlo physics event generators used by the LHC experiments, in view of the High-Luminosity LHC (HL-LHC) physics programme. This paper has been prepared by the HEP Software Foundation (HSF) Physics Event Generator Working Group as an input to the LHCC review of HL-LHC computing, which has started in May 2020.
△ Less
Submitted 18 February, 2021; v1 submitted 28 April, 2020;
originally announced April 2020.
-
HEP Software Foundation Community White Paper Working Group - Detector Simulation
Authors:
HEP Software Foundation,
:,
J Apostolakis,
M Asai,
S Banerjee,
R Bianchi,
P Canal,
R Cenci,
J Chapman,
G Corti,
G Cosmo,
S Easo,
L de Oliveira,
A Dotti,
V Elvira,
S Farrell,
L Fields,
K Genser,
A Gheata,
M Gheata,
J Harvey,
F Hariri,
R Hatcher,
K Herner,
M Hildreth
, et al. (40 additional authors not shown)
Abstract:
A working group on detector simulation was formed as part of the high-energy physics (HEP) Software Foundation's initiative to prepare a Community White Paper that describes the main software challenges and opportunities to be faced in the HEP field over the next decade. The working group met over a period of several months in order to review the current status of the Full and Fast simulation appl…
▽ More
A working group on detector simulation was formed as part of the high-energy physics (HEP) Software Foundation's initiative to prepare a Community White Paper that describes the main software challenges and opportunities to be faced in the HEP field over the next decade. The working group met over a period of several months in order to review the current status of the Full and Fast simulation applications of HEP experiments and the improvements that will need to be made in order to meet the goals of future HEP experimental programmes. The scope of the topics covered includes the main components of a HEP simulation application, such as MC truth handling, geometry modeling, particle propagation in materials and fields, physics modeling of the interactions of particles with matter, the treatment of pileup and other backgrounds, as well as signal processing and digitisation. The resulting work programme described in this document focuses on the need to improve both the software performance and the physics of detector simulation. The goals are to increase the accuracy of the physics models and expand their applicability to future physics programmes, while achieving large factors in computing performance gains consistent with projections on available computing resources.
△ Less
Submitted 12 March, 2018;
originally announced March 2018.
-
A Roadmap for HEP Software and Computing R&D for the 2020s
Authors:
Johannes Albrecht,
Antonio Augusto Alves Jr,
Guilherme Amadio,
Giuseppe Andronico,
Nguyen Anh-Ky,
Laurent Aphecetche,
John Apostolakis,
Makoto Asai,
Luca Atzori,
Marian Babik,
Giuseppe Bagliesi,
Marilena Bandieramonte,
Sunanda Banerjee,
Martin Barisits,
Lothar A. T. Bauerdick,
Stefano Belforte,
Douglas Benjamin,
Catrin Bernius,
Wahid Bhimji,
Riccardo Maria Bianchi,
Ian Bird,
Catherine Biscarat,
Jakob Blomer,
Kenneth Bloom,
Tommaso Boccali
, et al. (285 additional authors not shown)
Abstract:
Particle physics has an ambitious and broad experimental programme for the coming decades. This programme requires large investments in detector hardware, either to build new facilities and experiments, or to upgrade existing ones. Similarly, it requires commensurate investment in the R&D of software to acquire, manage, process, and analyse the shear amounts of data to be recorded. In planning for…
▽ More
Particle physics has an ambitious and broad experimental programme for the coming decades. This programme requires large investments in detector hardware, either to build new facilities and experiments, or to upgrade existing ones. Similarly, it requires commensurate investment in the R&D of software to acquire, manage, process, and analyse the shear amounts of data to be recorded. In planning for the HL-LHC in particular, it is critical that all of the collaborating stakeholders agree on the software goals and priorities, and that the efforts complement each other. In this spirit, this white paper describes the R&D activities required to prepare for this software upgrade.
△ Less
Submitted 19 December, 2018; v1 submitted 18 December, 2017;
originally announced December 2017.
-
Absolute luminosity measurements with the LHCb detector at the LHC
Authors:
The LHCb Collaboration,
R. Aaij,
B. Adeva,
M. Adinolfi,
C. Adrover,
A. Affolder,
Z. Ajaltouni,
J. Albrecht,
F. Alessio,
M. Alexander,
G. Alkhazov,
P. Alvarez Cartelle,
A. A. Alves Jr,
S. Amato,
Y. Amhis,
J. Anderson,
R. B. Appleby,
O. Aquines Gutierrez,
F. Archilli,
L. Arrabito,
A. Artamonov,
M. Artuso,
E. Aslanides,
G. Auriemma,
S. Bachmann
, et al. (549 additional authors not shown)
Abstract:
Absolute luminosity measurements are of general interest for colliding-beam experiments at storage rings. These measurements are necessary to determine the absolute cross-sections of reaction processes and are valuable to quantify the performance of the accelerator. Using data taken in 2010, LHCb has applied two methods to determine the absolute scale of its luminosity measurements for proton-prot…
▽ More
Absolute luminosity measurements are of general interest for colliding-beam experiments at storage rings. These measurements are necessary to determine the absolute cross-sections of reaction processes and are valuable to quantify the performance of the accelerator. Using data taken in 2010, LHCb has applied two methods to determine the absolute scale of its luminosity measurements for proton-proton collisions at the LHC with a centre-of-mass energy of 7 TeV. In addition to the classic "van der Meer scan" method a novel technique has been developed which makes use of direct imaging of the individual beams using beam-gas and beam-beam interactions. This beam imaging method is made possible by the high resolution of the LHCb vertex detector and the close proximity of the detector to the beams, and allows beam parameters such as positions, angles and widths to be determined. The results of the two methods have comparable precision and are in good agreement. Combining the two methods, an overall precision of 3.5% in the absolute luminosity determination is reached. The techniques used to transport the absolute luminosity calibration to the full 2010 data-taking period are presented.
△ Less
Submitted 11 January, 2012; v1 submitted 13 October, 2011;
originally announced October 2011.
-
Event Data Definition in LHCb
Authors:
Marco Cattaneo,
Gloria Corti,
Markus Frank,
Pere Mato Vila,
Silvia Miksch,
Stefan Roiser
Abstract:
We present the approach used for defining the event object model for the LHCb experiment. This approach is based on a high level modelling language, which is independent of the programming language used in the current implementation of the event data processing software. The different possibilities of object modelling languages are evaluated, and the advantages of a dedicated model based on XML…
▽ More
We present the approach used for defining the event object model for the LHCb experiment. This approach is based on a high level modelling language, which is independent of the programming language used in the current implementation of the event data processing software. The different possibilities of object modelling languages are evaluated, and the advantages of a dedicated model based on XML over other possible candidates are shown. After a description of the language itself, we explain the benefits obtained by applying this approach in the description of the event model of an experiment such as LHCb. Examples of these benefits are uniform and coherent map** of the object model to the implementation language across the experiment software development teams, easy maintenance of the event model, conformance to experiment coding rules, etc.
The description of the object model is parsed by means of a so called front-end which allows to feed several back-ends. We give an introduction to the model itself and to the currently implemented back-ends which produce information like programming language specific implementations of event objects or meta information about these objects. Meta information can be used for introspection of objects at run-time which is essential for functionalities like object persistency or interactive analysis. This object introspection package for C++ has been adopted by the LCG project as the starting point for the LCG object dictionary that is going to be developed in common for the LHC experiments.
The current status of the event object modelling and its usage in LHCb are presented and the prospects of further developments are discussed.
△ Less
Submitted 13 June, 2003;
originally announced June 2003.
-
The POOL Data Storage, Cache and Conversion Mechanism
Authors:
D. Dullmann,
M. Frank,
G. Govi,
I. Papadopoulos,
S. Roiser
Abstract:
The POOL data storage mechanism is intended to satisfy the needs of the LHC experiments to store and analyze the data from the detector response of particle collisions at the LHC proton-proton collider. Both the data rate and the data volumes will largely differ from the past experience. The POOL data storage mechanism is intended to be able to cope with the experiment's requirements applying a…
▽ More
The POOL data storage mechanism is intended to satisfy the needs of the LHC experiments to store and analyze the data from the detector response of particle collisions at the LHC proton-proton collider. Both the data rate and the data volumes will largely differ from the past experience. The POOL data storage mechanism is intended to be able to cope with the experiment's requirements applying a flexible multi technology data persistency mechanism. The developed technology independent approach is flexible enough to adopt new technologies, take advantage of existing schema evolution mechanisms and allows users to access data in a technology independent way. The framework consists of several components, which can be individually adopted and integrated into existing experiment frameworks.
△ Less
Submitted 11 June, 2003;
originally announced June 2003.
-
SEAL: Common Core Libraries and Services for LHC Applications
Authors:
J. Generowicz,
P. Mato,
L. Moneta,
S. Roiser,
M. Marino,
L. Tuura
Abstract:
The CERN LHC experiments have begun the LHC Computing Grid project in 2001. One of the project's aims is to develop common software infrastructure based on a development vision shared by the participating experiments. The SEAL project will provide common foundation libraries, services and utilities identified by the project's architecture blueprint report. This requires a broad range of function…
▽ More
The CERN LHC experiments have begun the LHC Computing Grid project in 2001. One of the project's aims is to develop common software infrastructure based on a development vision shared by the participating experiments. The SEAL project will provide common foundation libraries, services and utilities identified by the project's architecture blueprint report. This requires a broad range of functionality that no individual package suitably covers. SEAL thus selects external and experiment-developed packages, integrates them in a coherent whole, develops new code for missing functionality, and provides support to the experiments. We describe the set of basic components identified by the LHC Computing Grid project and thought to be sufficient for development of higher level framework components and specializations. Examples of such components are a plug-in manager, an object dictionary, object whiteboards, an incident or event manager. We present the design and implementation of some of these components and the underlying foundation libraries in some detail.
△ Less
Submitted 4 June, 2003;
originally announced June 2003.
