-
The O2 software framework and GPU usage in ALICE online and offline reconstruction in Run 3
Authors:
Giulio Eulisse,
David Rohr
Abstract:
ALICE has upgraded many of its detectors for LHC Run 3 to operate in continuous readout mode recording Pb--Pb collisions at 50 kHz interaction rate without trigger. This results in the need to process data in real time at rates 100 times higher than during Run 2. In order to tackle such a challenge we introduced O2, a new computing system and the associated infrastructure. Designed and implemented…
▽ More
ALICE has upgraded many of its detectors for LHC Run 3 to operate in continuous readout mode recording Pb--Pb collisions at 50 kHz interaction rate without trigger. This results in the need to process data in real time at rates 100 times higher than during Run 2. In order to tackle such a challenge we introduced O2, a new computing system and the associated infrastructure. Designed and implemented during the LHC long shutdown 2, O2 is now in production taking care of all the data processing needs of the experiment. O2 is designed around the message passing paradigm, enabling resilient, parallel data processing for both the synchronous (to LHC beam) and asynchronous data taking and processing phases. The main purpose of the synchronous online reconstruction is detector calibration and raw data compression. This synchronous processing is dominated by the TPC detector, which produces by far the largest data volume, and TPC reconstruction runs fully on GPUs. When there is no beam in the LHC, the powerful GPU-equipped online computing farm of ALICE is used for the asynchronous reconstruction, which creates the final reconstructed output for analysis from the compressed raw data. Since the majority of the compute performance of the online farm is in the GPUs, and since the asynchronous processing is not dominated by the TPC in the way the synchronous processing is, there is an ongoing effort to offload a significant amount of compute load from other detectors to the GPU as well.
△ Less
Submitted 2 February, 2024;
originally announced February 2024.
-
Second Analysis Ecosystem Workshop Report
Authors:
Mohamed Aly,
Jackson Burzynski,
Bryan Cardwell,
Daniel C. Craik,
Tal van Daalen,
Tomas Dado,
Ayanabha Das,
Antonio Delgado Peris,
Caterina Doglioni,
Peter Elmer,
Engin Eren,
Martin B. Eriksen,
Jonas Eschle,
Giulio Eulisse,
Conor Fitzpatrick,
José Flix Molina,
Alessandra Forti,
Ben Galewsky,
Sean Gasiorowski,
Aman Goel,
Loukas Gouskos,
Enrico Guiraud,
Kanhaiya Gupta,
Stephan Hageboeck,
Allison Reinsvold Hall
, et al. (44 additional authors not shown)
Abstract:
The second workshop on the HEP Analysis Ecosystem took place 23-25 May 2022 at IJCLab in Orsay, to look at progress and continuing challenges in scaling up HEP analysis to meet the needs of HL-LHC and DUNE, as well as the very pressing needs of LHC Run 3 analysis.
The workshop was themed around six particular topics, which were felt to capture key questions, opportunities and challenges. Each to…
▽ More
The second workshop on the HEP Analysis Ecosystem took place 23-25 May 2022 at IJCLab in Orsay, to look at progress and continuing challenges in scaling up HEP analysis to meet the needs of HL-LHC and DUNE, as well as the very pressing needs of LHC Run 3 analysis.
The workshop was themed around six particular topics, which were felt to capture key questions, opportunities and challenges. Each topic arranged a plenary session introduction, often with speakers summarising the state-of-the art and the next steps for analysis. This was then followed by parallel sessions, which were much more discussion focused, and where attendees could grapple with the challenges and propose solutions that could be tried. Where there was significant overlap between topics, a joint discussion between them was arranged.
In the weeks following the workshop the session conveners wrote this document, which is a summary of the main discussions, the key points raised and the conclusions and outcomes. The document was circulated amongst the participants for comments before being finalised here.
△ Less
Submitted 9 December, 2022;
originally announced December 2022.
-
HL-LHC Computing Review: Common Tools and Community Software
Authors:
HEP Software Foundation,
:,
Thea Aarrestad,
Simone Amoroso,
Markus Julian Atkinson,
Joshua Bendavid,
Tommaso Boccali,
Andrea Bocci,
Andy Buckley,
Matteo Cacciari,
Paolo Calafiura,
Philippe Canal,
Federico Carminati,
Taylor Childers,
Vitaliano Ciulli,
Gloria Corti,
Davide Costanzo,
Justin Gage Dezoort,
Caterina Doglioni,
Javier Mauricio Duarte,
Agnieszka Dziurda,
Peter Elmer,
Markus Elsing,
V. Daniel Elvira,
Giulio Eulisse
, et al. (85 additional authors not shown)
Abstract:
Common and community software packages, such as ROOT, Geant4 and event generators have been a key part of the LHC's success so far and continued development and optimisation will be critical in the future. The challenges are driven by an ambitious physics programme, notably the LHC accelerator upgrade to high-luminosity, HL-LHC, and the corresponding detector upgrades of ATLAS and CMS. In this doc…
▽ More
Common and community software packages, such as ROOT, Geant4 and event generators have been a key part of the LHC's success so far and continued development and optimisation will be critical in the future. The challenges are driven by an ambitious physics programme, notably the LHC accelerator upgrade to high-luminosity, HL-LHC, and the corresponding detector upgrades of ATLAS and CMS. In this document we address the issues for software that is used in multiple experiments (usually even more widely than ATLAS and CMS) and maintained by teams of developers who are either not linked to a particular experiment or who contribute to common software within the context of their experiment activity. We also give space to general considerations for future software and projects that tackle upcoming challenges, no matter who writes it, which is an area where community convergence on best practice is extremely useful.
△ Less
Submitted 31 August, 2020;
originally announced August 2020.
-
A next-generation LHC heavy-ion experiment
Authors:
D. Adamová,
G. Aglieri Rinella,
M. Agnello,
Z. Ahammed,
D. Aleksandrov,
A. Alici,
A. Alkin,
T. Alt,
I. Altsybeev,
D. Andreou,
A. Andronic,
F. Antinori,
P. Antonioli,
H. Appelshäuser,
R. Arnaldi,
I. C. Arsene,
M. Arslandok,
R. Averbeck,
M. D. Azmi,
X. Bai,
R. Bailhache,
R. Bala,
L. Barioglio,
G. G. Barnaföldi,
L. S. Barnby
, et al. (374 additional authors not shown)
Abstract:
The present document discusses plans for a compact, next-generation multi-purpose detector at the LHC as a follow-up to the present ALICE experiment. The aim is to build a nearly massless barrel detector consisting of truly cylindrical layers based on curved wafer-scale ultra-thin silicon sensors with MAPS technology, featuring an unprecedented low material budget of 0.05% X$_0$ per layer, with th…
▽ More
The present document discusses plans for a compact, next-generation multi-purpose detector at the LHC as a follow-up to the present ALICE experiment. The aim is to build a nearly massless barrel detector consisting of truly cylindrical layers based on curved wafer-scale ultra-thin silicon sensors with MAPS technology, featuring an unprecedented low material budget of 0.05% X$_0$ per layer, with the innermost layers possibly positioned inside the beam pipe. In addition to superior tracking and vertexing capabilities over a wide momentum range down to a few tens of MeV/$c$, the detector will provide particle identification via time-of-flight determination with about 20~ps resolution. In addition, electron and photon identification will be performed in a separate shower detector. The proposed detector is conceived for studies of pp, pA and AA collisions at luminosities a factor of 20 to 50 times higher than possible with the upgraded ALICE detector, enabling a rich physics program ranging from measurements with electromagnetic probes at ultra-low transverse momenta to precision physics in the charm and beauty sector.
△ Less
Submitted 2 May, 2019; v1 submitted 31 January, 2019;
originally announced February 2019.
-
HEP Software Foundation Community White Paper Working Group - Software Development, Deployment and Validation
Authors:
Benjamin Couturier,
Giulio Eulisse,
Hadrien Grasland,
Benedikt Hegner,
Michel Jouvin,
Meghan Kane,
Daniel S. Katz,
Thomas Kuhr,
David Lange,
Patricia Mendez Lorenzo,
Martin Ritter,
Graeme Andrew Stewart,
Andrea Valassi
Abstract:
The High Energy Phyiscs community has developed and needs to maintain many tens of millions of lines of code and to integrate effectively the work of thousands of developers across large collaborations. Software needs to be built, validated, and deployed across hundreds of sites. Software also has a lifetime of many years, frequently beyond that of the original developer, it must be developed with…
▽ More
The High Energy Phyiscs community has developed and needs to maintain many tens of millions of lines of code and to integrate effectively the work of thousands of developers across large collaborations. Software needs to be built, validated, and deployed across hundreds of sites. Software also has a lifetime of many years, frequently beyond that of the original developer, it must be developed with sustainability in mind. Adequate recognition of software development as a critical task in the HEP community needs to be fostered and an appropriate publication and citation strategy needs to be developed. As part of the HEP Softare Foundation's Community White Paper process a working group on Software Development, Deployment and Validation was formed to examine all of these issues, identify best practice and to formulare recommendations for the next decade. Its report is presented here.
△ Less
Submitted 15 June, 2018; v1 submitted 21 December, 2017;
originally announced December 2017.
-
A Roadmap for HEP Software and Computing R&D for the 2020s
Authors:
Johannes Albrecht,
Antonio Augusto Alves Jr,
Guilherme Amadio,
Giuseppe Andronico,
Nguyen Anh-Ky,
Laurent Aphecetche,
John Apostolakis,
Makoto Asai,
Luca Atzori,
Marian Babik,
Giuseppe Bagliesi,
Marilena Bandieramonte,
Sunanda Banerjee,
Martin Barisits,
Lothar A. T. Bauerdick,
Stefano Belforte,
Douglas Benjamin,
Catrin Bernius,
Wahid Bhimji,
Riccardo Maria Bianchi,
Ian Bird,
Catherine Biscarat,
Jakob Blomer,
Kenneth Bloom,
Tommaso Boccali
, et al. (285 additional authors not shown)
Abstract:
Particle physics has an ambitious and broad experimental programme for the coming decades. This programme requires large investments in detector hardware, either to build new facilities and experiments, or to upgrade existing ones. Similarly, it requires commensurate investment in the R&D of software to acquire, manage, process, and analyse the shear amounts of data to be recorded. In planning for…
▽ More
Particle physics has an ambitious and broad experimental programme for the coming decades. This programme requires large investments in detector hardware, either to build new facilities and experiments, or to upgrade existing ones. Similarly, it requires commensurate investment in the R&D of software to acquire, manage, process, and analyse the shear amounts of data to be recorded. In planning for the HL-LHC in particular, it is critical that all of the collaborating stakeholders agree on the software goals and priorities, and that the efforts complement each other. In this spirit, this white paper describes the R&D activities required to prepare for this software upgrade.
△ Less
Submitted 19 December, 2018; v1 submitted 18 December, 2017;
originally announced December 2017.
-
Future Computing Platforms for Science in a Power Constrained Era
Authors:
David Abdurachmanov,
Peter Elmer,
Giulio Eulisse,
Robert Knight
Abstract:
Power consumption will be a key constraint on the future growth of Distributed High Throughput Computing (DHTC) as used by High Energy Physics (HEP). This makes performance-per-watt a crucial metric for selecting cost-efficient computing solutions. For this paper, we have done a wide survey of current and emerging architectures becoming available on the market including x86-64 variants, ARMv7 32-b…
▽ More
Power consumption will be a key constraint on the future growth of Distributed High Throughput Computing (DHTC) as used by High Energy Physics (HEP). This makes performance-per-watt a crucial metric for selecting cost-efficient computing solutions. For this paper, we have done a wide survey of current and emerging architectures becoming available on the market including x86-64 variants, ARMv7 32-bit, ARMv8 64-bit, Many-Core and GPU solutions, as well as newer System-on-Chip (SoC) solutions. We compare performance and energy efficiency using an evolving set of standardized HEP-related benchmarks and power measurement techniques we have been develo**. We evaluate the potential for use of such computing solutions in the context of DHTC systems, such as the Worldwide LHC Computing Grid (WLCG).
△ Less
Submitted 28 July, 2015;
originally announced October 2015.
-
Optimizing CMS build infrastructure via Apache Mesos
Authors:
David Abdurachmanov,
Alessandro Degano,
Peter Elmer,
Giulio Eulisse,
David Mendez,
Shahzad Muzaffar
Abstract:
The Offline Software of the CMS Experiment at the Large Hadron Collider (LHC) at CERN consists of 6M lines of in-house code, developed over a decade by nearly 1000 physicists, as well as a comparable amount of general use open-source code. A critical ingredient to the success of the construction and early operation of the WLCG was the convergence, around the year 2000, on the use of a homogeneous…
▽ More
The Offline Software of the CMS Experiment at the Large Hadron Collider (LHC) at CERN consists of 6M lines of in-house code, developed over a decade by nearly 1000 physicists, as well as a comparable amount of general use open-source code. A critical ingredient to the success of the construction and early operation of the WLCG was the convergence, around the year 2000, on the use of a homogeneous environment of commodity x86-64 processors and Linux. Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications, or frameworks. It can run Hadoop, Jenkins, Spark, Aurora, and other applications on a dynamically shared pool of nodes. We present how we migrated our continuos integration system to schedule jobs on a relatively small Apache Mesos enabled cluster and how this resulted in better resource usage, higher peak performance and lower latency thanks to the dynamic scheduling capabilities of Mesos.
△ Less
Submitted 28 July, 2015; v1 submitted 20 July, 2015;
originally announced July 2015.
-
Heterogeneous High Throughput Scientific Computing with APM X-Gene and Intel Xeon Phi
Authors:
David Abdurachmanov,
Brian Bockelman,
Peter Elmer,
Giulio Eulisse,
Robert Knight,
Shahzad Muzaffar
Abstract:
Electrical power requirements will be a constraint on the future growth of Distributed High Throughput Computing (DHTC) as used by High Energy Physics. Performance-per-watt is a critical metric for the evaluation of computer architectures for cost- efficient computing. Additionally, future performance growth will come from heterogeneous, many-core, and high computing density platforms with special…
▽ More
Electrical power requirements will be a constraint on the future growth of Distributed High Throughput Computing (DHTC) as used by High Energy Physics. Performance-per-watt is a critical metric for the evaluation of computer architectures for cost- efficient computing. Additionally, future performance growth will come from heterogeneous, many-core, and high computing density platforms with specialized processors. In this paper, we examine the Intel Xeon Phi Many Integrated Cores (MIC) co-processor and Applied Micro X-Gene ARMv8 64-bit low-power server system-on-a-chip (SoC) solutions for scientific computing applications. We report our experience on software porting, performance and energy efficiency and evaluate the potential for use of such technologies in the context of distributed computing systems such as the Worldwide LHC Computing Grid (WLCG).
△ Less
Submitted 10 October, 2014;
originally announced October 2014.
-
Techniques and tools for measuring energy efficiency of scientific software applications
Authors:
David Abdurachmanov,
Peter Elmer,
Giulio Eulisse,
Robert Knight,
Tapio Niemi,
Jukka K. Nurminen,
Filip Nyback,
Goncalo Pestana,
Zhonghong Ou,
Kashif Khan
Abstract:
The scale of scientific High Performance Computing (HPC) and High Throughput Computing (HTC) has increased significantly in recent years, and is becoming sensitive to total energy use and cost. Energy-efficiency has thus become an important concern in scientific fields such as High Energy Physics (HEP). There has been a growing interest in utilizing alternate architectures, such as low power ARM p…
▽ More
The scale of scientific High Performance Computing (HPC) and High Throughput Computing (HTC) has increased significantly in recent years, and is becoming sensitive to total energy use and cost. Energy-efficiency has thus become an important concern in scientific fields such as High Energy Physics (HEP). There has been a growing interest in utilizing alternate architectures, such as low power ARM processors, to replace traditional Intel x86 architectures. Nevertheless, even though such solutions have been successfully used in mobile applications with low I/O and memory demands, it is unclear if they are suitable and more energy-efficient in the scientific computing environment. Furthermore, there is a lack of tools and experience to derive and compare power consumption between the architectures for various workloads, and eventually to support software optimizations for energy efficiency. To that end, we have performed several physical and software-based measurements of workloads from HEP applications running on ARM and Intel architectures, and compare their power consumption and performance. We leverage several profiling tools (both in hardware and software) to extract different characteristics of the power use. We report the results of these measurements and the experience gained in develo** a set of measurement techniques and profiling tools to accurately assess the power consumption for scientific workloads.
△ Less
Submitted 10 October, 2014;
originally announced October 2014.
-
Power-aware applications for scientific cluster and distributed computing
Authors:
David Abdurachmanov,
Peter Elmer,
Giulio Eulisse,
Paola Grosso,
Curtis Hillegas,
Burt Holzman,
Ruben L. Janssen,
Sander Klous,
Robert Knight,
Shahzad Muzaffar
Abstract:
The aggregate power use of computing hardware is an important cost factor in scientific cluster and distributed computing systems. The Worldwide LHC Computing Grid (WLCG) is a major example of such a distributed computing system, used primarily for high throughput computing (HTC) applications. It has a computing capacity and power consumption rivaling that of the largest supercomputers. The comput…
▽ More
The aggregate power use of computing hardware is an important cost factor in scientific cluster and distributed computing systems. The Worldwide LHC Computing Grid (WLCG) is a major example of such a distributed computing system, used primarily for high throughput computing (HTC) applications. It has a computing capacity and power consumption rivaling that of the largest supercomputers. The computing capacity required from this system is also expected to grow over the next decade. Optimizing the power utilization and cost of such systems is thus of great interest.
A number of trends currently underway will provide new opportunities for power-aware optimizations. We discuss how power-aware software applications and scheduling might be used to reduce power consumption, both as autonomous entities and as part of a (globally) distributed system. As concrete examples of computing centers we provide information on the large HEP-focused Tier-1 at FNAL, and the Tigress High Performance Computing Center at Princeton University, which provides HPC resources in a university context.
△ Less
Submitted 22 October, 2014; v1 submitted 28 April, 2014;
originally announced April 2014.
-
Explorations of the viability of ARM and Xeon Phi for physics processing
Authors:
David Abdurachmanov,
Kapil Arya,
Josh Bendavid,
Tommaso Boccali,
Gene Cooperman,
Andrea Dotti,
Peter Elmer,
Giulio Eulisse,
Francesco Giacomini,
Christopher D. Jones,
Matteo Manzali,
Shahzad Muzaffar
Abstract:
We report on our investigations into the viability of the ARM processor and the Intel Xeon Phi co-processor for scientific computing. We describe our experience porting software to these processors and running benchmarks using real physics applications to explore the potential of these processors for production physics processing.
We report on our investigations into the viability of the ARM processor and the Intel Xeon Phi co-processor for scientific computing. We describe our experience porting software to these processors and running benchmarks using real physics applications to explore the potential of these processors for production physics processing.
△ Less
Submitted 21 January, 2014; v1 submitted 5 November, 2013;
originally announced November 2013.
-
Initial explorations of ARM processors for scientific computing
Authors:
David Abdurachmanov,
Peter Elmer,
Giulio Eulisse,
Shahzad Muzaffar
Abstract:
Power efficiency is becoming an ever more important metric for both high performance and high throughput computing. Over the course of next decade it is expected that flops/watt will be a major driver for the evolution of computer architecture. Servers with large numbers of ARM processors, already ubiquitous in mobile computing, are a promising alternative to traditional x86-64 computing. We prese…
▽ More
Power efficiency is becoming an ever more important metric for both high performance and high throughput computing. Over the course of next decade it is expected that flops/watt will be a major driver for the evolution of computer architecture. Servers with large numbers of ARM processors, already ubiquitous in mobile computing, are a promising alternative to traditional x86-64 computing. We present the results of our initial investigations into the use of ARM processors for scientific computing applications. In particular we report the results from our work with a current generation ARMv7 development board to explore ARM-specific issues regarding the software development environment, operating system, performance benchmarks and issues for porting High Energy Physics software.
△ Less
Submitted 22 January, 2014; v1 submitted 1 November, 2013;
originally announced November 2013.
-
IGUANA Architecture, Framework and Toolkit for Interactive Graphics
Authors:
George Alverson,
Giulio Eulisse,
Shahzad Muzaffar,
Ianna Osborne,
Lassi A. Tuura,
Lucas Taylor
Abstract:
IGUANA is a generic interactive visualisation framework based on a C++ component model. It provides powerful user interface and visualisation primitives in a way that is not tied to any particular physics experiment or detector design. The article describes interactive visualisation tools built using IGUANA for the CMS and D0 experiments, as well as generic GEANT4 and GEANT3 applications. It cov…
▽ More
IGUANA is a generic interactive visualisation framework based on a C++ component model. It provides powerful user interface and visualisation primitives in a way that is not tied to any particular physics experiment or detector design. The article describes interactive visualisation tools built using IGUANA for the CMS and D0 experiments, as well as generic GEANT4 and GEANT3 applications. It covers features of the graphical user interfaces, 3D and 2D graphics, high-quality vector graphics output for print media, various textual, tabular and hierarchical data views, and integration with the application through control panels, a command line and different multi-threading models.
△ Less
Submitted 10 June, 2003;
originally announced June 2003.
