-
Towards podio v1.0 -- A first stable release of the EDM toolkit
Authors:
Juan Miguel Carceller,
Frank Gaede,
Gerardo Ganis,
Benedikt Hegner,
Clement Helsens,
Thomas Madlener,
André Sailer,
Graeme A Stewart,
Valentin Volkl
Abstract:
A performant and easy-to-use event data model (EDM) is a key component of any HEP software stack. The podio EDM toolkit provides a user friendly way of generating such a performant implementation in C++ from a high level description in yaml format. Finalizing a few important developments, we are in the final stretches for release v1.0 of podio, a stable release with backward compatibility for data…
▽ More
A performant and easy-to-use event data model (EDM) is a key component of any HEP software stack. The podio EDM toolkit provides a user friendly way of generating such a performant implementation in C++ from a high level description in yaml format. Finalizing a few important developments, we are in the final stretches for release v1.0 of podio, a stable release with backward compatibility for datafiles written with podio from then on. We present an overview of the podio basics, and go into slighty more technical detail on the most important topics and developments. These include: schema evolution for generated EDMs, multithreading with podio generated EDMs, the implementation of them as well as the basics of I/O. Using EDM4hep, the common and shared EDM of the Key4hep project, we highlight a few of the smaller features in action as well as some lessons learned during the development of EDM4hep and podio. Finally, we show how podio has been integrated into the Gaudi based event processing framework that is used by Key4hep, before we conclude with a brief outlook on potential developments after v1.0.
△ Less
Submitted 13 December, 2023;
originally announced December 2023.
-
Of Frames and schema evolution -- The newest features of podio
Authors:
Placido Fernandez Declara,
Frank Gaede,
Gerardo Ganis,
Benedikt Hegner,
Clement Helsens,
Thomas Madlener,
Andre Sailer,
Graeme A Stewart,
Valentin Volkl
Abstract:
The podio event data model (EDM) toolkit provides an easy way to generate a performant implementation of an EDM from a high level description in yaml format. We present the most recent developments in podio, most importantly the inclusion of a schema evolution mechanism for generated EDMs as well as the "Frame", a thread safe, generalized event data container. For the former we discuss some of the…
▽ More
The podio event data model (EDM) toolkit provides an easy way to generate a performant implementation of an EDM from a high level description in yaml format. We present the most recent developments in podio, most importantly the inclusion of a schema evolution mechanism for generated EDMs as well as the "Frame", a thread safe, generalized event data container. For the former we discuss some of the technical aspects in relation with supporting different I/O backends and leveraging potentially existing schema evolution mechanisms provided by them. Regarding the Frame we introduce the basic concept and highlight some of the functionality as well as important aspects of its implementation. The usage of podio for generating different EDMs for future collider projects (most importantly EDM4hep, the common EDM for the Key4hep project) has inspired new features. We present some of those smaller new features and end with a brief overview on current developments towards a first stable version as well as an outlook on future developments beyond that.
△ Less
Submitted 13 December, 2023;
originally announced December 2023.
-
Key4hep: Progress Report on Integrations
Authors:
Erica Brondolin,
Juan Miguel Carceller,
Wouter Deconinck,
Wenxing Fang,
Brieuc Francois,
Frank-Dieter Gaede,
Gerardo Ganis,
Benedikt Hegner,
Clement Helsens,
Xingtao Huang,
Sylvester Joosten,
Sang Hyun Ko,
Tao Lin,
Teng Li,
Weidong Li,
Thomas Madlener,
Leonhard Reichenbach,
André Sailer,
Swathi Sasikumar,
Juraj Smiesko,
Graeme A Stewart,
Alvaro Tolosa-Delgado,
Valentin Volkl,
Xiaomei Zhang,
Jiaheng Zou
Abstract:
Detector studies for future experiments rely on advanced software tools to estimate performance and optimize their design and technology choices. The Key4hep project provides a flexible turnkey solution for the full experiment life-cycle based on established community tools such as ROOT, Geant4, DD4hep, Gaudi, podio and spack. Members of the CEPC, CLIC, EIC, FCC, and ILC communities have joined to…
▽ More
Detector studies for future experiments rely on advanced software tools to estimate performance and optimize their design and technology choices. The Key4hep project provides a flexible turnkey solution for the full experiment life-cycle based on established community tools such as ROOT, Geant4, DD4hep, Gaudi, podio and spack. Members of the CEPC, CLIC, EIC, FCC, and ILC communities have joined to develop this framework and have merged, or are in the progress of merging, their respective software environments into the Key4hep stack. These proceedings will give an overview over the recent progress in the Key4hep project: covering the developments towards adaptation of state-of-the-art tools for simulation (DD4hep, Gaussino), track and calorimeter reconstruction (ACTS, CLUE), particle flow (PandoraPFA), analysis via RDataFrame, and visualization with Phoenix, as well as tools for testing and validation.
△ Less
Submitted 13 December, 2023;
originally announced December 2023.
-
The Key4hep software stack: Beyond Future Higgs factories
Authors:
Andre Sailer,
Benedikt Hegner,
Clement Helsens,
Erica Brondolin,
Frank-Dieter Gaede,
Gerardo Ganis,
Graeme A Stewart,
Jiaheng Zou,
Juraj Smiesko,
Placido Fernandez Declara,
Sang Hyun Ko,
Sylvester Joosten,
Tao Lin,
Teng Li,
Thomas Madlener,
Valentin Volkl,
Weidong Li,
Wenxing Fang,
Wouter Deconinck,
Xingtao Huang,
Xiaomei Zhang
Abstract:
The Key4hep project aims to provide a turnkey software solution for the full experiment lifecycle, based on established community tools. Several future collider communities (CEPC, CLIC, EIC, FCC, and ILC) have joined to develop and adapt their workflows to use the common data model EDM4hep and common framework. Besides sharing of existing experiment workflows, one focus of the Key4hep project is t…
▽ More
The Key4hep project aims to provide a turnkey software solution for the full experiment lifecycle, based on established community tools. Several future collider communities (CEPC, CLIC, EIC, FCC, and ILC) have joined to develop and adapt their workflows to use the common data model EDM4hep and common framework. Besides sharing of existing experiment workflows, one focus of the Key4hep project is the development and integration of new experiment independent software libraries. Ongoing collaborations with projects such as ACTS, CLUE, PandoraPFA and the OpenDataDector show the potential of Key4hep as an experiment-independent testbed and development platform. In this talk, we present the challenges of an experiment-independent framework along with the lessons learned from discussions of interested communities (such as LUXE) and recent adopters of Key4hep in order to discuss how Key4hep could be of interest to the wider HEP community while staying true to its goal of supporting future collider designs studies.
△ Less
Submitted 13 December, 2023;
originally announced December 2023.
-
Training and Onboarding initiatives in High Energy Physics experiments
Authors:
S. Hageboeck,
A. Reinsvold Hall,
N. Skidmore,
G. A. Stewart,
G. Benelli,
B. Carlson,
C. David,
J. Davies,
W. Deconinck,
D. DeMuth, Jr.,
P. Elmer,
R. B. Garg,
K. Lieret,
V. Lukashenko,
S. Malik,
A. Morris,
H. Schellman,
J. Veatch,
M. Hernandez Villanueva
Abstract:
In this paper we document the current analysis software training and onboarding activities in several High Energy Physics (HEP) experiments: ATLAS, CMS, LHCb, Belle II and DUNE. Fast and efficient onboarding of new collaboration members is increasingly important for HEP experiments as analyses and the related software become ever more complex with growing datasets. A meeting series was held by the…
▽ More
In this paper we document the current analysis software training and onboarding activities in several High Energy Physics (HEP) experiments: ATLAS, CMS, LHCb, Belle II and DUNE. Fast and efficient onboarding of new collaboration members is increasingly important for HEP experiments as analyses and the related software become ever more complex with growing datasets. A meeting series was held by the HEP Software Foundation (HSF) in 2022 for experiments to showcase their initiatives. Here we document and analyse these in an attempt to determine a set of key considerations for future experiments.
△ Less
Submitted 23 October, 2023; v1 submitted 11 October, 2023;
originally announced October 2023.
-
Polyglot Jet Finding
Authors:
Graeme Andrew Stewart,
Philippe Gras,
Benedikt Hegner,
Atell Krasnopolski
Abstract:
The evaluation of new computing languages for a large community, like HEP, involves comparison of many aspects of the languages' behaviour, ecosystem and interactions with other languages. In this paper we compare a number of languages using a common, yet non-trivial, HEP algorithm: the \akt\ clustering algorithm used for jet finding. We compare specifically the algorithm implemented in Python (pu…
▽ More
The evaluation of new computing languages for a large community, like HEP, involves comparison of many aspects of the languages' behaviour, ecosystem and interactions with other languages. In this paper we compare a number of languages using a common, yet non-trivial, HEP algorithm: the \akt\ clustering algorithm used for jet finding. We compare specifically the algorithm implemented in Python (pure Python and accelerated with numpy and numba), and Julia, with respect to the reference implementation in C++, from Fastjet. As well as the speed of the implementation we describe the ergonomics of the language for the coder, as well as the efforts required to achieve the best performance, which can directly impact on code readability and sustainability.
△ Less
Submitted 8 May, 2024; v1 submitted 29 September, 2023;
originally announced September 2023.
-
Software Citation in HEP: Current State and Recommendations for the Future
Authors:
Matthew Feickert,
Daniel S. Katz,
Mark S. Neubauer,
Elizabeth Sexton-Kennedy,
Graeme A. Stewart
Abstract:
In November 2022, the HEP Software Foundation and the Institute for Research and Innovation for Software in High-Energy Physics organized a workshop on the topic of Software Citation and Recognition in HEP. The goal of the workshop was to bring together different types of stakeholders whose roles relate to software citation, and the associated credit it provides, in order to engage the community i…
▽ More
In November 2022, the HEP Software Foundation and the Institute for Research and Innovation for Software in High-Energy Physics organized a workshop on the topic of Software Citation and Recognition in HEP. The goal of the workshop was to bring together different types of stakeholders whose roles relate to software citation, and the associated credit it provides, in order to engage the community in a discussion on: the ways HEP experiments handle citation of software, recognition for software efforts that enable physics results disseminated to the public, and how the scholarly publishing ecosystem supports these activities. Reports were given from the publication board leadership of the ATLAS, CMS, and LHCb experiments and HEP open source software community organizations (ROOT, Scikit-HEP, MCnet), and perspectives were given from publishers (Elsevier, JOSS) and related tool providers (INSPIRE, Zenodo). This paper summarizes key findings and recommendations from the workshop as presented at the 26th International Conference on Computing in High Energy and Nuclear Physics (CHEP 2023).
△ Less
Submitted 4 January, 2024; v1 submitted 25 September, 2023;
originally announced September 2023.
-
Potential of the Julia programming language for high energy physics computing
Authors:
J. Eschle,
T. Gal,
M. Giordano,
P. Gras,
B. Hegner,
L. Heinrich,
U. Hernandez Acosta,
S. Kluth,
J. Ling,
P. Mato,
M. Mikhasenko,
A. Moreno Briceño,
J. Pivarski,
K. Samaras-Tsakiris,
O. Schulz,
G. . A. Stewart,
J. Strube,
V. Vassilev
Abstract:
Research in high energy physics (HEP) requires huge amounts of computing and storage, putting strong constraints on the code speed and resource usage. To meet these requirements, a compiled high-performance language is typically used; while for physicists, who focus on the application when develo** the code, better research productivity pleads for a high-level programming language. A popular app…
▽ More
Research in high energy physics (HEP) requires huge amounts of computing and storage, putting strong constraints on the code speed and resource usage. To meet these requirements, a compiled high-performance language is typically used; while for physicists, who focus on the application when develo** the code, better research productivity pleads for a high-level programming language. A popular approach consists of combining Python, used for the high-level interface, and C++, used for the computing intensive part of the code. A more convenient and efficient approach would be to use a language that provides both high-level programming and high-performance. The Julia programming language, developed at MIT especially to allow the use of a single language in research activities, has followed this path. In this paper the applicability of using the Julia language for HEP research is explored, covering the different aspects that are important for HEP code development: runtime performance, handling of large projects, interface with legacy code, distributed computing, training, and ease of programming. The study shows that the HEP community would benefit from a large scale adoption of this programming language. The HEP-specific foundation libraries that would need to be consolidated are identified
△ Less
Submitted 6 October, 2023; v1 submitted 6 June, 2023;
originally announced June 2023.
-
Second Analysis Ecosystem Workshop Report
Authors:
Mohamed Aly,
Jackson Burzynski,
Bryan Cardwell,
Daniel C. Craik,
Tal van Daalen,
Tomas Dado,
Ayanabha Das,
Antonio Delgado Peris,
Caterina Doglioni,
Peter Elmer,
Engin Eren,
Martin B. Eriksen,
Jonas Eschle,
Giulio Eulisse,
Conor Fitzpatrick,
José Flix Molina,
Alessandra Forti,
Ben Galewsky,
Sean Gasiorowski,
Aman Goel,
Loukas Gouskos,
Enrico Guiraud,
Kanhaiya Gupta,
Stephan Hageboeck,
Allison Reinsvold Hall
, et al. (44 additional authors not shown)
Abstract:
The second workshop on the HEP Analysis Ecosystem took place 23-25 May 2022 at IJCLab in Orsay, to look at progress and continuing challenges in scaling up HEP analysis to meet the needs of HL-LHC and DUNE, as well as the very pressing needs of LHC Run 3 analysis.
The workshop was themed around six particular topics, which were felt to capture key questions, opportunities and challenges. Each to…
▽ More
The second workshop on the HEP Analysis Ecosystem took place 23-25 May 2022 at IJCLab in Orsay, to look at progress and continuing challenges in scaling up HEP analysis to meet the needs of HL-LHC and DUNE, as well as the very pressing needs of LHC Run 3 analysis.
The workshop was themed around six particular topics, which were felt to capture key questions, opportunities and challenges. Each topic arranged a plenary session introduction, often with speakers summarising the state-of-the art and the next steps for analysis. This was then followed by parallel sessions, which were much more discussion focused, and where attendees could grapple with the challenges and propose solutions that could be tried. Where there was significant overlap between topics, a joint discussion between them was arranged.
In the weeks following the workshop the session conveners wrote this document, which is a summary of the main discussions, the key points raised and the conclusions and outcomes. The document was circulated amongst the participants for comments before being finalised here.
△ Less
Submitted 9 December, 2022;
originally announced December 2022.
-
Offloading electromagnetic shower transport to GPUs
Authors:
G. Amadio,
J. Apostolakis,
P. Buncic,
G. Cosmo,
D. Dosaru,
A. Gheata,
S. Hageboeck,
J. Hahnfeld,
M. Hodgkinson,
B. Morgan,
M. Novak,
A. A. Petre,
W. Pokorski,
A. Ribon,
G. A. Stewart,
P. M. Vila
Abstract:
Making general particle transport simulation for high-energy physics (HEP) single-instruction-multiple-thread (SIMT) friendly, to take advantage of accelerator hardware, is an important alternative for boosting the throughput of simulation applications. To date, this challenge is not yet resolved, due to difficulties in map** the complexity of Geant4 components and workflow to the massive parall…
▽ More
Making general particle transport simulation for high-energy physics (HEP) single-instruction-multiple-thread (SIMT) friendly, to take advantage of accelerator hardware, is an important alternative for boosting the throughput of simulation applications. To date, this challenge is not yet resolved, due to difficulties in map** the complexity of Geant4 components and workflow to the massive parallelism features exposed by graphics processing units (GPU). The AdePT project is one of the R\&D initiatives tackling this limitation and exploring GPUs as potential accelerators for offloading some part of the CPU simulation workload. Our main target is to implement a complete electromagnetic shower demonstrator working on the GPU. The project is the first to create a full prototype of a realistic electron, positron, and gamma electromagnetic shower simulation on GPU, implemented as either a standalone application or as an extension of the standard Geant4 CPU workflow. Our prototype currently provides a platform to explore many optimisations and different approaches. We present the most recent results and initial conclusions of our work, using both a standalone GPU performance analysis and a first implementation of a hybrid workflow based on Geant4 on the CPU and AdePT on the GPU.
△ Less
Submitted 30 September, 2022;
originally announced September 2022.
-
The HEP Software Foundation Community
Authors:
Graeme A Stewart,
Peter Elmer,
Elizabeth Sexton-Kennedy
Abstract:
The HEP Software Foundation was founded in 2014 to tackle common problems of software development and sustainability for high-energy physics. In this paper we outline the motivation for the founding of the organisation and give a brief history of its development. We describe how the organisation functions today and what challenges remain to be faced in the future.
The HEP Software Foundation was founded in 2014 to tackle common problems of software development and sustainability for high-energy physics. In this paper we outline the motivation for the founding of the organisation and give a brief history of its development. We describe how the organisation functions today and what challenges remain to be faced in the future.
△ Less
Submitted 17 May, 2022;
originally announced May 2022.
-
Software and Computing for Small HEP Experiments
Authors:
Dave Casper,
Maria Elena Monzani,
Benjamin Nachman,
Costas Andreopoulos,
Stephen Bailey,
Deborah Bard,
Wahid Bhimji,
Giuseppe Cerati,
Grigorios Chachamis,
Jacob Daughhetee,
Miriam Diamond,
V. Daniel Elvira,
Alden Fan,
Krzysztof Genser,
Paolo Girotti,
Scott Kravitz,
Robert Kutschke,
Vincent R. Pascuzzi,
Gabriel N. Perdue,
Erica Snider,
Elizabeth Sexton-Kennedy,
Graeme Andrew Stewart,
Matthew Szydagis,
Eric Torrence,
Christopher Tunnell
Abstract:
This white paper briefly summarized key conclusions of the recent US Community Study on the Future of Particle Physics (Snowmass 2021) workshop on Software and Computing for Small High Energy Physics Experiments.
This white paper briefly summarized key conclusions of the recent US Community Study on the Future of Particle Physics (Snowmass 2021) workshop on Software and Computing for Small High Energy Physics Experiments.
△ Less
Submitted 27 December, 2022; v1 submitted 15 March, 2022;
originally announced March 2022.
-
HEP computing collaborations for the challenges of the next decade
Authors:
Simone Campana,
Alessandro Di Girolamo,
Paul Laycock,
Zach Marshall,
Heidi Schellman,
Graeme A Stewart
Abstract:
Large High Energy Physics (HEP) experiments adopted a distributed computing model more than a decade ago. WLCG, the global computing infrastructure for LHC, in partnership with the US Open Science Grid, has achieved data management at the many-hundred-Petabyte scale, and provides access to the entire community in a manner that is largely transparent to the end users. The main computing challenge o…
▽ More
Large High Energy Physics (HEP) experiments adopted a distributed computing model more than a decade ago. WLCG, the global computing infrastructure for LHC, in partnership with the US Open Science Grid, has achieved data management at the many-hundred-Petabyte scale, and provides access to the entire community in a manner that is largely transparent to the end users. The main computing challenge of the next decade for the LHC experiments is presented by the HL-LHC program. Other large HEP experiments, such as DUNE and Belle II, have large-scale computing needs and afford opportunities for collaboration on the same timescale. Many of the computing facilities supporting HEP experiments are shared and face common challenges, and the same is true for software libraries and services. The LHC experiments and their WLCG- partners, DUNE and Belle II, are now collaborating to evolve the computing infrastructure and services for their future needs, facilitated by the WLCG organization, OSG, the HEP Software Foundation and development projects such as HEP-CCE, IRIS-HEP and SWIFT-HEP. In this paper we outline the strategy by which the international HEP computing infrastructure, software and services should evolve through the collaboration of large and smaller scale HEP experiments, while respecting the specific needs of each community. We also highlight how the same infrastructure would be a benefit for other sciences, sharing similar needs with HEP. This proposal is in line with the OSG/WLCG strategy for addressing computing for HL-LHC and is aligned with European and other international strategies in computing for large scale science. The European Strategy for Particle Physics in 2020 agreed to the principles laid out above, in its final report.
△ Less
Submitted 14 March, 2022;
originally announced March 2022.
-
Constraints on future analysis metadata systems in High Energy Physics
Authors:
T. J. Khoo,
A. Reinsvold Hall,
N. Skidmore,
S. Alderweireldt,
J. Anders,
C. Burr,
W. Buttinger,
P. David,
L. Gouskos,
L. Gray,
S. Hageboeck,
A. Krasznahorkay,
P. Laycock,
A. Lister,
Z. Marshall,
A. B. Meyer,
T. Novak,
S. Rappoccio,
M. Ritter,
E. Rodrigues,
J. Rumsevicius,
L. Sexton-Kennedy,
N. Smith,
G. A. Stewart,
S. Wertz
Abstract:
In High Energy Physics (HEP), analysis metadata comes in many forms -- from theoretical cross-sections, to calibration corrections, to details about file processing. Correctly applying metadata is a crucial and often time-consuming step in an analysis, but designing analysis metadata systems has historically received little direct attention. Among other considerations, an ideal metadata tool shoul…
▽ More
In High Energy Physics (HEP), analysis metadata comes in many forms -- from theoretical cross-sections, to calibration corrections, to details about file processing. Correctly applying metadata is a crucial and often time-consuming step in an analysis, but designing analysis metadata systems has historically received little direct attention. Among other considerations, an ideal metadata tool should be easy to use by new analysers, should scale to large data volumes and diverse processing paradigms, and should enable future analysis reinterpretation. This document, which is the product of community discussions organised by the HEP Software Foundation, categorises types of metadata by scope and format and gives examples of current metadata solutions. Important design considerations for metadata systems, including sociological factors, analysis preservation efforts, and technical factors, are discussed. A list of best practices and technical requirements for future analysis metadata systems is presented. These best practices could guide the development of a future cross-experimental effort for analysis metadata tools.
△ Less
Submitted 19 May, 2022; v1 submitted 1 March, 2022;
originally announced March 2022.
-
HL-LHC Computing Review Stage 2, Common Software Projects: Data Science Tools for Analysis
Authors:
Jim Pivarski,
Eduardo Rodrigues,
Kevin Pedro,
Oksana Shadura,
Benjamin Krikler,
Graeme A. Stewart
Abstract:
This paper was prepared by the HEP Software Foundation (HSF) PyHEP Working Group as input to the second phase of the LHCC review of High-Luminosity LHC (HL-LHC) computing, which took place in November, 2021. It describes the adoption of Python and data science tools in HEP, discusses the likelihood of future scenarios, and recommendations for action by the HEP community.
This paper was prepared by the HEP Software Foundation (HSF) PyHEP Working Group as input to the second phase of the LHCC review of High-Luminosity LHC (HL-LHC) computing, which took place in November, 2021. It describes the adoption of Python and data science tools in HEP, discusses the likelihood of future scenarios, and recommendations for action by the HEP community.
△ Less
Submitted 4 February, 2022;
originally announced February 2022.
-
HL-LHC Computing Review Stage-2, Common Software Projects: Event Generators
Authors:
The HSF Physics Event Generator WG,
:,
Efe Yazgan,
Josh McFayden,
Andrea Valassi,
Simone Amoroso,
Enrico Bothmann,
Andy Buckley,
John Campbell,
Gurpreet Singh Chahal,
Taylor Childers,
Gloria Corti,
Rikkert Frederix,
Stefano Frixione,
Francesco Giuli,
Alexander Grohsjean,
Stefan Hoeche,
Phil Ilten,
Frank Krauss,
Michal Kreps,
David Lange,
Leif Lonnblad,
Zach Marshall,
Olivier Mattelaer,
Stephen Mrenna
, et al. (14 additional authors not shown)
Abstract:
This paper has been prepared by the HEP Software Foundation (HSF) Physics Event Generator Working Group (WG), as an input to the second phase of the LHCC review of High-Luminosity LHC (HL-LHC) computing, which is due to take place in November 2021. It complements previous documents prepared by the WG in the context of the first phase of the LHCC review in 2020, including in particular the WG paper…
▽ More
This paper has been prepared by the HEP Software Foundation (HSF) Physics Event Generator Working Group (WG), as an input to the second phase of the LHCC review of High-Luminosity LHC (HL-LHC) computing, which is due to take place in November 2021. It complements previous documents prepared by the WG in the context of the first phase of the LHCC review in 2020, including in particular the WG paper on the specific challenges in Monte Carlo event generator software for HL-LHC, which has since been updated and published, and which we are also submitting to the November 2021 review as an integral part of our contribution.
△ Less
Submitted 30 September, 2021;
originally announced September 2021.
-
Learning from the Pandemic: the Future of Meetings in HEP and Beyond
Authors:
Mark S. Neubauer,
Todd Adams,
Jennifer Adelman-McCarthy,
Gabriele Benelli,
Tulika Bose,
David Britton,
Pat Burchat,
Joel Butler,
Timothy A. Cartwright,
Tomáš Davídek,
Jacques Dumarchez,
Peter Elmer,
Matthew Feickert,
Ben Galewsky,
Mandeep Gill,
Maciej Gladki,
Aman Goel,
Jonathan E. Guyer,
Bo Jayatilaka,
Brendan Kiburg,
Benjamin Krikler,
David Lange,
Claire Lee,
Nick Manganelli,
Giovanni Marchiori
, et al. (14 additional authors not shown)
Abstract:
The COVID-19 pandemic has by-and-large prevented in-person meetings since March 2020. While the increasing deployment of effective vaccines around the world is a very positive development, the timeline and pathway to "normality" is uncertain and the "new normal" we will settle into is anyone's guess. Particle physics, like many other scientific fields, has more than a year of experience in holding…
▽ More
The COVID-19 pandemic has by-and-large prevented in-person meetings since March 2020. While the increasing deployment of effective vaccines around the world is a very positive development, the timeline and pathway to "normality" is uncertain and the "new normal" we will settle into is anyone's guess. Particle physics, like many other scientific fields, has more than a year of experience in holding virtual meetings, workshops, and conferences. A great deal of experimentation and innovation to explore how to execute these meetings effectively has occurred. Therefore, it is an appropriate time to take stock of what we as a community learned from running virtual meetings and discuss possible strategies for the future. Continuing to develop effective strategies for meetings with a virtual component is likely to be important for reducing the carbon footprint of our research activities, while also enabling greater diversity and inclusion for participation. This report summarizes a virtual two-day workshop on Virtual Meetings held May 5-6, 2021 which brought together experts from both inside and outside of high-energy physics to share their experiences and practices with organizing and executing virtual workshops, and to develop possible strategies for future meetings as we begin to emerge from the COVID-19 pandemic. This report outlines some of the practices and tools that have worked well which we hope will serve as a valuable resource for future virtual meeting organizers in all scientific fields.
△ Less
Submitted 29 June, 2021;
originally announced June 2021.
-
Software Training in HEP
Authors:
Sudhir Malik,
Samuel Meehan,
Kilian Lieret,
Meirin Oan Evans,
Michel H. Villanueva,
Daniel S. Katz,
Graeme A. Stewart,
Peter Elmer,
Sizar Aziz,
Matthew Bellis,
Riccardo Maria Bianchi,
Gianluca Bianco,
Johan Sebastian Bonilla,
Angela Burger,
Jackson Burzynski,
David Chamont,
Matthew Feickert,
Philipp Gadow,
Bernhard Manfred Gruber,
Daniel Guest,
Stephan Hageboeck,
Lukas Heinrich,
Maximilian M. Horzela,
Marc Huwiler,
Clemens Lange
, et al. (22 additional authors not shown)
Abstract:
Long term sustainability of the high energy physics (HEP) research software ecosystem is essential for the field. With upgrades and new facilities coming online throughout the 2020s this will only become increasingly relevant throughout this decade. Meeting this sustainability challenge requires a workforce with a combination of HEP domain knowledge and advanced software skills. The required softw…
▽ More
Long term sustainability of the high energy physics (HEP) research software ecosystem is essential for the field. With upgrades and new facilities coming online throughout the 2020s this will only become increasingly relevant throughout this decade. Meeting this sustainability challenge requires a workforce with a combination of HEP domain knowledge and advanced software skills. The required software skills fall into three broad groups. The first is fundamental and generic software engineering (e.g. Unix, version control,C++, continuous integration). The second is knowledge of domain specific HEP packages and practices (e.g., the ROOT data format and analysis framework). The third is more advanced knowledge involving more specialized techniques. These include parallel programming, machine learning and data science tools, and techniques to preserve software projects at all scales. This paper dis-cusses the collective software training program in HEP and its activities led by the HEP Software Foundation (HSF) and the Institute for Research and Innovation in Software in HEP (IRIS-HEP). The program equips participants with an array of software skills that serve as ingredients from which solutions to the computing challenges of HEP can be formed. Beyond serving the community by ensuring that members are able to pursue research goals, this program serves individuals by providing intellectual capital and transferable skills that are becoming increasingly important to careers in the realm of software and computing, whether inside or outside HEP
△ Less
Submitted 6 August, 2021; v1 submitted 28 February, 2021;
originally announced March 2021.
-
Software Sustainability & High Energy Physics
Authors:
Daniel S. Katz,
Sudhir Malik,
Mark S. Neubauer,
Graeme A. Stewart,
Kétévi A. Assamagan,
Erin A. Becker,
Neil P. Chue Hong,
Ian A. Cosden,
Samuel Meehan,
Edward J. W. Moyse,
Adrian M. Price-Whelan,
Elizabeth Sexton-Kennedy,
Meirin Oan Evans,
Matthew Feickert,
Clemens Lange,
Kilian Lieret,
Rob Quick,
Arturo Sánchez Pineda,
Christopher Tunnell
Abstract:
New facilities of the 2020s, such as the High Luminosity Large Hadron Collider (HL-LHC), will be relevant through at least the 2030s. This means that their software efforts and those that are used to analyze their data need to consider sustainability to enable their adaptability to new challenges, longevity, and efficiency, over at least this period. This will help ensure that this software will b…
▽ More
New facilities of the 2020s, such as the High Luminosity Large Hadron Collider (HL-LHC), will be relevant through at least the 2030s. This means that their software efforts and those that are used to analyze their data need to consider sustainability to enable their adaptability to new challenges, longevity, and efficiency, over at least this period. This will help ensure that this software will be easier to develop and maintain, that it remains available in the future on new platforms, that it meets new needs, and that it is as reusable as possible. This report discusses a virtual half-day workshop on "Software Sustainability and High Energy Physics" that aimed 1) to bring together experts from HEP as well as those from outside to share their experiences and practices, and 2) to articulate a vision that helps the Institute for Research and Innovation in Software for High Energy Physics (IRIS-HEP) to create a work plan to implement elements of software sustainability. Software sustainability practices could lead to new collaborations, including elements of HEP software being directly used outside the field, and, as has happened more frequently in recent years, to HEP developers contributing to software developed outside the field rather than reinventing it. A focus on and skills related to sustainable software will give HEP software developers an important skill that is essential to careers in the realm of software, inside or outside HEP. The report closes with recommendations to improve software sustainability in HEP, aimed at the HEP community via IRIS-HEP and the HEP Software Foundation (HSF).
△ Less
Submitted 16 October, 2020; v1 submitted 10 October, 2020;
originally announced October 2020.
-
HL-LHC Computing Review: Common Tools and Community Software
Authors:
HEP Software Foundation,
:,
Thea Aarrestad,
Simone Amoroso,
Markus Julian Atkinson,
Joshua Bendavid,
Tommaso Boccali,
Andrea Bocci,
Andy Buckley,
Matteo Cacciari,
Paolo Calafiura,
Philippe Canal,
Federico Carminati,
Taylor Childers,
Vitaliano Ciulli,
Gloria Corti,
Davide Costanzo,
Justin Gage Dezoort,
Caterina Doglioni,
Javier Mauricio Duarte,
Agnieszka Dziurda,
Peter Elmer,
Markus Elsing,
V. Daniel Elvira,
Giulio Eulisse
, et al. (85 additional authors not shown)
Abstract:
Common and community software packages, such as ROOT, Geant4 and event generators have been a key part of the LHC's success so far and continued development and optimisation will be critical in the future. The challenges are driven by an ambitious physics programme, notably the LHC accelerator upgrade to high-luminosity, HL-LHC, and the corresponding detector upgrades of ATLAS and CMS. In this doc…
▽ More
Common and community software packages, such as ROOT, Geant4 and event generators have been a key part of the LHC's success so far and continued development and optimisation will be critical in the future. The challenges are driven by an ambitious physics programme, notably the LHC accelerator upgrade to high-luminosity, HL-LHC, and the corresponding detector upgrades of ATLAS and CMS. In this document we address the issues for software that is used in multiple experiments (usually even more widely than ATLAS and CMS) and maintained by teams of developers who are either not linked to a particular experiment or who contribute to common software within the context of their experiment activity. We also give space to general considerations for future software and projects that tackle upcoming challenges, no matter who writes it, which is an area where community convergence on best practice is extremely useful.
△ Less
Submitted 31 August, 2020;
originally announced August 2020.
-
Challenges in Monte Carlo event generator software for High-Luminosity LHC
Authors:
The HSF Physics Event Generator WG,
:,
Andrea Valassi,
Efe Yazgan,
Josh McFayden,
Simone Amoroso,
Joshua Bendavid,
Andy Buckley,
Matteo Cacciari,
Taylor Childers,
Vitaliano Ciulli,
Rikkert Frederix,
Stefano Frixione,
Francesco Giuli,
Alexander Grohsjean,
Christian Gütschow,
Stefan Höche,
Walter Hopkins,
Philip Ilten,
Dmitri Konstantinov,
Frank Krauss,
Qiang Li,
Leif Lönnblad,
Fabio Maltoni,
Michelangelo Mangano
, et al. (16 additional authors not shown)
Abstract:
We review the main software and computing challenges for the Monte Carlo physics event generators used by the LHC experiments, in view of the High-Luminosity LHC (HL-LHC) physics programme. This paper has been prepared by the HEP Software Foundation (HSF) Physics Event Generator Working Group as an input to the LHCC review of HL-LHC computing, which has started in May 2020.
We review the main software and computing challenges for the Monte Carlo physics event generators used by the LHC experiments, in view of the High-Luminosity LHC (HL-LHC) physics programme. This paper has been prepared by the HEP Software Foundation (HSF) Physics Event Generator Working Group as an input to the LHCC review of HL-LHC computing, which has started in May 2020.
△ Less
Submitted 18 February, 2021; v1 submitted 28 April, 2020;
originally announced April 2020.
-
HEP Software Foundation Community White Paper Working Group - Data Processing Frameworks
Authors:
Paolo Calafiura,
Marco Clemencic,
Hadrien Grasland,
Chris Green,
Benedikt Hegner,
Chris Jones,
Michel Jouvin,
Kyle Knoepfel,
Thomas Kuhr,
Jim Kowalkowski,
Charles Leggett,
Adam Lyon,
David Malon,
Marc Paterno,
Simon Patton,
Elizabeth Sexton-Kennedy,
Graeme A Stewart,
Vakho Tsulaia
Abstract:
Data processing frameworks are an essential part of HEP experiments' software stacks. Frameworks provide a means by which code developers can undertake the essential tasks of physics data processing, accessing relevant inputs and storing their outputs, in a coherent way without needing to know the details of other domains. Frameworks provide essential core services for developers and help deliver…
▽ More
Data processing frameworks are an essential part of HEP experiments' software stacks. Frameworks provide a means by which code developers can undertake the essential tasks of physics data processing, accessing relevant inputs and storing their outputs, in a coherent way without needing to know the details of other domains. Frameworks provide essential core services for developers and help deliver a configurable working application to the experiments' production systems. Modern HEP processing frameworks are in the process of adapting to a new computing landscape dominated by parallel processing and heterogeneity, which pose many questions regarding enhanced functionality and scaling that must be faced without compromising the maintainability of the code. In this paper we identify a program of work that can help further clarify the key concepts of frameworks for HEP and then spawn R&D activities that can focus the community's efforts in the most efficient manner to address the challenges of the upcoming experimental program.
△ Less
Submitted 2 May, 2019; v1 submitted 19 December, 2018;
originally announced December 2018.
-
HEP Software Foundation Community White Paper Working Group - Training, Staffing and Careers
Authors:
HEP Software Foundation,
:,
Dario Berzano,
Riccardo Maria Bianchi,
Peter Elmer,
Sergei V. Gleyzer John Harvey,
Roger Jones,
Michel Jouvin,
Daniel S. Katz,
Sudhir Malik,
Dario Menasce,
Mark Neubauer,
Fernanda Psihas,
Albert Puig Navarro,
Graeme A. Stewart,
Christopher Tunnell,
Justin A. Vasel,
Sean-Jiun Wang
Abstract:
The rapid evolution of technology and the parallel increasing complexity of algorithmic analysis in HEP requires developers to acquire a much larger portfolio of programming skills. Young researchers graduating from universities worldwide currently do not receive adequate preparation in the very diverse fields of modern computing to respond to growing needs of the most advanced experimental challe…
▽ More
The rapid evolution of technology and the parallel increasing complexity of algorithmic analysis in HEP requires developers to acquire a much larger portfolio of programming skills. Young researchers graduating from universities worldwide currently do not receive adequate preparation in the very diverse fields of modern computing to respond to growing needs of the most advanced experimental challenges. There is a growing consensus in the HEP community on the need for training programmes to bring researchers up to date with new software technologies, in particular in the domains of concurrent programming and artificial intelligence. We review some of the initiatives under way for introducing new training programmes and highlight some of the issues that need to be taken into account for these to be successful.
△ Less
Submitted 17 January, 2019; v1 submitted 8 July, 2018;
originally announced July 2018.
-
HEP Software Foundation Community White Paper Working Group - Software Development, Deployment and Validation
Authors:
Benjamin Couturier,
Giulio Eulisse,
Hadrien Grasland,
Benedikt Hegner,
Michel Jouvin,
Meghan Kane,
Daniel S. Katz,
Thomas Kuhr,
David Lange,
Patricia Mendez Lorenzo,
Martin Ritter,
Graeme Andrew Stewart,
Andrea Valassi
Abstract:
The High Energy Phyiscs community has developed and needs to maintain many tens of millions of lines of code and to integrate effectively the work of thousands of developers across large collaborations. Software needs to be built, validated, and deployed across hundreds of sites. Software also has a lifetime of many years, frequently beyond that of the original developer, it must be developed with…
▽ More
The High Energy Phyiscs community has developed and needs to maintain many tens of millions of lines of code and to integrate effectively the work of thousands of developers across large collaborations. Software needs to be built, validated, and deployed across hundreds of sites. Software also has a lifetime of many years, frequently beyond that of the original developer, it must be developed with sustainability in mind. Adequate recognition of software development as a critical task in the HEP community needs to be fostered and an appropriate publication and citation strategy needs to be developed. As part of the HEP Softare Foundation's Community White Paper process a working group on Software Development, Deployment and Validation was formed to examine all of these issues, identify best practice and to formulare recommendations for the next decade. Its report is presented here.
△ Less
Submitted 15 June, 2018; v1 submitted 21 December, 2017;
originally announced December 2017.
-
A Roadmap for HEP Software and Computing R&D for the 2020s
Authors:
Johannes Albrecht,
Antonio Augusto Alves Jr,
Guilherme Amadio,
Giuseppe Andronico,
Nguyen Anh-Ky,
Laurent Aphecetche,
John Apostolakis,
Makoto Asai,
Luca Atzori,
Marian Babik,
Giuseppe Bagliesi,
Marilena Bandieramonte,
Sunanda Banerjee,
Martin Barisits,
Lothar A. T. Bauerdick,
Stefano Belforte,
Douglas Benjamin,
Catrin Bernius,
Wahid Bhimji,
Riccardo Maria Bianchi,
Ian Bird,
Catherine Biscarat,
Jakob Blomer,
Kenneth Bloom,
Tommaso Boccali
, et al. (285 additional authors not shown)
Abstract:
Particle physics has an ambitious and broad experimental programme for the coming decades. This programme requires large investments in detector hardware, either to build new facilities and experiments, or to upgrade existing ones. Similarly, it requires commensurate investment in the R&D of software to acquire, manage, process, and analyse the shear amounts of data to be recorded. In planning for…
▽ More
Particle physics has an ambitious and broad experimental programme for the coming decades. This programme requires large investments in detector hardware, either to build new facilities and experiments, or to upgrade existing ones. Similarly, it requires commensurate investment in the R&D of software to acquire, manage, process, and analyse the shear amounts of data to be recorded. In planning for the HL-LHC in particular, it is critical that all of the collaborating stakeholders agree on the software goals and priorities, and that the efforts complement each other. In this spirit, this white paper describes the R&D activities required to prepare for this software upgrade.
△ Less
Submitted 19 December, 2018; v1 submitted 18 December, 2017;
originally announced December 2017.
-
Crystal structure and magnetic modulation in beta-Ce2O2FeSe2
Authors:
Chun-Hai Wang,
C. M. Ainsworth,
S. D. Champion,
G. A. Stewart,
M. C. Worsdale,
T. Lancaster,
S. J. Blundell,
Helen E. A. Brand,
John S. O. Evans
Abstract:
We report a combination of X-ray and neutron diffraction studies, Mossbauer spectroscopy and muon spin relaxation (muSR) measurements to probe the structure and magnetic properties of the semiconducting beta-Ce2O2FeSe2 oxychalcogenide. We report a new structural description in space group Pna21 which is consistent with diffraction data and second harmonic generation measurements and reveal an orde…
▽ More
We report a combination of X-ray and neutron diffraction studies, Mossbauer spectroscopy and muon spin relaxation (muSR) measurements to probe the structure and magnetic properties of the semiconducting beta-Ce2O2FeSe2 oxychalcogenide. We report a new structural description in space group Pna21 which is consistent with diffraction data and second harmonic generation measurements and reveal an order-disorder transition on one Fe site at TOD ~ 330 K. Susceptibility measurements, Mossbauer and muSR reveal antiferromagnetic ordering below TN = 86 K and more complex short range order above this temperature. 12 K neutron diffraction data reveal a modulated magnetic structure with q = 0.444 bN*.
△ Less
Submitted 4 July, 2017;
originally announced July 2017.
-
Automating ATLAS Computing Operations using the Site Status Board
Authors:
Julia Andreeva,
Carlos Borrego Iglesias,
Simone Campana,
Alessandro Di Girolamo,
Ivan Dzhunov,
Xavier Espinal Curull,
Stavro Gayazov,
Erekle Magradze,
Michal Maciej Nowotka,
Lorenzo Rinaldi,
Pablo Saiz,
Jaroslava Schovancova,
Graeme Andrew Stewart,
Michael Wright
Abstract:
The automation of operations is essential to reduce manpower costs and improve the reliability of the system. The Site Status Board (SSB) is a framework which allows Virtual Organizations to monitor their computing activities at distributed sites and to evaluate site performance. The ATLAS experiment intensively uses the SSB for the distributed computing shifts, for estimating data processing and…
▽ More
The automation of operations is essential to reduce manpower costs and improve the reliability of the system. The Site Status Board (SSB) is a framework which allows Virtual Organizations to monitor their computing activities at distributed sites and to evaluate site performance. The ATLAS experiment intensively uses the SSB for the distributed computing shifts, for estimating data processing and data transfer efficiencies at a particular site, and for implementing automatic exclusion of sites from computing activities, in case of potential problems. The ATLAS SSB provides a real-time aggregated monitoring view and keeps the history of the monitoring metrics. Based on this history, usability of a site from the perspective of ATLAS is calculated. The paper will describe how the SSB is integrated in the ATLAS operations and computing infrastructure and will cover implementation details of the ATLAS SSB sensors and alarm system, based on the information in the SSB. It will demonstrate the positive impact of the use of the SSB on the overall performance of ATLAS computing activities and will overview future plans.
△ Less
Submitted 28 January, 2013; v1 submitted 1 January, 2013;
originally announced January 2013.
-
Unified storage systems for distributed Tier-2 centres
Authors:
Greig A. Cowan,
Graeme A. Stewart,
Andrew Elwell
Abstract:
The start of data taking at the Large Hadron Collider will herald a new era in data volumes and distributed processing in particle physics. Data volumes of hundreds of Terabytes will be shipped to Tier-2 centres for analysis by the LHC experiments using the Worldwide LHC Computing Grid (WLCG).
In many countries Tier-2 centres are distributed between a number of institutes, e.g., the geographic…
▽ More
The start of data taking at the Large Hadron Collider will herald a new era in data volumes and distributed processing in particle physics. Data volumes of hundreds of Terabytes will be shipped to Tier-2 centres for analysis by the LHC experiments using the Worldwide LHC Computing Grid (WLCG).
In many countries Tier-2 centres are distributed between a number of institutes, e.g., the geographically spread Tier-2s of GridPP in the UK. This presents a number of challenges for experiments to utilise these centres efficaciously, as CPU and storage resources may be sub-divided and exposed in smaller units than the experiment would ideally want to work with. In addition, unhelpful mismatches between storage and CPU at the individual centres may be seen, which make efficient exploitation of a Tier-2's resources difficult.
One method of addressing this is to unify the storage across a distributed Tier-2, presenting the centres' aggregated storage as a single system. This greatly simplifies data management for the VO, which then can access a greater amount of data across the Tier-2. However, such an approach will lead to scenarios where analysis jobs on one site's batch system must access data hosted on another site.
We investigate this situation using the Glasgow and Edinburgh clusters, which are part of the ScotGrid distributed Tier-2. In particular we look at how to mitigate the problems associated with ``distant'' data access and discuss the security implications of having LAN access protocols traverse the WAN between centres.
△ Less
Submitted 28 March, 2008;
originally announced March 2008.
-
Superconducting transition temperature of MgB_2 H_0.03 is higher than that of MgB_2
Authors:
V. V. Flambaum,
G. A. Stewart,
G. J. Russell,
J. Horvat,
S. X. Dou
Abstract:
Hydrogenation of MgB_2 powder has lead to an increase in the superconducting temperature, as determined by ac susceptibility. Applied dc fields reduce the transition temperature in the same ratio as for the pure powder.
Hydrogenation of MgB_2 powder has lead to an increase in the superconducting temperature, as determined by ac susceptibility. Applied dc fields reduce the transition temperature in the same ratio as for the pure powder.
△ Less
Submitted 17 December, 2001;
originally announced December 2001.
