-
The U.S. CMS HL-LHC R&D Strategic Plan
Authors:
Oliver Gutsche,
Tulika Bose,
Margaret Votava,
David Mason,
Andrew Melo,
Mia Liu,
Dirk Hufnagel,
Lindsey Gray,
Mike Hildreth,
Burt Holzman,
Kevin Lannon,
Saba Sehrish,
David Sperka,
James Letts,
Lothar Bauerdick,
Kenneth Bloom
Abstract:
The HL-LHC run is anticipated to start at the end of this decade and will pose a significant challenge for the scale of the HEP software and computing infrastructure. The mission of the U.S. CMS Software & Computing Operations Program is to develop and operate the software and computing resources necessary to process CMS data expeditiously and to enable U.S. physicists to fully participate in the…
▽ More
The HL-LHC run is anticipated to start at the end of this decade and will pose a significant challenge for the scale of the HEP software and computing infrastructure. The mission of the U.S. CMS Software & Computing Operations Program is to develop and operate the software and computing resources necessary to process CMS data expeditiously and to enable U.S. physicists to fully participate in the physics of CMS. We have developed a strategic plan to prioritize R&D efforts to reach this goal for the HL-LHC. This plan includes four grand challenges: modernizing physics software and improving algorithms, building infrastructure for exabyte-scale datasets, transforming the scientific data analysis process and transitioning from R&D to operations. We are involved in a variety of R&D projects that fall within these grand challenges. In this talk, we will introduce our four grand challenges and outline the R&D program of the U.S. CMS Software & Computing Operations Program.
△ Less
Submitted 4 December, 2023; v1 submitted 1 December, 2023;
originally announced December 2023.
-
US ATLAS and US CMS HPC and Cloud Blueprint
Authors:
Fernando Barreiro Megino,
Lincoln Bryant,
Dirk Hufnagel,
Kenyi Hurtado Anampa
Abstract:
The Large Hadron Collider (LHC) at CERN houses two general purpose detectors - ATLAS and CMS - which conduct physics programs over multi-year runs to generate increasingly precise and extensive datasets. The efforts of the CMS and ATLAS collaborations lead to the discovery of the Higgs boson, a fundamental particle that gives mass to other particles, representing a monumental achievement in the fi…
▽ More
The Large Hadron Collider (LHC) at CERN houses two general purpose detectors - ATLAS and CMS - which conduct physics programs over multi-year runs to generate increasingly precise and extensive datasets. The efforts of the CMS and ATLAS collaborations lead to the discovery of the Higgs boson, a fundamental particle that gives mass to other particles, representing a monumental achievement in the field of particle physics that was recognized with the awarding of the Nobel Prize in Physics in 2013 to François Englert and Peter Higgs. These collaborations continue to analyze data from the LHC and are preparing for the high luminosity data taking phase at the end of the decade. The computing models of these detectors rely on a distributed processing grid hosted by more than 150 associated universities and laboratories worldwide. However, such new data will require a significant expansion of the existing computing infrastructure. To address this, both collaborations have been working for years on integrating High Performance Computers (HPC) and commercial cloud resources into their infrastructure and continue to assess the potential role of such resources in order to cope with the demands of the new high luminosity era. US ATLAS and US CMS computing management have charged the authors to provide a blueprint document looking at current and possibly future use of HPC and Cloud resources, outlining integration models, possibilities, challenges and costs. The document will address key questions such as the optimal use of resources for the experiments and funding agencies, the main obstacles that need to be overcome for resource adoption, and areas that require more attention.
△ Less
Submitted 14 April, 2023;
originally announced April 2023.
-
HEPCloud, a New Paradigm for HEP Facilities: CMS Amazon Web Services Investigation
Authors:
Burt Holzman,
Lothar A. T. Bauerdick,
Brian Bockelman,
Dave Dykstra,
Ian Fisk,
Stuart Fuess,
Gabriele Garzoglio,
Maria Girone,
Oliver Gutsche,
Dirk Hufnagel,
Hyunwoo Kim,
Robert Kennedy,
Nicolo Magini,
David Mason,
Panagiotis Spentzouris,
Anthony Tiradani,
Steve Timm,
Eric W. Vaandering
Abstract:
Historically, high energy physics computing has been performed on large purpose-built computing systems. These began as single-site compute facilities, but have evolved into the distributed computing grids used today. Recently, there has been an exponential increase in the capacity and capability of commercial clouds. Cloud resources are highly virtualized and intended to be able to be flexibly de…
▽ More
Historically, high energy physics computing has been performed on large purpose-built computing systems. These began as single-site compute facilities, but have evolved into the distributed computing grids used today. Recently, there has been an exponential increase in the capacity and capability of commercial clouds. Cloud resources are highly virtualized and intended to be able to be flexibly deployed for a variety of computing tasks. There is a growing nterest among the cloud providers to demonstrate the capability to perform large-scale scientific computing. In this paper, we discuss results from the CMS experiment using the Fermilab HEPCloud facility, which utilized both local Fermilab resources and virtual machines in the Amazon Web Services Elastic Compute Cloud. We discuss the planning, technical challenges, and lessons learned involved in performing physics workflows on a large-scale set of virtualized resources. In addition, we will discuss the economics and operational efficiencies when executing workflows both in the cloud and on dedicated resources.
△ Less
Submitted 29 September, 2017;
originally announced October 2017.
-
The Physics of the B Factories
Authors:
A. J. Bevan,
B. Golob,
Th. Mannel,
S. Prell,
B. D. Yabsley,
K. Abe,
H. Aihara,
F. Anulli,
N. Arnaud,
T. Aushev,
M. Beneke,
J. Beringer,
F. Bianchi,
I. I. Bigi,
M. Bona,
N. Brambilla,
J. B rodzicka,
P. Chang,
M. J. Charles,
C. H. Cheng,
H. -Y. Cheng,
R. Chistov,
P. Colangelo,
J. P. Coleman,
A. Drutskoy
, et al. (2009 additional authors not shown)
Abstract:
This work is on the Physics of the B Factories. Part A of this book contains a brief description of the SLAC and KEK B Factories as well as their detectors, BaBar and Belle, and data taking related issues. Part B discusses tools and methods used by the experiments in order to obtain results. The results themselves can be found in Part C.
Please note that version 3 on the archive is the auxiliary…
▽ More
This work is on the Physics of the B Factories. Part A of this book contains a brief description of the SLAC and KEK B Factories as well as their detectors, BaBar and Belle, and data taking related issues. Part B discusses tools and methods used by the experiments in order to obtain results. The results themselves can be found in Part C.
Please note that version 3 on the archive is the auxiliary version of the Physics of the B Factories book. This uses the notation alpha, beta, gamma for the angles of the Unitarity Triangle. The nominal version uses the notation phi_1, phi_2 and phi_3. Please cite this work as Eur. Phys. J. C74 (2014) 3026.
△ Less
Submitted 31 October, 2015; v1 submitted 24 June, 2014;
originally announced June 2014.
-
CMS Workflow Execution using Intelligent Job Scheduling and Data Access Strategies
Authors:
Khawar Hasham,
Antonio Delgado Peris,
Ashiq Anjum,
Dave Evans,
Dirk Hufnagel,
Eduardo Huedo,
José M. Hernández,
Richard McClatchey,
Stephen Gowdy,
Simon Metson
Abstract:
Complex scientific workflows can process large amounts of data using thousands of tasks. The turnaround times of these workflows are often affected by various latencies such as the resource discovery, scheduling and data access latencies for the individual workflow processes or actors. Minimizing these latencies will improve the overall execution time of a workflow and thus lead to a more efficien…
▽ More
Complex scientific workflows can process large amounts of data using thousands of tasks. The turnaround times of these workflows are often affected by various latencies such as the resource discovery, scheduling and data access latencies for the individual workflow processes or actors. Minimizing these latencies will improve the overall execution time of a workflow and thus lead to a more efficient and robust processing environment. In this paper, we propose a pilot job based infrastructure that has intelligent data reuse and job execution strategies to minimize the scheduling, queuing, execution and data access latencies. The results have shown that significant improvements in the overall turnaround time of a workflow can be achieved with this approach. The proposed approach has been evaluated, first using the CMS Tier0 data processing workflow, and then simulating the workflows to evaluate its effectiveness in a controlled environment.
△ Less
Submitted 24 February, 2012;
originally announced February 2012.
