Using HEP experiment workflows for the benchmarking and accounting of WLCG computing resources
Authors:
Andrea Valassi,
Manfred Alef,
Jean-Michel Barbet,
Olga Datskova,
Riccardo De Maria,
Miguel Fontes Medeiros,
Domenico Giordano,
Costin Grigoras,
Christopher Hollowell,
Martina Javurkova,
Viktor Khristenko,
David Lange,
Michele Michelotto,
Lorenzo Rinaldi,
Andrea Sciabà,
Cas Van Der Laan
Abstract:
Benchmarking of CPU resources in WLCG has been based on the HEP-SPEC06 (HS06) suite for over a decade. It has recently become clear that HS06, which is based on real applications from non-HEP domains, no longer describes typical HEP workloads. The aim of the HEP-Benchmarks project is to develop a new benchmark suite for WLCG compute resources, based on real applications from the LHC experiments. B…
▽ More
Benchmarking of CPU resources in WLCG has been based on the HEP-SPEC06 (HS06) suite for over a decade. It has recently become clear that HS06, which is based on real applications from non-HEP domains, no longer describes typical HEP workloads. The aim of the HEP-Benchmarks project is to develop a new benchmark suite for WLCG compute resources, based on real applications from the LHC experiments. By construction, these new benchmarks are thus guaranteed to have a score highly correlated to the throughputs of HEP applications, and a CPU usage pattern similar to theirs. Linux containers and the CernVM-FS filesystem are the two main technologies enabling this approach, which had been considered impossible in the past. In this paper, we review the motivation, implementation and outlook of the new benchmark suite.
△ Less
Submitted 25 June, 2020; v1 submitted 3 April, 2020;
originally announced April 2020.
HEP Applications Evaluation of the EDG Testbed and Middleware
Authors:
I. Augustin,
F. Carminati,
J. Closier,
E. van Herwijnen,
J. J. Blaising,
D. Boutigny,
C. Charlot,
V. Garonne,
A. Tsaregorodtsev,
K. Bos,
J. Templon,
P. Capiluppi,
A. Fanfani,
R. Barbera,
G. Negri,
L. Perini,
S. Resconi,
M. Sitta,
M. Reale,
D. Vicinanza,
S. Bagnasco,
P. Cerello,
A. Sciaba,
O. Smirnova,
D. Colling
, et al. (2 additional authors not shown)
Abstract:
Workpackage 8 of the European Datagrid project was formed in January 2001 with representatives from the four LHC experiments, and with experiment independent people from five of the six main EDG partners. In September 2002 WP8 was strengthened by the addition of effort from BaBar and D0. The original mandate of WP8 was, following the definition of short- and long-term requirements, to port exper…
▽ More
Workpackage 8 of the European Datagrid project was formed in January 2001 with representatives from the four LHC experiments, and with experiment independent people from five of the six main EDG partners. In September 2002 WP8 was strengthened by the addition of effort from BaBar and D0. The original mandate of WP8 was, following the definition of short- and long-term requirements, to port experiment software to the EDG middleware and testbed environment. A major additional activity has been testing the basic functionality and performance of this environment. This paper reviews experiences and evaluations in the areas of job submission, data management, mass storage handling, information systems and monitoring. It also comments on the problems of remote debugging, the portability of code, and scaling problems with increasing numbers of jobs, sites and nodes. Reference is made to the pioneeering work of Atlas and CMS in integrating the use of the EDG Testbed into their data challenges. A forward look is made to essential software developments within EDG and to the necessary cooperation between EDG and LCG for the LCG prototype due in mid 2003.
△ Less
Submitted 5 June, 2003;
originally announced June 2003.
