Ganga: a tool for computational-task management and easy access to Grid resources
Authors:
J. T. MoĊcicki,
F. Brochu,
J. Ebke,
U. Egede,
J. Elmsheuser,
K. Harrison,
R. W. L. Jones,
H. C. Lee,
D. Liko,
A. Maier,
A. Muraru,
G. N. Patrick,
K. Pajchel,
W. Reece,
B. H. Samset,
M. W. Slater,
A. Soroko,
C. L. Tan,
D. C. Vanderster,
M. Williams
Abstract:
In this paper, we present the computational task-management tool Ganga, which allows for the specification, submission, bookkee** and post-processing of computational tasks on a wide set of distributed resources. Ganga has been developed to solve a problem increasingly common in scientific projects, which is that researchers must regularly switch between different processing systems, each with…
▽ More
In this paper, we present the computational task-management tool Ganga, which allows for the specification, submission, bookkee** and post-processing of computational tasks on a wide set of distributed resources. Ganga has been developed to solve a problem increasingly common in scientific projects, which is that researchers must regularly switch between different processing systems, each with its own command set, to complete their computational tasks. Ganga provides a homogeneous environment for processing data on heterogeneous resources. We give examples from High Energy Physics, demonstrating how an analysis can be developed on a local system and then transparently moved to a Grid system for processing of all available data. Ganga has an API that can be used via an interactive interface, in scripts, or through a GUI. Specific knowledge about types of tasks or computational resources is provided at run-time through a plugin system, making new developments easy to integrate. We give an overview of the Ganga architecture, give examples of current use, and demonstrate how Ganga can be used in many different areas of science.
△ Less
Submitted 9 June, 2009; v1 submitted 16 February, 2009;
originally announced February 2009.
An on-line Integrated Bookkee**: electronic run log book and Meta-Data Repository for ATLAS
Authors:
M. Barczyc,
D. Burckhart-Chromek,
M. Caprini,
J. Da Silva Conceicao,
M. Dobson,
J. Flammer,
R. Jones,
A. Kazarov,
S. Kolos,
D. Liko,
L. Mapelli,
I. Soloviev,
R. Hart NIKHEF,
A. Amorim,
D. Klose,
J. Lima,
L. Lucio,
L. Pedro,
H. Wolters,
E. Badescu NIPNE,
I. Alexandrov,
V. Kotov,
M. Mineev JINR,
Yu. Ryabov PNPI
Abstract:
In the context of the ATLAS experiment there is growing evidence of the importance of different kinds of Meta-data including all the important details of the detector and data acquisition that are vital for the analysis of the acquired data. The Online BookKeeper (OBK) is a component of ATLAS online software that stores all information collected while running the experiment, including the Meta-d…
▽ More
In the context of the ATLAS experiment there is growing evidence of the importance of different kinds of Meta-data including all the important details of the detector and data acquisition that are vital for the analysis of the acquired data. The Online BookKeeper (OBK) is a component of ATLAS online software that stores all information collected while running the experiment, including the Meta-data associated with the event acquisition, triggering and storage. The facilities for acquisition of control data within the on-line software framework, together with a full functional Web interface, make the OBK a powerful tool containing all information needed for event analysis, including an electronic log book.
In this paper we explain how OBK plays a role as one of the main collectors and managers of Meta-data produced on-line, and we'll also focus on the Web facilities already available. The usage of the web interface as an electronic run logbook is also explained, together with the future extensions.
We describe the technology used in OBK development and how we arrived at the present level explaining the previous experience with various DBMS technologies. The extensive performance evaluations that have been performed and the usage in the production environment of the ATLAS test beams are also analysed.
△ Less
Submitted 13 June, 2003;
originally announced June 2003.
