-
CHIPP: INAF pilot project for HTC, HPC and HPDA
Authors:
Giuliano Taffoni,
Ugo Becciani,
Bianca Garilli,
Gianmarco Maggio,
Fabio Pasian,
Grazia Umana,
Riccardo Smareglia,
Fabio Vitello
Abstract:
CHIPP (Computing HTC in INAF Pilot Project) is an Italian project funded by the Italian Institute for Astrophysics (INAF) and promoted by the ICT office of INAF. The main purpose of the CHIPP project is to coordinate the use of, and access to, already existing high throughput computing and high-performance computing and data processing resources (for small/medium size programs) for the INAF commun…
▽ More
CHIPP (Computing HTC in INAF Pilot Project) is an Italian project funded by the Italian Institute for Astrophysics (INAF) and promoted by the ICT office of INAF. The main purpose of the CHIPP project is to coordinate the use of, and access to, already existing high throughput computing and high-performance computing and data processing resources (for small/medium size programs) for the INAF community. Today, Tier2/Tier3 systems (1,200 CPU/core) are provided at the INAF institutes at Trieste and Catania, but in the future, the project will evolve including also other computing infrastructures. During the last two years, more than 30 programs have been approved for a total request of 30 Million CPU-h. Most of the programs are HPC, data reduction and analysis, machine learning. In this paper, we describe in details the CHIPP infrastructures and the results of the first two years of activity.
△ Less
Submitted 4 February, 2020;
originally announced February 2020.
-
Application of Google Cloud Platform in Astrophysics
Authors:
M. Landoni,
G. Taffoni,
A. Bignamini,
R. Smareglia
Abstract:
The availability of new Cloud Platform offered by Google motivated us to propose nine Proof of Concepts (PoC) aiming to demonstrated and test the capabilities of the platform in the context of scientifically-driven tasks and requirements. We review the status of our initiative by illustrating 3 out of 9 successfully closed PoC that we implemented on Google Cloud Platform. In particular, we illustr…
▽ More
The availability of new Cloud Platform offered by Google motivated us to propose nine Proof of Concepts (PoC) aiming to demonstrated and test the capabilities of the platform in the context of scientifically-driven tasks and requirements. We review the status of our initiative by illustrating 3 out of 9 successfully closed PoC that we implemented on Google Cloud Platform. In particular, we illustrate a cloud architecture for deployment of scientific software as microservice coupling Google Compute Engine with Docker and Pub/Sub to dispatch heavily parallel simulations. We detail also an experiment for HPC based simulation and workflow executions of data reduction pipelines (for the TNG-GIANO-B spectrograph) deployed on GCP. We compare and contrast our experience with on-site facilities comparing advantages and disadvantages both in terms of total cost of ownership and reached performances.
△ Less
Submitted 8 March, 2019;
originally announced March 2019.
-
Italian center for Astronomical Archives publishing solution: modular and distributed
Authors:
Marco Molinaro,
Nicola F. Calabria,
Robert Butora,
Sonia Zorba,
Riccardo Smareglia
Abstract:
The Italian center for Astronomical Archives tries to provide astronomical data resources as interoperable services based on IVOA standards. Its VO expertise and knowledge comes from active participation within IVOA and VO at European and international level, with a double-fold goal: learn from the collaboration and provide inputs to the community. The first solution to build an easy to configure…
▽ More
The Italian center for Astronomical Archives tries to provide astronomical data resources as interoperable services based on IVOA standards. Its VO expertise and knowledge comes from active participation within IVOA and VO at European and international level, with a double-fold goal: learn from the collaboration and provide inputs to the community. The first solution to build an easy to configure and maintain resource publisher conformant to VO standards proved to be too optimistic. For this reason it has been necessary to re-think the architecture with a modular system built around the messaging concept, where each modular component speaks to the other interested parties through a system of broker-managed queues. The first implemented protocol, the Simple Cone Search, shows the messaging task architecture connecting the parametric HTTP interface to the database backend access module, the logging module, and allows multiple cone search resources to be managed together through a configuration manager module. Even if relatively young, it already proved the flexibility required by the overall system when the database backend changed from MySQL to PostgreSQL+PgSphere. Another implementation test has been made to leverage task distribution over multiple servers to serve simultaneously: FITS cubes direct linking, cubes cutout and cubes positional merging. Currently the implementation of the SIA-2.0 standard protocol is ongoing while for TAP we will be adapting the TAPlib library. Alongside these tools a first administration tool (TASMAN) has been developed to ease the build up and maintenance of TAP_SCHEMA-ta including also ObsCore maintenance capability. Future work will be devoted at widening the range of VO protocols covered by the set of available modules, improve the configuration management and develop specific purpose modules common to all the service components.
△ Less
Submitted 21 May, 2018;
originally announced May 2018.
