-
Selection of gamma events from IACT images with deep learning methods
Authors:
E. O. Gres,
A. P. Kryukov,
A. P. Demichev,
J. J. Dubenskaya,
S. P. Polyakov,
A. A. Vlaskina,
D. P. Zhurov
Abstract:
Imaging Atmospheric Cherenkov Telescopes (IACTs) of gamma ray observatory TAIGA detect the Extesnive Air Showers (EASs) originating from the cosmic or gamma rays interactions with the atmosphere. Thereby, telescopes obtain images of the EASs. The ability to segregate gamma rays images from the hadronic cosmic ray background is one of the main features of this type of detectors. However, in actual…
▽ More
Imaging Atmospheric Cherenkov Telescopes (IACTs) of gamma ray observatory TAIGA detect the Extesnive Air Showers (EASs) originating from the cosmic or gamma rays interactions with the atmosphere. Thereby, telescopes obtain images of the EASs. The ability to segregate gamma rays images from the hadronic cosmic ray background is one of the main features of this type of detectors. However, in actual IACT observations simultaneous observation of the background and the source of gamma ray is needed. This observation mode (called wobbling) modifies images of events, which affects the quality of selection by neural networks.
Thus, in this work, the results of the application of neural networks (NN) for image classification task on Monte Carlo (MC) images of TAIGA-IACTs are presented. The wobbling mode is considered together with the image adaptation for adequate analysis by NNs. Simultaneously, we explore several neural network structures that classify events both directly from images or through Hillas parameters extracted from images. In addition, by employing NNs, MC simulation data are used to evaluate the quality of the segregation of rare gamma events with the account of all necessary image modifications.
△ Less
Submitted 30 January, 2024;
originally announced January 2024.
-
Using a Conditional Generative Adversarial Network to Control the Statistical Characteristics of Generated Images for IACT Data Analysis
Authors:
Julia Dubenskaya,
Alexander Kryukov,
Andrey Demichev,
Stanislav Polyakov,
Elizaveta Gres,
Anna Vlaskina
Abstract:
Generative adversarial networks are a promising tool for image generation in the astronomy domain. Of particular interest are conditional generative adversarial networks (cGANs), which allow you to divide images into several classes according to the value of some property of the image, and then specify the required class when generating new images. In the case of images from Imaging Atmospheric Ch…
▽ More
Generative adversarial networks are a promising tool for image generation in the astronomy domain. Of particular interest are conditional generative adversarial networks (cGANs), which allow you to divide images into several classes according to the value of some property of the image, and then specify the required class when generating new images. In the case of images from Imaging Atmospheric Cherenkov Telescopes (IACTs), an important property is the total brightness of all image pixels (image size), which is in direct correlation with the energy of primary particles. We used a cGAN technique to generate images similar to whose obtained in the TAIGA-IACT experiment. As a training set, we used a set of two-dimensional images generated using the TAIGA Monte Carlo simulation software. We artificiallly divided the training set into 10 classes, sorting images by size and defining the boundaries of the classes so that the same number of images fall into each class. These classes were used while training our network. The paper shows that for each class, the size distribution of the generated images is close to normal with the mean value located approximately in the middle of the corresponding class. We also show that for the generated images, the total image size distribution obtained by summing the distributions over all classes is close to the original distribution of the training set. The results obtained will be useful for more accurate generation of realistic synthetic images similar to the ones taken by IACTs.
△ Less
Submitted 28 November, 2022;
originally announced November 2022.
-
Using conditional variational autoencoders to generate images from atmospheric Cherenkov telescopes
Authors:
Stanislav Polyakov,
Alexander Kryukov,
Andrey Demichev,
Julia Dubenskaya,
Elizaveta Gres,
Anna Vlaskina
Abstract:
High-energy particles hitting the upper atmosphere of the Earth produce extensive air showers that can be detected from the ground level using imaging atmospheric Cherenkov telescopes. The images recorded by Cherenkov telescopes can be analyzed to separate gamma-ray events from the background hadron events. Many of the methods of analysis require simulation of massive amounts of events and the cor…
▽ More
High-energy particles hitting the upper atmosphere of the Earth produce extensive air showers that can be detected from the ground level using imaging atmospheric Cherenkov telescopes. The images recorded by Cherenkov telescopes can be analyzed to separate gamma-ray events from the background hadron events. Many of the methods of analysis require simulation of massive amounts of events and the corresponding images by the Monte Carlo method. However, Monte Carlo simulation is computationally expensive. The data simulated by the Monte Carlo method can be augmented by images generated using faster machine learning methods such as generative adversarial networks or conditional variational autoencoders. We use a conditional variational autoencoder to generate images of gamma events from a Cherenkov telescope of the TAIGA experiment. The variational autoencoder is trained on a set of Monte Carlo events with the image size, or the sum of the amplitudes of the pixels, used as the conditional parameter. We used the trained variational autoencoder to generate new images with the same distribution of the conditional parameter as the size distribution of the Monte Carlo-simulated images of gamma events. The generated images are similar to the Monte Carlo images: a classifier neural network trained on gamma and proton events assigns them the average gamma score 0.984, with less than 3% of the events being assigned the gamma score below 0.999. At the same time, the sizes of the generated images do not match the conditional parameter used in their generation, with the average error 0.33.
△ Less
Submitted 22 November, 2022;
originally announced November 2022.
-
Energy Reconstruction in Analysis of Cherenkov Telescopes Images in TAIGA Experiment Using Deep Learning Methods
Authors:
E. O. Gres,
A. P. Kryukov
Abstract:
Imaging Atmospheric Cherenkov Telescopes (IACT) of TAIGA astrophysical complex allow to observe high energy gamma radiation hel** to study many astrophysical objects and processes. TAIGA-IACT enables us to select gamma quanta from the total cosmic radiation flux and recover their primary parameters, such as energy and direction of arrival. The traditional method of processing the resulting image…
▽ More
Imaging Atmospheric Cherenkov Telescopes (IACT) of TAIGA astrophysical complex allow to observe high energy gamma radiation hel** to study many astrophysical objects and processes. TAIGA-IACT enables us to select gamma quanta from the total cosmic radiation flux and recover their primary parameters, such as energy and direction of arrival. The traditional method of processing the resulting images is an image parameterization - so-called the Hillas parameters method. At the present time Machine Learning methods, in particular Deep Learning methods have become actively used for IACT image processing. This paper presents the analysis of simulated Monte Carlo images by several Deep Learning methods for a single telescope (mono-mode) and multiple IACT telescopes (stereo-mode). The estimation of the quality of energy reconstruction was carried out and their energy spectra were analyzed using several types of neural networks. Using the developed methods the obtained results were also compared with the results obtained by traditional methods based on the Hillas parameters.
△ Less
Submitted 16 November, 2022;
originally announced November 2022.
-
Interrelation of equivariant Gaussian processes and convolutional neural networks
Authors:
Andrey Demichev,
Alexander Kryukov
Abstract:
Currently there exists rather promising new trend in machine leaning (ML) based on the relationship between neural networks (NN) and Gaussian processes (GP), including many related subtopics, e.g., signal propagation in NNs, theoretical derivation of learning curve for NNs, QFT methods in ML, etc. An important feature of convolutional neural networks (CNN) is their equivariance (consistency) with…
▽ More
Currently there exists rather promising new trend in machine leaning (ML) based on the relationship between neural networks (NN) and Gaussian processes (GP), including many related subtopics, e.g., signal propagation in NNs, theoretical derivation of learning curve for NNs, QFT methods in ML, etc. An important feature of convolutional neural networks (CNN) is their equivariance (consistency) with respect to the symmetry transformations of the input data. In this work we establish a relationship between the many-channel limit for CNNs equivariant with respect to two-dimensional Euclidean group with vector-valued neuron activations and the corresponding independently introduced equivariant Gaussian processes (GP).
△ Less
Submitted 17 September, 2022;
originally announced September 2022.
-
Processing Images from Multiple IACTs in the TAIGA Experiment with Convolutional Neural Networks
Authors:
Stanislav Polyakov,
Andrey Demichev,
Alexander Kryukov,
Evgeny Postnikov
Abstract:
Extensive air showers created by high-energy particles interacting with the Earth atmosphere can be detected using imaging atmospheric Cherenkov telescopes (IACTs). The IACT images can be analyzed to distinguish between the events caused by gamma rays and by hadrons and to infer the parameters of the event such as the energy of the primary particle. We use convolutional neural networks (CNNs) to a…
▽ More
Extensive air showers created by high-energy particles interacting with the Earth atmosphere can be detected using imaging atmospheric Cherenkov telescopes (IACTs). The IACT images can be analyzed to distinguish between the events caused by gamma rays and by hadrons and to infer the parameters of the event such as the energy of the primary particle. We use convolutional neural networks (CNNs) to analyze Monte Carlo-simulated images from the telescopes of the TAIGA experiment. The analysis includes selection of the images corresponding to the showers caused by gamma rays and estimating the energy of the gamma rays. We compare performance of the CNNs using images from a single telescope and the CNNs using images from two telescopes as inputs.
△ Less
Submitted 31 December, 2021;
originally announced December 2021.
-
Complete Decentralization of Distributed Data Storages Based on Blockchain Technology
Authors:
Andrey Demichev,
Alexander Kryukov
Abstract:
The work presents a solution for completely decentralized data management systems in geographically distributed environments with administratively unrelated or loosely related user groups and in conditions of partial or complete lack of trust between them. The solution is based on the integration of blockchain technology, smart contracts and provenance metadata driven data management. Architecture…
▽ More
The work presents a solution for completely decentralized data management systems in geographically distributed environments with administratively unrelated or loosely related user groups and in conditions of partial or complete lack of trust between them. The solution is based on the integration of blockchain technology, smart contracts and provenance metadata driven data management. Architecture, operation principles and algorithms developed provides fault-tolerant, safe and reliable management of provenance metadata, control of operations with data files, as well as resource access management in collaborative distributed computing systems. The latter refer to distributed systems formed by combining into a single pool of computer resources of various organizations (institutions) to work together in the framework of some project.
△ Less
Submitted 20 December, 2021;
originally announced December 2021.
-
Analysis of the HiSCORE Simulated Events in TAIGA Experiment Using Convolutional Neural Networks
Authors:
Anna Vlaskina,
Alexander Kryukov
Abstract:
TAIGA is a hybrid observatory for gamma-ray astronomy at high energies in range from 10 TeV to several EeV. It consists of instruments such as TAIGA-IACT, TAIGA-HiSCORE, and others. TAIGA-HiSCORE, in particular, is an array of wide-angle timing Cherenkov light stations. TAIGA-HiSCORE data enable to reconstruct air shower characteristics, such as air shower energy, arrival direction, and axis coord…
▽ More
TAIGA is a hybrid observatory for gamma-ray astronomy at high energies in range from 10 TeV to several EeV. It consists of instruments such as TAIGA-IACT, TAIGA-HiSCORE, and others. TAIGA-HiSCORE, in particular, is an array of wide-angle timing Cherenkov light stations. TAIGA-HiSCORE data enable to reconstruct air shower characteristics, such as air shower energy, arrival direction, and axis coordinates. In this report, we propose to consider the use of convolution neural networks in task of air shower characteristics determination. We use Convolutional Neural Networks (CNN) to analyze HiSCORE events, treating them like images. For this, the times and amplitudes of events recorded at HiSCORE stations are used. The work discusses a simple convolutional neural network and its training. In addition, we present some preliminary results on the determination of the parameters of air showers such as the direction and position of the shower axis and the energy of the primary particle and compare them with the results obtained by the traditional method.
△ Less
Submitted 19 December, 2021;
originally announced December 2021.
-
The Preliminary Results on Analysis of TAIGA-IACT Images Using Convolutional Neural Networks
Authors:
Elizaveta Gres,
and Alexander Kryukov
Abstract:
The imaging Cherenkov telescopes TAIGA-IACT, located in the Tunka valley of the republic Buryatia, accumulate a lot of data in a short period of time which must be efficiently and quickly analyzed. One of the methods of such analysis is the machine learning, which has proven its effectiveness in many technological and scientific fields in recent years. The aim of the work is to study the possibili…
▽ More
The imaging Cherenkov telescopes TAIGA-IACT, located in the Tunka valley of the republic Buryatia, accumulate a lot of data in a short period of time which must be efficiently and quickly analyzed. One of the methods of such analysis is the machine learning, which has proven its effectiveness in many technological and scientific fields in recent years. The aim of the work is to study the possibility of the machine learning application to solve the tasks set for TAIGA-IACT: the identification of the primary particle of cosmic rays and reconstruction their physical parameters. In the work the method of Convolutional Neural Networks (CNN) was applied to process and analyze Monte-Carlo events simulated with CORSIKA. Also various CNN architectures for the processing were considered. It has been demonstrated that this method gives good results in the determining the type of primary particles of Extensive Air Shower (EAS) and the reconstruction of gamma-rays energy. The results are significantly improved in the case of stereoscopic observations.
△ Less
Submitted 19 December, 2021;
originally announced December 2021.
-
Parametric non-interference in timed automata
Authors:
Étienne André,
Aleksander Kryukov
Abstract:
We consider a notion of non-interference for timed automata (TAs) that allows to quantify the frequency of an attack; that is, we infer values of the minimal time between two consecutive actions of the attacker, so that (s)he disturbs the set of reachable locations. We also synthesize valuations for the timing constants of the TA (seen as parameters) guaranteeing non-interference. We show that thi…
▽ More
We consider a notion of non-interference for timed automata (TAs) that allows to quantify the frequency of an attack; that is, we infer values of the minimal time between two consecutive actions of the attacker, so that (s)he disturbs the set of reachable locations. We also synthesize valuations for the timing constants of the TA (seen as parameters) guaranteeing non-interference. We show that this can reduce to reachability synthesis in parametric timed automata. We apply our method to a model of the Fischer mutual exclusion protocol and obtain preliminary results.
△ Less
Submitted 19 October, 2020;
originally announced October 2020.
-
AstroDS -- A Distributed Storage for Astrophysics of Cosmic Rays. Current Status
Authors:
Alexander Kryukov,
Igor Bychkov,
Elena Korosteleva,
Andrey Mikhailov,
Minh-Duc Nguyen
Abstract:
Currently, the processing of scientific data in astroparticle physics is based on various distributed technologies, the most common of which are Grid and cloud computing. The most frequently discussed approaches are focused on large and even very large scientific experiments, such as Cherenkov Telescope Array. We, by contrast, offer a solution designed for small to medium experiments such as TAIGA…
▽ More
Currently, the processing of scientific data in astroparticle physics is based on various distributed technologies, the most common of which are Grid and cloud computing. The most frequently discussed approaches are focused on large and even very large scientific experiments, such as Cherenkov Telescope Array. We, by contrast, offer a solution designed for small to medium experiments such as TAIGA. In such experiments, as a rule, historically developed specific data processing methods and specialized software are used. We have specifically designed a distributed (cloud) data storage for astroparticle physics data collaboration in medium-sized experiments. In this article, we discuss the current state of our work using the example of the TAIGA and CASCADE experiments. A feature of our approach is that we provide our users with scientific data in the form to which they are accustomed to in everyday work on local resources.
△ Less
Submitted 10 October, 2020;
originally announced October 2020.
-
Data Aggregation In The Astroparticle Physics Distributed Data Storage
Authors:
Minh-Duc Nguyen,
Alexander Kryukov,
Julia Dubenskaya,
Elena Korosteleva,
Igor Bychkov,
Andrey Mikhailov,
Alexey Shigarov
Abstract:
German-Russian Astroparticle Data Life Cycle Initiative is an international project whose aim is to develop a distributed data storage system that aggregates data from the storage systems of different astroparticle experiments. The prototype of such a system, which is called the Astroparticle Physics Distributed Data Storage (APPDS), has been being developed. In this paper, the Data Aggregation Se…
▽ More
German-Russian Astroparticle Data Life Cycle Initiative is an international project whose aim is to develop a distributed data storage system that aggregates data from the storage systems of different astroparticle experiments. The prototype of such a system, which is called the Astroparticle Physics Distributed Data Storage (APPDS), has been being developed. In this paper, the Data Aggregation Service, one of the core services of APDDS, is presented. The Data Aggregation Service connects all distributed services of APPDS together to find the necessary data and deliver them to users on demand.
△ Less
Submitted 5 August, 2019;
originally announced August 2019.
-
Deep Learning for Energy Estimation and Particle Identification in Gamma-ray Astronomy
Authors:
Evgeny Postnikov,
Alexander Kryukov,
Stanislav Polyakov,
Dmitry Zhurov
Abstract:
Deep learning techniques, namely convolutional neural networks (CNN), have previously been adapted to select gamma-ray events in the TAIGA experiment, having achieved a good quality of selection as compared with the conventional Hillas approach. Another important task for the TAIGA data analysis was also solved with CNN: gamma-ray energy estimation showed some improvement in comparison with the co…
▽ More
Deep learning techniques, namely convolutional neural networks (CNN), have previously been adapted to select gamma-ray events in the TAIGA experiment, having achieved a good quality of selection as compared with the conventional Hillas approach. Another important task for the TAIGA data analysis was also solved with CNN: gamma-ray energy estimation showed some improvement in comparison with the conventional method based on the Hillas analysis. Furthermore, our software was completely redeveloped for the graphics processing unit (GPU), which led to significantly faster calculations in both of these tasks. All the results have been obtained with the simulated data of TAIGA Monte Carlo software; their experimental confirmation is envisaged for the near future.
△ Less
Submitted 23 July, 2019;
originally announced July 2019.
-
Distributed data storage for modern astroparticle physics experiments
Authors:
Alexander Kryukov,
Minh-Duc Nguyen,
Igor Bychkov,
Andrey Mikhailov,
Alexey Shigarov,
Julia Dubenskaya
Abstract:
The German-Russian Astroparticle Data Life Cycle Initiative is an international project launched in 2018. The Initiative aims to develop technologies that provide a unified approach to data management, as well as to demonstrate their applicability on the example of two large astrophysical experiments - KASCADE and TAIGA. One of the key points of the project is the development of a distributed stor…
▽ More
The German-Russian Astroparticle Data Life Cycle Initiative is an international project launched in 2018. The Initiative aims to develop technologies that provide a unified approach to data management, as well as to demonstrate their applicability on the example of two large astrophysical experiments - KASCADE and TAIGA. One of the key points of the project is the development of a distributed storage, which, on the one hand, will allow data of several experiments to be combined into a single repository with unified interface, and on the other hand, will provide data to all participants of experimental groups for multi-messenger analysis. Our approach to storage design is based on the single write-multiple read (SWMR) model for accessing raw or centrally processed data for further analysis. The main feature of the distributed storage is the ability to extract data either as a collection of files or as aggregated events from different sources. In the last case the storage provides users with a special service that aggregates data from different storages into a single sample. Thanks to this feature, multi-messenger methods used for more sophisticated data exploration can be applied. Users can use both Web-interface and Application Programming Interface (API) for accessing the storage. In this paper we describe the architecture of a distributed data storage for astroparticle physics and discuss the current status of our work.
△ Less
Submitted 16 July, 2019;
originally announced July 2019.
-
Metadata Extraction from Raw Astroparticle Data of TAIGA Experiment
Authors:
Igor Bychkov,
Julia Dubenskaya,
Elena Korosteleva,
Alexandr Kryukov,
Andrey Mikhailov,
Minh-Duc Nguyen,
Alexey Shigarov
Abstract:
Today, the operating TAIGA (Tunka Advanced Instrument for cosmic rays and Gamma Astronomy) experiment continuously produces and accumulates a large volume of raw astroparticle data. To be available for the scientific community these data should be well-described and formally characterized. The use of metadata makes it possible to search for and to aggregate digital objects (e.g. events and runs) b…
▽ More
Today, the operating TAIGA (Tunka Advanced Instrument for cosmic rays and Gamma Astronomy) experiment continuously produces and accumulates a large volume of raw astroparticle data. To be available for the scientific community these data should be well-described and formally characterized. The use of metadata makes it possible to search for and to aggregate digital objects (e.g. events and runs) by time and equipment through a unified interface to access them. The important part of the metadata is hidden and scattered in folder/files names and package headers. Such metadata should be extracted from binary files, transformed to a unified form of digital objects, and loaded into the catalog. To address this challenge we developed a concept of the metadata extractor that can be extended by facility-specific extraction modules. It is designed to automatically collect descriptive metadata from raw data files of all TAIGA formats.
△ Less
Submitted 14 July, 2019;
originally announced July 2019.
-
A distributed data warehouse system for astroparticle physics
Authors:
Minh-Duc Nguyen,
Alexander Kryukov,
Julia Dubenskaya,
Elena Korosteleva,
Stanislav Polyakov,
Evgeny Postnikov,
Igor Bychkov,
Andrey Mikhailov,
Alexey Shigarov,
Oleg Fedorov,
Yulia Kazarina,
Dmitry Shipilov,
Dmitry Zhurov
Abstract:
A distributed data warehouse system is one of the actual issues in the field of astroparticle physics. Famous experiments, such as TAIGA, KASCADE-Grande, produce tens of terabytes of data measured by their instruments. It is critical to have a smart data warehouse system on-site to store the collected data for further distribution effectively. It is also vital to provide scientists with a handy an…
▽ More
A distributed data warehouse system is one of the actual issues in the field of astroparticle physics. Famous experiments, such as TAIGA, KASCADE-Grande, produce tens of terabytes of data measured by their instruments. It is critical to have a smart data warehouse system on-site to store the collected data for further distribution effectively. It is also vital to provide scientists with a handy and user-friendly interface to access the collected data with proper permissions not only on-site but also online. The latter case is handy when scientists need to combine data from different experiments for analysis. In this work, we describe an approach to implementing a distributed data warehouse system that allows scientists to acquire just the necessary data from different experiments via the Internet on demand. The implementation is based on CernVM-FS with additional components developed by us to search through the whole available data sets and deliver their subsets to users' computers.
△ Less
Submitted 5 December, 2018;
originally announced December 2018.
-
Particle identification in ground-based gamma-ray astronomy using convolutional neural networks
Authors:
E. B. Postnikov,
I. V. Bychkov,
J. Y. Dubenskaya,
O. L. Fedorov,
Y. A. Kazarina,
E. E. Korosteleva,
A. P. Kryukov,
A. A. Mikhailov,
M. D. Nguyen,
S. P. Polyakov,
A. O. Shigarov,
D. A. Shipilov,
D. P. Zhurov
Abstract:
Modern detectors of cosmic gamma-rays are a special type of imaging telescopes (air Cherenkov telescopes) supplied with cameras with a relatively large number of photomultiplier-based pixels. For example, the camera of the TAIGA-IACT telescope has 560 pixels of hexagonal structure. Images in such cameras can be analysed by deep learning techniques to extract numerous physical and geometrical param…
▽ More
Modern detectors of cosmic gamma-rays are a special type of imaging telescopes (air Cherenkov telescopes) supplied with cameras with a relatively large number of photomultiplier-based pixels. For example, the camera of the TAIGA-IACT telescope has 560 pixels of hexagonal structure. Images in such cameras can be analysed by deep learning techniques to extract numerous physical and geometrical parameters and/or for incoming particle identification. The most powerful deep learning technique for image analysis, the so-called convolutional neural network (CNN), was implemented in this study. Two open source libraries for machine learning, PyTorch and TensorFlow, were tested as possible software platforms for particle identification in imaging air Cherenkov telescopes. Monte Carlo simulation was performed to analyse images of gamma-rays and background particles (protons) as well as estimate identification accuracy. Further steps of implementation and improvement of this technique are discussed.
△ Less
Submitted 4 December, 2018;
originally announced December 2018.
-
Using Binary File Format Description Languages for Documenting, Parsing, and Verifying Raw Data in TAIGA Experiment
Authors:
I. Bychkov,
A. Demichev,
J. Dubenskaya,
O. Fedorov,
A. Hmelnov,
Y. Kazarina,
E. Korosteleva,
D. Kostunin,
A. Kryukov,
A. Mikhailov,
M. D. Nguyen,
S. Polyakov,
E. Postnikov,
A. Shigarov,
D. Shipilov,
D. Zhurov
Abstract:
The paper is devoted to the issues of raw binary data documenting, parsing and verifying in astroparticle data lifecycle. The long-term preservation of raw data of astroparticle experiments as originally generated is essential for re-running analyses and reproducing research results. The selected high-quality raw data should have detailed documentation and accompanied by open software tools for ac…
▽ More
The paper is devoted to the issues of raw binary data documenting, parsing and verifying in astroparticle data lifecycle. The long-term preservation of raw data of astroparticle experiments as originally generated is essential for re-running analyses and reproducing research results. The selected high-quality raw data should have detailed documentation and accompanied by open software tools for access to them. We consider applicability of binary file format description languages to specify, parse and verify raw data of the Tunka Advanced Instrument for cosmic rays and Gamma Astronomy (TAIGA) experiment. The formal specifications are implemented for five data formats of the experiment and provide automatic generation of source code for data reading libraries in target programming languages (e.g. C++, Java, and Python). These libraries were tested on TAIGA data. They showed a good performance and help us to locate the parts with corrupted data. The format specifications can be used as metadata for exchanging of astroparticle raw data. They can also simplify software development for data aggregation from various sources for the multi-messenger analysis.
△ Less
Submitted 4 December, 2018;
originally announced December 2018.
-
The Approach to Managing Provenance Metadata and Data Access Rights in Distributed Storage Using the Hyperledger Blockchain Platform
Authors:
Andrey Demichev,
Alexander Kryukov,
Nikolai Prikhodko
Abstract:
The paper suggests a new approach based on blockchain technologies and smart contracts to creation of a distributed system for managing provenance metadata, as well as access rights to data in distributed storages, which is fault-tolerant, safe and secure from the point of view of preservation of metadata records from accidental or intentional distortions. The implementation of the proposed approa…
▽ More
The paper suggests a new approach based on blockchain technologies and smart contracts to creation of a distributed system for managing provenance metadata, as well as access rights to data in distributed storages, which is fault-tolerant, safe and secure from the point of view of preservation of metadata records from accidental or intentional distortions. The implementation of the proposed approach is based on the permissioned blockchains and on the Hyperledger Fabric blockchain platform in conjunction with Hyperledger Composer.
△ Less
Submitted 30 November, 2018;
originally announced November 2018.
-
Simplification of tensor expressions in computer algebra
Authors:
A. Kryukov,
G. Shpiz
Abstract:
Computer algebra is widely used in various fields of mathematics, physics and other sciences. The simplification of tensor expressions is an important special case of computer algebra. In this paper, we consider the reduction of tensor polynomials to canonical form, taking into account the properties of symmetry under permutations of indices, the symmetries associated with the renaming of summatio…
▽ More
Computer algebra is widely used in various fields of mathematics, physics and other sciences. The simplification of tensor expressions is an important special case of computer algebra. In this paper, we consider the reduction of tensor polynomials to canonical form, taking into account the properties of symmetry under permutations of indices, the symmetries associated with the renaming of summation indices, and also linear relations between tensors of a general form. We give a definition of the canonical representation for polynomial (multiplicative) expressions of variables with abstract indices, which is the result of averaging of the original expression by the action of some finite group (the signature stabilizer). In practice, the proposed algorithms demonstrate high efficiency for expressions made of Riemann curvature tensors.
△ Less
Submitted 19 November, 2018;
originally announced November 2018.
-
Decentralized Data Storages: Technologies of Construction
Authors:
Alexander Kryukov,
Andrey Demichev
Abstract:
The paper presents a comparative overview of decentralized data storages of various types. It is shown that although they have a number of common properties that are typical of all peer-to-peer (P2P) networks, the problems to be solved and, accordingly, the technologies used to build different types of storages differ significantly.
The paper presents a comparative overview of decentralized data storages of various types. It is shown that although they have a number of common properties that are typical of all peer-to-peer (P2P) networks, the problems to be solved and, accordingly, the technologies used to build different types of storages differ significantly.
△ Less
Submitted 15 November, 2018;
originally announced November 2018.
-
ATENSOR - REDUCE program for tensor simplification
Authors:
V. A. Ilyin,
A. P. Kryukov
Abstract:
The paper presents a REDUCE program for the simplification of tensor expressions that are considered as formal indexed objects. The proposed algorithm is based on the consideration of tensor expressions as vectors in some linear space. This linear space is formed by all the elements of the group algebra of the corresponding tensor expression. Such approach permits us to simplify the tensor express…
▽ More
The paper presents a REDUCE program for the simplification of tensor expressions that are considered as formal indexed objects. The proposed algorithm is based on the consideration of tensor expressions as vectors in some linear space. This linear space is formed by all the elements of the group algebra of the corresponding tensor expression. Such approach permits us to simplify the tensor expressions possessing symmetry properties, summation (dummy) indices and multiterm identities by unify manner. The canonical element for the tensor expression is defined in terms of the basic vectors of this linear space. The main restriction of the algorithm is the dimension of the linear space that is equal to N!, where N is a number of indices of the tensor expression. The program uses REDUCE as user interface.
△ Less
Submitted 13 November, 2018;
originally announced November 2018.
-
Architecture of Distributed Data Storage for Astroparticle Physics
Authors:
Alexander Kryukov,
Andrey Demichev
Abstract:
For the successful development of the astrophysics and, accordingly, for obtaining more complete knowledge of the Universe, it is extremely important to combine and comprehensively analyze information of various types (e.g., about charged cosmic particles, gamma rays, neutrinos, etc.) obtained by using divers large-scale experimental setups located throughout the world. It is obvious that all kind…
▽ More
For the successful development of the astrophysics and, accordingly, for obtaining more complete knowledge of the Universe, it is extremely important to combine and comprehensively analyze information of various types (e.g., about charged cosmic particles, gamma rays, neutrinos, etc.) obtained by using divers large-scale experimental setups located throughout the world. It is obvious that all kinds of activities must be performed continually across all stages of the data life cycle to help support effective data management, in particular, the collection and storage of data, its processing and analysis, refining the physical model, making preparations for publication, and data reprocessing taking refinement into account. In this paper we present a general approach to construction and the architecture of a system to be able to collect, store, and provide users' access to astrophysical data. We also suggest a new approach to the construction of a metadata registry based on the blockchain technology.
△ Less
Submitted 6 November, 2018;
originally announced November 2018.
-
Web Toolkit for Scientific Research: State of the Art and the Prospect for Development
Authors:
Stanislav P. Polyakov,
Andrey P. Demichev,
Alexander P. Kryukov
Abstract:
The paper examines the current trends in designing of systems for convenient and secure remote job submission to various computer resources, including supercomputers, computer clusters, cloud resources, data storages and databases, and grid infrastructures by authorized users, as well as remote job monitoring and obtaining the results. Currently, high-perfomance computing and storage resources are…
▽ More
The paper examines the current trends in designing of systems for convenient and secure remote job submission to various computer resources, including supercomputers, computer clusters, cloud resources, data storages and databases, and grid infrastructures by authorized users, as well as remote job monitoring and obtaining the results. Currently, high-perfomance computing and storage resources are capable of solving independently the majority of practical problems in the field of science and technology. Therefore, the focus in the development of a new generation of middleware shifts from the global grid systems to building convenient and efficient web platforms for remote access to individual computing resources. The paper examines the general principles of the construction and briefly describes some of the specific implementations of the web platforms.
△ Less
Submitted 31 March, 2016;
originally announced March 2016.
-
Fault Tolerance of Small-World Regular and Stochastic Interconnection Networks
Authors:
A. Demichev,
V. Ilyin,
A. Kryukov,
S. Polyakov
Abstract:
Resilience of the most important properties of stochastic and regular (deterministic) small-world interconnection networks is studied. It is shown that in the broad range of values of the fraction of faulty nodes the networks under consideration possess high fault tolerance, the deterministic networks being slightly better than the stochastic ones.
Resilience of the most important properties of stochastic and regular (deterministic) small-world interconnection networks is studied. It is shown that in the broad range of values of the fraction of faulty nodes the networks under consideration possess high fault tolerance, the deterministic networks being slightly better than the stochastic ones.
△ Less
Submitted 13 January, 2014; v1 submitted 21 November, 2013;
originally announced December 2013.
-
A Quality and Cost Approach for Comparison of Small-World Networks
Authors:
A. Demichev,
V. Ilyin,
A. Kryukov,
S. Polyakov
Abstract:
We propose an approach based on analysis of cost-quality tradeoffs for comparison of efficiency of various algorithms for small-world network construction. A number of both known in the literature and original algorithms for complex small-world networks construction are shortly reviewed and compared. The networks constructed on the basis of these algorithms have basic structure of 1D regular latti…
▽ More
We propose an approach based on analysis of cost-quality tradeoffs for comparison of efficiency of various algorithms for small-world network construction. A number of both known in the literature and original algorithms for complex small-world networks construction are shortly reviewed and compared. The networks constructed on the basis of these algorithms have basic structure of 1D regular lattice with additional shortcuts providing the small-world properties. It is shown that networks proposed in this work have the best cost-quality ratio in the considered class.
△ Less
Submitted 4 January, 2013;
originally announced January 2013.
-
Problem of Application Job Monitoring in GRID Systems
Authors:
V. Kalyaev,
A. Kryukov
Abstract:
We present a new approach to monitoring of the execution process of an application job in the GRID environment. The main point of the approach is use of GRID ervices to access monitoring information with the security level available in GRID.
We present a new approach to monitoring of the execution process of an application job in the GRID environment. The main point of the approach is use of GRID ervices to access monitoring information with the security level available in GRID.
△ Less
Submitted 10 November, 2003;
originally announced November 2003.
-
OGSA/Globus Evaluation for Data Intensive Applications
Authors:
A. Demichev,
D. Foster,
V. Kalyaev,
A. Kryukov,
M. Lamanna,
V. Pose,
R. B. Da Rocha,
C. Wang
Abstract:
We present an architecture of Globus Toolkit 3 based testbed intended for evaluation of applicability of the Open Grid Service Architecture (OGSA) for Data Intensive Applications.
We present an architecture of Globus Toolkit 3 based testbed intended for evaluation of applicability of the Open Grid Service Architecture (OGSA) for Data Intensive Applications.
△ Less
Submitted 10 November, 2003;
originally announced November 2003.
