-
Data Provenance and Management in Radio Astronomy: A Stream Computing Approach
Authors:
Mahmoud S. Mahmoud,
Andrew Ensor,
Alain Biem,
Bruce Elmegreen,
Sergei Gulyaev
Abstract:
New approaches for data provenance and data management (DPDM) are required for mega science projects like the Square Kilometer Array, characterized by extremely large data volume and intense data rates, therefore demanding innovative and highly efficient computational paradigms. In this context, we explore a stream-computing approach with the emphasis on the use of accelerators. In particular, we…
▽ More
New approaches for data provenance and data management (DPDM) are required for mega science projects like the Square Kilometer Array, characterized by extremely large data volume and intense data rates, therefore demanding innovative and highly efficient computational paradigms. In this context, we explore a stream-computing approach with the emphasis on the use of accelerators. In particular, we make use of a new generation of high performance stream-based parallelization middleware known as InfoSphere Streams. Its viability for managing and ensuring interoperability and integrity of signal processing data pipelines is demonstrated in radio astronomy. IBM InfoSphere Streams embraces the stream-computing paradigm. It is a shift from conventional data mining techniques (involving analysis of existing data from databases) towards real-time analytic processing. We discuss using InfoSphere Streams for effective DPDM in radio astronomy and propose a way in which InfoSphere Streams can be utilized for large antennae arrays. We present a case-study: the InfoSphere Streams implementation of an autocorrelating spectrometer, and using this example we discuss the advantages of the stream-computing approach and the utilization of hardware accelerators.
△ Less
Submitted 12 December, 2011;
originally announced December 2011.
-
New Zealand involvement in Radio Astronomical VLBI Image Processing
Authors:
Stuart Weston,
Tim Natusch,
Sergei Gulyaev
Abstract:
With the establishment of the AUT University 12m radio telescope at Warkworth, New Zealand has now become a part of the international Very Long Baseline Interferometry (VLBI) community. A major product of VLBI observations are images in the radio domain of astronomical objects such as Active Galactic Nuclei (AGN). Using large geographical separations between radio antennas, very high angular resol…
▽ More
With the establishment of the AUT University 12m radio telescope at Warkworth, New Zealand has now become a part of the international Very Long Baseline Interferometry (VLBI) community. A major product of VLBI observations are images in the radio domain of astronomical objects such as Active Galactic Nuclei (AGN). Using large geographical separations between radio antennas, very high angular resolution can be achieved. Detailed images can be created using the technique of VLBI Earth Rotation Aperture Synthesis. We review the current process of VLBI radio imaging. In addition we model VLBI configurations using the Warkworth telescope, AuScope (a new array of three 12m antennas in Australia) and the Australian Square Kilometre Array Pathfinder (ASKAP) array currently under construction in Western Australia, and discuss how the configuration of these arrays affects the quality of images. Recent imaging results that demonstrate the modeled improvements from inclusion of the AUT and first ASKAP telescope in the Australian Long Baseline Array (LBA) are presented.
△ Less
Submitted 24 October, 2011;
originally announced October 2011.
-
Radio Astronomy Data Transfer and eVLBI using KAREN
Authors:
Stuart Weston,
Timothy Natusch,
Sergei Gulyaev
Abstract:
Kiwi Advanced Research and Education Network (KAREN) has been used to transfer large volumes of radio astronomical data between the Radio Astronomical Observatory at Warkworth, New Zealand and various international organizations involved in joint projects and VLBI observations. Here we report on the current status of connectivity and on the results of testing different data transfer protocols. We…
▽ More
Kiwi Advanced Research and Education Network (KAREN) has been used to transfer large volumes of radio astronomical data between the Radio Astronomical Observatory at Warkworth, New Zealand and various international organizations involved in joint projects and VLBI observations. Here we report on the current status of connectivity and on the results of testing different data transfer protocols. We investigate new UDP protocols such as 'tsunami' and UDT and demonstrate that the UDT protocol is more efficient than 'tsunami' and 'ftp'. We also report on the tests on direct data streaming from the radio telescope receiving system to the correlation centre without intermediate buffering or recording (real-time eVLBI).
△ Less
Submitted 12 August, 2011;
originally announced August 2011.
-
Radio Astronomy and eVLBI using KAREN
Authors:
Stuart Weston,
Tim Natusch,
Sergei Gulyaev
Abstract:
Kiwi Advanced Research and Education Network (KAREN) has been used to transfer large volumes of radio astronomical data between the AUT Radio Astronomical Observatory at Warkworth, New Zealand and the international organisations with which we are collaborating and conducting observations. Here we report on the current status of connectivity and on the results of testing different data transfer pro…
▽ More
Kiwi Advanced Research and Education Network (KAREN) has been used to transfer large volumes of radio astronomical data between the AUT Radio Astronomical Observatory at Warkworth, New Zealand and the international organisations with which we are collaborating and conducting observations. Here we report on the current status of connectivity and on the results of testing different data transfer protocols. We investigate new UDP protocols such as "tsunami" and UDT and demonstrate that the UDT protocol is more efficient than "tsunami" and ftp. We report on our initial steps towards real-time eVLBI and the attempt to directly stream data from the radio telescope receiving system to the correlation centre without intermediate buffering/recording.
△ Less
Submitted 31 October, 2010;
originally announced November 2010.
