Showing 1–50 of 90 results for author: Yatawatta, S

Reinforcement learning

Authors: Sarod Yatawatta

Abstract: Observing celestial objects and advancing our scientific knowledge about them involves tedious planning, scheduling, data collection and data post-processing. Many of these operational aspects of astronomy are guided and executed by expert astronomers. Reinforcement learning is a mechanism where we (as humans and astronomers) can teach agents of artificial intelligence to perform some of these ted… ▽ More Observing celestial objects and advancing our scientific knowledge about them involves tedious planning, scheduling, data collection and data post-processing. Many of these operational aspects of astronomy are guided and executed by expert astronomers. Reinforcement learning is a mechanism where we (as humans and astronomers) can teach agents of artificial intelligence to perform some of these tedious tasks. In this paper, we will present a state of the art overview of reinforcement learning and how it can benefit astronomy. △ Less

Submitted 16 May, 2024; originally announced May 2024.

Comments: To appear, Astronomy & Computing

Direction dependent calibration with diffuse radio sky models

Authors: Sarod Yatawatta

Abstract: A fundamental assumption of the data model used in radio interferometric calibration is that the sky model only consists of compact and discrete radio sources. This assumption breaks down when there are large scale diffuse structure such as the Galaxy visible in the observed data. No straightforward method currently exists to include such large scale diffuse structure in calibration and only indir… ▽ More A fundamental assumption of the data model used in radio interferometric calibration is that the sky model only consists of compact and discrete radio sources. This assumption breaks down when there are large scale diffuse structure such as the Galaxy visible in the observed data. No straightforward method currently exists to include such large scale diffuse structure in calibration and only indirect techniques such as excluding short baselines or filtering are used in practice as a remedy. In this paper, we propose a novel mechanism to include large scale diffuse sky models into direction dependent calibration of radio interferometers. We extend distributed calibration of radio interferometric data with both spectral and spatial regularization to include models for diffuse emission. We use shapelet basis functions to model both the diffuse sky structure as well as the spatial variation of systematic errors. The application of the direction dependent errors onto the diffuse sky is done in closed form using specific properties of shapelet basis functions, avoiding the need for expensive operations such as convolution. We provide extensive simulations showing the efficacy of our proposed method. We are able to overcome a major problem faced by existing calibration techniques, i.e., the suppression of large scale diffuse structure by not properly modeling such structure in calibration. △ Less

Submitted 9 January, 2024; originally announced January 2024.

Comments: Under review - A&A

First upper limits on the 21 cm signal power spectrum from cosmic dawn from one night of observations with NenuFAR

Authors: S. Munshi, F. G. Mertens, L. V. E. Koopmans, A. R. Offringa, B. Semelin, D. Aubert, R. Barkana, A. Bracco, S. A. Brackenhoff, B. Cecconi, E. Ceccotti, S. Corbel, A. Fialkov, B. K. Gehlot, R. Ghara, J. N. Girard, J. M. Grießmeier, C. Höfer, I. Hothi, R. Mériot, M. Mevius, P. Ocvirk, A. K. Shaw, G. Theureau, S. Yatawatta , et al. (2 additional authors not shown)

Abstract: The redshifted 21 cm signal from neutral hydrogen is a direct probe of the physics of the early universe and has been an important science driver of many present and upcoming radio interferometers. In this study we use a single night of observations with the New Extension in Nançay Upgrading LOFAR (NenuFAR) to place upper limits on the 21 cm power spectrum from cosmic dawn at a redshift of $z$ = 2… ▽ More The redshifted 21 cm signal from neutral hydrogen is a direct probe of the physics of the early universe and has been an important science driver of many present and upcoming radio interferometers. In this study we use a single night of observations with the New Extension in Nançay Upgrading LOFAR (NenuFAR) to place upper limits on the 21 cm power spectrum from cosmic dawn at a redshift of $z$ = 20.3. NenuFAR is a new low-frequency radio interferometer, operating in the 10-85 MHz frequency range, currently under construction at the Nançay Radio Observatory in France. It is a phased array instrument with a very dense uv coverage at short baselines, making it one of the most sensitive instruments for 21 cm cosmology analyses at these frequencies. Our analysis adopts the foreground subtraction approach, in which sky sources are modeled and subtracted through calibration and residual foregrounds are subsequently removed using Gaussian process regression. The final power spectra are constructed from the gridded residual data cubes in the uv plane. Signal injection tests are performed at each step of the analysis pipeline, the relevant pipeline settings are optimized to ensure minimal signal loss, and any signal suppression is accounted for through a bias correction on our final upper limits. We obtain a best 2$σ$ upper limit of $2.4\times 10^7$ $\text{mK}^{2}$ at $z$ = 20.3 and $k$ = 0.041 $h\,\text{cMpc}^{-1}$. We see a strong excess power in the data, making our upper limits two orders of magnitude higher than the thermal noise limit. We investigate the origin and nature of this excess power and discuss further improvements to the analysis pipeline that can potentially mitigate it and consequently allow us to reach thermal noise sensitivity when multiple nights of observations are processed in the future. △ Less

Submitted 30 April, 2024; v1 submitted 9 November, 2023; originally announced November 2023.

Comments: 27 pages, 21 figures, and 6 tables; accepted for publication in Astronomy and Astrophysics (A&A); language edits implemented; typos corrected

Journal ref: A&A 681, A62 (2024)

Transient RFI environment of LOFAR-LBA at 72-75 MHz: Impact on ultra-widefield AARTFAAC Cosmic Explorer observations of the redshifted 21-cm signal

Authors: B. K. Gehlot, L. V. E. Koopmans, S. A. Brackenhoff, E. Ceccotti, S. Ghosh, C. Höfer, F. G. Mertens, M. Mevius, S. Munshi, A. R. Offringa, V. N. Pandey, A. Rowlinson, A. Shulevski, R. A. M. J. Wijers, S. Yatawatta, S. Zaroubi

Abstract: Measurement of the redshifted 21-cm signal of neutral hydrogen from the Cosmic Dawn (CD) and Epoch of Reionisation (EoR) promises to unveil a wealth of information about the astrophysical processes during the first billion years of evolution of the universe. The AARTFAAC Cosmic Explorer (ACE) utilises the AARTFAAC wide-field imager of LOFAR to measure the power spectrum of the intensity fluctuatio… ▽ More Measurement of the redshifted 21-cm signal of neutral hydrogen from the Cosmic Dawn (CD) and Epoch of Reionisation (EoR) promises to unveil a wealth of information about the astrophysical processes during the first billion years of evolution of the universe. The AARTFAAC Cosmic Explorer (ACE) utilises the AARTFAAC wide-field imager of LOFAR to measure the power spectrum of the intensity fluctuations of the redshifted 21-cm signal from the CD at z~18. The RFI from various sources contaminates the observed data and it is crucial to exclude the RFI-affected data in the analysis for reliable detection. In this work, we investigate the impact of non-ground-based transient RFI using cross-power spectra and cross-coherence metrics to assess the correlation of RFI over time and investigate the level of impact of transient RFI on the ACE 21-cm power spectrum estimation. We detected moving sky-based transient RFI sources that cross the field of view within a few minutes and appear to be mainly from aeroplane communication beacons at the location of the LOFAR core in the 72-75 MHz band, by inspecting filtered images. This transient RFI is mostly uncorrelated over time and is only expected to dominate over the thermal noise for an extremely deep integration time of 3000 hours or more with a hypothetical instrument that is sky temperature dominated at 75 MHz. We find no visible correlation over different k-modes in Fourier space in the presence of noise for realistic thermal noise scenarios. We conclude that the sky-based transient RFI from aeroplanes, satellites and meteorites at present does not pose a significant concern for the ACE analyses at the current level of sensitivity and after integrating over the available 500 hours of observed data. However, it is crucial to mitigate or filter such transient RFI for more sensitive experiments aiming for significantly deeper integration. △ Less

Submitted 6 November, 2023; originally announced November 2023.

Comments: 13 pages, 9 figures, and 3 tables; accepted for publication in Astronomy and Astrophysics (A&A)

Journal ref: A&A 681, A71 (2024)

A novel radio imaging method for physical spectral index modelling

Authors: E. Ceccotti, A. R. Offringa, L. V. E. Koopmans, R. Timmerman, S. A. Brackenhoff, B. K. Gehlot, F. G. Mertens, S. Munshi, V. N. Pandey, R. J. van Weeren, S. Yatawatta, S. Zaroubi

Abstract: We present a new method, called "forced-spectrum fitting", for physically-based spectral modelling of radio sources during deconvolution. This improves upon current common deconvolution fitting methods, which often produce inaccurate spectra. Our method uses any pre-existing spectral index map to assign spectral indices to each model component cleaned during the multi-frequency deconvolution of WS… ▽ More We present a new method, called "forced-spectrum fitting", for physically-based spectral modelling of radio sources during deconvolution. This improves upon current common deconvolution fitting methods, which often produce inaccurate spectra. Our method uses any pre-existing spectral index map to assign spectral indices to each model component cleaned during the multi-frequency deconvolution of WSClean, where the pre-determined spectrum is fitted. The component magnitude is evaluated by performing a modified weighted linear least-squares fit. We test this method on a simulated LOFAR-HBA observation of the 3C196 QSO and a real LOFAR-HBA observation of the 4C+55.16 FRI galaxy. We compare the results from the forced-spectrum fitting with traditional joined-channel deconvolution using polynomial fitting. Because no prior spectral information was available for 4C+55.16, we demonstrate a method for extracting spectral indices in the observed frequency band using "clustering". The models generated by the forced-spectrum fitting are used to improve the calibration of the datasets. The final residuals are comparable to existing multi-frequency deconvolution methods, but the output model agrees with the provided spectral index map, embedding correct spectral information. While forced-spectrum fitting does not solve the determination of the spectral information itself, it enables the construction of accurate multi-frequency models that can be used for wide-band calibration and subtraction. △ Less

Submitted 14 August, 2023; originally announced August 2023.

Comments: 17 pages, 9 figures, 5 tables. Accepted for publication in MNRAS

Hint assisted reinforcement learning: an application in radio astronomy

Authors: Sarod Yatawatta

Abstract: Model based reinforcement learning has proven to be more sample efficient than model free methods. On the other hand, the construction of a dynamics model in model based reinforcement learning has increased complexity. Data processing tasks in radio astronomy are such situations where the original problem which is being solved by reinforcement learning itself is the creation of a model. Fortunatel… ▽ More Model based reinforcement learning has proven to be more sample efficient than model free methods. On the other hand, the construction of a dynamics model in model based reinforcement learning has increased complexity. Data processing tasks in radio astronomy are such situations where the original problem which is being solved by reinforcement learning itself is the creation of a model. Fortunately, many methods based on heuristics or signal processing do exist to perform the same tasks and we can leverage them to propose the best action to take, or in other words, to provide a `hint'. We propose to use `hints' generated by the environment as an aid to the reinforcement learning process mitigating the complexity of model construction. We modify the soft actor critic algorithm to use hints and use the alternating direction method of multipliers algorithm with inequality constraints to train the agent. Results in several environments show that we get the increased sample efficiency by using hints as compared to model free methods. △ Less

Submitted 10 January, 2023; originally announced January 2023.

Degree-Scale Galactic Radio Emission at 122 MHz around the North Celestial Pole with LOFAR-AARTFAAC

Authors: B. K. Gehlot, L. V. E. Koopmans, A. R. Offringa, H. Gan, R. Ghara, S. K. Giri, M. Kuiack, F. G. Mertens, M. Mevius, R. Mondal, V. N. Pandey, A. Shulevski, R. A. M. J. Wijers, S. Yatawatta

Abstract: Aims: Contamination from bright diffuse Galactic thermal and non-thermal radio emission poses crucial challenges in experiments aiming to measure the 21-cm signal of neutral hydrogen from the Cosmic Dawn and Epoch of Reionization. If not included in calibration, this diffuse emission can severely impact the analysis and signal extraction in 21-cm experiments. We examine large-scale diffuse Galacti… ▽ More Aims: Contamination from bright diffuse Galactic thermal and non-thermal radio emission poses crucial challenges in experiments aiming to measure the 21-cm signal of neutral hydrogen from the Cosmic Dawn and Epoch of Reionization. If not included in calibration, this diffuse emission can severely impact the analysis and signal extraction in 21-cm experiments. We examine large-scale diffuse Galactic emission at 122~MHz, around the North Celestial Pole, using the AARTFAAC-HBA system. Methods: In this pilot project, we present the first-ever wide-field image produced with a single sub-band of the data recorded with the AARTFAAC-HBA. We demonstrate two methods: multiscale CLEAN and shapelet decomposition, to model the diffuse emission revealed in the image. We use angular power spectrum metrics to quantify different components of the emission and compare the performance of the two diffuse structure modelling approaches. Results: We observe that the point sources dominate the angular power spectrum ($\ell(\ell+1)C_{\ell}/2π\equiv Δ^2(\ell)$) of the emission in the field on scales $\ell\gtrsim 60$ ($\lesssim 3$~degree). The angular power spectrum after subtraction of compact sources is flat within $20\lesssim \ell \lesssim200$ range, suggesting that the residual power is dominated by the diffuse emission on scales $\ell\lesssim200$. The residual diffuse emission has a brightness temperature variance of $Δ^2_{\ell=180} = (145.64 \pm 13.61)~{\rm K}^2$ at 122~MHz on angular scales of 1~degree, and is consistent with a power-law following $C_{\ell}\propto \ell^{-2.0}$ in $20\lesssim \ell \lesssim200$ range. We also find that, in the current setup, the multiscale CLEAN is suitable to model the compact and diffuse structures on a wide range of angular scales, whereas the shapelet decomposition method better models the large scales, which are of the order of a few degrees and wider. △ Less

Submitted 7 April, 2022; v1 submitted 1 December, 2021; originally announced December 2021.

Comments: 11 pages, 9 figures, accepted (A&A)

Journal ref: A&A 662, A97 (2022)

A numerical study of 21-cm signal suppression and noise increase in direction-dependent calibration of LOFAR data

Authors: M. Mevius, F. Mertens, L. V. E. Koopmans, A. R. Offringa, S. Yatawatta, M. A. Brentjens, E. Chapman, B. Ciardi, H. Gan, B. K. Gehlot, R. Ghara, A. Ghosh, S. K. Giri, I. T. Iliev, G. Mellema, V. N. Pandey, S. Zaroubi

Abstract: We investigate systematic effects in direction dependent gain calibration in the context of the Low-Frequency Array (LOFAR) 21-cm Epoch of Reionization (EoR) experiment. The LOFAR EoR Key Science Project aims to detect the 21-cm signal of neutral hydrogen on interferometric baselines of $50-250 λ$. We show that suppression of faint signals can effectively be avoided by calibrating these short base… ▽ More We investigate systematic effects in direction dependent gain calibration in the context of the Low-Frequency Array (LOFAR) 21-cm Epoch of Reionization (EoR) experiment. The LOFAR EoR Key Science Project aims to detect the 21-cm signal of neutral hydrogen on interferometric baselines of $50-250 λ$. We show that suppression of faint signals can effectively be avoided by calibrating these short baselines using only the longer baselines. However, this approach causes an excess variance on the short baselines due to small gain errors induced by overfitting during calibration. We apply a regularised expectation-maximisation algorithm with consensus optimisation (sagecal-co) to real data with simulated signals to show that overfitting can be largely mitigated by penalising spectrally non-smooth gain solutions during calibration. This reduces the excess power with about a factor 4 in the simulations. Our results agree with earlier theoretical analysis of this bias-variance trade off and support the gain-calibration approach to the LOFAR 21-cm signal data. △ Less

Submitted 3 November, 2021; originally announced November 2021.

Comments: This is a pre-copyedited, author-produced PDF of an article accepted for publication in MNRAS following peer review

Journal ref: MNRAS 2021

Spatially constrained direction dependent calibration

Authors: Sarod Yatawatta

Abstract: Direction dependent calibration of widefield radio interferometers estimates the systematic errors along multiple directions in the sky. This is necessary because with most systematic errors that are caused by effects such as the ionosphere or the receiver beam shape, there is significant spatial variation. Fortunately, there is some deterministic behavior of these variations in most situations. W… ▽ More Direction dependent calibration of widefield radio interferometers estimates the systematic errors along multiple directions in the sky. This is necessary because with most systematic errors that are caused by effects such as the ionosphere or the receiver beam shape, there is significant spatial variation. Fortunately, there is some deterministic behavior of these variations in most situations. We enforce this underlying smooth spatial behavior of systematic errors as an additional constraint onto spectrally constrained direction dependent calibration. Using both analysis and simulations, we show that this additional spatial constraint improves the performance of multi-frequency direction dependent calibration. △ Less

Submitted 13 December, 2021; v1 submitted 13 October, 2021; originally announced October 2021.

Comments: MNRAS Accepted 2021 December 12. Received 2021 December 11; in original form 2021 October 9

LOFAR imaging of Cygnus A -- Direct detection of a turnover in the hotspot radio spectra

Authors: J. P. McKean, L. E. H. Godfrey, S. Vegetti, M. W. Wise, R. Morganti, M. J. Hardcastle, D. Rafferty, J. Anderson, I. M. Avruch, R. Beck, M. E. Bell, I. van Bemmel, M. J. Bentum, G. Bernardi, P. Best, R. Blaauw, A. Bonafede, F. Breitling, J. W. Broderick, M. Bruggen, L. Cerrigone, B. Ciardi, F. de Gasperin, A. Deller, S. Duscha , et al. (53 additional authors not shown)

Abstract: The low-frequency radio spectra of the hotspots within powerful radio galaxies can provide valuable information about the physical processes operating at the site of the jet termination. These processes are responsible for the dissipation of jet kinetic energy, particle acceleration, and magnetic-field generation. Here we report new observations of the powerful radio galaxy Cygnus A using the Low… ▽ More The low-frequency radio spectra of the hotspots within powerful radio galaxies can provide valuable information about the physical processes operating at the site of the jet termination. These processes are responsible for the dissipation of jet kinetic energy, particle acceleration, and magnetic-field generation. Here we report new observations of the powerful radio galaxy Cygnus A using the Low Frequency Array (LOFAR) between 109 and 183 MHz, at an angular resolution of ~3.5 arcsec. The radio emission of the lobes is found to have a complex spectral index distribution, with a spectral steepening found towards the centre of the source. For the first time, a turnover in the radio spectrum of the two main hotspots of Cygnus A has been directly observed. By combining our LOFAR imaging with data from the Very Large Array at higher frequencies, we show that the very rapid turnover in the hotspot spectra cannot be explained by a low-energy cut-off in the electron energy distribution, as has been previously suggested. Thermal (free-free) absorption or synchrotron self absorption models are able to describe the low-frequency spectral shape of the hotspots, however, as with previous studies, we find that the implied model parameters are unlikely, and interpreting the spectra of the hotspots remains problematic. △ Less

Submitted 31 March, 2021; originally announced March 2021.

Comments: 9 pages, 3 figures, accepted for publication in MNRAS

Journal ref: MNRAS, 463, 3143 (2016)

Deep reinforcement learning for smart calibration of radio telescopes

Authors: Sarod Yatawatta, Ian M. Avruch

Abstract: Modern radio telescopes produce unprecedented amounts of data, which are passed through many processing pipelines before the delivery of scientific results. Hyperparameters of these pipelines need to be tuned by hand to produce optimal results. Because many thousands of observations are taken during a lifetime of a telescope and because each observation will have its unique settings, the fine tuni… ▽ More Modern radio telescopes produce unprecedented amounts of data, which are passed through many processing pipelines before the delivery of scientific results. Hyperparameters of these pipelines need to be tuned by hand to produce optimal results. Because many thousands of observations are taken during a lifetime of a telescope and because each observation will have its unique settings, the fine tuning of pipelines is a tedious task. In order to automate this process of hyperparameter selection in data calibration pipelines, we introduce the use of reinforcement learning. We test two reinforcement learning techniques, twin delayed deep deterministic policy gradient (TD3) and soft actor-critic (SAC), to train an autonomous agent to perform this fine tuning. For the sake of generalization, we consider the pipeline to be a black-box system where the summarized state of the performance of the pipeline is used by the autonomous agent. The autonomous agent trained in this manner is able to determine optimal settings for diverse observations and is therefore able to perform 'smart' calibration, minimizing the need for human intervention. △ Less

Submitted 12 May, 2021; v1 submitted 5 February, 2021; originally announced February 2021.

Comments: MNRAS Accepted 2021 May 12. Received 2021 May 11; in original form 2021 February 5

Scaling performance of the SAGECal calibration package: from LOFAR to SKA

Authors: H. Spreeuw, S. Yatawatta, B. Van Werkhoven, F. Diblen

Abstract: This decade, the Square Kilometre Array (SKA) will perform its first observations. Preparations for building dishes, antennas, correlators and infrastructure are well underway. Concurrently, software for the processing of SKA observations is developed at a number of levels. At a more basic level there are the telescope monitoring and control systems and also the correlator software. On top of that… ▽ More This decade, the Square Kilometre Array (SKA) will perform its first observations. Preparations for building dishes, antennas, correlators and infrastructure are well underway. Concurrently, software for the processing of SKA observations is developed at a number of levels. At a more basic level there are the telescope monitoring and control systems and also the correlator software. On top of that, in order to deliver science ready data products, software pipelines are needed for radio frequency interference (RFI) mitigation, averaging, calibration and imaging. Here, we focus on the SAGECal calibration package, in particular on the times needed to obtain calibration solutions. This is an important aspect, since this package is now used for the Epoch of Reionization (EoR) Key Science Project of LOFAR, but will also have to run optimally on SKA1 LOW. In terms of number of stations used for observing this amounts to a factor 10 increase, from 51 to 512 stations. Consequentially, the disk space needed to store an observation will increase by a factor 100, provided the number of frequency channels remains the same. In this paper we investigate the scaling behaviour of SAGECal, whose runtimes should ideally scale linearly with the number of stations. We also explain the algorithms inside SAGECal and use them to explain its scaling behaviour. △ Less

Submitted 30 October, 2020; originally announced October 2020.

Comments: 3 pages, 3 figures

SAGECal performance with large sky models

Authors: H. Spreeuw, S. Yatawatta, B. van Werkhoven, F. Diblen

Abstract: As astronomical instruments become more sensitive, the requirements for the calibration software become more stringent; without accurate calibration solutions, thermal noise levels in images will not be reached and the scientific output of the instrument is degraded. Calibration requires bright sources with known properties, in particular with respect to their brightnesses as a function of frequen… ▽ More As astronomical instruments become more sensitive, the requirements for the calibration software become more stringent; without accurate calibration solutions, thermal noise levels in images will not be reached and the scientific output of the instrument is degraded. Calibration requires bright sources with known properties, in particular with respect to their brightnesses as a function of frequency. However, for modern radio telescopes with a huge field of view, a single calibration source does not suffice; instead a sky model with tens of thousands of sources is needed. In this work, we investigate the compute load for such complicated sky models, with up to 50,000 sources, for the SAGECal calibration package. We have chosen half of the sources in these models to be point sources and half of them extended, which we represent by Gaussian profiles. △ Less

Submitted 29 October, 2020; originally announced October 2020.

Comments: 3 pages, 4 figures

The AARTFAAC Cosmic Explorer: observations of the 21-cm power spectrum in the EDGES absorption trough

Authors: B. K. Gehlot, F. G. Mertens, L. V. E. Koopmans, A. R. Offringa, A. Shulevski, M. Mevius, M. A. Brentjens, M. Kuiack, V. N. Pandey, A. Rowlinson, A. M. Sardarabadi, H. K. Vedantham, R. A. M. J. Wijers, S. Yatawatta, S. Zaroubi

Abstract: The 21-cm absorption feature reported by the EDGES collaboration is several times stronger than that predicted by traditional astrophysical models. If genuine, a deeper absorption may lead to stronger fluctuations on the 21-cm signal on degree scales (up to 1~Kelvin in rms), allowing these fluctuations to be detectable in nearly 50~times shorter integration times compared to previous predictions.… ▽ More The 21-cm absorption feature reported by the EDGES collaboration is several times stronger than that predicted by traditional astrophysical models. If genuine, a deeper absorption may lead to stronger fluctuations on the 21-cm signal on degree scales (up to 1~Kelvin in rms), allowing these fluctuations to be detectable in nearly 50~times shorter integration times compared to previous predictions. We commenced the "AARTFAAC Cosmic Explorer" (ACE) program, that employs the AARTFAAC wide-field imager, to measure or set limits on the power spectrum of the 21-cm fluctuations in the redshift range $z = 17.9-18.6$ ($Δν= 72.36-75.09$~MHz) corresponding to the deep part of the EDGES absorption feature. Here, we present first results from two LST bins: 23.5-23.75h and 23.5-23.75h, each with 2~h of data, recorded in `semi drift-scan' mode. We demonstrate the application of the new ACE data-processing pipeline (adapted from the LOFAR-EoR pipeline) on the AARTFAAC data. We observe that noise estimates from the channel and time-differenced Stokes~$V$ visibilities agree with each other. After 2~h of integration and subtraction of bright foregrounds, we obtain $2σ$ upper limits on the 21-cm power spectrum of $Δ_{21}^2 < (8139~\textrm{mK})^2$ and $Δ_{21}^2 < (8549~\textrm{mK})^2$ at $k = 0.144~h\,\textrm{cMpc}^{-1}$ for the two LST bins. Incoherently averaging the noise bias-corrected power spectra for the two LST bins yields an upper limit of $Δ_{21}^2 < (7388~\textrm{mK})^2$ at $k = 0.144~h\,\textrm{cMpc}^{-1}$. These are the deepest upper limits thus far at these redshifts. △ Less

Submitted 5 October, 2020; originally announced October 2020.

Comments: 16 pages, 10 figures, accepted for publication in MNRAS

Polarization-based online interference mitigation in radio interferometry

Authors: Sarod Yatawatta

Abstract: Mitigation of radio frequency interference (RFI) is essential to deliver science-ready radio interferometric data to astronomers. In this paper, using dual polarized radio interferometers, we propose to use the polarization information of post-correlation interference signals to detect and mitigate them. We use the directional statistics of the polarized signals as the detection criteria and formu… ▽ More Mitigation of radio frequency interference (RFI) is essential to deliver science-ready radio interferometric data to astronomers. In this paper, using dual polarized radio interferometers, we propose to use the polarization information of post-correlation interference signals to detect and mitigate them. We use the directional statistics of the polarized signals as the detection criteria and formulate a distributed, wideband spectrum sensing problem. Using consensus optimization, we solve this in an online manner, working with mini-batches of data. We present extensive results based on simulations to demonstrate the feasibility of our method. △ Less

Submitted 23 July, 2020; v1 submitted 29 May, 2020; originally announced June 2020.

Comments: EUSIPCO 2020 accepted

Stochastic Calibration of Radio Interferometers

Authors: Sarod Yatawatta

Abstract: With ever increasing data rates produced by modern radio telescopes like LOFAR and future telescopes like the SKA, many data processing steps are overwhelmed by the amount of data that needs to be handled using limited compute resources. Calibration is one such operation that dominates the overall data processing computational cost, nonetheless, it is an essential operation to reach many science g… ▽ More With ever increasing data rates produced by modern radio telescopes like LOFAR and future telescopes like the SKA, many data processing steps are overwhelmed by the amount of data that needs to be handled using limited compute resources. Calibration is one such operation that dominates the overall data processing computational cost, nonetheless, it is an essential operation to reach many science goals. Calibration algorithms do exist that scale well with the number of stations of an array and the number of directions being calibrated. However, the remaining bottleneck is the raw data volume, which scales with the number of baselines, and which is proportional to the square of the number of stations. We propose a 'stochastic' calibration strategy where we only read in a mini-batch of data for obtaining calibration solutions, as opposed to reading the full batch of data being calibrated. Nonetheless, we obtain solutions that are valid for the full batch of data. Normally, data need to be averaged before calibration is performed to accommodate the data in size-limited compute memory. Stochastic calibration overcomes the need for data averaging before any calibration can be performed, and offers many advantages including: enabling the mitigation of faint radio frequency interference; better removal of strong celestial sources from the data; and better detection and spatial localization of fast radio transients. △ Less

Submitted 3 March, 2020; v1 submitted 2 March, 2020; originally announced March 2020.

Comments: MNRAS Accepted 2020 March 2. Received 2020 March 2 ; in original form 2020 January 27

Improved upper limits on the 21-cm signal power spectrum of neutral hydrogen at $\boldsymbol{z \approx 9.1}$ from LOFAR

Authors: F. G. Mertens, M. Mevius, L. V. E Koopmans, A. R. Offringa, G. Mellema, S. Zaroubi, M. A. Brentjens, H. Gan, B. K. Gehlot, V. N. Pandey, A. M. Sardarabadi, H. K. Vedantham, S. Yatawatta, K. M. B. Asad, B. Ciardi, E. Chapman, S. Gazagnes, R. Ghara, A. Ghosh, S. K. Giri, I. T. Iliev, V. Jelić, R. Kooistra, R. Mondal, J. Schaye , et al. (1 additional authors not shown)

Abstract: A new upper limit on the 21-cm signal power spectrum at a redshift of $z \approx 9.1$ is presented, based on 141 hours of data obtained with the Low-Frequency Array (LOFAR). The analysis includes significant improvements in spectrally-smooth gain-calibration, Gaussian Process Regression (GPR) foreground mitigation and optimally-weighted power spectrum inference. Previously seen `excess power' due… ▽ More A new upper limit on the 21-cm signal power spectrum at a redshift of $z \approx 9.1$ is presented, based on 141 hours of data obtained with the Low-Frequency Array (LOFAR). The analysis includes significant improvements in spectrally-smooth gain-calibration, Gaussian Process Regression (GPR) foreground mitigation and optimally-weighted power spectrum inference. Previously seen `excess power' due to spectral structure in the gain solutions has markedly reduced but some excess power still remains with a spectral correlation distinct from thermal noise. This excess has a spectral coherence scale of $0.25 - 0.45$\,MHz and is partially correlated between nights, especially in the foreground wedge region. The correlation is stronger between nights covering similar local sidereal times. A best 2-$σ$ upper limit of $Δ^2_{21} < (73)^2\,\mathrm{mK^2}$ at $k = 0.075\,\mathrm{h\,cMpc^{-1}}$ is found, an improvement by a factor $\approx 8$ in power compared to the previously reported upper limit. The remaining excess power could be due to residual foreground emission from sources or diffuse emission far away from the phase centre, polarization leakage, chromatic calibration errors, ionosphere, or low-level radio-frequency interference. We discuss future improvements to the signal processing chain that can further reduce or even eliminate these causes of excess power. △ Less

Submitted 19 February, 2020; v1 submitted 17 February, 2020; originally announced February 2020.

Comments: 27 pages, 14 figues, accepted in MNRAS (updated with reference to accompanying paper)

Constraining the intergalactic medium at $z\approx$ 9.1 using LOFAR Epoch of Reionization observations

Authors: R. Ghara, S. K. Giri, G. Mellema, B. Ciardi, S. Zaroubi, I. T. Iliev, L. V. E. Koopmans, E. Chapman, S. Gazagnes, B. K. Gehlot, A. Ghosh, V. Jelic, F. G. Mertens, R. Mondal, J. Schaye, M. B. Silva, K. M. B. Asad, R. Kooistra, M. Mevius, A. R. Offringa, V. N. Pandey, S. Yatawatta

Abstract: We derive constraints on the thermal and ionization states of the intergalactic medium (IGM) at redshift $\approx$ 9.1 using new upper limits on the 21-cm power spectrum measured by the LOFAR radio-telescope and a prior on the ionized fraction at that redshift estimated from recent cosmic microwave background (CMB) observations. We have used results from the reionization simulation code GRIZZLY an… ▽ More We derive constraints on the thermal and ionization states of the intergalactic medium (IGM) at redshift $\approx$ 9.1 using new upper limits on the 21-cm power spectrum measured by the LOFAR radio-telescope and a prior on the ionized fraction at that redshift estimated from recent cosmic microwave background (CMB) observations. We have used results from the reionization simulation code GRIZZLY and a Bayesian inference framework to constrain the parameters which describe the physical state of the IGM. We find that, if the gas heating remains negligible, an IGM with ionized fraction $\gtrsim 0.13$ and a distribution of the ionized regions with a characteristic size $\gtrsim 8 ~h^{-1}$ comoving megaparsec (Mpc) and a full width at the half maximum (FWHM) $\gtrsim 16 ~h^{-1}$ Mpc is ruled out. For an IGM with a uniform spin temperature $T_{\rm S} \gtrsim 3$ K, no constraints on the ionized component can be computed. If the large-scale fluctuations of the signal are driven by spin temperature fluctuations, an IGM with a volume fraction $\lesssim 0.34$ of heated regions with a temperature larger than CMB, average gas temperature 7-160 K and a distribution of the heated regions with characteristic size 3.5-70 $h^{-1}$ Mpc and FWHM of $\lesssim 110$ $h^{-1}$ Mpc is ruled out. These constraints are within the 95 per cent credible intervals. With more stringent future upper limits from LOFAR at multiple redshifts, the constraints will become tighter and will exclude an increasingly large region of the parameter space. △ Less

Submitted 17 February, 2020; originally announced February 2020.

Comments: 22 pages, 15 Figures, 5 tables, Accepted for publication in MNRAS

GPU acceleration of the SAGECal calibration package for the SKA

Authors: Hanno Spreeuw, Ben van Werkhoven, Sarod Yatawatta, Faruk Diblen

Abstract: SAGECal has been designed to find the most accurate calibration solutions for low radio frequency imaging observations, with minimum artefacts due to incomplete sky models. SAGECAL is developed to handle extremely large datasets, e.g., when the number of frequency bands greatly exceeds the number of available nodes on a compute cluster. Accurate calibration solutions are derived at the expense of… ▽ More SAGECal has been designed to find the most accurate calibration solutions for low radio frequency imaging observations, with minimum artefacts due to incomplete sky models. SAGECAL is developed to handle extremely large datasets, e.g., when the number of frequency bands greatly exceeds the number of available nodes on a compute cluster. Accurate calibration solutions are derived at the expense of large computational loads, which require distributed computing and modern compute devices, such as GPUs, to decrease runtimes. In this work, we investigate if the GPU version of SAGECal scales well enough to meet the requirements for the Square Kilometre Array and we compare its performance with the CPU version. △ Less

Submitted 30 October, 2019; originally announced October 2019.

Comments: 4 pages, 4 figures, presented at ADASS 2019

A Stochastic LBFGS Algorithm for Radio Interferometric Calibration

Authors: Sarod Yatawatta, Lukas De Clercq, Hanno Spreeuw, Faruk Diblen

Abstract: We present a stochastic, limited-memory Broyden Fletcher Goldfarb Shanno (LBFGS) algorithm that is suitable for handling very large amounts of data. A direct application of this algorithm is radio interferometric calibration of raw data at fine time and frequency resolution. Almost all existing radio interferometric calibration algorithms assume that it is possible to fit the dataset being calibra… ▽ More We present a stochastic, limited-memory Broyden Fletcher Goldfarb Shanno (LBFGS) algorithm that is suitable for handling very large amounts of data. A direct application of this algorithm is radio interferometric calibration of raw data at fine time and frequency resolution. Almost all existing radio interferometric calibration algorithms assume that it is possible to fit the dataset being calibrated into memory. Therefore, the raw data is averaged in time and frequency to reduce its size by many orders of magnitude before calibration is performed. However, this averaging is detrimental for the detection of some signals of interest that have narrow bandwidth and time duration such as fast radio bursts (FRBs). Using the proposed algorithm, it is possible to calibrate data at such a fine resolution that they cannot be entirely loaded into memory, thus preserving such signals. As an additional demonstration, we use the proposed algorithm for training deep neural networks and compare the performance against the mainstream first order optimization algorithms that are used in deep learning. △ Less

Submitted 13 April, 2019; v1 submitted 11 April, 2019; originally announced April 2019.

Comments: Draft, final version in IEEE Data Science Workshop 2019 proceedings

Statistical Performance of Radio Interferometric Calibration

Authors: Sarod Yatawatta

Abstract: Calibration is an essential step in radio interferometric data processing that corrects the data for systematic errors and in addition, subtracts bright foreground interference to reveal weak signals hidden in the residual. These weak and unknown signals are much sought after to reach many science goals but the effect of calibration on such signals is an ever present concern. The main reason for t… ▽ More Calibration is an essential step in radio interferometric data processing that corrects the data for systematic errors and in addition, subtracts bright foreground interference to reveal weak signals hidden in the residual. These weak and unknown signals are much sought after to reach many science goals but the effect of calibration on such signals is an ever present concern. The main reason for this is the incompleteness of the model used in calibration. Distributed calibration based on consensus optimization has been shown to mitigate the effect due to model incompleteness by calibrating data covering a wide bandwidth in a computationally efficient manner. In this paper, we study the statistical performance of direction dependent distributed calibration, i.e., the distortion caused by calibration on the residual statistics. In order to study this, we consider the map** between the input uncalibrated data and the output residual data. We derive an analytical relationship for the influence of the input on the residual and use this to find the relationship between the input and output probability density functions. Using simulations we show that the smallest eigenvalue of the Jacobian of this map** is a reliable indicator of the statistical performance of calibration. The analysis developed in this paper can also be applied to other data processing steps in radio interferometry such as imaging and foreground subtraction as well as to many other machine learning problems. △ Less

Submitted 30 April, 2019; v1 submitted 27 February, 2019; originally announced February 2019.

Comments: MNRAS Accepted 2019 April 30. Received 2019 April 9; in original form 2018 December 6

The first power spectrum limit on the 21-cm signal of neutral hydrogen during the Cosmic Dawn at z=20-25 from LOFAR

Authors: B. K. Gehlot, F. G. Mertens, L. V. E. Koopmans, M. A. Brentjens, S. Zaroubi, B. Ciardi, A. Ghosh, M. Hatef, I. T. Iliev, V. Jelić, R. Kooistra, F. Krause, G. Mellema, M. Mevius, M. Mitra, A. R. Offringa, V. N. Pandey, A. M. Sardarabadi, J. Schaye, M. B. Silva, H. K. Vedantham, S. Yatawatta

Abstract: Observations of the redshifted 21-cm hyperfine line of neutral hydrogen from early phases of the Universe such as Cosmic Dawn and the Epoch of Reionization promise to open a new window onto the early formation of stars and galaxies. We present the first upper limits on the power spectrum of redshifted 21-cm brightness temperature fluctuations in the redshift range $z = 19.8 - 25.2$ ($54-68$ MHz fr… ▽ More Observations of the redshifted 21-cm hyperfine line of neutral hydrogen from early phases of the Universe such as Cosmic Dawn and the Epoch of Reionization promise to open a new window onto the early formation of stars and galaxies. We present the first upper limits on the power spectrum of redshifted 21-cm brightness temperature fluctuations in the redshift range $z = 19.8 - 25.2$ ($54-68$ MHz frequency range) using 14 hours of data obtained with the LOFAR-Low Band Antenna (LBA) array. We also demonstrate the application of a multiple pointing calibration technique to calibrate the LOFAR-LBA dual-pointing observations centred on the North Celestial Pole and the radio galaxy 3C220.3. We observe an unexplained excess of $\sim 30-50\%$ in Stokes $I$ noise compared to Stokes $V$ for the two observed fields, which decorrelates on $\gtrsim 12$ seconds and might have a physical origin. We show that enforcing smoothness of gain errors along frequency direction during calibration reduces the additional variance in Stokes $I$ compared Stokes $V$ introduced by the calibration on sub-band level. After subtraction of smooth foregrounds, we achieve a $2σ$ upper limit on the 21-cm power spectrum of $Δ_{21}^2 < (14561\,\text{mK})^2$ at $k\sim 0.038\,h\,\text{cMpc}^{-1}$ and $Δ_{21}^2 < (14886\,\text{mK})^2$ at $k\sim 0.038 \,h\,\text{cMpc}^{-1}$ for the 3C220 and NCP fields respectively and both upper limits are consistent with each other. The upper limits for the two fields are still dominated by systematics on most $k$ modes. △ Less

Submitted 20 July, 2019; v1 submitted 18 September, 2018; originally announced September 2018.

Comments: 19 pages, 16 figures, accepted for publication in MNRAS

Performance Analysis of Distributed Radio Interferometric Calibration

Authors: Sarod Yatawatta

Abstract: Distributed calibration based on consensus optimization is a computationally efficient method to calibrate large radio interferometers such as LOFAR and SKA. Calibrating along multiple directions in the sky and removing the bright foreground signal is a crucial step in many science cases in radio interferometry. The residual data contain weak signals of huge scientific interest and of particular c… ▽ More Distributed calibration based on consensus optimization is a computationally efficient method to calibrate large radio interferometers such as LOFAR and SKA. Calibrating along multiple directions in the sky and removing the bright foreground signal is a crucial step in many science cases in radio interferometry. The residual data contain weak signals of huge scientific interest and of particular concern is the effect of incomplete sky models used in calibration on the residual. In order to study this, we consider the map** between the input uncalibrated data and the output residual data. We derive an analytical relationship between the input and output probability density functions which can be used to study the performance of calibration. △ Less

Submitted 28 October, 2021; v1 submitted 1 May, 2018; originally announced May 2018.

Comments: published in the Proceedings of IEEE Sensor Array and Multichannel Signal Processing Workshop (IEEE SAM 2018), published by IEEE

Distributed Model Construction in Radio Interferometric Calibration

Authors: Sarod Yatawatta

Abstract: Calibration of a typical radio interferometric array yields thousands of parameters as solutions. These solutions contain valuable information about the systematic errors in the data (ionosphere and beam shape). This information could be reused in calibration to improve the accuracy and also can be fed into imaging to improve the fidelity. We propose a distributed optimization strategy to construc… ▽ More Calibration of a typical radio interferometric array yields thousands of parameters as solutions. These solutions contain valuable information about the systematic errors in the data (ionosphere and beam shape). This information could be reused in calibration to improve the accuracy and also can be fed into imaging to improve the fidelity. We propose a distributed optimization strategy to construct models for the systematic errors in the data using the calibration solutions. We formulate this as an elastic net regularized distributed optimization problem which we solve using the alternating direction method of multipliers (ADMM) algorithm. We give simulation results to show the feasibility of the proposed distributed model construction scheme. △ Less

Submitted 29 January, 2018; originally announced January 2018.

Comments: Draft, to be published in the Proceedings of the 2018 International Conference on Acoustics, Speech and Signal Processing (IEEE ICASSP 2018), published by IEEE

Data Multiplexing in Radio Interferometric Calibration

Authors: Sarod Yatawatta, Faruk Diblen, Hanno Spreeuw, L. V. E. Koopmans

Abstract: New and upcoming radio interferometers will produce unprecedented amounts of data that demand extremely powerful computers for processing. This is a limiting factor due to the large computational power and energy costs involved. Such limitations restrict several key data processing steps in radio interferometry. One such step is calibration where systematic errors in the data are determined and co… ▽ More New and upcoming radio interferometers will produce unprecedented amounts of data that demand extremely powerful computers for processing. This is a limiting factor due to the large computational power and energy costs involved. Such limitations restrict several key data processing steps in radio interferometry. One such step is calibration where systematic errors in the data are determined and corrected. Accurate calibration is an essential component in reaching many scientific goals in radio astronomy and the use of consensus optimization that exploits the continuity of systematic errors across frequency significantly improves calibration accuracy. In order to reach full consensus, data at all frequencies need to be calibrated simultaneously. In the SKA regime, this can become intractable if the available compute agents do not have the resources to process data from all frequency channels simultaneously. In this paper, we propose a multiplexing scheme that is based on the alternating direction method of multipliers (ADMM) with cyclic updates. With this scheme, it is possible to simultaneously calibrate the full dataset using far fewer compute agents than the number of frequencies at which data are available. We give simulation results to show the feasibility of the proposed multiplexing scheme in simultaneously calibrating a full dataset when a limited number of compute agents are available. △ Less

Submitted 29 November, 2017; v1 submitted 28 November, 2017; originally announced November 2017.

Comments: MNRAS Accepted 2017 November 28. Received 2017 November 28; in original form 2017 July 06

Adaptive ADMM in Distributed Radio Interferometric Calibration

Authors: Sarod Yatawatta, Faruk Diblen, Hanno Spreeuw

Abstract: Distributed radio interferometric calibration based on consensus optimization has been shown to improve the estimation of systematic errors in radio astronomical observations. The intrinsic continuity of systematic errors across frequency is used by a consensus polynomial to penalize traditional calibration. Consensus is achieved via the use of alternating direction method of multipliers (ADMM) al… ▽ More Distributed radio interferometric calibration based on consensus optimization has been shown to improve the estimation of systematic errors in radio astronomical observations. The intrinsic continuity of systematic errors across frequency is used by a consensus polynomial to penalize traditional calibration. Consensus is achieved via the use of alternating direction method of multipliers (ADMM) algorithm. In this paper, we extend the existing distributed calibration algorithms to use ADMM with an adaptive penalty parameter update. Compared to a fixed penalty, its adaptive update has been shown to perform better in diverse applications of ADMM. In this paper, we compare two such popular penalty parameter update schemes: residual balance penalty update and spectral penalty update (Barzilai-Borwein). We apply both schemes to distributed radio interferometric calibration and compare their performance against ADMM with a fixed penalty parameter. Simulations show that both methods of adaptive penalty update improve the convergence of ADMM but the spectral penalty parameter update shows more stability. △ Less

Submitted 16 October, 2017; originally announced October 2017.

Comments: Draft, to be published in the Proceedings of the 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP) (IEEE CAMSAP 2017), published by IEEE

Wide-field LOFAR-LBA power-spectra analyses: Impact of calibration, polarization leakage and ionosphere

Authors: B. K. Gehlot, L. V. E. Koopmans, A. G. de Bruyn, S. Zaroubi, M. A. Brentjens, K. M. B. Asad, M. Hatef, V. Jelic, M. Mevius, A. R. Offringa, V. N. Pandey, S. Yatawatta

Abstract: Contamination due to foregrounds (Galactic and Extra-galactic), calibration errors and ionospheric effects pose major challenges in detection of the cosmic 21 cm signal in various Epoch of Reionization (EoR) experiments. We present the results of a pilot study of a field centered on 3C196 using LOFAR Low Band (56-70 MHz) observations, where we quantify various wide field and calibration effects su… ▽ More Contamination due to foregrounds (Galactic and Extra-galactic), calibration errors and ionospheric effects pose major challenges in detection of the cosmic 21 cm signal in various Epoch of Reionization (EoR) experiments. We present the results of a pilot study of a field centered on 3C196 using LOFAR Low Band (56-70 MHz) observations, where we quantify various wide field and calibration effects such as gain errors, polarized foregrounds, and ionospheric effects. We observe a `pitchfork' structure in the 2D power spectrum of the polarized intensity in delay-baseline space, which leaks into the modes beyond the instrumental horizon (EoR/CD window). We show that this structure largely arises due to strong instrumental polarization leakage ($\sim30\%$) towards {Cas\,A} ($\sim21$ kJy at 81 MHz, brightest source in northern sky), which is far away from primary field of view. We measure an extremely small ionospheric diffractive scale ($r_{\text{diff}} \approx 430$ m at 60 MHz) towards {Cas\,A} resembling pure Kolmogorov turbulence compared to $r_{\text{diff}} \sim3 - 20$ km towards zenith at 150 MHz for typical ionospheric conditions. This is one of the smallest diffractive scales ever measured at these frequencies. Our work provides insights in understanding the nature of aforementioned effects and mitigating them in future Cosmic Dawn observations (e.g. with SKA-low and HERA) in the same frequency window. △ Less

Submitted 26 April, 2018; v1 submitted 22 September, 2017; originally announced September 2017.

Comments: 20 pages, 11 figures, accepted for publication in MNRAS

Upper limits on the 21-cm Epoch of Reionization power spectrum from one night with LOFAR

Authors: A. H. Patil, S. Yatawatta, L. V. E. Koopmans, A. G. de Bruyn, M. A. Brentjens, S. Zaroubi, K. M. B. Asad, M. Hatef, V. Jelic, M. Mevius, A. R. Offringa, V. N. Pandey, H. Vedantham, F. B. Abdalla, W. N. Brouw, E. Chapman, B. Ciardi, B. K. Gehlot, A. Ghosh, G. Harker, I. T. Iliev, K. Kakiichi, S. Majumdar, M. B. Silva, G. Mellema , et al. (3 additional authors not shown)

Abstract: We present the first limits on the Epoch of Reionization (EoR) 21-cm HI power spectra, in the redshift range $z=7.9-10.6$, using the Low-Frequency Array (LOFAR) High-Band Antenna (HBA). In total 13\,h of data were used from observations centred on the North Celestial Pole (NCP). After subtraction of the sky model and the noise bias, we detect a non-zero $Δ^2_{\rm I} = (56 \pm 13 {\rm mK})^2$ (1-… ▽ More We present the first limits on the Epoch of Reionization (EoR) 21-cm HI power spectra, in the redshift range $z=7.9-10.6$, using the Low-Frequency Array (LOFAR) High-Band Antenna (HBA). In total 13\,h of data were used from observations centred on the North Celestial Pole (NCP). After subtraction of the sky model and the noise bias, we detect a non-zero $Δ^2_{\rm I} = (56 \pm 13 {\rm mK})^2$ (1-$σ$) excess variance and a best 2-$σ$ upper limit of $Δ^2_{\rm 21} < (79.6 {\rm mK})^2$ at $k=0.053$$h$cMpc$^{-1}$ in the range $z=$9.6-10.6. The excess variance decreases when optimizing the smoothness of the direction- and frequency-dependent gain calibration, and with increasing the completeness of the sky model. It is likely caused by (i) residual side-lobe noise on calibration baselines, (ii) leverage due to non-linear effects, (iii) noise and ionosphere-induced gain errors, or a combination thereof. Further analyses of the excess variance will be discussed in forthcoming publications. △ Less

Submitted 28 February, 2017; originally announced February 2017.

Comments: 17 pages, 8 figures, accepted by ApJ

LBCS: the LOFAR Long-Baseline Calibrator Survey

Authors: N. Jackson, A. Tagore, A. Deller, J. Moldón, E. Varenius, L. Morabito, O. Wucknitz, T. Carozzi, J. Conway, A. Drabent, A. Kapinska, E. Orrù, M. Brentjens, R. Blaauw, G. Kuper, J. Sluman, J. Schaap, N. Vermaas, M. Iacobelli, L. Cerrigone, A. Shulevski, S. ter Veen, R. Fallows, R. Pizzo, M. Sipior , et al. (54 additional authors not shown)

Abstract: (abridged). We outline LBCS (the LOFAR Long-Baseline Calibrator Survey), whose aim is to identify sources suitable for calibrating the highest-resolution observations made with the International LOFAR Telescope, which include baselines >1000 km. Suitable sources must contain significant correlated flux density (50-100mJy) at frequencies around 110--190~MHz on scales of a few hundred mas. At least… ▽ More (abridged). We outline LBCS (the LOFAR Long-Baseline Calibrator Survey), whose aim is to identify sources suitable for calibrating the highest-resolution observations made with the International LOFAR Telescope, which include baselines >1000 km. Suitable sources must contain significant correlated flux density (50-100mJy) at frequencies around 110--190~MHz on scales of a few hundred mas. At least for the 200--300-km international baselines, we find around 1 suitable calibrator source per square degree over a large part of the northern sky, in agreement with previous work. This should allow a randomly selected target to be successfully phase calibrated on the international baselines in over 50% of cases. Products of the survey include calibrator source lists and fringe-rate and delay maps of wide areas -- typically a few degrees -- around each source. The density of sources with significant correlated flux declines noticeably with baseline length over the range 200--600~km, with good calibrators on the longest baselines appearing only at the rate of 0.5 per square degree. Coherence times decrease from 1--3 minutes on 200-km baselines to about 1 minute on 600-km baselines, suggesting that ionospheric phase variations contain components with scales of a few hundred kilometres. The longest median coherence time, at just over 3 minutes, is seen on the DE609 baseline, which at 227km is close to being the shortest. We see median coherence times of between 80 and 110 seconds on the four longest baselines (580--600~km), and about 2 minutes for the other baselines. The success of phase transfer from calibrator to target is shown to be influenced by distance, in a manner that suggests a coherence patch at 150-MHz of the order of 1 degree. △ Less

Submitted 9 September, 2016; v1 submitted 6 August, 2016; originally announced August 2016.

Comments: Accepted by Astronomy & Astrophysics. Error in figure 6 corrected

Journal ref: A&A 595, A86 (2016)

Probing Ionospheric Structures using the LOFAR radio telescope

Authors: M. Mevius, S. van der Tol, V. N. Pandey, H. K. Vedantham, M. A. Brentjens, A. G. de Bruyn, F. B. Abdalla, K. M. B. Asad, J. D. Bregman, W. N. Brouw, S. Bus, E. Chapman, B. Ciardi, E. R. Fernandez, A. Ghosh, G. Harker, I. T. Iliev, V. Jelić, S. Kazemi, L. V. E. Koopmans, J. E. Noordam, A. R. Offringa, A. H. Patil, R. J. van Weeren, S. Wijnholds , et al. (2 additional authors not shown)

Abstract: LOFAR is the LOw Frequency Radio interferometer ARray located at mid-latitude ($52^{\circ} 53'N$). Here, we present results on ionospheric structures derived from 29 LOFAR nighttime observations during the winters of 2012/2013 and 2013/2014. We show that LOFAR is able to determine differential ionospheric TEC values with an accuracy better than 1 mTECU over distances ranging between 1 and 100 km.… ▽ More LOFAR is the LOw Frequency Radio interferometer ARray located at mid-latitude ($52^{\circ} 53'N$). Here, we present results on ionospheric structures derived from 29 LOFAR nighttime observations during the winters of 2012/2013 and 2013/2014. We show that LOFAR is able to determine differential ionospheric TEC values with an accuracy better than 1 mTECU over distances ranging between 1 and 100 km. For all observations the power law behavior of the phase structure function is confirmed over a long range of baseline lengths, between $1$ and $80$ km, with a slope that is in general larger than the $5/3$ expected for pure Kolmogorov turbulence. The measured average slope is $1.89$ with a one standard deviation spread of $0.1$. The diffractive scale, i.e. the length scale where the phase variance is $1\, \mathrm{rad^2}$, is shown to be an easily obtained single number that represents the ionospheric quality of a radio interferometric observation. A small diffractive scale is equivalent to high phase variability over the field of view as well as a short time coherence of the signal, which limits calibration and imaging quality. For the studied observations the diffractive scales at $150$ MHz vary between $3.5$ and $30\,$ km. A diffractive scale above $5$ km, pertinent to about $90 \%$ of the observations, is considered sufficient for the high dynamic range imaging needed for the LOFAR Epoch of Reionization project. For most nights the ionospheric irregularities were anisotropic, with the structures being aligned with the Earth magnetic field in about $60\%$ of the observations. △ Less

Submitted 15 June, 2016; originally announced June 2016.

Comments: accepted for publication in Radio Science

Journal ref: Radio Sci. 51 (2016) 927-941

Fine tuning consensus optimization for distributed radio interferometric calibration

Authors: Sarod Yatawatta

Abstract: We recently proposed the use of consensus optimization as a viable and effective way to improve the quality of calibration of radio interferometric data. We showed that it is possible to obtain far more accurate calibration solutions and also to distribute the compute load across a network of computers by using this technique. A crucial aspect in any consensus optimization problem is the selection… ▽ More We recently proposed the use of consensus optimization as a viable and effective way to improve the quality of calibration of radio interferometric data. We showed that it is possible to obtain far more accurate calibration solutions and also to distribute the compute load across a network of computers by using this technique. A crucial aspect in any consensus optimization problem is the selection of the penalty parameter used in the alternating direction method of multipliers (ADMM) iterations. This affects the convergence speed as well as the accuracy. In this paper, we use the Hessian of the cost function used in calibration to appropriately select this penalty. We extend our results to a multi-directional calibration setting, where we propose to use a penalty scaled by the squared intensity of each direction. △ Less

Submitted 30 May, 2016; originally announced May 2016.

Comments: Draft, to be published in the Proceedings of the 24th European Signal Processing Conference (EUSIPCO-2016) in 2016, published by EURASIP

Systematic biases in low frequency radio interferometric data due to calibration: the LOFAR EoR case

Authors: A**kya H. Patil, Sarod Yatawatta, Saleem Zaroubi, Léon V. E. Koopmans, A. G. de Bruyn, Vibor Jelić, Benedetta Ciardi, Ilian T. Iliev, Maaijke Mevius, Vishambhar N. Pandey, Bharat K. Gehlot

Abstract: The redshifted 21 cm line of neutral hydrogen is a promising probe of the Epoch of Reionization (EoR). However, its detection requires a thorough understanding and control of the systematic errors. We study two systematic biases observed in the LOFAR EoR residual data after calibration and subtraction of bright discrete foreground sources. The first effect is a suppression in the diffuse foregroun… ▽ More The redshifted 21 cm line of neutral hydrogen is a promising probe of the Epoch of Reionization (EoR). However, its detection requires a thorough understanding and control of the systematic errors. We study two systematic biases observed in the LOFAR EoR residual data after calibration and subtraction of bright discrete foreground sources. The first effect is a suppression in the diffuse foregrounds, which could potentially mean a suppression of the 21 cm signal. The second effect is an excess of noise beyond the thermal noise. The excess noise shows fluctuations on small frequency scales, and hence it can not be easily removed by foreground removal or avoidance methods. Our analysis suggests that sidelobes of residual sources due to the chromatic point spread function and ionospheric scintillation can not be the dominant causes of the excess noise. Rather, both the suppression of diffuse foregrounds and the excess noise can occur due to calibration with an incomplete sky model containing predominantly bright discrete sources. We show that calibrating only on bright sources can cause suppression of other signals and introduce an excess noise in the data. The levels of the suppression and excess noise depend on the relative flux of sources which are not included in the model with respect to the flux of modeled sources. We discuss possible solutions such as using only long baselines to calibrate the interferometric gain solutions as well as simultaneous multi-frequency calibration along with their benefits and shortcomings. △ Less

Submitted 24 May, 2016; originally announced May 2016.

Comments: Submitted to MNRAS, 14 pages, 11 figures

Polarization leakage in epoch of reionization windows: II. Primary beam model and direction dependent calibration

Authors: K. M. B. Asad, L. V. E. Koopmans, V. Jelić, A. Ghosh, F. B. Abdalla, M. A. Brentjens, A. G. de Bruyn, B. Ciardi, B. K. Gehlot, I. T. Iliev, M. Mevius, V. N. Pandey, S. Yatawatta, S. Zaroubi

Abstract: Leakage of diffuse polarized emission into Stokes I caused by the polarized primary beam of the instrument might mimic the spectral structure of the 21-cm signal coming from the epoch of reionization (EoR) making their separation difficult. Therefore, understanding polarimetric performance of the antenna is crucial for a successful detection of the EoR signal. Here, we have calculated the accuracy… ▽ More Leakage of diffuse polarized emission into Stokes I caused by the polarized primary beam of the instrument might mimic the spectral structure of the 21-cm signal coming from the epoch of reionization (EoR) making their separation difficult. Therefore, understanding polarimetric performance of the antenna is crucial for a successful detection of the EoR signal. Here, we have calculated the accuracy of the nominal model beam of LOFAR in predicting the leakage from Stokes I to Q, U by comparing them with the corresponding leakage of compact sources actually observed in the 3C295 field. We have found that the model beam has errors of less than or equal to 10% on the predicted levels of leakage of ~1% within the field of view, i. e. if the leakage is taken out perfectly using this model the leakage will reduce to $10^{-3}$ of the Stokes I flux. If similar levels of accuracy can be obtained in removing leakage from Stokes Q, U to I, we can say, based on the results of our previous paper, that the removal of this leakage using this beam model would ensure that the leakage is well below the expected EoR signal in almost the whole instrumental k-space of the cylindrical power spectrum. We have also shown here that direction dependent calibration can remove instrumentally polarized compact sources, given an unpolarized sky model, very close to the local noise level. △ Less

Submitted 27 July, 2016; v1 submitted 15 April, 2016; originally announced April 2016.

Comments: 13 pages, 15 figures, accepted in MNRAS on 26 July 2016

A large light-mass component of cosmic rays at 10^{17} - 10^{17.5} eV from radio observations

Authors: S. Buitink, A. Corstanje, H. Falcke, J. R. Hörandel, T. Huege, A. Nelles, J. P. Rachen, L. Rossetto, P . Schellart, O. Scholten, S. ter Veen, S. Thoudam, T. N. G. Trinh, J. Anderson, A. Asgekar, I. M. Avruch, M. E. Bell, M. J. Bentum, G. Bernardi, P. Best, A. Bonafede, F. Breitling, J. W. Broderick, W. N. Brouw, M. Brüggen , et al. (79 additional authors not shown)

Abstract: Cosmic rays are the highest energy particles found in nature. Measurements of the mass composition of cosmic rays between 10^{17} eV and 10^{18} eV are essential to understand whether this energy range is dominated by Galactic or extragalactic sources. It has also been proposed that the astrophysical neutrino signal comes from accelerators capable of producing cosmic rays of these energies. Cosmic… ▽ More Cosmic rays are the highest energy particles found in nature. Measurements of the mass composition of cosmic rays between 10^{17} eV and 10^{18} eV are essential to understand whether this energy range is dominated by Galactic or extragalactic sources. It has also been proposed that the astrophysical neutrino signal comes from accelerators capable of producing cosmic rays of these energies. Cosmic rays initiate cascades of secondary particles (air showers) in the atmosphere and their masses are inferred from measurements of the atmospheric depth of the shower maximum, Xmax, or the composition of shower particles reaching the ground. Current measurements suffer from either low precision, or a low duty cycle and a high energy threshold. Radio detection of cosmic rays is a rapidly develo** technique, suitable for determination of Xmax with a duty cycle of in principle nearly 100%. The radiation is generated by the separation of relativistic charged particles in the geomagnetic field and a negative charge excess in the shower front. Here we report radio measurements of Xmax with a mean precision of 16 g/cm^2 between 10^{17}-10^{17.5} eV. Because of the high resolution in $Xmax we can determine the mass spectrum and find a mixed composition, containing a light mass fraction of ~80%. Unless the extragalactic component becomes significant already below 10^{17.5} eV, our measurements indicate an additional Galactic component dominating at this energy range. △ Less

Submitted 1 May, 2016; v1 submitted 4 March, 2016; originally announced March 2016.

Comments: 35 pages, 11 figures, updated version: Pierre Auger Observatory data ICRC 2015 added to Fig 2

Journal ref: Nature 531, 70 (2016)

LOFAR MSSS: Detection of a low-frequency radio transient in 400 hrs of monitoring of the North Celestial Pole

Authors: A. J. Stewart, R. P. Fender, J. W. Broderick, T. E. Hassall, T. Muñoz-Darias, A. Rowlinson, J. D. Swinbank, T. D. Staley, G. J. Molenaar, B. Scheers, T. L. Grobler, M. Pietka, G. Heald, J. P. McKean, M. E. Bell, A. Bonafede, R. P. Breton, D. Carbone, Y. Cendes, A. O. Clarke, S. Corbel, F. de Gasperin, J. Eislöffel, H. Falcke, C. Ferrari , et al. (77 additional authors not shown)

Abstract: We present the results of a four-month campaign searching for low-frequency radio transients near the North Celestial Pole with the Low-Frequency Array (LOFAR), as part of the Multifrequency Snapshot Sky Survey (MSSS). The data were recorded between 2011 December and 2012 April and comprised 2149 11-minute snapshots, each covering 175 deg^2. We have found one convincing candidate astrophysical tra… ▽ More We present the results of a four-month campaign searching for low-frequency radio transients near the North Celestial Pole with the Low-Frequency Array (LOFAR), as part of the Multifrequency Snapshot Sky Survey (MSSS). The data were recorded between 2011 December and 2012 April and comprised 2149 11-minute snapshots, each covering 175 deg^2. We have found one convincing candidate astrophysical transient, with a duration of a few minutes and a flux density at 60 MHz of 15-25 Jy. The transient does not repeat and has no obvious optical or high-energy counterpart, as a result of which its nature is unclear. The detection of this event implies a transient rate at 60 MHz of 3.9 (+14.7, -3.7) x 10^-4 day^-1 deg^-2, and a transient surface density of 1.5 x 10^-5 deg^-2, at a 7.9-Jy limiting flux density and ~10-minute time-scale. The campaign data were also searched for transients at a range of other time-scales, from 0.5 to 297 min, which allowed us to place a range of limits on transient rates at 60 MHz as a function of observation duration. △ Less

Submitted 30 November, 2015; originally announced December 2015.

Comments: 23 pages, 16 figures, 5 tables, accepted for publication in MNRAS

Journal ref: Monthly Notices of the Royal Astronomical Society, 2016, Volume 456, Issue 3, p.2321-2342

Wide-field LOFAR imaging of the field around the double-double radio galaxy B1834+620: A fresh view on a restarted AGN and doubeltjes

Authors: E. Orrú, S. van Velzen, R. F. Pizzo, S. Yatawatta, R. Paladino, M. Iacobelli, M. Murgia, H. Falcke, R. Morganti, A. G. de Bruyn, C. Ferrari

Abstract: The existence of double-double radio galaxies (DDRGs) is evidence for recurrent jet activity in AGN, as expected from standard accretion models. A detailed study of these rare sources provides new perspectives for investigating the AGN duty cycle, AGN-galaxy feedback, and accretion mechanisms. Large catalogues of radio sources provide statistical information about the evolution of the radio-loud A… ▽ More The existence of double-double radio galaxies (DDRGs) is evidence for recurrent jet activity in AGN, as expected from standard accretion models. A detailed study of these rare sources provides new perspectives for investigating the AGN duty cycle, AGN-galaxy feedback, and accretion mechanisms. Large catalogues of radio sources provide statistical information about the evolution of the radio-loud AGN population out to high redshifts. Using wide-field imaging with the LOFAR telescope, we study both a well-known DDRG as well as a large number of radio sources in the field of view. We present a high resolution image of the DDRG B1834+620 obtained at 144 MHz using LOFAR commissioning data. Our image covers about 100 square degrees and contains over 1000 sources. The four components of the DDRG B1834+620 have been resolved for the first time at 144 MHz. Inner lobes were found to point towards the direction of the outer lobes, unlike standard FR~II sources. Polarized emission was detected in the northern outer lobe. The high spatial resolution allows the identification of a large number of small double-lobed radio sources; roughly 10% of all sources in the field are doubles with a separation smaller than 1 arcmin. The spectral fit of the four components is consistent with a scenario in which the outer lobes are still active or the jets recently switched off, while emission of the inner lobes is the result of a mix-up of new and old jet activity. From the presence of the newly extended features in the inner lobes of the DDRG, we can infer that the mechanism responsible for their formation is the bow shock that is driven by the newly launched jet. We find that the density of the small doubles exceeds the density of FR-II sources with similar properties at 1.4 GHz, but this difference becomes smaller for low flux densities. △ Less

Submitted 2 October, 2015; originally announced October 2015.

Comments: 13 pages, 8 figures, Accepted for publication in Astronomy & Astrophysics

Journal ref: A&A 584, A112 (2015)

The LOFAR Multifrequency Snapshot Sky Survey (MSSS) I. Survey description and first results

Authors: G. H. Heald, R. F. Pizzo, E. Orrú, R. P. Breton, D. Carbone, C. Ferrari, M. J. Hardcastle, W. Jurusik, G. Macario, D. Mulcahy, D. Rafferty, A. Asgekar, M. Brentjens, R. A. Fallows, W. Frieswijk, M. C. Toribio, B. Adebahr, M. Arts, M. R. Bell, A. Bonafede, J. Bray, J. Broderick, T. Cantwell, P. Carroll, Y. Cendes , et al. (125 additional authors not shown)

Abstract: We present the Multifrequency Snapshot Sky Survey (MSSS), the first northern-sky LOFAR imaging survey. In this introductory paper, we first describe in detail the motivation and design of the survey. Compared to previous radio surveys, MSSS is exceptional due to its intrinsic multifrequency nature providing information about the spectral properties of the detected sources over more than two octave… ▽ More We present the Multifrequency Snapshot Sky Survey (MSSS), the first northern-sky LOFAR imaging survey. In this introductory paper, we first describe in detail the motivation and design of the survey. Compared to previous radio surveys, MSSS is exceptional due to its intrinsic multifrequency nature providing information about the spectral properties of the detected sources over more than two octaves (from 30 to 160 MHz). The broadband frequency coverage, together with the fast survey speed generated by LOFAR's multibeaming capabilities, make MSSS the first survey of the sort anticipated to be carried out with the forthcoming Square Kilometre Array (SKA). Two of the sixteen frequency bands included in the survey were chosen to exactly overlap the frequency coverage of large-area Very Large Array (VLA) and Giant Metrewave Radio Telescope (GMRT) surveys at 74 MHz and 151 MHz respectively. The survey performance is illustrated within the "MSSS Verification Field" (MVF), a region of 100 square degrees centered at J2000 (RA,Dec)=(15h,69deg). The MSSS results from the MVF are compared with previous radio survey catalogs. We assess the flux and astrometric uncertainties in the catalog, as well as the completeness and reliability considering our source finding strategy. We determine the 90% completeness levels within the MVF to be 100 mJy at 135 MHz with 108" resolution, and 550 mJy at 50 MHz with 166" resolution. Images and catalogs for the full survey, expected to contain 150,000-200,000 sources, will be released to a public web server. We outline the plans for the ongoing production of the final survey products, and the ultimate public release of images and source catalogs. △ Less

Submitted 3 September, 2015; originally announced September 2015.

Comments: 23 pages, 19 figures. Accepted for publication in Astronomy & Astrophysics. MSSS Verification Field images and catalog data may be downloaded from http://vo.astron.nl

Linear polarization structures in LOFAR observations of the interstellar medium in the 3C196 field

Authors: V. Jelić, A. G. de Bruyn, V. N. Pandey, M. Mevius, M. Haverkorn, M. A. Brentjens, L. V. E. Koopmans, S. Zaroubi, F. B. Abdalla, K. M. B. Asad, S. Bus, E. Chapman, B. Ciardi, E. R. Fernandez, A. Ghosh, G. Harker, I. T. Iliev, H. Jensen, S. Kazemi, G. Mellema, A. R. Offringa, A. H. Patil, H. K. Vedantham, S. Yatawatta

Abstract: This study aims to characterize linear polarization structures in LOFAR observations of the interstellar medium (ISM) in the 3C196 field, one of the primary fields of the LOFAR-Epoch of Reionization key science project. We have used the high band antennas (HBA) of LOFAR to image this region and Rotation Measure (RM) synthesis to unravel the distribution of polarized structures in Faraday depth. Th… ▽ More This study aims to characterize linear polarization structures in LOFAR observations of the interstellar medium (ISM) in the 3C196 field, one of the primary fields of the LOFAR-Epoch of Reionization key science project. We have used the high band antennas (HBA) of LOFAR to image this region and Rotation Measure (RM) synthesis to unravel the distribution of polarized structures in Faraday depth. The brightness temperature of the detected Galactic emission is $5-15~{\rm K}$ in polarized intensity and covers the range from -3 to +8 ${\rm rad~m^{-2}}$ in Faraday depth. The most interesting morphological feature is a strikingly straight filament at a Faraday depth of $+0.5~{\rm rad~m^{-2}}$ running from north to south, right through the centre of the field and parallel to the Galactic plane. There is also an interesting system of linear depolarization canals conspicuous in an image showing the peaks of Faraday spectra. We used the Westerbork Synthesis Radio Telescope (WSRT) at 350 MHz to image the same region. For the first time, we see some common morphology in the RM cubes made at 150 and 350~{\rm MHz}. There is no indication of diffuse emission in total intensity in the interferometric data, in line with results at higher frequencies and previous LOFAR observations. Based on our results, we determined physical parameters of the ISM and proposed a simple model that may explain the observed distribution of the intervening magneto-ionic medium. The mean line-of-sight magnetic field component, $B_\parallel$, is determined to be $0.3\pm0.1~{\rm μG}$ and its spatial variation across the 3C196 field is $0.1~{\rm μG}$. The filamentary structure is probably an ionized filament in the ISM, located somewhere within the Local Bubble. This filamentary structure shows an excess in thermal electron density ($n_e B_\parallel>6.2~{\rm cm^{-3}μG}$) compared to its surroundings. △ Less

Submitted 21 September, 2015; v1 submitted 26 August, 2015; originally announced August 2015.

Comments: 16 pages, 10 figures, aceppted for publication in A&A

Journal ref: A&A 583, A137 (2015)

LOFAR discovery of a quiet emission mode in PSR B0823+26

Authors: C. Sobey, N. J. Young, J. W. T. Hessels, P. Weltevrede, A. Noutsos, B. W. Stappers, M. Kramer, C. Bassa, A. G. Lyne, V. I. Kondratiev, T. E. Hassall, E. F. Keane, A. V. Bilous, R. P. Breton, J. -M. Grießmeier, A. Karastergiou, M. Pilia, M. Serylak, S. ter Veen, J. van Leeuwen, A. Alexov, J. Anderson, A. Asgekar, I. M. Avruch, M. E. Bell , et al. (69 additional authors not shown)

Abstract: PSR B0823+26, a 0.53-s radio pulsar, displays a host of emission phenomena over timescales of seconds to (at least) hours, including nulling, subpulse drifting, and mode-changing. Studying pulsars like PSR B0823+26 provides further insight into the relationship between these various emission phenomena and what they might teach us about pulsar magnetospheres. Here we report on the LOFAR discovery t… ▽ More PSR B0823+26, a 0.53-s radio pulsar, displays a host of emission phenomena over timescales of seconds to (at least) hours, including nulling, subpulse drifting, and mode-changing. Studying pulsars like PSR B0823+26 provides further insight into the relationship between these various emission phenomena and what they might teach us about pulsar magnetospheres. Here we report on the LOFAR discovery that PSR B0823+26 has a weak and sporadically emitting 'quiet' (Q) emission mode that is over 100 times weaker (on average) and has a nulling fraction forty-times greater than that of the more regularly-emitting 'bright' (B) mode. Previously, the pulsar has been undetected in the Q-mode, and was assumed to be nulling continuously. PSR B0823+26 shows a further decrease in average flux just before the transition into the B-mode, and perhaps truly turns off completely at these times. Furthermore, simultaneous observations taken with the LOFAR, Westerbork, Lovell, and Effelsberg telescopes between 110 MHz and 2.7 GHz demonstrate that the transition between the Q-mode and B-mode occurs within one single rotation of the neutron star, and that it is concurrent across the range of frequencies observed. △ Less

Submitted 12 May, 2015; originally announced May 2015.

Comments: 15 pages, 8 figures, 2 tables, accepted for publication in MNRAS

Simulating the 21cm forest detectable with LOFAR and SKA in the spectra of high-z GRBs

Authors: B. Ciardi, S. Inoue, F. B. Abdalla, K. Asad, G. Bernardi, J. S. Bolton, M. Brentjens, A. G. de Bruyn, E. Chapman, S. Daiboo, E. R. Fernandez, A. Ghosh, L. Graziani, G. J. A. Harker, I. T. Iliev, V. Jelic, H. Jensen, S. Kazemi, L. V. E. Koopmans, O. Martinez, A. Maselli, G. Mellema, A. R. Offringa, V. N. Pandey, J. Schaye , et al. (4 additional authors not shown)

Abstract: We investigate the feasibility of detecting 21cm absorption features in the afterglow spectra of high redshift long Gamma Ray Bursts (GRBs). This is done employing simulations of cosmic reionization, together with the instrumental characteristics of the LOw Frequency ARray (LOFAR). We find that absorption features could be marginally (with a S/N larger than a few) detected by LOFAR at z>7 if the G… ▽ More We investigate the feasibility of detecting 21cm absorption features in the afterglow spectra of high redshift long Gamma Ray Bursts (GRBs). This is done employing simulations of cosmic reionization, together with the instrumental characteristics of the LOw Frequency ARray (LOFAR). We find that absorption features could be marginally (with a S/N larger than a few) detected by LOFAR at z>7 if the GRB originated from PopIII stars, while the detection would be easier if the noise were reduced by one order of magnitude, i.e. similar to what is expected for the first phase of the Square Kilometer Array (SKA1-low). On the other hand, more standard GRBs are too dim to be detected even with ten times the sensitivity of SKA1-low, and only in the most optimistic case can a S/N larger than a few be reached at z>9. △ Less

Submitted 20 July, 2015; v1 submitted 28 April, 2015; originally announced April 2015.

Comments: 5 pages, 4 figures, MNRAS, in press

Polarization leakage in Epoch of Reionization windows: I. LOFAR observations of the 3C196 field

Authors: K. M. B. Asad, L. V. E. Koopmans, V. Jelić, V. N. Pandey, A. Ghosh, F. B. Abdalla, G. Bernardi, M. A. Brentjens, A. G. de Bruyn, S. Bus, B. Ciardi, E. Chapman, S. Daiboo, E. R. Fernandez, G. Harker, I. T. Iliev, H. Jensen, O. Martinez-Rubi, G. Mellema, M. Mevius, A. R. Offringa, A. H. Patil, J. Schaye, R. M. Thomas, S. van der Tol , et al. (3 additional authors not shown)

Abstract: Detection of the 21-cm signal coming from the epoch of reionization (EoR) is challenging especially because, even after removing the foregrounds, the residual Stokes $I$ maps contain leakage from polarized emission that can mimic the signal. Here, we discuss the instrumental polarization of LOFAR and present realistic simulations of the leakages between Stokes parameters. From the LOFAR observatio… ▽ More Detection of the 21-cm signal coming from the epoch of reionization (EoR) is challenging especially because, even after removing the foregrounds, the residual Stokes $I$ maps contain leakage from polarized emission that can mimic the signal. Here, we discuss the instrumental polarization of LOFAR and present realistic simulations of the leakages between Stokes parameters. From the LOFAR observations of polarized emission in the 3C196 field, we have quantified the level of polarization leakage caused by the nominal model beam of LOFAR, and compared it with the EoR signal using power spectrum analysis. We found that at 134--166 MHz, within the central 4$^\circ$ of the field the $(Q,U)\rightarrow I$ leakage power is lower than the EoR signal at $k<0.3$ Mpc$^{-1}$. The leakage was found to be localized around a Faraday depth of 0, and the rms of the leakage as a fraction of the rms of the polarized emission was shown to vary between 0.2-0.3\%, both of which could be utilized in the removal of leakage. Moreover, we could define an `EoR window' in terms of the polarization leakage in the cylindrical power spectrum above the PSF-induced wedge and below $k_\parallel\sim 0.5$ Mpc$^{-1}$, and the window extended up to $k_\parallel\sim 1$ Mpc$^{-1}$ at all $k_\perp$ when 70\% of the leakage had been removed. These LOFAR results show that even a modest polarimetric calibration over a field of view of $\lesssim 4^\circ$ in the future arrays like SKA will ensure that the polarization leakage remains well below the expected EoR signal at the scales of 0.02-1 Mpc$^{-1}$. △ Less

Submitted 13 May, 2015; v1 submitted 5 March, 2015; originally announced March 2015.

Comments: 19 pages, 13 figures. Accepted for publication in MNRAS

Distributed Radio Interferometric Calibration

Authors: Sarod Yatawatta

Abstract: Increasing data volumes delivered by a new generation of radio interferometers require computationally efficient and robust calibration algorithms. In this paper, we propose distributed calibration as a way of improving both computational cost as well as robustness in calibration. We exploit the data parallelism across frequency that is inherent in radio astronomical observations that are recorded… ▽ More Increasing data volumes delivered by a new generation of radio interferometers require computationally efficient and robust calibration algorithms. In this paper, we propose distributed calibration as a way of improving both computational cost as well as robustness in calibration. We exploit the data parallelism across frequency that is inherent in radio astronomical observations that are recorded as multiple channels at different frequencies. Moreover, we also exploit the smoothness of the variation of calibration parameters across frequency. Data parallelism enables us to distribute the computing load across a network of compute agents. Smoothness in frequency enables us reformulate calibration as a consensus optimization problem. With this formulation, we enable flow of information between compute agents calibrating data at different frequencies, without actually passing the data, and thereby improving robustness. We present simulation results to show the feasibility as well as the advantages of distributed calibration as opposed to conventional calibration. △ Less

Submitted 17 March, 2015; v1 submitted 3 February, 2015; originally announced February 2015.

Comments: MNRAS Accepted 2015 March 13. Received 2015 January 28; in original form 2014 November 6, low resolution figures

Measuring a Cherenkov ring in the radio emission from air showers at 110-190 MHz with LOFAR

Authors: A. Nelles, P. Schellart, S. Buitink, A. Corstanje, K. D. de Vries, J. E. Enriquez, H. Falcke, W. Frieswijk, J. R. Hörandel, O. Scholten, S. ter Veen, S. Thoudam, M. van den Akker, J. Anderson, A. Asgekar, M. E. Bell, M. J. Bentum, G. Bernardi, P. Best, J. Bregman, F. Breitling, J. Broderick, W. N. Brouw, M. Brüggen, H. R. Butcher , et al. (44 additional authors not shown)

Abstract: Measuring radio emission from air showers offers a novel way to determine properties of the primary cosmic rays such as their mass and energy. Theory predicts that relativistic time compression effects lead to a ring of amplified emission which starts to dominate the emission pattern for frequencies above ~100 MHz. In this article we present the first detailed measurements of this structure. Ring… ▽ More Measuring radio emission from air showers offers a novel way to determine properties of the primary cosmic rays such as their mass and energy. Theory predicts that relativistic time compression effects lead to a ring of amplified emission which starts to dominate the emission pattern for frequencies above ~100 MHz. In this article we present the first detailed measurements of this structure. Ring structures in the radio emission of air showers are measured with the LOFAR radio telescope in the frequency range of 110 - 190 MHz. These data are well described by CoREAS simulations. They clearly confirm the importance of including the index of refraction of air as a function of height. Furthermore, the presence of the Cherenkov ring offers the possibility for a geometrical measurement of the depth of shower maximum, which in turn depends on the mass of the primary particle. △ Less

Submitted 25 November, 2014; originally announced November 2014.

Comments: 20 pages, 10 figures, accpeted for publication in Astroparticle Physics

The LOFAR long baseline snapshot calibrator survey

Authors: J. Moldón, A. T. Deller, O. Wucknitz, N. Jackson, A. Drabent, T. Carozzi, J. Conway, A. D. Kapińska, P. McKean, L. Morabito, E. Varenius, P. Zarka, J. Anderson, A. Asgekar, I. M. Avruch, M. E. Bell, M. J. Bentum, G. Bernardi, P. Best, L. Bîrzan, J. Bregman, F. Breitling, J. W. Broderick, M. Brüggen, H. R. Butcher , et al. (60 additional authors not shown)

Abstract: Aims. An efficient means of locating calibrator sources for International LOFAR is developed and used to determine the average density of usable calibrator sources on the sky for subarcsecond observations at 140 MHz. Methods. We used the multi-beaming capability of LOFAR to conduct a fast and computationally inexpensive survey with the full International LOFAR array. Sources were pre-selected on t… ▽ More Aims. An efficient means of locating calibrator sources for International LOFAR is developed and used to determine the average density of usable calibrator sources on the sky for subarcsecond observations at 140 MHz. Methods. We used the multi-beaming capability of LOFAR to conduct a fast and computationally inexpensive survey with the full International LOFAR array. Sources were pre-selected on the basis of 325 MHz arcminute-scale flux density using existing catalogues. By observing 30 different sources in each of the 12 sets of pointings per hour, we were able to inspect 630 sources in two hours to determine if they possess a sufficiently bright compact component to be usable as LOFAR delay calibrators. Results. Over 40% of the observed sources are detected on multiple baselines between international stations and 86 are classified as satisfactory calibrators. We show that a flat low-frequency spectrum (from 74 to 325 MHz) is the best predictor of compactness at 140 MHz. We extrapolate from our sample to show that the density of calibrators on the sky that are sufficiently bright to calibrate dispersive and non-dispersive delays for the International LOFAR using existing methods is 1.0 per square degree. Conclusions. The observed density of satisfactory delay calibrator sources means that observations with International LOFAR should be possible at virtually any point in the sky, provided that a fast and efficient search using the methodology described here is conducted prior to the observation to identify the best calibrator. △ Less

Submitted 11 November, 2014; originally announced November 2014.

Comments: 13 pages, 9 figures. Accepted for publication in Astronomy and Astrophysics

Journal ref: A&A 574, A73 (2015)

Adaptive Weighting in Radio Interferometric Imaging

Authors: Sarod Yatawatta

Abstract: Radio interferometers observe the Fourier space of the sky, at locations determined by the array geometry. Before a real space image is constructed by a Fourier transform, the data is weighted to improve the quality of reconstruction. Two criteria for calculation of weights are maximizing sensitivity and minimizing point spread function (PSF) sidelobe levels. In this paper, we propose a novel weig… ▽ More Radio interferometers observe the Fourier space of the sky, at locations determined by the array geometry. Before a real space image is constructed by a Fourier transform, the data is weighted to improve the quality of reconstruction. Two criteria for calculation of weights are maximizing sensitivity and minimizing point spread function (PSF) sidelobe levels. In this paper, we propose a novel weighting scheme suitable for ultra deep imaging experiments. The proposed weighting scheme is used to maximize sensitivity while minimizing PSF sidelobe variation across frequency and multiple epochs. We give simulation results that show the superiority of the proposed scheme compared with commonly used weighting schemes in achieving these objectives. △ Less

Submitted 23 July, 2014; originally announced July 2014.

Comments: MNRAS Accepted 2014 July 22. Received 2014 July 15; in original form 2014 June 26

LOFAR tied-array imaging of Type III solar radio bursts

Authors: D. E. Morosan, P. T. Gallagher, P. Zucca, R. Fallows, E. P. Carley, G. Mann, M. M. Bisi, A. Kerdraon, A. A. Konovalenko, A. L. MacKinnon, H. O. Rucker, B. Thidé, J. Magdalenić, C. Vocks, H. Reid, J. Anderson, A. Asgekar, I. M. Avruch, M. J. Bentum, G. Bernardi, P. Best, A. Bonafede, J. Bregman, F. Breitling, J. Broderick , et al. (60 additional authors not shown)

Abstract: The Sun is an active source of radio emission which is often associated with energetic phenomena such as solar flares and coronal mass ejections (CMEs). At low radio frequencies (<100 MHz), the Sun has not been imaged extensively because of the instrumental limitations of previous radio telescopes. Here, the combined high spatial, spectral and temporal resolution of the Low Frequency Array (LOFAR)… ▽ More The Sun is an active source of radio emission which is often associated with energetic phenomena such as solar flares and coronal mass ejections (CMEs). At low radio frequencies (<100 MHz), the Sun has not been imaged extensively because of the instrumental limitations of previous radio telescopes. Here, the combined high spatial, spectral and temporal resolution of the Low Frequency Array (LOFAR) was used to study solar Type III radio bursts at 30-90 MHz and their association with CMEs. The Sun was imaged with 126 simultaneous tied-array beams within 5 solar radii of the solar centre. This method offers benefits over standard interferometric imaging since each beam produces high temporal (83 ms) and spectral resolution (12.5 kHz) dynamic spectra at an array of spatial locations centred on the Sun. LOFAR's standard interferometric output is currently limited to one image per second. Over a period of 30 minutes, multiple Type III radio bursts were observed, a number of which were found to be located at high altitudes (4 solar radii from the solar center at 30 MHz) and to have non-radial trajectories. These bursts occurred at altitudes in excess of values predicted by 1D radial electron density models. The non-radial high altitude Type III bursts were found to be associated with the expanding flank of a CME. The CME may have compressed neighbouring streamer plasma producing larger electron densities at high altitudes, while the non-radial burst trajectories can be explained by the deflection of radial magnetic fields as the CME expanded in the low corona. △ Less

Submitted 16 July, 2014; originally announced July 2014.

Comments: 9 pages, 6 figures

Journal ref: A&A 568, A67 (2014)

Lunar occultation of the diffuse radio sky: LOFAR measurements between 35 and 80 MHz

Authors: H. K. Vedantham, L. V. E. Koopmans, A. G. de Bruyn, S. J. Wijnholds, M. Brentjens, F. B. Abdalla, K. M. B. Asad, G. Bernardi, S. Bus, E. Chapman, B. Ciardi, S. Daiboo, E. R. Fernandez, A. Ghosh, G. Harker, V. Jelic, H. Jensen, S. Kazemi, P. Lambropoulos, O. Martinez-Rubi, G. Mellema, M. Mevius, A. R. Offringa, V. N. Pandey, A. H. Patil , et al. (69 additional authors not shown)

Abstract: We present radio observations of the Moon between $35$ and $80$ MHz to demonstrate a novel technique of interferometrically measuring large-scale diffuse emission extending far beyond the primary beam (global signal) for the first time. In particular, we show that (i) the Moon appears as a negative-flux source at frequencies $35<ν<80$ MHz since it is `colder' than the diffuse Galactic background i… ▽ More We present radio observations of the Moon between $35$ and $80$ MHz to demonstrate a novel technique of interferometrically measuring large-scale diffuse emission extending far beyond the primary beam (global signal) for the first time. In particular, we show that (i) the Moon appears as a negative-flux source at frequencies $35<ν<80$ MHz since it is `colder' than the diffuse Galactic background it occults, (ii) using the (negative) flux of the lunar disc, we can reconstruct the spectrum of the diffuse Galactic emission with the lunar thermal emission as a reference, and (iii) that reflected RFI (radio-frequency interference) is concentrated at the center of the lunar disc due to specular nature of reflection, and can be independently measured. Our RFI measurements show that (i) Moon-based Cosmic Dawn experiments must design for an Earth-isolation of better than $80$ dB to achieve an RFI temperature $<1$ mK, (ii) Moon-reflected RFI contributes to a dipole temperature less than $20$ mK for Earth-based Cosmic Dawn experiments, (iii) man-made satellite-reflected RFI temperature exceeds $20$ mK if the aggregate cross section of visible satellites exceeds $80$ m$^2$ at $800$ km height, or $5$ m$^2$ at $400$ km height. Currently, our diffuse background spectrum is limited by sidelobe confusion on short baselines (10-15% level). Further refinement of our technique may yield constraints on the redshifted global $21$-cm signal from Cosmic Dawn ($40>z>12$) and the Epoch of Reionization ($12>z>5$). △ Less

Submitted 16 July, 2014; originally announced July 2014.

Comments: 15 pages, 11 figures, 1 table

Initial LOFAR observations of Epoch of Reionization windows: II. Diffuse polarized emission in the ELAIS-N1 field

Authors: V. Jelic, A. G. de Bruyn, M. Mevius, F. B. Abdalla, K. M. B. Asad, G. Bernardi, M. A. Brentjens, S. Bus, E. Chapman, B. Ciardi, S. Daiboo, E. R. Fernandez, A. Ghosh, G. Harker, H. Jensen, S. Kazemi, L. V. E. Koopmans, P. Labropoulos, O. Martinez-Rubi, G. Mellema, A. R. Offringa, V. N. Pandey, A. H. Patil, R. M. Thomas, H. K. Vedantham , et al. (84 additional authors not shown)

Abstract: This study aims to characterise the polarized foreground emission in the ELAIS-N1 field and to address its possible implications for the extraction of the cosmological 21-cm signal from the Low-Frequency Array - Epoch of Reionization (LOFAR-EoR) data. We use the high band antennas of LOFAR to image this region and RM-synthesis to unravel structures of polarized emission at high Galactic latitudes.… ▽ More This study aims to characterise the polarized foreground emission in the ELAIS-N1 field and to address its possible implications for the extraction of the cosmological 21-cm signal from the Low-Frequency Array - Epoch of Reionization (LOFAR-EoR) data. We use the high band antennas of LOFAR to image this region and RM-synthesis to unravel structures of polarized emission at high Galactic latitudes. The brightness temperature of the detected Galactic emission is on average 4 K in polarized intensity and covers the range from -10 to +13rad m^-2 in Faraday depth. The total polarized intensity and polarization angle show a wide range of morphological features. We have also used the Westerbork Synthesis Radio Telescope (WSRT) at 350 MHz to image the same region. The LOFAR and WSRT images show a similar complex morphology, at comparable brightness levels, but their spatial correlation is very low. The fractional polarization at 150 MHz, expressed as a percentage of the total intensity, amounts to 1.5%. There is no indication of diffuse emission in total intensity in the interferometric data, in line with results at higher frequencies. The wide frequency range, good angular resolution and good sensitivity make LOFAR an exquisite instrument for studying Galactic polarized emission at a resolution of 1-2 rad m^-2 in Faraday depth. The different polarised patterns observed at 150 MHz and 350 MHz are consistent with different source distributions along the line of sight wring in a variety of Faraday thin regions of emission. The presence of polarised foregrounds is a serious complication for Epoch of Reionization experiments. To avoid the leakage of polarized emission into total intensity, which can depend on frequency, we need to calibrate the instrumental polarization across the field of view to a small fraction of 1%. △ Less

Submitted 8 July, 2014; originally announced July 2014.

Comments: 12 pages, 9 figures, accepted for publication in A&A

Journal ref: A&A 568, A101 (2014)

LOFAR Sparse Image Reconstruction

Authors: H. Garsden, J. N. Girard, J. L. Starck, S. Corbel, C. Tasse, A. Woiselle, J. P. McKean, A. S. van Amesfoort, J. Anderson, I. M. Avruch, R. Beck, M. J. Bentum, P. Best, F. Breitling, J. Broderick, M. Brüggen, H. R. Butcher, B. Ciardi, F. de Gasperin, E. de Geus, M. de Vos, S. Duscha, J. Eislöffel, D. Engels, H. Falcke , et al. (56 additional authors not shown)

Abstract: Context. The LOw Frequency ARray (LOFAR) radio telescope is a giant digital phased array interferometer with multiple antennas distributed in Europe. It provides discrete sets of Fourier components of the sky brightness. Recovering the original brightness distribution with aperture synthesis forms an inverse problem that can be solved by various deconvolution and minimization methods Aims. Recent… ▽ More Context. The LOw Frequency ARray (LOFAR) radio telescope is a giant digital phased array interferometer with multiple antennas distributed in Europe. It provides discrete sets of Fourier components of the sky brightness. Recovering the original brightness distribution with aperture synthesis forms an inverse problem that can be solved by various deconvolution and minimization methods Aims. Recent papers have established a clear link between the discrete nature of radio interferometry measurement and the "compressed sensing" (CS) theory, which supports sparse reconstruction methods to form an image from the measured visibilities. Empowered by proximal theory, CS offers a sound framework for efficient global minimization and sparse data representation using fast algorithms. Combined with instrumental direction-dependent effects (DDE) in the scope of a real instrument, we developed and validated a new method based on this framework Methods. We implemented a sparse reconstruction method in the standard LOFAR imaging tool and compared the photometric and resolution performance of this new imager with that of CLEAN-based methods (CLEAN and MS-CLEAN) with simulated and real LOFAR data Results. We show that i) sparse reconstruction performs as well as CLEAN in recovering the flux of point sources; ii) performs much better on extended objects (the root mean square error is reduced by a factor of up to 10); and iii) provides a solution with an effective angular resolution 2-3 times better than the CLEAN images. Conclusions. Sparse recovery gives a correct photometry on high dynamic and wide-field images and improved realistic structures of extended sources (of simulated and real LOFAR datasets). This sparse reconstruction method is compatible with modern interferometric imagers that handle DDE corrections (A- and W-projections) required for current and future instruments such as LOFAR and SKA △ Less

Submitted 6 March, 2015; v1 submitted 27 June, 2014; originally announced June 2014.

Comments: Published in A&A, 19 pages, 9 figures

Journal ref: A&A 575, A90 (2015)

The shape of the radio wavefront of extensive air showers as measured with LOFAR

Authors: A. Corstanje, P. Schellart, A. Nelles, S. Buitink, J. E. Enriquez, H. Falcke, W. Frieswijk, J. R. Hörandel, M. Krause, J. P. Rachen, O. Scholten, S. ter Veen, S. Thoudam, G. Trinh, M. van den Akker, A. Alexov, J. Anderson, I. M. Avruch, M. E. Bell, M. J. Bentum, G. Bernardi, P. Best, A. Bonafede, F. Breitling, J. Broderick , et al. (56 additional authors not shown)

Abstract: Extensive air showers, induced by high energy cosmic rays im**ing on the Earth's atmosphere, produce radio emission that is measured with the LOFAR radio telescope. As the emission comes from a finite distance of a few kilometers, the incident wavefront is non-planar. A spherical, conical or hyperbolic shape of the wavefront has been proposed, but measurements of individual air showers have been… ▽ More Extensive air showers, induced by high energy cosmic rays im**ing on the Earth's atmosphere, produce radio emission that is measured with the LOFAR radio telescope. As the emission comes from a finite distance of a few kilometers, the incident wavefront is non-planar. A spherical, conical or hyperbolic shape of the wavefront has been proposed, but measurements of individual air showers have been inconclusive so far. For a selected high-quality sample of 161 measured extensive air showers, we have reconstructed the wavefront by measuring pulse arrival times to sub-nanosecond precision in 200 to 350 individual antennas. For each measured air shower, we have fitted a conical, spherical, and hyperboloid shape to the arrival times. The fit quality and a likelihood analysis show that a hyperboloid is the best parametrization. Using a non-planar wavefront shape gives an improved angular resolution, when reconstructing the shower arrival direction. Furthermore, a dependence of the wavefront shape on the shower geometry can be seen. This suggests that it will be possible to use a wavefront shape analysis to get an additional handle on the atmospheric depth of the shower maximum, which is sensitive to the mass of the primary particle. △ Less

Submitted 8 June, 2014; v1 submitted 15 April, 2014; originally announced April 2014.

Comments: Accepted for publication in Astroparticle Physics