Showing 1–50 of 173 results for author: McEwen, J D

Using conditional GANs for convergence map reconstruction with uncertainties

Authors: Jessica Whitney, Tobías Liaudat, Matt Price, Matthijs Mars, Jason D. McEwen

Abstract: Understanding the large-scale structure of the Universe and unravelling the mysteries of dark matter are fundamental challenges in contemporary cosmology. Reconstruction of the cosmological matter distribution from lensing observables, referred to as 'mass-map**' is an important aspect of this quest. Mass-map** is an ill-posed problem, meaning there is inherent uncertainty in any convergence m… ▽ More Understanding the large-scale structure of the Universe and unravelling the mysteries of dark matter are fundamental challenges in contemporary cosmology. Reconstruction of the cosmological matter distribution from lensing observables, referred to as 'mass-map**' is an important aspect of this quest. Mass-map** is an ill-posed problem, meaning there is inherent uncertainty in any convergence map reconstruction. The demand for fast and efficient reconstruction techniques is rising as we prepare for upcoming surveys. We present a novel approach which utilises deep learning, in particular a conditional Generative Adversarial Network (cGAN), to approximate samples from a Bayesian posterior distribution, meaning they can be interpreted in a statistically robust manner. By combining data-driven priors with recent regularisation techniques, we introduce an approach that facilitates the swift generation of high-fidelity, mass maps. Furthermore, to validate the effectiveness of our approach, we train the model on mock COSMOS-style data, generated using Colombia Lensing's kappaTNG mock weak lensing suite. These preliminary results showcase compelling convergence map reconstructions and ongoing refinement efforts are underway to enhance the robustness of our method further. △ Less

Submitted 21 May, 2024; originally announced June 2024.

Comments: 2 pages, 1 figure, submitted for conference proceedings for '58th Recontres de Moriond', 2024

Euclid. I. Overview of the Euclid mission

Authors: Euclid Collaboration, Y. Mellier, Abdurro'uf, J. A. Acevedo Barroso, A. Achúcarro, J. Adamek, R. Adam, G. E. Addison, N. Aghanim, M. Aguena, V. Ajani, Y. Akrami, A. Al-Bahlawan, A. Alavi, I. S. Albuquerque, G. Alestas, G. Alguero, A. Allaoui, S. W. Allen, V. Allevato, A. V. Alonso-Tetilla, B. Altieri, A. Alvarez-Candal, A. Amara, L. Amendola , et al. (1086 additional authors not shown)

Abstract: The current standard model of cosmology successfully describes a variety of measurements, but the nature of its main ingredients, dark matter and dark energy, remains unknown. Euclid is a medium-class mission in the Cosmic Vision 2015-2025 programme of the European Space Agency (ESA) that will provide high-resolution optical imaging, as well as near-infrared imaging and spectroscopy, over about 14… ▽ More The current standard model of cosmology successfully describes a variety of measurements, but the nature of its main ingredients, dark matter and dark energy, remains unknown. Euclid is a medium-class mission in the Cosmic Vision 2015-2025 programme of the European Space Agency (ESA) that will provide high-resolution optical imaging, as well as near-infrared imaging and spectroscopy, over about 14,000 deg^2 of extragalactic sky. In addition to accurate weak lensing and clustering measurements that probe structure formation over half of the age of the Universe, its primary probes for cosmology, these exquisite data will enable a wide range of science. This paper provides a high-level overview of the mission, summarising the survey characteristics, the various data-processing steps, and data products. We also highlight the main science objectives and expected performance. △ Less

Submitted 22 May, 2024; originally announced May 2024.

Comments: Paper submitted as part of the A&A special issue`Euclid on Sky'

The future of cosmological likelihood-based inference: accelerated high-dimensional parameter estimation and model comparison

Authors: Davide Piras, Alicja Polanska, Alessio Spurio Mancini, Matthew A. Price, Jason D. McEwen

Abstract: We advocate for a new paradigm of cosmological likelihood-based inference, leveraging recent developments in machine learning and its underlying technology, to accelerate Bayesian inference in high-dimensional settings. Specifically, we combine (i) emulation, where a machine learning model is trained to mimic cosmological observables, e.g. CosmoPower-JAX; (ii) differentiable and probabilistic prog… ▽ More We advocate for a new paradigm of cosmological likelihood-based inference, leveraging recent developments in machine learning and its underlying technology, to accelerate Bayesian inference in high-dimensional settings. Specifically, we combine (i) emulation, where a machine learning model is trained to mimic cosmological observables, e.g. CosmoPower-JAX; (ii) differentiable and probabilistic programming, e.g. JAX and NumPyro, respectively; (iii) scalable Markov chain Monte Carlo (MCMC) sampling techniques that exploit gradients, e.g. Hamiltonian Monte Carlo; and (iv) decoupled and scalable Bayesian model selection techniques that compute the Bayesian evidence purely from posterior samples, e.g. the learned harmonic mean implemented in harmonic. This paradigm allows us to carry out a complete Bayesian analysis, including both parameter estimation and model selection, in a fraction of the time of traditional approaches. First, we demonstrate the application of this paradigm on a simulated cosmic shear analysis for a Stage IV survey in 37- and 39-dimensional parameter spaces, comparing $Λ$CDM and a dynamical dark energy model ($w_0w_a$CDM). We recover posterior contours and evidence estimates that are in excellent agreement with those computed by the traditional nested sampling approach while reducing the computational cost from 8 months on 48 CPU cores to 2 days on 12 GPUs. Second, we consider a joint analysis between three simulated next-generation surveys, each performing a 3x2pt analysis, resulting in 157- and 159-dimensional parameter spaces. Standard nested sampling techniques are simply not feasible in this high-dimensional setting, requiring a projected 12 years of compute time on 48 CPU cores; on the other hand, the proposed approach only requires 8 days of compute time on 24 GPUs. All packages used in our analyses are publicly available. △ Less

Submitted 21 May, 2024; originally announced May 2024.

Comments: 13 pages, 6 figures. Codes available at https://github.com/alessiospuriomancini/cosmopower, https://github.com/dpiras/cosmopower-jax, https://github.com/astro-informatics/harmonic/

Learned radio interferometric imaging for varying visibility coverage

Authors: Matthijs Mars, Marta M. Betcke, Jason D. McEwen

Abstract: With the next generation of interferometric telescopes, such as the Square Kilometre Array (SKA), the need for highly computationally efficient reconstruction techniques is particularly acute. The challenge in designing learned, data-driven reconstruction techniques for radio interferometry is that they need to be agnostic to the varying visibility coverages of the telescope, since these are diffe… ▽ More With the next generation of interferometric telescopes, such as the Square Kilometre Array (SKA), the need for highly computationally efficient reconstruction techniques is particularly acute. The challenge in designing learned, data-driven reconstruction techniques for radio interferometry is that they need to be agnostic to the varying visibility coverages of the telescope, since these are different for each observation. Because of this, learned post-processing or learned unrolled iterative reconstruction methods must typically be retrained for each specific observation, amounting to a large computational overhead. In this work we develop learned post-processing and unrolled iterative methods for varying visibility coverages, proposing training strategies to make these methods agnostic to variations in visibility coverage with minimal to no fine-tuning. Learned post-processing techniques are heavily dependent on the prior information encoded in training data and generalise poorly to other visibility coverages. In contrast, unrolled iterative methods, which include the telescope measurement operator inside the network, achieve state-of-the-art reconstruction quality and computation time, generalising well to other coverages and require little to no fine-tuning. Furthermore, they generalise well to realistic radio observations and are able to reconstruct the high dynamic range of these images. △ Less

Submitted 14 May, 2024; originally announced May 2024.

Learned harmonic mean estimation of the Bayesian evidence with normalizing flows

Authors: Alicja Polanska, Matthew A. Price, Davide Piras, Alessio Spurio Mancini, Jason D. McEwen

Abstract: We present the learned harmonic mean estimator with normalizing flows - a robust, scalable and flexible estimator of the Bayesian evidence for model comparison. Since the estimator is agnostic to sampling strategy and simply requires posterior samples, it can be applied to compute the evidence using any Markov chain Monte Carlo (MCMC) sampling technique, including saved down MCMC chains, or any va… ▽ More We present the learned harmonic mean estimator with normalizing flows - a robust, scalable and flexible estimator of the Bayesian evidence for model comparison. Since the estimator is agnostic to sampling strategy and simply requires posterior samples, it can be applied to compute the evidence using any Markov chain Monte Carlo (MCMC) sampling technique, including saved down MCMC chains, or any variational inference approach. The learned harmonic mean estimator was recently introduced, where machine learning techniques were developed to learn a suitable internal importance sampling target distribution to solve the issue of exploding variance of the original harmonic mean estimator. In this article we present the use of normalizing flows as the internal machine learning technique within the learned harmonic mean estimator. Normalizing flows can be elegantly coupled with the learned harmonic mean to provide an approach that is more robust, flexible and scalable than the machine learning models considered previously. We perform a series of numerical experiments, applying our method to benchmark problems and to a cosmological example in up to 21 dimensions. We find the learned harmonic mean estimator is in agreement with ground truth values and nested sampling estimates. The open-source harmonic Python package implementing the learned harmonic mean, now with normalizing flows included, is publicly available. △ Less

Submitted 9 May, 2024; originally announced May 2024.

Comments: 14 pages, 8 figures, harmonic code available at https://github.com/astro-informatics/harmonic

A covariant formulation for cosmological radiative transfer of the 21-cm line

Authors: Jennifer Y. H. Chan, Qin Han, Kinwah Wu, Jason D. McEwen

Abstract: The 21-cm hyperfine line of neutral hydrogen is a useful tool to probe the conditions of the Universe during the Dark Ages, Cosmic Dawn, and the Epoch of Reionisation. In most of the current calculations, the 21-cm line signals at given frequencies are computed, using an integrated line-of-sight line opacity, with the correction for cosmological expansion. These calculations have not fully capture… ▽ More The 21-cm hyperfine line of neutral hydrogen is a useful tool to probe the conditions of the Universe during the Dark Ages, Cosmic Dawn, and the Epoch of Reionisation. In most of the current calculations, the 21-cm line signals at given frequencies are computed, using an integrated line-of-sight line opacity, with the correction for cosmological expansion. These calculations have not fully captured the line and continuum interactions in the radiative transfer, in response to evolution of the radiation field and the variations of thermal and dynamic properties of the line-of-sight medium. We construct a covariant formulation for the radiative transfer of the 21-cm line and derive the cosmological 21-cm line radiative transfer (C21LRT) equation. The formulation properly accounts for local emission and absorption processes and the interaction between the line and continuum when the radiation propagates across the expanding Universe to the present observer. Our C21LRT calculations show that methods simply summing the line optical depth could lead to error of $5\%$ in the 21-cm signals for redshift $z \sim 12-35$ and of $>10\%$ for redshift $z \lesssim 8$. Proper covariant radiative transfer is therefore necessary for producing correct theoretical templates for extracting information of the structural evolution of the Universe through the Epoch of Reionisation from the 21-cm tomographic data. △ Less

Submitted 22 April, 2024; originally announced April 2024.

Comments: 16 pages, 11 figures, 3 tables

Differentiable and accelerated wavelet transforms on the sphere and ball

Authors: Matthew A. Price, Alicja Polanska, Jessica Whitney, Jason D. McEwen

Abstract: Directional wavelet dictionaries are hierarchical representations which efficiently capture and segment information across scale, location and orientation. Such representations demonstrate a particular affinity to physical signals, which often exhibit highly anisotropic, localised multiscale structure. Many physically important signals are observed over spherical domains, such as the celestial sky… ▽ More Directional wavelet dictionaries are hierarchical representations which efficiently capture and segment information across scale, location and orientation. Such representations demonstrate a particular affinity to physical signals, which often exhibit highly anisotropic, localised multiscale structure. Many physically important signals are observed over spherical domains, such as the celestial sky in cosmology. Leveraging recent advances in computational harmonic analysis, we design new highly distributable and automatically differentiable directional wavelet transforms on the $2$-dimensional sphere $\mathbb{S}^2$ and $3$-dimensional ball $\mathbb{B}^3 = \mathbb{R}^+ \times \mathbb{S}^2$ (the space formed by augmenting the sphere with the radial half-line). We observe up to a $300$-fold and $21800$-fold acceleration for signals on the sphere and ball, respectively, compared to existing software, whilst maintaining 64-bit machine precision. Not only do these algorithms dramatically accelerate existing spherical wavelet transforms, the gradient information afforded by automatic differentiation unlocks many data-driven analysis techniques previously not possible for these spaces. We publicly release both S2WAV and S2BALL, open-sourced JAX libraries for our transforms that are automatically differentiable and readily deployable both on and over clusters of hardware accelerators (e.g. GPUs & TPUs). △ Less

Submitted 14 March, 2024; v1 submitted 2 February, 2024; originally announced February 2024.

Comments: code available on the sphere at https://github.com/astro-informatics/s2wav and on the ball at https://github.com/astro-informatics/s2ball

Scalable Bayesian uncertainty quantification with data-driven priors for radio interferometric imaging

Authors: Tobías I. Liaudat, Matthijs Mars, Matthew A. Price, Marcelo Pereyra, Marta M. Betcke, Jason D. McEwen

Abstract: Next-generation radio interferometers like the Square Kilometer Array have the potential to unlock scientific discoveries thanks to their unprecedented angular resolution and sensitivity. One key to unlocking their potential resides in handling the deluge and complexity of incoming data. This challenge requires building radio interferometric imaging methods that can cope with the massive data size… ▽ More Next-generation radio interferometers like the Square Kilometer Array have the potential to unlock scientific discoveries thanks to their unprecedented angular resolution and sensitivity. One key to unlocking their potential resides in handling the deluge and complexity of incoming data. This challenge requires building radio interferometric imaging methods that can cope with the massive data sizes and provide high-quality image reconstructions with uncertainty quantification (UQ). This work proposes a method coined QuantifAI to address UQ in radio-interferometric imaging with data-driven (learned) priors for high-dimensional settings. Our model, rooted in the Bayesian framework, uses a physically motivated model for the likelihood. The model exploits a data-driven convex prior, which can encode complex information learned implicitly from simulations and guarantee the log-concavity of the posterior. We leverage probability concentration phenomena of high-dimensional log-concave posteriors that let us obtain information about the posterior, avoiding MCMC sampling techniques. We rely on convex optimisation methods to compute the MAP estimation, which is known to be faster and better scale with dimension than MCMC sampling strategies. Our method allows us to compute local credible intervals, i.e., Bayesian error bars, and perform hypothesis testing of structure on the reconstructed image. In addition, we propose a novel blazing-fast method to compute pixel-wise uncertainties at different scales. We demonstrate our method by reconstructing radio-interferometric images in a simulated setting and carrying out fast and scalable UQ, which we validate with MCMC sampling. Our method shows an improved image quality and more meaningful uncertainties than the benchmark method based on a sparsity-promoting prior. QuantifAI's source code: https://github.com/astro-informatics/QuantifAI. △ Less

Submitted 28 June, 2024; v1 submitted 30 November, 2023; originally announced December 2023.

Comments: 30 pages, 14 figures, 10 tables, code available at https://github.com/astro-informatics/QuantifAI

Differentiable and accelerated spherical harmonic and Wigner transforms

Authors: Matthew A. Price, Jason D. McEwen

Abstract: Many areas of science and engineering encounter data defined on spherical manifolds. Modelling and analysis of spherical data often necessitates spherical harmonic transforms, at high degrees, and increasingly requires efficient computation of gradients for machine learning or other differentiable programming tasks. We develop novel algorithmic structures for accelerated and differentiable computa… ▽ More Many areas of science and engineering encounter data defined on spherical manifolds. Modelling and analysis of spherical data often necessitates spherical harmonic transforms, at high degrees, and increasingly requires efficient computation of gradients for machine learning or other differentiable programming tasks. We develop novel algorithmic structures for accelerated and differentiable computation of generalised Fourier transforms on the sphere $\mathbb{S}^2$ and rotation group $\text{SO}(3)$, i.e. spherical harmonic and Wigner transforms, respectively. We present a recursive algorithm for the calculation of Wigner $d$-functions that is both stable to high harmonic degrees and extremely parallelisable. By tightly coupling this with separable spherical transforms, we obtain algorithms that exhibit an extremely parallelisable structure that is well-suited for the high throughput computing of modern hardware accelerators (e.g. GPUs). We also develop a hybrid automatic and manual differentiation approach so that gradients can be computed efficiently. Our algorithms are implemented within the JAX differentiable programming framework in the S2FFT software code. Numerous samplings of the sphere are supported, including equiangular and HEALPix sampling. Computational errors are at the order of machine precision for spherical samplings that admit a sampling theorem. When benchmarked against alternative C codes we observe up to a 400-fold acceleration. Furthermore, when distributing over multiple GPUs we achieve very close to optimal linear scaling with increasing number of GPUs due to the highly parallelised and balanced nature of our algorithms. Provided access to sufficiently many GPUs our transforms thus exhibit an unprecedented effective linear time complexity. △ Less

Submitted 20 May, 2024; v1 submitted 24 November, 2023; originally announced November 2023.

Comments: 30 pages, 7 figures, accepted by Journal of Computational Physics, code available at https://github.com/astro-informatics/s2fft

Fast emulation of anisotropies induced in the cosmic microwave background by cosmic strings

Authors: Matthew A. Price, Matthijs Mars, Matthew M. Docherty, Alessio Spurio Mancini, Augustin Marignier, Jason. D. McEwen

Abstract: Cosmic strings are linear topological defects that may have been produced during symmetry-breaking phase transitions in the very early Universe. In an expanding Universe the existence of causally separate regions prevents such symmetries from being broken uniformly, with a network of cosmic string inevitably forming as a result. To faithfully generate observables of such processes requires computa… ▽ More Cosmic strings are linear topological defects that may have been produced during symmetry-breaking phase transitions in the very early Universe. In an expanding Universe the existence of causally separate regions prevents such symmetries from being broken uniformly, with a network of cosmic string inevitably forming as a result. To faithfully generate observables of such processes requires computationally expensive numerical simulations, which prohibits many types of analyses. We propose a technique to instead rapidly emulate observables, thus circumventing simulation. Emulation is a form of generative modelling, often built upon a machine learning backbone. End-to-end emulation often fails due to high dimensionality and insufficient training data. Consequently, it is common to instead emulate a latent representation from which observables may readily be synthesised. Wavelet phase harmonics are an excellent latent representations for cosmological fields, both as a summary statistic and for emulation, since they do not require training and are highly sensitive to non-Gaussian information. Leveraging wavelet phase harmonics as a latent representation, we develop techniques to emulate string induced CMB anisotropies over a 7.2 degree field of view, with sub-arcminute resolution, in under a minute on a single GPU. Beyond generating high fidelity emulations, we provide a technique to ensure these observables are distributed correctly, providing a more representative ensemble of samples. The statistics of our emulations are commensurate with those calculated on comprehensive Nambu-Goto simulations. Our findings indicate these fast emulation approaches may be suitable for wide use in, e.g., simulation based inference pipelines. We make our code available to the community so that researchers may rapidly emulate cosmic string induced CMB anisotropies for their own analysis. △ Less

Submitted 14 March, 2024; v1 submitted 10 July, 2023; originally announced July 2023.

Comments: code available at https://github.com/astro-informatics/stringgen

Proximal nested sampling with data-driven priors for physical scientists

Authors: Jason D. McEwen, Tobías I. Liaudat, Matthew A. Price, Xiaohao Cai, Marcelo Pereyra

Abstract: Proximal nested sampling was introduced recently to open up Bayesian model selection for high-dimensional problems such as computational imaging. The framework is suitable for models with a log-convex likelihood, which are ubiquitous in the imaging sciences. The purpose of this article is two-fold. First, we review proximal nested sampling in a pedagogical manner in an attempt to elucidate the fra… ▽ More Proximal nested sampling was introduced recently to open up Bayesian model selection for high-dimensional problems such as computational imaging. The framework is suitable for models with a log-convex likelihood, which are ubiquitous in the imaging sciences. The purpose of this article is two-fold. First, we review proximal nested sampling in a pedagogical manner in an attempt to elucidate the framework for physical scientists. Second, we show how proximal nested sampling can be extended in an empirical Bayes setting to support data-driven priors, such as deep neural networks learned from training data. △ Less

Submitted 28 July, 2023; v1 submitted 30 June, 2023; originally announced July 2023.

Comments: 9 pages, 4 figures

Learned harmonic mean estimation of the marginal likelihood with normalizing flows

Authors: Alicja Polanska, Matthew A. Price, Alessio Spurio Mancini, Jason D. McEwen

Abstract: Computing the marginal likelihood (also called the Bayesian model evidence) is an important task in Bayesian model selection, providing a principled quantitative way to compare models. The learned harmonic mean estimator solves the exploding variance problem of the original harmonic mean estimation of the marginal likelihood. The learned harmonic mean estimator learns an importance sampling target… ▽ More Computing the marginal likelihood (also called the Bayesian model evidence) is an important task in Bayesian model selection, providing a principled quantitative way to compare models. The learned harmonic mean estimator solves the exploding variance problem of the original harmonic mean estimation of the marginal likelihood. The learned harmonic mean estimator learns an importance sampling target distribution that approximates the optimal distribution. While the approximation need not be highly accurate, it is critical that the probability mass of the learned distribution is contained within the posterior in order to avoid the exploding variance problem. In previous work a bespoke optimization problem is introduced when training models in order to ensure this property is satisfied. In the current article we introduce the use of normalizing flows to represent the importance sampling target distribution. A flow-based model is trained on samples from the posterior by maximum likelihood estimation. Then, the probability density of the flow is concentrated by lowering the variance of the base distribution, i.e. by lowering its "temperature", ensuring its probability mass is contained within the posterior. This approach avoids the need for a bespoke optimisation problem and careful fine tuning of parameters, resulting in a more robust method. Moreover, the use of normalizing flows has the potential to scale to high dimensional settings. We present preliminary experiments demonstrating the effectiveness of the use of flows for the learned harmonic mean estimator. The harmonic code implementing the learned harmonic mean, which is publicly available, has been updated to now support normalizing flows. △ Less

Submitted 19 January, 2024; v1 submitted 30 June, 2023; originally announced July 2023.

Comments: 9 pages, 6 figures. arXiv admin note: text overlap with arXiv:2111.12720

The Tiny Time-series Transformer: Low-latency High-throughput Classification of Astronomical Transients using Deep Model Compression

Authors: Tarek Allam Jr., Julien Peloton, Jason D. McEwen

Abstract: A new golden age in astronomy is upon us, dominated by data. Large astronomical surveys are broadcasting unprecedented rates of information, demanding machine learning as a critical component in modern scientific pipelines to handle the deluge of data. The upcoming Legacy Survey of Space and Time (LSST) of the Vera C. Rubin Observatory will raise the big-data bar for time-domain astronomy, with an… ▽ More A new golden age in astronomy is upon us, dominated by data. Large astronomical surveys are broadcasting unprecedented rates of information, demanding machine learning as a critical component in modern scientific pipelines to handle the deluge of data. The upcoming Legacy Survey of Space and Time (LSST) of the Vera C. Rubin Observatory will raise the big-data bar for time-domain astronomy, with an expected 10 million alerts per-night, and generating many petabytes of data over the lifetime of the survey. Fast and efficient classification algorithms that can operate in real-time, yet robustly and accurately, are needed for time-critical events where additional resources can be sought for follow-up analyses. In order to handle such data, state-of-the-art deep learning architectures coupled with tools that leverage modern hardware accelerators are essential. We showcase how the use of modern deep compression methods can achieve a $18\times$ reduction in model size, whilst preserving classification performance. We also show that in addition to the deep compression techniques, careful choice of file formats can improve inference latency, and thereby throughput of alerts, on the order of $8\times$ for local processing, and $5\times$ in a live production setting. To test this in a live setting, we deploy this optimised version of the original time-series transformer, t2, into the community alert broking system of FINK on real Zwicky Transient Facility (ZTF) alert data, and compare throughput performance with other science modules that exist in FINK. The results shown herein emphasise the time-series transformer's suitability for real-time classification at LSST scale, and beyond, and introduce deep model compression as a fundamental tool for improving deploy-ability and scalable inference of deep learning models for transient classification. △ Less

Submitted 15 March, 2023; originally announced March 2023.

Comments: 16 pages, 11 figures

Slepian Scale-Discretised Wavelets on Manifolds

Authors: Patrick J. Roddy, Jason D. McEwen

Abstract: Inspired by recent interest in geometric deep learning, this work generalises the recently developed Slepian scale-discretised wavelets on the sphere to Riemannian manifolds. Through the sifting convolution, one may define translations and, thus, convolutions on manifolds - which are otherwise not well-defined in general. Slepian wavelets are constructed on a region of a manifold and are therefore… ▽ More Inspired by recent interest in geometric deep learning, this work generalises the recently developed Slepian scale-discretised wavelets on the sphere to Riemannian manifolds. Through the sifting convolution, one may define translations and, thus, convolutions on manifolds - which are otherwise not well-defined in general. Slepian wavelets are constructed on a region of a manifold and are therefore suited to problems where data only exists in a particular region. The Slepian functions, on which Slepian wavelets are built, are the basis functions of the Slepian spatial-spectral concentration problem on the manifold. A tiling of the Slepian harmonic line with smoothly decreasing generating functions defines the scale-discretised wavelets; allowing one to probe spatially localised, scale-dependent features of a signal. By discretising manifolds as graphs, the Slepian functions and wavelets of a triangular mesh are presented. Through a wavelet transform, the wavelet coefficients of a field defined on the mesh are found and used in a straightforward thresholding denoising scheme. △ Less

Submitted 23 February, 2023; v1 submitted 12 February, 2023; originally announced February 2023.

Comments: 12 pages, 12 figures

Learned Interferometric Imaging for the SPIDER Instrument

Authors: Matthijs Mars, Marta M. Betcke, Jason D. McEwen

Abstract: The Segmented Planar Imaging Detector for Electro-Optical Reconnaissance (SPIDER) is an optical interferometric imaging device that aims to offer an alternative to the large space telescope designs of today with reduced size, weight and power consumption. This is achieved through interferometric imaging. State-of-the-art methods for reconstructing images from interferometric measurements adopt pro… ▽ More The Segmented Planar Imaging Detector for Electro-Optical Reconnaissance (SPIDER) is an optical interferometric imaging device that aims to offer an alternative to the large space telescope designs of today with reduced size, weight and power consumption. This is achieved through interferometric imaging. State-of-the-art methods for reconstructing images from interferometric measurements adopt proximal optimization techniques, which are computationally expensive and require handcrafted priors. In this work we present two data-driven approaches for reconstructing images from measurements made by the SPIDER instrument. These approaches use deep learning to learn prior information from training data, increasing the reconstruction quality, and significantly reducing the computation time required to recover images by orders of magnitude. Reconstruction time is reduced to ${\sim} 10$ milliseconds, opening up the possibility of real-time imaging with SPIDER for the first time. Furthermore, we show that these methods can also be applied in domains where training data is scarce, such as astronomical imaging, by leveraging transfer learning from domains where plenty of training data are available. △ Less

Submitted 15 January, 2024; v1 submitted 24 January, 2023; originally announced January 2023.

Comments: 21 pages, 14 figures

Journal ref: RAS Techniques and Instruments, Volume 2, Issue 1, January 2023, Pages 760-778

Sparse Bayesian mass-map** using trans-dimensional MCMC

Authors: Augustin Marignier, Thomas Kitching, Jason D. McEwen, Ana M. G. Ferreira

Abstract: Uncertainty quantification is a crucial step of cosmological mass-map** that is often ignored. Suggested methods are typically only approximate or make strong assumptions of Gaussianity of the shear field. Probabilistic sampling methods, such as Markov chain Monte Carlo (MCMC), draw samples form a probability distribution, allowing for full and flexible uncertainty quantification, however these… ▽ More Uncertainty quantification is a crucial step of cosmological mass-map** that is often ignored. Suggested methods are typically only approximate or make strong assumptions of Gaussianity of the shear field. Probabilistic sampling methods, such as Markov chain Monte Carlo (MCMC), draw samples form a probability distribution, allowing for full and flexible uncertainty quantification, however these methods are notoriously slow and struggle in the high-dimensional parameter spaces of imaging problems. In this work we use, for the first time, a trans-dimensional MCMC sampler for mass-map**, promoting sparsity in a wavelet basis. This sampler gradually grows the parameter space as required by the data, exploiting the extremely sparse nature of mass maps in wavelet space. The wavelet coefficients are arranged in a tree-like structure, which adds finer scale detail as the parameter space grows. We demonstrate the trans-dimensional sampler on galaxy cluster-scale images where the planar modelling approximation is valid. In high-resolution experiments, this method produces naturally parsimonious solutions, requiring less than 1% of the potential maximum number of wavelet coefficients and still producing a good fit to the observed data. In the presence of noisy data, trans-dimensional MCMC produces a better reconstruction of mass-maps than the standard smoothed Kaiser-Squires method, with the addition that uncertainties are fully quantified. This opens up the possibility for new mass maps and inferences about the nature of dark matter using the new high-resolution data from upcoming weak lensing surveys such as Euclid. △ Less

Submitted 16 June, 2023; v1 submitted 25 November, 2022; originally announced November 2022.

Impact of Rubin Observatory cadence choices on supernovae photometric classification

Authors: Catarina S. Alves, Hiranya V. Peiris, Michelle Lochner, Jason D. McEwen, Richard Kessler

Abstract: The Vera C. Rubin Observatory's Legacy Survey of Space and Time (LSST) will discover an unprecedented number of supernovae (SNe), making spectroscopic classification for all the events infeasible. LSST will thus rely on photometric classification, whose accuracy depends on the not-yet-finalized LSST observing strategy. In this work, we analyze the impact of cadence choices on classification perfor… ▽ More The Vera C. Rubin Observatory's Legacy Survey of Space and Time (LSST) will discover an unprecedented number of supernovae (SNe), making spectroscopic classification for all the events infeasible. LSST will thus rely on photometric classification, whose accuracy depends on the not-yet-finalized LSST observing strategy. In this work, we analyze the impact of cadence choices on classification performance using simulated multi-band light curves. First, we simulate SNe with an LSST baseline cadence, a non-rolling cadence, and a presto-color cadence which observes each sky location three times per night instead of twice. Each simulated dataset includes a spectroscopically-confirmed training set, which we augment to be representative of the test set as part of the classification pipeline. Then, we use the photometric transient classification library snmachine to build classifiers. We find that the active region of the rolling cadence used in the baseline observing strategy yields a 25% improvement in classification performance relative to the background region. This improvement in performance in the actively-rolling region is also associated with an increase of up to a factor of 2.7 in the number of cosmologically-useful Type Ia supernovae relative to the background region. However, adding a third visit per night as implemented in presto-color degrades classification performance due to more irregularly sampled light curves. Overall, our results establish desiderata on the observing cadence related to classification of full SNe light curves, which in turn impacts photometric SNe cosmology with LSST. △ Less

Submitted 15 March, 2023; v1 submitted 27 October, 2022; originally announced October 2022.

Comments: 22 pages, 14 figures. Changed to match version accepted by the Astrophysical Journal Supplement Series (accepted 06/02/2023)

Scalable and Equivariant Spherical CNNs by Discrete-Continuous (DISCO) Convolutions

Authors: Jeremy Ocampo, Matthew A. Price, Jason D. McEwen

Abstract: No existing spherical convolutional neural network (CNN) framework is both computationally scalable and rotationally equivariant. Continuous approaches capture rotational equivariance but are often prohibitively computationally demanding. Discrete approaches offer more favorable computational performance but at the cost of equivariance. We develop a hybrid discrete-continuous (DISCO) group convolu… ▽ More No existing spherical convolutional neural network (CNN) framework is both computationally scalable and rotationally equivariant. Continuous approaches capture rotational equivariance but are often prohibitively computationally demanding. Discrete approaches offer more favorable computational performance but at the cost of equivariance. We develop a hybrid discrete-continuous (DISCO) group convolution that is simultaneously equivariant and computationally scalable to high-resolution. While our framework can be applied to any compact group, we specialize to the sphere. Our DISCO spherical convolutions exhibit $\text{SO}(3)$ rotational equivariance, where $\text{SO}(n)$ is the special orthogonal group representing rotations in $n$-dimensions. When restricting rotations of the convolution to the quotient space $\text{SO}(3)/\text{SO}(2)$ for further computational enhancements, we recover a form of asymptotic $\text{SO}(3)$ rotational equivariance. Through a sparse tensor implementation we achieve linear scaling in number of pixels on the sphere for both computational cost and memory usage. For 4k spherical images we realize a saving of $10^9$ in computational cost and $10^4$ in memory usage when compared to the most efficient alternative equivariant spherical convolution. We apply the DISCO spherical CNN framework to a number of benchmark dense-prediction problems on the sphere, such as semantic segmentation and depth estimation, on all of which we achieve the state-of-the-art performance. △ Less

Submitted 28 January, 2023; v1 submitted 27 September, 2022; originally announced September 2022.

Comments: 19 pages, 7 figures, accepted by ICLR 2023

Bayesian model comparison for simulation-based inference

Authors: A. Spurio Mancini, M. M. Docherty, M. A. Price, J. D. McEwen

Abstract: Comparison of appropriate models to describe observational data is a fundamental task of science. The Bayesian model evidence, or marginal likelihood, is a computationally challenging, yet crucial, quantity to estimate to perform Bayesian model comparison. We introduce a methodology to compute the Bayesian model evidence in simulation-based inference (SBI) scenarios (also often called likelihood-f… ▽ More Comparison of appropriate models to describe observational data is a fundamental task of science. The Bayesian model evidence, or marginal likelihood, is a computationally challenging, yet crucial, quantity to estimate to perform Bayesian model comparison. We introduce a methodology to compute the Bayesian model evidence in simulation-based inference (SBI) scenarios (also often called likelihood-free inference). In particular, we leverage the recently proposed learnt harmonic mean estimator and exploit the fact that it is decoupled from the method used to generate posterior samples, i.e. it requires posterior samples only, which may be generated by any approach. This flexibility, which is lacking in many alternative methods for computing the model evidence, allows us to develop SBI model comparison techniques for the three main neural density estimation approaches, including neural posterior estimation (NPE), neural likelihood estimation (NLE), and neural ratio estimation (NRE). We demonstrate and validate our SBI evidence calculation techniques on a range of inference problems, including a gravitational wave example. Moreover, we further validate the accuracy of the learnt harmonic mean estimator, implemented in the HARMONIC software, in likelihood-based settings. These results highlight the potential of HARMONIC as a sampler-agnostic method to estimate the model evidence in both likelihood-based and simulation-based scenarios. △ Less

Submitted 8 November, 2023; v1 submitted 8 July, 2022; originally announced July 2022.

Comments: 13 pages, 5 figures. 2 min. Matches version published in RASTI. Summary video available at https://youtu.be/xbTS_5pGjaA. HARMONIC available at https://github.com/astro-informatics/harmonic

On Weak Lensing Response Functions

Authors: D. Munshi, R. Takahashi, J. D. McEwen

Abstract: We introduce the response function (RFs) approach to model the weak lensing statistics in the context of separate universe formalism. Numerical results for the RFs are presented for various semi-analytical models that include perturbative modelling and variants of halo models. These results extend the recent studies of the Integrated Bispectrum (IB) and Trispectrum to arbitrary order. We find that… ▽ More We introduce the response function (RFs) approach to model the weak lensing statistics in the context of separate universe formalism. Numerical results for the RFs are presented for various semi-analytical models that include perturbative modelling and variants of halo models. These results extend the recent studies of the Integrated Bispectrum (IB) and Trispectrum to arbitrary order. We find that due to the line-of-sight (los) projection effects, the expressions for RFs are not identical to the squeezed correlation functions of the same order. We compute the RFs in three-dimensions (3D) using the spherical Fourier-Bessel (sFB) formalism which provides a natural framework for incorporating photometric redshifts, and relate these expressions to tomographic and projected statistics. We generalise the concept of $k$-cut power spectrum to $k$-cut response functions. In addition to the response function for high-order spectra, we also define their counterparts in real space, since they are easier to estimate from surveys with low sky-coverage and non-trivial survey boundaries. △ Less

Submitted 7 July, 2022; originally announced July 2022.

Comments: 22 pages, 7 figures

Journal ref: JCAP10(2022)022

Weak Lensing Trispectrum and Kurt-Spectra

Authors: Dipak Munshi, Hayden Lee, Cora Dvorkin, Jason D. McEwen

Abstract: We introduce two kurt-spectra to probe fourth-order statistics of weak lensing convergence maps. Using state-of-the-art numerical simulations, we study the shapes of these kurt-spectra as a function of source redshifts and smoothing angular scales. We employ a pseudo-$C_{\ell}$ approach to estimate the spectra from realistic convergence maps in the presence of an observational mask and noise for s… ▽ More We introduce two kurt-spectra to probe fourth-order statistics of weak lensing convergence maps. Using state-of-the-art numerical simulations, we study the shapes of these kurt-spectra as a function of source redshifts and smoothing angular scales. We employ a pseudo-$C_{\ell}$ approach to estimate the spectra from realistic convergence maps in the presence of an observational mask and noise for stage-IV large-scale structure surveys. We compare these results against theoretical predictions calculated using the FFTLog formalism, and find that a simple nonlinear clustering model-the hierarchical ansatz-can reproduce the numerical trends for the kurt-spectra in the nonlinear regime. In addition, we provide estimators for beyond fourth-order spectra where no definitive analytical results are available, and present corresponding results from numerical simulations. △ Less

Submitted 17 October, 2022; v1 submitted 9 December, 2021; originally announced December 2021.

Comments: 32 pages, 6 figures, accepted for publication in JCAP

Journal ref: JCAP11(2022)020

Machine learning assisted Bayesian model comparison: learnt harmonic mean estimator

Authors: Jason D. McEwen, Christopher G. R. Wallis, Matthew A. Price, Alessio Spurio Mancini

Abstract: We resurrect the infamous harmonic mean estimator for computing the marginal likelihood (Bayesian evidence) and solve its problematic large variance. The marginal likelihood is a key component of Bayesian model selection to evaluate model posterior probabilities; however, its computation is challenging. The original harmonic mean estimator, first proposed by Newton and Raftery in 1994, involves co… ▽ More We resurrect the infamous harmonic mean estimator for computing the marginal likelihood (Bayesian evidence) and solve its problematic large variance. The marginal likelihood is a key component of Bayesian model selection to evaluate model posterior probabilities; however, its computation is challenging. The original harmonic mean estimator, first proposed by Newton and Raftery in 1994, involves computing the harmonic mean of the likelihood given samples from the posterior. It was immediately realised that the original estimator can fail catastrophically since its variance can become very large (possibly not finite). A number of variants of the harmonic mean estimator have been proposed to address this issue although none have proven fully satisfactory. We present the \emph{learnt harmonic mean estimator}, a variant of the original estimator that solves its large variance problem. This is achieved by interpreting the harmonic mean estimator as importance sampling and introducing a new target distribution. The new target distribution is learned to approximate the optimal but inaccessible target, while minimising the variance of the resulting estimator. Since the estimator requires samples of the posterior only, it is agnostic to the sampling strategy used. We validate the estimator on a variety of numerical experiments, including a number of pathological examples where the original harmonic mean estimator fails catastrophically. We also consider a cosmological application, where our approach leads to $\sim$ 3 to 6 times more samples than current state-of-the-art techniques in 1/3 of the time. In all cases our learnt harmonic mean estimator is shown to be highly accurate. The estimator is computationally scalable and can be applied to problems of dimension $O(10^3)$ and beyond. Code implementing the learnt harmonic mean estimator is made publicly available △ Less

Submitted 24 November, 2023; v1 submitted 24 November, 2021; originally announced November 2021.

Comments: 42 pages, 10 figures, code available at https://github.com/astro-informatics/harmonic

A New Estimator for Phase Statistics

Authors: D. Munshi, R. Takahashi, J. D. McEwen, T. D. Kitching, F. R. Bouchet

Abstract: We introduce a novel statistic to probe the statistics of phases of Fourier modes in two-dimensions (2D) for weak lensing convergence field $κ$. This statistic contains completely independent information compared to that contained in observed power spectrum. We compare our results against state-of-the-art numerical simulations as a function of source redshift and find good agreement with theoretic… ▽ More We introduce a novel statistic to probe the statistics of phases of Fourier modes in two-dimensions (2D) for weak lensing convergence field $κ$. This statistic contains completely independent information compared to that contained in observed power spectrum. We compare our results against state-of-the-art numerical simulations as a function of source redshift and find good agreement with theoretical predictions. We show that our estimator can achieve better signal-to-noise compared to the commonly employed statistics known as the line correlation function (LCF). Being a two-point statistics, our estimator is also easy to implement in the presence of complicated noise and mask, and can also be generalised to higher-order. While applying this estimator for the study of lensed CMB maps, we show that it is important to include post-Born corrections in the study of statistics of phase. △ Less

Submitted 16 September, 2021; originally announced September 2021.

Comments: 18 pages, 7 figures

Journal ref: JCAP05(2022)006

Considerations for optimizing photometric classification of supernovae from the Rubin Observatory

Authors: Catarina S. Alves, Hiranya V. Peiris, Michelle Lochner, Jason D. McEwen, Tarek Allam Jr, Rahul Biswas

Abstract: The Vera C. Rubin Observatory will increase the number of observed supernovae (SNe) by an order of magnitude; however, it is impossible to spectroscopically confirm the class for all the SNe discovered. Thus, photometric classification is crucial but its accuracy depends on the not-yet-finalized observing strategy of Rubin Observatory's Legacy Survey of Space and Time (LSST). We quantitatively ana… ▽ More The Vera C. Rubin Observatory will increase the number of observed supernovae (SNe) by an order of magnitude; however, it is impossible to spectroscopically confirm the class for all the SNe discovered. Thus, photometric classification is crucial but its accuracy depends on the not-yet-finalized observing strategy of Rubin Observatory's Legacy Survey of Space and Time (LSST). We quantitatively analyze the impact of the LSST observing strategy on SNe classification using simulated multi-band light curves from the Photometric LSST Astronomical Time-Series Classification Challenge (PLAsTiCC). First, we augment the simulated training set to be representative of the photometric redshift distribution per supernovae class, the cadence of observations, and the flux uncertainty distribution of the test set. Then we build a classifier using the photometric transient classification library snmachine, based on wavelet features obtained from Gaussian process fits, yielding similar performance to the winning PLAsTiCC entry. We study the classification performance for SNe with different properties within a single simulated observing strategy. We find that season length is important, with light curves of 150 days yielding the highest performance. Cadence also has an important impact on SNe classification; events with median inter-night gap <3.5 days yield higher classification performance. Interestingly, we find that large gaps (>10 days) in light curve observations do not impact performance if sufficient observations are available on either side, due to the effectiveness of the Gaussian process interpolation. This analysis is the first exploration of the impact of observing strategy on photometric supernova classification with LSST. △ Less

Submitted 29 October, 2021; v1 submitted 15 July, 2021; originally announced July 2021.

Comments: 18 pages, 13 figures. Changed to match version accepted by the Astrophysical Journal Supplement Series (accepted 28/10/2021). Software publicly available at https://github.com/LSSTDESC/snmachine

Journal ref: 2022 ApJS 258 23

Proximal nested sampling for high-dimensional Bayesian model selection

Authors: Xiaohao Cai, Jason D. McEwen, Marcelo Pereyra

Abstract: Bayesian model selection provides a powerful framework for objectively comparing models directly from observed data, without reference to ground truth data. However, Bayesian model selection requires the computation of the marginal likelihood (model evidence), which is computationally challenging, prohibiting its use in many high-dimensional Bayesian inverse problems. With Bayesian imaging applica… ▽ More Bayesian model selection provides a powerful framework for objectively comparing models directly from observed data, without reference to ground truth data. However, Bayesian model selection requires the computation of the marginal likelihood (model evidence), which is computationally challenging, prohibiting its use in many high-dimensional Bayesian inverse problems. With Bayesian imaging applications in mind, in this work we present the proximal nested sampling methodology to objectively compare alternative Bayesian imaging models for applications that use images to inform decisions under uncertainty. The methodology is based on nested sampling, a Monte Carlo approach specialised for model comparison, and exploits proximal Markov chain Monte Carlo techniques to scale efficiently to large problems and to tackle models that are log-concave and not necessarily smooth (e.g., involving l_1 or total-variation priors). The proposed approach can be applied computationally to problems of dimension O(10^6) and beyond, making it suitable for high-dimensional inverse imaging problems. It is validated on large Gaussian models, for which the likelihood is available analytically, and subsequently illustrated on a range of imaging problems where it is used to analyse different choices of dictionary and measurement model. △ Less

Submitted 9 September, 2022; v1 submitted 7 June, 2021; originally announced June 2021.

Journal ref: Statistics & Computing, 32, 87 (2002)

Slepian Scale-Discretised Wavelets on the Sphere

Authors: Patrick J. Roddy, Jason D. McEwen

Abstract: This work presents the construction of a novel spherical wavelet basis designed for incomplete spherical datasets, i.e. datasets which are missing in a particular region of the sphere. The eigenfunctions of the Slepian spatial-spectral concentration problem (the Slepian functions) are a set of orthogonal basis functions which are more concentrated within a defined region. Slepian functions allow o… ▽ More This work presents the construction of a novel spherical wavelet basis designed for incomplete spherical datasets, i.e. datasets which are missing in a particular region of the sphere. The eigenfunctions of the Slepian spatial-spectral concentration problem (the Slepian functions) are a set of orthogonal basis functions which are more concentrated within a defined region. Slepian functions allow one to compute a convolution on the incomplete sphere by leveraging the recently proposed sifting convolution and extending it to any set of basis functions. Through a tiling of the Slepian harmonic line, one may construct scale-discretised wavelets. An illustration is presented based on an example region on the sphere defined by the topographic map of the Earth. The Slepian wavelets and corresponding wavelet coefficients are constructed from this region and are used in a straightforward denoising example. △ Less

Submitted 23 December, 2022; v1 submitted 3 June, 2021; originally announced June 2021.

Comments: 12 pages, 14 figures

Journal ref: IEEE Transactions on Signal Processing, vol. 70, pp. 6142-6153, 2022

Paying Attention to Astronomical Transients: Introducing the Time-series Transformer for Photometric Classification

Authors: Tarek Allam Jr., Jason D. McEwen

Abstract: Future surveys such as the Legacy Survey of Space and Time (LSST) of the Vera C. Rubin Observatory will observe an order of magnitude more astrophysical transient events than any previous survey before. With this deluge of photometric data, it will be impossible for all such events to be classified by humans alone. Recent efforts have sought to leverage machine learning methods to tackle the chall… ▽ More Future surveys such as the Legacy Survey of Space and Time (LSST) of the Vera C. Rubin Observatory will observe an order of magnitude more astrophysical transient events than any previous survey before. With this deluge of photometric data, it will be impossible for all such events to be classified by humans alone. Recent efforts have sought to leverage machine learning methods to tackle the challenge of astronomical transient classification, with ever improving success. Transformers are a recently developed deep learning architecture, first proposed for natural language processing, that have shown a great deal of recent success. In this work we develop a new transformer architecture, which uses multi-head self attention at its core, for general multi-variate time-series data. Furthermore, the proposed time-series transformer architecture supports the inclusion of an arbitrary number of additional features, while also offering interpretability. We apply the time-series transformer to the task of photometric classification, minimising the reliance of expert domain knowledge for feature selection, while achieving results comparable to state-of-the-art photometric classification methods. We achieve a logarithmic-loss of 0.507 on imbalanced data in a representative setting using data from the Photometric LSST Astronomical Time-Series Classification Challenge (PLAsTiCC). Moreover, we achieve a micro-averaged receiver operating characteristic area under curve of 0.98 and micro-averaged precision-recall area under curve of 0.87. △ Less

Submitted 4 October, 2023; v1 submitted 13 May, 2021; originally announced May 2021.

Comments: Manuscript Accepted to RAS Techniques and Instruments. 15 pages, 12 figures

Bayesian variational regularization on the ball

Authors: Matthew A. Price, Jason D. McEwen

Abstract: We develop variational regularization methods which leverage sparsity-promoting priors to solve severely ill posed inverse problems defined on the 3D ball (i.e. the solid sphere). Our method solves the problem natively on the ball and thus does not suffer from discontinuities that plague alternate approaches where each spherical shell is considered independently. Additionally, we leverage advances… ▽ More We develop variational regularization methods which leverage sparsity-promoting priors to solve severely ill posed inverse problems defined on the 3D ball (i.e. the solid sphere). Our method solves the problem natively on the ball and thus does not suffer from discontinuities that plague alternate approaches where each spherical shell is considered independently. Additionally, we leverage advances in probability density theory to produce Bayesian variational methods which benefit from the computational efficiency of advanced convex optimization algorithms, whilst supporting principled uncertainty quantification. We showcase these variational regularization and uncertainty quantification techniques on an illustrative example. The C++ code discussed throughout is provided under a GNU general public license. △ Less

Submitted 12 May, 2021; originally announced May 2021.

Sparse image reconstruction on the sphere: a general approach with uncertainty quantification

Authors: Matthew A. Price, Luke Pratley, Jason D. McEwen

Abstract: Inverse problems defined naturally on the sphere are becoming increasingly of interest. In this article we provide a general framework for evaluation of inverse problems on the sphere, with a strong emphasis on flexibility and scalability. We consider flexibility with respect to the prior selection (regularization), the problem definition - specifically the problem formulation (constrained/unconst… ▽ More Inverse problems defined naturally on the sphere are becoming increasingly of interest. In this article we provide a general framework for evaluation of inverse problems on the sphere, with a strong emphasis on flexibility and scalability. We consider flexibility with respect to the prior selection (regularization), the problem definition - specifically the problem formulation (constrained/unconstrained) and problem setting (analysis/synthesis) - and optimization adopted to solve the problem. We discuss and quantify the trade-offs between problem formulation and setting. Crucially, we consider the Bayesian interpretation of the unconstrained problem which, combined with recent developments in probability density theory, permits rapid, statistically principled uncertainty quantification (UQ) in the spherical setting. Linearity is exploited to significantly increase the computational efficiency of such UQ techniques, which in some cases are shown to permit analytic solutions. We showcase this reconstruction framework and UQ techniques on a variety of spherical inverse problems. The code discussed throughout is provided under a GNU general public license, in both C++ and Python. △ Less

Submitted 11 May, 2021; originally announced May 2021.

Scattering Networks on the Sphere for Scalable and Rotationally Equivariant Spherical CNNs

Authors: Jason D. McEwen, Christopher G. R. Wallis, Augustine N. Mavor-Parker

Abstract: Convolutional neural networks (CNNs) constructed natively on the sphere have been developed recently and shown to be highly effective for the analysis of spherical data. While an efficient framework has been formulated, spherical CNNs are nevertheless highly computationally demanding; typically they cannot scale beyond spherical signals of thousands of pixels. We develop scattering networks constr… ▽ More Convolutional neural networks (CNNs) constructed natively on the sphere have been developed recently and shown to be highly effective for the analysis of spherical data. While an efficient framework has been formulated, spherical CNNs are nevertheless highly computationally demanding; typically they cannot scale beyond spherical signals of thousands of pixels. We develop scattering networks constructed natively on the sphere that provide a powerful representational space for spherical data. Spherical scattering networks are computationally scalable and exhibit rotational equivariance, while their representational space is invariant to isometries and provides efficient and stable signal representations. By integrating scattering networks as an additional type of layer in the generalized spherical CNN framework, we show how they can be leveraged to scale spherical CNNs to the high-resolution data typical of many practical applications, with spherical signals of many tens of megapixels and beyond. △ Less

Submitted 24 January, 2022; v1 submitted 4 February, 2021; originally announced February 2021.

Comments: 18 pages, 6 figures, accepted by ICLR, code at https://www.kagenova.com/products/fourpiAI/

Results of the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC)

Authors: R. Hložek, K. A. Ponder, A. I. Malz, M. Dai, G. Narayan, E. E. O. Ishida, T. Allam Jr, A. Bahmanyar, R. Biswas, L. Galbany, S. W. Jha, D. O. Jones, R. Kessler, M. Lochner, A. A. Mahabal, K. S. Mandel, J. R. Martínez-Galarza, J. D. McEwen, D. Muthukrishna, H. V. Peiris, C. M. Peters, C. N. Setzer

Abstract: Next-generation surveys like the Legacy Survey of Space and Time (LSST) on the Vera C. Rubin Observatory will generate orders of magnitude more discoveries of transients and variable stars than previous surveys. To prepare for this data deluge, we developed the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC), a competition which aimed to catalyze the development of ro… ▽ More Next-generation surveys like the Legacy Survey of Space and Time (LSST) on the Vera C. Rubin Observatory will generate orders of magnitude more discoveries of transients and variable stars than previous surveys. To prepare for this data deluge, we developed the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC), a competition which aimed to catalyze the development of robust classifiers under LSST-like conditions of a non-representative training set for a large photometric test set of imbalanced classes. Over 1,000 teams participated in PLAsTiCC, which was hosted in the Kaggle data science competition platform between Sep 28, 2018 and Dec 17, 2018, ultimately identifying three winners in February 2019. Participants produced classifiers employing a diverse set of machine learning techniques including hybrid combinations and ensemble averages of a range of approaches, among them boosted decision trees, neural networks, and multi-layer perceptrons. The strong performance of the top three classifiers on Type Ia supernovae and kilonovae represent a major improvement over the current state-of-the-art within astronomy. This paper summarizes the most promising methods and evaluates their results in detail, highlighting future directions both for classifier development and simulation needs for a next generation PLAsTiCC data set. △ Less

Submitted 22 December, 2020; originally announced December 2020.

Comments: 20 pages, 14 figures

Efficient Generalized Spherical CNNs

Authors: Oliver J. Cobb, Christopher G. R. Wallis, Augustine N. Mavor-Parker, Augustin Marignier, Matthew A. Price, Mayeul d'Avezac, Jason D. McEwen

Abstract: Many problems across computer vision and the natural sciences require the analysis of spherical data, for which representations may be learned efficiently by encoding equivariance to rotational symmetries. We present a generalized spherical CNN framework that encompasses various existing approaches and allows them to be leveraged alongside each other. The only existing non-linear spherical CNN lay… ▽ More Many problems across computer vision and the natural sciences require the analysis of spherical data, for which representations may be learned efficiently by encoding equivariance to rotational symmetries. We present a generalized spherical CNN framework that encompasses various existing approaches and allows them to be leveraged alongside each other. The only existing non-linear spherical CNN layer that is strictly equivariant has complexity $\mathcal{O}(C^2L^5)$, where $C$ is a measure of representational capacity and $L$ the spherical harmonic bandlimit. Such a high computational cost often prohibits the use of strictly equivariant spherical CNNs. We develop two new strictly equivariant layers with reduced complexity $\mathcal{O}(CL^4)$ and $\mathcal{O}(CL^3 \log L)$, making larger, more expressive models computationally feasible. Moreover, we adopt efficient sampling theory to achieve further computational savings. We show that these developments allow the construction of more expressive hybrid models that achieve state-of-the-art accuracy and parameter efficiency on spherical benchmark problems. △ Less

Submitted 8 March, 2021; v1 submitted 9 October, 2020; originally announced October 2020.

Comments: 20 pages, 4 figures, accepted by ICLR, code at https://www.kagenova.com/products/fourpiAI/

Morphology of Weak Lensing Convergence Maps

Authors: D. Munshi, T. Namikawa, J. D. McEwen, T. D. Kitching, F. R. Bouchet

Abstract: We study the morphology of convergence maps by perturbatively reconstructing their Minkowski Functionals (MFs). We present a systematics study using a set of three generalised skew-spectra as a function of source redshift and smoothing angular scale. Using an approach based on pseudo-$S_{\ell}$s (PSL) we show how these spectra will allow reconstruction of MFs in the presence of an arbitrary mask a… ▽ More We study the morphology of convergence maps by perturbatively reconstructing their Minkowski Functionals (MFs). We present a systematics study using a set of three generalised skew-spectra as a function of source redshift and smoothing angular scale. Using an approach based on pseudo-$S_{\ell}$s (PSL) we show how these spectra will allow reconstruction of MFs in the presence of an arbitrary mask and inhomogeneous noise in an unbiased way. Our theoretical predictions are based on a recently introduced fitting function to the bispectrum. We compare our results against state-of-the art numerical simulations and find an excellent agreement. The reconstruction can be carried out in a controlled manner as a function of angular harmonics $\ell$ and source redshift $z_s$ which allows for a greater handle on any possible sources of non-Gaussianity. Our method has the advantage of estimating the topology of convergence maps directly using shear data. We also study weak lensing convergence maps inferred from Cosmic Microwave Background (CMB) observations; and we find that, though less significant at low redshift, the post-Born corrections play an important role in any modelling of the non-Gaussianity of convergence maps at higher redshift. We also study the cross-correlations of estimates from different tomographic bins. △ Less

Submitted 12 October, 2020; originally announced October 2020.

Comments: 16 pages, 12 figures

Novel perspectives gained from new reconstruction algorithms

Authors: Luke Pratley, Melanie Johnston-Hollitt, Jason D. McEwen

Abstract: Since the 1970s, much of traditional interferometric imaging has been built around variations of the CLEAN algorithm, in both terminology, methodology, and algorithm development. Recent developments in applying new algorithms from convex optimization to interferometry has allowed old concepts to be viewed from a new perspective, ranging from image restoration to the development of computationally… ▽ More Since the 1970s, much of traditional interferometric imaging has been built around variations of the CLEAN algorithm, in both terminology, methodology, and algorithm development. Recent developments in applying new algorithms from convex optimization to interferometry has allowed old concepts to be viewed from a new perspective, ranging from image restoration to the development of computationally distributed algorithms. We present how this has ultimately led the authors to new perspectives in wide-field imaging, allowing for the first full individual non-coplanar corrections applied during imaging over extremely wide-fields of view for the Murchison Widefield Array (MWA) telescope. Furthermore, this same mathematical framework has provided a novel understanding of wide-band polarimetry at low frequencies, where instrumental channel depolarization can be corrected through the new $δλ^2$-projection algorithm. This is a demonstration that new algorithm development outside of traditional radio astronomy is valuable for the new theoretical and practical perspectives gained. These perspectives are timely with the next generation of radio telescopes coming online. △ Less

Submitted 26 September, 2020; originally announced September 2020.

Comments: 4 pages, 1 figure. URSI GASS 2020, Rome, Italy, 29 August - 5 September 2020

Planck intermediate results. LV. Reliability and thermal properties of high-frequency sources in the Second Planck Catalogue of Compact Sources

Authors: Planck Collaboration, Y. Akrami, M. Ashdown, J. Aumont, C. Baccigalupi, M. Ballardini, A. J. Banday, R. B. Barreiro, N. Bartolo, S. Basak, K. Benabed, J. -P. Bernard, M. Bersanelli, P. Bielewicz, J. R. Bond, J. Borrill, F. R. Bouchet, C. Burigana, E. Calabrese, P. Carvalho, H. C. Chiang, B. P. Crill, F. Cuttaia, A. de Rosa, G. de Zotti , et al. (95 additional authors not shown)

Abstract: We describe an extension of the most recent version of the Planck Catalogue of Compact Sources (PCCS2), produced using a new multi-band Bayesian Extraction and Estimation Package (BeeP). BeeP assumes that the compact sources present in PCCS2 at 857 GHz have a dust-like spectral energy distribution, which leads to emission at both lower and higher frequencies, and adjusts the parameters of the sour… ▽ More We describe an extension of the most recent version of the Planck Catalogue of Compact Sources (PCCS2), produced using a new multi-band Bayesian Extraction and Estimation Package (BeeP). BeeP assumes that the compact sources present in PCCS2 at 857 GHz have a dust-like spectral energy distribution, which leads to emission at both lower and higher frequencies, and adjusts the parameters of the source and its SED to fit the emission observed in Planck's three highest frequency channels at 353, 545, and 857 GHz, as well as the IRIS map at 3000 GHz. In order to reduce confusion regarding diffuse cirrus emission, BeeP's data model includes a description of the background emission surrounding each source, and it adjusts the confidence in the source parameter extraction based on the statistical properties of the spatial distribution of the background emission. BeeP produces the following three new sets of parameters for each source: (a) fits to a modified blackbody (MBB) thermal emission model of the source; (b) SED-independent source flux densities at each frequency considered; and (c) fits to an MBB model of the background in which the source is embedded. BeeP also calculates, for each source, a reliability parameter, which takes into account confusion due to the surrounding cirrus. We define a high-reliability subset (BeeP/base), containing 26 083 sources (54.1 per cent of the total PCCS2 catalogue), the majority of which have no information on reliability in the PCCS2. The results of the BeeP extension of PCCS2, which are made publicly available via the PLA, will enable the study of the thermal properties of well-defined samples of compact Galactic and extra-galactic dusty sources. △ Less

Submitted 14 September, 2020; originally announced September 2020.

Comments: 55 pages. Accepted for publication in A&A. The BeeP catalogue will be published in the Planck Legacy Archive (https://pla.esac.esa.int/pla)

Journal ref: A&A 644, A99 (2020)

Sifting Convolution on the Sphere

Authors: Patrick J. Roddy, Jason D. McEwen

Abstract: A novel spherical convolution is defined through the sifting property of the Dirac delta on the sphere. The so-called sifting convolution is defined by the inner product of one function with a translated version of another, but with the adoption of an alternative translation operator on the sphere. This translation operator follows by analogy with the Euclidean translation when viewed in harmonic… ▽ More A novel spherical convolution is defined through the sifting property of the Dirac delta on the sphere. The so-called sifting convolution is defined by the inner product of one function with a translated version of another, but with the adoption of an alternative translation operator on the sphere. This translation operator follows by analogy with the Euclidean translation when viewed in harmonic space. The sifting convolution satisfies a variety of desirable properties that are lacking in alternate definitions, namely: it supports directional kernels; it has an output which remains on the sphere; and is efficient to compute. An illustration of the sifting convolution on a topographic map of the Earth demonstrates that it supports directional kernels to perform anisotropic filtering, while its output remains on the sphere. △ Less

Submitted 29 October, 2020; v1 submitted 23 July, 2020; originally announced July 2020.

Comments: 5 pages, 3 figures

Journal ref: IEEE Signal Processing Letters, vol. 28, pp. 304-308, 2021

Planck intermediate results. LVII. Joint Planck LFI and HFI data processing

Authors: Planck Collaboration, Y. Akrami, K. J. Andersen, M. Ashdown, C. Baccigalupi, M. Ballardini, A. J. Banday, R. B. Barreiro, N. Bartolo, S. Basak, K. Benabed, J. -P. Bernard, M. Bersanelli, P. Bielewicz, J. R. Bond, J. Borrill, C. Burigana, R. C. Butler, E. Calabrese, B. Casaponsa, H. C. Chiang, L. P. L. Colombo, C. Combet, B. P. Crill, F. Cuttaia , et al. (114 additional authors not shown)

Abstract: We present the NPIPE processing pipeline, which produces calibrated frequency maps in temperature and polarization from data from the Planck Low Frequency Instrument (LFI) and High Frequency Instrument (HFI) using high-performance computers. NPIPE represents a natural evolution of previous Planck analysis efforts, and combines some of the most powerful features of the separate LFI and HFI analysis… ▽ More We present the NPIPE processing pipeline, which produces calibrated frequency maps in temperature and polarization from data from the Planck Low Frequency Instrument (LFI) and High Frequency Instrument (HFI) using high-performance computers. NPIPE represents a natural evolution of previous Planck analysis efforts, and combines some of the most powerful features of the separate LFI and HFI analysis pipelines. The net effect of the improvements is lower levels of noise and systematics in both frequency and component maps at essentially all angular scales, as well as notably improved internal consistency between the various frequency channels. Based on the NPIPE maps, we present the first estimate of the Solar dipole determined through component separation across all nine Planck frequencies. The amplitude is ($3366.6 \pm 2.7$)$μ$K, consistent with, albeit slightly higher than, earlier estimates. From the large-scale polarization data, we derive an updated estimate of the optical depth of reionization of $τ= 0.051 \pm 0.006$, which appears robust with respect to data and sky cuts. There are 600 complete signal, noise and systematics simulations of the full-frequency and detector-set maps. As a Planck first, these simulations include full time-domain processing of the beam-convolved CMB anisotropies. The release of NPIPE maps and simulations is accompanied with a complete suite of raw and processed time-ordered data and the software, scripts, auxiliary data, and parameter files needed to improve further on the analysis and to run matching simulations. △ Less

Submitted 9 July, 2020; originally announced July 2020.

Comments: 97 pages, 93 figures and 16 tables, abstract abridged for arXiv submission, accepted for publication in A&A

Journal ref: A&A 643, A42 (2020)

Weak Lensing Skew-Spectrum

Authors: D. Munshi, T. Namikawa, T. D. Kitching, J. D. McEwen, F. R. Bouchet

Abstract: We introduce the skew-spectrum statistic for weak lensing convergence $κ$ maps and test it against state-of-the-art high-resolution all-sky numerical simulations. We perform the analysis as a function of source redshift and smoothing angular scale for individual tomographic bins. We also analyse the cross-correlation between different tomographic bins. We compare the numerical results to fitti… ▽ More We introduce the skew-spectrum statistic for weak lensing convergence $κ$ maps and test it against state-of-the-art high-resolution all-sky numerical simulations. We perform the analysis as a function of source redshift and smoothing angular scale for individual tomographic bins. We also analyse the cross-correlation between different tomographic bins. We compare the numerical results to fitting-functions used to model the bispectrum of the underlying density field as a function of redshift and scale. We derive a closed form expression for the skew-spectrum for gravity-induced secondary non-Gaussianity. We also compute the skew-spectrum for the projected $κ$ inferred from Cosmic Microwave Background (CMB) studies. As opposed to the low redshift case we find the post-Born corrections to be important in the modelling of the skew-spectrum for such studies. We show how the presence of a mask and noise can be incorporated in the estimation of a skew-spectrum. △ Less

Submitted 23 June, 2020; originally announced June 2020.

Comments: 16 pages, 11 figures

Spherical Bayesian mass-map** with uncertainties: full sky observations on the celestial sphere

Authors: Matthew A. Price, Jason D. McEwen, L. Pratley, Thomas D. Kitching

Abstract: To date weak gravitational lensing surveys have typically been restricted to small fields of view, such that the $\textit{flat-sky approximation}$ has been sufficiently satisfied. However, with Stage IV surveys ($\textit{e.g. LSST}$ and $\textit{Euclid}$) imminent, extending mass-map** techniques to the sphere is a fundamental necessity. As such, we extend the sparse hierarchical Bayesian mass-m… ▽ More To date weak gravitational lensing surveys have typically been restricted to small fields of view, such that the $\textit{flat-sky approximation}$ has been sufficiently satisfied. However, with Stage IV surveys ($\textit{e.g. LSST}$ and $\textit{Euclid}$) imminent, extending mass-map** techniques to the sphere is a fundamental necessity. As such, we extend the sparse hierarchical Bayesian mass-map** formalism presented in previous work to the spherical sky. For the first time, this allows us to construct $\textit{maximum a posteriori}$ spherical weak lensing dark-matter mass-maps, with principled Bayesian uncertainties, without imposing or assuming Gaussianty. We solve the spherical mass-map** inverse problem in the analysis setting adopting a sparsity promoting Laplace-type wavelet prior, though this theoretical framework supports all log-concave posteriors. Our spherical mass-map** formalism facilitates principled statistical interpretation of reconstructions. We apply our framework to convergence reconstruction on high resolution N-body simulations with pseudo-Euclid masking, polluted with a variety of realistic noise levels, and show a significant increase in reconstruction fidelity compared to standard approaches. Furthermore we perform the largest joint reconstruction to date of the majority of publicly available shear observational datasets (combining DESY1, KiDS450 and CFHTLens) and find that our formalism recovers a convergence map with significantly enhanced small-scale detail. Within our Bayesian framework we validate, in a statistically rigorous manner, the community's intuition regarding the need to smooth spherical Kaiser-Squires estimates to provide physically meaningful convergence maps. Such approaches cannot reveal the small-scale physical structures that we recover within our framework. △ Less

Submitted 5 February, 2021; v1 submitted 16 April, 2020; originally announced April 2020.

Higher-Order Spectra of Weak Lensing Convergence Maps in Parameterized Theories of Modified Gravity

Authors: D. Munshi, J. D. McEwen

Abstract: We compute the low-$\ell$ limit of the family of higher-order spectra for projected (2D) weak lensing convergence maps. In this limit, these spectra are computed to an arbitrary order using {\em tree-level} perturbative calculations. We use the flat-sky approximation and Eulerian perturbative results based on a generating function approach. We test these results for the lower-order members of this… ▽ More We compute the low-$\ell$ limit of the family of higher-order spectra for projected (2D) weak lensing convergence maps. In this limit, these spectra are computed to an arbitrary order using {\em tree-level} perturbative calculations. We use the flat-sky approximation and Eulerian perturbative results based on a generating function approach. We test these results for the lower-order members of this family, i.e. the skew- and kurt-spectra against state-of-the-art simulated all-sky weak lensing convergence maps and find our results to be in very good agreement. We also show how these spectra can be computed in the presence of a realistic sky-mask and Gaussian noise. We generalize these results to three-dimensions (3D) and compute the {\em equal-time} higher-order spectra. These results will be valuable in analyzing higher-order statistics from future all-sky weak lensing surveys such as the {\em Euclid} survey at low-$\ell$ modes. As illustrative examples, we compute these statistics in the context of the {\em Horndeski} and {\em Beyond Horndeski} theories of modified gravity. They will be especially useful in constraining theories such as the Gleyzes-Langlois-Piazza-Vernizzi (GLPV) theories and Degenerate Higher-Order Scalar-Tensor (DHOST) theories as well as the commonly used normal-branch of Dvali-Gabadadze-Porrati (nDGP) model, clustering quintessence models, and scenarios with massive neutrinos. △ Less

Submitted 15 April, 2020; originally announced April 2020.

Comments: 22 pages, 5 figures

Offline and online reconstruction for radio interferometric imaging

Authors: Xiaohao Cai, Luke Pratley, Jason D. McEwen

Abstract: Radio astronomy is transitioning to a big-data era due to the emerging generation of radio interferometric (RI) telescopes, such as the Square Kilometre Array (SKA), which will acquire massive volumes of data. In this article we review methods proposed recently to resolve the ill-posed inverse problem of imaging the raw visibilities acquired by RI telescopes in the big-data scenario. We focus on t… ▽ More Radio astronomy is transitioning to a big-data era due to the emerging generation of radio interferometric (RI) telescopes, such as the Square Kilometre Array (SKA), which will acquire massive volumes of data. In this article we review methods proposed recently to resolve the ill-posed inverse problem of imaging the raw visibilities acquired by RI telescopes in the big-data scenario. We focus on the recently proposed online reconstruction method [4] and the considerable savings in data storage requirements and computational cost that it yields. △ Less

Submitted 8 April, 2020; originally announced April 2020.

Comments: 4 pages; 2 figures; URSI GASS 2020. arXiv admin note: substantial text overlap with arXiv:1712.04462

Planck intermediate results. LVI. Detection of the CMB dipole through modulation of the thermal Sunyaev-Zeldovich effect: Eppur si muove II

Authors: Planck Collaboration, Y. Akrami, M. Ashdown, J. Aumont, C. Baccigalupi, M. Ballardini, A. J. Banday, R. B. Barreiro, N. Bartolo, S. Basak, K. Benabed, J. -P. Bernard, M. Bersanelli, P. Bielewicz, J. R. Bond, J. Borrill, F. R. Bouchet, C. Burigana, E. Calabrese, J. -F. Cardoso, B. Casaponsa, H. C. Chiang, C. Combet, D. Contreras, B. P. Crill , et al. (104 additional authors not shown)

Abstract: The largest temperature anisotropy in the cosmic microwave background (CMB) is the dipole, which has been measured with increasing accuracy for more than three decades, particularly with the Planck satellite. The simplest interpretation of the dipole is that it is due to our motion with respect to the rest frame of the CMB. Since current CMB experiments infer temperature anisotropies from angular… ▽ More The largest temperature anisotropy in the cosmic microwave background (CMB) is the dipole, which has been measured with increasing accuracy for more than three decades, particularly with the Planck satellite. The simplest interpretation of the dipole is that it is due to our motion with respect to the rest frame of the CMB. Since current CMB experiments infer temperature anisotropies from angular intensity variations, the dipole modulates the temperature anisotropies with the same frequency dependence as the thermal Sunyaev-Zeldovich (tSZ) effect. We present the first, and significant, detection of this signal in the tSZ maps and find that it is consistent with direct measurements of the CMB dipole, as expected. The signal contributes power in the tSZ maps, which is modulated in a quadrupolar pattern, and we estimate its contribution to the tSZ bispectrum, noting that it contributes negligible noise to the bispectrum at relevant scales. △ Less

Submitted 7 September, 2020; v1 submitted 27 March, 2020; originally announced March 2020.

Comments: 15 pages, 8 figures. Added references, small clarifying and language edits. All results remain the same

Journal ref: A&A 644, A100 (2020)

The Weak Lensing Bispectrum Induced By Gravity

Authors: D. Munshi, T. Namikawa, T. D. Kitching, J. D. McEwen, R. Takahashi, F. R. Bouchet, A. Taruya, B. Bose

Abstract: Recent studies have demonstrated that {\em secondary} non-Gaussianity induced by gravity will be detected with a high signal-to-noise (S/N) by future and even by on-going weak lensing surveys. One way to characterise such non-Gaussianity is through the detection of a non-zero three-point correlation function of the lensing convergence field, or of its harmonic transform, the bispectrum. A recent s… ▽ More Recent studies have demonstrated that {\em secondary} non-Gaussianity induced by gravity will be detected with a high signal-to-noise (S/N) by future and even by on-going weak lensing surveys. One way to characterise such non-Gaussianity is through the detection of a non-zero three-point correlation function of the lensing convergence field, or of its harmonic transform, the bispectrum. A recent study analysed the properties of the squeezed configuration of the bispectrum, when two wavenumbers are much larger than the third one. We extend this work by estimating the amplitude of the (reduced) bispectrum in four generic configurations, i.e., {\em squeezed, equilateral, isosceles} and {\em folded}, and for four different source redshifts $z_s=0.5,1.0,1.5,2.0$, by using an ensemble of all-sky high-resolution simulations. We compare these results against theoretical predictions. We find that, while the theoretical expectations based on widely used fitting functions can predict the general trends of the reduced bispectra, a more accurate theoretical modelling will be required to analyse the next generation of all-sky weak lensing surveys. The disagreement is particularly pronounced in the squeezed limit. △ Less

Submitted 10 October, 2019; originally announced October 2019.

Comments: 12 pages, 2 figures

Journal ref: 2020, MNRAS, 493, 3985

Cleaning radio interferometric images using a spherical wavelet decomposition

Authors: Chris J. Skipper, Anna M. M. Scaife, Jason D. McEwen

Abstract: The deconvolution, or cleaning, of radio interferometric images often involves computing model visibilities from a list of clean components, in order that the contribution from the model can be subtracted from the observed visibilities. This step is normally performed using a forward fast Fourier transform (FFT), followed by a 'degridding' step that interpolates over the uv plane to construct the… ▽ More The deconvolution, or cleaning, of radio interferometric images often involves computing model visibilities from a list of clean components, in order that the contribution from the model can be subtracted from the observed visibilities. This step is normally performed using a forward fast Fourier transform (FFT), followed by a 'degridding' step that interpolates over the uv plane to construct the model visibilities. An alternative approach is to calculate the model visibilities directly by summing over all the members of the clean component list, which is a more accurate method that can also be much slower. However, if the clean components are used to construct a model image on the surface of the celestial sphere then the model visibilities can be generated directly from the wavelet coefficients, and the sparsity of the model means that most of these coefficients are zero, and can be ignored. We have constructed a prototype imager that uses a spherical-wavelet representation of the model image to generate model visibilities during each major cycle, and find empirically that the execution time scales with the wavelet resolution level, J, as O(1.07 J), and with the number of distinct clean components, N_C, as O(N_C). The prototype organises the wavelet coefficients into a tree structure, and does not store or process the zero wavelet coefficients. △ Less

Submitted 9 September, 2019; originally announced September 2019.

Comments: 12 pages, 12 figures

Planck 2018 results. V. CMB power spectra and likelihoods

Authors: Planck Collaboration, N. Aghanim, Y. Akrami, M. Ashdown, J. Aumont, C. Baccigalupi, M. Ballardini, A. J. Banday, R. B. Barreiro, N. Bartolo, S. Basak, K. Benabed, J. -P. Bernard, M. Bersanelli, P. Bielewicz, J. J. Bock, J. R. Bond, J. Borrill, F. R. Bouchet, F. Boulanger, M. Bucher, C. Burigana, R. C. Butler, E. Calabrese, J. -F. Cardoso , et al. (143 additional authors not shown)

Abstract: This paper describes the 2018 Planck CMB likelihoods, following a hybrid approach similar to the 2015 one, with different approximations at low and high multipoles, and implementing several methodological and analysis refinements. With more realistic simulations, and better correction and modelling of systematics, we can now make full use of the High Frequency Instrument polarization data. The low… ▽ More This paper describes the 2018 Planck CMB likelihoods, following a hybrid approach similar to the 2015 one, with different approximations at low and high multipoles, and implementing several methodological and analysis refinements. With more realistic simulations, and better correction and modelling of systematics, we can now make full use of the High Frequency Instrument polarization data. The low-multipole 100x143 GHz EE cross-spectrum constrains the reionization optical-depth parameter $τ$ to better than 15% (in combination with with the other low- and high-$\ell$ likelihoods). We also update the 2015 baseline low-$\ell$ joint TEB likelihood based on the Low Frequency Instrument data, which provides a weaker $τ$ constraint. At high multipoles, a better model of the temperature-to-polarization leakage and corrections for the effective calibrations of the polarization channels (polarization efficiency or PE) allow us to fully use the polarization spectra, improving the constraints on the $Λ$CDM parameters by 20 to 30% compared to TT-only constraints. Tests on the modelling of the polarization demonstrate good consistency, with some residual modelling uncertainties, the accuracy of the PE modelling being the main limitation. Using our various tests, simulations, and comparison between different high-$\ell$ implementations, we estimate the consistency of the results to be better than the 0.5$σ$ level. Minor curiosities already present before (differences between $\ell$<800 and $\ell$>800 parameters or the preference for more smoothing of the $C_\ell$ peaks) are shown to be driven by the TT power spectrum and are not significantly modified by the inclusion of polarization. Overall, the legacy Planck CMB likelihoods provide a robust tool for constraining the cosmological model and represent a reference for future CMB observations. (Abridged) △ Less

Submitted 15 September, 2020; v1 submitted 30 July, 2019; originally announced July 2019.

Comments: Revised to match version published in Astronomy & Astrophysics

Journal ref: A&A 641, A5 (2020)

Planck 2018 results. VII. Isotropy and Statistics of the CMB

Authors: Planck Collaboration, Y. Akrami, M. Ashdown, J. Aumont, C. Baccigalupi, M. Ballardini, A. J. Banday, R. B. Barreiro, N. Bartolo, S. Basak, K. Benabed, M. Bersanelli, P. Bielewicz, J. J. Bock, J. R. Bond, J. Borrill, F. R. Bouchet, F. Boulanger, M. Bucher, C. Burigana, R. C. Butler, E. Calabrese, J. -F. Cardoso, B. Casaponsa, H. C. Chiang , et al. (125 additional authors not shown)

Abstract: Analysis of the Planck 2018 data set indicates that the statistical properties of the cosmic microwave background (CMB) temperature anisotropies are in excellent agreement with previous studies using the 2013 and 2015 data releases. In particular, they are consistent with the Gaussian predictions of the $Λ$CDM cosmological model, yet also confirm the presence of several so-called "anomalies" on la… ▽ More Analysis of the Planck 2018 data set indicates that the statistical properties of the cosmic microwave background (CMB) temperature anisotropies are in excellent agreement with previous studies using the 2013 and 2015 data releases. In particular, they are consistent with the Gaussian predictions of the $Λ$CDM cosmological model, yet also confirm the presence of several so-called "anomalies" on large angular scales. The novelty of the current study, however, lies in being a first attempt at a comprehensive analysis of the statistics of the polarization signal over all angular scales, using either maps of the Stokes parameters, $Q$ and $U$, or the $E$-mode signal derived from these using a new methodology (which we describe in an appendix). Although remarkable progress has been made in reducing the systematic effects that contaminated the 2015 polarization maps on large angular scales, it is still the case that residual systematics (and our ability to simulate them) can limit some tests of non-Gaussianity and isotropy. However, a detailed set of null tests applied to the maps indicates that these issues do not dominate the analysis on intermediate and large angular scales (i.e., $\ell \lesssim 400$). In this regime, no unambiguous detections of cosmological non-Gaussianity, or of anomalies corresponding to those seen in temperature, are claimed. Notably, the stacking of CMB polarization signals centred on the positions of temperature hot and cold spots exhibits excellent agreement with the $Λ$CDM cosmological model, and also gives a clear indication of how Planck provides state-of-the-art measurements of CMB temperature and polarization on degree scales. △ Less

Submitted 14 September, 2020; v1 submitted 6 June, 2019; originally announced June 2019.

Comments: Paper VII of the Planck 2018 release, revised to closely match version published in Astronomy and Astrophysics

Journal ref: A&A 641, A7 (2020)

Planck 2018 results. IX. Constraints on primordial non-Gaussianity

Authors: Planck Collaboration, Y. Akrami, F. Arroja, M. Ashdown, J. Aumont, C. Baccigalupi, M. Ballardini, A. J. Banday, R. B. Barreiro, N. Bartolo, S. Basak, K. Benabed, J. -P. Bernard, M. Bersanelli, P. Bielewicz, J. R. Bond, J. Borrill, F. R. Bouchet, M. Bucher, C. Burigana, R. C. Butler, E. Calabrese, J. -F. Cardoso, B. Casaponsa, A. Challinor , et al. (135 additional authors not shown)

Abstract: We analyse the Planck full-mission cosmic microwave background (CMB) temperature and E-mode polarization maps to obtain constraints on primordial non-Gaussianity (NG). We compare estimates obtained from separable template-fitting, binned, and modal bispectrum estimators, finding consistent values for the local, equilateral, and orthogonal bispectrum amplitudes. Our combined temperature and polariz… ▽ More We analyse the Planck full-mission cosmic microwave background (CMB) temperature and E-mode polarization maps to obtain constraints on primordial non-Gaussianity (NG). We compare estimates obtained from separable template-fitting, binned, and modal bispectrum estimators, finding consistent values for the local, equilateral, and orthogonal bispectrum amplitudes. Our combined temperature and polarization analysis produces the following results: f_NL^local = -0.9 +\- 5.1; f_NL^equil = -26 +\- 47; and f_NL^ortho = - 38 +\- 24 (68%CL, statistical). These results include the low-multipole (4 <= l < 40) polarization data, not included in our previous analysis, pass an extensive battery of tests, and are stable with respect to our 2015 measurements. Polarization bispectra display a significant improvement in robustness; they can now be used independently to set NG constraints. We consider a large number of additional cases, e.g. scale-dependent feature and resonance bispectra, isocurvature primordial NG, and parity-breaking models, where we also place tight constraints but do not detect any signal. The non-primordial lensing bispectrum is detected with an improved significance compared to 2015, excluding the null hypothesis at 3.5 sigma. We present model-independent reconstructions and analyses of the CMB bispectrum. Our final constraint on the local trispectrum shape is g_NLl^local = (-5.8 +\-6.5) x 10^4 (68%CL, statistical), while constraints for other trispectra are also determined. We constrain the parameter space of different early-Universe scenarios, including general single-field models of inflation, multi-field and axion field parity-breaking models. Our results provide a high-precision test for structure-formation scenarios, in complete agreement with the basic picture of the LambdaCDM cosmology regarding the statistics of the initial conditions (abridged). △ Less

Submitted 14 May, 2019; originally announced May 2019.

Comments: 50 pages, 20 figures

Cosmic Shear: Inference from Forward Models

Authors: Peter L. Taylor, Thomas D. Kitching, Justin Alsing, Benjamin D. Wandelt, Stephen M. Feeney, Jason D. McEwen

Abstract: Density-estimation likelihood-free inference (DELFI) has recently been proposed as an efficient method for simulation-based cosmological parameter inference. Compared to the standard likelihood-based Markov Chain Monte Carlo (MCMC) approach, DELFI has several advantages: it is highly parallelizable, there is no need to assume a possibly incorrect functional form for the likelihood and complicated… ▽ More Density-estimation likelihood-free inference (DELFI) has recently been proposed as an efficient method for simulation-based cosmological parameter inference. Compared to the standard likelihood-based Markov Chain Monte Carlo (MCMC) approach, DELFI has several advantages: it is highly parallelizable, there is no need to assume a possibly incorrect functional form for the likelihood and complicated effects (e.g the mask and detector systematics) are easier to handle with forward models. In light of this, we present two DELFI pipelines to perform weak lensing parameter inference with lognormal realizations of the tomographic shear field -- using the C_l summary statistic. The first pipeline accounts for the non-Gaussianities of the shear field, intrinsic alignments and photometric-redshift error. We validate that it is accurate enough for Stage III experiments and estimate that O(1000) simulations are needed to perform inference on Stage IV data. By comparing the second DELFI pipeline, which makes no assumption about the functional form of the likelihood, with the standard MCMC approach, which assumes a Gaussian likelihood, we test the impact of the Gaussian likelihood approximation in the MCMC analysis. We find it has a negligible impact on Stage IV parameter constraints. Our pipeline is a step towards seamlessly propagating all data-processing, instrumental, theoretical and astrophysical systematics through to the final parameter constraints. △ Less

Submitted 29 July, 2019; v1 submitted 10 April, 2019; originally announced April 2019.

Comments: Physical Review D. accepted

Journal ref: Phys. Rev. D 100, 023519 (2019)

Load balancing for distributed interferometric image reconstruction

Authors: Luke Pratley, Jason D. McEwen

Abstract: We present a new algorithm to perform wide-field radio interferometric image reconstruction, with exact non-coplanar correction, that scales to big-data. This algorithm allows us to image 2 billion visibilities on 50 nodes of a computing cluster for a 25 by 25 degree field of view, in a little over an hour. We build on the recently developed distributed $w$-stacking $w$-projection hybrid algorithm… ▽ More We present a new algorithm to perform wide-field radio interferometric image reconstruction, with exact non-coplanar correction, that scales to big-data. This algorithm allows us to image 2 billion visibilities on 50 nodes of a computing cluster for a 25 by 25 degree field of view, in a little over an hour. We build on the recently developed distributed $w$-stacking $w$-projection hybrid algorithm, extending it to include a new distributed degridding algorithm that balances the computational load of the $w$-projection gridding kernels. The implementation of our algorithm is made publicly available in the PURIFY software package. Wide-field image reconstruction for data sets of this size cannot be performed effectively using the allocated computational resources without computational load balancing, demonstrating that our algorithms are critical for next-generation wide-field radio interferometers. △ Less

Submitted 18 March, 2019; originally announced March 2019.

Comments: 4 pages, 1 figure

$w$-stacking $w$-projection hybrid algorithm for wide-field interferometric imaging: implementation details and improvements

Authors: L. Pratley, M. Johnston-Hollitt, J. D. McEwen

Abstract: We present a detailed discussion of the implementation strategies for a recently developed $w$-stacking $w$-projection hybrid algorithm used to reconstruct wide-field interferometric images. In particular, we discuss the methodology used to deploy the algorithm efficiently on a supercomputer via use of a Message Passing Interface (MPI) $k$-means clustering technique to achieve efficient constructi… ▽ More We present a detailed discussion of the implementation strategies for a recently developed $w$-stacking $w$-projection hybrid algorithm used to reconstruct wide-field interferometric images. In particular, we discuss the methodology used to deploy the algorithm efficiently on a supercomputer via use of a Message Passing Interface (MPI) $k$-means clustering technique to achieve efficient construction and application of non co-planar effects. Additionally, we show that the use of conjugate symmetry increases the algorithms performance by imaging an interferometric observation of Fornax A from the Murchison Widefield Array (MWA). We perform exact non-coplanar wide-field correction for 126.6 million visibilities using 50 nodes of a computing cluster. The $w$-projection kernel construction takes only 15 minutes, demonstrating that the implementation is both fast and efficient. △ Less

Submitted 13 March, 2019; originally announced March 2019.

Comments: 10 Pages, 2 Figures. arXiv admin note: text overlap with arXiv:1807.09239