-
Rapid Parameter Estimation for Merging Massive Black Hole Binaries Using ODE-Based Generative Models
Authors:
Bo Liang,
Minghui Du,
He Wang,
Yuxiang Xu,
Chang Liu,
Xiaotong Wei,
Peng Xu,
Li-e Qiang,
Ziren Luo
Abstract:
Detecting the coalescences of massive black hole binaries (MBHBs) is one of the primary targets for space-based gravitational wave observatories such as LISA, Taiji, and Tianqin. The fast and accurate parameter estimation of merging MBHBs is of great significance for both astrophysics and the global fitting of all resolvable sources. However, such analyses entail significant computational costs. T…
▽ More
Detecting the coalescences of massive black hole binaries (MBHBs) is one of the primary targets for space-based gravitational wave observatories such as LISA, Taiji, and Tianqin. The fast and accurate parameter estimation of merging MBHBs is of great significance for both astrophysics and the global fitting of all resolvable sources. However, such analyses entail significant computational costs. To address these challenges, inspired by the latest progress in generative models, we proposed a novel artificial intelligence (AI) based parameter estimation method called Variance Preserving Flow Matching Posterior Estimation (VPFMPE). Specifically, we utilize triangular interpolation to maintain variance over time, thereby constructing a transport path for training continuous normalization flows. Compared to the simple linear interpolation method used in flow matching to construct the optimal transport path, our approach better captures continuous temporal variations, making it more suitable for the parameter estimation of MBHBs. Additionally, we creatively introduce a parameter transformation method based on the symmetry in the detector's response function. This transformation is integrated within VPFMPE, allowing us to train the model using a simplified dataset, and then perform parameter estimation on more general data, hence also acting as a crucial factor in improving the training speed. In conclusion, for the first time, within a comprehensive and reasonable parameter range, we have achieved a complete and unbiased 11-dimensional rapid inference for MBHBs in the presence of astrophysical confusion noise using ODE-based generative models. In the experiments based on simulated data, our model produces posterior distributions comparable to those obtained by nested sampling.
△ Less
Submitted 9 July, 2024;
originally announced July 2024.
-
Radio Frequency Interference Detection Using Efficient Multi-Scale Convolutional Attention UNet
Authors:
Fei Gu,
Longfei Hao,
Bo Liang,
Song Feng,
Shoulin Wei,
Wei Dai,
Yonghua Xu,
Zhixuan Li,
Yihang Dao
Abstract:
Studying the universe through radio telescope observation is crucial. However, radio telescopes capture not only signals from the universe but also various interfering signals, known as Radio Frequency Interference (RFI). The presence of RFI can significantly impact data analysis. Ensuring the accuracy, reliability, and scientific integrity of research findings by detecting and mitigating or elimi…
▽ More
Studying the universe through radio telescope observation is crucial. However, radio telescopes capture not only signals from the universe but also various interfering signals, known as Radio Frequency Interference (RFI). The presence of RFI can significantly impact data analysis. Ensuring the accuracy, reliability, and scientific integrity of research findings by detecting and mitigating or eliminating RFI in observational data, presents a persistent challenge in radio astronomy. In this study, we proposed a novel deep learning model called EMSCA-UNet for RFI detection. The model employs multi-scale convolutional operations to extract RFI features of various scale sizes. Additionally, an attention mechanism is utilized to assign different weights to the extracted RFI feature maps, enabling the model to focus on vital features for RFI detection. We evaluated the performance of the model using real data observed from the 40-meter radio telescope at Yunnan Observatory. Furthermore, we compared our results to other models, including U-Net, RFI-Net, and R-Net, using four commonly employed evaluation metrics: precision, recall, F1 score, and IoU. The results demonstrate that our model outperforms the other models on all evaluation metrics, achieving an average improvement of approximately 5\% compared to U-Net. Our model not only enhances the accuracy and comprehensiveness of RFI detection but also provides more detailed edge detection while minimizing the loss of useful signals.
△ Less
Submitted 30 March, 2024;
originally announced April 2024.
-
Gravitational Wave Signal Extraction Against Non-Stationary Instrumental Noises with Deep Neural Network
Authors:
Yuxiang Xu,
Minghui Du,
Peng Xu,
Bo Liang,
He Wang
Abstract:
Sapce-borne gravitational wave antennas, such as LISA and LISA-like mission (Taiji and Tianqin), will offer novel perspectives for exploring our Universe while introduce new challenges, especially in data analysis. Aside from the known challenges like high parameter space dimension, superposition of large number of signals and etc., gravitational wave detections in space would be more seriously af…
▽ More
Sapce-borne gravitational wave antennas, such as LISA and LISA-like mission (Taiji and Tianqin), will offer novel perspectives for exploring our Universe while introduce new challenges, especially in data analysis. Aside from the known challenges like high parameter space dimension, superposition of large number of signals and etc., gravitational wave detections in space would be more seriously affected by anomalies or non-stationarities in the science measurements. Considering the three types of foreseeable non-stationarities including data gaps, transients (glitches), and time-varying noise auto-correlations, which may come from routine maintenance or unexpected disturbances during science operations, we developed a deep learning model for accurate signal extractions confronted with such anomalous scenarios. Our model exhibits the same performance as the current state-of-the-art models do for the ideal and anomaly free scenario, while shows remarkable adaptability in extractions of coalescing massive black hole binary signal against all three types of non-stationarities and even their mixtures. This also provide new explorations into the robustness studies of deep learning models for data processing in space-borne gravitational wave missions.
△ Less
Submitted 29 May, 2024; v1 submitted 20 February, 2024;
originally announced February 2024.
-
Advancing Space-Based Gravitational Wave Astronomy: Rapid Parameter Estimation via Normalizing Flows
Authors:
Minghui Du,
Bo Liang,
He Wang,
Peng Xu,
Ziren Luo,
Yueliang Wu
Abstract:
Gravitational wave (GW) astronomy is witnessing a transformative shift from terrestrial to space-based detection, with missions like Taiji at the forefront. While the transition brings unprecedented opportunities for exploring massive black hole binaries (MBHBs), it also imposes complex challenges in data analysis, particularly in parameter estimation amidst confusion noise. Addressing this gap, w…
▽ More
Gravitational wave (GW) astronomy is witnessing a transformative shift from terrestrial to space-based detection, with missions like Taiji at the forefront. While the transition brings unprecedented opportunities for exploring massive black hole binaries (MBHBs), it also imposes complex challenges in data analysis, particularly in parameter estimation amidst confusion noise. Addressing this gap, we utilize scalable normalizing flow models to achieve rapid and accurate inference within the Taiji environment. Innovatively, our approach simplifies the data's complexity, employs a transformation map** to overcome the year-period time-dependent response function, and unveils additional multimodality in the arrival time parameter. Our method estimates MBHBs several orders of magnitude faster than conventional techniques, maintaining high accuracy even in complex backgrounds. These findings significantly enhance the efficiency of GW data analysis, paving the way for rapid detection and alerting systems and enriching our ability to explore the universe through space-based GW observation.
△ Less
Submitted 20 February, 2024; v1 submitted 10 August, 2023;
originally announced August 2023.
-
Unsupervised Galaxy Morphological Visual Representation with Deep Contrastive Learning
Authors:
Shoulin Wei,
Yadi Li,
Wei Lu,
Nan Li,
Bo Liang,
Wei Dai,
Zhijian Zhang
Abstract:
Galaxy morphology reflects structural properties which contribute to understand the formation and evolution of galaxies. Deep convolutional networks have proven to be very successful in learning hidden features that allow for unprecedented performance on galaxy morphological classification. Such networks mostly follow the supervised learning paradigm which requires sufficient labelled data for tra…
▽ More
Galaxy morphology reflects structural properties which contribute to understand the formation and evolution of galaxies. Deep convolutional networks have proven to be very successful in learning hidden features that allow for unprecedented performance on galaxy morphological classification. Such networks mostly follow the supervised learning paradigm which requires sufficient labelled data for training. However, it is an expensive and complicated process of labeling for million galaxies, particularly for the forthcoming survey projects. In this paper, we present an approach based on contrastive learning with aim for learning galaxy morphological visual representation using only unlabeled data. Considering the properties of low semantic information and contour dominated of galaxy image, the feature extraction layer of the proposed method incorporates vision transformers and convolutional network to provide rich semantic representation via the fusion of the multi-hierarchy features. We train and test our method on 3 classifications of datasets from Galaxy Zoo 2 and SDSS-DR17, and 4 classifications from Galaxy Zoo DECaLS. The testing accuracy achieves 94.7%, 96.5% and 89.9% respectively. The experiment of cross validation demonstrates our model possesses transfer and generalization ability when applied to the new datasets. The code that reveals our proposed method and pretrained models are publicly available and can be easily adapted to new surveys.
△ Less
Submitted 14 November, 2022;
originally announced November 2022.
-
An Empirical Evaluation On the Applicability of the DALiuGE Execution Framework
Authors:
Ying Mei,
Shoulin Wei,
Feng Wang,
Chen Wu,
Rodrigo Tobar,
Mohsim Shaikh,
Hui Deng,
Wei Dai,
Bo Liang,
Andreas Wicenec
Abstract:
The Square Kilometre Array (SKA) project is an international cooperation project to build the largest radio telescope worldwide. Data processing is one of the biggest challenges of building the SKA telescope. As a distributed execution framework, the Data Activated Liu Graph Engine (DALiuGE) was proposed to be one of the candidates for addressing the massive data of the SKA. DALiuGE has many disti…
▽ More
The Square Kilometre Array (SKA) project is an international cooperation project to build the largest radio telescope worldwide. Data processing is one of the biggest challenges of building the SKA telescope. As a distributed execution framework, the Data Activated Liu Graph Engine (DALiuGE) was proposed to be one of the candidates for addressing the massive data of the SKA. DALiuGE has many distinctive features, but its actual ability to handle scientific data is still not evident. In this paper, we perform an objective evaluation of the usability of DALiuGE concerning the execution performance, developer workload, and implementation difficulty of porting the SAGECal to DALiuGE. The evaluation results showed that the DALiuGE enables fast integration of astronomical software, but there are significant differences in the efficiency of different parallel granularities. Even with the deep optimization of the program, there is still a gap between the current DALiuGE and the traditional MPI in execution performance. Therefore, we come to a preliminary conclusion that the DALiuGE has no performance advantage in batch processing of massive data, while it may be more suitable for application scenarios with more customized computational tasks, such as SKA science regional centers.
△ Less
Submitted 24 December, 2021;
originally announced December 2021.
-
Revised $f_{\rm NL}$ parameter in Curvaton Scenario
Authors:
Lei-Hua Liu,
Bin Liang,
Ya-Chen Zhou,
Xiao-Dan Liu,
Wu-Long Xu,
Ai-Chen Li
Abstract:
We revise the Non-Gaussianity of canonical curvaton scenario with a generalized $δN$ formalism, in which it could handle the generic potentials. In various curvaton models, the energy density is dominant in different period including the secondary inflation of curvaton, matter domination and radiation domination. Our method could unify to deal with these periods since the non-linearity parameter…
▽ More
We revise the Non-Gaussianity of canonical curvaton scenario with a generalized $δN$ formalism, in which it could handle the generic potentials. In various curvaton models, the energy density is dominant in different period including the secondary inflation of curvaton, matter domination and radiation domination. Our method could unify to deal with these periods since the non-linearity parameter $f_{\rm NL}$ associated with Non-Gaussianity is a function of equation of state $w$. We firstly investigate the most simple curvaton scenario, namely the chaotic curvaton with quadratic potential. Our study shows that most parameter space satisfies with observational constraints. And our formula will nicely recover the well-known value of $f_{\rm NL}$ in the absence of non-linear evolution. From the micro origin of curvaton, we also investigate the Pseudo-Nambu-Goldstone curvaton. Our result clearly indicates that the second short inflationary process for Pseudo-Nambu-Goldstone curvaton is ruled out in light of observations. Finally, our method sheds a new way for investigating the Non-Gaussianity of curvaton mechanism, espeically for exploring the Non-Gaussianity in MSSM curvaton model.
△ Less
Submitted 15 February, 2021; v1 submitted 16 July, 2020;
originally announced July 2020.
-
Estimating red noise in quasi-periodic signals with MCMC-based Bayesian
Authors:
Bo Liang,
Yao Meng,
Song Feng,
Yunfei Yang
Abstract:
Multi-parameter Bayesian inferences based on Markov chain Monte Carlo (MCMC) samples have been widely used to estimate red noise in solar period-periodic signals. To MCMC, proper priors and sufficient iterations are prerequisites ensuring the accuracy of red noise estimation. We used MCMC-based Bayesian inferences to estimate 100 groups of red noise synthesized randomly for evaluating its accuracy…
▽ More
Multi-parameter Bayesian inferences based on Markov chain Monte Carlo (MCMC) samples have been widely used to estimate red noise in solar period-periodic signals. To MCMC, proper priors and sufficient iterations are prerequisites ensuring the accuracy of red noise estimation. We used MCMC-based Bayesian inferences to estimate 100 groups of red noise synthesized randomly for evaluating its accuracy. At the same time, the Brooks-Gelman algorithm was employed to precisely diagnose the convergence of the Markov chains generated by MCMC. The root-mean-square error of parameter inferences to the synthetic data is only 1.14. Furthermore, we applied the algorithm to analyze the oscillation modes in a sunspot and a flare. A 70 s period is detected in the sunspot umbra in addition to 3- and 5-minute periods, and a 40 s period is detected in the flare. The results prove that estimating red noise with MCMC-based Bayesian has more high accuracy in the case of proper priors and convergence. We also find that the number of iterations increases dramatically to achieve convergence as the number of parameters grows. Therefore, we strongly recommend that when estimating red noise with MCMC-based Bayesian, different initial values must be selected to ensure that the entire posterior distribution is covered.
△ Less
Submitted 22 February, 2020;
originally announced February 2020.
-
OpenCluster: A Flexible Distributed Computing Framework for Astronomical Data Processing
Authors:
Shoulin Wei,
Feng Wang,
Hui Deng,
Cuiyin Liu,
Wei Dai,
Bo Liang,
Ying Mei,
Congming Shi,
Yingbo Liu,
**g** Wu
Abstract:
The volume of data generated by modern astronomical telescopes is extremely large and rapidly growing. However, current high-performance data processing architectures/frameworks are not well suited for astronomers because of their limitations and programming difficulties. In this paper, we therefore present OpenCluster, an open-source distributed computing framework to support rapidly develo** h…
▽ More
The volume of data generated by modern astronomical telescopes is extremely large and rapidly growing. However, current high-performance data processing architectures/frameworks are not well suited for astronomers because of their limitations and programming difficulties. In this paper, we therefore present OpenCluster, an open-source distributed computing framework to support rapidly develo** high-performance processing pipelines of astronomical big data. We first detail the OpenCluster design principles and implementations and present the APIs facilitated by the framework. We then demonstrate a case in which OpenCluster is used to resolve complex data processing problems for develo** a pipeline for the Mingantu Ultrawide Spectral Radioheliograph. Finally, we present our OpenCluster performance evaluation. Overall, OpenCluster provides not only high fault tolerance and simple programming interfaces, but also a flexible means of scaling up the number of interacting entities. OpenCluster thereby provides an easily integrated distributed computing framework for quickly develo** a high-performance data processing system of astronomical telescopes and for significantly reducing software development expenses.
△ Less
Submitted 17 January, 2017;
originally announced January 2017.
-
NVST data archiving system based on fastbit nosql database
Authors:
Yingbo Liu,
Feng Wang,
Kaifan Ji,
Hui Deng,
Wei Dai,
Bo Liang
Abstract:
The New Vacuum Solar Telescope (NVST) is a 1-meter vacuum solar telescope that aims to observe the fine structures of active regions on the Sun. The main tasks of the NVST are high resolution imaging and spectral observations, including the measurements of the solar magnetic field. The NVST has been collecting more than 20 million FITS files since it began routine observations in 2012 and produces…
▽ More
The New Vacuum Solar Telescope (NVST) is a 1-meter vacuum solar telescope that aims to observe the fine structures of active regions on the Sun. The main tasks of the NVST are high resolution imaging and spectral observations, including the measurements of the solar magnetic field. The NVST has been collecting more than 20 million FITS files since it began routine observations in 2012 and produces a maximum observational records of 120 thousand files in a day. Given the large amount of files, the effective archiving and retrieval of files becomes a critical and urgent problem. In this study, we implement a new data archiving system for the NVST based on the Fastbit Not Only Structured Query Language (NoSQL) database. Comparing to the relational database (i.e., MySQL; My Structured Query Language), the Fastbit database manifests distinctive advantages on indexing and querying performance. In a large scale database of 40 million records, the multi-field combined query response time of Fastbit database is about 15 times faster and fully meets the requirements of the NVST. Our study brings a new idea for massive astronomical data archiving and would contribute to the design of data management systems for other astronomical telescopes.
△ Less
Submitted 22 December, 2016;
originally announced December 2016.
-
Low-cost high performance distributed data storage for multi-channel observations
Authors:
Ying-bo Liu,
Feng Wang,
Hui Deng,
Kai-fan Ji,
Wei Dai,
Shou-lin Wei,
Bo Liang,
Xiao-li Zhang
Abstract:
The New Vacuum Solar Telescope (NVST) is a 1-m solar telescope that aims to observe the fine structures in both the photosphere and the chromosphere of the Sun. The observational data acquired simultaneously from one channel for the chromosphere and two channels for the photosphere bring great challenges to the data storage of NVST. The multi-channel instruments of NVST, including scientific camer…
▽ More
The New Vacuum Solar Telescope (NVST) is a 1-m solar telescope that aims to observe the fine structures in both the photosphere and the chromosphere of the Sun. The observational data acquired simultaneously from one channel for the chromosphere and two channels for the photosphere bring great challenges to the data storage of NVST. The multi-channel instruments of NVST, including scientific cameras and multi-band spectrometers, generate at least 3 terabytes data per day and require high access performance while storing massive short-exposure images. It is worth studying and implementing a storage system for NVST which would balance the data availability, access performance and the cost of development. In this paper, we build a distributed data storage system (DDSS) for NVST and then deeply evaluate the availability of real-time data storage on a distributed computing environment. The experimental results show that two factors, i.e., the number of concurrent read/write and the file size, are critically important for improving the performance of data access on a distributed environment. Referring to these two factors, three strategies for storing FITS files are presented and implemented to ensure the access performance of the DDSS under conditions of multi-host write and read simultaneously. The real applications of the DDSS proves that the system is capable of meeting the requirements of NVST real-time high performance observational data storage. Our study on the DDSS is the first attempt for modern astronomical telescope systems to store real-time observational data on a low-cost distributed system. The research results and corresponding techniques of the DDSS provide a new option for designing real-time massive astronomical data storage system and will be a reference for future astronomical data storage.
△ Less
Submitted 22 December, 2016;
originally announced December 2016.
-
Distributed Data-Processing Pipeline for Mingantu Ultrawide Spectral Radioheliograph
Authors:
F. Wang,
Y. Mei,
H. Deng,
C. Y. Liu,
D. H. Liu,
S. L. Wei,
W. Dai,
B. Liang,
Y. B. Liu,
X. L. Zhang,
K. F. Ji
Abstract:
The Chinese Spectral RadioHeliograph (CSRH) is a synthetic aperture radio interferometer built in Inner Mongolia, China. As a solar-dedicated interferometric array, CSRH is capable of producing high quality radio images at frequency range from 400 MHz to 15 GHz with high temporal, spatial, and spectral resolution.To implement high cadence imaging at wide-band and obtain more than 2 order higher mu…
▽ More
The Chinese Spectral RadioHeliograph (CSRH) is a synthetic aperture radio interferometer built in Inner Mongolia, China. As a solar-dedicated interferometric array, CSRH is capable of producing high quality radio images at frequency range from 400 MHz to 15 GHz with high temporal, spatial, and spectral resolution.To implement high cadence imaging at wide-band and obtain more than 2 order higher multiple frequencies, the implementation of the data processing system for CSRH is a great challenge. It is urgent to build a pipeline for processing massive data of CSRH generated every day. In this paper, we develop a high performance distributed data processing pipeline (DDPP) built on the OpenCluster infrastructure for processing CSRH observational data including data storage, archiving, preprocessing, image reconstruction, deconvolution, and real-time monitoring. We comprehensively elaborate the system architecture of the pipeline and the implementation of each subsystem. The DDPP is automatic, robust, scalable and manageable. The processing performance under multi computers parallel and GPU hybrid system meets the requirements of CSRH data processing. The study presents an valuable reference for other radio telescopes especially aperture synthesis telescopes, and also gives an valuable contribution to the current and/or future data intensive astronomical observations.
△ Less
Submitted 20 December, 2016;
originally announced December 2016.