Showing 1–50 of 105 results for author: Lam, T L

Vision-Language Model-based Physical Reasoning for Robot Liquid Perception

Authors: Wenqiang Lai, Yuan Gao, Tin Lun Lam

Abstract: There is a growing interest in applying large language models (LLMs) in robotic tasks, due to their remarkable reasoning ability and extensive knowledge learned from vast training corpora. Grounding LLMs in the physical world remains an open challenge as they can only process textual input. Recent advancements in large vision-language models (LVLMs) have enabled a more comprehensive understanding… ▽ More There is a growing interest in applying large language models (LLMs) in robotic tasks, due to their remarkable reasoning ability and extensive knowledge learned from vast training corpora. Grounding LLMs in the physical world remains an open challenge as they can only process textual input. Recent advancements in large vision-language models (LVLMs) have enabled a more comprehensive understanding of the physical world by incorporating visual input, which provides richer contextual information than language alone. In this work, we proposed a novel paradigm that leveraged GPT-4V(ision), the state-of-the-art LVLM by OpenAI, to enable embodied agents to perceive liquid objects via image-based environmental feedback. Specifically, we exploited the physical understanding of GPT-4V to interpret the visual representation (e.g., time-series plot) of non-visual feedback (e.g., F/T sensor data), indirectly enabling multimodal perception beyond vision and language using images as proxies. We evaluated our method using 10 common household liquids with containers of various geometry and material. Without any training or fine-tuning, we demonstrated that our method can enable the robot to indirectly perceive the physical response of liquids and estimate their viscosity. We also showed that by jointly reasoning over the visual and physical attributes learned through interactions, our method could recognize liquid objects in the absence of strong visual cues (e.g., container labels with legible text or symbols), increasing the accuracy from 69.0% -- achieved by the best-performing vision-only variant -- to 86.0%. △ Less

Submitted 10 April, 2024; originally announced April 2024.

Comments: 8 pages, 6 figures, submitted to IROS 2024

Class Relevance Learning For Out-of-distribution Detection

Authors: Butian Xiong, Liguang Zhou, Tin Lun Lam, Yangsheng Xu

Abstract: Image classification plays a pivotal role across diverse applications, yet challenges persist when models are deployed in real-world scenarios. Notably, these models falter in detecting unfamiliar classes that were not incorporated during classifier training, a formidable hurdle for safe and effective real-world model deployment, commonly known as out-of-distribution (OOD) detection. While existin… ▽ More Image classification plays a pivotal role across diverse applications, yet challenges persist when models are deployed in real-world scenarios. Notably, these models falter in detecting unfamiliar classes that were not incorporated during classifier training, a formidable hurdle for safe and effective real-world model deployment, commonly known as out-of-distribution (OOD) detection. While existing techniques, like max logits, aim to leverage logits for OOD identification, they often disregard the intricate interclass relationships that underlie effective detection. This paper presents an innovative class relevance learning method tailored for OOD detection. Our method establishes a comprehensive class relevance learning framework, strategically harnessing interclass relationships within the OOD pipeline. This framework significantly augments OOD detection capabilities. Extensive experimentation on diverse datasets, encompassing generic image classification datasets (Near OOD and Far OOD datasets), demonstrates the superiority of our method over state-of-the-art alternatives for OOD detection. △ Less

Submitted 21 September, 2023; originally announced January 2024.

Decoding Modular Reconfigurable Robots: A Survey on Mechanisms and Design

Authors: Guanqi Liang, Di Wu, Yuxiao Tu, Tin Lun Lam

Abstract: The intrinsic modularity and reconfigurability of modular reconfigurable robots (MRR) confer advantages such as versatility, fault tolerance, and economic efficacy, thereby showcasing considerable potential across diverse applications. The continuous evolution of the technology landscape and the emergence of diverse conceptual designs have generated multiple MRR categories, each described by its r… ▽ More The intrinsic modularity and reconfigurability of modular reconfigurable robots (MRR) confer advantages such as versatility, fault tolerance, and economic efficacy, thereby showcasing considerable potential across diverse applications. The continuous evolution of the technology landscape and the emergence of diverse conceptual designs have generated multiple MRR categories, each described by its respective morphology or capability characteristics, leading to some ambiguity in the taxonomy. This paper conducts a comprehensive survey encompassing the entirety of MRR hardware and design, spanning from the inception in 1985 to 2023. This paper introduces an innovative, unified conceptual framework for understanding MRR hardware, which encompasses three pivotal elements: connectors, actuators, and homogeneity. Through the utilization of this trilateral framework, this paper provide an intuitive understanding of the diverse spectrum of MRR hardware iterations while systematically deciphering and classifying the entire range, offering a more structured perspective. This survey elucidates the fundamental attributes characterizing MRRs and their compositional aspects, providinig insights into their design, technology, functionality, and categorization. Augmented by the proposed trilateral framework, this paper also elaborates on the trajectory of evolution, prevailing trends, principal challenges, and potential prospects within the field of MRRs. △ Less

Submitted 15 October, 2023; originally announced October 2023.

A Joint Fermi-GBM and Swift-BAT Analysis of Gravitational-Wave Candidates from the Third Gravitational-wave Observing Run

Authors: C. Fletcher, J. Wood, R. Hamburg, P. Veres, C. M. Hui, E. Bissaldi, M. S. Briggs, E. Burns, W. H. Cleveland, M. M. Giles, A. Goldstein, B. A. Hristov, D. Kocevski, S. Lesage, B. Mailyan, C. Malacaria, S. Poolakkil, A. von Kienlin, C. A. Wilson-Hodge, The Fermi Gamma-ray Burst Monitor Team, M. Crnogorčević, J. DeLaunay, A. Tohuvavohu, R. Caputo, S. B. Cenko , et al. (1674 additional authors not shown)

Abstract: We present Fermi Gamma-ray Burst Monitor (Fermi-GBM) and Swift Burst Alert Telescope (Swift-BAT) searches for gamma-ray/X-ray counterparts to gravitational wave (GW) candidate events identified during the third observing run of the Advanced LIGO and Advanced Virgo detectors. Using Fermi-GBM on-board triggers and sub-threshold gamma-ray burst (GRB) candidates found in the Fermi-GBM ground analyses,… ▽ More We present Fermi Gamma-ray Burst Monitor (Fermi-GBM) and Swift Burst Alert Telescope (Swift-BAT) searches for gamma-ray/X-ray counterparts to gravitational wave (GW) candidate events identified during the third observing run of the Advanced LIGO and Advanced Virgo detectors. Using Fermi-GBM on-board triggers and sub-threshold gamma-ray burst (GRB) candidates found in the Fermi-GBM ground analyses, the Targeted Search and the Untargeted Search, we investigate whether there are any coincident GRBs associated with the GWs. We also search the Swift-BAT rate data around the GW times to determine whether a GRB counterpart is present. No counterparts are found. Using both the Fermi-GBM Targeted Search and the Swift-BAT search, we calculate flux upper limits and present joint upper limits on the gamma-ray luminosity of each GW. Given these limits, we constrain theoretical models for the emission of gamma-rays from binary black hole mergers. △ Less

Submitted 25 August, 2023; originally announced August 2023.

Explicit Attention-Enhanced Fusion for RGB-Thermal Perception Tasks

Authors: Mingjian Liang, Junjie Hu, Chenyu Bao, Hua Feng, Fuqin Deng, Tin Lun Lam

Abstract: Recently, RGB-Thermal based perception has shown significant advances. Thermal information provides useful clues when visual cameras suffer from poor lighting conditions, such as low light and fog. However, how to effectively fuse RGB images and thermal data remains an open challenge. Previous works involve naive fusion strategies such as merging them at the input, concatenating multi-modality fea… ▽ More Recently, RGB-Thermal based perception has shown significant advances. Thermal information provides useful clues when visual cameras suffer from poor lighting conditions, such as low light and fog. However, how to effectively fuse RGB images and thermal data remains an open challenge. Previous works involve naive fusion strategies such as merging them at the input, concatenating multi-modality features inside models, or applying attention to each data modality. These fusion strategies are straightforward yet insufficient. In this paper, we propose a novel fusion method named Explicit Attention-Enhanced Fusion (EAEF) that fully takes advantage of each type of data. Specifically, we consider the following cases: i) both RGB data and thermal data, ii) only one of the types of data, and iii) none of them generate discriminative features. EAEF uses one branch to enhance feature extraction for i) and iii) and the other branch to remedy insufficient representations for ii). The outputs of two branches are fused to form complementary features. As a result, the proposed fusion method outperforms state-of-the-art by 1.6\% in mIoU on semantic segmentation, 3.1\% in MAE on salient object detection, 2.3\% in mAP on object detection, and 8.1\% in MAE on crowd counting. The code is available at https://github.com/FreeformRobotics/EAEFNet. △ Less

Submitted 27 March, 2023; originally announced March 2023.

RGB-D-Inertial SLAM in Indoor Dynamic Environments with Long-term Large Occlusion

Authors: Ran Long, Christian Rauch, Vladimir Ivan, Tin Lun Lam, Sethu Vijayakumar

Abstract: This work presents a novel RGB-D-inertial dynamic SLAM method that can enable accurate localisation when the majority of the camera view is occluded by multiple dynamic objects over a long period of time. Most dynamic SLAM approaches either remove dynamic objects as outliers when they account for a minor proportion of the visual input, or detect dynamic objects using semantic segmentation before c… ▽ More This work presents a novel RGB-D-inertial dynamic SLAM method that can enable accurate localisation when the majority of the camera view is occluded by multiple dynamic objects over a long period of time. Most dynamic SLAM approaches either remove dynamic objects as outliers when they account for a minor proportion of the visual input, or detect dynamic objects using semantic segmentation before camera tracking. Therefore, dynamic objects that cause large occlusions are difficult to detect without prior information. The remaining visual information from the static background is also not enough to support localisation when large occlusion lasts for a long period. To overcome these problems, our framework presents a robust visual-inertial bundle adjustment that simultaneously tracks camera, estimates cluster-wise dense segmentation of dynamic objects and maintains a static sparse map by combining dense and sparse features. The experiment results demonstrate that our method achieves promising localisation and object segmentation performance compared to other state-of-the-art methods in the scenario of long-term large occlusion. △ Less

Submitted 23 March, 2023; originally announced March 2023.

Comments: 8 pages, 7 figures

Lifelong-MonoDepth: Lifelong Learning for Multi-Domain Monocular Metric Depth Estimation

Authors: Junjie Hu, Chenyou Fan, Liguang Zhou, Qing Gao, Honghai Liu, Tin Lun Lam

Abstract: With the rapid advancements in autonomous driving and robot navigation, there is a growing demand for lifelong learning models capable of estimating metric (absolute) depth. Lifelong learning approaches potentially offer significant cost savings in terms of model training, data storage, and collection. However, the quality of RGB images and depth maps is sensor-dependent, and depth maps in the rea… ▽ More With the rapid advancements in autonomous driving and robot navigation, there is a growing demand for lifelong learning models capable of estimating metric (absolute) depth. Lifelong learning approaches potentially offer significant cost savings in terms of model training, data storage, and collection. However, the quality of RGB images and depth maps is sensor-dependent, and depth maps in the real world exhibit domain-specific characteristics, leading to variations in depth ranges. These challenges limit existing methods to lifelong learning scenarios with small domain gaps and relative depth map estimation. To facilitate lifelong metric depth learning, we identify three crucial technical challenges that require attention: i) develo** a model capable of addressing the depth scale variation through scale-aware depth learning, ii) devising an effective learning strategy to handle significant domain gaps, and iii) creating an automated solution for domain-aware depth inference in practical applications. Based on the aforementioned considerations, in this paper, we present i) a lightweight multi-head framework that effectively tackles the depth scale imbalance, ii) an uncertainty-aware lifelong learning solution that adeptly handles significant domain gaps, and iii) an online domain-specific predictor selection method for real-time inference. Through extensive numerical studies, we show that the proposed method can achieve good efficiency, stability, and plasticity, leading the benchmarks by 8% to 15%. △ Less

Submitted 12 October, 2023; v1 submitted 9 March, 2023; originally announced March 2023.

Peer Learning for Unbiased Scene Graph Generation

Authors: Liguang Zhou, Junjie Hu, Yuhongze Zhou, Tin Lun Lam, Yangsheng Xu

Abstract: Unbiased scene graph generation (USGG) is a challenging task that requires predicting diverse and heavily imbalanced predicates between objects in an image. To address this, we propose a novel framework peer learning that uses predicate sampling and consensus voting (PSCV) to encourage multiple peers to learn from each other. Predicate sampling divides the predicate classes into sub-distributions… ▽ More Unbiased scene graph generation (USGG) is a challenging task that requires predicting diverse and heavily imbalanced predicates between objects in an image. To address this, we propose a novel framework peer learning that uses predicate sampling and consensus voting (PSCV) to encourage multiple peers to learn from each other. Predicate sampling divides the predicate classes into sub-distributions based on frequency, and assigns different peers to handle each sub-distribution or combinations of them. Consensus voting ensembles the peers' complementary predicate knowledge by emphasizing majority opinion and diminishing minority opinion. Experiments on Visual Genome show that PSCV outperforms previous methods and achieves a new state-of-the-art on SGCls task with 31.6 mean. △ Less

Submitted 4 March, 2023; v1 submitted 31 December, 2022; originally announced January 2023.

Attentional Graph Convolutional Network for Structure-aware Audio-Visual Scene Classification

Authors: Liguang Zhou, Yuhongze Zhou, Xiaonan Qi, Junjie Hu, Tin Lun Lam, Yangsheng Xu

Abstract: Audio-Visual scene understanding is a challenging problem due to the unstructured spatial-temporal relations that exist in the audio signals and spatial layouts of different objects and various texture patterns in the visual images. Recently, many studies have focused on abstracting features from convolutional neural networks while the learning of explicit semantically relevant frames of sound sig… ▽ More Audio-Visual scene understanding is a challenging problem due to the unstructured spatial-temporal relations that exist in the audio signals and spatial layouts of different objects and various texture patterns in the visual images. Recently, many studies have focused on abstracting features from convolutional neural networks while the learning of explicit semantically relevant frames of sound signals and visual images has been overlooked. To this end, we present an end-to-end framework, namely attentional graph convolutional network (AGCN), for structure-aware audio-visual scene representation. First, the spectrogram of sound and input image is processed by a backbone network for feature extraction. Then, to build multi-scale hierarchical information of input features, we utilize an attention fusion mechanism to aggregate features from multiple layers of the backbone network. Notably, to well represent the salient regions and contextual information of audio-visual inputs, the salient acoustic graph (SAG) and contextual acoustic graph (CAG), salient visual graph (SVG), and contextual visual graph (CVG) are constructed for the audio-visual scene representation. Finally, the constructed graphs pass through a graph convolutional network for structure-aware audio-visual scene recognition. Extensive experimental results on the audio, visual and audio-visual scene recognition datasets show that promising results have been achieved by the AGCN methods. Visualizing graphs on the spectrograms and images have been presented to show the effectiveness of proposed CAG/SAG and CVG/SVG that could focus on the salient and semantic relevant regions. △ Less

Submitted 31 December, 2022; originally announced January 2023.

Numerical-relativity simulation for tidal disruption of white dwarfs by a supermassive black hole

Authors: Alan Tsz Lok Lam, Masaru Shibata, Kenta Kiuchi

Abstract: We study tidal disruption of white dwarfs in elliptic orbits with the eccenticity of $\sim 1/3$--$2/3$ by a non-spinning supermassive black hole of mass $M_{\rm BH}=10^5M_\odot$ in fully general relativistic simulations targeting the extreme mass-ratio inspiral leading eventually to tidal disruption. Numerical-relativity simulations are performed by employing a suitable formulation in which the we… ▽ More We study tidal disruption of white dwarfs in elliptic orbits with the eccenticity of $\sim 1/3$--$2/3$ by a non-spinning supermassive black hole of mass $M_{\rm BH}=10^5M_\odot$ in fully general relativistic simulations targeting the extreme mass-ratio inspiral leading eventually to tidal disruption. Numerical-relativity simulations are performed by employing a suitable formulation in which the weak self-gravity of white dwarfs is accurately solved. We reconfirm that tidal disruption occurs for white dwarfs of the typical mass of $\sim 0.6M_\odot$ and radius $\approx 1.2 \times 10^4$\,km near the marginally bound orbit around a non-spinning black hole with $M_{\rm BH}\alt 4\times 10^5M_\odot$. △ Less

Submitted 3 April, 2023; v1 submitted 21 December, 2022; originally announced December 2022.

Comments: Accepted for publication in PRD

TMSTC*: A Turn-minimizing Algorithm For Multi-robot Coverage Path Planning

Authors: Junjie Lu, Bi Zeng, **gtao Tang, Tin Lun Lam

Abstract: Coverage path planning is a major application for mobile robots, which requires robots to move along a planned path to cover the entire map. For large-scale tasks, coverage path planning benefits greatly from multiple robots. In this paper, we describe Turn-minimizing Multirobot Spanning Tree Coverage Star(TMSTC*), an improved multirobot coverage path planning (mCPP) algorithm based on the MSTC*.… ▽ More Coverage path planning is a major application for mobile robots, which requires robots to move along a planned path to cover the entire map. For large-scale tasks, coverage path planning benefits greatly from multiple robots. In this paper, we describe Turn-minimizing Multirobot Spanning Tree Coverage Star(TMSTC*), an improved multirobot coverage path planning (mCPP) algorithm based on the MSTC*. Our algorithm partitions the map into minimum bricks as tree's branches and thereby transforms the problem into finding the maximum independent set of bipartite graph. We then connect bricks with greedy strategy to form a tree, aiming to reduce the number of turns of corresponding circumnavigating coverage path. Our experimental results show that our approach enables multiple robots to make fewer turns and thus complete terrain coverage tasks faster than other popular algorithms. △ Less

Submitted 5 December, 2022; originally announced December 2022.

Comments: 8 pages, 9 figures, submitted to RA-L

Search for gravitational-wave transients associated with magnetar bursts in Advanced LIGO and Advanced Virgo data from the third observing run

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, H. Abe, F. Acernese, K. Ackley, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, R. A. Alfaidi, A. Allocca, P. A. Altin , et al. (1645 additional authors not shown)

Abstract: Gravitational waves are expected to be produced from neutron star oscillations associated with magnetar giant flares and short bursts. We present the results of a search for short-duration (milliseconds to seconds) and long-duration ($\sim$ 100 s) transient gravitational waves from 13 magnetar short bursts observed during Advanced LIGO, Advanced Virgo and KAGRA's third observation run. These 13 bu… ▽ More Gravitational waves are expected to be produced from neutron star oscillations associated with magnetar giant flares and short bursts. We present the results of a search for short-duration (milliseconds to seconds) and long-duration ($\sim$ 100 s) transient gravitational waves from 13 magnetar short bursts observed during Advanced LIGO, Advanced Virgo and KAGRA's third observation run. These 13 bursts come from two magnetars, SGR 1935$+$2154 and Swift J1818.0$-$1607. We also include three other electromagnetic burst events detected by Fermi GBM which were identified as likely coming from one or more magnetars, but they have no association with a known magnetar. No magnetar giant flares were detected during the analysis period. We find no evidence of gravitational waves associated with any of these 16 bursts. We place upper bounds on the root-sum-square of the integrated gravitational-wave strain that reach $2.2 \times 10^{-23}$ $/\sqrt{\text{Hz}}$ at 100 Hz for the short-duration search and $8.7 \times 10^{-23}$ $/\sqrt{\text{Hz}}$ at $450$ Hz for the long-duration search, given a detection efficiency of 50%. For a ringdown signal at 1590 Hz targeted by the short-duration search the limit is set to $1.8 \times 10^{-22}$ $/\sqrt{\text{Hz}}$. Using the estimated distance to each magnetar, we derive upper bounds on the emitted gravitational-wave energy of $3.2 \times 10^{43}$ erg ($7.3 \times 10^{43}$ erg) for SGR 1935$+$2154 and $8.2 \times 10^{42}$ erg ($2.8 \times 10^{43}$ erg) for Swift J1818.0$-$1607, for the short-duration (long-duration) search. Assuming isotropic emission of electromagnetic radiation of the burst fluences, we constrain the ratio of gravitational-wave energy to electromagnetic energy for bursts from SGR 1935$+$2154 with available fluence information. The lowest of these ratios is $3 \times 10^3$. △ Less

Submitted 19 October, 2022; originally announced October 2022.

Comments: 30 pages with appendices, 5 figures, 10 tables

Report number: LIGO-P2100387

MultiRoboLearn: An open-source Framework for Multi-robot Deep Reinforcement Learning

Authors: Junfeng Chen, Fuqin Deng, Yuan Gao, Junjie Hu, Xiyue Guo, Guanqi Liang, Tin Lun Lam

Abstract: It is well known that it is difficult to have a reliable and robust framework to link multi-agent deep reinforcement learning algorithms with practical multi-robot applications. To fill this gap, we propose and build an open-source framework for multi-robot systems called MultiRoboLearn1. This framework builds a unified setup of simulation and real-world applications. It aims to provide standard,… ▽ More It is well known that it is difficult to have a reliable and robust framework to link multi-agent deep reinforcement learning algorithms with practical multi-robot applications. To fill this gap, we propose and build an open-source framework for multi-robot systems called MultiRoboLearn1. This framework builds a unified setup of simulation and real-world applications. It aims to provide standard, easy-to-use simulated scenarios that can also be easily deployed to real-world multi-robot environments. Also, the framework provides researchers with a benchmark system for comparing the performance of different reinforcement learning algorithms. We demonstrate the generality, scalability, and capability of the framework with two real-world scenarios2 using different types of multi-agent deep reinforcement learning algorithms in discrete and continuous action spaces. △ Less

Submitted 27 September, 2022; originally announced September 2022.

Comments: 6pages, 6 figures

Progressive Self-Distillation for Ground-to-Aerial Perception Knowledge Transfer

Authors: Junjie Hu, Chenyou Fan, Mete Ozay, Hua Feng, Yuan Gao, Tin Lun Lam

Abstract: We study a practical yet hasn't been explored problem: how a drone can perceive in an environment from different flight heights. Unlike autonomous driving, where the perception is always conducted from a ground viewpoint, a flying drone may flexibly change its flight height due to specific tasks, requiring the capability for viewpoint invariant perception. Tackling the such problem with supervised… ▽ More We study a practical yet hasn't been explored problem: how a drone can perceive in an environment from different flight heights. Unlike autonomous driving, where the perception is always conducted from a ground viewpoint, a flying drone may flexibly change its flight height due to specific tasks, requiring the capability for viewpoint invariant perception. Tackling the such problem with supervised learning will incur tremendous costs for data annotation of different flying heights. On the other hand, current semi-supervised learning methods are not effective under viewpoint differences. In this paper, we introduce the ground-to-aerial perception knowledge transfer and propose a progressive semi-supervised learning framework that enables drone perception using only labeled data of ground viewpoint and unlabeled data of flying viewpoints. Our framework has four core components: i) a dense viewpoint sampling strategy that splits the range of vertical flight height into a set of small pieces with evenly-distributed intervals, ii) nearest neighbor pseudo-labeling that infers labels of the nearest neighbor viewpoint with a model learned on the preceding viewpoint, iii) MixView that generates augmented images among different viewpoints to alleviate viewpoint differences, and iv) a progressive distillation strategy to gradually learn until reaching the maximum flying height. We collect a synthesized and a real-world dataset, and we perform extensive experimental analyses to show that our method yields 22.2% and 16.9% accuracy improvement for the synthesized dataset and the real world. Code and datasets are available on https://github.com/FreeformRobotics/Progressive-Self-Distillation-for-Ground-to-Aerial-Perception-Knowledge-Transfer. △ Less

Submitted 16 April, 2023; v1 submitted 29 August, 2022; originally announced August 2022.

Dense Depth Distillation with Out-of-Distribution Simulated Images

Authors: Junjie Hu, Chenyou Fan, Mete Ozay, Hualie Jiang, Tin Lun Lam

Abstract: We study data-free knowledge distillation (KD) for monocular depth estimation (MDE), which learns a lightweight model for real-world depth perception tasks by compressing it from a trained teacher model while lacking training data in the target domain. Owing to the essential difference between image classification and dense regression, previous methods of data-free KD are not applicable to MDE. To… ▽ More We study data-free knowledge distillation (KD) for monocular depth estimation (MDE), which learns a lightweight model for real-world depth perception tasks by compressing it from a trained teacher model while lacking training data in the target domain. Owing to the essential difference between image classification and dense regression, previous methods of data-free KD are not applicable to MDE. To strengthen its applicability in real-world tasks, in this paper, we propose to apply KD with out-of-distribution simulated images. The major challenges to be resolved are i) lacking prior information about scene configurations of real-world training data and ii) domain shift between simulated and real-world images. To cope with these difficulties, we propose a tailored framework for depth distillation. The framework generates new training samples for embracing a multitude of possible object arrangements in the target domain and utilizes a transformation network to efficiently adapt them to the feature statistics preserved in the teacher model. Through extensive experiments on various depth estimation models and two different datasets, we show that our method outperforms the baseline KD by a good margin and even achieves slightly better performance with as few as 1/6 of training images, demonstrating a clear superiority. △ Less

Submitted 7 December, 2023; v1 submitted 26 August, 2022; originally announced August 2022.

Context-aware Mixture-of-Experts for Unbiased Scene Graph Generation

Authors: Liguang Zhou, Yuhongze Zhou, Tin Lun Lam, Yangsheng Xu

Abstract: Scene graph generation (SGG) has gained tremendous progress in recent years. However, its underlying long-tailed distribution of predicate classes is a challenging problem. For extremely unbalanced predicate distributions, existing approaches usually construct complicated context encoders to extract the intrinsic relevance of scene context to predicates and complex networks to improve the learning… ▽ More Scene graph generation (SGG) has gained tremendous progress in recent years. However, its underlying long-tailed distribution of predicate classes is a challenging problem. For extremely unbalanced predicate distributions, existing approaches usually construct complicated context encoders to extract the intrinsic relevance of scene context to predicates and complex networks to improve the learning ability of network models for highly imbalanced predicate distributions. To address the unbiased SGG problem, we introduce a simple yet effective method dubbed Context-Aware Mixture-of-Experts (CAME) to improve model diversity and mitigate biased SGG without complicated design. Specifically, we propose to integrate the mixture of experts with a divide and ensemble strategy to remedy the severely long-tailed distribution of predicate classes, which is applicable to the majority of unbiased scene graph generators. The biased SGG is thereby reduced, and the model tends to anticipate more evenly distributed predicate predictions. To differentiate between various predicate distribution levels, experts with the same weights are not sufficiently diverse. In order to enable the network dynamically exploit the rich scene context and further boost the diversity of model, we simply use the built-in module to create a context encoder. The importance of each expert to scene context and each predicate to each expert is dynamically associated with expert weighting (EW) and predicate weighting (PW) strategy. We have conducted extensive experiments on three tasks using the Visual Genome dataset, showing that CAME outperforms recent methods and achieves state-of-the-art performance. Our code will be available publicly. △ Less

Submitted 1 January, 2023; v1 submitted 15 August, 2022; originally announced August 2022.

Learning to Coordinate for a Worker-Station Multi-robot System in Planar Coverage Tasks

Authors: **gtao Tang, Yuan Gao, Tin Lun Lam

Abstract: For massive large-scale tasks, a multi-robot system (MRS) can effectively improve efficiency by utilizing each robot's different capabilities, mobility, and functionality. In this paper, we focus on the multi-robot coverage path planning (mCPP) problem in large-scale planar areas with random dynamic interferers in the environment, where the robots have limited resources. We introduce a worker-stat… ▽ More For massive large-scale tasks, a multi-robot system (MRS) can effectively improve efficiency by utilizing each robot's different capabilities, mobility, and functionality. In this paper, we focus on the multi-robot coverage path planning (mCPP) problem in large-scale planar areas with random dynamic interferers in the environment, where the robots have limited resources. We introduce a worker-station MRS consisting of multiple workers with limited resources for actual work, and one station with enough resources for resource replenishment. We aim to solve the mCPP problem for the worker-station MRS by formulating it as a fully cooperative multi-agent reinforcement learning problem. Then we propose an end-to-end decentralized online planning method, which simultaneously solves coverage planning for workers and rendezvous planning for station. Our method manages to reduce the influence of random dynamic interferers on planning, while the robots can avoid collisions with them. We conduct simulation and real robot experiments, and the comparison results show that our method has competitive performance in solving the mCPP problem for worker-station MRS in metric of task finish time. △ Less

Submitted 24 August, 2022; v1 submitted 5 August, 2022; originally announced August 2022.

Journal ref: IEEE Robotics and Automation Letters, 7(4), 12315-12322, 2022

Feature Pyramid Attention based Residual Neural Network for Environmental Sound Classification

Authors: Liguang Zhou, Yuhongze Zhou, Xiaonan Qi, Junjie Hu, Tin Lun Lam, Yangsheng Xu

Abstract: Environmental sound classification (ESC) is a challenging problem due to the unstructured spatial-temporal relations that exist in the sound signals. Recently, many studies have focused on abstracting features from convolutional neural networks while the learning of semantically relevant frames of sound signals has been overlooked. To this end, we present an end-to-end framework, namely feature py… ▽ More Environmental sound classification (ESC) is a challenging problem due to the unstructured spatial-temporal relations that exist in the sound signals. Recently, many studies have focused on abstracting features from convolutional neural networks while the learning of semantically relevant frames of sound signals has been overlooked. To this end, we present an end-to-end framework, namely feature pyramid attention network (FPAM), focusing on abstracting the semantically relevant features for ESC. We first extract the feature maps of the preprocessed spectrogram of the sound waveform by a backbone network. Then, to build multi-scale hierarchical features of sound spectrograms, we construct a feature pyramid representation of the sound spectrograms by aggregating the feature maps from multi-scale layers, where the temporal frames and spatial locations of semantically relevant frames are localized by FPAM. Specifically, the multiple features are first processed by a dimension alignment module. Afterward, the pyramid spatial attention module (PSA) is attached to localize the important frequency regions spatially with a spatial attention module (SAM). Last, the processed feature maps are refined by a pyramid channel attention (PCA) to localize the important temporal frames. To justify the effectiveness of the proposed FPAM, visualization of attention maps on the spectrograms has been presented. The visualization results show that FPAM can focus more on the semantic relevant regions while neglecting the noises. The effectiveness of the proposed methods is validated on two widely used ESC datasets: the ESC-50 and ESC-10 datasets. The experimental results show that the FPAM yields comparable performance to state-of-the-art methods. A substantial performance increase has been achieved by FPAM compared with the baseline methods. △ Less

Submitted 28 May, 2022; originally announced May 2022.

Deep Depth Completion from Extremely Sparse Data: A Survey

Authors: Junjie Hu, Chenyu Bao, Mete Ozay, Chenyou Fan, Qing Gao, Honghai Liu, Tin Lun Lam

Abstract: Depth completion aims at predicting dense pixel-wise depth from an extremely sparse map captured from a depth sensor, e.g., LiDARs. It plays an essential role in various applications such as autonomous driving, 3D reconstruction, augmented reality, and robot navigation. Recent successes on the task have been demonstrated and dominated by deep learning based solutions. In this article, for the firs… ▽ More Depth completion aims at predicting dense pixel-wise depth from an extremely sparse map captured from a depth sensor, e.g., LiDARs. It plays an essential role in various applications such as autonomous driving, 3D reconstruction, augmented reality, and robot navigation. Recent successes on the task have been demonstrated and dominated by deep learning based solutions. In this article, for the first time, we provide a comprehensive literature review that helps readers better grasp the research trends and clearly understand the current advances. We investigate the related studies from the design aspects of network architectures, loss functions, benchmark datasets, and learning strategies with a proposal of a novel taxonomy that categorizes existing methods. Besides, we present a quantitative comparison of model performance on three widely used benchmarks, including indoor and outdoor datasets. Finally, we discuss the challenges of prior works and provide readers with some insights for future research directions. △ Less

Submitted 29 August, 2022; v1 submitted 11 May, 2022; originally announced May 2022.

Search for continuous gravitational wave emission from the Milky Way center in O3 LIGO--Virgo data

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, H. Abe, F. Acernese, K. Ackley, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, R. A. Alfaidi, A. Allocca, P. A. Altin , et al. (1645 additional authors not shown)

Abstract: We present a directed search for continuous gravitational wave (CW) signals emitted by spinning neutron stars located in the inner parsecs of the Galactic Center (GC). Compelling evidence for the presence of a numerous population of neutron stars has been reported in the literature, turning this region into a very interesting place to look for CWs. In this search, data from the full O3 LIGO--Virgo… ▽ More We present a directed search for continuous gravitational wave (CW) signals emitted by spinning neutron stars located in the inner parsecs of the Galactic Center (GC). Compelling evidence for the presence of a numerous population of neutron stars has been reported in the literature, turning this region into a very interesting place to look for CWs. In this search, data from the full O3 LIGO--Virgo run in the detector frequency band $[10,2000]\rm~Hz$ have been used. No significant detection was found and 95$\%$ confidence level upper limits on the signal strain amplitude were computed, over the full search band, with the deepest limit of about $7.6\times 10^{-26}$ at $\simeq 142\rm~Hz$. These results are significantly more constraining than those reported in previous searches. We use these limits to put constraints on the fiducial neutron star ellipticity and r-mode amplitude. These limits can be also translated into constraints in the black hole mass -- boson mass plane for a hypothetical population of boson clouds around spinning black holes located in the GC. △ Less

Submitted 9 April, 2022; originally announced April 2022.

Comments: 25 pages, 5 figures

Search for Gravitational Waves Associated with Fast Radio Bursts Detected by CHIME/FRB During the LIGO--Virgo Observing Run O3a

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, the CHIME/FRB Collaboration, :, R. Abbott, T. D. Abbott, F. Acernese, K. Ackley, C. Adams, N. Adhikari, R. X. Adhikari, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, A. Allocca , et al. (1633 additional authors not shown)

Abstract: We search for gravitational-wave transients associated with fast radio bursts (FRBs) detected by the Canadian Hydrogen Intensity Map** Experiment Fast Radio Burst Project (CHIME/FRB), during the first part of the third observing run of Advanced LIGO and Advanced Virgo (1 April 2019 15:00 UTC-1 Oct 2019 15:00 UTC). Triggers from 22 FRBs were analyzed with a search that targets compact binary coal… ▽ More We search for gravitational-wave transients associated with fast radio bursts (FRBs) detected by the Canadian Hydrogen Intensity Map** Experiment Fast Radio Burst Project (CHIME/FRB), during the first part of the third observing run of Advanced LIGO and Advanced Virgo (1 April 2019 15:00 UTC-1 Oct 2019 15:00 UTC). Triggers from 22 FRBs were analyzed with a search that targets compact binary coalescences with at least one neutron star component. A targeted search for generic gravitational-wave transients was conducted on 40 FRBs. We find no significant evidence for a gravitational-wave association in either search. Given the large uncertainties in the distances of the FRBs inferred from the dispersion measures in our sample, however, this does not conclusively exclude any progenitor models that include emission of a gravitational wave of the types searched for from any of these FRB events. We report $90\%$ confidence lower bounds on the distance to each FRB for a range of gravitational-wave progenitor models. By combining the inferred maximum distance information for each FRB with the sensitivity of the gravitational-wave searches, we set upper limits on the energy emitted through gravitational waves for a range of emission scenarios. We find values of order $10^{51}$-$10^{57}$ erg for a range of different emission models with central gravitational wave frequencies in the range 70-3560 Hz. Finally, we also found no significant coincident detection of gravitational waves with the repeater, FRB 20200120E, which is the closest known extragalactic FRB. △ Less

Submitted 22 March, 2022; originally announced March 2022.

Comments: 35 pages, 6 figures, 8 tables

Report number: P2100124

Whole-Body Control for Velocity-Controlled Mobile Collaborative Robots Using Coupling Dynamic Movement Primitives

Authors: Zhangjie Tu, Tianwei Zhang, Lei Yan, Tin lun Lam

Abstract: In this paper, we propose a unified whole-body control framework for velocity-controlled mobile collaborative robots which can distribute task motion into the arm and mobile base according to specific task requirements by adjusting weighting factors. Our framework focuses on addressing two challenging issues in whole-body coordination: 1) different dynamic characteristics of the mobile base and th… ▽ More In this paper, we propose a unified whole-body control framework for velocity-controlled mobile collaborative robots which can distribute task motion into the arm and mobile base according to specific task requirements by adjusting weighting factors. Our framework focuses on addressing two challenging issues in whole-body coordination: 1) different dynamic characteristics of the mobile base and the arm; 2) avoidance of violating both safety and configuration constraints. In addition, our controller involves Coupling Dynamic Movement Primitives to enable the essential capabilities for collaboration and interaction applications, such as obstacle avoidance, human teaching, and compliance control. Based on these, we design an adaptive motion mode for intuitive physical human-robot interaction through adjusting the weighting factors. The proposed controller is in closed-form and thus quite computationally efficient. Several typical experiments carried out on a real mobile collaborative robot validate the effectiveness of the proposed controller. △ Less

Submitted 6 November, 2022; v1 submitted 7 March, 2022; originally announced March 2022.

RGB-D SLAM in Indoor Planar Environments with Multiple Large Dynamic Objects

Authors: Ran Long, Christian Rauch, Tianwei Zhang, Vladimir Ivan, Tin Lun Lam, Sethu Vijayakumar

Abstract: This work presents a novel dense RGB-D SLAM approach for dynamic planar environments that enables simultaneous multi-object tracking, camera localisation and background reconstruction. Previous dynamic SLAM methods either rely on semantic segmentation to directly detect dynamic objects; or assume that dynamic objects occupy a smaller proportion of the camera view than the static background and can… ▽ More This work presents a novel dense RGB-D SLAM approach for dynamic planar environments that enables simultaneous multi-object tracking, camera localisation and background reconstruction. Previous dynamic SLAM methods either rely on semantic segmentation to directly detect dynamic objects; or assume that dynamic objects occupy a smaller proportion of the camera view than the static background and can, therefore, be removed as outliers. Our approach, however, enables dense SLAM when the camera view is largely occluded by multiple dynamic objects with the aid of camera motion prior. The dynamic planar objects are separated by their different rigid motions and tracked independently. The remaining dynamic non-planar areas are removed as outliers and not mapped into the background. The evaluation demonstrates that our approach outperforms the state-of-the-art methods in terms of localisation, map**, dynamic segmentation and object tracking. We also demonstrate its robustness to large drift in the camera motion prior. △ Less

Submitted 18 October, 2022; v1 submitted 6 March, 2022; originally announced March 2022.

Comments: 8 papges, 9 figures

Journal ref: IEEE Robotics and Automation Letters 2022

First joint observation by the underground gravitational-wave detector, KAGRA, with GEO600

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, H. Abe, F. Acernese, K. Ackley, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, R. A. Alfaidi, A. Allocca, P. A. Altin , et al. (1647 additional authors not shown)

Abstract: We report the results of the first joint observation of the KAGRA detector with GEO600. KAGRA is a cryogenic and underground gravitational-wave detector consisting of a laser interferometer with three-kilometer arms, and located in Kamioka, Gifu, Japan. GEO600 is a British--German laser interferometer with 600 m arms, and located near Hannover, Germany. GEO600 and KAGRA performed a joint observing… ▽ More We report the results of the first joint observation of the KAGRA detector with GEO600. KAGRA is a cryogenic and underground gravitational-wave detector consisting of a laser interferometer with three-kilometer arms, and located in Kamioka, Gifu, Japan. GEO600 is a British--German laser interferometer with 600 m arms, and located near Hannover, Germany. GEO600 and KAGRA performed a joint observing run from April 7 to 20, 2020. We present the results of the joint analysis of the GEO--KAGRA data for transient gravitational-wave signals, including the coalescence of neutron-star binaries and generic unmodeled transients. We also perform dedicated searches for binary coalescence signals and generic transients associated with gamma-ray burst events observed during the joint run. No gravitational-wave events were identified. We evaluate the minimum detectable amplitude for various types of transient signals and the spacetime volume for which the network is sensitive to binary neutron-star coalescences. We also place lower limits on the distances to the gamma-ray bursts analysed based on the non-detection of an associated gravitational-wave signal for several signal models, including binary coalescences. These analyses demonstrate the feasibility and utility of KAGRA as a member of the global gravitational-wave detector network. △ Less

Submitted 19 August, 2022; v1 submitted 2 March, 2022; originally announced March 2022.

Comments: Matches with published version

Report number: LIGO-P2100286

Journal ref: Progress of Theoretical and Experimental Physics, Volume 2022, Issue 6, 063F01 (2022)

Search for gravitational waves from Scorpius X-1 with a hidden Markov model in O3 LIGO data

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, H. Abe, F. Acernese, K. Ackley, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, R. A. Alfaidi, A. Allocca, P. A. Altin , et al. (1647 additional authors not shown)

Abstract: Results are presented for a semi-coherent search for continuous gravitational waves from the low-mass X-ray binary Scorpius X-1, using a hidden Markov model (HMM) to allow for spin wandering. This search improves on previous HMM-based searches of Laser Interferometer Gravitational-wave Observatory (LIGO) data by including the orbital period in the search template grid, and by analyzing data from t… ▽ More Results are presented for a semi-coherent search for continuous gravitational waves from the low-mass X-ray binary Scorpius X-1, using a hidden Markov model (HMM) to allow for spin wandering. This search improves on previous HMM-based searches of Laser Interferometer Gravitational-wave Observatory (LIGO) data by including the orbital period in the search template grid, and by analyzing data from the latest (third) observing run (O3). In the frequency range searched, from 60 to 500 Hz, we find no evidence of gravitational radiation. This is the most sensitive search for Scorpius X-1 using a HMM to date. For the most sensitive sub-band, starting at $256.06$Hz, we report an upper limit on gravitational wave strain (at $95 \%$ confidence) of $h_{0}^{95\%}=6.16\times10^{-26}$, assuming the orbital inclination angle takes its electromagnetically restricted value $ι=44^{\circ}$. The upper limits on gravitational wave strain reported here are on average a factor of $\sim 3$ lower than in the O2 HMM search. This is the first Scorpius X-1 HMM search with upper limits that reach below the indirect torque-balance limit for certain sub-bands, assuming $ι=44^{\circ}$. △ Less

Submitted 25 January, 2022; originally announced January 2022.

Comments: 23 pages, 5 figures

Report number: LIGO-P2100405

All-sky search for continuous gravitational waves from isolated neutron stars using Advanced LIGO and Advanced Virgo O3 data

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, H. Abe, F. Acernese, K. Ackley, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, R. A. Alfaidi, A. Allocca, P. A. Altin , et al. (1645 additional authors not shown)

Abstract: We present results of an all-sky search for continuous gravitational waves which can be produced by spinning neutron stars with an asymmetry around their rotation axis, using data from the third observing run of the Advanced LIGO and Advanced Virgo detectors. Four different analysis methods are used to search in a gravitational-wave frequency band from 10 to 2048 Hz and a first frequency derivativ… ▽ More We present results of an all-sky search for continuous gravitational waves which can be produced by spinning neutron stars with an asymmetry around their rotation axis, using data from the third observing run of the Advanced LIGO and Advanced Virgo detectors. Four different analysis methods are used to search in a gravitational-wave frequency band from 10 to 2048 Hz and a first frequency derivative from $-10^{-8}$ to $10^{-9}$ Hz/s. No statistically-significant periodic gravitational-wave signal is observed by any of the four searches. As a result, upper limits on the gravitational-wave strain amplitude $h_0$ are calculated. The best upper limits are obtained in the frequency range of 100 to 200 Hz and they are ${\sim}1.1\times10^{-25}$ at 95\% confidence-level. The minimum upper limit of $1.10\times10^{-25}$ is achieved at a frequency 111.5 Hz. We also place constraints on the rates and abundances of nearby planetary- and asteroid-mass primordial black holes that could give rise to continuous gravitational-wave signals. △ Less

Submitted 3 January, 2022; originally announced January 2022.

Comments: 23 main text pages, 17 figures

Report number: LIGO-P2100367

Narrowband searches for continuous and long-duration transient gravitational waves from known pulsars in the LIGO-Virgo third observing run

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, T. D. Abbott, F. Acernese, K. Ackley, C. Adams, N. Adhikari, R. X. Adhikari, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, A. Allocca, P. A. Altin, A. Amato , et al. (1636 additional authors not shown)

Abstract: Isolated neutron stars that are asymmetric with respect to their spin axis are possible sources of detectable continuous gravitational waves. This paper presents a fully-coherent search for such signals from eighteen pulsars in data from LIGO and Virgo's third observing run (O3). For known pulsars, efficient and sensitive matched-filter searches can be carried out if one assumes the gravitational… ▽ More Isolated neutron stars that are asymmetric with respect to their spin axis are possible sources of detectable continuous gravitational waves. This paper presents a fully-coherent search for such signals from eighteen pulsars in data from LIGO and Virgo's third observing run (O3). For known pulsars, efficient and sensitive matched-filter searches can be carried out if one assumes the gravitational radiation is phase-locked to the electromagnetic emission. In the search presented here, we relax this assumption and allow the frequency and frequency time-derivative of the gravitational waves to vary in a small range around those inferred from electromagnetic observations. We find no evidence for continuous gravitational waves, and set upper limits on the strain amplitude for each target. These limits are more constraining for seven of the targets than the spin-down limit defined by ascribing all rotational energy loss to gravitational radiation. In an additional search we look in O3 data for long-duration (hours-months) transient gravitational waves in the aftermath of pulsar glitches for six targets with a total of nine glitches. We report two marginal outliers from this search, but find no clear evidence for such emission either. The resulting duration-dependent strain upper limits do not surpass indirect energy constraints for any of these targets. △ Less

Submitted 27 June, 2022; v1 submitted 21 December, 2021; originally announced December 2021.

Comments: 37 pages, 9 figures, submitted to ApJ

Report number: LIGO-P2100267

Journal ref: ApJ, 932, 133 (2022)

Tests of General Relativity with GWTC-3

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, H. Abe, F. Acernese, K. Ackley, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, P. F. de Alarcón, S. Albanesi, R. A. Alfaidi, A. Allocca , et al. (1657 additional authors not shown)

Abstract: The ever-increasing number of detections of gravitational waves (GWs) from compact binaries by the Advanced LIGO and Advanced Virgo detectors allows us to perform ever-more sensitive tests of general relativity (GR) in the dynamical and strong-field regime of gravity. We perform a suite of tests of GR using the compact binary signals observed during the second half of the third observing run of th… ▽ More The ever-increasing number of detections of gravitational waves (GWs) from compact binaries by the Advanced LIGO and Advanced Virgo detectors allows us to perform ever-more sensitive tests of general relativity (GR) in the dynamical and strong-field regime of gravity. We perform a suite of tests of GR using the compact binary signals observed during the second half of the third observing run of those detectors. We restrict our analysis to the 15 confident signals that have false alarm rates $\leq 10^{-3}\, {\rm yr}^{-1}$. In addition to signals consistent with binary black hole (BH) mergers, the new events include GW200115_042309, a signal consistent with a neutron star--BH merger. We find the residual power, after subtracting the best fit waveform from the data for each event, to be consistent with the detector noise. Additionally, we find all the post-Newtonian deformation coefficients to be consistent with the predictions from GR, with an improvement by a factor of ~2 in the -1PN parameter. We also find that the spin-induced quadrupole moments of the binary BH constituents are consistent with those of Kerr BHs in GR. We find no evidence for dispersion of GWs, non-GR modes of polarization, or post-merger echoes in the events that were analyzed. We update the bound on the mass of the graviton, at 90% credibility, to $m_g \leq 1.27 \times 10^{-23} \mathrm{eV}/c^2$. The final mass and final spin as inferred from the pre-merger and post-merger parts of the waveform are consistent with each other. The studies of the properties of the remnant BHs, including deviations of the quasi-normal mode frequencies and dam** times, show consistency with the predictions of GR. In addition to considering signals individually, we also combine results from the catalog of GW signals to calculate more precise population constraints. We find no evidence in support of physics beyond GR. △ Less

Submitted 13 December, 2021; originally announced December 2021.

Report number: LIGO-P2100275

All-sky search for gravitational wave emission from scalar boson clouds around spinning black holes in LIGO O3 data

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, H. Abe, F. Acernese, K. Ackley, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, R. A. Alfaidi, A. Allocca, P. A. Altin , et al. (1647 additional authors not shown)

Abstract: This paper describes the first all-sky search for long-duration, quasi-monochromatic gravitational-wave signals emitted by ultralight scalar boson clouds around spinning black holes using data from the third observing run of Advanced LIGO. We analyze the frequency range from 20~Hz to 610~Hz, over a small frequency derivative range around zero, and use multiple frequency resolutions to be robust to… ▽ More This paper describes the first all-sky search for long-duration, quasi-monochromatic gravitational-wave signals emitted by ultralight scalar boson clouds around spinning black holes using data from the third observing run of Advanced LIGO. We analyze the frequency range from 20~Hz to 610~Hz, over a small frequency derivative range around zero, and use multiple frequency resolutions to be robust towards possible signal frequency wanderings. Outliers from this search are followed up using two different methods, one more suitable for nearly monochromatic signals, and the other more robust towards frequency fluctuations. We do not find any evidence for such signals and set upper limits on the signal strain amplitude, the most stringent being $\approx10^{-25}$ at around 130~Hz. We interpret these upper limits as both an "exclusion region" in the boson mass/black hole mass plane and the maximum detectable distance for a given boson mass, based on an assumption of the age of the black hole/boson cloud system. △ Less

Submitted 9 May, 2022; v1 submitted 30 November, 2021; originally announced November 2021.

Comments: 28 pages, 16 figures

Report number: P2100343

Journal ref: Phys. Rev. D 105, 102001, 2022

Search of the Early O3 LIGO Data for Continuous Gravitational Waves from the Cassiopeia A and Vela Jr. Supernova Remnants

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, R. Abbott, T. D. Abbott, F. Acernese, K. Ackley, C. Adams, N. Adhikari, R. X. Adhikari, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, S. Albanesi, A. Allocca, P. A. Altin, A. Amato, C. Anand, S. Anand , et al. (1389 additional authors not shown)

Abstract: We present directed searches for continuous gravitational waves from the neutron stars in the Cassiopeia A (Cas A) and Vela Jr. supernova remnants. We carry out the searches in the LIGO data from the first six months of the third Advanced LIGO and Virgo observing run, using the Weave semi-coherent method, which sums matched-filter detection-statistic values over many time segments spanning the obs… ▽ More We present directed searches for continuous gravitational waves from the neutron stars in the Cassiopeia A (Cas A) and Vela Jr. supernova remnants. We carry out the searches in the LIGO data from the first six months of the third Advanced LIGO and Virgo observing run, using the Weave semi-coherent method, which sums matched-filter detection-statistic values over many time segments spanning the observation period. No gravitational wave signal is detected in the search band of 20--976 Hz for assumed source ages greater than 300 years for Cas A and greater than 700 years for Vela Jr. Estimates from simulated continuous wave signals indicate we achieve the most sensitive results to date across the explored parameter space volume, probing to strain magnitudes as low as ~$6.3\times10^{-26}$ for Cas A and ~$5.6\times10^{-26}$ for Vela Jr. at frequencies near 166 Hz at 95% efficiency. △ Less

Submitted 22 March, 2022; v1 submitted 29 November, 2021; originally announced November 2021.

Comments: 24 pages, 8 figures. To appear in Physical Review D

Report number: LIGO-P2100298-v8

Searches for Gravitational Waves from Known Pulsars at Two Harmonics in the Second and Third LIGO-Virgo Observing Runs

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, H. Abe, F. Acernese, K. Ackley, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, R. A. Alfaidi, A. Allocca, P. A. Altin , et al. (1672 additional authors not shown)

Abstract: We present a targeted search for continuous gravitational waves (GWs) from 236 pulsars using data from the third observing run of LIGO and Virgo (O3) combined with data from the second observing run (O2). Searches were for emission from the $l=m=2$ mass quadrupole mode with a frequency at only twice the pulsar rotation frequency (single harmonic) and the $l=2, m=1,2$ modes with a frequency of both… ▽ More We present a targeted search for continuous gravitational waves (GWs) from 236 pulsars using data from the third observing run of LIGO and Virgo (O3) combined with data from the second observing run (O2). Searches were for emission from the $l=m=2$ mass quadrupole mode with a frequency at only twice the pulsar rotation frequency (single harmonic) and the $l=2, m=1,2$ modes with a frequency of both once and twice the rotation frequency (dual harmonic). No evidence of GWs was found so we present 95\% credible upper limits on the strain amplitudes $h_0$ for the single harmonic search along with limits on the pulsars' mass quadrupole moments $Q_{22}$ and ellipticities $\varepsilon$. Of the pulsars studied, 23 have strain amplitudes that are lower than the limits calculated from their electromagnetically measured spin-down rates. These pulsars include the millisecond pulsars J0437\textminus4715 and J0711\textminus6830 which have spin-down ratios of 0.87 and 0.57 respectively. For nine pulsars, their spin-down limits have been surpassed for the first time. For the Crab and Vela pulsars our limits are factors of $\sim 100$ and $\sim 20$ more constraining than their spin-down limits, respectively. For the dual harmonic searches, new limits are placed on the strain amplitudes $C_{21}$ and $C_{22}$. For 23 pulsars we also present limits on the emission amplitude assuming dipole radiation as predicted by Brans-Dicke theory. △ Less

Submitted 20 July, 2022; v1 submitted 25 November, 2021; originally announced November 2021.

Comments: 37 pages

Report number: LIGO-P2100049

The population of merging compact binaries inferred using gravitational waves through GWTC-3

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, T. D. Abbott, F. Acernese, K. Ackley, C. Adams, N. Adhikari, R. X. Adhikari, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, A. Allocca, P. A. Altin, A. Amato , et al. (1612 additional authors not shown)

Abstract: We report on the population properties of 76 compact binary mergers detected with gravitational waves below a false alarm rate of 1 per year through GWTC-3. The catalog contains three classes of binary mergers: BBH, BNS, and NSBH mergers. We infer the BNS merger rate to be between 10 $\rm{Gpc^{-3} yr^{-1}}$ and 1700 $\rm{Gpc^{-3} yr^{-1}}$ and the NSBH merger rate to be between 7.8… ▽ More We report on the population properties of 76 compact binary mergers detected with gravitational waves below a false alarm rate of 1 per year through GWTC-3. The catalog contains three classes of binary mergers: BBH, BNS, and NSBH mergers. We infer the BNS merger rate to be between 10 $\rm{Gpc^{-3} yr^{-1}}$ and 1700 $\rm{Gpc^{-3} yr^{-1}}$ and the NSBH merger rate to be between 7.8 $\rm{Gpc^{-3}\, yr^{-1}}$ and 140 $\rm{Gpc^{-3} yr^{-1}}$ , assuming a constant rate density versus comoving volume and taking the union of 90% credible intervals for methods used in this work. Accounting for the BBH merger rate to evolve with redshift, we find the BBH merger rate to be between 17.9 $\rm{Gpc^{-3}\, yr^{-1}}$ and 44 $\rm{Gpc^{-3}\, yr^{-1}}$ at a fiducial redshift (z=0.2). We obtain a broad neutron star mass distribution extending from $1.2^{+0.1}_{-0.2} M_\odot$ to $2.0^{+0.3}_{-0.3} M_\odot$. We can confidently identify a rapid decrease in merger rate versus component mass between neutron star-like masses and black-hole-like masses, but there is no evidence that the merger rate increases again before 10 $M_\odot$. We also find the BBH mass distribution has localized over- and under-densities relative to a power law distribution. While we continue to find the mass distribution of a binary's more massive component strongly decreases as a function of primary mass, we observe no evidence of a strongly suppressed merger rate above $\sim 60 M_\odot$. The rate of BBH mergers is observed to increase with redshift at a rate proportional to $(1+z)^κ$ with $κ= 2.9^{+1.7}_{-1.8}$ for $z\lesssim 1$. Observed black hole spins are small, with half of spin magnitudes below $χ_i \simeq 0.25$. We observe evidence of negative aligned spins in the population, and an increase in spin magnitude for systems with more unequal mass ratio. △ Less

Submitted 23 February, 2022; v1 submitted 5 November, 2021; originally announced November 2021.

Comments: v2: minor edits, most to Table 1 and caption; v3: rerun with public data; Data release: https://zenodo.org/record/5655785; v4: update Fig 14

Report number: LIGO-P2100239

Search for Gravitational Waves Associated with Gamma-Ray Bursts Detected by Fermi and Swift During the LIGO-Virgo Run O3b

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, T. D. Abbott, F. Acernese, K. Ackley, C. Adams, N. Adhikari, R. X. Adhikari, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, A. Allocca, P. A. Altin, A. Amato , et al. (1610 additional authors not shown)

Abstract: We search for gravitational-wave signals associated with gamma-ray bursts detected by the Fermi and Swift satellites during the second half of the third observing run of Advanced LIGO and Advanced Virgo (1 November 2019 15:00 UTC-27 March 2020 17:00 UTC).We conduct two independent searches: a generic gravitational-wave transients search to analyze 86 gamma-ray bursts and an analysis to target bina… ▽ More We search for gravitational-wave signals associated with gamma-ray bursts detected by the Fermi and Swift satellites during the second half of the third observing run of Advanced LIGO and Advanced Virgo (1 November 2019 15:00 UTC-27 March 2020 17:00 UTC).We conduct two independent searches: a generic gravitational-wave transients search to analyze 86 gamma-ray bursts and an analysis to target binary mergers with at least one neutron star as short gamma-ray burst progenitors for 17 events. We find no significant evidence for gravitational-wave signals associated with any of these gamma-ray bursts. A weighted binomial test of the combined results finds no evidence for sub-threshold gravitational wave signals associated with this GRB ensemble either. We use several source types and signal morphologies during the searches, resulting in lower bounds on the estimated distance to each gamma-ray burst. Finally, we constrain the population of low luminosity short gamma-ray bursts using results from the first to the third observing runs of Advanced LIGO and Advanced Virgo. The resulting population is in accordance with the local binary neutron star merger rate. △ Less

Submitted 5 November, 2021; originally announced November 2021.

Comments: 26 pages, 6 figures, 2 tables

Report number: P2100091

GWTC-3: Compact Binary Coalescences Observed by LIGO and Virgo During the Second Part of the Third Observing Run

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, T. D. Abbott, F. Acernese, K. Ackley, C. Adams, N. Adhikari, R. X. Adhikari, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, S. Akcay, T. Akutsu, S. Albanesi, A. Allocca, P. A. Altin , et al. (1637 additional authors not shown)

Abstract: The third Gravitational-Wave Transient Catalog (GWTC-3) describes signals detected with Advanced LIGO and Advanced Virgo up to the end of their third observing run. Updating the previous GWTC-2.1, we present candidate gravitational waves from compact binary coalescences during the second half of the third observing run (O3b) between 1 November 2019, 15:00 UTC and 27 March 2020, 17:00 UTC. There ar… ▽ More The third Gravitational-Wave Transient Catalog (GWTC-3) describes signals detected with Advanced LIGO and Advanced Virgo up to the end of their third observing run. Updating the previous GWTC-2.1, we present candidate gravitational waves from compact binary coalescences during the second half of the third observing run (O3b) between 1 November 2019, 15:00 UTC and 27 March 2020, 17:00 UTC. There are 35 compact binary coalescence candidates identified by at least one of our search algorithms with a probability of astrophysical origin $p_\mathrm{astro} > 0.5$. Of these, 18 were previously reported as low-latency public alerts, and 17 are reported here for the first time. Based upon estimates for the component masses, our O3b candidates with $p_\mathrm{astro} > 0.5$ are consistent with gravitational-wave signals from binary black holes or neutron star-black hole binaries, and we identify none from binary neutron stars. However, from the gravitational-wave data alone, we are not able to measure matter effects that distinguish whether the binary components are neutron stars or black holes. The range of inferred component masses is similar to that found with previous catalogs, but the O3b candidates include the first confident observations of neutron star-black hole binaries. Including the 35 candidates from O3b in addition to those from GWTC-2.1, GWTC-3 contains 90 candidates found by our analysis with $p_\mathrm{astro} > 0.5$ across the first three observing runs. These observations of compact binary coalescences present an unprecedented view of the properties of black holes and neutron stars. △ Less

Submitted 23 October, 2023; v1 submitted 5 November, 2021; originally announced November 2021.

Comments: 88 pages (10 pages author list, 31 pages main text, 1 page acknowledgements, 24 pages appendices, 22 pages bibliography), 17 figures, 16 tables. Update to match version to be published in Physical Review X. Data products available from https://gwosc.org/GWTC-3/

Report number: LIGO-P2000318

Journal ref: Phys. Rev. X; 13(4):041039; 2023

Constraints on the cosmic expansion history from GWTC-3

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, H. Abe, F. Acernese, K. Ackley, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, R. A. Alfaidi, A. Allocca, P. A. Altin , et al. (1654 additional authors not shown)

Abstract: We use 47 gravitational-wave sources from the Third LIGO-Virgo-KAGRA Gravitational-Wave Transient Catalog (GWTC-3) to estimate the Hubble parameter $H(z)$, including its current value, the Hubble constant $H_0$. Each gravitational-wave (GW) signal provides the luminosity distance to the source and we estimate the corresponding redshift using two methods: the redshifted masses and a galaxy catalog.… ▽ More We use 47 gravitational-wave sources from the Third LIGO-Virgo-KAGRA Gravitational-Wave Transient Catalog (GWTC-3) to estimate the Hubble parameter $H(z)$, including its current value, the Hubble constant $H_0$. Each gravitational-wave (GW) signal provides the luminosity distance to the source and we estimate the corresponding redshift using two methods: the redshifted masses and a galaxy catalog. Using the binary black hole (BBH) redshifted masses, we simultaneously infer the source mass distribution and $H(z)$. The source mass distribution displays a peak around $34\, {\rm M_\odot}$, followed by a drop-off. Assuming this mass scale does not evolve with redshift results in a $H(z)$ measurement, yielding $H_0=68^{+12}_{-7} {\rm km\,s^{-1}\,Mpc^{-1}}$ ($68\%$ credible interval) when combined with the $H_0$ measurement from GW170817 and its electromagnetic counterpart. This represents an improvement of 17% with respect to the $H_0$ estimate from GWTC-1. The second method associates each GW event with its probable host galaxy in the catalog GLADE+, statistically marginalizing over the redshifts of each event's potential hosts. Assuming a fixed BBH population, we estimate a value of $H_0=68^{+8}_{-6} {\rm km\,s^{-1}\,Mpc^{-1}}$ with the galaxy catalog method, an improvement of 42% with respect to our GWTC-1 result and 20% with respect to recent $H_0$ studies using GWTC-2 events. However, we show that this result is strongly impacted by assumptions about the BBH source mass distribution; the only event which is not strongly impacted by such assumptions (and is thus informative about $H_0$) is the well-localized event GW190814. △ Less

Submitted 19 November, 2021; v1 submitted 5 November, 2021; originally announced November 2021.

Comments: Main paper: 30 pages, 15 figure, 7 tables

Report number: LIGO-P2100185-v6

All-sky, all-frequency directional search for persistent gravitational-waves from Advanced LIGO's and Advanced Virgo's first three observing runs

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, T. D. Abbott, F. Acernese, K. Ackley, C. Adams, N. Adhikari, R. X. Adhikari, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, A. Allocca, P. A. Altin, A. Amato , et al. (1605 additional authors not shown)

Abstract: We present the first results from an all-sky all-frequency (ASAF) search for an anisotropic stochastic gravitational-wave background using the data from the first three observing runs of the Advanced LIGO and Advanced Virgo detectors. Upper limit maps on broadband anisotropies of a persistent stochastic background were published for all observing runs of the LIGO-Virgo detectors. However, a broadb… ▽ More We present the first results from an all-sky all-frequency (ASAF) search for an anisotropic stochastic gravitational-wave background using the data from the first three observing runs of the Advanced LIGO and Advanced Virgo detectors. Upper limit maps on broadband anisotropies of a persistent stochastic background were published for all observing runs of the LIGO-Virgo detectors. However, a broadband analysis is likely to miss narrowband signals as the signal-to-noise ratio of a narrowband signal can be significantly reduced when combined with detector output from other frequencies. Data folding and the computationally efficient analysis pipeline, {\tt PyStoch}, enable us to perform the radiometer map-making at every frequency bin. We perform the search at 3072 {\tt{HEALPix}} equal area pixels uniformly tiling the sky and in every frequency bin of width $1/32$~Hz in the range $20-1726$~Hz, except for bins that are likely to contain instrumental artefacts and hence are notched. We do not find any statistically significant evidence for the existence of narrowband gravitational-wave signals in the analyzed frequency bins. Therefore, we place $95\%$ confidence upper limits on the gravitational-wave strain for each pixel-frequency pair, the limits are in the range $(0.030 - 9.6) \times10^{-24}$. In addition, we outline a method to identify candidate pixel-frequency pairs that could be followed up by a more sensitive (and potentially computationally expensive) search, e.g., a matched-filtering-based analysis, to look for fainter nearly monochromatic coherent signals. The ASAF analysis is inherently independent of models describing any spectral or spatial distribution of power. We demonstrate that the ASAF results can be appropriately combined over frequencies and sky directions to successfully recover the broadband directional and isotropic results. △ Less

Submitted 19 October, 2021; originally announced October 2021.

Comments: 22 pages, 6 figures, 1 table

Report number: LIGO-P2100292

Abnormal Occupancy Grid Map Recognition using Attention Network

Authors: Fuqin Deng, Hua Feng, Mingjian Liang, Qi Feng, Ningbo Yi, Yong Yang, Yuan Gao, Junfeng Chen, Tin Lun Lam

Abstract: The occupancy grid map is a critical component of autonomous positioning and navigation in the mobile robotic system, as many other systems' performance depends heavily on it. To guarantee the quality of the occupancy grid maps, researchers previously had to perform tedious manual recognition for a long time. This work focuses on automatic abnormal occupancy grid map recognition using the residual… ▽ More The occupancy grid map is a critical component of autonomous positioning and navigation in the mobile robotic system, as many other systems' performance depends heavily on it. To guarantee the quality of the occupancy grid maps, researchers previously had to perform tedious manual recognition for a long time. This work focuses on automatic abnormal occupancy grid map recognition using the residual neural networks and a novel attention mechanism module. We propose an effective channel and spatial Residual SE(csRSE) attention module, which contains a residual block for producing hierarchical features, followed by both channel SE (cSE) block and spatial SE (sSE) block for the sufficient information extraction along the channel and spatial pathways. To further summarize the occupancy grid map characteristics and experiment with our csRSE attention modules, we constructed a dataset called occupancy grid map dataset (OGMD) for our experiments. On this OGMD test dataset, we tested few variants of our proposed structure and compared them with other attention mechanisms. Our experimental results show that the proposed attention network can infer the abnormal map with state-of-the-art (SOTA) accuracy of 96.23% for abnormal occupancy grid map recognition. △ Less

Submitted 18 October, 2021; originally announced October 2021.

FEANet: Feature-Enhanced Attention Network for RGB-Thermal Real-time Semantic Segmentation

Authors: Fuqin Deng, Hua Feng, Mingjian Liang, Hongmin Wang, Yong Yang, Yuan Gao, Junfeng Chen, Junjie Hu, Xiyue Guo, Tin Lun Lam

Abstract: The RGB-Thermal (RGB-T) information for semantic segmentation has been extensively explored in recent years. However, most existing RGB-T semantic segmentation usually compromises spatial resolution to achieve real-time inference speed, which leads to poor performance. To better extract detail spatial information, we propose a two-stage Feature-Enhanced Attention Network (FEANet) for the RGB-T sem… ▽ More The RGB-Thermal (RGB-T) information for semantic segmentation has been extensively explored in recent years. However, most existing RGB-T semantic segmentation usually compromises spatial resolution to achieve real-time inference speed, which leads to poor performance. To better extract detail spatial information, we propose a two-stage Feature-Enhanced Attention Network (FEANet) for the RGB-T semantic segmentation task. Specifically, we introduce a Feature-Enhanced Attention Module (FEAM) to excavate and enhance multi-level features from both the channel and spatial views. Benefited from the proposed FEAM module, our FEANet can preserve the spatial information and shift more attention to high-resolution features from the fused RGB-T images. Extensive experiments on the urban scene dataset demonstrate that our FEANet outperforms other state-of-the-art (SOTA) RGB-T methods in terms of objective metrics and subjective visual comparison (+2.6% in global mAcc and +0.8% in global mIoU). For the 480 x 640 RGB-T test images, our FEANet can run with a real-time speed on an NVIDIA GeForce RTX 2080 Ti card. △ Less

Submitted 17 October, 2021; originally announced October 2021.

Comments: 7 pages, 5 figures

AB-Mapper: Attention and BicNet Based Multi-agent Path Finding for Dynamic Crowded Environment

Authors: Huifeng Guan, Yuan Gao, Min Zhao, Yong Yang, Fuqin Deng, Tin Lun Lam

Abstract: Multi-agent path finding in dynamic crowded environments is of great academic and practical value for multi-robot systems in the real world. To improve the effectiveness and efficiency of communication and learning process during path planning in dynamic crowded environments, we introduce an algorithm called Attention and BicNet based Multi-agent path planning with effective reinforcement (AB-Mapp… ▽ More Multi-agent path finding in dynamic crowded environments is of great academic and practical value for multi-robot systems in the real world. To improve the effectiveness and efficiency of communication and learning process during path planning in dynamic crowded environments, we introduce an algorithm called Attention and BicNet based Multi-agent path planning with effective reinforcement (AB-Mapper)under the actor-critic reinforcement learning framework. In this framework, on the one hand, we utilize the BicNet with communication function in the actor-network to achieve intra team coordination. On the other hand, we propose a centralized critic network that can selectively allocate attention weights to surrounding agents. This attention mechanism allows an individual agent to automatically learn a better evaluation of actions by also considering the behaviours of its surrounding agents. Compared with the state-of-the-art method Mapper,our AB-Mapper is more effective (85.86% vs. 81.56% in terms of success rate) in solving the general path finding problems with dynamic obstacles. In addition, in crowded scenarios, our method outperforms the Mapper method by a large margin,reaching a stunning gap of more than 40% for each experiment. △ Less

Submitted 2 October, 2021; originally announced October 2021.

Meta Reinforcement Learning Based Sensor Scanning in 3D Uncertain Environments for Heterogeneous Multi-Robot Systems

Authors: Junfeng Chen, Yuan Gao, Junjie Hu, Fuqin Deng, Tin Lun Lam

Abstract: We study a novel problem that tackles learning based sensor scanning in 3D and uncertain environments with heterogeneous multi-robot systems. Our motivation is two-fold: first, 3D environments are complex, the use of heterogeneous multi-robot systems intuitively can facilitate sensor scanning by fully taking advantage of sensors with different capabilities. Second, in uncertain environments (e.g.… ▽ More We study a novel problem that tackles learning based sensor scanning in 3D and uncertain environments with heterogeneous multi-robot systems. Our motivation is two-fold: first, 3D environments are complex, the use of heterogeneous multi-robot systems intuitively can facilitate sensor scanning by fully taking advantage of sensors with different capabilities. Second, in uncertain environments (e.g. rescue), time is of great significance. Since the learning process normally takes time to train and adapt to a new environment, we need to find an effective way to explore and adapt quickly. To this end, in this paper, we present a meta-learning approach to improve the exploration and adaptation capabilities. The experimental results demonstrate our method can outperform other methods by approximately 15%-27% on success rate and 70%-75% on adaptation speed. △ Less

Submitted 28 September, 2021; originally announced September 2021.

Comments: 6 pages, 9 figures

Search for subsolar-mass binaries in the first half of Advanced LIGO and Virgo's third observing run

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, T. D. Abbott, F. Acernese, K. Ackley, C. Adams, N. Adhikari, R. X. Adhikari, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, A. Allocca, P. A. Altin, A. Amato , et al. (1612 additional authors not shown)

Abstract: We report on a search for compact binary coalescences where at least one binary component has a mass between 0.2 $M_\odot$ and 1.0 $M_\odot$ in Advanced LIGO and Advanced Virgo data collected between 1 April 2019 1500 UTC and 1 October 2019 1500 UTC. We extend previous analyses in two main ways: we include data from the Virgo detector and we allow for more unequal mass systems, with mass ratio… ▽ More We report on a search for compact binary coalescences where at least one binary component has a mass between 0.2 $M_\odot$ and 1.0 $M_\odot$ in Advanced LIGO and Advanced Virgo data collected between 1 April 2019 1500 UTC and 1 October 2019 1500 UTC. We extend previous analyses in two main ways: we include data from the Virgo detector and we allow for more unequal mass systems, with mass ratio $q \geq 0.1$. We do not report any gravitational-wave candidates. The most significant trigger has a false alarm rate of 0.14 $\mathrm{yr}^{-1}$. This implies an upper limit on the merger rate of subsolar binaries in the range $[220-24200] \mathrm{Gpc}^{-3} \mathrm{yr}^{-1}$, depending on the chirp mass of the binary. We use this upper limit to derive astrophysical constraints on two phenomenological models that could produce subsolar-mass compact objects. One is an isotropic distribution of equal-mass primordial black holes. Using this model, we find that the fraction of dark matter in primordial black holes is $f_\mathrm{PBH} \equiv Ω_\mathrm{PBH} / Ω_\mathrm{DM} \lesssim 6\%$. The other is a dissipative dark matter model, in which fermionic dark matter can collapse and form black holes. The upper limit on the fraction of dark matter black holes depends on the minimum mass of the black holes that can be formed: the most constraining result is obtained at $M_\mathrm{min}=1 M_\odot$, where $f_\mathrm{DBH} \equiv Ω_\mathrm{PBH} / Ω_\mathrm{DM} \lesssim 0.003\%$. These are the tightest limits on spinning subsolar-mass binaries to date. △ Less

Submitted 24 September, 2021; originally announced September 2021.

Report number: LIGO-P2100163-v8

Search for continuous gravitational waves from 20 accreting millisecond X-ray pulsars in O3 LIGO data

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, T. D. Abbott, F. Acernese, K. Ackley, C. Adams, N. Adhikari, R. X. Adhikari, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, T. Akutsu, S. Albanesi, A. Allocca, P. A. Altin, A. Amato, C. Anand , et al. (1612 additional authors not shown)

Abstract: Results are presented of searches for continuous gravitational waves from 20 accreting millisecond X-ray pulsars with accurately measured spin frequencies and orbital parameters, using data from the third observing run of the Advanced LIGO and Advanced Virgo detectors. The search algorithm uses a hidden Markov model, where the transition probabilities allow the frequency to wander according to an… ▽ More Results are presented of searches for continuous gravitational waves from 20 accreting millisecond X-ray pulsars with accurately measured spin frequencies and orbital parameters, using data from the third observing run of the Advanced LIGO and Advanced Virgo detectors. The search algorithm uses a hidden Markov model, where the transition probabilities allow the frequency to wander according to an unbiased random walk, while the $\mathcal{J}$-statistic maximum-likelihood matched filter tracks the binary orbital phase. Three narrow sub-bands are searched for each target, centered on harmonics of the measured spin frequency. The search yields 16 candidates, consistent with a false alarm probability of 30% per sub-band and target searched. These candidates, along with one candidate from an additional target-of-opportunity search done for SAX J1808.4$-$3658, which was in outburst during one month of the observing run, cannot be confidently associated with a known noise source. Additional follow-up does not provide convincing evidence that any are a true astrophysical signal. When all candidates are assumed non-astrophysical, upper limits are set on the maximum wave strain detectable at 95% confidence, $h_0^{95\%}$. The strictest constraint is $h_0^{95\%} = 4.7\times 10^{-26}$ from IGR J17062$-$6143. Constraints on the detectable wave strain from each target lead to constraints on neutron star ellipticity and $r$-mode amplitude, the strictest of which are $ε^{95\%} = 3.1\times 10^{-7}$ and $α^{95\%} = 1.8\times 10^{-5}$ respectively. This analysis is the most comprehensive and sensitive search of continuous gravitational waves from accreting millisecond X-ray pulsars to date. △ Less

Submitted 21 January, 2022; v1 submitted 19 September, 2021; originally announced September 2021.

Comments: 40 pages, 6 figures. This version contains minor typographical revisions to match published article

Report number: LIGO-P2100221

Journal ref: Phys. Rev. D 105, 022002 (2022)

View Blind-spot as Inpainting: Self-Supervised Denoising with Mask Guided Residual Convolution

Authors: Yuhongze Zhou, Liguang Zhou, Tin Lun Lam, Yangsheng Xu

Abstract: In recent years, self-supervised denoising methods have shown impressive performance, which circumvent painstaking collection procedure of noisy-clean image pairs in supervised denoising methods and boost denoising applicability in real world. One of well-known self-supervised denoising strategies is the blind-spot training scheme. However, a few works attempt to improve blind-spot based self-deno… ▽ More In recent years, self-supervised denoising methods have shown impressive performance, which circumvent painstaking collection procedure of noisy-clean image pairs in supervised denoising methods and boost denoising applicability in real world. One of well-known self-supervised denoising strategies is the blind-spot training scheme. However, a few works attempt to improve blind-spot based self-denoiser in the aspect of network architecture. In this paper, we take an intuitive view of blind-spot strategy and consider its process of using neighbor pixels to predict manipulated pixels as an inpainting process. Therefore, we propose a novel Mask Guided Residual Convolution (MGRConv) into common convolutional neural networks, e.g. U-Net, to promote blind-spot based denoising. Our MGRConv can be regarded as soft partial convolution and find a trade-off among partial convolution, learnable attention maps, and gated convolution. It enables dynamic mask learning with appropriate mask constrain. Different from partial convolution and gated convolution, it provides moderate freedom for network learning. It also avoids leveraging external learnable parameters for mask activation, unlike learnable attention maps. The experiments show that our proposed plug-and-play MGRConv can assist blind-spot based denoising network to reach promising results on both existing single-image based and dataset-based methods. △ Less

Submitted 10 September, 2021; originally announced September 2021.

Learn2Agree: Fitting with Multiple Annotators without Objective Ground Truth

Authors: Chongyang Wang, Yuan Gao, Chenyou Fan, Junjie Hu, Tin Lun Lam, Nicholas D. Lane, Nadia Bianchi-Berthouze

Abstract: The annotation of domain experts is important for some medical applications where the objective ground truth is ambiguous to define, e.g., the rehabilitation for some chronic diseases, and the prescreening of some musculoskeletal abnormalities without further medical examinations. However, improper uses of the annotations may hinder develo** reliable models. On one hand, forcing the use of a sin… ▽ More The annotation of domain experts is important for some medical applications where the objective ground truth is ambiguous to define, e.g., the rehabilitation for some chronic diseases, and the prescreening of some musculoskeletal abnormalities without further medical examinations. However, improper uses of the annotations may hinder develo** reliable models. On one hand, forcing the use of a single ground truth generated from multiple annotations is less informative for the modeling. On the other hand, feeding the model with all the annotations without proper regularization is noisy given existing disagreements. For such issues, we propose a novel Learning to Agreement (Learn2Agree) framework to tackle the challenge of learning from multiple annotators without objective ground truth. The framework has two streams, with one stream fitting with the multiple annotators and the other stream learning agreement information between annotators. In particular, the agreement learning stream produces regularization information to the classifier stream, tuning its decision to be better in line with the agreement between annotators. The proposed method can be easily added to existing backbones, with experiments on two medical datasets showed better agreement levels with annotators. △ Less

Submitted 9 March, 2023; v1 submitted 8 September, 2021; originally announced September 2021.

Comments: Accepted by the TML4H workshop at ICLR 2023

AcousticFusion: Fusing Sound Source Localization to Visual SLAM in Dynamic Environments

Authors: Tianwei Zhang, Huayan Zhang, Xiaofei Li, Junfeng Chen, Tin Lun Lam, Sethu Vijayakumar

Abstract: Dynamic objects in the environment, such as people and other agents, lead to challenges for existing simultaneous localization and map** (SLAM) approaches. To deal with dynamic environments, computer vision researchers usually apply some learning-based object detectors to remove these dynamic objects. However, these object detectors are computationally too expensive for mobile robot on-board pro… ▽ More Dynamic objects in the environment, such as people and other agents, lead to challenges for existing simultaneous localization and map** (SLAM) approaches. To deal with dynamic environments, computer vision researchers usually apply some learning-based object detectors to remove these dynamic objects. However, these object detectors are computationally too expensive for mobile robot on-board processing. In practical applications, these objects output noisy sounds that can be effectively detected by on-board sound source localization. The directional information of the sound source object can be efficiently obtained by direction of sound arrival (DoA) estimation, but depth estimation is difficult. Therefore, in this paper, we propose a novel audio-visual fusion approach that fuses sound source direction into the RGB-D image and thus removes the effect of dynamic obstacles on the multi-robot SLAM system. Experimental results of multi-robot SLAM in different dynamic environments show that the proposed method uses very small computational resources to obtain very stable self-localization results. △ Less

Submitted 2 August, 2021; originally announced August 2021.

Comments: Accepted by IROS-2021

GWTC-2.1: Deep Extended Catalog of Compact Binary Coalescences Observed by LIGO and Virgo During the First Half of the Third Observing Run

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, R. Abbott, T. D. Abbott, F. Acernese, K. Ackley, C. Adams, N. Adhikari, R. X. Adhikari, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, S. Albanesi, A. Allocca, P. A. Altin, A. Amato, C. Anand, S. Anand , et al. (1407 additional authors not shown)

Abstract: The second Gravitational-Wave Transient Catalog reported on 39 compact binary coalescences observed by the Advanced LIGO and Advanced Virgo detectors between 1 April 2019 15:00 UTC and 1 October 2019 15:00 UTC. We present GWTC-2.1, which reports on a deeper list of candidate events observed over the same period. We analyze the final version of the strain data over this period with improved calibra… ▽ More The second Gravitational-Wave Transient Catalog reported on 39 compact binary coalescences observed by the Advanced LIGO and Advanced Virgo detectors between 1 April 2019 15:00 UTC and 1 October 2019 15:00 UTC. We present GWTC-2.1, which reports on a deeper list of candidate events observed over the same period. We analyze the final version of the strain data over this period with improved calibration and better subtraction of excess noise, which has been publicly released. We employ three matched-filter search pipelines for candidate identification, and estimate the astrophysical probability for each candidate event. While GWTC-2 used a false alarm rate threshold of 2 per year, we include in GWTC-2.1, 1201 candidates that pass a false alarm rate threshold of 2 per day. We calculate the source properties of a subset of 44 high-significance candidates that have an astrophysical probability greater than 0.5. Of these candidates, 36 have been reported in GWTC-2. If the 8 additional high-significance candidates presented here are astrophysical, the mass range of events that are unambiguously identified as binary black holes (both objects $\geq 3M_\odot$) is increased compared to GWTC-2, with total masses from $\sim 14 M_\odot$ for GW190924_021846 to $\sim 182 M_\odot$ for GW190426_190642. The primary components of two new candidate events (GW190403_051519 and GW190426_190642) fall in the mass gap predicted by pair instability supernova theory. We also expand the population of binaries with significantly asymmetric mass ratios reported in GWTC-2 by an additional two events (the mass ratio is less than $0.65$ and $0.44$ at $90\%$ probability for GW190403_051519 and GW190917_114630 respectively), and find that 2 of the 8 new events have effective inspiral spins $χ_\mathrm{eff} > 0$ (at $90\%$ credibility), while no binary is consistent with $χ_\mathrm{eff} < 0$ at the same significance. △ Less

Submitted 10 May, 2022; v1 submitted 2 August, 2021; originally announced August 2021.

Comments: 8 figures, 8 tables, including updates to parameter estimates of events from GWTC-1 and GWTC-2 in an Appendix

Report number: LIGO-P2100063

PoseFusion2: Simultaneous Background Reconstruction and Human Shape Recovery in Real-time

Authors: Huayan Zhang, Tianwei Zhang, Tin Lun Lam, Sethu Vijayakumar

Abstract: Dynamic environments that include unstructured moving objects pose a hard problem for Simultaneous Localization and Map** (SLAM) performance. The motion of rigid objects can be typically tracked by exploiting their texture and geometric features. However, humans moving in the scene are often one of the most important, interactive targets - they are very hard to track and reconstruct robustly due… ▽ More Dynamic environments that include unstructured moving objects pose a hard problem for Simultaneous Localization and Map** (SLAM) performance. The motion of rigid objects can be typically tracked by exploiting their texture and geometric features. However, humans moving in the scene are often one of the most important, interactive targets - they are very hard to track and reconstruct robustly due to non-rigid shapes. In this work, we present a fast, learning-based human object detector to isolate the dynamic human objects and realise a real-time dense background reconstruction framework. We go further by estimating and reconstructing the human pose and shape. The final output environment maps not only provide the dense static backgrounds but also contain the dynamic human meshes and their trajectories. Our Dynamic SLAM system runs at around 26 frames per second (fps) on GPUs, while additionally turning on accurate human pose estimation can be executed at up to 10 fps. △ Less

Submitted 2 August, 2021; originally announced August 2021.

Comments: Accepted by IROS-2021

Report number: 1944

Object-to-Scene: Learning to Transfer Object Knowledge to Indoor Scene Recognition

Authors: Bo Miao, Liguang Zhou, Ajmal Mian, Tin Lun Lam, Yangsheng Xu

Abstract: Accurate perception of the surrounding scene is helpful for robots to make reasonable judgments and behaviours. Therefore, develo** effective scene representation and recognition methods are of significant importance in robotics. Currently, a large body of research focuses on develo** novel auxiliary features and networks to improve indoor scene recognition ability. However, few of them focus… ▽ More Accurate perception of the surrounding scene is helpful for robots to make reasonable judgments and behaviours. Therefore, develo** effective scene representation and recognition methods are of significant importance in robotics. Currently, a large body of research focuses on develo** novel auxiliary features and networks to improve indoor scene recognition ability. However, few of them focus on directly constructing object features and relations for indoor scene recognition. In this paper, we analyze the weaknesses of current methods and propose an Object-to-Scene (OTS) method, which extracts object features and learns object relations to recognize indoor scenes. The proposed OTS first extracts object features based on the segmentation network and the proposed object feature aggregation module (OFAM). Afterwards, the object relations are calculated and the scene representation is constructed based on the proposed object attention module (OAM) and global relation aggregation module (GRAM). The final results in this work show that OTS successfully extracts object features and learns object relations from the segmentation network. Moreover, OTS outperforms the state-of-the-art methods by more than 2\% on indoor scene recognition without using any additional streams. Code is publicly available at: https://github.com/FreeformRobotics/OTS. △ Less

Submitted 1 August, 2021; originally announced August 2021.

Comments: Accepted by IROS2021

BORM: Bayesian Object Relation Model for Indoor Scene Recognition

Authors: Liguang Zhou, Jun Cen, Xingchao Wang, Zhenglong Sun, Tin Lun Lam, Yangsheng Xu

Abstract: Scene recognition is a fundamental task in robotic perception. For human beings, scene recognition is reasonable because they have abundant object knowledge of the real world. The idea of transferring prior object knowledge from humans to scene recognition is significant but still less exploited. In this paper, we propose to utilize meaningful object representations for indoor scene representation… ▽ More Scene recognition is a fundamental task in robotic perception. For human beings, scene recognition is reasonable because they have abundant object knowledge of the real world. The idea of transferring prior object knowledge from humans to scene recognition is significant but still less exploited. In this paper, we propose to utilize meaningful object representations for indoor scene representation. First, we utilize an improved object model (IOM) as a baseline that enriches the object knowledge by introducing a scene parsing algorithm pretrained on the ADE20K dataset with rich object categories related to the indoor scene. To analyze the object co-occurrences and pairwise object relations, we formulate the IOM from a Bayesian perspective as the Bayesian object relation model (BORM). Meanwhile, we incorporate the proposed BORM with the PlacesCNN model as the combined Bayesian object relation model (CBORM) for scene recognition and significantly outperforms the state-of-the-art methods on the reduced Places365 dataset, and SUN RGB-D dataset without retraining, showing the excellent generalization ability of the proposed method. Code can be found at https://github.com/hszhoushen/borm. △ Less

Submitted 1 August, 2021; originally announced August 2021.

Comments: 8 pages, 5 figures, conference, Accepted by IROS2021

Journal ref: IROS2021

All-sky search for long-duration gravitational-wave bursts in the third Advanced LIGO and Advanced Virgo run

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, T. D. Abbott, F. Acernese, K. Ackley, C. Adams, N. Adhikari, R. X. Adhikari, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, K. Agatsuma, N. Aggarwal, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, A. Allocca, P. A. Altin, A. Amato , et al. (1605 additional authors not shown)

Abstract: After the detection of gravitational waves from compact binary coalescences, the search for transient gravitational-wave signals with less well-defined waveforms for which matched filtering is not well-suited is one of the frontiers for gravitational-wave astronomy. Broadly classified into "short" $ \lesssim 1~$\,s and "long" $ \gtrsim 1~$\,s duration signals, these signals are expected from a var… ▽ More After the detection of gravitational waves from compact binary coalescences, the search for transient gravitational-wave signals with less well-defined waveforms for which matched filtering is not well-suited is one of the frontiers for gravitational-wave astronomy. Broadly classified into "short" $ \lesssim 1~$\,s and "long" $ \gtrsim 1~$\,s duration signals, these signals are expected from a variety of astrophysical processes, including non-axisymmetric deformations in magnetars or eccentric binary black hole coalescences. In this work, we present a search for long-duration gravitational-wave transients from Advanced LIGO and Advanced Virgo's third observing run from April 2019 to March 2020. For this search, we use minimal assumptions for the sky location, event time, waveform morphology, and duration of the source. The search covers the range of $2~\text{--}~ 500$~s in duration and a frequency band of $24 - 2048$ Hz. We find no significant triggers within this parameter space; we report sensitivity limits on the signal strength of gravitational waves characterized by the root-sum-square amplitude $h_{\mathrm{rss}}$ as a function of waveform morphology. These $h_{\mathrm{rss}}$ limits improve upon the results from the second observing run by an average factor of 1.8. △ Less

Submitted 29 July, 2021; originally announced July 2021.

Report number: P2100063