Showing 51–100 of 3,591 results for author: Wang, T

Unsupervised Bayesian Generation of Synthetic CT from CBCT Using Patient-Specific Score-Based Prior

Authors: Junbo Peng, Yuan Gao, Chih-Wei Chang, Richard Qiu, Tonghe Wang, Aparna Kesarwala, Kailin Yang, Jacob Scott, David Yu, Xiaofeng Yang

Abstract: Background: Cone-beam computed tomography (CBCT) scans, performed fractionally (e.g., daily or weekly), are widely utilized for patient alignment in the image-guided radiotherapy (IGRT) process, thereby making it a potential imaging modality for the implementation of adaptive radiotherapy (ART) protocols. Nonetheless, significant artifacts and incorrect Hounsfield unit (HU) values hinder their app… ▽ More Background: Cone-beam computed tomography (CBCT) scans, performed fractionally (e.g., daily or weekly), are widely utilized for patient alignment in the image-guided radiotherapy (IGRT) process, thereby making it a potential imaging modality for the implementation of adaptive radiotherapy (ART) protocols. Nonetheless, significant artifacts and incorrect Hounsfield unit (HU) values hinder their application in quantitative tasks such as target and organ segmentations and dose calculation. Therefore, acquiring CT-quality images from the CBCT scans is essential to implement online ART in clinical settings. Purpose: This work aims to develop an unsupervised learning method using the patient-specific diffusion model for CBCT-based synthetic CT (sCT) generation to improve the image quality of CBCT. Methods: The proposed method is in an unsupervised framework that utilizes a patient-specific score-based model as the image prior alongside a customized total variation (TV) regularization to enforce coherence across different transverse slices. The score-based model is unconditionally trained using the same patient's planning CT (pCT) images to characterize the manifold of CT-quality images and capture the unique anatomical information of the specific patient. The efficacy of the proposed method was assessed on images from anatomical sites including head and neck (H&N) cancer, pancreatic cancer, and lung cancer. The performance of the proposed CBCT correction method was evaluated using quantitative metrics including mean absolute error (MAE), peak signal-to-noise ratio (PSNR), and normalized cross-correlation (NCC). Additionally, the proposed algorithm was benchmarked against two other unsupervised diffusion model-based CBCT correction algorithms. △ Less

Submitted 21 June, 2024; originally announced June 2024.

Search for the $e^+e^- \to φχ_{c1}(3872)$ process at BESIII

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere , et al. (639 additional authors not shown)

Abstract: Based on 368.5 pb$^{-1}$ of $e^+e^-$ collision data collected at center-of-mass energies 4.914 and 4.946 GeV by the BESIII detector, the $e^+e^- \to φχ_{c1}(3872)$ process is searched for the first time. No significant signal is observed and the upper limits at the 90\% confidence level on the product of the Born cross section $σ(e^+e^- \to φχ_{c1}(3872))$ and the branching fraction… ▽ More Based on 368.5 pb$^{-1}$ of $e^+e^-$ collision data collected at center-of-mass energies 4.914 and 4.946 GeV by the BESIII detector, the $e^+e^- \to φχ_{c1}(3872)$ process is searched for the first time. No significant signal is observed and the upper limits at the 90\% confidence level on the product of the Born cross section $σ(e^+e^- \to φχ_{c1}(3872))$ and the branching fraction $\mathcal{B}[χ_{c1}(3872)\toπ^+π^- J/ψ]$ at 4.914 and 4.946 GeV are set to be 0.85 and 0.96 pb, respectively. These measurements provide useful information for the production of the $χ_{c1}(3872)$ at $e^+e^-$ collider and deepen our understanding about the nature of this particle. △ Less

Submitted 21 June, 2024; originally announced June 2024.

Comments: 11 pages, 3 figures

CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamics

Authors: Jiawei Gao, Ziqin Wang, Zeqi Xiao, **gbo Wang, Tai Wang, **kun Cao, Xiaolin Hu, Si Liu, Jifeng Dai, Jiangmiao Pang

Abstract: Recent years have seen significant advancements in humanoid control, largely due to the availability of large-scale motion capture data and the application of reinforcement learning methodologies. However, many real-world tasks, such as moving large and heavy furniture, require multi-character collaboration. Given the scarcity of data on multi-character collaboration and the efficiency challenges… ▽ More Recent years have seen significant advancements in humanoid control, largely due to the availability of large-scale motion capture data and the application of reinforcement learning methodologies. However, many real-world tasks, such as moving large and heavy furniture, require multi-character collaboration. Given the scarcity of data on multi-character collaboration and the efficiency challenges associated with multi-agent learning, these tasks cannot be straightforwardly addressed using training paradigms designed for single-agent scenarios. In this paper, we introduce Cooperative Human-Object Interaction (CooHOI), a novel framework that addresses multi-character objects transporting through a two-phase learning paradigm: individual skill acquisition and subsequent transfer. Initially, a single agent learns to perform tasks using the Adversarial Motion Priors (AMP) framework. Following this, the agent learns to collaborate with others by considering the shared dynamics of the manipulated object during parallel training using Multi Agent Proximal Policy Optimization (MAPPO). When one agent interacts with the object, resulting in specific object dynamics changes, the other agents learn to respond appropriately, thereby achieving implicit communication and coordination between teammates. Unlike previous approaches that relied on tracking-based methods for multi-character HOI, CooHOI is inherently efficient, does not depend on motion capture data of multi-character interactions, and can be seamlessly extended to include more participants and a wide range of object types △ Less

Submitted 20 June, 2024; originally announced June 2024.

Improving Expert Radiology Report Summarization by Prompting Large Language Models with a Layperson Summary

Authors: Xingmeng Zhao, Tongnian Wang, Anthony Rios

Abstract: Radiology report summarization (RRS) is crucial for patient care, requiring concise "Impressions" from detailed "Findings." This paper introduces a novel prompting strategy to enhance RRS by first generating a layperson summary. This approach normalizes key observations and simplifies complex information using non-expert communication techniques inspired by doctor-patient interactions. Combined wi… ▽ More Radiology report summarization (RRS) is crucial for patient care, requiring concise "Impressions" from detailed "Findings." This paper introduces a novel prompting strategy to enhance RRS by first generating a layperson summary. This approach normalizes key observations and simplifies complex information using non-expert communication techniques inspired by doctor-patient interactions. Combined with few-shot in-context learning, this method improves the model's ability to link general terms to specific findings. We evaluate this approach on the MIMIC-CXR, CheXpert, and MIMIC-III datasets, benchmarking it against 7B/8B parameter state-of-the-art open-source large language models (LLMs) like Meta-Llama-3-8B-Instruct. Our results demonstrate improvements in summarization accuracy and accessibility, particularly in out-of-domain tests, with improvements as high as 5% for some metrics. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Achieving Cooling Without Repump Lasers Through Ion Motional Heating

Authors: Yue Xiao, Yongxu Peng, Linfeng Chen, Chunhui Li, Zongao Song, Xin Wang, Tao Wang, Yurun Xie, Bin Zhao, Tiangang Yang

Abstract: Laser cooling typically requires one or more repump lasers to clear dark states and enable recycling transitions. Here, we have achieved cooling of Be+ ions using a single laser beam, facilitated by one-dimensional heating through micromotion. By manipulating the displacement from the trap's nodal line, we precisely controlled the ion micromotion direction and speed, reaching up to 3144 m/s, which… ▽ More Laser cooling typically requires one or more repump lasers to clear dark states and enable recycling transitions. Here, we have achieved cooling of Be+ ions using a single laser beam, facilitated by one-dimensional heating through micromotion. By manipulating the displacement from the trap's nodal line, we precisely controlled the ion micromotion direction and speed, reaching up to 3144 m/s, which corresponds to a 7.1 GHz Doppler frequency shift in our experiment. This approach eliminates the necessity of a 1.25 GHz offset repump laser while kee** the Be+ ions cold in the perpendicular direction. Measurements were taken using cooling laser detuning and imaging of ion trajectories. Molecular dynamics simulations, based on machine learned time-dependent electric field E(X, Y, Z, t) inside the trap, accurately reproduced the experimental observation, illuminating the relationship between the direction of micromotion and the trap** electric filed vector. This work not only provides a robust method for managing the micromotion velocity of ions but also sheds light on laser cooling complex systems that require multiple repum** lasers. Additionally, it offers a method for controlling energy in the context of ion-molecule collision investigations. △ Less

Submitted 20 June, 2024; originally announced June 2024.

PIN: A Knowledge-Intensive Dataset for Paired and Interleaved Multimodal Documents

Authors: Junjie Wang, Yin Zhang, Yatai Ji, Yuxiang Zhang, Chunyang Jiang, Yubo Wang, Kang Zhu, Zekun Wang, Tiezhen Wang, Wenhao Huang, Jie Fu, Bei Chen, Qunshu Lin, Minghao Liu, Ge Zhang, Wenhu Chen

Abstract: Recent advancements in Large Multimodal Models (LMMs) have leveraged extensive multimodal datasets to enhance capabilities in complex knowledge-driven tasks. However, persistent challenges in perceptual and reasoning errors limit their efficacy, particularly in interpreting intricate visual data and deducing multimodal relationships. Addressing these issues, we introduce a novel dataset format, PI… ▽ More Recent advancements in Large Multimodal Models (LMMs) have leveraged extensive multimodal datasets to enhance capabilities in complex knowledge-driven tasks. However, persistent challenges in perceptual and reasoning errors limit their efficacy, particularly in interpreting intricate visual data and deducing multimodal relationships. Addressing these issues, we introduce a novel dataset format, PIN (Paired and INterleaved multimodal documents), designed to significantly improve both the depth and breadth of multimodal training. The PIN format is built on three foundational principles: knowledge intensity, scalability, and support for diverse training modalities. This innovative format combines markdown files and comprehensive images to enrich training data with a dense knowledge structure and versatile training strategies. We present PIN-14M, an open-source dataset comprising 14 million samples derived from a diverse range of Chinese and English sources, tailored to include complex web and scientific content. This dataset is constructed meticulously to ensure data quality and ethical integrity, aiming to facilitate advanced training strategies and improve model robustness against common multimodal training pitfalls. Our initial results, forming the basis of this technical report, suggest significant potential for the PIN format in refining LMM performance, with plans for future expansions and detailed evaluations of its impact on model capabilities. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Through the Theory of Mind's Eye: Reading Minds with Multimodal Video Large Language Models

Authors: Zhawnen Chen, Tianchun Wang, Yizhou Wang, Michal Kosinski, Xiang Zhang, Yun Fu, Sheng Li

Abstract: Can large multimodal models have a human-like ability for emotional and social reasoning, and if so, how does it work? Recent research has discovered emergent theory-of-mind (ToM) reasoning capabilities in large language models (LLMs). LLMs can reason about people's mental states by solving various text-based ToM tasks that ask questions about the actors' ToM (e.g., human belief, desire, intention… ▽ More Can large multimodal models have a human-like ability for emotional and social reasoning, and if so, how does it work? Recent research has discovered emergent theory-of-mind (ToM) reasoning capabilities in large language models (LLMs). LLMs can reason about people's mental states by solving various text-based ToM tasks that ask questions about the actors' ToM (e.g., human belief, desire, intention). However, human reasoning in the wild is often grounded in dynamic scenes across time. Thus, we consider videos a new medium for examining spatio-temporal ToM reasoning ability. Specifically, we ask explicit probing questions about videos with abundant social and emotional reasoning content. We develop a pipeline for multimodal LLM for ToM reasoning using video and text. We also enable explicit ToM reasoning by retrieving key frames for answering a ToM question, which reveals how multimodal LLMs reason about ToM. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Insights into the Production of $^{44}$Ti and Nickel Isotopes in Core-Collapse Supernovae

Authors: Tianshu Wang, Adam Burrows

Abstract: We report nucleosynthetic results for both $^{44}$Ti and nickel isotopes for eighteen three-dimensional (3D) core-collapse supernova (CCSN) simulations extended to $\sim$20 seconds after bounce. We find that many of our long-term models are able to achieve $^{44}$Ti/$^{56}$Ni ratios similar to that observed in Cassiopeia A, and modern supernova models can synthesize up to $2\times10^{-4}M_\odot$ o… ▽ More We report nucleosynthetic results for both $^{44}$Ti and nickel isotopes for eighteen three-dimensional (3D) core-collapse supernova (CCSN) simulations extended to $\sim$20 seconds after bounce. We find that many of our long-term models are able to achieve $^{44}$Ti/$^{56}$Ni ratios similar to that observed in Cassiopeia A, and modern supernova models can synthesize up to $2\times10^{-4}M_\odot$ of $^{44}$Ti. Neutrino-driven winds and the fact that there can be simultaneous accretion and explosion in 3D models of core-collapse supernovae play central roles in its production. We conclude that the $^{44}$Ti underproduction problem in previous CCSN models is no longer an issue. In addition, we discuss the production of both $^{57}$Ni and stable nickel/iron ratios and compare our results to observations of SN1987A and the Crab. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Comments: 15 pages, 7 figures. Submitted to ApJ

ABNet: Attention BarrierNet for Safe and Scalable Robot Learning

Authors: Wei Xiao, Tsun-Hsuan Wang, Daniela Rus

Abstract: Safe learning is central to AI-enabled robots where a single failure may lead to catastrophic results. Barrier-based method is one of the dominant approaches for safe robot learning. However, this method is not scalable, hard to train, and tends to generate unstable signals under noisy inputs that are challenging to be deployed for robots. To address these challenges, we propose a novel Attentio… ▽ More Safe learning is central to AI-enabled robots where a single failure may lead to catastrophic results. Barrier-based method is one of the dominant approaches for safe robot learning. However, this method is not scalable, hard to train, and tends to generate unstable signals under noisy inputs that are challenging to be deployed for robots. To address these challenges, we propose a novel Attention BarrierNet (ABNet) that is scalable to build larger foundational safe models in an incremental manner. Each head of BarrierNet in the ABNet could learn safe robot control policies from different features and focus on specific part of the observation. In this way, we do not need to one-shotly construct a large model for complex tasks, which significantly facilitates the training of the model while ensuring its stable output. Most importantly, we can still formally prove the safety guarantees of the ABNet. We demonstrate the strength of ABNet in 2D robot obstacle avoidance, safe robot manipulation, and vision-based end-to-end autonomous driving, with results showing much better robustness and guarantees over existing models. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Comments: 18 pages

Can Go AIs be adversarially robust?

Authors: Tom Tseng, Euan McLean, Kellin Pelrine, Tony T. Wang, Adam Gleave

Abstract: Prior work found that superhuman Go AIs like KataGo can be defeated by simple adversarial strategies. In this paper, we study if simple defenses can improve KataGo's worst-case performance. We test three natural defenses: adversarial training on hand-constructed positions, iterated adversarial training, and changing the network architecture. We find that some of these defenses are able to protect… ▽ More Prior work found that superhuman Go AIs like KataGo can be defeated by simple adversarial strategies. In this paper, we study if simple defenses can improve KataGo's worst-case performance. We test three natural defenses: adversarial training on hand-constructed positions, iterated adversarial training, and changing the network architecture. We find that some of these defenses are able to protect against previously discovered attacks. Unfortunately, we also find that none of these defenses are able to withstand adaptive attacks. In particular, we are able to train new adversaries that reliably defeat our defended agents by causing them to blunder in ways humans would not. Our results suggest that building robust AI systems is challenging even in narrow domains such as Go. For interactive examples of attacks and a link to our codebase, see https://goattack.far.ai. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Comments: 67 pages

BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversity

Authors: Zahra Gharaee, Scott C. Lowe, ZeMing Gong, Pablo Millan Arias, Nicholas Pellegrino, Austin T. Wang, Joakim Bruslund Haurum, Iuliia Zarubiieva, Lila Kari, Dirk Steinke, Graham W. Taylor, Paul Fieguth, Angel X. Chang

Abstract: As part of an ongoing worldwide effort to comprehend and monitor insect biodiversity, this paper presents the BIOSCAN-5M Insect dataset to the machine learning community and establish several benchmark tasks. BIOSCAN-5M is a comprehensive dataset containing multi-modal information for over 5 million insect specimens, and it significantly expands existing image-based biological datasets by includin… ▽ More As part of an ongoing worldwide effort to comprehend and monitor insect biodiversity, this paper presents the BIOSCAN-5M Insect dataset to the machine learning community and establish several benchmark tasks. BIOSCAN-5M is a comprehensive dataset containing multi-modal information for over 5 million insect specimens, and it significantly expands existing image-based biological datasets by including taxonomic labels, raw nucleotide barcode sequences, assigned barcode index numbers, and geographical information. We propose three benchmark experiments to demonstrate the impact of the multi-modal data types on the classification and clustering accuracy. First, we pretrain a masked language model on the DNA barcode sequences of the BIOSCAN-5M dataset, and demonstrate the impact of using this large reference library on species- and genus-level classification performance. Second, we propose a zero-shot transfer learning task applied to images and DNA barcodes to cluster feature embeddings obtained from self-supervised learning, to investigate whether meaningful clusters can be derived from these representation embeddings. Third, we benchmark multi-modality by performing contrastive learning on DNA barcodes, image data, and taxonomic information. This yields a general shared embedding space enabling taxonomic classification using multiple types of information and modalities. The code repository of the BIOSCAN-5M Insect dataset is available at https://github.com/zahrag/BIOSCAN-5M. △ Less

Submitted 24 June, 2024; v1 submitted 18 June, 2024; originally announced June 2024.

Minimal surfaces with low genus in lens spaces

Authors: Xingzhe Li, Tongrui Wang, Xuan Yao

Abstract: Given a Riemannian $\mathbb{RP}^3$ with a bumpy metric or a metric of positive Ricci curvature, we show that there either exist four distinct minimal real projective planes, or exist one minimal real projective plane together with two distinct minimal $2$-spheres. Our proof is based on a variant multiplicity one theorem for the Simon-Smith min-max theory under certain equivariant settings. In part… ▽ More Given a Riemannian $\mathbb{RP}^3$ with a bumpy metric or a metric of positive Ricci curvature, we show that there either exist four distinct minimal real projective planes, or exist one minimal real projective plane together with two distinct minimal $2$-spheres. Our proof is based on a variant multiplicity one theorem for the Simon-Smith min-max theory under certain equivariant settings. In particular, we show under the positive Ricci assumption that $\mathbb{RP}^3$ contains at least four distinct minimal real projective planes and four distinct minimal tori. Additionally, the number of minimal tori can be improved to five for a generic positive Ricci metric on $\mathbb{RP}^3$ by the degree method. Moreover, using the same strategy, we show that in the lens space $L(4m,2m\pm 1)$, $m\geq 1$, with a bumpy metric or a metric of positive Ricci curvature, there either exist $N(m)$ numbers of distinct minimal Klein bottles, or exist one minimal Klein bottle and three distinct minimal $2$-spheres, where $N(1)=4$, $N(m)=2$ for $m\geq 2$, and the first case happens under the positive Ricci assumption. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Comments: 34 pages, comments are welcome!

MSC Class: 53A10; 53C42

Cheeger type inequalities associated with isocapacitary constants on graphs

Authors: Bobo Hua, Florentin Münch, Tao Wang

Abstract: In this paper, we introduce Cheeger type constants via isocapacitary constants introduced by May'za to estimate first Dirichlet, Neumann and Steklov eigenvalues on a finite subgraph of a graph. Moreover, we estimate the bottom of the spectrum of the Laplace operator and the Dirichlet-to-Neumann operator for an infinite subgraph. Estimates for higher-order Steklov eigenvalues on a finite or infinit… ▽ More In this paper, we introduce Cheeger type constants via isocapacitary constants introduced by May'za to estimate first Dirichlet, Neumann and Steklov eigenvalues on a finite subgraph of a graph. Moreover, we estimate the bottom of the spectrum of the Laplace operator and the Dirichlet-to-Neumann operator for an infinite subgraph. Estimates for higher-order Steklov eigenvalues on a finite or infinite subgraph are also proved. △ Less

Submitted 18 June, 2024; originally announced June 2024.

MSC Class: 53A70; 05C50; 15A42; 39A12

Quantum Compiling with Reinforcement Learning on a Superconducting Processor

Authors: Z. T. Wang, Qiuhao Chen, Yuxuan Du, Z. H. Yang, Xiaoxia Cai, Kaixuan Huang, **gning Zhang, Kai Xu, Jun Du, Yinan Li, Yuling Jiao, Xingyao Wu, Wu Liu, Xiliang Lu, Huikai Xu, Yirong **, Ruixia Wang, Haifeng Yu, S. P. Zhao

Abstract: To effectively implement quantum algorithms on noisy intermediate-scale quantum (NISQ) processors is a central task in modern quantum technology. NISQ processors feature tens to a few hundreds of noisy qubits with limited coherence times and gate operations with errors, so NISQ algorithms naturally require employing circuits of short lengths via quantum compilation. Here, we develop a reinforcemen… ▽ More To effectively implement quantum algorithms on noisy intermediate-scale quantum (NISQ) processors is a central task in modern quantum technology. NISQ processors feature tens to a few hundreds of noisy qubits with limited coherence times and gate operations with errors, so NISQ algorithms naturally require employing circuits of short lengths via quantum compilation. Here, we develop a reinforcement learning (RL)-based quantum compiler for a superconducting processor and demonstrate its capability of discovering novel and hardware-amenable circuits with short lengths. We show that for the three-qubit quantum Fourier transformation, a compiled circuit using only seven CZ gates with unity circuit fidelity can be achieved. The compiler is also able to find optimal circuits under device topological constraints, with lengths considerably shorter than those by the conventional method. Our study exemplifies the codesign of the software with hardware for efficient quantum compilation, offering valuable insights for the advancement of RL-based compilers. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Quantized Andreev conductance in semiconductor nanowires

Authors: Yichun Gao, Wenyu Song, Yuhao Wang, Zuhan Geng, Zhan Cao, Zehao Yu, Shuai Yang, Jiaye Xu, Fangting Chen, Zonglin Li, Ruidong Li, Lining Yang, Zhaoyu Wang, Shan Zhang, Xiao Feng, Tiantian Wang, Yunyi Zang, Lin Li, Dong E. Liu, Runan Shang, Qi-Kun Xue, Ke He, Hao Zhang

Abstract: Clean one-dimensional electron systems can exhibit quantized conductance. The plateau conductance doubles if the transport is dominated by Andreev reflection. Here, we report quantized conductance observed in both Andreev and normal-state transports in PbTe-Pb and PbTe-In hybrid nanowires. The Andreev plateau is observed at $4e^2/h$, twice of the normal plateau value of $2e^2/h$. In comparison, An… ▽ More Clean one-dimensional electron systems can exhibit quantized conductance. The plateau conductance doubles if the transport is dominated by Andreev reflection. Here, we report quantized conductance observed in both Andreev and normal-state transports in PbTe-Pb and PbTe-In hybrid nanowires. The Andreev plateau is observed at $4e^2/h$, twice of the normal plateau value of $2e^2/h$. In comparison, Andreev conductance in the best-optimized III-V nanowires is non-quantized due to mode-mixing induced dips (a disorder effect), despite the quantization of normal-state transport. The negligible mode mixing in PbTe hybrids indicates an unprecedented low-disorder transport regime for nanowire devices, beneficial for Majorana researches. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Consistency^2: Consistent and Fast 3D Painting with Latent Consistency Models

Authors: Tianfu Wang, Anton Obukhov, Konrad Schindler

Abstract: Generative 3D Painting is among the top productivity boosters in high-resolution 3D asset management and recycling. Ever since text-to-image models became accessible for inference on consumer hardware, the performance of 3D Painting methods has consistently improved and is currently close to plateauing. At the core of most such models lies denoising diffusion in the latent space, an inherently tim… ▽ More Generative 3D Painting is among the top productivity boosters in high-resolution 3D asset management and recycling. Ever since text-to-image models became accessible for inference on consumer hardware, the performance of 3D Painting methods has consistently improved and is currently close to plateauing. At the core of most such models lies denoising diffusion in the latent space, an inherently time-consuming iterative process. Multiple techniques have been developed recently to accelerate generation and reduce sampling iterations by orders of magnitude. Designed for 2D generative imaging, these techniques do not come with recipes for lifting them into 3D. In this paper, we address this shortcoming by proposing a Latent Consistency Model (LCM) adaptation for the task at hand. We analyze the strengths and weaknesses of the proposed model and evaluate it quantitatively and qualitatively. Based on the Objaverse dataset samples study, our 3D painting method attains strong preference in all evaluations. Source code is available at https://github.com/kongdai123/consistency2. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Data Shapley in One Training Run

Authors: Jiachen T. Wang, Prateek Mittal, Dawn Song, Ruoxi Jia

Abstract: Data Shapley provides a principled framework for attributing data's contribution within machine learning contexts. However, existing approaches require re-training models on different data subsets, which is computationally intensive, foreclosing their application to large-scale models. Furthermore, they produce the same attribution score for any models produced by running the learning algorithm, m… ▽ More Data Shapley provides a principled framework for attributing data's contribution within machine learning contexts. However, existing approaches require re-training models on different data subsets, which is computationally intensive, foreclosing their application to large-scale models. Furthermore, they produce the same attribution score for any models produced by running the learning algorithm, meaning they cannot perform targeted attribution towards a specific model obtained from a single run of the algorithm. This paper introduces In-Run Data Shapley, which addresses these limitations by offering scalable data attribution for a target model of interest. In its most efficient implementation, our technique incurs negligible additional runtime compared to standard model training. This dramatic efficiency improvement makes it possible to perform data attribution for the foundation model pretraining stage for the first time. We present several case studies that offer fresh insights into pretraining data's contribution and discuss their implications for copyright in generative AI and pretraining data curation. △ Less

Submitted 29 June, 2024; v1 submitted 16 June, 2024; originally announced June 2024.

Causal feedback strategies for controlled stochastic Volterra systems: a unified treatment

Authors: Jiayin Gong, Tianxiao Wang

Abstract: This paper is concerned with a unified treatment of linear quadratic control problem for stochastic Volterra integral equations (SVIEs), motivated by the various approaches and scattered results in the existing literature. A novel class of optimal causal feedback strategy is introduced and characterized by means of a new Riccati system. To this end, a fundamental function space and an appropriate… ▽ More This paper is concerned with a unified treatment of linear quadratic control problem for stochastic Volterra integral equations (SVIEs), motivated by the various approaches and scattered results in the existing literature. A novel class of optimal causal feedback strategy is introduced and characterized by means of a new Riccati system. To this end, a fundamental function space and an appropriate multiplicative rule among functions are defined for the first time. In contrast with the existing works, our unified treatment not only provides a new approach, but also extends or improves the known conclusions in stochastic differential equations, convolution SVIEs, stochastic Volterra integro-differential equations (VIDEs), deterministic VIEs, deterministic VIDEs. In addition, an interesting phenomenon is reveal by the current study: for SVIEs the conventional structure of state feedback is replaced by a suitable causal form, and the original state process no longer plays indispensable role in the feedbacks while an auxiliary state process does. △ Less

Submitted 16 June, 2024; originally announced June 2024.

MINT: a Multi-modal Image and Narrative Text Dubbing Dataset for Foley Audio Content Planning and Generation

Authors: Ruibo Fu, Shuchen Shi, Hongming Guo, Tao Wang, Chunyu Qiang, Zhengqi Wen, Jianhua Tao, Xin Qi, Yi Lu, Xiaopeng Wang, Zhiyong Wang, Yukun Liu, Xuefei Liu, Shuai Zhang, Guanjun Li

Abstract: Foley audio, critical for enhancing the immersive experience in multimedia content, faces significant challenges in the AI-generated content (AIGC) landscape. Despite advancements in AIGC technologies for text and image generation, the foley audio dubbing remains rudimentary due to difficulties in cross-modal scene matching and content correlation. Current text-to-audio technology, which relies on… ▽ More Foley audio, critical for enhancing the immersive experience in multimedia content, faces significant challenges in the AI-generated content (AIGC) landscape. Despite advancements in AIGC technologies for text and image generation, the foley audio dubbing remains rudimentary due to difficulties in cross-modal scene matching and content correlation. Current text-to-audio technology, which relies on detailed and acoustically relevant textual descriptions, falls short in practical video dubbing applications. Existing datasets like AudioSet, AudioCaps, Clotho, Sound-of-Story, and WavCaps do not fully meet the requirements for real-world foley audio dubbing task. To address this, we introduce the Multi-modal Image and Narrative Text Dubbing Dataset (MINT), designed to enhance mainstream dubbing tasks such as literary story audiobooks dubbing, image/silent video dubbing. Besides, to address the limitations of existing TTA technology in understanding and planning complex prompts, a Foley Audio Content Planning, Generation, and Alignment (CPGA) framework is proposed, which includes a content planning module leveraging large language models for complex multi-modal prompts comprehension. Additionally, the training process is optimized using Proximal Policy Optimization based reinforcement learning, significantly improving the alignment and auditory realism of generated foley audio. Experimental results demonstrate that our approach significantly advances the field of foley audio dubbing, providing robust solutions for the challenges of multi-modal dubbing. Even when utilizing the relatively lightweight GPT-2 model, our framework outperforms open-source multimodal large models such as LLaVA, DeepSeek-VL, and Moondream2. The dataset is available at https://github.com/borisfrb/MINT . △ Less

Submitted 15 June, 2024; originally announced June 2024.

One-pass Multiple Conformer and Foundation Speech Systems Compression and Quantization Using An All-in-one Neural Model

Authors: Zhaoqing Li, Haoning Xu, Tianzi Wang, Shoukang Hu, Zengrui **, Shujie Hu, Jiajun Deng, Mingyu Cui, Mengzhe Geng, Xunying Liu

Abstract: We propose a novel one-pass multiple ASR systems joint compression and quantization approach using an all-in-one neural model. A single compression cycle allows multiple nested systems with varying Encoder depths, widths, and quantization precision settings to be simultaneously constructed without the need to train and store individual target systems separately. Experiments consistently demonstrat… ▽ More We propose a novel one-pass multiple ASR systems joint compression and quantization approach using an all-in-one neural model. A single compression cycle allows multiple nested systems with varying Encoder depths, widths, and quantization precision settings to be simultaneously constructed without the need to train and store individual target systems separately. Experiments consistently demonstrate the multiple ASR systems compressed in a single all-in-one model produced a word error rate (WER) comparable to, or lower by up to 1.01\% absolute (6.98\% relative) than individually trained systems of equal complexity. A 3.4x overall system compression and training time speed-up was achieved. Maximum model size compression ratios of 12.8x and 3.93x were obtained over the baseline Switchboard-300hr Conformer and LibriSpeech-100hr fine-tuned wav2vec2.0 models, respectively, incurring no statistically significant WER increase. △ Less

Submitted 14 June, 2024; originally announced June 2024.

Comments: Accepted by Interspeech 2024

Joint Speaker Features Learning for Audio-visual Multichannel Speech Separation and Recognition

Authors: Guinan Li, Jiajun Deng, Youjun Chen, Mengzhe Geng, Shujie Hu, Zhe Li, Zengrui **, Tianzi Wang, Xurong Xie, Helen Meng, Xunying Liu

Abstract: This paper proposes joint speaker feature learning methods for zero-shot adaptation of audio-visual multichannel speech separation and recognition systems. xVector and ECAPA-TDNN speaker encoders are connected using purpose-built fusion blocks and tightly integrated with the complete system training. Experiments conducted on LRS3-TED data simulated multichannel overlapped speech suggest that joint… ▽ More This paper proposes joint speaker feature learning methods for zero-shot adaptation of audio-visual multichannel speech separation and recognition systems. xVector and ECAPA-TDNN speaker encoders are connected using purpose-built fusion blocks and tightly integrated with the complete system training. Experiments conducted on LRS3-TED data simulated multichannel overlapped speech suggest that joint speaker feature learning consistently improves speech separation and recognition performance over the baselines without joint speaker feature estimation. Further analyses reveal performance improvements are strongly correlated with increased inter-speaker discrimination measured using cosine similarity. The best-performing joint speaker feature learning adapted system outperformed the baseline fine-tuned WavLM model by statistically significant WER reductions of 21.6% and 25.3% absolute (67.5% and 83.5% relative) on Dev and Test sets after incorporating WavLM features and video modality. △ Less

Submitted 14 June, 2024; originally announced June 2024.

Comments: Accepted by Interspeech 2024

SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understanding

Authors: Junwei Luo, Zhen Pang, Yongjun Zhang, Tingzhu Wang, Linlin Wang, Bo Dang, Jiangwei Lao, Jian Wang, **gdong Chen, Yihua Tan, Yansheng Li

Abstract: Remote Sensing Large Multi-Modal Models (RSLMMs) are develo** rapidly and showcase significant capabilities in remote sensing imagery (RSI) comprehension. However, due to the limitations of existing datasets, RSLMMs have shortcomings in understanding the rich semantic relations among objects in complex remote sensing scenes. To unlock RSLMMs' complex comprehension ability, we propose a large-sca… ▽ More Remote Sensing Large Multi-Modal Models (RSLMMs) are develo** rapidly and showcase significant capabilities in remote sensing imagery (RSI) comprehension. However, due to the limitations of existing datasets, RSLMMs have shortcomings in understanding the rich semantic relations among objects in complex remote sensing scenes. To unlock RSLMMs' complex comprehension ability, we propose a large-scale instruction tuning dataset FIT-RS, containing 1,800,851 instruction samples. FIT-RS covers common interpretation tasks and innovatively introduces several complex comprehension tasks of escalating difficulty, ranging from relation reasoning to image-level scene graph generation. Based on FIT-RS, we build the FIT-RSFG benchmark. Furthermore, we establish a new benchmark to evaluate the fine-grained relation comprehension capabilities of LMMs, named FIT-RSRC. Based on combined instruction data, we propose SkySenseGPT, which achieves outstanding performance on both public datasets and FIT-RSFG, surpassing existing RSLMMs. We hope the FIT-RS dataset can enhance the relation comprehension capability of RSLMMs and provide a large-scale fine-grained data source for the remote sensing community. The dataset will be available at https://github.com/Luo-Z13/SkySenseGPT △ Less

Submitted 8 July, 2024; v1 submitted 14 June, 2024; originally announced June 2024.

Comments: 30 pages, 5 figures, 19 tables, dataset and code see https://github.com/Luo-Z13/SkySenseGPT

Towards Effective and Efficient Non-autoregressive Decoding Using Block-based Attention Mask

Authors: Tianzi Wang, Xurong Xie, Zhaoqing Li, Shoukang Hu, Zengrui **g, Jiajun Deng, Mingyu Cui, Shujie Hu, Mengzhe Geng, Guinan Li, Helen Meng, Xunying Liu

Abstract: This paper proposes a novel non-autoregressive (NAR) block-based Attention Mask Decoder (AMD) that flexibly balances performance-efficiency trade-offs for Conformer ASR systems. AMD performs parallel NAR inference within contiguous blocks of output labels that are concealed using attention masks, while conducting left-to-right AR prediction and history context amalgamation between blocks. A beam s… ▽ More This paper proposes a novel non-autoregressive (NAR) block-based Attention Mask Decoder (AMD) that flexibly balances performance-efficiency trade-offs for Conformer ASR systems. AMD performs parallel NAR inference within contiguous blocks of output labels that are concealed using attention masks, while conducting left-to-right AR prediction and history context amalgamation between blocks. A beam search algorithm is designed to leverage a dynamic fusion of CTC, AR Decoder, and AMD probabilities. Experiments on the LibriSpeech-100hr corpus suggest the tripartite Decoder incorporating the AMD module produces a maximum decoding speed-up ratio of 1.73x over the baseline CTC+AR decoding, while incurring no statistically significant word error rate (WER) increase on the test sets. When operating with the same decoding real time factors, statistically significant WER reductions of up to 0.7% and 0.3% absolute (5.3% and 6.1% relative) were obtained over the CTC+AR baseline. △ Less

Submitted 14 June, 2024; originally announced June 2024.

Comments: 5 pages, 2 figures, 2 tables, Interspeech24 conference

ALMA Lensing Cluster Survey: Physical characterization of near-infrared-dark intrinsically faint ALMA sources at z=2-4

Authors: Akiyoshi Tsujita, Kotaro Kohno, Shuo Huang, Masamune Oguri, Ken-ichi Tadaki, Ian Smail, Hideki Umehata, Zhen-Kai Gao, Wei-Hao Wang, Fengwu Sun, Seiji Fujimoto, Tao Wang, Ryosuke Uematsu, Daniel Espada, Francesco Valentino, Yi** Ao, Franz E. Bauer, Bunyo Hatsukade, Fumi Egusa, Yuri Nishimura, Anton M. Koekemoer, Daniel Schaerer, Claudia Lagos, Miroslava Dessauges-Zavadsky, Gabriel Brammer , et al. (11 additional authors not shown)

Abstract: We present results from Atacama Large Millimeter/submillimeter Array (ALMA) spectral line-scan observations at 3-mm and 2-mm bands of three near-infrared-dark (NIR-dark) galaxies behind two massive lensing clusters MACS J0417.5-1154 and RXC J0032.1+1808. Each of these three sources is a faint (de-lensed $S_{\text{1.2 mm}}$ $<$ 1 mJy) triply lensed system originally discovered in the ALMA Lensing C… ▽ More We present results from Atacama Large Millimeter/submillimeter Array (ALMA) spectral line-scan observations at 3-mm and 2-mm bands of three near-infrared-dark (NIR-dark) galaxies behind two massive lensing clusters MACS J0417.5-1154 and RXC J0032.1+1808. Each of these three sources is a faint (de-lensed $S_{\text{1.2 mm}}$ $<$ 1 mJy) triply lensed system originally discovered in the ALMA Lensing Cluster Survey. We have successfully detected CO and [C I] emission lines and confirmed that their spectroscopic redshifts are $z=3.652$, 2.391, and 2.985. By utilizing a rich multi-wavelength data set, we find that the NIR-dark galaxies are located on the star formation main sequence in the intrinsic stellar mass range of log ($M_*$/$M_\odot$) = 9.8 - 10.4, which is about one order of magnitude lower than that of typical submillimeter galaxies (SMGs). These NIR-dark galaxies show a variety in gas depletion times and spatial extent of dust emission. One of the three is a normal star-forming galaxy with gas depletion time consistent with a scaling relation, and its infrared surface brightness is an order of magnitude smaller than that of typical SMGs. Since this galaxy has an elongated axis ratio of $\sim 0.17$, we argue that normal star-forming galaxies in an edge-on configuration can be heavily dust-obscured. This implies that existing deep WFC3/F160W surveys may miss a fraction of typical star-forming main-sequence galaxies due to their edge-on orientation. △ Less

Submitted 14 June, 2024; originally announced June 2024.

Comments: 23 pages, 10 figures, Submitted to ApJ

Perceiver-Prompt: Flexible Speaker Adaptation in Whisper for Chinese Disordered Speech Recognition

Authors: Yicong Jiang, Tianzi Wang, Xurong Xie, Juan Liu, Wei Sun, Nan Yan, Hui Chen, Lan Wang, Xunying Liu, Feng Tian

Abstract: Disordered speech recognition profound implications for improving the quality of life for individuals afflicted with, for example, dysarthria. Dysarthric speech recognition encounters challenges including limited data, substantial dissimilarities between dysarthric and non-dysarthric speakers, and significant speaker variations stemming from the disorder. This paper introduces Perceiver-Prompt, a… ▽ More Disordered speech recognition profound implications for improving the quality of life for individuals afflicted with, for example, dysarthria. Dysarthric speech recognition encounters challenges including limited data, substantial dissimilarities between dysarthric and non-dysarthric speakers, and significant speaker variations stemming from the disorder. This paper introduces Perceiver-Prompt, a method for speaker adaptation that utilizes P-Tuning on the Whisper large-scale model. We first fine-tune Whisper using LoRA and then integrate a trainable Perceiver to generate fixed-length speaker prompts from variable-length inputs, to improve model recognition of Chinese dysarthric speech. Experimental results from our Chinese dysarthric speech dataset demonstrate consistent improvements in recognition performance with Perceiver-Prompt. Relative reduction up to 13.04% in CER is obtained over the fine-tuned Whisper. △ Less

Submitted 14 June, 2024; originally announced June 2024.

Comments: Accepted by interspeech 2024

Watch the Watcher! Backdoor Attacks on Security-Enhancing Diffusion Models

Authors: Changjiang Li, Ren Pang, Bochuan Cao, **ghui Chen, Fenglong Ma, Shouling Ji, Ting Wang

Abstract: Thanks to their remarkable denoising capabilities, diffusion models are increasingly being employed as defensive tools to reinforce the security of other models, notably in purifying adversarial examples and certifying adversarial robustness. However, the security risks of these practices themselves remain largely unexplored, which is highly concerning. To bridge this gap, this work investigates t… ▽ More Thanks to their remarkable denoising capabilities, diffusion models are increasingly being employed as defensive tools to reinforce the security of other models, notably in purifying adversarial examples and certifying adversarial robustness. However, the security risks of these practices themselves remain largely unexplored, which is highly concerning. To bridge this gap, this work investigates the vulnerabilities of security-enhancing diffusion models. Specifically, we demonstrate that these models are highly susceptible to DIFF2, a simple yet effective backdoor attack, which substantially diminishes the security assurance provided by such models. Essentially, DIFF2 achieves this by integrating a malicious diffusion-sampling process into the diffusion model, guiding inputs embedded with specific triggers toward an adversary-defined distribution while preserving the normal functionality for clean inputs. Our case studies on adversarial purification and robustness certification show that DIFF2 can significantly reduce both post-purification and certified accuracy across benchmark datasets and models, highlighting the potential risks of relying on pre-trained diffusion models as defensive tools. We further explore possible countermeasures, suggesting promising avenues for future research. △ Less

Submitted 13 June, 2024; originally announced June 2024.

Search for $X(1870)$ via the decay $J/ψ\to ωK^+ K^-η$

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere , et al. (644 additional authors not shown)

Abstract: Using a sample of $(10087\pm 44)\times10^{6}$ $J/ψ$ events collected by the BESIII detector at the BEPCII collider, we search for the decay $X(1870)\to K^+ K^-η$ via the $J/ψ\to ωK^+ K^- η$ process for the first time. No significant $X(1870)$ signal is observed. The upper limit on the branching fraction of the decay $ J/ψ\to ωX(1870) \toωK^+ K^- η$ is determined to be $9.55\times 10^{-7}$ at the… ▽ More Using a sample of $(10087\pm 44)\times10^{6}$ $J/ψ$ events collected by the BESIII detector at the BEPCII collider, we search for the decay $X(1870)\to K^+ K^-η$ via the $J/ψ\to ωK^+ K^- η$ process for the first time. No significant $X(1870)$ signal is observed. The upper limit on the branching fraction of the decay $ J/ψ\to ωX(1870) \toωK^+ K^- η$ is determined to be $9.55\times 10^{-7}$ at the $90\%$ confidence level. In addition, the branching faction $B(J/ψ\toωK^+ K^- η)$ is measured to be $(3.33\pm0.02(\rm{stat.})\pm 0.12(\rm{syst.}))\times 10^{-4}$. △ Less

Submitted 13 June, 2024; originally announced June 2024.

STAR: A First-Ever Dataset and A Large-Scale Benchmark for Scene Graph Generation in Large-Size Satellite Imagery

Authors: Yansheng Li, Linlin Wang, Tingzhu Wang, Xue Yang, Junwei Luo, Qi Wang, Youming Deng, Wenbin Wang, Xian Sun, Haifeng Li, Bo Dang, Yongjun Zhang, Yi Yu, Junchi Yan

Abstract: Scene graph generation (SGG) in satellite imagery (SAI) benefits promoting understanding of geospatial scenarios from perception to cognition. In SAI, objects exhibit great variations in scales and aspect ratios, and there exist rich relationships between objects (even between spatially disjoint objects), which makes it attractive to holistically conduct SGG in large-size very-high-resolution (VHR… ▽ More Scene graph generation (SGG) in satellite imagery (SAI) benefits promoting understanding of geospatial scenarios from perception to cognition. In SAI, objects exhibit great variations in scales and aspect ratios, and there exist rich relationships between objects (even between spatially disjoint objects), which makes it attractive to holistically conduct SGG in large-size very-high-resolution (VHR) SAI. However, there lack such SGG datasets. Due to the complexity of large-size SAI, mining triplets <subject, relationship, object> heavily relies on long-range contextual reasoning. Consequently, SGG models designed for small-size natural imagery are not directly applicable to large-size SAI. This paper constructs a large-scale dataset for SGG in large-size VHR SAI with image sizes ranging from 512 x 768 to 27,860 x 31,096 pixels, named STAR (Scene graph generaTion in lArge-size satellite imageRy), encompassing over 210K objects and over 400K triplets. To realize SGG in large-size SAI, we propose a context-aware cascade cognition (CAC) framework to understand SAI regarding object detection (OBD), pair pruning and relationship prediction for SGG. We also release a SAI-oriented SGG toolkit with about 30 OBD and 10 SGG methods which need further adaptation by our devised modules on our challenging STAR dataset. The dataset and toolkit are available at: https://linlin-dev.github.io/project/STAR. △ Less

Submitted 3 July, 2024; v1 submitted 13 June, 2024; originally announced June 2024.

Comments: 18 pages, 11 figures

MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotations

Authors: Ruiyuan Lyu, Tai Wang, **gli Lin, Shuai Yang, Xiaohan Mao, Yilun Chen, Runsen Xu, Haifeng Huang, Chenming Zhu, Dahua Lin, Jiangmiao Pang

Abstract: With the emergence of LLMs and their integration with other data modalities, multi-modal 3D perception attracts more attention due to its connectivity to the physical world and makes rapid progress. However, limited by existing datasets, previous works mainly focus on understanding object properties or inter-object spatial relationships in a 3D scene. To tackle this problem, this paper builds the… ▽ More With the emergence of LLMs and their integration with other data modalities, multi-modal 3D perception attracts more attention due to its connectivity to the physical world and makes rapid progress. However, limited by existing datasets, previous works mainly focus on understanding object properties or inter-object spatial relationships in a 3D scene. To tackle this problem, this paper builds the first largest ever multi-modal 3D scene dataset and benchmark with hierarchical grounded language annotations, MMScan. It is constructed based on a top-down logic, from region to object level, from a single target to inter-target relationships, covering holistic aspects of spatial and attribute understanding. The overall pipeline incorporates powerful VLMs via carefully designed prompts to initialize the annotations efficiently and further involve humans' correction in the loop to ensure the annotations are natural, correct, and comprehensive. Built upon existing 3D scanning data, the resulting multi-modal 3D dataset encompasses 1.4M meta-annotated captions on 109k objects and 7.7k regions as well as over 3.04M diverse samples for 3D visual grounding and question-answering benchmarks. We evaluate representative baselines on our benchmarks, analyze their capabilities in different aspects, and showcase the key problems to be addressed in the future. Furthermore, we use this high-quality dataset to train state-of-the-art 3D visual grounding and LLMs and obtain remarkable performance improvement both on existing benchmarks and in-the-wild evaluation. Codes, datasets, and benchmarks will be available at https://github.com/OpenRobotLab/EmbodiedScan. △ Less

Submitted 13 June, 2024; originally announced June 2024.

Comments: Follow-up of EmbodiedScan. A multi-modal 3D dataset with the most-ever comprehensive language annotations for 3D-LLMs. Project page: https://tai-wang.github.io/mmscan/

Dynamics of Spinning Binary at 2PM

Authors: Gang Chen, Tianheng Wang

Abstract: We consider the covariant proposal for the gravitational Compton amplitude for a Kerr black hole. Employing the covariant three- and four-point Compton amplitudes, we assemble the classical one-loop integrand on the maximal cut at all orders in spin, utilizing the method of unitarity. Expanding in powers of spin, we evaluate the one-loop amplitude up to $\mathcal O(G^2 a^8)$. Supplemented with ext… ▽ More We consider the covariant proposal for the gravitational Compton amplitude for a Kerr black hole. Employing the covariant three- and four-point Compton amplitudes, we assemble the classical one-loop integrand on the maximal cut at all orders in spin, utilizing the method of unitarity. Expanding in powers of spin, we evaluate the one-loop amplitude up to $\mathcal O(G^2 a^8)$. Supplemented with extra contact contributions derived from the far-zone data of the Teukolsky solutions, the one-loop amplitude is in agreement with results available in the literature. We showcase the classical eikonal in the aligned-spin case at $\mathcal O(G^2 a^7)$. △ Less

Submitted 13 June, 2024; originally announced June 2024.

Comments: 16 pages + appendices

Report number: SNUTP24-002

Observation of $η_{c}$(1S, 2S) and $χ_{cJ}$ decays to 2$(π^{+}π^{-})η$ via $ψ$(3686) radiative transitions

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere , et al. (636 additional authors not shown)

Abstract: Based on $2.7 \times 10^9~ψ(3686)$ decays collected with the BESIII detector, the radiative decay $ψ(3686)\to\gamma2(π^{+}π^{-})η$ is investigated to measure properties of S- and P-wave charmonium states. The branching fraction of the decay $η_{c}(1S) \to 2(π^{+}π^{-})η$, which is found to have a strong dependence on the interference pattern between $η_c(1S)$ and non-$η_c(1S)$ processes, is measur… ▽ More Based on $2.7 \times 10^9~ψ(3686)$ decays collected with the BESIII detector, the radiative decay $ψ(3686)\to\gamma2(π^{+}π^{-})η$ is investigated to measure properties of S- and P-wave charmonium states. The branching fraction of the decay $η_{c}(1S) \to 2(π^{+}π^{-})η$, which is found to have a strong dependence on the interference pattern between $η_c(1S)$ and non-$η_c(1S)$ processes, is measured in both destructive and constructive interference scenarios for the first time. The mass and width of the $η_{c}(1S)$ are measured to be $M=(2984.14 \pm 0.13 \pm 0.38)$ MeV/$c^{2}$ and $Γ=(28.82 \pm 0.11 \pm 0.82)$ MeV, respectively. Clear signals for the decays of the $χ_{cJ}(J=0,1,2)$ and the $η_{c}(2S)$ to $2(π^{+}π^{-})η$ are also observed for the first time, and the corresponding branching fractions are measured. The ratio of the branching fractions between the $η_{c}(2S)$ and $η_{c}(1S)$ decays is significantly lower than the theoretical prediction, which might suggest different dynamics in their decays. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Incremental Learning and Self-Attention Mechanisms Improve Neural System Identification

Authors: Isaac Lin, Tianye Wang, Shang Gao, Shiming Tang, Tai Sing Lee

Abstract: Convolutional neural networks (CNNs) have been shown to be the state-of-the-art approach for modeling the transfer functions of visual cortical neurons. Cortical neurons in the primary visual cortex are are sensitive to contextual information mediated by extensive horizontal and feedback connections. Standard CNNs can integrate global spatial image information to model such contextual modulation v… ▽ More Convolutional neural networks (CNNs) have been shown to be the state-of-the-art approach for modeling the transfer functions of visual cortical neurons. Cortical neurons in the primary visual cortex are are sensitive to contextual information mediated by extensive horizontal and feedback connections. Standard CNNs can integrate global spatial image information to model such contextual modulation via two mechanisms: successive rounds of convolutions and a fully connected readout layer. In this paper, we find that non-local networks or self-attention (SA) mechanisms, theoretically related to context-dependent flexible gating mechanisms observed in the primary visual cortex, improve neural response predictions over parameter-matched CNNs in two key metrics: tuning curve correlation and tuning peak. We factorize networks to determine the relative contribution of each context mechanism. This reveals that information in the local receptive field is most important for modeling the overall tuning curve, but surround information is critically necessary for characterizing the tuning peak. We find that self-attention can replace subsequent spatial-integration convolutions when learned in an incremental manner, and is further enhanced in the presence of a fully connected readout layer, suggesting that the two context mechanisms are complementary. Finally, we find that learning a receptive-field-centric model with self-attention, before incrementally learning a fully connected readout, yields a more biologically realistic model in terms of center-surround contributions. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Comments: Preprint NeurIPS 2024

SE/BN Adapter: Parametric Efficient Domain Adaptation for Speaker Recognition

Authors: Tianhao Wang, Lantian Li, Dong Wang

Abstract: Deploying a well-optimized pre-trained speaker recognition model in a new domain often leads to a significant decline in performance. While fine-tuning is a commonly employed solution, it demands ample adaptation data and suffers from parameter inefficiency, rendering it impractical for real-world applications with limited data available for model adaptation. Drawing inspiration from the success o… ▽ More Deploying a well-optimized pre-trained speaker recognition model in a new domain often leads to a significant decline in performance. While fine-tuning is a commonly employed solution, it demands ample adaptation data and suffers from parameter inefficiency, rendering it impractical for real-world applications with limited data available for model adaptation. Drawing inspiration from the success of adapters in self-supervised pre-trained models, this paper introduces a SE/BN adapter to address this challenge. By freezing the core speaker encoder and adjusting the feature maps' weights and activation distributions, we introduce a novel adapter utilizing trainable squeeze-and-excitation (SE) blocks and batch normalization (BN) layers, termed SE/BN adapter. Our experiments, conducted using VoxCeleb for pre-training and 4 genres from CN-Celeb for adaptation, demonstrate that the SE/BN adapter offers significant performance improvement over the baseline and competes with the vanilla fine-tuning approach by tuning just 1% of the parameters. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Comments: to be published in INTERSPEECH 2024

FP-Inconsistent: Detecting Evasive Bots using Browser Fingerprint Inconsistencies

Authors: Hari Venugopalan, Shaoor Munir, Shuaib Ahmed, Tangbaihe Wang, Samuel T. King, Zubair Shafiq

Abstract: As browser fingerprinting is increasingly being used for bot detection, bots have started altering their fingerprints for evasion. We conduct the first large-scale evaluation of evasive bots to investigate whether and how altering fingerprints helps bots evade detection. To systematically investigate evasive bots, we deploy a honey site incorporating two anti-bot services (DataDome and BotD) and s… ▽ More As browser fingerprinting is increasingly being used for bot detection, bots have started altering their fingerprints for evasion. We conduct the first large-scale evaluation of evasive bots to investigate whether and how altering fingerprints helps bots evade detection. To systematically investigate evasive bots, we deploy a honey site incorporating two anti-bot services (DataDome and BotD) and solicit bot traffic from 20 different bot services that purport to sell "realistic and undetectable traffic". Across half a million requests from 20 different bot services on our honey site, we find an average evasion rate of 52.93% against DataDome and 44.56% evasion rate against BotD. Our comparison of fingerprint attributes from bot services that evade each anti-bot service individually as well as bot services that evade both shows that bot services indeed alter different browser fingerprint attributes for evasion. Further, our analysis reveals the presence of inconsistent fingerprint attributes in evasive bots. Given evasive bots seem to have difficulty in ensuring consistency in their fingerprint attributes, we propose a data-driven approach to discover rules to detect such inconsistencies across space (two attributes in a given browser fingerprint) and time (a single attribute at two different points in time). These rules, which can be readily deployed by anti-bot services, reduce the evasion rate of evasive bots against DataDome and BotD by 48.11% and 44.95% respectively. △ Less

Submitted 11 June, 2024; originally announced June 2024.

GemNet: Menu-Based, Strategy-Proof Multi-Bidder Auctions Through Deep Learning

Authors: Tonghan Wang, Yanchen Jiang, David C. Parkes

Abstract: Differentiable economics uses deep learning for automated mechanism design. Despite strong progress, it has remained an open problem to learn multi-bidder, general, and fully strategy-proof (SP) auctions. We introduce GEneral Menu-based NETwork (GemNet), which significantly extends the menu-based approach of RochetNet [Dütting et al., 2023] to the multi-bidder setting. The challenge in achieving S… ▽ More Differentiable economics uses deep learning for automated mechanism design. Despite strong progress, it has remained an open problem to learn multi-bidder, general, and fully strategy-proof (SP) auctions. We introduce GEneral Menu-based NETwork (GemNet), which significantly extends the menu-based approach of RochetNet [Dütting et al., 2023] to the multi-bidder setting. The challenge in achieving SP is to learn bidder-independent menus that are feasible, so that the optimal menu choices for each bidder do not over-allocate items when taken together (we call this menu compatibility). GemNet penalizes the failure of menu compatibility during training, and transforms learned menus after training through price changes, by considering a set of discretized bidder values and reasoning about Lipschitz smoothness to guarantee menu compatibility on the entire value space. This approach is general, leaving undisturbed trained menus that already satisfy menu compatibility and reducing to RochetNet for a single bidder. Mixed-integer linear programs are used for menu transforms and through a number of optimizations, including adaptive grids and methods to skip menu elements, we scale to large auction design problems. GemNet learns auctions with better revenue than affine maximization methods, achieves exact SP whereas previous general multi-bidder methods are approximately SP, and offers greatly enhanced interpretability. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Experimental Modeling of Chiral Active Robots and a Minimal Model of Non-Gaussian Displacements

Authors: Yuxuan Zhou, Maomao Ge, Ting Wang

Abstract: We design 3D-printed motor-driven active particles and find that their dynamics can be characterized using the model of overdamped chiral active Brownian particles (ABPs), as demonstrated by measured angular statistics and translational mean squared displacements (MSDs). Furthermore, we propose a minimal model that reproduces the double-peak velocity distributions and further predicts a transition… ▽ More We design 3D-printed motor-driven active particles and find that their dynamics can be characterized using the model of overdamped chiral active Brownian particles (ABPs), as demonstrated by measured angular statistics and translational mean squared displacements (MSDs). Furthermore, we propose a minimal model that reproduces the double-peak velocity distributions and further predicts a transition from the single-peak to the double-peak displacement distributions in short-time regimes. The model provides a clear physics picture of these phenomena, originating from the competition between the active motion and the translational diffusion. Our experiments confirm such picture. The minimal model enhances our understanding of activity-driven non-Gaussian phenomena. The designed particles could be further applied in the study of collective chiral motions. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Phase Diagram of growth modes in Graphene Growth on Cooper by Vapor Deposition

Authors: Tongtong Wang, Jian Zheng, Xin Wei, Dajun Shu

Abstract: Understanding the atomistic mechanism in graphene growth is crucial for controlling the number of layers or domain sizes to meet practical needs. In this work, focusing on the growth of graphene by chemical vapor deposition on copper substrates, the surface kinetics in the growth are systematically investigated by first-principles calculations. The phase diagram, predicting whether the growth mode… ▽ More Understanding the atomistic mechanism in graphene growth is crucial for controlling the number of layers or domain sizes to meet practical needs. In this work, focusing on the growth of graphene by chemical vapor deposition on copper substrates, the surface kinetics in the growth are systematically investigated by first-principles calculations. The phase diagram, predicting whether the growth mode is monolayer graphene or bilayer graphene under various experimental conditions, is constructed based on classical nucleation theory. Our phase diagram well illustrates the effect of high hydrogen pressure on bilayer graphene growth and clarifies the mechanism of the most widely used experimental growth approaches. The phase diagram can provide guidance and predictions for experiments and inspires the study of other two-dimensional materials with graphene-like growth mechanisms. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Existence and uniqueness of ground state solutions for the planar Schrödinger-Newton equation on the disc

Authors: Hui Guo, Zhiwen Long, Tao Wang

Abstract: This paper is concerned with the existence and qualitative properties of positive ground state solutions for the planar Schrödinger-Newton equation on the disc. First, we prove the existence and radial symmetry of all the positive ground state solutions by employing the symmetric decreasing rearrangement and Talenti's inequality. Next, we develop Newton's theorem and then use the contraction mappi… ▽ More This paper is concerned with the existence and qualitative properties of positive ground state solutions for the planar Schrödinger-Newton equation on the disc. First, we prove the existence and radial symmetry of all the positive ground state solutions by employing the symmetric decreasing rearrangement and Talenti's inequality. Next, we develop Newton's theorem and then use the contraction map** principle to establish the uniqueness of the positive ground state solution for the Schrödinger-Newton equation on the disc in the two dimensional case. Finally, we show that the unique positive ground state solution converges to the trivial solution as the radius $R$ tending to infinity, which is totally different from the higher dimensional case in \cite{Guo-Wang-Yi}. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: 16 pages

MSC Class: 35A02; 35B07; 35J08; 35J60

Strong and weak $CP$ tests in sequential decays of polarized $Σ^0$ hyperons

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere , et al. (644 additional authors not shown)

Abstract: The $J/ψ, ψ(3686) \to Σ^0 \barΣ^{0}$ processes and subsequent decays are studied using the world's largest $J/ψ$ and $ψ(3686)$ data samples collected with the BESIII detector. The strong-$CP$ symmetry is tested in the decays of the $Σ^0$ hyperons for the first time by measuring the decay parameters, $α_{Σ^0} = -0.0017 \pm 0.0021 \pm 0.0018$ and $\barα_{Σ^0} = 0.0021 \pm 0.0020 \pm 0.0022$. The wea… ▽ More The $J/ψ, ψ(3686) \to Σ^0 \barΣ^{0}$ processes and subsequent decays are studied using the world's largest $J/ψ$ and $ψ(3686)$ data samples collected with the BESIII detector. The strong-$CP$ symmetry is tested in the decays of the $Σ^0$ hyperons for the first time by measuring the decay parameters, $α_{Σ^0} = -0.0017 \pm 0.0021 \pm 0.0018$ and $\barα_{Σ^0} = 0.0021 \pm 0.0020 \pm 0.0022$. The weak-$CP$ test is performed in the subsequent decays of their daughter particles $Λ$ and $\barΛ$. Also for the first time, the transverse polarizations of the $Σ^0$ hyperons in $J/ψ$ and $ψ(3686)$ decays are observed with opposite directions, and the ratios between the S-wave and D-wave contributions of the $J/ψ, ψ(3686) \to Σ^0 \barΣ^{0}$ decays are obtained. These results are crucial to understand the decay dynamics of the charmonium states and the production mechanism of the $Σ^0-\barΣ^0$ pairs. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Enabling Large-Scale and High-Precision Fluid Simulations on Near-Term Quantum Computers

Authors: Zhao-Yun Chen, Teng-Yang Ma, Chuang-Chao Ye, Liang Xu, Ming-Yang Tan, Xi-Ning Zhuang, Xiao-Fan Xu, Yun-Jie Wang, Tai-** Sun, Yong Chen, Lei Du, Liang-Liang Guo, Hai-Feng Zhang, Hao-Ran Tao, Tian-Le Wang, Xiao-Yan Yang, Ze-An Zhao, Peng Wang, Sheng Zhang, Chi Zhang, Ren-Ze Zhao, Zhi-Long Jia, Wei-Cheng Kong, Meng-Han Dou, Jun-Chao Wang , et al. (7 additional authors not shown)

Abstract: Quantum computational fluid dynamics (QCFD) offers a promising alternative to classical computational fluid dynamics (CFD) by leveraging quantum algorithms for higher efficiency. This paper introduces a comprehensive QCFD method, including an iterative method "Iterative-QLS" that suppresses error in quantum linear solver, and a subspace method to scale the solution to a larger size. We implement o… ▽ More Quantum computational fluid dynamics (QCFD) offers a promising alternative to classical computational fluid dynamics (CFD) by leveraging quantum algorithms for higher efficiency. This paper introduces a comprehensive QCFD method, including an iterative method "Iterative-QLS" that suppresses error in quantum linear solver, and a subspace method to scale the solution to a larger size. We implement our method on a superconducting quantum computer, demonstrating successful simulations of steady Poiseuille flow and unsteady acoustic wave propagation. The Poiseuille flow simulation achieved a relative error of less than $0.2\%$, and the unsteady acoustic wave simulation solved a 5043-dimensional matrix. We emphasize the utilization of the quantum-classical hybrid approach in applications of near-term quantum computers. By adapting to quantum hardware constraints and offering scalable solutions for large-scale CFD problems, our method paves the way for practical applications of near-term quantum computers in computational science. △ Less

Submitted 19 June, 2024; v1 submitted 10 June, 2024; originally announced June 2024.

Comments: 31 pages, 10 figures

Can market volumes reveal traders' rationality and a new risk premium?

Authors: Francesca Mariani, Maria Cristina Recchioni, Tai-Ho Wang, Roberto Giacalone

Abstract: An empirical analysis, suggested by optimal Merton dynamics, reveals some unexpected features of asset volumes. These features are connected to traders' belief and risk aversion. This paper proposes a trading strategy model in the optimal Merton framework that is representative of the collective behavior of heterogeneous rational traders. This model allows for the estimation of the average risk av… ▽ More An empirical analysis, suggested by optimal Merton dynamics, reveals some unexpected features of asset volumes. These features are connected to traders' belief and risk aversion. This paper proposes a trading strategy model in the optimal Merton framework that is representative of the collective behavior of heterogeneous rational traders. This model allows for the estimation of the average risk aversion of traders acting on a specific risky asset, while revealing the existence of a price of risk closely related to market price of risk and volume rate. The empirical analysis, conducted on real data, confirms the validity of the proposed model. △ Less

Submitted 9 June, 2024; originally announced June 2024.

Measurement of the integrated luminosity of the data collected at 3.773 GeV by BESIII from 2021 to 2024

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere , et al. (634 additional authors not shown)

Abstract: We present a measurement of the integrated luminosity of $e^+e^-$ collision data collected with the BESIII detector at the BEPCII collider at a center-of-mass energy of $E_{\rm cm} = 3.773$~GeV. The integrated luminosities of the data sets taken from December 2021 to June 2022, from November 2022 to June 2023, and from October 2023 to February 2024 are determined to be $4.995 \pm 0.019$~fb$^{-1}$,… ▽ More We present a measurement of the integrated luminosity of $e^+e^-$ collision data collected with the BESIII detector at the BEPCII collider at a center-of-mass energy of $E_{\rm cm} = 3.773$~GeV. The integrated luminosities of the data sets taken from December 2021 to June 2022, from November 2022 to June 2023, and from October 2023 to February 2024 are determined to be $4.995 \pm 0.019$~fb$^{-1}$, $8.157 \pm 0.031$~fb$^{-1}$, and $4.191 \pm 0.016$~fb$^{-1}$, respectively, by analyzing large angle Bhabha scattering events. The uncertainties are dominated by systematic effects and the statistical uncertainties are negligible. Our results provide essential input for future analyses and precision measurements. △ Less

Submitted 9 June, 2024; originally announced June 2024.

Unified Text-to-Image Generation and Retrieval

Authors: Leigang Qu, Haochuan Li, Tan Wang, Wenjie Wang, Yongqi Li, Liqiang Nie, Tat-Seng Chua

Abstract: How humans can efficiently and effectively acquire images has always been a perennial question. A typical solution is text-to-image retrieval from an existing database given the text query; however, the limited database typically lacks creativity. By contrast, recent breakthroughs in text-to-image generation have made it possible to produce fancy and diverse visual content, but it faces challenges… ▽ More How humans can efficiently and effectively acquire images has always been a perennial question. A typical solution is text-to-image retrieval from an existing database given the text query; however, the limited database typically lacks creativity. By contrast, recent breakthroughs in text-to-image generation have made it possible to produce fancy and diverse visual content, but it faces challenges in synthesizing knowledge-intensive images. In this work, we rethink the relationship between text-to-image generation and retrieval and propose a unified framework in the context of Multimodal Large Language Models (MLLMs). Specifically, we first explore the intrinsic discriminative abilities of MLLMs and introduce a generative retrieval method to perform retrieval in a training-free manner. Subsequently, we unify generation and retrieval in an autoregressive generation way and propose an autonomous decision module to choose the best-matched one between generated and retrieved images as the response to the text query. Additionally, we construct a benchmark called TIGeR-Bench, including creative and knowledge-intensive domains, to standardize the evaluation of unified text-to-image generation and retrieval. Extensive experimental results on TIGeR-Bench and two retrieval benchmarks, i.e., Flickr30K and MS-COCO, demonstrate the superiority and effectiveness of our proposed method. △ Less

Submitted 9 June, 2024; originally announced June 2024.

CorrMAE: Pre-training Correspondence Transformers with Masked Autoencoder

Authors: Tangfei Liao, Xiaoqin Zhang, Guobao Xiao, Min Li, Tao Wang, Mang Ye

Abstract: Pre-training has emerged as a simple yet powerful methodology for representation learning across various domains. However, due to the expensive training cost and limited data, pre-training has not yet been extensively studied in correspondence pruning. To tackle these challenges, we propose a pre-training method to acquire a generic inliers-consistent representation by reconstructing masked corres… ▽ More Pre-training has emerged as a simple yet powerful methodology for representation learning across various domains. However, due to the expensive training cost and limited data, pre-training has not yet been extensively studied in correspondence pruning. To tackle these challenges, we propose a pre-training method to acquire a generic inliers-consistent representation by reconstructing masked correspondences, providing a strong initial representation for downstream tasks. Toward this objective, a modicum of true correspondences naturally serve as input, thus significantly reducing pre-training overhead. In practice, we introduce CorrMAE, an extension of the mask autoencoder framework tailored for the pre-training of correspondence pruning. CorrMAE involves two main phases, \ie correspondence learning and matching point reconstruction, guiding the reconstruction of masked correspondences through learning visible correspondence consistency. Herein, we employ a dual-branch structure with an ingenious positional encoding to reconstruct unordered and irregular correspondences. Also, a bi-level designed encoder is proposed for correspondence learning, which offers enhanced consistency learning capability and transferability. Extensive experiments have shown that the model pre-trained with our CorrMAE outperforms prior work on multiple challenging benchmarks. Meanwhile, our CorrMAE is primarily a task-driven pre-training method, and can achieve notable improvements for downstream tasks by pre-training on the targeted dataset. We hope this work can provide a starting point for correspondence pruning pre-training. △ Less

Submitted 9 June, 2024; originally announced June 2024.

The size of the Milky Way galaxy

Authors: Jianhui Lian, Gail Zasowski, Bingqiu Chen, Julie Imig, Tao Wang, Nicholas Boardman, Xiaowei Liu

Abstract: The size of a galaxy is one of the fundamental parameters that reflects its growth and assembly history. Traditionally, the size of the Milky Way has been characterized by the scale length of the disk, based on the assumption of an exponential density profile. Earlier scale length measurements suggest the Milky Way is an overly compact galaxy, compared to similar galaxies of its mass. These size m… ▽ More The size of a galaxy is one of the fundamental parameters that reflects its growth and assembly history. Traditionally, the size of the Milky Way has been characterized by the scale length of the disk, based on the assumption of an exponential density profile. Earlier scale length measurements suggest the Milky Way is an overly compact galaxy, compared to similar galaxies of its mass. These size measurements, however, ignore the presence of the bulge, and the assumption of a single-exponential disk profile faces growing challenges from the recent observations. The half-light radius is an alternative size measurement that is independent of the galaxy density profile and has been widely used to quantify the size of external galaxies. Here we report the half-light radius of the Milky Way, derived from a new measurement of the age-resolved Galactic surface brightness profile in an unprecedentedly wide radial range from ${\rm R=0}$ to 17~kpc. We find a broken surface brightness profile with a nearly flat distribution between 3.5 and 7.5 kpc, which results in a half-light radius of 5.75$\pm$0.38 kpc, significantly larger than the scale-length inferred from the canonical single-exponential disk profile but in good consistency with local disk galaxies of similar mass. Because our density profile can be decomposed by stellar age and extrapolated backwards in time, we can also confirm that the size history of the Milky Way is broadly consistent with high-redshift galaxies but with systematically smaller size at each look back time. Our results suggest that the Milky Way is a typical disk galaxy regarding its size and has likely experienced inefficient secular size growth. △ Less

Submitted 28 June, 2024; v1 submitted 8 June, 2024; originally announced June 2024.

Comments: 30 pages, 4figures, published online in Nature Astronomy on 27 June 2024, https://rdcu.be/dL3z5. Here is the version prior to the peer review

TraceableSpeech: Towards Proactively Traceable Text-to-Speech with Watermarking

Authors: Junzuo Zhou, Jiangyan Yi, Tao Wang, Jianhua Tao, Ye Bai, Chu Yuan Zhang, Yong Ren, Zhengqi Wen

Abstract: Various threats posed by the progress in text-to-speech (TTS) have prompted the need to reliably trace synthesized speech. However, contemporary approaches to this task involve adding watermarks to the audio separately after generation, a process that hurts both speech quality and watermark imperceptibility. In addition, these approaches are limited in robustness and flexibility. To address these… ▽ More Various threats posed by the progress in text-to-speech (TTS) have prompted the need to reliably trace synthesized speech. However, contemporary approaches to this task involve adding watermarks to the audio separately after generation, a process that hurts both speech quality and watermark imperceptibility. In addition, these approaches are limited in robustness and flexibility. To address these problems, we propose TraceableSpeech, a novel TTS model that directly generates watermarked speech, improving watermark imperceptibility and speech quality. Furthermore, We design the frame-wise imprinting and extraction of watermarks, achieving higher robustness against resplicing attacks and temporal flexibility in operation. Experimental results show that TraceableSpeech outperforms the strong baseline where VALL-E or HiFicodec individually uses WavMark in watermark imperceptibility, speech quality and resilience against resplicing attacks. It also can apply to speech of various durations. △ Less

Submitted 7 June, 2024; originally announced June 2024.

Comments: acceped by interspeech 2024

PPPR: Portable Plug-in Prompt Refiner for Text to Audio Generation

Authors: Shuchen Shi, Ruibo Fu, Zhengqi Wen, Jianhua Tao, Tao Wang, Chunyu Qiang, Yi Lu, Xin Qi, Xuefei Liu, Yukun Liu, Yongwei Li, Zhiyong Wang, Xiaopeng Wang

Abstract: Text-to-Audio (TTA) aims to generate audio that corresponds to the given text description, playing a crucial role in media production. The text descriptions in TTA datasets lack rich variations and diversity, resulting in a drop in TTA model performance when faced with complex text. To address this issue, we propose a method called Portable Plug-in Prompt Refiner, which utilizes rich knowledge abo… ▽ More Text-to-Audio (TTA) aims to generate audio that corresponds to the given text description, playing a crucial role in media production. The text descriptions in TTA datasets lack rich variations and diversity, resulting in a drop in TTA model performance when faced with complex text. To address this issue, we propose a method called Portable Plug-in Prompt Refiner, which utilizes rich knowledge about textual descriptions inherent in large language models to effectively enhance the robustness of TTA acoustic models without altering the acoustic training set. Furthermore, a Chain-of-Thought that mimics human verification is introduced to enhance the accuracy of audio descriptions, thereby improving the accuracy of generated content in practical applications. The experiments show that our method achieves a state-of-the-art Inception Score (IS) of 8.72, surpassing AudioGen, AudioLDM and Tango. △ Less

Submitted 7 June, 2024; originally announced June 2024.

Comments: accepted by INTERSPEECH2024

PromptFix: Few-shot Backdoor Removal via Adversarial Prompt Tuning

Authors: Tianrong Zhang, Zhaohan Xi, Ting Wang, Prasenjit Mitra, **ghui Chen

Abstract: Pre-trained language models (PLMs) have attracted enormous attention over the past few years with their unparalleled performances. Meanwhile, the soaring cost to train PLMs as well as their amazing generalizability have jointly contributed to few-shot fine-tuning and prompting as the most popular training paradigms for natural language processing (NLP) models. Nevertheless, existing studies have s… ▽ More Pre-trained language models (PLMs) have attracted enormous attention over the past few years with their unparalleled performances. Meanwhile, the soaring cost to train PLMs as well as their amazing generalizability have jointly contributed to few-shot fine-tuning and prompting as the most popular training paradigms for natural language processing (NLP) models. Nevertheless, existing studies have shown that these NLP models can be backdoored such that model behavior is manipulated when trigger tokens are presented. In this paper, we propose PromptFix, a novel backdoor mitigation strategy for NLP models via adversarial prompt-tuning in few-shot settings. Unlike existing NLP backdoor removal methods, which rely on accurate trigger inversion and subsequent model fine-tuning, PromptFix keeps the model parameters intact and only utilizes two extra sets of soft tokens which approximate the trigger and counteract it respectively. The use of soft tokens and adversarial optimization eliminates the need to enumerate possible backdoor configurations and enables an adaptive balance between trigger finding and preservation of performance. Experiments with various backdoor attacks validate the effectiveness of the proposed method and the performances when domain shift is present further shows PromptFix's applicability to models pretrained on unknown data source which is the common case in prompt tuning scenarios. △ Less

Submitted 6 June, 2024; originally announced June 2024.

Comments: NAACL 2024

Text-to-Drive: Diverse Driving Behavior Synthesis via Large Language Models

Authors: Phat Nguyen, Tsun-Hsuan Wang, Zhang-Wei Hong, Sertac Karaman, Daniela Rus

Abstract: Generating varied scenarios through simulation is crucial for training and evaluating safety-critical systems, such as autonomous vehicles. Yet, the task of modeling the trajectories of other vehicles to simulate diverse and meaningful close interactions remains prohibitively costly. Adopting language descriptions to generate driving behaviors emerges as a promising strategy, offering a scalable a… ▽ More Generating varied scenarios through simulation is crucial for training and evaluating safety-critical systems, such as autonomous vehicles. Yet, the task of modeling the trajectories of other vehicles to simulate diverse and meaningful close interactions remains prohibitively costly. Adopting language descriptions to generate driving behaviors emerges as a promising strategy, offering a scalable and intuitive method for human operators to simulate a wide range of driving interactions. However, the scarcity of large-scale annotated language-trajectory data makes this approach challenging. To address this gap, we propose Text-to-Drive (T2D) to synthesize diverse driving behaviors via Large Language Models (LLMs). We introduce a knowledge-driven approach that operates in two stages. In the first stage, we employ the embedded knowledge of LLMs to generate diverse language descriptions of driving behaviors for a scene. Then, we leverage LLM's reasoning capabilities to synthesize these behaviors in simulation. At its core, T2D employs an LLM to construct a state chart that maps low-level states to high-level abstractions. This strategy aids in downstream tasks such as summarizing low-level observations, assessing policy alignment with behavior description, and sha** the auxiliary reward, all without needing human supervision. With our knowledge-driven approach, we demonstrate that T2D generates more diverse trajectories compared to other baselines and offers a natural language interface that allows for interactive incorporation of human preference. Please check our website for more examples: https://text-to-drive.github.io/ △ Less

Submitted 6 June, 2024; originally announced June 2024.

Comments: 14 pages, 7 figures

Latent Neural Operator for Solving Forward and Inverse PDE Problems

Authors: Tian Wang, Chuang Wang

Abstract: Neural operators effectively solve PDE problems from data without knowing the explicit equations, which learn the map from the input sequences of observed samples to the predicted values. Most existed works build the model in the original geometric space, leading to high computational costs when the number of sample points is large. We present the Latent Neural Operator (LNO) solving PDEs in the l… ▽ More Neural operators effectively solve PDE problems from data without knowing the explicit equations, which learn the map from the input sequences of observed samples to the predicted values. Most existed works build the model in the original geometric space, leading to high computational costs when the number of sample points is large. We present the Latent Neural Operator (LNO) solving PDEs in the latent space. In particular, we first propose Physics-Cross-Attention (PhCA) transforming representation from the geometric space to the latent space, then learn the operator in the latent space, and finally recover the real-world geometric space via the inverse PhCA map. Our model retains flexibility that can decode values in any position not limited to locations defined in training set, and therefore can naturally perform interpolation and extrapolation tasks particularly useful for inverse problems. Moreover, the proposed LNO improves in both prediction accuracy and computational efficiency. Experiments show that LNO reduces the GPU memory by 50%, speeds up training 1.8 times, and reaches state-of-the-art accuracy on four out of six benchmarks for forward problems and a benchmark for inverse problem. △ Less

Submitted 9 June, 2024; v1 submitted 6 June, 2024; originally announced June 2024.