Showing 51–100 of 1,317 results for author: Ren, Z

Task-Driven Exploration: Decoupling and Inter-Task Feedback for Joint Moment Retrieval and Highlight Detection

Authors: ** Yang, ** Wei, Huan Li, Ziyang Ren

Abstract: Video moment retrieval and highlight detection are two highly valuable tasks in video understanding, but until recently they have been jointly studied. Although existing studies have made impressive advancement recently, they predominantly follow the data-driven bottom-up paradigm. Such paradigm overlooks task-specific and inter-task effects, resulting in poor model performance. In this paper, we… ▽ More Video moment retrieval and highlight detection are two highly valuable tasks in video understanding, but until recently they have been jointly studied. Although existing studies have made impressive advancement recently, they predominantly follow the data-driven bottom-up paradigm. Such paradigm overlooks task-specific and inter-task effects, resulting in poor model performance. In this paper, we propose a novel task-driven top-down framework TaskWeave for joint moment retrieval and highlight detection. The framework introduces a task-decoupled unit to capture task-specific and common representations. To investigate the interplay between the two tasks, we propose an inter-task feedback mechanism, which transforms the results of one task as guiding masks to assist the other task. Different from existing methods, we present a task-dependent joint loss function to optimize the model. Comprehensive experiments and in-depth ablation studies on QVHighlights, TVSum, and Charades-STA datasets corroborate the effectiveness and flexibility of the proposed framework. Codes are available at https://github.com/EdenGabriel/TaskWeave. △ Less

Submitted 14 April, 2024; originally announced April 2024.

Spin entanglement of multinucleons: experimental prospects

Authors: Dong Bai, Zhongzhou Ren

Abstract: Multiprotons and multineutrons are among the most exotic and mysterious things ever produced on earth. They provide an exceptional opportunity to understand nuclear forces and nuclear dynamics at extreme conditions, as well as neutron stars in the heaven. Quantum entanglement, referred to as ``spooky action at a distance'' by Einstein, is a ubiquitous yet deep property of quantum systems. It not o… ▽ More Multiprotons and multineutrons are among the most exotic and mysterious things ever produced on earth. They provide an exceptional opportunity to understand nuclear forces and nuclear dynamics at extreme conditions, as well as neutron stars in the heaven. Quantum entanglement, referred to as ``spooky action at a distance'' by Einstein, is a ubiquitous yet deep property of quantum systems. It not only occupies a central position in quantum information science but also is investigated intensively in high energy physics, condensed matter physics, and quantum gravity. In comparison, the study of nuclear entanglement is still in infancy, and the entanglement properties of multiprotons and multineutrons in free space are generally unknown. Here, we study the crucial problem of how to measure spin entanglement of these multinucleons in nuclear experiments, with special emphases on two- and three-nucleon states. These findings open a freshly new direction for the multinucleon research. They are also useful for understanding entanglement properties of other exotic nuclear objects. △ Less

Submitted 13 April, 2024; originally announced April 2024.

Comments: 6 pages, 4 figures

GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh

Authors: **g Wen, Xiaoming Zhao, Zhongzheng Ren, Alexander G. Schwing, Shenlong Wang

Abstract: We introduce GoMAvatar, a novel approach for real-time, memory-efficient, high-quality animatable human modeling. GoMAvatar takes as input a single monocular video to create a digital avatar capable of re-articulation in new poses and real-time rendering from novel viewpoints, while seamlessly integrating with rasterization-based graphics pipelines. Central to our method is the Gaussians-on-Mesh r… ▽ More We introduce GoMAvatar, a novel approach for real-time, memory-efficient, high-quality animatable human modeling. GoMAvatar takes as input a single monocular video to create a digital avatar capable of re-articulation in new poses and real-time rendering from novel viewpoints, while seamlessly integrating with rasterization-based graphics pipelines. Central to our method is the Gaussians-on-Mesh representation, a hybrid 3D model combining rendering quality and speed of Gaussian splatting with geometry modeling and compatibility of deformable meshes. We assess GoMAvatar on ZJU-MoCap data and various YouTube videos. GoMAvatar matches or surpasses current monocular human modeling algorithms in rendering quality and significantly outperforms them in computational efficiency (43 FPS) while being memory-efficient (3.63 MB per subject). △ Less

Submitted 11 April, 2024; originally announced April 2024.

Comments: CVPR 2024; project page: https://wenj.github.io/GoMAvatar/

Search for prompt production of pentaquarks in charm hadron final states

Authors: LHCb collaboration, R. Aaij, A. S. W. Abdelmotteleb, C. Abellan Beteta, F. Abudinén, T. Ackernley, B. Adeva, M. Adinolfi, P. Adlarson, H. Afsharnia, C. Agapopoulou, C. A. Aidala, Z. Ajaltouni, S. Akar, K. Akiba, P. Albicocco, J. Albrecht, F. Alessio, M. Alexander, A. Alfonso Albero, Z. Aliouche, P. Alvarez Cartelle, R. Amalric, S. Amato, J. L. Amey , et al. (1090 additional authors not shown)

Abstract: A search for hidden-charm pentaquark states decaying to a range of $Σ_{c}\bar{D}$ and $Λ_{c}\bar{D}$ final states, as well as doubly-charmed pentaquark states to $Σ_{c}D$ and $Λ_{c}^{+}D$, is made using samples of proton-proton collision data corresponding to an integrated luminosity of $5.7fb^{-1}$ recorded by the LHCb detector at $\sqrt{s} = 13Te\kern -0.1em V$. Since no significant signals are… ▽ More A search for hidden-charm pentaquark states decaying to a range of $Σ_{c}\bar{D}$ and $Λ_{c}\bar{D}$ final states, as well as doubly-charmed pentaquark states to $Σ_{c}D$ and $Λ_{c}^{+}D$, is made using samples of proton-proton collision data corresponding to an integrated luminosity of $5.7fb^{-1}$ recorded by the LHCb detector at $\sqrt{s} = 13Te\kern -0.1em V$. Since no significant signals are found, upper limits are set on the pentaquark yields relative to that of the $Λ_{c}^{+}$ baryon in the $Λ_{c}^{+}\to pK^{-}π^{+}$ decay mode. The known pentaquark states are also investigated, and their signal yields are found to be consistent with zero in all cases. △ Less

Submitted 2 June, 2024; v1 submitted 10 April, 2024; originally announced April 2024.

Comments: All figures and tables, along with any supplementary material and additional information, are available at https://cern.ch/lhcbproject/Publications/p/LHCb-PAPER-2023-018.html (LHCb public pages)

Report number: LHCb-PAPER-2023-018, CERN-EP-2024-071

Skill Transfer and Discovery for Sim-to-Real Learning: A Representation-Based Viewpoint

Authors: Haitong Ma, Zhaolin Ren, Bo Dai, Na Li

Abstract: We study sim-to-real skill transfer and discovery in the context of robotics control using representation learning. We draw inspiration from spectral decomposition of Markov decision processes. The spectral decomposition brings about representation that can linearly represent the state-action value function induced by any policies, thus can be regarded as skills. The skill representations are tran… ▽ More We study sim-to-real skill transfer and discovery in the context of robotics control using representation learning. We draw inspiration from spectral decomposition of Markov decision processes. The spectral decomposition brings about representation that can linearly represent the state-action value function induced by any policies, thus can be regarded as skills. The skill representations are transferable across arbitrary tasks with the same transition dynamics. Moreover, to handle the sim-to-real gap in the dynamics, we propose a skill discovery algorithm that learns new skills caused by the sim-to-real gap from real-world data. We promote the discovery of new skills by enforcing orthogonal constraints between the skills to learn and the skills from simulators, and then synthesize the policy using the enlarged skill sets. We demonstrate our methodology by transferring quadrotor controllers from simulators to Crazyflie 2.1 quadrotors. We show that we can learn the skill representations from a single simulator task and transfer these to multiple different real-world tasks including hovering, taking off, landing and trajectory tracking. Our skill discovery approach helps narrow the sim-to-real gap and improve the real-world controller performance by up to 30.2%. △ Less

Submitted 7 April, 2024; originally announced April 2024.

Comments: 9 pages, 6 figures. Project page: https://congharvard.github.io/steady-sim-to-real/

Hyperparameter Optimization for SecureBoost via Constrained Multi-Objective Federated Learning

Authors: Yan Kang, Ziyao Ren, Lixin Fan, Linghua Yang, Yongxin Tong, Qiang Yang

Abstract: SecureBoost is a tree-boosting algorithm that leverages homomorphic encryption (HE) to protect data privacy in vertical federated learning. SecureBoost and its variants have been widely adopted in fields such as finance and healthcare. However, the hyperparameters of SecureBoost are typically configured heuristically for optimizing model performance (i.e., utility) solely, assuming that privacy is… ▽ More SecureBoost is a tree-boosting algorithm that leverages homomorphic encryption (HE) to protect data privacy in vertical federated learning. SecureBoost and its variants have been widely adopted in fields such as finance and healthcare. However, the hyperparameters of SecureBoost are typically configured heuristically for optimizing model performance (i.e., utility) solely, assuming that privacy is secured. Our study found that SecureBoost and some of its variants are still vulnerable to label leakage. This vulnerability may lead the current heuristic hyperparameter configuration of SecureBoost to a suboptimal trade-off between utility, privacy, and efficiency, which are pivotal elements toward a trustworthy federated learning system. To address this issue, we propose the Constrained Multi-Objective SecureBoost (CMOSB) algorithm, which aims to approximate Pareto optimal solutions that each solution is a set of hyperparameters achieving an optimal trade-off between utility loss, training cost, and privacy leakage. We design measurements of the three objectives, including a novel label inference attack named instance clustering attack (ICA) to measure the privacy leakage of SecureBoost. Additionally, we provide two countermeasures against ICA. The experimental results demonstrate that the CMOSB yields superior hyperparameters over those optimized by grid search and Bayesian optimization regarding the trade-off between utility loss, training cost, and privacy leakage. △ Less

Submitted 5 April, 2024; originally announced April 2024.

Search for the $B_s^0 \rightarrow μ^+μ^-γ$ decay

Authors: LHCb collaboration, R. Aaij, A. S. W. Abdelmotteleb, C. Abellan Beteta, F. Abudinén, T. Ackernley, A. A. Adefisoye, B. Adeva, M. Adinolfi, P. Adlarson, C. Agapopoulou, C. A. Aidala, Z. Ajaltouni, S. Akar, K. Akiba, P. Albicocco, J. Albrecht, F. Alessio, M. Alexander, Z. Aliouche, P. Alvarez Cartelle, R. Amalric, S. Amato, J. L. Amey, Y. Amhis , et al. (1068 additional authors not shown)

Abstract: A search for the fully reconstructed $B_s^0 \rightarrow μ^+μ^-γ$ decay is performed at the LHCb experiment using proton-proton collisions at $\sqrt{s}=13$\,TeV corresponding to an integrated luminosity of $5.4\,\mathrm{fb^{-1}}$. No significant signal is found and upper limits on the branching fraction in intervals of the dimuon mass are set \begin{align} {\cal B}(B_s^0 \rightarrow μ^+μ^-γ) <… ▽ More A search for the fully reconstructed $B_s^0 \rightarrow μ^+μ^-γ$ decay is performed at the LHCb experiment using proton-proton collisions at $\sqrt{s}=13$\,TeV corresponding to an integrated luminosity of $5.4\,\mathrm{fb^{-1}}$. No significant signal is found and upper limits on the branching fraction in intervals of the dimuon mass are set \begin{align} {\cal B}(B_s^0 \rightarrow μ^+μ^-γ) < 4.2\times10^{-8},~&m(μμ)\in[2m_μ,~1.70]\,\mathrm{GeV/c^2} ,\nonumber {\cal B}(B_s^0 \rightarrow μ^+μ^-γ) < 7.7\times10^{-8},~&m(μμ)\in[1.70,~2.88]\,\mathrm{GeV/c^2},\nonumber {\cal B}(B_s^0 \rightarrow μ^+μ^-γ) < 4.2\times10^{-8},~&m(μμ)\in[3.92 ,~m_{B_s^0}]\,\mathrm{GeV/c^2},\nonumber \end{align} at 95\% confidence level. Additionally, upper limits are set on the branching fraction in the $[2m_μ,~1.70]\,\mathrm{GeV/c^2}$ dimuon mass region excluding the contribution from the intermediate $φ(1020)$ meson, and in the region combining all dimuon-mass intervals. △ Less

Submitted 4 April, 2024; originally announced April 2024.

Comments: All figures and tables, along with any supplementary material and additional information, are available at https://cern.ch/lhcbproject/Publications/p/LHCb-PAPER-2023-045.html

Report number: LHCb-PAPER-2023-045, CERN-EP-2024-065

Talaria: Interactively Optimizing Machine Learning Models for Efficient Inference

Authors: Fred Hohman, Chaoqun Wang, **mook Lee, Jochen Görtler, Dominik Moritz, Jeffrey P Bigham, Zhile Ren, Cecile Foret, Qi Shan, Xiaoyi Zhang

Abstract: On-device machine learning (ML) moves computation from the cloud to personal devices, protecting user privacy and enabling intelligent user experiences. However, fitting models on devices with limited resources presents a major technical challenge: practitioners need to optimize models and balance hardware metrics such as model size, latency, and power. To help practitioners create efficient ML mo… ▽ More On-device machine learning (ML) moves computation from the cloud to personal devices, protecting user privacy and enabling intelligent user experiences. However, fitting models on devices with limited resources presents a major technical challenge: practitioners need to optimize models and balance hardware metrics such as model size, latency, and power. To help practitioners create efficient ML models, we designed and developed Talaria: a model visualization and optimization system. Talaria enables practitioners to compile models to hardware, interactively visualize model statistics, and simulate optimizations to test the impact on inference metrics. Since its internal deployment two years ago, we have evaluated Talaria using three methodologies: (1) a log analysis highlighting its growth of 800+ practitioners submitting 3,600+ models; (2) a usability survey with 26 users assessing the utility of 20 Talaria features; and (3) a qualitative interview with the 7 most active users about their experience using Talaria. △ Less

Submitted 3 April, 2024; originally announced April 2024.

Comments: Proceedings of the 2024 ACM CHI Conference on Human Factors in Computing Systems

Generative Retrieval as Multi-Vector Dense Retrieval

Authors: Shiguang Wu, Wenda Wei, Mengqi Zhang, Zhumin Chen, Jun Ma, Zhaochun Ren, Maarten de Rijke, Pengjie Ren

Abstract: Generative retrieval generates identifiers of relevant documents in an end-to-end manner using a sequence-to-sequence architecture for a given query. The relation between generative retrieval and other retrieval methods, especially those based on matching within dense retrieval models, is not yet fully comprehended. Prior work has demonstrated that generative retrieval with atomic identifiers is e… ▽ More Generative retrieval generates identifiers of relevant documents in an end-to-end manner using a sequence-to-sequence architecture for a given query. The relation between generative retrieval and other retrieval methods, especially those based on matching within dense retrieval models, is not yet fully comprehended. Prior work has demonstrated that generative retrieval with atomic identifiers is equivalent to single-vector dense retrieval. Accordingly, generative retrieval exhibits behavior analogous to hierarchical search within a tree index in dense retrieval when using hierarchical semantic identifiers. However, prior work focuses solely on the retrieval stage without considering the deep interactions within the decoder of generative retrieval. In this paper, we fill this gap by demonstrating that generative retrieval and multi-vector dense retrieval share the same framework for measuring the relevance to a query of a document. Specifically, we examine the attention layer and prediction head of generative retrieval, revealing that generative retrieval can be understood as a special case of multi-vector dense retrieval. Both methods compute relevance as a sum of products of query and document vectors and an alignment matrix. We then explore how generative retrieval applies this framework, employing distinct strategies for computing document token vectors and the alignment matrix. We have conducted experiments to verify our conclusions and show that both paradigms exhibit commonalities of term matching in their alignment matrix. △ Less

Submitted 31 March, 2024; originally announced April 2024.

Comments: 12 pages, 5 figures, 8 tables, accepted at SIGIR 2024

A Survey of Privacy-Preserving Model Explanations: Privacy Risks, Attacks, and Countermeasures

Authors: Thanh Tam Nguyen, Thanh Trung Huynh, Zhao Ren, Thanh Toan Nguyen, Phi Le Nguyen, Hongzhi Yin, Quoc Viet Hung Nguyen

Abstract: As the adoption of explainable AI (XAI) continues to expand, the urgency to address its privacy implications intensifies. Despite a growing corpus of research in AI privacy and explainability, there is little attention on privacy-preserving model explanations. This article presents the first thorough survey about privacy attacks on model explanations and their countermeasures. Our contribution to… ▽ More As the adoption of explainable AI (XAI) continues to expand, the urgency to address its privacy implications intensifies. Despite a growing corpus of research in AI privacy and explainability, there is little attention on privacy-preserving model explanations. This article presents the first thorough survey about privacy attacks on model explanations and their countermeasures. Our contribution to this field comprises a thorough analysis of research papers with a connected taxonomy that facilitates the categorisation of privacy attacks and countermeasures based on the targeted explanations. This work also includes an initial investigation into the causes of privacy leaks. Finally, we discuss unresolved issues and prospective research directions uncovered in our analysis. This survey aims to be a valuable resource for the research community and offers clear insights for those new to this domain. To support ongoing research, we have established an online resource repository, which will be continuously updated with new and relevant findings. Interested readers are encouraged to access our repository at https://github.com/tamlhp/awesome-privex. △ Less

Submitted 26 June, 2024; v1 submitted 31 March, 2024; originally announced April 2024.

Comments: Revision

CAUSE: Counterfactual Assessment of User Satisfaction Estimation in Task-Oriented Dialogue Systems

Authors: Amin Abolghasemi, Zhaochun Ren, Arian Askari, Mohammad Aliannejadi, Maarten de Rijke, Suzan Verberne

Abstract: An important unexplored aspect in previous work on user satisfaction estimation for Task-Oriented Dialogue (TOD) systems is their evaluation in terms of robustness for the identification of user dissatisfaction: current benchmarks for user satisfaction estimation in TOD systems are highly skewed towards dialogues for which the user is satisfied. The effect of having a more balanced set of satisfac… ▽ More An important unexplored aspect in previous work on user satisfaction estimation for Task-Oriented Dialogue (TOD) systems is their evaluation in terms of robustness for the identification of user dissatisfaction: current benchmarks for user satisfaction estimation in TOD systems are highly skewed towards dialogues for which the user is satisfied. The effect of having a more balanced set of satisfaction labels on performance is unknown. However, balancing the data with more dissatisfactory dialogue samples requires further data collection and human annotation, which is costly and time-consuming. In this work, we leverage large language models (LLMs) and unlock their ability to generate satisfaction-aware counterfactual dialogues to augment the set of original dialogues of a test collection. We gather human annotations to ensure the reliability of the generated samples. We evaluate two open-source LLMs as user satisfaction estimators on our augmented collection against state-of-the-art fine-tuned models. Our experiments show that when used as few-shot user satisfaction estimators, open-source LLMs show higher robustness to the increase in the number of dissatisfaction labels in the test collection than the fine-tuned state-of-the-art models. Our results shed light on the need for data augmentation approaches for user satisfaction estimation in TOD systems. We release our aligned counterfactual dialogues, which are curated by human annotation, to facilitate further research on this topic. △ Less

Submitted 27 March, 2024; originally announced March 2024.

Enhanced Generative Recommendation via Content and Collaboration Integration

Authors: Yidan Wang, Zhaochun Ren, Weiwei Sun, Jiyuan Yang, Zhixiang Liang, Xin Chen, Ruobing Xie, Su Yan, Xu Zhang, Pengjie Ren, Zhumin Chen, Xin Xin

Abstract: Generative recommendation has emerged as a promising paradigm aimed at augmenting recommender systems with recent advancements in generative artificial intelligence. This task has been formulated as a sequence-to-sequence generation process, wherein the input sequence encompasses data pertaining to the user's previously interacted items, and the output sequence denotes the generative identifier fo… ▽ More Generative recommendation has emerged as a promising paradigm aimed at augmenting recommender systems with recent advancements in generative artificial intelligence. This task has been formulated as a sequence-to-sequence generation process, wherein the input sequence encompasses data pertaining to the user's previously interacted items, and the output sequence denotes the generative identifier for the suggested item. However, existing generative recommendation approaches still encounter challenges in (i) effectively integrating user-item collaborative signals and item content information within a unified generative framework, and (ii) executing an efficient alignment between content information and collaborative signals. In this paper, we introduce content-based collaborative generation for recommender systems, denoted as ColaRec. To capture collaborative signals, the generative item identifiers are derived from a pretrained collaborative filtering model, while the user is represented through the aggregation of interacted items' content. Subsequently, the aggregated textual description of items is fed into a language model to encapsulate content information. This integration enables ColaRec to amalgamate collaborative signals and content information within an end-to-end framework. Regarding the alignment, we propose an item indexing task to facilitate the map** between the content-based semantic space and the interaction-based collaborative space. Additionally, a contrastive loss is introduced to ensure that items with similar collaborative GIDs possess comparable content representations, thereby enhancing alignment. To validate the efficacy of ColaRec, we conduct experiments on three benchmark datasets. Empirical results substantiate the superior performance of ColaRec. △ Less

Submitted 27 March, 2024; originally announced March 2024.

Uncovering Selective State Space Model's Capabilities in Lifelong Sequential Recommendation

Authors: Jiyuan Yang, Yuanzi Li, **gyu Zhao, Hanbing Wang, Muyang Ma, Jun Ma, Zhaochun Ren, Mengqi Zhang, Xin Xin, Zhumin Chen, Pengjie Ren

Abstract: Sequential Recommenders have been widely applied in various online services, aiming to model users' dynamic interests from their sequential interactions. With users increasingly engaging with online platforms, vast amounts of lifelong user behavioral sequences have been generated. However, existing sequential recommender models often struggle to handle such lifelong sequences. The primary challeng… ▽ More Sequential Recommenders have been widely applied in various online services, aiming to model users' dynamic interests from their sequential interactions. With users increasingly engaging with online platforms, vast amounts of lifelong user behavioral sequences have been generated. However, existing sequential recommender models often struggle to handle such lifelong sequences. The primary challenges stem from computational complexity and the ability to capture long-range dependencies within the sequence. Recently, a state space model featuring a selective mechanism (i.e., Mamba) has emerged. In this work, we investigate the performance of Mamba for lifelong sequential recommendation (i.e., length>=2k). More specifically, we leverage the Mamba block to model lifelong user sequences selectively. We conduct extensive experiments to evaluate the performance of representative sequential recommendation models in the setting of lifelong sequences. Experiments on two real-world datasets demonstrate the superiority of Mamba. We found that RecMamba achieves performance comparable to the representative model while significantly reducing training duration by approximately 70% and memory costs by 80%. Codes and data are available at \url{https://github.com/nancheng58/RecMamba}. △ Less

Submitted 24 March, 2024; originally announced March 2024.

Explore until Confident: Efficient Exploration for Embodied Question Answering

Authors: Allen Z. Ren, Jaden Clark, Anushri Dixit, Masha Itkina, Anirudha Majumdar, Dorsa Sadigh

Abstract: We consider the problem of Embodied Question Answering (EQA), which refers to settings where an embodied agent such as a robot needs to actively explore an environment to gather information until it is confident about the answer to a question. In this work, we leverage the strong semantic reasoning capabilities of large vision-language models (VLMs) to efficiently explore and answer such questions… ▽ More We consider the problem of Embodied Question Answering (EQA), which refers to settings where an embodied agent such as a robot needs to actively explore an environment to gather information until it is confident about the answer to a question. In this work, we leverage the strong semantic reasoning capabilities of large vision-language models (VLMs) to efficiently explore and answer such questions. However, there are two main challenges when using VLMs in EQA: they do not have an internal memory for map** the scene to be able to plan how to explore over time, and their confidence can be miscalibrated and can cause the robot to prematurely stop exploration or over-explore. We propose a method that first builds a semantic map of the scene based on depth information and via visual prompting of a VLM - leveraging its vast knowledge of relevant regions of the scene for exploration. Next, we use conformal prediction to calibrate the VLM's question answering confidence, allowing the robot to know when to stop exploration - leading to a more calibrated and efficient exploration strategy. To test our framework in simulation, we also contribute a new EQA dataset with diverse, realistic human-robot scenarios and scenes built upon the Habitat-Matterport 3D Research Dataset (HM3D). Both simulated and real robot experiments show our proposed approach improves the performance and efficiency over baselines that do no leverage VLM for exploration or do not calibrate its confidence. Webpage with experiment videos and code: https://explore-eqa.github.io/ △ Less

Submitted 26 May, 2024; v1 submitted 23 March, 2024; originally announced March 2024.

Comments: Robotics: Science and Systems (RSS) 2024

Improving the Robustness of Large Language Models via Consistency Alignment

Authors: Yukun Zhao, Lingyong Yan, Weiwei Sun, Guoliang Xing, Shuaiqiang Wang, Chong Meng, Zhicong Cheng, Zhaochun Ren, Dawei Yin

Abstract: Large language models (LLMs) have shown tremendous success in following user instructions and generating helpful responses. Nevertheless, their robustness is still far from optimal, as they may generate significantly inconsistent responses due to minor changes in the verbalized instructions. Recent literature has explored this inconsistency issue, highlighting the importance of continued improveme… ▽ More Large language models (LLMs) have shown tremendous success in following user instructions and generating helpful responses. Nevertheless, their robustness is still far from optimal, as they may generate significantly inconsistent responses due to minor changes in the verbalized instructions. Recent literature has explored this inconsistency issue, highlighting the importance of continued improvement in the robustness of response generation. However, systematic analysis and solutions are still lacking. In this paper, we quantitatively define the inconsistency problem and propose a two-stage training framework consisting of instruction-augmented supervised fine-tuning and consistency alignment training. The first stage helps a model generalize on following instructions via similar instruction augmentations. In the second stage, we improve the diversity and help the model understand which responses are more aligned with human expectations by differentiating subtle differences in similar responses. The training process is accomplished by self-rewards inferred from the trained model at the first stage without referring to external human preference resources. We conduct extensive experiments on recent publicly available LLMs on instruction-following tasks and demonstrate the effectiveness of our training framework. △ Less

Submitted 22 March, 2024; v1 submitted 21 March, 2024; originally announced March 2024.

Comments: Accepted by LREC-COLING 2024

Tracking of charged particles with nanosecond lifetimes at LHCb

Authors: LHCb collaboration, R. Aaij, A. S. W. Abdelmotteleb, C. Abellan Beteta, F. Abudinén, T. Ackernley, J. A. Adams, A. A. Adefisoye, B. Adeva, M. Adinolfi, P. Adlarson, C. Agapopoulou, C. A. Aidala, Z. Ajaltouni, S. Akar, K. Akiba, P. Albicocco, J. Albrecht, F. Alessio, M. Alexander, Z. Aliouche, P. Alvarez Cartelle, R. Amalric, S. Amato, J. L. Amey , et al. (1060 additional authors not shown)

Abstract: A method is presented to reconstruct charged particles with lifetimes between 10 ps and 10 ns, which considers a combination of their decay products and the partial tracks created by the initial charged particle. Using the $Ξ^-$ baryon as a benchmark, the method is demonstrated with simulated events and proton-proton collision data at $\sqrt{s}=13$ TeV, corresponding to an integrated luminosity of… ▽ More A method is presented to reconstruct charged particles with lifetimes between 10 ps and 10 ns, which considers a combination of their decay products and the partial tracks created by the initial charged particle. Using the $Ξ^-$ baryon as a benchmark, the method is demonstrated with simulated events and proton-proton collision data at $\sqrt{s}=13$ TeV, corresponding to an integrated luminosity of 2.0 fb${}^{-1}$ collected with the LHCb detector in 2018. Significant improvements in the angular resolution and the signal purity are obtained. The method is implemented as part of the LHCb Run 3 event trigger in a set of requirements to select detached hyperons. This is the first demonstration of the applicability of this approach at the LHC, and the first to show its scaling with instantaneous luminosity. △ Less

Submitted 14 March, 2024; originally announced March 2024.

Comments: All figures and tables, along with any supplementary material and additional information, are available at https://cern.ch/lhcbproject/Publications/p/LHCb-DP-2023-004.html (LHCb public pages)

Report number: CERN-EP-2024-077, LHCb-DP-2023-004

Perceive With Confidence: Statistical Safety Assurances for Navigation with Learning-Based Perception

Authors: Anushri Dixit, Zhiting Mei, Meghan Booker, Mariko Storey-Matsutani, Allen Z. Ren, Anirudha Majumdar

Abstract: Rapid advances in perception have enabled large pre-trained models to be used out of the box for processing high-dimensional, noisy, and partial observations of the world into rich geometric representations (e.g., occupancy predictions). However, safe integration of these models onto robots remains challenging due to a lack of reliable performance in unfamiliar environments. In this work, we prese… ▽ More Rapid advances in perception have enabled large pre-trained models to be used out of the box for processing high-dimensional, noisy, and partial observations of the world into rich geometric representations (e.g., occupancy predictions). However, safe integration of these models onto robots remains challenging due to a lack of reliable performance in unfamiliar environments. In this work, we present a framework for rigorously quantifying the uncertainty of pre-trained perception models for occupancy prediction in order to provide end-to-end statistical safety assurances for navigation. We build on techniques from conformal prediction for producing a calibrated perception system that lightly processes the outputs of a pre-trained model while ensuring generalization to novel environments and robustness to distribution shifts in states when perceptual outputs are used in conjunction with a planner. The calibrated system can be used in combination with any safe planner to provide an end-to-end statistical assurance on safety in a new environment with a user-specified threshold $1-ε$. We evaluate the resulting approach - which we refer to as Perceive with Confidence (PwC) - with experiments in simulation and on hardware where a quadruped robot navigates through indoor environments containing objects unseen during training or calibration. These experiments validate the safety assurances provided by PwC and demonstrate significant improvements in empirical safety rates compared to baselines. △ Less

Submitted 12 March, 2024; originally announced March 2024.

Comments: Videos and code can be found at https://perceive-with-confidence.github.io

Harmonious Group Choreography with Trajectory-Controllable Diffusion

Authors: Yuqin Dai, Wanlu Zhu, Ronghui Li, Ze** Ren, Xiangzheng Zhou, Xiu Li, Jun Li, Jian Yang

Abstract: Creating group choreography from music has gained attention in cultural entertainment and virtual reality, aiming to coordinate visually cohesive and diverse group movements. Despite increasing interest, recent works face challenges in achieving aesthetically appealing choreography, primarily for two key issues: multi-dancer collision and single-dancer foot slide. To address these issues, we propo… ▽ More Creating group choreography from music has gained attention in cultural entertainment and virtual reality, aiming to coordinate visually cohesive and diverse group movements. Despite increasing interest, recent works face challenges in achieving aesthetically appealing choreography, primarily for two key issues: multi-dancer collision and single-dancer foot slide. To address these issues, we propose a Trajectory-Controllable Diffusion (TCDiff), a novel approach that harnesses non-overlap** trajectories to facilitate coherent dance movements. Specifically, to tackle dancer collisions, we introduce a Dance-Beat Navigator capable of generating trajectories for multiple dancers based on the music, complemented by a Distance-Consistency loss to maintain appropriate spacing among trajectories within a reasonable threshold. To mitigate foot sliding, we present a Footwork Adaptor that utilizes trajectory displacement from adjacent frames to enable flexible footwork, coupled with a Relative Forward-Kinematic loss to adjust the positioning of individual dancers' root nodes and joints. Extensive experiments demonstrate that our method achieves state-of-the-art results. △ Less

Submitted 6 June, 2024; v1 submitted 10 March, 2024; originally announced March 2024.

On Protecting the Data Privacy of Large Language Models (LLMs): A Survey

Authors: Biwei Yan, Kun Li, Minghui Xu, Yueyan Dong, Yue Zhang, Zhaochun Ren, Xiuzhen Cheng

Abstract: Large language models (LLMs) are complex artificial intelligence systems capable of understanding, generating and translating human language. They learn language patterns by analyzing large amounts of text data, allowing them to perform writing, conversation, summarizing and other language tasks. When LLMs process and generate large amounts of data, there is a risk of leaking sensitive information… ▽ More Large language models (LLMs) are complex artificial intelligence systems capable of understanding, generating and translating human language. They learn language patterns by analyzing large amounts of text data, allowing them to perform writing, conversation, summarizing and other language tasks. When LLMs process and generate large amounts of data, there is a risk of leaking sensitive information, which may threaten data privacy. This paper concentrates on elucidating the data privacy concerns associated with LLMs to foster a comprehensive understanding. Specifically, a thorough investigation is undertaken to delineate the spectrum of data privacy threats, encompassing both passive privacy leakage and active privacy attacks within LLMs. Subsequently, we conduct an assessment of the privacy protection mechanisms employed by LLMs at various stages, followed by a detailed examination of their efficacy and constraints. Finally, the discourse extends to delineate the challenges encountered and outline prospective directions for advancement in the realm of LLM privacy protection. △ Less

Submitted 14 March, 2024; v1 submitted 8 March, 2024; originally announced March 2024.

Comments: 18 pages, 4 figures

A Mixed-Integer Conic Program for the Moving-Target Traveling Salesman Problem based on a Graph of Convex Sets

Authors: Allen George Philip, Zhongqiang Ren, Sivakumar Rathinam, Howie Choset

Abstract: This paper introduces a new formulation that finds the optimum for the Moving-Target Traveling Salesman Problem (MT-TSP), which seeks to find a shortest path for an agent, that starts at a depot, visits a set of moving targets exactly once within their assigned time-windows, and returns to the depot. The formulation relies on the key idea that when the targets move along lines, their trajectories… ▽ More This paper introduces a new formulation that finds the optimum for the Moving-Target Traveling Salesman Problem (MT-TSP), which seeks to find a shortest path for an agent, that starts at a depot, visits a set of moving targets exactly once within their assigned time-windows, and returns to the depot. The formulation relies on the key idea that when the targets move along lines, their trajectories become convex sets within the space-time coordinate system. The problem then reduces to finding the shortest path within a graph of convex sets, subject to some speed constraints. We compare our formulation with the current state-of-the-art Mixed Integer Conic Program (MICP) solver for the MT-TSP. The experimental results show that our formulation outperforms the MICP for instances with up to 20 targets, with up to two orders of magnitude reduction in runtime, and up to a 60\% tighter optimality gap. We also show that the solution cost from the convex relaxation of our formulation provides significantly tighter lower bounds for the MT-TSP than the ones from the MICP. △ Less

Submitted 10 March, 2024; v1 submitted 7 March, 2024; originally announced March 2024.

Comments: 7 pages, 4 figures

TS-RSR: A provably efficient approach for batch bayesian optimization

Authors: Zhaolin Ren, Na Li

Abstract: This paper presents a new approach for batch Bayesian Optimization (BO) called Thompson Sampling-Regret to Sigma Ratio directed sampling (TS-RSR), where we sample a new batch of actions by minimizing a Thompson Sampling approximation of a regret to uncertainty ratio. Our sampling objective is able to coordinate the actions chosen in each batch in a way that minimizes redundancy between points whil… ▽ More This paper presents a new approach for batch Bayesian Optimization (BO) called Thompson Sampling-Regret to Sigma Ratio directed sampling (TS-RSR), where we sample a new batch of actions by minimizing a Thompson Sampling approximation of a regret to uncertainty ratio. Our sampling objective is able to coordinate the actions chosen in each batch in a way that minimizes redundancy between points whilst focusing on points with high predictive means or high uncertainty. Theoretically, we provide rigorous convergence guarantees on our algorithm's regret, and numerically, we demonstrate that our method attains state-of-the-art performance on a range of challenging synthetic and realistic test functions, where it outperforms several competitive benchmark batch BO algorithms. △ Less

Submitted 2 May, 2024; v1 submitted 7 March, 2024; originally announced March 2024.

Comments: Revised presentation and organization of theoretical results

Confidence on the Focal: Conformal Prediction with Selection-Conditional Coverage

Authors: Ying **, Zhimei Ren

Abstract: Conformal prediction builds marginally valid prediction intervals that cover the unknown outcome of a randomly drawn new test point with a prescribed probability. However, a common scenario in practice is that, after seeing the data, practitioners decide which test unit(s) to focus on in a data-driven manner and seek for uncertainty quantification of the focal unit(s). In such cases, marginally va… ▽ More Conformal prediction builds marginally valid prediction intervals that cover the unknown outcome of a randomly drawn new test point with a prescribed probability. However, a common scenario in practice is that, after seeing the data, practitioners decide which test unit(s) to focus on in a data-driven manner and seek for uncertainty quantification of the focal unit(s). In such cases, marginally valid conformal prediction intervals may not provide valid coverage for the focal unit(s) due to selection bias. This paper presents a general framework for constructing a prediction set with finite-sample exact coverage conditional on the unit being selected by a given procedure. The general form of our method works for arbitrary selection rules that are invariant to the permutation of the calibration units, and generalizes Mondrian Conformal Prediction to multiple test units and non-equivariant classifiers. We then work out the computationally efficient implementation of our framework for a number of realistic selection rules, including top-K selection, optimization-based selection, selection based on conformal p-values, and selection based on properties of preliminary conformal prediction sets. The performance of our methods is demonstrated via applications in drug discovery and health risk prediction. △ Less

Submitted 24 March, 2024; v1 submitted 6 March, 2024; originally announced March 2024.

Amplitude analysis of the $Λ_b^0\to pK^-γ$ decay

Authors: LHCb collaboration, R. Aaij, A. S. W. Abdelmotteleb, C. Abellan Beteta, F. Abudinén, T. Ackernley, B. Adeva, M. Adinolfi, P. Adlarson, C. Agapopoulou, C. A. Aidala, Z. Ajaltouni, S. Akar, K. Akiba, P. Albicocco, J. Albrecht, F. Alessio, M. Alexander, A. Alfonso Albero, Z. Aliouche, P. Alvarez Cartelle, R. Amalric, S. Amato, J. L. Amey, Y. Amhis , et al. (1084 additional authors not shown)

Abstract: The resonant structure of the radiative decay $Λ_b^0\to pK^-γ$ in the region of proton-kaon invariant-mass up to 2.5 GeV$/c^2$ is studied using proton-proton collision data recorded at centre-of-mass energies of 7, 8, and 13 TeV collected with the LHCb detector, corresponding to a total integrated luminosity of 9 fb$^{-1}$. Results are given in terms of fit and interference fractions between the d… ▽ More The resonant structure of the radiative decay $Λ_b^0\to pK^-γ$ in the region of proton-kaon invariant-mass up to 2.5 GeV$/c^2$ is studied using proton-proton collision data recorded at centre-of-mass energies of 7, 8, and 13 TeV collected with the LHCb detector, corresponding to a total integrated luminosity of 9 fb$^{-1}$. Results are given in terms of fit and interference fractions between the different components contributing to this final state. Only $Λ$ resonances decaying to $pK^-$ are found to be relevant, where the largest contributions stem from the $Λ(1520)$, $Λ(1600)$, $Λ(1800)$, and $Λ(1890)$ states. △ Less

Submitted 21 June, 2024; v1 submitted 6 March, 2024; originally announced March 2024.

Comments: All figures and tables, along with machine-readable versions and any supplementary material and additional information, are available at https://cern.ch/lhcbproject/Publications/p/LHCb-PAPER-2023-036.html (LHCb public pages)

Report number: LHCb-PAPER-2023-036, CERN-EP-2023-253

Journal ref: JHEP 06 (2024) 098

First observation of the $Λ^0_b \to D^+ D^- Λ$ decay

Authors: LHCb collaboration, R. Aaij, A. S. W. Abdelmotteleb, C. Abellan Beteta, F. Abudinén, T. Ackernley, J. A. Adams, A. A. Adefisoye, B. Adeva, M. Adinolfi, P. Adlarson, C. Agapopoulou, C. A. Aidala, Z. Ajaltouni, S. Akar, K. Akiba, P. Albicocco, J. Albrecht, F. Alessio, M. Alexander, Z. Aliouche, P. Alvarez Cartelle, R. Amalric, S. Amato, J. L. Amey , et al. (1068 additional authors not shown)

Abstract: The $Λ^0_b \to D^+ D^- Λ$ decay is observed for the first time using proton-proton collision data collected by the LHCb experiment at a center-of-mass energy of $13 \mathrm{TeV}$, corresponding to an integrated luminosity of $5.3 \mathrm{fb}^{-1}$. Using the $B^0 \to D^+ D^- K_{\mathrm{S}}^0$ decay as a reference channel, the product of the relative production cross-section and decay branching fra… ▽ More The $Λ^0_b \to D^+ D^- Λ$ decay is observed for the first time using proton-proton collision data collected by the LHCb experiment at a center-of-mass energy of $13 \mathrm{TeV}$, corresponding to an integrated luminosity of $5.3 \mathrm{fb}^{-1}$. Using the $B^0 \to D^+ D^- K_{\mathrm{S}}^0$ decay as a reference channel, the product of the relative production cross-section and decay branching fractions is measured to be $$ {\cal R}=\frac{σ_{Λ^0_b}}{σ_{B^0}} \times \frac{{\cal B}(Λ^0_b \to D^+ D^- Λ)}{{\cal B}(B^0 \to D^+ D^- K_{\mathrm{S}}^0)}=0.179 \pm 0.022 \pm 0.014 $$ where the first uncertainty is statistical and the second is systematic. The known branching fraction of the reference channel, ${\cal B}(B^0 \to D^+ D^- K_{\mathrm{S}}^0)$, and the cross-section ratio, $σ_{Λ^0_b} / σ_{B^0}$, previously measured by $\mathrm{LHCb}$ are used to derive the branching fraction of the $Λ^0_b \to D^+ D^- Λ$ decay $$ {\cal B}(Λ^0_b \to D^+ D^- Λ)=(1.24 \pm 0.15 \pm 0.10 \pm 0.28 \pm 0.11) \times 10^{-4}, $$ where the third and fourth contributions are due to uncertainties of ${\cal B}(B^0 \to D^+ D^- K_{\mathrm{S}}^0)$ and $σ_{Λ^0_b} / σ_{B^0}$, respectively. Inspection of the $D^+ Λ$ and $D^+ D^-$ invariant-mass distributions suggests a rich presence of intermediate resonances in the decay. The $Λ^0_b \to D^{*+} D^- Λ$ decay is also observed for the first time as a partially reconstructed component in the $D^+ D^- Λ$ invariant mass spectrum. △ Less

Submitted 6 March, 2024; originally announced March 2024.

Comments: All figures and tables, along with machine-readable versions and any supplementary material and additional information, are available at https://cern.ch/lhcbproject/Publications/p/LHCb-PAPER-2023-042.html (LHCb public pages)

Report number: LHCb-PAPER-2023-042, CERN-EP-2024-041

Generative News Recommendation

Authors: Shen Gao, Jiabao Fang, Quan Tu, Zhitao Yao, Zhumin Chen, Pengjie Ren, Zhaochun Ren

Abstract: Most existing news recommendation methods tackle this task by conducting semantic matching between candidate news and user representation produced by historical clicked news. However, they overlook the high-level connections among different news articles and also ignore the profound relationship between these news articles and users. And the definition of these methods dictates that they can only… ▽ More Most existing news recommendation methods tackle this task by conducting semantic matching between candidate news and user representation produced by historical clicked news. However, they overlook the high-level connections among different news articles and also ignore the profound relationship between these news articles and users. And the definition of these methods dictates that they can only deliver news articles as-is. On the contrary, integrating several relevant news articles into a coherent narrative would assist users in gaining a quicker and more comprehensive understanding of events. In this paper, we propose a novel generative news recommendation paradigm that includes two steps: (1) Leveraging the internal knowledge and reasoning capabilities of the Large Language Model (LLM) to perform high-level matching between candidate news and user representation; (2) Generating a coherent and logically structured narrative based on the associations between related news and user interests, thus engaging users in further reading of the news. Specifically, we propose GNR to implement the generative news recommendation paradigm. First, we compose the dual-level representation of news and users by leveraging LLM to generate theme-level representations and combine them with semantic-level representations. Next, in order to generate a coherent narrative, we explore the news relation and filter the related news according to the user preference. Finally, we propose a novel training method named UIFT to train the LLM to fuse multiple news articles in a coherent narrative. Extensive experiments show that GNR can improve recommendation accuracy and eventually generate more personalized and factually consistent narratives. △ Less

Submitted 5 March, 2024; originally announced March 2024.

Comments: Accepted by WWW 2024

Learning to Use Tools via Cooperative and Interactive Agents

Authors: Zhengliang Shi, Shen Gao, Xiuyi Chen, Yue Feng, Lingyong Yan, Haibo Shi, Dawei Yin, Pengjie Ren, Suzan Verberne, Zhaochun Ren

Abstract: Tool learning empowers large language models (LLMs) as agents to use external tools and extend their utility. Existing methods employ one single LLM-based agent to iteratively select and execute tools, thereafter incorporating execution results into the next action prediction. Despite their progress, these methods suffer from performance degradation when addressing practical tasks due to: (1) the… ▽ More Tool learning empowers large language models (LLMs) as agents to use external tools and extend their utility. Existing methods employ one single LLM-based agent to iteratively select and execute tools, thereafter incorporating execution results into the next action prediction. Despite their progress, these methods suffer from performance degradation when addressing practical tasks due to: (1) the pre-defined pipeline with restricted flexibility to calibrate incorrect actions, and (2) the struggle to adapt a general LLM-based agent to perform a variety of specialized actions. To mitigate these problems, we propose ConAgents, a Cooperative and interactive Agents framework, which coordinates three specialized agents for tool selection, tool execution, and action calibration separately. ConAgents introduces two communication protocols to enable the flexible cooperation of agents. To effectively generalize the ConAgents into open-source models, we also propose specialized action distillation, enhancing their ability to perform specialized actions in our framework. Our extensive experiments on three datasets show that the LLMs, when equipped with the ConAgents, outperform baselines with substantial improvement (i.e., up to 14% higher success rate). △ Less

Submitted 22 June, 2024; v1 submitted 5 March, 2024; originally announced March 2024.

Comments: working in process

Complete Interband Transitions for Non-Hermitian Spin-Orbit-Coupled Cold-Atom Systems

Authors: Dong Liu, Zejian Ren, Wai Chun Wong, Entong Zhao, Chengdong He, Ka Kwan Pak, Gyu-Boong Jo, Jensen Li

Abstract: Recently, synthetic spin-orbit coupling has been introduced into cold-atom systems for more flexible control of the Hamiltonian, which was further made time-varying through two-photon detuning to achieve dynamic control of the cold-atom state. While an intraband transition can be adiabatically obtained, a complete interband transition, rather than a superposition of different bands, obtained throu… ▽ More Recently, synthetic spin-orbit coupling has been introduced into cold-atom systems for more flexible control of the Hamiltonian, which was further made time-varying through two-photon detuning to achieve dynamic control of the cold-atom state. While an intraband transition can be adiabatically obtained, a complete interband transition, rather than a superposition of different bands, obtained through fast swee** is usually guaranteed by having the positions of the initial and final states be far away from any band gap in the quasimomentum space. Here, by introducing an additional non-Hermitian parameter through an atom-loss contrast together with two-photon detuning as two controllable external parameters, both intraband and complete interband transitions can be achieved independent of the positions of the initial and final states. In addition, a point-source diagram approach in the 2D external parameter space is developed to visualize and predict the locations of any nonadiabatic transitions. This control protocol can have potential applications in quantum state control and quantum simulations using cold-atom systems. △ Less

Submitted 4 March, 2024; originally announced March 2024.

Comments: 21 pages, 4 figures

An Iterative Associative Memory Model for Empathetic Response Generation

Authors: Zhou Yang, Zhaochun Ren, Yufeng Wang, Chao Chen, Haizhou Sun, Xiaofei Zhu, Xiangwen Liao

Abstract: Empathetic response generation aims to comprehend the cognitive and emotional states in dialogue utterances and generate proper responses. Psychological theories posit that comprehending emotional and cognitive states necessitates iteratively capturing and understanding associated words across dialogue utterances. However, existing approaches regard dialogue utterances as either a long sequence or… ▽ More Empathetic response generation aims to comprehend the cognitive and emotional states in dialogue utterances and generate proper responses. Psychological theories posit that comprehending emotional and cognitive states necessitates iteratively capturing and understanding associated words across dialogue utterances. However, existing approaches regard dialogue utterances as either a long sequence or independent utterances for comprehension, which are prone to overlook the associated words between them. To address this issue, we propose an Iterative Associative Memory Model (IAMM) for empathetic response generation. Specifically, we employ a novel second-order interaction attention mechanism to iteratively capture vital associated words between dialogue utterances and situations, dialogue history, and a memory module (for storing associated words), thereby accurately and nuancedly comprehending the utterances. We conduct experiments on the Empathetic-Dialogue dataset. Both automatic and human evaluations validate the efficacy of the model. Variant experiments on LLMs also demonstrate that attending to associated words improves empathetic comprehension and expression. △ Less

Submitted 2 June, 2024; v1 submitted 27 February, 2024; originally announced February 2024.

Comments: 12 pages, 4 figures

Nissist: An Incident Mitigation Copilot based on Troubleshooting Guides

Authors: Kaikai An, Fangkai Yang, Junting Lu, Liqun Li, Zhixing Ren, Hao Huang, Lu Wang, Pu Zhao, Yu Kang, Hua Ding, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Qi Zhang

Abstract: Effective incident management is pivotal for the smooth operation of enterprises-level cloud services. In order to expedite incident mitigation, service teams compile troubleshooting knowledge into Troubleshooting Guides (TSGs) accessible to on-call engineers (OCEs). While automated pipelines are enabled to resolve the most frequent and easy incidents, there still exist complex incidents that requ… ▽ More Effective incident management is pivotal for the smooth operation of enterprises-level cloud services. In order to expedite incident mitigation, service teams compile troubleshooting knowledge into Troubleshooting Guides (TSGs) accessible to on-call engineers (OCEs). While automated pipelines are enabled to resolve the most frequent and easy incidents, there still exist complex incidents that require OCEs' intervention. However, TSGs are often unstructured and incomplete, which requires manual interpretation by OCEs, leading to on-call fatigue and decreased productivity, especially among new-hire OCEs. In this work, we propose Nissist which leverages TSGs and incident mitigation histories to provide proactive suggestions, reducing human intervention. Leveraging Large Language Models (LLM), Nissist extracts insights from unstructured TSGs and historical incident mitigation discussions, forming a comprehensive knowledge base. Its multi-agent system design enhances proficiency in precisely discerning user queries, retrieving relevant information, and delivering systematic plans consecutively. Through our user case and experiment, we demonstrate that Nissist significant reduce Time to Mitigate (TTM) in incident mitigation, alleviating operational burdens on OCEs and improving service reliability. Our demo is available at https://aka.ms/nissist_demo. △ Less

Submitted 10 May, 2024; v1 submitted 27 February, 2024; originally announced February 2024.

Comments: Work in progress

Exploiting Emotion-Semantic Correlations for Empathetic Response Generation

Authors: Zhou Yang, Zhaochun Ren, Yufeng Wang, Xiaofei Zhu, Zhihao Chen, Tiecheng Cai, Yunbing Wu, Yisong Su, Sibo Ju, Xiangwen Liao

Abstract: Empathetic response generation aims to generate empathetic responses by understanding the speaker's emotional feelings from the language of dialogue. Recent methods capture emotional words in the language of communicators and construct them as static vectors to perceive nuanced emotions. However, linguistic research has shown that emotional words in language are dynamic and have correlations with… ▽ More Empathetic response generation aims to generate empathetic responses by understanding the speaker's emotional feelings from the language of dialogue. Recent methods capture emotional words in the language of communicators and construct them as static vectors to perceive nuanced emotions. However, linguistic research has shown that emotional words in language are dynamic and have correlations with other grammar semantic roles, i.e., words with semantic meanings, in grammar. Previous methods overlook these two characteristics, which easily lead to misunderstandings of emotions and neglect of key semantics. To address this issue, we propose a dynamical Emotion-Semantic Correlation Model (ESCM) for empathetic dialogue generation tasks. ESCM constructs dynamic emotion-semantic vectors through the interaction of context and emotions. We introduce dependency trees to reflect the correlations between emotions and semantics. Based on dynamic emotion-semantic vectors and dependency trees, we propose a dynamic correlation graph convolutional network to guide the model in learning context meanings in dialogue and generating empathetic responses. Experimental results on the EMPATHETIC-DIALOGUES dataset show that ESCM understands semantics and emotions more accurately and expresses fluent and informative empathetic responses. Our analysis results also indicate that the correlations between emotions and semantics are frequently used in dialogues, which is of great significance for empathetic perception and expression. △ Less

Submitted 27 February, 2024; originally announced February 2024.

Comments: 12 pages, 3 figures, Findings of EMNLP 2023

Superconducting-transition-temperature dependence of superfluid density and conductivity in pressurized cuprate superconductors

Authors: **yu Zhao, Shu Cai, Yiwen Chen, Genda Gu, Hongtao Yan, **g Guo, **yu Han, Pengyu Wang, Yazhou Zhou, Yanchun Li, Xiaodong Li, Zhian Ren, Qi Wu, Xingjiang Zhou, Yang Ding, Tao Xiang, Ho-kwang Mao, Liling Sun

Abstract: What factors fundamentally determine the value of superconducting transition temperature (Tc) in high temperature superconductors has been the subject of intense debate. Following the establishment of an empirical law known as Homes'law, there is a growing consensus in the community that the Tc value of the cuprate superconductors is closely linked to its superfluid density and conductivity. Howev… ▽ More What factors fundamentally determine the value of superconducting transition temperature (Tc) in high temperature superconductors has been the subject of intense debate. Following the establishment of an empirical law known as Homes'law, there is a growing consensus in the community that the Tc value of the cuprate superconductors is closely linked to its superfluid density and conductivity. However, all the data supporting this empirical law have been obtained from the ambient-pressure superconductors. In this study, we present the first high-pressure results about the connection of these two quantities with Tc, through the studies on the Bi1.74Pb0.38Sr1.88CuO6+delta and Bi2Sr2CaCu2O8+delta, in which the value of their high-pressure resistivity (the reciprocal of conductivity) is achieved by adopting our newly established method, while the value of superfluid density is extracted using the Homes'law. We highlight that the Tc values are strongly linked the two joint response factors of magnetic field and electric field, i.e. superfluid density and conductivity, respectively, implying that the physics governing the determination of Tc is influenced by the intrinsic electromagnetic fields of the system. △ Less

Submitted 28 February, 2024; v1 submitted 27 February, 2024; originally announced February 2024.

Comments: 11 pages, 3 figures

MELoRA: Mini-Ensemble Low-Rank Adapters for Parameter-Efficient Fine-Tuning

Authors: Pengjie Ren, Chengshun Shi, Shiguang Wu, Mengqi Zhang, Zhaochun Ren, Maarten de Rijke, Zhumin Chen, Jiahuan Pei

Abstract: Parameter-efficient fine-tuning (PEFT) is a popular method for tailoring pre-trained large language models (LLMs), especially as the models' scale and the diversity of tasks increase. Low-rank adaptation (LoRA) is based on the idea that the adaptation process is intrinsically low-dimensional, i.e., significant model changes can be represented with relatively few parameters. However, decreasing the… ▽ More Parameter-efficient fine-tuning (PEFT) is a popular method for tailoring pre-trained large language models (LLMs), especially as the models' scale and the diversity of tasks increase. Low-rank adaptation (LoRA) is based on the idea that the adaptation process is intrinsically low-dimensional, i.e., significant model changes can be represented with relatively few parameters. However, decreasing the rank encounters challenges with generalization errors for specific tasks when compared to full-parameter fine-tuning. We present MELoRA, a mini-ensemble low-rank adapters that uses fewer trainable parameters while maintaining a higher rank, thereby offering improved performance potential. The core idea is to freeze original pretrained weights and train a group of mini LoRAs with only a small number of parameters. This can capture a significant degree of diversity among mini LoRAs, thus promoting better generalization ability. We conduct a theoretical analysis and empirical studies on various NLP tasks. Our experimental results show that, compared to LoRA, MELoRA achieves better performance with 8 times fewer trainable parameters on natural language understanding tasks and 36 times fewer trainable parameters on instruction following tasks, which demonstrates the effectiveness of MELoRA. △ Less

Submitted 24 June, 2024; v1 submitted 27 February, 2024; originally announced February 2024.

Comments: ACL2024

MSC Class: 68T50 ACM Class: I.2.7

How Large Language Models Encode Context Knowledge? A Layer-Wise Probing Study

Authors: Tianjie Ju, Weiwei Sun, Wei Du, Xinwei Yuan, Zhaochun Ren, Gongshen Liu

Abstract: Previous work has showcased the intriguing capability of large language models (LLMs) in retrieving facts and processing context knowledge. However, only limited research exists on the layer-wise capability of LLMs to encode knowledge, which challenges our understanding of their internal mechanisms. In this paper, we devote the first attempt to investigate the layer-wise capability of LLMs through… ▽ More Previous work has showcased the intriguing capability of large language models (LLMs) in retrieving facts and processing context knowledge. However, only limited research exists on the layer-wise capability of LLMs to encode knowledge, which challenges our understanding of their internal mechanisms. In this paper, we devote the first attempt to investigate the layer-wise capability of LLMs through probing tasks. We leverage the powerful generative capability of ChatGPT to construct probing datasets, providing diverse and coherent evidence corresponding to various facts. We employ $\mathcal V$-usable information as the validation metric to better reflect the capability in encoding context knowledge across different layers. Our experiments on conflicting and newly acquired knowledge show that LLMs: (1) prefer to encode more context knowledge in the upper layers; (2) primarily encode context knowledge within knowledge-related entity tokens at lower layers while progressively expanding more knowledge within other tokens at upper layers; and (3) gradually forget the earlier context knowledge retained within the intermediate layers when provided with irrelevant evidence. Code is publicly available at https://github.com/Jometeorie/probing_llama. △ Less

Submitted 4 March, 2024; v1 submitted 25 February, 2024; originally announced February 2024.

Comments: Accepted at LREC-COLING 2024 (Long Paper)

Modification of $χ_{c1}$(3872) and $ψ$(2$S$) production in $p$Pb collisions at $\sqrt{s_{NN}} = 8.16$ TeV

Authors: LHCb collaboration, R. Aaij, A. S. W. Abdelmotteleb, C. Abellan Beteta, F. Abudinén, T. Ackernley, B. Adeva, M. Adinolfi, P. Adlarson, C. Agapopoulou, C. A. Aidala, Z. Ajaltouni, S. Akar, K. Akiba, P. Albicocco, J. Albrecht, F. Alessio, M. Alexander, A. Alfonso Albero, Z. Aliouche, P. Alvarez Cartelle, R. Amalric, S. Amato, J. L. Amey, Y. Amhis , et al. (1082 additional authors not shown)

Abstract: The LHCb collaboration measures production of the exotic hadron $χ_{c1}$(3872) in proton-nucleus collisions for the first time. Comparison with the charmonium state $ψ$(2$S$) suggests that the exotic $χ_{c1}$(3872) experiences different dynamics in the nuclear medium than conventional hadrons, and comparison with data from proton-proton collisions indicates that the presence of the nucleus may mod… ▽ More The LHCb collaboration measures production of the exotic hadron $χ_{c1}$(3872) in proton-nucleus collisions for the first time. Comparison with the charmonium state $ψ$(2$S$) suggests that the exotic $χ_{c1}$(3872) experiences different dynamics in the nuclear medium than conventional hadrons, and comparison with data from proton-proton collisions indicates that the presence of the nucleus may modify $χ_{c1}$(3872) production rates. This is the first measurement of the nuclear modification factor of an exotic hadron. △ Less

Submitted 19 June, 2024; v1 submitted 22 February, 2024; originally announced February 2024.

Comments: All figures and tables, along with any supplementary material and additional information, are available at https://cern.ch/lhcbproject/Publications/p/LHCb-PAPER-2023-026.html (LHCb public pages)

Report number: LHCb-PAPER-2023-026, CERN-EP-2024-033

Journal ref: Phys. Rev. Lett. 132 (2024) 242301

Structural and resistivity properties of Fe$_{1-x}$Co${_x}$Se single crystals grown by the molten salt method

Authors: Qiaoyu Wang, Mingwei Ma, Binbin Ruan, Menghu Zhou, Yadong Gu, Qingsong Yang, Lewei Chen, Yunqing Shi, Junkun Yi, Genfu Chen, Zhian Ren

Abstract: A series of tetragonal Fe$_{1-x}$Co${_x}$Se single crystals with a complete Co do** range (0$\leq$x$\leq$0.52) up to its solid solubility limit in FeSe have been grown by an eutectic AlCl${_3}$/KCl molten salt method. The typical lateral size of as-grown Fe$_{1-x}$Co${_x}$Se single crystals is 1$-$5 mm. The chemical composition and homogeneity of the crystals was examined by both inductively cou… ▽ More A series of tetragonal Fe$_{1-x}$Co${_x}$Se single crystals with a complete Co do** range (0$\leq$x$\leq$0.52) up to its solid solubility limit in FeSe have been grown by an eutectic AlCl${_3}$/KCl molten salt method. The typical lateral size of as-grown Fe$_{1-x}$Co${_x}$Se single crystals is 1$-$5 mm. The chemical composition and homogeneity of the crystals was examined by both inductively coupled plasma atomic emission spectroscopy and energy dispersive spectrometer. X-ray diffraction analysis demonstrates that the crystal lattice parameters $a$ and $c$ are both linearly decreased with increasing Co do** level x. In the whole do** range, all the samples show metallic behaviour in contrast to a metal insulator transition of Cu-doped FeSe according to the resistivity measurements △ Less

Submitted 22 February, 2024; originally announced February 2024.

VN Network: Embedding Newly Emerging Entities with Virtual Neighbors

Authors: Yongquan He, Zihan Wang, Peng Zhang, Zhaopeng Tu, Zhaochun Ren

Abstract: Embedding entities and relations into continuous vector spaces has attracted a surge of interest in recent years. Most embedding methods assume that all test entities are available during training, which makes it time-consuming to retrain embeddings for newly emerging entities. To address this issue, recent works apply the graph neural network on the existing neighbors of the unseen entities. In t… ▽ More Embedding entities and relations into continuous vector spaces has attracted a surge of interest in recent years. Most embedding methods assume that all test entities are available during training, which makes it time-consuming to retrain embeddings for newly emerging entities. To address this issue, recent works apply the graph neural network on the existing neighbors of the unseen entities. In this paper, we propose a novel framework, namely Virtual Neighbor (VN) network, to address three key challenges. Firstly, to reduce the neighbor sparsity problem, we introduce the concept of the virtual neighbors inferred by rules. And we assign soft labels to these neighbors by solving a rule-constrained problem, rather than simply regarding them as unquestionably true. Secondly, many existing methods only use one-hop or two-hop neighbors for aggregation and ignore the distant information that may be helpful. Instead, we identify both logic and symmetric path rules to capture complex patterns. Finally, instead of one-time injection of rules, we employ an iterative learning scheme between the embedding method and virtual neighbor prediction to capture the interactions within. Experimental results on two knowledge graph completion tasks demonstrate that our VN network significantly outperforms state-of-the-art baselines. Furthermore, results on Subject/Object-R show that our proposed VN network is highly robust to the neighbor sparsity problem. △ Less

Submitted 20 February, 2024; originally announced February 2024.

Comments: 10 pages, 5 figures

ACM Class: I.2.4; I.2.6

Journal ref: CIKM (2020) 505-514

Healthcare Copilot: Eliciting the Power of General LLMs for Medical Consultation

Authors: Zhiyao Ren, Yibing Zhan, Baosheng Yu, Liang Ding, Dacheng Tao

Abstract: The copilot framework, which aims to enhance and tailor large language models (LLMs) for specific complex tasks without requiring fine-tuning, is gaining increasing attention from the community. In this paper, we introduce the construction of a Healthcare Copilot designed for medical consultation. The proposed Healthcare Copilot comprises three main components: 1) the Dialogue component, responsib… ▽ More The copilot framework, which aims to enhance and tailor large language models (LLMs) for specific complex tasks without requiring fine-tuning, is gaining increasing attention from the community. In this paper, we introduce the construction of a Healthcare Copilot designed for medical consultation. The proposed Healthcare Copilot comprises three main components: 1) the Dialogue component, responsible for effective and safe patient interactions; 2) the Memory component, storing both current conversation data and historical patient information; and 3) the Processing component, summarizing the entire dialogue and generating reports. To evaluate the proposed Healthcare Copilot, we implement an auto-evaluation scheme using ChatGPT for two roles: as a virtual patient engaging in dialogue with the copilot, and as an evaluator to assess the quality of the dialogue. Extensive results demonstrate that the proposed Healthcare Copilot significantly enhances the capabilities of general LLMs for medical consultations in terms of inquiry capability, conversational fluency, response accuracy, and safety. Furthermore, we conduct ablation studies to highlight the contribution of each individual module in the Healthcare Copilot. Code will be made publicly available on GitHub. △ Less

Submitted 20 February, 2024; originally announced February 2024.

Not all distributional shifts are equal: Fine-grained robust conformal inference

Authors: Jiahao Ai, Zhimei Ren

Abstract: We introduce a fine-grained framework for uncertainty quantification of predictive models under distributional shifts. This framework distinguishes the shift in covariate distributions from that in the conditional relationship between the outcome ($Y$) and the covariates ($X$). We propose to reweight the training samples to adjust for an identifiable covariate shift while protecting against worst-… ▽ More We introduce a fine-grained framework for uncertainty quantification of predictive models under distributional shifts. This framework distinguishes the shift in covariate distributions from that in the conditional relationship between the outcome ($Y$) and the covariates ($X$). We propose to reweight the training samples to adjust for an identifiable covariate shift while protecting against worst-case conditional distribution shift bounded in an $f$-divergence ball. Based on ideas from conformal inference and distributionally robust learning, we present an algorithm that outputs (approximately) valid and efficient prediction intervals in the presence of distributional shifts. As a use case, we apply the framework to sensitivity analysis of individual treatment effects with hidden confounding. The proposed methods are evaluated in simulation studies and three real data applications, demonstrating superior robustness and efficiency compared with existing benchmarks. △ Less

Submitted 25 February, 2024; v1 submitted 20 February, 2024; originally announced February 2024.

Comments: 25 pages, 5 figures

Enhancing Empathetic Response Generation by Augmenting LLMs with Small-scale Empathetic Models

Authors: Zhou Yang, Zhaochun Ren, Wang Yufeng, Shizhong Peng, Haizhou Sun, Xiaofei Zhu, Xiangwen Liao

Abstract: Empathetic response generation is increasingly significant in AI, necessitating nuanced emotional and cognitive understanding coupled with articulate response expression. Current large language models (LLMs) excel in response expression; however, they lack the ability to deeply understand emotional and cognitive nuances, particularly in pinpointing fine-grained emotions and their triggers. Convers… ▽ More Empathetic response generation is increasingly significant in AI, necessitating nuanced emotional and cognitive understanding coupled with articulate response expression. Current large language models (LLMs) excel in response expression; however, they lack the ability to deeply understand emotional and cognitive nuances, particularly in pinpointing fine-grained emotions and their triggers. Conversely, small-scale empathetic models (SEMs) offer strength in fine-grained emotion detection and detailed emotion cause identification. To harness the complementary strengths of both LLMs and SEMs, we introduce a Hybrid Empathetic Framework (HEF). HEF regards SEMs as flexible plugins to improve LLM's nuanced emotional and cognitive understanding. Regarding emotional understanding, HEF implements a two-stage emotion prediction strategy, encouraging LLMs to prioritize primary emotions emphasized by SEMs, followed by other categories, substantially alleviates the difficulties for LLMs in fine-grained emotion detection. Regarding cognitive understanding, HEF employs an emotion cause perception strategy, prompting LLMs to focus on crucial emotion-eliciting words identified by SEMs, thus boosting LLMs' capabilities in identifying emotion causes. This collaborative approach enables LLMs to discern emotions more precisely and formulate empathetic responses. We validate HEF on the Empathetic-Dialogue dataset, and the findings indicate that our framework enhances the refined understanding of LLMs and their ability to convey empathetic responses. △ Less

Submitted 18 February, 2024; originally announced February 2024.

Comments: 12 pages, 4 figures

KnowTuning: Knowledge-aware Fine-tuning for Large Language Models

Authors: Yougang Lyu, Lingyong Yan, Shuaiqiang Wang, Haibo Shi, Dawei Yin, Pengjie Ren, Zhumin Chen, Maarten de Rijke, Zhaochun Ren

Abstract: Despite their success at many natural language processing (NLP) tasks, large language models still struggle to effectively leverage knowledge for knowledge-intensive tasks, manifesting limitations such as generating incomplete, non-factual, or illogical answers. These limitations stem from inadequate knowledge awareness of LLMs during vanilla fine-tuning. To address these problems, we propose a kn… ▽ More Despite their success at many natural language processing (NLP) tasks, large language models still struggle to effectively leverage knowledge for knowledge-intensive tasks, manifesting limitations such as generating incomplete, non-factual, or illogical answers. These limitations stem from inadequate knowledge awareness of LLMs during vanilla fine-tuning. To address these problems, we propose a knowledge-aware fine-tuning (KnowTuning) method to improve fine-grained and coarse-grained knowledge awareness of LLMs. We devise a fine-grained knowledge augmentation stage to train LLMs to identify difficult fine-grained knowledge in answers. We also propose a coarse-grained knowledge comparison stage to train LLMs to distinguish between reliable and unreliable knowledge, in three aspects: completeness, factuality, and logicality. Extensive experiments on both generic and medical question answering (QA) datasets confirm the effectiveness of KnowTuning, through automatic and human evaluations, across various sizes of LLMs. We further verify that KnowTuning generates more facts with less factual error rate under fine-grained facts evaluation. △ Less

Submitted 17 April, 2024; v1 submitted 16 February, 2024; originally announced February 2024.

Language Models as Science Tutors

Authors: Alexis Chevalier, Jiayi Geng, Alexander Wettig, Howard Chen, Sebastian Mizera, Toni Annala, Max Jameson Aragon, Arturo Rodríguez Fanlo, Simon Frieder, Simon Machado, Akshara Prabhakar, Ellie Thieu, Jiachen T. Wang, Zirui Wang, Xindi Wu, Mengzhou Xia, Wenhan Jia, Jiatong Yu, Jun-Jie Zhu, Zhiyong Jason Ren, Sanjeev Arora, Danqi Chen

Abstract: NLP has recently made exciting progress toward training language models (LMs) with strong scientific problem-solving skills. However, model development has not focused on real-life use-cases of LMs for science, including applications in education that require processing long scientific documents. To address this, we introduce TutorEval and TutorChat. TutorEval is a diverse question-answering bench… ▽ More NLP has recently made exciting progress toward training language models (LMs) with strong scientific problem-solving skills. However, model development has not focused on real-life use-cases of LMs for science, including applications in education that require processing long scientific documents. To address this, we introduce TutorEval and TutorChat. TutorEval is a diverse question-answering benchmark consisting of questions about long chapters from STEM textbooks, written by experts. TutorEval helps measure real-life usability of LMs as scientific assistants, and it is the first benchmark combining long contexts, free-form generation, and multi-disciplinary scientific knowledge. Moreover, we show that fine-tuning base models with existing dialogue datasets leads to poor performance on TutorEval. Therefore, we create TutorChat, a dataset of 80,000 long synthetic dialogues about textbooks. We use TutorChat to fine-tune Llemma models with 7B and 34B parameters. These LM tutors specialized in math have a 32K-token context window, and they excel at TutorEval while performing strongly on GSM8K and MATH. Our datasets build on open-source materials, and we release our models, data, and evaluations. △ Less

Submitted 16 February, 2024; originally announced February 2024.

Comments: 8 pages without bibliography and appendix, 26 pages total

LLM-based Federated Recommendation

Authors: Jujia Zhao, Wenjie Wang, Chen Xu, Zhaochun Ren, See-Kiong Ng, Tat-Seng Chua

Abstract: Large Language Models (LLMs), with their advanced contextual understanding abilities, have demonstrated considerable potential in enhancing recommendation systems via fine-tuning methods. However, fine-tuning requires users' behavior data, which poses considerable privacy risks due to the incorporation of sensitive user information. The unintended disclosure of such data could infringe upon data p… ▽ More Large Language Models (LLMs), with their advanced contextual understanding abilities, have demonstrated considerable potential in enhancing recommendation systems via fine-tuning methods. However, fine-tuning requires users' behavior data, which poses considerable privacy risks due to the incorporation of sensitive user information. The unintended disclosure of such data could infringe upon data protection laws and give rise to ethical issues. To mitigate these privacy issues, Federated Learning for Recommendation (Fed4Rec) has emerged as a promising approach. Nevertheless, applying Fed4Rec to LLM-based recommendation presents two main challenges: first, an increase in the imbalance of performance across clients, affecting the system's efficiency over time, and second, a high demand on clients' computational and storage resources for local training and inference of LLMs. To address these challenges, we introduce a Privacy-Preserving LLM-based Recommendation (PPLR) framework. The PPLR framework employs two primary strategies. First, it implements a dynamic balance strategy, which involves the design of dynamic parameter aggregation and adjustment of learning speed for different clients during the training phase, to ensure relatively balanced performance across all clients. Second, PPLR adopts a flexible storage strategy, selectively retaining certain sensitive layers of the language model on the client side while offloading non-sensitive layers to the server. This approach aims to preserve user privacy while efficiently saving computational and storage resources. Experimental results demonstrate that PPLR not only achieves a balanced performance among clients but also enhances overall system performance in a manner that is both computationally and storage-efficient, while effectively protecting user privacy. △ Less

Submitted 16 February, 2024; v1 submitted 15 February, 2024; originally announced February 2024.

Alpha-like correlations in $^{20}$Ne, comparison of quartetting wave function and THSR approaches

Authors: G. Röpke, C. Xu, B. Zhou, Z. Z. Ren, Y. Funaki, H. Horiuchi, M. Lyu, A. Tohsaki, T. Yamada

Abstract: $^{20}$Ne can be considered as a double-magic $^{16}$O core nucleus surrounded by four nucleons, the constituents of an $α$-like quartet. Similar to other nuclei ($^{212}$Po, $^{104}$Ti, etc.) with a quartet on top of a double-magic core nucleus, significant $α$-like correlations are expected. Correlations in the ground state of $^{20}… ▽ More $^{20}$Ne can be considered as a double-magic $^{16}$O core nucleus surrounded by four nucleons, the constituents of an $α$-like quartet. Similar to other nuclei ($^{212}$Po, $^{104}$Ti, etc.) with a quartet on top of a double-magic core nucleus, significant $α$-like correlations are expected. Correlations in the ground state of $^{20}$Ne are investigated using different approaches. The quartetting wave function approach (QWFA) predicts a large $α$-like cluster contribution near the surface of the nuclei. The Tohsaki-Horiuchi-Schuck-Röpke (THSR) approach describes $α$-like clustering in nuclear systems. The results of the QWFA in the Thomas-Fermi and shell-model approximation are compared with THSR calculations for the container model. Results for the $α$ formation probability and the rms radii are shown. △ Less

Submitted 11 February, 2024; originally announced February 2024.

Comments: 15 pages, 15 figures. arXiv admin note: text overlap with arXiv:1810.01274

Impact of Domain Knowledge and Multi-Modality on Intelligent Molecular Property Prediction: A Systematic Survey

Authors: Taojie Kuang, Pengfei Liu, Zhixiang Ren

Abstract: The precise prediction of molecular properties is essential for advancements in drug development, particularly in virtual screening and compound optimization. The recent introduction of numerous deep learning-based methods has shown remarkable potential in enhancing molecular property prediction (MPP), especially improving accuracy and insights into molecular structures. Yet, two critical question… ▽ More The precise prediction of molecular properties is essential for advancements in drug development, particularly in virtual screening and compound optimization. The recent introduction of numerous deep learning-based methods has shown remarkable potential in enhancing molecular property prediction (MPP), especially improving accuracy and insights into molecular structures. Yet, two critical questions arise: does the integration of domain knowledge augment the accuracy of molecular property prediction and does employing multi-modal data fusion yield more precise results than unique data source methods? To explore these matters, we comprehensively review and quantitatively analyze recent deep learning methods based on various benchmarks. We discover that integrating molecular information significantly improves molecular property prediction (MPP) for both regression and classification tasks. Specifically, regression improvements, measured by reductions in root mean square error (RMSE), are up to 4.0%, while classification enhancements, measured by the area under the receiver operating characteristic curve (ROC-AUC), are up to 1.7%. We also discover that enriching 2D graphs with 1D SMILES boosts multi-modal learning performance for regression tasks by up to 9.1%, and augmenting 2D graphs with 3D information increases performance for classification tasks by up to 13.2%, with both enhancements measured using ROC-AUC. The two consolidated insights offer crucial guidance for future advancements in drug discovery. △ Less

Submitted 27 June, 2024; v1 submitted 11 February, 2024; originally announced February 2024.

Measurement of the Branching Fraction of $B^{0} \rightarrow J/ψπ^{0}$ Decays

Authors: LHCb collaboration, R. Aaij, A. S. W. Abdelmotteleb, C. Abellan Beteta, F. Abudinén, T. Ackernley, J. A. Adams, A. A. Adefisoye, B. Adeva, M. Adinolfi, P. Adlarson, C. Agapopoulou, C. A. Aidala, Z. Ajaltouni, S. Akar, K. Akiba, P. Albicocco, J. Albrecht, F. Alessio, M. Alexander, Z. Aliouche, P. Alvarez Cartelle, R. Amalric, S. Amato, J. L. Amey , et al. (1067 additional authors not shown)

Abstract: The ratio of branching fractions between $B^{0} \rightarrow J/ψπ^{0}$ and $B^{+} \rightarrow J/ψK^{*+}$ decays is measured with proton-proton collision data collected by the LHCb experiment, corresponding to an integrated luminosity of 9 fb$^{-1}$. The measured value is… ▽ More The ratio of branching fractions between $B^{0} \rightarrow J/ψπ^{0}$ and $B^{+} \rightarrow J/ψK^{*+}$ decays is measured with proton-proton collision data collected by the LHCb experiment, corresponding to an integrated luminosity of 9 fb$^{-1}$. The measured value is $\frac{\mathcal{B}_{B^{0} \rightarrow J/ψπ^{0}}}{\mathcal{B}_{B^{+} \rightarrow J/ψK^{*+}}} = (1.153 \pm 0.053 \pm 0.048 ) \times 10^{-2}$, where the first uncertainty is statistical and the second is systematic. The branching fraction for $B^{0} \rightarrow J/ψπ^{0}$ decays is determined using the branching fraction of the normalisation channel, resulting in $\mathcal{B}_{B^{0} \rightarrow J/ψπ^{0}} = (1.670 \pm 0.077 \pm 0.069 \pm 0.095) \times 10^{-5}$, where the last uncertainty corresponds to that of the external input. This result is consistent with the current world average value and competitive with the most precise single measurement to date. △ Less

Submitted 23 May, 2024; v1 submitted 8 February, 2024; originally announced February 2024.

Comments: All figures and tables, along with any supplementary material and additional information, are available at https://cern.ch/lhcbproject/Publications/p/LHCb-PAPER-2023-041.html (LHCb public pages)

Report number: LHCb-PAPER-2023-041, CERN-EP-2024-009

Journal ref: J. High Energ. Phys. 2024, 65 (2024)

Observation of the $B_c^+ \to J/ψπ^+ π^0$ decay

Authors: LHCb collaboration, R. Aaij, A. S. W. Abdelmotteleb, C. Abellan Beteta, F. Abudinén, T. Ackernley, J. A. Adams, A. A. Adefisoye, B. Adeva, M. Adinolfi, P. Adlarson, C. Agapopoulou, C. A. Aidala, Z. Ajaltouni, S. Akar, K. Akiba, P. Albicocco, J. Albrecht, F. Alessio, M. Alexander, Z. Aliouche, P. Alvarez Cartelle, R. Amalric, S. Amato, J. L. Amey , et al. (1064 additional authors not shown)

Abstract: The first observation of the $B_c^+ \to J/ψπ^+ π^0$ decay is reported with high significance using proton-proton collision data, corresponding to an integrated luminosity of 9fb$^{-1}$, collected with the LHCb detector at centre-of-mass energies of 7, 8, and 13 TeV. The ratio of its branching fraction relative to the $B_c^+ \to J/ψπ^+$ channel is measured to be… ▽ More The first observation of the $B_c^+ \to J/ψπ^+ π^0$ decay is reported with high significance using proton-proton collision data, corresponding to an integrated luminosity of 9fb$^{-1}$, collected with the LHCb detector at centre-of-mass energies of 7, 8, and 13 TeV. The ratio of its branching fraction relative to the $B_c^+ \to J/ψπ^+$ channel is measured to be $$ \frac{ {\cal{B}}( B_c^+ \to J/ψπ^+π^0 ) } { {\cal{B}}( B_c^+ \to J/ψπ^+ ) } = 2.80 \pm 0.15 \pm 0.11 \pm 0.16 \,, $$ where the first uncertainty is statistical, the second systematic and the third related to imprecise knowledge of the branching fractions for $B^+ \to J/ψK^{*+}$ and $B^+ \to J/ψK^+$ decays, which are used to determine the $π^0$ detection efficiency. The $π^+π^0$ mass spectrum is found to be consistent with the dominance of an intermediate $ρ^+$ contribution in accordance with a model based on QCD factorisation. △ Less

Submitted 15 May, 2024; v1 submitted 8 February, 2024; originally announced February 2024.

Comments: 30 pages, 6 figures. All figures and tables, along with any supplementary material and additional information, are available at https://cern.ch/lhcbproject/Publications/p/LHCb-PAPER-2023-046.html (LHCb public pages)

Report number: LHCb-PAPER-2023-046, CERN-EP-2024-019

Journal ref: JHEP04 (2024) 151

Interpretable domain knowledge enhanced machine learning framework on axial capacity prediction of circular CFST columns

Authors: Dian Wang, Zhigang Ren, Gen Kondo

Abstract: This study introduces a novel machine learning framework, integrating domain knowledge, to accurately predict the bearing capacity of CFSTs, bridging the gap between traditional engineering and machine learning techniques. Utilizing a comprehensive database of 2621 experimental data points on CFSTs, we developed a Domain Knowledge Enhanced Neural Network (DKNN) model. This model incorporates advan… ▽ More This study introduces a novel machine learning framework, integrating domain knowledge, to accurately predict the bearing capacity of CFSTs, bridging the gap between traditional engineering and machine learning techniques. Utilizing a comprehensive database of 2621 experimental data points on CFSTs, we developed a Domain Knowledge Enhanced Neural Network (DKNN) model. This model incorporates advanced feature engineering techniques, including Pearson correlation, XGBoost, and Random tree algorithms. The DKNN model demonstrated a marked improvement in prediction accuracy, with a Mean Absolute Percentage Error (MAPE) reduction of over 50% compared to existing models. Its robustness was confirmed through extensive performance assessments, maintaining high accuracy even in noisy environments. Furthermore, sensitivity and SHAP analysis were conducted to assess the contribution of each effective parameter to axial load capacity and propose design recommendations for the diameter of cross-section, material strength range and material combination. This research advances CFST predictive modelling, showcasing the potential of integrating machine learning with domain expertise in structural engineering. The DKNN model sets a new benchmark for accuracy and reliability in the field. △ Less

Submitted 5 March, 2024; v1 submitted 6 February, 2024; originally announced February 2024.

Comments: Journal Research Article

Scientific Language Modeling: A Quantitative Review of Large Language Models in Molecular Science

Authors: Pengfei Liu, Jun Tao, Zhixiang Ren

Abstract: Efficient molecular modeling and design are crucial for the discovery and exploration of novel molecules, and the incorporation of deep learning methods has revolutionized this field. In particular, large language models (LLMs) offer a fresh approach to tackle scientific problems from a natural language processing (NLP) perspective, introducing a research paradigm called scientific language modeli… ▽ More Efficient molecular modeling and design are crucial for the discovery and exploration of novel molecules, and the incorporation of deep learning methods has revolutionized this field. In particular, large language models (LLMs) offer a fresh approach to tackle scientific problems from a natural language processing (NLP) perspective, introducing a research paradigm called scientific language modeling (SLM). However, two key issues remain: how to quantify the match between model and data modalities and how to identify the knowledge-learning preferences of models. To address these challenges, we propose a multi-modal benchmark, named ChEBI-20-MM, and perform 1263 experiments to assess the model's compatibility with data modalities and knowledge acquisition. Through the modal transition probability matrix, we provide insights into the most suitable modalities for tasks. Furthermore, we introduce a statistically interpretable approach to discover context-specific knowledge map** by localized feature filtering. Our pioneering analysis offers an exploration of the learning mechanism and paves the way for advancing SLM in molecular science. △ Less

Submitted 6 February, 2024; originally announced February 2024.

GWAI: Harnessing Artificial Intelligence for Enhancing Gravitational Wave Data Analysis

Authors: Tianyu Zhao, Yue Zhou, Ruijun Shi, Zhoujian Cao, Zhixiang Ren

Abstract: Gravitational wave (GW) astronomy has opened new frontiers in understanding the cosmos, while the integration of artificial intelligence (AI) in science promises to revolutionize data analysis methodologies. However, a significant gap exists, as there is currently no dedicated platform that enables scientists to develop, test, and evaluate AI algorithms efficiently. To address this gap, we introdu… ▽ More Gravitational wave (GW) astronomy has opened new frontiers in understanding the cosmos, while the integration of artificial intelligence (AI) in science promises to revolutionize data analysis methodologies. However, a significant gap exists, as there is currently no dedicated platform that enables scientists to develop, test, and evaluate AI algorithms efficiently. To address this gap, we introduce GWAI, a pioneering AI-centered software platform designed for gravitational wave data analysis. GWAI contains a three-layered architecture that emphasizes simplicity, modularity, and flexibility, covering the entire analysis pipeline. GWAI aims to accelerate scientific discoveries, bridging the gap between advanced AI techniques and astrophysical research. △ Less

Submitted 5 February, 2024; originally announced February 2024.

Comments: 10 pages, 5 figures

Measurements of the branching fraction ratio $\cal{B}(φ\to μ^+μ^-)/\cal{B}(φ\to e^+e^-)$ with charm meson decays

Authors: LHCb collaboration, R. Aaij, A. S. W. Abdelmotteleb, C. Abellan Beteta, F. Abudinén, T. Ackernley, B. Adeva, M. Adinolfi, P. Adlarson, C. Agapopoulou, C. A. Aidala, Z. Ajaltouni, S. Akar, K. Akiba, P. Albicocco, J. Albrecht, F. Alessio, M. Alexander, A. Alfonso Albero, Z. Aliouche, P. Alvarez Cartelle, R. Amalric, S. Amato, J. L. Amey, Y. Amhis , et al. (1080 additional authors not shown)

Abstract: Measurements of the branching fraction ratio ${\cal{B}(φ\to μ^+ μ^-)/\cal{B}(φ\to e^+e^-)}$ with ${D_{s}^{+} \to π^{+} φ}$ and ${D^{+} \to π^{+} φ}$ decays, denoted $R^{s}_{φπ}$ and $R^{d}_{φπ}$, are presented. The analysis is performed using a dataset corresponding to an integrated luminosity of 5.4$\,\rm{fb}^{-1}$ of $pp$ collision data collected with the LHCb experiment. The branching fractions… ▽ More Measurements of the branching fraction ratio ${\cal{B}(φ\to μ^+ μ^-)/\cal{B}(φ\to e^+e^-)}$ with ${D_{s}^{+} \to π^{+} φ}$ and ${D^{+} \to π^{+} φ}$ decays, denoted $R^{s}_{φπ}$ and $R^{d}_{φπ}$, are presented. The analysis is performed using a dataset corresponding to an integrated luminosity of 5.4$\,\rm{fb}^{-1}$ of $pp$ collision data collected with the LHCb experiment. The branching fractions are normalised with respect to the ${B^{+} \to K^{+} J/ψ(\to e^+e^-)}$ and ${B^{+} \to K^{+} J/ψ(\to μ^+μ^-)}$ decay modes. The combination of the results yields $$ R_{φπ} = 1.022 \pm 0.012 \,({\rm stat}) \, \pm 0.048 \,({\rm syst}). $$ The result is compatible with previous measurements of the $φ\to \ell^{+}\ell^{-}$ branching fractions and predictions based on the Standard Model. △ Less

Submitted 1 May, 2024; v1 submitted 2 February, 2024; originally announced February 2024.

Comments: All figures and tables, along with any supplementary material and additional information, are available at https://cern.ch/lhcbproject/Publications/p/LHCb-PAPER-2023-038.html (LHCb public pages)

Report number: LHCb-PAPER-2023-038, CERN-EP-2024-001