Showing 1–50 of 144 results for author: Zeng, K

PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigators

Authors: Kuo-Hao Zeng, Zichen Zhang, Kiana Ehsani, Rose Hendrix, Jordi Salvador, Alvaro Herrasti, Ross Girshick, Aniruddha Kembhavi, Luca Weihs

Abstract: We present PoliFormer (Policy Transformer), an RGB-only indoor navigation agent trained end-to-end with reinforcement learning at scale that generalizes to the real-world without adaptation despite being trained purely in simulation. PoliFormer uses a foundational vision transformer encoder with a causal transformer decoder enabling long-term memory and reasoning. It is trained for hundreds of mil… ▽ More We present PoliFormer (Policy Transformer), an RGB-only indoor navigation agent trained end-to-end with reinforcement learning at scale that generalizes to the real-world without adaptation despite being trained purely in simulation. PoliFormer uses a foundational vision transformer encoder with a causal transformer decoder enabling long-term memory and reasoning. It is trained for hundreds of millions of interactions across diverse environments, leveraging parallelized, multi-machine rollouts for efficient training with high throughput. PoliFormer is a masterful navigator, producing state-of-the-art results across two distinct embodiments, the LoCoBot and Stretch RE-1 robots, and four navigation benchmarks. It breaks through the plateaus of previous work, achieving an unprecedented 85.5% success rate in object goal navigation on the CHORES-S benchmark, a 28.5% absolute improvement. PoliFormer can also be trivially extended to a variety of downstream applications such as object tracking, multi-object navigation, and open-vocabulary navigation with no finetuning. △ Less

Submitted 28 June, 2024; originally announced June 2024.

Take a Step Further: Understanding Page Spray in Linux Kernel Exploitation

Authors: Ziyi Guo, Dang K Le, Zhenpeng Lin, Kyle Zeng, Ruoyu Wang, Tiffany Bao, Yan Shoshitaishvili, Adam Doupé, Xinyu Xing

Abstract: Recently, a novel method known as Page Spray emerges, focusing on page-level exploitation for kernel vulnerabilities. Despite the advantages it offers in terms of exploitability, stability, and compatibility, comprehensive research on Page Spray remains scarce. Questions regarding its root causes, exploitation model, comparative benefits over other exploitation techniques, and possible mitigation… ▽ More Recently, a novel method known as Page Spray emerges, focusing on page-level exploitation for kernel vulnerabilities. Despite the advantages it offers in terms of exploitability, stability, and compatibility, comprehensive research on Page Spray remains scarce. Questions regarding its root causes, exploitation model, comparative benefits over other exploitation techniques, and possible mitigation strategies have largely remained unanswered. In this paper, we conduct a systematic investigation into Page Spray, providing an in-depth understanding of this exploitation technique. We introduce a comprehensive exploit model termed the \sys model, elucidating its fundamental principles. Additionally, we conduct a thorough analysis of the root causes underlying Page Spray occurrences within the Linux Kernel. We design an analyzer based on the Page Spray analysis model to identify Page Spray callsites. Subsequently, we evaluate the stability, exploitability, and compatibility of Page Spray through meticulously designed experiments. Finally, we propose mitigation principles for addressing Page Spray and introduce our own lightweight mitigation approach. This research aims to assist security researchers and developers in gaining insights into Page Spray, ultimately enhancing our collective understanding of this emerging exploitation technique and making improvements to the community. △ Less

Submitted 6 June, 2024; v1 submitted 3 June, 2024; originally announced June 2024.

CLAQ: Pushing the Limits of Low-Bit Post-Training Quantization for LLMs

Authors: Haoyu Wang, Bei Liu, Hang Shao, Bo Xiao, Ke Zeng, Guanglu Wan, Yanmin Qian

Abstract: Parameter quantization for Large Language Models (LLMs) has attracted increasing attentions recently in reducing memory costs and improving computational efficiency. Early approaches have been widely adopted. However, the existing methods suffer from poor performance in low-bit (such as 2 to 3 bits) scenarios. In this paper, we present a novel and effective Column-Level Adaptive weight Quantizatio… ▽ More Parameter quantization for Large Language Models (LLMs) has attracted increasing attentions recently in reducing memory costs and improving computational efficiency. Early approaches have been widely adopted. However, the existing methods suffer from poor performance in low-bit (such as 2 to 3 bits) scenarios. In this paper, we present a novel and effective Column-Level Adaptive weight Quantization (CLAQ) framework by introducing three different types of adaptive strategies for LLM quantization. Firstly, a K-Means clustering based algorithm is proposed that allows dynamic generation of quantization centroids for each column of a parameter matrix. Secondly, we design an outlier-guided adaptive precision search strategy which can dynamically assign varying bit-widths to different columns. Finally, a dynamic outlier reservation scheme is developed to retain some parameters in their original float point precision, in trade off of boosted model performance. Experiments on various mainstream open source LLMs including LLaMA-1, LLaMA-2 and Yi demonstrate that our methods achieve the state-of-the-art results across different bit settings, especially in extremely low-bit scenarios. Code is available at https://github.com/fayuge/CLAQ. △ Less

Submitted 2 June, 2024; v1 submitted 27 May, 2024; originally announced May 2024.

SN 2023zaw: the low-energy explosion of an ultra-stripped star, with non-radioactive heating

Authors: Thomas Moore, James Gillanders, Matt Nicholl, Mark Huber, Stephen Smartt, Shubham Srivastav, Heloise Stevance, Ting-Wan Chen, Kenneth Chambers, Joseph Anderson, Michael Fulton, Samantha Oates, Charlotte Angus, Giuliano Pignata, Nicolas Erasmus, Hua Gao, Joanna Bulger, Chien-Cheng Lin, Thomas Lowe, Eugene Magnier, Paloma Minguez, Chow-Choong Ngeow, Xinyue Sheng, Stuart A. Sim, Ken Smith , et al. (4 additional authors not shown)

Abstract: Most stripped envelope supernova progenitors are formed through binary interaction, losing hydrogen and/or helium from their outer layers. An emerging class of supernovae with the highest degree of envelope-strip** are thought to be the product of strip** by a NS companion. However, relatively few examples are known and the outcomes of such systems can be diverse and are poorly understood at p… ▽ More Most stripped envelope supernova progenitors are formed through binary interaction, losing hydrogen and/or helium from their outer layers. An emerging class of supernovae with the highest degree of envelope-strip** are thought to be the product of strip** by a NS companion. However, relatively few examples are known and the outcomes of such systems can be diverse and are poorly understood at present. Here, we present spectroscopic observations and high cadence multi-band photometry of SN 2023zaw, a low ejecta mass and rapidly evolving supernova. SN 2023zaw was discovered in a nearby spiral galaxy at D = 39.7 Mpc, with significant Milky Way extinction, $E(B-V) = 0.21$, and significant (but uncertain) host extinction. Bayesian evidence comparison reveals that nickel is not the only power source and an additional energy source is required to explain our observations. Our models suggest an ejecta mass of $M_{\rm ej} \sim 0.07\,\rm M_\odot$ and a synthesised nickel mass of $M_{\rm ej} \sim 0.007\,\rm M_\odot$ is required to explain the explosion. However an additional heating from a magnetar or interaction with circumstellar material is required to power the early light curve. △ Less

Submitted 22 May, 2024; originally announced May 2024.

Swipe2Pair: Secure and Fast In-Band Wireless Device Pairing

Authors: Yaqi He, Kai Zeng, Long Jiao, Brian L. Mark, Khaled N. Khasawneh

Abstract: Wireless device pairing is a critical security mechanism to bootstrap the secure communication between two devices without a pre-shared secret. It has been widely used in many Internet of Things (IoT) applications, such as smart-home and smart-health. Most existing device pairing mechanisms are based on out-of-band channels, e.g., extra sensors or hardware, to validate the proximity of pairing dev… ▽ More Wireless device pairing is a critical security mechanism to bootstrap the secure communication between two devices without a pre-shared secret. It has been widely used in many Internet of Things (IoT) applications, such as smart-home and smart-health. Most existing device pairing mechanisms are based on out-of-band channels, e.g., extra sensors or hardware, to validate the proximity of pairing devices. However, out-of-band channels are not universal across all wireless devices, so such a scheme is limited to certain application scenarios or conditions. On the other hand, in-band channel-based device pairing seeks universal applicability by only relying on wireless interfaces. Existing in-band channel-based pairing schemes either require multiple antennas separated by a good distance on one pairing device, which is not feasible in certain scenarios, or require users to repeat multiple sweeps, which is not optimal in terms of usability. Therefore, an in-band wireless device pairing scheme providing high security while maintaining high usability (simple pairing process and minimal user intervention) is highly desired. In this work, we propose an easy-to-use mutual authentication device pairing scheme, named Swipe2Pair, based on the proximity of pairing devices and randomization of wireless transmission power. We conduct extensive security analysis and collect considerable experimental data under various settings across different environments. Experimental results show that Swipe2Pair achieves high security and usability. It only takes less than one second to complete the pairing process with a simple swipe of one device in front of the other. △ Less

Submitted 5 May, 2024; originally announced May 2024.

MambaMOS: LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space Model

Authors: Kang Zeng, Hao Shi, Jiacheng Lin, Siyu Li, **tao Cheng, Kaiwei Wang, Zhiyong Li, Kailun Yang

Abstract: LiDAR-based Moving Object Segmentation (MOS) aims to locate and segment moving objects in point clouds of the current scan using motion information from previous scans. Despite the promising results achieved by previous MOS methods, several key issues, such as the weak coupling of temporal and spatial information, still need further study. In this paper, we propose a novel LiDAR-based 3D Moving Ob… ▽ More LiDAR-based Moving Object Segmentation (MOS) aims to locate and segment moving objects in point clouds of the current scan using motion information from previous scans. Despite the promising results achieved by previous MOS methods, several key issues, such as the weak coupling of temporal and spatial information, still need further study. In this paper, we propose a novel LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space Model, termed MambaMOS. Firstly, we develop a novel embedding module, the Time Clue Bootstrap** Embedding (TCBE), to enhance the coupling of temporal and spatial information in point clouds and alleviate the issue of overlooked temporal clues. Secondly, we introduce the Motion-aware State Space Model (MSSM) to endow the model with the capacity to understand the temporal correlations of the same object across different time steps. Specifically, MSSM emphasizes the motion states of the same object at different time steps through two distinct temporal modeling and correlation steps. We utilize an improved state space model to represent these motion differences, significantly modeling the motion states. Finally, extensive experiments on the SemanticKITTI-MOS and KITTI-Road benchmarks demonstrate that the proposed MambaMOS achieves state-of-the-art performance. The source code of this work will be made publicly available at https://github.com/Terminal-K/MambaMOS. △ Less

Submitted 19 April, 2024; originally announced April 2024.

Comments: The source code will be made publicly available at https://github.com/Terminal-K/MambaMOS

CMNEE: A Large-Scale Document-Level Event Extraction Dataset based on Open-Source Chinese Military News

Authors: Mengna Zhu, Zijie Xu, Kaisheng Zeng, Kaiming Xiao, Mao Wang, Wenjun Ke, Hongbin Huang

Abstract: Extracting structured event knowledge, including event triggers and corresponding arguments, from military texts is fundamental to many applications, such as intelligence analysis and decision assistance. However, event extraction in the military field faces the data scarcity problem, which impedes the research of event extraction models in this domain. To alleviate this problem, we propose CMNEE,… ▽ More Extracting structured event knowledge, including event triggers and corresponding arguments, from military texts is fundamental to many applications, such as intelligence analysis and decision assistance. However, event extraction in the military field faces the data scarcity problem, which impedes the research of event extraction models in this domain. To alleviate this problem, we propose CMNEE, a large-scale, document-level open-source Chinese Military News Event Extraction dataset. It contains 17,000 documents and 29,223 events, which are all manually annotated based on a pre-defined schema for the military domain including 8 event types and 11 argument role types. We designed a two-stage, multi-turns annotation strategy to ensure the quality of CMNEE and reproduced several state-of-the-art event extraction models with a systematic evaluation. The experimental results on CMNEE fall shorter than those on other domain datasets obviously, which demonstrates that event extraction for military domain poses unique challenges and requires further research efforts. Our code and data can be obtained from https://github.com/Mzzzhu/CMNEE. △ Less

Submitted 18 April, 2024; originally announced April 2024.

Comments: 13 pages, 7 figures, accepted to LREC-COLING 2024

Gaussian Shading: Provable Performance-Lossless Image Watermarking for Diffusion Models

Authors: Zi** Yang, Kai Zeng, Kejiang Chen, Han Fang, Weiming Zhang, Nenghai Yu

Abstract: Ethical concerns surrounding copyright protection and inappropriate content generation pose challenges for the practical implementation of diffusion models. One effective solution involves watermarking the generated images. However, existing methods often compromise the model performance or require additional training, which is undesirable for operators and users. To address this issue, we propose… ▽ More Ethical concerns surrounding copyright protection and inappropriate content generation pose challenges for the practical implementation of diffusion models. One effective solution involves watermarking the generated images. However, existing methods often compromise the model performance or require additional training, which is undesirable for operators and users. To address this issue, we propose Gaussian Shading, a diffusion model watermarking technique that is both performance-lossless and training-free, while serving the dual purpose of copyright protection and tracing of offending content. Our watermark embedding is free of model parameter modifications and thus is plug-and-play. We map the watermark to latent representations following a standard Gaussian distribution, which is indistinguishable from latent representations obtained from the non-watermarked diffusion model. Therefore we can achieve watermark embedding with lossless performance, for which we also provide theoretical proof. Furthermore, since the watermark is intricately linked with image semantics, it exhibits resilience to lossy processing and erasure attempts. The watermark can be extracted by Denoising Diffusion Implicit Models (DDIM) inversion and inverse sampling. We evaluate Gaussian Shading on multiple versions of Stable Diffusion, and the results demonstrate that Gaussian Shading not only is performance-lossless but also outperforms existing methods in terms of robustness. △ Less

Submitted 6 May, 2024; v1 submitted 7 April, 2024; originally announced April 2024.

Comments: 17 pages, 11 figures, accepted by CVPR 2024

UAlign: Pushing the Limit of Template-free Retrosynthesis Prediction with Unsupervised SMILES Alignment

Authors: Kaipeng Zeng, Bo yang, Xin Zhao, Yu Zhang, Fan Nie, Xiaokang Yang, Yaohui **, Yanyan Xu

Abstract: Motivation: Retrosynthesis planning poses a formidable challenge in the organic chemical industry. Single-step retrosynthesis prediction, a crucial step in the planning process, has witnessed a surge in interest in recent years due to advancements in AI for science. Various deep learning-based methods have been proposed for this task in recent years, incorporating diverse levels of additional chem… ▽ More Motivation: Retrosynthesis planning poses a formidable challenge in the organic chemical industry. Single-step retrosynthesis prediction, a crucial step in the planning process, has witnessed a surge in interest in recent years due to advancements in AI for science. Various deep learning-based methods have been proposed for this task in recent years, incorporating diverse levels of additional chemical knowledge dependency. Results: This paper introduces UAlign, a template-free graph-to-sequence pipeline for retrosynthesis prediction. By combining graph neural networks and Transformers, our method can more effectively leverage the inherent graph structure of molecules. Based on the fact that the majority of molecule structures remain unchanged during a chemical reaction, we propose a simple yet effective SMILES alignment technique to facilitate the reuse of unchanged structures for reactant generation. Extensive experiments show that our method substantially outperforms state-of-the-art template-free and semi-template-based approaches. Importantly, our template-free method achieves effectiveness comparable to, or even surpasses, established powerful template-based methods. Scientific contribution: We present a novel graph-to-sequence template-free retrosynthesis prediction pipeline that overcomes the limitations of Transformer-based methods in molecular representation learning and insufficient utilization of chemical information. We propose an unsupervised learning mechanism for establishing product-atom correspondence with reactant SMILES tokens, achieving even better results than supervised SMILES alignment methods. Extensive experiments demonstrate that UAlign significantly outperforms state-of-the-art template-free methods and rivals or surpasses template-based approaches, with up to 5\% (top-5) and 5.4\% (top-10) increased accuracy over the strongest baseline. △ Less

Submitted 19 April, 2024; v1 submitted 24 March, 2024; originally announced April 2024.

Distributed Swarm Learning for Edge Internet of Things

Authors: Yue Wang, Zhi Tian, FXin Fan, Zhipeng Cai, Cameron Nowzari, Kai Zeng

Abstract: The rapid growth of Internet of Things (IoT) has led to the widespread deployment of smart IoT devices at wireless edge for collaborative machine learning tasks, ushering in a new era of edge learning. With a huge number of hardware-constrained IoT devices operating in resource-limited wireless networks, edge learning encounters substantial challenges, including communication and computation bottl… ▽ More The rapid growth of Internet of Things (IoT) has led to the widespread deployment of smart IoT devices at wireless edge for collaborative machine learning tasks, ushering in a new era of edge learning. With a huge number of hardware-constrained IoT devices operating in resource-limited wireless networks, edge learning encounters substantial challenges, including communication and computation bottlenecks, device and data heterogeneity, security risks, privacy leakages, non-convex optimization, and complex wireless environments. To address these issues, this article explores a novel framework known as distributed swarm learning (DSL), which combines artificial intelligence and biological swarm intelligence in a holistic manner. By harnessing advanced signal processing and communications, DSL provides efficient solutions and robust tools for large-scale IoT at the edge of wireless networks. △ Less

Submitted 29 March, 2024; originally announced March 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2210.16705

Provably Secure Disambiguating Neural Linguistic Steganography

Authors: Yuang Qi, Kejiang Chen, Kai Zeng, Weiming Zhang, Nenghai Yu

Abstract: Recent research in provably secure neural linguistic steganography has overlooked a crucial aspect: the sender must detokenize stegotexts to avoid raising suspicion from the eavesdropper. The segmentation ambiguity problem, which arises when using language models based on subwords, leads to occasional decoding failures in all neural language steganography implementations based on these models. Cur… ▽ More Recent research in provably secure neural linguistic steganography has overlooked a crucial aspect: the sender must detokenize stegotexts to avoid raising suspicion from the eavesdropper. The segmentation ambiguity problem, which arises when using language models based on subwords, leads to occasional decoding failures in all neural language steganography implementations based on these models. Current solutions to this issue involve altering the probability distribution of candidate words, rendering them incompatible with provably secure steganography. We propose a novel secure disambiguation method named SyncPool, which effectively addresses the segmentation ambiguity problem. We group all tokens with prefix relationships in the candidate pool before the steganographic embedding algorithm runs to eliminate uncertainty among ambiguous tokens. To enable the receiver to synchronize the sampling process of the sender, a shared cryptographically-secure pseudorandom number generator (CSPRNG) is deployed to select a token from the ambiguity pool. SyncPool does not change the size of the candidate pool or the distribution of tokens and thus is applicable to provably secure language steganography methods. We provide theoretical proofs and experimentally demonstrate the applicability of our solution to various languages and models, showing its potential to significantly improve the reliability and security of neural linguistic steganography systems. △ Less

Submitted 26 March, 2024; originally announced March 2024.

Site Symmetry and Multiorbital Flat Bands on Kagome and Pyrochlore Lattices

Authors: Keyu Zeng, Ziqiang Wang

Abstract: Flat bands in electronic band structures are intriguing platforms for strong correlation and topological physics, primarily due to the suppressed kinetic energy of electrons. Various methods have been developed to create flat bands, utilizing lattice geometry or finely tuned parameters. Despite this, the investigation of orbital symmetry in multiorbital materials is a relatively new area of focus.… ▽ More Flat bands in electronic band structures are intriguing platforms for strong correlation and topological physics, primarily due to the suppressed kinetic energy of electrons. Various methods have been developed to create flat bands, utilizing lattice geometry or finely tuned parameters. Despite this, the investigation of orbital symmetry in multiorbital materials is a relatively new area of focus. In this work, we propose a site symmetry based systematic approach to emerging multiorbital flat bands in lattices made of corner-connecting motifs such as the kagome and pyrochlore lattices. As a conceptual advance, the one-orbital flat bands are shown to originate as mutual eigenstates of isolated molecular motifs. Further develo** the mutual eigenstate method for multiorbitals transforming differently under the site symmetries such as mirror and inversion, we derive multiorbital flat bands from the skew-symmetric interorbital Hamiltonian and introduce an isolated molecule enabled group-theoretic description of the flat band wavefunctions. Realizations of the multiorbital flatbands in relevant materials are shown to be possible under the Slater-Koster formalism. Our findings provide new directions for exploring flatband electronic structures for novel correlated and topological quantum states. △ Less

Submitted 14 March, 2024; v1 submitted 5 March, 2024; originally announced March 2024.

Learning or Self-aligning? Rethinking Instruction Fine-tuning

Authors: Mengjie Ren, Boxi Cao, Hongyu Lin, Cao Liu, Xianpei Han, Ke Zeng, Guanglu Wan, Xunliang Cai, Le Sun

Abstract: Instruction Fine-tuning~(IFT) is a critical phase in building large language models~(LLMs). Previous works mainly focus on the IFT's role in the transfer of behavioral norms and the learning of additional world knowledge. However, the understanding of the underlying mechanisms of IFT remains significantly limited. In this paper, we design a knowledge intervention framework to decouple the potentia… ▽ More Instruction Fine-tuning~(IFT) is a critical phase in building large language models~(LLMs). Previous works mainly focus on the IFT's role in the transfer of behavioral norms and the learning of additional world knowledge. However, the understanding of the underlying mechanisms of IFT remains significantly limited. In this paper, we design a knowledge intervention framework to decouple the potential underlying factors of IFT, thereby enabling individual analysis of different factors. Surprisingly, our experiments reveal that attempting to learn additional world knowledge through IFT often struggles to yield positive impacts and can even lead to markedly negative effects. Further, we discover that maintaining internal knowledge consistency before and after IFT is a critical factor for achieving successful IFT. Our findings reveal the underlying mechanisms of IFT and provide robust support for some very recent and potential future works. △ Less

Submitted 2 March, 2024; v1 submitted 28 February, 2024; originally announced February 2024.

Multi-Bit Distortion-Free Watermarking for Large Language Models

Authors: Massieh Kordi Boroujeny, Ya Jiang, Kai Zeng, Brian Mark

Abstract: Methods for watermarking large language models have been proposed that distinguish AI-generated text from human-generated text by slightly altering the model output distribution, but they also distort the quality of the text, exposing the watermark to adversarial detection. More recently, distortion-free watermarking methods were proposed that require a secret key to detect the watermark. The prio… ▽ More Methods for watermarking large language models have been proposed that distinguish AI-generated text from human-generated text by slightly altering the model output distribution, but they also distort the quality of the text, exposing the watermark to adversarial detection. More recently, distortion-free watermarking methods were proposed that require a secret key to detect the watermark. The prior methods generally embed zero-bit watermarks that do not provide additional information beyond tagging a text as being AI-generated. We extend an existing zero-bit distortion-free watermarking method by embedding multiple bits of meta-information as part of the watermark. We also develop a computationally efficient decoder that extracts the embedded information from the watermark with low bit error rate. △ Less

Submitted 26 February, 2024; originally announced February 2024.

Event-level Knowledge Editing

Authors: Hao Peng, Xiaozhi Wang, Chunyang Li, Kaisheng Zeng, Jiangshan Duo, Yixin Cao, Lei Hou, Juanzi Li

Abstract: Knowledge editing aims at updating knowledge of large language models (LLMs) to prevent them from becoming outdated. Existing work edits LLMs at the level of factual knowledge triplets. However, natural knowledge updates in the real world come from the occurrences of new events rather than direct changes in factual triplets. In this paper, we propose a new task setting: event-level knowledge editi… ▽ More Knowledge editing aims at updating knowledge of large language models (LLMs) to prevent them from becoming outdated. Existing work edits LLMs at the level of factual knowledge triplets. However, natural knowledge updates in the real world come from the occurrences of new events rather than direct changes in factual triplets. In this paper, we propose a new task setting: event-level knowledge editing, which directly edits new events into LLMs and improves over conventional triplet-level editing on (1) Efficiency. A single event edit leads to updates in multiple entailed knowledge triplets. (2) Completeness. Beyond updating factual knowledge, event-level editing also requires considering the event influences and updating LLMs' knowledge about future trends. We construct a high-quality event-level editing benchmark ELKEN, consisting of 1,515 event edits, 6,449 questions about factual knowledge, and 10,150 questions about future tendencies. We systematically evaluate the performance of various knowledge editing methods and LLMs on this benchmark. We find that ELKEN poses significant challenges to existing knowledge editing approaches. Our codes and dataset are publicly released to facilitate further research. △ Less

Submitted 21 April, 2024; v1 submitted 20 February, 2024; originally announced February 2024.

Comments: 18 pages, 2 figures

MF-MOS: A Motion-Focused Model for Moving Object Segmentation

Authors: **tao Cheng, Kang Zeng, Zhuoxu Huang, Xiaoyu Tang, ** Wu, Chengxi Zhang, Xieyuanli Chen, Rui Fan

Abstract: Moving object segmentation (MOS) provides a reliable solution for detecting traffic participants and thus is of great interest in the autonomous driving field. Dynamic capture is always critical in the MOS problem. Previous methods capture motion features from the range images directly. Differently, we argue that the residual maps provide greater potential for motion information, while range image… ▽ More Moving object segmentation (MOS) provides a reliable solution for detecting traffic participants and thus is of great interest in the autonomous driving field. Dynamic capture is always critical in the MOS problem. Previous methods capture motion features from the range images directly. Differently, we argue that the residual maps provide greater potential for motion information, while range images contain rich semantic guidance. Based on this intuition, we propose MF-MOS, a novel motion-focused model with a dual-branch structure for LiDAR moving object segmentation. Novelly, we decouple the spatial-temporal information by capturing the motion from residual maps and generating semantic features from range images, which are used as movable object guidance for the motion branch. Our straightforward yet distinctive solution can make the most use of both range images and residual maps, thus greatly improving the performance of the LiDAR-based MOS task. Remarkably, our MF-MOS achieved a leading IoU of 76.7% on the MOS leaderboard of the SemanticKITTI dataset upon submission, demonstrating the current state-of-the-art performance. The implementation of our MF-MOS has been released at https://github.com/SCNU-RISLAB/MF-MOS. △ Less

Submitted 30 January, 2024; originally announced January 2024.

Comments: Accepted by ICRA2024

MorphGrower: A Synchronized Layer-by-layer Growing Approach for Plausible Neuronal Morphology Generation

Authors: Nianzu Yang, Kaipeng Zeng, Haotian Lu, Yexin Wu, Zexin Yuan, Danni Chen, Shengdian Jiang, Jiaxiang Wu, Yimin Wang, Junchi Yan

Abstract: Neuronal morphology is essential for studying brain functioning and understanding neurodegenerative disorders. As acquiring real-world morphology data is expensive, computational approaches for morphology generation have been studied. Traditional methods heavily rely on expert-set rules and parameter tuning, making it difficult to generalize across different types of morphologies. Recently, MorphV… ▽ More Neuronal morphology is essential for studying brain functioning and understanding neurodegenerative disorders. As acquiring real-world morphology data is expensive, computational approaches for morphology generation have been studied. Traditional methods heavily rely on expert-set rules and parameter tuning, making it difficult to generalize across different types of morphologies. Recently, MorphVAE was introduced as the sole learning-based method, but its generated morphologies lack plausibility, i.e., they do not appear realistic enough and most of the generated samples are topologically invalid. To fill this gap, this paper proposes MorphGrower, which mimicks the neuron natural growth mechanism for generation. Specifically, MorphGrower generates morphologies layer by layer, with each subsequent layer conditioned on the previously generated structure. During each layer generation, MorphGrower utilizes a pair of sibling branches as the basic generation block and generates branch pairs synchronously. This approach ensures topological validity and allows for fine-grained generation, thereby enhancing the realism of the final generated morphologies. Results on four real-world datasets demonstrate that MorphGrower outperforms MorphVAE by a notable margin. Importantly, the electrophysiological response simulation demonstrates the plausibility of our generated samples from a neuroscience perspective. Our code is available at https://github.com/Thinklab-SJTU/MorphGrower. △ Less

Submitted 27 May, 2024; v1 submitted 17 January, 2024; originally announced January 2024.

Seeing the Unseen: Visual Common Sense for Semantic Placement

Authors: Ram Ramrakhya, Aniruddha Kembhavi, Dhruv Batra, Zsolt Kira, Kuo-Hao Zeng, Luca Weihs

Abstract: Computer vision tasks typically involve describing what is present in an image (e.g. classification, detection, segmentation, and captioning). We study a visual common sense task that requires understanding what is not present. Specifically, given an image (e.g. of a living room) and name of an object ("cushion"), a vision system is asked to predict semantically-meaningful regions (masks or boundi… ▽ More Computer vision tasks typically involve describing what is present in an image (e.g. classification, detection, segmentation, and captioning). We study a visual common sense task that requires understanding what is not present. Specifically, given an image (e.g. of a living room) and name of an object ("cushion"), a vision system is asked to predict semantically-meaningful regions (masks or bounding boxes) in the image where that object could be placed or is likely be placed by humans (e.g. on the sofa). We call this task: Semantic Placement (SP) and believe that such common-sense visual understanding is critical for assitive robots (tidying a house), and AR devices (automatically rendering an object in the user's space). Studying the invisible is hard. Datasets for image description are typically constructed by curating relevant images and asking humans to annotate the contents of the image; neither of those two steps are straightforward for objects not present in the image. We overcome this challenge by operating in the opposite direction: we start with an image of an object in context from web, and then remove that object from the image via inpainting. This automated pipeline converts unstructured web data into a dataset comprising pairs of images with/without the object. Using this, we collect a novel dataset, with ${\sim}1.3$M images across $9$ object categories, and train a SP prediction model called CLIP-UNet. CLIP-UNet outperforms existing VLMs and baselines that combine semantic priors with object detectors on real-world and simulated images. In our user studies, we find that the SP masks predicted by CLIP-UNet are favored $43.7\%$ and $31.3\%$ times when comparing against the $4$ SP baselines on real and simulated images. In addition, we demonstrate leveraging SP mask predictions from CLIP-UNet enables downstream applications like building tidying robots in indoor environments. △ Less

Submitted 15 January, 2024; originally announced January 2024.

Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real World

Authors: Kiana Ehsani, Tanmay Gupta, Rose Hendrix, Jordi Salvador, Luca Weihs, Kuo-Hao Zeng, Kunal Pratap Singh, Ye** Kim, Winson Han, Alvaro Herrasti, Ranjay Krishna, Dustin Schwenk, Eli VanderBilt, Aniruddha Kembhavi

Abstract: Reinforcement learning (RL) with dense rewards and imitation learning (IL) with human-generated trajectories are the most widely used approaches for training modern embodied agents. RL requires extensive reward sha** and auxiliary losses and is often too slow and ineffective for long-horizon tasks. While IL with human supervision is effective, collecting human trajectories at scale is extremely… ▽ More Reinforcement learning (RL) with dense rewards and imitation learning (IL) with human-generated trajectories are the most widely used approaches for training modern embodied agents. RL requires extensive reward sha** and auxiliary losses and is often too slow and ineffective for long-horizon tasks. While IL with human supervision is effective, collecting human trajectories at scale is extremely expensive. In this work, we show that imitating shortest-path planners in simulation produces agents that, given a language instruction, can proficiently navigate, explore, and manipulate objects in both simulation and in the real world using only RGB sensors (no depth map or GPS coordinates). This surprising result is enabled by our end-to-end, transformer-based, SPOC architecture, powerful visual encoders paired with extensive image augmentation, and the dramatic scale and diversity of our training data: millions of frames of shortest-path-expert trajectories collected inside approximately 200,000 procedurally generated houses containing 40,000 unique 3D assets. Our models, data, training code, and newly proposed 10-task benchmarking suite CHORES will be open-sourced. △ Less

Submitted 5 December, 2023; originally announced December 2023.

Comments: First six authors contributed equally. Project page: https://spoc-robot.github.io/

MAVEN-Arg: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation

Authors: Xiaozhi Wang, Hao Peng, Yong Guan, Kaisheng Zeng, Jianhui Chen, Lei Hou, Xu Han, Yankai Lin, Zhiyuan Liu, Ruobing Xie, Jie Zhou, Juanzi Li

Abstract: Understanding events in texts is a core objective of natural language understanding, which requires detecting event occurrences, extracting event arguments, and analyzing inter-event relationships. However, due to the annotation challenges brought by task complexity, a large-scale dataset covering the full process of event understanding has long been absent. In this paper, we introduce MAVEN-Arg,… ▽ More Understanding events in texts is a core objective of natural language understanding, which requires detecting event occurrences, extracting event arguments, and analyzing inter-event relationships. However, due to the annotation challenges brought by task complexity, a large-scale dataset covering the full process of event understanding has long been absent. In this paper, we introduce MAVEN-Arg, which augments MAVEN datasets with event argument annotations, making the first all-in-one dataset supporting event detection, event argument extraction (EAE), and event relation extraction. As an EAE benchmark, MAVEN-Arg offers three main advantages: (1) a comprehensive schema covering 162 event types and 612 argument roles, all with expert-written definitions and examples; (2) a large data scale, containing 98,591 events and 290,613 arguments obtained with laborious human annotation; (3) the exhaustive annotation supporting all task variants of EAE, which annotates both entity and non-entity event arguments in document level. Experiments indicate that MAVEN-Arg is quite challenging for both fine-tuned EAE models and proprietary large language models (LLMs). Furthermore, to demonstrate the benefits of an all-in-one dataset, we preliminarily explore a potential application, future event prediction, with LLMs. MAVEN-Arg and codes can be obtained from https://github.com/THU-KEG/MAVEN-Argument. △ Less

Submitted 18 June, 2024; v1 submitted 15 November, 2023; originally announced November 2023.

Comments: Accepted at ACL 2024. Camera-ready version

When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks

Authors: Hao Peng, Xiaozhi Wang, Jianhui Chen, Weikai Li, Yunjia Qi, Zimu Wang, Zhili Wu, Kaisheng Zeng, Bin Xu, Lei Hou, Juanzi Li

Abstract: In-context learning (ICL) has become the default method for using large language models (LLMs), making the exploration of its limitations and understanding the underlying causes crucial. In this paper, we find that ICL falls short of handling specification-heavy tasks, which are tasks with complicated and extensive task specifications, requiring several hours for ordinary humans to master, such as… ▽ More In-context learning (ICL) has become the default method for using large language models (LLMs), making the exploration of its limitations and understanding the underlying causes crucial. In this paper, we find that ICL falls short of handling specification-heavy tasks, which are tasks with complicated and extensive task specifications, requiring several hours for ordinary humans to master, such as traditional information extraction tasks. The performance of ICL on these tasks mostly cannot reach half of the state-of-the-art results. To explore the reasons behind this failure, we conduct comprehensive experiments on 18 specification-heavy tasks with various LLMs and identify three primary reasons: inability to specifically understand context, misalignment in task schema comprehension with humans, and inadequate long-text understanding ability. Furthermore, we demonstrate that through fine-tuning, LLMs can achieve decent performance on these tasks, indicating that the failure of ICL is not an inherent flaw of LLMs, but rather a drawback of existing alignment methods that renders LLMs incapable of handling complicated specification-heavy tasks via ICL. To substantiate this, we perform dedicated instruction tuning on LLMs for these tasks and observe a notable improvement. We hope the analyses in this paper could facilitate advancements in alignment methods enabling LLMs to meet more sophisticated human demands. △ Less

Submitted 15 November, 2023; originally announced November 2023.

Comments: Under review

FireMatch: A Semi-Supervised Video Fire Detection Network Based on Consistency and Distribution Alignment

Authors: Qinghua Lin, Zuoyong Li, Kun Zeng, Haoyi Fan, Wei Li, Xiaoguang Zhou

Abstract: Deep learning techniques have greatly enhanced the performance of fire detection in videos. However, video-based fire detection models heavily rely on labeled data, and the process of data labeling is particularly costly and time-consuming, especially when dealing with videos. Considering the limited quantity of labeled video data, we propose a semi-supervised fire detection model called FireMatch… ▽ More Deep learning techniques have greatly enhanced the performance of fire detection in videos. However, video-based fire detection models heavily rely on labeled data, and the process of data labeling is particularly costly and time-consuming, especially when dealing with videos. Considering the limited quantity of labeled video data, we propose a semi-supervised fire detection model called FireMatch, which is based on consistency regularization and adversarial distribution alignment. Specifically, we first combine consistency regularization with pseudo-label. For unlabeled data, we design video data augmentation to obtain corresponding weakly augmented and strongly augmented samples. The proposed model predicts weakly augmented samples and retains pseudo-label above a threshold, while training on strongly augmented samples to predict these pseudo-labels for learning more robust feature representations. Secondly, we generate video cross-set augmented samples by adversarial distribution alignment to expand the training data and alleviate the decline in classification performance caused by insufficient labeled data. Finally, we introduce a fairness loss to help the model produce diverse predictions for input samples, thereby addressing the issue of high confidence with the non-fire class in fire classification scenarios. The FireMatch achieved an accuracy of 76.92% and 91.81% on two real-world fire datasets, respectively. The experimental results demonstrate that the proposed method outperforms the current state-of-the-art semi-supervised classification methods. △ Less

Submitted 9 November, 2023; originally announced November 2023.

Selective Visual Representations Improve Convergence and Generalization for Embodied AI

Authors: Ainaz Eftekhar, Kuo-Hao Zeng, Jiafei Duan, Ali Farhadi, Ani Kembhavi, Ranjay Krishna

Abstract: Embodied AI models often employ off the shelf vision backbones like CLIP to encode their visual observations. Although such general purpose representations encode rich syntactic and semantic information about the scene, much of this information is often irrelevant to the specific task at hand. This introduces noise within the learning process and distracts the agent's focus from task-relevant visu… ▽ More Embodied AI models often employ off the shelf vision backbones like CLIP to encode their visual observations. Although such general purpose representations encode rich syntactic and semantic information about the scene, much of this information is often irrelevant to the specific task at hand. This introduces noise within the learning process and distracts the agent's focus from task-relevant visual cues. Inspired by selective attention in humans-the process through which people filter their perception based on their experiences, knowledge, and the task at hand-we introduce a parameter-efficient approach to filter visual stimuli for embodied AI. Our approach induces a task-conditioned bottleneck using a small learnable codebook module. This codebook is trained jointly to optimize task reward and acts as a task-conditioned selective filter over the visual observation. Our experiments showcase state-of-the-art performance for object goal navigation and object displacement across 5 benchmarks, ProcTHOR, ArchitecTHOR, RoboTHOR, AI2-iTHOR, and ManipulaTHOR. The filtered representations produced by the codebook are also able generalize better and converge faster when adapted to other simulation environments such as Habitat. Our qualitative analyses show that agents explore their environments more effectively and their representations retain task-relevant information like target object recognition while ignoring superfluous information about other objects. Code and pretrained models are available at our project website: https://embodied-codebook.github.io. △ Less

Submitted 9 March, 2024; v1 submitted 7 November, 2023; originally announced November 2023.

Comments: See project website: https://embodied-codebook.github.io

Mastering the Task of Open Information Extraction with Large Language Models and Consistent Reasoning Environment

Authors: Ji Qi, Kaixuan Ji, Xiaozhi Wang, Jifan Yu, Kaisheng Zeng, Lei Hou, Juanzi Li, Bin Xu

Abstract: Open Information Extraction (OIE) aims to extract objective structured knowledge from natural texts, which has attracted growing attention to build dedicated models with human experience. As the large language models (LLMs) have exhibited remarkable in-context learning capabilities, a question arises as to whether the task of OIE can be effectively tackled with this paradigm? In this paper, we exp… ▽ More Open Information Extraction (OIE) aims to extract objective structured knowledge from natural texts, which has attracted growing attention to build dedicated models with human experience. As the large language models (LLMs) have exhibited remarkable in-context learning capabilities, a question arises as to whether the task of OIE can be effectively tackled with this paradigm? In this paper, we explore solving the OIE problem by constructing an appropriate reasoning environment for LLMs. Specifically, we first propose a method to effectively estimate the discrepancy of syntactic distribution between a LLM and test samples, which can serve as correlation evidence for preparing positive demonstrations. Upon the evidence, we introduce a simple yet effective mechanism to establish the reasoning environment for LLMs on specific tasks. Without bells and whistles, experimental results on the standard CaRB benchmark demonstrate that our $6$-shot approach outperforms state-of-the-art supervised method, achieving an $55.3$ $F_1$ score. Further experiments on TACRED and ACE05 show that our method can naturally generalize to other information extraction tasks, resulting in improvements of $5.7$ and $6.8$ $F_1$ scores, respectively. △ Less

Submitted 16 October, 2023; originally announced October 2023.

One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models

Authors: Hang Shao, Bei Liu, Bo Xiao, Ke Zeng, Guanglu Wan, Yanmin Qian

Abstract: Various Large Language Models~(LLMs) from the Generative Pretrained Transformer(GPT) family have achieved outstanding performances in a wide range of text generation tasks. However, the enormous model sizes have hindered their practical use in real-world applications due to high inference latency. Therefore, improving the efficiencies of LLMs through quantization, pruning, and other means has been… ▽ More Various Large Language Models~(LLMs) from the Generative Pretrained Transformer(GPT) family have achieved outstanding performances in a wide range of text generation tasks. However, the enormous model sizes have hindered their practical use in real-world applications due to high inference latency. Therefore, improving the efficiencies of LLMs through quantization, pruning, and other means has been a key issue in LLM studies. In this work, we propose a method based on Hessian sensitivity-aware mixed sparsity pruning to prune LLMs to at least 50% sparsity without the need of any retraining. It allocates sparsity adaptively based on sensitivity, allowing us to reduce pruning-induced error while maintaining the overall sparsity level. The advantages of the proposed method exhibit even more when the sparsity is extremely high. Furthermore, our method is compatible with quantization, enabling further compression of LLMs. We have released the available code. △ Less

Submitted 23 April, 2024; v1 submitted 14 October, 2023; originally announced October 2023.

Comments: Accepted to ICASSP2024

Open X-Embodiment: Robotic Learning Datasets and RT-X Models

Authors: Open X-Embodiment Collaboration, Abby O'Neill, Abdul Rehman, Abhinav Gupta, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, A**kya Jain, Albert Tung, Alex Bewley, Alex Herzog, Alex Irpan, Alexander Khazatsky, Anant Rai, Anchit Gupta, Andrew Wang, Andrey Kolobov, Anikait Singh, Animesh Garg, Aniruddha Kembhavi, Annie Xie , et al. (267 additional authors not shown)

Abstract: Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning method… ▽ More Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train generalist X-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective X-robot policies. We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks). We show that a high-capacity model trained on this data, which we call RT-X, exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms. More details can be found on the project website https://robotics-transformer-x.github.io. △ Less

Submitted 1 June, 2024; v1 submitted 13 October, 2023; originally announced October 2023.

Comments: Project website: https://robotics-transformer-x.github.io

Exploring Large Language Models for Multi-Modal Out-of-Distribution Detection

Authors: Yi Dai, Hao Lang, Kaisheng Zeng, Fei Huang, Yongbin Li

Abstract: Out-of-distribution (OOD) detection is essential for reliable and trustworthy machine learning. Recent multi-modal OOD detection leverages textual information from in-distribution (ID) class names for visual OOD detection, yet it currently neglects the rich contextual information of ID classes. Large language models (LLMs) encode a wealth of world knowledge and can be prompted to generate descript… ▽ More Out-of-distribution (OOD) detection is essential for reliable and trustworthy machine learning. Recent multi-modal OOD detection leverages textual information from in-distribution (ID) class names for visual OOD detection, yet it currently neglects the rich contextual information of ID classes. Large language models (LLMs) encode a wealth of world knowledge and can be prompted to generate descriptive features for each class. Indiscriminately using such knowledge causes catastrophic damage to OOD detection due to LLMs' hallucinations, as is observed by our analysis. In this paper, we propose to apply world knowledge to enhance OOD detection performance through selective generation from LLMs. Specifically, we introduce a consistency-based uncertainty calibration method to estimate the confidence score of each generation. We further extract visual objects from each image to fully capitalize on the aforementioned world knowledge. Extensive experiments demonstrate that our method consistently outperforms the state-of-the-art. △ Less

Submitted 12 October, 2023; originally announced October 2023.

Comments: EMNLP2023 Findings Long Paper

Advective Diffusion Transformers for Topological Generalization in Graph Learning

Authors: Qitian Wu, Chenxiao Yang, Kaipeng Zeng, Fan Nie, Michael Bronstein, Junchi Yan

Abstract: Graph diffusion equations are intimately related to graph neural networks (GNNs) and have recently attracted attention as a principled framework for analyzing GNN dynamics, formalizing their expressive power, and justifying architectural choices. One key open questions in graph learning is the generalization capabilities of GNNs. A major limitation of current approaches hinges on the assumption th… ▽ More Graph diffusion equations are intimately related to graph neural networks (GNNs) and have recently attracted attention as a principled framework for analyzing GNN dynamics, formalizing their expressive power, and justifying architectural choices. One key open questions in graph learning is the generalization capabilities of GNNs. A major limitation of current approaches hinges on the assumption that the graph topologies in the training and test sets come from the same distribution. In this paper, we make steps towards understanding the generalization of GNNs by exploring how graph diffusion equations extrapolate and generalize in the presence of varying graph topologies. We first show deficiencies in the generalization capability of existing models built upon local diffusion on graphs, stemming from the exponential sensitivity to topology variation. Our subsequent analysis reveals the promise of non-local diffusion, which advocates for feature propagation over fully-connected latent graphs, under the assumption of a specific data-generating condition. In addition to these findings, we propose a novel graph encoder backbone, Advective Diffusion Transformer (ADiT), inspired by advective graph diffusion equations that have a closed-form solution backed up with theoretical guarantees of desired generalization under topological distribution shifts. The new model, functioning as a versatile graph Transformer, demonstrates superior performance across a wide range of graph learning tasks. △ Less

Submitted 10 October, 2023; originally announced October 2023.

Comments: 39 pages

A Task-oriented Dialog Model with Task-progressive and Policy-aware Pre-training

Authors: Lucen Zhong, Hengtong Lu, Caixia Yuan, Xiaojie Wang, Jiashen Sun, Ke Zeng, Guanglu Wan

Abstract: Pre-trained conversation models (PCMs) have achieved promising progress in recent years. However, existing PCMs for Task-oriented dialog (TOD) are insufficient for capturing the sequential nature of the TOD-related tasks, as well as for learning dialog policy information. To alleviate these problems, this paper proposes a task-progressive PCM with two policy-aware pre-training tasks. The model is… ▽ More Pre-trained conversation models (PCMs) have achieved promising progress in recent years. However, existing PCMs for Task-oriented dialog (TOD) are insufficient for capturing the sequential nature of the TOD-related tasks, as well as for learning dialog policy information. To alleviate these problems, this paper proposes a task-progressive PCM with two policy-aware pre-training tasks. The model is pre-trained through three stages where TOD-related tasks are progressively employed according to the task logic of the TOD system. A global policy consistency task is designed to capture the multi-turn dialog policy sequential relation, and an act-based contrastive learning task is designed to capture similarities among samples with the same dialog policy. Our model achieves better results on both MultiWOZ and In-Car end-to-end dialog modeling benchmarks with only 18\% parameters and 25\% pre-training data compared to the previous state-of-the-art PCM, GALAXY. △ Less

Submitted 1 October, 2023; originally announced October 2023.

Comments: Accepted at NLPCC 2023

OmniEvent: A Comprehensive, Fair, and Easy-to-Use Toolkit for Event Understanding

Authors: Hao Peng, Xiaozhi Wang, Feng Yao, Zimu Wang, Chuzhao Zhu, Kaisheng Zeng, Lei Hou, Juanzi Li

Abstract: Event understanding aims at understanding the content and relationship of events within texts, which covers multiple complicated information extraction tasks: event detection, event argument extraction, and event relation extraction. To facilitate related research and application, we present an event understanding toolkit OmniEvent, which features three desiderata: (1) Comprehensive. OmniEvent sup… ▽ More Event understanding aims at understanding the content and relationship of events within texts, which covers multiple complicated information extraction tasks: event detection, event argument extraction, and event relation extraction. To facilitate related research and application, we present an event understanding toolkit OmniEvent, which features three desiderata: (1) Comprehensive. OmniEvent supports mainstream modeling paradigms of all the event understanding tasks and the processing of 15 widely-used English and Chinese datasets. (2) Fair. OmniEvent carefully handles the inconspicuous evaluation pitfalls reported in Peng et al. (2023), which ensures fair comparisons between different models. (3) Easy-to-use. OmniEvent is designed to be easily used by users with varying needs. We provide off-the-shelf models that can be directly deployed as web services. The modular framework also enables users to easily implement and evaluate new event understanding models with OmniEvent. The toolkit (https://github.com/THU-KEG/OmniEvent) is publicly released along with the demonstration website and video (https://omnievent.xlore.cn/). △ Less

Submitted 25 September, 2023; originally announced September 2023.

Recovering lossless propagation of polaritons with synthesized complex frequency excitation

Authors: Fuxin Guan, Xiangdong Guo, Shu Zhang, Kebo Zeng, Yue Hu, Chenchen Wu, Shaobo Zhou, Yuanjiang Xiang, Xiaoxia Yang, Qing Dai, Shuang Zhang

Abstract: Surface plasmon polaritons and phonon polaritons offer a means of surpassing the diffraction limit of conventional optics and facilitate efficient energy storage, local field enhancement, high sensitivities, benefitting from their subwavelength confinement of light. Unfortunately, losses severely limit the propagation decay length, thus restricting the practical use of polaritons. While optimizing… ▽ More Surface plasmon polaritons and phonon polaritons offer a means of surpassing the diffraction limit of conventional optics and facilitate efficient energy storage, local field enhancement, high sensitivities, benefitting from their subwavelength confinement of light. Unfortunately, losses severely limit the propagation decay length, thus restricting the practical use of polaritons. While optimizing the fabrication technique can help circumvent the scattering loss of imperfect structures, the intrinsic absorption channel leading to heat production cannot be eliminated. Here, we utilize synthetic optical excitation of complex frequency with virtual gain, synthesized by combining the measurements taken at multiple real frequencies, to restore the lossless propagations of phonon polaritons with significantly reduced intrinsic losses. The concept of synthetic complex frequency excitation represents a viable solution to compensate for loss and would benefit applications including photonic circuits, waveguiding and plasmonic/phononic structured illumination microscopy. △ Less

Submitted 18 September, 2023; v1 submitted 28 August, 2023; originally announced August 2023.

Comments: 20 pages, 4 figures

Location Privacy and Spectrum Efficiency Enhancement in Spectrum Sharing Systems

Authors: Long Jiao, Yao Ge, Kai Zeng, B. C. Hilburn

Abstract: In this work, we investigate the benefits of secondary user (SU) network beamforming on improving primary user (PU) location privacy in spectrum sharing systems, where the beamformer in the SU network is designed to suppress the aggregate interference to improve the location privacy of PUs. We consider two problems: improving SU network communication throughput subject to the specified PU location… ▽ More In this work, we investigate the benefits of secondary user (SU) network beamforming on improving primary user (PU) location privacy in spectrum sharing systems, where the beamformer in the SU network is designed to suppress the aggregate interference to improve the location privacy of PUs. We consider two problems: improving SU network communication throughput subject to the specified PU location privacy requirements, and enhancing PU location privacy given the quality of service (QoS) requirements of SU networks. In the first problem, we provide an algorithm to achieve high data rates with the constrained PU location privacy level. Numerical results show that for a given PU location privacy requirement, the proposed scheme is able to interfere/exclude only a few SU nodes from the PU band and the network throughput can be greatly improved. In the second problem, to fully explore the potential of SU network beamforming for enhancing PU location privacy, we propose a two-step scheme to decouple the beamforming and privacy zone design so that the PU location privacy can be improved while satisfying the SU network throughput requirement. According to numerical evaluations, the proposed scheme can maintain/achieve higher PU location privacy than the benchmark beamforming schemes while satisfying a QoS requirement for the SU network. △ Less

Submitted 26 August, 2023; originally announced August 2023.

Optically levitated gyroscopes with a MHz rotating micro-rotor

Authors: Kai Zeng, Xiangming Xu, Yulie Wu, Xuezhong Wu, Dingbang Xiao

Abstract: The optically levitated particles have been driven to rotate at an ultra-high speed of GHz, and the gyroscopic application of these levitated particles to measure angular motion have long been explored. However, this gyroscope has not been proven either theoretically or experimentally. Here, a rotor gyroscope based on optically levitated high-speed rotating particles is proposed. In vacuum, an ell… ▽ More The optically levitated particles have been driven to rotate at an ultra-high speed of GHz, and the gyroscopic application of these levitated particles to measure angular motion have long been explored. However, this gyroscope has not been proven either theoretically or experimentally. Here, a rotor gyroscope based on optically levitated high-speed rotating particles is proposed. In vacuum, an ellipsoidal vaterite particle with 3.58 $μ$m average diameter is driven to rotate at MHz, and the optical axis orientation of the particle is measured by the particle rotational signal. The external inputted angular velocity makes the optical axis deviate from the initial position, which changes the frequency and amplitude of the rotational signal. The inputted angular velocity is hence detected by the rotational signal, and the angular rate bias instability of the prototype is measured to be $0.08^o/s$. It is the smallest rotor gyroscope in the world, and the bias instability can be further improved up to $10^{-9o}/h$ theoretically by cooling the motion and increasing the angular moment of the levitated particle. Our work opens a new application paradigm of the levitated optomechanical systems and possibly bring the rotor gyroscope to the quantum realm. △ Less

Submitted 17 August, 2023; originally announced August 2023.

Journal ref: Microsystems & Nanoengineering, 10 78 (2024)

Loday-Quillen-Tsygan theorem on Quivers

Authors: Keyou Zeng

Abstract: The well-known Loday-Quillen-Tsygan theorem calculates the Lie algebra homology of the infinite general linear Lie algebra $\mathfrak{gl}(A)$ over an unital associative algebra $A$. We generalize the Loday-Quillen-Tsygan theorem to an infinite Lie algebra associated with a (framed) quiver, where we assign to each vertex $v$ an infinite general linear Lie algebra $\mathfrak{gl}(A_v)$, to each edge… ▽ More The well-known Loday-Quillen-Tsygan theorem calculates the Lie algebra homology of the infinite general linear Lie algebra $\mathfrak{gl}(A)$ over an unital associative algebra $A$. We generalize the Loday-Quillen-Tsygan theorem to an infinite Lie algebra associated with a (framed) quiver, where we assign to each vertex $v$ an infinite general linear Lie algebra $\mathfrak{gl}(A_v)$, to each edge $e$ an infinite matrix module and to each framed vertex a (anti)-fundamental representation. Given this data, each loop or path ending on framed vertices of the quiver defined a stratified factorization algebra over $S^1$ or $[0,1]$ respectively. We show that the corresponding Lie algebra homology can be expressed as summing the factorization homology over all loops and framed paths of the quiver. △ Less

Submitted 19 July, 2023; originally announced July 2023.

Wide-band Unambiguous Quantum Sensing via Geodesic Evolution

Authors: Ke Zeng, Xiaohui Yu, Martin B. Plenio, Zhen-Yu Wang

Abstract: We present a quantum sensing technique that utilizes a sequence of $π$ pulses to cyclically drive the qubit dynamics along a geodesic path of adiabatic evolution. This approach effectively suppresses the effects of both decoherence noise and control errors while simultaneously removing unwanted resonance terms, such as higher harmonics and spurious responses commonly encountered in dynamical decou… ▽ More We present a quantum sensing technique that utilizes a sequence of $π$ pulses to cyclically drive the qubit dynamics along a geodesic path of adiabatic evolution. This approach effectively suppresses the effects of both decoherence noise and control errors while simultaneously removing unwanted resonance terms, such as higher harmonics and spurious responses commonly encountered in dynamical decoupling control. As a result, our technique offers robust, wide-band, unambiguous, and high-resolution quantum sensing capabilities for signal detection and individual addressing of quantum systems, including spins. To demonstrate its versatility, we showcase successful applications of our method in both low-frequency and high-frequency sensing scenarios. The significance of this quantum sensing technique extends to the detection of complex signals and the control of intricate quantum environments. By enhancing detection accuracy and enabling precise manipulation of quantum systems, our method holds considerable promise for a variety of practical applications. △ Less

Submitted 19 July, 2023; originally announced July 2023.

Synthesized complex-frequency excitation for ultrasensitive molecular sensing

Authors: Kebo Zeng, Chenchen Wu, Xiangdong Guo, Fuxin Guan, Yu Duan, Lauren L Zhang, Xiaoxia Yang, Na Liu, Qing Dai, Shuang Zhang

Abstract: Detecting trace molecules remains a significant challenge. Surface-enhanced infrared absorption (SEIRA) based on plasmonic nanostructures, particularly graphene, has emerged as a promising approach to enhance sensing sensitivity. While graphene-based SEIRA offers advantages such as ultrahigh sensitivity and active tunability, intrinsic molecular dam** weakens the interaction between vibrational… ▽ More Detecting trace molecules remains a significant challenge. Surface-enhanced infrared absorption (SEIRA) based on plasmonic nanostructures, particularly graphene, has emerged as a promising approach to enhance sensing sensitivity. While graphene-based SEIRA offers advantages such as ultrahigh sensitivity and active tunability, intrinsic molecular dam** weakens the interaction between vibrational modes and plasmons. Here, we demonstrate ultrahigh-sensitive molecular sensing based on synthesized complex-frequency waves (CFW). Our experiment shows that CFW can amplify the molecular signals (~1.2-nm-thick silk protein layer) detected by graphene-based sensor by at least an order of magnitude and can be universally applied to molecular sensing in different phases. Our approach is highly scalable and can facilitate the investigation of light-matter interactions, enabling diverse potential applications in fields such as optical spectroscopy, metasurfaces, optoelectronics, biomedicine and pharmaceutics. △ Less

Submitted 18 July, 2023; originally announced July 2023.

Comments: 21 pages, 4 figures

KoLA: Carefully Benchmarking World Knowledge of Large Language Models

Authors: Jifan Yu, Xiaozhi Wang, Shangqing Tu, Shulin Cao, Daniel Zhang-Li, Xin Lv, Hao Peng, Zijun Yao, Xiaohan Zhang, Hanming Li, Chunyang Li, Zheyuan Zhang, Yushi Bai, Yantao Liu, Amy Xin, Nianyi Lin, Kaifeng Yun, Linlu Gong, Jianhui Chen, Zhili Wu, Yunjia Qi, Weikai Li, Yong Guan, Kaisheng Zeng, Ji Qi , et al. (10 additional authors not shown)

Abstract: The unprecedented performance of large language models (LLMs) necessitates improvements in evaluations. Rather than merely exploring the breadth of LLM abilities, we believe meticulous and thoughtful designs are essential to thorough, unbiased, and applicable evaluations. Given the importance of world knowledge to LLMs, we construct a Knowledge-oriented LLM Assessment benchmark (KoLA), in which we… ▽ More The unprecedented performance of large language models (LLMs) necessitates improvements in evaluations. Rather than merely exploring the breadth of LLM abilities, we believe meticulous and thoughtful designs are essential to thorough, unbiased, and applicable evaluations. Given the importance of world knowledge to LLMs, we construct a Knowledge-oriented LLM Assessment benchmark (KoLA), in which we carefully design three crucial factors: (1) For \textbf{ability modeling}, we mimic human cognition to form a four-level taxonomy of knowledge-related abilities, covering $19$ tasks. (2) For \textbf{data}, to ensure fair comparisons, we use both Wikipedia, a corpus prevalently pre-trained by LLMs, along with continuously collected emerging corpora, aiming to evaluate the capacity to handle unseen data and evolving knowledge. (3) For \textbf{evaluation criteria}, we adopt a contrastive system, including overall standard scores for better numerical comparability across tasks and models and a unique self-contrast metric for automatically evaluating knowledge-creating ability. We evaluate $28$ open-source and commercial LLMs and obtain some intriguing findings. The KoLA dataset and open-participation leaderboard are publicly released at https://kola.xlore.cn and will be continuously updated to provide references for develo** LLMs and knowledge-related systems. △ Less

Submitted 30 June, 2024; v1 submitted 15 June, 2023; originally announced June 2023.

Comments: Accepted by ICLR 2024

The Devil is in the Details: On the Pitfalls of Event Extraction Evaluation

Authors: Hao Peng, Xiaozhi Wang, Feng Yao, Kaisheng Zeng, Lei Hou, Juanzi Li, Zhiyuan Liu, Weixing Shen

Abstract: Event extraction (EE) is a crucial task aiming at extracting events from texts, which includes two subtasks: event detection (ED) and event argument extraction (EAE). In this paper, we check the reliability of EE evaluations and identify three major pitfalls: (1) The data preprocessing discrepancy makes the evaluation results on the same dataset not directly comparable, but the data preprocessing… ▽ More Event extraction (EE) is a crucial task aiming at extracting events from texts, which includes two subtasks: event detection (ED) and event argument extraction (EAE). In this paper, we check the reliability of EE evaluations and identify three major pitfalls: (1) The data preprocessing discrepancy makes the evaluation results on the same dataset not directly comparable, but the data preprocessing details are not widely noted and specified in papers. (2) The output space discrepancy of different model paradigms makes different-paradigm EE models lack grounds for comparison and also leads to unclear map** issues between predictions and annotations. (3) The absence of pipeline evaluation of many EAE-only works makes them hard to be directly compared with EE works and may not well reflect the model performance in real-world pipeline scenarios. We demonstrate the significant influence of these pitfalls through comprehensive meta-analyses of recent papers and empirical experiments. To avoid these pitfalls, we suggest a series of remedies, including specifying data preprocessing, standardizing outputs, and providing pipeline evaluation results. To help implement these remedies, we develop a consistent evaluation framework OMNIEVENT, which can be obtained from https://github.com/THU-KEG/OmniEvent. △ Less

Submitted 15 June, 2023; v1 submitted 12 June, 2023; originally announced June 2023.

Comments: Accepted at Findings of ACL 2023

Benchmarking Foundation Models with Language-Model-as-an-Examiner

Authors: Yushi Bai, Jiahao Ying, Yixin Cao, Xin Lv, Yuze He, Xiaozhi Wang, Jifan Yu, Kaisheng Zeng, Yijia Xiao, Haozhe Lyu, Jiayin Zhang, Juanzi Li, Lei Hou

Abstract: Numerous benchmarks have been established to assess the performance of foundation models on open-ended question answering, which serves as a comprehensive test of a model's ability to understand and generate language in a manner similar to humans. Most of these works focus on proposing new datasets, however, we see two main issues within previous benchmarking pipelines, namely testing leakage and… ▽ More Numerous benchmarks have been established to assess the performance of foundation models on open-ended question answering, which serves as a comprehensive test of a model's ability to understand and generate language in a manner similar to humans. Most of these works focus on proposing new datasets, however, we see two main issues within previous benchmarking pipelines, namely testing leakage and evaluation automation. In this paper, we propose a novel benchmarking framework, Language-Model-as-an-Examiner, where the LM serves as a knowledgeable examiner that formulates questions based on its knowledge and evaluates responses in a reference-free manner. Our framework allows for effortless extensibility as various LMs can be adopted as the examiner, and the questions can be constantly updated given more diverse trigger topics. For a more comprehensive and equitable evaluation, we devise three strategies: (1) We instruct the LM examiner to generate questions across a multitude of domains to probe for a broad acquisition, and raise follow-up questions to engage in a more in-depth assessment. (2) Upon evaluation, the examiner combines both scoring and ranking measurements, providing a reliable result as it aligns closely with human annotations. (3) We additionally propose a decentralized Peer-examination method to address the biases in a single examiner. Our data and benchmarking results are available at: http://lmexam.xlore.cn. △ Less

Submitted 4 November, 2023; v1 submitted 7 June, 2023; originally announced June 2023.

Comments: NeurIPS 2023 Datasets and Benchmarks

Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction

Authors: Ji Qi, Chuchun Zhang, Xiaozhi Wang, Kaisheng Zeng, Jifan Yu, **xin Liu, Jiuding Sun, Yuxiang Chen, Lei Hou, Juanzi Li, Bin Xu

Abstract: The robustness to distribution changes ensures that NLP models can be successfully applied in the realistic world, especially for information extraction tasks. However, most prior evaluation benchmarks have been devoted to validating pairwise matching correctness, ignoring the crucial measurement of robustness. In this paper, we present the first benchmark that simulates the evaluation of open inf… ▽ More The robustness to distribution changes ensures that NLP models can be successfully applied in the realistic world, especially for information extraction tasks. However, most prior evaluation benchmarks have been devoted to validating pairwise matching correctness, ignoring the crucial measurement of robustness. In this paper, we present the first benchmark that simulates the evaluation of open information extraction models in the real world, where the syntactic and expressive distributions under the same knowledge meaning may drift variously. We design and annotate a large-scale testbed in which each example is a knowledge-invariant clique that consists of sentences with structured knowledge of the same meaning but with different syntactic and expressive forms. By further elaborating the robustness metric, a model is judged to be robust if its performance is consistently accurate on the overall cliques. We perform experiments on typical models published in the last decade as well as a popular large language model, the results show that the existing successful models exhibit a frustrating degradation, with a maximum drop of 23.43 F1 score. Our resources and code are available at https://github.com/qijimrc/ROBUST. △ Less

Submitted 24 October, 2023; v1 submitted 23 May, 2023; originally announced May 2023.

Comments: Accepted by EMNLP 2023 Main Conference

Quiver: Supporting GPUs for Low-Latency, High-Throughput GNN Serving with Workload Awareness

Authors: Zeyuan Tan, Xiulong Yuan, Congjie He, Man-Kit Sit, Guo Li, Xiaoze Liu, Baole Ai, Kai Zeng, Peter Pietzuch, Luo Mai

Abstract: Systems for serving inference requests on graph neural networks (GNN) must combine low latency with high throughout, but they face irregular computation due to skew in the number of sampled graph nodes and aggregated GNN features. This makes it challenging to exploit GPUs effectively: using GPUs to sample only a few graph nodes yields lower performance than CPU-based sampling; and aggregating many… ▽ More Systems for serving inference requests on graph neural networks (GNN) must combine low latency with high throughout, but they face irregular computation due to skew in the number of sampled graph nodes and aggregated GNN features. This makes it challenging to exploit GPUs effectively: using GPUs to sample only a few graph nodes yields lower performance than CPU-based sampling; and aggregating many features exhibits high data movement costs between GPUs and CPUs. Therefore, current GNN serving systems use CPUs for graph sampling and feature aggregation, limiting throughput. We describe Quiver, a distributed GPU-based GNN serving system with low-latency and high-throughput. Quiver's key idea is to exploit workload metrics for predicting the irregular computation of GNN requests, and governing the use of GPUs for graph sampling and feature aggregation: (1) for graph sampling, Quiver calculates the probabilistic sampled graph size, a metric that predicts the degree of parallelism in graph sampling. Quiver uses this metric to assign sampling tasks to GPUs only when the performance gains surpass CPU-based sampling; and (2) for feature aggregation, Quiver relies on the feature access probability to decide which features to partition and replicate across a distributed GPU NUMA topology. We show that Quiver achieves up to 35 times lower latency with an 8 times higher throughput compared to state-of-the-art GNN approaches (DGL and PyG). △ Less

Submitted 18 May, 2023; originally announced May 2023.

Autoencoders for discovering manifold dimension and coordinates in data from complex dynamical systems

Authors: Kevin Zeng, Carlos E. Pérez De Jesús, Andrew J. Fox, Michael D. Graham

Abstract: While many phenomena in physics and engineering are formally high-dimensional, their long-time dynamics often live on a lower-dimensional manifold. The present work introduces an autoencoder framework that combines implicit regularization with internal linear layers and $L_2$ regularization (weight decay) to automatically estimate the underlying dimensionality of a data set, produce an orthogonal… ▽ More While many phenomena in physics and engineering are formally high-dimensional, their long-time dynamics often live on a lower-dimensional manifold. The present work introduces an autoencoder framework that combines implicit regularization with internal linear layers and $L_2$ regularization (weight decay) to automatically estimate the underlying dimensionality of a data set, produce an orthogonal manifold coordinate system, and provide the map** functions between the ambient space and manifold space, allowing for out-of-sample projections. We validate our framework's ability to estimate the manifold dimension for a series of datasets from dynamical systems of varying complexities and compare to other state-of-the-art estimators. We analyze the training dynamics of the network to glean insight into the mechanism of low-rank learning and find that collectively each of the implicit regularizing layers compound the low-rank representation and even self-correct during training. Analysis of gradient descent dynamics for this architecture in the linear case reveals the role of the internal linear layers in leading to faster decay of a "collective weight variable" incorporating all layers, and the role of weight decay in breaking degeneracies and thus driving convergence along directions in which no decay would occur in its absence. We show that this framework can be naturally extended for applications of state-space modeling and forecasting by generating a data-driven dynamic model of a spatiotemporally chaotic partial differential equation using only the manifold coordinates. Finally, we demonstrate that our framework is robust to hyperparameter choices. △ Less

Submitted 6 December, 2023; v1 submitted 1 May, 2023; originally announced May 2023.

Moving Forward by Moving Backward: Embedding Action Impact over Action Semantics

Authors: Kuo-Hao Zeng, Luca Weihs, Roozbeh Mottaghi, Ali Farhadi

Abstract: A common assumption when training embodied agents is that the impact of taking an action is stable; for instance, executing the "move ahead" action will always move the agent forward by a fixed distance, perhaps with some small amount of actuator-induced noise. This assumption is limiting; an agent may encounter settings that dramatically alter the impact of actions: a move ahead action on a wet f… ▽ More A common assumption when training embodied agents is that the impact of taking an action is stable; for instance, executing the "move ahead" action will always move the agent forward by a fixed distance, perhaps with some small amount of actuator-induced noise. This assumption is limiting; an agent may encounter settings that dramatically alter the impact of actions: a move ahead action on a wet floor may send the agent twice as far as it expects and using the same action with a broken wheel might transform the expected translation into a rotation. Instead of relying that the impact of an action stably reflects its pre-defined semantic meaning, we propose to model the impact of actions on-the-fly using latent embeddings. By combining these latent action embeddings with a novel, transformer-based, policy head, we design an Action Adaptive Policy (AAP). We evaluate our AAP on two challenging visual navigation tasks in the AI2-THOR and Habitat environments and show that our AAP is highly performant even when faced, at inference-time with missing actions and, previously unseen, perturbed action space. Moreover, we observe significant improvement in robustness against these actions when evaluating in real-world scenarios. △ Less

Submitted 24 April, 2023; originally announced April 2023.

Comments: 21 pages, 17 figures, ICLR 2023

Assessing the Socio-economic Impacts of Secure Texting and Anti-Jamming Technologies in Non-Cooperative Networks

Authors: Osoro B Ogutu, Edward J Oughton, Kai Zeng, Brian L. Mark

Abstract: Operating securely over 5G (and legacy) infrastructure is a challenge. In non-cooperative networks, malicious actors may try to decipher, block encrypted messages, or specifically jam wireless radio systems. Such activities can disrupt operations, from causing minor inconvenience, through to fully paralyzing the functionality of critical infrastructure. While technological mitigation measures do e… ▽ More Operating securely over 5G (and legacy) infrastructure is a challenge. In non-cooperative networks, malicious actors may try to decipher, block encrypted messages, or specifically jam wireless radio systems. Such activities can disrupt operations, from causing minor inconvenience, through to fully paralyzing the functionality of critical infrastructure. While technological mitigation measures do exist, there are very few methods capable of assessing the socio-economic impacts from different mitigation strategies. This leads to a lack of robust evidence to inform cost-benefit analysis, and thus support decision makers in industry and government. Consequently, this paper presents two open-source simulation models for assessing the socio-economic impacts of operating in untrusted non-cooperative networks. The first focuses on using multiple non-cooperative networks to transmit a message. The second model simulates a case where a message is converted into alternative plain language to avoid detection, separated into different portions and then transmitted over multiple non-cooperative networks. A probabilistic simulation of the two models is performed for a 15 km by 15 km spatial grid with 5 untrusted non-cooperative networks and intercepting agents. The results are used to estimate economic losses for private, commercial, government and military sectors. The highest probabilistic total losses for military applications include US$300, US$150, and US$75, incurred for a 1, 3 and 5 site multi-transmission approach, respectively, for non-cooperative networks when considering 1,000 texts being sent. These results form a framework for deterministic socio-economic impact analysis of using non-cooperative networks and secure texting as protection against radio network attacks. The simulation data and the open-source codebase is provided for reproducibility. △ Less

Submitted 10 April, 2023; v1 submitted 29 March, 2023; originally announced March 2023.

Overcoming losses in superlenses with synthetic waves of complex frequency

Authors: Fuxin Guan, Kebo Zeng, Zhaoyu Nie, Xiangdong Guo, Shaojie Ma, Qing Dai, John B. Pendry, Xiang Zhang, Shuang Zhang

Abstract: Superlenses made of plasmonic materials and metamaterials have been exploited to image features of sub-diffractional scale. However, their intrinsic losses impose a serious restriction on the imaging resolution, which is a long-standing problem that has hindered wide-spread applications of superlenses. Optical waves of complex frequency exhibiting a temporally attenuating behavior have been propos… ▽ More Superlenses made of plasmonic materials and metamaterials have been exploited to image features of sub-diffractional scale. However, their intrinsic losses impose a serious restriction on the imaging resolution, which is a long-standing problem that has hindered wide-spread applications of superlenses. Optical waves of complex frequency exhibiting a temporally attenuating behavior have been proposed to offset the intrinsic losses in superlenses via virtual gain, but the experimental realization has been missing due to the challenge involved in preparing the illumination with temporal decay. Here, by employing multi-frequency measurement, we successfully implement a synthetic optical wave of complex frequency to experimentally observe deep-subwavelength superimaging patterns enabled by the virtual gain. Our work represents a practical approach to overcoming the intrinsic losses of plasmonic systems for imaging and sensing applications. △ Less

Submitted 22 March, 2023; originally announced March 2023.

Comments: 17 pages, 3 figures

Twisted Holography and Celestial Holography from Boundary Chiral Algebra

Authors: Keyou Zeng

Abstract: We study the Kaluza-Klein reduction of various $6d$ holomorphic theories. The KK reduction is analyzed in the BV formalism, resulting in theories that come from the holomorphic topological twist of $3d$ $\mathcal{N} = 2$ supersymmetric field theories. Effective interactions of the KK theories at the classical level can be obtained at all orders using homotopy transfer theorem. We also analyze a de… ▽ More We study the Kaluza-Klein reduction of various $6d$ holomorphic theories. The KK reduction is analyzed in the BV formalism, resulting in theories that come from the holomorphic topological twist of $3d$ $\mathcal{N} = 2$ supersymmetric field theories. Effective interactions of the KK theories at the classical level can be obtained at all orders using homotopy transfer theorem. We also analyze a deformation of the theories that comes from deforming the spacetime geometry to $SL_2(\mathbb{C})$ due to the brane back-reaction. We study the boundary chiral algebras for the various KK theories. Using Koszul duality, we argue that by properly choosing a boundary condition, the boundary chiral algebra coincides with the universal defect chiral algebra of the original theory. This perspective provides a unified framework for accessing the chiral algebras that arise from both twisted holography and celestial holography programs. △ Less

Submitted 13 February, 2023; originally announced February 2023.

Comments: 99 pages

Turbulence control in plane Couette flow using low-dimensional neural ODE-based models and deep reinforcement learning

Authors: Alec J. Linot, Kevin Zeng, Michael D. Graham

Abstract: The high dimensionality and complex dynamics of turbulent flows remain an obstacle to the discovery and implementation of control strategies. Deep reinforcement learning (RL) is a promising avenue for overcoming these obstacles, but requires a training phase in which the RL agent iteratively interacts with the flow environment to learn a control policy, which can be prohibitively expensive when th… ▽ More The high dimensionality and complex dynamics of turbulent flows remain an obstacle to the discovery and implementation of control strategies. Deep reinforcement learning (RL) is a promising avenue for overcoming these obstacles, but requires a training phase in which the RL agent iteratively interacts with the flow environment to learn a control policy, which can be prohibitively expensive when the environment involves slow experiments or large-scale simulations. We overcome this challenge using a framework we call "DManD-RL" (data-driven manifold dynamics-RL), which generates a data-driven low-dimensional model of our system that we use for RL training. With this approach, we seek to minimize drag in a direct numerical simulation (DNS) of a turbulent minimal flow unit of plane Couette flow at Re=400 using two slot jets on one wall. We obtain, from DNS data with $\mathcal{O}(10^5)$ degrees of freedom, a 25-dimensional DManD model of the dynamics by combining an autoencoder and neural ordinary differential equation. Using this model as the environment, we train an RL control agent, yielding a 440-fold speedup over training on the DNS, with equivalent control performance. The agent learns a policy that laminarizes 84% of unseen DNS test trajectories within 900 time units, significantly outperforming classical opposition control (58%), despite the actuation authority being much more restricted. The agent often achieves laminarization through a counterintuitive strategy that drives the formation of two low-speed streaks, with a spanwise wavelength that is too small to be self-sustaining. The agent demonstrates the same performance when we limit observations to wall shear rate. △ Less

Submitted 28 January, 2023; originally announced January 2023.

Khaos: The Impact of Inter-procedural Code Obfuscation on Binary Diffing Techniques

Authors: Peihua Zhang, Chenggang Wu, Mingfan Peng, Kai Zeng, Ding Yu, Yuanming Lai, Yan Kang, Wei Wang, Zhe Wang

Abstract: Software obfuscation techniques can prevent binary diffing techniques from locating vulnerable code by obfuscating the third-party code, to achieve the purpose of protecting embedded device software. With the rapid development of binary diffing techniques, they can achieve more and more accurate function matching and identification by extracting the features within the function. This makes existin… ▽ More Software obfuscation techniques can prevent binary diffing techniques from locating vulnerable code by obfuscating the third-party code, to achieve the purpose of protecting embedded device software. With the rapid development of binary diffing techniques, they can achieve more and more accurate function matching and identification by extracting the features within the function. This makes existing software obfuscation techniques, which mainly focus on the intra-procedural code obfuscation, no longer effective. In this paper, we propose a new inter-procedural code obfuscation mechanism Khaos, which moves the code across functions to obfuscate the function by using compilation optimizations. Two obfuscation primitives are proposed to separate and aggregate the function, which are called fission and fusion respectively. A prototype of Khaos is implemented based on the LLVM compiler and evaluated on a large number of real-world programs including SPEC CPU 2006 & 2017, CoreUtils, JavaScript engines, etc. Experimental results show that Khaos outperforms existing code obfuscations and can significantly reduce the accuracy rates of five state-of-the-art binary diffing techniques (less than 19%) with lower runtime overhead (less than 7%). △ Less

Submitted 27 January, 2023; originally announced January 2023.

Quadratic Duality for Chiral Algebras

Authors: Zheng** Gui, Si Li, Keyou Zeng

Abstract: We introduce a notion of quadratic duality for chiral algebras. This can be viewed as a chiral version of the usual quadratic duality for quadratic associative algebras. We study the relationship between this duality notion and the Maurer-Cartan equations for chiral algebras, which turns out to be parallel to the associative algebra case. We also present some explicit examples. We introduce a notion of quadratic duality for chiral algebras. This can be viewed as a chiral version of the usual quadratic duality for quadratic associative algebras. We study the relationship between this duality notion and the Maurer-Cartan equations for chiral algebras, which turns out to be parallel to the associative algebra case. We also present some explicit examples. △ Less

Submitted 21 December, 2022; originally announced December 2022.

Comments: 23 pages. Comments are welcome

Electronic nematicity without charge density waves in titanium-based kagome metal

Authors: Hong Li, Siyu Cheng, Brenden R. Ortiz, Hengxin Tan, Dominik Werhahn, Keyu Zeng, Dirk Johrendt, Binghai Yan, Ziqiang Wang, Stephen D. Wilson, Ilija Zeljkovic

Abstract: Layered crystalline materials that consist of transition metal atoms on a kagome network have emerged as a versatile platform to study unusual electronic phenomena. For example, in the vanadium-based kagome superconductors AV3Sb5 (where A can stand for K, Cs, or Rb) there is a parent charge density wave phase that appears to simultaneously break both the translational and the rotational symmetry o… ▽ More Layered crystalline materials that consist of transition metal atoms on a kagome network have emerged as a versatile platform to study unusual electronic phenomena. For example, in the vanadium-based kagome superconductors AV3Sb5 (where A can stand for K, Cs, or Rb) there is a parent charge density wave phase that appears to simultaneously break both the translational and the rotational symmetry of the lattice. Here, we show a contrasting situation where electronic nematic order - the breaking of rotational symmetry without the breaking of translational symmetry - can occur without a corresponding charge density wave. We use spectroscopic-imaging scanning tunneling microscopy to study the kagome metal CsTi3Bi5 that is isostructural to AV3Sb5 but with a titanium atom kagome network. CsTi3Bi5 does not exhibit any detectable charge density wave state, but comparison to density functional theory calculations reveals substantial electronic correlation effects at low energies. Comparing the amplitudes of scattering wave vectors along different directions, we discover an electronic anisotropy that breaks the six-fold symmetry of the lattice, arising from both in-plane and out-of-plane titanium-derived d orbitals. Our work uncovers the role of electronic orbitals in CsTi3Bi5, suggestive of a hexagonal analogue of the nematic bond order in Fe-based superconductors. △ Less

Submitted 27 July, 2023; v1 submitted 29 November, 2022; originally announced November 2022.

Comments: This is the submitted version. Final manuscript will appear in Nature Physics

Journal ref: Nature Physics 19, 1591 (2023)